You only have to switch on the TV to assume that cars with ‘autonomous’ driving systems can drive you to work while you, almost literally, take a back seat.
But in reality, while driving support technologies help the driver to maintain a steady speed and keep a safe distance from the car in front, self-driving cars are still far from being available.
In 2018, as part of its ongoing commitment to independently assess the benefits of new vehicle safety technologies, Euro NCAP launched a campaign on Assisted Driving (AD) in addition to its star rating scheme.
What was discovered in 2018 was pretty alarming; without driver intervention, most ‘Highway Assist’ systems tested failed to cope with even straightforward driving situations, ones which even an inexperienced driver would be able to navigate. Driver oversight is paramount to keeping the highways safe.
For 2020, Euro NCAP has launched an extended, more comprehensive AD grading system, a natural progression from the one carried out in 2018, which aims to inform consumers about the best AD systems currently available, and more importantly, highlight their technological limitations to consumers.
Euro NCAP test facilities involved in the launch include Thatcham Research in the UK, ADAC and BASt in Germany, UTAC Ceram in France, IDIADA in Spain, TNO/TASS in the Netherlands, Asta Zero in Sweden and CSI in Italy. For 2020, ten cars have been tested, including: the Audi Q8, the BMW 3 Series, the Ford Kuga, the Mercedes GLE, the Nissan Juke, the Renault Clio, the Tesla Model 3, the Volkswagen Passat, the Volvo V60 and the Peugeot 2008.
The test series looks at how the latest ‘Highway Assist’ systems in cars perform in a series of scenarios, typical of ‘real-world’ driving. The systems combine Adaptive Cruise Control, Lane Centering and Speed Assist Systems and are supposed to support the driver in monotonous driving situations on motorways and adapt to the traffic conditions.
Notable results include:
- From the Very Good graded cars, the Mercedes has a top score in all three assessment areas.
- BMW significantly increased their level of Assistance compared to the 2018 publication while maintaining a balance between Driver Engagement and Vehicle Assistance.
- The Tesla Model 3 has a top score in Vehicle Assistance but lacks Driver Engagement, resulting in a Moderate grading.
- Ford shows that with a mid-class Kuga a Good grading is achievable due to a combination of Vehicle Assistance and a good Safety Back-up.
- All three cars with Entry systems lack Emergency Assist which would have increased their grading when scored.
The cars were tested on three criteria – Driver Engagement, Vehicle Assistance and Safety Backup, where they could score a maximum of 100 points for each category. As was discovered in the 2018 campaign, good performance in one area is not helpful, unless it is accompanied by good performance in the other. Their scores out of 100 place them in grades from Entry, Moderate, Good and Very Good.
As Euro NCAP’s Secretary General, Michiel van Ratingen explains, “The goal was to not discourage manufacturers from developing their AD systems, so we avoided categorising any systems as ‘Poor’ and also kept in mind while grading them that some manufacturers are creating their systems for less premium cars than others.”
An Entry level vehicle assistance system typically provides assistance in the less challenging scenarios and comprises only a basic ACC and Lane Centering system, with no additional features.
Comparatively, Very Good means the vehicle has state-of-the-art ACC and Lane Centering systems with additional functions to support the driver and keep them engaged in the driving task. These vehicles also provide a high-level safety back-up in challenging scenarios, utilizing the extended sensor set these vehicles are equipped with, meaning that in the case of an unresponsive driver, the car would be able to find the best and safest solution.
The good news is that since the 2018 tests, Euro NCAP has spotted noticeable differences and improvements in multiple criteria from certain manufacturer AD systems, despite some misleading marketing of the systems, which we will discuss in more detail later.
What makes a good AD system, according to NCAP, is a good balance between steering support and driver input. Van Ratingen explains, “We test the cars in multiple very challenging scenarios that are not uncommon on the highway. Many of these systems will alone be unable to cope with the situations presented as they will see the object too late to avoid the crash. They need the driver to react.
The key is finding the balance between assistance and keeping the driver engaged. “Too little help from the car and there would be little benefit to the driver. Too much support, and the driver may be lured into thinking they don’t need to maintain attention,” says van Ratingen.
What this means is that with increasing levels of assistance from the car, it becomes more and more critical for the system to ensure that the driver remains engaged in the driving task. “The more support and assistance the car gives the driver, the more effort actually has to be put in by the manufacturer to make sure the driver is engaged,” van Ratingen says.
In Euro NCAP’s recent testing, high-end vehicles including BMW, Mercedes and Audi have shown an improvement in the assistance of their systems over the two years, as they have all made a conscious effort to further balance their advancements with increased driver engagement.
The BMW 3 Series, for example, excels in the level of Vehicle Assistance with a similar level of Driver Engagement resulting in a high-performing, balanced system. “We tested the 3 Series in 2018 and have tested it again to see a significant improvement, showing that the manufacturer shared our concerns, and we hope that will continue,” explains van Ratingen.
This is a similar story for the Ford Kuga, which is a far cheaper car than the 3 Series but provides a good level of Vehicle Assistance and Driver Engagement resulting in another good, balanced system. “For a reasonably priced mainstream SUV, the Kuga delivers a balanced system that aims to keep the driver engaged in the driving task at hand,” Euro NCAP’s Technical Director, Richard Schram, says.
Although clearly at a higher price point than the Ford, the Tesla Model 3 offers a very high level of Vehicle Assistance and Safety Backup in the case of an emergency, but was found to fall behind other manufacturer systems in terms of balancing Vehicle Assistance systems with Driver Engagement.
Schram explains, “Good AD systems should offer steering support while the driver maintains full control and works with the driver’s intentions, not against them.”
NCAP says the steering input applied to the steering wheel to override the system should not feel like the driver is fighting against the vehicle, which they found in the Model 3. “When the driver steers away from the middle of the lane, for any reason, the system should stay engaged but always overridable, to ensure the feeling of cooperation, not a hand-over of control. Instead, Tesla’s system resisted driver steering input and then disengaged, limiting co-operative driving.”
The main point is that the driver needs to be engaged in any given situation, “and in the case of all of the cars tested, the driver engagement just isn’t up to scratch,” says van Ratingen. But to excel in that area, the systems firstly need to ‘see’ the driver and use a driver-facing camera to evaluate the driver’s state of alertness and engagement.
Euro NCAP recently recognised the importance of these Direct Driver Monitoring systems (DMS) in its revised crash-test safety standards, which starting this year require DMS for a five-star rating. “For higher performance in Vehicle Assistance, it is crucial that DMS is part of a system,” says Schram.
But while some vehicles may excel in certain areas and lag behind in others, the findings of the tests concluded that all vehicles tested lack these DMS systems to really understand whether the driver is engaged.
Schram explains, “If the system detects fatigue or distraction, it should act accordingly. Finally, if there is no driver response, the system should enter an emergency mode. But not all cars we tested did this successfully.”
Misleading marketing
Two years is a long time in the field of driving assistance and, since 2018, the technology in each vehicle has developed. But it’s important to reiterate that it’s still not completely autonomous, as proven by the results of NCAP’s most recent testing.
So why, then, are some manufacturers making such sensational claims in the media about their ‘autonomous’ cars?
“What some of the manufacturers are saying to their customers is really misleading from our perspective,” says Schram. “Currently, there is no such thing as full automation, so a good system name should not be misleading and should contain the word ‘Assistance’ to clearly identify the system design and its purpose to assist the driver.” Moreover, the system name should never contain ‘auto’ or refer to automation, but Euro NCAP’s findings discovered this isn’t the case.
“In fact, we’re still finding that many manufacturers are using names that are just not accurate but they’re choosing them because they’re more catchy,” says Schram. An example of this is Ford’s ‘Co-Pilot 360.’ “The name doesn’t cover the content of what the system is actually doing, and misleads the consumer into thinking the car can fully operate by itself, without the need for human intervention,” Schram explains.
The Tesla Model 3 system ‘Autopilot’ is also named incorrectly, suggests Schram, as it implies the car offers full automation. “In our study, we also found its promotional material suggests automation where the handbook small print correctly indicates the limitations of the system capabilities, so you can see how this might confuse consumers,” he says.
Looking to other manufacturers, Volvo V60’s system is named Pilot Assist, which Schram feels accurately portrays the system’s functionality. “At least some of the manufacturers have got it right, but then all you have to do is look at ProPilot from Nissan, which is another inappropriate name,” Schram says. “Strangely enough, it is called Pro Pilot Assist in the US. Nissan Europe did consider after our last campaign in 2018 to change their name, but unfortunately, we’re still waiting.”
Euro NCAP’s key conclusions from the 2020 tests include:
- No car on the market today is able to provide full autonomy.
- Many manufacturers are using misleading marketing or ‘jargon’ to refer to its AD systems, causing consumers to believe they are more autonomous than they really are.
- Similarly to 2018, the driver assist systems available today are able to support the driver but are not designed to take complete control in all critical situations.
- The driver needs to remain fully responsible for safe driving, using the AD systems for assistance only.
- Used correctly, this technology can help the driver to maintain a safe distance, speed and to stay within the lane.
- All vehicles lack Direct Driver Monitoring systems to really understand whether the driver is Engaged. For higher performance in Vehicle Assistance, it is crucial that DMS is part of a system.
The 2018 campaign concluded that self-driving cars are still far from being available and we can see that the story is very similar in 2020. “What we found was the driver assist systems available are able to support the driver but are not at all capable nor designed to take complete control in all critical situations,” explains van Ratingen.
When it comes to AD, Euro NCAP’s guidance is clear – every system on the market today is an assistance system, therefore this should be made clear to consumers. As van Ratingen explains, “There is no such thing as an autonomous driving personal vehicle yet. The driver needs to be responsible and engaged at all times behind the wheel, and the assistance systems are there to do just that – assist.”
Euro NCAP’s goal and reason for launching this campaign is to develop a narrative and explain to the consumer what they can and cannot expect from AD systems. “We don’t have hard and fast evidence across all brands and markets to show that these AD systems are beneficial from the point of view of casualty reduction,” van Ratingen continues. “We are still learning how these systems are currently designed, what their physical limitations are, and what safety benefit can be expected from drivers using them.”
The best systems tested by Euro NCAP were those which gave the driver, through their naming, description and performance on the road, a realistic sense of the assistance they could offer without ever lulling the driver into a sense of complacency.
“All of the cars were found again to be seriously lacking in much of our more difficult test criteria meaning there is a long way to go before cars can cope with situations that humans intuitively respond to,” says Schram. “Drivers should not think they can rely on the technology completely, even for short periods.”
There is little doubt that, one day, fully autonomous cars will bring the benefits expected of them and, with driver error taken out of the equation, will offer drivers a far safer experience. “But until we reach that stage, Euro NCAP will continue to highlight the importance of a balance between what a vehicle is truly capable of and what the driver perceives the vehicle is capable of,” says van Ratingen.
For now, AD technology is rapidly developing and van Ratingen says that for this reason, Euro NCAP’s work is never done. “We will continue to further develop these protocols into the next stages of autonomy to ensure manufacturers are developing their systems with the right things in mind and that their capabilities are made clear to the people buying them.”
SOURCE: Euro NCAP
