Advanced driver assistance systems (ADAS) of various complexity are becoming increasingly common in the automotive industry, and this has made thorough scrutiny imperative. In the US, the New Car Assessment Program (NCAP)—a consumer guide that rates cars based on their safety profile—has been testing ADAS since 2009 and continues to examine how its metrics can be updated as the technology itself evolves. Other regional NCAPs, including China and Europe, are now similarly committed.
SAE Level 2 ADAS currently includes core safety functions like impending collision detection and prevention. However, some industry analysts contend that many of these partial autonomy systems currently offer limited to no real-world safety benefits. In fact, the US National Highway Traffic Safety Administration attributed 367 traffic collisions to ADAS between July 2021 and March 2022.
Meanwhile, software-defined vehicle architectures enable automakers to expand autonomy beyond core safety to more advanced functions, including lane keeping assist and adaptive cruise control. The danger is that widening the operating domain while even relatively simple functions are far from perfect risks alienating industry stakeholders, whose trust has been repeatedly shaken by bad press. To rebuild and expand that trust, reliable testing has never been more important.
Gathering ground truth data
Andrew Pick, Director of Track Test Systems at AB Dynamics, tells Automotive World that ADAS testing cannot be conducted on public roads because of the inherent safety risk. Nevertheless, OEMs need “ground truth data” upon which they can validate system integrity. As such, AB Dynamics manufactures test objects for proving-ground tracks based on common real-world road hazards, including pedestrians and motorcycles. However, in a rapidly evolving market, this just provides the baseline.
“The main difficulty automakers face is the sheer volume of tests, combinations, and permutations they need work through,” says Pick. This applies to every single model equipped with ADAS, and each regional market will have its own specific test requirements. To manage this complexity, AB Dynamics has been developing an extensive protocol library in an attempt to simplify and automate what is otherwise a costly and time-consuming process. A June 2024 update to its Track Applications Suite added 237 tests for China NCAP, 60 for the EU General Safety Regulation, and 120 United Nations Economic Commission for Europe protocols. “In total, our database now spans more than 1,000 test cases.”
Thanks to artificial intelligence and machine learning, sensors are becoming more discerning, meaning tests must be similarly sophisticated. Repeatability and realism, continues Pick, are the crucial goals. The protocol database factors in information both about the vehicle tested—for example, specific dimensions or layout geometry that could affect response—and the objects used. “If the customer can define exactly what they want, we can make it happen reliably on the proving ground. The realism comes from the test objects themselves.” AB Dynamics’ test objects have identical radar signatures and comparable movement to what they replicate: “That includes the motion of pedestrians’ walking legs or turning heads,” he explains.
The value of automated testing
During a test, vehicles are controlled by AB Dynamics’ range of driving robots—steering, pedal, and gear change automation—instead of human drivers. An automaker’s selected test protocol then defines the speed and relative positioning of the vehicle and other objects during the scenario at a coordinated ‘point of interest’ on the track. This is where the response of the ADAS feature in question will be assessed. Aspects of the scenario can be adapted by automakers in real time according to their requirements.
“Because we’re in complete control of the ecosystem, we can also manage the results by post-processing test data to compare how well ADAS performed against KPIs,” says Pick Typically, this means measuring the time gap between a collision warning and the system taking action, such as applying the brakes, or recording the impact speed if a collision occurs. “Sometimes ADAS’ goal is to mitigate a collision rather than prevent one,” he clarifies.
AB Dynamics claims that the repeatability of automated ADAS testing using driving robots is much superior to human testing. “People just aren’t capable of the same consistency: a driver could get the pathway, braking rate or speed right during a test, but getting all three perfect simultaneously is really tough,” Pick states. “A single successful test run could take six or more attempts.” Comparatively, robotic drivers are able to perform strings of consecutive test protocols flawlessly and in a matter of minutes.
Advancing ADAS with software
As vehicle autonomy advances beyond Level 2, the number of variables in a test scenario increase significantly. In addition to coordinating test objects and providing a library of preconfigured test protocols, AB Dynamics’ Track Applications Suite includes a Scenario Generator that can import previous test data and reshape it in simulation into any conceivable scenario. Pick notes that this tool can be used to establish a virtual baseline for new ADAS functions in a safe digital environment. It also provides a data foundation for validating those simulated results in future real-world testing.
While Level 4 autonomous driving systems exist in a limited capacity, ADAS technology is far from reaching its peak in the mass market. According to MarketsandMarkets, global ADAS hardware production is expected to almost double from 334 million units per annum in 2024 to 655 million units in 2030. At the same time, mainstream automakers like Mercedes-Benz and BMW are both exhibiting Level 3 systems—Drive Pilot and Personal Pilot, respectively—and validating safety is only going to become more complicated. Pick emphasises that the testing equipment industry must be prepared to move in step with these advances.
“New ADAS functionalities are increasingly defined by software,” he observes. “As acceptance for these technologies builds, the limit of their previous operating domain can be expanded.” Companies like AB Dynamics will have a vital role to play in proving to regulators and consumers that this advancement does not come at the cost of safety. Pick suggests that future ADAS test scenarios could include objects in blind spots and upcoming hazards obscured by blocked sensors.
Software-defined ADAS could unlock billions of dollars in added value. However, with a limited pool of talent from which they can draw, automakers may need to build ecosystems that incorporate software developers from Big Tech. While this could enable faster R&D, these workers will need to be familiarised with the core parameters and data around which they can iterate new ADAS features. In this way, Pick concludes that the testing industry will be crucial for bridging automotive and tech. “Software developers like to be agile, and automakers want to validate vehicles as quickly as possible. We can help them both work in exactly the way they want.”
