On-road and track tests are helping IIHS craft a consumer ratings program for advanced driver assistance systems. Evaluations of adaptive cruise control and active lane-keeping show variable performance in typical driving situations, such as approaching stopped vehicles and negotiating hills and curves. The early results underscore the fact that today’s systems aren’t robust substitutes for human drivers.
One of the questions researchers looked to answer is, do the systems handle driving tasks as humans would? Not always, tests showed. When they didn’t perform as expected, the outcomes ranged from the irksome, such as too-cautious braking, to the dangerous, for example, veering toward the shoulder if sensors couldn’t detect lane lines.
Adaptive cruise control (ACC) maintains a set speed and following distance from the vehicle in front. It is designed to slow for cars ahead and can come to a full stop but may not react to already-stopped vehicles. ACC doesn’t react to traffic signals or other traffic controls. Active lane-keeping provides sustained steering input to keep the vehicle within its lane, but drivers must continue to hold the wheel.
On SAE International’s scale of zero autonomy to Level 5 full autonomy, the combination of ACC and active lane-keeping is Level 2. They can assist with steering, speed control and following distance, but the human driver is still in charge and must stay on task.
“The new tests are an outgrowth of our research on Level 2 autonomy,” says Jessica Jermakian, IIHS senior research engineer. “We zeroed in on situations our staff have identified as areas of concern during test drives with Level 2 systems, then used that feedback to develop road and track scenarios to compare vehicles.”
The 2017 BMW 5-series with “Driving Assistant Plus,” 2017 Mercedes-Benz E-Class with “Drive Pilot,” 2018 Tesla Model 3 and 2016 Model S with “Autopilot” (software versions 8.1 and 7.1, respectively) and 2018 Volvo S90 with “Pilot Assist” were evaluated. All five have automatic emergency braking systems rated superior by IIHS.
Adaptive cruise control
Engineers evaluated ACC systems in four different series of track tests to see how they handle stopped lead vehicles and lead vehicles exiting the lane, and how the systems accelerate and decelerate.
One series involved driving at 31 mph toward a stationary vehicle target with ACC off and autobrake turned on to evaluate autobrake performance. Only the two Teslas hit the stationary target in this test.
The same test was repeated with ACC engaged and set to close, middle and far following distance in multiple runs.
With ACC active, the 5-series, E-Class, Model 3 and Model S braked earlier and gentler than with emergency braking and still avoided the target. The cars slowed with relatively gradual decelerations of 0.2-0.3 gs, braking in the same manner no matter the distance setting. Braking before impact was earlier for the Teslas than for the 5 series and E-Class.
The S90 braked more abruptly than the other models with ACC active, similar to its autobrake performance. In the ACC test, the S90 braked at a forceful 1.1g, just 1.1 seconds before impact to avoid the collision.
A third scenario involved following a lead vehicle that slows down to a stop and then accelerates. Every ACC system decelerated smoothly in this test.
A fourth scenario involved the test vehicle following a lead vehicle, which then changed lanes to reveal a stationary inflatable target vehicle in the path ahead when the time to collision was about 4.3 seconds.
None of the vehicles crashed into the target, and the 5 series, E-Class and Teslas all braked earlier and gentler than the S90, similar to the active ACC test.
Track tests are good for evaluating capability and performance in a controlled environment but not for assessing performance in traffic. Under ideal conditions, advanced driver assistance systems may function better than they do in more complex driving situations.
A case in point is the stopped-vehicle ACC tests. On the track, the 5 series, E-Class and Teslas braked to avoid the target vehicle. This was the case even though the owner’s manuals for all the test vehicles warn that ACC may not brake when it encounters vehicles that are already stopped when they come into sensor range.
Out on the road, engineers noted instances in which each vehicle except the Model 3 failed to respond to stopped vehicles ahead.
Jermakian recounts her experience with the E-Class on U.S. 33 near the IIHS-HLDI Vehicle Research Center (VRC). Traveling about 55 mph with ACC and active lane-keeping engaged but not following a lead vehicle, the E-Class system briefly detected a pickup truck stopped at a traffic light ahead but promptly lost sight of it and continued at speed until she hit the brakes.
“At IIHS we are coached to intervene without warning, but other drivers might not be as vigilant,” Jermakian says. “ACC systems require drivers to pay attention to what the vehicle is doing at all times and be ready to brake manually.”
Unnecessary or overly cautious braking is an issue IIHS noted in the Model 3. In 180 miles, the car unexpectedly slowed down 12 times, seven of which coincided with tree shadows on the road. The others were for oncoming vehicles in another lane or vehicles crossing the road far ahead.
“The braking events we observed didn’t create unsafe conditions because the decelerations were mild and short enough that the vehicle didn’t slow too much. However, unnecessary braking could pose crash risks in heavy traffic, especially if it’s more forceful,” Jermakian says.
“Plus, drivers who feel that their car brakes erratically may choose not to use adaptive cruise control and would miss out on any safety benefit from the system.”
The outlook is promising for the potential safety benefits of ACC. The technology is often bundled with forward collision warning and autobrake, and research by IIHS and HLDI has found crash-reduction benefits for these systems combined. A federally sponsored study found that drivers using ACC have longer, safer following distances than drivers who don’t use ACC. Still, IIHS tests indicate that current ACC systems aren’t ready to handle speed control in all traffic situations.
Active lane-keeping
Engineers focused on two situations that challenge active lane-keeping systems — curves and hills — in tests on open roads with no other vehicles around. They also observed how the systems performed in traffic.
All five systems provide steering assistance that centers the vehicle within clearly marked lanes. They also may use a lead vehicle as a guide when traveling at lower speeds or when the lead vehicle is blocking the system’s view of the lane markers ahead.
To test active lane-keeping on curves, engineers conducted six trials with each vehicle on three different sections of road with radii ranging from 1,300 to 2,000 feet.
Only the Model 3 stayed within the lane on all 18 trials. The Model S was similar but overcorrected on one curve, causing it to cross the line on the inside of the curve in one trial. None of the other systems tested provided enough steering input on their own to consistently stay in their lane, often requiring the driver to provide additional steering to successfully navigate the curve.
The E-Class stayed within the lane in 9 of 17 runs and strayed to the lane marker in five trials. The system disengaged itself in one trial and crossed the line in two. The 5 series stayed within the lane in 3 of 16 trials and was more likely to disengage than steer outside the lane. The S90 stayed in the lane in 9 of 17 runs and crossed the lane line in eight runs.
When trying out new vehicles in hilly Central Virginia, home to the VRC, engineers noted early on that advanced driver assistance systems that rely on seeing road markings to keep vehicles in their lanes were sometimes flummoxed by hills. As a vehicle crests a hill, the lane markers on the road beyond are obscured.
For the on-road tests, engineers mapped out a course that included three hills with different slopes. Drivers made six trial runs on each hill in each vehicle.
The E-Class stayed in its lane in 15 of 18 trials and on the line in one trial, continuously providing steering support without erratic moves when lane lines weren’t visible. The Model 3 also stayed in the lane in all but one trial, when it hugged the line.
In contrast, the 5-series, Model S and S90 struggled. The 5-series steered toward or across the lane line regularly, requiring drivers to override the steering support to get it back on track. Sometimes the car disengaged steering assistance on its own. The car failed to stay in the lane on all 14 valid trials.
The Model S was errant in the hill tests, staying in the lane in 5 of 18 trials. When cresting hills, the Model S swerved left and right until it determined the correct place in the lane, jolting test drivers. It rarely warned them to take over as it hunted for the lane center. The car regularly veered into the adjacent lanes or onto the shoulder.
When drivers intervened to avoid potential trouble, the active lane-keeping system disengaged. Steering assistance only resumed after drivers re-engaged Autopilot.
The S90 stayed in the lane in 9 of 16 trials. The car crossed the lane line in two trials and in four trials disengaged steering assistance when it crested hills but automatically re-engaged when the system once again detected the markings.
One issue drivers noted among some of the vehicles was a propensity to follow a lead vehicle into the exit lane in slow-moving traffic, even though the driver intended to stay the course. When a car is traveling too slow to track lane lines, active lane-keeping systems use the vehicle in front as a guide. If the lead vehicle exits, the trailing car might, too.
The evidence for safety benefits of active lane-keeping systems isn’t as pronounced as for ACC. Still, the potential to prevent crashes and save lives is large. IIHS research shows that preventing lane-departure crashes could save nearly 8,000 lives in a typical year (see “New estimates of benefits of crash avoidance features on passenger vehicles,” May 20, 2010). Lane-departure warning systems are associated with an 11 percent reduction in the rates of single-vehicle, sideswipe and head-on crashes of all severities and a 21 percent reduction in the rates of injury crashes of the same types (see “Stay within the lines: Lane departure warning, blind spot detection help drivers avoid trouble,” Aug. 23, 2017).
More research before ratings
IIHS continues to run track and on-road tests as it moves toward a consumer rating system for advanced driver assistance systems. Apart from questions about whether the systems perform as drivers expect, one of the many factors to consider is how much of the driving task can safely be handed over to technology without drivers checking out altogether?
“Designers are struggling with trade-offs inherent in automated assistance,” says David Zuby, IIHS chief research officer. “If they limit functionality to keep drivers engaged, they risk a backlash that the systems are too rudimentary. If the systems seem too capable, then drivers may not give them the attention required to use them safely.”
Real-world crashes involving vehicles with Level 2 automation demonstrate the matter isn’t settled.
“We’re not ready to say yet which company has the safest implementation of Level 2 driver assistance, but it’s important to note that none of these vehicles is capable of driving safely on its own,” Zuby says. “A production autonomous vehicle that can go anywhere, anytime isn’t available at your local car dealer and won’t be for quite some time. We aren’t there yet.”
How active lane-keeping systems performed in IIHS road tests on 3 curves and 3 hills
| Number of times vehicle | ||||||||
|---|---|---|---|---|---|---|---|---|
 went over line |
 touched line |
 system disengaged |
 stayed within lane |
|||||
| on curves | on hills | on curves | on hills | on curves | on hills | on curves | on hills | |
| BMW 5 series | 3 | 6 | 1 | 1 | 9 | 7 | 3 | 0 |
| Mercedes-Benz E-Class | 2 | 1 | 5 | 1 | 1 | 1 | 9 | 15 |
| Tesla Model 3 | 0 | 0 | 0 | 1 | 0 | 0 | 18 | 17 |
| Tesla Model S | 1 | 12 | 0 | 1 | 0 | 0 | 17 | 5 |
| Volvo S90 | 8 | 2 | 0 | 1 | 0 | 4 | 9 | 9 |
