Continental is conducting intensive simulator- and road-based testing to research the user acceptance of new displays and control concepts

Holistic human–machine interface with multimodal elements ensures communication with and without words

Babenhausen, June 8, 2017. Letting go of the steering wheel and reading a book in peace and quiet while the car chauffeurs you through the rush-hour traffic after a long day at work: both vehicle technology and the human factor play a key role to realize this dream. To find the right solutions to the upcoming challenges that automotive manufacturers will be facing in this area, the technology company Continental is intensively researching and testing a range of display and control concepts for automated driving.

Diverse questions concerning car–driver interaction

As the car takes control of more and more driving tasks, the human gradually transitions from the role of driver to observer. This raises entirely new questions and challenges concerning the interaction between humans and vehicles. To what extent do drivers have to know what is happening on the road around and ahead of them? What is the best way for cars to prompt drivers to intervene? How do drivers interact with cars in order to perform certain maneuvers? And what kind of non-driving-related activities will drivers be able to do while their car is driving itself? The wishes of drivers, their safety, and of course the safety of all road users around them are of top priority here.

“On the road towards fully automated driving, our biggest challenge will be the new role of drivers and the resulting new needs and requirements,” explains Dr. Karsten Michels, who heads System and Advanced Development in the Interior division at Continental.

Up to now, drivers are solely occupied with the task of driving; in the future, however, they will become critical users and monitors in the cockpit. To meet this challenge, they have to know at all times how the vehicle is behaving and the vehicle’s current driving mode.

“Transparency and an awareness of the current situation are our watchwords when it comes to developing new concepts for a holistic human–machine dialog. Only in this way can drivers place their trust in fully automated driving systems,” says Michels. “What’s important here is the appropriate degree of trust: If drivers overestimate the system’s capabilities, in extreme cases they might not take back control of the vehicle at the end of an automated driving phase even if they are required to do so,” says Guido Meier-Arendt, Principal Technical Expert human-machine interface in Continental’s Interior division.

The technical components of interaction concepts for automated driving not only have to be interlinked dynamically and in a multimodal manner, but also have to be intuitive to use and capable of taking into account the driver’s current state of readiness at all times. For example, drivers might need more time and assistance when taking back control of the vehicle if they have been engaged in non-driving-related activities over a long period of automated driving.

Particularly for the sensitive transition process at the end of an automated driving phase, Continental is already working on innovative concepts that require extensive testing – both in the lab and on the road – before they can be used for real. Because it doesn’t matter how good the technology is, human behavior will always play a critical role in the transition to fully automated driving (level 5). Extensive testing performed by test subjects, who are not trained test drivers, enables a realistic estimation and evaluation of the technology.

For real-life driving out on the roads, Continental uses a method that gives the test subjects the feeling that they are working with a highly automated system. The extensively prepared test vehicle uses numerous sensors to record the behavior of the driver and analyze their reactions. The test subject sits behind a dummy steering wheel on the left of the vehicle; sitting at the actual steering wheel on the right is a specially trained driver.

During manual driving phases, the trained driver is informed of the test subject’s actions via a combiner head-up display and converts these to actual maneuvers. During automated driving phases, the trained driver takes full control of the steering wheel, creating the illusion of highly automated driving. This method means that, even today, certain elements of the human–machine interface for automated driving can be tested in real-life driving situations. This is because, precisely due to the interaction required, highly automated test drives on public roads, without trained test drivers, can put test subjects at risk. The results are helping the developers at Continental to formulate the necessary control algorithms for fully automated driving.

The driving simulator – Continental’s “ergonomics lab” – allows the researchers to observe and analyze their test subjects in even greater detail, for example by measuring their heart rates or by precisely recording their eye movements. This data is gathered without any outside interference and enables the researchers to establish whether the test subject is comfortable or whether a particular situation is causing them stress. The lab environment also means that the test subjects can be put under much more pressure, for example to test their response to hazardous situations and critical maneuvers that would not be possible on real-life roads.

Elements of holistic human–machine interaction

Already today, Continental is employing multimodal – that is, multisensory – solutions to transfer information to drivers. Visual presentations via a range of displays in the cockpit, head-up displays, or changes in the color of LED light strips are supported by audio signals that, for example, inform drivers about the end of an automated driving phase or alert drivers via a specific audio signal. If the interior cameras establish that the driver is still not paying attention, haptic elements are activated – for example, the seat starts to vibrate or the seat-belt tensioner tightens.

To control automated driving phases, Continental has also developed a control element designed to rule out the risk of confusion regarding the drive modes and to further enhance transparency. “With our test series and products, we are continuously improving the holistic human–machine interface to ensure seamless communication and to lay the foundations for ensuring that drivers know exactly what their responsibilities are at any given time and what the system is currently doing,” says Meier-Arendt.

Continental is developing and producing the necessary components and systems for automated driving worldwide – in the U.S.A. as well as in Japan, China, and Europe. The engineers involved are working on six key elements: sensor technology, cluster connectivity, human-machine dialog, system architecture, reliability and the acceptance of automated driving.