Multimodalities of the ASSAM Walking Assistant

Usability research by Bartiméus and the Accessibility Foundation

Multimodal interaction, communication and navigation can help blind and visually impaired people to function better in daily life and find their way around in familiar and unfamiliar surroundings. Multimodality is understood as the optional presentation of the same information content in more than one sensory mode. The principles of multimodality can be used to compensate the lack of one sensory mode in a person by presenting the information in another mode.

The ASSAM eWalker with the Walking Assistant aims to compensate for declining physical and cognitive capabilities of elderly persons. How? By its multimodalities. These multimodalities consist of, among others: a laser scanner, a tablet, a tactile actuator, and an iHandlebar . The Walking Assistance by the electric iWheels is a very promising innovation, because it provides physical support in addition to the travel and navigation aid software. This way, it can be used by people with both visual and mobility impairments.

The Walking Assistant has been developed as part of the European project Assistance for Safe Mobility (ASSAM), and can help blind people to find their way in an indoor and outdoor environment. Our blind test users include Hans (63), Roel (64) and Bianca (39). They have tested the Walking Assistant during a usability research session at Bartiméus Accessibility Foundation in Utrecht, The Netherlands. With their help we want to find out if the multimodal features of the Walking Assistance prototype are sufficient and ready to use for blind and visually impaired people. What do our test users think about the multimodalities of the Walking Assistance? What is working and what needs to be improved?

Roel is positive about the Walking Assistance: “I have some mobility issues and I think the ASSAM Walking Assistant can help me to move around. If I get tired from walking, I can use the walker to take some rest.”

Roel likes the fact that the Walking Assistant compensates for declining physical capabilities. In daily life, he finds it difficult to walk without the help of a walking stick. The Walking Assistant can support him to keep in balance. Like the other test users, Roel is blind. He uses his walking cane and a navigation app on his iPhone to move around independently. What are the features of the ASSAM Walking Assistant that are useful for him and the other test users? To answer this question, we have to look at the single features of the Walking Assistant one by one. With its multimodalities, the assistance system provides physical assistance, obstacle avoidance, navigation and security aid.

Physical assistance

In the manual mode, the Walking Assistant behaves like a normal walker and provides physical assistance. It supports users when walking in out- and indoor environments. For a person like Roel, the Walking Assistant can be a solution. Bianca and Hans do not need a walker yet and would not trade their current aids (cane or dog) for the walker, but they do say that they can definitely use assistance with navigation and obstacle detection in daily life. As it is well known that both visual impairments and mobility impairments increase substantially with age, Bianca and Hans agree that they find that the Walking Assistant would be useful for them in case they face declining physical capabilities. We think there is room for more research into the combination between the blind cane or the guide dog and a walker.

Obstacle avoidance

The laser scanner of the ASSAM Walking Assistant detects obstacles and works together with the electric wheels of the walker by assuming control of the steering to navigate around obstacles. The iWheels provide information about the walkers’ position, speed, direction, tilt and more. The wheels are also responsible for power management of the walker.

The users are very positive about the possibility to detect and avoid obstacles with the Walking Assistant. Hans, Bianca and Roel say that, occasionally, they all bump into objects or persons when they move around, especially in crowded places with a lot of noise. They think the obstacle avoidance can definitely help them in these situations.

Even though Hans liked to experience the walkers’ obstacle avoidance, he emphasized the importance and power of his own hearing. “With my own ears I can hear where walls and obstacles are. In quiet places I don’t really need an automatic obstacle avoidance.” He thinks he would use the walker in outdoor situations and not really in indoor environments. Also Bianca and Roel had some advice to take in mind regarding the obstacle avoidance of the walker:

Bianca prefers to get a signal about the presence of obstacles instead of the automatic steering and navigating around obstacles. Although this is an option, she gets the feeling she loses control when the Walking Assistant handles the situation on its own.

The laser scanner on the eWalker only seems to detect obstacles that are exactly in front of the scanner. The blind test users suggest a scanner that can detect all obstacles in their walking path. This could be covered by a newer version of the scanner.

The laser scanner does not detect the glass of windows. The indoor site where we tested the Walking Assistant was full of windows. As a result the test users were bumping into windows with the walker.

Navigation Aid

The Navigation Aid software of the eWalker gives cognitive assistance for declining visual and mental capabilities. Different features help to navigate and transform the instructions into nonvisual information. This is achieved by fusing odometry, GPS and map information. Feedback on the environment is given via an audio interface and vibrating handlebars. The vibration element inside the iHandlebars can be used for notifications to the user.

During the users’ research the navigation application and the vibrating handlebars did not work yet. We installed a small vibrating element ourselves to start the discussion with the test users. Hans, Roel and Bianca are dependent on navigation in their daily life. They use apps like NAVIGON and BlindSquare to navigate and locate themselves. They all emphasised the importance of navigation possibilities for blind and visually impaired people. For them navigation is an essential feature of the ASSAM Walking Assistant. They have some ideas about it:

Accurate self-localization is essential. By this they do not only mean their position, but the possibility to receive information about where they are and maybe even help to tell them what they could do there.

The navigation aid needs to be very precise and give clear instructions about the route and environment.

Although this is an option, Hans warned for a signal and notification overload by presenting navigation instructions by speech and vibrations at the same time.

The test users all preferred to get instructions by an audio interface using speech instructions and not by the vibrating handlebars. They think the vibrating handlebars are better suited for sending warnings about objects. In loud environments this could be different. More research would be good.

Literature also has some useful ideas about presenting multimodal information:

Use multimodal interaction to allow users to interact with a system following their individual preferences and suited to their special needs.

Use multimodal presentation of information to allow users with different preferences and abilities to interpret data in their preferred way.

Security Assistance

In case of an emergency situation the ASSAM Walking Assistant is able to make a connection with a care centre. During the usability research we did not test this possibility. Bianca, Roel and Hans agreed that this feature can be very useful for the target group, elderly persons with declining physical and cognitive capabilities, but that they would not use the security assistance at this point in their life. They said that most likely in a case of an emergency, they would ask bystanders for help instead of making a digital connection with a care centre.

Promising

The multimodal features of the ASSAM Walking Assistant are very promising, but not yet fully ready to use for blind and visually impaired people. To make the Walking Assistant accessible for the blind and visually impaired, some adjustments could be made and some additional research would be useful. We concluded that currently, it would be difficult for people with visual disabilities to start all the equipment like the tablet. The apps are not yet made accessible for blind people. An optional voice-prompt could lead a blind person through the process. With the right technological adjustments, the ASSAM Walking Assistant can definitely become a great aid for people with both physical and visual impairments.