Can Kinect replace the white cane?

Technology has offered tantalizing prospects to replace the white cane—the traditional low-tech accessory that people who are blind use to detect obstacles in their path. Researchers have experimented with systems based on sonar and on lasers, and while some of these have shown promise, they all require highly specialized and often costly hardware. Now, a multinational team of researchers—Nadia Kanwal of Pakistan, Erkan Bostanci of Turkey, and Keith Currie and Adrian F. Clark, both from the UK—has developed a prototype system that uses the Kinect sensor, a battery, and a laptop PC, a readily available and comparatively inexpensive combination.

The prototype system is strapped to the user’s body; for the production model, the researchers envision a smaller apparatus.

The new system uses the Kinect sensor’s infrared depth sensing capabilities and its RGB video camera to provide information on objects and their distance from a person who is blind or has low vision. It works like this: once its infrared and video sensors have been calibrated, the Kinect sensor’s RGB camera detects the corners of objects in its field of view. This data is correlated with data from the depth sensor, and the result provides information about the presence of an obstacle ahead and its distance from the user.

Lead researcher Kanwal, an assistant professor of computer science at Lahore College for Women University, has spent years exploring the use of adaptive technologies to assist people with special needs. Speaking about the potential value of the Kinect-based system, she says, “A blind person wants to adapt anything that makes them more independent in mobility; therefore, a system that can identify and warn about potential hazards would be much appreciated.”

The Kinect-based system checks for two types of obstacles: those, such as walls, that can completely block a person’s progress, and smaller ones, such as chairs, that can be avoided by changing direction. When an obstruction is detected, the system warns the user to either stop (so he doesn’t run into a wall) or move right or left (so she can navigate safely around a piece of furniture).

The system is carried on the user’s body; much of the equipment is transported in a backpack or shoulder bag, while the Kinect sensor is strapped to the front of the user’s body. While the researchers acknowledge that this setup is cumbersome, they envision a smaller, more portable production model.

The research team tested the prototype with a blindfolded, normally-sighted subject and then with a subject who has been blind since birth. Both participants successfully navigated through a test environment filled with various objects. Interestingly, the blind participant felt that the system would be most useful when combined with the traditional white cane, whose sweeping action can detect smaller objects and changes in the walking surface that the Kinect-based system misses. Conversely, the Kinect system can detect large objects sooner and thus can help the user traverse more smoothly through a crowded space.

The researchers used the Kinect for Xbox 360 in their prototype, but are anxious to incorporate the latest sensor, the Kinect for Xbox One, which offers improvements in both depth sensing video resolution.

Although the researchers have more work to do on the system before it is production ready, the ability to use Kinect for Windows to effectively and inexpensively combine video and depth information offers great potential to help users move more freely through their environment. It might not replace the white cane, but it could go a long way toward complementing it and making the world more navigable for people who are blind.