Virtual Assistant in Development Could Help Alzheimer’s Patients at Home

By Tim Regan | August 31, 2017

A new emotionally intelligenttool from computer scientists at the University of Waterloo in Waterloo, Canada, could one day help seniors with Alzheimer’s disease complete everyday tasks throughout their homes or nursing facilities, enabling them to age at home longer.

The virtual assistant, Act@Home, helps someone with Alzheimer’s or dementia perform an activity of daily living (ADL), such as getting dressed, taking medication or making a cup of coffee.

The technology is one of the latest in a wave of initiatives pushing robotics in the home to aid seniors and keep them out of higher-acuity settings.Act@Home was funded by the American Alzheimer’s Association and the Age-Well Network of Centres of Excellence, an innovation and technology program paid for in part by the Canadian government.

Emotional intelligence

Act@Home at its core is a kind of software that detects whether someone with Alzheimer’s needs help. The virtual assistant works by studying input from cameras, microphones or other sensors placed around the home to detect a senior’s emotional cues such as facial expressions, posture or tone of voice.

One Act@Home prototype involves a wall-mounted camera that watches a senior as they wash their hands.

For example, a senior with dementia might know that in order to wash up, they need to stand in front of the sink. But the senior might then forget what they’re doing or why they’re doing it, lose motivation, start over from the beginning or just stop in the middle of washing.

In that prototype, if Act@Home detects the senior is having trouble or acting frustrated, it activates a video screen on the wall that displays a virtual woman who verbally reminds them what they’re doing and how to do it.

“When they stop making progress or they’re unable to get something done, we can then automatically provide them with some kind of assistance,” Jesse Hoey, a professor at the David R. Cheriton School of Computer Science at Waterloo and lead researcher on the project, tells Home Health Care News. “There’s a lot of context and modeling that has to go into it for the whole thing to come together.”

Over time, caregivers could install more microphones, cameras or sensors on other surfacesrelated to other ADLs that a senior might have trouble with, like finding the bathroom in the middle of the night.

“As the disease progresses, they might start having trouble with other activities of daily living, and then the system could kind of grow organically,” Hoey says. “So, it starts monitoring the person in other parts of the home.”

Though the hand-washing prototype uses a digital woman who talks to seniors, other prompts don’t need to involve a human element at all, Hoey says. Reminders could also arrive in the form of a set of lights turning on or another signal — whatever works best to get the message across.

Figuring out what motivates people with dementia is hard, which is why Hoey and his fellow researchers interviewed seniors with Alzheimer’s to come up with the kinds of reminders they should use. They also ran trials of the hand-washing prototype to refine it to its current-day form.

But the prompts still need some work before they’re suitable for everyone with dementia, Hoey says.

“There are barriers to widespread deployment that come down to…the inability to align emotionally with the people [who] are trying to use them,” he says. “[Some of] the people [who] are trying to use [the prompts] don’t really understand what they are, and don’t really have a way of building a mental model of what this thing is, why it’s there and what it’s doing.”

Once that gets figured out, Act@Home could be ready for final design, mass production and sales as soon as a few years from now, Hoey says.

On the cloud, in the home

The assistant is able to run on the cloud, meaning it can be used on virtually any device with an internet connection. One of the researchers’ prototypes even works on a Roomba robotic vacuum that’s been retrofitted with a fiberglass body and a screen on top to display information.

“We have another arm of the project which is looking at building mobile robots that would sort of follow the person around,” Hoey says. “Rather than putting the sensors in the environment or on the walls, the sensors are on a robot that are able to move and go with the person.”

Still, more research is planned to better integrate the system into home networks. Age-Well is funded through 2020, meaning there’s more time to do that.

“There are lots of smart home possibilities out there already,” Hoey says. “I think you’ll see probably in the next couple of years these kind of solutions being integrated into the lives of people living with Alzheimer’s.”

Hoey and his team of researchers aren’t the only ones experimenting with virtual assistant technology to aid seniors living at home. Libertana Home Health, a Beverly Hills, California-based agency, recently launched a pilot program using Amazon’s Echo Dot device.