This project is submitted for

Description

People who have totally lost the use of their muscles cannot operate their own wheelchairs.

The Eyedrivomatic system takes advantage of existing eye tracking technology to allow users to drive their chairs again - using only their eyes. It's a low cost, open source way to give mobility back to people who thought they had lost it forever.

Details

The Problem

Spinal
injury is the most common cause of quadriplegia, though there are
many diseases that can produce the same result - inability to produce
controlled movement in any of your limbs or head. For people in this
situation independent control of their wheelchairs is not possible –
the technology just isn't available. Independent mobility increases
quality of life significantly, and it's loss is keenly felt by those
robbed of it, through accident or disease.

The
scale of the problem is enormous. ALS is responsible for only a
tiny fraction of the people in this situation, yet there are five
thousand ALS sufferers at any one time in the UK alone, and twenty
thousand in the US. Most of them will end up sat immobile in their
wheelchairs, unable to move on their own. Eyedrivomatic could help
most of them.

The Concept

To create an inexpensive and accessible way for
people with motorised wheelchairs to take control of their mobility
via eye movement, using open source methods as much as possible.

Our team

Patrick Joyce - Team captain

Steve Evans - Test pilot

David Hopkinson - Film maker

Cody Barnes - Software Developer

Steve and I both have Motor Neuron Disease, or ALS as it's also known. We both have electric wheelchairs, but while I still have some remaining movement in my fingers, and am just able to operate mine, Steve only has his eyes left, and until Eyedrivomatic, had to rely on his carer to operate his.

Background

It is usual for people with MND/ALS in the UK to
be provided with electric wheelchairs loaned by the health service,
for as long as we need them. Steve and I have wheelchairs, and we
both also have eyegaze equipment (which is becoming increasingly more
available). At the moment it is not possible to operate one with the
other. Eyegaze technology is only intended for operating a computer,
not the chair.

The Original Idea

As I don't actually own either the wheelchair or
the eyegaze, my idea was to make something that would interface with
the user's chair mounted computer, and physically move the joystick.
Crucially this would mean not making any modifications to the loaned
hardware, and could work with any wheelchair and eyegaze combination.

The solution I came up with is called
Eyedrivomatic, and is a two part system, a 'brain box', containing an
arduino and four relays, and an 'electronic hand', containing servos
to move the wheelchair's joystick.

I decided on this approach, rather than interfacing directly with the
wheelchair's electronics for several reasons. I know the protocols for
my own chair, but not all chairs use the same protocol. Also, my chair has a socket for plugging in external joysticks, but that is by no means common on other chairs. And the sockets are of a proprietary design. Interfacing directly would, on most chairs, involve physically cutting in to the wheelchair's wiring.

This is an issue because many people don’t actually own their chairs – especially in the UK where they are loaned by the health service and will be reclaimed after the user’s death.

An integrated
version of eyedrivomatic in new chairs would be brilliant, but this hacking –
yet non-interfering – approach to existing tech means the problem can be
solved cheaply and easily for those who need it as soon as possible.

This is especially important as ALS usually has a very rapid onset
and people losing their mobility need a practical and inexpensive solution as soon as
possible with what is immediately available.

Two years in development,
eyedrivomatic now works well, and is;

Easy to
use

Safe

Expandable
- the system is able to control other external devices according to
the needs of the user

Totally
open, both open hardware and open source software

Capable of
being built at home - by an unskilled person

Soon to be a manufactured product as
well

I put in a four relay module as standard equipment
in the brain box. These relays are for operating external switches of
the user's choosing. Many wheelchairs have tilt/recline...

Project Logs

The hacker/maker community has just suffered another great loss. I was informed this morning that Patrick Joyce has also passed away. Although I know Patrick from his work on the Eyedrivomatic, he was involved in several projects, several of them listed here on Hackaday.io, that so clearly demonstrated his compassion, creativity, intelligence and inability to give-up. I am very grateful to Patrick and to Steve for allowing me to work on this project with them and intend to continue with their work. There are many people out there suffering with MND or other conditions that have left them unable to move. And thanks to Patrick's vision, those people have gained a small part of their independence back. This sounds cliche, but it is true none the less... Patrick and Steve did their part to make the world a better place.

I am sorry to report that after more than a decade of fighting Motor Neurone Disease (MND), on April 14th, Steve Evans has passed away. MND is a brutal terminal disease that slowly destroys the ability of its victims to function and removes almost all self-reliance while leaving the mind fully in tact. Despite his own troubles, Steve selflessly found a way through Eyedrivomatic to help many others with similar struggles. Up to the end, Steve was busy communicating with users and researchers and coordinating manufacturing and delivery. I cannot understate how much Steve was a key part of the Eyedrivomatic team, and the loss of Steve is a major blow to the project.

There will be a short delay in the production of any new units while we figure out how to pick-up where Steve left off. However despite our loss, the Eyedrivomatic team intends to continue development and has much planned for this year.

If you are interested in helping with this project, please send me a note. We have much still to do and without Steve, we could use some help! If you could benefit from Eyedrivomatic, I would also like to hear from you. If you can't participate and have no personal use for Eyedrivomatic, but would like to share your appreciation of Steve's work, I would like to hear from you too (and would be happy to forward your notes to Steve's family).

BREAKING NEWS

Good news and bad news,
bad first. The 'powerbee executive' power pack component, that
provides power to the servos, is no longer available. Most other usb
powerbanks have internal circuitry that turns them off when power
demand drops. Unfortunately they don't switch back on when power
demand resumes. Eyedrivomatic, inconveniently, only requires power
sporadically. We are currently looking for a solution, and will post
updates here soon. Good news now. The all singing, all dancing,
stand alone pc application that we're writing, is nearly finished!
(many thanks to our genius coder – Cody Barnes). More news
soon....

I've been very lazy since the end of last years Hackaday Prize
competition. However, over the last month i've pulled myself
together, and got back on the horse, so to speak. By the end of the
competition we'd put everything up on github, thingiverse and
instructables. All the files necessary to build your own
eyedrivomatic system. But it wasn't as good as we wanted. The
hardware was ok, but the software had two issues. It was tricky to
set up, and only worked in conjunction with a commercial software
package called Grid2.

The tricky setup issue
was due to the com port number that a pc assigns to an arduino (like
the one in the brain box ). The processing app for eyedrivomatic
needs to know that number. Each user had to discover his own com port
number, and manually alter the code for the app. Simple for a geek.
Hard indeed for a normal person – daunting even. That problem
has now disappeared, thanks to our brand new Eyedrivomatic Port
Finder app. Written by W. A. Smith, and modified by Steve Thomas,
port finder finds out your com port number, and stores it in a config
file on your pc – ready for the main eyedrivomatic app to use. Its
brilliantly simple. You run port finder once only, and then the main
eyedrivomatic app runs perfectly. Simple. The new port finder app
is up our Github repository, along with the modified main
eyedrivomatic app. Its just the Grid2 issue to deal with now.
Give me a few more weeks, and eyedrivomatic will work on any pc, with
or without Grid2, simply!

Since the project's inception, the waiting list of quadriplegics wanting their own system has been growing steadily. So it was with great pleasure that i posted the first three beta systems out on Monday. One of them is headed for Jim Durfer, a US desert storm veteran with ALS. His system is yet to arrive. Jono Stenburg, a twenty four year old film maker with cerebral palsy, and Steve Thomas, an ex computer programmer, also with ALS, live here in the UK. This is a video of their first go......

Frantic? That word
doesn't begin to describe what it's been like round here lately. Despite the huge workload of preparing for the finals, in
an act of insanity, three weeks ago, I decided to completely redesign
the brain box. One of the parts in the old brain box had become hard
to get hold of. This prompted the redesign, but I ended up
redesigning all of it. Dupont cables are out, along with the LED's.
The system now uses a 4 channel relay shield on top of the arduino
uno, with a servo/sensor shield on top of that. This simplifies the
build process enormously. Everything plugs in, or gets screwed in to
a terminal block.

I also took the
opportunity to add another safety feature. Previously, when the
eyedrivomatic app was started, the servos would occasionally give a
random jerk. Which was potentially dangerous. So in the new brain box
I used one of the relays to deny power to the servos, until both the
app, and the firmware were running smoothly.

The electronic hand has
also had two revisions in the last few weeks. It is now at Mk6! I've
also got some exciting news about our three beta testers. But i'll
put that in the next update!

Most of our previous videos have been filmed outside. Partly for the light, and partly, with some of the early prototypes, because they were not quite accurate enough for narrow interior spaces. No longer. With the new Mk5 electronic hand, and updated software, Eyedrivomatic is now precise, accurate and reliable. To demonstrate this, here is Steve at home, driving about inside.

Myself and Steve both
have ALS. When you've had it for a while, like we have, your health
becomes very fragile. Steve had pneumonia recently, which prevented
him from working on Eyedrivomatic for a month. Last week was my turn.
I can no longer swallow, and for the last two years I have been fed
through a tube in my stomach. Last week I stopped being able to
tolerate the food at all. After three days without food I was rushed
into hospital. The doctors fiddled with my medication, and the
situation improved a little. I am out of hospital now, and feeding
again - enough to keep me alive, though with a distressing amount of
nausea attached. This makes perfecting Eyedrivomatic all the more
urgent. While it works well now, there are too many loose ends for
others to take over easily were I to die unexpectedly. The system
works well, but it needs more fine tuning on the electronic hand, and
time spent making the software more plug and play.

The Mk5 electronic hand
is now in it's fifth sub prototype. The left right servo was moving
excessively compared to the forwards backwards one, so I redesigned
it. I moved the pivot point for the main cradle one centimetre lower,
which hopefully will restore the balance between the x and y
movement. I'm posting it to Steve for testing today. Meanwhile i'm
redesigning it a further time as it's too hard to build at the
moment. An easy build process is very important to us, as the end
user is unlikely to be an engineer.

We've had a lot of
interest from disabled people wanting their own Eyedrivomatic
systems. As soon as the Mk5b has proved itself, we'll start producing
units for beta testing.

I printed my design for
the Mk5 electronic hand, unfortunately the design wasn't quite as
perfect as i'd hoped. There have been two sub prototypes since then
to improve clearances and performance. The third one (the one with
red servo arms), is good enough to use for preliminary testing, so
i'll be posting it to Steve today.

The Mk5 is much harder to
assemble than the Mk4, so i'm already working on an easier to build
version.

I'm really keen to expand our testing team – to start
getting eyedrivomatic out to the people that need it, so i've started
the process of recruiting beta testers. As soon as the new electronic hand proves
itself in the field, i'll produce four more systems and get them out
to the new crew....

What about a photoresistor, that detects the level of light, and turns on/turns off the safety function automatically? So immediately when the users drive outside in strong sunlight, the safety function is activated, and when user drives inside and light level falls to a safe level to reliable detect the users eye, safety function turns off. Of course this could be configurable. Like Safety=Off, Safety=On, Safety=Auto.

And just curious, isn't is possible to configure eyegaze to move the mouse position to a safe position (like stop button) once the Contact with user's Eyes is lost?

thanks for theidea! The problem is the the way light affects eyegaze equipment is quite complex. Its not simply more or less light. Eyegaze uses ir light to detect contrast. A change in the ambient ir level can throw off the calibration, so moving at all causes problems. Also it is not a problem if the eyegaze equipment loses focus because it takesthe mouse away. The real problem is whenit loses focus, but doesn't realise it

But, indeed, one very very useful thing about your approach is that it 'should' just work with any joystick. You don't need to do any further hacking to work on a new joystick. Mine you have to work out each new joystick protocol.

I'm very interested in how your project progresses, and very excited to see that instructable!

Bob, you're a genius! I love it. I am particularly excited about the robot arm. It was in my mind to make an arm for my wheelchair . However, your use of the actuator as the main arm is simply genius. I would never have thought of that. I even have one in my project bits box. Thank you lots!

If you mean, how long does it take to build an eyedrivomatic system yourself? Once you have all the components and 3d printed parts, maybe an afternoon. If you mean how long have we spent on the project - two years.

Hi Patrick, I did a multi-input system for people with cerebral palsy and others, to control wheelchairs, robotics arms and computers some months ago. You can see some of the videos (sry, only in Spanish) here: and Remember to be careful with mechanical devices moving the wheelchair's joystick, be sure to include a safety system to cut the power, just in case, and that for some wheelchairs the 9g servos are not enough.

Good luck and congratulations for your project, it looks wonderful! :)

Thats really cool. I love the robot arm. And thanks for the advice. We are very aware of the safety issues around eyedrivomatic. The safety thing has always come first in the design process. The 9g servos that we are using are high torque ones, and the next version of the hand will be able to take standard and micro servos depending on your joystick weight. Myself and Steve both have lightened joysticks, so for us smaller was more beautiful...

We are not quite ready to put the how to instructions up yet, because the project is still in active and rapid development. Also, because Eyedrivomatic is potentially dangerous, we want to make sure its as safe as it can possibly be, before we let others use it. But don't worry. It won't be long - within a month, and certainly before August, I expect to have all the instructions on here for anyone to use.

The reason for using arduino for this project is that the servos in the electronic hand only requirea simple microprocessor. Arduino is cheap and simple. But the main reason is that I am an electronics beginner, and arduino is all I know.