Abstract

Abstract: Standard computer peripherals are often challenging to
use for mobility impaired, and in particular controlling computer mouse
movements efficiently and without risking strain injuries. Projected touch
screen technology show promise in this respect. This paper presents a Projected
Interactive PC-control pilot (PIP) solution for computer interaction. The paper
focus on testing and improving the usability of the PIP solution through
iterative user tests with mobility impaired children. It describes prototype
improvements aimed at fulfilling the requirements of users with reduced motor
skills, and discusses challenges and key findings from the extensive usability
tests.
Our results demonstrates that interactive touch based solutions may enable
heavily impaired children to independently partake in creative activities and
play, and points to the value of creating a touch based computer interaction
solution tailored to the needs of this user group.

1. Introduction

Computer technology is central to the modern societies, and continuous use of
technology is becoming a prevalent part of everyday life. For disabled people,
technology may be of great value - compensating and providing alternate ways of
doing things. At the same time, standard technologies may be difficult to use,
and system adjustments along with alternative peripherals are often needed.

Most standard peripherals today are designed to avoid strain injuries for non-disabled computer users, and not to lessen the impact of symptoms such as impaired body movement, muscle stiffness and weakness, spasticity and pain. Ergonomically designed equipment exists - typically computer mice and keyboards. Computer mice are often optimized towards the gaming industry, providing alternative sensitivities and weights but using standard designs. Computer interaction peripherals designed to accommodate specific users are available as assistive technologies. Examples are keyboards with fewer keys, larger keys or enhanced key label visibility,
key guards, foot or head operated mice and software for enhancing mouse control (medGadget 2005; IBM 2007).

However, through the project Parkinson's IT challenges (PIKT), it became evident that few
existing alternative peripherals show the innovative potential of projected
touch screen technology for motor disabled users; a projected
interactive PC-management (PIP) solution indicated touch technologies may
provide a beneficial alternative interaction technique for people with mobility
impairments (Begnum 2010).

Mobility impaired is a user group with many sub-populations, and thus various
challenges and demands towards technical solutions. A number of disabilities
fall within the category of mobility impairment, ranging from reduced stamina to
complete paralysis. These disabilities may affect body movements and muscle
coordination, give various degrees of paralysis, stiffness and/or spasticity, as
well as loss of muscle strength. The focus of the PIKT project were on users
with Parkinson's disease, with mild to moderate mobility disabilities. This
paper examines the usefulness of the technology for children with different
types of significant and severe motor disabilities.

Existing interactive touch
technologies are not tailored for motor disabled users. Our aim was to
investigate the usefulness, as well as further the development, of an
affordable, easy-to-use PIP solution tailored to user group needs, offering an
efficient and effective way to interact with a computer without unnecessary
fatigue, pain and strain.

2. Background

Creativity and play is an important part of child development (Lemus 2008). They are fun and social activities in which friendships and social groups are built, even helping to reduce stress (Ginsburg 2007). Many researches argue creativity and play is important for brain growth, motor control and general child development socially, emotionally and cognitively. For example, there seems to be strong evidence that play can serve literacy by: (a) providing settings that promote literacy activity, skills, and strategies; (b) serving as a language experience that can build connections between oral and written modes of expression; and (c) providing opportunities to teach and learn literacy (Roskos 2001),
and Frost (Frost 1998) discuss the role of play, recess, games and art in cognitive, language, social, emotional, and physical development.

Disabled children have the same desire to play and use their creativity as non-disabled, but not equal opportunities. Many activities, such as drawing a picture, can be inhibited due to poor motor control. Technological solutions could provide disabled children with the means to overcome some of these obstacles (Brænde 2003). For example, games can be played at the computer and art software exists which is specialized for children, like Tux-paint. However, when using a standard keyboard/mouse interface mobility impaired children face the same accessibility issues as for regular computer usage. The challenges are particularly great when it comes to tasks which require a high degree of accuracy motor coordination skill, such as for example drawing.

The PIP solution consist of a point-and-click device with an infrared LED, a projector, a
Bluetooth enabled computer and Wii-mote controller. The projector, acting as the
computer monitor, projects the screen on any chosen surface, typically downwards
from a high position onto a table in front of the user. With a raised projector,
a large screen surface is projected, reducing the need for detailed motor
coordination skills as the surface area reaches dimensions such as 30" across. A
Nintendo Wii-mote controller is mounted next to the projector, pointing its
infrared camera towards the same surface as the projected screen.

The control and clicking device consists of a simple circuit and is attached to the index finger. A button at the bottom closes the circuit every time the user "touches" a surface. At the top, over the nail, sits the LED, emitting light when
the circuit is closed. The
Wii-mote controller above the table is connected via Bluetooth to the computer, registering every time the iFinger's button is pressed,
i.e. when someone with the iFinger touches the screen on the projected surface. The position of the click is communicated to the computer, initiating a pointer movement or a click.

Utilizing the potential of this affordable technology was showcased by Johnny Chung Lee, as he was working at the Carnegie Mellon University. He posted videos of the low-cost touch screens functionality on YouTube which has been watched by millions of people and was elected best educational video in 2007 (Lee 2008).

The control and clicking device was improved since first tested
by PC users with PD in the PIKT project. Instead of a glove solution, a finger
attached pointing device was created (iFinger). Our iFinger material is stretchy to fit different finger sizes and the design allow for switching from finger to finger including the left handed. Low fidelity
clicking device prototypes was tested on students from the Oslo University College (HiO), and general flaws and errors detected.
The battery was changed to a smaller variant, a click-button was added to increase affordance and LED type, placement and position
was optimized. These changes resulted in prototype "A". Prior to this study some challenges regarding inaccurate aiming/positioning of the infrared light from the diode was detected in prototype A, and as a result, prototype B was built.

3. Usability tests

Focus was on testing usability and benefit of the touch based interactive
technology for different types and
severities of motor disability, and to optimize the technical improvements of
the iFinger unit accordingly.
Tests were thus conducted iteratively, improving the technology based on
detected user
needs prior to each new test iteration, aiming to increase usability and benefit
with each iteration.

The decision was made to use children as testers. They represent a realistic test audience for the here-and-now prototype evaluation - and not focused on future potential.
In addition, children seem more open to new interaction ideas, adapting to new techniques more quickly than adults.

Further, the choice was made to test the device for play: drawing and gaming.
Research on virtual reality and children with Cerebral Palsy shows creative
games (such as drawing and music programs) in particular increase feeling of
coping, contentment and motivation (Reid 2004; Lager 2009). The idea was that a
play test scenario would make it easier to recruit testers, motivate the
testers, make longer tests possible (less tiring when inspiring) as well as test
the solution more thoroughly than for office use. Also, drawing requires fine
motor skills, thus challenging and testing pointer control. When gaming, the
pointing/control device is stressed with regards to the rapid clicking and
movements made. The number of interaction events per minute when gaming is also
far greater than for routine desktop procedures.

Tux Paint was selected as the drawing program. Tux Paint (New
Breed Software 2010) is designed for children 3-12 years of age, has a user
friendly and intuitive design and amusing sound effects.

Figure 1. Tux Paint Screen shot

Minabc.no is an online gaming portal with entertaining colorful flash-based games designed for children of all ages. The user receives visual and/or auditive feedback for every action taken. Most games have the possibility for tailored degree of difficulty, including the size of clickable elements. Many games also have motivational factors included, such as an image becoming gradually more visible as tasks are completed. 5 different games from this portal were used in our tests: "Find pairs" (either identical animals or one animal and the related letter), "Count the fish" (fish swimming on screen), "Figure Memory", "Drag and drop puzzle" and "Connect-the-dots numerical drawing".

Figure 2. Screen shots of the 5 games tested

Seven children with various severe mobility disabilities were selected by a Norwegian school for students with neurological disabilities and/or multiple disabilities as testers, ranging from 6 to 15 years of age. Table 1 presents the seven testers. Prior to tests, we visited the children at their school to get to know them and establish communication with testers and participating teachers, map out physical needs regarding game and iFinger adaptations and explain test set ups. All tests in the test series also took place at the school. Each tester had a teacher at his/her side during tests, aiding verbal and non-verbal communication with the facilitator. In addition there were two observers placed at a distance. Tests were non-structured, reflecting play. A test journal was used for each tester, suggesting scenarios to complete and logging each test. Observers were provided with copies in which resulting data was noted. In addition, hypothesis for following tests were registered, such as tester specific challenges and assumptions on prototype improvements.

In total, the PIP solution was iteratively tested through an
iterative test series by 7 children with various severe mobility disabilities -
a total of 17 different tests conducted in 6 iterations:

In addition to the iterative tests, one of the child testers (with Cerebral Palsy) and
her teacher were subsequently followed through a 7 weeks period of independent
use at the school. at the school. The main goal of this was to observe how teacher and
student would utilize the solution in the education, and if and how an external
institution could use the PIP solution without expert knowledge.

4. Results

Prototype B drawing test: Barney, Rose and Phoebe

Barney managed to draw a circle following a circle shape, and demonstrated pointer control and click precision. He drew freely and enjoyed playing using the PIP solution. Rose familiarized herself with the PIP solution, but was very withdrawn. Phoebe did not have the strength to use the click button on prototype B, and tested with prototype A instead. She found drawing extremely fun, and drew painting upon painting, learning even advanced features of Tux Paint. She had some interaction difficulties linked to button insensitivity and prototype A inaccuracy.

The added click-button in prototype B was not sensitive enough for the testers, due to reduced muscle strength. Prototype C was modified using two buttons of smaller size, together forming a larger and more sensitive clickable area. The new button design was also more robust, placed in between two surfaces. More iFinger size diversity were needed.

Prototype C drawing test: Sarah, Pam and Joey

Physical test surroundings did not fit Sarah, who had challenges using the set-up table. In addition, her finger positioning was inexpedient, resulting in jagged drawing and low click responsiveness. Sarah had no difficulties understanding the software or the PIP solution, and thus overcame the interaction difficulties - for example repeating clicks until registered. She very much enjoyed using the geometrical figures, and especially liked drawing red hearts. Pam mastered both pointer precision and the click technique after some practice. Pam very much enjoyed painting, and so did Joey. Joey had some trouble understanding the software, but was an eager tester and artist.

Clicking techniques amongst the children made it expedient to further increase the clickable area. This was done in prototype D. Some calibration issues were also discovered due to a loosely fastened Wii control.

Prototype D drawing test: Barney, Rose and Phoebe

As button sensitivity was increased, Barney now had better click response. He also drew with a standard computer mouse for a part of the test to compare against the iFinger. The iFinger produced significantly improved precision. Barney now had no interaction difficulties aside from some random involuntary twitches affecting click registration. Rose had some cognitive challenges using Tux Paint, and needed assistance changing colors etc. She had some interaction difficulties related to precision, but found the drawing test fun. The iFinger was still not fitting Phoebe well, resulting in difficulties reaching the click area. Still, click responsiveness and sensitivity was notably improved, making drawing more exciting and fun. iFinger robustness was tested, with luck, as Phoebe experienced some large, speeded spasm where she banged the iFinger violently against the table.

Prototype D now used two larger sized buttons, reducing the resistance of pressure from 1.6 nm in previous prototypes to 1.3 nm, but ease of clicking still needed some improvement. The design had weak affordance, as several testers tried clicking with their thumb on the iFinger button. This was improved in prototype E. Steel edges on the buttons were painful, and were removed.

Prototype E drawing test: Sarah, Pam and Hillary

The final prototype was built in two versions, using two different battery technologies (SR44 in addition to the LR1). The iFinger surface facing towards the projected screen was enlarged in prototype E, increasing ease of clicking, but reducing click precision. Even so, Sarah's precision was improved compared to prototype C. Both she and Pam enjoyed drawing, although still having some interaction challenges. Hillary tried iFinger for the first time, instantly displaying good click and pointer precision, and even managed handwriting. She controlled the software with ease, changing colors and modes, opening new sheets etc. No interaction difficulties were identified.

Prototype E game test: Barney and Joey

Barney tested the games "Find pairs", "Drag and drop puzzle", "Count the fish" and "Figure Memory" with great success. Joey tested "Find pairs" and "Figure Memory". Joey was at times disturbed by violent involuntary movements, but with practice, both click technique and precision was managed.

Prototype E game test: Phoebe, Pam and Sarah

Phoebe tested the game "Find pairs", but even on the easiest level, this game was too complicated cognitively for her. She interacted seemingly without any purpose, and eventually the test were ended. Pam and Sarah also tested "Find pairs". A couple of games were to challenging cognitively for Pam, but she managed to complete the easy and medium versions of "Find pairs". Even though her precision was good, the position of her hand was not optimal, and she did not have the accuracy skills needed for "Count the fish". Sarah quickly selected matching figures and letters, herself increasing game difficulty to the maximum. Sarah had to practice interaction and precision skills for a while to master the games "Drag and drop puzzle" and "Count the fish". Her memory was exercised during "Figure Memory". For Sarah, the gaming test was a huge success.

Long-term institutional usage

Sarah was selected to try out the solution over a longer period of time after the tests series were completed. In collaboration with the school board and the art teacher, the solution was set up in the art education classroom. Even though mounted on lockable wheels, and thus movable, Sarah mainly used the solution during art lessons. The IT responsible at the school was given a quick introduction to the set up and use of the PIP-solution.

Feedback from the school after the long-term test was that Sarah had on several occasions used the PIP solution for the entire class (45 minutes) without fatigue and continued to enjoy using it. The school managed to independently utilize and maintain the technology, moving and calibrating it when needed.

5. Discussion

The child testers had various severe and complex motor disabilities, affecting both movements, fine motor skills and cognitive understanding.
Still, all testers were able to master using the PIP-solution, where as not all master
standard computer equipment. All testers spontaneous started drawing using the
iFinger when placed at the test table, finding the testing fun and exciting, and
even desired that we should come back and test some more. They often arrived
smiling, remembering how to use the PIP solution and looking forward to the
test. During briefings, they emphasized the positive feelings of being able to
create something and "to draw on the table, like the other kids".

The PIKT project showed mouse clicking and pointer control were challenging
users with reduced motor skills, in particular retaining mouse pointer control
while initiating a click, and largely unsolved. The PIP solution however seemed
to support both areas simultaneously. Our tests show the PIP interaction
solution provide children not able to draw with a pencil on paper the
possibility of mastering unassisted freehand drawing.

Users could lean in
on the projected surface to support their movements, reducing fatigue and strain, stabilizing the pointer while steering and steadying hand and fingers while clicking.
However, since testers often had limited motion range, projecting a larger
screen to enlarge interactive elements was not always helpful. Troubles during
the tests were mainly related to technical prototype weaknesses and not using
the iFinger in the correct manner. The PIP solution was iteratively and
incrementally improved to better suit the needs of people with reduced motor
skills. Enlarged clickable area, increased
sensitivity and improved affordance in the final prototype solved most of the
interaction difficulties. Further aspects needing attention are:

Hygienic and robustness of iFinger design, e.g. be waterproof in case of saliva.

Investigate additional iFinger aiming mechanisms.

Teachers and school personnel continued giving positive feedback
long after the study were completed and the long-term test scenario demonstrated
how the PIP solution can be utilized by a school institution, independently
managing the solution without needing extensive training or technical skills.
This shows that the solution is accessible both financially as technically. Our
tests show the iFinger device is treated very roughly when used. We envisage an
institution/user having multiple iFingers, simply instantly replacing one if
broken.

Creating a touch based computer interaction solution for users with reduced
motor skills seem both beneficial and possible, although not trivial. Our tests
emphasized the importance of a high level of personal adjustment possibilities
for the user group. The PIP pilot offer choice of screen surface (floor, wall, table...) and size, projected
height and direction, and elasticity and size of pointer unit. The possibilities
for personal adjustment should however be furthered, especially with regards to
the pointing device; possibilities for individual preference on arm/hand device
mounting (on hand, one or several fingers, easy to take on/off...), strategy for
pointing and clicking, click sensitivity and size of clickable area. In
addition, specific additions should be considered, e.g. users with involuntary
hand contractions probably need help keeping the hand open.

It would be interesting to test the PIP-solution in a more multi-modal interaction environment, e.g. alongside a keyboard, for general PC control and usage. If iFinger precision is further improved, the solution may be a highly interesting interaction system for
more advanced gaming - possibly giving an aim advantage and a more realistic feel.

6. Conclusion

This paper demonstrates how a projected interactive touch screen technology is promising
for users with significant mobility impairments, providing pointer and click control literary at your finger tips. Prototypes were iteratively and incrementally improved through a series of 17 tests on 7 different children of ages 6-15 years with various mobility impairments. Our tests
illustrate how the touch based interactive technology have potential as a
compensator and enabler for the challenges facing people with motor
disabilities. It empowered motor disabled children, facilitating them to
independently part-take in games and creative arts they could not exercise with
standard computer peripherals and providing them with the means to creatively
express themselves through drawing. The possibilities of individual adaptation
proved a vital solution attribute in relation to the targeted use group, e.g.
with regards to mouse clicking sensitivity.

7. Acknowledgements

The authors would like to thank all those who contributed to this study. In particular, we would like to acknowledge the contributions of Petter Skolla
and Jonas Taftø Rødfoss
through their bachelor project work.

Ginsburg, K.R., MD, MSEd and the Committee on Communications and the Committee on Psychosocial Aspects of Child and Family Health (2007), "The Importance of Play in Promoting Healthy Child Development and Maintaining Strong Parent-Child Bonds", American Academy of Pediatrics, Vol. 119, No. 1, pp. 182-191.