Access to Mathematics by Blind Students

Arthur I.
Karshmer and Chris Bledsoenformation Technology Program
University of South Florida
Lakeland, FL USA

Introduction

Reading and writing mathematics is fundamentally different than reading and
writing text. While Braille is adequate for the representation of text, it
is not up to the task of representing mathematics. The two basic reasons for
this are:

Linearity

Text is linear in nature while mathematical equations are two dimensional.
What you have been reading in this text is a good example of this problem.
In contrast, examine the following relatively simple equation

Figure 1: A relatively simple equation

One will immediately notice that the equation contains a superscript and
a fraction - both being two dimensional in nature. The equation could have
been written in a linear form, for example:

a = sqrt(((x super 2) - y) / z)

For this relatively simple equation, a linear representation is adequate
for reading to a blind user. But, with any increase in complexity it becomes
apparent that linear representations are no longer useful.

Character Set

Text can be generally represented in a somewhat limited number of characters
which normally include upper and lowercase letters, the 10 digits, various
punctuation marks and a small set of special characters. Equations on the
other hand can contain all of the normal text characters plus a large number
of special characters.

Normal 6-dot Braille can intrinsically represent 64 unique characters. Through
the use of special "escape" sequences Braille can support a much
larger set of characters. But this comes at some cost. This means that the
basic characters that can be represented in Braille can have different meaning
in different contexts. While there is effectively no limit to the number of
characters one can represent in the fashion, the reading and understanding
of these special sequences becomes very involved. The more new characters
one needs, the more escape sequences one requires. So, for example the letter
"a" can have different meanings in different contexts. It could
be an "a," an "A", a "1," and so on. This makes
for difficult reading and/or writing of mathematical equations in Braille.
Now, add to the mix the multi-dimensional nature of all but the simplest equations,
and the escape sequences required to represent them. It becomes clear that
Braille math representation is very difficult given the character set that
can be represented in 6-dot Braille.

In spite of these limitations, numerous Braille mathematical notations have
been developed over the years, primarily based on 6-dot Braille. In the United
States the Braille math standard is named after it inventor, Dr. Abraham Nemeth.
Other countries have their own standards and some countries have more than
one. These representations use the tricks described above and are therefore
not easy to learn.

Add to these problems the fact that the vast majority of mathematics teachers
don't know, and don't care to learn math Braille notations and you can easily
see the problems associated with learning math if you are severely visually
impaired or blind.

Finally, while text can be recorded by sighted readers in the form of talking
books, this technique does not work well with transmitting mathematical equations
(Gillan, et al. - 2001). Having a human reader at your side does help, but
is an impractical solution on several levels.

Other techniques have been tried to solve these basic problems, and these
approaches will be presented in the remainder of this paper. While not an
exhaustive review of other techniques, we present several examples of methods
reported in the literature.

General Approaches

Most of the work reported in developing techniques to help visually impaired
students deal with mathematics fall within the following general categories
:

Tactile as in Braille and other raised representations.

Audio aids that read equations to the student with tools to help in
the reading process.

Tonal representations of equations and graphs.

Haptic or forced feedback devices that represent shapes of objects
and curves.

Integrated approaches.

While more exotic approaches have been postulated and are possible in the
future, we will examine work that falls into the basic categories listed
above.

Tactile Representations

We have already discussed the use of traditional 6-dot Braille and its
limitations. Now we turn to other tactile methods. First and foremost amongst
these other tactile methods is 8-dot Braille representation. By using 8
dots, we extend our basic character set from 64 to 256 characters. Additionally,
most 8-dot systems attempt to have some sort of graphics associated with
the dot pattern (Weber - 1994; Gardner - 2001).

In an effort to alleviate the problem of requiring teachers to learn Braille,
there have been projects that allow a teacher to prepare instructional materials
in familiar electronic ways and then have their work automatically translated
into Braille. Three such projects are, 1) the ASTER project (Raman - 1994),
2) the LABRADOOR project (Miesenberger, et al. - 1998) and 3) the MAVIS
project (Karshmer, et al. - 1999). Other markup languages have also been
used to accomplish the same end as seen in the MATHS project which used
SGML (Edwards - 2001) and the MAVIS project which now uses XML (Karshmer,
et al. - 2001), By using standard markup languages, these projects offer
teachers powerful tools to create instructional materials and make them
usable by the blind. Another feature of many of these projects is their
ability to generate refreshable Braille output.

Tactile output is also very helpful for the blind math student in other
ways. For example, sighted students can easily examine the graphical output
of a sine function, but for the blind this is not an easily achieved option.
The ultimate goal here would be the development .of a high resolution refreshable
tactile device that could present images to be touched by the user. There
are several industrial efforts to create such a device that would be inexpensive
enough to be practical, but it hasn't been accomplished yet. In the meantime,
there are other devices that can prepare the equivalent tactile sheet on
paper or paper-like substances, One of the more promising of these devices
is the Tiger Printer (Walsh and Gardner - 2001).

Audio Aids

Audio has been one of the most popular and successful output media for
the blind student over the years and is employed in a wide variety of computer-based
device interfaces. Certainly, Jaws for Windows (Freedom Scientific - 2001)
is a necessary tool for any blind computer user. But this tool is primarily
designed for the general user interface and is not well suited for more
technical data interfaces.

The problems associated with the automated reading of equations have been
the subject of several research projects over the years. Professor Abraham
Nemeth not only defined the U.S. standard for math Braille representation,
but also developed a simple spoken structure for reading equations (Nemeth
- 1996). An interesting overview of the problem of reading math (Hayes -
1996) offers an overview of techniques developed to that date. Finally,
the MAVIS project carried out a series of psychological experiments to better
understand the process and its inherent problems (Gillan et al., - 2001).

Two early projects in the delivery of technical information to the blind
student through audio output were ASTER (Raman - 1994) and Talking Emacs
(York and Karshmer - 1991). ASTER reads complex equations to the listener
with special sonification to highlight parts of the equation while the Talking
Emacs project reads C code to the user providing descriptive information
about the code in addition to the basic program material. Given that the
project was based on the Emacs editor work was planned to enhance the editor's
base Lisp code to also read equations.

More sophisticated equation reading were key to both the MATHS and MAVIS
projects. Additional features such as the use of prosody and highly flexible
equation browsers to allow the user to navigate through complex equations.
The MATHS system also uses non-speech sounds to give an overview of the
equation. MAVIS (see Figure 2), on the other hand uses a hierarchical structure
to group sub sections of an equation into meaningful "chunks,"
for browsing. Both MAVIS and MATHS also offer refreshable Braille output
and specialized video output for low vision users.

Figure 2 - The MAVIS Math Genie

Tonal Representations

Reading equations is not the whole problem associated with mathematics
and blind students. The output of a function, for example, can also be a
multi-dimensional structure. These forms of mathematical structure are even
more difficult to describe in words than an equation.

Sonification of graphs has been the subject of several interesting research
projects over the years (Mansur - 1975; Kennel - 1996). In theses systems,
musical tones are used to represent the shape of graphical object. Both
two- and three-dimensional objects have been sonified with a these techniques.

The use of strictly tonal representations have had limited success especially
when used for complex graphs. Hearing alone seems to lack the information
carrying capability to describe complex objects. While not a complete solution,
sonification can be used with other non-visual output media to describe
graphs.

Haptic Devices

The development of high resolution refreshable dot grids would be the best
way to represent both text and non-text data. Unfortunately, such devices
are still way beyond an affordable price. Other devices like the Tiger printer
can produce relatively inexpensive raised dot paper output, but they are
not dynamic in nature once they are embossed.

Another solution to this problem can be found in haptic or forced feedback
devices. Such devices can work in either two- or three-dimensional space,
and under computer control allow the user the "feel" shapes (Yu,
et al. - 2000; Kennedy, et al., 1992; Sjostrom - 1999). Again, high-quality
haptic devices are reasonably expensive, but new less expensive devices
are now coming on the market. Prices can range from US$ 6,000 down to US$
100. Naturally, the less expensive devices don't offer the resolution of
their more expensive counterparts.

Integrated Approaches

While all of the approaches to delivering math education to blind students
have a great deal of merit, the truly successful ones offer a number of
interface techniques. In the domain of reading equations, for example, the
MATHS and MAVIS projects used multiple output media to accomplish their
tasks. These included speech, sound, Braille and structured navigational
techniques. In the case of MAVIS research from math reading experiments
laid the groundwork for less ambiguous equation reading and the ability
to speak equations into the system by the user.

Figure 3 - Math Braille Translation

Another example of integrated approaches is seen in the combination of
haptic and sonic devices for describing graphical output (Grabowski and
Barner - 1998; Sahyun and Gardner - 1998 In these projects, forced feedback
devices are coupled with sonifcation techniques to deliver a high quality
rendering of the graphical output. An interesting project from the University
of Bologna in Italy includes both haptic and sonic output to describe objects,
but also incorporates vibrators to represent colors in the form of a tactile
RGB display (Bonivento - 1999).

Finally, a new project at the University of South Florida, with partners
in New Mexico, Texas and Austria, is integrating several input and output
media to build a tool to translate a variety of national math Braille representations
(see Figure 3). The tool will allow blind mathematicians in different countries
to use tools developed in other projects to exchange mathematical works
with blind colleagues in other countries. Here the math browser developed
in the MAVIS project is integrated into a tool that translates different
math Braille notations through standard markup languages to other national
math Braille notations.

Conclusions

Making mathematics accessible to the blind is a challenging and difficult
process. The computer and its range of output devices has become the foundation
of numerous projects that have brought this goal closer to a reality. With
I/O devices such as high-quality speech, musical tones, refreshable Braille,
haptic feedback and high reliability speech input, new and effective tools
will soon be on the market. Other research into direct neural connectivity,
will in the future, make the picture even brighter.

Some of the work reported in this paper was funded under grants from the U.S.
National Science Foundation (HRD 9800209 & HRD 9906130) and the U.S. Department
of Education, National Institute of Disabilities and Rehabilitation Research
(H133G010046).

Karshmer, A.I., Gupta, G., Pontelli, E., Miesenberger, K., and H-F Guo, "The
Development of a Tool to Enhance Communications Between Blind and Sighted
Mathematicians, Students and Teachers: A Global Translation Appliance,"
Proceedings of the HCI Conference, New Orleans, 2001