Tuesday, May 13, 2014

New eye tracking research helps piano students become better sight readers

A student works with eyetracker.

Cicero said our eyes are the windows to our soul, but struggling music
students and their teachers may find it handy to know eyes can also
pinpoint our struggles reading music.

This came to light from the
curiosity of a professor of piano at UAA, Timothy Smith, looking for
ways to help students pass a challenging part of their piano exam.
Successful students must sight read a four-voice chorale, thick with
notes. That means playing it without advance practice. Such complex
music requires them to “chunk” together a lot of notes, or get more data
with a single glance, while staying on tempo and playing accurately.
Not easy.

Smith has been an Evelyn Woods speed-reading fan, and
even taught the course one summer. Could studying eye gaze and fixation
on musical scores, as the speed-reading program did for words, be of
help to his struggling students? He posed that question to two UAA
computer science professors skilled at programming eye tracking
software.

Kenrick Mock and Bogdan Hoanca had earned a patent in 2011 for their computer security software.
Essentially they programmed an eye-tracking device to allow a computer
user to sign on without a password by tracking the user’s gaze on a
particular set of icons. The eye gaze pattern replaced the password.

To
figure out if eye tracking would help piano students, Mock and Hoanca
refashioned their software over digital musical scores and Smith began
recording student eye movements. A year of testing followed.

As it
turns out, the answer is yes. The three professors now have a patent
pending on eye tracking software capable of delivering feedback to a
music student that compares to what a human teacher can deliver.

Here’s how they got there.

Typically, a music teacher assesses student performance by taking in the music as it is played.
“I
would just sit back and listen,” Smith said. “How musical is it? Is it
flowing? Or is it really a chore? Most of the time, it was just my
appraisal of what I heard and what I saw.”

A sample of an eye tracking printout.

Of course, he noticed students’ eyes. Sometimes he even sat opposite them behind the piano, watching their eyes.
“Students
who are less adept, you see them glance at their hands,” Smith said.
“The idea of staying focused and also being able to generate the notes
by feel or with your periphery—that’s the goal. The eye tracker can tell
us when the eyes go off the page and how many times they do.”

Smith
scores his students from 1-10 on their overall performance. That score
is made up of about a half-dozen metrics that Smith evaluates aurally.
He sent his list to the computer scientists, and they evaluated the eye
tracking data to see if it could measure the same things. Turns out it
could, very well.

Besides accuracy and tempo, Smith listens for
pausing, regression (playing a wrong note, and playing it again),
fixation, length of fixation and ability to read ahead. The eye tracking
software reports exactly where the student’s eyes spend time. Advanced
players tend to move forward linearly with clean up and down movements.
Novice players? Their eyes are all over the place, including going back
to mistakes, glancing off the page, lingering on troublesome notes.

Part
of the trio just delivered a paper on their results at a national eye
tracking conference. A company there presented a $100 eye tracker, which
compares to the $30,000-$40,000 eye tracking equipment programmed at
UAA. “There is a lot of potential out there for something like this to
be inexpensive,” Hoanca said.

The ultimate goal is a low-cost
device like a tablet, loaded with eye tracking software overlaid on
musical scores. The team imagines having a library of these ready for
students to check out. Once they achieve a score of 8 or 10, they can
advance to the next one.

The student will play the music and
record her eye movements. She can play the eye-tracking video back
immediately, and also receive quick feedback on what the results show.
Besides one overall score, it could deliver individual scores on each
metric, plus some text feedback, like “You weren’t steady with the
delivery of your beat pattern."

Will this replace music teachers? They don’t think so.

“It’s
one thing to have music lessons once or a few times a week,” Hoanca
said. “It’s something else to have a device anywhere and anytime you’re
practicing that can give you this kind of feedback.”