ECE professor earns award for study on pianists’ hand positions

Professor Marge Skubic illustrates how the Microsoft Kinect device, above, was set up to capture pianists’ hand positions.

Electrical and computer engineering Professor Marge Skubic’s work has appeared in numerous publications and earned her multiple accolades over the years. But her latest award came from an unusual source for an engineer.

Skubic worked with assistant professor of piano pedagogy Paola Savvidou, assistant teaching professor of physical therapy Brad Willis and recent College of Engineering graduate Mengyuan Li on “Assessing Injury Risk in Pianists: Using Objective Measures to Promote Self-awareness,” and the group’s paper recently earned the 2016 Article of the Year honor from the Music Teachers National Association (MTNA).

“I did get a kick out of the idea that as an engineering professor, I got a best paper award in a music journal,” Skubic quipped. “I didn’t expect it.”

Skubic’s work with the Microsoft Kinect device and expertise in sensors from her extensive work with them in the field of eldercare provided exactly what Savvidou was looking for to gather data and begin work on a screening tool to assess the posture and hand positions of pianists. This latest paper used the Kinect depth camera to take overhead depth images of the hand positions of 15 participants to determine whether or not the participants were using proper form as they played the same selection of piano pieces. Depth images store the distance to the nearest object at each pixel, differentiating them from grayscale images, and further work allows the image of the hands to be segmented out.

Using the Kinect, seen above, as opposed to devices mounted on the hands provided a passive, unobtrusive way of collecting data without affecting the natural form of the participants.

Using the Kinect as opposed to devices mounted on the hands provided a passive, unobtrusive way of collecting data without affecting the natural form of the participants.

A lack of proper form often leads to injuries, including maladies such as tendinitis and carpal tunnel. Proper form is considered a non-flexed wrist position, curved hand position and having fingertips in contact with the surface of the keys.

“If you have a piano student that may practice eight or 10 hours a day, No. 1, it can set them up for injury,” Skubic said. “It could be even permanent injury that can knock out their chance of having a performance career. The other problem of course is that you don’t get the right technique for playing, and that also affects the sound, the quality of the music.”

The paper was the latest report on the quartet’s work. Li, who graduated with her master’s degree from MU Engineering in 2015, used the project as the basis for her thesis, as well as a paper titled “The Detection of Potentially Harmful Hand Postures in Pianists Using Kinect Depth Images.” The paper was accepted by the Conference of the IEEE Engineering in Medicine and Biology Society in 2014.

“We had a large number of papers; we took many students there. We had some really good papers, and the interesting thing is the paper that seemed to have gotten the most buzz was this paper about pianists,” Skubic said. “There was a lot of interest in it, maybe because it was so unusual.”

Savvidou said in an Arts and Sciences news release that her eventual goal is to create a screening tool that automatically detects poor form. Skubic, meanwhile, posited that this project could open the door for similar Kinect-based studies into hand posture in the future.

“It’s a repetitive stress injury, really. And I think there’s some opportunity to translate that to computer keyboard work,” Skubic said.