A new way of piano learning

Smart Piano is an Android-based app that
helps beginne﻿r piano-players learn new pieces of sheet music.

Introduction

Beginner piano-players encounter several obstacles when starting to learn the instrument. Our app is designed to: •Offer a compact, paperless sheet music library. •Provide visual aid to learning the tempo and finger positions of the music. •Give the user control over tempo and provide performance feedback.

gui design

The ‘Tempo bar’ How do we progress through the music? 1) Move the tempo bar, hold the sheet music. 2) Move the music, hold the tempo bar still (think guitar-hero). This is meant to be a learning aid first and a game second, so we want the experience to be as realistic as possible.

Keyboard Another key learning component of the app is to aid players in learning the finger position for certain keys. While we have the framework for this feature done, we would like to add two modifications to increase the functionality: •Illuminate the keys, instead of dots.•Provide actual finger suggestions for each key.Below is an illustration of current format compared to our goal:

Matched filtering

Matched filtering is an optimal linear filter used in signal processing. Our filter uses a note template library to compare to a piece of sheet music, and performs an exclusive-OR operation that returns the percentage of matching pixels. We use templates for quarter and half-notes to read the sheet music and return information to the GUI.

Note Detection Results: We can adjust the threshold percentage anywhere from 0-100%. Lower thresholds have less misses, but more noise and false-alarms. Likewise, a higher threshold has less false-alarms, but is more prone to misses.

System flow diagram of how matched filtering works.

Accuracy & templates

Matched-filtering itself is a robust method. However, not all quarter-notes on a piece of music are the same. With only 1 template we have a match percentage of 85.4%. Two templates, however, result in an average of 91-92%, as shown below. With four, we find a 97% match rate.

The two graphs above illustrate the same data differently. To the left, we see the match percentage of each template combo for each note in a staff. To the right, we have a histogram showing the distribution of note hits compared to the accuracy. More hits to the right of this graph is better. The four-template combo clearly outperforms the others.

failure analysis - measuring "false-alarms" & Misses

We also measured the percentage of “false-alarms’, which occur when the filter thinks there is a match, but there isn’t, as well as “misses”, where the filter misses a note. This is particularly important for our app because a half note on a line looks very similar to a quarter note-template. By plotting the false-alarms and misses against each other, we can find a ‘region’ of high accuracy, where we get neither false alarms nor misses. Below is our failure analysis of the four quarter note and four half note templates we used.

future of smart piano

Manual Tempo Control

The next version of Smart Piano will allow the user to define the tempo of the song.

Automatic Tempo Control

In a later version, we envision the tempo of the song automatically adjusting to the rate at which the user is playing via audio feedback.

Score Feedback

Smart Piano will tell the user if the note played is correct or incorrect. The app would then provide a score after the song is finished. Users could create profiles to keep track of their progress and statistics.