Recurrent Neural Networks (their most frecuent form, LSTMs) are particularly interesting in that they learn patterns in sequences. A great article showing their tremendous power is Andrej Karpathy’s Unreasonable Effectivenes of RNNs. If you haven’t, you should go read it now.

After seeing the wonderful things they can do, I decided to try and use them to tackle a problem I’ve been interested in for years: programmatic music composition. This is something I’ve attempted in the past using Markov models, Genetic Algorithms, and other techniques with pretty disappointing results.

The Experiment

I was really surprised when a project I thought would take me a whole weekend actually took me ~5 hours of coding at a busy Starbucks. (!)

Let me show you a short 8-second sample first, and then tell you how I did it:

That’s one of the first pieces my RNN composed. Not very good, right? But bear in mind that it learned how to generate that just by looking at examples of existing music. I didn’t have to program (or know about) any rule about how to structure notes to form a melody. Let’s see how:

First thing I did was looking for a good representation for the music to be composed. I eventually found the abc notation, which was particularly good for my purposes because it includes concepts of chords and melodies, which makes it easier to procedurally create more meaningful sounds.

Here’s a simple example I just wrote:

X:1T:"Hello world in abc notation"M:4/4K:C"Am"C,D,E,F,|"F"G,A,B,C|"C"DEFG|"G"ABec

I won’t go over the details of the format, but if you know some musical notation it will look familiar. If you’re curious about how this sounds, you can listen to this in Ubuntu by saving the above snippet into a hello.abc file and running:

I then needed the data to train the RNN on. After googling a bit I came across the Nottingham Music Database, which has over 1000 Folk Tunes in abc notation. I downloaded this pretty small dataset, compared the >1M data points typically used to train Neural Nets for real.

OK, so the RNN can learn to produce valid abc notation files, but that doesn’t say anything about their actual musical quality. I’ll let you judge by yourself (I think they’re quite shitty but at least non-random. I don’t think I could have programmed an algorithm to generate better non-trivial music)

Results

I’ll list 4 more non-hand-picked pieces I just generated from the fully-trained network, along with their abc version and music sheet. The names are hilarious.

Conclusions

Machine Learning is at a tipping point. The tools are getting better. Complexities are being abstracted. You don’t have to have a PhD to make apps that use state of the art network models. Verygoodframeworks and models are available for everyone to use. It’s a good time to hack something in Machine Learning. Even if you’re not an expert, you can achieve fun results.