Air Drumsby Alexis Collins (aac38) and Joe Kerekes (jck46)

Introduction

One Sentence Sound Bite

“Air Drums” is an electronic drum kit played in the air that eliminates the need for tactile drum pads.

Summary

We created an electronic percussion set with three upright percussion sounds and a floor bass drum sound. The upright instruments require one/two modified drumsticks to play, while the floor drum requires an electronic bass pedal. The important difference between our drum kit and store bought drum kits is the lack of drum pads. Instead, our drum kit can be played entirely in the air, eliminating the distracting tapping noise that comes from the drum pads. This means you can play the Air Drums in your room with a set of headphones without annoying your next-door neighbor!

High Level Design

Rationale

The inspiration for our project came from multiple occasions of being scolded by our roommates for playing the Rock Band drums too loudly (all while wearing headphones). The rationale behind our project was to create a drum kit that would not make any noise when played. While ordinary electronic drum kits claim to be silent since the noise they produce can simply be filtered through headphones, this is not the whole truth. The loud rap of the drum sticks hitting the drum pads carries a great distance. The loud beat has been known to be just as annoying to roommates and apartment mates as the sound of actual drums. For this reason we wanted to eliminate the drum pads and therefore the loud rapping noise. The idea was wireless drum sticks with a speaker component that can be plugged into headphones. This will eliminate all ambient noise as there is no actual contact with a surface of any kind. In addition, Air Jam, the air guitar project from the 476 project page, inspired the idea to create an entire air band.

Background Math

The background math for this project was required for the drum sound synthesis. Instead of using direct digital synthesis recorded sound playback, as we did in previous labs, we chose to use the Karplus-Strong digital signal processing algorithm to create accurate drum sounds. The Karplus-Strong algorithm is a digital approximation of the way a wave decays in a string or on a drum. Using a shift register, when the instrument is struck the register is filled with random noise which is the same as the fourier transform of an impulse of energy to the instrument. For drums this shift register is band passed using a butterworth band pass filter, for cymbals we use a low pass filter after subtracting the delayed version of the signal from itself. The frequency of the sound is given by the sampling frequency over the length of the shift register.

Logical Structure

The structure of this project is broken down into three separate tasks: Bluetooth communication, drum sound synthesis, and IR position parsing. The first step, Bluetooth communication, consists of a Wiimote, and a Bluetooth dongle connected to a laptop with Blue Soleil Bluetooth software installed. The Wiimote is used for its CMOS camera technology which can determine the positions of multiple IR LEDs. We have two such IR LEDs which are connected to the tips of drumsticks so we can determine drum tip position. The Bluetooth dongle and Blue Soleil connect to the Wiimote through a Bluetooth connection. From this connection we use the open source “Wiimote Lib”, written in C#, to query the Wiimote for the position values of up to four IR LEDs in the reference frame of the Wiimote’s CMOS camera. After receiving this data we open a serial port and transmit this data to the Atmega32 on the STK500 board through the serial connection.

This leads us to the next step, IR position parsing. In this portion of the code, written on the Atmega32, we receive the serial data transmitted from the laptop software. This information is sent out in packets of information that include the number of the LED whose data is being transmitted, and whether it is the x or y position of the IR LED. This is followed by the actual data. Knowing the format of these packets, we parse these packets in a large switch statement, update the x or y position variable for one of the IR LEDs, and request another packet. From this data we determine which “drum” is being hit depending on its relative location on the CMOS sensor’s scope. The scope is simply divided into three equal widths on the x axis. When the y position in each of these scopes is lower than the half way mark and changes from down to up, we play the corresponding drum sound. In addition we play the bass drum noise everytime the bass pedal is hit. This requires no serial communication however since it is connected directly to an input on the Atmega32.

The final task we accomplished in this project was drum sound synthesis. We created four separate drum sounds, a bongo, a mid Tom, a crash cymbal and a bass drum. Each sound we generated using some variation of the Karplus-Strong algorithm which uses different filters on random noise to create sounds. This essentially simulates different instruments and produces a fairly accurate synthesized tone. The algorithm uses different shift registers initialized to hold random noise. When our “drum” is hit we perform some type of filter on the random noise in the shift register a number of times until the sound decays, based on a decay factor that we introduce. The length of the registers determines the frequency of the sound which is actually produced using a PWM signal from the microcontroller. The calculations for the sound are actually performed in an interrupt that runs at an eighth of the clock frequency. This is necessary since we have a limited amount of space to store variables and the larger the initial interrupt frequency, a longer shift register is required to produce different frequency tones. Finally, as the drum sound is created it is sent from the microcontroller to a set of speakers via the PWM signal.

Hardware/Software Tradeoffs

Our initial plan was to use two Nintendo Wiimotes as the drum sticks in our project. When we realized the amount of added C# code this would have required (plus our very limited experience with C#) this seemed less possible. Furthermore when we began testing by holding a single Wiimote and using multiple IR LEDs for position detection, we noticed that the range of the Wiimote’s position was very limited. Because of the very directional IR LEDs, when we changed the yaw of the Wiimote, the camera was facing the IR emission from a different angle and this decreased the position range even further. Instead, we decided to use the Wiimote for its CMOS camera and detect the LEDs as drumsticks. This dramatically increased the position range that we could measure, since we could rotate the LEDs from side to side and the Wiimote could still detect the IR dot in the same x,y position.

Another design tradeoff in our project was the limited amount of memory space on the Atmega32. This space issue significantly limited the size of our shift register arrays, and required us to decrease the frequency of our output sound interrupt by one eighth. This allowed us to reduce the size of the shift registers and produce accurate frequencies despite the size limitations.

Relationship of Design to Standards

The standards relevant to this project include both IEEE standards for Bluetooth as well as ANSI standards pertaining to safety with radio frequencies in hand held wireless communication devices.

IEEE Standard 802.15.1-2005: Bluetooth 1.2 became an IEEE standard in 2005 and defines specifications for wireless connectivity in the Personal Operating Space. We operate the Bluetooth connection within the 10 meter space and used a store bought Bluetooth dongle to interface the Wiimote to the PC. We followed this standard by leaving the Bluetooth connection unmodified by us, and instead we simply received the data.

62209-1 Ed. 1.0 b:2005: This standard assures safety in hand held wireless devices that utilize radio frequencies. We comply with this standard since our project’s radiation falls well below the maximum absorption rate limit required for safe human use of handheld wireless devices.

ISO/IEEE 11073-30300:2004: This standard provides a structure for using Infrared communication in hand held devices. The most important part of this standard is adherence to the eye safety clause. We do not exceed the power limit for hazardous operating ranges and therefore meet the Class 1 AEL limit. This means we do not require a warning label on our device.

Patent Discussion

Though this patent certainly relates to our project there are several important differences. First off, we make no claim that our drum set be portable. Secondly, the means for creating drum sticks are completely different than those alluded to in this patent. There would be no way to use IR communication in the schema they devised.

There are other patents pertaining to electronic drums, but other than the above, “drumless” drums seem to be relatively unique.

Program/Hardware Design

Hardware

There are numerous aspects to the hardware design including the sound output, the Wiimote interface, the drumstick assembly, and the bass pedal interface.

Sound Output

Sound output was handled by PWM outputting through pin B.3. Using a 10-pin header we connected port B to the white board, and filtered the signal using low pass RC circuit. We then connected the filtered sound to the TRS adapter to send to our speakers.

Wiimote Interface

The Wiimote operates wirelessly via the 2.4 GHz Bluetooth communications protocol. Using a Bluetooth adapter on a laptop and the WiimoteLib library provided by Brian Peek we can interface with the Wiimote through the computer. Using serial communication over UART we send the raw data from the Wiimote, specifically the IR LED coordinates to the Mega32. The Mega32 can then analyze the position of the IR LED tipped drum sticks and determine whether an instrument has been struck or not.

Drumsticks

To simulate playing a real drum kit we decided to mount our IR LEDs on the tips of real drum sticks. The assemble was simple: the IR LED was soldered to a 330 ohm resistor and then soldered to a battery clip attached to a standard 9V battery. The whole assembly was then tapped to the drumstick as can be seen in the picture above. The battery clip allowed us to disconnect the LED when not in use to prevent draining the battery power. Finally the addition of the electronics did not change the feel of the sticks much and so the system provides for a relatively natural playing feel.

Bass Pedal

The bass pedal was taken from a Rock Band drum kit. It is essentially a push button operating using a reed switch and a strong magnet. A reed switch is a switch that is operated by magnetic fields. Inside the reed switch are two magnetic and conducting reeds which are separated. When a magnetic field is applied by pressing down on the pedal, the reeds are pulled together and complete the circuit. When the magnetic field is removed, the reeds pull apart. The bass pedal was used to emulate a push button on the STK500 board since it provides a much more authentic feel. It is connected via a mono 1/8" TRS audio cable to a ciruit that active low as seen in the schematic in appendix B.

Software

There are two aspects to the software design of the program. The main component is the embedded C code used to generate our Karplus-Strong based sounds and to interpret IR LED positioning and resulting played instruments. The supplementary part is the C# WiimoteLib test program provided with the library that we modified to include UART communication over serial so that we could send the raw data to Mega32.

Embedded C Mega32 Code

Our system was designed using interrupt driven timing, PWM sound generation, and fixed point 8:8 arithmetic. PWM was run by timer0 set to run at full speed, fast PWM mode, toggling oc0 (pin B3). Timer2 was set to run at 1/8th speed and to trigger an ISR on overflow. The timer2 ISR runs (16Mhz/8)/2^256 = 7812 times a second and calculates the Karplus-Strong algorithm for all our instruments based on flags of which instruments were struck. The ISR running a 7812 times a second allows us to reproduce in PWM a signal of up to 7812/2 = 3906 Hz, which is more than enough for our drums and cymbals.

In timer2 we calculate the result of the Karplus-Strong algorithm based on our inputs. Using message passing between main and the interrupt we can know whether an instrument is played. If an instrument is played, we replace the information in that instrument's shift register with pre filtered (to soften the sound) random values to represent a burst of energy provided to the instrument as explained above. We then update our 'out' variable which represents the sum of each instrument and the total sound of our drum kit. The shift register value is then updated according to the instrument specific Karplus-Strong algorithm. Finally we update our shift register indexes, which are used to avoid actually shifting our registers as this would be too slow. Once all the instrument registers have been updated and the current sound 'out' calculated, it is then written to OCR0 for PWM output, which is constantly running in the background.

Mult_opt was provided by Professor Land and is an optimized 8:8 fixed point multiplication function written in assembly. It was used because of its speed increase over the provided Mega32 functions; x3 over the normal mult and x6 over floating point mult.

Uart_rec and uart_send, provided by Professor Land, are functions for asynchronous serial communication. For receiving, the interrupt triggers when data is sent over the serial while UCSRB.7=1 (receive ISR enable). We continue to receive individual characters until a return character is sent representing the end of transmission. A flag is set to indicate complete reception and the receive ISR enable is turned off. Sending was not needed for our project. Gets_int and puts_int are also used for serial communication. Gets_int resets the receive buffer indices and turns on the receive ISR enable. Again puts_int was not necessary as we don't communicate from the Mega32 to the laptop.

Our main function, after calling initialize(), enters an infinite while(1) loop and controls the data receive, parsing, and hit determination. After receiving data from the C# program over serial UART, it was parsed according to our data packet setup as explained below.

Data received from the Wiimote are four sets of coordinates representing separate IR LED coordinates. X, horizontal coordinates range from 0-1000, while y, vertical coordinates range from 0-760. We split the space horizontally into three separate areas to represent locations for our instruments. 0-333 (space to the user's right) represents our crash cymbal, 334-667 represents our snare/hi tom sounding drum, and 668-1000 represents our bongo. The screen is then split vertically at the halfway point y=380. When an LED crosses over the y threshold, a flag is set and when the drum stick is raised (changes direction), as determined by a simple delta function (old-current) the instrument based on the LED's x location is considered struck. The bass on the floor is a simple switch and so is played when the pedal is released after closing it.

Using a publically available Wiimote library we modified the provided test code, which continuously monitors the wiimote's state, to open a serial port and to send IR LED coordinate data to the Mega32. We formatted our data packets as four number characters followed by a single letter character. The numbers represent the coordinate, while the letter represents which coordinate and LED the number corresponds to.

Code

Meaning

####x

LED1 XCoord: ####

####y

LED1 YCoord: ####

####a

LED2 XCoord: ####

####b

LED2 YCoord: ####

####c

LED3 XCoord: ####

####d

LED3 YCoord: ####

####e

LED4 XCoord: ####

####f

LED4 YCoord: ####

Despite only using two LEDs in our project, the wiimote has the ability to follow four separate LEDs and occasionally it loses track of which is which and randomly changes what the LED is labelled as. For example, while tracking LED2, the wiimote has sometimes switched it to LED4 and thus if we didn't send that data to the Mega32, no sound would be played.

Problems Encountered

We encountered numerous problems as we developed Air Drums over the course of the month. These issues included, serial communication problems, variable mapping issues, calculating Karplus-Strong algorithms in the actual PWM interrupt and not a separate one, overflowing indices resulting in better sounding drums, and finally random noise in the bass drum producing high pitched wailing sounds.

When we initially implemented serial communication in the C# program in order to send the raw wiimote data, we had already tested the link by using hypertrm to verify proper communication. However, in the C# program, after a few seconds of communication, the C# program would crash. It was finally determined that the C# program was attempting to write too often to the serial port and thus started storing data into a buffer, which would then overflow. With the help of CS friend we resolved this by discarding the buffer after one pass of the data. This prevents a buffer overflow and insures the most recent data.

Originally we had calculated Karplus-Strong algorithms in the actual PWM interrupt every eigth time in the interrupt. This proved disastrous, prompting several other errors, namely the mapping issues and overflowing indices sounding better. When we had two instruments and tried adding a third, simply declaring another shift register broke the sounds, as variables were mapped over one another for some reason. The bad indices were the result of sheer chance. We had accidentally started indexing arrays from one, instead of zero. Surprisingly the sound was still nice (for our two instruments). However when we noticed and changed the indices to the correct values so there were no overflows, the sound became incomprehensible. On the advice of Professor Land we used another interrupt to calculate Karplus-Strong algorithms for instruments and thankfully the above mentioned problems went away.

The last issue was that ocassionally our bass drum would become a high pitched wail sound. On the advice of our TA Rob Zimmerman, we figured out that the random number generator was perhaps generating amplitudes that resulted in sound overflows of some type. After some testing we determined that numbers above 17500 would result in the loud squeeking and so we modded the rand() function with 17500 to ensure that we never go above. After this fix, the bass drum never squeeked again.

Results of the Design

Accuracy and Speed Testing

The final code produced a usable drum kit with three upright percussion sounds and one bass drum. All sounds were clear, able to be played concurrently with little lag. However, we decided to quantitatively test our design. We conducted three tests of our design, reaction time of the system from physical striking to hearing the sound, accuracy of playing individual instruments, and how fast we can play successive notes.

First Test

During intermediate testing of our system there was significant lag in playback so we felt we could measure the difference from striking an air drum physically and then hearing the sound. However, with final code, the delay is virtually non existent if the system detects the hit, and any attempt to measure it was based solely on human reaction time.

Second Test

The second test was an accuracy test to see whether the system failed to detect what a user would reasonably expect to be a hit. We played a set of ten strikes per instrument. The results are as follows:

Instrument

Number Hit out of 10

Accuracy

Notes

Bongo

10

100%

Played the crash sound at 3

Hi Tom

9

90%

Missed 6th

Crash

8

80%

Missed 3rd and 7th

Bass

10

100%

~

Third Test

The third and final test was a speed test to measure how fast our system can detect and play continuous strikes. This was measured by quickly 'hiting' the bongo air drum and measuring how long it takes to reach 40 played strikes. We repeated this test four times and the results were averaged as follows:

Test Set

Time to play 40 / s

Played per second

1

9.02

4.43

2

7.46

5.36

3

9.30

4.30

4

11.25

3.56

Avg:

4.41

Safety Considerations

For safety, the user handles the drum sticks and so we used a basic standard 9V DC battery to power the LEDs attached to the sticks. The wires are covered in electrical tape and a resistor is used to protect the LED. Other safety considerations are that small DC voltages only are required for correct operation aside from the power supply to the STK500.

Interference

We did not experience any interference as no one to our knowledge was working in the same high freq wireless band other than one other group that used Wiimotes. Since bluetooth requires handshaking we were not interefering with each other during development.

Usability

When we completed our system we invited other people to play our system. People found it enjoyable, although it took a few moments to become accustomed to the placement of the drums in the air. We placed stripes of tape to indicate general locations to help people visualize the drum locations.

Conclusions

Our Expectations

From our original proposal our project evolved quite a bit. We expected to use the Wiimotes as the physical drumsticks and rely on three dimensional positioning to account for our interpretation of the drum heads. We quickly changed from using two Wiimotes to using a single Wiimote and two drumsticks with attached IR LEDs. We also eliminated three dimensional positioning as we decided this was unnecessary and needlessly complicated. Instead we focused more on the Karplus-Strong algorithm and the synthesized drum sounds. Obtaining accurate sounding drum sounds seemed to be harder than anticipated. It was partially the use of the algorithm and then a bit of tweaking by ear. For this reason our drum sounds are not entirely perfect. If we were to do this project again we would certainly budget more time to improving our synthesized drum sounds. In addition, there was a bit of lag between drum hits and the actual noise sounding. This lag and the occasional glitches seemed to be caused by a loss of data from serial communication. If we were to improve this project in the future we would consider using a different method of Bluetooth to serial conversion for the Wiimote. A dedicated Bluetooth module would most likely have eliminated the lag from our multitasking PC and since we were not writing the serial code ourselves it probably would have eliminated the loss of data.

IP Considerations

When we were creating the synthesized drum noises we looked at both Bruce Land’s matlab code on the Karplus-Strong algorithm for drums, as well as Yuan Ning and Adam Beece’s code from last year’s Karplus-Strong implementation of guitar synthesis on the Atmega32. We adapted Bruce’s drum code to code from the microcontroller for part of the bass drum sound. We had a very similar implementation to Yuan and Adam’s Air Jam code for synthesis since we used both PWM and an interrupt driven timer to create musical sounds.

In addition we used code from the public domain. The “Wiimote Lib” was an open source library written in C# designed for the wiimote. To obtain the IR data from the Wiimote and send it out over the serial port, we adapted this code to fit our needs. They do not require any type of compensation or royalty for use of the code, but it is suggested that we can contact the creators at CodePlex to show them our design.

The similarity to the above patent makes patent opportunities questionable. Though we could not find a project that particularly used a Wiimote and IR LED drum sticks, it is entirely plausible that a patent is already being created, or has been created.

Ethical Considerations

In the lab, there was one other group using Bluetooth communication, in addition to many Bluetooth devices in the lab. To minimize any possible interference with the other Bluetooth group, we sat as far away from them as possible and from any Bluetooth enabled laptops. In addition, the two Bluetooth groups assisted each other with any information we had relating to the two projects. We constantly stayed in contact and if we made any interesting advancements, we were sure to communicate them to the other group. Unfortunately, while we had similar problems, they were usually solved in completely different ways since we were using two entirely different operating systems to run our Bluetooth to serial programs. But nonetheless group communication provided both groups with different perspectives on different implementations of similar code. Aside from Bluetooth communication, when playing our synthesized drum noises in the lab we kept our speaker volume as low as possible to minimize distractions to other groups. We also warned people when we were worried that an annoying noise would be produced. In one particular case, we had to test when said annoying noise would occur and we kept the volume very low and shut off the system immediately after the noise was produced. We tried to be very mindful of the close space in the lab and respect the space of others around us.