"programming": ["audio", "low-level", "iOS"]

Tag Archives: callback

Over the past month or so, I’ve been working with a friend and colleague, George Hufnagl, on building an iOS app for audiophiles that includes several useful functions and references for both linear and interactive media. The app is called “Pocket Audio Tools” and will be available sometime in mid-August for iPhone, with iPad native and OS X desktop versions to follow. Among the functions included in the app is one that displays frequency values for all pitches in the MIDI range with adjustable A4 tuning. As an extension of this, we decided to include playback of a pure sine wave for any pitch selected as further reference. While standard audio playback on iOS is quite straightforward, building a tone generator requires manipulation of the actual sample data, and for that level of control, you need to access the lowest audio layer on iOS — Audio Units.

To do this, we set up an Objective-C class (simply derived from NSObject) that takes care of all the operations we need to initialize and play back our sine tone. To keep things reasonably short, I’m just going to focus on the specifics of initializing and using Audio Units.

Here is what we need to initialize our tone generator; we’re setting up an output unit with input supplied by a callback function.

Above, we set our type and sub type to an output Audio Unit in which its I/O interfaces directly with iOS (specified by RemoteIO). Currently on iOS, no third-party AUs are allowed, so the only manufacturer is Apple. The two flag fields are set to 0 as per the Apple documentation. Next we search for the default Audio Component by passing in NULL as the first argument to the function call, and then we initialize the instance.

Once an Audio Unit instance has been created, it’s properties are customized through calls to AudioUnitSetProperty(). Here we set up the callback function used to populate the buffers with the sine wave for our tone generator.

After setting the callback function RenderTone (we’ll define this function later) and our user data (self in this case because we want the instance of the class we’re defining to be our user data in the callback routine), we set this property to the AU instance. The scope of the Audio Unit refers to the context for which the property applies, in this case input. The “0” argument is the element number, and we want the first one, so we specify “0”.

Next we need to specify the type of stream the audio callback function will be expecting. Again we use the AudioUnitSetProperty() function call, this time passing in an instance of AudioStreamBasicDescription.

The format identifier in our case is linear PCM since we’ll be dealing with non-compressed audio data. We’ll also be using native floating point as our sample data, and because we’ll be using only one channel, our buffers don’t need to be interleaved. The bytes per packet field is set to 4 since we’re using floating point numbers (which is of course 4 bytes in size), and bytes per frame is the same because in mono format, 1 frame is equal to 1 sample. In uncompressed formats, we need 1 frame per packet, and we specify 1 channel for mono. Bits per channel is just calculated as 8 * sizeof our data type, which is float.

This completes the setup of our Audio Unit instance, and we then just initialize it with a call to

AudioUnitInitialize(_componentInstance);

Before looking at the callback routine, there is one extra feature that I included in this tone generator. Simply playing back a sine wave results in noticeable clicks and pops in the audio due to the abrupt amplitude changes, so I added fade ins and fade outs to the playback. This is accomplished simply by storing arrays of fade in and fade out amplitude values that shape the audio based on the current playback state. For our purposes, setting a base amplitude level of 0.85 and using linear interpolation to calculate the arrays using this value is sufficient. Now we can examine the callback function.

We only need to consider the following arguments: inRefCon, inNumberFrames, and ioData. First we need to cast our user data to the right type (our tone generator class), and then get our buffer that we will be filling with data. From here, we need to determine the amplitude of our sine wave based on the playback state.

One important thing to bear in mind is that the callback function is continually called after we start the Audio Unit with

AudioOutputUnitStart(_componentInstance);

We don’t need to start and stop the entire instance each time a tone is played because that could very well introduce some delays in playback, so the callback continually runs in the background while the Audio Unit is active. That way, calls to play and stop the tone are as simple as changing the playback state.

The reason for the extra check in stopTone() is to prevent clicks from occuring if the playback time is very short. In other words, if the playback state has not yet reached its sustain point, we don’t want any sound output.

That completes this look at writing a tone generator for output on iOS. Apple’s Audio Units layer is a sophisticated and flexible system that can be used for all sorts of audio needs including effects, analysis, and synthesis. Keep an eye out for Pocket Audio Tools, coming soon to iOS!