Monday, August 2, 2010

MonoTouch_AudioUnit_SampleCodes

Abstract:I demonstrate three AudioUnit sample codes using MonoTouch: sine waveform generator, AIFF audio file player, and monitoring microphone. Comparing to other frameworks (Audio Queue Service and OpenAL), AudioUnit can realize real-time audio rendering and the lowest output latency (~ 3 msec), which is suitable for instrument or sound-triggerd application. The sample codes are in the github under the MIT license so that you can use it freely [1].

In my software modem project, I once decided to use AudioQueue Serivce to access speaker and microphone. But I need smaller output latency than 23 msec, which is the lowest achievable value with Audio Queue Service. Therefore I try to use AudioUnit which is the lowest-level audio API where its output latency is up to 3 msec.

MonoTouch AudioUnit marshaling

AudioUnit is a set of C language methods included in AudioToolbox framework.Therefore interoperating with unmanaged code is required to use it from MonoTouch [2]. Details of the interoperation is described in the wrapper classe source codes (a project MonoTouch.AudioUnit in the sample code) which correspond to AudioUnit method and struct definitions one by one.

Because Audio unit sample type is different between a simulator (32-bit float) and a device (32-bit int), the difference is absorbed in the code (line #21-28). Each time audio unit requires buffer rendering, a render callback event handler (line #45) is invoked as following:

1: void callback(object sender, AudioUnitEventArgs args)

2: {

3: // Generating sin waveform

4: double dphai = 440 * 2.0 * Math.PI / _sampleRate;

5:

6: // Getting a pointer to a buffer to be filled

7: IntPtr outL = args.Data.mBuffers[0].mData;

8: IntPtr outR = args.Data.mBuffers[1].mData;

9:

10: // filling sin waveform.

11: // AudioUnitSampleType is different between a simulator (float32) and a real device (int32).

Audio buffer is a 32-bit float or 32-bit integer array. Therefore arbitrary waveform can be written in the buffer within unsafe block as above. Array length can be set using AudioSesion framework, whose default value is 512 and must be more than 128. Smaller array length can achieve less output latency while it needs more processor power.

Other examples
I omit explanation of other examples, playing audio file and recording from the microphone. Please read the source code :-P

16 comments:

One suggestion, rather than an if/then in the callback, I'd prefer to see two separate callback methods, one for Simulator and one for Device, and assign the appropriate handler to the RenderCallback event handler..

It is possible by replacing AudioStreamBasicDescription format of the sample code http://github.com/reinforce-lab/MonoTouch.AudioUnit/blob/master/MonoTouch_RemoteIO_PlayThroughWithMonitoring/RemoteIOPlayThrough.cs , ExtAudioFile supports various audio formats including AAC.

I have not tried to handle AppleIMA4 format, so AudioUnit wrapper class may have some problem.AudioUnit wrapper classes throw an exception when AudioUnit library function returns an error code, witch is OSStatus enum value (you can get value means here : http://www.benjiegillam.com/2009/07/osstatus-iphone-sdk-meanings/ ). Chekking an error code within the function CreateWithURL by setting a breaking point may be starting point.

> 1. mic > speaker I am just writing ;-P. You can get a C/C++ sample code from http://developer.apple.com/mac/library/samplecode/RecordAudioToFile/Introduction/Intro.html .To record, getting a remote io audio unit by calling audio component functions, and call AURender function to get mic signal in the callback function.AURender function needs a memory allocated audio list buffer, which I am just writting a wrapper class for it (it may take a month, it is not high priority task item...).

>2. UV meters how ?I do not know. Perhaps, I have to read the technical document http://developer.apple.com/mac/library/technotes/tn2002/tn2091.html .

One question, The "PlayThrough" example, I got AAC recording working just fine. But how can I mute (or set volume to zero) on the speaker or better mic that goes to speaker without muting sound to recording?

Design service for you

Reinforce Lab. provides software and hardware design services. Our service using .NET technology is the most cost-effective for the multiple platforms including Windows, iPhone, ARM7 micro controllers and Android.