I'm reviewing all kinds of Android sound API and I'd like to know which one I should use.
My goal is to get low latency audio or, at least, deterministic behavior regarding delay of playback.

We've had a lot of problems and it seems that Android sound API is crap, so I'm exploring possibilities.

The problem we have is that there is significant delay between sound_out.write(sound_samples); and actual sound played from the speakers. Usually it is around 300 ms. The problem is that on all devices it's different; some don't have that problem, but most are crippled (however, CS call has zero latency). The biggest issue with this ridiculous delay is that on some devices this delay appears to be some random value (i.e. it's not always 300ms).

I'm reading about OpenSL ES and I'd like to know if anybody had experience with it, or it's the same shit but wrapped in different package?

I prefer to have native access, but I don't mind Java layer indirection as long as I can get deterministic behavior: either the delay has to be constant (for a given device), or I'd like to get access to current playback position instead of guessing it with a error range of ±300 ms...

EDIT:1.5 years later I tried multiple android phones to see how I can get best possible latency for a real time voice communication. Using specialized tools I measured the delay of waveout path. Best results were over 100 ms, most phones were in 180ms range. Anybody have ideas?

You may be interested to look into Android source code. There are some underlying clases like AudioPolicyManager and audio_policy which, as I understand, are used to work with audio on low level. They may be useful.
–
Victor RoninSep 6 '12 at 19:11

2

@JamesMoore: As of Jellybean the doc you quoted is no longer strictly accurate. Jellybean includes scheduler improvements that OpenSL ES can take advantage of but Java cannot. Please see my answer below.
–
Ian Ni-LewisSep 13 '12 at 17:54

@Phonon: 30ms is achievable on some Jellybean devices. It depends a lot on how you measure it, though. Audio output latency is only part of the issue. Touch latency is also important, and varies from device to device.
–
Ian Ni-LewisSep 13 '12 at 17:56

3 Answers
3

SoundPool is the lowest-latency interface on most devices, because the pool is stored in the audio process. All of the other audio paths require inter-process communication. OpenSL is the best choice if SoundPool doesn't meet your needs.

Why OpenSL? AudioTrack and OpenSL have similar latencies, with one important difference: AudioTrack buffer callbacks are serviced in Dalvik, while OpenSL callbacks are serviced in native threads. The current implementation of Dalvik isn't capable of servicing callbacks at extremely low latencies, because there is no way to suspend garbage collection during audio callbacks. This means that the minimum size for AudioTrack buffers has to be larger than the minimum size for OpenSL buffers to sustain glitch-free playback.

On most Android releases this difference between AudioTrack and OpenSL made no difference at all. But with Jellybean, Android now has a low-latency audio path. The actual latency is still device dependent, but it can be considerably lower than previously. For instance, http://code.google.com/p/music-synthesizer-for-android/ uses 384-frame buffers on the Galaxy Nexus for a total output latency of under 30ms. This requires the audio thread to service buffers approximately once every 8ms, which was not feasible on previous Android releases. It is still not feasible in a Dalvik thread.

This explanation assumes two things: first, that you are requesting the smallest possible buffers from OpenSL and doing your processing in the buffer callback rather than with a buffer queue. Second, that your device supports the low-latency path. On most current devices you will not see much difference between AudioTrack and OpenSL ES. But on devices that support Jellybean+ and low-latency audio, OpenSL ES will give you the lowest-latency path.

Ian, I actually performed tests and it looks that OpenSL performs worse.
–
PavelFeb 28 '13 at 21:24

Can you describe the tests you performed? There are a number of ways you can set up OpenSL ES so that its latency is higher than AudioTrack. See my answer to stackoverflow.com/questions/14842803/… for details on how to set up OpenSL to be low latency.
–
Ian Ni-LewisApr 25 '13 at 18:29

I used Android's OpenSL waveout code from WebRTC for playing audio. The idea of my test was simple: I generate two distinctive short burst of noise from loudspeaker and I pick it up on the microphone of the same device. Then I measure the time from when I wrote these bursts of noise to waveout and the time when I detected it on wavein. We need lowest possible waveout latency for real-time VoIP calls, we also need all kinds of controls available for AudioTrack, like switching earpiece/loudspeaker, switching automatic gain or acoustic echo cancellation; Is all of that unavailable with OpenSL?
–
PavelApr 25 '13 at 20:07

The WebRTC code doesn't enable fast path low latency. (Why would it?) Watch youtube.com/watch?v=d3kfEeMZ65c. The important takeaway is that if you do not set the magic buffer size and sample rate
–
Ian Ni-LewisMay 31 '13 at 20:00

IIRC, OpenSL is passed through the same interface as AudioTrack, so at best it will match AudioTrack. (FWIW, I'm currently using OpenSL for "low latency" output)

The sad truth is there is no such thing as low latency audio on Android. There isn't even a proper way to flag and/or filter devices based on audio latency.

What interface you'll want to use to minimize latency is going to depend on what you are trying to do.
If you want to have an audio stream you'll be looking at either OpenSL or AudioTrack.

If you want to trigger some static oneshots you may want to use SoundPool. For static oneshots SoundPool will have low latency as the samples are preloaded to the hardware. I think it's possible to preload oneshots using OpenSL as well, but I haven't tried.

I tried some tests, and they confirm: at best OpenSL gets the same result as AudioTrack.
–
PavelFeb 28 '13 at 21:28

As above, this is true if OpenSL is not using the fast path. If you configure it so that it can use the fast path, your latency should be significantly better on most devices.
–
Ian Ni-LewisMay 31 '13 at 20:05

The lowest latency you can get is from SoundPool. There's a limit on how big of a sound you can play that way, but if you're under the limit (1Mb, IIRC) it's pretty low latency. Even that's probably not 40ms, though.

But it is faster than what you can get by streaming, at least in my experience.

Caveat: You may see occasional crashes in SoundPool on Samsung devices. I'm have a theory that it only happens when you access the SoundPool from multiple threads, but I haven't verified this.

EDIT: OpenSL ES apparently has extremely HIGH latency on Kindle Fire, while SoundPool is much better, but the reverse may be true on other platforms.

The OpenSL ES buffer size is chosen by the OEM. Kindle Fire apparently has a very large buffer, which makes sense--larger buffers mean fewer buffer servicing calls, which means longer battery life. And you don't need low latency if all you're doing is playing music and movies.
–
Ian Ni-LewisSep 6 '12 at 23:02