Audio Fidelity: Latency

Audio Fidelity: Latency

Relativity theorizes that there is no such thing as simultaneity between cause and effect. Effect happens after cause. All processes take a nonzero amount of time to complete. This delay has a name (of course). It's called latency. In audio, latency (or filter latency) is the amount of time between a change in signal on the input of an audio filter and the corresponding change on the filter's output.

With latency, lower is almost always better. The most obvious problem with high latency is a kind of cognitive dissonance, where aural and visual stimuli arrive out of sync. People's lips move out of time with their voices, or a gunshot bang happens noticably after the muzzle flash. This, in itself, isn't that unusual. Sound travels approximately 1100ft/sec, so if you're standing 110' from the television, the audio will lag about 100ms behind the video even if the sync is perfect. However, at 110' away, we also hear more background noise, signal attenuation, and a lot of visual cues that the speaker is some distance away. If you add processing latency, the average person will notice a timing discrepancy as little as 50 milliseconds late, and even as low as 20ms latency, the signal will seem "off". The sound would behave as if the person is standing over twenty feet away, yet few of us sit that far from our computers.

On the PC, latency is one of the most important audio performance metrics. The PC audio path consists of several "filters" which perform processing on the signal. The application, the API layer, driver system effects, the kernel streaming layer, the driver, and finally the hardware. Each of these has latency associated with it. The hardware or software may also need time to initialize. The mixer needs to copy and merge buffers. Kernel transitions generate interrupts and take time. The thread that needs to process the samples may not be scheduled to run yet. (When you come down to it, there are no real-time consumer operating systems, and you can never be guaranteed to get CPU time when you need it). The most useful latency metric is usually the full end-to-end system latency, which covers all parts of this. System latency is the delay of all parts of the audio system in the OS and the hardware. The system latency is the delay between an application calling an API to play sound, and that sound coming out of the speaker jack, and represents everything outside the control of an app.

Latency is one of those things in audio that can't be eliminated, so it must be minimized. If you build anything that handles audio, you'll want to reduce latency as much as possible. It's always easier to add a delay than to remove one.