Audio-visual equipment comprises the largest segment of the global consumer electronics market, accounting for over 90% of the $250 billion-plus sector's total value, according to research firm Datamonitor. Within this sector, audio processing becomes a key technology, given that high-quality audio is a fundamental requirement in CD players, hi-fi systems, sound bars, car infotainment systems, portable audio players, and media players, as well as being a critical element alongside video in digital TVs, home theater systems, DVD and Blu-ray Disc™ players, and tablets. Consumers want superb audio quality in all of these products.

Connected Audio EverywhereIncreasingly, consumers want their audio products to be well connected with the right audio input, output, and connectivity interfaces. "Must-have" interfaces for many devices include a microphone, line-in/out and S/PDIF as well as speakers and a headphone interface. The connectivity options are also more varied than ever; consumers want to stream audio from the Internet, play back music from a USB stick, enjoy HD audio from Blu-ray Discs, and receive multi-channel broadcast audio on a digital TV.

While there are more product opportunities than ever for high-quality audio features, the growing complexity of interfaces and high expectations regarding audio quality are challenging system-on-chip (SoC) design teams to look for new ways to bring audio products to market faster and at a lower cost.

Audio Processing Needs Hardware and SoftwareThere are many aspects to audio processing. It incorporates encoding and decoding using a variety of software codecs, established audio functions such as volume, surround balance, equalizers, treble and bass boost, and advanced audio post-processing using proprietary algorithms from leading audio labs.

There are hundreds of audio codecs supporting many file formats, but their basic purpose remains the same – to enable audio data to be stored using the minimum amount of storage and transported using the minimum bandwidth while maintaining the best quality. Design teams targeting consumer products must provide support for many codecs and certainly the most popular ones, such as MP3, FLAC, AAC-LC, Ogg/Vorbis, WMA, Dolby Digital Plus, and DTS-HD Master Audio.

Once decoded, audio post-processing is used to make the audio sound even better. Post-processing can take a 2.0 channel stereo source and create a 5.1 channel surround sound experience for six speakers – two front speakers, two rear speakers, a center speaker and a Low Frequency Effects (LFE) channel for the subwoofer (the .1). Innovative audio research companies like Dolby Laboratories and DTS invest in creating algorithms and proprietary standards to enhance their customers' audio experience.

Implementing audio processing starts with a hardware platform including IP, for example a single- or dual-core audio processor, analog/digital converter IP, and digital interfaces. Design teams can implement the codecs and post-processing in software running on the audio processor and connect them to the relevant interfaces using software device drivers.

With all the building blocks in place, the hardware platform and a software library of basic audio functions, codecs and drivers, design teams can create solutions to deliver advanced audio features and tackle complex consumer use cases.

Common Use Cases

Audio PlaybackThe most basic function of an audio system is to play back an audio file! Figure 1 shows how software and hardware integrate to support audio playback of MP3, FLAC, or any other file format. The subsystem decodes the file and performs the required post processing to adjust volume, bass, and treble before sending the output simultaneously to an analog headphone and a digital S/PDIF output.

Figure 1: Audio playback sequence

Digital TV: Watch and Record Different ChannelsCreating a subsystem that allows consumers to watch one TV channel while recording another is a more complex use-case example.

Figure 2 shows how hardware and software combine to take two streams and decode them in parallel before down-mixing. The viewer watches one channel while programming on the other channel is decoded, down-mixed, encoded, and streamed to the hard disk for viewing later.

Figure 2: Watching and recording two different TV channels at the same time

Post Processing: Advanced Volume Leveling A third use-case illustrates how more sophisticated post processing enhances the audio experience for consumers.

One of the more annoying sound artifacts when watching broadcast TV is the extreme variation in volume that can occur when switching between different channels, or even on the same channel when the broadcast switches to commercials and back. Until recently, viewers had no choice but to grab the remote and adjust the volume to suit, and then repeat the process again at the end of the commercial break.

An example of how advanced sound processing can further enhance the user experience is advanced volume leveling. While standard volume control mechanisms respond to changes in volume by boosting the audio signal at low amplitudes and reducing it at high amplitudes, such an approach does not address the nuances of what is happening within the individual frequency bands. Advanced audio processing can make adjustments based on perceived loudness, allowing users to adjust the volume to their preferred setting once and then enjoy listening to a sound level that is consistent with the original recording.

Advanced volume leveling will become something that users come to expect even on portable audio players. Design teams need a way to take emerging, more complex audio functionality and quickly and cost-effectively integrate it into their SoC designs for audio products.

Synopsys provides audio post-processing software components for its ARC™ audio processors, including TruVolume from DTS (formerly an SRS Labs product) and Dolby Volume from Dolby Laboratories, which aim to produce a constant volume level, regardless of any volume variation from the broadcast. Both solutions manage playback volumes by analyzing and adjusting the volume levels in real time. Dolby and DTS have their own algorithms for this, and Synopsys offers both, allowing its customers the choice of either.

Synopsys continues to extend the SoundWave Audio Subsystem by frequently adding new software features – like advanced volume leveling – to its portfolio. Synopsys talks to its customers' product teams to understand user requirements and its audio specialist research and development team works closely with all of the leading audio labs, including Dolby Laboratories and DTS, to bring their latest audio processing techniques to market.

Typically, the audio labs develop new post-processing algorithms that Synopsys' R&D team optimizes to run with maximum efficiency on the ARC audio processors. The audio labs then certify the Synopsys implementations using their internal tools and processes. This reduces risk for customers and saves them from having to go through the time consuming codec-level certification process.

ARC Audio ProcessorsThe 32-bit ARC audio processors are at the heart of the SoundWave Audio Subsystem. The ARC processors have a good track record of successful application in set-top boxes, digital TVs, and mobile multimedia products. They are optimized for processing multiple high-definition, multi-channel audio streams in parallel.

Design teams can easily configure and optimize the ARC cores to match their applications, resulting in area-efficient and high-performance hardware solutions.

The dual-core ARC processor provides more processing capacity at lower clock frequencies than a single-core processor and offers the benefit of allowing users to further partition their systems. Some design teams choose to offer a small second core for their end customers to use, enabling them to add their own software without interfering with the rest of the system.

ARC processors offer excellent tolerance-to-memory latency, which is just one of the parameters that we optimize for.

System OptimizationThe Synopsys R&D team pays a lot of attention to optimizing software for power, performance, memory, and latency.

While it might be possible to run a core at gigahertz clock speeds, design teams would much prefer to run them at the lowest possible speed because the chip will run cooler and consume less power. By optimizing software to consume the minimum required number of processor cycles, design teams have the option of keeping clock speeds low or running multiple pieces of code, for example, audio codecs, simultaneously on the same core.

By their nature, audio applications process a lot of data, creating a high demand on system memory. Data needs to be stored (typically in RAM) before, during and after processing, and the program code itself initially resides in ROM. We aim to minimize the use of both RAM and ROM, which directly relates to chip area, and therefore cost.

Synopsys designs its software to work with the hardware so that the system is highly tolerant to the impact of memory latency. Even when sharing DDR memory with other parts of the SoC, the ARC processor load remains low, making it unnecessary to increase its clock speed.

Certified ARC Audio Software Synopsys first thoroughly tests all codecs against standards and certification requirements to ensure that they are fully compliant, and then has the codecs certified by the owner of the standard, whether it's Dolby Laboratories, DTS, Microsoft, Digital Wave, or one of a number of other partners. Synopsys delivers each codec as 'C' source code for easier integration, along with a test harness that runs out of the box on an FPGA or simulator. Comprehensive documentation, including datasheets, release notes, quick start guides, reference documents and code examples, is included with Synopsys software IP.

Developers can run multiple instances of codecs in parallel, and Synopsys supplies multi-codec use-cases, including Blu-ray Disc audio, for reference. The Synopsys R&D team develops and supports all of the codecs that it supplies.

As well as providing software codec IP for all of the popular audio formats, Synopsys supports more advanced audio post-processing algorithms from the leading audio labs, including Dolby Laboratories' Dolby Virtual Speaker and Pro Logic IIz, DTS' TruSurround HD, and WOW HD plus all the audio formats required for Blu-ray Disc.

Integrated Software StackFigure 3 shows the SoundWave subsystem's Media Streaming Framework (MSF), which enables developers to easily integrate and use all of the software functions. The MSF utilizes all of the embedded audio features, including source/sink, decoding/encoding, and post-processing elements.

The GStreamer plug-in is an application-programming interface (API) that contains all the available features in the audio subsystem. The plug-in takes care of all communication between the subsystem and host processors, enabling plug-and-play integration between application software running on the host and the audio subsystem. System integrators can easily embed all available audio functions into their application software using the SoundWave GStreamer plug-in.

SummarySoftware plays an increasingly important role in enabling design teams to deliver high-quality audio solutions for consumer audio products.

Synopsys has invested in creating a world-class research and development center focused on delivering software IP for the audio market. We optimize our broad portfolio of audio software for ARC audio processors, including multiple codecs and post-processing applications, in order to achieve the best performance at the lowest power for audio applications. By packaging software IP as part of a complete solution of hardware and software, we enable design teams to integrate advanced audio features, such as volume leveling, into their SoCs with lower risk and higher productivity.

About the AuthorHenk Hamoen is senior product marketing manager for the DesignWare SoundWave Audio Subsystem and ARC Audio IP product lines at Synopsys. Henk has over 15 years of experience in the semiconductor industry and has held various marketing and program management positions at Virage Logic, NXP Semiconductors and Philips. Henk holds a B.Sc. EE from Saxion University of Applied Sciences in The Netherlands.

Having read this article, will you take a moment to let us know how informative the article was to you.Exceptionally informative (I emailed the article to a friend)
Very informative
Informative
Somewhat informative
Not at all informative