Why Everyone Is Talking About xHE-AAC
xHE-AAC is the latest Advanced Audio Coding (AAC) technology, and is available to all licensees of the AAC patent pool administered by Via Licensing Alliance. It was developed by MPEG Audio (ISO/IEC SC29/WG6), with significant contributions from Fraunhofer IIS and Dolby notably, and has been included in the AAC license at no extra cost since 2018.
The codec offers improved audio quality at stereo bitrates of 12Kbps to 500Kbps or above, allows adaptive streaming, and provides mandatory loudness and dynamic range control. For users, this means a much better streaming experience with uninterrupted, seamless switching when on the move and consistent volume whatever the device or environment.
It is already supported in Android, iOS and FireOS for many years and, since October 2022, also in Microsoft Windows 11 for PC and Xbox.
Netflix and Meta Endorse xHE-AAC
Netflix started streaming xHE-AAC to compatible Android Mobile devices in 2021. In a post on the Netflix Technology Blog, Phill Williams and Vijay Gondi described the codec as “a sonic delight” to users.
Consistent with the North American broadcasting standard, Netflix aims to play all dialogue at the same level, meaning loudness management is critical. As the blog post states: “When it is working effectively, once you set your volume to a comfortable level, you never have to change it, even as you switch from a movie to a documentary, to a live concert.”
The blog post adds that Dynamic Range Control (DRC) optimizes “the dynamic range of a program to provide the best listening experience on any device, in any environment.” Added to loudness management, they say, it can provide an optimal listening experience wherever the user is.
In testing, Netflix found less dissatisfaction with the overall volume level, less changes to volume levels, and 16% fewer people switching to headphones rather than built-in speakers when listening to high dynamic range content using xHE-AAC.
Meta has also embraced xHE-AAC. In a post in April 2023, audio software engineer Zack Cohen described the codec as “the vehicle for delivering high-quality audio at scale,” saying it delivers a superior audio experience on both Facebook and Instagram.
One of the challenges Meta faces is that it delivers diverse types of media–from short-form, user-generated content such as Reels to premium video on demand and live broadcasts. “As a result of xHE-AAC’s loudness management, people can spend more time immersed in their favorite content and less time fiddling with the volume control,” Cohen wrote.
He also highlighted the concept of immediate playout frames (IPFs) in xHE-AAC, which enable seamless audio adaptive bit rate (ABR). Switching between audio lanes during playback provides high-quality audio at any bandwidth and avoids playback stalls, according to Cohen: “After launching audio ABR on Facebook for Android, we were able to improve user experience by reducing the number of sessions where playback stalls.”
The Evolution of AAC
Why is xHE-AAC so important? To answer that question, we need to look back at the development of the AAC family of audio codecs. Over the past two decades or more, the family has been enhanced with continuous improvements, each of them backward compatible and covered by the AAC patent pool.
The original MPEG-2 AAC codec was refined through the MPEG-4 standardization process. One of the early developments was low complexity AAC (AAC-LC), published in 1999. With the later development of streaming services, it became ubiquitous in devices around the world.
AAC-LC was followed High Efficiency AAC (HE-AAC) , which incorporated spectral band replication (SBR) to enhance audio at low bit rates, and then HE-AAC v2, which added parametric stereo (PS), to efficiently encode stereo signals. HE-AAC v2 was well suited to portable media devices, and was widely taken up in the industry.
Now xHE-AAC is delivering further enhancements that dramatically improve users’ experience when streaming media. These enhancements include better compression, better coding for speech signals and MPEG-D DRC, as the comments by Meta and Netflix show.
The AAC Patent Pool
Altogether, these technologies are protected by more than 6,000 patents owned by more than a dozen licensors, including AT&T, Dolby, Fraunhofer IIS, NTT, Philips, and Samsung. In fact, thanks to the addition of new patents, the portfolio in the pool has more than doubled in the past five years.
Through Via LA’s AAC patent pool, nearly 1,000 companies have a license to all of these patents, including computer, telecoms, audiovisual, and streaming media entities. New licensees continue to be added. For example, smart device brand OPPO announced that it had secured a license in February this year.
The AAC patent license also covers AAC-LD (Low Delay), AAC-ELD (Enhanced Low Delay) and AAC-ELD v2. These enable high-quality audio for videoconferencing and other communications applications where low-latency performance is critical.
Although AAC has continually incorporated new codecs, licensees can be reassured that they always have coverage for the latest iterations.
Never Bet Against Bitrate
At each stage of the evolution of AAC, the efficiency of audio compression (i.e., the audio quality per bit) has increased. This means that users experience smoother streaming, without buffering or loss of quality, even on congested 4G and 5G networks and in emerging markets where 2G/3G networks still dominate.
Jan Nordmann, Senior Director Business Development, New Media at Fraunhofer USA Digital Media Technologies DMT, says, “With the use of xHE-AAC by Netflix and the Meta owned Facebook and Instagram services, we estimate that over two billion hours of xHE-AAC content are streamed every month to more than two billion people worldwide, and the number will only increase. Due to the mandatory support for MPEG-D DRC loudness and dynamic range metadata, xHE-AAC provides a more consistent consumer experience on any device and in any listening environment.”
Christof Fersch, Director of Standards and Engineering at Dolby in Nuremberg, adds, “Even as bandwidth on networks increases, compression is still vital to keep down costs and deliver content in a seamless fashion, even in congested networks. More efficient audio coding also means that video quality can be improved. Never bet against bitrate!”
Streaming is the Future
Consumer demand for streaming and live content is increasing, especially as content producers offer higher quality media, from movies to concerts, to music, audiobooks and podcasts. Ensuring that users get a quality, consistent experience wherever they are and whatever device they are using is therefore vital and xHE-AAC delivers that.
Related Articles
Following on the news that Via Licensing has acquired codec patent pool administrators MPEG LA to form Via LA as a "unified entity," Via LA President Heath Hoglund discusses how the pool will operate, as well as resulting synergies including streamlined IP distribution and economies of scale in this interview with Streaming Learning Center's Jan Ozer.
08 May 2023