What Is AV1?
This is another installment in our series of "What Is...?" articles, designed to offer definitions, history, and context around significant terms and issues in the online video industry.
The AV1 codec will be the first codec released by the Alliance for Open Media (AOM), and it's scheduled to ship sometime between December 2016 and March 2017. It is positioned to replace Google's VP9 and to compete with the standards-based High Efficiency Video Codec (HEVC). While the codec's performance won't be known until it ships, one significant competitive advantage is the Alliance membership, which ensures the prompt deployment of AV1 playback in browsers, mobile devices, OTT, and smart TVs and the distribution of AV1-encoded content by YouTube, Netflix, and Amazon. HEVC enjoys a multiple year lead in hardware-based deployments, however, and some pundits question whether AV1 was created without infringing upon patents owned by H.264, HEVC, and other video-related IP owners.
The Alliance for Open Media
The Alliance for Open Media was announced on September 1, 2015 with charter members Amazon, Cisco, Google, Intel Corporation, Microsoft, Mozilla, and Netflix. At the time, the formation consolidated the development of three potentially competitive open source codecs; Cisco's Thor, Google's VP10, and Mozilla's Daala. According to the initial press release, the goal was to produce a "next-generation video format" that is:
- Interoperable and open
- Optimized for the web
- Scalable to any modern device at any bandwidth
- Designed with a low computational footprint and optimized for hardware
- Capable of consistent, highest-quality, real-time video delivery
- Flexible for both commercial and non-commercial content, including user-generated content.
According to the AOM license terms, all licensors receive a "perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as expressly stated in this License) patent license to its Necessary Claims to make, use, sell, offer for sale, import or distribute any Implementation." There is no requirement for licensees to disclose any of their own code. The Alliance will pursue all codec developments via an open source repository.
On April 5, 2016, the Alliance announced that IP provider ARM and semiconductor companies AMD and NVIDIA joined the Alliance to help ensure that the codec was hardware-friendly, and to facilitate and accelerate AV1 hardware support.
In terms of makeup, the Alliance members enjoy leading positions in the following markets:
- Codec development - Cisco (Thor) Google (VPX), Mozilla (Daala)
- Desktop and mobile browsers - Google (Chrome), Mozilla (Firefox), Microsoft (Edge)
- Content - Amazon (Prime), Google (YouTube), Netflix
- Hardware co-processing - AMD (CPUs, graphics), ARM (SoCs, other chips), Intel (CPUs), NVIDIA (SoC, GPUs)
- Mobile - Google (Android), Microsoft (Windows Phone)
- OTT - Amazon (Amazon Fire TV), Google (Chromecast, Android TV)
These positions should enable the Alliance to ensure the fastest possible integration of AV1 into the members' products and services, and to influence industry acceptance as a whole. For example, once Netflix and YouTube start deploying AV1-content to browser-based viewers, this should influence the manufacturers of Smart TVs, set top boxes, and competitive OTT devices to support the format as well, which will be simplified by the availability of AV1-compatible hardware from other Alliance members.
About the AV1 Codec
As mentioned, AV1 is expected to ship as early as December 2016, and no later than March 2017. The initial AV1 implementation will incorporate some features of Daala and Thor, but most of the code will come from VP10, which was scheduled to ship by the end of 2016.
The Alliance is targeting an improvement of 50 percent over VP9/HEVC with reasonable increases in encoding and playback complexity. One focus is UHD video, including higher bitrate, wider color gamut, and increased frame rates, with the group targeting the ability to play 4K 60fps in a browser on a reasonably fast computer. The base version of the codec will support 10-bit and 12-bit encoding, as well as the BT.2020 color space. Another focus is providing a codec for WebRTC (Real-Time Communications), an initiative supported by Alliance members Google and Mozilla, and similar applications including Microsoft's Skype.
Once available, YouTube expects to transition to AV1 as quickly as possible, particularly for video configurations such as UHD, HDR, and high frame rate videos where the codec is expected to deliver significant bandwidth savings over VP9. Based upon its experience with implementing VP9, YouTube estimates that they could start shipping AV1 streams within six months after the bitstream is finalized. The Alliance expects the first AV1-compatible hardware components to ship within 12 months after bitstream finalization.
Market Positioning
AV1 is positioned to compete directly with HEVC in most major streaming-related markets, including browser-based streaming, mobile, OTT, smart TVs, and set-top boxes, which are the primary markets served by Alliance members. Precise positioning won't be known until the codec is released and tested and comparative quality and CPU playback requirements can be assessed.
AV1's most distinct competitive advantage is that it is royalty free, while HEVC comes encumbered with known royalties on devices via the MPEG LA and HEVC Advance patent pools. The HEVC Advance pool also charges content royalties for HEVC-encoded videos distributed on physical media, or for web video distributed via subscription or pay-per-view. Originally a member of the HEVC Advance pool, Technicolor withdrew from the pool, preferring to license its IP directly. There are also multiple known HEVC IP owners who have not joined a pool or announced if and how they intend to license their HEVC-related IP. The disparate actions taken by the various HEVC rights holders makes licensing HEVC both expensive and cumbersome.
Beyond deployment cost and complexity, if the Alliance meets its goals, AV1 should also have a significant quality advantage over HEVC, though this may be offset by higher CPU playback requirements and/or longer encoding times. As previously stated, the Alliance membership also creates a competitive advantage with the ability to quickly implement AV1 and influence others to do the same.
HEVC's most significant advantage is that it's a multi-vendor standard with a significant headstart and hardware support in many key markets, including mobile (hardware support on Apple, software support on Android), smart TVs, set-top boxes, and OTT devices, as well as very significant support among mainstream encoding vendors. However, HEVC playback is not available in any desktop/notebook browsers, while VP9, AV1's technology precursor, is available on the most recent versions of all browsers except for Apple Safari.
It does not appear that AV1 will compete with Google's VP9, as most major users, like Google and Microsoft, have announced their intentions to support AV1 once available. Both will continue to support VP9 to the extent that it enables them to distribute to computers and devices that AV1 can't serve due to the lack of hardware support or other technical reasons.
AV1 Intellectual Property Status
Some technology observers assert that it's likely that AV1 will include some intellectual property (IP) that conflicts with existing patents. For example, in an article in Streaming Media, Black Stone IP's Elvir Causevic and Ed Fish stated, "The essential point is that a lot of inventive work went into core technologies that underlie HEVC, such as novel quad tree structures, sophisticated tiling, hardware optimization, and scalability. One would have to assume that at least some of these core innovations would be used or extended in the codec the Alliance is proposing, and therefore patents relating to that functionality would continue to apply."
It's also clear that at least some Alliance members foresee potential legal challenges. In a comment discussing the Alliance for Open Media (AOM) published on LinkedIn, Netflix's director of encoding technology David Ronca stated, "If AOM is a technical success, and the codec survives legal challenges, then the era of royalty-based codecs will end (as it should)."(Emphasis supplied by Ronca.)
Beyond the financial war chest necessary to fight a patent infringement suit, the Alliance members bring another bargaining chip to the table. As Causevic and Fish also state, "many of these companies have meaningful pre-existing portfolios and cross licenses with a very large set of companies throughout the industry." In other words, just because there is an infringement doesn't mean that it will automatically take AV1 off the market, or convert it into a royalty bearing codec. At this point, the only certainty is that the more success the AV1 codec achieves, the greater the likelihood of an IP-related challenge.
Related Articles
Today the Alliance for Open Media froze the AV1 bitstream and released an unoptimized software encoder and decoder; AV1 decode should arrive in several browsers and some content from member companies over the next few months, with hardware implementations in about a year.
28 Mar 2018
Netflix and YouTube could start using AV1 as soon as early 2018, while hardware implementations will take much longer.
30 Aug 2017
Hardware acceleration and field programmable gate arrays may be the answer to the rising costs of encoding for multiple codecs including H.264, H.265, VP9, and soon AV1
03 Aug 2017
Some of the best minds in the streaming industry say AV1 will be challenged by infringement claims, but some of the best minds could be wrong.
31 Jul 2017
For educational video, some choose to shoot in 4k so they can zoom and enhance in post, but stream in mobile-friendly 720p. However, there's no reason to rush to 4K just yet.
01 May 2017
Online is different from broadcast and doesn't need formal standards. HEVC isn't considered by many online video streamers, as the future belongs to VP9 and AV1.
08 Jun 2016
VP9 is the open-source codec from Google, and provides a royalty-free alternative to HEVC. It's more efficient than H.264, and while it's less efficient than HEVC, it compares well on quality.
26 May 2016
Not sure what to make of the new format on the block? Read this to get up to speed on how HEVC was created, the challenges it now faces, and when it will go into everyday use.
14 Feb 2013
If you're going to understand online video, you need to understand codecs. Here's a high-level look at what codecs are, as well as the variety of codecs relevant to streaming media.
21 Mar 2011