AV1 Is Finally Here, but Intellectual Property Questions Remain
After three years and multiple delays, the Alliance for Open Media froze its AV1 codec bitstream, thus setting up a cacophony of HEVC/AV1 face-offs in booths and meeting rooms at the upcoming NAB Show. These comparisons will largely dictate which codec will replace H.264 as the "it" codec for streaming to desktops, mobile devices, Smart TVs, and OTT devices, though the intellectual property (IP) picture remains uncertain.
By way of background, the Alliance for Open Media is a Joint Development Foundation Project formed in 2015 in response to the uncertain royalty situation surrounding HEVC. Software released by AOM is offered on an irrevocable, non-exclusive, and royalty-free basis so long as the licensee is in conformance with the terms of the Alliance for Open Media Patent License 1.0. As shown in Figure 1, members include dominant players across the codec development, video encoding, delivery, and playback ecosystem, ensuring the fastest possible technology adoption in all major submarkets.
Figure 1. AOM members and initial deployers. (Click to enlarge)
Bitstream Freeze
On March 29, AOM announced the public release of the AOM Video 1.0 AV1 specification, aka the bitstream freeze. They also released an "unoptimized, experimental software decoder and encoder to create and consume the bitstream, reference streams for product validation, and binding specifications to allow content creation and streaming tools for user-generated and commercial video."
According to the AOM representatives, at NAB several members will show 30-40% better quality for UHD 4K videos than VP9/HEVC. Encoding times are currently about 100X slower than VP9, which they feel will drop to 5X by the end of the year. For decode, AOM is currently about 5X slower then VP9 on the x86 platform, which should drop to 2X by the end of 2018.\
Figure 2. The AV1 development timeline
Figure 2 shows the anticipated deployment timeline. According our conversations, AOM expects AV1 decode in several browsers and some content from member companies over the next few months. This will be followed by hardware implementations in about 12 months that can be integrated into devices that will ship in early to mid-2020.
Given the AOM membership, AV1's success in the streaming marketplace seems almost assured, particularly given that HEVC has made little progress to date in markets other than streaming to Smart TVs. Since its announcement however, AV1 has been plagued by concerns regarding its intellectual property status.
The Changing IP Picture
Specifically, multiple sources have questioned AOM's ability to ship a codec that doesn't infringe upon existing IP that might be used in the H.264 and HEVC codecs. For example, a FAQ on HEVC patent group Velos Media's website states, "we know that VP9 incorporates patented technologies, including some of the patents being licensed by Velos Media for HEVC. And, while AV1 has not yet been publicly released, it may also incorporate patented technology from many parties."
During our launch-related conversations, AOM execs elaborated on their development approach and their efforts to ensure that AV1 didn't infringe upon existing IP. This started with a recital of the companies in the Alliance and the IP and codec development histories that they brought to the table. For example, Google brought all On2, Google, and Motorola IP that it acquired in 2011. Microsoft brought all the IP from the Windows Media development effort, while Cisco brought its own Thor-related development efforts, plus those it acquired from Scientific Atlanta and Tandberg. Intel brought its own technology plus technology acquired from RealNetworks in 2012, while BBC, IBM, and Apple brought their own technology. Mozilla brought its Daala-related and other IP, plus years of codec development experience.
All technology was vetted in a rigorous patent review process before being integrated into the final spec. The AOM sources emphasized that when designing the codec, if they couldn't work around a specific technique protected by a patent, they invited the patent owner to join the Alliance, even if they were a member of an existing patent pool. In this regard, note that Apple, Cisco, Google and Microsoft are all licensors in the MPEG LA H.264 patent pool, and may be contributing the same IP to AOM.
In addition, AOM created a legal defense fund so if one of the smaller member companies or an AV1 licensee is sued, there will be sufficient funds for a defense. That said, AOM will not indemnify licensees if AV1 is found infringing on any third-party technology.
I asked attorney David W. Long, owner of the Essential Patent blog, about the AOM intellectual property announcements. After the normal cautions that all parties should check with their own patent counsel for specific advice, Long confirmed that each patent owner is free to license its technology to multiple groups or pools as they wish, including charging for some groups, and on a royalty-free basis for others. Long felt that the legal defense fund was a creative solution for protecting potential AV1 licensees, though he again cautioned that he had not researched this specific issue and there might be some unknown restrictions or limitations.
Given the statements made by Velos, I asked Long what steps potential AV1 licensees should take to protect their IP interests. While he felt that Velos' statements probably fell short of a true notice of infringements, he also suggested that potential licensees should do their own diligence to obtain a "reasonable good faith belief that there won't be an infringement. For example, they can't simply rely on AOM's bare statements alone regarding the AOM patent review process, but should seek to understand the underlying facts and reasoning behind AOM's conclusion.
In terms of the IP-related fear, uncertainty, and doubt facing the AV1 codec, it's worth noting that Google's VP8 and VP9 faced similar market noise. While Nokia did sue Google claiming that VP8 infringed upon its H.264-related IP, Nokia ultimately lost the suit, and no other claims have emerged.
It's also worth noting that none of the HEVC pools guarantee that other claims from third parties won't arise; they simply provide a convenient mechanism for licensing the IP actually in their pool. In other words, when it comes to IP, there are few, if any, guarantees. Cold comfort, if any, but that's the reality.
In terms of AV1's performance, we'll know a lot more after NAB. Unfortunately, the IP situation will take much longer to resolve.
Related Articles
The licenses cover devices such as smartphones, computers, TVs, set-top boxes, and graphics cards, but not encoded content—at least for now.
27 Mar 2019
FFmpeg 4.0 gives many video engineers their first chance to test the new AV1 codec against H.264, HEVC, and VP9. The results? In our tests, quality was impressive, but glacially slow encoding times make AV1 a non-starter for most publishers until hardware acceleration becomes available.
31 Aug 2018
Latest codec quality comparison study finds AV1 tops on quality, but far behind on speed, and that VP9 beats out HEVC. Moscow State also launched Subjectify.us, a new service for subjectively comparing video quality.
30 Jan 2018
With Apple joining the Alliance for Open Media, there's been a lot of talk about AV1 vs. HEVC. But we no longer live in an either/or world, and can now give viewers what they want with minimal cost and effort compared to years gone by.
17 Jan 2018
Mozilla's Timothy Terriberry, Brightcove's David Sayed, and Twitch's Tarek Amara debate the future of streaming codecs at Streaming Media West 2017.
04 Dec 2017
Netflix and YouTube could start using AV1 as soon as early 2018, while hardware implementations will take much longer.
30 Aug 2017
Some of the best minds in the streaming industry say AV1 will be challenged by infringement claims, but some of the best minds could be wrong.
31 Jul 2017
Scheduled to be the first codec released by the Alliance for Open Media, AV1 is positioned to replace VP9 and compete with HEVC. While we don't know many details yet, the backing of the Alliance should give AV1 a significant competitive advantage.
03 Jun 2016
The cross-industry open source Alliance welcomed new members and is making strides with AV1. Meanwhile, HEVC's future seems more uncertain than ever.
12 Apr 2016