Commentary: The Streaming Industry Gangs up on HEVC with the Alliance for Open Media
Yesterday a new consortium called the Alliance for Open Media announced that it's working on creating a new royalty-free codec for internet-delivered video. Here are some thoughts on the organization's efforts after a conversation with Alliance members Ian LeGrow of Microsoft and Matt Frost of Google that we reported on yesterday.
Obviously it makes little sense for Cisco, Google, and Mozilla to develop codecs independently, and in this regard, working together should allow the Alliance to deliver a better codec in less time. Beyond that, the impact of the upcoming “Alliance codec” needs to be considered in the three primary markets that deploy codecs.
First, let’s recognize that at a high level, the internet, mobile, and OTT markets are the only ones that demand a royalty-free codec. The traditional broadcast market has happily bounced along deploying royalty-bearing codecs like MPEG-1, MPEG-2, H.264, and now HEVC since it turned digital. While the recent announcements from HEVC Advance have spawned much controversy, many H.264 users wrote royalty checks to several parties, so it’s clearly not unprecedented.
In the internet space, H.264 did quite well even though it was royalty-bearing, though that was because the $5 million annual cap was close to free for companies like Adobe, Apple, Google, and Microsoft. The $25 million cap proposed by MPEG LA is another kettle of fish, while the uncapped royalty proposed by HEVC Advance likely eliminates any likelihood that any major browser vendors will include HEVC playback it its browser. More on that below.
Note that this doesn't preclude HEVC playback in browsers as browsers can utilize codecs available in the OS without paying a royalty—that’s how Mozilla, which never licensed H.264, decodes H.264 video. So if HEVC decode appears in Windows or the Mac OS, all browsers will be able to access that decode capability. But while HEVC seemed a certainty in Windows 10, that no longer seems to be the case, as you’ll see.
So what does the future hold now that another codec will be joining the fray? Let’s look at the desktop/notebook, mobile, and broadcast markets separately.
Desktop/Notebook
This is the traditional desktop market that uses full-sized, plugged-in boxes with plenty of space and power for multiple-core CPUs that can decode HEVC, VP9/10, and presumably the Alliance codec without codec-specific hardware. As Flash declines, the primary decoding engine for all these computers will be the browser.
With Microsoft in the Alliance, this should ensure that the new codec is supported in Microsoft Edge down the road, perhaps even in Internet Explorer sometime sooner. Microsoft never supported VP8 or VP9, which hindered both codec’s utility. I asked Microsoft’s LeGrow if Microsoft joining the Alliance meant that the company might support VP9 before the Alliance codec was available, and he responded, “We are not announcing our future codec support plans today; we’ll follow-up with them when we are ready.” A bit prickly, but it was early on the west coast when I asked him, so it’s understandable.
Not to judge, but Microsoft’s position on many things HTML5 is reminiscent of my eldest daughter’s approach to homework; she’ll do it tomorrow. Specifically, Microsoft seems to be using these technologies to spur adaption of its latest OSs, for example making the Media Source Extensions and Encrypted Media Extensions available only on Windows 8/10, not Windows 7, which still enjoys the lion’s share of Windows installed base. If Microsoft would incorporate VP9 playback into an Internet Explorer version that all Windows users could access, it would be a very significant move that would boost the Alliance and HTML5 playback immeasurably.
Back on point, once the Alliance codec is available, Apple will be the only major browser vendor that may not support it. I asked both spokespersons if Apple was involved, and got the stock answer that the Alliance is in discussions with many companies but couldn’t say who they were. LeGrow did mention that they anticipated a second round of members later in September.
Apple has never supported VP8/VP9 in either its browser or desktop/mobile OS, but is in an interesting position. Before HEVC Advance arrived, Apple was firmly in the HEVC camp, in fact part of the MPEG LA royalty pool. With HEVC Advance in the mix, Apple stands to pay far more in HEVC royalties than it will ever receive from MPEG LA. At the same time, the fact that the Alliance codec is from a group, rather than controlled by a competitor (Google), may make it easier for Apple to embrace. Even if you’re Apple, at this point, resistance is futile; ultimately, with all other browsers in line, Apple risks harming Safari’s market share if they don’t support the Alliance codec in Safari.
HEVC in Windows 10?
As mentioned above, once the operating system supports HEVC decode, all browser vendors can leverage that capability. In this regard, Microsoft has announced that Windows 10 will support HEVC. I asked LeGrow if the Alliance will change this, and he replied, “Working with the Alliance has no impact on Microsoft’s plans to support HEVC in Windows 10. HEVC is supported on new Windows 10 PCs that include the necessary decode hardware.”
The concept of “necessary decode hardware,” was a new one, so I did a little digging. On November 10, Windows Central reported that Windows 10 will support HEVC, “in-box,” quoting a tweet from Gabriel Aul, VP of the Windows and Devices Group (WDG) Engineering Systems Team.
In August, 2015, on the Windows Community site several Windows 10 users complained that HEVC decode didn’t work. A response from Andy GI, who appears to be a Microsoft employee speaking for Microsoft said, “Initially we included HEVC (H.265) software decode in Windows 10 pre-release for Windows for PC and phone. We made a late decision to limit HEVC support to devices that had hardware support. As HEVC is very computationally complex, this will provide a more consistent experience.”
Later statements indicated that HEVC decode might be returned in future versions. At this point, however, Windows 10, which has about a 5% share according to NetMarketShare, does not appear to have software-based HEVC decode.
Of course, back in November 2014, when Aul made his promise, HEVC would cost $25 million. With HEVC Advance, an OS would quality as an “Other Device,” which costs $1.10 per unit (PDF). A quick Google search revealed that there are 1.5 billion Windows PCs out there, and likely well over 800 million using Windows 7 and 8 who are eligible for the free upgrade to Windows 10. So under the current royalty policy, the potential near-term price tag for HEVC decode has increased from $25 million to close to a billion, which has to be enough to raise eyebrows in Redmond.
According to Microsoft’s statements, the decision to pull HEVC decode out of Windows 10 had nothing to do with MPEG Advance’s royalty policy. While pricing details weren’t known when Windows 10 finally shipped, HEVC Advance’s initial press release back in March would have signaled that additional royalties were coming. And today, turning HEVC back on is a billion dollar decision.
At this point, we can anticipate that the Alliance codec will be very widely supported by all browsers except for Safari, and that Apple might indeed come around. Given the economics, it’s tempting to predict that HEVC will play very little role on computers and notebooks, but the counterpoint is that if you exclude YouTube, you couldn’t fill a MINI Cooper with the number of websites that are currently using VP8 or VP9. Still, by adding Microsoft to the mix, as well as Netflix and Amazon, it appears that unless something dramatic happens, the Alliance codec will dominate in computer-based playback and that HEVC will play only a minor role.
Mobile Markets
Mobile markets are different in two critical ways; first, hardware support is essential to support battery life during UHD codec playback, and second, both Apple and Google have already added HEVC to their latest product offerings, Apple exclusively for FaceTime.
Again, since that time, the cost of HEVC has increased dramatically from the $0.20/unit with a $25 million cap asked by MPEG LA to $0.80 unit with no cap asked by HEVC Advance. According to Android Authority, more than 1 billion Android smartphones shipped in 2014, along with 193 million iOS devices. Apple would have to pay the royalty on all those devices with HEVC, Google would pay only on HEVC-enabled devices they directly ship, and Android licensees would pay the rest.
On a per-unit basis, the cost isn’t too bad, but without a cap, the numbers quickly get obscene, particularly since both devices currently support H.264, and Android 4.4+ supports VP8/VP9. Given the current paucity of HEVC content, how much of a premium or user benefit does HEVC playback actually deliver today?
If Google can disable HEVC on its devices, it can save its ecosystem hundreds of millions of dollars annually, while Apple can simply use H.264 for FaceTime and save close to $100 million on shipments of the iPhone 6 alone. Since only a relatively small percentage of users actually know which codec is used in the device, these moves seem possible, if not likely. That said, I asked Google’s Frost if the Alliance formation would impact Google’s planned use of HEVC in Android. His response was, “with regard to Android's plans relating to HEVC, the announcement does not affect any plans relating to HEVC.”
Given the expected codec release target of January 2017, you wouldn’t expect hardware support until 12-18 months after that, or 2018. Between then and now there are huge dollars involved, and it will be interesting to see what HEVC-related moves Apple and Google make during the interim.
Broadcast
The broadcast market is the one I know least well, so I’ll limit my comments. HEVC is the clear heir apparent in the traditional broadcast space, particularly in closed-loop applications like contribution that don’t have a customer-facing component. For the foreseeable future, this will remain true. I don’t expect broadcast-related companies to embrace the Alliance codec for the very same reason the Alliance was created—they don’t have a seat at the table. But that’s not saying that they won’t, and it will be interesting to see how many broadcast-related companies join the Alliance over the next few months.
The growing intersection of broadcast and internet video is on Smart TVs and retail OTT devices like Roku, Apple TV, and the like. While most of these devices will ultimately support HEVC, YouTube’s use of VP9 has engendered lots of VP9 support by Smart TV vendors, including those from Samsung. All Android-powered Smart TVs must support VP9, and presumably the Alliance codec down the road, as would dongles from Google and Amazon. Netflix’s and Amazon’s presence will certainly provide an incentive for additional support of the Alliance codec.
On devices where HEVC and the Alliance codec are supported, both companies can ship movies for free using VP9 or the Alliance codec, or use HEVC and pay .5% of the sales price to HEVC Advance. You would expect both services to query each device to determine which formats it supports, and send VP9-encoded video, or that from the Alliance codec, when available. You would also expect other movie distributors to follow suit.
Final Thoughts
You can’t write an article about video codecs without mentioning the patent bogeyman, so here goes. In March, 2013, MPEG LA granted Google a “a license to technologies that “may” be essential to VP8,” and “one-next generation VPx video codec,” which obviously is VP9 and not VP10.
The original goal of Cisco's Thor project was to create a codec from scratch without IP issues, but that effort was estimated to take years. The Alliance seems to be leveraging VP10 to accelerate that schedule. If that codec uses techniques included in VP8/9 that were licensed from MPEG LA, those licenses don’t apply, or there could be general patent claims from totally different sources. If the codec uses IP from companies that didn’t sign the original license, there also could be issues. The Alliance obviously knows all this, but I felt it worthy of mention.
It’s also interesting to note that before Google acquired On2 and open-sourced VP8, there was a potential for H.264 royalties on free internet video. Shortly after the WebM announcement, MPEG LA announced that H.264 would be royalty free for free internet video in perpetuity. They’ve consistently denied a relationship between the two, but the timing seems to indicate otherwise.
The codec world almost universally decried HEVC Advance’s proposed royalty policies, though I’ve been one of the few exceptions. That’s because I believe that like baseball players and rock stars, IP owners should be able to charge whatever the market will bear. However, as we’ve seen, the market can have a way of biting back.
By contributing VP9 to the Alliance, Google has converted it from a proprietary technology that was tough for companies like Microsoft and Apple to support to an alliance-based project that will be much more palatable. Perhaps that will be enough to trigger a reconsideration by HEVC Advance. Of course, now that the genie is out of the bottle, there’s no telling how far and into what markets it will resonate.
As the saying goes, pigs get fat, but hogs get slaughtered. At the risk of sounding hyperbolic, this could be the day we look back and cite as the start of the decline of HEVC. I think it’s highly unlikely, but it’s certainly possible.
Related Articles
The cross-industry open source Alliance welcomed new members and is making strides with AV1. Meanwhile, HEVC's future seems more uncertain than ever.
12 Apr 2016
After widespread criticism of its initial royalty proposal, HEVC Advance has reduced royalty rates and decoupled them from attributable revenue, as well as provided incentives for early adoption and a royalty credit for small publishers.
18 Dec 2015
This year, HEVC has gone from the heir apparent to an unmanageable mess. Meanwhile, open source codec efforts are focused and progressing. What happened?
01 Dec 2015
MLB Advanced Media says the proposed royalty isn't reasonable. With pressure mounting, the new patent pool promises an adjustment.
24 Sep 2015
In this Q&A with two video IP and technology investment banking professionals, we ask about patent risk, VP 8/9, and the possibility of a third HEVC patent group forming.
16 Sep 2015
The Alliance for Open Media—including Amazon, Cisco, Google, Intel, Microsoft, Mozilla, and Netflix—hopes to deliver a video codec for internet delivery by late 2016 or early 2017; Cisco's Thor technology will be included
01 Sep 2015
What can battle the two HEVC patent pools and the third one being formed? Cisco's open source video codec Thor is on the horizon.
18 Aug 2015