Video: Key Considerations of Migrating to VP9 for Live Encoding
Learn more about ad insertion for live events at Streaming Media's next event. Watch the complete video of this presentation from Video Engineering Summit at Streaming Media West, VES105. Live Streaming with VP9 at Twitch TV, in the Streaming Media Conference Video Portal.
At Twitch, we approached the decision to migrate from H.264 to to HEVC or VP9 from three angles:
- legal risk
- available decoder support
- and the compression efficiency and the compression gain.
Legal Risks with HEVC
For HEVC, the legal risk (Figure 1, below) is probably something I don't need to talk about much. The patent mechanism, the number of pools, and the amount of confusion it's bringing to the market is beyond OTT and thus for itself.
Figure 1. HEVC legal risks
As you can see there are already three pools and there are even people with patents which are not in any pool. Some of these pools have already defined cost for content (Figure 2, below). Specifically, MPEG-LA is free, HEVC Advanced only recently moved for content to be free, and Velos Media is not defined yet.
Figure 2. HEVC royalty structure
Other patent holders need to be treated case by case, and that's really not a comfortable position. And, although, for content, we seem to be moving towards free content, in terms of the devices where people will watch us, if people on the device side won't support HEVC, there is no point in us streaming that for them.
And to be honest, although it is very expensive, the highest risk is that it is unknown. So that's what gives lots of incentive to the industry not to deploy HEVC to a big extent nowadays.
Legal Risks with VP9
VP9 is not risk-free. Although VP9 says it's royalty-free, there is always a possibility for anyone to come and challenge Google for that. Although Google is using VP9 on a massive scale, and there is talk that Google has already signed a deal with MPEG-LA to cover the two generations which are VP8 and VP9, but there is still a little risk.
Decoder Support
As for decoder support, I'm dividing this into three areas: browser support, hardware support, and general devices support. As you can see in Figure 3 (below), the five left columns are for desktop and laptop browsers and the rest is mobile devices. You can see the top row, H.264, is pretty much deployed everywhere. That's the universal encoder so far and probably the last universal encoder.
Figure 3. Decoder support: browsers
HEVC is not supported on Firefox or Chrome, but VP9 is, and Chrome is probably the top browser used by most platforms. So there is a little advantage in terms of playability for VP9 with Chrome and Firefox. For mobile devices, H.264 is everywhere (Figure 4, below). HEVC has been supported on iOS since 2017, and although it is required on Android devices starting 5.0 and beyond, lots of devices will still have hardware HEVC decoding. Again, that comes back to licensing issues.
Figure 4. Decoder support: mobile
VP9, obviously, is not supported on iOS, and probably is never going to be, but it is fairly mature and available on lots of devices. The graph in Figure 5 (below) shows that, in general, if you look at the installed base on desktop, laptop, mobile tablets and connected TVs, VP9 has pretty much double the penetration.
Figure 5. Decoder support: installed base
This data comes from 2017. Since then we’ve seen a slight increase on both sides, and we’ve also seen HEVC penetration start to go much deeper. But this still gives us a good base for VP9 to start with.
Viewership
Where does Twitch stand in all this? Today, we encode only H.264. If we start adding HEVC, and we start adding VP9, Figure 6 (below) shows the percentage of viewed minutes our viewers would watch using either of these standards. For HEVC on browsers, since we rely a lot on chrome or Firefox, it's only 2.8%, whereas 50% of our content could be delivered using VP9. For mobile devices, 7% of our content could be watched in HEVC. That's mainly the IOS devices. For VP9, with all the Android devices, it could reach 10%.
Figure 6. Viewership at Twitch
That's another reason why we thought VP9 would be the way to go.
Finding a Software Encoder
Our next challenge was to find a good encoder that runs in real-time at 1080p60 which is our highest rendition. Before we did that, we did some software calculations and ran the experiments documented in Figure 7 (below). The first column shows our benchmark, H.264 (x264) medium preset. We encoded whatever is available in HEVC software, just to have a rough guide, and we used x265.
Figure 7. Compression gain: software encoder implementation
In order to achieve real-time encoding we needed to run ultra-fast, and in testing the performance was not much better than x264. VP9, the library available is libvpx VP9, which is the Google library. To know that we’ve achieved real-time, we need to run speed 6, and that gives the worst results than x264.
We feel that VP9 could deliver something for us, but we're stuck in the sense that there are no VP9 software encoders out there. There are lots of people working on HEVC encoding. Lots of companies have built ASIC, FPGA solutions, and some software solutions as well. At the end of 2017, only four companies, as far as I know, were doing any work related to VP9.
Partnering with Third-Party Providers
So we needed to dig a little bit deeper, working with third parties. We started talking to NGCodec about their Video Encoder IP, and we realized that they were starting to develop a VP9 model for real-time implementation (Figure 6, below). We ran an evaluation of that towards end of 2017 and found that it does deliver, consistently almost, 25% compression savings over H.264.
Figure 8. Third-party providers of technology compression for VP9
That encoder will be running on the VU9P FPGA, which is the Xilinx FPGA, and Xilinx has been working with us to deliver the Scaler IP as well as helping us with the integration. All this will be running on the AWS server. I’m sure lots of you are familiar with the F1 instance, which is an FPGA instance on the cloud that you can use. That's the hardware we're going to deploy this solution.
Related Articles
Streaming Media's Tim Siglin interviews Twitch's Tarek Amara at Streaming Media West 2018.
06 Dec 2018
Google Chrome's Matt Frost explains why YouTube uses VP9 "down the resolution stack"--not just for HDR and 4K--and explains how better-quality encoding benefits watch times and ad revenues.
31 Jul 2017
Viacom's Jeff Tapper explains migrating to VP9 on platforms that support it will help you reach new markets and save money by delivering better quality video at lower bandwidths.
05 Dec 2016
Frost & Sullivan analyst Avni Rambhia assesses the key factors content owners face today in determining whether to stick with AVC or move to HEVC.
24 Nov 2016
Frost & Sullivan analyst Avni Rambhia identifies the key inhibiting factors that are impeding the widespread migration from AVC to HEVC.
10 Nov 2016
Frost & Sullivan analyst Avni Rambhia explores the "three 'Rs'" of HEVC adoption: Revenue, Resolution, and thRoughput.
07 Nov 2016
Companies and Suppliers Mentioned