NAB 2018: Reza Rassool Talks RealNetworks HD
At NAB, Jan Ozer met with about a dozen companies with stakes in HEVC and/or AV1. This is another in a series of video interviews he conducted with them.
Jan Ozer: I'm here today at NAB with Reza Rassool, CTO at RealNetworks. We've been spending a lot of time at NAB talking about HEVC and AV1 and RealNetworks has an alternative to that called RealNetworks HD. Why don't you walk me through what the codec is doing, when it came out, and who's using it, and why.
Reza Rassool: Let me tell you about the RealMedia codec. It's a very different approach to a codec compared to the MPEG class of codecs. Still, it's a two-dimensional codec, uses the same approach. It’s DCT-based. It's motion estimation but an arithmetic coding and uses similar technique. But we took an approach of pursuing a low-complexity solution.
It was our belief that the MPEG class of codecs and all of the codecs that were designed by committee ended up pursuing a high-complexity result. We thought perhaps, it's unnecessarily high in complexity. We sent our codec team a mission. We challenged them to come up with a codec that can be competitive with HEVC but that can be decoded in software, on a mobile processor, and used not draining the battery hideously. So, a practical software codec that could be competitive with HEVC.
This project kicked off in 2014 and it was actually run out of our Beijing office. So, the net result--and you're probably going to scoop on this—is that we just published the competitive data and I'll send you that. In fact, I tweeted it to you just before the interview. Have a look at the results. We outperform HEVC by a good margin.
Jan Ozer: According to which parameter? Quality?
Reza Rassool: So, we use the Netflix VMAF quality score plotted against bit rate and you see the classic curves. And, as the curves sort of part as the quality increases, the RMHD codec outperforms AVC and HEVC by a measurable amount.
It turns out for 4K video, we do a whole lot better than for 1080p video. At standard-definition video 480p, there's probably not much of an advantage. This is very similar to the finding that AV1 is getting in comparison to HEVC. At higher quality, it starts to stretch its legs and outrun HEVC but at the lower end, it's not much of a difference.
Jan Ozer: What's the encode time and what's the decode load?
Reza Rassool: The encode time is very rapid. It outperforms HEVC or x265 by a good measure. I think it's five times faster. We also set ourselves a goal that any consumer with the RealPlayer running on a modern laptop should be able to encode a 1080p stream in real time. We've just achieved that. In fact, if you're using a super-fast processor, you can almost encode 4K in real time if you're using a thread ripper. But we are continuously optimizing the encoder as most encoders do. You get opportunities to optimize, and we believe we're going to achieve 4K real-time encoding very soon.
Jan Ozer: You know the codec is one thing. Enabling it to be implementable is probably the bigger challenge sometimes. If I say, "Great, I want to use your codec," how do I get playback in a browser? How do I get playback in a mobile device? How do I get playback on STBs and smart TVs?
Reza Rassool: This is a challenge. Let me tackle the browser issue first. It's not in a browser at the moment. It's in a browser with a little bit of help of a plugin or an ActiveX component. In the RealPlayer, there is an embedded browser, and in that we can render the decoded frame into the browser. That's not a solution that is widely publishable, but in the mobile space, we deliver the codec in a couple of forms. In the SDK form, it comes wrapped in a player, and application developers can deploy that in their mobile apps.
We've also got an OEM SDK where handset manufacturers like Huawei. So, if you have your Huawei phone, the Mate 9 phone includes the RealMedia codec in its operating system. So, it becomes a native codec of the platform. So, these phones, because the RealMedia codec is arguably the de facto codec of China. RealMedia codecs of all of their generations. This is 11 generations, and for historical reasons, the RealPlayer and RealMedia codec thrived in China. We can go into why that happened, and you probably know the story of that already. We've got strong adoption of this 11th version of the coded by Huawei, and we are engaging with other OEMs that had adopted RMVB in previous years.
It's bringing them the benefits and saying, "Hey, here's an alternative." Here's an alternative to HEVC. It's software-decodable, and it’s straight to market, so you don't have to wait for chip development that might take 18 months. How much of the battery does it consume? On a mobile processor like the Mate 9 phone, you can watch a two-hour movie on one charge of the battery and that was another one of our metrics. We are continuously optimizing the codec even further. As we get closer relationships with the OEMs, we find ways to utilize extended instruction sets, or maybe undocumented instruction sets on the devices themselves to improve the performance of the codec.
Jan Ozer: I understood that RealNetworks sold all their video-related codec IP to Intel.
Reza Rassool: That was the RMVB-related codec in 2010. It was a good deal. Sold the IP. Retained the rights to continue selling it so it was an interesting deal. We also transferred the team as well as the Seattle team to Intel. So that deal was 2010 deal. In this new codec, we spawned a whole new team in Beijing. And, it's now come to fruition. Not only in the OEM space, but now in OTT. The OTT operators are also looking for an alternate to a codec that not only charges a per-device fee, but puts a tax on their streaming business as well. The RealMedia codec offers a great opportunity for an OTT operator.
There's another unique advantage that the RealMedia codec has for OTT operators. In the OTT space, Jan, very different from IPTV or cable television. The OTT operator does not own the set-top box. When you go and buy a Fire TV or Roku, that's your choice. You bring it home and you say, "I'd like to watch Netflix on this. I'd like to watch Hulu or the equivalent services in China, and so on." Then, when you connect that, the OTT operator discovers the hardware capabilities of your box.
One of our big OTT operators is CIBN, the state-owned broadcast. It's like the CBS of China. They adopted the RealMedia codec, because, when they analyze their network, they said, "We've got 74 different set-top box models that are connecting to our network with all sorts of different hardware capabilities." Only 32% of them had HEVC in hardware. They ended up with a situation where they've lost control of the codec and having a software codec in the form of an SDK that can be bundled into their OTT app. It gives the OTT operator back that control that they had in the old cable days, where they actually constructed the box with a particular codec.
It's that nuance, which we hadn't anticipated when we created the codec, and that seems to be one of the buying decisions in OTT.
Jan Ozer: Real quickly, let's get on a couple of different topics. There seems to be an impression that you can't build a codec without stepping on someone's IP. Now you have IP that you developed before, but you also innovated with the new codec. What's your sense of the ability of any codec to be constructed without violating anyone's IP rights?
Reza Rassool: It's really tough. It's a horrible minefield. Hat's off to the AV1 guys at the Alliance for Open Media for taking a different approach and saying, "Park your rights at the door if you want to come and play."
The jury's still out. For us, we built upon a legacy of 10 previous iterations of codecs, and we knew the minefield of the current technologies of the competitive approaches and we were very careful to make sure that we used alternate techniques where needed.
But the approach that we took with RMHD was to favor spatial compression over temporal compression. What do I mean by that? As you go up from standard definition to 720p to 1080p to 4K, spatial resolution is increasing all the time. But for commercial applications, the frame rate was not increasing at the same rate. We thought, there's an opportunity for our codec to double down on the spatial compression and do a much better job of that.
Put the complexity and the compute power in spatial compression and maybe ease back off of the temporal compression. The practical result was that we only look for motion between adjacent frames--none of these long motion vectors that are referencing frames, 16 frames away. That reduced the complexity of the codec a tremendous amount. The memory footprint is really small now, the threading complexity is also much simpler. You know the slice parallelism and all of the gnarly stuff you have to do in other codec designs where you've got these long references, and it seemed to pay off. As we tested it, it turned out that, just looking at motion between adjacent frames, more than enough.
Jan Ozer: Tests I've done show that more than one B-frame really doesn't add that much quality, and using multiple reference frames really bumps up the encoding time without a significant increase in quality.
Reza Rassool: There you go.
Jan Ozer: I guessed you published your findings on the RealNetworks website?
Reza Rassool: Yes, on the RealNetworks website.
Jan Ozer: Getting back to the player side, how do you see the browser playing out? Mobile and OTT are good markets, but the easiest market to adopt for new codecs is always the browser. Are you talking to Google and Mozilla and Microsoft?
Reza Rassool: Of course, they've all got their vested interest in AV1 and they say, "Why does the industry need another codec?" I just have to keep on my mantra: low complexity, low complexity, low complexity. Moore's Law wins out in the end. Moore's Law doubles the CPU power and reduces storage costs each year, and so a software solution will always be more agile than a silicon solution. The design-by-committee approach will either deliberately or accidentally end up with an overly complex result.
Related Articles
Streaming Media's Tim Siglin interviews ABC News Product Manager Raj Moorjani at Streaming Media East 2018
07 Jun 2018
Streaming Media's Jan Ozer interviews RealNetworks Founder Rob Glaser at Streaming Media East 2018.
21 May 2018
H.264 still leads, HEVC is starting to gain traction, and AV1 had its coming-out party. To add to the confusion, other codecs offer alternatives to all three. Jan Ozer makes sense of all the codec news from this year's NAB.
17 Apr 2018
Mux CEO Jon Dahl talks about the company's QoE flagship QoE product and Mux Video, a new API to video hosting and streaming.
17 Apr 2018
NGcodec CEO Oliver Gunasekara talks about how his company uses field programmable gate arrays (FPGA) available on standard cloud computers to accelerate high-quality encoding of HEVC and VP9 video, and what this might mean for AV1.
17 Apr 2018
V-Nova co-founder and CEO Guido Meardi describes how his company has taken the PERSEUS codec launched three years ago and produced a finished product playable in all browsers and mobile devices, and many STB and OTT platforms.
17 Apr 2018
Streaming Media's Jan Ozer interviews Capella Systems' Ikuyo Yamada in the Capella booth at NAB and discusses HEVC, AV1, and Dolby Vision.
17 Apr 2018
Jan Ozer spoke with Greg Heil, CEO and co-founder of encoding.com about the results of the Encoding.com Global Media Format Report (2017), particularly as it relates to the actual usage of HEVC and VP9 by Encoding.com customers.
17 Apr 2018
Streaming Media's Jan Ozer interviews Beamr VP of strategy Tom Vaughan in the Beamr booth at NAB 2018 about HEVC and AV1
17 Apr 2018
Streaming Media's Jan Ozer interviews Alliance for Open Media executive director Gabe Frost at NAB 2018, talking about AV1's launch, its future, and how it will likely co-exist with HEVC
16 Apr 2018
Mozilla researchers Nathan Egge and Michael Bebenita discuss Daala's role in the AV1 codec, and how a codec can be developed without stepping on third-party IP. They also discuss early performance numbers and when AV1 decode will be available in Firefox.
16 Apr 2018
Streaming Media's Jan Ozer interviews Bitmovin CEO Stefan Lederer in the Bitmovin booth at NAB 2018 to talk about the company's $30 million funding, AV1, and its new machine learning-based encoding.
16 Apr 2018