April 16, 2018
By Jan Ozer Contributing Editor
Featured Articles

NAB 2018: Mozilla Talks Daala, Firefox, and AV1

At NAB, Jan Ozer met with about a dozen companies with stakes in HEVC and/or AV1. This is the first in a series of video interviews he conducted with them.

Jan Ozer: Jan Ozer here from the Bitmovin booth at NAB. I'm with two engineers from the Daala codec encoding team from Mozilla, team manager Nathan Egge and Mozilla research engineering Manager Michael Bebenita. We're going to talk about Mozilla's contribution to the AV1 codec and performance and when AV1 decode will show up in Firefox. Gentlemen, tell me what portions of AV1 came from Daala?

Nathan Egge: Yes. There are three major contributions that came from Daala. We've got our Chroma-from-Luma Prediction, we've got the Multi-Symbol Entropy Coder, and then we contributed part of the Constrained Directional Enhancement Filter.

Jan Ozer: Okay. I guess we should start off by saying what Daala was and how far along were you with that when you joined the alliance.

Nathan Egge: Daala is Mozilla's effort to create a royalty-free video codec, and we'd put about three years into it between 2012 and 2015 when the alliance was formed, and we were getting some pretty good results. We were competitive against certain test clips and certain use cases.

When the Alliance was formed it was a group effort by a bunch of companies who kind of pooled their resources, and we looked at it as an opportunity to take the Daala technology and put that into what ultimately became AV1.

Jan Ozer: You guys want to talk about what you did with Daala and the IP situation? I know you can't talk about AV1 in general, but when you're managing codec development, everybody says, you can't develop a codec today without stepping on somebody's IP. What did you do within the Daala development cycle to make sure that didn't happen?

Nathan Egge: For Daala, we took a look at the patent landscape, and we noticed there were certain areas that maybe had a lot of patents on them and took a look at the state-of-the-art, and then did something that was somewhat different. For example, in Daala we had a lapped transform, and no other video codecs had been doing lapped transforms. We did some research to figure out how we could use lapped transforms to do video compression and end up developing a bunch of other techniques in support of that. From an IP point of view, the idea was to do something that did not infringe by design.

Jan Ozer: When you formed the group, there were a lot of companies who each brought individual IP that was codec-related. Microsoft had the VC-1 technology and WMV. It sounded like during the development that you almost picked and chose companies based upon the IP they could deliver to the development.

Nathan Egge: I don't think we picked and chose companies, but we definitely picked and chose technologies. All the member companies had the same interest in seeing royalty-free video and each brought the contributions they could to the AV1 code base.

Jan Ozer: What do the numbers look like? Everybody wants to know how much better is it than HEVC? How much longer does it take to encode? What's the decode tax on the CPU side?

Nathan Egge: In terms of performance, there have been a number of independent studies. I think the Moscow State University published something in January that showed that we were about 30% better than H.265 and VP9. They had actually a four-codec comparison and they had looked at different profiles there.

In terms of encode time, depending on CPU level use, it varies. We were anywhere between 50 and 200 times slower than VP9 for equivalent rates, and that's because we're a research codebase where we haven't done a lot of encoder optimizations.

Jan Ozer: Where do you see that ending up in a year?

Nathan Egge: It'll certainly be better. As part of the Alliance we're working on doing real-time encoding for video conferencing and things like that, and so I'm certain that there'll be some optimization point where you can get real-time video out of it in the near term. That's our big use case.

Jan Ozer: The Moscow University has been criticized because the codebase they used was old. When was that AV1 code pulled from?

Nathan Egge: That code was pulled from June 2017. In that time we've actually made a number of strides in improving things. I'd imagine that new studies are going to be able to show that with the current codebase, we're still getting about 30% better than these other sort of modern codecs.

Jan Ozer: Bitmovin has a survey that's up on the Bitmovin blog that compares HEVC with AV1 if anybody's curious. We know the encode side. Well, what's the CPU tax as compared to VP9? How much more powerful do you need a decoder to play it back today?

Nathan Egge: I don't have exact figures for that, but I can tell you that with the demo that we showed here with Bitmovin, we were able to do real-time playback of 1080p content in the browser, unlike a conventional laptop.

Jan Ozer: It's a Dell laptop and it's not all that powerful.

Nathan Egge: 50% CPU utilization.

Jan Ozer: And that’s at 1080p. I looked at that demo earlier and playback requirements used to be a lot higher; and that was at 480p. You've already made the decode side more efficient.

Nathan Egge: We've actually done some progress on that even since that demo as well. We've been working on reducing the memory footprint and adding a bunch of transforms to other tools.

Jan Ozer: The big question is, we know all the details on the codec. What about the decode side? I know you guys aren't part of the Mozilla Firefox team, but what's your expectation in terms of when we'll see decode in Firefox and some of the other browsers in the AV1 committee?

Nathan Egge: You can you download Firefox right now and it has support for AV1 in it. It's using an older hash of lib AOM and we're working and updating that to a more modern hash. We're also working on adding FLAC so you can prep it on and it will eventually ship in the mainstream version of Firefox in a disabled form until there's wide support for it.

Jan Ozer: When would you expect that to be?

Nathan Egge: I believe that you'll see people shipping products by the end of the year with AV1, and you'll definitely see it on in Firefox by default by the end of the year.

Jan Ozer: Okay. When will we see HEVC in Firefox?

Nathan Egge: Hopefully never.

Jan Ozer: Sorry, I had to ask. Okay, listen gentlemen, thanks for taking time during a busy week and good luck with Firefox and the codec.