Whither SVT-AV1
Moscow State University (MSU) issues multiple codec reports each year. I find these reports credible because they use very diverse content and realistic encoding parameters that the university derives in consultation with the codec developers. I wrote about the 2020 crop of reports at go2sm.com/moscow2020.
One very noteworthy observation from these reports is the consistently dismal performance of Scalable Video Technology (SVT)-based codecs. In 1-fps tests in the “MSU Video Codecs Comparison 2020 Part 1: FullHD, Objective” report, SVT-HEVC and SVT-VP9 placed well behind x265 and WebM’s VP9; they’re even behind the x264 codec they were designed to replace. SVT-AV1 was four percentage points behind the Alliance for Open Media’s (AOMedia) libaom and 25 percentage points behind Visionular’s Aurora1 codec. Granted, SVT is more about performance than quality, but SVT-HEVC and SVT-VP9 trailed x264 and x265 again in 30-fps tests. (VP9 and Aurora didn’t make the performance cut.)
These results are most interesting as they relate to SVT-AV1, the AV1 codec that AOMedia has gone all-in on. Specifically, in August 2020, AOMedia formed a software working group to leverage SVT-AV1 to “create AV1 encoder implementations that deliver excellent video compression across applications in ways that remove computational complexity trade-offs for an ever-growing video delivery marketplace.” It’s only been about a year, so it’s hard to criticize these efforts, but the laggard performance of the SVT-based VP9 and HEVC codecs raises questions as to whether AOMedia chose the right architecture for AV1’s continued advancement.
I raised these issues with David Ronca, Facebook’s director of video encoding and the former director of encoding technologies at Netflix. He responded, “As I understand it, SVT-HEVC and SVT-VP9 were designed for live encoding, and [are] not really useful for other use cases. The threading performance is nearly linear, which means more cores = much faster encodes. Other codecs are not as efficient with many threads. SVT-AV1 performs very favorably to libaom, and delivers solid gains all the way into x264 computational complexity. We (the SVT community) feel very good about SVT-AV1, and are continuing to invest in it. It’s among the top most well-designed encoders.”
Regarding codec comparisons in general, Ronca added, “Intel shared a codec comparison at SPIE that is how modern codecs should be evaluated. The fundamental metric is per-shot, single-threaded, complexity vs. efficiency.” Ronca was referring to a white paper titled “The SVT-AV1 Encoder: Overview, Features and Speed-Quality Tradeoffs,” which compared SVT-AV1 to libaom, x264, x265, and libvpx (VP9). The paper was published in August 2020 and will cost you $21 to download. Or, you can check out Ronca making the key points in a video interview with John Porterfield on YouTube (go2sm.com/av1ronca; about 25 minutes in).
The testing for the paper was extensive, well-documented, and well-presented, although a touch idiosyncratic in my view, which raises questions as to how the results will translate to real-world applications. For example, the researchers performed all encodes using constant rate factor encoding, which is very frequently used for academic codec comparisons, but very seldom used for actual production. When I tested AV1 encoders for my article “AV1 Has Arrived: Comparing Codecs From AOMedia, Visionular, and Intel/Netflix,” I tested SVT-AV1 using two-pass encoding. Intel objected, saying that two-pass rate control had not yet been completed. As I explain in the article, I left SVT-AV1 in because I had already invested substantial time working with the codec and because it was freely available on GitHub without any warnings about two-pass performance. The white paper also didn’t test the Aurora1 codec, which performed best in my tests.
I asked Ronca about these issues, and he responded, “If I was going to deploy AV1 and my use-case was 2-pass, fixed GOP, I might want to test libvpx, SVT-AV1, and maybe some closed source AV1 encoders in 2-pass mode with fixed GOP.”
The white paper concludes, “Preliminary experimental data with respect to faster presets show great promise that SVT-AV1 will eventually be the encoder of choice for a very-wide range of [video-on-demand] applications.” That may yet be the case, but I’m reminded that all codec comparisons are academic until they test your content with your encoder using your encoding parameters. The closer they are to your encoding schema, the more accurate they are likely to be.
Related Articles
Netflix Senior Software Engineer Cyril Concolato walks viewers through Netflix' workflow for AVI deployment6 using CMAF in this clip from Streaming Media East Connect 2021.
13 Aug 2021
The good news: As always, Moscow State's codec studies are some of the most comprehensive available. The bad news: Unless you're TikTok or Tencent, you won't have access to some of the best performers.
11 Jun 2021
Within 24 months, hardware support appeared, encoding became affordable, and AV1 became a much more realistic competitor to HEVC. Here's how the currently available AV1 codecs measure up.
18 Sep 2020
Bitmovin's Paul MacDougall and Streaming Media's Tim Siglin discuss benefits built into the AV1 codec that enhance the efficiency of 8K delivery, and what that means in terms of codec adoption and for CDNs and consumers going forward in this clip from Streaming Media East Connect 2020.
18 Jun 2020