Rethink TV's Alex Davies Talks New M&E Codecs Market Forecast
Jan Ozer: I’m here today with Alex Davies, Senior Analyst with Rethink Technology Research, who just issued an updated report, Media and Entertainment Codecs Market Forecast 2023-2030. We reported on the previous edition of this report back when it came out in 2021. So tell us about this report.
Alex Davies: The original report was a big project. This one is a lighter update, based on a similar timeline. We realized that we hadn’t officially updated the timeline for a while. Part of that was building on a bunch of the changes in the video devices forecasts and projections that have taken place since COVID.
Jan Ozer: Let’s start with an overview of the timeline chart.
Alex Davies: There are two parts to this forecast. The first part is the timeline. You have to start with an idea of what you think the market is going to look like. And part of that is looking back in the past. There’s a lot of lessons from history that we can pull in. The green stuff at the top (of Figure 1) is the AOMedia realm. The red stuff down at the bottom is MPEG, and the orange stuff is LCEVC.
Figure 1. The Rethink forecast timeline. Click the image to see it at full size
Broadly, this is where we ended up for 2020-22. The MPEG-2 usage is extremely small, but about 8% of transcoding was still AVC in 2020. HEVC was between 10 and 20, and VVC, EVC, and LCEVC was just about finished. At the same time, AV1 is around but it’s not doing much. And in 2020, that was our first smartphone silicon from Mediatek, and we figured that that would be something of a turning point. And that was just about when we were publishing the first version of this. Moving into 2021, there were a few changes in AV1, which, to our mind at least, closely tied LCEVC and AV1 together. That is something that we view a bit differently now. But AV1 was ticking along. And notably, 2021 was when we saw the first VVC silicon turn up. MX Player, which is a big Indian OTT service, started using VVC at this point too.
Jan Ozer: What’s your perspective? When AV1 first came out in 2018, we heard silicon in one year and phones and TVs in two years, which would’ve made it April 2020. What happened to that? Did it come true? And if not, why not?
Alex Davies: AV1 has been, depending on how you look at it, a great success, or a ticking time bomb and a bit of a failure. The established MPEG crowd were laughing, and essentially saying, “There’s no way you can make a brand new codec without infringing on people’s patents. What do these Silicon Valley types know? This will get bogged down in litigation for years and years and years." AOMedia was quietly adamant that it would be free. And then of course, the Sisvel pool turned up and started charging a royalty, and it’s up to 51 customers now. They like to say that it’s royalty-free, but quite clearly it isn’t. And in the background, a number of IP holders are adamant that AV1 infringes on their patents. But these very big companies that have established licensing programs say this AV1 implementation infringes on their patents. As a part-time journalist, I couldn’t wait for the lawsuits to happen. But if you’re in the industry and trying to make roadmap decisions, then that uncertainty isn’t good.
Jan Ozer: Do you see that as the reason that the codec hasn’t moved forward as quickly as was predicted back in 2018?
Alex Davies: No one likes uncertainty, especially if you’re making a silicon-manufacturing decision that could cost you hundreds of millions of dollars that you wouldn’t get back. The parallel is HEVC arriving and not really doing anything for quite a long time because of the uncertainty.
Jan Ozer: Okay. Let’s get back to the timeline.
Alex Davies: AV1, in theory, is mandated for Android TV devices. 2022 was when we saw the first AV1-native set-tops for Android TV begin arriving in volume. In the MPEG camp, that’s when the VVC pools were finalized, the DVB added VVC to its spec, and market interest in LCEVC is increased. In 2023 (Figure 2), we’ve seen a bit of an explosion in terms of relevant points. AV1 software decoders are now mainstream in browsers and smartphones, even if not everyone is using them yet. I mean, they could all be out there. We’ve got established silicon support; Mediatek has really been leading there. Something like 23% of Instagram reels are done in AV1—a detail you wouldn’t get from the mainstream tech news.
Figure 2. The 2023-25 timeline. Click the image to see it at full size
For the AOMedia stuff, we’re stretching and looking toward the end of the year—post-IBC and in the run up for CES, when (in theory) Qualcomm finally announces its flagship. That should be the beginning of accelerated growth for AV1. That is also when we would see the first VVC Smart TVs hit the shelves. The holiday season is when the ball gets rolling. That’s also when we expect the first flagship smartphones with VVC hardware to start shipping in volume.
We’re also banking that at some point this year, LCEVC finally signs its first major video provider for a contract for that enhancement layer. The LCEVC curve is beginning.
The high-drama MPEG news was Velos bowing out of the HEVC pools, and also confirming it wouldn’t do VVC in a pool. It’s gone back to its licensing heritage. And the other big thing was Google quietly adding HEVC into Chrome. That’s a pretty major change, but it might be a bit too early to be able to see the kinds of differences that decision has made yet. LG also signed the first VVC license, so InterDigital has got the first public VVC licensing agreement in place. And then the MPEG LA and Via Pools merged, which made Dolby the kingmaker with a foot in like every camp, including Sisvel and LCEVC.
Looking into the future now, 2024 is the point where we expect all new Android TV and Google TV devices to be AV1 native. In 2025, we expect to see the first VVC operator set-tops start trickling in. Based on previous silicon adoption rates, this is the point where AV1 hardware decoders should be mainstream, and the key window video services and developers should be watching.
And if we look at MPEG, Brazil is implementing VVC in SBTVD 3.0, which is like ATSC 3.0, but distinct to Brazil.
By 2025, LCEVC should have three major flagship video services licensed and in place. But there will be a lot that aren’t public because it can all get done in software. There’s no real need to announce it outright. And this is when work on the MPEG-6 project will commence, based on the latency we see in the historical patterns. By then, AV2 should be finalized and VVC software decoders will be common. In late 2025, VVC will start ticking up on its growth curve. In 2026, if history holds true, the first AV2 silicon will appear.
2027 is when VVC hardware decoders will go mainstream, and in 2028 AV2 software decoders will be commonplace. 2028 will also mark the earliest point that AVC dips below 50% of the codec workload, and the codec introduced in 2003 will begin dropping off. And that really speaks to a whole lot of points, really. In the graph we have (Figure 3), you see that AVC is declining quite steeply, but five or six other codecs are ticking up. For most of this timeline, there’s only one codec that you have to worry about; by 2028, there will be two or three, which is just a bit different for people to consider.
Figure 3. The 2026-2030 timeline. Click the image to see it at full size
Jan Ozer: Tell us a bit about the “2023 M&E Entertainment Codec Workload” graph (Figure 4).
Figure 4. 2023 M&E Entertainment Codec Workload
Alex Davies: The top line is AVC. In 2020, it was at 79%. In the 2025 range, the other codecs start picking up. If we compared the 2020/2021 version and this one, the 2023, the space between HEVC crossing AVC is a lot bigger. HEVC’s adoption is now quicker, and it gets to a higher point. Similarly, that means that AVCs fall is a bit sharper, and that leaves some growth room for VVC, and also for AV1 and AV2. But the AV1 and AV2 stuff is not so tied to VVC’s growth as the others.
Jan Ozer: It feels like your report is measuring device adoption. How have you seen that relate to publisher adoption of the codec, both from a time perspective and whether they do it or not. Everybody says HEVC was awful, nobody adopted, but it’s in every device and really, it’s a publisher’s decision to, to do it, if they’re distributing to the living room and they’re doing HDR, they’re almost exclusively on HEVC, but you’re reporting on device adoption and even software. How does that relate to the publisher decision in your view?
Alex Davies: Historically the publishers only really had to worry about AVC. And if you were targeting that high-quality viewing experience, which was conventionally on a television and probably powered by a set-top or a Blu-ray player, you knew that you had to choose HEVC, and if you had any plans for 4K, then you almost certainly had to to go for it. There are a few ways that I can waffle now, but essentially our view of viewing an audience is that they’ve been fracturing for a while, so the actual demand for viewing flagship high-quality content on the best screen is falling. But nonetheless, you have to choose one of these established codecs, because it’s not just your decision, it’s this whole supplier ecosystem and vendors and things.
But VVC should appear natively in those devices. So, you then have to ask yourself, if you’re a publisher, "What do I gain if I move from HEVC to VVC? And there’s lots of benchmarks and things that will show the bandwidth savings. But there’s a separate argument here for the bandwidth not being the main driver anymore, and the fair share debate—like, fair contribution stuff—I have a deeply cynical view of that. But separately, the environmental angle for “Do you need 8K?” is the question that a lot of these publishers have to start thinking about. And if the sole justification for VVC is 8K—which it isn’t—but if you try to boil down a pitch into something pithy, you might arrive at that conclusion. So, if the sole reason for it is 8K, and you’re not gonna do 8K because the EU is banned 8K televisions, then you’re probably fine on HEVC.
From the established broadcast high-quality content world, the push is still on HEVC. And our graph shows that. But for the fractured global audiences that are watching weird videos that you can’t broadcast out on old terrestrial infrastructure, all of that stuff that’s greenfield opportunities for things like AV1, especially when the big guys like Instagram are using it. And I’m demonstrating that it’s useful and the political situation surrounding TikTok is a bit fraught. Five years ago, I don’t think anyone would’ve bet that TikTok could be a kingmaker for the video codec world, but there we are, like the ByteDance codec, the HEVC one does really well. But you just wouldn’t have banked that a Chinese firm would be so influential.
Jan Ozer: Well, talking about Instagram reels—the subhead for the report is “$2.4 billion codec royalty market lifted by the arrival of VVC and AV2, but AOMedia legal showdown awaits.” Tell us about that.
Alex Davies: So, the juicy headline was to get people interested. We’ve got a graph we’ll touch on. But part of this was looking at the total addressable market (TAM) and the service-obtainable market (SOM), and working out if each device segment had a healthy number of OEMs, say 3-5. The top line in this graph is our TAM. If you started this from scratch and said, “Let’s assume each of our device segments has five OEMs selling into it globally, it’s healthy. They specialize.” If every one of those devices is paying and you model in the device caps and things, you end up with this number that’s really quite large—in the 10 billion range, on 2.64 billion devices in 2030.
So that’s our TAM—our best-case scenario. The bottom line is the SOM which is two-ish billion ticking up slightly, but it is just under two and it finishes at 2.4 2.41. So that is the actual state of things. And the reason is because there’s not actually five OEMs in every category, and not everyone who should pay a royalty according to these pools pays them. So there’s this disconnect really between if you start with with fresh eyes you quickly learn that there’s a lot of layers to this. And then the realistic picture is a lot smaller. And then this is the one that I wish people would talk about, but they don’t ever—the licensing terms.
I wouldn’t say it’s an industry-level understanding, but everyone knows that there’s a lot more patents outside of these pools that people are paying for. It’s just that no one tells you how much they’re paying. And the bilateral licensing trend where two guys say, “You can use my patents and I’ll use your patents and we’ll promise not to sue each other”—you can’t really put a dollar amount on that kind of thing. So, if you crudely take this industry concept that only about half the relevant patents are in these pools and you think about the OEMs and whatnot, you can just say, well, it means that there’s also double the amount of revenue that’s ticking around. The TAM of licensing is there, but really the line is a lot lower down.
And to be honest, it would just be me stabbing in the dark to try and root that out. So we put this in only as an illustrative line, but the actual number for the SOM for codec royalties is higher because you have to include the licensing. It’s just that no one talks about it. It’s protected by many layers of NDA and corporate ethics statements and things like that, so it’s somewhere between those lines.
Jan Ozer: So, circle us back to the AV1-AOMedia legal issue.
Alex Davies: With those patents that are essentially licensed, those are claims from the IP holder that say, “This device or this company uses our patents and we’ve come to this nice, sensible agreement that they’ll pay us however many cents per device to use them.” So, at the same time, the AOMedia crowd publishes AV1 and says, “We’ve made a royalty-free codec.” All of those licensors who have established agreements for AVC and HEVC with dozens to hundreds of companies have looked at it and gone, “That’s not true. That’s patently false.” It’s led to this awkward dynamic where we expect something to happen, but there are too many different ways for it to play out for us to be able to confidently predict it because Apple, Google, and Meta could comfortably pay this industry off if they needed to, or if they wanted to. Apple’s got more free cash than most countries, so if it came to it, that trio could afford to indemnify AV1, and essentially buy Sisvel out. But the minute it does, it’s gonna have to buy out every other potential licensor because there hasn’t been the same pool process as the others because Sisvel just appeared. There’s a whole pile of uncertainty. So, AV2 could be really useful and quite valuable, but AV2’s adoption is going to be properly hamstrung if we haven’t had that legal clarity appear yet. It’s the spiciest bit of the roadmap for sure.
Jan Ozer: Is the illusion of royalty-free the prime driver of AV1 adoption at this point? You’re saying it’s a fiction. We have HEVC and VVC, and they’re working well in the broadcast market and the software market. Now that Chrome has adopted HEVC, what’s the rationale for AV1 if it’s just another royalty-bearing codec, as you predict it will be in the future?
Alex Davies: If it works for them and they know that they can pay the licensing fee if they had to down the line, they will, but there’s direct bandwidth savings that they can make if they offload 30% of reels into AV1. This gets into the unicast vs. multicast and caching debate, which I’ll try to stay away from. But the reason that AV1 is finding traction is because there’s interest among these publishers to use it, and as far as they’re concerned, they’re getting away with it currently. Do you want to be the scrappy codec designer that wants to try and sue Apple? Probably not. I’d imagine there’s been discussions over pints about class action or the equivalent. If you can use it, you’d be daft not to use it and get a saving. For some of them it just comes down to dollars, and there’s a spreadsheet somewhere that says, “I can save this much if I offload this much traffic into AV1, so I’ll do that.” The tools exist and it’s viable.
Jan Ozer: One of the wild-card topics I wanted to talk about was content royalties. We’ve been talking hardware and software adoption from a player perspective, and you merged into publisher adoption. Reels really could have been an HEVC production. And in fact, if they cared about mobile playback, and they were looking at the ScientiaMobile report that came out in December, they would have found that 84% of mobile devices had hardware HEVC support, whereas only 2.4% had AV1. Your answer merged into the software the publisher side, and most of the adoption you’re talking about traditionally in the report is the device and the playback side. It’s going to be interesting to see when that’s resolved. And like you, I did work with Sisvel before I joined NETINT, and I really respect the company, I really respect the process, and I really respect the patent holders in the pools, and I fully expected things to come to a head before now. But we’re just starting to hear from some TV and large appliance manufacturers that AV1 uncertainty is starting to slow their adoption. So maybe that will force some action in this part. What’s your sense of the impact of LCEVC? It’s a totally different licensing model. You saw the article we wrote on content royalties. LCEVC is all about content royalties and to a lot of people, that makes a lot of sense, particularly with the very low cap that they have. What’s your sense of LCEVC’s role going forward? Not only from a technology perspective, but also from a business model perspective?
Alex Davies: It seems logical that if I buy a car and I want to drive it places, I pay for the toll roads or the toll bridges along the way. And given the weird state of device royalties and the different tiers and regions and cap levels, it would be simpler if this cost was something for the video service to pay for. So LCEVC adoption has taken longer than we figured, but the sales pitch does seem really convincing: “Take your current workflow, add some code to your client SDK, and you’re off to the races, and you can pay us this much.” There’s a really obvious benefit to this because it’s like, “How much am I paying V-Nova? It’s this much. And how much am I gonna save in bandwidth? It’s that much.” It’s a really simple bar chart. The threat of content royalties has existed in the MPEG realm for a long time.
Jan Ozer: Did you hear of anybody paying content royalties outside of what’s called for?
Alex Davies: Not that I can recall. When MPEG LA first announced proposed content royalties for HEVC, there was an immense pushback. Maybe the sentiment towards these video services has changed sufficiently over the years. Back in the day, they were disrupting, bad, horrible cable companies, and no one likes Comcast, so go Netflix! But nowadays, a lot of these operators are on public stages saying, “We’re drowning under the weight of all this. Netflix is terrible. They’re gonna have to compensate us.”
So with the fair share stuff, when something like LCEVC comes in and says, "Hey, I can save you this much bandwidth and it’s only gonna cost you this much," that becomes a really strong sales pitch. But for companies like Netflix who build their own caching appliances and give them to these operators to shut them up, essentially, they’re still getting it in the neck. At Mobile World Congress 2023, Orange Telecom, Deutsche Telekom, and Telefonica—these supposed leaders of the mobile industry—all three of them got up and started saying, “Oh, it’s terrible. There’s so much bandwidth.” And I think it speaks volumes. It’s really irritating that they start talking like, “Hey, get your checkbook out” instead of, “Hey, maybe look at new technologies that could solve this problem.” And codecs are part of that. But another big part of it is caching. So it’s this telco/CDN trend, and it’s open caching from the SVTA. We did a report on that in 2022. But the idea is to take the content and cache it deeper in the network so that you’re not paying these huge peering charges.
So, in combination with the codec side of things, the LCEVC approach just seems like a winner. But they’ve had this this available for a while now, and we still haven’t had a big public customer. There’s obviously more to it behind the scenes. But on paper, it’s a winner, and it’s going to get pretty good traction in the market. But a lot of those video services are going to be looking at other bits of their bandwidth costs, not just the codecs. Maybe they get better returns on pushing through open caching, but with that project, there’s a whole pile of issues that I’ll try to avoid for now.
Jan Ozer: The last technical topic I want to talk about is AI-based codecs. You may not have even seen this given that your focus was publisher and consumer electronics vendors. But with all the video that’s being produced now from machine reading, whether it’s automatic cars or autonomous robots in factories, we should talk about Leonardo and his his artificial intelligence-based codec standardization effort. What did you hear about that, and what’s your personal prediction in the context of this particular report? When are we gonna see an artificial intelligence-based codec?
Alex Davies: They have strong interest. I don’t think I found anyone that thought they’d be much of a concern in this timeframe. You have to have sufficient adoption of the devices. When people come out talking about VR and metaverse stuff, you then have to think, “Well, how many of these devices need to be sold?” How many of these devices would benefit from this new AI-based codec? We’re spitting out another form of one-zero, but it’s a more bandwidth-efficient implementation rather than like video compression. But the point is there’s a latency between when these playback devices are available and when they are acquired at scale. And with regular codecs, just for plain old video, you can safely assume in mature markets that every house will eventually buy a Smart TV. If they had a regular television, they’ll buy a Smart TV, just as if they had a regular phone, they’ll probably buy a smartphone. But for things like the VR headsets, you have to pick another analog to compare them to. And even things like games consoles actually selling in quite low volumes. If you then say that for these super-immersive, very high-intensity experiences that you might need an AI codec for in the early days of them, then there’s actually a very small pool of devices that you can target.
I don’t think anyone has said, “No, that won’t be a thing,” but I don’t think any mainstream types are worried about them currently. But there does seem to be a point when, if you plot this out and you say, “Well, I can’t reliably send someone 100-megabit-per-second stream,” there needs to be a better way to do this. And the technical requirements of the use case mean that you need something like these AI-based implementations. And the devices we’ve got now are actually quite capable. When people start getting their hands on those fancy new Apple screens and see just what they can do with them, that could be telling. But until then, how much further can you go really than one 4K panel in front of each eye at 120 frames per second? The 8K-per-eye argument is probably limited. It all comes back to display capabilities and use case requirements.
Jan Ozer: You’re basically saying there’s huge latency in the consumer markets. Tesla could put end-to-end devices that use any codec they wanted in next year’s model, but you don’t have that option with ordinary people’s living rooms. So, finally, tell us about the report. Give us the elevator pitch: Who’s it for, and why do they need to buy it?
Alex Davies: Rethink TV is the service, and we learned a while ago that bundling everything together makes a lot of sense. It’s a subscription. If you buy this one, you get all the others. And what we do is we look at a few core areas. Generally, it’s devices and traffic and the CDN stuff. And then we do a bunch of technology forecasts on the base of those. Codecs is one, open caching is another, and Multicast ABR was a fun one. It’s a service. You can find us at RethinkResearch.biz. And the other thing for the video world, if you are interested in a very niche B2B trade news service, we have Faultline. That’s been going for quite a while, and we built a nice little niche for ourselves there.
Related Articles
If you're a video publisher who doesn't rank high in the VVC patent list, it's likely that you won't consider VVC until 2026 or later. If you're looking to augment your H.264 encodes with a more efficient codec before then, your most likely choices are HEVC and AV1, with LCEVC also an option. However, if you're a product or service provider in the streaming media ecosystem, it's long past the time to start considering where and when VVC is going to fit in for you and your target customers.
01 Aug 2023
In late September, Google quietly let slip the news that it now offers playback support for HEVC in Chrome when it's already available in the underlying platform. This is bigger news than Google is letting on, but its immediate impact remains to be seen. We surveyed a number of experts and analysts to gauge its meaning for increased HEVC usage and more, and how it affects DRM and HDR.
24 Oct 2022
With the recent launch of version 1.0, SVT-AV1 appears to have caught up with libaom in quality, with very definite performance advantages. Its two-pass rate control is tested and proven. If you're creating an AV1 encoding workflow today that emphasises encoding speed and quality, SVT-AV1 should definitely be on your short list.
08 Sep 2022
Royalty will be paid by streaming service providers; the codec will be free for encoder/player developers
20 May 2021
Rethink Technology Research's "Media & Entertainment Transcoding Workload and Device Royalty Forecast" breaks down anticipated codec penetration through 2030
04 May 2021
Companies and Suppliers Mentioned