SMNYC 2024: Streaming Learning Center's Jan Ozer Talks AI, Codecs, and Patent Pools
At Streaming Media NYC 2024, Tim Siglin, Founding Executive Director, Help Me Stream Research Foundation, and Contributing Editor, Streaming Media, interviews Jan Ozer, Owner, Streaming Learning Center. Ozer discusses how the Streaming Learning Center has expanded into a platform offering courses on streaming-related topics to help technical people in the industry become familiar with terms, technologies, best practices, and tools. He also talks about machine learning, the shift towards Neural Programming Units (NPUs), and the legal and financial implications of choosing a codec due to potential patent infringements.
What is the Streaming Learning Center?
Siglin asks Ozer to describe the Streaming Learning Center and what it does.
Ozer says that Streaming Learning Center is a blog he started around 13 years ago that contained articles about compression, codecs, and encoding. Then, about five years ago, it began to offer courses in streaming-related topics. “It just helps technical people coming into the industry get familiar with all the terms and technologies and best practices and tools that they need to use to become conversant and effective in their roles, either on the user side or on the builder side,” he says. “So those are my two targets.”
What is the impact of AI on codecs?
Siglin says, “From a builder standpoint, obviously, quite a bit has changed in the last couple of years. For instance, machine learning, which some people call AI, has become more popular, at least from a buzzword standpoint. What's your take on that?”
Ozer says that while machine learning and AI have exploded into the public consciousness since ChatGPT came out, companies that have been in the business for a while, such as Harmonic and Vizioner, have been writing about and using machine learning in their codecs since as early as 2016. “So I think it has been a big part of codec development for a while, primarily in content adaptive technology,” he says. “So the ability to gauge a technology and estimate what bitrate would be necessary to encode it effectively and then [to] only use that bit rate.”
Siglin asks, “So how has it progressed from that 2016 timeframe to now?”
“It’s hard for me to say,” Ozer replies. “The one thing I discovered is that it's impossible to separate the efficiency delivered by AI from the efficiency delivered by other techniques. So everybody's going to say machine learning, or AI, or generative in their spec sheets, but you're never going to be able to tell, is it 5%? Is it 10%? I spoke at a panel on NAB, and I told people to ignore when they see an encoder with AI. If you're evaluating it, evaluate it like any other encoder and gauge its overall effectiveness, not what contribution AI has overall.”
Siglin says, “We've got workflows that have machine learning, but are we seeing a similar scenario where new codecs will say, ‘Look, we've got AI, therefore we're better, or is it strictly AI and the workflow?”
“The whole thing that stops codec deployment is the decode side,” Ozer says. “And there's a huge preference for hardware decode in mobile. There's a necessity for hardware decode in the living room, where all premium content is really targeted. So that's where all the money goes. And it's very challenging to change traditional codecs in a way that makes them incompatible with those decoders. There's a company called Deep Render, and they're building their codec to be decoded on NPUs (Neural Programming Units). And those have been shipping in edge devices since 2015 or 2016. It's interesting because those are general purpose devices, and you can use them for any machine learning, any type of AI on [the] unwinding that you need. It feels like the move is going to be toward them, as opposed to codec-specific silicon. It may be that VVC or even HEVC could be the last dedicated silicon decode-based codec, and we could be moving towards NPUs. A big part of the recent Apple announcement with the M4 chip was 18 TOPS or 36 TOPS NPU unit on the chip. And that can more than handle decode for any of the new codecs that are coming out that are AI-based.”
The legal and financial implications of 2024 codec patent pools
Siglin asks, “Where do we stand in the patent pool space at this point? On the encoding side especially.”
“We are at a huge inflection point,” Ozer says. “Avanti came out with Avanti video pool that is charging royalties based on content. So it's not encoder, it's not decoder, it's encoded bitstreams. There was a recent case in Germany, Broadcom versus Netflix, where the court found Netflix violated Broadcom's patents because they, not encoded, not decoded, but transmitted HEVC encoded Bitstreams within Germany, which is content. So it's content royalties. There's a suit in California where Nokia is suing Amazon for H.264 and H.265 bitstreams, not where they're encoded, not where they decode. And so we've got three or four sources of law that are going to say if you're encoding in a specific format, you're liable to the patent owners for some fee as yet undetermined. So, I think if you look at codec adoption in 2024, I could be the only person who cares about this. Everybody I speak to is either unaware of it or doesn't think it's going to be significant. But I really think before I adopted codecs in 2003, back then you needed to be a codec expert to choose a new one. And even in 2013, or 2018, or 2019, you needed to be a codec expert. Now you need to be a legal expert, a chief financial officer, because you've got to evaluate all these pools and all these lawsuits, and it helps if you know a little bit about encoding.”
Siglin says, “So to the extent that we've mastered encoding as an industry, what we've not mastered is the financial implications of how we choose to encode and the potential legal ramifications.”
“Exactly,” Ozer says.
See more highlights and interviews from SMNYC.
Watch full sessions from SMNYC on-demand.
Join us in August 2024 for more thought leadership, actionable insights, and lively debate at Streaming Media Connect.
Related Articles
Viewer demand for high-quality 4K video content has skyrocketed, and there is an expectation that 8K video will also surge in the coming years. However, with increased resolution comes much larger file sizes, which is challenging for media providers because of the increased storage and processing power needed. Naturally, media companies are seeking ways to process video more efficiently. Frank Schönberger of MainConcept explores how content providers and video services leverage technological developments in video compression and AI to improve efficiency at every stage of the broadcast workflow, from the point of capture to delivery to the end user.
18 Jun 2024
How close to obsolescence is AVC in 2024, with AV1 and other more efficient and increasingly well-established "emerging" codecs like VP9 and HEVC on the rise? Meta's Hassene Tmar, Netflix's Andrey Norkin, AMD's Sean Gardner, and Help Me Stream's Tim Siglin weigh in during this clip from Streaming Media Connect 2024.
29 Mar 2024
Migrating to new codecs is fraught with challenges and complications for companies with massive content libraries like Netflix and Meta, or live linear providers like United Cloud. What drives change-of-codec decisions and what considerations and trade-offs complicate these decisions vis a vis legacy users and more? Netflix's Andrey Norkin, Meta's Hassene Tmar, United Cloud's Boban Kasalovic, and Help Me Stream's Tim Siglin discuss these issues and more in this clip from Streaming Media Connect.
15 Mar 2024
Avanci Video launched in October 2023 as the latest patent pool contender to stake a claim to a piece of the streaming codec pie. Avanci isn't new to licensing; just to streaming. So how does it impact other existing patent pool players? Two leading patent attorneys with expertise in this market, Garrard R. Beeney of Sullivan & Cromwell, and Robert J.L. Moore of Moore IP Solutions explore what it meant when Avanci joined the patent pool party in this clip from November 2023's Streaming Media Connect.
15 Jan 2024
Via Licensing shook the MPEG licensing world in May 2023 when it acquired MPEG LA to form the Via LA juggernaut. And how is that deal shaking out now, six months on, as licensors and patent holders alike survey the landscape? Sisvel Group's Mattia Fogliacco and patent attorney Robert J.L. Moore of Moore IP Solutions discuss the implications in this clip from November 2023's Streaming Media Connect.
12 Jan 2024
What are the key factors that drive and impede new and emerging codec deployment on premium OTT streaming platforms like Netflix, Disney, and Apple TV+? Streaming Learning Center's Jan Ozer provides a much-needed codec deployment reality check in this clip from his presentation at Streaming Media East 2023.
25 Aug 2023
Companies and Suppliers Mentioned