Moving Picture, Audio, and Data Coding by Artificial Intelligence: "A New Way to Make Money"

The video compression industry is ready to reinvent itself, and new standards-setting body Moving Picture, Audio and Data Coding by Artificial Intelligence (MPAI) is the mechanism to drive it, according to Thierry Fautier, President-Chair of the Ultra HD Forum and an influential voice in the shifting sands of codec development.

Speaking on an IABM-hosted Future Trends webinar about Imaging & Immersive, Fautier (who is also VP Video Strategy for Harmonic) was joined by MPAI founder and Leonardo Chiariglione, who launched the MPEG standards committee for ISO/IEC in 1988 and resigned two months ago.

"The problem confronting our industry is the fragmentation of license pools," underlined Fautier.  "This is blocking the evolution of technology."

Fautier elaborated on the current impasse at MPEG. The media coding industry is trying to move from a position of "pulling their hair out," he said, about how to implement codecs like HEVC and potentially VVC commercially.

Rival AOM, meanwhile, is "only selecting the tools that will be patent-free. [This means that] at the end of the day nobody is making money in a process which is very friendly to big companies like Netflix, Amazon, Apple and Google."

The third approach is that of MPAI. The non-profit organisation plans to develop new technical specifications of data coding, using AI to bridge the gap between those tech specs and their practical use through Intellectual Property Rights Guidelines, such as Framework Licences.

"Here is an opportunity for people to make money," Fautier said. "There will be more buy-in from license sources and licensees. You need to be upfront with those who use the technology. I believe MPAI's approach is the right one. It's a very exciting time where the video industry is ready to reinvent itself."

Whereas Fautier insists the industry is ready to pay certain fees for using technology provided it has insurance for the amount they pay, Chiariglione said MPAI was open-minded to open source.

"In my long MPEG days, I fought hard to convince a significant number of members to think that MPEG shouldn't just be something that has only one business model. We shouldn't be religious about it."

Immersive Imaging Rising

The IABM's CTO Stan Moote shared research from its latest industry report identifying COVID-related technologies including virtualization, cloud and remote production as the tech enablers at the top of media industry lists. Imaging and immersive applications like 8K HDR, VR, and MR were a little further down but still high on people's agendas.

"Immersive applications are pointless without practical compression techniques," Moot said. "What is at stake in the battle for video compression is what makes sense from a technical and business perspective."

Fragmentation is only getting more pronounced. This week the Chinese consortia behind AVS2 is on verge of releasing AVS3 targeting 8K content.

Fautier called AVS3 an alternative in case people are not satisfied with 'western' technology, "though we don't know if it will be a success outside China."

Meanwhile, HEVC patent holder InterDigital has announced that it has developed software to simplify and accelerate AI-based video compression research. CompressAI allows researchers to quickly design, train, test, and evaluate AI-based codecs. It said the open source platform (publicly available on Github) is already being used to support the development of next generation image and video codecs.

The demand for more immersive content seems to have accelerated under COVID-19.

"While 5.1 was the standard we're seeing that expand," said Greg Chin of Avid on the IABM panel. "There's no request for stereo-only mixes. There is a large and primary immersive component to shows now."

The lion's share on the audio side is taken by Dolby Atmos, but Chin said there was interest in every single areas of immersive sound, including binaural and ambisonic techniques.

Raul Aldrey, chief business officer of MediaKind, said that opportunities for implementing immersive applications were growing into 2020, and while there has been an enforced dip in the first half of the year, these plans haven't gone away.

"We're seeing these things come back to the forefront of discussion," he said.

A current example is cycling's premier event Tour de France. In cooperation with cycling app Zwift fans can join the race on their own bikes at home and compete head to head with the pros.

"There's a pressing need for live events to deliver an experience to fans at home in the absence of fans at the stadia," Aldrey said, predicting that there will be more and more of these immersive experiences crossing over into fan engagement.

Betting is also likely to merge into the mainstream of the live sports experience, enabled by real-time interaction and the growing number of states and countries where sports betting on TV is legal.

Peter Kirkup, group technical solutions manager at AR/MR solutions developer disguise, said the pandemic has demonstrated that audiences are hungry for the immersion they would experience in a live context.

"I think the new normal we will all go into will benefit from the outcome of all of this innovation," he said. "There will be new capabilities and new technologies on top of new normal which will enable us to create new more amazing things. We've already seen extended reality use cases for viewers watching a live stream of an event at home. This will accelerate even after, partly because audience expectations are now set much higher."

Aldrey also predicted that 8K will be driven by the postponed Tokyo Games. "There aren't many devices [capable of receiving 8K] in the market, but the Olympics will be used to drive 8K and other immersive experiences into the mainstream. I would say we'll see a cascade of these experiences and events beginning next year."