In Search of Better Encoding Quality for WebRTC
Suffice it to say that WebRTC is finally out of kindergarten and moving into the elementary grades. Which grade exactly? Well, that likely depends on which web browser you’re using and which server technology or platform your WebRTC implementation uses.
The majority of my client projects to date use WebRTC ingest (or publishing) via a web browser, in which JavaScript application programming interfaces can capture audio and video from locally connected devices like a webcam and microphone. As such, the video quality of the live stream is relying on the browser vendor’s encoding implementation, which could utilize your GPU or CPU. (Tip: On Chrome, you can use the URL chrome://gpu to get stats on your local machine’s GPU capabilities as supported by Chrome.) Browser-based encoding for WebRTC has limitations, specifically with variable frame sizes, frame rates, and bitrates while publishing. These variabilities can wreak havoc on server-side recordings or within video-switching software that is able to consume WebRTC feeds.
Of these limitations, the video bitrate is most troubling when it comes to high-quality video. I find that WebRTC streams published by current web browsers have an upper limit for video bitrate, usually around 2Mbps. In Chrome, you can monitor the outbound bandwidth by tapping the real-time reports generated by the URL chrome://webrtc-internals. Try it for yourself anytime you’re using a local capture of your webcam in a browser-based WebRTC publishing session. You’ll likely see similar results, regardless of which frame rate and resolution you choose for the capture. If you push the same video content with a non-browser encoder solution, such as Millicast’s OBS WebRTC software (a free download, based on the same code as the original OBS software), you can achieve much higher quality. This OBS variant can push to Millicast’s platform as well as any server running Janus WebRTC Server, which is open source.
As an example, Figure 1 (below) shows the real-time bitrate graph of a Chrome-captured WebRTC publishing session for a virtual webcam and mic driven by Telestream’s Wirecast into Millicast’s platform. Here, the video bitrate is averaging 2Mbps.
Figure 1. Chrome encoded session
Figure 2 (below) shows the same video content being streamed directly via WebRTC into Millicast with its OBS WebRTC software, which is using FFmpeg and x264 under the hood to encode the content in real time. (Note that OBS can also utilize the GPU for NVENC encoding.) The output settings for OBS were set to 8Mbps for the video bitrate, and the results in Figure 2 show an average 8Mbps bitrate being utilized for WebRTC playback.
Figure 2. OBS WebRTC encoded session
It’s beyond the scope of this column to discuss the implications of simulcast WebRTC on encoding performance within software solutions such as OBS, but bear in mind that both client- and server-side WebRTC solutions can assume the role of providing tiered resolutions for adaptive streaming playback.
What do we do in the meantime if there isn’t an option to use a WebRTC publisher for a WebRTC playout scenario? Many WebRTC platform-as-a-service (PaaS) vendors and streaming server products will accept Real-Time Messaging Protocol (RTMP) ingest sessions and transcode AAC audio to WebRTC-compatible Opus audio. As such, you can achieve higher-quality WebRTC playout streams by encoding with any existing RTMP encoding solution.
In one of last year’s columns (go2sm.com/webrtcproblem), I highlighted the lack of a consistent signaling approach among WebRTC server vendors and made a plea for WebRTC vendors to implement a standard connection setup. I’ve since seen WHIP (WebRTC HTTP Ingestion Protocol), a proposal from CoSMo Software’s Alex Gouaillard and Sergio Garcia Murillo. WHIP offers the potential to unify the next wave of real-time encoding solutions by providing an alternative and improvement to RTMP, allowing a wider range of audio and video codecs. Hopefully, popular products in the live video-switching software space, such as Wirecast, vMix, Vimeo Studio 6, Boinx Software’s mimoLive, and the master branch of OBS, will incorporate WHIP to give every web broadcaster the opportunity to work with WebRTC. On the ingest side, I expect to see similar adoption by more WebRTC PaaS vendors and media server products.
Related Articles
Look beyond the hype and the monkeys. NFTs, blockchain, and other elements of the new decentralized web—Web3—have serious implications for video creators and publishers.
17 Jun 2022
Wowza's new ultra-low latency offering can deliver either an RTMP stream or a WebRTC stream, with WebRTC outperforming the legacy protocol
21 Oct 2021
Where and how does "real time" fit into your workflows in this "COVID and beyond" era? Similar to global wars accelerating advances in medicine such as plastic surgery, COVID has pushed all of us working in streaming media to rethink, innovate, optimise, and rebuild many components of our day-to-day responsibilities.
14 Oct 2021
As a metric, Apple's Advanced Video Quality Tool (AVQT) showed some bright spots, but it's hard to see it bumping VMAF or SSIMPLUS from real-world workflows without a lot more verification.
14 Oct 2021