Key Considerations for Live Encoding

As more and more live streaming content is consumed on a widely disparate group of devices—including both lowerand upper-end mobile phones and tablets, as well as smart TVs and 4K-capable set-top boxes (STBs)—the decisions around choosing a live streaming encoder have grown more complex.

Consider the fact that a large number of live streaming viewers will consume a single live event across at least two devices—whether sitting in the comfort of their living room, watching the big game on a smart TV, while also interacting with “watch party” friends on a second screen, or just watching part of the event on a mobile device while in transit between work and home—and it’s easy to see that live streaming workflows are much more complex than they were even four or five years ago.

Roundtripping Between Devices

Fortunately, newer live streaming solutions have benefited from two decades of consumer-facing features added to video-on-demand (VOD) media streaming workflows. Beyond the standardization around HTTP-based delivery, pioneered by the likes of Adobe, Apple, Microsoft, and others more than a decade ago, user-focused VOD benefits are now available as standard options on most of today’s live streaming encoder platforms. As an example, consider how roundtripping content between devices has jumped from being a VOD-only feature to becoming
table stakes for live streaming solutions.

The VOD roundtrip benefit addressed a consumer frustration: when VOD content viewing started on one device and was completed on another device. Consumers had complained that they were forced to start at the beginning of the content on the second device, manually searching until they found the same spot. The advent of roundtripping content (essentially keeping track of time-based pointers in a single-users playlist that was shared between devices) allowed consumers to start watching content on one device and then seamlessly shift to another device.

In live streaming delivery solutions, this roundtripping benefit is expanded to catch-up services (essentially a DVR-like functionality that allows the viewer to miss a few minutes of a game as they transition to another device, but “catch up” on that gap to check whether any major event occurred in the gap). Some solutions allow for full-event catchup (e.g., major sporting events, inclusive of must-see commercials) while others allow for a minimal catch-up window of several minutes.

There are challenges, of course, several of which were covered in a session at the recent Streaming Media East 2023 session on Meeting the Live-to-VOD Challenge, but also new benefits as livestreamed catch-up functionality has added the ability to create on-the-fly highlight reels of key events, which are then delivered as VOD clips.

Some content owners choose to make VOD content available to non-paying viewers as a way to entice them to subscribe or pay for an ongoing pay-per-view event. As an enhancement beyond the typical DVR-based catchup services or manually created highlight reels mentioned above, experience dealing with large-scale live events has brought about additional enhancements.

Recently, a number of companies have announced machine learning options—often using the term AI or artificial intelligence—with the intent of automating the creation and packaging of these highlight reels for live sports. The solutions are programmed to identify ball movement, referees’ whistles, crowd roars, and other important aspects of a key play or goal.

Given the higher likelihood that viewers will consume live-event content across multiple devices—whether at work, home, or in the transition between locations—the table stakes in encoding have risen significantly over the past few years. Ensuring a high-quality origination stream from the encoder to the consumer is vital for successful live streaming—especially now that consumers are demanding catch-up functionality with content quality expectations for this functionality that rivals VOD content encodings that have the luxury of multipass encoding.

Ensuring Origination Stream Quality: Key Factors to Consider

So, what factors should be considered when attempting to ensure a high-quality origination stream? Each live event streaming workflow shares similarities, but I’ve listed some key factors to consider below.

Venue bandwidth: Given the transitory nature of live-event content consumption, it’s not just the bandwidth issues on the consumer delivery front but also at the venue that need to be considered. There are a number of solutions designed to assist with intermittent network availability—most based around the concept of forward error correction, which can extrapolate parts of the content even if packets are missing—but nothing guarantees a quality ingest into a modern encoder more than a rock-solid IP content acquisition network.

Encoding parameters: The choices you make when setting up a live-stream encoder are also critical in enhancing the viewing experience your consumers, so choosing a live streaming solution that can encode anything for very-low-bandwidth 3G mobile all the way up to 4K high-dynamic range (HDR) streams—ideally at the same time, as a way to save on both power consumption and processing time—is a key factor in your live streaming encoding and delivery workflow.

Multiple protocols: Not only should the live encoder be able to handle the multiple bandwidths and resolutions required for viewers to properly experience your live event; it should also be able to handle multiple input (acquisition) and delivery (origin server) transmission protocols. We’ll cover acquisition below, but on the origin server front, make sure the minimum delivery outputs cover at least classic RTMP as well as one of the several HTTP-based delivery options. This flexibility will both provide a redundancy of origin streams and also add flexibility as you choose a streaming delivery partner to help deliver streams to all your viewers.

Scalability: Besides planning for your live-event streams to reach every type of viewer device—desktop, laptop, smart TV, smartphone—the question of scalability is probably the next major concern when choosing both a live streaming encoder and streaming delivery partner. Scaling up to meet viewership demand in the past was often seen as a “black art,” but the last two decades of live event streaming have generated a number of seasoned delivery partners. The great thing about working with a partner is that they’ve not only been through scaling up a similar live event stream but also may have worked with the very live stream encoder you’re using, meaning they may make a number of suggestions on how to create the best origin stream.

Acquisition decisions: When it comes to content acquisition, the best device is the one that you have on hand. In some instances that might be a higher-end smartphone, in others it may be a laptop—remember just how much remote production (REMI) was done during the pandemic lockdowns via business and consumer laptops—or it may be a group of high-end broadcast cameras.

Input sources: Most of today’s hardware-based live streaming encoders allow input via IP (including both Wi-Fi and Ethernet), as well as via the standard digital video connectivity (such as SDI or HDMI), and many of these come in very compact—and reasonably priced, for the feature set—options.

Software encoders: Don’t want to spend capital on buying a hardware encoder? You’re in luck, as the new breed of software-based online encoders offers many of the features found in hardware encoders while also providing acquisition from all sorts of low-latency, bi-directional video—be it FaceTime, Meet, Skype, Teams and a number of others—as well as plain, old RTMP and newer WebRTC streams.

Monitoring: Before and during any event, monitoring is a key component of your live streaming workflow. This can take on the form of on-device monitoring, which is a feature often included in hardware-based encoders, or it can be monitoring via an app provided by the hardwareor software-encoder solution. I’ve personally used one app-based monitoring solution for a very small hardware encoder, which has ultra-low-latency monitoring that’s proven helpful in catching audio- and video-sync issues before a live event stream goes public.

Additional Resources

We’ve only hit the highlights you should consider when choosing a live event streaming encoder, but here are a few additional resources. Several articles in the recent 2023 Streaming Media Sourcebook cover live streaming options.

In addition, this special checklist section includes an accompanying article, “The Ultimate Guide to Live Encoding,” which follows this one and highlights additional considerations when selecting the right encoder for your workflow; I strongly encourage readers to review that article carefully while making purchasing decisions around live stream encoders and streaming delivery service partners.