Stream Stitching: What Pros Must Know About the Ad-Blocker Beater

Premium content owners that monetize their content via video ads face an enormous financial constraint: the effectiveness of ad blocking software.

"Ad blocking software and its impact on video strategies is a topic that we hear in four out of five meetings today," says Matt Smith (right), chief evangelist for Anvato. "In short, broadcasters, publishers, and programmers are seeing as much as one-third of their video views being compromised as a result of this technology."

How did we get here, and what can an industry built on pre-roll, interstitial, and post-roll advertising do to combat this massive decline in ad playback?

The answer, according to some, is the concept of "stitching" together the ad and primary content into a single stream. This process is known by several names, but the most common is server-side ad insertion (SSAI).

SSAI allows the playlist, also known as a manifest file, to be delivered to an end user's device, but with a twist. Rather than the manifest making calls to multiple locations—one for primary content, one or more for ad serving services— the manifest only requests content from a single location. In theory, this makes it difficult for ad blocking technologies to differentiate between primary content and ad content.

There are a few approaches to SSAI, some of which only manipulate the manifest and others that truly stitch the ad directly into the live or on-demand bitstream, but for simplicity, this article won't differentiate between the two. Instead we will generically call the whole process stream stitching.

According to Smith, "more than $21 billion in ad payload (display and video, to be clear) is being lost in 2015 as a result" of ad blocking. How did we get to this point?

We've Come Full Circle

In the early days, before ad serving technologies were required, the ads were "baked in" to the stream. For a live event, an ad might be played into the master control or video mixer via a digital disk recorder (DDR), while an ad destined to play in a piece of on-demand content would be encoded as part of the on-demand content.

These kinds of baked-in ads were referred to as static ads, and often were only available for pre-roll delivery. One of the promises of online video delivery, though, is an ability to customize content to any viewer—a viewer in India might see an ad for a Mumbai sweet shop, while a farmer in Montana might see an ad for the local John Deere dealership—which meant the static ad solution was lacking.

One way to address the issue would be to create hundreds of versions of the same on-demand content, with each version playing in a particular locale. On a global scale, though, that would be unwieldy at best, and the level of granularity for ad targeting would still be lacking.

On the other hand, most streaming companies weren't equipped with sophisticated ways to serve up thousands of playlists, with each playlist containing the same on-demand primary content with diverse ads based on demographic or geographic information.

Companies serving up dynamic banner ads, though, already had the technology to granularly target content to browsers. A few of these companies saw video ad serving as a large opportunity and began to offer services to premium video content owners.

Because HTML rendering in a web browser works roughly the same way for images and banner ads, with the use of inline content delivery with a "call" or request to a specific URL coded into the HTML pages, these still image ad services had an Achilles' heel: Anyone who understood the way HTML code worked could simply write a program to block particular calls to particular URLs.

And did they ever. The open-source community created ad blocking software apps, and the crowdsourcing community has continued to improve those over the years. These ad blockers have proven to be very effective against both banner ads and more recent video ads.

Ad serving companies tried to write generic code, which then was interpreted by the ad server as a request for a particular type of ad, but the ad blocking community would find these code strings and block the request.

Then ad serving companies tried obfuscating the ad request, at first by using URL or IP addresses that weren't part of the standard list of known ad serving companies' URLs or IP addresses. Those were also blocked effectively. Obfuscation went a step further, with ad requests using multiple redirects before hitting the actual ad server. Even these options have been effectively thwarted by the open-source ad blocking community.

Part of the problem with video ad serving was the need for a seamless end-user experience. No one wants to wait for an ad to load, so most client-side ad insertion (CSAI) required a dualplayer approach: one player for ads and another for primary content.

The dual-player approach had a number of technical challenges, but did allow some ads to be prepopulated to the local user's computer. But that also presented an opportunity for the ad blocking community to identify the caches and eliminate the ad altogether.

Advances in CSAI continued, including some clever solutions by Adobe that are used as part of its Primetime Player service, yet it seems that a better solution was in order—one that bears a surprising resemblance to the original concept of baking in ads into the stream. Essentially it brings us full circle, but with a twist.

Beyond the Dual Player

That solution, at least as it stands today, is SSAI, the ad stitching technique described earlier.

Brightcove acquired one of the original stream stitching technology companies, Unicorn Media, a few years ago. Unicorn had labeled its product Once, because it was able to deliver streams to the hundreds of Android and iOS devices from a single source. Brightcove now offers a derivative solution called Lift.

Mike Green, vice president of marketing and business development at Brightcove, cites a specific example from Vox Media, a Brightcove customer, to explain the benefits of Lift and SSAI in general, as opposed to CSAI.

"By implementing Brightcove Lift to stitch ads directly into the video content via server-side ad insertion, Vox Media removed some of the variables and obstacles of client-side ad calls that commonly impact performance for digital publishers," Green says.

According to Green, Vox faced CSAI challenges prior to implementing Lift. One challenge involved a less-than-seamless end-user experience for mobile devices. "Trying to do client-side ad calls with HLS on Android mobile web was so poor that Vox had actually decided to remove ads there altogether," Green says. "With Lift, Vox was able to restore monetization on Android and more broadly, on mobile web, the company significantly improved playback consistency and quality, and shortened the time to first frame between ads and video content."

Anvato's Smith says that he can't think of a scenario where his clients would prefer CSAI to SSAI.

"CSAI was good in its day, but introduced characteristics that needed to be improved upon," Smith says, "including introduction of latency, mismatched quality between program and ad payload, and others."

Smith says that the ad blocking technologies used against CSAI may have other unintended consequences.

"Client-side ad blockers see many outbound requests that video players make as a call for an ad," Smith says. "Think of it in military terms as friendly fire. The initial request for playback may be viewed in this way by the ad blockers, in the way that they see subsequent calls for video payload in a CSAI model. As a result, program content is sometimes not able to reach the viewer, and the ad payload almost certainly isn't."

Smith notes that "massive media brands need these solutions to scale without limitations" so if an event drives large-scale over-the-top (OTT) audiences, "programmers and publishers need the scale and elasticity of the cloud to be able to deliver highly targeted (per user) ad payload at scale."

Smith says Anvato has tested this "at scale with millions of requests per minute," in order to meet the needs of these customers.

Online video platforms (OVPs) tend to deal with ad blockers with a very straightforward approach, which has nonetheless proven very effective.

"Ad blockers maintain a list of rules (filters) to prevent the client from successfully making requests to ad providers," Green says. "These rules can be as simple as specifying a domain of a known ad provider or as complex as regular expressions to look for patterns in a URL."

Some ad blockers provide default "lists" of rules and, hence, rely on the power of the community to continuously enhance these lists to account for the broad range of ad types, providers, and methods.

In addition, Green says there are some very effective approaches to ad blocking with HTML5 video players.

"Some ad blockers also modify the actual HTML5 DOM itself," says Green, "to remove content that has embedded advertising or to prevent successfully requested ads from being placed in the page."

Limitations of Stream Stitching?

When asked to switch from a CSAI-based to an SSAI-based delivery solution, some companies ask whether it's worth the trouble. There are several potential objections, chief among them the potential need to preprocess ad content to match the same adaptive bitrate (ABR) bitrates and segmentation length.

This means that a true stream stitching solution might need to aggregate the ads and primary content at the same streaming server, prior to broadcasting the stitched stream.

"SSAI does require new ad content to be preprocessed and prepared for delivery for new campaigns or third-party filled ads," Green says. "The impact is small given that only the first few users will trigger the processing in which case the ad will be served the next time it needs to run on any platform or device in the region."

Green says there are workarounds and ways to further reduce impact, such as "predefined ads or known ad creatives [which] can be sent to our SSAI system ahead of the campaign to make this a non-issue."

The fact that most premium publishers or broadcasters have what Green calls "a known/ defined universe of creatives" means that the OVP can handle this, especially since premium content publishers "aren't big users of open exchanges at all" when it comes to ad serving.

Green says the trade-off between preparing content versus a consistent user experience for mobile devices is a small one to make.

"Preparing the content properly for delivery is key to the best user experience," Green says, "ensuring that the requesting device is getting the appropriate ad and content renditions for the specific connection or device type."

Smith agrees, at least regarding the best practice approach for preseeding ads, where an ad is conditioned ahead of time for playback across all screens, which may include changing bitrates and segment length.

SSAI comes at no sacrifice to dynamic and personalized ads for every user, and only the first few users of a new campaign may go unmonetized, which is by orders of magnitude less of a negative impact than the ad breakage experienced on client-side delivery. Processing the ad in a server-side solution ensures that it will be seen in the most optimal viewing experience for users, spanning across any device.

All the while it ensures delivery of the ad scales to any size audience by being backed by a single scalable infrastructure specialized for ad and content delivery.

"The absolute best SSAI practice is to preseed the ad payload prior to the first request for an ad," Smith says. "In case this isn't feasible, for whatever reason, we can have a backup set of house ads or promos that can play for the first two or three users who make an ad call for an ad unit that ‘isn't aboard' yet.

"We call this a cache miss," Smith continues. "In this case, we deliver those first few users a house ad or promo while preparing the proper ad. We then feed that ad to the cache so that every subsequent user is able to see the appropriate ad."

While this initial ad delivery limitation could be perceived as a flaw, Smith stresses that the preseeding step can and should become standard operating procedure.

What's the Next Battlefront?

Green says there's another battle lurking: packet inspection.

"Advanced ad blockers have implemented more sophisticated methods by inspecting the actual Internet packets," Green says, "not simply the request for bits and bytes but the actual bits and bytes. These ad blockers are effectively acting as a client proxy/VPN."

While this approach can be more effective, it also raises a question of privacy.

"Would a user want a third party to actively ‘snoop' all of their internet traffic?" Green says, noting that the traffic would be not only URLs but actual bits and bytes. He also points out that Apple banned a popular ad blocker app that was doing this on iOS devices, due to privacy concerns.

Still that may be a small price to pay for some users, especially those who want to see zero ads while watching premium content.

Until then, current SSAI solutions may suffice, at least when it comes to retaining a partial amount of lost ad revenue.

"A major U.S. broadcaster with a site that was heavily concentrated in tech savvy young men had an ad blocking problem," Green says. "They implemented a straight A/B test (3 separate instances of the test) and found a gain of 50 percent in ads delivered with Lift as opposed to their client-side solution."

Perhaps 50 percent is better than nothing, but in the ongoing war between ad serving and ad blocking technologies, the balance of power seems to shift every few weeks.

This article appears in the January/February 2016 issue of Streaming Media magazine as "A Stitch in Time?"

Tim Siglin's article first appeared on OnlineVideo.net