July 17, 2015
By Jan Ozer Contributing Editor
Featured Articles

HTML5 Comes of Age: It's Finally Time to Tell Flash Good-bye

We’ve been hearing that Flash is dead since Apple introduced the iPad back in 2010. Yet it’s still the predominant technology used by premium websites, particularly those that are ad-supported. This is because the first generation of HTML5-based video playback technology, essentially the famous video tag, didn’t enable features critical to the distribution of premium content, such as adaptive streaming, digital rights management (DRM), live streaming, or even true streaming as opposed to progressive download.

At long last, three new HTML5 technologies, the Media Source Extensions (MSE), Encrypted Media Extensions (EME), and Dynamic Adaptive Streaming over HTTP (DASH), will help premium content producers overcome these obstacles, and transition to HTML5 for the adaptive delivery of live and VOD content with DRM. It won’t be easy, and it won’t be particularly clean, but it will definitely be possible.

Definitions

Let’s start with some high-level definitions of the three technologies. MSE is a W3C HTML Working Group specification for a JavaScript interface to play back media data within a browser. It’s more functional than the video tag, and browsers and devices that support MSE can play chunks of video (or byte-range requests within a single file), which enables the adaptive delivery of live and VOD content.

DASH is a standardized file format, much like Apple’s HTTP Live Streaming (HLS) or Microsoft’s Smooth Streaming. Like all HTTP-based adaptive streaming formats, there are two elements: fragmented video files (or byte-range requests within a single file), and manifest files, which identify the location of the various files within an adaptive group and the location of the chunks or byte-range requests of the individual segments. In use, most DASH content is contained in MP4 files, while the manifest files are MPD, which stands for media presentation description, files.

MSE and DASH go together hand in glove. That is, for a browser or device to play DASH files, it must support MSE. So MSE is the playback spec, while DASH is the specified file format.

EME is another JavaScript API that enables HTML5-based DRM by extending MSE with application programming interfaces (APIs) to control the playback of protected content. Before EME, DRM was handled by a plug-in such as Flash, Silverlight, or Widevine, which communicated with the license server to unlock protected content. EME eliminates the middleman and allows the browser or device to communicate directly with the license server.

MSE and EME

When creating encrypted files, a standard called the common encryption scheme (CENC) details the standard encryption and key mapping techniques used to store the DRM-related data. On the playback side, EME works by incorporating what’s called a content decryption module, or CDM, into the browser or mobile operating system. For example, Google includes the Widevine CDM in Chrome and Android. As currently implemented, each browser or platform includes only one or two CDMs. So where most producers previously supported only a single DRM for distribution to all or most of their target platforms, EME’s one platform-one CDM dynamic will force most producers to support multiple DRM technologies.

Note that not all HTML5-based technologies seeking to supplant Flash and Silverlight rely on MSE/DASH/EME. In particular, OpenTelly’s THEOplayer enables the playback of encrypted HLS streams in an HTML5 browser without Flash or Silverlight plug-ins. As you’ll see, this is important, because while the penetration of HTML5-compatible browsers is closing in on 98 percent, the percentage of browsers that support MSE and EME is much, much lower.

Support for MSE and EME

As with all things HTML5, MSE and EME only work in browsers that support the new specifications. While precise numbers for desktop browsers are elusive, the general principles and some rough estimates are shown in Table 1, which is derived from data supplied by website StatCounter in May 2015.

Google Chrome started supporting MSE/EME back in version 23, so most of the installed base of Chrome browsers (now at version 43) supports these specs. Microsoft Internet Explorer 11, with a 15 percent total share, supports MSE, but only on Windows 8.1, which currently has about an 11 percent market share of all PCs and desktops. If you assume that all Windows 8.1 computers also run IE 11, which is reasonable, that’s 11 percent added to the Yes column, 4 percent to the No column. IE versions 8–10, which account for 16 percent of browsers according to StatCounter, do not support either spec.

Regarding Firefox, Mozilla released support for DASH as early as version 21, and YouTube now uses DASH in Firefox version 38. EME support is nascent and reportedly buggy, with only 32-bit versions of the browser supported on Windows, and not 64-bit versions. Though it will be a while before all Firefox browsers on all platforms support EME, we still count it as a yes in Table 1.

Note that Safari supports MSE/EME, but only in OS X Yosemite, so the full 2 percent of the installed base there is unlikely. Finally, while Opera supports MSE (and not EME), it’s for the WebM codec only, and most producers plan to use H.264. These numbers take us to 86 percent total browser share, with the vast bulk of the remaining 14 percent not supporting MSE or EME.

In mobile markets, iOS doesn’t support MSE or EME, but Android has supported MSE since version 4.1, and EME since version 4.3. According to Google’s developer dashboard, this means that close to 90 percent of Android devices should support MSE, and about 55 percent would support EME. Windows Phone 8.1 supports MSE, but not EME.

EME Fragmentation

As mentioned, most browsers and mobile operating systems will support only one, or at most, two DRMs. This is shown in Table 2. Within this dynamic, producers have two options; they can support multiple DRMs, or force their viewers to watch only on selected platforms or devices, as in Chrome, but not IE, Firefox, or Safari.

Supporting multiple DRMs shouldn’t be a problem logistically or contractually. That is, the CENC standard can include key-related information for multiple DRMs in a single file. On the contractual side, many DRM providers, such as BuyDRM, DRM Today, EZ-DRM, and Verimatrix, now support multiple DRM technologies. So instead of contracting directly with Microsoft, Adobe, Widevine, and Apple, a video distributor should be able to find one or two DRM providers to access all necessary DRMs.

DRM technology support

Table 2. DRM technologies supported by the browser/platform

What isn’t clear is Apple’s plan for FairPlay. Specifically, while it appears that Apple has made FairPlay available to Netflix and Hulu, Apple hasn’t announced whether it will license FairPlay more generally, and if so, whether it will be through third-party distribution, or only directly.

The Known Knowns

With this information as background, several realities about supporting HTML5 become evident. Let’s run through them.

MSE/EME/DASH WON’T BE A UNIVERSAL SOLUTION IN THE NEAR TERM

Today, most producers output files for Flash or Silverlight playback for desktop distribution and HLS for iOS. With Apple’s recalcitrance to adapt MSE/EME for iOS, even though DASH might replace Flash or Silverlight on the desktop, it won’t on Apple devices, at least for browser-based playback. As with Flash- and Silverlight-compatible formats, distributors might be able to produce an app that plays DASH files.

On the other hand, many video distributors create one set of files and transmux as necessary with tools such as the Wowza Streaming Engine, which already supports DASH. Briefly, transmuxing is a lightweight, real-time operation that involves changing the container format of the encoded file and creating the necessary manifest files. It does not require transcoding or re-encoding files and can be performed by the server without introducing significant latency or additional load. So while switching from Flash or Silverlight to MSE/EME might not reduce the file creation requirements, it likely won’t expand them either.

FALLBACK IS ESSENTIAL FOR MSE/EME

Given that current support for MSE/EME is hovering around 65 percent, it’s clear that producers can’t cut over to the new standards without addressing noncompliant browsers and platforms. Typically, this involves a technique called fallback, or returning to whichever technology the browser supports. In operation, the player queries the browser to determine its capabilities; those that support MSE/EME receive the DASH files. Browsers that don’t support MSE/EME receive files in whichever format they do support, which is usually

Flash or Smooth Streaming for Silverlight. Interestingly, several of the off-the-shelf players discussed below can transmux DASH files into the required format in real time within the browser. This includes both transmuxing the content and falling back to the required DRM. Absent this feature, the distributor would have to transmux at the server or create multiple packages of files and DRM for MSE/EME and Flash or Silverlight.

PRODUCERS SHOULD CONSIDER OFF-THE-SHELF PLAYERS

Many Flash and HTML5 producers use off-the-shelf players to reduce development cost and time to market. With capabilities such as player-side transmuxing and multiple DRM support key to implementing HTML5-based video playback, those considering moving to HTML5 should consider doing the same. There are several players available; let’s start with those that participated in a Streaming Media East panel last May titled Replacing Flash: Adaptive Streaming and DRM in HTML5, Bitmovin, CastLabs, and OpenTelly.

Bitmovin

The Austrian company Bitmovin offers the bitdash player and bitcodin cloud trancoding service. The player schema deploys DASH on compatible browsers with fallback to Flash or HLS on legacy platforms and iOS. The scheme anticipates supplying both DASH and HLS encoded files, with Flash fallback supported via in-the-player transmuxing. For Android, the company supports DASH playback on MSE-compatible versions, or via an app based upon WebView or ExoPlayer.