The State of Streaming Media Protocols 2013

The Pantos spec, as it is known in the industry, is a series of working drafts for HLS submitted by two Apple employees as an information draft for the Internet Engineering Task Force. As of the time of this article, the Pantos spec is currently at informational version 10.

Much has changed between the early versions and the most recent v10 draft, but one constant remains: HLS is based on the MPEG-2 Transport Streams (M2TS), has been in use for almost 2 decades, and is deployed widely for varied broadcast and physical media delivery solutions.

In that time frame, however, little has changed for basic M2TS transport stream capabilities. For instance, M2TS still lacks an integrated solution for digital rights management (DRM). As such, all HLS versions cannot use "plain vanilla" M2TS, and even the modified M2TS used by Apple lacks timed-text or closed-captioning features found in more recent fragmented elementary stream streaming formats.

Yet Apple has been making strides in addressing the shortcomings of both M2TS and the early versions of HLS: In recent drafts, the HLS informational draft allows for the use of elementary streams, which are segmented at the time of demand rather than beforehand. This use of elementary streams means that one Achilles' heel of HLS -- the need to store thousands, tens of thousands, or hundreds of thousands of small segments of long-form streaming content -- is now eliminated.

Google, with its Android mobile operating system platform, has adopted HLS for Android OS 4. Some enterprising companies have even gone back and created HLS playback for earlier versions of Android OS-based devices.

SMOOTH STREAMING AHEAD?

No discussion of HTTP protocol-based streaming delivery would be complete without a mention of Microsoft Smooth Streaming. After all, the Protected Interoperable File Format (PIFF) is the basis for the Common File Format (CFF) that is being used for UltraViolet, and the Common Encryption Scheme (CES) is based partly on Microsoft's 2008 idea that common encryptions and encoding could be implemented for use around the fragmented MP4 standard.

As we enter 2013, rationalization of PIFF-CFF-CES and the upcoming Common Streaming Format (CSF) will continue to pare down the number of options, which is a good thing if we are to get back to the business of creating content and delivering it to anyone who wants to view it.

Yet Microsoft isn't resting on its laurels, as the company announced in late 2012 that it would be supporting Smooth Streaming via the Open Source Media Framework (OSMF) that is part of Adobe's Strobe initiative for Flash. Yes, you heard that right: Not only does Flash Player support DASH in beta (thanks to Adobe), but it now supports Smooth Streaming (thanks to Microsoft).

Is RTSP Dead?

One of the most venerable streaming protocols, the Real-Time Streaming Protocol, has been implemented natively into every type of device: set-top boxes, smartphones, tablets, and PCs. Yet these implementations are often fraught with buggy code, limited support, and a number of oddities.

In testing performed on a number of Android OS devices, we have been surprised to find that RTSP-based video playback -- served from a standards-based server -- could not be played with built-in applications and services, despite the requirement for the base Android OS to be able to play this content. Content that would play consistently on numerous RTSP implementations would stop dead in its tracks on other devices.

This was true of any standards-based streaming protocol in the late 1990s, but these days, consumers just expect their content to stream with limited buffering and at varying data rates. RTSP offers neither of these as certainty, but is quite inexpensive to implement, so we suspect that it will be with us for at least a few more years before retiring to greener pastures to make way for DASH and more recent streaming protocols.

Where Does RTMP Fit?

At a recent informational meeting I attended with a major software company, a slide that was shown caught my attention, more for the lack of what was shown than for what the slide contained.

This particular slide listed the typical agnosticism -- codec, protocol, player -- of their soon-to-be-released update, and it had the regular litany of compatibilities. Yet what caught my attention was that the sparsest, by far, was the protocol list.

Only two protocols were noted -- HTTP and RTMP -- so I asked why RTMP was listed when other non-HTTP protocols were not. To summarize their response, they said RTSP and other non-HTTP protocols weren't being requested at all, but RTMP was still a valid industry solution.

Part of the reason lies in the fact that RTMP is "true" streaming with very low latencies and session "statefulness" that can't yet be found in HTTP-based delivery. In addition, RTMP is firmly entrenched on the vast majority of devices -- with the exception of the iOS devices -- thanks to the inclusion of the Flash Player on handsets, tablets, mobile devices, and PCs.

Yet, for all that entrenchment, as we've noted previously, Adobe continues to lean toward the HTTP delivery model in all the important ways including monetization functionality. We're not ruling out RTMP, but we do understand that the scalability and interoperability of HTTP solutions such as MPEG DASH and HLS offer compelling reasons to surf the fine streaming waves coming out of Apache servers everywhere.

Conclusion

So what does 2013 hold? We think the year is DASH's to lose.

Once DASH is officially supported in Flash, we see the possibility that DASH will be firmly enough entrenched to begin "hockey stick" growth for online video delivery. If DASH 264 can be implemented as quickly as it appears it will be ratified, and if there is some form of rationalization between HLS and DASH, including the ability to include Apple's DRM scheme in the Common Encryption Scheme, we might just note 2013 not only as the beginning of true online video delivery growth but also as the point at which cable and satellite providers begin to pay attention to delivery to all devices -- including set-top boxes -- for a true TV Everywhere experience.

This article appears in the forthcoming 2013 Streaming Media Industry Sourcebook.