Top 20 Streaming Technologies of the Last 20 Years
Apple HTTP Live Streaming (HLS)
Percentage vote: 57%
Definition. HTTP Live Streaming (HLS), Apple’s adaptive bitrate (ABR) technology, can be used for both unbounded (live) and bounded (on-demand) content playback. Apple’s HLS uses AVC/H.264 video encoding but it is the only major HTTP streaming technology to encapsulate audio and video elementary streams into a multiplexed format. Apple maintains a series of external-facing updates to HLS, currently at version 12, in an ongoing series of documents informally called the “Pantos spec” after the name of its author. The original HLS used MPEG-2 Transport Stream, requiring that both audio and video be present in the transport stream prior to segmentation, although more recent versions allow late binding of audio and video streams thanks to the implementation of byte-range addressing in which a single MP4 file can be segmented on the fly rather than requiring prior segmentation.
Why it matters. HLS started as an Apple-only de facto standard for its iOS-based smartphones and tablets, but HLS has seen widespread adoption across a variety of platforms. Even though HLS is not a standards-based, segmentation-and-delivery technology, it has become the most popular adaptive bitrate technology, used in a number of Android-based mobile devices as well as some third-party apps and HTML5 video players.
The Rest of the Best
The remaining 11 technologies received votes of less than a 50%, but several of these technologies are critical to today’s streaming media success. Because space is limited, only a few of these technologies will be detailed, while others will merely be listed along with a pertinent URL so you can further explore them.
RealAudio/RealMedia
Percentage vote: 43%
Definition & Why it matters. RealAudio/RealMedia is aproprietary video container format (it can be used with or without audio) for use in RealPlayer with a variety of video codecs, including RealVideo 10 and RV9 EHQ. RealMedia (.rm) replaced RealAudio (.ra) and RealVideo (.rv) but continued to use the proprietary audio and video codecs within the Real Media format. While quite popular for a number of years, a number of major providers have moved away from the audio-only container format.
On2 VP6, VP7, VP8 (vpx and libvpx)
Percentage vote: 43%
Definition & Why it matters. On2 Technologies came out with a number of True Motion VPx series codecs. The following highlights, adapted from the Streaming Media Glossary, describe the VP6, VP7, and VP8 versions of the codecs developed before On2 was purchased by Google.
VP6 is a video compression codec created by On2 Technologies and licensed for Macromedia Flash, known as Flash 8 Video. It uses a modified, nonstandard, fixed-point integer DCT, with complexity adjusted as a function of target quantization.
VP7 is the successor for VP6, licensed by On2 Technologies in April 2005 for use in Adobe (Macromedia) Flash video playback. It is also used for Skype video collaboration, in part due to its low-latency characteristics. On2 Technologies‚ and all its compression technology‚ was acquired by Google in 2010. On2 claimed VP7 had higher quality than H.264, but this was true only in certain instances.
VP8 is the successor for VP7. No external tests were performed on VP8 prior to Google's acquisition of On2 in 2010, but subsequent tests reveal a quality almost equivalent to H.264 commercial codecs. It was modified and reintroduced by Google as the WebM open source codec, after which time Google developed the VP9 video codec, released via an open source license.
VP9 uses the Matroska file format, called WebM in Google documentation, to deliver video via streaming or file-based means. VP9 was first released in mid-2013 and is now supported natively in Google’s Chrome browser and Mozilla’s Firefox browser. In addition, according to the WebM Project, “Experimental VP9 decode support was added to VLC 2.1.2” in late 2013. Google positioned VP9 as royalty-free and showcased support of VP9 for Ultra HD content encoding and decoding at the 2014 Consumer Electronics Show.
Perceptual processing/video optimization
Percentage vote: 43%
Definition & Why it matters. Perceptual processing is a form of video optimization that accounts for two types of typically disparate compression: spatial and temporal compression. Temporal compression is compression over time, and is widely used by a number of interframe compression standards, such as H.264 and H.265, to gain significant compression advantages by looking ahead from a keyframe (or I-frame) across a long group of pictures (GoP) and compressing only what changes on a temporal basis (known as P and B blames). Spatial is used to gain compression on an I-frame or even more rudimentary compressions such as JPEG for still images. Some forms of perceptual processing take into account the context of where the compressed video will be delivered (e.g., small screens on a smartphone over a cellular data network) while other forms of perceptual processing are aware of the types of content and assign processing based on more- or less-complex parts of a frame (or frames over time).
VideoLAN Client (VLC)
Percentage vote: 43%
Definition & Why it matters. VideoLAN Client (VLC) is a third-party player, created and maintained by Anevia, that provides playback for a wide variety of formats. VLC is a prime example of open source software being maintained as well as being enhanced to meet emerging trends in the streaming media industry, such as adaptive bitrate (ABR) and royalty-free codecs (e.g., VPx, Ogg, etc.).
Media Cleaner Pro
Percentage vote: 29%
Definition & Why it matters. When the Media Cleaner Pro product was sold to Media 100, then spun out to Autodesk and Discreet, it marked the end of a transitionary period from video delivered via CD-ROM (the focus of MCP versions 1–2) and progressive downloads (MCP versions 3–4) to the beginning of streaming-centric codecs and formats designed to truly deliver media as a stream rather than a download. MCP pioneered the concept of watch folders, and the alumni of Terran Interactive went on to deliver a number of de facto technologies that we recognize as today’s streaming industry.
MCP 3 came out in 1999, just after streaming began, but the team at Terran has continued to make significant waves in the industry. For instance, alumni Ben Waggoner, who joined Terran to head up the MCP consulting division but was laid off when Media 100 acquired the company, went on to consult on “the design of many different products, including Windows Media Encoder, Adobe Media Encoder, Telestream’s Episode Pro, Canopus ProCoder, Rhozet Carbon Coder, and Sorenson Squeeze.” Waggoner then joined back up with Terran co-founders Dana LoPiccolo-Giles and Darren Giles at Amazon, after the Giles’ and John Geyer sold CustomFlix, an on-demand DVD publishing company, to Amazon, in a move that ultimately rebranded the company as CreateSpace.
Libav
Percentage vote: 29%
Definition & Why it matters. Acting as an underlying library of A/V technologies, Libav allows FFmpeg and other open source projects easy access to a wide variety of codecs and container formats. According to the developers, “Libav is free software licensed under the LGPL or GPL depending on your choice of configuration options. If you use Libav or its constituent libraries, you must adhere to the terms of the license in question.”
Honorable Mentions
JavaScript (e.g., video.js and node.js)
Percentage vote: 43%
Microsoft Silverlight
Percentage vote: 29%
Server virtualization
Percentage vote: 29%
AnyCast
Percentage vote: 29%
AV1
Percentage vote: 29%
Conclusion
There are a number of technologies that didn’t quite make the cut—including Microsoft Smooth Streaming and the emerging Websockets approach to sidestepping TCP windowing issues—but the top 20 list we’ve covered here shows that the streaming industry of today continues to innovate beyond just the standards-based solutions available for today’s march toward OTT global conquest. Don’t forget to check out our top 20 standards and patents article, which includes a number of internationally agreed-upon approaches to streaming that have found their way into all aspects of the streaming media industry.
[This article appears in the 2018 Streaming Media Industry Sourcebook as "20 for 20: Top Technologies of the Last 20 Years."]
Related Articles
Streaming Media launched during the dot-com bubble, and while some of the early players faded away when their funding ran out, others helped chart the future of the industry.
23 Apr 2018