20 for 20: The Most Important Standards of the Last 20 Years
Throughout the history of Streaming Media magazine, we’ve often made this basic argument: Standards are good for, well, standardization of an industry, but not necessarily good for the industry’s overall innovative streak.
In other words, standards help provide commonality to scale up a stabilized technology (e.g., codecs or delivery approaches), but they often run the risk of stifling research and development of “better” technologies.
Having said that, we surveyed readers to rank a list of more than 30 overall standards, including several patents that we felt were instrumental to the growth of the streaming industry over the last 2 decades.
The following 20 standards rose to the top of overall responses.
THE TOP OF THE TOP 20
These standards received votes from more than 50% of respondents: RTMP, MPEG-2 Transport Streams (M2TS), HTML (including HTML5), HTTP, H.264 (AVC, MPEG-4 Part 10 via JVT), RTP/RTSP, TCP, UDP (and reliable UDP), MPEG-4 System, MP4 (ISO Base Media File Format).
Real-Time Messaging Protocol (RTMP)
Percentage vote: 100%
Definition. The Real-Time Messaging Protocol (RTMP) was acquired by Adobe as part of its acquisition of Macromedia. According to Adobe, RTMP “was designed for high-performance transmission of audio, video, and data between Adobe Flash Platform technologies, including Adobe Flash Player and Adobe AIR. RTMP is now available as an open specification to create products and technology that enable delivery of video, audio, and data in the open AMF, SWF, FLV, and F4V formats compatible with Adobe Flash Player.”
Why it matters. RTMP forms the basis for many low-latency origin streams used for key live-streaming services. The specification is available for download, but Adobe notes that it covers “the RTMP protocol only” and doesn’t include “information or license around any other Flash Media Server technology.”
MPEG-2 Transport Streams (M2TS)
Percentage vote: 100%
Definition. An MPEG-2 Transport Stream (M2TS) is a standard containerized and packetized format to store audio, video, program, and system information (and to transmit via the PSIP protocol) in error-prone environments. M2TS has widespread use in broadcast systems (DVB, ATSC) and even in some physical discs (Blu-Ray). ITU-T R H.222 provides for both transport and program streams.
Why it matters. From a streaming perspective, the M2TS container format is the basis of Apple HTTP Live Streaming (HLS), which has become a de facto standard for delivery of HTTP-based streams. An M2TS must be multiplexed, meaning that small portions of audio and video files are sent at the same time, then de-multiplexed (demuxed) and played back in an order determined by a playlist or manifest file.
Hypertext Markup Language (HTML)
Percentage vote: 86%
Definition. Hypertext Markup Language (HTML) is the formatting code used to create web pages, but recent advances, such as the HTML5 standard, also allow for integrated video and audio tags, as well as drawing and interactive elements as part of the Canvas substandard.
Why it matters. The use of audio and video tags within HTML eliminates the need for web browsers to allow plug-in video players. This not only makes the browser itself more secure, but potentially allows for optimized resource allocation during an end-user viewing experience, since some plug-in player manufacturers, as Steve Jobs famously stated, “should focus more on creating great HTML5 tools for the future, and less on criticizing Apple for leaving the past behind.”
Hypertext Transfer Protocol (HTTP)
Percentage vote: 86%
Definition. Hypertext Transfer Protocol is the protocol used to distribute web page content. Some streaming media productions are also sent to end-users via HTTP, such as the recently ratified DASH (Dynamic Adaptive Streaming via HTTP).
Why it matters. HTTP is the core transport protocol for web pages, using specific ports (80 and 8080) to deliver a wide variety of content to web browsers. These ports are open on all routers, unlike the specialized ports of particular streaming technologies or even the ports for multicast and RTP/RTSP streaming standards. Streaming delivered via HTTP is protected in Europe from Deep Packet Inspection (DPI), but in the U.S., the ability of network administrators to use DPI to “sniff out” video sent via HTTP has given rise to the Net Neutrality debates.
H.264 (aka AVC, MPEG-4 Part 10)
Percentage vote: 86%
Definition.A video compression codec known by various names (Advanced Video Coding or AVC, H.264, or MPEG-4 Part 10) that can be used for both low-latency solutions such as videoconferencing and very high resolution/longer latency solutions like Blu-ray discs, HDTV, and on-demand streaming content.
Why it matters. Having a single video codec, adopted by a wide number of companies across a matrix of low-versus-high latency and low-versus-high resolution use cases, allowed the industry to focus efforts on enhancing delivery methodologies. The current landscape, where competing codecs—such as H.265 (HEVC), VPx, and AV1—are all vying for attention, reminds the industry why sometimes an internationally ratified standard makes more sense than competing ad hoc solutions.
Real-Time Transport Protocol/Real-Time Streaming Protocol (RTP/RTSP)
Percentage vote: 71%
Definition.Real-Time Transport Protocol is an internet protocol that defines the transmission of real-time data, such as audio and video. RTP provides mechanisms for the sending (server) and receiving (client) applications to support streaming of any kind of data. Real-Time Streaming Protocol (RTSP) is a standardized method of transferring audio, video, and other time-based media (such as timed-text or closed-caption titles) over internet-technology-based networks.
Why it matters. RTP and its companion control protocol (RTSP) are still widely used, and gaining traction with the advent of HTML5-focused WebRTC (Real-Time Communications) audio and video conferencing. RTSP doesn’t actually transmit the streaming data, but does align the content in a time-sensitive manner. Since RTP and RTSP are based around UDP, they view time sensitivity as more important than TCP’s approach of “playing nicely” with other non-time-sensitive data on a network.
Transmission Control Protocol (TCP)
Percentage vote: 71%
Definition. Transmission Control Protocol (TCP) is a standards-based protocol designed for transmitting text and ASCII data across the internet and other IP-based networks. It provides automatic detection and retransmission of lost packets, resulting in accurate end-to-end transmission but also often resulting in considerable transmission delay.
Why it matters. TCP is primarily designed to deliver non-time-sensitive content, and is a bit “chatty” in that it confirms receipt of content sent from server to client. This means it is more reliable, but less timely, than UDP. Streaming delivery via TCP is a relatively new development, mainly due to the use of HTTP-based, small-file delivery solutions like MPEG-DASH or Apple’s HTTP Live Streaming (HLS) technology.
User Datagram Protocol (UDP)
Percentage vote: 57%
Definition. User Datagram Protocol is a method of communicating between computers that does not guarantee that every bit arrives at its end destination. It’s favored for time-sensitive data such as streaming media, including RTSP delivery. Using UDP to deliver time-sensitive video content has led to a number of derivatives, often called Reliable UDP (R-UDP), that attempt to mimic some part of TCP’s confirmation of delivery.
Why it matters. UDP delivery is often opposed by network administrators, who prefer the “plays fair” approach of TCP data. In use cases such as videoconferencing or video telephony, though, where shorter delay (lower latency) delivery times are key, it is not practical to confirm that every data packet has arrived. Yet enough data has to arrive so the video is subjectively viewable and the corresponding audio plays smoothly. Several approaches, such as R-UDP and Forward Error Correction (FEC), have been attempted to address the loss of some video data. These additions to UDP hold great promise, but each causes additional latency in the initial encoding.
MPEG-4 System (MPEG-4)
Percentage vote: 57%
Definition. A system containing audio and video codecs (AAC, H.264), container formats (MP4), metadata, and system protocols for MPEG-4 audio and video storage and delivery. Based on the initial QuickTime interactive authoring system and its Sprites concept.
Why it matters. MPEG-4 System is the overarching solution for interactivity that the MPEG standards committee put forward as an alternative to Macromedia Flash (later Adobe Flash). It defined a number of key elements, including timed-text and closed-captioning titles as well as innovations such as motion paths (and accompanying blurs or motion elements) for alpha-channeled still elements. MPEG-4 started with a fairly limited video codec, which was replaced by MPEG-4 Part 10 (aka AVC or H.264).
MPEG-4 ISO Base Media File Format (MP4 aka MPEG-4 Part 12)
Percentage vote: 57%
Definition. MPEG-4 Part 12: Also known as the ISO Base Media File Format, the standards-based MPEG-4 Part 12 (ISO/IEC 14496-12) container format is identical to the text published for ISO/IEC 15444-12 (a part of the JPEG 2000 still image standard). The MP4 extension, which is based on the QuickTime container format, is used for some (not all) ISOBMFF files. For instance, Adobe Systems introduced the F4V file format for Flash Video, noting it was based on the ISOBMFF. Yet the F4V file format was not registered by the MP4 registration authority, although the F4V technical specification is publicly available. ISOBMFF may contain H.264 video compression and MP3 or AAC audio compression. Microsoft announced in 2009 a file format based on ISOBMFF with the extension ISMV, better known as Smooth Streaming and the subsequent Protected Interoperable File Format (PIFF). However, no mention was made of the WMA compression format in ISOBMFF, so it may be unsupported by some platforms.
Related Articles
Streaming Media launched during the dot-com bubble, and while some of the early players faded away when their funding ran out, others helped chart the future of the industry.
23 Apr 2018
Thanks to a fractured HEVC licensing system companies no longer have the financial incentive to innovate, but Leonardo Chiariglione suggests steps to reverse the damage.
31 Jan 2018
Think RTMP is out of date? Despite the ubiquity of HTTP video streaming, there are still use cases where RTMP is the standard.
02 Dec 2014
In this guest post, Bitmovin's CTO and co-founder explains how HTTP/2.0 will solve the problems of previous generations and seamlessly integrate with DASH.
07 Aug 2014
Not sure what to make of the new format on the block? Read this to get up to speed on how HEVC was created, the challenges it now faces, and when it will go into everyday use.
14 Feb 2013