What Is HLS (HTTP Live Streaming)?
HTTP Live Streaming (HLS) Executive Summary
HTTP Live Streaming (or HLS) is an adaptive streaming communications protocol created by Apple to communicate with iOS and Apple TV devices and Macs running OSX in Snow Leopard or later. HLS can distribute both live and on-demand files and is the sole technology available for adaptively streaming to Apple devices, which is an increasingly important target segment to streaming publishers.
HLS is widely supported in streaming servers from vendors like Adobe, Microsoft, RealNetworks, and Wowza, as well as real time transmuxing functions in distribution platforms like those from Akamai. The popularity of iOS devices and this distribution-related technology support has also led to increased support on the player side, most notably from Google in Android 3.0.
In the Apple App Store, if you produce an app that delivers video longer then ten minutes or greater than 5MB of data, you must use HTTP Live Streaming, and provide at least one stream at 64Kbps or lower bandwidth. Any streaming publisher targeting iOS devices via a website or app should know the basics of HLS and how it’s implemented.
How HLS Works
At a high level, HLS works like all adaptive streaming technologies; you create multiple files for distribution to the player, which can adaptively change streams to optimize the playback experience. As an HTTP-based technology, no streaming server is required, so all the switching logic resides on the player.
To distribute to HLS clients, you encode the source into multiple files at different data rates and divide them into short chunks, usually between 5-10 seconds long. These are loaded onto an HTTP server along with a text-based manifest file with a .M3U8 extension that directs the player to additional manifest files for each of the encoded streams.
Figure 1. HLS uses multiple encoded files with index files directing the player to different streams and chunks of audio/video data within those streams.
The player monitors changing bandwidth conditions. If these dictate a stream change, the player checks the original manifest file for the location of additional streams, and then the stream-specific manifest file for the URL of the next chunk of video data. Stream switching is generally seamless to the viewer.
HLS File Preparation
HLS currently supports H.264 video using the Baseline profile up to Level 3.0 for iPhone and iPod Touch clients, and the Main profile Level 3.1 for the iPad 1 and 2. Audio can be HE-AAC or AAC-LC up to 48 khz, stereo. The individual manifest files detail the profile used during encoding so the player will only select and retrieve compatible streams. This allows producers to create a single set of HLS files that will serve iPhone/iPod touch devices with Baseline streams and iPads with streams encoded using the Main profile.
Though encoded using the H.264 video codec and AAC audio codec, audio/video streams must be segmented into chunks in an MPEG-2 transport stream with a .ts extension. All files are then uploaded to an HTTP server for deployment. In a live scenario, the .ts chunks are continuously added and the .M3U8 manifest files continually updated with the locations of alternative streams and file chunks.
Before producing files for HLS, you should read through Apple’s Tech Note: Recommended Encoding Settings for HTTP Live Streaming Media, which contains detailed recommended configurations (resolution, data rate, keyframe interval) for distributing both 4:3 and 16:9 video to all compatible iDevice and Apple TV players. To supplement this data, you should also check out Adaptive Streaming in the Field, which details the configurations used by several publishers when producing their files for HLS distribution.
Content Protection/Closed Captions in HLS
HLS doesn’t natively support digital rights management (DRM) though you can encrypt the data and provide key access using HTTPS authentication. There are several third-party DRM solutions becoming available, including from AuthenTec, SecureMedia, and WideVine. HLS can support closed captions included in the MPEG-2 transport stream.
Deploying HLS Streams
Delivery via HTTP has several advantages; no streaming server is required and the audio/video chunks should leverage HTTP caching servers located in the premises of internet service providers, cellular providers, and other organizations, which should improve video quality for viewers served from these caches. HTTP content should also pass through most firewalls.
Apple recommends using the HTML5 video tag for deploying HLS video on a website.
On the Playback Side
On computers and iPad devices, the Safari browser can play HLS streams within a web page, with Safari launching a full-screen media player on iPhones and iPod touch devices. Starting with version 2, all Apple TV devices include an HTTP Live Streaming client.
Producing HLS
As discussed, the HLS experience has two components; a set of chunked files in .ts format and the required manifest files. In an on-demand environment, you can encode the alternative files using any standalone H.264 encoding tool, with the latest version of Sorenson Squeeze offering a multiple file HLS encoding template. More recently, Telestream updated Episode to include command line HLS multiple file creation. Cloud encoding services like those provided by Encoding.com can also typically produce HLS-compatible files.
Once you have the encoded streams, you can use Apple tools to create the chunked files and playlists. Here are the HLS tools available for download from Apple:
- Media Stream Segmenter - inputs an MPEG-2 transport stream and produces chunked .ts files and index files. It can also encrypt the media and produced encryption keys.
Media File Segmenter - Inputs H.264 files and produces chunked .ts files and index files. It can also encrypt the media and produced encryption keys.
Variant Playlist Creator - Compiles the individual index files created by the Media Stream or Media File Segmenter into a master .M3U8 file that identifies the alternate streams.
Metadata Tag Generator - Creates ID3 metadata tags that can either be written to a file or inserted into outgoing stream segments.
Media Stream Validator - Examines index files, stream alternates, and chunked.ts files to validate HLS compatibility.
For live HLS distribution, you need an encoding tool that can encode the files into H.264 format, create the MPEG-2 transport stream chunks and create and update the manifest files. When Apple first announced HLS in 2009, only two live encoders were available; one each from Inlet (now Cisco) and Envivio. Now most vendors of encoding hardware also offer live HLS-compatible products, including Digital Rapids, Elemental Technologies, Haivision, Seawell Networks, and ViewCast.
Real Time Transmuxing
The other approach to live or on-demand streaming to HLS-compatible players is via transmuxing, which is offered by multiple streaming server vendors and CDNs. Specifically, these servers input an H.264-stream originally compatible with Flash or Silverlight (or other formats) and then dynamically re-wrap the file into the required MPEG-2 transport stream chunks and create the required manifest files.
Server-based implementations include:
Akamai also offers “in the network” repackaging of H.264 input files for HLS deployment.
In these applications, any live encoding tool that can deliver multiple streams of input to the server, like the Adobe Flash Live Media Encoder, Haivision, Microsoft Expression Encoder Pro, or Telestream Wirecast, can serve as the encoding front end for multiple-platform adaptive distribution including HLS.
Figure 2. Microsoft’s Transform Manager inputting Silverlight-compatible streams and producing HLS compatible output.
Not surprisingly given the level of technology support, many of the larger online video platforms are now starting to support HLS distribution, including Brightcove, Kaltura, and Ooyala.
Conclusion
The iOS platform is a critical target for virtually all streaming publishers, and HLS can deliver the best possible experience to that platform, and others that support HLS playback. Fortunately, the streaming industry had embraced HLS with tools and technologies that make this very simple and affordable.
HLS Resources
One of the reasons that HLS has been so successful is that Apple has created multiple documents that comprehensively address the creation and deployment of HLS files. Apple has created an HTTP Live Streaming master page with links to all these resources that you can access here.
Other Resources
Akamai - Encoding for iPad (whitepaper)
RealNetworks - Delivering content to Apple iPhone, iPod Touch and iPad using RealNetworks Helix Solutions (whitepaper)
StreamingMedia.com - How to Encode Video for the iPad, iPhone, and iPod touch (video tutorial)
Related Articles
When will the industry jettison HTTP-segment-based streaming and buffer-based playback, both of which hold us back? How about right now, our columnist proposes.
10 Sep 2019
A high-level view of state of streaming media in 2019, and the concepts and and technologies that make it work.
12 Feb 2019
Now that virtually all major players support Apple's HLS, the shortcomings of that de facto standard may lead to more widespread implementation of MPEG-DASH
01 May 2013
Not sure what to make of the new format on the block? Read this to get up to speed on how HEVC was created, the challenges it now faces, and when it will go into everyday use.
14 Feb 2013
HTTP, RTSP, RTMP, Smooth Streaming, HLS, HDS, and more: What exactly are streaming protocols, and how do they interact with other communications protocols?
22 Aug 2012
Jan Ozer's Streaming Media West presentation on adaptive bitrate can turn casual encoders into instant experts.
30 Jan 2012
MPEG DASH is the latest hot topic in the online video space. Here we break down what it is, and what its implications might be for video delivery in the future.
22 Nov 2011
A look at what adaptive streaming is, the primary technology providers, and the factors you should consider when choosing an adaptive streaming technology
28 Apr 2011
A look behind H.264, the world's most popular video codec, including encoding parameters and royalty concerns
04 Apr 2011
An explanation of HTML5 and HTML5 Video, including history, patent issues, and current use by Apple, Microsoft, Google, Adobe, and others.
10 Mar 2011
A high-level view of streaming media technology, history, and the online video market landscape
26 Feb 2011
Companies and Suppliers Mentioned