How to Create Interactive HTML5 Video
(Note: Mobile browsers typically do not allow a video clip to start automatically without the viewer interacting with the user interface. As such, test any playlist functionality you’re building on each target platform.)
Like many of the features discussed in this article, playlists are not natively supported by the HTML5 <video> tag; instead, playlists are implemented by customized JavaScript code. JW Player 6 has built-in support for playlists in all editions, and extended advertising features for playlists is available in the Enterprise or “Ads” edition. As with the thumbnail feature, Video.js can include playlists by way of a separate JavaScript plug-in that is available on its GitHub repository.
The JW Player 6 JavaScript API has a playlist property in the initial object passed to the setup() method. This property is an array of multiple video items, each with its own sources and tracks properties. After a playlist has initialized in JW Player, the viewer can choose a video from the playlist interface as shown in Figure 6. The videos will play continuously without viewer intervention.
After a playlist has initialized in JW Player, the viewer can choose a video from the playlist interface.
As you plan and construct a browser-based video experience for content catalogs, how you segment your video can be critical to a positive user experience. For example, do you create chapters within one video clip, or do you create separate clips and treat them as a playlist?
Depending on your streaming server and player codebase, you may be able to carve out different options on the fly. For example, with Wowza Streaming Engine 4 and JW Player 6, you can use start and duration parameters in HTTP streaming URLs to divide one video clip into independent items for a playlist, as shown in Figure 6. Alternatively, you can use a chapter track to showcase the same divisions within one clip, as shown earlier in Figure 5.
Enhancement #5: Captions
Last but not least, captions for video provide assistance for the hearing impaired, and provide an alternate means for viewers to experience the content without listening to the audio. Captions can also enhance the seek experience by enabling a viewer to scan for keywords displayed in the captions. For example, if you’re watching a how-to video on hardwood floor installation and want to jump to a section where the narrator mentions the word “fir,” you can scrub the video for that caption text. Figure 7 shows the display of caption text using JW Player 6.
Caption text displayed using JW Player 6.
(Note: In the United States, if your online video content has been aired on broadcast TV with captions, you likely need to include closed captioning capabilities with the online version. For more information on the Twenty-First Century Communications and Video Accessibility Act of 2010 (CVAA).)
As mentioned earlier in this article, most HTML5 browsers do not yet natively support a <track> element for the <video> tag. Thankfully, most of off-the-shelf web video players support the display of captions on top of the video by way of custom JavaScript and CSS controls. While the formatting of caption data may vary from one player vendor to the next, the industry seems to be rallying around the WebVTT format for captions. The listing below is an example of the format:
WEBVTT
1
00:00:11.170 --> 00:00:14.330
The Columbia is one of America’s great and useful rivers.
2
00:00:14.330 --> 00:00:18.350
The largest North American river to flow into the Pacific Ocean,
3
00:00:18.350 --> 00:00:22.920
it begins with a small mountain stream in the snow fields at the Canadian Rockies
4
00:00:22.920 --> 00:00:26.660
and enters the United States near the northeast corner of Washington.
Regardless of which web video player implementation you use, mobile browsers that don’t support the <track> tag will once again prevent the display of custom overlays -- including caption text -- when the video is viewed in full screen mode.
As you might expect, Apple supports specific captioning capabilities that can be directly embedded into the video stream. For video on demand (VOD) and live streaming via Apple HLS, you can use CEA-608 tracks to carry the captioning data. Streaming media servers such as Wowza Streaming Engine 4 can automatically merge caption data from SRT or WebVTT files with VOD streams delivered over HTTP streaming protocols such as Apple HLS, Adobe HDS, or MPEG-DASH.
You may also add subtitle or caption tracks to MPEG-4 video files before you store the content for delivery over standard HTTP range-request, such as Akamai NetStorage and Amazon S3. The native video player chrome of Apple iOS shows a caption button on the control bar whenever compatible tracks are loaded during playback. You may need to view the video in fullscreen mode to see the caption button enabled on the control bar.
(Note: Wowza Media Systems provides a tutorial on closed captions via HLS. The example shows how the captions button appears during VOD playback on iOS devices.)
Interactivity in Future Browsers
When Adobe Flash Player less-than-gracefully exited from the mobile space in late 2011, the onus was put on to HTML5 browser vendors to replace the interactive feature set of Flash Player-based experiences. It’s now 2014, and we’ve yet to see browser vendors rally around a true extended standard for HTML5 video.
From late 2012 to early 2013, most HTML5 desktop browsers implemented a fullscreen API in JavaScript, making rich layered web experiences possible by scripting interactive elements outside of the browser window chrome. Finally, we could overlay maps, graphics, and other interactive elements on top of full screen video.
Using the full screen API in JavaScript, you can create enhanced video experiences in the browser. For example, you can develop a web video player that has clickable hot spots that track a character’s face. Clickable video, as it’s often called, can more closely tie-in informative or promotional content to relevant subject matter as the viewer is watching the video. This example of enhanced interactivity, though, cries for standardization -- especially if it’s going to be viable for advertising networks. It isn’t rocket science to build a data specification that supports hot spots across specific x, y coordinates in time-based media. (In fact, much of the spec is already there: HTML has supported a <map> element for use with a “hot spot” <img> tag for well over a decade.)
Unfortunately, we haven’t seen support for the fullscreen API reach a critical mass of mobile devices. (Apple, you’re slowing the pace of progress here.) So, it remains impossible to create consistent interactive video landscapes across all desktop and mobile browsers.
Right now, mobile device manufacturers continue to push native apps for their platforms, as it’s in their financial interest to do so. For device manufacturers that control both a mobile operating system and a major desktop browser (think Apple and Google), building mobile browsers that support feature-rich web applications is not a high priority. Unless you are a major video outlet such as Netflix or Amazon Prime Instant Video with resources to build, maintain, and market a native app and a browser-based video experience, the cost to build enhanced interactive video players for both mobile and desktop platforms will be prohibitive.
For outlets with smaller budgets, I would recommend that you build video player enhancements with mobile browser restrictions in mind. Alternatively, if you are willing to forego cross-platform consistency in your viewing experience, you can push for a better desktop browser video experience for your audience, while recognizing the limitations of viewing on mobile devices.
This article appears in the June 2014 issue of Streaming Media magazine as "How to Create Interactive HTML5 Video."
Related Articles
Media source extensions, encrypted media extensions, and DASH offer new possibilities for player-based DRM and increase HTML5's appeal to video publishers.
17 Jul 2015
How are viewers supposed to scan a QR code on a speeding race car? When it comes to selling with online video, even big brands miss the obvious.
22 Aug 2014
We examine off-the-shelf video players MediaElement.js, Video.js, jPlayer, Flowplayer, and JW Player to show which of these five favorites is the best for any situation.
13 Jan 2014
Companies and Suppliers Mentioned