Repurposing TV for Streaming

Downstream commercial insertion into a DTV transmission is made possible by Digital Program Insertion (DPI) splice points that are added to MPEG-2 transport stream during creation of the emission multiplex. Downstream program switching can then be done in the compressed domain.

Audio, video, timing, assembly instructions, and other information contained in the 188-byte packets are multiplexed into a Transport Steam. The MPEG-2 transport stream may have a data rate of up to 19.39Mb/s.

Figure 2 presents an overview of the program signal flow for the assembly of a digital television program.

Figure 2. Assembly phase for TV operations

Audio embedders and de-embedders are frequently used to distribute audio channels as part of a serial digital signal, referred to as SDI, in a broadcast facility. This technique eliminates the need for dedicated cables for multichannel audio distribution since video and audio are both present in the SDI signal.

Variable delays introduced by the separate signal compression paths of audio and video must be accounted for. Conversion systems and PCR and MCR switchers, as well as many other pieces of audio and video equipment, have buffering capabilities that can be used to restore "lip sync" timing for content leaving a facility. But this doesn’t insure that a DTV receiver will maintain "lip sync" when the audio and video are decoded and presented. Unfortunately, presentation time stamps have not been implemented by DTV receiver manufacturers that reliably maintain audio and video synchronization. As a result, DTV programs are often presented with annoying audio/video timing skews.

Workflow StrategyThrowing content over the wall to the Website design department may get TV content online, but will result in less than optimum presentation quality and esthetics as well as a duplication of production effort. Intelligent repurposing of TV content for streaming distribution has created a need for converged and integrated workflows.

Even though each reception platform will require a different program assembly technique to present audio, video, graphics and data, there are common production processes that can be leveraged to improve production efficiency. In addition, departments that were traditionally not part of television broadcast operations are now integral to TV content repurposing and distribution process and must be considered when developing a repurposing workflow.

Format Conversion
Format conversion of audio and video is at the heart of any repurposing of TV content. Essentially this is a two-step process that can be categorized as presentation formatting and compression transcoding.

Presentation reformatting (or conversion) is any transformation of audio or video attributes to content to produce a format that is compatible with a reception device—for example, 1080i to VGA video or 5.1 surround sound to mono.

Squeezing content into a limited bandwidth delivery channel requires compression. The codec must produce a bit stream at an appropriate bitrate, for the data capacity of the delivery channel. For example, MPEG-2 video compression is the foundation of DTV transmission in the U.S.; an SD signal will have a compressed bit rate of around 3Mbps. However, web and cell distribution channels require significantly lower bitrates, so an advanced codec (AVC or VC-1 for example) is used to attain a usable bitrate.

Any change in presentation or compression format will introduce undesirable artifacts. The fundamental processing tenet is that no format conversion is a good conversion. Hence, the number of conversions should be kept to a minimum during the repurposing workflow.

In addition, presentation problems may arise when using TV graphics on new media reception devices. Video scaling issues are of particular importance. For example, will a TV graphic be legible on a cell phone? When repurposing TV content on a PC, would it be best to display the lower-third graphic outside of the media played window, elsewhere on the web page? As a result graphics and data elements should be designed and integrated into the content stream in reception platform presentation-friendly ways.

Platform-Specific Processes
Examining a hypothetical multiplatform production workflow will shed light on the television content repurposing process. First, content presentation formats for each distribution channel need to be established. For example, the house and transmission DTV format will be 1080i. Content repurposed for the Internet will be produced and delivered as quarter-screen (QVGA) 320x240 at 60 progressive fps; this pixel grid format is a good quality versus data rate compromise and maintains acceptable video quality in variable size windows. Cell phone content will use the QCIF format 176x144 at 15 fps. Audio will be full 5.1 surround for DTV, and PCM (2.0) stereo the Internet and PCM (1.0) mono for cell phones.

Choice of presentation formats must take into account that content will be compressed and then assembled not just for each delivery platform, but also for each target decoder. In this example, video and audio will be compressed for the Internet using AVC/H.264 and AAC, while the cell phone stream will use WMV and MP3 compression. Since Flash and QuickTime are not interoperable, the compressed steams will be loaded by "pages" coded to support a particular media player on the target reception device.

Compression Codecs & Multiple Source Conversion
The repurposing workflow and infrastructure implementation goal is to minimize artifacts produced by format conversions—therefore, a fundamental design philosophy is to keep format conversions to a minimum. Workflow analysis will help decide where format conversions should occur to avoid unnecessary conversions.

Figure 3 shows three different approaches to compressing a single content source for distribution to each of the three screen device types. In all three cases the details of the audio and video processing chain are not important, just the sequence of conversions. Note that the scaler processing box applies to video; for audio the analogous processing step would be a downmix. TV content generally skips the scaling and downmix processing step.

Figure 3 A, B, and C. Three content conversion workflow scenarios

In Figure 3A, a brute force method is employed. One uncompressed TV source is distributed to three encoding devices. As mentioned earlier, audio and video are processed separately. Each device produces the appropriate bitrate in the desired compression format.

A cascaded transcoding approach is illustrated in Figure 3B. The baseband source is compressed for DTV transmission. The compressed source is fed to a scaler, then a transcoder and encoded at the target bitrate, compression codec, or both. (Note: transcoding denotes use of a different compression encoder; for example converting MPEG-2 DTV video to AVC/MPEG-4 for web distribution.) The process can be repeated as often as required.

The preferred solution, shown in Figure 3C, uses a single, multicodec compression encoder to simultaneously produce all three bit streams. Broadcast equipment manufacturers are now producing integrated, configurable devices with multiple format and compression encoding capabilities. As a result, high quality equipment that directly supports the integrated conversion functionality required in a three-screen universe is becoming increasingly available.

Template-Based Graphics Creation
Graphics playout systems often use template-based methodologies that assemble the finished graphic by populating fields in a display template with data on the fly, as it goes to air. Information that populates a template is contained in a database or provided by real-time data feeds (sports scoring or news services). Templates are loaded into playout servers prior to air.Figure 4 depicts the template based graphics production and playout process.
Figure 4. Graphics template production workflow process

Assembly TechniquesAt this point, the converged workflow diverges. Not only will it diverge based on delivery channel, but it will also diverge based on target device. This is due to the fact that different media players are resident on different reception devices. Some can decode and present content in a variety of formats transparently while others can only process content in a single format.

Figure 5 illustrates how the TV PCR and MCR signal flow can be expanded to include web and cell streaming.

Figure 5. Live event simulcast with automated commercial insertion

In the PCR, the production switcher and audio mixer produce a television program feed that includes graphics and a full 5.1 surround sound audio mix. This feed is sent to the MCR. The PCR also produces a clean feed (no graphics) signal and 2.0 stereo and 1.0 mono audio mixes for web and cell phone delivery respectively.