4K Webcast Workflow: Acquisition to Delivery

4K Signal Processing

Now that you have your 4K/UHD signal flowing over either HDMI, HD-SDI, or fiber, you need to do some form of signal processing. This might be a connection to a video switcher when you want to mix multiple 4K video signals, a hardware encoder, video capture card, and/or a 4K recorder.

Many 4K video switchers lack internal up/down/cross converters, so you need to make sure that all your signal types match; others may allow for full or limited scaling ability depending on your switcher model and connection type. This consideration is very relevant on current HD models too, with some models like my new Roland V-60HD that doesn’t support 30P but does support 60P.

Interestingly, you can also mix interlaced and progressive signals from HDMI inputs because the V-60HD has internal deinterlacers. Initially, I thought that the 30P limitation was going to be an issue as I like to webcast in 30P, but it turns out that I just needed to change the output on my Sony video cameras to output 30P signals over 60P, with a pull-down, which was a menu option that I’d previously overlooked.

I mention this workflow as a segue to discussing the delivery frame rate because, even if I want to film and record in 60P for later on-demand viewing, 30P is a much more manageable frame rate for live streaming when you consider all of the signal processing that happens in the cloud and ultimately on your viewers’ devices.

The new Blackmagic Design ATEM Television Studio Pro 4K (Figure 3, below) is the first ATEM switcher that has internal scalers on each input. It also supports Fairlight audio plugins for advanced audio controls.

Figure 3. The Blackmagic Design ATEM Television Studio Pro 4K

After your signal leaves your 4K video switcher (or if a video switcher isn’t a part of your 4K workflow), you need a video capture card or hardware encoder to encode and then send your video signal to your cloud-based webcast service. HD hardware like the Teradek Cube and VidiU, Matrox Monarch, AJA HELO, and many others use the H.264 codec and RTMP protocols to encode and webcast video.

H.264 can also be used to encode 4K content, but it isn’t as efficient a codec as newer ones like HEVC and AV1. The quick-and-dirty explanation of interframe compression is that these three codecs are a group of picture codecs that create an I-frame at set intervals (15 frames is typical) and then B- and P-frames that reference the changes from this I-frame. HEVC and AV1 take the efficiency one step higher by being more content-aware, allocating more bits where there is more motion, and grouping similar macroblocks together in areas that don’t have much change, like a flat wall.

The additional efficiency afforded by HEVC and AV1 translates into smaller file sizes, which is important for both webcast producers who need to upload video over the internet and for end-viewers who need to download this 4K content. I actually did my first tests with the HEVC codec on HD footage for the sole purpose of bypassing the 5GB weekly limit on my client’s Vimeo account (a limit I don’t have on my own account as a Vimeo Pro member).

The initial HD encode in H.264 was 16GB, and my clients took it upon themselves to re-encode the footage to fit within their weekly limit. They did such a poor job of selecting a codec, resolution, and bitrate that it was painful to watch my beautiful three-camera, live-switch footage of this professional ballet performance of Romeo and Juliet, and I had to intervene. Upgrading their hosting package was one option I quickly considered asking them to do, but because Vimeo supports HEVC video uploads, I figured it would be a great test case to experience the efficiencies of the HEVC codec first-hand. The experiment was a success, but it did take more than 20 hours to export my 2 hours of 1080/60P HD footage on my Core i7-3930K hexacore video editing system with a 980Ti graphics card and 64GB of RAM.

A similar HD encode using the H.264 codec, thanks to Adobe Premiere Pro’s hardware acceleration with this codec, would have been faster than real-time. While it might sound like a great idea to move all your workflows to HEVC for smaller file sizes, there are serious workflow considerations involved and you may require additional specialized hardware if you plan on working with the HEVC or any newer advanced codec. Companies have been quick to fill this need, and you can find HEVC as an option on the Teradek Cube 755/855 or the BoxCaster Pro (Figure 4, below). BoxCaster also supports HEVC on its webcast CDN, which is powered by Amazon AWS.

Figure 4. BoxCaster Pro

On the software side, vMix does support the both H.264 and HEVC codecs for 4K recordings, but only H.264 for webcast. But before you can start working with 4K content, you first need to pair your webcast computer with a 4K capture card. Desktop and workstation webcast encoders can install PCIe 4K capture cards like the AJA KONA 4 or KONA 3G/Quad, Blackmagic Mini Recorder 4K or DeckLink 4K Extreme, and Magewell AIO 4K Plus.

If you don’t have a desktop or workstation system and you’re working from a laptop, you might still be able to use a PCIe 4K capture card, but you will need to install it in a Thunderbolt enclosure, like the Sonnet Echo Express. Just be careful to consider that the Thunderbolt 2 connector is the same as a mini DisplayPort connector, while the newer Thunderbolt 3 connector is same as the USB-C connector.

Laptop users can also employ 4K USB capture cards like the Magewell USB Capture 4K Plus or the Epiphan AV.io 4K. These USB models have an added benefit: They can also scale the 4K input to a different resolution and frame rate if you don’t want to webcast a 4K workflow, or you want to record the signal in 4K on a recorder like the Atomos Ninja Flame, Shogun Flame, or Ninja Inferno, or Blackmagic Video Assist 4K.

It goes without saying that processing 4K signals in any workflow requires a fast system. Last year, the Intel Core i7-7700K was a very common quad-core processor for webcast laptops, but this year the sweet spot is going to be the hexacore Intel i9-8950HK mobile processor that Intel claims will deliver a 59% boost in speed for 4K video editing.

Before you rush out and buy yourself the newest HEVC hardware, keep in mind that HEVC is becoming more accepted by on-demand video hosting companies like Vimeo and YouTube, but adoption by webcast companies is much more limited. At the time of this writing, pre-NAB 2018, I am aware of only one workflow with Amazon EC2 and Wowza that can support HEVC, but I have not personally had the opportunity to test this out.

A more important long-term consideration against rushing out to cobble together a reliable HEVC workflow is that the Alliance for Open Media has just frozen the AV1 bitstream and released an unoptimized software encoder and decoder. AV1 is a competitor of HEVC with two distinct advantages. The first advantage is 30–40% better quality, and the second is that AV1 is a royalty-free codec, whereas HEVC requires licencing. If you can hang on another year or two, AV1 hardware implementations will likely be released or announced at NAB 2019 and AV1 hardware around NAB 2020.

The 4K live-streaming workflows are still a work in progress. Most likely, if you need to deliver a 4K live stream today, you might want to start with a 2160/30P H.264 stream at 15–30Mbps on a webcast service like IBM Cloud Video, Wowza Streaming Cloud, YouTube Live, or DaCast. It remains to be seen whether HEVC or AV1 will be the next dominant codec and which hardware and software solutions will be released to support either or both of these two next-generation codecs for resolutions of 4K and beyond. Ultimately, wider support of either of these codecs is needed from webcast providers before 4K live streaming at lower bitrates and higher quality becomes a reality.