Streaming Live From the Battlefield: Military Video in 2019
For many military units, the tools of the trade for communications aren't much different from the ones you'd buy at your average big-box electronics store or mobile phone kiosk. In fact, based on interviews for this issue’s focus on law enforcement and military use of streaming, the benefits of the latest smartphone may be just as appealing to members of a special forces unit that needs to travel light and fast over a long distance—but still maintain consistent communications and even send video or audio streams or recording—as they are to, say, an average American teen.
Five years on from our “Video in the War Zone” article, which synopsized our understanding of the then-current state of military streaming, we’re taking another look at the types of use cases in which the armed forces—and the support teams around them—leverage streaming technologies to protect and defend national interests.
In a nod to those operatives and vendors who were willing to have off-the-record discussions for this article, the use cases have been generalized so as not to provide specific operational details. At the same time, though, the key takeaways are presented in a manner in which the streaming industry can take actionable steps to further the use of streaming in military applications while simultaneously benefiting enterprise and entertainment customers that need quality live streaming and subsequent content archiving.
There are several companies worth noting in this space that directly offer products and services to military providers, both in the U.S. and in its allied countries. They include Vitec— which has diverse products such as playing-card-sized, fanless, ruggedized encoders that rose to fame by being velcroed into Humvees to add an “extra set of eyes” for vehicle occupants as well as encrypted digital signage solutions— and Red5 Pro, whose peer-assisted WebRTC solutions provide ultra-low-latency streaming for drone and advanced analytics solutions.
Finally, before we dive into a few military use cases, readers should note that video from the battlefield is often the last step in delivering live streams. The vast majority of live streaming and subsequent archiving of content is used by the broader intelligence, surveillance, and reconnaissance (ISR) analyst community as decisions are made around the three C’s: command, control, and communications. Think of Jack Ryan at his desk for days on end, poring over video footage, before hopping that private jet to an exotic location for a hair-raising but relatively short field mission.
HEVC Gains Momentum
At the outset, let’s look at one of the major advancements that has shaped the use of live-streaming video in the military and ISR market vertical: advancements in video compression.
The hype around HEVC/H.265 has always been that it could deliver equivalent-quality streams—meaning the same content, bandwidth, and network topologies—at half the data rates of traditional AVC/H.264 streams. Yet, the overall uptake in HEVC has been slower than earlier adoptions of new video codecs (including historical uptake rates for AVC, MPEG-2, and even VP-9 codecs) partly because of the continued improvement in data networks and partly because of the widespread adoption of— and “it’s good enough” mentality around—AVC.
While the advent of IP-based over-the-air (OTA) broadcast—thanks to the ATSC 3.0 spec we covered in an article earlier this year—should result in an uptick in deployments for mass consumer usage of HEVC, the biggest gains in HEVC adoption in recent years have come from its use in closed-loop environments that need to balance the quality and deliverability of higher-resolution data streams.
The military market vertical is almost a poster child for HEVC adoption, and, indeed, it’s an area in which HEVC usage has thrived. One of the compelling factors, from a battlefield standpoint, is the benefit HEVC brings for higher-quality imagery over a short geographical distance. The use of microwave and other line-of-sight technologies to move extremely low-latency video imagery across a few square miles of a conflict hot spot means that the video compression won’t face any barriers in terms of network hops adding latency. And yet, given the fixed-bandwidth nature of these line-of-sight technologies, every bit counts when making real-time command and control decisions.
One interviewee noted that HEVC allows those using military theater video streaming to not have to decide between getting content faster or getting it at a higher quality, since HEVC inherently provides those direct benefits over AVC. And yet, those same benefits come at a cost when transmitting HEVC live video streams over congested or intermittent networks. Compared to AVC and its inherent ability to more easily recover from minimal packet loss, one drawback of HEVC is that it’s much more sensitive to even the slightest amount of packet loss.
One interviewee described the visual effects issues with HEVC as almost having a snowball effect, with HEVC’s use of more vector frames, but fewer I-frames, causing cascading results that can last for the length of a GOP or longer. And in war-zone situations, the loss of image clarity—or, at extremes, the almost complete loss of visual imagery across a multisecond GOP—could exacerbate a life-or-death situation.
Backchannel Negotiations
The nature of the problem noted above is not limited to military applications. In fact, as we’ve seen in deep dives into broadcast workflows, the use of HEVC as a contribution source from remote locations and even second-tier global sports venues has further shaped the need for FEC and error concealment techniques when the contribution feed encounters IP network congestion issues.
For most contribution feeds, the data pipe has enough extra overhead that an FEC solution can be deployed, with redundant packets being sent as a form of safety net in case network congestion occurs. In addition, a bi-directional link between the remote site and the broadcast headend or master control location allows solutions like SRT or Zixi to communicate back to the encoder as a feedback loop regarding the network topology.
This results in one of two typical scenarios: 1) Within a reasonable window of time, packets can be retransmitted to fill in for lost packets, or 2) the information about latencies and packet loss can be used to reduce subsequent encoding bandwidth and quality, at least until the network environment improves.
In military applications, however, there is often neither extra bandwidth nor a backchannel on which to negotiate retransmission of particular packets. Given the issues previously mentioned regarding HEVC, then, there’s a very real possibility that the live video stream in HEVC could actually look worse than an equivalent AVC live stream.
To combat this issue, various error-concealment approaches are being deployed. One such solution is a clever byproduct of content aware encoding and Context Aware Encoding (CAE) solutions, which analyze each pixel across a frame to decide where to best deploy the encoder’s computational budget. The need to balance out limited computational resources has only grown with the advent of HEVC, which typically requires double the computing power of AVC to effectively enhance the video quality while still maintaining bandwidth savings.
While most CAE solutions only look at a single frame (or perhaps a few frames before and after the current frame being encoded) to establish both a priority ranking of important pixels as well as an overall computational bit budget, they often dump this information when a single frame has been encoded. It makes sense to do so, on one level, since single-frame encoding is often required to deliver extremely low-latency delays (less than three frames or 100 milliseconds) when transmitting live streams. However, it’s always seemed to me, after working for a CAE solution company about 5 years ago, that the pixel-priority ranking and bit-budgeting data must have some additional value.
And, in fact, they do. One company showed me a solution in which it leverages that derived pixel-priority data at the viewing application level to act as a form of error concealment. In what it described as viewer-side error recovery and concealment, the viewing application uses historic frame data to temporarily replace artifacts with objects from prior frames. Because this occurs at the viewing application level, there’s no need for additional packets to be resent—in fact, there’s no need for a backchannel at all—and the solution, therefore, doesn’t add any more latency to the viewing experience or to the encoding process, since no look-ahead information from future frames is used in the error concealment. By memorizing the values of pixels in historic frames, interpolating the value of the pixel, and injecting an actual pixel into the playback viewer, the overall benefit is an elimination of the snowball effect of HEVC encoding mentioned previously.
Related Articles
From training to remote observations, the use of streaming video in the medical and pharmaceutical markets is growing.
08 Oct 2019
The online video revolution isn't just changing how we watch our favorite shows. Learn how video is informing and entertaining government and public sector employees.
14 Aug 2019
A new television standard, expected to debut in live broadcasts by 2020, brings the best of streaming—such as 4K video—to over-the-air delivery.
17 Jan 2019
For the armed forces, streaming is a matter of national security. Here's an exclusive look at how the military, from analysts to ground troops, is using streaming video.
31 Dec 2014
Companies and Suppliers Mentioned