Streaming Live From the Battlefield: Military Video in 2019
The interpolation approach provides a prettier viewing experience, but it also raises an interesting question, one that I asked during the interview with the company: Will the injection of playback pixels cause an analyst to make a false real-time assumption? Apparently, this issue has been raised before, and the company said it provides government agencies with a demonstration of a side-by-side viewer, with one side including the interpolated and injected pixels and the other side showing the raw stream, which, it’s worth noting, can be from any HEVC encoder, not just a single company’s encoder. The demonstration shows that artifacts, which would normally render a critical area completely unrecognizable, are masked in a way that helps with the decision-making process.
Metadata for the (Decisioning) Win
Speaking of the decisioning process, there’s another key factor in this real-time command and control workflow: metadata.
From an ISR perspective, the ability to have a team of analysts monitor on-the-ground live feeds, in addition to feeds from support aircraft, is critical to making a split-second decision. But equally important is the metadata that accompanies the streams in the form of key-length-value (KLV)-transmitted information.
As such, informed decisioning requires that not only do multiple live streams need to be synchronized, but that the KLV metadata accompanying the live streams must also be delivered at the same time as the visual imagery. What’s interesting is that KLV is actually a Society of Motion Picture and Television Engineers (SMPTE) data-encoding standard (336M), which was traditionally used to embed information such as broadcast flags in OTA video feeds.
Because KLV triplets are fairly versatile— identifying not only the data, but also its length and value—the KLV standard is heavily used in ISR for video metadata such as geospatial and other critical intelligence detail that’s transmitted from the war theater to analyst teams and centralized command and control locations.
With the KLV data, it’s possible to provide visualizations of the location of the surveillance-gathering aircraft—be it a manned or unmanned aerial vehicle (UAV) or even the intended target—alongside the synchronized video streams. Given this combined use of metadata and visualization, consistent latencies across the primary video streams are key.
The Streamer in Your Pocket
This brings us to the use of consumer mobile devices, or what the military refers to as commercial off-the-shelf (COTS) technologies. At the outset of the article, I mentioned that the same phone you buy at an electronics store or your local AT&T Wireless, T-Mobile, or Verizon store might also be in use in a military application.
Companies working in the military streaming space can spend years getting their products to a military specification (mil spec) level, since they need to be ruggedized and handle shocks such as those from large-caliber-gun fire and being dropped during combat. Meeting mil spec ruggedization requires beefing up the frames of equipment. This adds extra weight, which is antithetical to elite military unit missions that might require on-the-ground foot travel—without accompanying support vehicles or even any vehicle at all—in remote or treacherous terrains.
The use of COTS devices by elite military units has its benefits: Devices can blend in with civilian devices, can be much more easily repaired in the field, and are lightweight enough to be strapped to a uniform without adding bulk or limiting operator movement. And, according to interviews conducted for this article, there’s a general feeling that consumer technologies have continued to outpace military video technologies.
But the use of COTS devices to augment mil spec equipment also comes with a challenge around security. As such, 5 years ago, the U.S. Army Communications Electronics Research Development and Engineering Center’s Space and Terrestrial Communications Directorate put out a request for information (RFI) to various cellular network equipment manufacturers to adapt “commercial cellular/wireless technology to provide soldiers at the tactical edge with extended secure wireless communications across the battlefield.”
Interviewees noted that, in 2019, the use of secure apps on consumer phones continues to increase. Typically, these applications are found on Android OS devices, rather than iOS devices, partly due to the easier ability to sideload specialized applications onto Android OS devices. Elite units that need to travel light and go over long distances use these COTS devices to send back live audio and video streams, as well as KLV metadata.
Not only are the apps secured with standard encryption protocols, such as SSLs, but the transmitted content is also secured. According to at least one interviewee, secure apps are architected without any dependency on Apple or Google players, and live-streamed feeds are scrambled prior to transmission to further ensure that audio, video, or metadata content— if intercepted mid-transmission—cannot be easily decrypted to gain a tactical advantage.
The scrambling and encryption add a slight amount of latency, but it appears to be in the single-digit millisecond range, so, well below the threshold of a single frame of video. In addition, a number of these mobile apps are set up to be used on secure Wi-Fi networks, which means that streaming content could be tunneled across public fixed-line data networks as well as across cellular data networks.
Storing Intelligence
The final topic we’ll discuss briefly here is archiving media content. According to interviewees, today’s military and ISR deployments rely heavily on HEVC for anything that needs to be archived. This includes archiving of AVC-native streams, although this is dependent on the specific codecs for both AVC and HEVC since not every company makes codecs that comply to both standards.
Because HEVC is capable of 30%–50% bandwidth savings for equivalent-quality video in both live streaming and archived recordings, I asked whether large-scale repositories of AVC-based content are routinely re-encoded into HEVC for long-term storage. I was told that it’s abnormal for large-scale archiving clients to do a post-archiving process to convert stored content into HEVC.
There are three primary reasons for this. First, because of the computational requirements to encode in HEVC, it’s less likely that idle computational processing is available. While this may change with the advent of HEVC-based ASICs, and perhaps even in the interim step of FPGA or GPU parallelized transcoding, most HEVC transcoding today is performed by CPUs, which requires longer transcoding times.
Second, it was noted that, while broadcasters may often do quality-control checks on transcoded content, it’s rare to see the enterprise, education, or ISR community do comparative assessments, especially given the sheer size of visual record archives.
Finally, there are still some fears around concatenation errors when it comes to converting between codecs. One interviewee even noted that those fears—perhaps stemming from the old days of video degradation when making copies of analog videos—are hard enough to combat that it’s led companies to create duplicates of content they’re transcoding. In other words, the large-scale archiving process to transcode previously recorded content to lower-data-rate versions actually uses more storage, since the retention of the original version—deemed to be “better” quality—adds to the already large storage requirement.
We’d like to offer special thanks to the interviewees who were willing to speak on the phone or in person, providing anecdotal details and demonstrations on which this article is based. Streaming Media magazine readers who are interested in learning more about the use of streaming in military environments can search for streaming companies that have governmental (federal, state, and local) sales forces or that reference ISR or military solutions.
[This article appears in the October 2019 issue of Streaming Media Magazine as "Streaming From the Battlefield."]
Related Articles
From training to remote observations, the use of streaming video in the medical and pharmaceutical markets is growing.
08 Oct 2019
The online video revolution isn't just changing how we watch our favorite shows. Learn how video is informing and entertaining government and public sector employees.
14 Aug 2019
A new television standard, expected to debut in live broadcasts by 2020, brings the best of streaming—such as 4K video—to over-the-air delivery.
17 Jan 2019
For the armed forces, streaming is a matter of national security. Here's an exclusive look at how the military, from analysts to ground troops, is using streaming video.
31 Dec 2014
Companies and Suppliers Mentioned