Live Production for Mobile Viewing

Why Android?

Again, like smartphone growth, it's a game of sheer numbers: Of the 101 million smartphones per quarter mentioned above, the research firm comScore notes that 33% of US smartphone sales are based on Android OS. And, while Apple dominates the tablet market, with 78% of all global tablet traffic generated from iPads, the Android Honeycomb (OS 3.0+) operating system that the Motorola Xoom tablet uses seems to be making inroads.

The vast majority of those devices support the Flash Player plug-in and a number of those devices also support Adobe AIR (the latter is for games and stand-alone applications).

On the handset front, at the end of 2010, Android-based handset shipments surpassed the number of shipping iPhones devices.

Apple's iPhone only holds 25.2% of US smartphone market share, meaning that Flash-equipped smartphones now outnumber iOS handsets.

It's not necessarily true that all Android devices support Flash Player, nor is it true that all users would prefer to use Flash Player. In fact, all Android devices are supposed to have native streaming capability (via built-in applications such as YouTube and protocols such as RTSP) which should make live streaming to an Android device fairly straightforward.

Many Android OS devices also have the ability to play Flash-based video content and Flash Player is often included on many Android devices.

Based on those two facts-native streaming support and the ability to use Flash Player-we thought Android OS handsets and tablets would be a good test bed for live streaming.

What Did We Do?

We performed live streaming delivery tests, using RTMP and RTSP streaming protocols from a streaming server. We chose to use Flash Player for RTMP playback and the native Video app (an underlying player on most Android devices, not to be confused with the YouTube app) for RTSP streaming playback.

To perform tests that any reader could replicate, we chose to stream a publicly available file, via the Wowza Media Server 2.2.4. The clip, distributed under a Creative Commons license, is called "Sita Sings the Blues," an animation by Nina Paley.

To also address the variations in the types of live playback capabilities, we chose to stream both the 480p (SD) and 720p (HD) versions of the file, using Wowza Media Server 2.2.4 to push out RTSP and RTMP content in two separate tests.

Why choose a higher-resolution video to stream? The answer is partly due to our test environment being Wi-Fi centric and partly due to the fact that we wanted to tax all the devices with content that would require a heavy amount of processing (CPU or GPU) to decode, in order to determine which devices could withstand the heat, so to speak.

A third reason is hardware acceleration. Without hardware acceleration, it's difficult to even play today's standard-definition clips, let alone the high-definition content that most of the new tablets-and even a limited number of handsets-are geared toward playing.

The downside of hardware acceleration is that it uses up a bit more battery, so we wanted to balance performance against battery life. Boy, were we in for a surprise, regarding the preconceived notions of battery impact versus performance.

So What Did we Find?

In our tests, several patterns emerged. First, on the performance front, content delivered in RTMP (the Flash Player "native" protocol) was consistently delivered at a higher frame rate than content delivered by RTSP to the native Video app, even when both were served from the same Wowza Media Server.

None of the delivery solutions were perfect, by any means. Content played back as RTMP, RTSP and even HTTP dropped some number of frames for every clip played. That's to be expected in a mobile environment, and as mentioned above, our test clips were intentionally robust enough to tax the devices.

Yet we were surprised to find that-although the content was streaming out from the server at full frame rates and resolutions-several devices just couldn't muster the performance necessary to play back content with the native video app and RTSP protocol.

One of the primary reasons was the lack of hardware acceleration support for all devices in our test bed. For those without hardware acceleration support, playback was consistently 2-3 frames per second slower than the original content. For devices playing back RTMP-based delivery of the Sita clip, which was encoded at 23.98 (or 24) frames per second, the three devices without hardware support—Droid 2, Droid X, and Nexus S-failed to meet the full-motion decoding mark, falling off by numerous frames.

As noted in the following table, the Nexus S fell off by six frames per second for the Sita clip, while the baseline device, the Droid 2 as well as the Droid X, fell off almost ten frames per second for Sita playback. We also had an initial issue with the Motorola Xoom tablet, where its initial firmware and operating system (Android 3.0) failed to achieve 100% content playback framerates even with the dual-core NVIDIA Tegra processor. Once the firmware and OS update was applied, however, the included hardware acceleration support pushed the live stream playback of 720p content up to full frame rates.

Said another way, software-only decodes have a noticeably lower frame rate, averaging 21 frames per second (fps) versus 24 fps of hardware-assisted playback. In other words, for software-only decoding, video playback is noticeably less fluid but has slightly better battery life.

Slightly better is key, as we saw roughly a 15% difference in battery life over an average 3.5-hour span.

The question, then, is whether one would sacrifice a few minutes of battery life to be able to play back content at full-frame rate/full-resolution? This question may vex content creators for some time, as battery life seems to be given the most press in discussions around the topic of battery life versus performance.

For content creators doing a one-hour live stream, this may be solved by educating the customer about the minimal battery impact and recommending they use a plug-in such as Flash Player to view the live stream. For those content creators doing multihour live streams, the solution may revolve around sending out multiple stream types-something Wowza Media Server and Flash Media Server are capable of doing-so that customers who want full-motion have the option of RTMP, while customers who want to preserve battery life will choose the lesser-frame rate option via RTSP.

Speaking of RTSP, though, we found something else out that was quite interesting: Not every Android OS-based device could play back RTSP streams natively.

We were surprised to find, in some instances, that an Android device without a Flash Player plug-in could not play chosen content. For the flagship product from Google, the Nexus S, we found that RTSP-based video playback (and other standards-based media delivery on other Android test devices) could not be played with built-in applications and services, despite the requirement for the base Android OS to be able to play this content.

Just as it did in the initial test round, the current testing shows the Nexus S couldn't play some YouTube content (content that all other devices could play). In addition, the Nexus S showed an inability to play most RTSP content with the built-in video player. We couldn't get Nexus S to play the mobile version of our test YouTube clip in either the built-in YouTube app or the web browser, but we could get it to play content via Flash Player, showing just how consistent Flash Player was able to perform in a role as universal media player technology across every single Android device we tested.

The question we set out to answer-mainly that Flash Player appears to provide a performance benefit that outweighs the battery impact on any given handset or tablet-appears to be a resounding "yes" in Adobe's favor. In the end, we think it all boils down to compatibility.