Elemental Cloud: Streaming Media's Comprehensive Hands-On Review
The results? Unfortunately, the nature of cloud encoding requires more disclosures and descriptions than a red herring prospectus for an internet startup. By way of comparison, when comparing desktop encoders you’d load them up, throw your tests at them, record the encoding times, and compare the quality. With cloud encoders, you have to incorporate factors such as upload time, queue time, service plan, the maximum number of available nodes, their status (running or not), and rules for spinning up additional nodes.
Upload time is a universal issue for all encoders, and you would think that it would apply equally to all cloud encoders. However, Zencoder starts encoding during the upload, while Encoding.com and Elemental don’t (Encoding.com has a planned feature called Instant Video that does, but it wasn’t ready during my testing). If you’re uploading a lengthy file on a slow connection, Zencoder could be almost done encoding before the other two even get started. On the other hand, if you’re encoding relatively short and compact files already in the cloud, upload time would be fast and largely irrelevant.
The encoding times presented in Table 1 ignore upload times. Why? Because every user’s scenario will be different, making it impossible to derive a single set of tests that would be universally relevant.
Table 1. Performance comparisons
Queue time (or wait time, as Zencoder calls it) is the time that a file sits in the queue, awaiting an encoder. With Zencoder, this seldom strayed above 4 or 5 seconds. With Encoding.com, it periodically took 1 to 2 minutes, with up to 6 to 7 minutes reported on a late Sunday night when the service appeared to spin up new cloud nodes to encode the newly submitted files. In all cases, queue and wait times are included in the overall time, since these are unavoidable.
With Elemental Cloud, there is no concept of queue time because I had dedicated resources. For my tests, I encoded using the fastest, most expensive configuration. That is, I always had nodes up and spinning before submitting the jobs, so I never experienced the 6-to-7-minute node startup time. I took this approach because one of the key reasons to use Elemental is retain control of your encoding times and because, unlike the queue times for Zencoder and Encoding.com, this decision is totally under the user’s control.
In terms of service plan, I encoded using Twin Turbo mode for all Encoding.com jobs, which ensured the fastest possible encoding, though it was at an additional $2 per GB. Note that Zencoder doesn’t offer a similar mode or I would have used that as well.
Finally, I’ll discuss the tests, of which there were three: The first was a 52-minute file encoded to 11 separate outputs in an adaptive streaming group, output to MP4 files for real-time messaging protocol-based (RTMP) Flash Dynamic Streaming. The second was a 210-minute SD file encoded to a single SD output. The third test involved six files, averaging about 45 minutes in duration, encoded to the same 11 outputs as the first test. I ran each test multiple times irregularly over 2 days and present the results in Table 1, which contains the average of the last three results. For the record, the 6-to-7-minute wait times that I experienced with Encoding.com were not in the last three encodes and are not included.
As you can see, Elemental was slowest in all three tests; Encoding.com was fastest in the tests involving multiple presets; and Zencoder was fastest in encoding the long SD file. On a positive note, Elemental was the most reliable performer, with by far the least variation between the fastest and slowest of the three recorded times.
Elemental Cloud Quality
What about quality? Here I looked at three scenarios. The first was the 640x360x30 at 240Kbps variant of the adaptive group produced for the first and third tests. In practice, this would be one of the streams sent to mobile devices connecting via cellular.
The second scenario was 1280x720x30 at 800Kbps, which is an aggressive encoding configuration that might be used by a website attempting to send the lowest possible data rate for a 720p video. By comparison, at this resolution most websites, including ESPN and YouTube, are encoded at 2.5Mbps, which is more than three times the tested data rate.
The final test scenario is the most relevant: 640x360x30 at 1200Kbps, which is a generous, but not an overly excessive, configuration from a data rate perspective. For example, while CNN publishes at around 800Kbps at this resolution, ESPN publishes at 1.4Mbps.
You can see all test files and still image comparisons. If I had to pick a winner in the 640x360x30 at 240Kbps comparison, it would be Encoding.com, although the difference between the contenders is commercially irrelevant, which I define as “indistinguishable without comparative videos,” and which web viewers never have. You can see this in Figure 3, where the frames in the background window are a bit more distinct, as are the faces of the dancers to the right. But this is the comparative frame that showed the most differential; most others showed little or none.
Figure 3. Encoding.com won the 640x360x30 at 240Kbps comparison.
In the other two comparisons, there was even less of a differential. Overall, as I’ve said many times, though all vendors tout the absolute top quality (as you would expect), they are all mature products running highly optimized H.264 code, so it’s not surprising that the differences are minimal and not commercially relevant.
Where does this leave us? The Elemental Cloud offering will appeal most strongly to those currently running Elemental hardware who need a cloud-based overflow valve for their appliance-based encodes. Elemental will also appeal to those seeking the benefits of PaaS as compared to SaaS, which includes the ability to create a private cloud and reserve resources.
This article appears in the October/November 2013 issue of Streaming Media magazine as "Review: Elemental Cloud."
Related Articles
We've upgraded our testing methodology to accurately reflect what broadcasters and other high-volume video publishers ask of their encoding systems. Our first subjects: Elemental Server and Telestream Vantage Lightspeed Server.
05 Dec 2014
It's a mobile video world, and Elemental promises to ease video complexity, add time-shifting services, and reduce costs with Delta.
12 Sep 2014
Brightcove customers should look to this new service for inexpensive real-time event transcoding, but several rough edges mar the experience.
12 Dec 2013
While it delivered impressive results in our tests, Encoding.com isn't perfect: Read our review before paying extra for Turbo mode.
19 Aug 2013
Digging into the competing cloud encoding solutions shows that they're far from similar, making comparisons difficult.
16 Aug 2013
If you need a tool to accelerate H.264 encoding, Elemental Server should be on the top of your list.
13 Apr 2012