Roundup Review: Looking at Sub-$2,000 Hardware H.264 Encoders
Each H.264 encoder was set to use 1280x720p 29.97 fps at various bitrates: 5Mbps, 2.5Mbps, 1Mbps, and 500Kbps. The output stream from the encoders was pushed via RTMP to the local streaming server, where it was recorded. The recorded MP4 files were then converted to ProRes HQ 720p 29.97 fps, and the ProRes file played from the Odyssey7Q+ was also downsampled to 720p 29.97 fps. With each bitrate test, I trimmed the in and out points of each of the recordings to match a trimmed version from the Odyssey7Q+.
This painstaking process typically took 3 to 4 hours to complete for each bitrate tested. Moscow University’s Video Quality Measurement Tool (VQMT), was used to compare the downsampled 720p high-quality source to the 720p renditions created by each encoder. (As a side note, you can read more about VQMT in various articles written by Jan Ozer on streamingmedia.com.
As a relative baseline metric, a GOP-restricted x264 encoded version of the source file was compressed for each bitrate tested and posted in the results.
Due to space constraints, samples of the encoded output are not included in the print version of this article. You can find screen grabs and MP4 recordings of the samples used at videorx.com/h264-hardware-test.
Slow- to Average-Motion-Content Results
The first sample tested was a clip titled “Honey Bees 96fps in 4K.” As described earlier, this clip was downsampled to 1080p 29.97 fps and recorded from Telestream Switch to the Convergent Design Odyssey7Q+. The entire clip was played out over SDI to each of the encoders in 5Mbps, 2.5Mbps, 1Mbps, and 500Kbps renditions.
Each encoder used a consistent 60-frame GOP/keyframe interval. The video bitrate was set to 5000Kbps (or 5MBps) in all encoders. The Teradek Cube’s automatic rate control (default) produced even lower bitrate values than expected, and as such, I used a manual rate control which was set to the same bitrate value as the average bitrate setting.
5MBPS
For most 720p 29.97 fps content, a 5Mbps video bitrate will be more than adequate for a high-quality live stream. As expected, the Video Quality Metric (VQM) scores (Chart A) for such high bitrates on hardware encoders was closest to the VQM of a non-live x264 encoded version of the original source file. The HELO produced the best VQM score, with the Talon G2 as a close second. The Teradek Cube 655, though, should be recognized for a decent VQM score despite the actual average bitrate being almost 2Mbps lower than specified. For 3Mbps, the Cube 655’s score should rank it much higher. Having said that, the Cube should have been better at producing a bitrate that was closer to the requested 5Mbps.
2.5MBPS
Things start to get more interesting as we get closer to a real-world bitrate for 720p. In production webcasts, I typically use a 2.5Mbps video bitrate for talking-head- and PowerPoint-style presentations managed on my Roland V-1SDI mixer. In this round of tests (Chart B), we can see that the x264 comparison video is not quite as good as the 5Mbps version and the VQM scores are all close to each other. Again, the Teradek Cube 655 had the lowest VQM score, but with an actual bitrate that was barely 60 percent of the requested bitrate. If we take the Teradek Cube 655’s score from the 5Mbps round, which produced a bitrate of 3Mbps, closer to this test’s bitrate, we can see that the Cube 655 had comparable results to the others in this round.
1MBPS
With this round of tests, similar trends reappeared (Chart C). The Teradek Cube 655’s actual bitrate was significantly lower than the desired preset. While its score was lower quality than the other encoders in this batch, its VQM score does well against the actual bitrates for the 500Kbps round discussed next. The other three encoders had scores relatively close to each other when adjusted for actual bitrate.
500KBPS
Using a bitrate of 500Kbps on a 720p resolution is not likely to produce high-quality output, but it can help differentiate how well an encoder can reduce complex imagery. As shown in Chart D, the AJA HELO and the Teradek Cube 655 seemed to compensate for low-bitrate scenarios by dropping frames. Because frames are lost in the output recording, the VQM score will be worse for such encoders even if the frames that are retained are higher quality. The Matrox Monarch HDX and Osprey Talon G2 retained more lower-quality frames.
FAST-MOTION-CONTENT RESULTS
For this category of tests, I used the only sports action clip available on 4ksamples.com and used the same encoding settings as the previous test. The only exception was the compression settings for the Teradek Cube 655. Because the slow- to average-motion content consistently produced lower actual bitrates for the Cube 655 compared to other encoders, I thought I would try restricting the default wide range of quantizers, or QP values. I was able to get actual bitrates that were higher, but still not as close to the desired bitrate preset as I was with other encoders.
5MBPS
In the 5Mbps tests (Chart E), the Osprey Talon G2 and the AJA HELO did best with high-motion content. The scores are close to one another, and the actual bitrates were nearly identical across the board. The average viewer would not easily distinguish any significant differences among these clips.
2.5MBPS
In this batch of tests (Chart F), the same trend that only started to surface with 1Mbps presets and slow- to average-motion content appear even in the 2.5Mbps category. The Teradek Cube 655 and the AJA HELO start to drop frames to retain quality, at the expense of smoother motion. Only the Osprey Talon G2 had an average video bitrate that was within 100Kbps of the desired bitrate. With this round, we start to see a quality difference between the lowest score (better) of the Osprey Talon G2 and the highest score (worse) of the AJA HELO.
1MBPS
As the encoders are pushed to encode over lower bitrates at 720p, the differences become more pronounced (Chart G). Most viewers would consider the output from all of the encoders unacceptable. The Matrox Monarch HDX produced a better score than the reference x264 file, but the hardware encoder used 30 percent more bitrate in the process. There’s a more significant difference visually now between the top two performers and the bottom two.
Again, the Teradek Cube 655 and AJA HELO dropped frames to retain quality, and for a VQMT comparison, that will produce higher scores (worse). Because the encoder will hold on any given frame where there was a drop, VQMT will have a completely different frame to compare from the original source clip compared to the encoded output.
500KBPS
Similar trends continue as the bitrate is pushed below a threshold necessary for reasonable quality at 720p (Chart H). The Teradek Cube 655 and AJA HELO drop frames in an effort to keep quality on non-dropped frames, while the Osprey Talon G2 and the Matrox Monarch HDX reduce the overall quality of frames without dropping as many frames. Again, the Matrox Monarch HDX produced a better score than the reference x264 file, but with a higher bitrate than specified.
Conclusions
When I first embarked on my journey to compare these four encoders, I was expecting to see greater differences between the units than I observed. The Teradek Cube 655 and the AJA HELO offer much finer control over the encoding parameters than the Matrox Monarch HDX and the Osprey Talon G2. For portability, the Teradek Cube 655 is the easiest to mount and configure with a camera or mixer, but it requires more attention to the live output to fine-tune the bitrate to match your intended target.
As a first-generation product, I was pleasantly surprised with the AJA HELO. Its web interface was the fastest to apply new settings—a feature highly desirable while on location. The quality from the AJA HELO and the Teradek Cube 655 was best when the bitrates were high enough to accommodate the pixel dimensions and frame rate of the 720p output. The Matrox Monarch HDX and the Osprey Talon G2 did remarkably well at retaining smooth motion across very constrained bitrates. The Matrox Monarch HDX had a tendency to use higher-than-specified bitrates under such demands.
[This article appears in the October 2017 issue of Streaming Media Magazine as "Roundup: Sub-$2,000 Hardware H.264 Encoders."]
Related Articles
A range of factors—from video formats needed to encoder portability—will impact the decision-making process for a live streaming solution.
04 Mar 2015
In the market for a VOD encoder? Learn the important product categories, the top choices in each, and the right time to upgrade to a more powerful encoder.
03 Mar 2015
A wealth of options makes for tough choices. Here are the questions you need to ask to make sure you choose wisely.
03 Apr 2014
Companies and Suppliers Mentioned