Review: Anystream Agility
How and What I Tested
Now that you understand the basic workflow, let’s discuss the testing process. My primary focus was output quality, which is a combination of deinterlacing quality, codec feature support, and pure encoding quality. I also ran some high-volume (at least for me) encodes to test throughput and stability.
As part of this latter test, Anystream set up a two computer rendering farm in my office, which I used to produce the test files. This process exposed me to Anystream’s excellent support infrastructure and personnel, which is critical to the success of any such enterprise product and which I’ll briefly describe later.
Deinterlacing Quality
It sounds pedantic—particularly regarding a system that costs in the low five figures—but deinterlacing quality is key to overall streaming quality, a given that affects all interlaced files submitted for encoding. Agility offers two deinterlacing options: single field-only, where the top field will be interpolated, which is recommended for fast-motion video, and two fields-blend, which combines both fields into a single frame and is recommended for slow-moving or stationary video. Under the hood, the encoder monitors the source media and adaptively changes deinterlacing techniques to optimize quality.
To test Agility’s deinterlacing capabilities, I used a 1-minute DV file containing multiple scenes with diagonal lines and sharp edges that separate (in this PC world) the adults from the children, at least when it comes to deinterlacing quality. The best deinterlacing quality I’ve seen in a batch-encoding tool is from Telestream Episode, and Agility is definitely a step behind Episode.
To be fair, Agility ranked better than Sorenson Squeeze and about the same as Adobe Premiere Pro, Adobe After Effects, and Apple Compressor in "Better" but not "Best" deinterlacing mode, but there’s still room for improvement.
Codec Testing
Now on to encoding quality. For the record, to test encoding quality, I use a predeinterlaced, prescaled file to take deinterlacing and scaling quality out of the equation. I’ve used the same file for about 2 years and have comparative files in most batch encoding tools. To test a range of different scene types, the file contains 42 discrete scenes between 5 and 10 seconds long, with varying ranges of motion and detail.
My target encoding parameters are aggressive: 640x480 resolution at 30 fps, with a video data rate of 468Kbps and audio data rate of 32Kbps. I produce using 2-pass CBR for Windows Media Video and H.264, and 2-pass VBR, when available, for Flash. I vary the parameters for Flash because 2-pass VBR output is one of the key differentiators between products such as Adobe’s suite of Flash encoding tools, which only support 1-pass encoding, and tools such as On2 Flix Pro and On2’s licensees, which can output 2-pass.
To assess quality, I compared both single frame grabs from the compressed files and quality during real-time playback. I used Inlet Technologies’ Semaphore product to analyze various aspects of the encoded files, including data rate variability, which can affect delivery smoothness; whether the encoder inserted key frames at scene changes, which obviously affects overall quality; and whether the encoder dropped any frames during encoding. I also reviewed the available encoding controls for completeness, and, in the course of producing my comparative file, noted how effectively the Agility encoder hits the target data rate.
Windows Media Video
The Windows Media Controls are competent, but Agility is not yet compatible with SDK 11, and produces files using the older SDK 10, in large part (according to Anystream) because the SDK 11 is not yet qualified for Windows Server 2003. In any event, sometime in early 2008 Anystream will release an update that supports SDK 11, but only via the Power Toy or manual registry changes, not through controls enabling custom "tweaks."
To produce the Windows Media file to my target specs, I had to bump the target data rate by about 10%, from 468Kbps to 510Kbps. Agility users should check the final bitrate of compressed WMV files, since there’s a chance that your data rate is 90% or so of your target bitrate, which could reduce the quality of your video output.