The HD Showdown: Codec Vendors Battle It Out for Supreme Quality

Microsoft
Microsoft scaled and deinterlaced with AviSynth, a free program available under a GNU General Public License (www.avisynth.org). AviSynth is unusual in that there is no user interface; instead, you create a script that identifies the source file and any filters and configuration parameters applied to the file. Then, you load the script into an AviSynth-compatible program such as Expression Encoder 2, which executes the script, receives the processed file from AviSynth, and then encodes the file. If your encoding program isn’t AviSynth-compatible, you can use a different tool, such as VirtualDub, discussed later, to render a raw-processed intermediate file for inputting into your encoding tool.

AviSynth has a huge library of third-party filters for preprocessing. Microsoft used DGDecode.dll to decode the file (http://neuron2.net/dgmpgdec/QuickStart.html), FFT3DFilter for noise reduction and sharpening (http://avisynth.org.ru/fft3dfilter/fft3dfilter.html), and VagueDenoiser.dll (http://avisynth.org.ru/vague/vaguedenoiser.html) for further noise reduction. Within Expression Encoder itself, Microsoft’s compressionist used the Windows Media 9 Advanced Profile and customized most of the tweaks available in Expression Encoder. Again, you can download the AviSynth script file and Expression Encoder 2 project file at www.doceo.com/HDcomps.html.

On2
On2’s compressionists also used a free tool, VirtualDub, for preprocessing in conjunction with third-party filters (www.virtualdub.org). Specifically, they used the MSU Denoiser (http://compression.ru/video/ denoising/index_en.html) and the Smart Deinterlace filter available at http://neuron2.net/smart/smart.html, saving out an uncompressed AVI file using the YV12 format.

Then, they compressed the file in On2 Flix Pro using the new VP6 codec announced at Streaming Media East in May. Looking at the Flix Pro encoding configuration, the only setting that jumped out at me was the keyframe setting of 9999, which was obviously intended to ensure keyframes only at scene changes.

As with Microsoft Expression Encoder, Flix Pro can perform many of the same functions that On2 performed in VirtualDub, including deinterlacing and noise reduction. This is a tacit admission that the third-party filters do a better job than those included in their own programs, which is both interesting and instructive.

H.264
While both Microsoft and On2 went the free route, I went totally Cadillac. For preprocessing, I used Adobe After Effects as the engine and Algolith’s AlgoSuite plug-in pack for deinterlacing, scaling, and noise reduction. The only problem with this approach is that Algolith has withdrawn AlgoSuite from the market, so unless you already have it, you can’t get it. Then, I encoded the file in Rhozet Carbon Coder, which costs $5,000 and uses the MainConcept H.264 codec.

I encoded with the High Profile, CABAC entropy encoding, and every "damn the encoding time, full quality ahead" option supported by Carbon Coder. As is my practice, I encoded with a keyframe setting of 300 and enabled Scene Change Detection so that Carbon Coder would insert keyframes at scene changes.

So, I used three encoders and three different preprocessing techniques and programs. I didn’t compare the output quality of the three different preprocessing approaches, though it’s an interesting issue I may follow up on in the future. At a high level, if you’re considering revamping your approach to preprocessing, AviSynth feels more convenient when you’re working with a compatible program that can input an AviSynth script; otherwise, you’ll have to use another compatible program, such as VirtualDub, to produce your intermediate file.

VirtualDub is a bit easier to use if you’re not into scripting, but it doesn’t import MOV files, a problem for many working in the cross-platform environment. Overall, the key lesson here is that if you’re trying to achieve maximum quality, you probably shouldn’t rely on the key functions such as scaling, deinterlacing, and noise reduction that are included in most programs, including Flix Pro and Expression Encoder 2.

Comparing the Output
After receiving the files from Microsoft and On2, I analyzed them in Inlet Semaphore, primarily to ensure that none of the encoded files had any dropped frames, and none did. For what it’s worth, both the VP6 and H.264 files exceeded a peak data rate of 1.38Mbps while the VC-1 file peaked just above 1.06. Given that all contenders had to meet a fixed overall data rate target, this should have provided no advantage, since higher data rates in the high-motion regions meant lower data rates in the other regions.