The Right Profile

Table 1 shows a summary of the results. Overall, since lower scores are better, Flash edged out Apple’s H.264 for top quality, 1.39 to 1.72. However, as you can see, Flash scored 106 total points to 100 for WMV-9 AP and H.264. It’s nice being first and all, but with this small a difference, you have to wonder if anyone will notice. As it turns out, analyzing each category separately put a different spin on the issue. So let’s jump in. Business
This segment is comprised of six primarily talking-head clips. The Windows Media Video Advanced Profile swept all components of this category, courtesy of very sharp images and rock-solid backgrounds.

As you can see by the scoring, H.264 had problems with both image quality and temporal quality, specifically background noise, which is illustrated by Figure 6 (p. 86), an image that I’ve darkened to highlight the artifacts in the back wall. The Apple image in the bottom right shows significant bleeding and noticeable background blockiness, while the Flash frame, on the bottom left, showed significant banding, or grouping, of pixels into noticeable bands. In comparison, the Advanced Profile video on the upper right shows the smoothest wall of all.

I’ve included the raw points score and percentage difference to add some context to the raw scores. In my view, a points difference of 10% or less is probably unnoticeable in the absence of side-by-side comparisons, so it is commercially irrelevant. Between 11 and 20%, viewers may suspect that quality isn’t up to par; and beyond 20%, most will notice and start to wonder what’s up.

Accordingly, I would ignore the 6% difference between Flash and WMV-AP, particularly because still-frame quality was identical and because, by choosing an appropriate background, you can minimize these types of background artifacts. In this regard, avoid backgrounds with wide-open spaces on a bright, reflective surface, like the one shown in Figure 6. Either add low-detail pictures, posters, or similar objects, or drop the lights on the back wall to reduce the reflection.

On the other hand, at a 28% difference between WMV-AP and H.264 (see Table 2), producers using this format should consider bumping their data rates significantly to produce equivalent still-frame quality, and manage their backgrounds to minimize temporal artifacts.

Action
The action category contains seven high-motion clips including golf, kickboxing, skateboarding, martial arts, and the pita clip shown in Figure 3 that has tortured codecs since I shot the video in Israel back in 1999. In a surprising turn, Apple won this category (and the next) with good still-frame quality and fewer temporal artifacts during real-time playback than its nearest competitor, Flash—though the difference between the two, at least in this category, was not commercially meaningful (see Table 3).

On the other hand, WMV-AP’s performance was significantly below that of both H.264 and Flash, primarily due to blockiness in the still-frame comparison, and lots of motion artifacts. A good example of the still-frame issue is shown in Figure 7 (p. 86), where blocks are noticeable in the two WMV images, while the Flash and H.264 images are clear.

Entertainment
These clips include ballet, opera, country music, and a marimba player, with most clips having at least moderate camera or subject motion. Here again, Apple came in first, with significant margins over both Flash and WMV-AP (see Table 4). Note that still-frame quality was very similar with the top three codecs, with only one point separating them.

On the other hand, Apple proved four points better than Flash in temporal quality, and five points better than WMV-AP, though for different reasons. Again, WMV-AP performed very well on clips with low to moderate motion, especially when it came to minimizing noise in background walls. However, once motion became significant, blockiness got very severe. In contrast, Flash’s downfall was background noise—fortunately, you can minimize this by choosing the appropriate background.

Quality Summary
Obviously, Microsoft has made good progress in codec quality, and the customization options now available could pay real dividends for Windows Media producers. Clearly, however, the new release doesn’t change the balance of power in the codec space.

Flash has made little progress since we last analyzed the format, dropping behind QuickTime in quality at these encoding parameters. Given the different value propositions offered by the two formats, it’s hard to conclude that this is competitively meaningful. You can read more about this in Chris Hock’s online interview on StreamingMedia.com. On the other hand, quality always matters, and this dynamic bears watching.

Obviously, Apple has also significantly improved its codec in the past year, improving competitive quality and eliminating the random, highly distorted frames I experienced frequently when producing the codec survey in early 2006. Still, the bipolar nature of its performance was puzzling.

Originally, I though that this disparity might have been caused by VBR encoding gone bad, though I had definitely opted for the "streaming" option in Compressor while setting my parameters. To be sure, I went back and verified that I had used CBR on the test file, and even encoded just the low-motion business file seperately from high-motion clips, and got very similar results. I also encoded the file with Sorenson Squeeze with similar results, though squeeze doesn't have a two-pass CBR option with the Apple codec.

Recognizing that Apple is widely used for movie trailers, I checked the average bits per pixel per second for movie trailers that I had viewed in the web survey referred to frequently in this article. I find the average bits per pixel per second metric a good way to compare videos compressed using different resolutions and data rates becuase, as the name suggests, it measures how much data the producer allocated to each pixel in the video.

I found three movie trailers-for Borat, Babel, and A prairie Home Companion-encoded into QuickTime format, and the average bits per pixel per second was 10.7. In comparison, the test employed here has an average bits per pixel per second of 3.3, about 30% of the bits used for the movie trailers. I'm sure if I tripled the data rate of these files, the low-motion sequences in these files would look as good as the movie trailers. Overall, however, despite the Jekyll and Hyde performance, QuickTime/H.264 was clearly impressive.

So, clearly there's little reason to switch from Flash or QuickTime to WMV. On the other hand, the improvements in quality you can achieve by using the latest WMV codecs make switching to them a no-brainer. As long as you pay attention to the details, your audience will thank you.