3 Essential Tools to Own When Comparing Video Encoding Quality
I just finished a consulting project that involved shooting and encoding 32 1080p and 4K files into about 350 different configurations in H.264 and HEVC format and evaluating the moving and still image quality of each file. To meet my deadlines and time estimates—to say nothing of maintaining my sanity—I used three key tools.
The first one is an objective quality metric. I’ve encoded video for more than 20 years and have always relied upon subjective comparisons. However, when you’re evaluating the comparative quality of more than 350 files, a trusted, objective metric is worth its weight in Rolaids and NoDoz, particularly if it also enables subjective comparisons. The tool I acquired is the Moscow State University Video Quality Metric Tool (VQMT), my review of which appeared in the winter 2014 issue of Streaming Media European Edition.
The VQMT can produce more than 20 different quality metrics, including peak signal to noise ratio (PSNR), structured similarity index (SSIM), and Video Quality Metric (VQM, see description). For the record, I found VQM to be most useful in accurately predicting meaningful quality differences between the encoded files. VQMT also presents an excellent interface for comparing the frame quality of two or three encoded files, making it a one-stop shop for identifying and documenting qualitative differences between files. Even at $999, it’s a steal for high-volume jobs.
Fortunately, the second critical tool, FFmpeg, is free, though like all command line programs, there is a learning curve, particularly if you’re not command line literate. During my project, FFmpeg proved to be the Swiss Army knife of file conversion tools, converting H.264, HEVC, and VP9 files to the YUV format that the VQMT tool uses for file comparisons, and also converting YUV files into AVI files I could import into Adobe Premiere for supplementary subjective comparisons and frame grabs.
FFmpeg is a cross-platform, command line encoding tool that many developers use to create their own enterprise encoding tools. You can download the software and access multiple learning resources at ffmpeg.org. While there were other GUI-based tools I could have used to perform these conversions, most lack the simple and massive automatability that command line tools enable via batch files.
The batch file is the third tool necessary for high volume producers.
As an example, one great feature of the VQMT is the ability to batch process multiple files. During my project, I would batch encode 16 files, then use VQMT to produce PSNR, SSIM, and VQM ratings for each file. On the larger 4K files, each test took about 3 minutes to run, so if performed via the GUI, analyzing the files would have taken an exceptionally boring 2.5 hours. By using the command line, I could create the batch, walk away, and copy and paste the results into my scoring spreadsheet. The VQM scores identified major qualitative differences, so I could load those files into VQMT and continue my analysis.
Put simply, batch files are files that contain commands to run multiple command line operations. As an example, the command line to analyze a file under VQMT has three elements, the program name (VQMT.exe), input file name (encodedfile.mp4) and test (PSNR). To analyze 16 input files by three tests, you create a 48-line batch file, including the input files and required tests, and run it under the DOS command line (VQMT only supports Windows). The batch file presents the command line parameters to the program, one after another, until all processes are complete.
There a plenty of free resources on batch files; one I liked was an article at WikiHow you can find at bit.ly/batch_file. Note that not all encoding, analysis, and conversion tools are accessible by command line, so make sure whatever tool you use for high-volume encoding work is command line-accessible.
By their very nature, large, high-volume encoding projects are cumbersome and time-consuming. Whether you’re producing them for a client, or for in-house use, using the right tools can help ensure both productivity and efficiency.
This article appears in the November/December 2014 issue of Streaming Media magazine as “The Right Tools for the Job.”
Related Articles
A Streaming Media survey shows big changes afoot in the next five years, with a move to the cloud, away from Flash, and towards 4K and HEVC.
28 Sep 2015
How can video compressionists assess the quality of different files? Only by combining objective mathematical comparisons with their own professional judgments.
13 Jul 2015
We've upgraded our testing methodology to accurately reflect what broadcasters and other high-volume video publishers ask of their encoding systems. Our first subjects: Elemental Server and Telestream Vantage Lightspeed Server.
05 Dec 2014