December 20, 2018
By Jan Ozer Contributing Editor
Featured Articles

Elecard Video Quality Estimator: Review

The Elecard Video Quality Estimator is a tool for analyzing compressed files and computing quality metrics like VMAF, PSNR, SSIM, and others. The tool, referred to as VideoQuest in the user interface, runs in GUI mode ($850) and command-line mode, though the console application for command line operation costs an additional $400. The software runs on both Mac and Windows computers and we the latter on an HP Z840 workstation.

For perspective, VideoQuest is one of several video analysis tools from Elecard, including StreamEye, a comprehensive video analysis tool; Stream Analyzer, which can analyze the syntax of encoded media streams; and YUV Viewer, which lets you view and compare YUV files. Within this family, VideoQuest lets you easily compare two compressed files to a single source, which is very useful for comparing different encoding techniques or codecs.

You start GUI operation by loading one or two compressed files, with native support for MPEG-1/2, H.264, HEVC, and VP9 in multiple containers, including MPEG-2 Transport and Program Streams, and MP4, MKV, and AVI containers (Figure 1). Strangely, you have to load the source file in a raw format, which means a quick trip to FFmpeg (or another tool) for such conversion.

Elecard VideoQuest Loading Files
Figure 1. Getting started by loading your files and picking your metrics.

VideoQuest supports many raw formats like IYUV, NV21, and RGB24, so that’s not an issue, but you can’t see the raw file before choosing a format and opening the file, so you have to make an educated guess and then hope for the best, and if you choose wrong, you have to start over and load all three files. Other similar tools let you visualize the file while selecting it, so you can verify that you’ve chosen the right format, avoiding this problem. Like many video analysis tools, VideoQuest can only compare files of like resolution, another reason that a working knowledge of FFmpeg is almost essential for successful operation.

While in the New task screen (Figure 1), you also choose which metrics to run, which you can choose to start running as you load the files into the program. As you can see in the figure, you can also select the region of interest for metric computation, a nice feature that lets you eliminate edge regions that may contain lots of errors but aren’t really relevant to an actual viewer. You can also select auto synchronization of frames within the files, though in my tests VideoQuest synchronized the frames automatically, which saved a lot of time.

Once you press OK, the main interface shown Figure 2 opens with three major panels. On the left is the Stream panel, which contains information about each stream and frame if you click that tab on the bottom right of the panel. The graphics panel is on the bottom and contains bitrate and frame views along with views of each metric you selected in the New task screen. Dominating the interface and on the top right is the display panel, where you can see frames from the source and two test files, as well as a Diff frame that lets you display differing views of the source and test files (more later). You control the views in the various windows via the toolbar at the top and associated menu commands.

Figure 2. The main VideoQuest user interface [click to enlarge].

The Streams panel on the left provide a lot of stream-related data, including information you can’t get from MediaInfo, like the number of I, P, and B frames in the file, the minimum, maximum, and average bitrate and QP allocation. When you click on the Frame tab, the program provides frame-related data from the two test files, including frame type, size, and average QP value. For example, you can tell if the frame from Video A is a keyframe that’s ten times larger than the B-frame from video B, data that’s not available in many competitive tools, and information which makes for better informed quality comparison.

In Figure 2, the Graphics panel shows the frame types and bitrates of the two compressed files. While the bitrate visualization is too small to be useful (and can’t be adjusted to better show the comparative bitrates), the frame visualization is outstanding and one of the few tools that can actually look inside a VP9 file.

Graphics panel functionality improves when displaying the different metrics that you’ve run on your two test files as shown in Figure 3. Here we’re comparing the VMAF values of the two files, and you can set the magnification levels to better visualize key differences. For example, I set the display at 80:100 on the upper left to focus on the upper range relevant to the VMAF scores, and could change this to 35:45 to better visualize the respective PSNR values. You can also visualize the Y, U, V, or YUV value for all metrics. Dragging the playhead through this panel also moves through the video files in the display panel, enabling precise and well-informed navigation.

Figure 3. Comparing VMAF scores for my two test videos [click to enlarge].

Visualizing Key Differences

The display panel is where you’ll spend the bulk of your time with VideoQuest, and here the program offers features and configurability unavailable in other programs. For example, in Figure 2, I’ve shown all four windows in layered mode so I can easily compare the two compressed files and the source. In Figure 4, I’ve positioned the two test files side by side, zoomed to 4x resolution, inserted a 32x32 grid, and am viewing the Y values that contain detail but no color information. This makes it easy to see that there’s much more blurriness surrounding and between the players on the left, which isn’t as evident in full color. If I click and drag either window, VideoQuest synchronizes the movement of both, so you can quickly and easily examine the entire frame.

Figure 4. The display panel is highly customizable [click to enlarge].

The Diff panel shown in Figure 5 lets you visualize differences between the two compressed files or either file and the source file using the four methods shown atop Figure 4: PSNR, PSNR Clip, Subtraction, and Temperature. In the figure, I’m comparing Stream A with the Raw clip via the temperature visualization. Though I wish the colors were different, since light blue against black is tough to perceive, I can easily see that the extent of changes between the Raw file and Stream B are much greater than between the Raw file and Stream A.

Figure 5. The useful Diff visualization showing the differences between Stream A and the raw file [click to enlarge].

Once the analysis is complete, you can download metric files as well as stream information or frame grabs from any screen. The metric information is output Euro style, which switches commas and decimal points, which was initially confusing but quickly resolved with a trip to the Options dialog which let me switch to U.S. style. Once I changed from the semi-colon delimiter to the comma delimiter in my CSV file, I easily input the files into Excel and Google Sheets.

If you purchase the console version of the program you can analyze your files via a command line. I didn’t test this option, other than to see that it uses an XML configuration file to set parameters rather than the command line itself.

Comparables

VideoQuest will inevitably be compared to the Moscow State University Video Quality Measurement Tool (VQMT), which we’ve looked at several times (see here and here). They’re both very useful tools in different ways. VQMT is easier to use but not as flexible and provides less file-related data and less useful visualizations. VideoQuest has some rough edges but is deeper and more configurable. If you’re interested in blasting out a number of file comparisons and metrics, VQMT is the tool for you. If you’re looking to really explore and explain the differences between two encoded files, VideoQuest is preferred. I’m glad I have both in my tool chest.