CodecCompTesting

Code Compression Testing

High Level Goal

The goal is to create a framework for testing image compression. We should be able to create data sets for any given codec and then compare them.

We may want to extend this system to include testing of video codecs, will keep this in mind when designing it.

Specific Technical Goals

Given a codec, we want to create an RD curve which includes bits-per-pixel (BPP) values for each quality level. For example, if a codec offers quality level options 0-100, we'll create a point on an RD curve for each quality level. The Y axis will represent quality (often labelled "dB", more or less "signal to noise"), which is the result of a single or multiple quality metrics such as PSNR-HVS-M or Y-SSIM. The X axis will represent bits-per-pixel, which is the number of bits in the encoded image divided by the total number of output pixels.

We want to be able to generate graphs containing one or more RD curve from specified codecs. This will allow for comparison.

The Code

Right now there is an existing test framework here:

https://github.com/bdaehlie/web_image_formats

The test harness is written in Python, encoder library wrappers are written in C. The plan is to build on this setup.

Planned Work (higher level)

At a higher level, we need to:

• Make a number of fixes/improvements to test methodology based on feedback from the last study
• Switch data output to RD curve data
• Add graphing capabilities
• Add to quality metric options
  • Port some existing quality metric options to C programs. We don't want any more MATLAB programs, they're slow and introduce frustrating dependencies.
• Add to collection of images we run tests on.

Planned Work (specific)

• coming soon