Automating CPIQ analysis Using Imatest IT and Python

The Python interface to Imatest IT provides a simple means of invoking Imatest’s tests. This post will show how Imatest runs can be automated, then the results of those tests can be collected and easily processed. For this example, we will run five Imatest modules across three light levels, then extract CPIQ quality loss metrics.

Development Setup

Image Collection

For each device, we capture the following set of images of different test charts at different light levels: (Note: full details will be available with the publication of the standard in March of 2017)

DeviceX_eSFR_5000KLED_1000lux.JPG

DeviceX_eSFR_TL84_100lux.JPG

DeviceX_eSFR_Tung_10lux.JPG

DeviceX_Dot_5000KLED_1000lux.JPG

DeviceX_Dot_TL84_100lux.JPG

DeviceX_Dot_Tung_10lux.JPG

DeviceX_SG_5000KLED_1000lux.JPG

DeviceX_SG_TL84_100lux.JPG

DeviceX_SG_Tung_10lux.JPG

DeviceX_Coins_5000KLED_1000lux.JPG

DeviceX_Coins_TL84_100lux.JPG

DeviceX_Coins_Tung_10lux.JPG

DeviceX_Unif_100lux.JPG

Running All The Tests

Once we have these images collected in a folder named “DeviceX”, we define our lighting conditions, initialize the Imatest IT library and run our processing function on this and a number of other devices:

On the fly INI file changes

The most difficult part of this processing is to manually update region selections for the calls to Multitest, which does not yet have automatic region detection for the Colorchecker SG target (coming soon…).

In this case, we are we replacing the [sg] roi, nwid_save, and nht_save keys with the values stored in this multicharts-rois.json file:

Extracting Scores

Finally, we extract Key Performance Indicators (KPI), which are all in perceptual quality loss (QL) in units of just noticeable differences (JND). This script is a bit long for this post, so view the source on Github if you are interested./

To add these automation capabilities to your existing Imatest Master license, upgrade to Imatest Ultimate. You can also request a free 30-day trial of Imatest IT to see how this works for you before you purchase.