Super-resolution imaging

Super-resolution (SR) is the process of combining a sequence of low resolution images in order to produce a higher resolution image or sequence.

We assume a similar acquisition model as in the case of multichannel blind deconvolution. However for SR, the original image undergoes three degradations during the measurement: blurring, resolution decimation, and corruption by noise.

The blur may be caused by several external effects, such as, atmospheric turbulence, camera lens, relative camera-scene motion, etc.
Yet we will assume that we can model them as convolution with an unknown point spread function (PSF) hk. Resolution decimation is performed by the decimation operator D(.) that models the function of CCD sensors. It consists of convolution with a sensor PSF followed by a sampling operator, which is defined as multiplication by a sum of delta functions placed on a grid. In addition the decimation operator may contain "warping" which models geometric transformations that occurs during the acquisition. The above model is the state of the art as it takes all possible degradations into account. We have K blurred, low-resolution, and noisy images zk and the goal of SR is to estimate the original high-resolution image u.

Superresolution requires registration of images with sub-pixel accuracy, which is very difficult to achieve. Assuming unknown PSFs (blurs) in our model provides a method, which is robust to such misalignments. Our algorithm performs automatic sub-pixel registration by calculating blurs in the high-resolution grid. Alternating minimization (AM) scheme is used to find a maximum a posteriori (MAP) estimator of the high-resolution image and blurs.

The SR application (without blind deconvolution) running on Android 5.0 smartphones is available here. First you need to unzip the file before installation. Note that the mobile application was tested only on Google Nexus 5. It might run also on other smartphones with Android 5.0 or more recent. This is a beta version and only for experimental testing. We do not provide any support.

Examples:

Superresolution of real photos taken by a standard digital camera

Original low-resolution images

Roughly registered and cropped

Reconstructed high-resolution image and PSFs

The best image obtained by camera zoom

Superresolution of video acquired by a standard digital camera

Click on the images below to download the whole video sequences (XVID).