ARTICLE

Intrinsic Images – Introduction and Reading List

The problem of decomposing an image into reflectance and shading – known as intrinsic images – has received much attention in the last few years. This article introduces the problem of retrieving intrinsic images as well as recent literature in this area.

Introduction

The idea of intrinsic images was already picked up by Barrow and Tenenbaum [1] in 1978 and is based on the observation that humans are able to derive intrinsic characteristics from images. With intrinsic characteristics, Barrow and Tenenbaum essentially refer to a set of features such as "color, orientation, distance, size, shape, [and] illumination" [1, p.3]. The problem of retrieving these characteristics, or intrinsic images, is formulated as follows:

Problem (Intrinsic Images). Given an image (or possibly several images of the scene), retrieve a set of intrinsic images, all registered with the original image, each describing one intrinsic characteristic [1].

This problem statement is still very general, however, it captures the main goal of computing intrinsic images. It is clear, that the set of possible intrinsic characteristics may be extended by several other aspects, like motion, occlusion boundaries etc., depending on the application. In practice, when speaking of intrinsic images, most researchers are primarily interested in two characteristics: reflectance (or albedo) and shading (e.g [3,4,5,6,7]).

In this context of decomposing an image into reflectance and shading, another work of the early 70s becomes increasingly interesting: Land and McCann's Retinex Theory [2]. In their work, Land and McCann propose a simple algorithm for determining the relative difference in reflectance of two surfaces in a "Mondrian world" (i.e. a world looking like Mondrians's paintings). The basic notion on which most Retinex-based algorithms for color constancy, illumination estimation and intrinsic images are based is the following: strong gradients and edges are most likely caused by reflectance changes while small gradients are caused by shading, this means that shading is assumed to be smooth.

The above considerations can easily be formalized when assuming surfaces to be approximately Lambertian (in other words: all surfaces are matt). Then, an image can be written as product of reflectance and shading:

$I = R \cdot S$

where $I$ is the image, $R$ the reflectance and $S$ the shading image and the product is defined element-wise. Often, authors transform the above equation into log-space [3,4,5,6,7]:

$\log(I) = \log(R) + \log(S)$.

Obviously, this problem is highly underconstrained making it comparably difficult to solve. Therefore, some authors use appropriate priors on either reflectance or shading, for example shading is often assumed to be smooth while the reflectance values are often considered to be sparse (see references below). Another difficulty lies in creating appropriate datasets. To date (to the best of my knowledge), there are only three datasets for intrinsic image decompositions out there, see below.

Reading List

The following paragraphs are intended to give a brief overview over the literature on intrinsic images. Some basic references of the 70s and 80s were already introduced above:

Due to the large number of publications on this topic, discussing any of the above approaches in detail is unrealistic for this reading list, however, some pointers to the corresponding implementations, if available, are necessary.

Figure 1 shows examples from the datasets introduced in [4] and [6]. The images of the "Intrinsic Images in the Wild" dataset can be browsed online at opensurfaces.cs.cornell.edu/.

Figure 1 (click to enlarge): Images from the dataset introduced in [4], row 1 and 2, and from the dataset introduced in [6], row 3 and 4. Each row shows the original image, the reflectance image and the shading image.

ABOUTTHEAUTHOR

In September, I was honored to receive the MINT-Award IT 2018, sponsored by ZF and audimax, for my master thesis on weakly-supervised shape completion. For CVPR 2019, however, I am working on a different topic: adversarial robustness and generalization of deep neural networks.
18thOCTOBER2018 , David Stutz

What is your opinion on this article? Did you find it interesting or useful? Let me know your thoughts in the comments below: