Human reflectance judgements

Our user interface for collecting annotations shows the user
an image and asks them, for a particular pair of pixels
(indicated with crosshairs and labeled Points 1 and 2), which
of the two points has a darker surface color. The user can then
select one of three options: Point 1, Point 2, and About the
same. We ask users to specify their confidence in their
assessment as Guessing, Probably, or Definitely, as was done by
[Branson et al. 2010].

We aggregate judgements from 5 users for each pair of points
and use the CUBAM machine learning model [Welinder et al. 2010]
to model two forms of bias.

Intrinsic image decompositions

The input image is decomposed into a "reflectance" and "shading" layer. Note that the reflectance layer is listed twice: color (left) and grayscale (center). Decompositions are ordered by error and then runtime (best on top). The parameters for each algorithm are the same for all photos; they are set to the values that produce lowest mean error (WHDR) for all photos. See our publication for more details.