Abstract

Deep neural networks have become increasingly successful at
solving classic perception problems such as object recognition, semantic
segmentation, and scene understanding, often reaching or surpassing human-level
accuracy. This success is due in part to the ability of DNNs to learn useful
representations of high-dimensional inputs, a problem that humans must also
solve. We examine the relationship between the representations learned by these
networks and human psychological representations recovered from similarity
judgments. We find that deep features learned in service of object classification
account for a significant amount of the variance in human similarity judgments
for a set of animal images. However, these features do not capture some
qualitative distinctions that are a key part of human representations. To remedy
this, we develop a method for adapting deep features to align with human
similarity judgments, resulting in image representations that can potentially be
used to extend the scope of psychological experiments.