Thursday, November 12, 2015

The Human Kernel

Interesting way of doing supervised learning of kernel. From the introduction:

Our work is intended as a preliminary step towards building probabilistic kernel machines that en-
capsulate human-like support and inductive biases. Since state of the art machine learning methods
perform conspicuously poorly on a number of extrapolation problems which would be easy for
humans [10], such efforts have the potential to help automate machine learning and improve perfor-
mance on a wide range of tasks – including settings which are difficult for humans to process (e.g.,
big data and high dimensional problems). Finally, the presented framework can be considered in
a more general context, where one wishes to efficiently reverse engineer interpretable properties of
any model (e.g., a deep neural network) from its predictions.

Bayesian nonparametric models, such as Gaussian processes, provide a
compelling framework for automatic statistical modelling: these models have a
high degree of flexibility, and automatically calibrated complexity. However,
automating human expertise remains elusive; for example, Gaussian processes
with standard kernels struggle on function extrapolation problems that are
trivial for human learners. In this paper, we create function extrapolation
problems and acquire human responses, and then design a kernel learning
framework to reverse engineer the inductive biases of human learners across a
set of behavioral experiments. We use the learned kernels to gain psychological
insights and to extrapolate in human-like ways that go beyond traditional
stationary and polynomial kernels. Finally, we investigate Occam's razor in
human and Gaussian process based function learning.