Machine Learning Research

In many machine perception problems, the set of inputs which are
likely to be encountered is limited. Consequently, generic and untuned
representations are wasteful because they allocate representational
resources to atypical inputs. Moreover, generic representations do
not take noise and other distortions into account, and are therefore
comparatively brittle. My research programme addresses these two key
limitations by developing adaptive and robust representations for
visual and audio signals. Establishing a good representation is
arguably the key step in building successful computer vision and
audition algorithms, and so these methods have wide applicability.

Modern machine learning approaches sit at the core of my research
programme because they provide automatic methods for adapting
representations to match the statistics of the input, and because they
handle noise corrupted signals gracefully by maintaining a
representation of the associated uncertainty. More specifically, I use
the Bayesian approach through which uncertainty is handled using the
rules of probability.

My machine learning projects include:

Bayesian Signal Processing. I view Machine Learning and Signal
Processing as two sides of the same coin: they are both interested in
making inferences from data. Traditionally signal processing has
focussed on efficient, and therefore often feedforward methods for
processing the data. Whereas machine learning has focussed on more
complicated and more computationally intensive methods. I have
established concrete theoretical connections between the two
fields. For instance, I have shown that classical signal processing
methods for time-frequency analysis and demodulation, are equivalent
to Bayesian inference problems. This break through has allowed
techniques from both fields to be combined, thereby improving on both
approaches.

Approximate inference. The quantities of interest in
Bayesian inference are often hard to compute mathematically or on a
computer (i.e. they are often analytically and computationally
intractable). Therefore, a large part of a Bayesian's time is spent
devising fast and accurate approximations. I use and develop a suite
of different approximation methods including, variational free-energy
methods, expectation propagation, Markov chain Monte Carlo, and
moment-matching schemes.

Circular Variables. Circular variables (i.e. angular variables
that range between between π and -π) show up all over the
place. They arise naturally in signal processing, neuroscience, brain
recording data, geophysics, engineering, and any place where complex
variables are used. Often they form a time-series like a succession of
wind direction measurements, a set of joint angle measurements during
a movement, or a time-varying complex variable such as a wavelet
coefficient. I build statistical models for time-series of circular
variables which can remove noise and impute missing data (e.g. data
which is corrupted due to a faulty sensor) and which can adaptively
process the data in an efficient manner.