Abstract: Neural networks are popular state-of-the-art models for many different
tasks.They are often trained via back-propagation to find a value of the
weights that correctly predicts the observed data. Although back-propagation
has shown good performance in many applications, it cannot easily output an
estimate of the uncertainty in the predictions made. Estimating the uncertainty
in the predictions is a critical aspect with important applications, and one
method to obtain this information is following a Bayesian approach to estimate
a posterior distribution on the model parameters. This posterior distribution
summarizes which parameter values are compatible with the data, but is usually
intractable and has to be approximated. Several mechanisms have been considered
for solving this problem. We propose here a general method for approximate
Bayesian inference that is based on minimizing{\alpha}-divergences and that
allows for flexible approximate distributions. The method is evaluated in the
context of Bayesian neural networks on extensive experiments. The results show
that, in regression problems, it often gives better performance in terms of the
test log-likelihoodand sometimes in terms of the squared error. In
classification problems, however, it gives competitive results.

Comments:

47 pages, 10 figures (41 pages for the main article, 6 for the supplementary material)