Mean Field Inference

Abstract

Graphical models are important and useful, but come with a serious practical problem. For many models, we cannot compute either the normalizing constant or the maximum a posteriori state. It will help to have some notation. Write X for a set of observed values, H1, …, HN for the unknown (hidden) values of interest. We will assume that these are discrete. We seek the values of H1, …, HN that maximizes P(H1, …, HN|X). There is an exponential number of such possible values, so we must exploit some kind of structure in the problem to find the maximum. In the case of a model that could be drawn as a forest, this structure was easily found; for models which can’t, mostly that structure isn’t there. This means the model is formally intractable—there is no practical prospect of an efficient algorithm for finding the maximum.