"... Recently, researchers have demonstrated that "loopy belief propagation" --- the use of Pearl's polytree algorithm in a Bayesian network with loops --- can perform well in the context of error-correcting codes. The most dramatic instance of this is the near Shannon-limit performa ..."

Recently, researchers have demonstrated that &quot;loopy belief propagation&quot; --- the use of Pearl&apos;s polytree algorithm in a Bayesian network with loops --- can perform well in the context of error-correcting codes. The most dramatic instance of this is the near Shannon-limit performance of &quot;Turbo Codes&quot; --- codes whose decoding algorithm is equivalent to loopy belief propagation in a chain-structured Bayesian network. In this paper we ask: is there something special about the error-correcting code context, or does loopy propagation work as an approximate inference scheme in a more general setting? We compare the marginals computed using loopy propagation to the exact ones in four Bayesian network architectures, including two real-world networks: ALARM and QMR. We find that the loopy beliefs often converge and when they do, they give a good approximation to the correct marginals. However, on the QMR network, the loopy beliefs oscillated and had no obvious relationship ...

...aluate loopy propagation rather than exhaustively compare the performance of alternative algorithms. (For a more careful evaluation of likelihood weighted sampling in the case of the QMR network, see =-=[8]-=-.) 3 The networks We used two synthetic networks, PYRAMID and toyQMR, and two real world networks, ALARM and QMR. The synthetic networks are sufficiently small that we can perform exact inference, usi...

"... One of the major obstacles to using Bayesian methods for pattern recognition has been its computational expense. This thesis presents an approximation technique that can perform Bayesian inference faster and more accurately than previously possible. This method, "Expectation Propagation," ..."

One of the major obstacles to using Bayesian methods for pattern recognition has been its computational expense. This thesis presents an approximation technique that can perform Bayesian inference faster and more accurately than previously possible. This method, &quot;Expectation Propagation,&quot; unifies and generalizes two previous techniques: assumeddensity filtering, an extension of the Kalman filter, and loopy belief propagation, an extension of belief propagation in Bayesian networks. The unification shows how both of these algorithms can be viewed as approximating the true posterior distribution with a simpler distribution, which is close in the sense of KL-divergence. Expectation Propagation exploits the best of both algorithms: the generality of assumed-density filtering and the accuracy of loopy belief propagation. Loopy belief propagation, because it propagates exact belief states, is useful for limited types of belief networks, such as purely discrete networks. Expectation Propagati...

...iables [84], and more [71]. One class of algorithms that has drawn attention recently is the family of variational methods [51], which have proven to be extremely successful where they can be applied =-=[47]-=-. 3.8 Conclusion In recent years, BNs have had a growing number of applications, both academic and commercial. Many of the early successes were in the area of medical diagnosis (e.g. [40]). Since then...

Abstract. In Datalog, missing values are represented by Skolem constants. More generally, in logic programming missing values, or existentially quantified variables, are represented by terms built from Skolem functors. The CLP(BN) language represents the joint probability distribution over missing values in a database or logic program by using constraints to represent Skolem functions. Algorithms from inductive logic programming (ILP) can be used with only minor modification to learn CLP(BN) programs. An implementation of CLP(BN) is publicly available as part of YAP Prolog at

"... We show how to use a variational approximation to the logistic function to perform approximate inference in Bayesian networks containing discrete nodes with continuous parents. Essentially, we convert the logistic function to a Gaussian, which facilitates exact inference, and then iteratively adjust ..."

We show how to use a variational approximation to the logistic function to perform approximate inference in Bayesian networks containing discrete nodes with continuous parents. Essentially, we convert the logistic function to a Gaussian, which facilitates exact inference, and then iteratively adjust the variational parameters to improve the quality of the approximation. We demonstrate experimentally that this approximation is much faster than sampling, but comparable in accuracy. We also introduce a simple new technique for handling evidence, which allows us to handle arbitrary distributionson observed nodes, as well as achieving a significant speedup in networks with discrete variables of large cardinality. 1

...ower bound on the likelihood [NH98]. However, the upper bound can be used in conjunction with the lower bound to filter out runs of MCMC which result in marginals which fall outside the bounds, as in =-=[JJ99]-=-. Note that we can also exploit the quadratic approximation to fit the parameters of the logistic node, w and b, using linear regression, instead of the slower IRLS (Iteratively Reweighted Least Squar...

"... Recently, variational approximations such as the mean field approximation have received much interest. We extend the standard mean field method by using an approximating distribution that factorises into cluster potentials. This includes undirected graphs, directed acyclic graphs and junction ..."

Recently, variational approximations such as the mean field approximation have received much interest. We extend the standard mean field method by using an approximating distribution that factorises into cluster potentials. This includes undirected graphs, directed acyclic graphs and junction trees. We derive generalised mean field equations to optimise the cluster potentials. We show that the method bridges the gap between the standard mean field approximation and the exact junction tree algorithm. In addition, we address the problem of how to choose the structure and the free parameters of the approximating distribution. From the generalised mean field equations we derive rules to simplify the approximation in advance without affecting the potential accuracy of the model class. We also show how the method fits into some other variational approximations that are currently popular. 1 INTRODUCTION Graphical models, such as Bayesian networks, Markov fields, and Bolt...

...inty in Artificial Intelligence (UAI-2000) intractable for exact computation, and approximations are necessary. In this context, the variational methods gain increasingly interest (Saul et al., 1996; =-=Jaakkola and Jordan, 1999-=-; Jordan et al., 1999; Murphy, 1999). An advantage of these methods is that they provide bounds on the approximation error and they fit excellently into a generalised-EM framework for learning (Saul e...

by
Haipeng Guo, William Hsu
- In In the joint AAAI-02/KDD-02/UAI-02 workshop on Real-Time Decision Support and Diagnosis Systems, 2002

"... As Bayesian networks are applied to more complex and realistic real-world applications, the development of more efficient inference algorithms working under real-time constraints is becoming more and more important. This paper presents a survey of various exact and approximate Bayesian network ..."

As Bayesian networks are applied to more complex and realistic real-world applications, the development of more efficient inference algorithms working under real-time constraints is becoming more and more important. This paper presents a survey of various exact and approximate Bayesian network inference algorithms. In particular, previous research on real-time inference is reviewed. It provides a framework for understanding these algorithms and the relationships between them. Some important issues in real-time Bayesian networks inference are also discussed.

...he use of exact inference algorithm. For example, Jaakkola and Jordan found that in “QMR-DT”, one of the largest BNs in practice, the median size of the maximal clique of the moralized graph is 15=-=1.5 [JJ99]-=-. Faced with the intractability of exact inference to large, complex networks, many researchers have investigated approximate inference algorithms. 3.2 Approximate Inference Approximate BN inference a...

"... Loopy and generalized belief propagation are popular algorithms for approximate inference in Markov random fields and Bayesian networks. Fixed points of these algorithms have been shown to correspond to extrema of the Bethe and Kikuchi free energy, both of which are approximations of the exact Helmh ..."

Loopy and generalized belief propagation are popular algorithms for approximate inference in Markov random fields and Bayesian networks. Fixed points of these algorithms have been shown to correspond to extrema of the Bethe and Kikuchi free energy, both of which are approximations of the exact Helmholtz free energy. However, belief propagation does not always converge, which motivates approaches that explicitly minimize the Kikuchi/Bethe free energy, such as CCCP and UPS. Here we describe a class of algorithms that solves this typically non-convex constrained minimization problem through a sequence of convex constrained minimizations of upper bounds on the Kikuchi free energy. Intuitively one would expect tighter bounds to lead to faster algorithms, which is indeed convincingly demonstrated in our simulations. Several ideas are applied to obtain tight convex bounds that yield dramatic speed-ups over CCCP.

"... We propose a new method to program robots based on Bayesian inference and learning. The capacities of this programming method are demonstrated through a succession of increasingly complex experiments. Starting from the learning of simple reactive behaviors, we present instances of behavior combinati ..."

We propose a new method to program robots based on Bayesian inference and learning. The capacities of this programming method are demonstrated through a succession of increasingly complex experiments. Starting from the learning of simple reactive behaviors, we present instances of behavior combinations, sensor fusion, hierarchical behavior composition, situation recognition and temporal sequencing. This series of experiments comprises the steps in the incremental development of a complex robot program. The advantages and drawbacks of this approach are discussed along with these different experiments and summed up as a conclusion. These different robotics programs may be seen as an illustration of probabilistic programming applicable whenever one must deal with problems based on uncertain or incomplete knowledge. The scope of possible applications is obviously much broader than robotics.

"... We introduce a framework for advanced behavioral animation in virtual humans, which addresses the challenging open problem of simulating social interactions between pedestrians in urban settings. Based on hierarchical decision networks, our novel framework combines probability, decision, and graph t ..."

We introduce a framework for advanced behavioral animation in virtual humans, which addresses the challenging open problem of simulating social interactions between pedestrians in urban settings. Based on hierarchical decision networks, our novel framework combines probability, decision, and graph theories for complex behavior modeling and intelligent action selection subject to manifold internal and external factors in the presence of uncertain knowledge. It yields autonomous characters that can make nontrivial interpretations and arrive at rational decisions dependent on multiple considerations. We demonstrate our framework in behavioral animation scenarios involving interacting autonomous pedestrians, including an elaborate emergency response animation. 1.

...le elimination [Aji and McEliece 2000; Kschischang et al. 2001], the junction tree algorithm [Cowell et al. 1999; Huang and Darwiche 1994], and a variety of approximation methods [Jordan et al. 1998; =-=Jaakkola and Jordan 1999-=-]. These algorithms run in exponential-time in the worst case, but when the treewidth w of the network graph is small, they run in time and space n O(1) 2 O(w) ,wheren is the number of nodes in the ne...