BAYESIAN PROBABILITY THEORY

The starting point in modern Bayesian probability theory is that
probability is interpreted as a degree of belief (for
bibliographic notes, see [7,57]). Richard Cox
showed that certain very general requirements for the calculus of
beliefs result in the rules of probability theory [19].
Decision theory also leads to the same rules [129,114,102]
with the same interpretation. There are other domains, most notably
measure theory, where the same rules appear, but from the point of
view of learning systems and decisions in the face of uncertainty,
degree of belief is the appropriate interpretation.

Beliefs are always subjective, and therefore all the probabilities
appearing in Bayesian probability theory are conditional. In
particular, under the belief interpretation probability is not an
objective property of some physical setting, but is conditional to the
prior assumptions and experience of the learning system. It is
completely reasonable to talk about ``the probability that there is a
tenth planet in the solar system'' although this planet either exists
or does not exist and there is no sense in interpreting the
probability as a frequency of observing a tenth planet. Sometimes the
probabilities can be roughly equated with empirical frequencies, but
this can be considered as a special case of the belief interpretation
as was shown by Cox [19].

Accessible introductions to practical applications of Bayesian
probability theory can be found, for instance, in
[75,103,28].