Bayes' Theorem

One and only one of the Bi must occur
because they are a partition of B.

Inference

Bayes theorem is relevant to inference because
we may be entertaining a number of exclusive and exhaustive hypotheses
H1, H2, ..., Hk, and
wish to know which is the best explanation of some observed data D.
In that case P(Hi|D) is called the posterior probability
of Hi, "posterior" because
it is the probability after the data has been observed.

∑j=1..k P(D|Hj) P(Hj) = P(D)

P(Hi|D) = P(D|Hi) P(Hi) / P(D)
--posterior

Note that the Hi can even be an infinite enumerable set.

P(Hi) is called the prior probability
of Hi, "prior" because
it is the probability before D is known.

Notes

T. Bayes.
An essay towards solving a problem in the doctrine of chance.
Phil. Trans. of the Royal Soc. of London, 53, pp.370-418, 1763.
Reprinted in Biometrika, 45, pp.296-315, 1958.

Conditional Probability

The probability of B given A is written P(B|A).
It is the probability of B provided that A is true;
we do not care, either way, if A is false.
Conditional probability is defined by:

P(A&B) = P(A).P(B|A) = P(B).P(A|B)

P(A|B) = P(A&B) / P(B)

P(B|A) = P(A&B) / P(A)

These rules are a special case of Bayes' theorem for k=2.

There are four combinations for two Boolean variables:

A

not A

margin

B

A & B

not A & B

(A or not A)& B = B

not B

A & not B

not A & not B

(A or not A)& not B = not B

margin

A = A&(B or not B)

not A = not A &(B or not B)

LA 1999

We can still ask what is the probability of A, say, alone

P(A) = P(A & B) + P(A & not B)

P(B) = P(A & B) + P(not A & B)

Independence

A and B are said to be independent
if the probability of A does not depend on B and v.v..
In that case P(A|B)=P(A) and P(B|A)=P(B) so

P(A&B) = P(A).P(B)

P(A & not B) = P(A).P(not B)

P(not A & B) = P(not A).P(B)

P(not A & not B) = P(not A).P(not B)

A Puzzle

I have a dice (made it myself, so it might be "tricky")
which has 1, 2, 3, 4, 5 & 6 on different faces.
Opposite faces sum to 7.
The results of rolling the dice 100 times (good vigorous rolls on carpet) were:

1- 20:
3 1 1 3 3 5 1 4 4 2 3 4 3 1 2 4 6 6 6 6

21- 40:
3 3 5 1 3 1 5 3 6 5 1 6 2 4 1 2 2 4 5 5

41- 60:
1 1 1 1 6 6 5 5 3 5 4 3 3 3 4 3 2 2 2 3

61- 80:
5 1 3 3 2 2 2 2 1 2 4 4 1 4 1 5 4 1 4 2

81-100:
5 5 6 4 4 6 6 4 6 6 6 3 1 1 1 6 6 2 4 5

Can you learn anything about the dice from these results?
What would you predict might come up at the next roll?
How certain are you of your prediction?