Comments 0

Document transcript

Be able to define the following terms and answer basic questions about them:



Probability

o

Random variables

o

Axioms of probability

o

Joint, marginal, conditional probability distributions

o

Independence and conditionalindependence

o

Bayes rule



Bayesian inference

o

Likelihood, prior, posterior

o

Maximum likelihood (ML), maximum a posteriori (MAP) inference

o

Naïve Bayes

o

Learning



Bayesian networks

o

Structure and parameters

o

Conditional independence assumptions

o

Calculating jointand conditional probabilities

o

Complexity of inference

o

Inference by sampling, learning (very high level)



Markov decision processes

o

Markov assumption, transition model, policy

o

Bellman equation

o

Value iteration

o

Policy iteration



Reinforcement learning

o

Model-based vs. model-free approaches

o

Exploration vs. exploitation

o

TD Q-learning



Classifiers

o

Training, testing, generalization

o

Naïve Bayes

o

Decision trees

o

Nearest neighbor classifiers

o

Linear classifiers

o

Nonlinear classifiers

Sample exam

questions

1.

Use the axioms of probability to prove thatP(¬A) = 1–

P(A).

2.

A couple has two children, and one of them is a boy. What is the probability of the otherone being a boy?

3.

Consider the following joint probability distribution:

P(A = true, B = true) = 0.12

P(A = true, B = false) = 0.18

P(A = false, B = true) = 0.28

P(A = false, B = false) = 0.42

What are the marginal distributions of A and B? Are A and B independent and why?

4.

What is the relationship between the Boolean satisfiability (SAT) problem and

Bayesiannetwork inference? How can we find an approximate solution to the SAT problem byapproximate Bayesian network inference?

5.

We have a bag of three biased coins,a,b, andc, with probabilities of coming up heads of20%, 60%, and 80%, respectively.One coin is drawn randomly from the bag (with equallikelihood of drawing each of the three coins), and then the coin is flipped three times togenerate the outcomes X1, X2, and X3.

a.

Draw the Bayesian network corresponding to this setup and define the necessaryconditional probability tables (CPTs).

b.

Calculate which coin was most likely to have been drawn from the bag if theobserved flips come out heads twice and tails once.

6.

Consider the “Burglary” Bayesian network.

a.

How many independent parameters doesthis network have? How manyindependent entries does the full joint distribution table have?

b.

Is this network a polytree?

c.

If no evidence is observed, are Burglary and Earthquake independent?

d.

If we observe Alarm = true, are Burglary and Earthquakeindependent? Justifyyour answer by deriving whether the probabilities involved satisfy the definitionof conditional independence.

7.

Two astronomers in different parts of the world make measurements M1

and M2

of thenumber of stars N in some small region of the sky, using their telescopes. Normally, thereis a small probabilitye

of error by up to one star in each direction. Each telescope canalso (with a much smaller probabilityf) be badly out of focus (events F1

and F2), in whichcase the scientist will

undercount by three or more stars (or if N is less than 3, fail todetect any stars at all).

a.

Draw a Bayesian network for this problem.

b.

Write out a conditional distribution for P(M1

| N) for the case where N

{1,2,3}and M1



{0,1,2,3,4}. Each entry in

the conditional distribution table should beexpressed as a function of the parameterse

and/orf.

c.

Suppose M1

= 1 and M2

= 3. What are the possible numbers of stars if you assumeno prior constraint on the values of N?

d.

What is the most likely number of stars, given these observations?Explain how tocompute this, or if it is not possible to compute, explain what additionalinformation is needed and how it would affect the result.

8.

Consider the following 4x3 world, withtransition model as explained in class. Calculatewhich squares can be reached from (1,1) by the action sequence [Up, Up, Right, Right,Right] and with what probabilities.

9.

Consider the following

game, called “High/Low”. There is an infinite deck of cards, halfof whichare2’s, one quarter

are

3’s, and one quarter are

4’s. The game starts with a 3showing. After each card, you say “High” or “Low,” and a new card is flipped. If you arecorrect, you win the points shown on the new card. If there is a tie,

you get zero points. Ifyou are wrong, the game ends.

a.

Write down the transition model and the reward function for this game.

b.

Draw the expectimax tree for the first two rounds of this game

and write down theexpected utility of every node (use a discount factor of 1). What is the optimalpolicy assuming the game only lasts two rounds?