= = 2 3. variations on this classic theme. PROBABILITY MODELS 15

Transcription

1 PROBABILITY MODELS Conditional probability Suppose, for a random experiment, you are given some information about its outcome. For example, in a single roll of a fair 6-sided die, suppose I tell you that an even number was rolled. How do you update your probabilities in the face of such information? Adopting the frequentist philosophy, we should update probabilities so that if we were to repeat the experiment indefinitely and only pay attention to outcomes satisfying the condition an even number occurs, then the proportion of times an event E occurs among those trials converges to our new probability assignment of E. In this section, we make this intuition formal. Let (Ω, E, P) be a probability model and define, for any A, B E, the conditional probability of B given A by { P[A B]/P[A], P[A] > 0 (4) P[B A] : arbitrary, otherwise. We can interpret (4) in at least two ways: Frequentist interpretation: observe outcomes repeatedly from (Ω, E, P) and let n A, n B, and n A B be the number of times A, B and A B occur, respectively, in n trials. Then, from before, we have n A /n P[A]. Furthermore, n A B /n A is the fraction of times that B occurred whenever A occurred. Therefore, n A B n A n A B/n n A /n P[A B] P[A] : P[B A]. Bayesian interpretation: Update your belief that B occurred/will occur based on the information in event A. Example 4.1. In the classic game show Let s Make a Deal, the host Monty Hall would present the contestant with three doors, labeled 1, 2, and 3. Behind one of the doors was a grand prize, e.g., a new car; behind the other two doors was a goat. 1 The game begins with the contestant choosing a door. Monty Hall then reveals a goat behind one of the unchosen doors so that now there are two unknown doors (the chosen door and the remaining unchosen door). Hall then gives the contestant the option of switching his choice of doors or keeping his original choice. After this decision is made, the contents behind each door are revealed and the contestant wins whatever is behind his chosen door (either a car or a goat). Should the contestant switch doors? Does it matter? Initially, the contestant has three doors to choose from. Assuming the contestant has no extra information about which door is more likely to contain the car, the probability that the initially chosen door has a car behind it is 1/3, which coincides with the probability of winning the car given that he stays with his initial choice. On the other hand, of the two initially unchosen doors, there is a probability 2/3 that one has the car behind it and probability 1 that there is a goat. We can formailze this by considering the possible combinations behind the unchosen doors GG, CG, GC, each occurring with initial probability 1/3. Therefore, P[W switch] P[W switch, GG]P[GG] + P[W switch, GG c ]P[GG c ] A modern version of this show airs now and is hosted by Wayne Brady. The games on that show are variations on this classic theme.

2 16 HARRY CRANE Problem 4.2. Many people, even after seeing the above calculation, do not fully believe the conclusion that switching is the optimal strategy. Think about a clear, concise, intuitive way to convince someone of this without using language or tools of probability. Example 4.3 (Equally likely outcomes). Let Ω be finite, E 2 Ω, and P be the uniform distribution on E, i.e., P[E] : #E/#Ω for E E. Then, assuming A, P[A B] #(A B)/#Ω #(A B) P[B A] P[A] #A/#Ω #A. In particular, P[A A] 1 and P[{ω} A] 1/#Ω for all ω Ω. Therefore, for computing P[ A], we can treat A as the sample space and update the set of events to E A : 2 A. Exercise 4.4. Suppose P[A] > 0, then P[ A] induces a probability measure on subsets of A. Example?? illustrates a very important property of the discrete uniform distribution. Given a uniform probability model on a finite sample space Ω and a non-empty subset A Ω, the conditional probability measure P[ A] induced on A is the discrete uniform probability measure on sample space A. This property, sometimes called consistency under subsampling, is not satisfied by all probability measures. It is a very useful property to keep in mind Law of cases and subcases. Suppose an event C E can be partitioned into subcases C 1, C 2,... such that C 1, C 2,... are mutually exclusive and i C i C. Note that, in this case, i P[C i C] 1. For an event A E, how can we compute P[A C] from P[C i C], i 1? Note that the events A C 1, A C 2,... are mutually exclusive (because C 1, C 2,... are) and (A C i) A C (because C i C). Thus, P[A C] P[A C]/P[C] P[A C i ]/P[C] P[C i ]P[A C i ] P[C] P[C i C]P[A C i ]. That is, the conditional probability of A in case C is the weighted average of conditional probability of A in each subcase. When C Ω, this is called the law of cases, or law of total probability: (5) P[A] P[A Ω i ]P[Ω i ], where Ω 1, Ω 2,... is a partition of Ω. i Example 4.5 (Craps). Craps is a game played with 2 6-sided dice. The outcome of a given roll of the dice is the sum of the number of dots. The rules are as follows: On the first roll:

3 PROBABILITY MODELS 17 you lose ( crap out ) if roll a total of 2, 3, or 12; you win if roll a total of 7 or 11; otherwise, the roll total (4, 5, 6, 8, 9, or 10) becomes the point and you continue to roll until you roll the point before a 7, you win; otherwise, you lose. What is the probability that you win at craps? The strategy we use for computing this quantity is to condition on the first roll. This is a standard technique for analyzing processes that involve independent trials, but the rules of the game allow for an indefinite number of repetitions. Let W be the event you Win and let T i be the event that you get a total of i on the first roll, i 2, 3,..., 12. Note that the events T 2,..., T 12 are mutually exclusive (T i T j for i j) and exhaustive ( 12 i2 T i Ω). Hence, T 2,..., T 12 is a partition of Ω. Now, by the law of cases, P[W] 12 i2 P[T i ]P[W T i ]. Recall the P[T i ], i 2,..., 12, from Example 2.2, for which we must fill in the row with P[W T i ]: i P[T i ] P[W T i ] The table is completing by conditioning on the first step. For example, The complete table is given above. P[W T 4 ] ( )P[W T 4] 3/9. Alternatively, we can notice that once the point is established at i 4, 5, 6, 8, 9, or 10, we reduce the set of important outcomes to {i is rolled} and {7 is rolled}. Intuitively, we can consider the new sample space Ω i {i, 7} on which we assign probability P({i} {i or 7}) P[T i ] P[T i ] + P[T 7 ] P[W T i], i 4, 5, 6, 8, 9, 10. It is important to realize that this is not a complete argument, because we have not accounted for the fact that we might never roll i or 7. To complete the argument, we would have to prove that the number of rolls before i or 7 shows up is finite with probability one. This amounts to summing a geometric series, and is left as an exercise Independent and exchangeable events. The rolls in the above model for craps are assumed to be independent. Definition 4.6 (Independent Events). For a probability model (Ω, E, P), events A, B E are independent if (6) P[A B] P[A]P[B]. Remark 4.7. Drawing with replacement from the same set (usually) assumes independence, while drawing without replacement assumes exchangeability.

4 18 HARRY CRANE Example 4.8 (Exchangeability). Suppose two balls are drawn without replacement from a box containing balls labeled 1,..., 6. We assume that, on each draw, all possible outcomes are equally likely. Then we have P[first ball a 2] 1/6 P[second ball a 2] but P[first ball a 2 second ball a 1] 1/5. In general, when drawing without replacement n times from {1,..., N}, for any distinct indices n 1,..., n k, the chances associated with draws n 1, n 2,..., n k are the same as that chances associated with draws 1,..., k. The draws are said to be exchangeable. Example 4.9 (Card shuffling). What is the probability that all 4 aces appear next to each other in a well-shuffled deck of 52 cards? By exchangeability, P[4 aces next to each other] 49 P[4 aces on the top]. Let A, B E be events. By definition of P[B A] in (4), we have (7) P[A B] P[B A]P[A]. For three events A, B, C E, we have P[A B C] P[C A B]P[A B] P[C A B]P[B A]P[A]. Proposition 4.10 (Multiplication rule for conditional probabilities). For a probability model (Ω, E, P), let A 1, A 2,..., A n E, then n n j 1 (8) P A j P A j A i, where we define P[A 1 ] : P[A 1 ]. Proof. By induction, n+1 P A j P [ ] A n+1 A [n] P A n+1 n n A j P A j, where A [n] : n A j. Example 4.11 (Guessing game). Consider a deck of four cards labeled A, K, Q, T. Cards are shuffled and laid face down. They will be turned over one by one. Before each card is turned over, you predict next card is A. If it is, you win, otherwise you lose; or next card is not A. If it is, you lose, otherwise continue to next card. General analysis of the guessing game: Suppose there are n cards with one special card marked. You win if you predict when the card comes up. What strategy gives you the best chance to win? Claim: Regardless of your strategy, P[Win] 1/n. Proof by induction. Let W denote the event that you win. Clearly, for n 1, P[W] 1 and the claim is true. Now, we assume our claim is true for an n 1 card deck and use this to prove it for an n card deck. Let A 1 be the event that you assert that the first card is special and let S 1 be the event that the first card is special.

5 PROBABILITY MODELS 19 1st assertion outcome/probability/win { or lose S1 1/n win A 1 S c (n 1)/n lose 1 S 1 1/n { lose A c 1 you win w.p. 1/(n 1) by induction S c (n 1)/n 1 you lose w.p. (n 2)/(n 1). Therefore, and also P[W n A c 1 ] P[Sc 1 ] P[W n 1 S c 1 ] n 1 n 1 n 1 1 n P[W n A 1 ] 1 n. Hence, your strategy on the first card does not matter. After the first card, the remaining game is the same, but with a deck of n 1 cards. We already know that the strategy for the first card does not matter in that game. Therefore, the strategy for the second card (assuming we get that far) does not matter, and so on Odds. Definition The (chance) odds of an event A E happening are (9) odds(a) : P[A] 1 P[A]. Conversely, if odds(a) x, then P[A] x/(1 + x). For a bet on A, the payoff odds are defined by net gain if win net gain if lose. Example 4.13 (Diagnosis). A doctor is initially 60% certain that a patient has some disease, so she orders a test. The following statistics are known about the accuracy of the test. If the patient has the disease, the test almost always gives a positive result. For a diabetic patient without the disease, there are 30% false positives. Our patient is diabetic and obtained a positive result. What is the probability the patient has the disease? Let D : {has disease} and + (resp. ) denote the event that the test is postive (resp. negative). Then P[D +] P[+ D]P[D] P[+ D]P[D] + P[+ D c ]P[D c ] (0.4)

Ch. 13.2: Mathematical Expectation Random Variables Very often, we are interested in sample spaces in which the outcomes are distinct real numbers. For example, in the experiment of rolling two dice, we

Section 6.2 Definition of Probability Probability is a measure of the likelihood that an event occurs. For example, if there is a 20% chance of rain tomorrow, that means that the probability that it will

8 Finite and discrete probability distributions To understand the algorithmic aspects of number theory and algebra, and applications such as cryptography, a firm grasp of the basics of probability theory

A Few Basics of Probability Philosophy 57 Spring, 2004 1 Introduction This handout distinguishes between inductive and deductive logic, and then introduces probability, a concept essential to the study

The Casino Lab Casinos rely on the laws of probability and expected values of random variables to guarantee them profits on a daily basis. Some individuals will walk away very wealthy, while others will

Probability OPRE 6301 Random Experiment... Recall that our eventual goal in this course is to go from the random sample to the population. The theory that allows for this transition is the theory of probability.

222 CHAPTER 6. PROBABILITY 6.3 Conditional Probability and Independence Conditional Probability Two cubical dice each have a triangle painted on one side, a circle painted on two sides and a square painted

In experiments with multivariate outcome variable, knowledge of the value of one variable may help predict another. For now, the word prediction will mean update the probabilities of events regarding the

A P S T A T S A Fabulous PROBABILITY C A S I N O L A B AP Statistics Casino Lab 1 AP STATISTICS CASINO LAB: INSTRUCTIONS The purpose of this lab is to allow you to explore the rules of probability in the

Unit 19: Probability Models Summary of Video Probability is the language of uncertainty. Using statistics, we can better predict the outcomes of random phenomena over the long term from the very complex,

Conditional Probability, Hypothesis Testing, and the Monty Hall Problem Ernie Croot September 17, 2008 On more than one occasion I have heard the comment Probability does not exist in the real world, and

Bayesian Tutorial (Sheet Updated 20 March) Practice Questions (for discussing in Class) Week starting 21 March 2016 1. What is the probability that the total of two dice will be greater than 8, given that

Great Theoretical Ideas In Computer Science Victor Adamchi CS 5-5 Carnegie Mellon University Probability - I The Story The theory of probability was originally developed by a French mathematician Blaise

STA 371G: Statistics and Modeling Decision Making Under Uncertainty: Probability, Betting Odds and Bayes Theorem Mingyuan Zhou McCombs School of Business The University of Texas at Austin http://mingyuanzhou.github.io/sta371g

Discrete Math in Computer Science Homework 7 Solutions (Max Points: 80) CS 30, Winter 2016 by Prasad Jayanti 1. (10 points) Here is the famous Monty Hall Puzzle. Suppose you are on a game show, and you

Stat 20: Intro to Probability and Statistics Lecture 16: More Box Models Tessa L. Childers-Day UC Berkeley 22 July 2014 By the end of this lecture... You will be able to: Determine what we expect the sum

Combinatorics: The Fine Art of Counting Week 7 Lecture Notes Discrete Probability Continued Note Binomial coefficients are written horizontally. The symbol ~ is used to mean approximately equal. The Bernoulli

Lecture 13 Understanding Probability and Long-Term Expectations Thinking Challenge What s the probability of getting a head on the toss of a single fair coin? Use a scale from 0 (no way) to 1 (sure thing).

Decision Making Under Uncertainty Professor Peter Cramton Economics 300 Uncertainty Consumers and firms are usually uncertain about the payoffs from their choices Example 1: A farmer chooses to cultivate

THE EXPECTED VALUE FOR THE SUM OF THE DRAWS In the game of Keno there are 80 balls, numbered 1 through 80. On each play, the casino chooses 20 balls at random without replacement. Suppose you bet on the

Week 2: Conditional Probability and Bayes formula We ask the following question: suppose we know that a certain event B has occurred. How does this impact the probability of some other A. This question

Probability Models.S1 Introduction to Probability Operations Research Models and Methods Paul A. Jensen and Jonathan F. Bard The stochastic chapters of this book involve random variability. Decisions are

Chapter 4 & 5 practice set. The actual exam is not multiple choice nor does it contain like questions. MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question.

AP Stats - Probability Review Multiple Choice Identify the choice that best completes the statement or answers the question. 1. I toss a penny and observe whether it lands heads up or tails up. Suppose

4.3 Probability (4-3) 727 4.3 PROBABILITY In this section In the two preceding sections we were concerned with counting the number of different outcomes to an experiment. We now use those counting techniques

What does it mean for something to be random? An event is called random if the process which produces the outcome is sufficiently complicated that we are unable to predict the precise result and are instead

1 3.1 Sample Spaces and Tree Diagrams Probability This section introduces terminology and some techniques which will eventually lead us to the basic concept of the probability of an event. The Rare Event

PROBABILITY PROBABILITY Documents prepared for use in course B0.305, New York University, Stern School of Business Types of probability page 3 The catchall term probability refers to several distinct ideas.

Math/Stats 425 Introduction to Probability 1. Uncertainty and the axioms of probability Processes in the real world are random if outcomes cannot be predicted with certainty. Example: coin tossing, stock

A Probability puzzler!! Basic Probability Theory I Dr. Tom Ilvento FREC 408 Our Strategy with Probability Generally, we want to get to an inference from a sample to a population. In this case the population

4.4 Conditional Probability It is often necessary to know the probability of an event under restricted conditions. Recall the results of a survey of 100 Grade 12 mathematics students in a local high school.

Introduction and Overview Probability and Statistics is a topic that is quickly growing, has become a major part of our educational program, and has a substantial role in the NCTM Standards. While covering

Expectation Statistics and Random Variables Math 425 Introduction to Probability Lecture 4 Kenneth Harris kaharri@umich.edu Department of Mathematics University of Michigan February 9, 2009 When a large

Practice Test Chapter 9 Name MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question. Find the odds. ) Two dice are rolled. What are the odds against a sum

Probability definitions 1. Probability of an event = chance that the event will occur. 2. Experiment = any action or process that generates observations. In some contexts, we speak of a data-generating

PROBABILITY 53 Chapter 3 PROBABILITY The theory of probabilities is simply the Science of logic quantitatively treated. C.S. PEIRCE 3. Introduction In earlier Classes, we have studied the probability as

6.42/8.62J Mathematics for Computer Science Srini Devadas and Eric Lehman May 3, 25 Lecture otes Expected Value I The expectation or expected value of a random variable is a single number that tells you

191 The probability distribution on a single variable must sum to 1 It is also true that any joint probability distribution on any set of variables must sum to 1 Recall that any proposition a is equivalent

4.1 REVIEW AND PREVIEW RARE EVENT RULE FOR INFERENTIAL STATISTICS If, under a given assumption, the of a particular observed is extremely, we conclude that the is probably not. 4.2 BASIC CONCEPTS OF PROBABILITY

V. RANDOM VARIABLES, PROBABILITY DISTRIBUTIONS, EXPETED VALUE A game of chance featured at an amusement park is played as follows: You pay $ to play. A penny and a nickel are flipped. You win $ if either

36 Odds, Expected Value, and Conditional Probability What s the difference between probabilities and odds? To answer this question, let s consider a game that involves rolling a die. If one gets the face

1 What Is Probability? The idea: Uncertainty can often be "quantified" i.e., we can talk about degrees of certainty or uncertainty. This is the idea of probability: a higher probability expresses a higher

Chapter 5 A Survey of Probability Concepts True/False 1. Based on a classical approach, the probability of an event is defined as the number of favorable outcomes divided by the total number of possible

6.7. Probability. The study of probability has increased in popularity over the years because of its wide range of practical applications. In probability, each repetition of an experiment is called a trial,

Chapter 3 Cartesian Products and Relations The material in this chapter is the first real encounter with abstraction. Relations are very general thing they are a special type of subset. After introducing

C HAPTER 3 PROBABILITY Random Experiments I. WHAT IS PROBABILITY? The weatherman on 0 o clock news program states that there is a 20% chance that it will snow tomorrow, a 65% chance that it will rain and

MAS113 Introduction to Probability and Statistics 1 Introduction 1.1 Studying probability theory There are (at least) two ways to think about the study of probability theory: 1. Probability theory is a

Lab 11 Simulations In this lab you ll learn how to create simulations to provide approximate answers to probability questions. We ll make use of a particular kind of structure, called a box model, that

Week 5: Expected value and Betting systems Random variable A random variable represents a measurement in a random experiment. We usually denote random variable with capital letter X, Y,. If S is the sample

Section 6.2 and 6.3 Learning Objectives After this section, you should be able to DEFINE and APPLY basic rules of probability CONSTRUCT Venn diagrams and DETERMINE probabilities DETERMINE probabilities

CSE 21 - Winter 2014 Homework Homework 8 Solutions 1 Of 330 male and 270 female employees at the Flagstaff Mall, 210 of the men and 180 of the women are on flex-time (flexible working hours). Given that

Section 7C: The Law of Large Numbers Example. You flip a coin 00 times. Suppose the coin is fair. How many times would you expect to get heads? tails? One would expect a fair coin to come up heads half

Can Baccarat be Beaten By Card Counting George Joseph Las Vegas CAN "BACCARAT" BE BEATEN BY CARD COUNTING Both Baccarat and the game of 21 (Blackjack) are dealt from decks of cards which are not shuffled

The "Dutch Book" argument, tracing back to independent work by F.Ramsey (1926) and B.deFinetti (1937), offers prudential grounds for action in conformity with personal probability. Under several structural