Tossing a Coin

Suppose that you toss this coin 100 times. Which of the following results is more likely to occur?

Result-1: You obtain 95 Heads and 5 Tails

Result-2: You obtain 48 Heads and 52 Tails

If the coin is a normal everyday coin, in which neither side is particularly prone to showing up more than the other side, you would expect that in a large number of tosses, Heads and Tails should show up roughly an equal number of times. This means of the two results above, Result-2 seems to be the more likely one, as the number of Heads is roughly equal to the number of Tails, which concurs with the fact that neither Head nor Tail is a preferred outcome.

On the other hand, in Result-1, the number of Heads is much larger than the number of Tails. Clearly, such a result is extremely biased towards Heads, which is not very likely given that Heads and Tails are equally preferred outcomes. Note that we are not saying that Result-1 is impossible. We are only saying that it is improbable, or unlikely. In other words, the likelihood of Result-1 is much lower than the likelihood of Result-2.

The study of Probability enables us to quantify likelihoods. It enables us to answer questions like: How likely is Result-2? How unlikely is Result-1? And so on.

The action of tossing a coin has two possible outcomes: Head or Tail. You don’t know which outcome you will obtain on a particular toss, but you do know that it will be either Head or Tail (we rule out the possibility of the coin landing on its edge!). Contrast this with a science experiment. For example, if your experiment is to drop an object, you know the outcome for sure: the object will fall towards the ground. However, tossing a coin is a random experiment, as you do know the set of outcomes, but you do not know the exact outcome for a particular execution of the random experiment.

The set of all possible outcomes of a random experiment is known as its sample space. Thus, if your random experiment is tossing a coin, then the sample space is {Head, Tail}, or more succinctly, {H, T}.

If the coin is fair, which means that no outcome is particularly preferred, or every outcome is equally likely, then we know that for a large number of tosses, the number of Heads and the number of Tails should be roughly equal. That is, the number of Heads should be roughly 1/2 of the total number of tosses, and so should be the number of Tails. This numerical quantity of 1/2 can be used as a measure of likelihood, or probability. We can say that:

The probability of the outcome Head is 1/2, because for a large number of tosses, the number of Heads will be (roughly) 1/2 of the total number of tosses.

Similarly, the probability of the outcome Tail is 1/2.

We will use the term relative occurrenceof an outcometo signify the ratio of the number of times that particular outcome is obtained to the total number of times the random experiment is performed. For example, if a coin is tossed 100 times, and 51 Heads and 41 Tails are obtained, then:

Relative occurrence of Heads: 51/100 or 0.51

Relative occurrence of Tails: 49/100 or 0.49

Now, we re-iterate the two points above. For a fair coin,

The probability of the outcome Head is 1/2, because for a large number of tosses, the relative occurrence of Heads will be roughly 1/2.

Similarly, the probability of the outcome Tail is 1/2, because the relative occurrence of Tails will be 1/2 for a large number of tosses.

Note the significance of the phrase “a large number of”. Why are we considering a large number of tosses? This is because for a small number of tosses, the relative occurrence may not give the true measure of probability. Let us understand this with an example.

Suppose that your friend gives you a coin, and tells you to determine whether this coin is fair or not. That is, you are to determine whether Head and Tail are equally likely outcomes, or one of the two outcomes is preferred over the other. You think to yourself: I will use the relative occurrence of each of the two outcomes to determine its likelihood, or probability. You start tossing the coin, and recording the outcome at each toss. Suppose that you obtain the following results.

After the 10th toss, you have 7 Heads and 3 Tails. Up to this point, the relative occurrence of Heads is 7/10 or 0.7, while the relative occurrence of Tails is 3/10, or 0.3

You continue tossing the coin. At the end of the 20th toss, you have 12 Heads and 8 Tails. The relative occurrences of Heads and Tails are 0.6 and 0.4 respectively.

At the end of the 100th toss, you have 49 Heads and 51 Tails. Now, the relative occurrences are 0.49 for Heads and 0.51 for Tails.

What do you think of these observations? If you had stopped at the 10th toss, the relative occurrence values would not have given the true probability of each outcome. You would have concluded that Head is much more likely than Tail. However, at the end of the 100th toss, it would become clear to you that the relative occurrence values of Heads and Tails are roughly equal, so they are equally likely, or equally probable, outcomes, which means that the coin is fair.

We see that to obtain a measure of the likelihood or probability of a particular outcome, we can use the value of its relative occurrence when the random experiment is repeated a large number of times.

Example 1: A coin is tossed a certain number of times. The relative occurrence of Heads is 0.75. Can we say that the coin is biased towards Heads?

Solution: No, we cannot, because the experiment (tossing the coin) may have been repeated a very small number of times, and thus the relative occurrence in such a scenario will not give the true probability.

Example 2: Coin-A is tossed 200 times, and the relative occurrence of Tails is 0.47. Coin-B is tossed an unknown number of times, but it is known that the relative occurrence of Heads is 0.50. Which coin is fairer?

Solution: It is not possible to comment on the fairness of Coin-B, because the number of times it was tossed is not known. On the other hand, Coin-A seems to be fair, as the relative occurrence of Heads over a large number of tosses is almost 1/2.

When we say that the experiment needs to be performed a sufficiently large number of times to determine the true probability of an outcome, how large a number do we need? Well, ideally speaking, the experiment should be repeated (almost) an infinite number of times. Suppose that we are trying to measure the probability of the outcome Head in the coin-tossing experiment. Let N be the total number of times we toss the coin, and let \({N_H}\) be the number of Heads obtained. Then, the relative occurrence of Heads is \(\frac{{{N_H}}}{N}\).

The true probability of the outcome Head will be given by \(\frac{{{N_H}}}{N}\) when Ntends to infinity.

In practice, you cannot toss the coin infinitely many times. How can you then calculate the probability of (say) the outcome Head?

Empirical way. One way is to toss the coin a sufficiently large number of times (say 500). The relative occurrence of H for such a large number of tosses will be a very good approximation for the true probability of H.

Assumption of equal likelihood. If for some reason, we can assume that both outcomes are equally likely, then there’s no problem and we don’t need to do any empirical calculations: the probability of H will be exactly 1/2, and so will be the probability of T.

Of course, you may ask the question: how do we decide whether both the outcomes are equally likely? Don’t we have to test that empirically - by tossing the coin a large number of times? That is, does not the assumption of equal likelihood itself need to be justified empirically?

Technically speaking, yes. However, in our present studies, whenever a situation mentions a fair coin, we assume that the two outcomes are equally likely, and that this equal likelihood of the two outcomes has been proven empirically or otherwise.

Before moving on, we note that the probability of an outcome is denoted by P(outcome). For example, in tossing a coin, the probabilities of H and T are denoted as P(H) and P(T). For a fair coin,

\[P\left( H \right) = P\left( T \right) = \frac{1}{2}\]

For any coin (whether fair or not), we will have:

\[P\left( H \right) + P\left( T \right) = 1\]

Can you understand why the sum of the two probabilities should be equal to 1? Think in terms of relative occurrences. The relative occurrences of the two outcomes must sum to 1.