Bayes' theorem

In probability theory and applications, Bayes' theorem shows the relation between a conditional probability and its reverse form. For example, the probability of a hypothesis given some observed pieces of evidence, and the probability of that evidence given the hypothesis. This theorem is named after Thomas Bayes (how to say: /ˈbeɪz/ or "bays") and is often called Bayes' law or Bayes' rule.

Formula

The equation used is:

[math]\displaystyle{ P(A|B) = \frac{P(B | A)\, P(A)}{P(B)}. }[/math]

Where:

P(A) is the prior probability or marginal probability of A. It is "prior" in the sense that it does not take into account any information about B.
P(A|B) is the conditional probability of A, given B.^[1] It is also called the posterior probability because it is derived from (or depends upon) the specified value of B.
P(B|A) is the conditional probability of B given A. It is also called the likelihood.
P(B) is the prior or marginal probability of B, and acts as a normalizing constant.

In many scenarios, P(B) is calculated indirectly using the formula [math]\displaystyle{ P(B) = P(B|A)P(A) + P(B|A^c)P(A^c) }[/math], which simply states that the probability of B is the sum of the conditional probabilities based on whether A has occurred or not.^[2]

Example

A simple example is as follows: There is a 40% chance of rain on Sunday. If it rains on Sunday, there is a 10% chance of rain on Monday. If it did not rain on Sunday, then there is an 80% chance that it will rain on Monday.

"Raining on Sunday" is event A, and "Raining on Monday" is event B.

P( A ) = 0.40 = Probability of rain on Sunday.
P( A` ) = 0.60 = Probability of no rain on Sunday.
P( B | A ) = 0.10 = Probability of rain on Monday, if it rained on Sunday.
P( B` | A ) = 0.90 = Probability of no rain on Monday, if it rained on Sunday.
P( B | A` ) = 0.80 = Probability of rain on Monday, if it did not rain on Sunday.
P( B` |A` ) = 0.20 = Probability of no rain on Monday, if it did not rain on Sunday.

The first thing we would normally calculate is the probability of it raining on Monday: This would be the sum of the probability of "Rain on Sunday and rain on Monday" and "No rain on Sunday and rain on Monday":

[math]\displaystyle{ 0.40\times0.10 + 0.60\times0.80 = 0.52 = 52% }[/math] chance

However, we were asked to calculate the probability that it rained on Sunday, given that it rained on Monday, then this is where Bayes' theorem comes in. It allows us to calculate the probability of an earlier event, given the result of a later event.

The equation used is:

[math]\displaystyle{ P(A|B) = \frac{P(B | A)\, P(A)}{P(B)}. }[/math]

In our case, "Rain on Sunday" is event A, and "Rain on Monday" is event B.

P(B|A) = 0.10 = Probability of rain on Monday, if it rained on Sunday.
P(A) = 0.40 = Probability of rain on Sunday.
P(B) = 0.52 = Probability of rain on Monday.

So, to calculate the probability it rained on Sunday, given that it rained on Monday, we use the formula:

[math]\displaystyle{ P(A|B) = \frac{P(B | A)\, P(A)}{P(B)}. }[/math]

or:

[math]\displaystyle{ P(A|B) = \frac{0.10*0.40}{0.52} = .0769 }[/math]

In other words, if it rained on Monday, then there is a 7.69% chance it rained on Sunday.

Intuitive explanation

To calculate the probability of it having rained on Sunday, given that it rained on Monday, we can take the following steps:

We know that it rained on Monday. Therefore, the total probability is P(B).
The probability it rained on Sunday is P(A).
The probability it rained on Monday, given that it rained on Sunday is P(B|A).
The probability of raining on Sunday and raining Monday is P(A)*P(B|A).
Therefore, the total probability of it having rained on Sunday, given that it rained on Monday, is the chance of it raining on Sunday and Monday divided by the total chance of it having rained on Monday.

Therefore,

[math]\displaystyle{ P(A|B) = \frac{P(B | A)\, P(A)}{P(B)}. }[/math]

Another way to see this, which shows where Bayes' theorem comes from, is to consider the probability P(AB) that it rains on both Sunday and Monday. This can be calculated in two different ways, which give the same answer for P(AB):

[math]\displaystyle{ P(A)\, P(B|A) = P(B)\, P(A|B) }[/math]

In this regard, Bayes' theorem is just another way to write that equation.

Related pages

References

↑ "List of Probability and Statistics Symbols". Math Vault. 2020-04-26. Retrieved 2020-09-11.
↑
Bayes' Theorem Media
- Figure 1: Using a frequency box to show P(\text{User}\vert \text{Positive}) visually by comparison of shaded areas
- Figure 2: A geometric visualisation of Bayes' theorem
- Figure 3: Illustration of frequentist interpretation with tree diagrams
- Figure 4: Tree diagram illustrating the beetle example. R, C, P and \overline{P} are the events rare, common, pattern and no pattern. Percentages in parentheses are calculated. Three independent values are given, so it is possible to calculate the inverse tree.
- Figure 5: Bayes' theorem applied to an event space generated by continuous random variables X and Y. There exists an instance of Bayes' theorem for each point in the domain. In practice, these instances might be parametrized by writing the specified probability densities as a function of x and y.
- Figure 6: A way to conceptualize event spaces generated by continuous random variables X and Y
"Conditional Probability". www.stat.yale.edu. Retrieved 2020-09-11.

[1] "List of Probability and Statistics Symbols". Math Vault. 2020-04-26. Retrieved 2020-09-11.

[2] 
Bayes' Theorem Media

Figure 1: Using a frequency box to show P(\text{User}\vert \text{Positive}) visually by comparison of shaded areas

Figure 2: A geometric visualisation of Bayes' theorem

Figure 3: Illustration of frequentist interpretation with tree diagrams

Figure 4: Tree diagram illustrating the beetle example. R, C, P and \overline{P} are the events rare, common, pattern and no pattern. Percentages in parentheses are calculated. Three independent values are given, so it is possible to calculate the inverse tree.

Figure 5: Bayes' theorem applied to an event space generated by continuous random variables X and Y. There exists an instance of Bayes' theorem for each point in the domain. In practice, these instances might be parametrized by writing the specified probability densities as a function of x and y.

Figure 6: A way to conceptualize event spaces generated by continuous random variables X and Y

"Conditional Probability". www.stat.yale.edu. Retrieved 2020-09-11.

[3] Figure 1: Using a frequency box to show P(\text{User}\vert \text{Positive}) visually by comparison of shaded areas

[4] Figure 2: A geometric visualisation of Bayes' theorem

[5] Figure 3: Illustration of frequentist interpretation with tree diagrams

[6] Figure 4: Tree diagram illustrating the beetle example. R, C, P and \overline{P} are the events rare, common, pattern and no pattern. Percentages in parentheses are calculated. Three independent values are given, so it is possible to calculate the inverse tree.

[7] Figure 5: Bayes' theorem applied to an event space generated by continuous random variables X and Y. There exists an instance of Bayes' theorem for each point in the domain. In practice, these instances might be parametrized by writing the specified probability densities as a function of x and y.

[8] Figure 6: A way to conceptualize event spaces generated by continuous random variables X and Y

[1]

[2]

Bayes' theorem

Formula

Example

Intuitive explanation

Related pages

References

Bayes' Theorem Media