Bayes' Rule: If \(B\) is any event with \(P(B) > 0\), then
\[P(A|B) = \frac{P(B|A)P(A)}{P(B)}.\]
▼ Proof:
Let \(A\) and \(B\) be events with \(P(B) > 0\). If \(P(A) = 0\) then both sides of the equations are 0 and the equation is true.
If \(P(A) \neq 0\) then by definition of conditional probability,
\[P(A|B) = \frac{P(A \cap B)}{P(B)}.\]
Since \(P(A) \neq 0\), we can multiply by \(P(A)/P(A)\). This gives us
\[P(A|B) = \frac{P(A \cap B)P(A)}{P(B)P(A)}.\]
Since \(P(A \cap B) = P(B \cap A)\) and \(P(B \cap A)/P(A) = P(B|A)\) we have
\[P(A|B) = \frac{P(B|A)P(A)}{P(B)}.\]
Question: Suppose a survey of your neighborhood reveals the following about pet ownership:
70% of your neighbors own dogs
40% of your neighbors own cats
50% of your neighbors who own a cat also own a dog
Later, you take a walk and see a neighbor walking her dog. What is the probability she owns a cat?
▼ Solution:
Let \(D\) be the event that a random neighbor owns a dog. By (1), \(P(D) = 0.7\). Let \(C\) be the event that a random neighbor owns a cat. By (2), \(P(C) = 0.4\).
Last, (3) says that a random cat owner has a 50% chance of owning a dog. So, given that someone owns a cat, the probability they own a dog is 0.5. We can write this as \(P(D|C) = 0.5\).
The neighbor you see walking her dog has a dog, which is a given. The probability she has a cat can be expressed \(P(C|D)\). To compute \(P(C|D)\), use Bayes' Rule.
\[P(C|D) = \frac{P(D|C)P(C)}{P(D)} = \frac{(0.5)(0.4)}{0.7} \approx 0.286\]
The probability she owns a cat is approximately 0.286.
Generalized Bayes' Rule: Let \(A_1, A_2, \dots, A_n\) be a partition of the sample space \(\Omega\). Then for any of the events \(A_i,\)
\[P(A_i|B) = \frac{P(B|A_i)P(A_i)}{\sum_{j=1}^n P(B|A_j)P(A_j)}\]
▼ Proof:
By Bayes' Rule,
\[P(A_i|B) = \frac{P(B|A_i)P(A_i)}{P(B)}\]
The only thing that needs to be shown is \(P(B) = \sum_{j=1}^n P(B|A_j)P(A_j).\) Since \(A_1, A_2, \dots, A_n\) is a partition of \(\Omega,\)
\[\Omega = \bigcup_{j=1}^n A_j\]
This implies
\[B = B \cap \bigcup_{j=1}^n A_j = \bigcup_{j=1}^n (B \cap A_j)\]
Also, by the definition of partition, the sets are pairwise disjoint, to by the definition of probability
\[P(B) = \sum_{j=1}^n P(B \cap A_j)\]
For each \(j,\) \(P(B \cap A_j) = P(B|A_j)P(A_j)\) by the multiplication rule. Making this substitution for every \(j\) we get
\[P(B) = \sum_{j=1}^n P(B | A_j)P(A_j)\]
Corollary: If \(B\) is any event with \(P(B) > 0\), then
\[P(A|B) = \frac{P(B|A)P(A)}{P(B|A)P(A)+P(B|A^C)P(A^C)}\]
▼ Proof:
This follows from the generalized version of Bayes' Rule and the fact that for any set \(A,\) \(\{A, A^C\}\) is a partition of \(\Omega.\)
Question: You have \(3\) dice. One is a 6 sided die, one is a 10 sided die, and one is a 20 sided die. A die is chosen at random and rolled. The outcome is a \(2.\) What is the probability that the randomly chosen die was the \(6\) sided die?
▼ Solution:
Let \(D_6\) be the event that the \(6\) sided die was chosen, \(D_{10}\) be the event that the \(10\) sided die was chosen, and \(D_{20}\) be the event that the \(20\) sided die was chosen. Let \(A\) be the event that a \(2\) is rolled when a die is chosen at random and rolled. Since we are given the information that a \(2\) was rolled, we are asked to find \(P(D_6|A).\) The events \(\{D_6, D_{10}, D_{12}\}\) form a partition, so by the generalized version of Bayes' Rule
\begin{align}
P(D_6|A) & = \frac{P(A|D_6)P(D_6)}{P(A|D_6)P(D_6)+P(A|D_{10})P(D_{10})+P(A|D_{20})P(D_{20})} \\
& = \frac{(1/6)(1/3)}{(1/6)(1/3)+(1/10)(1/3)+(1/20)(1/3)} \\
& = \frac{(1/6)}{(1/6)+(1/10)+(1/20)} \\
& = \frac{10}{19} \\
\end{align}
Check your understanding:
1. The third and fourth grade students at a particular school were given a math test. There are the same number of students in both grades. On problem \(6\) on the test, \(80\%\) of the students were able to answer the question correctly. However, \(90\%\) of the fourth graders answered problem \(6\) correctly. If Carrie missed question \(6,\) what is the probability she is in the fourth grade?
Unanswered
Solution: Let \(G\) be the event Carrie is in the fourth grade, and let \(M\) be the event Carrie missed question \(6.\) We know Carrie missed question \(6,\) so we need to find \(P(G|M).\) Using Bayes' rule,
\begin{align}
P(G|M) & = \frac{P(M|G)P(G)}{P(M)} \\
\end{align}
There are the same number of students in the third and fourth grades, so \(P(G) = 0.5.\) Since \(80\%\) of the students in the school answered problem \(6\) correctly, \(P(M) = 0.2.\) Also, \(90\%\) of the fourth grade class was able to answer problem \(6\) correctly, so \(P(M|G) = 0.1.\) Plugging in these values, we get
\[P(G|M) = \frac{0.1 \cdot 0.5}{0.2} = 0.25\]
2. A medical test can be given to detect a certain blood disease thought to affect \(3\%\) of the population. If a person has the disease, there is a \(99\%\) chance that the test will come back positive. If a person does not have the disease, there is a \(95\%\) chance the test will come back negative. Bob is given the test and it comes back positive. What is the probability that Bob has the blood disease?
Unanswered
Solution: Let \(A\) be the event that the blood test is positive, and let \(B\) be the event that Bob has the disease. Since Bob's test was positive, we want to find \(P(B|A).\) Using Bayes' rule,
\[P(B|A) = \frac{P(A|B)P(B)}{P(A)}\]
If Bob had the disease, there is a \(99\%\) chance that his test would come back positive. So, \(P(A|B) = 0.99.\) The blood disease affects \(3\%\) of the population, so \(P(B) = 0.03.\) The probability the test comes back positive depends whether someone has the disease. So,
\begin{align}
P(A) & = P(A|B)P(B)+P(A|B^C)P(B^C) \\
& = 0.99 \cdot 0.3 + 0.95 \cdot 0.97
\end{align}
Plugging in the values, we get
\[\frac{P(A|B)P(B)}{P(A)} = \frac{0.99 \cdot 0.3}{0.99 \cdot 0.3 + 0.95 \cdot 0.97} = 0.031\]
So the probability Bob has the disease is \(3.1\%.\)
3. Urn \(1\) contains \(3\) black balls and \(5\) blue balls. Urn \(2\) contains \(4\) black balls and \(1\) blue ball. One of the two urns is chosen at random, and then a ball is chosen randomly from the urn. The ball chosen is black. What is the probability the ball was picked from urn \(1?\)
Unanswered
Solution: Let \(A\) be the event that urn 1 is chosen, and \(B\) be the event that a black ball is chosen. We must find \(P(A|B).\) By Bayes' rule,
\[P(A|B) = \frac{P(B|A)P(A)}{P(B)}\]
Since urn 1 has 3 black balls and 5 blue balls, \(P(B|A) = \frac{3}{8}.\) Either urn is chosen with probability \(\frac{1}{2},\) so \(P(A) = \frac{1}{2}.\) The probability that a black ball is chosen depends on the urn, so
\begin{align}
P(B) & = P(B|A)P(A)+P(B|A^C)P(A^C) \\
& = \frac{3}{8} \cdot \frac{1}{2} + \frac{4}{5} \cdot \frac{1}{2}
\end{align}
Therefore, the probability that urn 1 was chosen is
\[\frac{P(B|A)P(A)}{P(B)} = \frac{\frac{3}{8} \cdot \frac{1}{2}}{\frac{3}{8} \cdot \frac{1}{2} + \frac{4}{5} \cdot \frac{1}{2}} = 0.32\]
4. Group A consists of \(2\) men and \(3\) women. Group B consists of \(3\) men and \(2\) women. Group C consists of \(4\) men and \(1\) woman. First, a group is chosen at random. Then a person is chosen from the group. If a woman is chosen, what is the probability that the woman is from group A?
Unanswered
Solution: Let \(A\) be the event the person chosen is from Group A, \(B\) be the event the person chosen is from Group B, and \(C\) be the event the person chosen is from Group C. Let \(W\) be the event that a woman is chosen. We want to find \(P(A|W).\) By generalized Bayes' rule,
\[P(A|W) = \frac{P(W|A)P(A)}{P(W|A)P(A)+P(W|B)P(B)+P(W|C)P(C)}\]
All \(3\) groups are equally likely to be chosen, so \(P(A) = P(B) = P(C).\) Since they are equally likely, they can be cancelled. So,
\[P(A|W) = \frac{P(W|A)}{P(W|A)+P(W|B)+P(W|C)}\]
Now plug in the ratios of the number of women to the number of people in the group.
\[\frac{P(W|A)}{P(W|A)+P(W|B)+P(W|C)} = \frac{\frac{3}{5}}{\frac{3}{5}+\frac{2}{5}+\frac{1}{5}}\]
Since every group has \(5\) members, the \(5\)s in the denominators can be cancelled.
\[P(A) = \frac{3}{3+2+1} = \frac{1}{2}\]