Conditional Probability, Bayes’ Theorem, and Independence
SD03a — Probabilities
Author
Samer El Kababji
Course: Statistics and Probability for Data Science (DS14270). Source deck: Samer El Kababji, SD03a — Probabilities.
This lesson develops the deck’s conditional-probability, Bayes, and independence calculations. The practice questions and the extra computational checks extend the deck’s concepts.
Conditional probability
Conditional probability restricts attention to outcomes for which a given event has occurred. If \(P(B)>0\), then
\[
P(A\mid B)=\frac{P(A\cap B)}{P(B)}.
\]
Rearranging gives the multiplication rule:
\[
P(A\cap B)=P(A\mid B)P(B)=P(B\mid A)P(A).
\]
The denominator \(P(B)\) changes the reference population from the entire sample space to event \(B\).
Worked example: production lines
The deck describes two production lines, \(A\) and \(A^c\), and two outcomes, defective \(B\) and nondefective \(B^c\). The counts are
For a partition \(A_1,\ldots,A_k\) and an event \(B\) with \(P(B)>0\),
\[
P(A_j\mid B)
=\frac{P(B\mid A_j)P(A_j)}
{\displaystyle\sum_{i=1}^{k}P(B\mid A_i)P(A_i)}.
\]
\(P(A_j)\) is the prior probability. After observing \(B\), \(P(A_j\mid B)\) is the posterior probability. The likelihood \(P(B\mid A_j)\) measures how compatible the observation is with \(A_j\).
Worked example: DVD-player repairs
An electronics store’s DVD-player sales are 50% brand 1, 30% brand 2, and 20% brand 3. The respective probabilities of requiring warranty repair are 25%, 20%, and 10%.
Let \(A_j\) denote brand \(j\) and let \(B\) denote warranty repair. The priors and likelihoods are
Events \(A\) and \(B\) are independent if knowing that one occurred does not change the probability of the other. When the relevant conditional probabilities are defined,
\[
P(A\mid B)=P(A)
\]
is equivalent to the symmetric product rule
\[
P(A\cap B)=P(A)P(B).
\]
Given another event or variable \(Y\), conditional independence is expressed as
\[
P(A\cap B\mid Y)=P(A\mid Y)P(B\mid Y).
\]
Mutual exclusivity and independence describe different relationships. If \(A\) and \(B\) are disjoint and both have positive probability, then observing \(A\) makes \(B\) impossible. Such events are dependent, not independent.
Worked example: boiler relief valves
A boiler has five identical relief valves. Each valve opens on demand with probability \(0.96\), and the valves operate independently.
The complement of “at least one opens” is “none opens,” so
\[
P(\text{at least one opens})=1-(1-0.96)^5
=1-(0.04)^5\approx0.999999898.
\]
The complement of “at least one fails” is “all open,” so
\[
P(\text{at least one fails})=1-(0.96)^5\approx0.1846.
\]
From the production table, compute \(P(B\mid A)\) and explain its denominator.
In the DVD example, find \(P(A_2\mid B)\) and \(P(A_3\mid B)\).
A supplier provides 70% of a factory’s parts and has a 2% defect rate. A second supplier provides 30% and has a 5% defect rate. Find the overall defect probability and the probability that a defective part came from the second supplier.
If \(P(C)=0.4\), \(P(D)=0.5\), and \(P(C\cap D)=0.2\), are \(C\) and \(D\) independent?
Two events have positive probabilities and are mutually exclusive. Can they be independent? Justify your answer mathematically.
NoteCheck your answers
\(P(B\mid A)=2/8=1/4\). The denominator is 8 because conditioning restricts the table to components from line \(A\).
\(P(A_2\mid B)=0.060/0.205\approx0.293\) and \(P(A_3\mid B)=0.020/0.205\approx0.098\).
\(P(D)=0.70(0.02)+0.30(0.05)=0.029\). The posterior probability is \(P(S_2\mid D)=0.30(0.05)/0.029\approx0.517\).
Yes. \(P(C)P(D)=0.4(0.5)=0.2=P(C\cap D)\).
No. Mutual exclusivity gives \(P(C\cap D)=0\), but independence would require \(P(C\cap D)=P(C)P(D)>0\).
