SD03a — Probabilities
Course: Statistics and Probability for Data Science (DS14270).
Source deck: Samer El Kababji, SD03a — Probabilities.
This lesson develops the deck’s sample-space notation, probability axioms, event operations, and addition rules. The practice questions and the extra computational checks extend the deck’s concepts.
An experiment is a process with more than one possible outcome. Its sample space \(S\) is the set of every possible outcome.
Suppose three welds are inspected in sequence. Write \(N\) for nondefective and \(D\) for defective. Each outcome is a string of length three, so
\[ S=\{NNN,NND,NDN,NDD,DNN,DND,DDN,DDD\}. \]
There are \(2^3=8\) outcomes because each of the three positions has two possibilities.
An event is a subset of \(S\). The event that all three welds are nondefective is the simple event \(\{NNN\}\). The compound event that exactly two welds are defective is
\[ A=\{NDD,DND,DDN\}. \]
weld_grid <- expand.grid(
weld_1 = c("N", "D"),
weld_2 = c("N", "D"),
weld_3 = c("N", "D")
)
weld_outcomes <- apply(weld_grid, 1, paste0, collapse = "")
defect_count <- nchar(gsub("N", "", weld_outcomes))
S <- sort(weld_outcomes)
A <- sort(weld_outcomes[defect_count == 2])
S[1] "DDD" "DDN" "DND" "DNN" "NDD" "NDN" "NND" "NNN"A[1] "DDN" "DND" "NDD"For a finite sample space with outcomes \(O_1,\ldots,O_n\), assign probabilities \(p_1,\ldots,p_n\) satisfying
\[ 0\leq p_i\leq 1, \qquad \sum_{i=1}^{n}p_i=1, \qquad P(O_i)=p_i. \]
For any event \(E\subseteq S\), add the probabilities of the outcomes it contains:
\[ P(E)=\sum_{O_i\in E}P(O_i). \]
Worked example — three independent weld results. If \(N\) and \(D\) are equally likely for each inspection, all eight sequences have probability \(1/8\). Therefore
\[ P(A)=P(NDD)+P(DND)+P(DDN) =\frac18+\frac18+\frac18=\frac38. \]
outcome_probability <- setNames(rep(1 / length(S), length(S)), S)
P_A <- sum(outcome_probability[A])
c(number_of_outcomes = length(S), P_A = P_A)number_of_outcomes P_A
8.000 0.375 stopifnot(isTRUE(all.equal(P_A, 3 / 8)))Equal probabilities are an assumption here. In another experiment, the outcomes may have different probabilities while still summing to one.
The complement \(A^c\) contains all outcomes in \(S\) that are not in \(A\):
\[ A^c=S\setminus A, \qquad P(A^c)=1-P(A). \]
For the weld event,
\[ A^c=\{NNN,NND,NDN,DNN,DDD\}, \qquad P(A^c)=1-\frac38=\frac58. \]
A_complement <- setdiff(S, A)
P_A_complement <- sum(outcome_probability[A_complement])
A_complement[1] "DDD" "DNN" "NDN" "NND" "NNN"c(P_A = P_A, P_A_complement = P_A_complement,
total = P_A + P_A_complement) P_A P_A_complement total
0.375 0.625 1.000 The complement rule is especially useful for probabilities such as “at least one,” whose complement is often a single simpler event.
For events \(A\) and \(B\):
\[ A\cap B=\{\text{outcomes in both }A\text{ and }B\}, \]
\[ A\cup B=\{\text{outcomes in }A\text{ or }B\text{ or both}\}. \]
Thus, \(P(A\cap B)\) is the probability that both events occur, while \(P(A\cup B)\) is the probability that at least one occurs.
The addition rule prevents outcomes in the intersection from being counted twice:
\[ P(A\cup B)=P(A)+P(B)-P(A\cap B). \]
If \(A\cap B=\varnothing\), the events are mutually exclusive or disjoint, and the rule reduces to
\[ P(A\cup B)=P(A)+P(B). \]
Roll two fair six-sided dice. Let
\[ A=\{\text{the sum is }6\}, \qquad B=\{\text{at least one die is }6\}. \]
There are 36 equally likely ordered outcomes. Event \(A\) has five outcomes:
\[ (1,5),(2,4),(3,3),(4,2),(5,1). \]
Event \(B\) has \(6+6-1=11\) outcomes; \((6,6)\) is subtracted once because it was counted twice. No outcome can have sum 6 and contain a die showing 6, so \(A\cap B=\varnothing\). Therefore
\[ P(A\cup B)=\frac5{36}+\frac{11}{36} =\frac{16}{36}=\frac49. \]
dice <- expand.grid(die_1 = 1:6, die_2 = 1:6)
A_dice <- with(dice, die_1 + die_2 == 6)
B_dice <- with(dice, die_1 == 6 | die_2 == 6)
probability <- function(event) mean(event)
c(
P_A = probability(A_dice),
P_B = probability(B_dice),
P_intersection = probability(A_dice & B_dice),
P_union = probability(A_dice | B_dice)
) P_A P_B P_intersection P_union
0.1388889 0.3055556 0.0000000 0.4444444 stopifnot(isTRUE(all.equal(probability(A_dice | B_dice), 4 / 9)))Probability is built from three axioms:
\[ P\!\left(\bigcup_{i=1}^{\infty}E_i\right) =\sum_{i=1}^{\infty}P(E_i). \]
These axioms imply useful propositions:
\[ P(\varnothing)=0, \qquad P(E)+P(E^c)=1, \]
and, for three events,
\[ \begin{aligned} P(A\cup B\cup C) ={}&P(A)+P(B)+P(C)\\ &-P(A\cap B)-P(A\cap C)-P(B\cap C)\\ &+P(A\cap B\cap C). \end{aligned} \]
The final triple-intersection term is added back because subtracting all three pairwise intersections removes outcomes shared by all three too many times.