Definition

The random variables \(X\) and \(Y\) are independent if the events \(\{X \in A\}\) and \(\{Y \in B\}\) are independent for all subsets \(A\) and \(B\) in \(\mathbb{R}.\) In particular, \[P(X \in A, Y \in B) = P(X \in A) \cdot P(Y \in B)\]

Example: Suppose a coin is flipped twice. Let \(X\) be \(1\) if the first flip is heads and \(0\) if the first flip is tails. Similarly, let \(Y\) be \(1\) if the second flip is heads and \(0\) if the second flip is tails. Then \(X\) and \(Y\) are independent random variables.

The sample space in this example is \(\Omega = \{HH, HT, TH, TT\}.\) The probability is \(P(\omega) = \frac{1}{4}\) for every \(\omega \in \Omega.\) The random variables \(X\) and \(Y\) are defined by \[X(HH) = 1, X(HT) = 1, X(TH) = 0, X(TT) = 0\] \[Y(HH) = 1, Y(HT) = 0, Y(TH) = 1, Y(TT) = 0\] The only important subsets of \(\mathbb{R}\) to check are \(\{0\}\) and \(\{1\}\) since these are the only points on which \(X\) and \(Y\) have non-zero probability. As an example, we check \(A = \{0\}\) and \(B = \{0\}.\)

First, compute the left hand side. \[P(X \in A, Y \in B) = P(X = 0, Y = 0) = P(\{TH, TT\} \cap \{HT, TT\}) = P(\{TT\}) = \frac{1}{4}\] Now check that the right hand side is the same. \[P(X \in A)P(Y \in B) = P(\{TH, TT\})P(\{HT, TT\}) = \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}\] So, when \(A = \{0\}\) and \(B = \{0\},\) \(P(X \in A, Y \in B) = P(X \in A)P(Y \in B).\)

Similar computations work for \(A = \{0\}\) and \(B = \{1\},\) \(A = \{1\}\) and \(B = \{0\},\) and \(A = \{1\}\) and \(B = \{1\}.\)

Collections of Independent Random Variables

A collection of random variables \((X_i : i \in I)\) over some index set \(I\) is said to be independent if for every finite subset \(J \subset I\) and any subsets of \(\mathbb{R}\) indexed over \(J,\) \((A_j : j \in J),\) \[P\left(\bigcap_{j \in J}\{X_j \in A_j\}\right) = \prod_{j \in J}P(X_j \in A_j)\]