A Bernoulli random variable is a random variable \(X\) with the distribution \begin{align} & P(X = 1) = p \\ & P(X = 0) = 1-p \end{align} where \(p\) is a number between \(0\) and \(1.\) We will usually say \(X\) is Bernoulli\((p).\)
A Bernoulli random variable can be thought of as the outcome of an experiment. The outcome is a success with probability \(p,\) where we say the experiment is successful if \(X = 1.\) The experiment fails, or \(X = 0,\) with probability \(1-p.\)
Let \(X\) be Bernoulli\((p)\). The expected value of \(X\) is \begin{align} E[X] & = 1 \cdot p + 0 \cdot (1-p) \\ & = p \end{align} The variance of \(X\) is \begin{align} \text{Var}[X] & = E[X^2] - E[X]^2 \\ & = 1^2 \cdot p + 0^2 \cdot (1-p) - p^2 \\ & = p-p^2 \\ & = p(1-p) \end{align}
A binomial random variable is a random variable \(X\) that takes values from \(0\) to \(n\) with the following probabilities: \begin{align} P(X = k) = {n \choose k} p^k(1-p)^{n-k} \end{align} where \(0 \leq i \leq n\) is an integer and \(0 \leq p \leq 1.\) We will write this distribution as Binomial\((n, p).\)
Let \(X\) have Binomial\((n,p)\) distribution, and let \(Y_1, Y_2, \dots, Y_n\) be independent random variables with Bernoulli\((p)\) distribution. Then for every integer \(k,\) \(0 \leq k \leq n,\)
\[P(X = k) = P(Y_1 + Y_2 + \dots + Y_n = k)\]
In other words, \(Y_1 + Y_2 + \dots + Y_n\) has Binomial\((n, p)\) distribution.
To show this is true, we need to show
\[P(Y_1 + Y_2 + \dots + Y_n = k) = {n \choose k} p^k(1-p)^{n-k}\]
for \(1 \leq k \leq n.\)
First, suppose \(Y_1 = Y_2 = \dots = Y_k = 1\) and \(Y_{k+1} = Y_{k+2} = \dots = Y_n = 0.\) Then \(Y_1 + Y_2 + \dots + Y_n = k.\) Also, the probability that this happens is
\begin{align}
P(Y_1 = 1, \dots, Y_k = 1, Y_{k+1} = 0, \dots, Y_n = 0) & = \prod_{i=1}^k P(Y_i = 1) \cdot \prod_{i = k+1}^n P(Y_i = 0) \\
& = p^k(1-p)^{n-k}
\end{align}
In general, for \(Y_1 + Y_2 + \dots + Y_n = k\) to happen, exactly \(k\) of the \(Y_i\)'s must be 1 and the other \(n-k\) must be \(0.\) One such way is when the first \(k\) of the \(Y_i\)'s are \(1\) as shown above. There are \({n \choose k}\) ways for exactly \(k\) of the \(Y_i\)'s to be \(1\), and each way has a probability of \(p^k (1-p)^{n-k}\) of occuring. Thus,
\[P(Y_1 + Y_2 + \dots + Y_n = k) = {n \choose k} p^k(1-p)^{n-k}\]
The expected value and variance of a Binomial\((n, p)\) can be easily found by using the representation as a sum of Bernoulli\((p)\) random variables. Let \(Y_1, Y_2, \dots, Y_n\) be indpendent Bernoulli\((p)\) random variables. Then \[E\left[\sum_{i=1}^n Y_i\right] = \sum_{i=1}^n E[Y_i] = \sum_{i=1}^n p = np\] Also, because the variables are independent, \[\text{Var}\left[\sum_{i=1}^n Y_i\right] = \sum_{i=1}^n Var[Y_i] = \sum_{i=1}^n p(1-p) = np(1-p)\] So, if \(X\) has a Binomial\((n, p),\) \[E[X] = np, \text{Var}(X) = np(n-p)\]
A geometric random variable is a random variable \(X\) with pmf \[P(X = k) = (1-p)^{k-1}p\] for every integer \(k \geq 1.\) We will write this distribution as Geometric\((p).\)
Let \(Y_1, Y_2, \dots\) be a sequence of independent random variables with Bernoulli\((p)\) distributions. Let \(N\) be the number of the first successful trial. \[N = \text{min}\{n : Y_n = 1\}\] Then \(N\) has the Geometric\((p)\) distribution. So, a random variable with Geometric\((p)\) distribution as the number of independent trials you would need to run until the first successful trial.
The expected value and variance can be computed directly. Let \(X\) have Geometric\((p)\) distribution. \begin{align} E[X] & = \sum_{k = 1}^\infty kP(X = k) \\ & = \sum_{k = 1}^\infty k(1-p)^{k-1} p \\ & = p\sum_{k = 1}^\infty k(1-p)^{k-1} \\ & = p \sum_{k = 1}^\infty \frac{d}{dp}(-(1-p)^k) \\ & = -p \frac{d}{dp} \sum_{k = 1}^\infty (1-p)^k \\ & = -p \frac{d}{dp} \frac{1-p}{p} \\ & = -p \cdot \frac{-1}{p^2} \\ & = \frac{1}{p} \\ \end{align} Finding the variance can be done in a similar way. \begin{align} E[X^2] & = \sum_{k = 1}^\infty k^2 P(X = k) \\ & = \sum_{k = 1}^\infty k^2 (1-p)^{k-1} p \\ & = \sum_{k = 1}^\infty (1 + k - 1)k (1-p)^{k-1} p \\ & = \sum_{k = 1}^\infty k (1-p)^{k-1} p + (k - 1)k (1-p)^{k-1} p \\ & = E[X] + \sum_{k = 2}^\infty (k - 1)k (1-p)^{k-1} p \\ & = \frac{1}{p} + (1-p)p\sum_{k = 2}^\infty (k - 1)k (1-p)^{k-2} \\ & = \frac{1}{p} + (1-p)p\sum_{k = 2}^\infty \frac{d^2}{dp^2} (1-p)^k \\ & = \frac{1}{p} + (1-p)p \frac{d^2}{dp^2}\sum_{k = 2}^\infty (1-p)^k \\ & = \frac{1}{p} + (1-p)p \frac{d^2}{dp^2}\frac{(1-p)^2}{p} \\ & = \frac{1}{p} + (1-p)p \cdot \frac{2}{p^3} \\ & = \frac{1}{p} + \frac{2(1-p)}{p^2} \\ & = \frac{2-p}{p^2} \\ \end{align} So, \[\text{Var}(X) = E[X^2] - E[X]^2 = \frac{2-p}{p^2} - \frac{1}{p^2} = \frac{1-p}{p^2}\] In summary, if \(X\) is Geometric\((p),\) \[E[X] = \frac{1}{p}, \text{Var}(X) = \frac{1-p}{p^2}\]
Check your understanding:
1. A random variable has a Bernoulli distribution with parameter \(p = 0.7.\) What is the variance of the random variable?
Unanswered
2. If \(Y\) is Binomial\((5,0.2),\) then what is \(P(Y \leq 2)?\)
Unanswered
3. Let \(X\) be Binomial\((4,0.6)\) and \(Y\) be Geometric\((0.2).\) Find \(E[X+Y].\)
Unanswered
4. A student taking a math exam has a \(12\%\) chance of making an arithmetic mistake on any given problem. What is the probability the student makes an arithmetic mistake on the first \(3\) problems?
Unanswered