I am taking a course in probability and I have trouble computing the variance of a random variable.

There are 2 cases we saw in class that I would like to understand:

First I will state the definition of variance of expected value:

  • If $X$ is a random variable who's values are in $\mathbb{N}$, then $$\mathbb{E}(X) = \Sigma_{n≥0}\mathbb{P} (X > n)$$

  • If $X$ is a random variable and $\mathbb{E}(X)$ exists, then the variance of X is: $$Var(X) = \mathbb{E}\Big((X-\mathbb{E}(X))^2\Big)$$

Now here are the examples I'd like to understand:

  1. binomial distribution: If $X$ is a random variable that follows a binomial distribution of parametres $n$ and $p$ then we can write $X = X_1+X_2+X_3+...+X_n$ where the $X_i$'s are bernoulli variables of parametre $p$. Then $$Var(X) = np(1-p)$$

  2. negative binomial distribution (Pascal law): If X is a random variable that follows a pascal law of parameters $r$ and $p$ the $X+r= X_1+...+X_r$ where $X_i$'s are independant geometric variables. Then $$Var(X)=Var(X+r) = r\dfrac{1-p}{p^2}$$

I don't understand how one comes up with such results using the defintion of variance.

  • 1
    $\begingroup$ 1 ) By using $\text{Var}(X_1+\cdots+X_n)=\text{Var}X_1+\cdots+\text{Var}X_n$. This is not true in general, but it is true if the $X_i$ are uncorrelated (wich is the case if they are independent). Secondly $Var{X_i}=p(1-p)$ for every $i$ (wich is easy to find) so the summation gives $np(1-p)$. Sortlike on 2) with the addition that $\text{Var}(X+r)=\text{Var}(X)$ if $r$ is a constant. $\endgroup$ – drhab Nov 23 '16 at 13:15
  • 1
    $\begingroup$ A useful formula for variance is $V (X)=E (X^2)-(E (X))^2$ $\endgroup$ – Dave Nov 23 '16 at 13:20
  • $\begingroup$ Showing that the above formula is equivalent to the definition of variance is a good exercise, as well. $\endgroup$ – Math1000 Nov 23 '16 at 14:10

If $X$ is Bernoulli, with $P(\text{Success}) = p,$ then $E(X) = (0)(1-p) + 1p = p.$ Because $0^2 = 0$ and $1^2 = 1$, we also have $E(X^2) = p.$ From @Dave's hint $Var(X) = p - p^2 = p(1-p).$

Binomial: Then for $Y \sim Binom(n, p),$ we have $$E(Y) = E\left(\sum_{i=1}^n X_i \right) = \sum_{i=1}^n E(X_i) = np.$$

Also, direct proofs, with $$E(Y) = \sum_{i=0}^n i{n \choose i}p^i(1-p)^{n-i} = \sum_{i=1}^n i{n \choose i}p^i(1-p)^{n-i} = \cdots = np(1) = np,$$ (with a change in index $j = i-1$) are given in many elementary texts.

Similarly, using independence, $$Var(Y) = Var\left(\sum_{i=1}^n X_i \right) = \sum_{i=1}^n Var(X_i) = np(1-p).$$

Again here, proofs using about the same method as above to find $E(Y(Y-1)),$ and then $Var(Y)$ from that and $E(Y),$ are given in many elementary texts.

For the negative binomial, it seems intuitive that the avarage waiting time for the $r$th Success should be $r/p.$ A rigorous derivation of the expectation of a negative binomial random variable often uses some sort of trick involving differentiation of a sum. Probably the easiest route is to find the moment generating function and differentiate it to get the mean and variance.

Note: You may find some useful details on Wikipedia 'negative binomial distribution'. But if you are looking at various references on negative binomial, be aware that some parameterizations count trials until the $r$th Success and others count only the Failures before the $r$th Success.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.