I have a Geometric Distribution, where the stochastic variable $X$ represents the number of failures before the first success.

The distribution function is $P(X=x) = q^x p$ for $x=0,1,2,\ldots$ and $q = 1-p$.

Now, I know the definition of the expected value is: $E[X] = \sum_{i}{x_i p_i}$

So, I proved the expected value of the Geometric Distribution like this:

$E[X]=\sum _{ i=0 }^{ \infty }{ iP(X=i) } = \sum _{i=0}^{\infty}{i q^i p} = p\sum _{i=0}^{\infty}{i q^i} = pq \sum _{i=0}^{\infty}{iq^{i-1}}$

$\qquad = pq \sum _{i=0}^{\infty}{\frac{d}{dq}q^i} = pq \frac{d}{dq}(\sum _{i=0}^{\infty}{q^i}) = pq \frac{d}{dq}(\frac{1}{1-q})$

$\qquad = pq \frac{1}{(1-q)^2} = \frac{pq}{p^2} = \frac{q}{p}$

So now, I would like to prove that $Var[X] = \frac{q}{p^2}$. I know I have to use a simular trick as above (with the derivation).

$Var[X] = E[X^2] - E[X]^2 = \sum _{i=0}^{\infty}{i^2 q^i p} - (\frac{q}{p})^2 = p \sum _{i=0}^{\infty}{i^2 q^i} - (\frac{q}{p})^2 = pq \sum _{i=0}^{\infty}{i^2 q^{i-1}} - (\frac{q}{p})^2$

$\qquad = pq \sum _{i=0}^{\infty}{\frac{d}{dq}i q^i} - (\frac{q}{p})^2 = pq \frac{d}{dq} \sum _{i=0}^{\infty}{iq^i}-(\frac{q}{p})^2$

Then I'm stuck. How can I get another $q$ out of the sum? Won't it mess up the first derivation?


I have a proof which follows the approach of @Math1000 but it in a slightly different way. It may be useful if you're not familiar with generating functions.

However, I'm using the other variant of geometric distribution. In my case $X$ is the number of trials until success. Therefore $E[X]=\frac{1}{p}$ in this case. Anyways both variants have the same variance.

So assuming we already know that $E[X]=\frac{1}{p}$. Then the variance can be calculated as follows: $$ Var[X]=E[X^2]-(E[X])^2=\boxed{E[X(X-1)]} + E[X] -(E[X])^2 = \boxed{E[X(X-1)]} + \frac{1}{p} - \frac{1}{p^2} $$ So the trick is splitting up $E[X^2]$ into $E[X(X-1)]+E[X]$, which is easier to determine. To determine $\boxed{E[X(X-1)]}$ we have to determine the value of the following series for $p\in(0,1)$: $$ \sum_{k=1}^\infty k(k-1)p(1-p)^{k-1} $$

Here's how it can be done (as an alternative to Math1000's approach): $$ \begin{align} \sum_{k=1}^\infty k(k-1)p(1-p)^{k-1} &= p\sum_{k=1}^\infty k(k-1)(1-p)^{k-1} \qquad\text{Subst. }q:=(1-p)\\\\ &= p\sum_{k=1}^\infty (k-1)kq^{k-1} \\\\ &= p\frac{d}{dq}\left(\sum_{k=1}^\infty (k-1)q^k\right) \\\\ &= p\frac{d}{dq}\left(q^2\sum_{k=1}^\infty (k-1)q^{k-2}\right) \\\\ &= p\frac{d}{dq}\left(q^2\sum_{k=2}^\infty (k-1)q^{k-2}\right) \\\\ &= p\frac{d}{dq}\left(q^2\frac{d}{dq}\left(\sum_{k=2}^\infty q^{k-1}\right)\right) \\\\ &= p\frac{d}{dq}\left(q^2\frac{d}{dq}\left(\sum_{k=1}^\infty q^{k}\right)\right) \\\\ &= p\frac{d}{dq}\left(q^2\frac{d}{dq}\left(\frac{1}{1-q}-1\right)\right) \\\\ &= p\frac{d}{dq}\left(\frac{q^2}{(1-q)^2}\right) \\\\ &= p\left(\frac{-2q}{(q-1)^3}\right)\qquad\text{Backsub. }q=(1-p) \\\\ &= p\left(\frac{-2(1-p)}{((1-p)-1)^3}\right) = p\left(\frac{-2+2p}{-p^3}\right) \\\\ &= \frac{-2+2p}{-p^2} =\frac{2(p-1)}{-p^2} = \frac{2(1-p)}{p^2}. \\\\ \end{align} $$ Now putting the result back into the equation for $Var[X]$ gives us: $$ Var[X]=\boxed{E[X(X-1)]} + E[X] -(E[X])^2 =\frac{2(1-p)}{p^2} + \frac{1}{p} - \frac{1}{p^2} = \frac{2-2p+p-1}{p^2} = \frac{1-p}{p^2}. $$


No answer to your question but a suggestion to follow an alternative route (too much for a comment).

Let $S$ denote the event that the first experiment is a succes and let $F$ denote the event that the first experiment is a failure. Then make use of: $$\mathbb EX^n=\mathbb E(X^n|S)P(S)+\mathbb E(X^n|F)P(F)=\mathbb E(1+X)^nq$$ This for $n=1$ and $n=2$ respectivily.

It leads to expressions for $\mathbb EX$, $\mathbb EX^2$ and consequently $\text{Var}X=\mathbb EX^2-(\mathbb EX)^2$.


$\mathbb E[X] = \frac{1-p}p$ as you computed above. Here is a trick to make the computation of $\mathrm{Var}(X)$ easier: $$ \mathrm{Var}(X) = \mathbb E[X^2] - \mathbb E[X]^2 = \mathbb E[X(X-1)] + \mathbb E[X] - \mathbb E[X]^2. $$ Since the generating function of $X$ is $$\begin{align*} P(s) &:= \mathbb E\left[s^X\right]\\ &= \sum_{n=0}^\infty (1-p)^n p s^n\\ &= p\sum_{n=0}^\infty ((1-p)s)^n\\ &= \frac p{1-(1-p)s}, \end{align*}$$ we have $$\begin{align*} \mathbb E[X(X-1)] &= \lim_{s\uparrow1} P''(s)\\ &= \lim_{s\uparrow1} \frac{2p(1-p)^2}{\left(1-(1-p)s\right)^3}\\ &= \frac{2p(1-p)^2}{p^3}\\ &= \frac{2(1-p)^2}{p^2} \end{align*}$$ Hence $$\begin{align*} \mathrm{Var}(X) &= \frac{2(1-p)^2}{p^2} +\frac{1-p}p - \left(\frac{1-p}p\right)^2\\ &= \frac{2(1-p)^2 + p(1-p) -(1-p)^2 }{p^2}\\ &= \frac{(1-p)(1-p+p)}{p^2}\\ &= \frac{1-p}{p^2}. \end{align*}$$


I just happened to see it later, but actually you were really close. You just have to use the derivation-trick another time. So continuing from where you've been you'd do: $$ \begin{align*} \ldots &= pq \frac{d}{dq}\left[ \sum _{i=0}^{\infty}{iq^i}\right]-(\frac{q}{p})^2 \\ &= pq \frac{d}{dq}\left[ \sum _{i=0}^{\infty}q{iq^{i-1}}\right]-(\frac{q}{p})^2 \\ &= pq \frac{d}{dq}\left[ q\sum _{i=0}^{\infty}\frac{d}{dq}\left[q^{i}\right]\right]-(\frac{q}{p})^2 \\ &= pq \frac{d}{dq}\left[ q\frac{d}{dq}\left[\sum _{i=0}^{\infty}{q^{i}}\right]\right]-(\frac{q}{p})^2 \\ &=\ldots \end{align*} $$ Then you just continue as follows:

  1. Use the rule for the geometric series
  2. Derivate with respect to $q$
  3. Derivate another time with respect to $q$
  4. Then you just have to collect the terms and you should get there.

It's very similar to the proof of $\mathbb E[X] = \frac{q}{p}$ that you've already worked out.

Given $X \sim \mathcal{Geo}(k;\ p)\ ,where\ k \in \{0, 1, 2, 3, ..., K\} and\ p \in (0,1]$, below is the the proof of $Var[X]$:

$$\begin{align} Var[X] &= E[X^2]-E[X]^2 \\ \text{linearity of expectation:}\qquad &= E[X(X-1)] + E[X] - E[X]^2 \\ \text{law of the unconscious statistician:}\qquad &= \sum_{k=0}^\infty k(k-1) \ \mathcal{Geo}(k;\ p)+E[X]-E[X]^2 \\ \text{If we let } \gamma =E[X]-E[X]^2 \text{ and }q=1-p:\qquad &=\sum_{k=0}^\infty\ k(k-1)\ pq^{k}+\gamma \\ &=pq^2\sum_{k=0}^\infty\ k(k-1)q^{k-2}+\gamma \\ &=pq^2\sum_{k=0}^\infty\ \frac{\partial^2}{\partial q^2}(\int_0^1k(k-1)q^{k-2}\ dq^2)+\gamma \\ \text{power rule of second order derivative:}\qquad &=pq^2\sum_{k=0}^\infty\ \frac{\partial^2}{\partial q^2}q^k+\gamma \\ \text{linearity of differentiation:}\qquad &=pq^2\frac{\partial^2}{\partial q^2}\sum_{k=0}^\infty\ q^k+\gamma \\ \text{recall the sum of geometric series}\sum_{k=0}^\infty\ q^k=\frac{1-q^{k+1}}{1-q}:\qquad \lim_{k \to \infty}&=pq^2\frac{\partial^2}{\partial q^2}\frac{1-q^{k+1}}{1-q}+\gamma \\ &=pq^2\frac{\partial^2}{\partial q^2}\frac{1}{1-q}+\gamma \\ &=pq^2\frac{2}{(1-q)^3}+\gamma \\ &=q^2\frac{2}{p^2}+\frac{q}{p}-\frac{q^2}{p^2} \\ &=\frac{2q^2+pq-q^2}{p^2} \\ &=\frac{q(q+p)}{p^2} \\ &=\frac{1-p}{p^2} \end{align}$$ proved.


It's my first post here, so please forgive the mess.

You can use moment generating function to find the first two moments of the distribution, i.e. $E[x]$ and $E[x^2]$.

First, let's see what the mgf for the geometric distribution looks like. By the definition: $$M(t) = \sum_{x=0} e^{tx}p(x)$$ Therefore, $$M(t) = \sum_{x=0} e^{tx}q^xp = p\sum_{x=0}(qe^t)^x$$ Now, assuming $qe^t < 1$, the infinite sum simplifies to $\frac{1}{1-qe^t}$ and we get: $$M(t) = \frac{p}{1-qe^t}$$

By definition, $$E[X^r] = M^{(r)}(0)$$ First, $$M'(t) = -\frac{p}{(1-(1-p)e^t)^2}(-(1-p))e^t = \frac{p(1-p)e^t}{(1-(1-p)e^t)^2}$$ And, $$E[X] = M'(0) = \frac{p(1-p)}{p^2} = \frac{1-p}{p}$$ Now, let's calculate the second derivative of the mgf w.r.t $t$: $$M''(t) = \frac{p(p-1)\left[(p-1)e^t-1\right]e^t}{((p-1)e^t+1)^3}$$ and $$E[X^2] = M''(0) = \frac{(p-2)(p-1)}{p^2}$$ And finally: $$Var[X] = E[X^2] - (E[X])^2 = \frac{(1-p)\left[(2-p) - (1-p)\right]}{p^2} = \frac{1-p}{p^2}$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.