5
$\begingroup$

How can one use memorylessness and the law of total expectation and the law of total variance to find the expectation and variance of the geometric distribution?

I will post my own answer, but as always, that shouldn't stop anyone else from posting theirs.

$\endgroup$
3
$\begingroup$

Memorylessness means that either $X=0$, which happens with probability $p$, or that, with probability $1-p$, $X=1+X'$ where $X'$ has the same distribution as $X$. That is,

$$ X=U\cdot(1+X'),\qquad U\sim\mathrm{Ber}(1-p),\qquad U\ \text{independent of}\ X'. $$

This yields every moment of $X$, for example, $E[U]=1-p$ hence $$ E[X]=E[U]\cdot(1+E[X])\implies E[X]=\frac{E[U]}{1-E[U]}=\frac{1-p}p. $$ Likewise, $$ E[X^2]=E[U^2]\cdot E[(1+X)^2]=E[U]\cdot(1+2E[X]+E[X^2]), $$ which implies $$ E[X^2]=\frac{E[U]}{1-E[U]}\cdot\frac{1+E[U]}{1-E[U]}, $$ and $$ \mathrm{var}(X)=\frac{E[U]}{(1-E[U])^2}=\frac{1-p}{p^2}. $$ More generally, for every $x$ in $(0,1]$, $$ E[s^X]=E[s^{U(1+X)}]=p+(1-p)E[s^{1+X}]\implies E[s^X]=\frac{p}{1-(1-p)s}, $$ hence, differentiating $n$ times, $$ E[X(X-1)\cdots(X-n+1)s^X]=n!\,\frac{p(1-p)^n}{(1-(1-p)s)^{n+1}}, $$ and in particular, for $s=1$, $$ E[X(X-1)\cdots(X-n+1)]=n!\,\frac{(1-p)^n}{p^n}, $$ from which every moment of $X$ can be deduced.

$\endgroup$
3
$\begingroup$

$\newcommand{\var}{\operatorname{var}}$ $\newcommand{\E}{\mathbb E}$

I will consider the geometric distribution supported on the set $\{0,1,2,3,\ldots\}$. This is the distribution of the number $X$ of failures before the first success in a sequence of independent Bernoulli trials. Call the probability of success on each trial $p$.

Then $$ X = \begin{cases} 0 & \text{with probability }p \\ 1 & \text{with probability }p(1-p) \\ 2 & \text{with probability }p(1-p)^2 \\ 3 & \text{with probability }p(1-p)^3 \\ \vdots & {}\qquad \vdots \end{cases} $$

Memorylessness of this distribution means that $\Pr(X\ge w+x\mid X\ge w)=\Pr(X\ge x)$, i.e. the probability distribution of the number of remaining trials, given the number of failures so far, does not depend on the number of failures so far.

Let $\displaystyle A=\begin{cases} 1, & \text{if }X\ge 1 \\[6pt] 0, & \text{if }X=0. \end{cases}$

Then $$ \E(X) = E(E(X\mid A)) = E\left.\begin{cases} 0 & \text{if }A=0 \\ 1+\E(X) & \text{if }A=1 \end{cases}\right\} = p\cdot0+(1-p)(1+\E(X)). $$ Thus we have $$ \E(X) = 1-p+(1-p)\E(X). $$ Therefore $$ \E(X) = \frac{1-p}{p}. $$

Now the variance: $$ \var(X) = \var(\E(X\mid A)) + \E(\var(X\mid A)) $$ $$ = \var\left.\begin{cases} 0 & \text{if }A=0 \\ 1 + \E(X) & \text{if }A=1 \end{cases}\right\} + \E\left.\begin{cases} 0 & \text{if }A=0 \\ \var(X) & \text{if }A=1 \end{cases}\right\} $$ $$ = \var\left.\begin{cases} 0 & \text{if }A=0 \\ 1/p & \text{if }A=1 \end{cases}\right\} + p\cdot0 + (1-p)\var(X) $$ $$ = \frac{1-p}{p} + p\cdot0 + (1-p)\var(X) = (1-p)\left(\frac1p+\var(X)\right). $$ So we get $$ \var(X) = (1-p)\left(\frac1p+\var(X)\right). $$ Therefore $$ \var(X) = \frac{1-p}{p^2}. $$

$\endgroup$
  • $\begingroup$ How did you combine the fact that variance = 0 if A = 0 and variance = 1/p if A = 1 to get $\frac{(1-p)}{p}$? $\endgroup$ – 1110101001 Jun 11 '15 at 8:33
  • 1
    $\begingroup$ @1110101001 : Just after the word "Then", we have $\operatorname{E}(X) = \cdots = \cdots$, and there appears a sum of two terms, but one of those two terms is $0$. So we get $\operatorname{E}(X) = (1-p)(1+\operatorname{E}(X))$. That expands to $(1-p)+(1-p)\operatorname{E}(X)$. We now have $\operatorname{E}(X) = (1-p)+(1-p)\operatorname{E}(X)$, Subtracting that last term from both sides, we get $\operatorname{E}(X) - (1-p)\operatorname{E}(X) = (1-p)$. The left side of that equation simplifies to $p\operatorname{E}(X)$. Then we have $p\operatorname{E}(X)=1-p$. ${}\qquad{}$ $\endgroup$ – Michael Hardy Jun 11 '15 at 14:11
  • $\begingroup$ I was referring to the derivation for the variance, not the expected value -- you wrote $$ = \var\left.\begin{cases} 0 & \text{if }A=0 \\ 1/p & \text{if }A=1 \end{cases}\right\} + p\cdot0 + (1-p)\var(X) $$ How did you combine the two cases of variance into $\frac{1-p}{p}$ $\endgroup$ – 1110101001 Jun 11 '15 at 18:34
  • 1
    $\begingroup$ @1110101001 : Suppose $\displaystyle W=\left.\begin{cases} a & \text{with probability }p, \\ b & \text{with probability }1-p. \end{cases}\right\}$ Then $\operatorname{var}(W)=p(1-p)(a-b)^2$. Apply that to this instance. To see that that's what it is, do it first in the case where $a$ and $b$ are $0$ and $1$ (it doesn't matter which is which). Then recall that variance doesn't change if you add a constant to a random variable, and recall why happens to the variance when you multiply the random variable by a constant. ${}\qquad{}$ $\endgroup$ – Michael Hardy Jun 11 '15 at 19:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.