Expectation and variance of the geometric distribution How can one use memorylessness and the law of total expectation and the law of total variance to find the expectation and variance of the geometric distribution?
I will post my own answer, but as always, that shouldn't stop anyone else from posting theirs.
 A: $\newcommand{\var}{\operatorname{var}}$ $\newcommand{\E}{\mathbb E}$
I will consider the geometric distribution supported on the set $\{0,1,2,3,\ldots\}$.  This is the distribution of the number $X$ of failures before the first success in a sequence of independent Bernoulli trials.  Call the probability of success on each trial $p$.
Then
$$
X = \begin{cases} 0 & \text{with probability }p \\
1 & \text{with probability }p(1-p) \\
2 & \text{with probability }p(1-p)^2 \\
3 & \text{with probability }p(1-p)^3 \\
\vdots & {}\qquad \vdots \end{cases}
$$
Memorylessness of this distribution means that $\Pr(X\ge w+x\mid X\ge w)=\Pr(X\ge x)$, i.e. the probability distribution of the number of remaining trials, given the number of failures so far, does not depend on the number of failures so far.
Let $\displaystyle A=\begin{cases} 1, & \text{if }X\ge 1 \\[6pt] 0, & \text{if }X=0. \end{cases}$
Then
$$
\E(X) = E(E(X\mid A)) = E\left.\begin{cases} 0 & \text{if }A=0 \\ 1+\E(X) & \text{if }A=1 \end{cases}\right\} = p\cdot0+(1-p)(1+\E(X)).
$$
Thus we have
$$
\E(X) = 1-p+(1-p)\E(X).
$$
Therefore
$$
\E(X) = \frac{1-p}{p}.
$$
Now the variance:
$$
\var(X) = \var(\E(X\mid A)) + \E(\var(X\mid A))
$$
$$
= \var\left.\begin{cases} 0 & \text{if }A=0 \\  1 + \E(X) & \text{if }A=1 \end{cases}\right\} + \E\left.\begin{cases} 0 & \text{if }A=0 \\  \var(X) & \text{if }A=1 \end{cases}\right\}
$$
$$
= \var\left.\begin{cases} 0 & \text{if }A=0 \\  1/p & \text{if }A=1 \end{cases}\right\} + p\cdot0 + (1-p)\var(X)
$$
$$
= \frac{1-p}{p} + p\cdot0 + (1-p)\var(X) = (1-p)\left(\frac1p+\var(X)\right).
$$
So we get
$$
\var(X) = (1-p)\left(\frac1p+\var(X)\right).
$$
Therefore
$$
\var(X) = \frac{1-p}{p^2}.
$$
A: Memorylessness means that either $X=0$, which happens with probability $p$, or that, with probability $1-p$, $X=1+X'$ where $X'$ has the same distribution as $X$. That is,

$$
X=U\cdot(1+X'),\qquad U\sim\mathrm{Ber}(1-p),\qquad U\ \text{independent of}\ X'.
$$

This yields every moment of $X$, for example, $E[U]=1-p$ hence
$$
E[X]=E[U]\cdot(1+E[X])\implies E[X]=\frac{E[U]}{1-E[U]}=\frac{1-p}p.
$$
Likewise,
$$
E[X^2]=E[U^2]\cdot E[(1+X)^2]=E[U]\cdot(1+2E[X]+E[X^2]),
$$
which implies
$$ 
E[X^2]=\frac{E[U]}{1-E[U]}\cdot\frac{1+E[U]}{1-E[U]},
$$
and
$$
\mathrm{var}(X)=\frac{E[U]}{(1-E[U])^2}=\frac{1-p}{p^2}.
$$
More generally, for every $x$ in $(0,1]$,
$$
E[s^X]=E[s^{U(1+X)}]=p+(1-p)E[s^{1+X}]\implies E[s^X]=\frac{p}{1-(1-p)s},
$$
hence, differentiating $n$ times,
$$
E[X(X-1)\cdots(X-n+1)s^X]=n!\,\frac{p(1-p)^n}{(1-(1-p)s)^{n+1}},
$$
and in particular, for $s=1$,
$$
E[X(X-1)\cdots(X-n+1)]=n!\,\frac{(1-p)^n}{p^n},
$$
from which every moment of $X$ can be deduced.
