Why is $\Pr(X=E[X])$ small for a random variable $X$ that follows a geometric distribution with probability $p$?

Question

I was experimenting around with some exercises for my computational statistics and started reading about how, for some random variable $X$ that follows a geometric distribution with probability $p$, the expected value of said variable is given by $1\over p$ (when counting attempts until, and including, the first success). This made some sense intuitively, considering that, if something has the probability of happening every 20 tries ($p=\frac{1}{20}$), then with good odds it should happen after 20 tries.

However, I thought about putting the value 20 into the formula for finding the probability of $X = 20$, considering the same $p=\frac{1}{20}$. Using the version of the formula where $k$ is the number of attempts until the first success:

$$ \Pr(X=20) = (1-p)^{20-1}\cdot p $$

The result is a very small number. I was under the impression that, since the average would be $E[X]=20$, the probability of $X$ when close to said average would increase, not decrease. Why is this the case?

Answer 1

You're looking at the probability of succeeding in exactly 20 tries. Which of course should be a small number, because there is infinitely more ways to do it differently, each with a non-zero weight and summing up to 1 if we include the exactly 20 tries case. There is no reason why doing it in 21 tries or 19 tries should have a wildly different probability. In fact, they only differ by a factor of $(1-1/20)$ or a 5% difference. So, there is no peak in the distribution there. The distribution just goes smoothly down towards infinity.

The clash with your intuition gets resolved if you consider that you shouldn't look at the probabilty of exactly 20 tries, but rather at the probability of succeeding in 20 tries maximum. And this is given by

$$\Pr(X\leq20) = \sum_{k=1}^{20} (1-p)^{k-1}\cdot p \approx 0.6594$$

and this is quite a high probability. Larger than 50% even, which means the median of the distribution is lower.

Answer 2

As you found, just you need to check why the following term is small:

$$ f(p) = \mathbb{P}\left(X = \frac{1}{p}\right) = (1-p)^{\frac{1}{p}-1}p $$

Consider the limits when $p$ goes to zero. As $\lim_{p\to 0}(1-p)^{\frac{1}{p}-1} = \frac{1}{e}$, so the probability goes to zero.

If $p$ goes to one, we have $\lim_{p\to 1}(1-p)^{\frac{1}{p}-1} = 1$. So the probability goes to one.

So, your observation for small $p$s is correct, but wrong for near one probability.

Also, you can plot $f(p)$ to see the above observation. As, you can see $f(p)$ is strictly increasing from 0 to 1.

Note that, as Henry mentioned, the plot is only correct when $p$ is reciprocal of an integer.

Answer 3

One way of see this at an intuitive level is to consider an exponentially distributed random variable $Y$ and then spot that rounding up $X=\lceil Y \rceil$ is a geometric random variable on $1,2,3,\ldots$.

You then get the same effect on $Y$ as you have observed on $X$, in that if $Y$ has mean $\mu$, the density at the mean is $\frac1{\mu }e^{-1}$ and this falls as $\mu$ increases. This is what you should expect, as increasing $\mu$ reduces the rate and stretches the distribution rightwards and so pulls it down vertically. In a handwaving sense, you are doing something similar to $X$.