# Why is $\Pr(X=E[X])$ small for a random variable $X$ that follows a geometric distribution with probability $p$?

I was experimenting around with some exercises for my computational statistics and started reading about how, for some random variable $$X$$ that follows a geometric distribution with probability $$p$$, the expected value of said variable is given by $$1\over p$$ (when counting attempts until, and including, the first success). This made some sense intuitively, considering that, if something has the probability of happening every 20 tries ($$p=\frac{1}{20}$$), then with good odds it should happen after 20 tries.

However, I thought about putting the value 20 into the formula for finding the probability of $$X = 20$$, considering the same $$p=\frac{1}{20}$$. Using the version of the formula where $$k$$ is the number of attempts until the first success:

$$\Pr(X=20) = (1-p)^{20-1}\cdot p$$

The result is a very small number. I was under the impression that, since the average would be $$E[X]=20$$, the probability of $$X$$ when close to said average would increase, not decrease. Why is this the case?

• Note that if $p$ is not the reciprocal of an integer then $\Pr(X=E[X])=0$ Commented May 24, 2021 at 22:39
• What exactly would be a reciprocal in this context @Henry? Commented May 24, 2021 at 22:44
• If you do not not want $\Pr(X=E[X])=0$ then you need $E[X]$ in $1,2,3,4,\ldots$ (i.e. an integer, since geometric random variables take integer values) and so $p$ in $1,\frac12,\frac13, \frac14\ldots$ (i.e. the reciprocal of an integer) Commented May 24, 2021 at 22:47

You're looking at the probability of succeeding in exactly 20 tries. Which of course should be a small number, because there is infinitely more ways to do it differently, each with a non-zero weight and summing up to 1 if we include the exactly 20 tries case. There is no reason why doing it in 21 tries or 19 tries should have a wildly different probability. In fact, they only differ by a factor of $$(1-1/20)$$ or a 5% difference. So, there is no peak in the distribution there. The distribution just goes smoothly down towards infinity.

The clash with your intuition gets resolved if you consider that you shouldn't look at the probabilty of exactly 20 tries, but rather at the probability of succeeding in 20 tries maximum. And this is given by

$$\Pr(X\leq20) = \sum_{k=1}^{20} (1-p)^{k-1}\cdot p \approx 0.6594$$

and this is quite a high probability. Larger than 50% even, which means the median of the distribution is lower.

• Oh, that makes a lot of sense! I also seem to have been mistaken in thinking that the expected value of a random variable would be exactly the median (because I spent a lot more time on normal distributions in this course), but it seems like it's not the case. Thank you for the explanation! Commented May 25, 2021 at 12:00
• I figured that was what you struggled with. If possible in those cases, you should try to visualize the distribution. Commented May 25, 2021 at 17:34

As you found, just you need to check why the following term is small:

$$f(p) = \mathbb{P}\left(X = \frac{1}{p}\right) = (1-p)^{\frac{1}{p}-1}p$$

Consider the limits when $$p$$ goes to zero. As $$\lim_{p\to 0}(1-p)^{\frac{1}{p}-1} = \frac{1}{e}$$, so the probability goes to zero.

If $$p$$ goes to one, we have $$\lim_{p\to 1}(1-p)^{\frac{1}{p}-1} = 1$$. So the probability goes to one.

So, your observation for small $$p$$s is correct, but wrong for near one probability.

Also, you can plot $$f(p)$$ to see the above observation. As, you can see $$f(p)$$ is strictly increasing from 0 to 1.

Note that, as Henry mentioned, the plot is only correct when $$p$$ is reciprocal of an integer.

• Shouldn't the plot be $(1-p)^{(1/p - 1)}$ instead of $(1-p)^{(1/p)}$? Commented May 24, 2021 at 22:42
• Your Wolfram Alpha link is for $p(1-p)^{\frac1p}$ rather than $p(1-p)^{\frac1p-1}$. The latter is maximised when $p=1$ giving $1$, and the next possible $p$ is $\frac12$ which would give $0.25$ since you can only use reciprocals of integers Commented May 24, 2021 at 22:43
• @Carmo: you're right. It's updated.
– OmG
Commented May 24, 2021 at 22:49
• @Henry thanks. updated.
– OmG
Commented May 24, 2021 at 22:49
• @Carmo Yes, exactly.
– OmG
Commented May 25, 2021 at 4:02

One way of see this at an intuitive level is to consider an exponentially distributed random variable $$Y$$ and then spot that rounding up $$X=\lceil Y \rceil$$ is a geometric random variable on $$1,2,3,\ldots$$.

You then get the same effect on $$Y$$ as you have observed on $$X$$, in that if $$Y$$ has mean $$\mu$$, the density at the mean is $$\frac1{\mu }e^{-1}$$ and this falls as $$\mu$$ increases. This is what you should expect, as increasing $$\mu$$ reduces the rate and stretches the distribution rightwards and so pulls it down vertically. In a handwaving sense, you are doing something similar to $$X$$.

• That is an interesting comparison. I had completely forgotten the relationship between the geometric and exponential distributions. Commented May 25, 2021 at 12:12