# Most and least likely outcomes of a Bernoulli distribution experiment

Let's say I have some experiment that I repeat $n$ times. Each time I repeat it, there is a $p$ chance that the outcome will be successful. If I set $X$ to be the amount of successful trials, how can I find the most probable and least probable $X$?

I figured that the most probable would be $np$, but I'm not quite sure.

-
Compute the ratio $P(X=k+1)/P(X=k)$, in simplified form. –  André Nicolas Feb 4 '12 at 19:14

Suppose $X$ is a Binomial variable with parameters $n$ and $p$, as you have.

The probability mass function of $X$ is: $$p_X(k)= {n\choose k} \cdot p^k (1-p)^{n-k}= {n!\over (n-k)! k!} p^k (1-p)^{n-k},\quad k=0,1,2,\ldots,n.$$

We now derive a recurrence relation for the probability mass function of $X$. We have: \eqalign{ {P(X=k+1)\over P(X=k)}&= { {n!\over (n-(k+1))! (k+1)!} p^{k+1} (1-p)^{n-(k+1)} } \cdot { (n-k)! k!\over n!} {1\over p^k (1-p)^{n-k}}\cr &= {n-k\over k+1} {p\over(1-p)} } Thus, we have the

Recursion Formula for a Binomial Variable

If $X$ is a Binomial variable with parameters $n$ and $p$, then $$P(X=k+1)= {n-k\over k+1} {p\over(1-p)} P(X=k).$$

It follows from the recursion formula that $p_X(k+1)\ge p_X(k)$ if and only if $${n-k\over k+1}\ge{1-p\over p }.$$ This holds if and only if $$p(n-k) \ge(1-p)(k+1).$$ This holds if and only if $$np \ge (1-p)(k+1) +pk$$ But $$(1-p)(k+1) +pk = k +1 -p.$$ Thus $p_X(k+1)\ge p_X(k)$ if and only if $$np \ge k+1-p.$$ So $p_X(k+1)\ge p_X(k)$ if and only if $$k\le p(n+1)-1.$$

Note, then, that $p_X(k)$ is increasing until it reaches a maximum value at the point when when $k$ the largest integer less than or equal to $(n+1)p$, and then decreases.

The minimum value of $p_X(k)$ is the smaller of $p_X(n)=p^n$ and $p_X(0)=(1-p)^n$.

Note $np$ is the expected value of $X$.

-

You're close, but of course $np$ is not always an integer, which causes a problem.

Let $f(k) = {n \choose k} p^k (1-p)^{n-k}$ be the probability that there are $k$ successes in your experiment. Then look at the ratio $f(k)/f(k-1)$. You get:

$${f(k) \over f(k-1)} = {{n \choose k} \over {n \choose k-1}} {p^k \over p^{k-1}} {(1-p)^{n-k} \over (1-p)^{n-k+1}}$$

and cancelling powers of $p$ and $1-p$, this is

$${{n \choose k} \over {n \choose k-1}} {p \over 1-p}.$$

Now write the ratio of binomial coefficients in terms of factorials:

$${{n \choose k} \over {n \choose k-1}} = {n! \over k! (n-k)!} {(k-1)! (n-k+1)! \over n!} = {n! \over n!} {(k-1)! \over k!} {(n-k+1)! \over (n-k)!} = {n-k+1 \over k}.$$

Therefore, the original ratio we were interested in is

$${f(k) \over f(k-1)} = {n-k+1 \over k} {p \over 1-p}.$$

This decreases as $k$ increases. In particular, as $k$ increases it goes from being greater than $1$ to less than $1$, and so the sequence $f(0), f(1), \ldots, f(n)$ first increases and then decreases. (This is called being "unimodal".) Now, $f(k) \ge f(k-1)$ if and only if $${n-k+1 \over k} {p \over 1-p} \ge 1$$ and we'll solve this inequality for k. Clear denominators: $$(n-k+1)p \ge k(1-p)$$ Expand: $$np-kp+p \ge k-kp$$ Cancel like terms: $$np+1 \ge k$$ or $k \le np+p = (n+1)p$. So $f(k)$ increases until $k$ reaches $\lfloor (n+1)p \rfloor$ and then decreases.

For example, consider $k = 1/3, p = 6$. Our formula tells us that $f(k) \ge f(k-1)$ if and only if $k \le (6+1)/3$. So we'll have $f(2) \ge f(1)$ but $f(3) \le f(2)$; that is, 2 is th4e most likely number of successed. And indeed the probabilities $f(0), \ldots, f(6)$ are, in order, $0.088, 0.263, 0.329, 0.219, 0.082, 0.016, 0.001$, confirming the result.

But there's a special case when $(n+1)p$ is an integer. Then if $k = (n+1)p$ we actually get $f(k)/f(k-1) = 1$ -- so $(n+1)p$ and $(n+1)p-1$ are both maxima. The easiest example is when $p = 1/2$ and $n$ is odd -- say $n = 5$ -- then this predicts that $2$ and $3$ are both maxima, which they are. And of course if you flip five coins you expect two and three heads to be equally likely and more likely than any other number.

Since $f(0), f(1), f(2), \ldots, f(n)$ is increasing and then decreasing, the least likely number of successes is either $0$ or $n$. It's $0$ if $p > 1/2$ and $n$ if $p < 1/2$; if $p = 1/2$ then both are equally likely.

-