9
$\begingroup$

Suppose that I perform a stochastic task $n$ times (like tossing a coin) and that $p$ is the probability that one of the possible outcomes. If $K$ is the stochastic variable that measures how many times this outcome occurred during my experiment, then the probability that $K$ is equal to a specific $k$, with $0\leq k\leq n$, is $$\mathrm{Pr}\{K=k\} = \binom n k p^k (1-p)^k =: b_n(k), $$ which makes intuitive sense.

Now suppose that I want to know what the average, or expected, value of $K$ is going to be: the formulae tell me that $$\begin{split} \langle K\rangle &= \sum_{k=0}^n k b_n(k) = \sum_{k=0}^n k \binom n k p^k (1-p)^k \\ &= \sum_{k=0}^n k \frac{n(n-1)!}{k(k-1)!(n-k)!} p p^{k-1} (1-p)^{n-k} \\ &= np \sum_{\kappa=1}^\nu \frac{\nu!}{\kappa!(\nu-\kappa)!} p^\kappa (1-p)^{\nu-\kappa} = np(p + 1- p)^{\nu} \\ &= np \end{split}$$ where I’ve made the substitutions $\kappa = k-1$ and $\nu = n-1$. Apart from the mathematical certainty of this derivation, it also makes perfect intuitive sense that $np$ should be the expected value, since it is the product of the probability of the outcome times the number of trials performed: if there’s a $1/6$ chance that I roll a 5 on a fair dice, and I throw it $600$ times, then I expect to see a 5 about $100$ of those times.

If instead I want to know how I should expect the outcomes to vary around the expected value, I may compute the variance of $K$: with the same substitutions as before, $$\begin{split} \mathrm{Var}[K] &= \langle K^2\rangle - \langle K \rangle^2 = \left(\sum_{k=0}^n k^2 b_n(k) \right) -n^2p^2 \\ &= \left( \sum_{k=1}^n k^2 \binom n k p^k (1-p)^{n-k}\right) -n^2p^2 \\ &= \left( np \sum_{\kappa = 0}^\nu (\kappa +1) \binom \nu \kappa p^\kappa (1-p)^{\nu-\kappa} \right)-n^2p^2 \\ &= \left(np \Big(\sum_{\kappa=0}^\nu \kappa b_\nu(\kappa) +(p + 1-p)^\nu \Big) \right) -n^2p^2 \\ &= np(\nu p + 1) - n^2 p^2 = np( np - p + 1 - np) \\ &= np(1-p) \end{split}$$ Again, the derivation is mathematically straightforward; but why should I expect that this be the formula for the variance of $K$? Why does multiplying the expected value of $K$ times the probability that my outcome doesn’t occur give me a measure of the dispersion of $K$? In other words, how can I justify this formula for variance intuitively in a similar way as I can with the formula for the mean?

EDIT. Up until now, I’ve gotten answers that are just perfectly good explanations of how to derive the formula for the variance of $K$ in ways that differ from the one presented above. That’s not what I’m asking. The answer I’m looking for should contain as few formulae as possible, and use simple enough words to explain not why the formula is mathematically true, but why it’s reasonable and couldn’t possibly be otherwise. Something like the intuitive explanation for$\langle K\rangle = np$ that I gave above.

$\endgroup$
  • 6
    $\begingroup$ Nice question, +1. If someone comes up with a good answer, that could have implications for the teaching of probability and statistics. $\endgroup$ – Adrian Keister Jun 21 '18 at 13:08
  • 3
    $\begingroup$ That’s also part of the reason why I’m asking it. It is so much better to introduce these concepts to the uninitiated first through intuitive reasoning, then through formalism; however this one formula in particular seems to survive any attempt to give it a sensible explanation in words. $\endgroup$ – giobrach Jun 21 '18 at 13:17
  • 3
    $\begingroup$ I think it's intrinsically harder to justify a variance formula in an intuitive sense, as opposed to a mean value. At first glance you can justify some of the implications of the formula, such as the variance should be zero when $p=0,1$ and you should have maximum variance when $p=0.5$. I'm not saying it's impossible though, I'm genuinely interested in seeing such a justification. $\endgroup$ – Paul Jun 21 '18 at 13:43
  • 1
    $\begingroup$ Shouldn't $\binom{n}{k}p^k(1-p)^k$ be $\binom{n}{k}p^k(1-p)^{n-k}$? $\endgroup$ – robjohn Aug 11 at 3:30
4
$\begingroup$

A binomial random variable is a sum of independent Bernoulli random variables. So if you accept that a Bernoulli random variable has variance $p(1-p)$, then the formula for the variance of a Binomial random variable follows from the "variance of sum" rule. Moreover, the variance of a Bernoulli random variable can be seen at a glance using the formula $\text{Var}(X) = E(X^2) - E(X)^2$.

$\endgroup$
  • 2
    $\begingroup$ I literally included a complete derivation of the formula in the question itself. As per the title, I’m interested in the intuition behind it, not the mathematical steps necessary to its justifcation. So, why should I accept that formula for Bernoulli variance? Why is it intuitively obvious? $\endgroup$ – giobrach Jun 21 '18 at 13:19
  • 1
    $\begingroup$ I would also like to see a more direct intuitive explanation, which hopefully somebody else will be able to provide, but with this approach it's at least possible to see the formula at a glance. $\endgroup$ – littleO Jun 21 '18 at 13:23
  • 1
    $\begingroup$ @littleO For the individual Bernouilli random variable $B$ it should be no surprise that the variance is $0$ when $p=0$ or $p=1$ (you are almost certain what the value is going to be) or that it is maximised when $p=\frac12$. As for calculating it, $B^2=B$ so $E[B^2]=E[B]=p$ so the variance is $E[B^2]-(E[B])^2=p-p^2=p(1-p)$ and this looks highly plausible. Add together $n$ i.i.d. of then and the variance intuitively increases with $n$ (no offsets) so $np(1-p)$. You may not find this immediately intuitive, but I find the three components of $n$, $p$ and $(1-p)$ completely reasonable $\endgroup$ – Henry Jun 21 '18 at 14:00
2
$\begingroup$

I understand the logic to be the same in the case of Bernoulli random variables as in the case of Binomial r.v.s because of independence. In the case of Bernoulli r.v.s, just as in the case of the binomial, the concept of variance is implicit in the notion of expected value itself. That is, the more the expected value per trial deviates from 1 or 0, the more we have variability in our results. Variability is maximised the less we can say, either, that we have all ones or all zeros, and the closer we get to equal ones and equal zeros. This occurs as the mean value, p, approaches 1/2. When our expected value is neither one nor zero (that is, when there is randomness in our outcomes) we start to record distances of x from the mean value. Our distance from the expected value is the same as our expected value when x=0, when our outcome does not occur, which occurs with a probability (1-p). At the same time, our distance can be the complement of this with the complementary probability. Calculating the expected difference of x from its mean means taking account of both distances and their probabilities which, because of symmetry, yields a duplication of the product of p (the expected value of x), and the probability of the non occurrence of the outcome. This symmetry recalls the fact that distances from x = 0 or x = 1 are equally relevant to the variability, because they equally prescribe the deviation from certainty we have in relation to the event, however the contribution they provide to this sum has in both instances to be tempered by the level of certainty we can have in relation to them (their own probability). The squaring of complement terms necessary to calculate the variance reduces this duplicate to one. Thus the variance is simply the expected value of x and the probability of the non occurrence of the outcome.

$\endgroup$
  • 2
    $\begingroup$ Please edit and use MathJax to properly format math expressions. $\endgroup$ – Lee David Chung Lin Aug 11 at 2:53
  • 1
    $\begingroup$ ...around so. I made an example. $\endgroup$ – peterh Aug 11 at 3:09
  • 1
    $\begingroup$ Thanks. I'm still learning all of this, will give it a go. $\endgroup$ – Mathem-antics Aug 11 at 7:58
1
$\begingroup$

You have $Var(X+Y) = Var(X)+Var(Y)$ for i.i.d vars and we have $$ K = \sum I_i, $$ With i.i.d $I_i = 1$ because the nubber of succcesses is has the same as distribution as the number of times $I_i$ is 1 and they sum the the number of successes.

Now with probability $p$ $I_i=1$ and zero with probability $1-p$. and also $$ Var(I_i) = p(1-p) $$ Once can motivate this formula by noting the variance is quadratic in $p$ and if almost all mass is at 1 then we have almost zero variance and if we have almost all mass at 0 then symmetrically we have almost zero variance from there you get the formula. The n is from the principle that variance of a sum of independent variables is the same as the sum of the individual variance.

$\endgroup$
  • 1
    $\begingroup$ See my comment to littleO’s answer. $\endgroup$ – giobrach Jun 21 '18 at 13:22
0
$\begingroup$

For any distribution,

$$E((X-E(X))^2)=E(X^2)-E(X)^2.$$

The variance is the second order moment from which you subtract the squared average.

In a Bernouilli scheme ($B=0/1$ drawings), $B^2=B$ so that

$$\text{Var}(B)=E((B-E(B))^2)=E(B)-\overline E(B)^2=p-p^2=pq$$

and

$$\text{Var}\left(\sum B\right)=npq.$$

$\endgroup$
  • 4
    $\begingroup$ This doesn't answer the question. $\endgroup$ – Paul Jun 21 '18 at 14:01
  • 1
    $\begingroup$ @Paul: please substantiate this comment. $\endgroup$ – Yves Daoust Jun 21 '18 at 14:08
  • 1
    $\begingroup$ Take a look at OP's edit. $\endgroup$ – Paul Jun 21 '18 at 14:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.