0
$\begingroup$

For independent variables, it is true that $$\mu_{ax+b} = a\,\mu_x + b$$ and that $$\mu_{x+y} = \mu_x + \mu_y$$

This makes a lot of intuitive sense: shifting a mean is adding to it, scaling a mean is multiplying something by it.

But then I found that $$\sigma_{ax}^2 = a^2\,\sigma_x^2$$ and that

$$\sigma_{x+y}^2 = \sigma_x^2 + \sigma_y^2$$

$$\sigma_{x-y}^2 = \sigma_x^2 + \sigma_y^2$$

Why are these true? It's not immediately obvious why we multiply by the square of the scale factor. Even more elusive is the reason why the sum and difference of variances between independent variables is the same value.

Can someone shed light on why these concepts related to the manipulation of variances hold? Is there an intuitive explanation?

$\endgroup$
1
$\begingroup$

First, we can look at the formula

$$ \sigma_x^2 = \mathbb{E}[(X - \mu)^2] = \mathbb{E}[X^2] - \mu^2 $$

So

$$ \sigma_{ax}^2 = \mathbb{E}[(aX)^2] - \mathbb{E}[aX]^2 = a^2\left(\mathbb{E}[X^2] - \mu^2\right) = a^2 \sigma_x^2 $$

Similarly,

$$ \begin{align} \sigma_{x+y}^2 &= \mathbb{E}[(X+Y)^2] - \mathbb{E}[X+Y]^2\\ &= \mathbb{E}[X^2 + 2XY + Y^2] - \left(\mathbb{E}[X] + \mathbb{E}[Y]\right)^2\\ &= \mathbb{E}[X^2] - \mathbb{E}[X]^2 + \mathbb{E}[Y^2] - \mathbb{E}[Y]^2 + 2(\mathbb{E}[XY] - \mathbb{E}[X]\mathbb{E}[Y])\\ &= \mathbb{E}[X^2] - \mathbb{E}[X]^2 + \mathbb{E}[Y^2] - \mathbb{E}[Y]^2 \qquad \text{ if } X \text{ and } Y \text{ are independent}\\ &= \sigma_x^2 + \sigma_y^2 \end{align} $$

So, while the expectation is linear (it's defined by the integral, which is a linear operator). The variance is quadratic.

As far as an intuitive notion, you could see if this post helps you. Usually, we think about the standard deviation for the intuition. The problem is we want to measure total variability of our data (that's why it's squared) not just the spread (which could average out to zero).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.