0
$\begingroup$

My contention is that it's true. I thought of two ways of proving it, unsure which one is better (and/or correct):

  1. Suppose $X_i \sim N(\mu, \sigma^2)$, then it's moment generating function (MGF) is $$\Psi_{X_i}(t) = \exp\{\mu t + \frac12 \sigma^2 t^2\}$$ If we've got the iid property, then MGF of the sum is equal to the product of the MGF's which is: $$\Psi_{\sum_{i=1}^n X_i}(t) = \prod_{i=1}^n \exp\{\mu t + \frac12 \sigma^2 t^2\} = \exp\{n \mu t + n \frac12 \sigma^2 t^2\} $$ Which gives us that $\sum_{i=1}^n X_i \sim N(n \mu, n \sigma^2)$. It should probably hold in general (can't think of an argument, why?)

  2. From the definition of variance

\begin{align} Var( \sum_{i=1}^n X_i ) &= \mathbb{E} \Bigg (\sum_{i=1}^n X_i- \mathbb{E} \sum_{i=1}^n X_i \Bigg)^2\\ &= \mathbb{E} \Bigg( (\sum_{i=1}^n X_i)^2- 2 \sum_{i=1}^n X_i \sum_{i=1}^n\mathbb{E} X_i + (\sum_{i=1}^n\mathbb{E} X_i)^2 \Bigg)\\ &=\sum_{i=1}^n\mathbb{E} X_i^2- (\sum_{i=1}^n\mathbb{E} X_i)^2\\ &=n \mathbb{E}X_1^2 - n^2 (\mathbb{E}X_1)^2 \end{align} since $\mathbb{E}[\sum_{i \neq j} X_i X_j ]=0 $ and we have the iid property, i.e.: $\mathbb{E}X_1 = \mathbb{E}X_i \space \forall i$.

Whereas \begin{align} \sum_{i=1}^n Var(X_i) &=\sum_{i=1}^n \mathbb{E} \Bigg( X_i - \mathbb{E}X_i \Bigg)^2 \\ &= \sum_{i=1}^n \mathbb{E} \Bigg( X_i^2- 2X_i \mathbb{E}X_i + (\mathbb{E}X_i)^2 \Bigg)\\ &= n \mathbb{E}X_1^2 - n (\mathbb{E}X_1)^2 \end{align} I can't seem to get the $n^2$ here the same way I got (above). This would suggest that my contention is wrong. Is it or did I make a mistake in the calculations?

$\endgroup$
3
$\begingroup$

$$\begin{align} \operatorname{var}\left( \sum_{i=1}^n X_i \right) &= \mathbb{E} \Bigg (\sum_{i=1}^n X_i- \mathbb{E} \sum_{i=1}^n X_i \Bigg)^2\\ &= \mathbb{E} \Bigg (\sum_{i=1}^n (X_i- \mathbb{E}X_i) \Bigg)^2\\ &= \sum_{i=1}^n \mathbb{E} (X_i- \mathbb EX_i)^2 + \mathbb E\sum_{i=1}^n ~\sum_{j=1, j \neq i}^n (X_i -\mathbb{E} X_i) (X_j -\mathbb{E} X_j)\\ &=\sum_{i=1}^n\operatorname{var}(X_i)+ \sum_{i=1}^n~ \sum_{j=1, j \neq i}^n \mathbb E(X_i -\mathbb{E} X_i)\mathbb E(X_j -\mathbb{E} X_j)\\ &= \sum_{i=1}^n\operatorname{var}(X_i) \end{align}$$

$\endgroup$
2
$\begingroup$
  1. Let us begin with the $n=2$ case to show some explicit calculation. Let me change the notation a bit and consider the iid random variables $X$,$Y$ s.t. $\mathbb E[X]=\mu=\mathbb E[Y]$. Then

$Var(X+Y)=\mathbb E[(X+Y-2\mu)^2]=\mathbb E[(X-\mu)^2]+\mathbb E[(Y-\mu)^2]+\mathbb E[2XY+2{\mu}^2-2\mu(X+Y)]=Var(X)+Var(Y)+2{\mu}^2+2{\mu}^2-2\mu(\mu+\mu)= Var(X)+Var(Y)$,

as $\mathbb E[XY]=\mathbb E[X]\mathbb E[Y]={\mu}^2$ ($X$ and $Y$ are independent). For $n=2$ your statement holds without giving any explicit form for the distribution of $X$.

  1. Let us move to the general case ($n>1$). Point 2. in your question is bugged. More precisely, when you compute $Var(\sum_{i=1}^nX_i)$ you arrive at

$\mathbb E[(\sum_{i=1}^nX_i)^2]=\sum_{i=1}^n \mathbb E[ {X}^2_i]$,

which is wrong (check it for $n=2$ to convince yourself). In fact

$\mathbb E[(\sum_{i=1}^nX_i)^2]=\sum_{i=1}^n \mathbb E[ {X}^2_i]+2\sum_{i\neq j}E[X_i]E[X_j]$.

This last equality can be proved by induction and using the fact that $E[X_iX_j]=E[X_i]E[X_j]$ for all $i\neq j$. But then

$2\sum_{i\neq j}E[X_i]E[X_j]=2\sum_{i\neq j}\mu^2=2\mu^2n(n-1)$,

as each of the $n$ indices (let us say $i$) is multiplied with all the remaining $n-1$ indices (i.e. all those indices in $1,\dots,n$ distinct from $i$).

In summary, we have

$\sum_{i=1}^n Var(X_i)=\sum_{i=1}^n\mathbb E[X^2_i]-n\mu^2$ (as you correctly computed)

and

$Var(\sum_{i=1}^nX_i)=$(you did this correctly) $=\mathbb E[(\sum_{i=1}^nX_i)^2]-2n^2\mu^2+n\mu^2=\sum_{i=1}^n \mathbb E[ {X}^2_i]+2\mu^2n(n-1)-2n^2\mu^2+n\mu^2=\sum_{i=1}^n \mathbb E[ {X}^2_i]-n\mu^2,$

as stated.

I hope this can help.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.