3
$\begingroup$

For any random variable, X, Var(aX) = a^2*Var(X), which is easy to demonstrate.

Suppose you have a series of IID's, and want to find the variance. So, in that case for example, Var(X+X+X+X+X) = Var(X) + Var(X) + Var(X) + Var(X) + Var(X) = 5Var(X) since there is no covariance involved. But isn't Var(X+X+X+X+X) = Var(5X) = 25Var(X)?

Follow-up, when doing the variance of a sum of dependent random variables would you add two times every possible pairwise covariance to the individual variances?

$\endgroup$
  • $\begingroup$ I think reading this Wikipedia article en.wikipedia.org/wiki/Variance would answer all your questions and give you a better understanding of simple concepts such as the variance $\endgroup$ – Stelios Kounis Jul 7 at 19:54
  • $\begingroup$ This is a very common mistake. $\endgroup$ – leonbloy Jul 7 at 20:41
3
$\begingroup$

Follow-up, when doing the variance of a sum of dependent random variables would you add two times every possible pairwise covariance to the individual variances?

Yes. And the covariance of $X$ and $X$ is the variance: $\operatorname{Cov}(X,X)=\operatorname{Var}(X)$, where $X$ fully depend on itself. That´s why

\begin{align} \operatorname{Var}(X+X)& =\operatorname{Var}(X)+\operatorname{Var}(X) + 2\cdot \operatorname{Cov}(X,X) \\[6pt] &=2\cdot \operatorname{Var}(X) + 2 \cdot \operatorname{Var}(X) \\[6pt] &=4\cdot \operatorname{Var}(X) \end{align}

| cite | improve this answer | |
$\endgroup$
3
$\begingroup$

If $X_1,...,X_5$ are independent, then $$V(X_1+X_2+X_3+X_4+X_5) = V(X_1)+V(X_2)+V(X_3)+V(X_4)+V(X_5)$$ For obvious reasons, $X$ is not independent from $X$, so it is your second formula that is correct: $$V(X+X+X+X+X)=V(5X)=25V(X)$$

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I guess it really just comes down to the fact that multiplying the same random variable by 5 is not the same as having 5 independent samples from the same underlying distribution. Yeah? That was my presumption initially, but just wanted to verify. $\endgroup$ – OzarkNathan Jul 7 at 19:59
  • $\begingroup$ You are correct, those are two very different things. $\endgroup$ – charlus Jul 7 at 20:03
1
$\begingroup$

$\newcommand{\v}{\operatorname{var}}\newcommand{\c}{\operatorname{cov}}$

would you add two times every possible pairwise covariance

Here is a way to think about that. First, suppose you know that $$ \v(X+Y) = \v(X) + \v(Y) + 2\c(X,Y). $$ That can be applied repeatedly, as follows: \begin{align} & \v(X_1+X_2+X_3+X_4+X_5) \\[6pt] = {} & \v(X_1+X_2) + \v(X_3+X_4+X_5) \tag1 \\ & {} + 2\c(X_1+X_2,\,X_3+X_4+X_5) \tag2 \\[10pt] & \text{The first term on line $(1)$ is:} \\[8pt] & \v(X_1+X_2) = \v(X_1)+\v(X_2) + 2\c(X_1,X_2). \\[8pt] & \text{The second term on line $(1)$ is:} \\[8pt] & \v(X_3+X_4+ X_5) \\[4pt] = {} & \v(X_3)+ \v(X_4+X_5) \\ & {} + 2\c(X_3,\,X_4+X_5). \\[10pt] & \text{The thing on line $(2)$ is:} \\[8pt] & \c(X_1+X_2,\,\,X_3+X_4+X_5) \\[8pt] = {} & \c(X_1,\,\,X_3+X_4+X_5) + \c(X_2,\,\,X_3+X_4+X_5) \tag3 \\[8pt] = {} & \c(X_1,X_3) + \c(X_1,X_4) + \cdots \end{align} Keep going like that until you see just how many covariances you need, and which ones.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.