2
$\begingroup$

I want to calculate the variance of a sum of random variables $ (X_1+X_2) $.

Doing statistics, I have to learn maths starting from the end and it is quite difficult, yet very interesting (please consider that I only have the very basic skills in maths).

For now I am doing this calculus manually with the formula $ \mathrm{var}(X_1+X_2) = \mathrm{var}(X_1) + \mathrm{var}(X_2) + 2\mathrm{cov}(X_1,X_2)$.

But I am now facing much larger sums (with some minus) and being able to do so with matrix calculation would save me a lot of time (and would be very satisfying too).

I searched every resource in matrix calculus but couldn't find anything usable with my knowledge.

How can I do this calculus from the variance-covariance matrix $$ \begin{pmatrix} \mathrm{var}(X_1) & \mathrm{cov}(X_1,X_2) \\ \mathrm{cov}(X_1,X_2) & \mathrm{var}(X_2) \\ \end{pmatrix} $$ [preferentially extended to substractions and n terms, like $ (X_1+X_2-X_3) $]?

NB: this is not a statistic question and doesn't belong to stats.stackexchange. I want to understand the thought process of turning scalar calculation to matrix.

$\endgroup$
3
$\begingroup$

The variance-covariance matrix of $X$ is $\frac1n(X-\bar X)^T(X-\bar X)$.

Now, you want to compute the variance of the vector $u=X\beta$ for some vector $\beta$. This variance is

$$Var(u)=\frac1n(u-\bar u)^T(u-\bar u)=\frac1n(X\beta-\bar X\beta)^T(X\beta-\bar X\beta)\\ =\frac1n\beta^T(X-\bar X)^T(X-\bar X)\beta=\beta^TVar(X)\beta$$

$\endgroup$
  • $\begingroup$ +1 Elegant solution $\endgroup$ – caverac Nov 22 '18 at 13:12
  • $\begingroup$ Great answer, thanks ! Learned a lot with this ! $\endgroup$ – Dan Chaltiel Nov 22 '18 at 13:28
2
$\begingroup$

The key point here is that

$$ \mathbb{V}{\rm ar}[X] = \mathbb{C}{\rm ov}[X, X] $$

so that you can express your first expression as

\begin{eqnarray} \mathbb{V}{\rm ar}[a_1 X_1 + a_2 X_2] &=& a_1^2\mathbb{V}{\rm ar}[X_1] + a_2^2\mathbb{V}{\rm ar}[X_2] + 2 a_1a_2 \mathbb{C}{\rm ov}[X_1, X_2] \\ &=& a_1^2 \mathbb{C}{\rm ov}[X_1, X_1] + a_2^2 \mathbb{C}{\rm ov}[X_2, X_2] + 2a_1a_2 \mathbb{C}{\rm ov}[X_1, X_2] \\ &=& a_1^2 \mathbb{C}{\rm ov}[X_1, X_1] + a_2^2 \mathbb{C}{\rm ov}[X_2, X_2] + a_1a_2 \mathbb{C}{\rm ov}[X_1, X_2] + a_2a_1 \mathbb{C}{\rm ov}[X_2, X_1] \\ &=& \sum_{i=1}^2 \sum_{j=1}^2 a_i a_j \mathbb{C}{\rm ov}[X_i, X_j] \end{eqnarray}

In general

\begin{eqnarray} \mathbb{V}{\rm ar}[a_1 X_1 + \cdots a_n X_n] &=& \sum_{i=1}^n \sum_{j=1}^n a_i a_j \mathbb{C}{\rm ov}[X_i, X_j] \end{eqnarray}

$\endgroup$
  • 1
    $\begingroup$ +1, but I think the OP wanted to see how this is done with matrices, that is how to write it as $a^TVar(X)a$. $\endgroup$ – Jean-Claude Arbaut Nov 22 '18 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.