# Matrix calculus for statistics

I want to calculate the variance of a sum of random variables $$(X_1+X_2)$$.

Doing statistics, I have to learn maths starting from the end and it is quite difficult, yet very interesting (please consider that I only have the very basic skills in maths).

For now I am doing this calculus manually with the formula $$\mathrm{var}(X_1+X_2) = \mathrm{var}(X_1) + \mathrm{var}(X_2) + 2\mathrm{cov}(X_1,X_2)$$.

But I am now facing much larger sums (with some minus) and being able to do so with matrix calculation would save me a lot of time (and would be very satisfying too).

I searched every resource in matrix calculus but couldn't find anything usable with my knowledge.

How can I do this calculus from the variance-covariance matrix $$\begin{pmatrix} \mathrm{var}(X_1) & \mathrm{cov}(X_1,X_2) \\ \mathrm{cov}(X_1,X_2) & \mathrm{var}(X_2) \\ \end{pmatrix}$$ [preferentially extended to substractions and n terms, like $$(X_1+X_2-X_3)$$]?

NB: this is not a statistic question and doesn't belong to stats.stackexchange. I want to understand the thought process of turning scalar calculation to matrix.

The variance-covariance matrix of $$X$$ is $$\frac1n(X-\bar X)^T(X-\bar X)$$.

Now, you want to compute the variance of the vector $$u=X\beta$$ for some vector $$\beta$$. This variance is

$$Var(u)=\frac1n(u-\bar u)^T(u-\bar u)=\frac1n(X\beta-\bar X\beta)^T(X\beta-\bar X\beta)\\ =\frac1n\beta^T(X-\bar X)^T(X-\bar X)\beta=\beta^TVar(X)\beta$$

• +1 Elegant solution – caverac Nov 22 '18 at 13:12
• Great answer, thanks ! Learned a lot with this ! – Dan Chaltiel Nov 22 '18 at 13:28

The key point here is that

$$\mathbb{V}{\rm ar}[X] = \mathbb{C}{\rm ov}[X, X]$$

so that you can express your first expression as

$$\begin{eqnarray} \mathbb{V}{\rm ar}[a_1 X_1 + a_2 X_2] &=& a_1^2\mathbb{V}{\rm ar}[X_1] + a_2^2\mathbb{V}{\rm ar}[X_2] + 2 a_1a_2 \mathbb{C}{\rm ov}[X_1, X_2] \\ &=& a_1^2 \mathbb{C}{\rm ov}[X_1, X_1] + a_2^2 \mathbb{C}{\rm ov}[X_2, X_2] + 2a_1a_2 \mathbb{C}{\rm ov}[X_1, X_2] \\ &=& a_1^2 \mathbb{C}{\rm ov}[X_1, X_1] + a_2^2 \mathbb{C}{\rm ov}[X_2, X_2] + a_1a_2 \mathbb{C}{\rm ov}[X_1, X_2] + a_2a_1 \mathbb{C}{\rm ov}[X_2, X_1] \\ &=& \sum_{i=1}^2 \sum_{j=1}^2 a_i a_j \mathbb{C}{\rm ov}[X_i, X_j] \end{eqnarray}$$

In general

$$\begin{eqnarray} \mathbb{V}{\rm ar}[a_1 X_1 + \cdots a_n X_n] &=& \sum_{i=1}^n \sum_{j=1}^n a_i a_j \mathbb{C}{\rm ov}[X_i, X_j] \end{eqnarray}$$

• +1, but I think the OP wanted to see how this is done with matrices, that is how to write it as $a^TVar(X)a$. – Jean-Claude Arbaut Nov 22 '18 at 12:57