# Covariance of sum of two dependent random vectors

$$\newcommand{\cov}{\operatorname{cov}}\newcommand{\corr}{\operatorname{corr}}$$I have two dependent multivariate random variables, or random vectors, $$X$$ and $$Y$$, their respective covariance matrices $$\Sigma_X = \cov(X,X)$$, $$\Sigma_Y = \cov(Y,Y)$$ and a non-zero correlation matrix $$P_{XY} = \corr(X, Y)$$.

I would like to compute the covariance matrix of the sum of the two random vectors:

\begin{align} Z {}={}& X + Y \\ \Sigma_Z {}={}& \Sigma_X + \Sigma_Y + 2 \cov(X, Y) \end{align}

Intuitively, I would assume the cross-covariance can be computed as follows:

$$\cov(X, Y) = P_{XY} \sqrt{\Sigma_X} \sqrt{\Sigma_Y}$$

But I can see that this matrix is not necessarily symmetric, and therefore the resulting $$\Sigma_Z$$ wouldn't be either. Am I wrong? Is it at all possible to compute $$\Sigma_Z$$ from the information I have? If not, is there some way to approximate it?

EDIT: I think it isn't correct to write:

$$\Sigma_Z = \Sigma_X + \Sigma_Y + 2 \cov(X, Y)$$

as $$\cov(X,Y) \ne \cov(Y,X)$$, but instead $$\cov(X,Y) = \cov(Y,X)^T$$.

Rewriting the equation for $$\Sigma_Z$$ to:

\begin{align} \Sigma_Z {}={}& \Sigma_X + \Sigma_Y + \cov(X, Y) + \cov(Y, X) \\ {}={}& \Sigma_X + \Sigma_Y + \cov(X, Y) + \cov(X, Y)^T \end{align}

I can now prove that indeed $$\Sigma_Z$$ is guaranteed to be symmetric. As antkam points out in the chat below, it isn't necessary that $$\cov(X, Y) + \cov(X, Y)^T$$ be positive semi-definite for $$\Sigma_Z$$ to be positive semi-definite.

• Wait... what exactly is your question? I thought you're trying to calculate $\Sigma_Z = cov(Z,Z)$? Of course this matrix is symmetric and psd. Why would you need to prove this? May 9, 2019 at 14:42
• Of course $\Sigma_Z$ is symmetric and PSD, which is why I was concerned about my above definition of $\Sigma_Z$ . I saw this cannot possibly be right as I wrote it, hence my question here :) I understand your answer below and think you are correct. Now I'm trying to fit this into my algorithm (obtaining $P_{XY}$ is another step I have to solve, but unrelated to the question here). I'll upvote/accept your answer once I've got a good feeling I understand the full picture :) May 9, 2019 at 14:49
• hahaha, ok I see now. so your OP actually contains two errors: (1) the equation $\Sigma_Z = ...$ (which I did not address) and (2) the equation $cov(X,Y)=...$ (which I did address). anyway, you can calculate $cov(X,Y)$ using my method and then $cov(Y,X)$ is just the transpose. May 9, 2019 at 14:53
• BTW, in general if matrices $A = B + C + D$ and $A,B,C$ are symmetric and psd, that implies $D$ is symmetric but does not imply $D$ is also psd. (E.g. $A = B=C = I$, the identity, then $D = -I$.) I don't know offhand whether $cov(X,Y) + cov(Y,X)$ is psd, but perhaps I'm just missing something obvious. May 9, 2019 at 14:56
• Thanks for the pointer. Perhaps it isn't the case that $cov(X, Y) + cov(Y, X)$ is PSD then. May 9, 2019 at 15:01

Your original intuition does not work, i.e., $$cov(X, Y) \neq P_{XY} \sqrt{\Sigma_X} \sqrt{\Sigma_Y}$$, because the definition of $$corr(X_i, Y_j)$$ involves scalar divisions, which cannot be reversed like this. For one thing, you have three matrices, so how do you pick the order of multiplication?

Instead, I would go back to the per-entry definitions:

$$[cov(X,Y)]_{ij} = cov(X_i, Y_j) = corr(X_i, Y_j) \ \sigma(X_i) \ \sigma(Y_j) = [P_{XY}]_{ij} \sqrt{ [\Sigma_X]_{ii}} \sqrt{[\Sigma_Y]_{jj}}$$

As you can see, the non-diagonal entries of $$\Sigma_X, \Sigma_Y$$ don't enter the calculations at all. So I'd think you cannot write this in terms of standard matrix multiplications.

However, if you have a $$Diag$$ function that keeps the diagonal entries but zeroes out all the non-diagonal entries, then you can write:

$$Cov(X,Y) = \sqrt{Diag(\Sigma_X)} \ P_{XY} \ \sqrt{Diag(\Sigma_Y)}$$

The multiplication is in that order because left-multiplying by a diagonal matrix scales the rows and right-multiplying scales the columns. (I know some typical software packages have a similar function although in the case of python np.diag you actually have to call it twice, so be careful!)

• Actually, I initially implemented $cov(X,Y) = diag(P_{XY})\sqrt{diag(\Sigma_X)}\sqrt{diag(\Sigma_Y)}$ which is close to your suggestion, but realized experimentally that this constrained $cov(X,Y)$ to be axis-aligned (if I e.g. look at a covariance-ellipse plot). Need to think about this some more. Thanks! May 9, 2019 at 14:00
• The product $diag(.) diag(..) diag(...)$ would still be diagonal, which $cov(X,Y)$ isn't (necessarily). Anyway, if you have further questions, feel free to comment again. You do understand the per-entry equations, right? In the worst (most tedious) case you can always write a loop to calculate entry-by-entry... May 9, 2019 at 14:21