2
$\begingroup$

$\newcommand{\cov}{\operatorname{cov}}\newcommand{\corr}{\operatorname{corr}}$I have two dependent multivariate random variables, or random vectors, $X$ and $Y$, their respective covariance matrices $\Sigma_X = \cov(X,X)$, $\Sigma_Y = \cov(Y,Y)$ and a non-zero correlation matrix $P_{XY} = \corr(X, Y)$.

I would like to compute the covariance matrix of the sum of the two random vectors:

\begin{align} Z {}={}& X + Y \\ \Sigma_Z {}={}& \Sigma_X + \Sigma_Y + 2 \cov(X, Y) \end{align}

Intuitively, I would assume the cross-covariance can be computed as follows:

$$\cov(X, Y) = P_{XY} \sqrt{\Sigma_X} \sqrt{\Sigma_Y}$$

But I can see that this matrix is not necessarily symmetric, and therefore the resulting $\Sigma_Z$ wouldn't be either. Am I wrong? Is it at all possible to compute $\Sigma_Z$ from the information I have? If not, is there some way to approximate it?

EDIT: I think it isn't correct to write:

$$ \Sigma_Z = \Sigma_X + \Sigma_Y + 2 \cov(X, Y) $$

as $\cov(X,Y) \ne \cov(Y,X)$, but instead $\cov(X,Y) = \cov(Y,X)^T$.

Rewriting the equation for $\Sigma_Z$ to:

\begin{align} \Sigma_Z {}={}& \Sigma_X + \Sigma_Y + \cov(X, Y) + \cov(Y, X) \\ {}={}& \Sigma_X + \Sigma_Y + \cov(X, Y) + \cov(X, Y)^T \end{align}

I can now prove that indeed $\Sigma_Z$ is guaranteed to be symmetric. As antkam points out in the chat below, it isn't necessary that $\cov(X, Y) + \cov(X, Y)^T$ be positive semi-definite for $\Sigma_Z$ to be positive semi-definite.

$\endgroup$
6
  • $\begingroup$ Wait... what exactly is your question? I thought you're trying to calculate $\Sigma_Z = cov(Z,Z)$? Of course this matrix is symmetric and psd. Why would you need to prove this? $\endgroup$
    – antkam
    May 9, 2019 at 14:42
  • $\begingroup$ Of course $\Sigma_Z$ is symmetric and PSD, which is why I was concerned about my above definition of $\Sigma_Z$ . I saw this cannot possibly be right as I wrote it, hence my question here :) I understand your answer below and think you are correct. Now I'm trying to fit this into my algorithm (obtaining $P_{XY}$ is another step I have to solve, but unrelated to the question here). I'll upvote/accept your answer once I've got a good feeling I understand the full picture :) $\endgroup$
    – lucab
    May 9, 2019 at 14:49
  • $\begingroup$ hahaha, ok I see now. so your OP actually contains two errors: (1) the equation $\Sigma_Z = ...$ (which I did not address) and (2) the equation $cov(X,Y)=...$ (which I did address). anyway, you can calculate $cov(X,Y)$ using my method and then $cov(Y,X)$ is just the transpose. $\endgroup$
    – antkam
    May 9, 2019 at 14:53
  • $\begingroup$ BTW, in general if matrices $A = B + C + D$ and $A,B,C$ are symmetric and psd, that implies $D$ is symmetric but does not imply $D$ is also psd. (E.g. $A = B=C = I$, the identity, then $D = -I$.) I don't know offhand whether $cov(X,Y) + cov(Y,X)$ is psd, but perhaps I'm just missing something obvious. $\endgroup$
    – antkam
    May 9, 2019 at 14:56
  • $\begingroup$ Thanks for the pointer. Perhaps it isn't the case that $cov(X, Y) + cov(Y, X)$ is PSD then. $\endgroup$
    – lucab
    May 9, 2019 at 15:01

1 Answer 1

0
$\begingroup$

Your original intuition does not work, i.e., $cov(X, Y) \neq P_{XY} \sqrt{\Sigma_X} \sqrt{\Sigma_Y}$, because the definition of $corr(X_i, Y_j)$ involves scalar divisions, which cannot be reversed like this. For one thing, you have three matrices, so how do you pick the order of multiplication?

Instead, I would go back to the per-entry definitions:

$$[cov(X,Y)]_{ij} = cov(X_i, Y_j) = corr(X_i, Y_j) \ \sigma(X_i) \ \sigma(Y_j) = [P_{XY}]_{ij} \sqrt{ [\Sigma_X]_{ii}} \sqrt{[\Sigma_Y]_{jj}}$$

As you can see, the non-diagonal entries of $\Sigma_X, \Sigma_Y$ don't enter the calculations at all. So I'd think you cannot write this in terms of standard matrix multiplications.

However, if you have a $Diag$ function that keeps the diagonal entries but zeroes out all the non-diagonal entries, then you can write:

$$Cov(X,Y) = \sqrt{Diag(\Sigma_X)} \ P_{XY} \ \sqrt{Diag(\Sigma_Y)}$$

The multiplication is in that order because left-multiplying by a diagonal matrix scales the rows and right-multiplying scales the columns. (I know some typical software packages have a similar function although in the case of python np.diag you actually have to call it twice, so be careful!)

$\endgroup$
2
  • $\begingroup$ Actually, I initially implemented $cov(X,Y) = diag(P_{XY})\sqrt{diag(\Sigma_X)}\sqrt{diag(\Sigma_Y)}$ which is close to your suggestion, but realized experimentally that this constrained $cov(X,Y)$ to be axis-aligned (if I e.g. look at a covariance-ellipse plot). Need to think about this some more. Thanks! $\endgroup$
    – lucab
    May 9, 2019 at 14:00
  • $\begingroup$ The product $diag(.) diag(..) diag(...)$ would still be diagonal, which $cov(X,Y)$ isn't (necessarily). Anyway, if you have further questions, feel free to comment again. You do understand the per-entry equations, right? In the worst (most tedious) case you can always write a loop to calculate entry-by-entry... $\endgroup$
    – antkam
    May 9, 2019 at 14:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .