3
$\begingroup$

$\newcommand{\Cov}{\operatorname{Cov}}$

$X_i$ from $i=1,\ldots,n$ is a set of random variables

The following confuses me:

$\Cov\left(\sum_{i=1}^n X_i, \sum_{j=1}^n X_j\right) =$ the sum of all possible covariance pairs (so $n\cdot n$ terms) (source is the book I'm currently reading)

In my thoughts, the expansion might as well be

$$\Cov(X_1, X_1)+\Cov(X_2, X_2)+\cdots+\Cov(X_n, X_n)$$

Or

$$\Cov(X_1 + X_2 + \cdots +X_n, X_1 + X_2 + \cdots +X_n)$$

$\endgroup$
4
  • 1
    $\begingroup$ It's the second one. The covariance operator is bilinear; that means it's linear in each variable separately, like the dot product. $\endgroup$
    – Qiaochu Yuan
    Sep 24, 2012 at 0:25
  • $\begingroup$ isnt the dot product more like the first one, where you multiply the ith term times the ith term and then add everything up $\endgroup$
    – Wuschelbeutel Kartoffelhuhn
    Sep 24, 2012 at 0:45
  • 1
    $\begingroup$ Only if $\text{Cov}(X_i, X_j) = 0$ when $i \neq j$. Think of the $X_i$ as vectors which might not be orthogonal to each other. $\endgroup$
    – Qiaochu Yuan
    Sep 24, 2012 at 0:55
  • $\begingroup$ @WuschelbeutelKartoffelhuhn : The dot product $(a_1\vec{v}_1 + \cdots + a_n\vec{v}_n) \cdot (a_1\vec{v}_1 + \cdots + a_n\vec{v}_n)$ is the sum of all $n^2$ pairs $a_i a_j \vec{v}_i\cdot\vec{v}_j$. $\endgroup$
    – Michael Hardy
    Sep 24, 2012 at 2:40

2 Answers 2

Reset to default
4
$\begingroup$

The other comments and answers are absolutely correct. But rather than just stating covariance identities, maybe it will be helpful to actually expand it out using the definition of covariance.

$\begin{align} \textrm{Cov}\left[\sum_{i=1}^{n}X_i,\sum_{j=1}^{n}X_j\right]&=\textrm{E}\left[\left(\sum_{i=1}^{n}X_i-\textrm{E}\left[\sum_{i=1}^{n}X_i\right]\right)\left(\sum_{j=1}^{n}X_j-\textrm{E}\left[\sum_{j=1}^{n}X_j\right]\right)\right]\\ &= \textrm{E}\left[\left(\sum_{i=1}^{n}\left(X_i-\textrm{E}\left[X_i\right]\right)\right)\left(\sum_{j=1}^{n}\left(X_j-\textrm{E}\left[X_j\right]\right)\right)\right]\\ &= \textrm{E}\left[\sum_{i=1}^{n}\left(\left(X_i-\textrm{E}\left[X_i\right]\right)\sum_{j=1}^{n}\left(X_j-\textrm{E}\left[X_j\right]\right)\right)\right]\\ &= \textrm{E}\left[\sum_{i=1}^{n}\sum_{j=1}^{n}\left(X_i-\textrm{E}\left[X_i\right]\right)\left(X_j-\textrm{E}\left[X_j\right]\right)\right]\\ &= \sum_{i=1}^{n}\sum_{j=1}^{n}\textrm{E}\left[\left(X_i-\textrm{E}\left[X_i\right]\right)\left(X_j-\textrm{E}\left[X_j\right]\right)\right]\\ &= \sum_{i=1}^{n}\sum_{j=1}^{n}\textrm{Cov}\left[X_i,X_j\right]\\ \end{align}$

$\endgroup$
3
$\begingroup$

$\newcommand{\Cov}{\operatorname{Cov}}$

Since $\Cov(A+B,C)=\Cov(A,C)+\Cov(B,C)$ and $\Cov(D,E+F)=\Cov(D,E)+\Cov(D,F)$ etc.:

  • $\Cov\left(\sum_{i=1}^n X_i, \sum_{j=1}^n X_j\right)$

  • $\Cov(X_1, X_1) + \Cov(X_1, X_2) + \ldots + \Cov(X_1, X_n) + \Cov(X_2, X_1) + \ldots +\Cov(X_n, X_n)$

  • $\Cov(X_1 + X_2 + \cdots +X_n, X_1 + X_2 + \cdots +X_n)$

all give the same result. But my second bullet has $n^2$ terms in the sum while yours has $n$; they will be the same if the $X_i$ are independent or at least pairwise uncorrelated, but otherwise will usually be different.

Incidentally, the covariance of a random variable with itself can be called its variance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.