3
$\begingroup$

$\newcommand{\Cov}{\operatorname{Cov}}$

$X_i$ from $i=1,\ldots,n$ is a set of random variables

The following confuses me:

$\Cov\left(\sum_{i=1}^n X_i, \sum_{j=1}^n X_j\right) =$ the sum of all possible covariance pairs (so $n\cdot n$ terms) (source is the book I'm currently reading)

In my thoughts, the expansion might as well be

$$\Cov(X_1, X_1)+\Cov(X_2, X_2)+\cdots+\Cov(X_n, X_n)$$

Or

$$\Cov(X_1 + X_2 + \cdots +X_n, X_1 + X_2 + \cdots +X_n)$$

$\endgroup$
4
  • 1
    $\begingroup$ It's the second one. The covariance operator is bilinear; that means it's linear in each variable separately, like the dot product. $\endgroup$ – Qiaochu Yuan Sep 24 '12 at 0:25
  • $\begingroup$ isnt the dot product more like the first one, where you multiply the ith term times the ith term and then add everything up $\endgroup$ – Wuschelbeutel Kartoffelhuhn Sep 24 '12 at 0:45
  • 1
    $\begingroup$ Only if $\text{Cov}(X_i, X_j) = 0$ when $i \neq j$. Think of the $X_i$ as vectors which might not be orthogonal to each other. $\endgroup$ – Qiaochu Yuan Sep 24 '12 at 0:55
  • $\begingroup$ @WuschelbeutelKartoffelhuhn : The dot product $(a_1\vec{v}_1 + \cdots + a_n\vec{v}_n) \cdot (a_1\vec{v}_1 + \cdots + a_n\vec{v}_n)$ is the sum of all $n^2$ pairs $a_i a_j \vec{v}_i\cdot\vec{v}_j$. $\endgroup$ – Michael Hardy Sep 24 '12 at 2:40
3
$\begingroup$

$\newcommand{\Cov}{\operatorname{Cov}}$

Since $\Cov(A+B,C)=\Cov(A,C)+\Cov(B,C)$ and $\Cov(D,E+F)=\Cov(D,E)+\Cov(D,F)$ etc.:

  • $\Cov\left(\sum_{i=1}^n X_i, \sum_{j=1}^n X_j\right)$

  • $\Cov(X_1, X_1) + \Cov(X_1, X_2) + \ldots + \Cov(X_1, X_n) + \Cov(X_2, X_1) + \ldots +\Cov(X_n, X_n)$

  • $\Cov(X_1 + X_2 + \cdots +X_n, X_1 + X_2 + \cdots +X_n)$

all give the same result. But my second bullet has $n^2$ terms in the sum while yours has $n$; they will be the same if the $X_i$ are independent or at least pairwise uncorrelated, but otherwise will usually be different.

Incidentally, the covariance of a random variable with itself can be called its variance.

$\endgroup$
4
$\begingroup$

The other comments and answers are absolutely correct. But rather than just stating covariance identities, maybe it will be helpful to actually expand it out using the definition of covariance.

$\begin{align} \textrm{Cov}\left[\sum_{i=1}^{n}X_i,\sum_{j=1}^{n}X_j\right]&=\textrm{E}\left[\left(\sum_{i=1}^{n}X_i-\textrm{E}\left[\sum_{i=1}^{n}X_i\right]\right)\left(\sum_{j=1}^{n}X_j-\textrm{E}\left[\sum_{j=1}^{n}X_j\right]\right)\right]\\ &= \textrm{E}\left[\left(\sum_{i=1}^{n}\left(X_i-\textrm{E}\left[X_i\right]\right)\right)\left(\sum_{j=1}^{n}\left(X_j-\textrm{E}\left[X_j\right]\right)\right)\right]\\ &= \textrm{E}\left[\sum_{i=1}^{n}\left(\left(X_i-\textrm{E}\left[X_i\right]\right)\sum_{j=1}^{n}\left(X_j-\textrm{E}\left[X_j\right]\right)\right)\right]\\ &= \textrm{E}\left[\sum_{i=1}^{n}\sum_{j=1}^{n}\left(X_i-\textrm{E}\left[X_i\right]\right)\left(X_j-\textrm{E}\left[X_j\right]\right)\right]\\ &= \sum_{i=1}^{n}\sum_{j=1}^{n}\textrm{E}\left[\left(X_i-\textrm{E}\left[X_i\right]\right)\left(X_j-\textrm{E}\left[X_j\right]\right)\right]\\ &= \sum_{i=1}^{n}\sum_{j=1}^{n}\textrm{Cov}\left[X_i,X_j\right]\\ \end{align}$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.