Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

$\newcommand{\Cov}{\operatorname{Cov}}$

$X_i$ from $i=1,\ldots,n$ is a set of random variables

The following confuses me:

$\Cov\left(\sum_{i=1}^n X_i, \sum_{j=1}^n X_j\right) =$ the sum of all possible covariance pairs (so $n\cdot n$ terms) (source is the book I'm currently reading)

In my thoughts, the expansion might as well be

$$\Cov(X_1, X_1)+\Cov(X_2, X_2)+\cdots+\Cov(X_n, X_n)$$

Or

$$\Cov(X_1 + X_2 + \cdots +X_n, X_1 + X_2 + \cdots +X_n)$$

share|improve this question
1  
It's the second one. The covariance operator is bilinear; that means it's linear in each variable separately, like the dot product. –  Qiaochu Yuan Sep 24 '12 at 0:25
    
isnt the dot product more like the first one, where you multiply the ith term times the ith term and then add everything up –  Wuschelbeutel Kartoffelhuhn Sep 24 '12 at 0:45
1  
Only if $\text{Cov}(X_i, X_j) = 0$ when $i \neq j$. Think of the $X_i$ as vectors which might not be orthogonal to each other. –  Qiaochu Yuan Sep 24 '12 at 0:55
    
@WuschelbeutelKartoffelhuhn : The dot product $(a_1\vec{v}_1 + \cdots + a_n\vec{v}_n) \cdot (a_1\vec{v}_1 + \cdots + a_n\vec{v}_n)$ is the sum of all $n^2$ pairs $a_i a_j \vec{v}_i\cdot\vec{v}_j$. –  Michael Hardy Sep 24 '12 at 2:40
add comment

2 Answers 2

up vote 2 down vote accepted

$\newcommand{\Cov}{\operatorname{Cov}}$

Since $\Cov(A+B,C)=\Cov(A,C)+\Cov(B,C)$ and $\Cov(D,E+F)=\Cov(D,E)+\Cov(D,F)$ etc.:

  • $\Cov\left(\sum_{i=1}^n X_i, \sum_{j=1}^n X_j\right)$

  • $\Cov(X_1, X_1) + \Cov(X_1, X_2) + \ldots + \Cov(X_1, X_n) + \Cov(X_2, X_1) + \ldots +\Cov(X_n, X_n)$

  • $\Cov(X_1 + X_2 + \cdots +X_n, X_1 + X_2 + \cdots +X_n)$

all give the same result. But my second bullet has $n^2$ terms in the sum while yours has $n$; they will be the same if the $X_i$ are independent or at least pairwise uncorrelated, but otherwise will usually be different.

Incidentally, the covariance of a random variable with itself can be called its variance.

share|improve this answer
add comment

The other comments and answers are absolutely correct. But rather than just stating covariance identities, maybe it will be helpful to actually expand it out using the definition of covariance.

$\begin{align} \textrm{Cov}\left[\sum_{i=1}^{n}X_i,\sum_{j=1}^{n}X_j\right]&=\textrm{E}\left[\left(\sum_{i=1}^{n}X_i-\textrm{E}\left[\sum_{i=1}^{n}X_i\right]\right)\left(\sum_{j=1}^{n}X_j-\textrm{E}\left[\sum_{j=1}^{n}X_j\right]\right)\right]\\ &= \textrm{E}\left[\left(\sum_{i=1}^{n}\left(X_i-\textrm{E}\left[X_i\right]\right)\right)\left(\sum_{j=1}^{n}\left(X_j-\textrm{E}\left[X_j\right]\right)\right)\right]\\ &= \textrm{E}\left[\sum_{i=1}^{n}\left(\left(X_i-\textrm{E}\left[X_i\right]\right)\sum_{j=1}^{n}\left(X_j-\textrm{E}\left[X_j\right]\right)\right)\right]\\ &= \textrm{E}\left[\sum_{i=1}^{n}\sum_{j=1}^{n}\left(X_i-\textrm{E}\left[X_i\right]\right)\left(X_j-\textrm{E}\left[X_j\right]\right)\right]\\ &= \sum_{i=1}^{n}\sum_{j=1}^{n}\textrm{E}\left[\left(X_i-\textrm{E}\left[X_i\right]\right)\left(X_j-\textrm{E}\left[X_j\right]\right)\right]\\ &= \sum_{i=1}^{n}\sum_{j=1}^{n}\textrm{Cov}\left[X_i,X_j\right]\\ \end{align}$

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.