Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top


$X_i$ from $i=1,\ldots,n$ is a set of random variables

The following confuses me:

$\Cov\left(\sum_{i=1}^n X_i, \sum_{j=1}^n X_j\right) =$ the sum of all possible covariance pairs (so $n\cdot n$ terms) (source is the book I'm currently reading)

In my thoughts, the expansion might as well be

$$\Cov(X_1, X_1)+\Cov(X_2, X_2)+\cdots+\Cov(X_n, X_n)$$


$$\Cov(X_1 + X_2 + \cdots +X_n, X_1 + X_2 + \cdots +X_n)$$

share|cite|improve this question
It's the second one. The covariance operator is bilinear; that means it's linear in each variable separately, like the dot product. – Qiaochu Yuan Sep 24 '12 at 0:25
isnt the dot product more like the first one, where you multiply the ith term times the ith term and then add everything up – Wuschelbeutel Kartoffelhuhn Sep 24 '12 at 0:45
Only if $\text{Cov}(X_i, X_j) = 0$ when $i \neq j$. Think of the $X_i$ as vectors which might not be orthogonal to each other. – Qiaochu Yuan Sep 24 '12 at 0:55
@WuschelbeutelKartoffelhuhn : The dot product $(a_1\vec{v}_1 + \cdots + a_n\vec{v}_n) \cdot (a_1\vec{v}_1 + \cdots + a_n\vec{v}_n)$ is the sum of all $n^2$ pairs $a_i a_j \vec{v}_i\cdot\vec{v}_j$. – Michael Hardy Sep 24 '12 at 2:40
up vote 2 down vote accepted


Since $\Cov(A+B,C)=\Cov(A,C)+\Cov(B,C)$ and $\Cov(D,E+F)=\Cov(D,E)+\Cov(D,F)$ etc.:

  • $\Cov\left(\sum_{i=1}^n X_i, \sum_{j=1}^n X_j\right)$

  • $\Cov(X_1, X_1) + \Cov(X_1, X_2) + \ldots + \Cov(X_1, X_n) + \Cov(X_2, X_1) + \ldots +\Cov(X_n, X_n)$

  • $\Cov(X_1 + X_2 + \cdots +X_n, X_1 + X_2 + \cdots +X_n)$

all give the same result. But my second bullet has $n^2$ terms in the sum while yours has $n$; they will be the same if the $X_i$ are independent or at least pairwise uncorrelated, but otherwise will usually be different.

Incidentally, the covariance of a random variable with itself can be called its variance.

share|cite|improve this answer

The other comments and answers are absolutely correct. But rather than just stating covariance identities, maybe it will be helpful to actually expand it out using the definition of covariance.

$\begin{align} \textrm{Cov}\left[\sum_{i=1}^{n}X_i,\sum_{j=1}^{n}X_j\right]&=\textrm{E}\left[\left(\sum_{i=1}^{n}X_i-\textrm{E}\left[\sum_{i=1}^{n}X_i\right]\right)\left(\sum_{j=1}^{n}X_j-\textrm{E}\left[\sum_{j=1}^{n}X_j\right]\right)\right]\\ &= \textrm{E}\left[\left(\sum_{i=1}^{n}\left(X_i-\textrm{E}\left[X_i\right]\right)\right)\left(\sum_{j=1}^{n}\left(X_j-\textrm{E}\left[X_j\right]\right)\right)\right]\\ &= \textrm{E}\left[\sum_{i=1}^{n}\left(\left(X_i-\textrm{E}\left[X_i\right]\right)\sum_{j=1}^{n}\left(X_j-\textrm{E}\left[X_j\right]\right)\right)\right]\\ &= \textrm{E}\left[\sum_{i=1}^{n}\sum_{j=1}^{n}\left(X_i-\textrm{E}\left[X_i\right]\right)\left(X_j-\textrm{E}\left[X_j\right]\right)\right]\\ &= \sum_{i=1}^{n}\sum_{j=1}^{n}\textrm{E}\left[\left(X_i-\textrm{E}\left[X_i\right]\right)\left(X_j-\textrm{E}\left[X_j\right]\right)\right]\\ &= \sum_{i=1}^{n}\sum_{j=1}^{n}\textrm{Cov}\left[X_i,X_j\right]\\ \end{align}$

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.