3
$\begingroup$

$\newcommand{\Cov}{\operatorname{Cov}}$

$X_i$ from $i=1,\ldots,n$ is a set of random variables

The following confuses me:

$\Cov\left(\sum_{i=1}^n X_i, \sum_{j=1}^n X_j\right) =$ the sum of all possible covariance pairs (so $n\cdot n$ terms) (source is the book I'm currently reading)

In my thoughts, the expansion might as well be

$$\Cov(X_1, X_1)+\Cov(X_2, X_2)+\cdots+\Cov(X_n, X_n)$$

Or

$$\Cov(X_1 + X_2 + \cdots +X_n, X_1 + X_2 + \cdots +X_n)$$

$\endgroup$
4
  • 1
    $\begingroup$ It's the second one. The covariance operator is bilinear; that means it's linear in each variable separately, like the dot product. $\endgroup$ Sep 24, 2012 at 0:25
  • $\begingroup$ isnt the dot product more like the first one, where you multiply the ith term times the ith term and then add everything up $\endgroup$ Sep 24, 2012 at 0:45
  • 1
    $\begingroup$ Only if $\text{Cov}(X_i, X_j) = 0$ when $i \neq j$. Think of the $X_i$ as vectors which might not be orthogonal to each other. $\endgroup$ Sep 24, 2012 at 0:55
  • $\begingroup$ @WuschelbeutelKartoffelhuhn : The dot product $(a_1\vec{v}_1 + \cdots + a_n\vec{v}_n) \cdot (a_1\vec{v}_1 + \cdots + a_n\vec{v}_n)$ is the sum of all $n^2$ pairs $a_i a_j \vec{v}_i\cdot\vec{v}_j$. $\endgroup$ Sep 24, 2012 at 2:40

2 Answers 2

4
$\begingroup$

The other comments and answers are absolutely correct. But rather than just stating covariance identities, maybe it will be helpful to actually expand it out using the definition of covariance.

$\begin{align} \textrm{Cov}\left[\sum_{i=1}^{n}X_i,\sum_{j=1}^{n}X_j\right]&=\textrm{E}\left[\left(\sum_{i=1}^{n}X_i-\textrm{E}\left[\sum_{i=1}^{n}X_i\right]\right)\left(\sum_{j=1}^{n}X_j-\textrm{E}\left[\sum_{j=1}^{n}X_j\right]\right)\right]\\ &= \textrm{E}\left[\left(\sum_{i=1}^{n}\left(X_i-\textrm{E}\left[X_i\right]\right)\right)\left(\sum_{j=1}^{n}\left(X_j-\textrm{E}\left[X_j\right]\right)\right)\right]\\ &= \textrm{E}\left[\sum_{i=1}^{n}\left(\left(X_i-\textrm{E}\left[X_i\right]\right)\sum_{j=1}^{n}\left(X_j-\textrm{E}\left[X_j\right]\right)\right)\right]\\ &= \textrm{E}\left[\sum_{i=1}^{n}\sum_{j=1}^{n}\left(X_i-\textrm{E}\left[X_i\right]\right)\left(X_j-\textrm{E}\left[X_j\right]\right)\right]\\ &= \sum_{i=1}^{n}\sum_{j=1}^{n}\textrm{E}\left[\left(X_i-\textrm{E}\left[X_i\right]\right)\left(X_j-\textrm{E}\left[X_j\right]\right)\right]\\ &= \sum_{i=1}^{n}\sum_{j=1}^{n}\textrm{Cov}\left[X_i,X_j\right]\\ \end{align}$

$\endgroup$
3
$\begingroup$

$\newcommand{\Cov}{\operatorname{Cov}}$

Since $\Cov(A+B,C)=\Cov(A,C)+\Cov(B,C)$ and $\Cov(D,E+F)=\Cov(D,E)+\Cov(D,F)$ etc.:

  • $\Cov\left(\sum_{i=1}^n X_i, \sum_{j=1}^n X_j\right)$

  • $\Cov(X_1, X_1) + \Cov(X_1, X_2) + \ldots + \Cov(X_1, X_n) + \Cov(X_2, X_1) + \ldots +\Cov(X_n, X_n)$

  • $\Cov(X_1 + X_2 + \cdots +X_n, X_1 + X_2 + \cdots +X_n)$

all give the same result. But my second bullet has $n^2$ terms in the sum while yours has $n$; they will be the same if the $X_i$ are independent or at least pairwise uncorrelated, but otherwise will usually be different.

Incidentally, the covariance of a random variable with itself can be called its variance.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .