In descriptive statistics, the empirical covariance of a point cloud $\left\{(x_i,y_i) : 1\le i\le n \right\}$ is defined as $$\operatorname{cov}(x,y):=\frac{1}{n}\sum_{i=1}^n\left(x_i-\overline{x}\right)\left(y_i-\overline{y}\right)$$ where $\overline{z}=\frac{1}{n}\sum_{i=1}^nz_i$ is the arithmetic mean.
That's easy to interpret:
- $z_i$ can somehow considered to be large if $z_i-\overline{z}>0$, i.e. $z_i$ is greater than the mean.
- Analogously, $z_i$ can somehow considered to be small if $z_i-\overline{z}<0$, i.e. $z_i$ is smaller than the mean.
- Thus, we can divide the $xy$-plane into four quadrants:
Or even simpler:
- So, $\left(x_i-\overline{x}\right)\left(y_i-\overline{y}\right)<0$ means that the pair $(x_i,y_i)$ lies in the left upper or right lower quadrant, i.e. $x_i$ is small and $y_i$ is large, or vice versa.
- Analogously, $\left(x_i-\overline{x}\right)\left(y_i-\overline{y}\right)>0$ means that the pair $(x_i,y_i)$ lies in the right upper or left lower quadrant, i.e. $x_i$ is small and $y_i$ is small, or both are large.
- Thus, the mean over all pairs indicates how these pairs change together. If the mean is negative, the $x$- and $y$-values tend to show opposite behavior. If the man is positive, they tend to show similar behavior.
Now I would really like to transport this understanding to the covariance of random variables $X$ and $Y$, which is defined as $$\operatorname{cov}(X,Y):=\operatorname{E}\left[\left(X-\operatorname{E}\left[X\right]\right)\left(Y-\operatorname{E}\left[Y\right]\right)\right]$$ How can we interpret the sign of $\operatorname{cov}(X,Y)$ here?
