Covariance is defined as:


Is the following equation derived from the one above? If yes, how? By treating the expected value as an arithmetic mean? Expected value is indeed equal to arithmetic mean when the probabilities are equal.


The question is - why do we divide by $n-1$ instead of $n$? If it's an arithmetic mean, we are adding up $n$ terms and dividing by the number of terms $n$.


You question is answered in this Wikipedia section.

The first quantity is the [true] covariance and the second quantity is sample covariance, which is an estimate of the true covariance. You are correct in your intuitive interpretation of the sample covariance being an average. The $n-1$ makes the sample covariance unbiased, which means that if you think of the $x_i$ and $y_i$ as i.i.d. random variables drawn from the distributions of $X$ and $Y$ respectively, then the expectation of the sample covariance is the true covariance. This is an unfortunate quirk that arises due to the fact that the sample mean $\bar{x}$ is not the true mean $\mathbb{E}[X]$.

  • $\begingroup$ 'if you think of the xi and yi as i.i.d. random variables' - did you mean values of random variables? $\endgroup$ – user216094 Mar 20 '15 at 20:07
  • $\begingroup$ @user216094 It's a small distinction I guess. What I mean is that $X_1,\ldots, X_n$ are random variables with the same distribution as $X$ (similarly for $Y$), and the data you observe is one realization $x_1,\ldots, x_n$. But in the definition of unbiased, I mean that $$\frac{1}{n-1}\mathbb{E}\left[\sum_{i=1}^n (X_i-\bar{X})(Y_i-\bar{Y})\right] = \operatorname{Cov}(X,Y),$$ where $\bar{X}:= \frac{1}{n}\sum_{i=1}^n X_i$, and similarly for $\bar{Y}$. $\endgroup$ – angryavian Mar 20 '15 at 20:11
  • $\begingroup$ $x_i$ and $y_i$ are specific values of random variable $X$ and $Y$, respectively. Because we've collected some real data, they are specific numbers. Then why do you call them random variables? I don't understand your point here. $\endgroup$ – user216094 Mar 20 '15 at 20:17
  • $\begingroup$ @user216094 Yes, you are right, they are specific numbers, and the sample covariance is indeed a number. (Note also that true covariance is also a number.) However, I was explaining reason why $n-1$ is used instead of $n$, and it comes from the notion of being an unbiased estimator, which involves this interpretation of the data (the specific numbers you see) being generated from a probability distribution. $\endgroup$ – angryavian Mar 20 '15 at 20:27
  • $\begingroup$ Ok, but now I'm referring to this part only: ''if you think of the xi and yi as i.i.d. random variables'. Is it better to call xi and yi values of random variables? $\endgroup$ – user216094 Mar 20 '15 at 20:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.