# Is Sample Covariance Tied to a Specific Distribution

In many sources on data analysis, the author(s) talk about calculating covariance of the data, and the formula is given as such

$$\Sigma = cov(X) = E[(X-E[X])(X-E[X])^T]$$

This formulation is given without any reference to any distribution. And as example, the author can go right to a programming environment, say R, and for famous data set Iris, and do

I am curious if the disconnect between specific distribution and data is an omission, or if covariance calculation on data, any data, can be performed for any distribution. As it happens, multivariate normal is

$$f(x;\mu,\Sigma) = \frac{ 1}{(2\pi)^{k/2} \det(\Sigma)^{1/2}} \exp \bigg\{ -\frac{ 1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu) \bigg\}$$

and I wonder if $\Sigma$ of the distribution is related to covariance calculated from data. For example maximum likelihood estimator for $\Sigma$ is

$$\frac{1}{n} \sum_{j=1}^{n} (x_j-\bar{x})(x_j-\bar{x})^T$$

This statement is almost similar to the one using $E[\cdot]$, except we have summing and dividing by $n$ which can be seen as the calculation of expectation?

I have also seen expectation and variance (covariance in multidimensional case) being talked about / derived utilizing method of moments, which makes no assumption on the underlying distribution. I guess I am trying to understand if all of the "summary statistics" have any theoretical basis (i.e. tied to a probability distribution)

Any clarification or pointers are welcome.

• Actually I think sample covariance and maximum likelihood estimator for $\Sigma$ are exactly the same thing.. Isnt expected value also a max likeli. estimator for $\mu$? – BBDynSys Feb 6 '14 at 13:26