# Covariance Matrix

I'm studying Pattern Classification and there are some terms that I'm not familiar with.

first one is Covariance Matrix.

Say I have database of K samples, each one is a length d feature vector.

How do I build the Covariance Matrix from my database and what is the meaningful of that matrix?

-
This is a rather large topic. Have you tried consulting standard sources such as statistics textbooks or even wikipedia? In short, the covariance matrix encodes many useful properties of the underlying distribution, the rough shape of the distribution being one of them. – EuYu Dec 9 '12 at 9:10
wikipedia was too generic for me.. – Bush Dec 9 '12 at 9:21
The article details exactly how you construct a covariance matrix. As for what the matrix means, it is a measure of the spread of the data. Is there something specific that you would like to know? – EuYu Dec 9 '12 at 9:26

The covariance matrix consists of covariances $\sigma_{i,j}$ and standard deviations $\sigma_{i,i} = \sqrt{\mathrm{Variance}(X_i)}$ on the diagonals.

$$\begin{bmatrix} \color{Olive}{\sigma_{1,1}} & \sigma_{1,2} & \sigma_{1,3} \\ \sigma_{2,1} & \color{Olive}{\sigma_{2,2}} & \sigma_{2,3} \\ \sigma_{3,1} & \sigma_{3,2} & \color{Olive}{\sigma_{3,3}} \end{bmatrix}$$

Notice that $\sigma_{i,j} = \sigma_{j,i}$. (Think about the meaning of "correlation", which derives from $\sigma_{i,j}$. I can't be correalted with you if you're not correlated with me.)

You might want to check out http://stats.stackexchange.com as well.

-