Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I am trying to figure out how to use the Mahalanobis distance still. I am having trouble figuring out how to produce my own covariance matrix. I guess the relevant link is http://en.wikipedia.org/wiki/Estimation_of_covariance_matrices, but I am still stuck. What I have is a set of points in space, every single point in my data set. I want to find the distance between any two of them using this type of distance, but how do I form the covariance matrix? The webpage is not very clear to me.

share|improve this question

1 Answer 1

up vote 1 down vote accepted

Let the points be $\{\mathbb{x}_i\}_{i=1}^N$. Then you can compute the sample covariance as follows:

\begin{align} \Sigma &= \frac{1}{N-1}\sum_{i=1}^N (\mathbf{x}_i - \mu)(\mathbf{x}_i - \mu)^T \end{align}

where

\begin{align} \mu &= \frac{1}{N}\sum_{i=1}^N \mathbf{x}_i \end{align} is the sample mean.

Note that I have used $N-1$ in the denominator for $\Sigma$ instead of $N$ to make the estimator unbiased.

share|improve this answer
    
Those are pretty much the same equations on the link I gave. But as far as I understand, a covariance matrix is a square matrix. What would an entry at row i, column j look like? –  mathguy Aug 27 '12 at 3:50
    
Each term in the expression for $\Sigma$ is a square matrix, since it is the outer-product of a vector with itself. The $(k,l)$ entry would look like: $\frac{1}{N-1}\sum_{i=1}^N (x_{i,k} - \mu_k)(x_{i,l} - \mu_l)$, where $x_{i,k}$ is the $k^{th}$ entry of $\mathbf{x}_i$. This is nothing but the sample covariance between components $k$ and $l$ of $\mathbf{x}$. –  Kartik Audhkhasi Aug 27 '12 at 4:00
    
Oh I see..My thinking was that I would get a 1x1 matrix, but ofcourse I am not multiplying it like that. Thanks! –  mathguy Aug 27 '12 at 5:15
    
You would get a $1\times 1$ matrix if you were taking inner products of the column vectors. That would just give you the trace (sum of diagonal entries) of the covariance matrix. –  Kartik Audhkhasi Aug 27 '12 at 5:30

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.