8
$\begingroup$

I am really stuck on calculating the Mahalanobis distance. I have two vectors, and I want to find the Mahalanobis distance between them. Wikipedia gives me the formula of $$ d\left(\vec{x}, \vec{y}\right) = \sqrt{\left(\vec{x}-\vec{y}\right)^\top S^{-1} \left(\vec{x}-\vec{y}\right) } $$

Suppose my $\vec{y}$ is $(1,9,10)$ and my $\vec{x}$ is $(17, 8, 26)$ (These are just random), well $\vec{x}-\vec{y}$ is really easy to calculate (it would be $(16, -1, 16)$), but how in the world do I calculate $S^{-1}$. I understand that it is a covariance matrix, but that is about it. I see an "X" everywhere, but that only seems like one vector to me. What do I do with the Y vector? Can someone briefly walk me through this one example? Or any other example involving two vectors.

|cite|improve this question
$\endgroup$
  • 3
    $\begingroup$ The matrix $S$ must be part of your problem description. Is it assumed to be an identity matrix, maybe? $\endgroup$ – Sasha Aug 23 '12 at 17:37
  • $\begingroup$ @Sasha so what you are saying is that $S$ is not defined by $\vec x$ and $\vec y$? $\endgroup$ – Sidd Singal Aug 23 '12 at 17:39
  • $\begingroup$ No, it is not. It has be additionally specified. $\endgroup$ – Sasha Aug 23 '12 at 17:40
  • $\begingroup$ @Sasha Thank you so much! I guess I won't be able to use this then. Basically, I just have a bunch of vectors in an $n$-dimensional space and I needed to find the closest point to any given point in the space. $\endgroup$ – Sidd Singal Aug 23 '12 at 17:44
6
$\begingroup$

I think, there is a misconception in that you are thinking, that simply between two points there can be a mahalanobis-distance in the same way as there is an euclidean distance. For instance, in the above case, the euclidean-distance can simply be compute if $S$ is assumed the identity matrix and thus $S^{-1}$ the same. The difference, which Mahalanobis introduced is, that he would compute a distance, where the measurements (plural!) are taken with a correlated metric, so to say. So $S$ is not assumed to be the identity matrix (which can be understood as special correlation-matrix where all correlations are zero), but where the metric itself is given in correlated coordinates, aka correlation-values in the $S $ matrix, which are also cosines betwen oblique coordinate axes (in the euclidean metric they are orthogonal and their cosine/their correlation is zero by definition of the euclidean).

But now - what correlation does Mahalanobis assume? This are the empirical correlations between the x and the y - thus we need that correlations from external knowledge or from the data itself. So I'd say in answering to your problem, that the attempt to use Mahalanobis distance requires empirical correlations, thus a multitude of x- and y measurements, such that we can compute such correlations/ such a metric: it does not make sense to talk of Mahalanobis-distance without a base for actual correlations/angles between the axes of the coordinatesystem/the measure of the obliqueness in the metric.

Edit:When all correlations are zero, $S$ is diagonal, but not necessarily identity matrix. For $S$ to be equal to identity matrix all sampled variables must have equal value of standard deviation. The $i$-th diagonal element of the matrix $S$ represents the metric for $i$-th variable in units of $\sigma_i$ (or proportional to $\sigma_i$)

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks, I think understand now from you and Sasha. I'm still in high school trying to learn a lot about math what high school won't teach, so a lot of this is new to me. Anyways, I am not sure if this next question is naive, but can I calculate these correlations from, lets say, all the vectors that exist in the vector space my data produces? $\endgroup$ – Sidd Singal Aug 23 '12 at 17:52
  • $\begingroup$ I have more context over here (this is where the Mahalanobis distance was suggested) stats.stackexchange.com/questions/34827/… $\endgroup$ – Sidd Singal Aug 23 '12 at 17:54
  • 1
    $\begingroup$ @Gottfried Helms: an anonymous user proposed a rather long edit to your answer. Are you just not logged on or is that another user? Should that edit be ignored? $\endgroup$ – Dennis Gulko May 13 '13 at 10:25
  • $\begingroup$ @Dennis: I'm not yet used to this sort of approve-edits. I've to look at it later, don't have my head free for this at the moment. Thanks for your hint anyway, I would not have been aware of this. $\endgroup$ – Gottfried Helms May 13 '13 at 10:30
0
$\begingroup$

Maybe a simple geometric interpretation will help: Suppose you have a large set of vectors, in your case sets of 'triplets'. Each point can be represented in a 3 dimensional space, and the distance between them is the Euclidean distance.

But suppose when you look at your cloud of 3d points, you see that a two dimensional plane describes the cloud pretty well. So project all your points perpendicularly onto this 2d plane, and now look at the 'distances' between them.

Or maybe a single vector can be defined from the origin through 3d space. Similaly, project ALL your points onto this 1d vector, and look at their 'distances'.

These 1d and 2d 'distances' are called Mahalanobis distances in the reduced 1d and 2d spaces.

I like mental 'pictures' to explain math... hope that helps.

|cite|improve this answer
$\endgroup$
0
$\begingroup$

You need at least 2 data point to be able to compute the Covariance in 2 dimensions and you need to have at least 3 data points for computation of Covariance in 3 dimension. Once you have the required data points, you can use mahalanobis distance calculator in http://people.revoledu.com/kardi/tutorial/Similarity/MahalanobisDistance.html to help you solve this problem.

|cite|improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.