# which is the best distance function?

We all know there are distance functions, like Kullback Leibler distance, Bhattacharyya measure, Euclidean distance, Wasserstein distance, and so on. Take a sample distance: $D=\sum\limits_n\left|P_n\left(\text{model}\right)-P_n\left(\text{sample}\right)\right|$. Specifically, if we have a model distribution (probability density function)$P\left(\text{model}\right)=[0.2,0.8]$, $n=2$, we want to calculate the distance between every sample distribution and this model distribution. If sample distribution is $P\left(\text{sample}1\right)=[0.3,0.7]$. Then $D=\left|0.3-0.2\right|+\left|0.7-0.8\right|=0.2$. But I do not think it is a good distance measure since I think $0.8$ and $0.7$ are more similar than $0.2$ and $0.3$. What I mean is in model distribution, one component is 0.8, another is 0.2, same difference $a$ has more influence in 0.2 since 0.8 is quite big than 0.2. So the weight of components are different. How should I incorporate weight into distance equation? I try to make a equation , like $D=\sum\limits_n\frac{\left|P_n\left(\text{model}\right)-P_n\left(\text{sample}\right)\right|}{P_n\left(\text{model}\right)}$, But it seems to be wrong.

-
I, personally, do not know there are distance functions like Kullback Leibler distance, Bhattacharyya distance, etc., etc. – Gerry Myerson Aug 2 '11 at 5:57
Somewhat related... but more of an interesting (imo) aside, en.wikipedia.org/wiki/Logarithm#Psychology "Psychological studies found that mathematically unsophisticated individuals tend to estimate quantities logarithmically" – Tyler Aug 2 '11 at 5:59
@ Gerry Myerson,hi, Thank you for your comment. I add two links about these two terms. – Heather Aug 2 '11 at 6:50
What the best distance is, will largely depend on the application you need it for. – Raskolnikov Aug 2 '11 at 6:52
I am pointing out that we don't all know these things, since I am a part of "we" and I never heard of any of them (except Euclidean distance). – Gerry Myerson Aug 2 '11 at 7:07

$$D=\sum\limits_n \frac{\left(P_n \left(\text{model}\right)-P_n \left(\text{sample}\right)\right)^2}{P_n \left(\text{model}\right)} \; .$$
This is used in many statistical tests and is known to be approximately $\chi^2$-distributed.