1
$\begingroup$

I am calculating Hellinger distance between two probability distributions. I believe that the hellinger distance is bounded by 0 to 1. Now I have seen implementations of "similarity" calculated by 1/(1+distance) rather than simply doing (1-distance) to get the similarity.

I've been looking for an explanation but can't seem to find one. Could anyone explain or shed some light on this?

$\endgroup$
0
$\begingroup$

(1) For a distance $d$ that's not bounded, the similarity $S_1 = 1/(1+d)$ guarantees that $S_1 \in[0,1]$, whereas $S_2 = 1-d$ can be negative for large $d$. This reason doesn't really matter if $d\in [0,1]$, and in that case $S_2$ may be better.

(2) Depending on the application, you may want to consider outliers differently. (E.g., the similarity might be getting used in some kind of function being optimized). This is similar to choosing a distance function for an optimization problem, between $L_1$ vs $L_2$ distances. Notice that $\partial_d S_1 = -(1+d)^{-2} $, so for very large $d$, the distance basically stops changing (i.e., the difference between a large $d$ and, say, $2d$ translates to very little difference for $S_1$.) This means $S_1$ is a bit more robust to outliers, wrt distance. On other hand, $\partial_d S_2 = -1$, which is constant! The penalty is always proportional to the distance, regardless of the magnitude. Note that both functions are similar, as are their derivatives, near $0$, but away from there, $S_1$ progressively desensitizes itself to larger distances (at a quadratic rate). TL;DR: the behaviour of the two transforms is quite different (especially when $d$ has a very large range).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.