I've been looking for a way to compare two values, and every idea I think of falls short in some way. Essentially, I want a function $f$ that meets these conditions for non-negative scalars $x, y$:

If $x = 1000$ and $y = 1100$, $f(x,y)$ is not significantly large.
If $x = 0$ and $y = 1$, $f(x,y)$ is not significantly large.
If $x = 100$ and $y = 300$, $f(x,y)$ is significantly large.
If $x = 0$ and $y = 100$, $f(x,y)$ is significantly large.

Possible options I've considered:

  1. $f(x,y) = x-y$
  2. $f(x,y) = \frac{x-y}{min(x,y)}$
  3. $f(x,y) = \frac{x-y}{(x+y)/2}$
  4. $f(x,y) = \frac{x-y}{max(min(x,y),1)}$

Problems with each option:

  1. $f(6000,6100)$ = $f(100,0)$
  2. Division by zero
  3. $f(1,0) = f(100,0)$
  4. This is the best option I've considered, and is often how K/D ratios are calculated in video games. Still, it feels odd and I believe that there exists a better solution.

Some other properties of my mythical function $f$:

$f(x,y) = -f(y,x)$
$f(x,x) = 0$

I realize I have not been precise with the values I want $f$ to return, but $f(x,y)$ should essentially be the answer to the question "how big is the difference between $x$ and $y$?"

To give an example (and my current use case): I want to compare the usage of words between two texts of similar size. Obviously, "and" will be used a lot by both texts, and one may use it hundreds of times more than the other, but it is overall insignificant. Similarly, one text may use one rare word once while the other doesn't, but this too is insignificant. However, if one text uses a certain word far more often than the other text, this is statistically significant, just like if one text uses a rare word multiple times.

  • $\begingroup$ You probably want something like a relative difference (essentially, the difference, scaled by some function of the two inputs). $\endgroup$
    – Xander Henderson
    Dec 24, 2017 at 4:08
  • $\begingroup$ You are correct, that is the article I looked at while making this post, should have probably cited it. Functions 2 and 3 are two of the options from the several common choices listed on that article. $\endgroup$ Dec 24, 2017 at 4:10
  • $\begingroup$ I don't think there's all that many great answers, since based on 2 and 4 you're not getting scale invariance. But given 1 and 3 you don't seem to want additive invariance. So whatever you get is going to be pretty arbitrary. $\endgroup$
    – Jason Carr
    Dec 24, 2017 at 4:12
  • $\begingroup$ I agree that this function will be intrinsically arbitrary, and there might not be an ideal solution. However, there likely could be constant(s) in $f$ that could be tweaked to get the value returned to accurately answer the question "how big is the difference between the two values?". $\endgroup$ Dec 24, 2017 at 4:15
  • $\begingroup$ @JasonCarr see my edit for option #4 as well as my expansion on my use case. $\endgroup$ Dec 24, 2017 at 4:25

1 Answer 1


This may seem slightly complicated, though it seems to give the sort of function you are looking for.

Taking a Bayesian approach, suppose as a prior that the proportion of occurrences of the first type among first and second types is uniformly distributed on $[0,1]$, i.e. has a $\text{Beta}(1,1)$ distribution

With your observations, your posterior distribution for that proportion would be $\text{Beta}(x+1,y+1)$ distributed, and your posterior probability that the first type is less common that the second would be $q=\int_0^{1/2} \frac{p^{x}(1-p)^y}{\text{B}(x+1,y+1)}\,dp$

That will give a probability estimate $q$ in $(0,1)$. Taking the log-odds (logit) would give $\log\left(\frac{q}{1-q}\right)$ in $(-\infty,\infty)$ which may meet your needs as a function

For example in R, you could construct this function

reldist <- function(m,n){pbeta(0.5,m+1,n+1,lower.tail=TRUE, log.p=TRUE) - 

which for example would give

> reldist(1,2)
[1] 0.7884574

x,y         reldist
---------   ---------- 
1,2         0.7884574
1000,1100   4.215137
100,300     55.41071
0,100       70.00787
100,100     0
2,1         -0.7884574
1100,1000   -4.215137

You can draw significance at any point you wish. An arbitrary two-tailed probability of $5\%$ (so one-tailed of $2.5\%$) would correspond to log-odds critical points of about $\pm\log\left(\frac{0.025}{0.975}\right) \approx \pm 3.66$ so seeing $1000,1100$ might be seen as just significant on this basis; by contrast an arbitrary two-tailed probability of $0.1\%$ would correspond to log-odds critical points of about $\pm 7.6$


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .