Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have a set of $N$ votes $O_1, O_2,..O_N$ distributed into $n$ bins. So...

$$n \le N$$ $$0 \le O_i \le N$$ $$\sum_{i=1}^{n} O_i = N$$

I want to generate some sort of metric for how uniformly distributed they are. I need that metric to be a value between zero and one. (This is the tricky part. I have read info on this topic, both here and on other sites, but I haven't found anything that addresses the maximum/worst case, which would allow me to scale it.) My application isn't really statistical$^†$ but I think that Pearson's chi-squared test would be a reasonable choice here.

$$\chi^2 = \sum_{i=1}^{n} \frac{(O_i - E_i)^2}{E_i}$$

Since $E_i=\frac{N}{n}$ for all $i$,

$$\chi^2 = \sum_{i=1}^{n} \frac{(O_i - \frac{N}{n})^2}{\frac{N}{n}}$$ $$= \frac{n}{N}\sum_{i=1}^{n} (O_i - \frac{N}{n})^2$$ $$= \frac{n}{N}\sum_{i=1}^{n} (\frac{1}{n})^2(nO_i - N)^2$$ $$= \frac{1}{nN}\sum_{i=1}^{n} (nO_i - N)^2$$

Now, I want to scale that $\chi^2$ function appropriately. In the best case (perfect uniform distribution), $O_i = E_i$ for all $i$, and so $\chi^2 = 0$. My intuition tells me that the worst case is where one measurement is N and the other $n-1$ are zero; is that right? If so, then

$$\chi^2_{worst} = \frac{1}{nN}[(nN - N)^2 + (n-1)(-N)^2]$$ $$= \frac{N}{n}[(n - 1)^2 + (n-1)]$$ $$= \frac{N}{n}[n^2 -n]$$ $$= N(n-1)$$

And so $\frac{\chi^2}{\chi^2_{worst}}$ gives me a value between 0 and one.

My questions are:

  1. Is this a reasonable approach? I just need some sort of metric so I can compare different distributions.
  2. See the "is that right", in bold above.

$^†$If you need more info: I'm working with sets of $n$ models. Each set of models has been used to classify a set of $N$ input patterns. I want to tell how well a particular set of models is doing at capturing the diversity of the patterns. The worst case would be if one model in the set matches all of the patterns, and the other models are unused. The best case would be where all models get the same number of matches. The "votes" that I refer to indicate when a model has matched an input pattern. And the reason that it needs to be between zero and one is that I'm going to penalise bad solutions. I could just cap the penalty, but it would be nice to know that it will never exceed a certain value.

share|improve this question
1  
If you're looking for a method of comparing distributions, you might want to try the Kolmogorov-Smirnov test, en.wikipedia.org/wiki/KS_Test, which gives a measure of closeness of two probability distributions based on their maximum difference. It's available in most decent statistical software packages. For the KS test, your intuition of the worst case being N samples in one bin and 0 in all others is indeed correct. –  ymbirtt Oct 15 '13 at 10:20
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.