# How to do a statistically sound ranking using varying numbers of likes/dislikes?

Assume the following problem: A popular website allows for users to either like or dislike its listed items (may it be movies, goods, people, or whatever). Each user may cast up to one like/dislike on any item she likes, but she has not to rate each and every item at display. (However, let us assume that each item has received at least one vote, i.e., ignore any items with neither like nor dislike.)

The website now likes to rank its items according to the given votes. Yet, it is quite clear that the two most trivial (and most frequently used) approaches do not yield meaningful results:

• If items are solely ranked by the total number of likes, the ranking disregards the number of dislikes.
• If items are ranked by the item's likes to dislikes ratio, the ranking ignores that items with many votes should have an edge over items with few votes. (An item with 199 likes and one dislike should end up better than an item with merely 2 likes and no dislikes.)

So my question is: How to do a sound (theoretically justifiably) ranking in the given scenario? I assume that concepts from statistical inference should be used, but I am not too familiar with that field so I don't know what concepts and models to choose.

Any help is appreciated. References to papers/publication addressing this ranking problem are also welcome.

• how about adding one dislike to all items and then using the likelihood ratio? Oct 25, 2012 at 13:52
• @Seyhmus Güngören: Do you mind to elaborate?
– MRA
Oct 25, 2012 at 15:54
• I can post a possible answer if you like. Oct 25, 2012 at 15:57
• A possible answer would be highly appreciated. ;)
– MRA
Oct 25, 2012 at 16:10
• Actually, I found my question answered (for the most part) at [this CrossValidated question][1]. [1]: stats.stackexchange.com/questions/15979/…
– MRA
Oct 29, 2012 at 14:24

Assume that you have a detection problem with two hypothesis

$$\mathcal{H}_0:x=w$$

and

$$\mathcal{H}_1:x=s+w$$

where $s$ is the signal, and $w$ is the noise. Furthermore assume that the noise has a distribution $w\sim f_0$ and the signal plus noise hast a distribution $s+w\sim f_1$. Then so called the probability likelihood ratio test

$$L(y)=\frac{f_1(y)}{f_0(y)}$$

is the optimum test in the sense of both Neyman-Pearson as well as Bayesian such that it minimizes the given cost function. I dont write the proofs as they are apperantly not necessary.

The densities are continuous, introduce no mass and have no zeros at some real numbers except for $\infty$ and $-\infty$. As a result likelihood ratio is well defined.

A simple hypothesis testing problem is therefore uniquely determined by the likelihood ratio test. One needs no other statistics or something else then using the likelihood ratio test to get the optimum performance.

Assume that likes is $w$ and dislikes is $s+w$, then you have the hypothesis testing problem, to say if likes dominate or dislikes dominate. As the theory says you should be able to determine this amount only by the likelihood ratio test unless your likelihood ratio is not well defined.
• Thanks for your answer. If I understand correctly you propose a likelihood test to decide whether a particular item is generally liked ($H_0$) or disliked ($H_1$). I agree that this should be possible. However, I do not see how this helps to rank all items accordingly. Do you suggest ranking by likelihood ratios? If so, can you point me to some literature that discusses why such an approach avoids the fallacies given in the opening post?
• Also, thinking about it, wouldn't most (simple) models of a user's voting behavior pretty much boil down to a simple "likes to dislikes ratio" test? E.g., model $\Pr[like\ x_i]=\delta_i$, $\Pr[dislike\ x_i]=\epsilon_i$, and $\Pr[ignore\ x_i]=1-\delta_i-\epsilon_i$ with like hypothesis $H_0: \delta_i>\epsilon_i$ and dislike hypothesis $H_1: \epsilon_i>\delta_i$?