1
$\begingroup$

Assume the following problem: A popular website allows for users to either like or dislike its listed items (may it be movies, goods, people, or whatever). Each user may cast up to one like/dislike on any item she likes, but she has not to rate each and every item at display. (However, let us assume that each item has received at least one vote, i.e., ignore any items with neither like nor dislike.)

The website now likes to rank its items according to the given votes. Yet, it is quite clear that the two most trivial (and most frequently used) approaches do not yield meaningful results:

  • If items are solely ranked by the total number of likes, the ranking disregards the number of dislikes.
  • If items are ranked by the item's likes to dislikes ratio, the ranking ignores that items with many votes should have an edge over items with few votes. (An item with 199 likes and one dislike should end up better than an item with merely 2 likes and no dislikes.)

So my question is: How to do a sound (theoretically justifiably) ranking in the given scenario? I assume that concepts from statistical inference should be used, but I am not too familiar with that field so I don't know what concepts and models to choose.

Any help is appreciated. References to papers/publication addressing this ranking problem are also welcome.

$\endgroup$
7
  • $\begingroup$ how about adding one dislike to all items and then using the likelihood ratio? $\endgroup$
    – Seyhmus Güngören
    Oct 25, 2012 at 13:52
  • $\begingroup$ @Seyhmus Güngören: Do you mind to elaborate? $\endgroup$
    – MRA
    Oct 25, 2012 at 15:54
  • $\begingroup$ I can post a possible answer if you like. $\endgroup$
    – Seyhmus Güngören
    Oct 25, 2012 at 15:57
  • $\begingroup$ A possible answer would be highly appreciated. ;) $\endgroup$
    – MRA
    Oct 25, 2012 at 16:10
  • $\begingroup$ Actually, I found my question answered (for the most part) at [this CrossValidated question][1]. [1]: stats.stackexchange.com/questions/15979/… $\endgroup$
    – MRA
    Oct 29, 2012 at 14:24

1 Answer 1

Reset to default
1
$\begingroup$

Assume that you have a detection problem with two hypothesis

$$\mathcal{H}_0:x=w$$

and

$$\mathcal{H}_1:x=s+w$$

where $s$ is the signal, and $w$ is the noise. Furthermore assume that the noise has a distribution $w\sim f_0$ and the signal plus noise hast a distribution $s+w\sim f_1$. Then so called the probability likelihood ratio test

$$L(y)=\frac{f_1(y)}{f_0(y)}$$

is the optimum test in the sense of both Neyman-Pearson as well as Bayesian such that it minimizes the given cost function. I dont write the proofs as they are apperantly not necessary.

The densities are continuous, introduce no mass and have no zeros at some real numbers except for $\infty$ and $-\infty$. As a result likelihood ratio is well defined.

A simple hypothesis testing problem is therefore uniquely determined by the likelihood ratio test. One needs no other statistics or something else then using the likelihood ratio test to get the optimum performance.

Translation to your problem:

Assume that likes is $w$ and dislikes is $s+w$, then you have the hypothesis testing problem, to say if likes dominate or dislikes dominate. As the theory says you should be able to determine this amount only by the likelihood ratio test unless your likelihood ratio is not well defined.

As I suggested some sort of modification of your statistics such that you will not get zeros at the nominator and denominator should solve your problem.

$\endgroup$
4
  • $\begingroup$ Thanks for your answer. If I understand correctly you propose a likelihood test to decide whether a particular item is generally liked ($H_0$) or disliked ($H_1$). I agree that this should be possible. However, I do not see how this helps to rank all items accordingly. Do you suggest ranking by likelihood ratios? If so, can you point me to some literature that discusses why such an approach avoids the fallacies given in the opening post? $\endgroup$
    – MRA
    Oct 26, 2012 at 10:05
  • $\begingroup$ Your first remark in the question is definitely correct. One should use both likes and dislikes in the measure for the ranking. In the second comment you have null points in case you have zeros. As long as you have the way to deal with zeros, you re fine. The likelihood ratio has all the information about deciding if likes or dislikes dominate with respect to one other. It is similar to assign +1 to likes and -1 to dislikes and sum them up. This will be the sufficient statistic for your problem as soon as likes and dislikes have the equal weights. I have "Steven Kay, detection theory" $\endgroup$
    – Seyhmus Güngören
    Oct 26, 2012 at 10:24
  • $\begingroup$ Also, thinking about it, wouldn't most (simple) models of a user's voting behavior pretty much boil down to a simple "likes to dislikes ratio" test? E.g., model $\Pr[like\ x_i]=\delta_i$, $\Pr[dislike\ x_i]=\epsilon_i$, and $\Pr[ignore\ x_i]=1-\delta_i-\epsilon_i$ with like hypothesis $H_0: \delta_i>\epsilon_i$ and dislike hypothesis $H_1: \epsilon_i>\delta_i$? $\endgroup$
    – MRA
    Oct 26, 2012 at 10:29
  • $\begingroup$ Yes. They have similar meanings. However voting is coming from another concept, that is data fusion. There is another theory for that. $\endgroup$
    – Seyhmus Güngören
    Oct 26, 2012 at 10:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.