Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I'm working on a computer program but I need help coming up with a formula. This program has list of customers that wrote reviews for products. For each review written by this customers there is a number of votes and also a number indicating how many of these votes marked the review as helpful (think of Amazon).

And I would like to come up with a shorter list of expert reviewers. The way I envisioned my solution was to assign weights to each customers based on the votes given to the reviews, so I thought a ratio between the number of votes and number of those votes marked as "helpful" could be meaningful.

However, I ran into instances in which one reviewer had 1 vote and that one vote was marked as helpful, thus the ratio = 1/1 =1

And there are some other instances in which a customer had 19 votes and all of those votes were marked as helpful. that is 19/19 = 1

It's evident that mathematically they are similar; however I would like to assign a bigger weight to the customer who got 19 votes versus only 1. Similarly, I believe a review that got 24 helpful votes out of 25 (24/25 = .96) should have a bigger weight than the customer who got 1 vote which was marked as helpful.

Do you have any thoughts on how I could come up with a formula that takes into consideration the ratio and also the number of reviews?

Thank you!

share|improve this question
    
I appreciate everybody's help. The answer is indeed subjective and the solutions proposed are a great start. Another consideration is that records with few reviews (1/1) should probably be filtered out. –  Ulises May 6 '11 at 4:02

3 Answers 3

up vote 4 down vote accepted

You need to define carefully what you want, then you can certainly make a formula to do that. For example, maybe unhelpful votes are rare (as they are on this site), and you feel they should give a big penalty. You could have a score of helpful-10*unhelpful. This will make 19/19 beat 1/1 badly, and even 24/25 will beat 1/1, but 8/9 will lose to 1/1. Is that what you want?

Maybe a good way to think about it is what (n-1)/n score should rank even with 1/1? With 2/2? What (n-2)/n score should be even with 1/1? If you get a number of these, a formula can be constructed.

share|improve this answer

The simplest way is to add some numbers to the numerator and denominator: for example if you added $5$ and $10$ then you would get the reviewers ranked as follows:

Original Adjusted Decimal 
24/25     29/35    0.829
19/19     24/29    0.828
 1/1       6/10    0.600

or you could choose other adjustments.

share|improve this answer

Since this is pretty subjective, what you are looking for is a way to quantify your subjective opinion about what qualifies as a good reviewer and what qualifies as a bad one. In that case, I suggest you try to visualize it by drawing a heat map.

Make a graphic with on the horizontal axis the number of reviews and on the vertical axis the number of helpful reviews. Since there can't be more helpful reviews than there are reviews, only the half of the quadrant below the line $y=x$ will be colored. Then, you colour all the points on the graphic for which you want to attribute the same weight. For instance, if all the people with the same number of helpful reviews have to receive the same weight, you give them the same color. If all the people with the same ratio of helpful to total reviews need to get the same weight, you give them the same color. If not, you find another color scheme. The colors is just to make it more pictorial, it will in the end of course correspond to the weights.

Once you have done that, you can either directly work with this heat map to associate weights, or you can look for the function of reviews and helpful reviews that best approximates your heat map.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.