9
$\begingroup$

First of all I must state that I am not a mathematician, so please correct me if I use wrong terminology.

I am building a web application which needs to calculate the rating for each entity based on both the quantity and score of the reviews for that entity.

In other words, I don't want to just calculate the rating based on the average score as that would make an entity with one hundred 9 score (review score can be from 0 to 10) reviews rate lower than an entity with only one 9.5 score review.

I need an algorithm to calculate rating and add rating "weight" to the final rating based on how many reviews the entity has, so that for instance in the above example the entity with 100 9 score reviews would get a rating that is higher than the entity with only one 9.5 score review. In other words, the final entity rating score will be based on the "relationship" between quality and quantity.

There is another important thing to note: an entity can not have a rating higher than 10, so the rating "weight" added by the quantity can not be linear.

In the algorithm we can use any data about the reviews/rating, that is individual review score, total number of reviews, sum of all reviews, number of good reviews (score 8 or higher) so far, etc, in each iteration of the rating calculation process.

Any kind of help or info regarding this would be appreciated. Thank you.

$\endgroup$
  • $\begingroup$ If I understand correctly you want to build a score based on both "quality" and "quantity". $\endgroup$ – Marc Bogaerts Sep 23 '14 at 12:11
  • $\begingroup$ Yes. I want the rating score to be based on the relationship between quality and quantity of the entity, if that makes sense. $\endgroup$ – Andrius Bartulis Sep 23 '14 at 12:45
15
$\begingroup$

What you can do, for instance is take the rate of reviews (w weighted mean), divide it by two (in order to reduce the scoring to a scale of $[0,5]$ and add this value to $5(1-e^{-q})$. So the formula becomes $$\text{score}=5p/10+5(1-e^{-q/Q})$$ where $p$ is the review rating and $q$ is the quantity of ratings and you chose for $Q$ an appropriate number that shows what importance you attach to the notion "quantity". An example: An item has $3$ times a revision score of $6$ and $2$ times a revision score of $7$. Then $p=(3.6+2.7)/5=6.4$ if we take $Q=10$ then $5(1-e^{-5/10})\approx 3.88$ so the total score is $3.2+3.9=7.1$ rounded $7$. On the other hand if somebody has $20$ scorings of $6$ then $p=6$ and $5(1-e^{-20/10})\approx 4.58$ so the final score is $3+4.6$ rounded giving $8$. The choice of $Q$ depends on what you call "few", "moderate", "many". As a rule of thumb consider a value $M$ that you consider "moderate" and take $Q=-M/\ln(1/2)\approx 1.44M$. So if you think $100$ is a moderate value then take $Q=144$. Finally you can also replace the equal weight on quantity an quality by a skewed one so that the final formula becomes:$$\text{score}=Pp+10(1-P)(1-e^{-q/Q}))$$ where $P\in [0,1]$ (in the original formula we had $P=0.5$).

$\endgroup$
  • $\begingroup$ Thanks. This kind of a formula is exactly what I was looking for. From what I can see, the maximum score will never go above 10, which is what I want. I will implement this in code and let you know if it works as expected. Thanks again. $\endgroup$ – Andrius Bartulis Sep 24 '14 at 7:11
  • $\begingroup$ @AndriusBartulis Good idea, I'm looking forward to it. $\endgroup$ – Marc Bogaerts Sep 24 '14 at 8:06
  • $\begingroup$ @MarcBogaerts Hi Marc, thanks for this. I have set p = 0.5 and then do this in Ruby: (p * average) + 10 * (1 - p) * (1 - ((Math::E)**(-number_scores / q))) When I set q = 5 I can see that if I have somebody who has 6, 7, 8, 9 or 10 scores of 10/10 give the exact same result. Is that normal or have I made an error? what I see is that somebody who $\endgroup$ – rept Oct 12 '17 at 12:50
  • $\begingroup$ @MarcBogaerts grrr sorry, had error in the formula, needs to be number_scores.to_f to convert it to a float first. Thanks again for the answer. $\endgroup$ – rept Oct 12 '17 at 16:16
  • $\begingroup$ @MarcBogaerts Hi Marc, If each user has a point (weight) similar to that of stack exchange, can this formula be modified to include the user's point also? $\endgroup$ – BEFL Apr 2 '18 at 12:21
1
$\begingroup$

You can do all the things you speculate about. You need to decide how to weight them. You say you think 100 reviews of 9 are better than one review of 9.5, but how about 10 reviews of 9? Five? How about 90 reviews of 9 and 10 of 2-how does that compare to one review of 9.5?

One simple thing to do is give points for lots of reviews: no points for no reviews, 1 point for 1 review, 2 points for 2-5 reviews, and so on. You will have to scale it to your population. I am guessing that small numbers of reviews are common, which is why I started with small brackets, but you need to look at your data. Now add these points to the average score, and you have a total in the range 0 to 20.

Another approach is to find the average review-say it is a 7. Now just count the total number of points and subtract 7 times the number of reviews. Your single review of 9.5 is then 2.5 points above average for the number of reviews it has gotten. The 100 reviews of 9 are 200 points better, the 90 reviews of 9 and 10 reviews of 2 are $90 \cdot 2 +10 \cdot (-5)=130$ points better than average, etc.

$\endgroup$
  • $\begingroup$ Thanks for your answer. I can see how this could work, but I will stick to Nimda's solution as it normalizes the total score to a scale that does not go beyond a value of 10. $\endgroup$ – Andrius Bartulis Sep 24 '14 at 7:13

protected by Community Jan 11 at 16:30

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.