Algorithm to calculate rating based on multiple reviews (using both review score and quantity)

Question

First of all I must state that I am not a mathematician, so please correct me if I use wrong terminology.

I am building a web application which needs to calculate the rating for each entity based on both the quantity and score of the reviews for that entity.

In other words, I don't want to just calculate the rating based on the average score as that would make an entity with one hundred 9 score (review score can be from 0 to 10) reviews rate lower than an entity with only one 9.5 score review.

I need an algorithm to calculate rating and add rating "weight" to the final rating based on how many reviews the entity has, so that for instance in the above example the entity with 100 9 score reviews would get a rating that is higher than the entity with only one 9.5 score review. In other words, the final entity rating score will be based on the "relationship" between quality and quantity.

There is another important thing to note: an entity can not have a rating higher than 10, so the rating "weight" added by the quantity can not be linear.

In the algorithm we can use any data about the reviews/rating, that is individual review score, total number of reviews, sum of all reviews, number of good reviews (score 8 or higher) so far, etc, in each iteration of the rating calculation process.

Any kind of help or info regarding this would be appreciated. Thank you.

Answer 1

What you can do, for instance, is take the rate of reviews (w weighted mean), divide it by two (to reduce the scoring to a scale of $[0,5]$ and add this value to $5(1-e^{-q})$. So the formula becomes $$\text{score}=5p/10+5(1-e^{-q/Q})$$ where $p$ is the review rating and $q$ is the number of ratings, and you chose for $Q$ an appropriate number that shows what importance you attach to the notion "quantity."

An example: An item has $3$ times a revision score of $6$ and $2$ times a revision score of $7$.

Then $p=(3⋅6+2⋅7)/5=6.4$

If we take $Q=10$, then $5(1-e^{-5/10})\approx 3.88$, so the total score is $3⋅2+3⋅9=7.1$ rounded $7$.

On the other hand, if somebody has $20$ scorings of $6$, then $p=6$ and $5(1-e^{-20/10})\approx 4.58$, so the final score is $3+4.6$ rounded giving $8$.

The choice of $Q$ depends on what you call "few," "moderate," "many."

As a rule of thumb, consider a value $M$ that you consider "moderate", and take $Q=-M/\ln(1/2)\approx 1.44M$.

So if you think $100$ is a moderate value, then take $Q=144$.

Finally, you can also replace the equal weight on quantity and quality by a skewed one so that the final formula becomes:$$\text{score}=Pp+10(1-P)(1-e^{-q/Q}))$$ where $P\in [0,1]$ (in the original formula we had $P=0.5$).

Answer 2

You can do all the things you speculate about. You need to decide how to weight them. You say you think 100 reviews of 9 are better than one review of 9.5, but how about 10 reviews of 9? Five? How about 90 reviews of 9 and 10 of 2-how does that compare to one review of 9.5?

One simple thing to do is give points for lots of reviews: no points for no reviews, 1 point for 1 review, 2 points for 2-5 reviews, and so on. You will have to scale it to your population. I am guessing that small numbers of reviews are common, which is why I started with small brackets, but you need to look at your data. Now add these points to the average score, and you have a total in the range 0 to 20.

Another approach is to find the average review-say it is a 7. Now just count the total number of points and subtract 7 times the number of reviews. Your single review of 9.5 is then 2.5 points above average for the number of reviews it has gotten. The 100 reviews of 9 are 200 points better, the 90 reviews of 9 and 10 reviews of 2 are $90 \cdot 2 +10 \cdot (-5)=130$ points better than average, etc.