Granularity, Skewing, Beauties and Beasts: why your ratings scale matters


What does a change in rating scale mean for Netflix?

This week, Netflix announced a Yes/No rating scale instead of a 5-point rating.  Why does a ratings scale matter? We asked Voluntas Head of Research and Insight, Katy Wilburn, about the impact of the scale change on future ratings.

The big difference that it makes is that the ratings scale has a massive impact on the overall positive score. People intuitively see ratings of 4 or 5 on a 5-point scale as positive.  Whereas comparatively, the ‘Yes’ on the Yes/No scale is the positive and ‘No’ is the negative.  The problem is, in my experience, it’s much easier to get people to say ‘Yes’ on a Yes/ No scale than it is to get people to say 4 or 5 on a 5-point scale.

What does it means for Netflix?

All their videos are now likely to have a higher positive score on the Yes/No scale than on the 5-point scale.  But for people, trying to decide whether to watch a video or not, they will have lost that granularity of distinguishing between a rating of 2, 3, 4, or 5 stars.

Head of Research and Insight, Katy Wilburn

Head of Research and Insight, Katy Wilburn

Here’s a real world example:

I went to see Disney’s Beauty and the Beast at the weekend with my son, and to be honest, I’m not a Disney princess type person. But, if someone asked me to score the film out of 5, I’d have to give it some credit:

First of all, the CGI was good.  Secondly, the comparison with the original cartoon was quite good.

So, I would have scored it 3 stars on that basis.

But, if someone had asked me on the way out of the cinema “would you recommend that someone come see this film?”  I would have said “Yes” and the reason why is because it wasn’t sufficiently bad enough for me to say “No”. 

The Pros and Cons

One of the pros of Yes/No is that it isn’t a graded decision to watch a video on Netflix – it is either a yes or no, you watch it or you don’t.  So that’s a ‘pro’ point for Yes/No, because the decision you are making to watch a film or not is Yes/No.  There is alignment between the ratings you give and the choice to view or not – both are binary. So that alignment is a reason why you might change your grading scale to Yes or No.

The impact to Netflix

The impact it will start having for Netflix is that people will stop paying attention to the ratings.  The reason is, that on a Yes/No scale, ratings are skewed toward the positive.  What you are going to find is much higher ratings.  So, out of 100 people, if 95 said ‘Yes’ and 5 said ‘No’, the ratings are artificially inflated.  Unless it’s a really, really bad video, people simply aren’t going to say “No”.  The ratings will be up in the 80s or 90s – however, because there is no granularity, unfortunately, the rating system simply won’t mean anything anymore.

Further Reading

Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *