Reviewing the meta-reviewers
Are meta-review sites trustworthy? Well, it depends on the site and their weighting policy, according to this great article at Game Revolution. They explain the grading schemes behind Rotten Tomatoes, Metacritic, and GameRankings and elaborate on why each is accurate or not.
Rotten Tomatoes’ scores don’t go over well for a couple of reasons: First, they only use reader-submitted reviews, which may number as few as 5 or 6. Furthermore, there are only 2 scores: Fresh or Rotten. To get a score of Fresh, a game has to score 80% or better – meaning that a game that gets a 79 is Rotten. Wow, a 79 is a C+ in grade school, certainly not a failure.
Metacritic gets a slightly better review, though there are still problems. For one, they use one set of grading criteria for games, and a different set for everything else. Yet, their conversion-to-letter-grade scale is a third set, not matching either of the others. They also don’t show the original scores from other sites. While these problems are “loco,” according to the article, it all “sort of works” and the Metacritic scores come close to the general consensus about a game.
Finally, the article covers GameRankings, who are at least transparent, showing the original scores from the other sites they take their averages from. However, they do post ratings for Japanese games and forget to mention the games are not available anywhere else. And sometimes, their aggregate scores are really only from one site, when only one site reviews the game. GameRankings is rated as more accurate than Metacritic or Rotten Tomatoes, but still not perfect.
3 Comments »
Leave a comment!
“They also don’t show the original scores from other sites. ”
Actually MetaCritic *does* post all the scores from the individual reviews
Wow, that was an awful article, rife with obnoxious writing, unfunny illustrations, and trivial complaints. When only one site has reviewed a game, what should Game Rankings do besides assign it an aggregate score equal to the single review score? The article doesn’t suggest an alternative, probably because there isn’t a better solution. Game Rankings could wait for a game to receive a certain number of scores before assigning an aggregate score, but what should that threshold be? 5? 20? I’d rather know immediately what the average score is, even if it’s based on only one or two reviews. And Game Rankings acknowledges the issue with this compromise: “A title needs at least 20 reviews in the database before it is placed into the overall rankings.”
This Game Revolution site requires much debugging before it emerges from beta.
Just a point from skimming this, the point about 79% being a good score is invalid. First off 79% is a B+ not a C+, but that is besides the point. The point is that rotten tomatoes came up with that 80% marker for good reason. For movies it’s 60%, not because C’s are good, but because it adequately balances what can be considered a good or bad movie. Video games garner more positive reviews across the board than the more subjective film industry, so a higher score threshold is needed to adequately seperate the good from the bad in gaming.