Sign in to follow this  
Jake

Idle Thumbs 101: Introduction to Video Games

Recommended Posts

You make some good points, and I don't discount Metacritic altogether, but I really take issue with the notion that marketing doesn't have more to do with a game's Metacritic score than the game itself. Maybe in the long run, yes, but in the first month or so?

 

Also, don't forget, as Wikipedia Brown pointed out, that Metacritic weighs scores from different sites differently, supposedly to provide a "true" aggregate rather than a base average. This is much less transparent and straightforward than Michelin ratings, which is why I brought up student evals.

Share this post


Link to post
Share on other sites

The Obsidian thing was really just the result of a bad business decision. Obsidian agreed to a contract where if they scored 85+ they'd get all the money, and if they got under 85 they'd get none of the money. That kind of outcome is totally avoidable by having the bonus scale to score. And that kind of outcome will also occur whenever you have a firm line. Say you tie the bonus to the game selling three million copies. The game sells 299,999 copies instead. One missed sale has huge consequences in that case. 

 

The problem is kind of what I was (badly) driving at when I got all ranty, right? The people that made a bad decision were Obsidian by negotiating an all or nothing contract tied to an aggregation website, but as referenced in Tom Chick's post Adam Sessler is upset and blames Metacritic for literally taking food off of developers' tables. So people are mad at Metacritic coming AND going, and mostly all for bad or wrong reasons. Consumer information is not at fault. No one cost themselves any money but Obsidian. I hate harping back to Game Rankings, but if they were aggregator du jour they would have also denied a bonus because they're even lower than an 84.

Share this post


Link to post
Share on other sites

The Obsidian thing was really just the result of a bad business decision. Obsidian agreed to a contract where if they scored 85+ they'd get all the money, and if they got under 85 they'd get none of the money. That kind of outcome is totally avoidable by having the bonus scale to score. And that kind of outcome will also occur whenever you have a firm line. Say you tie the bonus to the game selling three million copies. The game sells 299,999 copies instead. One missed sale has huge consequences in that case. 

 

(That'd be actually more like 2.7 million missed sales...but I'm pretty sure I get what you meant.)

Share this post


Link to post
Share on other sites

I love Tom Chick as a reviewer, but I don't think he's very consistent in his attiude about Metacritc.

He says he's personally interested in seeing aggregate measures of a game's critical reception, which is fine.  But surely he's aware that when he gives a game 3 stars, he means something very different from what most critics mean when they give a game 60%.  So his actual opinion isn't really being reflected in his contribution to the Metacritic score.  The 3 stars usually makes perfect sense in the context of his written review, but he's either ignoring how Metacritic interprets it (as I think he should) or he's submitting his score as a kind of protest vote in hopes other critics will use the whole scale as he does. Personally, I think if reviewers give scores they should only be using them as a way to communicate with their readers, and not worrying about what Metacritc would do with it, and that feels like what Tom is actually doing, regardless of what he says about the value of Metacritic.

The most interesting part to that Tom Chick post Gormongous linked to is in the comments where someone points out that Rotten Tomatoes used to have a games section, but it turned out to be useless.  The "Fresh" score was 60% or higher, and four out of five games had a tomatometer rating of 100% fresh.  How's that for proof that the review score system is broken?

Share this post


Link to post
Share on other sites

One thing that actually is really weird about Metacritic is that it normalizes scores for music and movies, but not for games. Normalization is basically "grading on a curve", and could be used so that game scores from different publications are distributed across the whole scale. It prevents scores from clustering around one area of a distribution. It's very weird to me that they would normalize some media-types, but not others, and I really wonder why. 

Share this post


Link to post
Share on other sites

The review system is definitely broken, like everyone is treating it like a logarithmic scale or something. It seems like anything lower than a 7 is really just a measure of how terrible a game is. So 7 is okay, 6 is bad, 5 is really bad, 4 is so bad its unplayable, 3 is dog shit, 2 is AIDS, and 1 is an inverted penis.

 

That leaves 8, 9, and 10 as the only scores that actually measure how good a game is. On a scale that small, a game that gets a 10 could only be marginally better than an 8 or 9. So we use decimals at that level to make that scale feel bigger, giving games 9.25 or 9.75 because there aren't enough fucking numbers to truly capture all those little nuances that contribute to a game's quality.

 

My question is, if a game gets a 2.5, what the fuck does that mean? Is it like not quite as bad as AIDS but worse than dog shit?

Share this post


Link to post
Share on other sites

I love Tom Chick as a reviewer, but I don't think he's very consistent in his attiude about Metacritc.

He says he's personally interested in seeing aggregate measures of a game's critical reception, which is fine.  But surely he's aware that when he gives a game 3 stars, he means something very different from what most critics mean when they give a game 60%.  So his actual opinion isn't really being reflected in his contribution to the Metacritic score.  The 3 stars usually makes perfect sense in the context of his written review, but he's either ignoring how Metacritic interprets it (as I think he should) or he's submitting his score as a kind of protest vote in hopes other critics will use the whole scale as he does. Personally, I think if reviewers give scores they should only be using them as a way to communicate with their readers, and not worrying about what Metacritc would do with it, and that feels like what Tom is actually doing, regardless of what he says about the value of Metacritic.

The most interesting part to that Tom Chick post Gormongous linked to is in the comments where someone points out that Rotten Tomatoes used to have a games section, but it turned out to be useless.  The "Fresh" score was 60% or higher, and four out of five games had a tomatometer rating of 100% fresh.  How's that for proof that the review score system is broken?

 

Well, you have to understand that Tom has two dogs in the review score race. One is that he feels Metacritic is a potentially valuable and useful tool. And the other is that he feels that reviews can and should use a full scale and not the narrow 7-9 range that's actually being used by most reviewers. And if they did, his scores might or might not be outliers (certainly he's fond of some games that aren't widely acclaimed and not particularly impressed with others that are in most places critical darlings), but the Metacritic version of his score would convey the intended meaning.

Share this post


Link to post
Share on other sites

who here has ever bought a game based on a metacritic score?

 

i never have, a number means nothing to me and i generally disagree with it anyway 

Share this post


Link to post
Share on other sites

I kind of feel one of the root problems with game criticism, or rather game reviews, is that everyone involved seems to have the idea that the purpose of a review is to basically be a product guide, and these days it's very rare indeed to find a game that's downright broken. Uninspired, derivative, sure, but some people really want more of a particular kind of game, and the game is fun enough (again, because everyone's pretty good at this these days), so you can't really say that it's failing, exactly. I think this came about because players really really really want a product guide because they are paying just an obscene amount of money for a product, so the attitude that this is Art and it should be judged on subjective standards in the context of the medium is useless to people whose chief concern is that $50-60 is really a lot of money to be risking on something that is not guaranteed Fun on a Bun. You can see the traditional product guide review approach having serious problems with downloadable games, where everything is at multiple price points and quantifying a game's properties is almost certainly a recipe for disaster; I also note most mobile review sites use a four-point scale that's a lot more honest.

Share this post


Link to post
Share on other sites

who here has ever bought a game based on a metacritic score?

 

i never have, a number means nothing to me and i generally disagree with it anyway 

Back when I had more time than games I used to sort the Steam store by metacritic to see whether there was something good I might have missed/be missing. It's definitely a metric that I considered.

Share this post


Link to post
Share on other sites

ho here has ever bought a game based on a metacritic score?

 

i never have, a number means nothing to me and i generally disagree with it anyway 

 

I've used it as a quick check to make sure something isn't totally broken. Like, if I haven't been paying attention to a game's release, I may look to make sure the score isn't in the red, because if it is, that means it's not a functioning software product.

 

Outside of extremes like that, I don't find the number useful, though.

 

Reviews don't really factor into my purchasing decisions either, for that matter. 

Share this post


Link to post
Share on other sites

I'll check Metacritic for games where I've only had one source of information. When I listened to the 3MA podcast about March of the Eagles, I was intrigued and thought my dad might like it. I went to Metacritic and it has a 71, with the spread from an 80 to a 50. Most of the reviews echo some of the concerns that were voiced, namely that the single player was lackluster, and the multiplayer was phenomenal but required a dedicated group of people, likely a certain base knowledge from other paradox games, and the connection was iffy at the time of release.

 

So I guess I use Metacritic as it was created? An aggregation service. I don' t know if you'd consider that buying based on the score or not.

 

It's also interesting to see the peaks and valleys of Metacritic. The very, very highest highs, and lows.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this