netmonkey

Fuck Metacritic

Recommended Posts

Read: http://www.giantbomb.com/news/eidos-reviews-magic/560/

Seriously? And this is not the first time it happens. While it may not be Metacritic's fault, they're definitely driving some evil doing out in the world, and it must be stopped. There are other alternatives to reviews (Idle Thumbs, Giant Bomb, Rotten Tomatoes, Amazon user reviews, word of mouth, etc.), but the game industry bureaucracy seems to be obsessed with two things: Wal-Mart shelf space (in the United States) and Metacritic scores. If we as consumers stop caring about the two, then we can prove publishers that we don't take shit from them.

And as a gamemaker, I'm not letting my work be treated like a piece of meat, even if sucks.

Please, discuss.

Share this post


Link to post
Share on other sites

MetaCritic wouldn't be a problem if the majority of reviewers didn't use the 7.5-9.5 scale to begin with. Look at their film/music reviews: they're all across the board with the ratings and getting a 90, after a fair number of reviews, is quite a feat of unanimity. But with games? Fairly common and completely predictable. That's not MetaCritic's fault.

Share this post


Link to post
Share on other sites

Not sure why you're focussing on Metacritic, it sounds to me like the whole reviewing system is fucked. Unscrupulous deals being set up between game publishers and review publishers. THAT'S what sucks like shit.

Metacritic will show a "correct" aggregate score in a few days time, but they can't be blamed for people deliberately trying to undermine their stats.

*shrugs*

Share this post


Link to post
Share on other sites

Soooo... the best thing would be for everyone to start writing reviews without scores? It'd fuck up Metacritic, at least.

That'd also mean people would have to read the reviews. :tup:

Share this post


Link to post
Share on other sites

that, or give relative ratings.

For example: game X is not as good as Y, but better than well known game Z (note: the hypothetical Z, not the game 'Z' by the Bitmap Brothers, which was actually quite nice, better than KKND, but not as good as C&C1 or Warcraft 2).

Share this post


Link to post
Share on other sites

I think something akin to the Mitchelin stars system would be the best approach.

i.e.:

1 star = definately worth a look if you're passing by

2 star = worth making a detour for, very good

3 star = worth planning a whole trip for the sole sake of paying a visit.

(no stars = no comment, effectively. Not bad just... not incredibly noteworthy)

Frankly this is how I think about games anyway. Either something is an absolute must-buy and play; something I seriously consider buying if I have the money to hand; or something that I would very much like to play if the opportunity arises. And the rest, which I don't plan for but are occasionally good but often dissapointing.

It's not about the absolute quality of the game, it's about how much effort you should put into playing it ASAP.

Share this post


Link to post
Share on other sites

There's a world of difference between reviews and criticism. Both have a place.

Reviews aren't the be all and end all; for instance Haze got 55 percent on metacritic, yet apparently sold well.

Quite often things like this happen and developers get screwed out of royalties due to the publisher putting a metacritic clause in the contract (i.e. 80 pecent or above or you don't get anything beyond dev costs). This isn't metacritic's fault, Their aim is to be a useful consumer site, not a business tool for publishers.

What's needed are developers who are clued up and experienced in contract negotiation, and they've been in pretty short supply here in the UK.

Share this post


Link to post
Share on other sites

I don't think things like this are worth dropping scoring completely. Even with the Michelin stars thing, someone could translate that to a numerical scale (maybe not linear). Hell, you could even put :tup::tmeh::tdown: on a scale.

When most reviewers use scores, I can at least make quick judgements about some games if they are receiving exceptionally low (or high) scores everywhere. If a game is getting 50-60 all around, probably not worth my time. If someone gives it 90, though, I might look into that particular review to see if it might be something *I* would like nevermind that most people don't. If a game is getting 80-90 everywhere, I should probably read at least a few reviews (but still might decide not to play because it's the wrong genre for me or whatever). etc.

What I'm trying to say that scoring is neither useless nor essential, but as I don't have time to read reviews of every game ever made, I find it somewhat helpful.

Share this post


Link to post
Share on other sites
Reviews aren't the be all and end all; for instance Haze got 55 percent on metacritic, yet apparently sold well.

That might be because of actions like the Edios one. So that the sheeple pick up the game, and toss it away after 1 hour never to be played again.

Like the the Iron Man game...

I think metacritic should abandon numbers or at least the absolute scale. Simply make scores relative based on all other scores. More good new games will make the previously good games probably less good. Of course that doesn't change the fact that one man's "good" isn't the same as the "good" from somebody else.

Share this post


Link to post
Share on other sites

Just for the record, Haze didn't sell well, and it also almost killed Free Radical Design.

Share this post


Link to post
Share on other sites
This is such an important issue that I decided to write an editorial about it

four years ago

.

http://www.idlethumbs.net/display.php?id=75

Might be fun to read.

Haha! I must have read this article when it came out years ago, of course! I knew the thumbs would be on my side, which is why I started the post here.

My point is that we're still on the same page. I know the Thumbs folk are supporters of this system (seen as idle reviews had no numbers at all), but I think we need to take some serious action. Though I'm wondering, how? Many studios are owned by the big three/four, and indie developers have other means of getting their game. Can we stop publishers from this nonsense?

Perhaps it's all about campaigning against these metasites and educating gamers, who a lot of them do check metacritic/gamerankings, that they don't need to, and that there are other sites there where they can get what they want without being consumer whores. Game writers who blog and non-corporate-dependent sites like giant bomb, shacknews, kotaku, etc. Can show this support.

Honestly, I'm a sucker for rottentomatoes, but I don't see the tomatometer as a means to judge the film. For me it's become more of like "how do the critics feel?" and I see it more like a three thumbs system: :tup::tmeh::tdown: . I still drill down and read the reviews, because I want to know the opinions of the critics, and this day and age movies are not really reviewed less on technical achievement but more on story, characters, and such, anyway.

Share this post


Link to post
Share on other sites
Heh! I posted about this very idea 18 months ago :)

http://thunderpeel2001.blogspot.com/2007/06/gaming-review-scores-off.html

Heh, I knew I'd had the idea implanted from somewhere. I also thought of film reviews (most film reviews seem to use the four-star system now), but the Mitchelin star system seemed even more, I dunno, pure to me.

I don't think things like this are worth dropping scoring completely. Even with the Michelin stars thing, someone could translate that to a numerical scale (maybe not linear). Hell, you could even put :tup::tmeh::tdown: on a scale.

You could, of course. But the thing about Mitchelin stars is that even just one is an accolade. For many people it is a lifetime achievement (although it would probably be a bit more common in a games situation). Translating it to, say, a logarythmic scale, would be awkward and most likely lose useful information for the reader.

:greatscott:

heh, after reading the old thread linked in the comments to Wrestle's blog post (wow... convoluted), I saw my previous suggestion. That could still work, I think, only call them "achievements". This game has unlocked the 'Fun with Friends' achievement. And so on. Genius!

Share this post


Link to post
Share on other sites
Haha! I knew the thumbs would be on my side.

My point is that we're still on the same page. I know the Thumbs folk are supporters of this system (seen as idle reviews had no numbers at all), but I think we need to take some serious action. Though I'm wondering, how?

Perhaps it's all about campaigning against these metasites and educating gamers.

Honestly, I'm a sucker for rottentomatoes, but I don't see the tomatometer as a means to judge the film. For me it's become more of like "how do the critics feel?".

Sorry, don't want to make a big stink, and I could be wrong, but I think you're talking rubbish. You admit to regularly using RottenTomatoes, but qualify that by saying you're different. You understand that you shouldn't take the score as being absolute, so that's ok.

Other people, however, need to be educated.

My problem with this is simple: What makes you so sure that there are masses of people who need "saving"? How do you know that people are only taking Metacritic into account and not being sensible about it? It sounds like it could be you simply projected your fears, when you might not actually have anything to worry about.

If anything I think people should be educated to the attempted deliberate corruption of the scores of publishers, but trying to get them to realise that reviewer scores aren't the word of God... Well, who's to say they we all don't know that already?

Chris's editorial, for example, is more of warning about publishers and the industry becoming too focussed on aggregate scores, and the news article you linked to in your original post seems to have come from a direct result of that.

It would be a terrible thing if Metacritic became the Alexa for the gaming industry.

Share this post


Link to post
Share on other sites
MetaCritic wouldn't be a problem if the majority of reviewers didn't use the 7.5-9.5 scale to begin with. Look at their film/music reviews: they're all across the board with the ratings and getting a 90, after a fair number of reviews, is quite a feat of unanimity. But with games? Fairly common and completely predictable. That's not MetaCritic's fault.

Maybe you are just ignoring all the low-score games? They don't get as much attention, so it's easy to think most games are getting very high scores. For a random example, I searched for all Action games for XBox360 (released before october), sorted by score.

http://apps.metacritic.com/search/process?ty=3&ts=&tfs=game_all&sb=4&game_platform=xbox360&game_genres=action&release_date_s=&release_date_e=10%2F01%2F2008&x=29&y=9&metascore_s=&metascore_e=

There's 210 games total. There don't seem to be many exteremely low ratings, but

The first 50 games are rated below 55

the first 100 below 70

the first 150 below 80

the first 200 below 90

less than ten games are over 90

So about half of the games are rated below 70. I don't think that's so horrible or somehow more wrong than movie ratings.

Compare this to PC (same genre), which might have more crappy games:

There are 708 games, first 125 don't even have a rating (probably really crap because most reviewers don't bothers to review them)

first 200 are below 50

first 300 below 60

first 400 below 70

first 500 below 75

first 600 below 85

about thirty games are above 90

As I suspected, the PC has even bigger percentage of games (4/7) below 70 (including those with not enough reviews to be rated). I don't see anything totally off here.

IMHO, the problem is only with a few games that publishers put lots of marketing behind (and try to influence scores), but that are actually crap. Those are exceptional and don't mean the whole system sucks.

A game that recently seemed to receive too much attention for its crappiness: Golden Axe. Metacritic score on both PS3 and XBox: 45/100 (yeah, maybe should be even lower, but that's nitpicking).

Share this post


Link to post
Share on other sites

I agree with Thunderpeel.

I've come across a bunch of creative people in different sectors who talk about "audience education", and it's usually means "I'm fucking up but can't tell I'm doing it". Petulantly, it becomes "Noone understands me!".

Reviews tend to be more of a checklist, but moving beyond that goes into the territory of criticism, something video games have historically lacked. Zero Punctuation is a start, likewise RPS, but most of the games press is still dominated by scores.

They don't necessarily correlate to sales. Transformers was an execrable game, yet sold bucketloads. The problem is publishers using reviews to fuck developers (metacritic clauses are happening a lot now), which is kind of their business. Developers need to get better at business. It's not like there's going to be a shortage of work for hire now games are starting to outsell music a DVDs.

There is no review system that will work properly or be without some exploitative hole. If reviews didn't have metrics, publishers might well make them, or if not they'll just find another way to screw developers.

Just for the record, Haze didn't sell well, and it also almost killed Free Radical Design.

Ah, ok. I might have been fed that by an overly optimistic but nonetheless pissed off FRD developer. Not much of this kind of thing is on the record as far as anyone outside FRD and Ubi are concerned ;)

Share this post


Link to post
Share on other sites
The problem is publishers using reviews to fuck developers (metacritic clauses are happening a lot now), which is kind of their business.

This is what's really broken. I'm hoping that my venturing into rottentomatoes is irrelevant to film contracts or to manipulating most of the critics in the country to stop them from posting reviews, unless there's evidence out there that I'm not aware of. Sure, it doesn't matter whether the film sucks or not, most of the time it makes a ton of of money.

I don't know why publishers are getting the idea that some sort of average review score correlates to their sales. Perhaps it's because the system is totally broken in the first place, as it has been discussed, and therefore average scoring for video games is broken.

What makes you so sure that there are masses of people who need "saving"? How do you know that people are only taking Metacritic into account and not being sensible about it? It sounds like it could be you simply projected your fears, when you might not actually have anything to worry about.

They don't really need to be saved. I just hope they're aware that these points are bogus, but it's uncertain. I'm not sure where teens are getting their scoop on games these days, whether it's the same place as the ones we do, or if it's still gamespot/ign/1up/etc.

Regardless, my biggest concern is that because this is system has been broken for so long that publishers are now seriously exploiting it as Nachimir mentions. And it can only get worse; publishers only seem to be focused exploiting something until it crashes hard into the ground, and they continue to be the driving force of the console games industry. They might as well dig themselves into a hole, bringing us all along with them.

Share this post


Link to post
Share on other sites

With regards to MetaCritic's numerical point system, one point I'm surprised nobody's made is that MetaCritic does in fact have a secondary scoring system that is essentially a three-star rating: red, yellow, and green.

I personally use this rather than the specific number. If it's red I'll stay well away, if it's yellow I'll probably only look into it if I'm particularly interested in it for whatever reason, and if it's green I'll most probably read a few of the aggregated reviews and also read most of the little excerpts (which are in themselves an excellent little provision IMO).

Of course many people will go by the specific scores, but I kind of suspect a lot of people don't. The traffic light system is a subtle but very good feature. :tup:

For what it is, MetaCritic does a pretty good job and allows people to use the service in the way they find most useful. Certainly far superior to GameRankings I reckon.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now