Comments on: The state of things http://www.paulcallaghan.net/blog/2009/11/11/the-state-of-things/ My home, on the web Tue, 01 Jun 2010 04:59:26 +0000 hourly 1 http://wordpress.org/?v=3.0 By: Blitz http://www.paulcallaghan.net/blog/2009/11/11/the-state-of-things/comment-page-1/#comment-1331 Blitz Sat, 14 Nov 2009 00:12:30 +0000 http://www.paulcallaghan.net/?p=341#comment-1331 <a href="#comment-1306" rel="nofollow">@Paul</a> Hey, Thanks for clearing that up. I don't think I have any better ratings data for you though :) I don't think any of the gaelic games ever got reviewed in any official online media, unless it was in gaelic? I only ever remember seeing one review online for hurling and i think that was just some guys blog :P Cheers @Paul
Hey,
Thanks for clearing that up. I don’t think I have any better ratings data for you though :) I don’t think any of the gaelic games ever got reviewed in any official online media, unless it was in gaelic? I only ever remember seeing one review online for hurling and i think that was just some guys blog :P
Cheers

]]> By: Paul http://www.paulcallaghan.net/blog/2009/11/11/the-state-of-things/comment-page-1/#comment-1306 Paul Thu, 12 Nov 2009 10:16:32 +0000 http://www.paulcallaghan.net/?p=341#comment-1306 <a href="#comment-1302" rel="nofollow">@Blitz</a> I'm looking at releasing the spreadsheet at some point. To gather the numbers, I pretty much just typed the name of the developer into the metacritic advanced search and copied the results straight into a spreadsheet. Some games, as you've pointed out, weren't represented - AFL being a big example - or didn't have a score. In those cases, I left them out or searched gamerankings.com for a representative sample in cases where there was more than one SKU (as there was for Lucinda Green). In cases where there was more than one version (both Heroes of the Pacific and Heroes over Europe are good examples) I treated the highest rated version as the 'original' and the other versions as 'ports' of that. All versions of Heroes of the Pacific rated 76% so it's just a function of a sorting algorithm that I took the Xbox version as the original. In the case of Heroes over Europe, the PC version rated 66% compared to 64% on PS3 and 62% on Xbox 360 so that's treated as the lead platform and the others as ports. If you have ratings data for the AFL titles and the Gaelic Football titles, I'm happy to add them in. @Blitz

I’m looking at releasing the spreadsheet at some point.

To gather the numbers, I pretty much just typed the name of the developer into the metacritic advanced search and copied the results straight into a spreadsheet. Some games, as you’ve pointed out, weren’t represented – AFL being a big example – or didn’t have a score. In those cases, I left them out or searched gamerankings.com for a representative sample in cases where there was more than one SKU (as there was for Lucinda Green).

In cases where there was more than one version (both Heroes of the Pacific and Heroes over Europe are good examples) I treated the highest rated version as the ‘original’ and the other versions as ‘ports’ of that. All versions of Heroes of the Pacific rated 76% so it’s just a function of a sorting algorithm that I took the Xbox version as the original. In the case of Heroes over Europe, the PC version rated 66% compared to 64% on PS3 and 62% on Xbox 360 so that’s treated as the lead platform and the others as ports.

If you have ratings data for the AFL titles and the Gaelic Football titles, I’m happy to add them in.

]]> By: Paul http://www.paulcallaghan.net/blog/2009/11/11/the-state-of-things/comment-page-1/#comment-1304 Paul Thu, 12 Nov 2009 10:03:59 +0000 http://www.paulcallaghan.net/?p=341#comment-1304 @Christian I agree that metacritic isn't the best source of data, but sadly it is all I have easy access to. Ideally, if metacritic had an api I could do some deeper data mining. It would be interesting to see overall trends across years, along with whether or not certain outlets rated some types of games - those that fall into the categories you point out - more harshly than they deserve. Removing outliers in my data (or at least knowing where they were) would be a useful addition. As a start - what licensed games released by Melbourne developers on PC / Console would you consider to have been rated harshly? If there are one or two I'd be interested in taking a deeper look at the reviews overall to see if removing the outliers significantly changed the overall trend. Like I said during my talk though - everyone in the room will have their own reason for the overall downward trend. I don't want to draw conclusions on why because there isn't enough data to support a conclusion other than 'based on metacritic aggregate scores, games developed in Melbourne have trended downwards since 2000.' @Christian
I agree that metacritic isn’t the best source of data, but sadly it is all I have easy access to. Ideally, if metacritic had an api I could do some deeper data mining. It would be interesting to see overall trends across years, along with whether or not certain outlets rated some types of games – those that fall into the categories you point out – more harshly than they deserve. Removing outliers in my data (or at least knowing where they were) would be a useful addition.

As a start – what licensed games released by Melbourne developers on PC / Console would you consider to have been rated harshly? If there are one or two I’d be interested in taking a deeper look at the reviews overall to see if removing the outliers significantly changed the overall trend.

Like I said during my talk though – everyone in the room will have their own reason for the overall downward trend. I don’t want to draw conclusions on why because there isn’t enough data to support a conclusion other than ‘based on metacritic aggregate scores, games developed in Melbourne have trended downwards since 2000.’

]]> By: Blitz http://www.paulcallaghan.net/blog/2009/11/11/the-state-of-things/comment-page-1/#comment-1302 Blitz Thu, 12 Nov 2009 06:47:20 +0000 http://www.paulcallaghan.net/?p=341#comment-1302 Hey, Thanks for putting these up. You mentioned that you had the raw data available too, and I was wondering how you collated that data, as there seem to be some anomolies coming from my knowledge at Ir gurus/TG. In titles by platform you have only 2 titles on PS3, and both of those are ports. Transmission released both Ashes Cricket 2009 (a license) and Heroes over Europe (original) in 2009, for 360, PS3, and PC. Where these left out somehow? Or are they considered "ports" of the PC version (can't be the 360 version since you don't have any originals in there either)? In the title breakdown by year, is that by SKU or by Title? The comments indicate it is by SKU. This doesn't quite add up for 2005 either as Heroes of the Pacific was released on Xbox, PS2, PC that year, and you only have 2 originals there (unless once again the PS2/Xbox are considered ports of the PC version?). Not to mention that the numbers themselves would be completely wrong if you add that AFL 2005 was also released over 3 SKUs and GG:F on PS2 in 2005 (7 SKU's that year from IRGurus, you only have 6 total), but i'll forgive that somewhat since AFL 2005 and GG:F don't even have an entry on metacritic. That said, AFL 2005 (3 SKUs 2005) Gaelic Games Football (PS2 2005), AFL 2007 (PS2 2007) GG:F 2 (PS2 2007) GG:Hurling (PS2 2007) are probably all missing from your stats since they're not listed on metacritic that i can see. How did you handle titles such as AFL 2006 and Equestrian Challenge which have a metacritic entry but not enough ratings to have an average score? Cheers Hey,
Thanks for putting these up. You mentioned that you had the raw data available too, and I was wondering how you collated that data, as there seem to be some anomolies coming from my knowledge at Ir gurus/TG.
In titles by platform you have only 2 titles on PS3, and both of those are ports. Transmission released both Ashes Cricket 2009 (a license) and Heroes over Europe (original) in 2009, for 360, PS3, and PC. Where these left out somehow? Or are they considered “ports” of the PC version (can’t be the 360 version since you don’t have any originals in there either)?
In the title breakdown by year, is that by SKU or by Title? The comments indicate it is by SKU. This doesn’t quite add up for 2005 either as Heroes of the Pacific was released on Xbox, PS2, PC that year, and you only have 2 originals there (unless once again the PS2/Xbox are considered ports of the PC version?). Not to mention that the numbers themselves would be completely wrong if you add that AFL 2005 was also released over 3 SKUs and GG:F on PS2 in 2005 (7 SKU’s that year from IRGurus, you only have 6 total), but i’ll forgive that somewhat since AFL 2005 and GG:F don’t even have an entry on metacritic.
That said, AFL 2005 (3 SKUs 2005) Gaelic Games Football (PS2 2005), AFL 2007 (PS2 2007) GG:F 2 (PS2 2007) GG:Hurling (PS2 2007) are probably all missing from your stats since they’re not listed on metacritic that i can see. How did you handle titles such as AFL 2006 and Equestrian Challenge which have a metacritic entry but not enough ratings to have an average score?
Cheers

]]> By: Christian http://www.paulcallaghan.net/blog/2009/11/11/the-state-of-things/comment-page-1/#comment-1287 Christian Wed, 11 Nov 2009 16:52:14 +0000 http://www.paulcallaghan.net/?p=341#comment-1287 Great talk, Paul, and very energising while being sobering. One very important proviso is that increasingly, the types of games that are made in Australia (ports / sequels / Wii / racing / sports) are reviewed lower and lower by a game review industry (as opposed to the fallacious 'game journalism') that often pre-reviews titles in these categories. While working at Gamespot and IGN dealing with these games, both editorial teams explicitly told me that I was free to go all the way down the scale with these games because they needed to legitimate the higher marks given to A games. This is the very definition of a false economy. There have been -amazing- game designs in these liscensed categories made in Melbourne + Australia as you well know - but I would absolutely not assume that game critics for Metacritic rated sites would have played them for more than a minute, if at all. So an adjunct point might be that Australia could use a 'tasteforming' blog like Offworld or Rock Paper Shotgun with a super strong dedicated team that pushes Australian issues and drives traffic (and therefore ratings) for games these big sites will ignore no matter how good they are. Great talk, Paul, and very energising while being sobering. One very important proviso is that increasingly, the types of games that are made in Australia (ports / sequels / Wii / racing / sports) are reviewed lower and lower by a game review industry (as opposed to the fallacious ‘game journalism’) that often pre-reviews titles in these categories. While working at Gamespot and IGN dealing with these games, both editorial teams explicitly told me that I was free to go all the way down the scale with these games because they needed to legitimate the higher marks given to A games. This is the very definition of a false economy. There have been -amazing- game designs in these liscensed categories made in Melbourne + Australia as you well know – but I would absolutely not assume that game critics for Metacritic rated sites would have played them for more than a minute, if at all.

So an adjunct point might be that Australia could use a ‘tasteforming’ blog like Offworld or Rock Paper Shotgun with a super strong dedicated team that pushes Australian issues and drives traffic (and therefore ratings) for games these big sites will ignore no matter how good they are.

]]> By: Tweets that mention PaulCallaghan.net » The state of things -- Topsy.com http://www.paulcallaghan.net/blog/2009/11/11/the-state-of-things/comment-page-1/#comment-1282 Tweets that mention PaulCallaghan.net » The state of things -- Topsy.com Wed, 11 Nov 2009 11:46:43 +0000 http://www.paulcallaghan.net/?p=341#comment-1282 [...] This post was mentioned on Twitter by Paul Callaghan and Paul Callaghan, Freeplay. Freeplay said: RT @paul_callaghan: New blog post: The state of things http://bit.ly/3WuUkw #igdamelb [...] [...] This post was mentioned on Twitter by Paul Callaghan and Paul Callaghan, Freeplay. Freeplay said: RT @paul_callaghan: New blog post: The state of things http://bit.ly/3WuUkw #igdamelb [...]

]]>