MMO Data Analysis

This post stems from Syl’s recent rant on games giving (or trying to give) players what they want.  I feel a lot of empathy for the position.  There’s a lot to be said of EQ and the stubbornness of the design but you really have to give Verant credit at the time, what with sticking to their “vision”.  On a side note, it appears that vision isn’t enough to get Pantheon going.

I’m a data analyst by training and a project integrator by career.  That means I take a bunch of numbers and stats from a given system, analyse and interpret them, then assist in the design of a new system and then finally help the end users with transition to the new stuff.  I live in a world of numbers, patterns and statistics.

Systems, just like games, are for the most part a black box design.  You know what goes in and what comes out but the inner workings are a mystery.  From the outside, you can’t see the system interdependencies.  All you see is that if you put in X, then Y comes out.  Theorycrafters (of which I did a bit of in WoW and a lot of in SWTOR) try a ton of mathematics to reverse engineer a blackbox.  This is well under 0.01% of the game population though.  Other people may use the spreadsheet generated by the theorycrafter but next to no one mucks with the code.

As a general rule, MMO players are horrible designers.  By and large, they are sheep who follow the sparkly path.  They like the black box and that when they press a button they get some fancy graphics.  If you give a player a choice between a high risk, high reward activity and a low risk, low reward in nearly all cases they take the easy road and just repeat is ad-nausea.  It isn’t that the design for the high risk is bad, it’s that human nature is risk-averse.  We don’t go around kissing spiders after all.

A MMO designer has a ton of metrics generated from the game from all sorts of venues.  Heat maps, activity counters, percentage of completion, distance traveled and many more.  Each individual piece says something but given that MMOs are such massively complex beasts, you’re missing the context of the data.  WoW’s Firelands tier of Cataclysm raiding had less than 1% completion rate on heroic, under 10% on normal.  That would be a red flag and you’d look at raiding numbers.  Is it because the raids are too hard?  Not enough people?  Gating is too complex?  Not enough gear?  The raid isn’t rewarding enough?  Other competing activities are more engaging?  Each one of those questions has numbers but it take a pretty bright person to find the links between them.

So as a player, what can you really do about this?  Posting on official forums is often times useless as the din of the dumb is much too loud.  A decent developer instead keeps track of fan site message boards, which usually attract people with a vested interest in the game in that you can better communicate with context.  You can start a blog on the game and start some further discussion (I still rank exceptionally high for Neverwinter).  You can “vote with your wallet” but moving your expenses from a game you don’t like to a game you do.  99% of the time when you quit a game today, there’s an exit interview where you can explain the position.  I know from personal experience that those questionnaires are highly valued as it’s much easier to get an old player back than a new player in.

There’s an old adage that goes if you try to please everyone, you end up pleasing no one.  Games that lose their focus on their core end up in this bucket.  It’s rare to find a game that is drastically flawed from the start.  If you’re playing the long game then you need to start small and get bigger – please a few and convert some later.  If you’re playing a short game, then you want a flash in the pan. Of course, there’s always the split between planning and reality…

4 thoughts on “MMO Data Analysis

  1. I don’t believe in armchair game design myself. So am actually all for devs that take (are allowed to take) decisive charge and keep a narrow design focus, make it a niche theme or something rather than monstrous buffet. At least then you can really get rid of all the finger pointing towards players, too.


    • Maybe WoW was too big for one person to make that call? When the game shifted, they were at 8m with only EQ2 as competition. I am honestly curious as to who designed WotLK and with what direction they had in mind.


  2. If you can communicate to me Pantheon’s vision in less than a thousand words, then you’re better at it than whoever wrote and ran their Kickstarter. PvP and seemingly core classes as stretch goals? A ton of focus on lore potential over expressing any real mechanics? Duel-wielding shields?

    They probably needed glasses if they wanted any focus in their vision.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s