There’s Testing and Then There’s Testing

Over the years, I’ve been fortunate enough to be in most every beta I opted in for.  I started it back when beta was actually a beta and not a sales pitch.  Most betas today are so polished that they are really just soft launches and not test platforms.

Then you have stress tests.  I work in the IT field.  Oddly enough, yesterday I was writing up an FRTM and test plan.  Stress tests are for two purposes.  First is authentication (logon screen) and the second is core systems.  Think of it as front door and inside the house.  Stress tests on the front door today should not be public.  There are hundreds of testing options for testing capacity on authentication.  The actual fallout of having your public test this feature is extremely poor press and word of mouth.   NDAs are all fine and dandy but people talk anyhow and a 21 gig character creation tool sells nothing.

What you want from a true stress test is putting a bunch of people inside the house and trying the various appliances.  You want 1000 people trying to PvP.  You want 1000 people to craft at the same time.  You want 1000 people looting at the exact same time.  Your transactional services need to be pushed to capacity and black box testing can only get you so far.

If you want to spend 3 days stress testing and the only thing you’re testing is the logon screen, then you just wasted 3 days.

Launch Windows

The average public beta of a new MMO is 60 days.  This gives some balancing numbers and time to tweak systems.  Most NDA’s drop on public beta to create buzz and word of mouth.  SWTOR is a recent example of a game with such a window, character wipes and a fairly strong NDA.  It was a rather bad beta, compared to previous ones.  RIFT is the complete opposite.  Beta was polished and the core task for testers was balance.  Balance takes time and takes recursive tests and non-logical functions.  That’s why you put people in the game, to do stupid things.

Considering we have a few big MMOs launching this year, I am somewhat curious as to the status of beta windows and NDA drops.  The goal isn’t to put something out the door.  That ship has sailed.   Fingers crossed.

Open Beta is Still Beta

A few more hours into Neverwinter and I’m still having fun.  The soft launch/beta issue is still quite evident, what with daily downtimes, sometimes more than once a day.  And there are bugs/exploits abound.  Bannings have even started for some people exploiting the Foundry.  I think bans are heavy handed, especially in a agreed-upon BETA status, though perhaps they are only banning people that haven’t spent money yet.  That exploits are being found though, that’s a good thing.  Means players are playing!

I have this thing with crafting in MMOs.  In Ultima Online I had 2 characters who only crafted, one of which was a GM tinker.  That might resonate with some.  EQ2, EvE and a couple others have complex systems but by and large, crafting in MMOs today is more or less a joke.  SWTOR embraced that thought and had crafting done by NPCs.  Neverwinter uses the same concept, crafting as a something you can do in combat, and puts in a web-service to manage it.  I am certain that I have more hours in the crafting queue than I have  played as a character in-game.  Right now, I have two 18 hour queues running.  And I still can’t make gear that’s high enough level for my character to wear.  /sigh.  I’m going to stick with it though, just to see what comes of it.  A good thing is the “rare” crafting items you can make, that appear on timers.  That’s cool.

Neverwinter Cleric

Chest piece at level 29

The game also comes with an LFG tool, one for skirmishes and one for dungeons.  The first one is simple enough, lasts about 10 minutes.  You’re likely to be in the queue longer than in the fight.  The dungeon queue is a tad bit longer but the dungeons themselves are quite long.  They are basically corridors of trash followed by a mini-boss.  Usually 3 minis then the big one.  They tend to all follow the same pattern.  At a given health percentage/time delay, they summon allies.  The problem here is that tanks are not able to pick them up quickly enough as threat management isn’t yet balanced.  On some end boss fights, I’ll spend the last 30% or more running around trying to not get hit by the spawns.  And that’s with every -threat skill I can get.  The health pools/damage abilities of the bosses aren’t yet balanced either.  One skirmish boss took about 5 minutes to kill.  Another dungeon boss could 2 shot some players.  Beta is Beta.

Another quirk for group play is the loot system.  Every magical item (green, blue, purple) is un-identified, meaning you have no idea if it’s an upgrade or not.  Every group I’ve been in, there’s been 1 guy(gal) who rolls a /need on every drop.  I can’t tell if it’s an upgrade or not and there’s plenty of restricted gear – i.e. a mage can’t wear rogue gear.  Why a mage can roll need on gear that they cannot use is crazy.  And boss drops, for me at least, have been static.  Meaning I’ve killed the guy 4-5 times and seen the same item drop every time.  Finally, there are chests in these runs.  Some can be only opened once, others multiple times.  It would be nice for some consistency.

Neverwinter Auction House

Gear can be expensive

I talked a bit about the F2P aspect, and the cash stop.  From what I’ve seen so far, there’s next to no need to spend actual money yet.  Sure, you need to spend real cash to get a 100%+ speed mount (default is 50%) or to upgrade your companion past level 15, but they aren’t really requirements as you can easily get by without it.  Even inventory management isn’t too bad.  The Astral Diamonds used for the Auction House are a bit different though.  If you complete 3 dailies (Foundry, Skirmish and Dungeon) you get 1000 diamonds a piece.  Run 3-4 skirmishes during the event, and you get 1000 per as well.  If everything lines up perfectly, you can finish the day with 5000-6000 diamonds.  I don’t see how you can reach the 24,000 daily cap yet but maybe that’s in the higher levels.  The best gear on the AH is about 200K per piece and by my math, you’re going to need to buy diamonds with cash or get lucky with drops.  The “cheap” stuff is 50K per piece and seems well enough.  We’ll see how that ends up in the end.  One thing I think will sell the most is Enchanted Keys, which open Nightmare chests.  You get these chest randomly (I have 20 or so) and the items are usable anytime once you have a key.  And the key only comes from the cash store.  Cryptic has been smart enough to send a system message whenever a player unlocks the “rare” mount from these, which appears to be every other minute.  A whole lot of “he won it, I should be able to as well”.  Still, for the average player, I don’t think there’s a huge motivation to spend money.  Maybe with more classes or races?

The next post on the game will discuss the Foundry system and player development.

In the meantime, here are my suggestions to the devs:

  • If crafting starts at level 10, let me make level 10 gear then.
  • Add a default option for loot rolls that you can only need equipment your class can use
  • Add some randomization to boss drops
  • Make all chests mutli-use or single-use.  If single-use, have a /roll option on the appropriate items

It Is What It Is

You know when you put your name in a hat for something and nearly a year later you get a call about it?  That feeling of “huh, I guess I was interested then“.  FireFall and Wildstar both piqued my interest about this time last year, when my SWTOR beta blues were hitting hard.  This week I got a pre-beta (what the heck is that if not an alpha?) for FireFall and decided to load it up.

Primer first.  FireFall is an FPS with PvE and PvP elements.  The RPG elements are skill unlocks and player customization.  The more skills you have, the better your loadouts.  You can also craft, but most of that is off-limits in the beta, as are the top-tier loadouts.  A few common classes, medic, engineer, tank, assault and sniper.  You can swap between them for little cost and the typical advantages/disadvantages apply.

Other than the actual beta-type issues (supremely interesting bugs), the core mechanics seem solid enough.  Shooting things to shoot more things makes sense.  The aiming portion feels solid.  The damage portion does not, but I think that’s a numbers game that beta will solve.  Skills make sense, cooldowns are long enough to provide meaning.  Movement is fluid, quests are indicated by checkpoint flags.  There’s plenty of random stuff happening on the map too, so there’s always something to do.

The hiccups for are as follows:

  • The game is based on group play.  Beta currently has few people.  This isn’t player driven content (like Planetside 2) but you still need players.
  • The variance between classes is significant.  Assaults/heavies deal much more damage than other types, way more than snipers.  Balance in any PvP game in terms of basic numbers is important.
  • Parallel to that, skill should play a larger factor.  I play most FPS games on the hardest level and get by well enough.  Standing and shooting has the same impact as moving and shooting, for the most part.  AI aiming/damage seems like it’s perpetually cheating.
  • I really don’t get how the game is monetized other than cosmetics.  Again, it’s beta, so a lot of stuff is going to change.  Still, I hope that cosmetics/convenience are the only things for sale.
  • Tutorials.  The game throws everything at you on the first mission.  All you unlock later on is more skills on the bar but 90% of the game is right there to start and they practically expect you to know it all.  I don’t mean combat, I mean crafting, loadouts, changing skills, using skills and all the town related items.  It took me 20 minutes to figure out how to load a 2nd skill.  My 2 year old was pressing random buttons and found the menu.
  • Self-healing.  There needs to be some type of self heal available at lower levels.  Being at 5% health with no healing options (or nearby players) is a drag.

The pluses!

  • Sound is great.  Sound is a stupid important part of an FPS.
  • The art style is cohesive and entertaining.  It reminds me a LOT of Borderlands, without the black outline.  Not only stylish but lowers the video card requirements
  • Integrated systems.  Everything seems linked in some fashion.  Spawns can trigger other spawns, some skills work best when combined with another one.
  • The skill progression is interesting, if somewhat confusing.  I like talent trees.  I like that your talent tree changes drastically when you change loadouts (classes) or armatures (sub-classes).
  • I like a challenge.  FireFall is a challenge.  The wall for skill for any player is high, very high.  I died about 20 times in the first hour.  I expect that to be slightly worse for the average player.
  • 3D matters.  If your enemy cannot shoot, then get to higher ground.  If they can, then use line of sight to your advantage.  This is smart.

There’s quite a lot of potential here.  Beta should be going on for another 6 months or more, but with no EXP resets, what you play/buy now will be there forever.  I think this will become the norm for F2P games, where the perpetual beta/slow rollout means that the server loads make sense.  No more day zeros of loading 100 servers then 2 months out, no one is left.

Sign up, give it a shot.

Edit: I wanted to point out that I found a bug on the main quest chain after you get your first upgrade that essentially stopped you from any progress whatsoever story-wise.  You can still level and explore, but the main quest is unavailable.  Hopefully it’s fixed by the time anyone tries it out.