I am anxious by nature, and one of the mechanisms I use to control that is planning. I used to overplan, to the n-th degree, but over time I’ve learned to let some things just slide. I think in reality, I’m just better at managing odds and the low percentage events get a whole lot less attention than they used to.
At work I oversee a team that supports a critical service for a whole bunch of clients. Outages mean freakouts and long hours, so we go to great lengths to manage the risk. IT, after all, is a commodity now. And you only notice a commodity when it isn’t there (like water, electricity). Planning of large changes takes a fair amount of lead time, and we need to do a lot of testing to make sure it works. Part of that testing includes load/stress/failure, where we throw everything we can at system and see what happens. We test at daily load, peak load, and critical load – meaning what do we normally see in a day, what is the highest number we see in a day, and what have we planned for before it melts.
For starting companies, launching a new product, this can be really hard to do. Maybe the architecture/platform is new. Maybe there wasn’t enough research to estimate the load. Maybe you get really popular before you can grow.
For larger, established companies, these items should not occur. The ol’ error 37 in Diablo 3. The inability for Sim City to work for nearly a month. New MMOs that melt for the first week. Typically, this is borne from a) poor testing and b) poor market analysis for load (you are popular).
How do you know if you’ll be popular? Today you can check pre-sales and the number of accounts registered. Social media trends. Analysis from gaming blogs. Plenty of data to give you a pulse. If you’re big enough, then you have extra hardware on stand-by anyhow, since you’re running a cloud-type data center. May not be able to turn them on in 5 minutes, but a day or two should be reasonable.
Which brings me to the VIP demo for Anthem, and the servers “melting”.
The reason this is confusing is that the VIP demo is only for players who have
- Active on Origin Premier
- Have received an invite and linked it to their account
That is a fraction of the launch day user base. It’s an even smaller fraction than those that will use the open beta. Plus… it’s not like EA has no experience running online games – Battlefield V is only a month or so old.
So maybe the server architecture is too complex to spin up. Maybe they had already planned to add capacity and the equipment came in late. Maybe their stress testing wasn’t accurate and this is the fall back plan (my $$ on this one).
Regardless, it’s good news that they are able to react this quickly. Glad the days of waiting weeks for server capacity are behind us. And really, the entire point of this demo is to test the infrastructure for load and bugs. Better now than on launch.