FF14 – Server Loads and 2021

FF14 has a weird server architecture, at least by my understanding of how this should work. There are 3 data centers which impact overall latency, and each holds 8 worlds. To say that there’s been an uptick of players is an understatement… and the servers are struggling. Queues are present daily (you see your position, but not time estimate). S/E tried to fix this with a near 12 hour patch on Monday, with no real success (nor failure!)

As of this draft, of the 24 worlds, 12 of them will not allow new characters to be rolled – and one data center is completely locked. Side note, I transferred to this data center with less than 24 hours before they locked it…blame me I guess.

My field of work intersects with DC architecture. I know there are a lot of efforts to move out of the mid-00’s data center design, into the PaaS-type of design. This is not easy, as there are often specific lines of code in software that relies on specific OS/HW configurations. The ol’ hook/pull coding nightmare that all the cool kids thought was going to save the planet with their custom code. Upgrading is hard, you can’t break anything, and odds are you can’t fully test the load until you turn on the switch.

Yeah well, we’re still picking up the pieces of those decisions… and we’re going to be in a long tail of migration for a good chunk. Most large enterprises still have legacy code somewhere that they just can’t port over for A/B/C reasons after all. And in the gaming space, this particular code portability is the primary driver for sequels. No need to port if it’s brand new code!

Side note – a few years ago WoW updated their database architecture, which had a pretty darn big impact on skill calculations. They were using a DB structure that dated from Vanilla (well, more like 4 years prior to launch), and it was quite painful to move forward. They did it though! And the overall system load (if you read those reports) was significantly reduced. That requires some serious planning and smart people to deliver.

FF14, from the information that’s published, is talking about OS configuration issues and replacing some equipment with HPC. I do understand why they would not be moving to commercial cloud… that is a really bad idea for an MMO. Commercial (or even semi-private) cloud is absolutely not the silver bullet that some folks may believe – just like a buffet is not the best type of restaurant for all your food needs. Given the on-going supply chain issues, this is going to be a long-term effort to expand given capacity, let alone upgrade the components.

Hats off to S/E for the forthright nature in this issue. Fingers crossed that this is actually something they can actually address before November. Cause if queues exist before launch… well delaying launch is going to be right on the table.

FF14 – Main Story Quest

It’s an MMO, there are quests everywhere. FF14 still takes a rather unique path here as it’s a mandatory process to go through the story quest (MSQ) in order to unlock a bunch of features. This is somewhat different than other MMO’s, where the quests themselves are more of a step in order to reach max level, and that once you get that final ding, you can ignore all the levelling content.

I remember back when I did the ARR MSQ and was confused when I found myself forced to group at an early level. I had no real issues with the step, it’s an MMO after all, but it was still jarring compared to others. And that the game had an LFG tool built into the entire process was really something. It still remains one of the few MMOs that goes to great lengths to harmonize the leveling experience and the end-game experience… what you see at level 20 is pretty darn close to what you see at the end.

FF14 has “groups” of MSQ, those that are related to the leveling portion, and then the post-expansion patches. The former group is usually enough to get you to the max level for a given expansion (ARR had some gaps, not sure if still present). The latter group is meant to be a mix of storyline and dungeon reveal. Again, you need to go through this portion to unlock said dungeons in the LFG (roulette) tool, so you’re certainly motivated from a mechanical perspective.

The link above refers to the arches of the quests. Each can have 3-5 substeps within, and FF14 loves to have steps cross the map. Leveling, this isn’t too bad as it points you around. Post-leveling, it can be somewhat ridiculous to teleport everywhere for a 30s cutscene, only to return to where you were. The 3.1 content has 1 minor battle with 2 small enemies, then an instanced battle with support NPCs that takes about 20 minutes to complete. The whole 8 quest arc takes about 2 hours to get through – that’s a lot of exposition.

The length of the MSQ can be seen as a downside, especially for attracting new players who want to be with their friends. From ARR -> end of Shadowbringers, there’s close to 200 hours of content to get through. You can buy a boost to get to the starting line of Shadowbringers, which still leaves you with a good 40hrs or so. Yes, this puts a burden on the players to get to max content, but on the flipside, it also makes sure that they understand all the expansion mechanics and group functions. It eases the transition to end game, rather than creating a quit wall.

The sort of benefit here is that the MSQ only needs to be done once per character, meaning you can level any other job/class while just worrying about experience. People say WoW is alt-unfriendly, well FF14 is both the friendliest and most punishing. The only reason I can think of having an alt is if you want a different looking character. There are niche reasons (RP, more retainers, raid lockouts, FC farming) but those would really wait until you have many jobs at max level on a main. And sanity wise, if you want a different looking character, pay the $10 for the race change.

So now I’m 2 hrs into the max-level journey at 60, with 8/44 quests complete before Stormblood. There are 4 dungeons and 2 raids in there too, so the estimate to get through this is about 25 hours. Let the journey continue.

FF14 – Getting It Back

Last week I uninstalled WoW and D3. I tend to keep my desktop icons relevant to the active games that I play, or at least indicators of those that I should be playing. WoW has been on the desktop since beta. Looking back on my posts for WoW, aside from really liking the world/art/music, there hasn’t been a positive post on the state of the game since Legion. Cripes, that was nearly 5 years ago! The whole ABK was just that larger realization.

I figured I’d give FF14 a shot again. The irony here was that I had not played since just before Legion, so just before the mid-pack of Heavensward. Playing 2 MMOs at the same time, in the way that I play them, required just way too much time. Sunk cost in WoW and all… FF14 went on the shelf.

Getting back into FF14 meant a choice – start anew or where I left off. Having completed 2 entire expansions (and ARR is quite long), I figured I could better use that time with an account restore. To do so, I needed to remember my old password. Let’s not shy from the fact that Square-Enix has invested a total of $10.25 in web UX design. Finding anything that makes sense here is an exercise in frustration.

I did finally find the password recovery page, which has security questions. 2 days. That’s how long it took for me to enter the correct answer to the security question. The button for “forgot security answer” doesn’t go anywhere. The answers appear to be case sensitive. And it would appear that I was forced to reset my answers in the past, as this is the only place, across all similar questions, that I put in that specific answer (which came to me while doing something else).

With the account restored, I needed to figure out the rest of the bits and bobs on the account. Back to bad UX design, actually getting another expansion is a 3 step process. First, I need to figure out which one to buy… Endwalker says it includes it all, but you can’t actually access it until the game launches later in the year. Fine, base version of Shadowbringers it is (making sure it is NOT the Steam version, which I don’t think you can buy anywhere but Steam). Buy that, then realize that it isn’t linked to my account. I need to find the game key, then add it to my account. Ok, done. Now to add game time, which is another weird process (since there are 2 levels of accounts, one which supports alts, which are an insane option in this game).

FINALLY, all the bits in place to actually log in!

Did I mention that this meant multiple downloads? I get a quick start download to get the client running, but it’s multiple large Gb downloads after that. I can’t think of any other game or platform that does this. Sure, maybe you’ll get a day 1 patch after you’re pre-loaded the game. But this is not pre-load, and we’re at the tail end of the current expansion. ARGH.

This time, I’m able to log in. Brynhildr is where I set roots long ago. I log in to a barrage of visual spam. Where WoW is a ghost town in almost every area that isn’t the current expansion’s capital, FF14 has people a plenty. The neat part here was that it automatically prompted me to join a “returning players” channel. That was a nice surprise, and woooo, were there things in that chat which were helpful! Still, as much as I liked that character (and time invested) I wasn’t interested in staying there. I logged out and went through the relatively simple (again, horrible UX) process of transferring to another server – Cactuar.

That took about 15 minutes to process, and I was finally off to the races (or perhaps, re-learning the races). If I exclude the time it took for me to restore my account, getting the rest all set up was about 2 hours of work. Perhaps this is a relic of the Playstation UI to help players get in the game. Still, with a few month’s work and a relatively small investment, the UI for the store/account system could be dramatically improved. This feels like the same comment I made way back when. One can hope!

Next steps are all about following the MSQ through the 2 expansions. I don’t think I can get all of that done before Endwalker comes out. Where I could plow through WoW’s content in less than a week to get to end game, it’s more like 40+hrs per expansion here. Let’s see how much fun this ride can deliver.

12 Minutes

Way back in the olden days (2015), a conceptual game called 12 Minutes was pitched. The concept was that you’d be stuck in the same 12 minute time loop and need to find a way to break out. The game released on Thursday, and I dove in headfirst.

Time loop as a story mechanic is really hard to get right. You often need some sort of book to keep track of what’s going on. Only a few movies have tried it. Primer has like 40+ timelines on the go at one point. ARQ is a simplified time loop film, and really focuses on the anticipation of the next steps, similar to Edge of Tomorrow (which is almost a video game). Groundhog Day, well… that’s a form of hell if you think about it.

In terms of games with this model… is Mario Bros a time loop game? I mean, you die, and the loop restarts. Most incremental games have actions in one loop impacting the next. It’s so common we don’t think about it anymore. But in the strictest sense, where the purpose of the game is to break the loop, really Outer Wilds is the only one that comes to mind that did this in any way successfully. Like GotY successful.

That’s a heck of a bar to reach for any indie game! I guess that explains all the years of dev work. And the results…

A game that merges interactive fiction and point and click adventure mechanics, with incremental changes on each loop. There’s no “speed run” this game, you need to go through at least a dozen loops before you have enough bits and bobs to get close to the finish line. More than once I thought I was close, then quickly realized that the rabbit hole went even deeper.

The voice acting is solid (hard not to, how did they get these actors?), and there are some QoL aspects that let you speed through dialogue if you’ve heard it already. The graphics are passable, but it’s not like that truly matters in this type of game. Every puzzle has a logical answer, with plenty of breadcrumbs unveiled as the game goes. Even the music here is darn near perfectly placed to convey the emotional tone.

Side track on puzzlers for a minute. Some of them suffer due to size, or ridiculous solutions (Sierra!). 12 Minutes takes place in an apartment, one you can’t leave as it resets the loop. This smaller footprint dramatically reduces frustration as you know the answer is right in front of you, it can’t be anywhere else. And with only a dozen or so items to collect, it’s more about the order of using the items than the items themselves. Like if you do an action in front of an NPC, they will see you, but if you close the door, then you’re good to go. And if you’ve seen text before, it highlights in a different color, so you know what’s new or not. And the dialogue is the real puzzle here, every puzzle is about unlocking a new part of dialogue.

I will say that the only downside here is when you’re focusing on actions near the end of a loop and need to restart. Getting back to that point can feel a bit long, especially when you’re talking about 1 action that takes 2 seconds to perform. It does mean that you have to be considerate of time management and sort of plan out what you’re trying to do before you do it. There was a point where I was stuck in what felt the same loop for an hour, just not quite sure of where to go next. But that’s no different than being stuck in the belly of a whale, trying to tickle it’s “uvula”. 1988 and I am STILL ticked off at that.

I can’t talk about the plot at all. Really, any indication here would spoil the ride. I can say it’s a good ride, bordering on great.

I think the best compliment I can give 12 Minutes is that it’s one of the best examples of interactive fiction to come along in a long time, supported through a time loop mechanic that really delivers. This is the sort of game that can only come from an indie developer, and one I’m sure others will try to copy.

Free Guy

The tldr; is that this movie could not have possible worked without Fornite. Full stop.

If you enjoy Ryan Reynolds being himself, well then, you’re going to enjoy this movie. If you want a social critique on MMO behaviors, then yeah, this is the movie for you. If you’re looking for a film full of easter eggs and call-outs to gaming, then this is a thousand times more appropriate than Ready Player One.

I have no idea who was smoking what when this pitch came through. An NPC in an MMO gains sentience, and then the McGuffin quest to find proof of the AI. Sounds like a few people on a couch, enjoying the air, and then this comes out. Without Fortnite taking over the west, there’s no way for movie goers to even understand the fundamentals here. The whole over 49 demographic is going to come away from this with barely any understanding of what’s going on.

It is worth pointing out that Taika Waititi goes for the fences in this one. It’s so borderline absurd, it’s realistic to the immature genius of game development studios. That he doesn’t use an accent here, really let’s him lay into the kiwi vibe something fierce. There’s not a scene he doesn’t completely own.

This is also a weird movie with the romantic aspect is a key turning point. Fine, the epilogue is laying it on thick (more on this) in terms of the main characters, but it’s also an interesting twist plot-wise, that acts as a deux ex machina. Gamers understanding MMO technical limitations will find this particular scene a little tough to swallow, but in the larger scheme, it works.

The whole movie is about breaking a 4th wall (or I guess a 5th) in blurring the line between the art and the consumer. The epilogue just puts it all out there… log off and talk to people. Feels like a giant critique of streamer/gamer culture, which doesn’t feel like it’s heavy handed. It does feel like this is a more cartoony version of a Black Mirror episode, and packaged in such a way that people can take something away from it, aside from depression.

Call me pleasantly surprised with the balancing act this film achieves. Plus, you know, Ryan Reynolds.

MMOs – Time is a Flat Circle

An interesting bit from Stephan Frost, in relation to MMO development. If you recall, he was one of the leads on Wildstar, a spectacular example of MMO failure.

In the conceptual layer, the items described apply to any long-dev cycle of an evolving product, not just games. You’re competing against a finish line that is always moving, mainly because you’re competing against established market forces. Big bang successful launches are ultra rare in existing markets, they instead start small and grow over time. MMO’s are rarely given that luxury, as they are competing against expectations from long-established products. New World has 50 other MMOs to compare against… it’s got to be focused and refined to have a chance. You need a super clean vision and new flawless execution to have a remote change.

Wildstar, to kick that dead horse, had an incoherent vision and launched about 6 months too early. I had a blast in the leveling portion and really liked the crafting system. Hell, it had social housing and tons of cosmetics. It was a solid setting, with a decently solid 1-to-max experience. And then it shifted to the same place WoW has turned, the ultra competitive + fine tuned playerbase. It had no social tools for nearly a year, and all group content was either against a clock or required twitch reflexes to get through. Raids being only 20/40 was an insane choice that only works worse the farther we move away from it.

Could Wildstar have ever succeeded, even with more time in the cooker? I like to think so, as the bones were solid. Had social tools been there from the start, had the “timer” dungeons been an extra tier instead of the wall of quit (WoW ran with this as Mythic+), and had raids been much smaller, perhaps we’d be having a different conversation. People can understand balance issues at launch, but systematic issues are tougher pill to swallow.

Is this due to dev cycles, or poor vision? Maybe they just copied the wrong mechanics and should have pulled from FF14’s bag of tricks. I don’t think we’ll ever truly find out, at least not until there’s a deep dive on the topic with such a purpose.

Which is a right shame, as we’re seeing a larger push for games as a service, MMOs in nearly all respects, and those lessons just don’t seem to be learned. Or perhaps they are simply not being listened to while the bean counters run the show.

Credit vs Subsidies

Canada is heading to elections. It’s opportunistic in order to try and acquire a majority government. We have 2 main parties (to various degrees of left/right) and then some supporting smaller parties (both hard left). There’s a cadence here of a few steps left, a few steps right. Keeps some semblance of balance and generally the country keeps path. We went through the global recession with tight fiscal control, and then the recent social uprisings with liberal social values. I find myself aligning more to centrist values, so depending on the election cycle, the edges of either of the parties platforms.

One particular item announced in the competing platforms is child care. This is an odd topic for those who don’t have kids, or never had. Sort of like schools when you’re an adult, or hospitals when you’re young. Anyhoot, the standing government (left) has set up agreements nationally to introduce a $10/day day care program. There’s a lot of money behind this, and the gist is that the provinces will be able to pay out for day care services (and meet their criteria).

The other party (right) wants to scrap that program and institute a credit program that covers 75% of fees to parents, up to $6000 annually. This gives parents the flexibility to select personalized services (not regulated) or homestead parents.

Day Care is one of those things where it’s a tough choice for a parent. Licensed day cares provide stability (except perhaps in Quebec) with known hours, a structured system, and lots of kids/supervision. Unlicensed day cares provide tons of flexibility, much smaller groups (if they are not underground), and the added risk of the provider being available (if they get sick, it closes). We opted for an unlicensed one, through reference, and it was a great experience for the kids.

Right, so now the math.

Day Care Costs

Depending on where you live, the costs change dramatically. I was recently paying $60/day for it. Some pay a lot less. On average, it’s not too far from $1000 a month for infants, and $900 for toddlers. So let’s say $950/month. That’s $11,400 a year.

The $10/day play comes up to $3,600 a year. The 75% credit should be $2,900(8,500 off), but it caps at $6,000 credit… so you’re paying $5,400. That’s a solid difference.

Anecdotally, where I live the daily costs are ~$70/day, which is $25,000 a year. The first plan would cost me $3,600 a year, the 2nd would cost me $19,000 a year.

Credit vs Subsidies

The core value behind both concepts is about choice. Credits give people the choice to select the item they want, which would then potentially drive market innovation or variety, with it’s inherent risks. Subsidies instead put the power of choice in the government’s hands, and ensure that the options present meet a certain “base level standard”.

Many services are subsidized – nearly the entire agricultural market. This ensures some level of food price stability, as the providers are ensured a certain amount of return. This can get out of whack something fierce with effective lobbying – the US beef and corn markets are just astounding.

Credits make sense when there’s a complex supply chain and the market is not domestic, or if the item itself is not applicable to the general population. Like a home office credit pre-pandemic. It shifts the burden to the citizen to manage their finances, and the “rich” with decent lawyers can take major advantage of credits to offset income.

Political Platforms

The challenge I find in political platforms today is that they are less about the practical aspects, and more about the ideological mandates. Platforms are complex beasts, there are dozens of items listed in each, and some will resonate, and some won’t. New parents are going to grasp on one part, while retirees will look at something else. The urbanite has a much different set of goals as someone who owns a farm. Finding the balance between both is next to impossible, or perhaps so difficult that it’s ignored.

What is going to happen instead, with a 32 day election period, is that we’re going to have a half dozen sound bites on ideological items, and the actual platforms won’t be considered. The good news is that this will all be over relatively quickly.

I Miss Funerals

Morbid as that is, let me explain.

The funeral itself acts as a pivot point, a step where mourning moves from an individual layer to a social one. A good funeral is a celebration of the person’s life, where people get together and talk about how that person impacted their lives. They’ll mourn the fact that the person won’t be in their lives in the future. It’s a bittersweet event that reminds us of the fragility of it all.

It’s the social aspect that is the true value here. The ability to share with other people, to know that we’re not alone and to share in the memories.

COVID took all of that away for over a year. Oh, people kept passing, that’s for sure! We’ve lost more than enough people during that time as life never takes a break. But the ability to meet other people and truly share in the process wasn’t possible.

I went to my first funeral post-pandemic (mid?) a couple weeks ago for my uncle. Funeral home, a list of 100 people, the tiny sandwiches and all. They streamed the ceremony, which is both great for people who can’t make it, and also quite odd as the physical emotions can’t be streamed. Progress I suppose. I still have social anxiety in small spaces with a lot of people. The ceremony was fine, but the post-even really was not a comfortable space. Really didn’t give a chance to appreciate the event as much as I should have.

This weekend I had another funeral, a more traditional one, for my great-aunt. 90 minute full mass in a church that’s 10 degrees too hot. We had masks (it was well out of town), there was a lot of spacing, and they streamed that one too. I dislike churches to start, and while the eulogy was super to hear, the rest was just me wishing for it to end. Post-funeral was different, we had family members invited to our family cottage for a post-even BBQ. My anxiety was much better here, I opted to cook all the food outside the garage and made a concerted effort on keeping distance. That allowed for a much more cathartic event. I got to see people I hadn’t in a long while and share stories that I had never heard before. It truly was a a great event.

Tangent – I will note the behavior changes that people are taking now. We’re a family of huggers. It’s like this instinctive thing. Seeing people hesitate to give a hug is a very weird thing. Hearing people pre-face any hug with a “I got both shots” is almost surreal. It helps with the anxiety for one. And interestingly, I didn’t talk to anyone who didn’t have both. May be spurred by the fact that we had a shared family member pass from COVID.

I had forgotten what a funeral was, all caught up on the chaos of trying to get through another day. I had postponed mourning, not truly accepting that a big step is to share that joy/grief with others. I honestly missed them without even realizing it.

RIP – Overwatch

An opinion article, fine, but also one that really does a good job highlighting the compounding Blizzard woes of poor pipeline management.

There are multiple factors here, and all of them are compounding on each other. Nearly all of them deal with Blizzard’s management approach.

Content drought

The pace of content for any GaaS is predicated on keeping people engaged. OW has been borderline maintenance mode, in line with the WoW content pipeline approach. Is game development hard? HELL YEAH. Did it get harder with COVID, triple hell yeah. At we 18 months into the COVID model? Yup, and any management team that wasn’t able to adapt (not reach 100% of prior, simply adapt) really shouldn’t be a management team. So while yes, there are delays, no it shouldn’t be an all-stop.

Competition

Valorant is one, no doubt. But Apex, Fortnite and a pile of others still manage to release content on some cadence and take some eyeballs. Gamers are locusts and lemmings. Cool that you had something 3 months ago, but there’s a bright and shiny over here.

Fair to point that Blizzard made a point of keeping track with WoW to launch updates to compete with FF14, Wildstar and other MMOs. The shoe is clearly on the other foot now.

Sequel

Any “live” game keeps eyeballs as long as they know there’s a roadmap. Announcing a sequel, means that you’re pulling people from the current one to the next one. Halo 2 players moved to Halo 3. But those game launch cycles are 1 to 2 years. Overwatch 2 was announced at BlizzCon 2019. It will not launch until 2022, at the earliest.

The serious downside to sequels is that you need an A and B team. Each alternates between dev and support. It was made really clear that this did not exist for OW2, and that they needed resources shared between both. Why not add more people? First, the skills are hard to find, and second, that costs money.

Leadership

There’s no denying that Jeff Kaplan is the reason that Overwatch even exists. The game director leaving mid-stream is never a good sign. And we can’t shy away that this whole lawsuit is going to cause some leadership ripples (for now, this seems to be hitting Diablo 4).

Sponsors

It seems you only need to wait a day to learn about another sponsor dropping OWL. This is a financial consideration after all… HotS was shut down due to funding. OWL cannot survive without sponsors, and if OWL doesn’t exist, then there’s not much coming in terms of funding for future content.

End Result

This is a year and a half of content drought, nothing in the pipeline, leadership woes, serious competition, and diminishing funds. This doesn’t mean that OW2 is dead in the water, but it does provide some serious indications that OW1 is on it’s last legs as an e-sports item. It’ll move into the HotS bucket of maintenance mode.

I really didn’t think that OW would have reached this space this quickly. A fascinating set of circumstances.

ActiBlizz – A Picture Says a Thousand Words

A bingo card

More “departures” at Blizzard, and this particular photo is acting as a sort of hit list of easy targets for Activision to take action upon.

I have done some dumb things in my life, of that there’s no question. We all have. I’ve paid my fair share of consequences, enough to understand that while in a leadership position, there are some fundamental values and ethics that we need to share. And for a very long time, money allowed people to get away with almost anything. If you survived the middle trenches, it really didn’t matter what you did from that point forward.

There’s a fundamental concept in physics, that for every action there is a commensurate reaction. This applies to sociology as well, just that the timeframes are different. Sometimes that triggering event seems minor compared to other things, but it’s just a lack of awareness of the pressure on that given topic. Dam failures are not often caused by massive floods, it’s gradually eating away and lack of maintenance.

SUPER TANGENT TIME! I do want to point that the last US President is viewed as the final trigger for sexual harassment deluge. MeToo started in the mid-2000’s but truly picked up in 2017. The trickle turned into a torrent of allegations everywhere, and you can’t shake a stick without hearing/reading about it. The normalcy that he presented in sexual assault/harassment really pushed this thing over the edge. And where this current leads, I have no clue.

So more Blizzard leadership exits, and I’m somewhat sure that this will continue for the months to come. It’s a good message to leaders that there are consequences for actions, but it’s also years late. And for those who actually want to buy Blizzard products, the leadership exists here mean that dev cycles will slow to a crawl, which certainly isn’t helping things. It’s all bad news, with a sliver of justice for those who were abused in the past.

Let’s see how far this toilet will flush.