There’s a pretty good article over on Ten Ton Hammer offering a look at the current state of beta testing. As many others have noted, not much testing goes on in the public stages of MMO betas, and the article offers a variety of perspectives from both developers and players.
My old comrade Scott Hartsman chimes in with a voluminous set of observations (without me around to edit him, Scott will just talk forever!) on how a big-budget game can manage to make it into beta and be lacking in the fun department:
Let’s say you’re a AAA game with 3-4 years of time and money invested, enough money to support a large team having worked on it for that long. Games like this frequently need to go for years before enough pieces come together before you can start making decisions about what’s fun and what isn’t.
By the time beta begins, you’ve made decision after decision that have compounded on each other. Your assumptions’ assumptions’ have assumptions about what your game is. The whole product, systems, content, operations, marketing, PR, community ramp, you name it — is built upon them. Changing core assumptions about the product itself is unlikely to be possible without significant delays, costing progressively more money per month. (Remember, the months toward the end of the dev cycle are the most expensive ones by far.)
The game is, for the most part, what it is. You’re capable of making shifts, but the more complex the game, the more minor the shifts you can make with any confidence. If assumptions that you made years ago turn out to be wrong, you’re left to scramble, or in most cases, do your best to ameliorate the now-certain fallout.
If you haven’t verified your gameplay at the point of having a beta, you’ve already left your fate to chance. (This is, of course, all presuming that your game has passed the technical bar in terms of stability, which is all too often not the case. And, again, is another flaw with the launch-big-or-die model.)
The guy from ohai speaks true. If you start out with mediocre core gameplay and pile tons of mediocre decisions on top of it, soon you’ll find yourself in a plate of spaghetti and weaksauce you won’t be able to escape (how’s that for mixing metaphors?). For instance, if the act of killing enemies over and over is the centerpiece of your gameplay, then you better make damn sure basic combat feels interesting and cool because otherwise nothing will be able to detract attention from the suckiness.
On a more positive note, I recently had a happy beta experience and felt I should share. I installed the Chinese open beta for Aion after finding instructions on how to apply an English-language patch. Considering the hoops jumped through to get the client running in English, I expected a train wreck and was hoping I could at least get a feel for the visuals and gameplay. Instead, I found myself pleasantly surprised that the language stuff worked and the game played smoothly. Though the experience wasn’t perfect, there was a thoughtful layer of polish to the UI and other elements of the game that I found refreshing. After not expecting much more than a grindfest, I’m now looking forward to the US version.
So beta–or live preview, or whatever you choose to call it–still has the potential to positively influence the market toward your game, so long as you when you lift your skirt what you reveal to the world is solid (ugh, that metaphor is simply uncomfortable).
Look, every shipped triple-A MMO is going to have bugs. These games simply have too many systems and too much content to test every possible use case before launch, and everyone from Blizzard on down spends the first few months after a major release fixing things. And players will accept that, as long as you’re up front about the issues and the core gameplay experience is rich and enjoyable enough to mitigate the bugs until they’re stomped out.
But public perception based on that first public showing has a measurable impact on your game’s ultimate fate. Now more than ever, major signs of weakness are going to haunt your sales potential for eternity, and even the greatest patches can only do so much to fix that lingering initial perception.