Feeding the iteration engine

During my humble time in the games industry, the most prominent feedback I would receive about ‘how to make successful games’ was ‘iteration’; that is, constantly polishing, grinding and working on a video game would continue to make it better, more focused, and more qualitative. One problem with this, however, was that there was never a methodology behind it; but rather a wandering vague guideline to iterate to be great.

As with most things, the games industry didn’t create this mantra, and certainly web-companies have done a better job at embracing this concept in their battle cry “ship early, ship often.” This methodology has been proven multiple times as a cornerstone of the creative process. 37Signals’ book Rework, a clear counter-culture process is presented for product development:

1. Ship a product super early
2. Let the users tell you what’s wrong and what needs to be fixed
3. Fix all the stuff in #2
4. Repeat
5. Profit

Lean startups have been adopting this philosophy more aggressively lately; with their champion FakeGrimlock offering sound advice in bite size form:

Make sure that you're on FAKEGRIMLOCK's #NOEATFRIDAY list; As he says "you on the list, or you on the run."

In the modern digital age, iteration is sound advice, however once adopted, you’ll quickly find that the provided tomes contain plenty of information on the why but quite lacking on the how. There’s a disconnect between the dogmatic message and the mechanical operation. Especially in game development, where most often, devs ignore proper iteration during pre-production and production phases,focusing only on beta/post-ship feedback; This is a great shame, after all, content creators should have the ability to iterate on their craft as well.

Iterative gaming wins

Often, game developers are adverse to early feedback of their product, fearing that if exposed to the criticism of the outside world too early, will cause the game to fail. Interestingly enough, all the data shows the contrary, that games which ship early in alpha, beta, and continue to grind and make improvements in the public eye have been shown to have stronger user engagement, and have MORE success over time. Minecraft and Realm of the Mad God are perfect success stories in this space, having launched in early forms, and didn’t see success until being in development which included the public feedback loop, which quickly vaulted them into massive success stories.

While desktop and Web games have this ability directly, mobile game development exhibits this phenomena in a different way; where smaller companies, trying to power-level the limited space distribution platform, ship multiple games in a short time to attempt to stay on the top of the rankings. Implicitly, this creates a user-feedback cycle; This fast amount of iteration allowed the developer to get a great deal more feedback about their ecosystem and habits of their customers, in addition to doing a breadth first search on what content will become popular. An often difficult task to predict, given the general random nature of the marketplaces.

Poster-child success story Angry Birds is heralded as the secret sauce that caused the mobile gaming boom, but often forgotten is that developer Rovio Entertainment had shipped 30 other mobile games over the course of 3 years. And this has proven to be a process that works, with early 2012 success story Temple Run being the 10th title from game developer Imangi on the mobile platform, resulting in over 100 million downloads of the app.

Iteration means focusing on what counts

This approach to development methodology can be a breath of fresh air for most game studios who complain about feature creep. As Donald Knuth was once attributed to saying "premature optimization is the root of all evil.” And web developers / lean startups are right on target with this effort. Rather than spending years grinding internally, only focusing on internal feedback about priorities and power-struggles associated with internal politics, these developers were getting their product directly into the hands of the consumers, and prioritizing feedback based upon that first. These developers don’t have the capacity to prematurely optimize; There is such strong dialogue with their potential user base, they knew exactly what to work on, and the impact it would have on the end product. Which is the important part; after all, the customers are the ones keeping your company in business.

Your Iteration Engine Checklist:

Your train to success should be driven by the iteration engine, and like any complex system, you need to understand the parameters before building anything. As such, here’s you’re iteration engine checklist:

Pre Production:

• Make a place to keep, sort, and find iterations & feedback. As you start iterating and collecting feedback, you’ll soon want a log of what didn’t work, and why. And as your project rambles on, you’ll need to refer back to that data over time, so find a good place to store your successes and failures.
• Iterate the hell out of your idea. Share it with others, get their feedback; Automatically assume that you’re too close to the idea, and need others to validate it. Start with your co-developers for feedback, then slowly grow your circle to friends/family before opening up to unknown people.

• Embrace a 1-off technology for prototyping. Don’t be afraid to generate tech that you’re going to throw away; You learn a lot from those efforts. Have your designers work in HTML5, Flash, Unity, or XNA to quickly build a prototype, and maintain a fork of their efforts so they aren’t blocked on product features.
• Start with someone elses’ idea and make it better. Some of the most successful games of all time have been more spinoffs of already successful games. Halo, World Of Warcraft and Braid are great examples of this. They were able to duplicate the successful parts, fix the crummy parts, and add their own unique twists and improvements to make a much better product.
• Make something fun without the features. Your goal out of pre production should be to have a fun game mechanic, that feels fun without the need for millions of bells and whistles. Take this time to prove your hook, interest, and goals; the extended feature set should add to your product quality, not define it.
• Clearly define what your ARE / ARE NOT. During iteration, it’s easy to ramble on down a path which may not be directed towards your initial goals; while sometimes this can yield amazing success, it’s important to put an upper limit on your efforts. If you’re trying to make a platformer, don’t start programming an FPS. Your definition of what the target is should be constantly evaluated and appear in every iterative decision process.

Production:

• Design in parallel. Keep designers working in the prototype build, in parallel to commercial builds. This will allow your designers to prove ideas and gameplay that won’t force new task ripples through code, art and production groups, this is a critical path that keeps feature creep at bay; You can test if something is fun without having to move heaven and earth to test it in-game.
• Embrace tools for small content-creation loops. Content creators should have a tight feedback loop to getting content into the game. Optimize your content creation tools to allow Artists to get content in as fast as possible, as easily as possible. Reduce the number of steps, clicks, and exports needed.
• Always Be Stable Builds (ABSB). Production builds should be continuous and easy to create. Make sure that you have a way to kick off manual builds, alongside automated builds that occur daily.
• Bonus points: Make sure that the developer knows what build they are running at all times.
• Double Bonus: Zip up the source code and asset trees for that build, so that you can properly roll back changes and issue tests on historic code.
• Track your stats. Track crashes and automate testing in as many ways as possible; Make this part of your build system; Incorporate Google’s testing unit testing framework. Can you chart the FPS changes over the past month? Do you know if a checked-in asset is causing out-of-memory crashes? Where’s your memory going today?
• Bonus Points: Embrace great data visualization tools to view your daily stats
• Don’t get comfortable with your game. Once you get comfortable, you’re too close to give feedback. Don’t playtest your own game too much; instead get friends, family, close testers to keep trying your game and give you feedback. Don’t let them become familiar with the game, or their feedback becomes useless. You should listen to your user feedback and iterate quickly to update your build with better content.
• Make time to clean up. By constantly iterating, your art and code can become a mangled mass of items that don’t exist any more. Make sure that on regular intervals you’ve scheduled gardening duty where everyone pitches in to make sure the garden is free of goblins that will inhibit future endeavors.

Post Ship:

• Create a shippable product. Make sure you’re extending your ABSB to the customer. There comes a point which you will need to patch up all the holes and issues before shipping a polished, stable build. Shipping unstable builds reduces the user experience and does nothing but cause stress.
• Ship Early. The only way to get feedback is to have other people play your game. Ship early versions of your game sooner than you’re comfortable doing; Put your ego aside and embrace the feedback.
• Ship Often. Make shipping updates part of your product lifecycle. And then keep doing it. Make sure that your technology supports easy patching and content updates; make the user happy to see that a new patch is available.
• Understand your user. Ensure you’re deeply tracking user-clicks and actions, mining the data for meaning. Issuing tests and checking results should be part of your normal process.  How many players make it to the 3rd level? How many have purchased armor? Constantly run and A|B tests to find the best results.

All aboard the iteration train!

The iteration engine can be considered a fractal algorithm. Directly you should be focusing on how to track user feedback playing the game, and then quickly respond to it in order to make better products. Truthfully though, this action extends to production and pre production as well, where you need to iterate on your ideas and content creation processes, just at a smaller level.

The iterative engine is not about saying ‘no’, it’s about proving ‘yes’ .

More importantly, it’s about proving success.

~Main

You can find Colt McAnlis here: