The Art of the Play Test: Designing Tests and Keeping Records

Board game development is incredibly iterative by nature. As a neophyte developer, you may go into a project with a clear idea of what your game is going to be like only to find that your ideas don’t work at all and that your failures have opened a whole new path for you. This is super common. The meandering, wild, iterative nature of game development is why I liken it to a journey. You grow, you change, and you find new ways of solving intractable problems. Playing your own games lots and lots before releasing them to the general public is the best way to do that.

Because the path to seeing a creative project to completion is so vague and subject to change, you can’t focus on your plan. You have to focus on your methods. You need to have a way of grappling with your game’s design in a flexible and systematic way that leaves you with a ton of data for decision making. That, my friends, is the Art of the Play Test.

A play test copy of my current project – Highways & Byways

It’s really important to stay organized when you start play testing. One method I’ve had success with is versioning the game every time I make a change that’s not superficial. If it changes the gameplay or user experience, I consider my game to be a new version. For Highways & Byways, it looked like this:

1: First version of the game.

2: Split roads into “hard” and “easy.” Created traffic mechanic.

3: Nixed the traffic mechanic. Implemented construction mechanic.

4: Reduced board crowding and fiddliness of construction mechanic.

5: Added Vehicle Cards.

You get the idea. Pretty much every time I had a new idea I needed to test, I created a new version. The intention was to test things one by one so that if something went wrong (and something very often does), I knew exactly what to blame. The exception is, of course, early versions where the game is borderline unplayable and multiple changes need to be made to keep it playable. Then it’s okay to make multiple edits at once.

Okay, so let’s assume you’re versioning your game. That’s a good start! That will let you keep a changelog that details what’s different between each version of your game. That is really handy to have if you find yourself wondering what you did and when you did it, or if you find yourself wanting to backtrack in development a little bit. You need to go a step further, though. You need to track every one of your play tests.

Everybody who’s made a game knows how incredibly important it is to play test. Games are based around complex, difficult-to-predict interactions. It is only through trial and error that we can confirm absolutely that our games are functional and fun. What you don’t hear people saying as much is that play testing is time-consuming and that it’s really easy to run out of people to play test with. You need to not only play test, but make every single play test productive and worthwhile.

First, start by opening up a Microsoft Excel spreadsheet or Google Sheet. For each version of your game, make a new tab on that spreadsheet named after the version number. For each play test you do for each version, you’ll want to create a new row. There is a lot of information you should try to capture when you play test. Ultimately, what you need to track is dependent upon your preferences, your game, and your business needs. Here are some questions I try to track the answers to:

Who played? You want to track whether players are new or veteran to your game. Veterans will know how to make coherent strategies. Newbies won’t. You want both groups of players to feel like they’re having a good time.

How many people played? Some games play better with fewer people. Some play better with more. The ideal number of players is often called a “sweet spot.” Tracking this will help you find your game’s sweet spot later on. It’s also good for seeing if games run too long with more players or if a certain number of players breaks the game.

Who won? You’ll want to track this for two reasons. One, you want to compare it with the player’s strategy. Two, if someone becomes an exceptional player – good or bad – you’ll want to ask them questions about how they’re playing your game.

How long did the game take? Not only will this help you keep your game from ending too quickly or running too long, it’s something you’ll have to put on the box to sell it to people.

Describe each player’s strategy. This will help you identify game-winning and game-losing strategies. You can then nerf or buff certain parts of your game as necessary.

Were there any game-breaking flaws? If so, describe them. If your game broke, you need to make a new version and try again right now. “Broken” here meaning that the game was rendered unplayable or just plain terrible for some reason.

Did you catch any minor errors? If so, describe them. Document any tiny errors or gripes that your players have. Decide what to do with them later.

Were there any ambiguities in the rules? If so, describe them. Poor communication sucks the fun out of games. If you find your players getting hung up on certain issues, you’ll want to write them down here so you can clarify your game’s rules or tweak mechanics as needed.

Did you catch any typos, graphic issues, or small errors? If so, describe them. This is obvious, but nothing says “unprofessional” like a typo.

I practice what I preach. This is my tracking method.

In Highways & Byways, I use the following columns to track my play test data:

Version

Game Number

Players in Game

Start Space

Player Strategy

Car

Roads

Game-Breaking Flaws?

Minor errors?

Ambiguities?

Resolution

Comments

Now just because I’m using examples from my own project, that doesn’t mean it’s golden advice! Game development is very personal, and many decisions are yours to make. No one can tell you what is right for you.

Commit to the journey. Plan to throw out your plans. Be ready for changes when it’s time to change. Stay organized and take great notes. If you do this, you’ll have a great head-start in your game development project!

Join my community of over 1,100 game developers, artists, and passionate creators.

5 thoughts on “The Art of the Play Test: Designing Tests and Keeping Records”

This was an interesting read. As a board game blogger and board game club host I get a loads of playtest requests. My group and I tend to do it quite a lot. Local developers here in Finland have not yet achieved the level of feedback forms you have. I think I’ll loan some ideas from your list and try to implement that on our feedback.

I wish you all the best in your development endeavors with Highways & Byways.

I don’t tend to use formal surveys or anything when I test, but I’ve always got these questions in the back of my mind. During each play-test, I’m watching the behaviors and strategies of players. After each play-test, I ask questions about what their experience was like. I find going in with clear goals helps a lot.