Developing Better Sets

Sam StoddardFriday, March 07, 2014

his week in Making Magic, Mark Rosewater talked a bit about how iteration helps design, and some advice for people making their own sets. Development is, by its very nature, a process based on iteration, so in a similar vein, I wanted to give some advice to those same people, or to people working on Cubes, about how to better develop their own sets. This article will go over a few rules that I have learned over the last two years about developing Magic sets.

Development is divided into two parts—structure (where we work on the Limited environment) and format (where we work on the Constructed format.) Depending on the size of the set (large sets get more time than small sets), development has between ten and fourteen weeks of playtesting for structure, and another five to seven weeks for format development. And then there is around eight weeks of "devign" before all that to make sure the set is ready to get handed over. There are hard and fast deadlines (after all, we need art, and packaging, promos, etc.), but that doesn't mean there isn't time to experiment—in fact, the reason why we have the amount of time we do, and not just a few weeks, is so we have time to experiment.

Nothing is Right the First Time

There is a saying I was taught in my writing courses for college, and it definitely applies to developing Magic sets—"The good is the enemy of the great." The problem with focusing just on things being good and balanced, especially early on, is that it doesn't leave you with much actionable feedback. I think that, for most people, their natural inclination is to get things just about right before each playtest, but that removes one of the most important parts of playtesting—doing science! While our goal for the play experience with a set when it actually ships is for it to be new, fun, and different, the fun part isn't actually as important early on. It's nice, of course, but the goal is for the best final product, which means taking risks and seeing which ones work and which ones don't.

A lot of what makes individual Magic sets great is that they are able to take something that would not play well in other environments, and find a way for it to play well. Innistrad's use of morbid and flashback create a higher-than-normal focus on the graveyard that would be detrimental to many sets, but when the set is built around it, we can focus on turning that into an upside. Similarly, Theros is very much about building up creatures, which can be frustrating, but works as an environment when we build around it.

Many a great Magic mechanic came from a large amount of frustration early on in the process because we were able to figure out how to reduce the things we didn't like and improve and increase on the things we did. We want new Magic sets to add something new to the game, and that means testing boundaries. Testing things out means occasionally things falling flat. A lot of this testing gets out in design, but they rightfully pass the things they feel have real promise on to development, to see if we can figure out the right mix.

Try the Wrong Things

It might seem counterintuitive, but starting off with the wrong thing can be more useful than what you believe to be the right thing—as long as you know in what direction you think you are off. After all, it's easier to fix something if you know what is wrong with it, rather than knowing if something is right if it doesn't feel wrong.

Let's say you are working on a Cube, and you want to add an enchantment theme to white—if you guestimate what you think the right number of inclusions are and nobody plays the deck, what have you learned? It could be that people didn't open enough of the cards, or that the individual packs just happened to break in such a way that more powerful strategies came out. Or, it could be that you need more cards to support the theme. What if someone tried it and the deck wasn't strong enough; how much would you need to add to get it to be competitive? If you overshoot by a healthy margin and things are too strong, but it's otherwise working, then you can get much closer to the right number in one more iteration by just cutting back a few cards.

Oftentimes, I will be asked by players questions like, "Did you try bestow at a lower cost?" Well, of course we did. We wouldn't be doing our jobs as developers if we didn't try out various permutations of cards and seeing just how far we could push things. While bestow was much more powerful at much lower costs, it was also less fun. How do we know that? Well, we tried it, and it was less fun—with too many games coming down to nothing else but who could build a more impressive Voltron by turn five. After trying that out, and having some solid data points on the low end of costing and how that felt, we knew the direction we wanted to move in with the mechanic, and brought it up to the costs that you see today.

When you are making changes to a set, it is important to leave yourself in a position where you have an idea of where to adjust things. Early on in development, if you aren't doing at least some science with each playtest, you aren't doing enough to improve the set for the long haul.

Don't Try to Fix Everything at Once

This can be one of the hardest things for new developers to learn—or, at least, it was hard for me. While it can be attractive to go through a set and make a ton of changes to improve the game play, doing too much at once adds a lot of noise to the system. It lets you tell that things have improved, but it makes it hard to know which changes were really for the better, and which were for the worst. We budget out time for the set to develop, and it's better to use a system of continuing quality improvements over time to hone things, rather than a lot of wild swings at what the perfect mix might be.

Let's say you are working on a set's Limited, and things aren't all going well. People are complaining that the set is too swingy, and the person who gets ahead in the early game always wins. So, to fix it you weaken a few black card and red cards, strengthen a few blue and white cards, give red and green a new mechanic, and put in more color fixing. During the next playtest, people had more fun but now feel that the control decks are too good. What do you do now? By changing so much at one time, you made it hard to tell which of your changes worked, and which didn't. Maybe you made control too good, or maybe you made aggro too weak. If you had, instead, only made half as many changes—let's say just weakening black and red and adding more color fixing, then you would better know how your changes impacted the set, and you could've used that feedback to see if the red and green mechanic really needed to change.

That isn't to say you can't make some larger, sweeping changes between playtests, but it's usually best to do so with a clear purpose, and with strict parameters. You could imagine working on Odyssey and trying a different mix of threshold numbers and enablers to get to the right mix, but you wouldn't want to radically change how flashback works at the same time. In the end, playtests that result in clear, actionable feedback, are just more productive than ones that don't.

Today's Solutions are Tomorrow's Problems

Don't get too enamored with your own development choices. The goal of the set is to iterate and improve things over time, and that often means that what might be one of the best aspects of your set early in the development process is holding your set back near the end. You have to be willing to let it go.

When a set comes over from design, there are a lot of rough edges. It's design's job to test the boundaries of what can be done and development's job to figure out how to best implement those decisions. This leads development to figure out clever ways to tie mechanics together or get color pairs to work. These kinds of discoveries are good, but that doesn't mean they are as good as they could be.

Let me give a more concrete example—let's say you are working on a set with a new mechanic based around hand-size matters. You say, "Aha! I know exactly what mechanic would work well with that—buyback! It means that I can cast spells but not lose a card in my hand!" You put it in the set, and things go great for a while. The mechanic is working great, people are having more fun, and you have the time to figure out which of the hand-size matters cards are fun, and which aren't. You even get the chance to figure out some other cards that play in a similar space to buyback, but work differently.

Then, you start getting feedback in your playtests that buyback is repetitive and leading to stale game play. You tweak the buyback cards, which helps, but now people complain that not only is the game play not exciting, but their cards don't even look appealing. As the developer, it is important to consider the fact that buyback was never the end-all-be-all-silver-bullet answer to the set, but instead just was in a close-enough range that you could work on other aspects of the set, and those had improved to the point where buyback is now the worst aspect.