Monday, August 16, 2010

As I coach new developers, I've taken to scribbling out the same useful diagram for visualizing the creative process again and again on coffee-ringed napkins. In order to limit my future abuse of culinary paper wares, I've reproduced my images in a more formal fashion in this essay.

The conversation usually starts with the following statement: "Creativity is like a snake swallowing a series of tennis balls."

And when confused looks inevitably result, I sketch some variant of this odd little picture:

Using this as a starting point, we start chatting about joys and pitfalls of creativity.

The Brainstorming Phase

Failures in brainstorming

The Culling Phase

Failures in culling

Cycling

Failures in cycling

The Brainstorming Phase

We all start with an idea. It could be a small inspiration or a large insight. Immediately, you begin a process of brainstorming and daydreaming. This is a time of infinite possibility and promise. I use the term brainstorming broadly to include any activities that expand the options or possibilities of a project. The traditional image of a group of designer types sitting in a room with a whiteboard is indeed part of the brainstorming phase, but it is really only one of a much broader spectrum of activities.

Brainstorming starts out small and expands over time

There are several activities that occur during this phase:

Ideas: Generate new ideas related to your initial insight. These are proto-experiments. A half-jotted note in a notebook "Solar powered underpants!" is an example of an idea. (This is a real idea. It has been hot in Seattle lately)

Thought experiments: Invest energy in your ideas to understand how they might be built and thinking through the theoretical impact of each idea. A spec concerning "A Method for inserting a small fan in one's shorts to reduce ambient temperature" is an example of a thought experiment.

Real world experiments: Build working models from physical materials or usable code so you can experience your idea first hand. A working model of underpants plus a small metal fan (plus a ready supply of bandages) is an example of a real world experiment.

Cross fertilization: As you work through ideas, you see new possibilities. I've discovered through much trial and error that escaping to a coffee shop with air conditioning is immensely more effective than other attempted alternatives.

A multitude of experiments arise during brainstorming

Brainstorming is ultimately the act of kick starting experiments. Even when you dream up a completely off-the-wall idea, you are stating that "There is some potential in this direction." You've formed a postulate at the fuzziest possible level. Future steps during the brainstorming involve making more detailed predictions and modeling the results.

When brainstorming is successful, we end up with a portfolio of experiments These go by a variety of different names in software development: features, user stories, use cases and 'the thing Bob made while screwing around on the weekend.'

Problems with brainstorming

The single most common flaw during the brainstorming period is that creators do not build enough testable experiments. This mistake comes in a variety of flavors.

The creator already knows what needs to be done

Experimentation is considered expensive

The original idea is brittle

The creator already knows what needs to be done

Creator pursues a single solution

Why waste time on expensive experiments when the right answer is obvious? The flaw in this thinking is that creativity is an iterative process in which you synthesize the final result from a variety of sources and thousands of potential solutions. It is not purely a deductive process with a single right answer.

When you fail to experiment broadly, you are building your solution from an anemic set of mental and technical resources. It is the equivalent of trying to design a bridge when the only material you've tested is paper. You can certainly build a bridge, but it will not be nearly as good compared to someone who experimented with a broad range of materials and construction techniques including steel or concrete.

To understand the power of a portfolio of experiments, consider some simple statistics. If 4 out of 5 experimental systems are bound for failure and you create only one experiment, you have a 20% chance of overall success. On the other hand if you create 10 experiments, you have a 89% chance of finding a success. In practice, your chance of success is even higher since ideas cross pollinate. By learning and adapting to your new knowledge you'll uncover new options that are often far superior than the original set of experiments.

(Note: If the goal of experimentation is hands on knowledge, try including a wide range of participants that can bring a variety of pertinent skills to the table.)

Experimentation is considered expensive

Each individual experiment is expensive

In the example above, a savvy counter of beans might note that 10 experiments is likely to cost 10 times as much as a single experiment. Yet all this extra money only increases the additive chance of success by a mere 450%. So the team compromises and invests in a handful of expensive experiments.

The solution here is to use less expensive experiments, not fewer experiments. What can you make in a day? What can you make in an hour? Instead of using teams of 5 to 10, what can you learn with a team of 1or 2? By focusing on lightweight experimentation and rapid turnover between experiments you can pack more experiments into your brainstorming phase.

One technique I love that keeps experiments small is Post-it note design docs. Since your experiments must fit on a sticky note, you are forced to keep the scope small and easily implementable.

The original idea is brittle

No matter how hard you try, you only can come up with a handful of ideas

Sometimes you start with an idea that seems brilliant, but as you try to expand upon it, you keep running into walls. You know you've found a brittle idea when you try to bend the concept in interesting new directions and it collapses again and again.

Over time, you get learn to recognize brilliant yet brittle ideas early in the creative process. My test is to try to think up 20 or 30 crazy variations on the idea. If I can imagine that most of those variations would be exciting to build, then I know I've discovered a robust idea that is worth investing in further. If I can't, then I have a brittle idea.

It is too easy to invest months, even years of your life trying to "make it work." Instead there are a couple techniques for making the idea more robust:

Put the idea aside: The single best thing you can do is to put the brittle idea on the back-burner. Over time, if the idea is in fact brilliant, it will find its way back into your creative process. A different perspective, be it brought on by time or new experiences, can be an essential ingredient in softening the idea's previous constraints.

I wrote up an idea for a game called Cute God a while back, but none of the prototypes really gelled. It was a design with a thousand problems and very few good solutions. Instead of belaboring the point, I consciously stopped redesigning it. Years later, some of the original combinatorics ideas found their way into a game called Triple Town that should be released later this year. Ideas you cull are never really erased. Instead they turn into fertile soil from which the next generation of ideas are grown.

Radical simplification: What is core experience that you are trying to achieve? Write a single sentence or draw a picture that represents that experience. Put that on the wall in front of you. Now brainstorm a dozen different incredibly simple ways of creating the essence of that experience for the user. Toss out all your complex systems and constraints and start over with just the core.

Recently a friend and I went through this exercise on a game concept that was born from a pencil drawing of dozens of stick figures attacking one another. The brilliant, yet brittle design was a Toribash-style fighting game where you could move individual joints for each of the stick figures in an epic multiplayer battle. As an exercise, we went back to the original drawing and asked "What was the simplest way to get that image up on the player's screen" and "How do we evoke the coolness of a dozen stick figures blasting one another with shotguns." Out goes the rag doll physics. Out goes the complex UI. Out goes the multiplayer. The resultant idea was robust, easier to implement and still captured the emotional joy of the original inspiration. At the very least, this exercise helped the designer look at the problem in a new light and question their original (brittle) constraints.

This process can all be summed up "If your design is difficult, cheat."

Culling Phase

There is an entire second phase of of the creative process called culling. Inevitably, not all experiments are good experiments. Some show immediate value and others are plagued by obvious flaws.

During the Culling Phase, you need to kill flawed experiments so you can double down on good ones.

Culling boils down all your experiments to a refined nugget of value

No one really talks much about this unpleasant part of creativity. We lionize the eureka moments of brainstorming and end up ignoring the agonizing process of trimming and shaping those meandering experiments into something coherent and valuable. This is a huge mistake.

Critically culling your experiments is an essential step to any successful creative act. Culling focuses the creative act, ensures projects are finish-able and ultimately yields a more powerful final experience. When I start painting, I place down a thousand brush strokes. But only one stroke appears on top and is visible to the viewer. Each previous stroke is an experiment that leads me towards that final visible stroke. Once I've learned enough from my experiments, I make an informed decision, place the optimal mark and move on.

Those who fail to cull, fail to create meaningful projects. You can spot a newbie designer from a thousand yards by suggesting they kill a feature that doesn't seem to be contributing much. Their nostrils flair and their voice rises. A litany of denials, excuses and accusations pour forth. And you know immediately that their project is going to be an incoherent piece of crap. This is a good coaching moment. :-)

Good experiments get more love and the questionable ones get trimmed

Culling is composed of several activities

Determining your culling criteria: You need to know how you are culling up front so you don't end up making arbitrary decisions. The single best way I've found of defining culling criteria is to write down what success looks like. For example, in Bunni 2, we say that the game is a 'social stickerbook'. Anything that doesn't fit that vision is worth questioning.

Deciding what to invest in further: Look for opportunities to amplify obvious value in your existing experiments. For example, in Half Life 2, one experiment was this risky concept called a 'gravity gun'. When a real world prototype was made, players loved it. Valved decided to invest further and tried to figure out how that gun could be used throughout the entire game.

Deciding what to remove. If something doesn't fit your culling criteria, it is better to cut it early and spend those resources elsewhere. In Bunni 2, we dabbled with a combat system. However, after we built a simple version, we realized that it really didn't fit our culling criteria so we put it on the backburner.

Problems with Culling

There are several common issues that come up during the culling phase

No explicit culling criteria

Experiments are not tangible.

Assuming more is better

Fixing every problem

Focusing on problems not opportunities

Judging features not core experiences.

No culling criteria

Often people get so caught up in the amazing optimism of brainstorming, they fail to agree upon explicit culling criteria. Instead the particular politics or opinions of the day hold sway and features are randomly invested in. Culling does occur, but in a haphazard fashion that is just as likely to kill good features as bad.

When you leave your culling criteria vague, you are saying that it is okay for everyone to have slightly different opinions about what is good and what is out. This leads to unnecessary conflicts. You need to bite the bullet and have the hard conversation about what your shared vision for the project should be. I like the term 'culling criteria' since it ensures buy off from everyone that, yes, you will mothball experiments for the greater good of the project. The act of explicitly stating a small set of common goals ensures that everyone buys into something bigger than their individual efforts and passions.

I put 'goals' at the top of almost every single design document I write. Though this is the shortest part of the design, it is often the section most critical to success. Experiments will blossom in dozens of directions, but the goals keep the project on track.

The experiments are not tangible

Mere opinion is the only indicator if an experiment is good or bad

Often due to fear of creating expensive experiments, creative folks spend far too much time in the land of ideas and thought experiments. Some call this documentation and invest in it like some religious protection from mistakes. As a result, very few real world experiments are built. With nothing concrete to react to and judge in a critical fashion, it becomes difficult to apply the culling criteria in an objective fashion.

The solution is to make your ideas real as quickly as possible. Paper prototypes, 24-hour game jams, role-playing with a friend...create your idea in the physical world. To get a bit geeky, a vast portion of conscious cognition emerges as a post-processed rationalization of our body's physical interactions with the world. Feed your subconscious cognition by creating systems you can touch, see and play with directly.

Instead of merely wearing down a single golden path in your mental thought experiments, you'll accumulate the wisdom that only comes from a thousand real world observations. You'll see players smile when they stumble. You sense the stickiness of a dialog that asks you to 'click continue'. You'll give yourself the richest possible source of information about the problem space in the shortest amount of time.

Recently I wrote out a design for a simple word game called Panda Poet. It was completely obvious how the game played on paper. The rules were crisply defined and I could easily play through the experience in my head and imagine the delight that would result. All it really needed was a quick implementation and the project would be done.

So we implemented the first prototype. The result was completely unplayable. The interface was fundamentally broken. The feedback loops were not functional. In the first 10 minutes of playing a physical prototype, I learned 10 times as much about the problem space than I had learned in the previous days spent designing on paper. Panda Poet was almost a failure, but by building multiple real world prototypes, we learned enough to salvage the design. It should be out later this year.

Assuming more is better

If you started something, you feel obligated to release it.

As a creator it is easy to get caught in the belief that more is more. Yet, the people consuming our creations rarely feel this. They have limited time and limited mental resources to spend on our work. Instead, they see all the extra 'stuff' as mental noise that actively harms their enjoyment of the project.

Imagine that you have a painting of a duck and you start randomly adding static in the form of blobs from other pictures to it. You can add a lot of static and still tell it is a duck. But the static detracts from the image. The end user would likely be much happier if you just gave them a great picture of a duck.

The same thing occurs when you fail to kill experiments. You are actively adding low quality noise to your creation.

Creators that fear rigorously editing their creations suffer from the sunk cost fallacy. They assume that since they spent effort making something, value will be lost if it is removed. Often they consider their work 'precious'. However, there is no inherent utilitarian value in a feature simply because it took a lot of effort to build. The user doesn't see your effort. They only see the messy and imperfect noise that come from not rigorously culling your flawed experiments.

Practice killing your ideas: David McClure has a great saying, "Kill a feature every week". Culling with wisdom and discipline is a skill worth training much like any healthy habit. You practice and it slowly and steadily gets easier. You begin to see each feature and experiment as a small step in a much larger process, not a rare and precious thing that must be protected.

Be objective: Another technique is to use objective, data-centric criteria. By boiling down decisions to numbers and metrics, you give yourself permission to make emotionally difficult decisions. This can be taken too far, but is particularly useful if you have a group of people that have divergent subjective opinions on a topic. Again, real world experiments facilitate this approach. There is an objective measurable reality to how people react to art. Understand this fact and used it to facilitate your culling.

One implementation of this technique is the use of Fun ratings. I have a survey built into many of the games I work on that asks users how fun the game is on a scale of 1 to 5. It is one thing to tell someone that their game sucks. That statement comes across as a purely subjective and potentially insulting opinion. It is another thing to have 10,000 people say that your game rates 3.1 out of 5 and to know from historical data that you need to reach 3.9 in order to have a chance at financial or critical success.

Fixing every problem

Every problem with every feature must be solved

A related issue is that creators often attempt to fixed all the problems with their failed experiments. This is particularly common on projects that are thought of a series of modular features. The creator lists out the problems with each feature and then methodically fixes each in an attempt to bring the feature and therefore the project up to a reasonable level of quality.

The impact of this technique is painful to witness.

The expense of 'completing' the projects blossoms and progress across the board slows to a crawl as multiple objectives are pursued simultaneously. Adding more resources often only slows down the project more (ala the Mythical Man Month.)

Quality still decreases. It is rare that each feature is completely aligned to the central value of the project. You end up with a project that is being pulled in a variety of directions all at once. The result is like student art where a new artist meticulously polishes every single shape in the drawing, but the end result is a hideously disjointed experience.

A good rule of thumb for game development is that every experimental feature you start takes 30 times as much effort to finish. So one day prototyping yields one month polishing. If you try to polish everything, the surface area of your project grows huge in very short order.

I once worked on a failed online game that suffered from this issue. There were about 5 different project owners, each of which assumed that their job was to point out the flaws in the game and mandate (on threat of death) that the team fix them all. The team would scramble to plug holes one after another. In very short order, the game turned into an incoherent mass of half polished features. In hindsight, a cleaner set of culling criteria that resulted in killing broken features would have resulted in a more focused project of higher overall quality.

The solution is to focus less on the problems and more on the opportunities. You win when you generate value. If you release an unpolished version of a something genuinely interesting and wonderful, it is amazing what people will forgive.

In Steambirds, the levels were tossed together with a semi-random assortment of planes. The writing on the mission text was highly questionable. The graphics were one iteration past the initial prototype art. There was an invisible wall that caused players to die inexplicably. It could have easily turned into a 12-month project.

However, what we did right was polish the heck out of the core mechanic of movement and attacking. Of all our experiments, it was a robust and interesting gem that resulted in a powerful user experience. In the end, by focusing on the heart of the game, none of the other issues really mattered. We saved months of labor by focusing on what worked and killing or ignoring what didn't.

Cycling

Brainstorming and culling occurs in an iterative cycle. In each cycle, you create experiments and then cull back to the valuable core. Then you repeat. Each complete cycle spawns a new spurt of ideas and experiments that must be culled in turn.

Each cycle results in accumulating more value for the customers of your labor. When you've generated enough value, you stop.

Cycling problems

There are a couple issues that occur during this process.

Not iterating.

Not delivering value to the customer.

Not iterating

Often creators stop after a single cycle of brainstorming and culling. I see this quite often in groups that come from a waterfall-centric culture. Planning is treated as the equivalent of brainstorming and initial implementation. Culling is treated as scope reduction and bug fixing. After one long cycle, the product releases.

The solution here is shorter, lower cost cycles. Any of the various agile methodologies cover this ground extensively. To facilitate more cycles, I ask the question: How do I decrease cycle time so I can fit more learning cycles into my project? I've asked this question for art, for games, for UI design and for application development. In all situations, the more cycles I can pack in, the happier I am with the end result.

Not delivering value to the customer

The exact opposite of not enough iteration is to continue to iterate indefinitely and never release the value you have accumulated to a wider audience. There is always room for improvement and always new value to generate. You can get stuck in the trap of constantly cycling through the creative process and never feeling that your product is good enough to release.

Here are two good solutions I recommend

Timeboxing: Set a release date and release regardless of how far you've gotten. Pixar's Darla Anderson has a great quote that "we don't finish our films. We just release them" The predetermined release date is a forcing function that ensure they pack in as much value as possible before they are forced to put something out. It also ensure that you stop working on a flawed experiment. The emotional distance that comes from releasing can be extremely helpful in realizing that you need to take a break from a particular great white whale that is eating your life.

Release criteria: Another alternative is to set objective goals that trigger a release. For Flash games, I know that when I've hit a 3.9 fun rating, I can release the game. Certainly, I could invest further, but it really isn't worth it.

Conclusion

All these thoughts and pictures can be summarized in a very short list.

Brainstorm: Create lots of low cost, real word experiments.

Cull: Rigorously apply agreed upon culling criteria to weed out the weak ideas and reinvest in your most promising experiments.

Cycle: Repeat the process until you generate meaningful value.

Practice: Across multiple projects, practice all stages of the creative process so you constantly improve the myriad of skills involved in brainstorming, culling and cycling.

Why people read this blog

About Me

I've been a game designer, pixel artist, toolmaker, physicist and MBA. My first job in college was on a game called Tyrian at a tiny company called Epic Megagames. These days, I'm the Chief Creative Officer at Spry Fox.