Bumping the Lamp During Testing

Most testers have heard of “bump the lamp”. It’s hard to go through a testing career and not be exposed to this concept at some point. I recently found an experience where I was able to keep this kind of thinking front and center.

You might want to read the linked article if you have never come across the concept before just so you have a focus for the context.

The in-a-nutshell version involves the film Who Framed Roger Rabbit (back from 1988!). In one scene, Roger (an animated character) and Eddie Valiant (an actual actor) walk into a dark room. Eddie (played by Bob Hoskins) bumps his head on a lamp which causes shadows to continuously move throughout the remainder of the scene. Well, Roger, being animated of course, cast no shadows. But it wasn’t necessarily all that easy to tell anyway given how the scene was filmed. Nevertheless, the director and animators decided this wasn’t good enough. They spent a great deal of time and effort to make sure that the shadows caused by the bumping lamp actually matched up between the real actor in the scene, the room itself, and the animated character. This was, keep in mind, before the CGI technology that exists today which would make this trivial at best.

What you should get from the story itself is that it involves going a bit beyond and providing an experience. Sometimes that experience is not necessarily something a user (or viewer) would even think to point to as an aspect of quality, at least until it was specifically pointed out to them. Yet, at some level, they did know it was there. And it did matter. And, quite frankly, even if no one in your audience recognized it at all, the fact that you thought of it does matter. It’s a level of quality: a level of getting things right. It’s a matter of caring for the details simply because you want the details to be the best they can be. Put simply, it’s taking pride in your work.

As the composer Claude Debussy once said, “Music is the space between the notes.” I believe quality is often the space between the functionality and the behavior it provides.

My Own Recent Bump the Lamp

Now I’ll tell you a little about some of “bump the lamp” moments in an insanely fun part of my own career.

As I mentioned before, I do off-work game testing for various companies. (See one particular interesting bug that was found with Star Wars: The Old Republic.) I recently came off a stint of testing the game Grand Theft Auto 5 for the PC which I think is one of the best games to be developed in quite some time, particularly when “best” is measured relative to the money spent on it versus the amount of game you actually get. It’s also one of the most visually stunning.

There was a LOT of testing involved in this game but what was most interesting to me were the things that were put in place as a result of suggestions due to testing. Our goal was to help this be the definitive game in the Grand Theft Auto series, not simply the latest installment. To that end consider a few things that you’ll find if you play the game:

When you have your character go into water, only the parts of the clothing that were submerged appear darker, as if they were wet.

If you have your character sprint a lot during the day when the sun is out, they develop sweat stains on their shirt.

When you go near a garage or house at night, the security lights will click on if the house has them.

If you stop your car in the road and block traffic, drivers behind you will give you the finger and make some choice comments.

If you don’t move your car after a street light turns green, drivers will begin honking at you. Eventually they will drive around you angrily, often shouting something unflattering to you. In the more violent neighborhoods, they may hit your car or even shoot at you.

When you open car doors, the dome light of the car turns on.

When it starts to rain in the game, the NPC pedestrians will cover their heads and run for cover under buildings, overpasses, awnings, whatever.

You have GPS in your cars. You will, however, lose your GPS signal when entering a tunnel or some underground areas.

If you drive into the mountain areas, the radio stations from the city will often sound garbled and may cut out entirely.

If you follow some female NPCs around the city, particularly in the ritzier areas, they will sometimes get nervous, look over their shoulder, and start walking faster.

When you hit the key to look behind your car while driving, your character actually looks in the rearview mirror. Likewise if you change radio stations, your character is seen visibly changing the stations.

You can play your own custom MP3 music library in game, rather than using the game’s music, but you can also have your own music interspersed with DJ chatter, news, and commercials to maintain immersion.

If you crash a car into a house at night, the interior lights will come on in various rooms, making it look like you woke up the residents.

If you try to pick up a prostitute when a cop car is nearby, they won’t approach you. They instead try to look inconspicuous. Along with this, prostitutes won’t trust you if you pull up in a car that is heavily damaged or bullet riddled.

If you drive at night with your headlights off, oncoming cars will flash their own lights, to alert you. They will also flash at you and honk their horns if you drive in their lane against the flow of traffic.

Dialogue becomes softer if a character is in a car and the windows are rolled up or not broken. Likewise, if the radio is on in a car and you are standing outside the car, the radio will sound louder when the door is open but more muted if the door is closed.

You can rob stores, but if you do it often enough to the same store, and are recognizable (no mask), the clerks get wise to you and the cops may be waiting for you the next time you try it.

There are tons more examples. For example, you can take selfies of your own character, using the in-game cell phone.

Mind you, none of these things were stated as direct requirements. The mandate for testing was to explore the world. But during our exploring we had to think about the experience of a user (i.e., game player). What did we expect to happen, given the rules of the game engine? What did we expect not to happen? Often we stumbled across opportunities for improvement simply by exploring the application in context.

Now, granted, someone could argue that an open world, sandbox style story-driven computer game is not the same as most of the applications they test. There’s also a thought that says “The very worst waste of time is to do very well what need not be done at all.” That’s true. But it leaves a wide open range of what “need be done.”

The best way to delight other people — particularly those using your products — is to exceed their expectations. The best way I know of to exceed expectations is to set them in a way that you know you can exceed them in the first place. In the above case, testing was arguing for how to make the world more realistic and more surprising and inviting and fun for the user. In each case, one of the motivators was looking at what our own reactions were. As testers, we tuned into our emotions. If we got frustrated that something didn’t seem to make sense, we noted that. If we got disappointed because we thought an opportunity to do something cool was missed, we noted that too. (Emotions in software testing are important.)

I realize this post was purely anecdotal for everyone but I do firmly believe that testers need to look for ways to help developers and the business “bump the lamp.” Bumping the lamp is about an attention to detail, a concern for a high level of quality, and a desire to excite and delight your users who realize you really did work to provide them with the best experience you could.

About Jeff Nyman

Anything I put here is an approximation of the truth. You're getting a particular view of myself ... and it's the view I'm choosing to present to you. If you've never met me before in person, please realize I'm not the same in person as I am in writing. That's because I can only put part of myself down into words.
If you have met me before in person then I'd ask you to consider that the view you've formed that way and the view you come to by reading what I say here may, in fact, both be true. I'd advise that you not automatically discard either viewpoint when they conflict or accept either as truth when they agree.