I'll be giving a 3-day course on software test design later this month with lots of hands-on exercises. I'd like to include one or more exercises that would involve testing an online game for approximately 20 minutes. I am interested in your suggestions of an online game I could use for this exercise. Ideally, it will be a very easy-to-understand game that is generally operational (e.g., not totally broken) but has a fair number of relatively easy-to-find bugs. It will be important that we not waste much time in the course getting access to the game, explaining how it works, etc. we want to dive right in and make this a quick exercise. Additional context:

Logging into a free online gaming site (like addictinggames.com or similar)

Opening a game I would specify in order to test it

Exploring the game for just a couple minutes to understand the basic objectives of the game,

Designing combinatorial (similar to pairwise) tests

Executing those tests and learning more about the game in doing so

Sharing issues discovered with the class in a discussion that will highlight most of these conclusions:

Lesson 1: Using combinatorial test design techniques is great for maximizing variation between different tests but it is insufficient.

Lesson 2: You can parameterize just about any system or application or process if you think about it in the right way. Being able to parameterize Systems Under Tests (in obvious and not so obvious ways) is an extremely important skill to have as a tester.

Lesson 3: More important for efficient and effective testing is whether or not you include good testing ideas into the mix of variables that get modified from test to test.

Lesson 4: As emphasized by Exploratory Testing proponents, you should keep your eyes open to interesting lessons learned as you are executing your tests and not hesitate to enrich your test scripts by exploring a bit while you're executing them. If you do, you will probably uncover interesting findings. You should consider whether it would make sense to incorporate these new test ideas and other findings in additional tests.

Lesson 5: Requirements will always be incomplete; testing is a lot more than "verifying requirements."

Does it need to be a game? Or can it be a website?
–
Steve MiskiewiczDec 5 '11 at 18:05

Dan, I would prefer a game if possible to have some fun, get non-game testers out of their shells and out of their comfort zones a little bit. Part of the challenge I have in mind will be to "parameterize" user playing styles and possible user actions which is not a traditional test design technique most of them will be familiar with. Thanks.
–
JustinDec 7 '11 at 7:35

6 Answers
6

your idea is great but has a big flaw : What if you prepare your training and the makers of the game update it the day before, fixing the bugs ?

There is a solution : You should find an open source game which you could download, and then run a controlled version hosted by yourself. You could even add bugs yourself and provide an "update" in mid-course to check for improvments.

i am an engineer who has just started testing as career. i used to play the game farmvill a lot and often found bugs with it.. it would be interesting if i were to try out the techniques u mentioned and document the out come. so u can try out farmvile.. it has bugs

I have a question for you...what if your "testers" find no bugs in the software under test despite all the planning techniques. How will you explain this to your students? And how will you go about resolving this?

Are you planning on testing the software (using the same planning techniques you want to teach) and find some bugs so you can compare?

devonps, I or one of my colleagues are in the classroom together with participants when we run these test design training sessions. If the testers struggle, we would encourage them to try out different approaches and ideas. Rarely do we need to do more than that. If we need to, we can gently guide participants in a direction that is likely to be fruitful.
–
JustinJan 21 at 21:41

I don't know about online games, but if you have access to an Ubuntu machine, you could browse their software centre. There are a whole bunch of applications of all kinds, including several games. These range from relatively simple 2D scrolling games to OpenArena, a 3D FPS. They're mostly free, and available for download. I'm sure that given the range of free open-source games there, you can find some that are buggy enough for your tastes. You can also try contacting the creator(s) of the games and ask for newer versions to test as well, as some may be particularly receptive to having some playtesting and otherwise for their applications.

As a bonus a great buggy real life app is hosted by Adam Goucher. The Parking Calculalor is probably one of the buggiest apps I've ever seen and its the first thing I thought of when you posted your question. It's actually found on an airport website but has been hosted by Adam so you don't have to hit their prod site.

1) you can shoot yourself off the board
2) you can shoot through the ground (and hit the target)
3) you can shoot your character through the small pieces of ground
4) you can shot your character infinitely up if you go far enough out of bounds

also for game testing.. what you can do is download the beta version or the trial version.
these have a lot of bugs. but game QA is not only about playing the game. its playing the game in every possible ways to cover most part of the game world. because we dont know where there can be a coalition bug or the game goes out of world. so for a game tester it is important to play the game in every possible way.