Robots Are Finding the Intersection of Cool and Useful

Image: jeffedoe/Flickr

My fascination with videogame-playing robots started innocently enough. Two years ago, London-based software developer Dave Hunt wrote on his blog, “Wouldn’t it be cool to automate playing Pac-Man using [the testing framework known as] Selenium?” A few days prior, Google had released a fully playable in-browser Google Doodle version of Pac-Man to commemorate Pac-Man’s 30th anniversary.

As if the Doodle wasn’t cool enough, Dave thought it would be even more cool to write a program to play the game automatically, as a virtual robot, mimicking a real user playing the game. If you’re a software developer like Dave, the Pac-Man Doodle is a fun challenge. If you can write a program to play a videogame, it should be easy for you to write one to fill out and control serious web and mobile applications.

Developers create test scripts with Selenium to mimic a real user and to verify their web applications work correctly in web browsers. If a Selenium test is successful, the developer knows that everything is working and that any changes made recently to the application haven’t broken existing functionality.

However, testing tools will need to adapt as a larger portion of application development shifts to development for mobile phones and tablets. I originally designed Selenium to simulate mouse clicks and keystrokes, not tapping, pinching, zooming on touch screens, or interacting with GPS locators, accelerometers, and multiple cameras.

And with that all in mind, I decided to create a real testing robot — with a synthetic finger (technically an iPad stylus), controlled by motors and electronics that could interact with a mobile device. And while I was at it, I thought, “Wouldn’t it be cool if I could automate playing Angry Birds?”

I presented my “Angry Birds Playing Robot” at several software conferences last year and have been working on it ever since. The mechanical design is getting sturdier and faster – I upgraded to using a delta robot design, considered the fastest type of robot arm design available. Other developers have contributed code to help improve the brains. And even though I created it originally as a lark, after many conversations with friends in the open source software community, I realized that this robot has a very practical use in the future of mobile testing.

The notion that software testing is like controlling a robot is not new. Many commercial testing tools, like IBM’s Rational Robot, make the connection directly. And yet, when the word “robot” is mentioned, most people imagine futuristic robots like R2-D2 or the Terminator. Or they think of real robots in homes vacuuming floors or in factories welding cars. People rarely imagine a simple robot helping them at work. Although the idea of an “office robot” is not something many movies have been written about, it’s plausible that the first real robot that software developers and testers use at work will be sitting next to them, hovering over phones and tablets, tapping screens, and testing their apps.

Testing in the future will get harder to replicate in the lab or in a software simulator. My favorite example is Zipcar’s iPhone app, which can open your car for you. To truly test it, you need an iPhone and a car — not something easily virtualized in the cloud. You can also use your phone to buy coffee at Starbucks or control your TV. I believe that in the future, the phone will be the remote control for everything. But with the “appification” of the world, comes a host of new problems. Digital systems are more complex and brittle than their analog predecessors. With complexity comes the higher risk of bugs. The way to hedge that risk is to do more testing – lots of testing – faster and more “real-world.” And that is where the robots come in.

Today, developers use a combination of automated and manual techniques to test their apps. Automated testing in software emulators and simulators is great for covering many different test cases on multiple device configurations in the shortest period of time. Manual testing (with a dedicated QA team, outsourced test vendor, or the goodwill of friends and family) is the truest way to make sure the app works on a real device in the exact same way users will use it. A robot tester is the best of both worlds – it’s a real-world test on a real device – and because it’s a robot, it also performs the task precisely and quickly.

There’s something about robots that stirs the imagination in all of us. However, a cool robot that plays games only gets you so far. It’s better to be a useful robot that gets real work done and frees people up to work on other tasks. An automated testing robot lives at the intersection at cool and useful – and it’s a fun place to be.