Tuesday, January 19, 2010

Cucumber, Webrat and XML

I've written before about using cucumber and soap4r to test SOAP web services. That still works, but it's annoying when you're trying to test a SOAP web service you're writing. In order to test a web service you're writing with soap4r, you have to pretend its remote and run it outside your standard test environment. So in order to run tests that way, I had to:

start my server

put the right data into the database my server is pointing at

reconcile the data in the server database with my test data

run the tests

re-reconcile the data in the server database to clean it up

I wound up spending a lot of time on maintenance and tracking down issues that really didn't have anything to do with the software. Plus, the setup requiremented certainly limited the incentive to run the tests. A typical run would look like this:

run tests. (FAIL)

Whoops. Forgot. Start the server.

run tests. (FAIL)

whoops. I didn't clean out from the other manual testing I was doing on that server so my database has unexpected data in it. Clean out database.

run tests (actually find something useful or interesting)

etc...

Step back a moment, and let's look at what we're testing. We're exercising the code, yes. We're also testing our system configuration (Is the web server properly configured for ssl? Does it run on the expected ports?). This is useful in some scenarios, but when it's just my development server, testing the system configuration seems less than ideal. I'm really more interested in testing the code at this point.

So why are we doing this? Long story short, I blame one key limitation in soap4r, and that is that soap4r expects to make API calls to a full domain name. You have to call "http://localhost/webservice/api/MyCall", and you cannot call "webservice/api/MyCall". This makes it really hard to run in the standard cucumber test environment. It's fine as long as you're consuming a truly remote web service (e.g., a Google API or something), but it's rough when you're trying to test your own web service.

But there is another way. You can simply skip the problem component (soap4r in this case). Instead, I simply use cucumber and webrat's existing infrastructure and feed it my self-formed SOAP requests. For example, this is how I send a call to our Activation web service:

When /^"([^\"]*)" activates a widget with description "([^\"]*)" with password "([^\"]*)"$/ do |email, description, password|