The Quest Continues: Introducing Capybara Webkit

We recently
blogged
about our experiences and ongoing efforts to discover and improve tools for
Javascript testing. In closing, we mentioned that we were fairly happy with
Akephalos, but we didn’t believe
the quest was over. Today, the quest continues.

Akephalos

Akephalos is a well-written driver that integrates soundly with the htmlunit
virtual browser. However, as well-integrated as Akephalos is into both Capybara
and htmlunit, it can only ever be as strong as htmlunit. As our applications
became more sophisticated and large, we began to run into issues with htmlunit.

Bugs: as previously mentioned, there are bugs in htmlunit, specifically with
jQuery’s live. Although all browser implementations have bugs, it’s more
useful if tests experience the same bugs as actual browsers.

Performance: when most of your tests use Javascript, test suites with htmlunit
start to crawl. It takes a while to start up a test with Akephalos, and a
large test suite can easily take 10 or 15 minutes.

These issues made us realize that virtual browsers like envjs and htmlunit are
never going to keep pace with actual browsers. If we want to write applications
using features from the latest and greatest browsers, we’re going to need to
test with them, too.

Selenium

That brought us back to the old mirage: Selenium. Selenium is well-supported,
well-maintained, and works with real browsers. Integration with Ruby tests has
improved considerably since the old Webrat days, and speed is better in some
cases, as real browsers have been heavily optimized.

However, Selenium still has some ongoing issues that aren’t easy to solve:

Timing: because Selenium uses a real browser, it doesn’t have as much control
as a virtual browser does. This leads to issues with timing, causing erratic
test failures.

Overhead: real browsers come with features we don’t need, like a UI and
plugins. This slows things down a bit.

Gotchas: real browsers come with strange edge cases. Using Selenium and
Firefox, sometimes the tests crash because Firefox tries to update the
browser. Firefox doesn’t allow more than one instance to run at once. These
gotchas are inevitable, because real browsers weren’t designed to be driven
programatically.

We were forced to use Selenium because we needed all the capabilities of a real
browser, but all these issues kept us looking for something better.

capybara-webkit

What we need was a real rendering engine coupled with a full Javascript and DOM
implementation, but without all the cruft of a GUI browser. What we really
wanted was a headless implementation of WebKit’s rendering engine that could be
driven by Capybara tests. After Tristan “Websockets” Dunn showed me
PhantomJs, I realized this might be possible using
Qt’s WebKit implementation.

It was possible. I’d like to introduce a headless WebKit driver for capybara: capybara-webkit.

capybara-webkit has the following benefits:

Compatibility: uses a real rendering engine (WebKit)

Simplicity: runs headlessly, and designed to be controlled programatically

The holy grail

We designed capybara-webkit to solve the problems we had with other solutions,
and so far it fits the bill nicely. We believe the quest is far from over, but
we invite you to try out capybara-webkit on your own and see if this latest tool
can take you a little further on your quest for the perfect testing solution.

Please keep in mind that this is a new driver and isn’t battle-tested as well as
Akephalos or Selenium. If you discover any bugs, please report them in Github
Issues. We’d also be
happy to accept tested patches.

Next Steps & Related Reading

Detect emerging problems in your codebase with. We’ll deliver solutions for
fixing them, and demonstrate techniques for building a Ruby on Rails application
that will be fun to work on for years to come.