After first describing how the test-last aspect of record-and-playback tools has little chance of success on any project, agile or not, Hendrickson explains why this is particularly detrimental to an agile project. On an agile project, test-last workflow has at least the following problems:

Waste: the same information is duplicated in both the manual and automated regression tests. Actually, it’s duplicated elsewhere too. But for now, let’s just focus on the duplication in the manual and automated tests.

Feedback Latency: the bulk of the testing in this workflow is manual, and that means it takes days or weeks to discover the effect of a given change. If we’re working in 4 week sprints, waiting 3 - 4 weeks for regression test results just does not work.

Hendrickson explains how the test scripts fundamental to these record-and-playback tools, which inevitably contain a spaghetti mix of business-level expectations and implementation-specific details about the UI code, turn an agile project's responsiveness to change into a maintenance nightmare. Succinctly stated:

Agile teams need tools that separate the essence of the test from the implementation details. Such a separation is a hallmark of good design and increases maintainability.

Thirdly, due largely to their high costs of and propriety-coding requirements, typical record-and-playback tools lead most organizations to the creation of a dedicated group of "Test Automation Specialists", charged as the keepers of the automated tests. Hendrickson asserts how this works against the collaboration required for effective agility:

Agile teams increase their effectiveness and efficiency by breaking down silos, not by creating test automation superheroes. That means the test automation effort becomes a collaboration. Business stakeholders, analysts, and black box testers contribute tests expressed in an automatable form (e.g. a Fit table) while the programmers write the code to hook the tests up to the implementation.

Hendrickson finishes off with a great discussion about what Agile teams do need from their test automation tools:

Agile teams need test automation tools/frameworks that:

Support starting the test automation effort immediately, using a test-first approach.

Separate the essence of the test from the implementation details.

Support and encourage good programming practices for the code portion of the test automation.

Great question (I personally was hoping for it). My opinion: not exactly.

One of Elisabeth's primary points is that traditional commercial tools (of which Selenium certainly is not!) use a proprietary language/syntax for the scripts - not Selenium.

Another point is that the traditionals are commercial - ie, cost a lot of cash - which encourages organizations to create "test automation specialists" in order to save on license fees - again, not Selenium.

The question then, is Selenium part of the "Next Generation", or is it not ;-)

I spoke at the most recent Selenium user meeting and asked the delegates if they were "Developers that test" or "Testers and code". When I asked this 3 years ago at the STAR conference I found that very few identified with either of these. At the Selenium meeting the majority of delegates identified with both groups. This is an important trend.

I don't see the test automation specialization that Elisebeth writes about at PushToTest customers. Instead I see enterprise development organizations wanting to repurpose the unit test assets their developers write into function tests, load and performance tests, and service monitors. This is happening at AMD, Ford, The Jackson Labratory and others enterprises.

InfoQ Weekly Newsletter

Join a community of over 250 K senior developers by signing up for our newsletter. If you are based in the EEA, please contact us so we can provide you with the protections afforded to you under EEA protection laws.

Is your profile up-to-date? Please take a moment to review and update.

Email Address

Note: If updating/changing your email, a validation request will be sent

Company name:

Keep current company name

Update Company name to:

Company role:

Keep current company role

Update company role to:

Company size:

Keep current company Size

Update company size to:

Country/Zone:

Keep current country/zone

Update country/zone to:

State/Province/Region:

Keep current state/province/region

Update state/province/region to:

Subscribe to our newsletter?

Subscribe to our architect newsletter?

Subscribe to our industry email notices?

By subscribing to this email, we may send you content based on your previous topic interests. See our privacy notice for details.

You will be sent an email to validate the new email address. This pop-up will close itself in a few moments.

We notice you're using an ad blocker

We understand why you use ad blockers. However to keep InfoQ free we need your support. InfoQ will not provide your data to third parties without individual opt-in consent. We only work with advertisers relevant to our readers. Please consider whitelisting us.