Where the rubber meets the road in an enterprise adopting Agile practices

About the Blog

Software is our business, and perfecting the art and science of delivering it is our mission. The contributors to this blog are passionate about the impact that great teams and good software can have on an organization’s bottom line. They bring decades of experience designing, developing and delivering great software, and each is playing a critical role in Borland’s own transformation.

July 2009

July 20, 2009

Greetings. In my last post - part two of a three-part series on Agile Testing - I talked about the importance of test automation in Agile (Agile Testing: what does it take? Pt. 2). Before finishing that thread, I would like to expand on this topic a bit more. You see, today is a milestone for me. We are releasing a significant update to my product line - a suite of test automation products - and I think it is a great moment to share with you what makes me so excited about the work I do. First of all, I consider myself fortunate to work with a great team. Collectively, we are responsible for defining the strategy of Borland's Lifecycle Quality Management (LQM) products. In the last couple of years the general direction of these products has been greatly influenced by Borland's own transition from a traditional, waterfall-based organization to one that lives and breathes Agile.

During our transition we not only realized that we needed to change many of the long- established processes and cultural norms, we also realized that we needed to enhance our products in order to better support Agile development scenarios in addition to continuing to improve our support for the core needs of all QA teams. Also, as an ISV we obviously have the same need for quality as any of our customers and, consequently, we not only use our own products to test our software but leverage our many years of experience in the testing domain to continuously improve our products.

As you may have seen, Borland today announced the latest versions of the Silk products which represent a significant step in supporting 21st century testing. The new releases not only provide true test automation for state-of-the-art applications, but they now offer some really exciting enhancements to better support Agile teams. Now I can already hear many of you moan about how I am trying to sell the Borland products. But that part of the post is done. The link above will let you take a closer look at the products if you are interested. My real intention for this post is to share the experiences that were the genesis for these products and through that sharing, hopefully, show you why am so excited about them.

There is no hiding in Agile

As we adopted more and more Agile practices in the Linz office, we noticed some issues surfacing and then becoming more and more pressing. Amongst these was our need for better support of Agile workflows in our Silk products. For example, we quickly learned that Agile teams work in a pull model as opposed to the traditional method of assigning work items. This was especially relevant from a test management perspective. As a consequence of our own teams' experiences, we realized we needed to provide much more collaborative views that allowed the team members to see what was going on so that they could select testing-related tasks from a list of available work items that had not yet been picked up by another team member, or have easy and quick access to related information. These requirements led to a number of enhancements in SilkCentral Test Manager such as Agile project templates, a new Web-based manual testing interface and an integration to Version One to support additional Agile test planning tools. These enhancements quickly proved to be an enormous help in our teams' daily work process by supporting self-organization, providing the necessary visibility into the many activities, and making collaboration much easier and efficient.

Another important need we quickly identified was the ability to leverage existing tests as well as the ability to easily integrate new tests, e.g. unit tests. It quickly became clear that it was a major hurdle for the Agile teams to have to maintain test plans both externally (for example as part of a unit testing framework) as well as in the test management tool. This gave us the idea for external test package support, which now allows users to just point to the external test plan and execute all the tests that are part of it. Gone are the days where we needed to manually create test stubs in the test management tool and add the necessary details in a subsequent manual step which, of course, is cumbersome when dealing with a large number of tests. Now it is as simple as pointing to a suite of existing unit or FitNesse test plans and executing them directly from SilkCentral Test Manager. And voila! All your results get automatically pulled in!

Finally, probably the most important requirement we uncovered, was the need to automate increasingly more tests. Anyone who has built test automation frameworks in the past knows that this is a very challenging task. The brittleness of test scripts combined with the potentially fast- changing GUI of an application are just two of the factors that have made the creation of a robust test automation framework extremely difficult. Our teams quickly realized they needed to do something about it.

Addressing the challenges

We immediately identified two closely related areas of improvement that we would have to address. First of all, from the perspective of a tester we looked for ways to improve the testability of our applications. Introducing unique identifiers for each of the objects in the products we develop was a significant step to help improve testability of the applications as well as the stability of test scripts. This also quickly lead to an improved understanding from the product development point of view about how we needed to enhance our test tools' capabilities to best leverage these kinds of testability enhancements of an application. In order to be able to build solid test scripts we needed to have stable and consistent object recognition. So the first obvious improvement we set out to make to the Silk suite was to ensure that the products provided a way to consistently find simple locators which were unique for each control - the same for each call, the same for each build, and the same for each instantiation of the application. Ideally, they would be readable and follow standard naming conventions well. With that in mind we enhanced all of our products so that they would intelligently optimize their object recognition strategies accordingly for testability.

Another major challenge our teams faced was around testing AJAX and other Web 2.0 applications. Due to their asynchronous nature they cannot be tested reliably with a traditional serialized test logic approach. It became obvious that synchronization needed to be done in the test tool rather than at the script level. Consequently, we enhanced SilkTest with everything that was needed to remove the necessity for the tester to try and mimic the (nondeterministic) nature of asynchronous applications allowing for much leaner test script code.

Finally, with all of our teams moving to two week iterations, speed became of paramount importance. The teams needed to be able to run their ever-increasing regression sets in much smaller time frames. There was an increased need to test more - and more often. We quickly realized that we needed to directly leverage RIA technologies to speed up replay of test scripts instead of leveraging an on-the-surface approach like using Javascript engine-injection for browsers. This approach, used by many other testing tools, just didn't work for our teams. The security restrictions, incredibly slow replay speed, and inability to handle actions that happen outside the browser's DOM (like Windows message boxes that come up) are just some of the problems we ran into with other tools. So what do you do if you are a test tool vendor? You build what you need!

These are just a couple of the challenges that our teams experienced that influenced the direction of the product line. One of the key advantages you have when you get immediate feedback through internal usage, and then have the opportunity to and refine and improve products on a sprint-by-sprint basis is that customers (and we are one of our customers) reap the benefits very quickly.

July 07, 2009

Recently I received an email commenting on David Anderson's "Where Everyone's a Pig" article. The author agreed to let me share part of their message along with my reply. They said, in part:

To a significant amount Scrum is intended to put a protective shell around a group of persons to ensure that the vulnerable process of self organization is not disrupted from outside forces. The metaphor of chickens and pigs is intended to underline that notion. I don’t see chickens and pigs as opponents at all.

To which I replied:

As with many agile practices, the pig/chicken distinction must be adapted to fit the organizational context and maturity of the team. In early stages of agile adoption this "protective shell", as you call it, is a defense mechanism intended to help the team counteract the larger organization's resistance to, or even hostility toward, agility. This is often based on fear and misunderstanding, but it can still be a real threat.

David's point is that a mature agile organization does not have that problem. He suggests that one way to remove the opposition to agile methods is to include the rest of the organization in your agile process. Viewing participants as pigs/chickens maintains an us/them attitude and therefore can undermine full organization-wide collaboration. Although it may be useful to provide some degree of safety to new agile teams, the need to "keep management at arm's length" is a symptom of a larger organizational dysfunction.

Ultimately, if anyone, inside the team or outside the team, is an impediment to timely delivery of value to the customer, we need to ask why they are doing that and what goal (presumably more important to them) is being served by their actions. Problem-solving based on whole-system thinking is not served by the pig/chicken distinction.

What do you think? Is there value in the pig/chicken metaphor? Does it encourage dysfunction?