(1) Introduction

Capgemini and Sogeti, in collaboration with HPE, have recently released the eighth edition of the World Quality Report (WQR). This year’s report reveals an increase in the adoption of agile testing methodologies and DevOps to help businesses roll out new products and services at greater speed.

Study findings pertaining to DevOps show an increased maturity with a better appreciation of the true purpose and application of DevOps. 42% of the respondents reported an uptick in continuous monitoring with predictive analysis along with an increased use of more cloud-based environments.

Co-location and the lack of required skill sets are the two biggest risks associated with agile and DevOps adoption. However, organizations are able to find solutions to these challenges in order to increase the implementation of agile methodologies. Nevertheless, 44% of businesses are reluctant to include testing teams in the initial planning phase, as they believe it will inhibit release speed, inadvertently putting the security of the business at risk.

Success stories in distributed agile methodology are enabling faster proliferation of agile principles. As per the WQR, 2016, more than 42% of the organizations that adopt the agile delivery model today use distributed agile methodologies. Distributed agile allows the organization to achieve higher cost savings associated with the global delivery model. From a skill set perspective, the supply and demand gap is also narrowing and more than 80% of the project managers believe that they have right skill sets needed for agile and DevOps delivery.

(2) Increased Use of Agile/DevOps Principles

As per WQR 2016, over 40% of participants cite that more than 50% of the projects are using Agile/DevOps principles. The percentage of those not using Agile/DevOps has dropped from 18% in 2015 to 12% in 2016.

(3) Increased Complexity of Testing in the Agile Landscape

The report finds that DevOps implementation challenges go far beyond issues with QA, and businesses risk isolating the benefits of DevOps if they are unable to break down business silos. DevOps and Agile delivery methodologies are gaining wide acceptance at a rapid pace. More and more organizations are adopting the new ways of delivering IT projects based on agile principles.

With the advent of mobile, digital, and cloud-based technologies, the IT landscape is changing fast. There is a continuous demand from the business to deliver new and improved IT solutions to end-users. Time to market is shrinking and application complexity is increasing. At the same time, the quality expectations are rising. For applications, especially those based on clouds, a higher level of testing is needed to ensure performance and security requirements are met. The testing complexity is enhanced further due to multiple access channels and various mobile and handheld devices that need to be supported by modern-day digital applications.

Identifying the right focus areas for testing and involving testing teams early during planning and inception stages are considered the top two challenges for agile implementation. This creates difficulty with allocating the appropriate level of test types and test depth to meet project requirements.

(4) Challenges Cited in Setting Up Agile Test COEs

While there has been increased use of Agile/DevOps lifecycles, there have been increased challenges cited as well. The top three challenges in setting up Agile Test COE as cited by participants include:

Necessity to co-locate the team which denies the cost savings

Independent test vendors’ unwillingness to pair up testers with developers

Agile is being used as excuse to not have documentation.

(5) How is QA Transforming to Meet the Demands of Agile / DevOps Lifecycles

It is important to note that while Agile and DevOps practices strive for faster software delivery, quality, stability, security of applications is always a first priority in any business. Any failure of IT applications has a direct impact on corporate image. The aim of QA is to develop the internal capability to deliver software faster to support the needs of the business; however the software must also satisfy the needs of the end user. This is achieved by developing an environment where an application can be tested early, optimally, and quickly. It is also equally important to be able to monitor quality continuously in production and test environments.

In order to maximize the effectiveness of the QA function, organizations should ensure that the basic principles of Agile (i.e., communicate, collaborate, and integrate) are practiced in a continuous and consistent manner. Additionally, QA teams should become an integral part of the technical team. This helps QA to become the facilitator of quality.

Organizations also need to define the desired level of quality. Instead of reaching for perfection in software, the focus should instead be on a “satisfying user experience within the given resources and the given time frame.” Testers should choose metrics that bring to light software defects early in the lifecycle. They should also adopt lean and agile principles to develop software in small increments. The focus should be on automated regression testing for high-risk areas using risk-based testing. Automation should be leveraged to the maximum extent.

Behavior Driven Development (BDD) is fast gaining popularity as it is a customer-centric testing methodology that implements an acceptance criteria-based test strategy. Zero touch test automation is the ultimate goal of QA in order to meet time-to-market objectives while maintaining test coverage.

(6) What is the Difference between Testing in DevOps and Testing in Traditional Lifecycle?

While the traditional testing placed a very high emphasis on process standardization and optimization, DevOps QA focuses more on agility and time-to-market. Skill sets needed for QA professionals are also undergoing considerable change. Today’s testers adopt continuous testing techniques, leverage automation, and deploy emerging technologies such as cloud-based environments and service virtualization.

The mindset of a QA in DevOps is very different from that of a QA organization ten years ago. In the past, the primary job of a tester was to find bugs. Today, QA groups are charged with the responsibility of preventing defects from reaching the end user. QA takes on a critical role in DevOps because it has the visibility and the directive to push code out when it is working, and roll it back when it is not.

The skillset required for a QA engineer requires a fine mix of technical and business skills. Testers are quality advocates, influencing both development and operational processes. They don’t just find bugs. They look for any opportunity to improve repeatability and predictability.

QA owns continuous improvement and quality tracking across the entire development lifecycle. Testers should have a thorough knowledge on application architecture and design. They should be able to participate in design and architectural discussions and check for product scalability. Testers should create high quality and maintainable automation framework. They should also be capable of building, deploying, managing, and customizing own test environments and test data

(7) Conclusion

Agile development has now attained critical mass and is being widely adopted. The initial hesitation does not prevail anymore. The QA function is a key enabler in this journey. The use of predictive analytics to determine and optimize test coverage alongside usage BDD techniques provides the perfect combination of shift left and shift right techniques. The co-existence of traditional IT along with agile delivery principles is giving rise to a new methodology based on bi-modal development.