Transcription

1 White Paper Requirements-Based Testing: Encourage Collaboration Through Traceability Executive Summary It is a well-documented fact that incomplete, poorly written or poorly communicated requirements are responsible for 50 to 70 percent of software project failures. Indeed, industry research shows that the root cause of 56 percent of all errors identified in projects are introduced in the requirements phase. 1 As a result, many organizations are looking to reduce their risk by implementing formal requirements management practices. The adoption of a formalized requirements management discipline can be leveraged by the quality assurance organization. By implementing a requirements-based testing methodology and ensuring complete requirements test coverage, companies can increase efficiency, reduce the risk of failure and improve overall software quality. Requirements-based testing is also popular with organizations following iterative or agile development styles, because it ensures complete coverage of changing requirements, in which a continuous develop-build-deploy cycle is in use. 1 What Is Requirements-Based Testing? Gary E. Mogyorodi, Bloodworth Integrated Technology, Inc., STSC Crosstalk, March 2003 Page 1 of 6 Requirements-Based Testing

2 In this white paper, you will learn how Integrity, a PTC product that is a unified platform for application lifecycle management, can help your QA team implement a requirementsbased testing process that enables staff to: Know what to test and ensure all project requirements have complete test planning and test execution coverage Establish traceability between all artifacts in the application lifecycle, such as requirements, specifications, test cases and code Manage the resolution of defects identified during testing Measure project readiness through reports and dashboards incorporating real-time metrics and trends Description of Requirements-Based Testing QA can confidently measure progress as the project evolves and ensure all requirements have been validated. When all requirements have been thoroughly tested, and the defects identified during testing have been resolved, the QA department can provide formal release sign-off. A requirements-based testing methodology helps identify problems early in the application lifecycle, which can substantially contribute to development cost savings. The cost of fixing a software error rises dramatically as a software project progresses. If an issue is caught during the requirements phase, the cost ratio to fix the error is one to one; at coding, it is seven to one; and at production, it ranges from 40 to one to 100 to one. 2 2 B. Boehm and V. Basili, Software Defect Reduction Top 10 List, IEEE Computer A requirements-based testing methodology begins with requirements management. Business analysts, business users, product managers and other business professionals compile and manage requirements and specification documents as an accurate and complete description of the needs of the system. The QA department relies on these requirements as a basis to plan and execute its testing efforts. By tracking test planning and execution of test cases directly against the requirements, Cost to find and fix a requirements defect Phase Average Cost of Repair Defect Requirements $139 1 Normalized Value Design $455 3 Coding $1,000 7 System Testing $7, Production $14, Page 2 of 6 Requirements-Based Testing

3 The Challenges with Current Testing Methods Manual Processes Many challenges exist when using manual methods to manage testing within the QA department, including: Inconsistent quality and problems passing audits, due to lack of enforcement for processes or standards across multiple projects Time wasted manually preparing reports and evaluating the status of testing Inability to ensure validation of requirements because no direct link to test cases exists Project disruption caused by changes to requirements when inadequate impact analysis is done Errors and omissions caused by incomplete test coverage against requirements Inaccurate testing as a result of requirement churn during the project lifecycle Multiple Point Products and Silos Over time, many organizations incrementally adopt point products to solve specific team-based needs. Implementing multiple products for requirements management, test management and defect tracking creates disconnected silos of information, resulting in the following collaboration challenges: There is no single source of reporting available to management that spans all phases of the application lifecycle It is difficult to implement business rules and enforce processes across domains Changes that take place in one domain are not accounted for in another Requirements-Based Testing with Integrity A unified platform for application lifecycle management, such as Integrity, can help QA organizations conform into lockstep with all other phases of the application lifecycle and also help them overcome the challenges presented by manual or multiple point system approaches. Change Management Managing change effectively throughout the application lifecycle is critical for success. Process controls within the Integrity platform ensure that only authorized individuals make changes at the appropriate time. Comprehensive impact analysis can be performed to assess the risk of accepting a change. When changes are accepted, related items that may be impacted are flagged for review, ensuring appropriate corrective measures are taken. Connecting the Requirements Phase to Downstream Testing Efforts Most organizations today define their requirements in business tools such as Microsoft Word, Excel or Project. These requirements documents can be easily imported into Integrity. Alternatively, users can take advantage of Integrity s document-centric interface to author requirements directly within the system. Workflow within Integrity then initiates an approval cycle for these requirements and, upon their approval, test authors can create test cases linked directly to the requirement, just as developers do for functional specifications. Through its direct, downstream connection with development and QA, Integrity enables proposed requirements changes to be analyzed for impact and ensures that all changes made to requirements are communicated downstream. A detailed overview of Integrity s requirements management capabilities is provided in the white paper, An Innovative Approach to Managing Software Requirements. Page 3 of 6 Requirements-Based Testing

4 Plan Testing with Requirements in Mind Now that the software application is well defined by a set of requirements, the QA organization can implement a focused testing effort that is based directly on the requirements, to ensure that the system is being verified against original intent and that it will deliver on the business need. This next step begins with test development. Test authors prepare one or more test cases for each requirement to ensure the requirement will be met in the final deliverable. Test developers enhance the test case to include details regarding the test steps a test engineer will perform, as well as the expected results and pass/fail criteria that will be used to evaluate the test. For automated testing, a link is established to the test script in the external automated test execution product. By linking requirements to test cases, a feedback loop between business analysts and QA is established. This feedback loop helps refine the requirements and, in general, improves the product. By asking questions to clarify precisely what needs to be tested, errors or gaps in the requirements may be uncovered. Conduct Automated and Manual Tests with Confidence Once test cases are prepared and logically grouped together, a QA manager will assign testing activities to test engineers,indicating that, they are ready to run. In the case of automated testing, Integrity interacts with external test execution products such as HP Quick Test Professional and ITKO Lisa through its Automated Test Execution Framework. Integrity connects with the external system, provides it with information needed to run the tests, and then captures the results. The test results are published in Integrity for review and interpretation. When running manual tests, test engineers use the Test Results Editor. They are provided with a focused view of the test steps needed to be run, the expected results and the pass/fail criteria. Detailed comments on the results can be logged. The test engineer will also review the results of automated testing in this Editor. Figure 1: Create a defect linked to a failed test case and its corresponding requirement. Page 4 of 6 Requirements-Based Testing

5 Resolve Defects and Lower Project Costs One of the key measurements of product readiness is the number and severity of open defects. As referenced earlier, the earlier defects are detected and corrected in the development lifecycle, the lower the cost. Defects are prioritized and assigned to a developer for corrective action. The system establishes traceability between the defect and all changes the developer makes to resolve them. This closed loop provides complete traceability from the requirements down to the lines of code changed to repair defects identified during testing. Charts/Reports/Dashboards QA managers, project managers and executives need accurate and up-to-date information to plan and schedule resources, take quick corrective action and generally manage schedules and budgets with confidence. Too often reporting is done manually, resulting in time delays. There s also the risk of subjectivity, given the human collection factor. Point tools each have their own data repository and therefore cannot provide an aggregate view of the application lifecycle. Integrity includes extensive graphing, charting and reporting capabilities. Role-specific dashboards display the real-time information team members need to do their jobs. Information can be displayed for a single project or filtered to include all the projects of interest. A single click provides drill down into the lifecycle details behind the chart or report. For example, if a QA manager sees three high-priority defects outstanding, she will be taken directly to those defects by clicking on that section of the chart, thereby allowing her to further investigate and take quick corrective action. Figure 2 & 3: Charts provide a real-time view into critical testing metrics. Page 5 of 6 Requirements-Based Testing

6 Summary By implementing automated requirements-based testing processes within Integrity, your QA organization can help ensure that software projects approved for release are on time, and on budget and meet the needs of the business by: Assuring requirements receive complete test planning and test execution coverage Reducing disruptions that arise due to requirement churn Accessing a single source of real-time information to measure project readiness Ensuring processes and standards are enforced across all projects Establishing traceability between all artifacts and visibility across the entire development lifecycle Integrity Business Unit Locations North America United Kingdom +44 (0) Germany +49 (0) Asia Pacific Japan For more information visit: /products/integrity 2012, Parametric Technology Corporation (PTC). All rights reserved. Information described herein is furnished for informational use only and is subject to change without notice. The only warranties for PTC products and services are set forth in the express warranty statements accompanying such products and services and nothing herein should be construed as constituting an additional warranty. References to customer successes are based upon a single user experience and such customer s testimonial. Analyst or other forward-looking statements about PTC products and services or the markets in which PTC participates are those of the analysts themselves and PTC makes no representations as to the basis or accuracy thereof. PTC, the PTC logo, and all PTC product names and logos are trademarks or registered trademarks of PTC and/or its subsidiaries in the United States and in other countries. All other product or company names are property of their respective owners. The timing of any product release, including any features or functionality, is subject to change at PTC s discretion Integrity: Requirements-Based Testing-WP-EN-0312 Page 6 of 6 Requirements-Based Testing

Improve Quality and Decrease Time to Market with Better Requirements Management Requirements Engineering: Right Requirements, Right Products Nearly 20% of development cost is due to rework because of ill-defined

Software: Driving Innovation for Engineered Products Software in products holds the key to innovations that improve quality, safety, and ease-of-use, as well as add new functions. Software simply makes

Software Development for Medical Devices Overcoming the Challenges of Compliance, Quality and Cost Software is fast becoming the differentiator for manufacturers of medical devices. The rewards of software

Managing Product Variants in a Software Product Line with PTC Integrity Software Product Line (SPL) engineering has become indispensable to many product engineering organizations. It enables those organizations

Governing the Control and Delivery of Change in IT An MKS White Paper By: Gary Guttridge Principal Change Manage IT Ltd. Page 1 of 19 1. INTRODUCTION When a business requirement passes along the application

RUN THE RIGHT RACE Keep pace with quickening release cycles. Discover automation with the human touch. CHOOSE A TEST TO RUN BELOW 26032015 FUNCTIONAL TESTING With Borland everyone from business analysts

SAP Brief SAP Extensions SAP Service Virtualization by HP Objectives Faster Development Through Virtualization Remove the obstacles that slow down application delivery Remove the obstacles that slow down

Service Virtualization: Reduce the time and cost to develop and test modern, composite applications Business white paper Table of contents Why you need service virtualization 3 The challenges of composite

Agile and the cloud: why automating application deployment matters Business white paper Executive summary Agile development methodologies and the cloud computing model have increased the pace of deployment

How do you manage the growing complexity of software development? Is your software development organization as responsive to your business needs as it could be? Borland Core SDP enables your IT organization

SAP Solution in Detail SAP Product Lifecycle Management SAP Portfolio and Project Management Plan Well and Manage Efficiently Within Budget and on Time Profitable product and service development in today

Business Intelligence Visualization Starter Pack from SAP Overview Enabling Self-Service Data Exploration and Visualization In today s environment, almost every corporation has to work with enormous data

Leveraging PTC s Integrity Platform for IEC 62304 Compliance SPK and Associates, LLC Our medical device customers report that up to 70% of new product innovation is being created in the software domain.

Master Data Governance Find Out How SAP Business Suite powered by SAP HANA Delivers Business Value in Real Time Disclaimer This document is not subject to your license agreement or any other service or

Data Sheet QADIRECTOR A framework for centralized test management. Managing QA testing for a large IT organization involves many challenges, but the most basic one is to collect the information you need

Overcoming the Challenges of Compliance, Quality and Cost An MKS White Paper Introduction Software is fast becoming the differentiator for manufacturers of medical devices. The rewards available from software

SalesManagement.org Improving Sales Pipeline Performance Through Enhanced Visibility Leveraging Analytics to Focus on the Right Opportunities OVERVIEW For companies with complex sales cycles in high technology,

SAP Brief SAP Travel OnDemand Objectives SAP Travel OnDemand An Easier Way to Travel Focus on your business while controlling travel costs Focus on your business while controlling travel costs With the

TeamCompanion Solution Overview Visual Studio Information in this document, including URL and other Internet Web site references, is subject to change without notice. Unless otherwise noted, the example

Business white paper Best practices in project and portfolio management Practical advice for achieving greater value and business benefits Table of contents 3 Introduction 3 The importance of best practices

SOLUTION BRIEF Service Assurance May 2010 assure the quality and availability of business services to your customers we can is a mature, integrated portfolio of management products for delivering exceptional

Data governance is a means to define the policies, standards, and data management services to be employed by the organization. Information Management & Data Governance OVERVIEW A thorough Data Governance

PRODUCT BRIEF: CA SERVICE DESK MANAGER CA Service Desk Manager CA SERVICE DESK MANAGER IS A VERSATILE, COMPREHENSIVE IT SUPPORT SOLUTION THAT HELPS YOU BUILD SUPERIOR INCIDENT AND PROBLEM MANAGEMENT PROCESSES

SAP Brief SAP s for Small Businesses and Midsize Companies SAP Business One Objectives Business Management Made Simpler Successfully manage and grow your small business Successfully manage and grow your

SAP Hybris Solution Overview: SAP Hybris Cloud for Customer ACHIEVE DIGITAL TRANSFORMATION WITH SALES AND SERVICE SOLUTIONS The Digital customer requires companies to be one step ahead of them, be better

Continuous IT Compliance: A Stepwise Approach to Effective Assurance BEST PRACTICES WHITE PAPER Introduction Regardless of industry, most IT organizations today must comply with a variety of government,

Development Testing for Agile Environments November 2011 The Pressure Is On More than ever before, companies are being asked to do things faster. They need to get products to market faster to remain competitive

R49 Using SAP Payment Engine for payment transactions Process Diagram Purpose, Benefits, and Key Process Steps Purpose The purpose of this scenario is to show you how to check the result of payment orders

Solution in Detail R&D Engineering Executive Summary Contact Us Commercial Deliver More for Less More for Less Efficiency Companies that sell projects often need to professionalize core business processes

WHITEPAPER Best Practices in Release and Deployment Management Mark Levy Through 2016, a lack of effective release management will contribute up to 80% of production incidents in large organizations with

HP and netforensics Security Information Management solutions Business blueprint Executive Summary Every day there are new destructive cyber-threats and vulnerabilities that may limit your organization

How Silk Central brings flexibility to agile development The name agile development is perhaps slightly misleading as it is by its very nature, a carefully structured environment of rigorous procedures.

FireScope + ServiceNow: CMDB Integration Use Cases While virtualization, cloud technologies and automation have slashed the time it takes to plan and implement new IT services, enterprises are still struggling