Download Presentation

Automated Software Test Generation : Model-Based Testing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

What are the problems of Software Testing?

What is wrong with manual testing?

Testers are oftenperceived as bottlenecks to the delivery of software products. They are being asked to test more and more code in less and less time.

What is Test Automation?

Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions.

Model-Based Testing

One way to generate test casesautomatically is “model-based testing” where a model of the system is used for test case generation.

Model-Based Testing

“Model-based testing is a testing technique where the runtime behavior of an implementation under test is checked against predictions made by a formal specification, or model.”- Colin Campbell, Microsoft Research

In other words:

A model describes how a system should behave in response to an action.

Supply the action and see if the system responds as you expect.

Models

Simply, a model of software is a depiction of its behavior. Behavior can be described in termsof the

input sequences accepted by the system,

the actions,

conditions,

output logic, or theflow of data through the application’s modules and routines.

By using modeling a shareable, reusable, precise description of the system is acquired.

Models in SW Testing

Finite State Machines

Statecharts

UML

Markov Chains

Grammars

Finite State Machines

Formally a finite state machine representing a software system is defined as a quintuple (I, S, T,F, L), where

􀂉 I is the set of inputs of the system.

􀂉 S is the set of all states of the system.

􀂉 T is a function that determines whether a transition occurs when an input is applied tothe system in a particular state.

􀂉 F is the set of final states the system can end up in when it terminates.

􀂉 L is the state into which the software is launched.

Example: Light Switch

Consider a simple light switch simulator:

The lights can be turned on and off usingone input.

The intensity of the light can be adjusted using two inputs for decreasing and increasingthe intensity.

There are three levels of light intensity: dim, normal, and bright.

If the lights arebright, increasing the intensity should not affect the intensity.

The simulator starts with the lights off. Finally, when the lights are

turned on, the intensity is normal by default, regardless of the intensityof the light when it waslast turned off.

<increase intensity> changes [dim] to [normal], or [normal] to [bright]

<decrease intensity> changes [bright] to [normal], or [normal] to [dim]

The inputs do not affect the state of the system under anycondition not described above

F = [off]

L = [off]

State Transition Matrix And State Transition Diagram

Example: Notepad

Fundamental tasks of MBT

Understanding The System: Forming a mental representation of the system’s

functionality is a prerequisite to building models.This is a nontrivial task as most systems todaytypically complex functionality. Moreover, software is deployedwithin gigantic operating systems among a clutter of other applications, dynamically linkedlibraries, and file systems all interacting with and/or affecting it in some manner.

Determine the components/features that need to be tested based on test objectives.

Start exploring target areas in the system.

Establish communication with requirements, design, and development teams if possible.

Identify the users of the system.

Enumerate the inputs and outputs of each user.

Study the domains of each input.

Document input applicability information.

Fundamental tasks of MBT(cont’d)

Document conditions under which responses occur.

Study the sequences of inputs that need to be modeled.

Understand internal data interactions and computation.

Maintain one living document.

Choosingthe Model:Not surprisingly, there are no software models today that fit all intents and purposes. Consequently, for each situation decisions need to be made as to what model (or collection of models) are most suitable.

HTML files for a web browser or mathematical expressions for a scientific calculator: Grammar

Phone systems etc.: Finite State Machine

Parallel Systems: Statecharts (each component is a FSM)

Systems that have intention of statistical analysis of failure data: Markov Chain

Fundamental tasks of MBT(cont’d)

Building the Model:

The general procedure that underlies many MBT methodologies:

Make a list of the inputs.

input applicability constraintsfor each input.Forexample, consider a simple telephone. The input “take the handset off the hook” can onlybe applied when the handset is hung up. (It is impossible to pick up the handset if thehandset has already been picked up.)

input behavior constraintsfor each input according to the current state context. For example, the input “take thehandset of the hook” can cause a number of behaviors. If the context is that the phone isidle, the resulting behavior will be that the system will generate a dial tone. If the contextis that the phone is ringing due to an incoming call, then the system will connect the twophones to allow conversation. Two different behaviors are caused by the same input.

Generating Tests

In the case offinite state machines, traverse thestate transition diagram. The sequences of arc labels along the generated paths are, by definition,tests. For example, in the state transition diagram below, the sequence of inputs “a, b, d, e, f, i, j,k” qualifies as a test of the represented system.

A state machine (left) and a test path in the state machine (right)

Running Tests

First, test scripts are written tosimulate the application of inputs.

Next, the test suite can be easily translated into an executable test script.

Collecting Results

Evaluating test results is perhaps the most difficult testing process. Testers must determinewhether the software generated the correct output given the sequence of test inputs applied to it.

In practice, this means verifying screen output, verifying the values of internally stored data andestablishing that time and space requirements were met.

Verification of state can be used. For example,the model itself will catalog each state change that occurs (or should occur) in the software as testinputs are being applied. Thus, the model acts as a very precise and detailed specification,informing the tester what inputs should be available and the values of each data abstraction thatcomprises the state of the software under test.

Collecting Results (cont’d)

Key Concerns

State Space Explosion: Number of states can be increased in a dramatical manner.

Solutions:

Abstraction: Abstraction for MBT involves the incorporation ofcomplex data into a simple structure. For example a dialog box that requires the entry of multiplefields of data followed by the OK button, could be modeled as two abstract inputs: “enter validdata” and “enter invalid data.”

Exclusion: Simply dropping information without bothering to abstract it. This is mostlydone when decomposing a set of software behaviors into multiple models. Each model willcontain certain inputs and exclude others. The inputs excluded by a modeler would have to betested in some other way, obviously.

Key Concerns (cont’d)

Meeting Coverage Criteria

For state machines, coverage is expressed in terms of inputs, states, and transitions. For example,there is the criterion of covering all states.

A model serves as a unifying point of reference that all teams and individuals involved in the development process can share, reuse, and benefit from. For example, confusion as to whether the system under test needs to satisfy a particular requirement can be resolved by examining the model.

Most popular models have a rich theoretical background that makes numerous tasks such as generating large suites of test cases easy to automate. For example, graph theory readily solves automated test generation for finite state machine models.

Difficulties and Drawbacks of MBT

Skills, time, and other resources need to be allocated for makingpreparations, overcoming common difficulties, and working around the major drawbacks.

This overhead needs to be weighed againstpotential rewards in order to determine whether a model-based technique is sensible to the task athand.

The most prominent problem for state models (and most other similar models) is state space explosion. Briefly, models of almost any non-trivial software functionality can grow beyond management even with tool support.