4.
Introduction to Performance Testing <ul><li>Performance testing is the process of determining the speed or effectiveness of a computer, network, software program or device. </li></ul><ul><li>Before going into the details, we should understand the factors that governs Performance testing: </li></ul><ul><ul><li>Throughput </li></ul></ul><ul><ul><li>Response Time </li></ul></ul><ul><ul><li>Tuning </li></ul></ul><ul><ul><li>Benchmarking </li></ul></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

5.
Throughput <ul><li>Capability of a product to handle multiple transactions in a give period. </li></ul><ul><li>Throughput represents the number of requests/business transactions processed by the product in a specified time duration. </li></ul><ul><li>As the number of concurrent users increase, the throughput increases almost linearly with the number of requests. As there is very little congestion within the Application Server system queues. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

6.
Throughput <ul><li>In the heavy load zone or Section B, as the concurrent client load increases, throughput remains relatively constant. </li></ul><ul><li>In Section C (the buckle zone) one or more of the system components have become exhausted and throughput starts to degrade. For example, the system might enter the buckle zone when the network connections at the Web server exhaust the limits of the network adapter or if the requests exceed operating system limits for file handles. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

7.
Response Time <ul><li>It is equally important to find out how much time each of the transactions took to complete. </li></ul><ul><li>Response time is defined as the delay between the point of request and the first response from the product. </li></ul><ul><li>The response time increases proportionally to the user load. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

8.
Tuning <ul><li>Tuning is the procedure by which product performance is enhanced by setting different values to the parameters of the product, operating system and other components. </li></ul><ul><li>Tuning improves the product performance without having to touch the source code of the product. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

9.
Benchmarking <ul><li>A very well-improved performance of a product makes no business sense if that performance does not match up to the competitive products. </li></ul><ul><li>A careful analysis is needed to chalk out the list of transactions to be compared across products so that an apple-apple comparison becomes possible. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

10.
Performance Testing- Definition <ul><li>The testing to evaluate the response time (speed), throughput and utilization of system to execute its required functions in comparison with different versions of the same product or a different competitive product is called Performance Testing. </li></ul><ul><li>Performance testing is done to derive benchmark numbers for the system. </li></ul><ul><li>Heavy load is not applied to the system </li></ul><ul><li>Tuning is performed until the system under test achieves the expected levels of performance. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

11.
Difference between Performance, Load and Stress Testing <ul><li>Load Testing </li></ul><ul><li>Process of exercising the system under test by feeding it the largest tasks it can operate with. </li></ul><ul><li>Constantly increasing the load on the system via automated tools to simulate real time scenario with virtual users. </li></ul><ul><li>Examples: </li></ul><ul><li>Testing a word processor by editing a very large document. </li></ul><ul><li>For Web Application load is defined in terms of concurrent users or HTTP connections. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

12.
Difference between Performance, Load and Stress Testing <ul><li>Stress Testing </li></ul><ul><li>Trying to break the system under test by overwhelming its resources or by taking resources away from it. </li></ul><ul><li>Purpose is to make sure that the system fails and recovers gracefully. </li></ul><ul><li>Example: </li></ul><ul><li>Double the baseline number for concurrent users/HTTP connections. </li></ul><ul><li>Randomly shut down and restart ports on the network switches/routers that connects servers. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

14.
Why Performance Testing <ul><li>Identifies problems early on before they become costly to resolve. </li></ul><ul><li>Reduces development cycles. </li></ul><ul><li>Produces better quality, more scalable code. </li></ul><ul><li>Prevents revenue and credibility loss due to poor Web site performance. </li></ul><ul><li>Enables intelligent planning for future expansion. </li></ul><ul><li>To ensure that the system meets performance expectations such as response time, throughput etc. under given levels of load. </li></ul><ul><li>Expose bugs that do not surface in cursory testing, such as memory management bugs, memory leaks, buffer overflows, etc. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

15.
When is it required? <ul><li>Design Phase: Pages containing lots of images and multimedia for reasonable wait times. Heavy loads are less important than knowing which types of content cause slowdowns. </li></ul><ul><li>Development Phase: To check results of individual pages and processes, looking for breaking points, unnecessary code and bottlenecks. </li></ul><ul><li>Deployment Phase: To identify the minimum hardware and software requirements for the application. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

16.
What should be tested? <ul><li>High frequency transactions: The most frequently used transactions have the potential to impact the performance of all of the other transactions if they are not efficient. </li></ul><ul><li>Mission Critical transactions: The more important transactions that facilitate the core objectives of the system should be included, as failure under load of these transactions has, by definition, the greatest impact. </li></ul><ul><li>Read Transactions: At least one READ ONLY transaction should be included, so that performance of such transactions can be differentiated from other more complex transactions. </li></ul><ul><li>Update Transactions: At least one update transaction should be included so that performance of such transactions can be differentiated from other transactions. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

18.
<ul><li>Determine the performance testing objectives </li></ul><ul><li>Describe the application to test using a application model </li></ul><ul><ul><li>Describe the Hardware environment </li></ul></ul><ul><ul><li>Create a Benchmark (Agenda) to be recorded in Phase 2. </li></ul></ul><ul><ul><ul><li>Define what tasks each user will perform </li></ul></ul></ul><ul><ul><ul><li>Define (or estimate) the percentage of users per task. </li></ul></ul></ul>1.Planning >>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

21.
<ul><li>Monitoring the scenario: We monitor scenario execution using the various online runtime monitors. </li></ul><ul><li>Analysing test results: During scenario execution, the tool records the performance of the application under different loads. We use the graphs and reports to analyse the application’s performance. </li></ul>6. Analyze 5.Monitor >>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

23.
Questions to Review your Skills <ul><li>What are the factors that governs Performance Testing? </li></ul><ul><li>How are Throughput and Response time related with user load? </li></ul><ul><li>How do we decide whether the application passed or failed the load test? </li></ul><ul><li>What do you mean by Capacity, Stability and Scalability of an application. </li></ul><ul><li>What is the difference between Performance, Load and Stress testing? </li></ul><ul><li>What is Longevity, endurance, spike and Volume Testing? </li></ul><ul><li>At what point in SDLC, performance testing is required? </li></ul><ul><li>How to identify the transactions in a complete application for load testing? </li></ul><ul><li>Define the 6 steps involved in Performance Testing Process? </li></ul><ul><li>Explain the Load Test configuration of a web system and what is the role of Load Generators in it? </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

26.
Why Planning <ul><li>As in any type of system testing, a well-defined test plan is the first essential step to successful testing. Planning load testing helps to: </li></ul><ul><ul><li>Build test scenarios that accurately emulate your working environment: Load testing means testing the application under typical working conditions, and checking for system performance, reliability, capacity, and so forth. </li></ul></ul><ul><ul><li>Understand which resources are required for testing: Application testing requires hardware, software, and human resources. Before beginning testing, we should know which resources are available and decide how to use them effectively. </li></ul></ul><ul><ul><li>Define success criteria in measurable terms: Focused testing goals and test criteria ensure successful testing. For example, it’s not enough to define vague objectives like “Check server response time under heavy load.” A more focused success criterion would be “Check that 50 customers can check their account balance simultaneously & that server response time will not exceed 1- minute” </li></ul></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

27.
Why Planning <ul><li>Load test planning is a three-step process: </li></ul><ul><ul><li>Analyzing the Application </li></ul></ul><ul><ul><ul><li>Analysis ensures that the testing environment we create using LoadRunner will accurately reflect the environment and configuration of the application under test. </li></ul></ul></ul><ul><ul><li>Defining Testing Objectives </li></ul></ul><ul><ul><ul><li>Before testing, we should define exactly what we want to accomplish. </li></ul></ul></ul><ul><ul><li>Gathering Requirements </li></ul></ul><ul><ul><ul><li>All the requirements and resources should be evaluated and collected beforehand to avoid any last minute hurdles. </li></ul></ul></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

28.
Analyzing the Application <ul><li>Load testing does not require as much knowledge of the application as functional testing does. </li></ul><ul><li>Load tester should have some operational knowledge of the application to be tested. </li></ul><ul><li>Load tester should have the idea on how the application is actually used in production to make an informed estimate. </li></ul><ul><li>Load tester must know the application architecture (Client Server, Local Deployment, Live URL), Platform and Database used. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

29.
Defining Testing Objectives <ul><li>Determining and recording performance testing objectives involves communicating with the team to establish and update these objectives as the project advances through milestones </li></ul><ul><li>Performance, Load or Stress testing: Type and scope of testing should be clear as each type of testing has different requirements. </li></ul><ul><li>Goal Setting: General load testing objectives should be defined. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

30.
Defining Testing Objectives <ul><li>Common Objectives: </li></ul><ul><ul><li>Measuring end-user response time </li></ul></ul><ul><ul><li>Defining optimal hardware configuration </li></ul></ul><ul><ul><li>Checking reliability </li></ul></ul><ul><ul><li>Assist the development team in determining the performance characteristics for various configuration options </li></ul></ul><ul><ul><li>Ensure that the new production hardware is no slower than the previous release </li></ul></ul><ul><ul><li>Provide input data for scalability and capacity-planning efforts </li></ul></ul><ul><ul><li>Determine if the application is ready for deployment to production </li></ul></ul><ul><ul><li>Detect bottlenecks to be tuned </li></ul></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

31.
Defining Testing Objectives <ul><li>Stating Objectives in Measurable Terms: </li></ul><ul><li>Once you decide on your general load testing objectives, you should identify more focused goals by stating your objectives in measurable terms. </li></ul><ul><li>To provide a baseline for evaluation, determine exactly what constitutes acceptable and unacceptable test results. </li></ul><ul><li>For example: </li></ul><ul><ul><li>General Objective: </li></ul></ul><ul><ul><ul><li>Product Evaluation: choose hardware for the Web server. </li></ul></ul></ul><ul><ul><li>Focused Objective: </li></ul></ul><ul><ul><ul><li>Product Evaluation: run the same group of 300 virtual users on two different servers, HP and NEC. When all 300 users simultaneously browse the pages of your Web application, determine which hardware gives a better response time. </li></ul></ul></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

32.
Gathering Requirements <ul><li>Users: Identify all the types of people and processes that can put load on the application or system. </li></ul><ul><ul><li>Defining the types of primary end users of the application or system such as purchasers, claims processors, and sales reps </li></ul></ul><ul><ul><li>Add other types of users such as system administrators, managers, and report readers who use the application or system but are not the primary users. </li></ul></ul><ul><ul><li>Add types of non-human users such as batch processes, system backups, bulk data loads and anything else that may add load or consume system resources. </li></ul></ul><ul><li>Transactions: For each type of user we identified in the previous step, identify the tasks that the user performs. </li></ul><ul><li>Production Environment: </li></ul><ul><ul><li>Performance and capacity of an application is significantly affected by the hardware and software components on which it executes. </li></ul></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

33.
Gathering Requirements <ul><li>Production Environment: </li></ul><ul><ul><li>Speed, capacity, IP address and name, version numbers and other significant information. </li></ul></ul><ul><li>Test Environment: </li></ul><ul><ul><li>Should be similar to the production environment as is possible to be able to get meaningful performance results. </li></ul></ul><ul><ul><li>It is important that the databases be set up with the same amount of data in the same proportions as the production environment as that can substantially affect the performance. </li></ul></ul><ul><li>Scenarios: </li></ul><ul><ul><li>Select the use cases to include </li></ul></ul><ul><ul><li>Determine how many instances of each use case will run concurrently </li></ul></ul><ul><ul><li>Determine how often the use cases will execute per hour </li></ul></ul><ul><ul><li>Select the test environment </li></ul></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

36.
Questions to Review your Skills <ul><li>Why planning is required before starting load test? </li></ul><ul><li>What are the three steps involved in load test planning? </li></ul><ul><li>What information should be collected about the application to be load tested? </li></ul><ul><li>What are the common testing objectives? </li></ul><ul><li>State the following testing objective in measurable term: “Ensure that the new production hardware is not slower than the previous release” </li></ul><ul><li>Why is the knowledge of Production Environment necessary before load test? </li></ul><ul><li>What are the factors that need to be considered for creating a scenario? </li></ul><ul><li>How to choose a load test tool? </li></ul><ul><li>What are the requirements to setup a load test lab? </li></ul><ul><li>What are main points in a Load test checklist? </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

39.
<ul><li>Do you have the testing resources? </li></ul><ul><li>Testing personnel </li></ul><ul><li>Client machines </li></ul>How do you coordinate and synchronize users? How do you collect and analyze results? How do you achieve test repeatability? All of you, click the GO button again Manual Testing Limitations >>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<< Testers Load Generation System Under Test Analysis? 123.20 Web server Database server Coordinator

40.
Manual Testing Limitations <ul><li>Manual Testing Limitations </li></ul><ul><li>Expensive, requiring large amounts of both personnel and machinery. </li></ul><ul><li>Complicated, especially co-ordinating and synchronising multiple testers </li></ul><ul><li>Involves a high degree of organization, especially to record and analyse results meaningfully </li></ul><ul><li>Repeatability of the manual tests is limited </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

42.
Benefits of Automation <ul><li>Using Automated Tools </li></ul><ul><li>Reduces personnel requirements by replacing human users with virtual users or Vusers. These Vusers emulate the behaviour of real users </li></ul><ul><li>Because numerous Vusers can run on a single computer, the tool reduces the amount of hardware required for testing. </li></ul><ul><li>Monitors the application performance online, enabling you to fine-tune your system during test execution. </li></ul><ul><li>It automatically records the performance of the application during a test. You can choose from a wide variety of graphs and reports to view the performance data. </li></ul><ul><li>Because the tests are fully automated, you can easily repeat them as often as you need. </li></ul>>>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<