Technical Discussions – Crosscheck Networks Canadahttp://www.crosschecknet.ca
Enabling the Data EconomyWed, 05 Dec 2018 20:59:55 +0000en-UShourly1https://wordpress.org/?v=4.5.16REST Tutorial 1 – JSON Functional Unit Testinghttp://www.crosschecknet.ca/json-functional-unit-testing/
http://www.crosschecknet.ca/json-functional-unit-testing/#respondWed, 18 Dec 2013 14:59:55 +0000http://www.crosschecknet.ca/?p=1652Lets start with the most basic of testing, – Unit functional Test of a REST based JSON service. Unit tests are often done by developers as and while they are developing the service, although there are many QA groups I have met that still do unit testing only. Most of this is available in the […]

]]>Lets start with the most basic of testing, – Unit functional Test of a REST based JSON service. Unit tests are often done by developers as and while they are developing the service, although there are many QA groups I have met that still do unit testing only. Most of this is available in the free Personal Edition of SOAPSonar. A REST/JSON API does not offer the same WSDL richness and structure that is offered by SOAP. Its lighter weight though better suited to mobile and certain other environments. Our application is mashup, with one component needing to display current exchange rates against the Canadian Dollar. For this example, I am going to use a simple exchange rate service http://rate-exchange.appspot.com/currency

1. Launch SOAPSonar with administrator rights. Unlike SOAP, we cannot capture the WSDL. Instead, select File, New, Test Group. The right-click on tests and add a New REST Test. (JSON will work as well). Right-click again and Rename New Test_1 to CAD to USD and name you test group.

2. In the URI in the Request tab enter http://rate-exchange.appspot.com/currency?from=cad&to=usd (How do you know the string, that’s in the manual for the API). Under method, make sure it is set to GET. Select the round green checkmark – Commit Test Settings and then the round blue Arrow – Send Current Request to Server. Notice SOAPSonar breaks out the protocol, Host, Path and URI parameters for you? Did you get a response? Does it look about right? You have just run your first unit test.

3. Now we want to add a few more units to the test. Right-clickon CAD to USD and Clone. Right-click on CAD to USD_1 and Rename it to CAD to EUR. The select CAD to EUR and change the to=usd to to=eur in order to change your get request. Select the round green checkmark – Commit Test Settings and then the round blue Arrow – Send Current Request to Server. Notice the URI changes to include your new eur get request?

4. Our Test case calls for 2 additional currencies. So follow steps 3 to clone and create 2 additional tests for currency INR and CNY. In each case chance the to=INRand to =CNY and Select the round green checkmark – Commit Test Settings and then the round blue Arrow – Send Current Request to Server.

5. Now we have successfully created 4 manual test cases to test the API. But this could have just as easily been done through a browser. Test automation software, is there to save testers time and effort. Leaving SOAPSonar in QA mode on the right, switch from Project View to Run View and Drag all 4 test cases into DefaultGroup. Select the round green checkmark – Commit Settings and then the round blue Arrow – Run Suite. Under success Criteria Evaluation how many Pass do you see? You just ran all 4 test cases together.

6. Select Analyze Results in Report View. Can you tell me what response time was for each test case, how large it was and was was the response code? Your boss walks up and asks you for a report on what you doing, select Generate Report and select detailed report for generation. Notice you can also export that report.

7. Go back to Run View. Right-clickon DefaultGroup and select Clone. If you run the test case now, how many have you run?

]]>http://www.crosschecknet.ca/json-functional-unit-testing/feed/0SOAP Tutorial 1 – Functional Unit Testinghttp://www.crosschecknet.ca/soap-tutorial-1-functional-unit-testing/
http://www.crosschecknet.ca/soap-tutorial-1-functional-unit-testing/#respondTue, 17 Dec 2013 22:29:50 +0000http://www.crosschecknet.ca/?p=1634Most basic of testing - Unit functional Test of a SOAP service. A similar example will be done for a REST/JSON service under the REST set of tutorials. Unit tests are often done by developers as and while they are developing the service, although there are many QA groups I have met that still do unit testing only.

]]>Lets start with the most basic of testing, – Unit functional Test of a SOAP service. A similar example will be done for a REST/JSON service under the REST set of tutorials. Unit tests are often done by developers as and while they are developing the service, although there are many QA groups I have met that still do unit testing only. Most of this is available in the free Personal Edition of SOAPSonar.A SOAP API offer some additional WSDL structure that aids testing without a client. If you have not installed and activated SOAPSonar, please follow this Tutorial first. For this example, I am going to use a SOAP service hosted here http://www.html2xml.nl/Services/Calculator/Version1/Calculator.asmx

1.Launch SOAPSonar using full administrator permissions. Make sure you leave SOAPSonar in QA mode (right hand top). In the main Capture WSDL bar enter the service we wish to capture by pasting http://www.html2xml.nl/Services/Calculator/Version1/Calculator.asmx and then add ?WSDL with no spaces and press enter. Notice the SOAP API Calculator.asmx is now imported which as 4 WSDL Services, Add, Subtract, Multiply, Divide.

2. For those who interested in seeing the code, If you select Configuration, Documents, View Documents, you can see the Main WSDL and the Embedded Schema that you just imported. More on WSDL Scoring Report Card in later tutorials perhaps.

3. Lets run through a simple Addition unit test while in project view still. Under WSDL, Calculator, Add, select Add_1. Notice SOAPSonar populates a UI for you with Body a= and B= fields. Put 6 in first field and 4 in the second then round green check box “Commit Settings” and the round blue arrow “Send Current Request to Server“. Did you get a response? Was it the right response of 10? Congratulations, you just unit tested a single service using SOAPSonar as the client.

4. Lets try that for the other services. Select Subtract_1 and enter the same a=6 and b=4 then round green check box “Commit Settings” and the round blue arrow “Send Current Request to Server“. And Multiply_1 and Divide_1 as well. Did they all work? Did you notice that 6/4 is truncated to 1 with no decimal places?

5. One way to add more test cases is to clone the service. This is usually used if you wish to establish separate test criteria for the service. Right click on Divide_1 and select Clone. Do you have Divide_2 now?

6. Lets rename that so we remember what we doing better. Right Click, Renameand call it “whole number“. Leave a=6 but make b =3. Then round green check box “Commit Settings” and the round blue arrow “Send Current Request to Server“. Did you get 2?

7. Now, this is slow and repetitive and a good QA tool is about saving the time taken for testing. I have 5 test cases and want to run them all together. Select Run View and lets simply drag all 5 test cases under the default group. Then round green check box “Commit Settings” and the round blue arrow “Send Current Request to Server“. Patient, you now running 5 test cases.

8. Under success criteria graph, how many pass and how many failures did you get? Since we have not set up any success criteria yet, I would hope you have 5 pass.

9. Select Analyze Results in Report View. Can you see the details of your 5 test cases now. Which one tool the longest to respond, what was the response codes for each? You boss wants to know what you doing. Take a moment to look under Generate Report drop down. Which of those can you send to them? How about the Detailed Functional Test Report? Notice you can export these reports?

10. But wait, maybe I should show more test cases. Select Run View again. Right click on Default Test and Rename it to Tutorial 1. Then right-click it again and Clone it. Rename the new Group Tutorial 2 and you ready for the next tutorial – Automating Data Sources

11. Save your project file as SOAP_Tutorial and take a stretch. Yes this is not very exciting yet, but perhaps you can start thinking of how automation can save you time?

]]>http://www.crosschecknet.ca/soap-tutorial-1-functional-unit-testing/feed/0SOAP Tutorial 4 – API Workflow Simulation and Testinghttp://www.crosschecknet.ca/api-testing-workflow-simulation/
http://www.crosschecknet.ca/api-testing-workflow-simulation/#respondFri, 09 Aug 2013 18:03:01 +0000http://www.crosschecknet.ca/?p=1036When it comes to web services API testing in a very simplistic view, folks are divided into two different groups. One group of users is interested in simply having something to invoke the service, primarily for debugging purposes. The other group of users however is interested in more sophisticated and robust validation of API’s functionality. […]

]]>When it comes to web services API testing in a very simplistic view, folks are divided into two different groups. One group of users is interested in simply having something to invoke the service, primarily for debugging purposes. The other group of users however is interested in more sophisticated and robust validation of API’s functionality. Crosscheck Networks focus is on Enterprise Level tools for insuring high quality of the end product – Robust and Sustainable API.

One of the fundamental benefits of a web services-based API is reuse and componentization. One web services call may however depend on the response from another. The result is a meshed or chained set of interdependent operations called in sequence. Automated testing requires test cases to be built, linking these chained services.

In tutorial #2 – Functional Testing, we looked at how functional testing of a single Web Services API. In this tutorial, we look at chaining these into a Automated test case using workflow simulation. A common example of a multi-step process would be:

1. Login

2. Create Customer Record

3. Assign product to a client

4. etc etc etc

5. Logout

Here the services must be chained and information from one service must be passed to the next in order to simulate a complete workflow. SOAPSonar is capable of simulating of complex workflows without any scripting or coding. Saving the automated test case, that then requires very little maintenance as new releases become available.

The web service that we will use is hosted by http://www.w3schools.com and is called Temperature Convert. NOTE: This is a SOAP based service. The process of chaining two REST services or ESB based service etc. is much the same. We will chain two requests, passing response from the first call to the second.

1. First, we need to capture the WSDL of the service to determine what operations are exposed by the service.

3. We are ready to submit our FahrenheitToCelsius request. Select SEND CURRENT REQUEST TO SERVER.

4. Now we need to capture result returned by FahrenheitToCelsius and store in a variable. This variable will be used to pass the value to the next operation in the chain. Execute the operation and save the value. In the Response Panel, select the RUNTIME VARIABLES Tab as shown below.

5. Right click on the FahrenheitToCelsiusResult value 37.77777777 and select the Add Variable Reference. Confirm the name and select OK. You should see a Variable Name $rv:FahrenheitToCelsius_1 : Element : FahrenheitToCelsiusResult$ and a Capture Element appears in the Variable Reference(1) Tab.

10. Select SEND CURRENT REQUEST TO SERVER Executing the CelsiusToFahrenheit test results in the FahrenheitToCelsiustest case Response Value being evaluated and sent as an input to CelsiusToFahrenheit with the following result converting back to 100

11. Now to view your automated chained test case visually, go to Run View and drag and drop CelsiusToFahrenheit test case (only) into DefaultGroup. Expanding tree view, you will see that SOAPSonar automatically recognized that FahrenheitToCelsiustest Test Case is a prerequisite to CelsiusToFahrenheit and must be run first, as part of the automated test.

12. Select RUN SUITE. The ANALYZE RESULTS IN REPORT VIEWwill show the sequence of the call as well as the response times for each step in the chained link

13) Save the project and name it.

You know know how to use SOAPSonar to chain the response from one API service and use that response as the input for a second service. Could you do this with a Automated data source and success criteria?

]]>http://www.crosschecknet.ca/api-testing-workflow-simulation/feed/0Tutorial – External Data Sources in SOAPSonarhttp://www.crosschecknet.ca/soapsonar-introduction-tutorial-3-external-data-sources/
http://www.crosschecknet.ca/soapsonar-introduction-tutorial-3-external-data-sources/#commentsFri, 02 Aug 2013 20:28:21 +0000http://www.crosschecknet.ca/?p=964Entering input values for messages can be a manual task. However, when a large number of input values are involved this process can be automated by using external data sources such as RDBMS, Microsoft Excel Spreadsheets or flat files that contain input values. In this tutorial, we will develop a test suite that uses an […]

]]>Entering input values for messages can be a manual task. However, when a large number of input values are involved this process can be automated by using external data sources such as RDBMS, Microsoft Excel Spreadsheets or flat files that contain input values. In this tutorial, we will develop a test suite that uses an external Excel Data Source.

This tutorial is a continuation of SOAPSonar Introduction Tutorial 2 – Functional Testing and has prerequisites from that Tutorial

Before starting this Tutorial, the user should ensure they switch SOAPSonar to QA Mode and Project view

1. Create & Populate an Excel Spreadsheet TestData.xls with the first row labeled as State and Population. Populate the values in this spread sheet. As sample Excel file is shown below.

2. Under the Configuration Node in the Project Tree, select Data Sources as shown in the Figure below. Select Add Automation Data Source > Excel Data Source. Browse to the TestData.xls and COMMIT

3. Verify that the Excel spreadsheet has been properly loaded, click on the spreadsheet name in t he list of data sources and then click on the REFRESH followed VIEW DATA buttons. The Data contents of the spreadsheet will be displayed in the pop-up window.

4. For testing a Web Services API that returns state population we need to provide State as an input parameter. This value will populate directly from the above created Automation Data Source (ADS). Lets clone a new test first to work with and rename it Autostate in the same way we clones Maryland in Tutorial 2

5. Go to your request and Put your cursor after “for=state:”. Right click to open submenu. Go to [ADS] Automation Data Source -> Quick Select->TestData.xlsx->State. Notice as SOAPSonar automatically recognizes that the data source was added to the project and allows you to automate test input parameters.

6. Once you select the Automatic Data Source (ADS) State as inputs for the request, unique reference IDs for the data sources starting with $ads: will appear. COMMIT

7. Go to Success Criteria tab delete anything entered here. We will create a new success criteria later in this tutorial. SAVE

8. Go to the Run View clear out any other tests and Drag-and-Drop the Autostate to Test Suite under the Default Group as shown in the Figure below. COMMIT. The RUN. Once the test suite is executed, click on Analyze Results in Report View. The test case iterates through all four State values and substitute values in the request as expected.

Testers can generate comprehensive test suite values by simply pointing to reusable data source such as an Excel spreadsheet or a RDBMS. Many of these are available free or commercially. Greatly reducing the time needed to run a test and increasing the number of possible inputs resulting in more testing and higher quality

So far we looked at Automating Requests using ADS. Now, let’s look at automating response validation from the service.

9) Go back to Project View and then Success Criteria Tab and add a new XPath Match criteria. SOAPSonar will automatically recognize that Web Service API response will be in JSON Format. Right-click on the JSON value 37253956 node and select Compare Element Value( to validate Population).

11. Go to Run View and RUN the test. Now SOAPSonar will not only look at the HTTP response code but also compare Value that was returned by the service with expected results stored in Excel file. Did all 4 pass?

12. Lets set one to fail. Go back and edit your Testdata.xls file with a line 6. Fill in say state 25 and put 25 under the population as well. Save your Excel file and rerun the test. Can you see 5 tests where now run and 1 failed?

If you go to Analyze Results in Report View you can see Test 5 is red, selecting that you can see the response for state 25 is 6547629 and not 25 as we have in our data source.

]]>http://www.crosschecknet.ca/soapsonar-introduction-tutorial-3-external-data-sources/feed/2SOAPSonar Introduction Series #2 – Functional Testinghttp://www.crosschecknet.ca/soapsonar-introduction-series-2-functional-testing/
http://www.crosschecknet.ca/soapsonar-introduction-series-2-functional-testing/#respondFri, 02 Aug 2013 14:40:04 +0000http://www.crosschecknet.ca/?p=921Functional testing is the first pillar of API testing. As a part of this test, we will set up and run a number of tests that make up a test suite. We will also learn how to configure success criteria to determine PASS/FAIL for each test case in the test suite. Our Test API US […]

]]>Functional testing is the first pillar of API testing. As a part of this test, we will set up and run a number of tests that make up a test suite. We will also learn how to configure success criteria to determine PASS/FAIL for each test case in the test suite.

Our Test API

US Census Bureau has begun to provide Open API access to the Economic Census is the U.S. To use the census API you will need to register and request a Key. For more information on this please visit census website

Let’s say we would like to test that API correctly returns the current population of California. From documentation on the site, California is state:06 and we need to issue a get request for P0010001 with the above key

1. Configure REST test cases in SOAPSonar we first need to create a new Test Group to hold all Test cases – Click File->New->Test Group as shown in the Figure below.

2. Right click on the Tests folder and Select New REST Test. Right click on New Test and name it California.

3. Now we need to provide Protocol, Host, Path, URI Parameters and the request Method.

There are two ways to do it – One you can simply type it all in URI Window. The second is to enter individual values in the boxes provided. Either way you do it, SOAPSonar will make sure that all values are in Sync:

In the URI window enter http://api.census.gov/data/2010/sf1?key=xxxxxxxxx&get=P0010001&for=state:06 (please enter your own key) and GET

4. COMMIT the values for test case and SAVE your project.

5. You are now ready to send your first REST request in SOAPSonar. RUN the test case The API response will be processed and displayed on the bottom Response tab.
In this case response will be send in a JSON format and should look something like[[“P0010001″,”state”], [“37253956″,”06”]]. Population of state 06 (California) is 37,253,956

If you don’t get this, something is wrong. On occasion, if the service is down, you may get a timeout after 10 seconds.

For every request, the response from the web service API must be evaluated. To determine whether a response is valid or invalid, a tester should setup pre-defined filters that examine HTTP return codes or any business specific content contained in a response.

6. Next, click to the Success Criteria Tab, to define evaluation criteria that SOAPSonar will use to determine if the TC Pass or Fail:

Click Add Criteria Rule-> Document->Xpath Match:

7. Click on the XPath Match rule to edit it.

SOAPSonar will automatically recognize that response is in JSON format. ( Tip: click on UPDATE TREE if the tree is not populated)

8. To validate that State and Population values returned as expected Right-click on the JSON value 06 node and select Compare Element Value( to validate State).

SOAPSonar will automatically create a rule that this element must contain) 06 value and COMMIT your changes.

10. Now lets add a test case for Maryland. We could repeat steps 2-9 using State:24. The faster way to do it is to clone California by Right Click on California and select Clone. Rename it Maryland and modify State:24. COMMIT your changes and RUNthe GET request. Go to success criteria, and refresh . Under tab Criteria Rules make sure 1st. Element is 24 and 2nd Element is 5773552 (The expected responses.

All test configuration and results validation setup is complete and you are ready to run the test.

11. Once the test cases are defined, the user can select any combination of test cases and build a Test Suite. To build a test suite, click on the Run View as shown in the Figure below. From the left-most navigation panel, Drag-and-Drop “California” and “Maryland” test cases into the Default Test Suite. Commit your changes. Test cases that appear under DefaultGroup are selected to run as the Default Suite.

12. RUN the test case and then select Analyze Results in Report View on the Real-time Run Monitor.

13. As seen in the Report View Panel below, all tests PASS (as seen in the Test Result Column) with the responses from the Service operations as expected.

If for any reason Test Case fails you may go to “Success Criteria Evaluation” Tab to learn the reason why.

14. By selecting the appropriate report and clicking on the GENERATEicon a PDF test report is automatically generated detailing the tests passed, response times etc.

]]>http://www.crosschecknet.ca/soapsonar-introduction-series-2-functional-testing/feed/0Distributed Agent Load Testinghttp://www.crosschecknet.ca/distributed-agent-load-testing/
http://www.crosschecknet.ca/distributed-agent-load-testing/#respondWed, 31 Jul 2013 01:55:11 +0000http://pixelperfectdesignstudio.com/crosschecknetworks/?p=551SOAPSonar Enterprise Server edition includes Distributed Agent Load Testing Performance of up to 50 remote agents per license. Each agent supporting up to 50 additional virtual Clients (virtual clients can be limited by physical network constraints and this should be considered in any performance test) Additional agents can be added. Agents can be distributed across […]

]]>SOAPSonar Enterprise Server edition includes Distributed Agent Load Testing Performance of up to 50 remote agents per license. Each agent supporting up to 50 additional virtual Clients (virtual clients can be limited by physical network constraints and this should be considered in any performance test) Additional agents can be added. Agents can be distributed across machines and placed in multiple locations, yet triggered from SOAPSonar. Distributing agents enables the testing of the impact of load on network and location. Ensuring the performance of client applications (Consumers) at their intended physical location, is known.

To get your agent working please follow these steps:
On the menu select Agents – Download Agent SOAPSonar Installer

Agent Installation

Install agent on the desired remote machines as administrator.

Note the IP address and port setting for each agent.

Confirm there is no firewall is blocking communications between SOAPSonar and each agent and that the port is open.

ensure the agent is running (left bottom corner)

In SOAPSonar

On the menu select Agents -> Configure Performance Agents and enter the IP address and same port for each agent machine the agent is installed on.

Export Agent Collection, naming it for your load test

Put SOAPSonar into Performance Mode and Run View and click on the tab Performance Loading Agents

Import Agent Collection and select the file you exported above

Activate each agent you wish to run by clicking on enabled icon to ensure its green

The Local, Agent is SOAPSonar machine you triggering the test from

Set-up Group Performance Settings and Suite Settings as per “LOAD TESTING WITH SOAPSONAR” allocating the number of virtual users per instance.

Run your test by clicking on the blue commit arrow and the agent initialization window will popup.

Once all agents are initialized press Start Test button to start you performance test.

Reports

In the performance monitor you can select individual agents or view aggregated reports after test is complete

Conclusion

Distributed agents enable the testing and evaluation of impact of location and load on application servers. We usually recommend to our clients that they run simulated load tests before deployment and frequent regression tests via distributed agents to identify and isolate real world scenarios. Assessing the impact of access via a remote office, mobile network, home office, cloud, overseas location or private network and determining the best architecture for deployment.