REST API Testing with Qualys Web Application Scanning

With more web applications exposing RESTful (or REST) APIs for ease of use, flexibility and scalability, it has become more important for web application security teams to test and secure those APIs. But APIs (including REST APIs) introduce some behaviors that make it difficult for web application scanners to test them for vulnerabilities.

About REST APIs

With advancement in the design of web applications, newer and newer design choices are coming up for developers of web applications. Web applications have evolved from static web pages to AJAX (Asynchronous Javascript And XML) applications. The latest in the series of advancements is for developers to expose the functionality of their application using REST (REpresentational State Transfer) APIs, whereas earlier forms of web services were exposed using SOAP (Simple Object Access Protocol) or WSDL (Web Service Description Language). RESTful Web services typically expose their functionality via HTTP verbs such as GET, PUT, POST and request/response format such as JSON, XML or HTML. Many newer mobile and IoT (Internet Of Things) applications are exposing their functionality using REST APIs.

Automated Discovery of REST APIs

The first challenge in testing REST APIs is to find and catalog them so that all input points can be tested. Unlike traditional web applications where a web crawler is launched to organically discover various links and forms, RESTful web services can be exposed using specification files such as WADL (Web Application Description Language), Swagger, RAML (RESTful API Modeling Language) or using proxy capture of the REST API client. However, many REST services do not provide specification files, but instead provide a sample client to test the APIs.

Once the REST APIs have been discovered, the next challenge is to test them for vulnerabilities.

How Qualys WAS Tests REST APIs

There are three ways to pass inputs to REST APIs: via URL query parameters, via URL path variables, or via the body of the HTTP request. All three expose potential injection points for malicious payloads, and Qualys WAS can test all of them.

Specifically, Qualys WAS supports fuzzing of requests with GET, POST and PUT verbs (with support for DELETE under investigation), and it supports fuzzing of the body of HTTP requests with “Content-Type” as “application/json”. Fuzzing involves passing specially crafted inputs to the API that are designed expose security vulnerabilities. All are described below.

To illustrate the three ways to pass inputs, let’s imagine a hypothetical contact management REST API that exposes a number of services including this one that returns a list of current contacts:

GET request to http://www.example.com/api/v1/contacts

URL Query Parameters

With URL query parameters, inputs are passed as parameters in the URL after the “?”, typically with an HTTP GET request.

Some examples are:

GET request to http://www.example.com/api/v1/contacts?sort=firstname
This returns a list of contacts sorted by firstname.

GET request to http://www.example.com/api/v1/contacts?per_page=5&page=5&sort=email
This provides a list of contacts paginated by 5 contacts per page and sorted by email with content of 5th page.

URL Path Variables

With URL path variables, inputs are passed directly in the URL, often via URL rewriting (where URL parameters are rewritten as URL path variables for better readability and search engine optimization). REST APIs can also include URL path variables that are not the result of URL rewriting. Testing is the same in either case.

Some examples are:

GET request to http://www.example.com/api/v1/contacts/50
This provides details of contact whose ID is 50. This is also an example of a URL rewrite of http://www.example.com/api/v1/contacts?id=50

GET request to http://www.example.com/api/v1/contacts/50/notes
This provides details of all the notes associated with the contact whose ID is 50. A typical response for this request would be [{ “id”: “1”, “body”: “Carpenter with good wooden work ability.”, “contact_id”: “50” }, { “id”: “2”, “body”: “Sample project at Mumbai Bandra.”, “contact_id”: “50” }]

GET request to http://www.example.com/api/v1/contacts/50/notes/1
This provides details of note with ID 1 associated with the contact whose ID is 50. A typical response for this request would be { “id”: “1”, “body”: “Carpenter with good wooden work ability.”, “contact_id”: “50” }

Additional Challenges

Authentication: Typical web applications use form/cookie or server-based authentication while REST APIs use a variety of other authentication methods such as certificate, API_KEY or authentication token.

The current REST API testing in Qualys WAS supports two kinds of authentication methods:

Server Auth – This includes Basic, NTLM and Digest authentication methods. This is similar in behavior to how these methods are used in traditional web applications.

API_KEY – This supports passing an API_KEY in a header in all the requests that are created by WAS. This method is useful in testing REST APIs that take a custom header to identify a user/session.

Rate Limiting: Traditional web applications do not have any limit on the number of requests performed by test application during a fixed interval and it completely depends on how much load the server is capable of handling. While in case of RESTful Web services, server will limit the number of API calls done during a fixed interval and if the requests exceed the limit, server will typically send HTTP response code 429 (Too Many Requests). We recommend that web application security teams check if rate limiting is blocking their scans.

Related

This is good news. But the Burp Tab in Qualys WAS UI portal allows to upload a valid XML scan report not proxy capture. Is burp proxy capture the only way to do this or there are some other options in Qualys WAS that we can try? Or there is something else that I am missing here?