Automate On-Demand Reporting with Selenium

The Terminator exposed it, the DARPA Grand Challenge rewards it, and Selenium puts its future in your hands. "It" is man versus machine… well, sort of. With Selenium, you are in control. Why link Selenium to the Terminator? Because it is that powerful.

As explained in the blog post above, Selenium scripts are often used to automate complex web app interactions such as authentication when scanning them via QualysGuard Web Application Scanning. Here we introduce a different use-case where we automate a QualysGuard subscriber’s interaction with the QualysGuard Vulnerability Management user interface, in order to demonstrate a best practice and make it easy to adopt — simply by running a Selenium script.

Make it happen

In a previous blog post about customizing Scorecard Reports, a fellow community member came up with a pretty good list of criteria of vulnerabilities to watch out for. Let’s take a closer look at creating a dynamic search list tailored to externally facing hosts. For such hosts, an initial starting point for discovering their "worst off the worst" vulnerabilities are those with the following criteria:

Remote (no authentication necessary) vulnerability.

Associated with a penetration testing toolkit (such as CORE or Exploit-DB).

Confirmed, severity 5 (easy segue to "game over").

Now I can show you screen shots of how to create this, but it’s so much easier to just create a Selenium script — by the way, it’s easier for you to run the script, too!

Open Firefox, log in to QualysGuard. While in QualysGuard, make sure you do not already have a search list with the name, "Remote exploit-available confirmed sev5 (Selenium)", or the script will error out.

From Firefox, Tools menu –> Selenium IDE

From Selenium IDE, File –> Open –> Open test case –> "create/Selenium test case, QualysGuard, create dynamic search list – remote exploit-available confirmed sev5.html". Check out the source, it is commented so you can see how it breaks apart building the search list.

Optional step. The script works at any speed, but if you would like to actually watch it work, I recommend slowing the execution down. Drag the speed bar from Fast to Slow so it’s easier to follow the script.

We are ready to run the script, also known as a test case in Selenium. Click on the "Play current test case" button. (Note this will play the test case that is shown in the right column under "Table | Source".)

Congratulations! You now have the dynamic search list we architected from our example. It’s called "Remote exploit-available confirmed sev5 (Selenium)".

You should see the dynamic search list under the "Search Lists" subtab (blue bar).

Note the criteria is optimized for externally facing hosts, as we described above:

Make it actionable

Pretty neat stuff, huh? But a search list on its own is not the most useful. Let’s create a report template that builds on it. Scratch that, let’s automate building of a report template that uses it.

Make sure you are still logged into QualysGuard inside of Firefox.

The test case creates a report template named "Remote exploit-available confirmed sev5 (Selenium)". So you want to make sure you do not already have a report template with the same name.

Load up the test case ("create/Selenium test case, QualysGuard, create report – remote exploit-available confirmed sev5.html") in the Selenium IDE. Check out the source, it is commented to explain how it accomplishes building the report.

Run it! It works at any speed.

After the test case completes, you will have a report template that uses the dynamic search list we just created.

Note the template filters against the search list we just created:

Make it easy

Now that we have built individual test cases for creating a search list and report template, let’s merge them into one step. Rather than copy and pasting the rows from one script into another, the Selenium IDE offers a much cleaner way via test suites.

Test suites offer a more coupled integration of test cases. This enables visible separation for troubleshooting individual test cases, while remaining transparently functional as a whole to the user. When we want to run a test suite, we click a different button from the Selenium IDE:

Let’s get started on running one together.

Confirm you are still logged into QualysGuard.

Before we recreate the report template and search list, we will need to delete the existing ones, as QualysGuard requires unique names of each. You can either delete them manually, or run the test suite, "Selenium test suite, QualysGuard, delete remote exploit-available confirmed sev5 report.html" that does it for you — try it, it’s not just for the lazy! Remember to click on the play button with multiple lines to its right, and it will delete both the report template and the search list.

Run the entire suite. It will start to create the search list from the first test case, and then automatically progress to create the report template from the second test case.

Congrats! You now have both a search list and report template. Easy, huh?

Make yours the next big hit

Now that we are able to create search lists and report templates associated with those search lists in one simple step, what’s next? Collaborate! Share your favorite search list, or report template, or both. Feel free to comment the criteria, or QIDs (one can also automate creation of static search lists, too).

If you are feeling ambitious, and want to create Selenium scripts of these reports, feel free to modify the ones we demoed. I am here to help if you have questions, just comment on this post. To get things started, I shared one more test suite that you may find useful (great for creating policies in Remediation):

You can find the above script (and more to come!) by searching for the tag, selenium_script. Help us grow QualysGuard automation by contributing! If you need help or have a request, just comment on this post… I’ll be back.

One response to “Automate On-Demand Reporting with Selenium”

This shows the real power of tools and sharing in a community like this.

If you come up with a good use for Selenium automation for QualysGuard feature building or for use in WAS, feel free to post the xml on the Community by toggling the IDE from "Table" to "Source". This is the easiest way to share what you have created with the rest of us.