Next Level Reconnaissance with Shodan and Censys

Posted on December 2nd, 2015

During the initial phases of a penetration test, online resources such as Shodan can serve as a starting point to identifying the technology footprint of your target organization.

A new resource has been released, known as Censys.io, that collects data on internet facing websites and systems through daily ZMap and ZGrab scans of the IPv4 address space.
These online resources can be leveraged together or separately with various search parameters (such as TLD, CIDR, etc.) to query the available databases, helping to identify any information ​and services​ exposed to the Internet.

The four primary ways these resources are leveraged:

Query a CIDR to enumerate systems and services (take note of any additional domains that are enumerated)

Query a Top Level Domain (TLD) to enumerate systems and services (take note of any additional CIDR ranges that are enumerated)

Query for a single target system

Query for a search string (Example: Apache, etc.)

It is often useful to show a ​customer​ the external host and service exposure, and this process will commonly lead to the scope being expanded to include previously “unknown” hosts/services. Both Shodan and Censys.io offer a GUI to interact with the data, though both provide direct access to the data through an API.

The combination of Shodan and Censys allows for a more thorough capture of external services due to the variance in results. These resources can be quickly accessed and leveraged as a starting point for initial reconnaissance of a target organization.

Shodan:

Shodan is an excellent resource for external service enumeration, and supports queries of both IPv4 and IPv6 data. Shodan offers both a paid and free edition for API access and extended GUI access, as well as a Python module to make data extraction easier. Below is an example of interacting with Shodan’s free API access in Python:

With the premium Shodan API access, you can implement search filters when performing queries, drastically increasing the functionality and level of detail you can use when searching for data. A more advanced example of Shodan scripting which requires paid API access is demonstrated below. The output has been abbreviated, but you can see this can give you a quick snapshot of external services, network ranges, TLDs, etc. that can be used to drive your reconnaissance and scoping process further.

**Recon-ng and Maltego also can tie into Shodan and offer increased functionality by configuring the tool with an API key.

Censys:

Censys.io is a new free online resource that was created by researchers at the University of Michigan. This project offers a web services API, as well as a Python module to improve API interaction. The Censys.io API can be utilized with a free account once created. In order to perform API queries, you will need the API ID and Secret key values associated with your account. Both of these unique key values can be found under the “My Account” section. Below is a quick example of interacting with the Censys.io API in Python:

Below is a POC script that extracts data from the API, by default only grabbing the first page of results (100 results). To circumvent this restriction, we added some simple logic to parse the total and make the appropriate number of queries.