Backtrack includes a bunch of items that are not in the menu system such as the list-urls.py Python script. This script which has been enhanced by Muts simply queries a specific URL and extracts all of the URL’s from the page. These types of scripts really are handy and knowing your way around Backtrack can save you a ridiculous amount of time.

list-urls.py Python Script Code:

Click to expand the list-urls.py efPython code below.

list-urls.py Python Code

Close

#!/usr/bin/python
“””Extract list of URLs in a web page

This program is part of “Dive Into Python”, a free Python book for
experienced programmers. Visit http://diveintopython.org/ for the
latest version.
“””

As you can see from the code above the script is fairly simply but think how much time it would save you if you were tasked with extracting thousands of URL’s from a single web page. Below we show an example of the list-urls.py script found in the /pentest/enumeration/list-urls/ directory on Backtrack 5 r2.

As you can see above there were 4 URL’s available on the www.question-defense.com web page when this example was performed. The list-urls.py script is easy to use and performs a single task which is extracting URL’s from a web page.