bew is a Web mirroring tool that will recursively get a whole set of Web pages using the HEAD mechanism so that it only downloads pages that have changed. It mostly works, but there are still a lot of features to be added. Support for checking external links is also implemented.

CCTE is a small utility to provide a way to run automated tests on Web applications. It acts like a Web browser GETing form data to different URLs, and logging output. It has two ways to understand if a CGI did OK: HTTP status, and user-definable strings for both (success and error).

Checkbot is a tool to verify links on a set of HTML pages. Checkbot can check a single document, or a set of documents on one or more servers. Checkbot creates a report which summarizes all links which caused some kind of warning or error.

Checklinks is yet Another HTML Link Checker (in Perl 5). Features include SSI (.shtml) support, direct file reads where possible, HTML 4.0, HTTP 1.1, and aliases and other server options. Other useful but more common features include regular expressions to restrict the URLs searched and results reported, and a detailed verbose report. Checklinks was written with Apache in mind, and you can feed it your srm.conf file to auto-configure many settings.

Dead Link Check (DLC) is a Perl script designed to find information on validity of HTTP references. The script may use/generate a cache file for avoiding redoing network requests if the user wants to check added entries. The script works by reading entries from a file (or a list of links from the command line) and output results in file(s) (or STDOUT). DLC was created as an extension to Public Bookmark Generator (PBM), but can be used on its own.

FPSE clone is a Microsoft Frontpage Server Extensions clone. FPSE is should work with nearly any web server, because it is a regular CGI written in perl. It is also more secure than its Microsoft counterpart, because it does not require SUID priviledges.

GNU Wget is a utility for noninteractive download of
files from the Web. It supports HTTP and FTP
protocols, as well as retrieval through HTTP
proxies. It can follow HTML links, download many
pages, and convert the links for local viewing. It
can also mirror FTP hierarchies or only those files
that have changed. Wget has been designed for
robustness over slow network connections; if a
download fails due to a network problem, it will
keep retrying until the whole file has been
retrieved.

Hammerhead is a stress testing tool for Web sites. It initiates connections from multiple IP aliases and simulates a user from each alias. It is fully configurable, and there are numerous other options for creating problems with a site. Extensive data collection is also available.

ht://Check is a link checker derived from
ht://Dig. It can retrieve information through
HTTP/1.1 and store it in a MySQL database so that
after a "crawl", ht://Check can return broken
links, anchors not found, content-types, and HTTP
status codes summaries. ht://Check also performs
accessibility checks in accordance with the
principles of the University of Toronto's Open
Accessibility Checks (OAC) project, allowing users
to discover site-wide barriers like images without
proper alternatives, missing titles, etc. A PHP
interface lets the user query and view the results
directly via the Web.