I have written a perl script that wanders the WWW collecting URLs, keeping
tracking of where it's been and new hosts that it finds. Eventually,
after hacking up the code to return some slightly more useful information
(currently it just returns URLs), I will produce a searchabe index of this.
There is a complete list of all the sites it has found at
<a href="http://www.mit.edu:8001/afs/sipb/user/mkgray/ht/comprehensive.html">
A complete list of sites found by the W4 (World Wide Web Wanderer)
</a>
I'll announce here when we get this index properly running, however it probably
won't be until sometime in August, as I am going on vacation. Until then...
Matthew Gray
mkgray@athena.mit.edu
Visit the SIPB WWW Plexus Server. URL: http://www.mit.edu:8001/