Congrats on the update. Do you plan on doing (or have you already done) an analysis blog post of how your index has changed over time in terms of things like % nofollowed, number of pages with rel=canonical, etc? I’d be interested to see how quickly the web follows the search engines’ new initiatives.

Hello there. First off, thanks for creating such an interesting tool. I am, however, experiencing a problem with certain URLs. For example while http://davecardwell.co.uk/ is fine, http://davecardwell.co.uk/cv/ gives the error:

We were unable to fetch the URL you entered. Please verify the URL is active and try again.

The pages are generated essentially the same, have exactly the same response headers, and I can see the request and 200 response in my apache2 logs. Any ideas?