Viewing live crawl data

Sitebulb will perform some configuration behind the scenes, and then shortly you'll be taken to the in-crawl page, where you can see the progress of your crawl.

If you just leave Sitebulb to do its thing, it will complete the crawl and then compile the reports for you.

On this screen there's a lot going on, but the main things to know are that the bits at the top are showing you current progress, whereas the 'URL Log' at the bottom is a list of the URLs that have just been crawled.

You can't interact with any of the data on this page, but if you want to start interrogating the data already then you can hit the blue button View Realtime URL Data, which will take you to a screen where you can check out the crawl data (you can sort or filter on any column).

It is important to understand that Sitebulb is much more than just a website crawler. Once it has finished performing the extraction element (crawling the website and scraping necessary data), it will then perform a number of complex analyses, which give rise to all the reports, graphs and data you will see in the Audit.