The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.