October 27, 2010

SlideShare: Building a Scalable Web Crawler with Hadoop

Note: this post has been marked as obsolete.

Common Crawl on building an open Web-Scale crawl using Hadoop.

Common Crawl Foundation

Common Crawl builds and maintains an open repository of web crawl data that can be accessed and analyzed by anyone.

This release was authored by:

No items found.