Search results

Common Crawl - Blog - WikiReverse- Visualizing Reverse Links with the Common Crawl Archive

This is a guest blog post by Ross Fairbanks, a software developer based in Barcelona. He mainly develops in Ruby and is interested in open data and cloud computing. This guest post describes his open data project and why he built it. Ross Fairbanks.

Common Crawl - Blog - URL Search Tool!

A couple months ago we announced the creation of the Common Crawl URL Index and followed it up with a guest post by Jason Ronallo describing how he had used the URL Index.

Common Crawl - Blog - Towards Social Discovery - New Content Models; New Data; New Toolsets

This is a guest blog post by Matthew Berk, Founder of Lucky Oyster. Matthew has been on the front lines of search technology for the past decade. Matthew Berk. Matthew Berk is a founder at Bean Box and Open List, worked at Jupiter Research and Marchex.

Common Crawl - Blog - August 2015 Crawl Archive Available

For full details, refer to Ilya's. guest blog post. Please. donate. to Common Crawl if you appreciate our free datasets! We're also seeking corporate sponsors to partner with Common Crawl for our non-profit work in big open data!

Common Crawl - Blog - March 2015 Crawl Archive Available

For full details, refer to Ilya's. guest blog post. Please. donate. to Common Crawl if you appreciate our free datasets! We're also seeking corporate sponsors to partner with Common Crawl for our non-profit work in big open data!

Common Crawl - Blog - June 2015 Crawl Archive Available

For full details, refer to Ilya's. guest blog post. Please. donate. to Common Crawl if you appreciate our free datasets! We're also seeking corporate sponsors to partner with Common Crawl for our non-profit work in big open data!

Common Crawl - Blog - May 2015 Crawl Archive Available

For full details, refer to Ilya's. guest blog post. Please. donate. to Common Crawl if you appreciate our free datasets! We're also seeking corporate sponsors to partner with Common Crawl for our non-profit work in big open data!

Common Crawl - Blog - April 2015 Crawl Archive Available

For full details, refer to Ilya's. guest blog post. Please. donate. to Common Crawl if you appreciate our free datasets! We're also seeking corporate sponsors to partner with Common Crawl for our non-profit work in big open data!

Common Crawl - Blog - July 2015 Crawl Archive Available

For full details, refer to Ilya's. guest blog post. Please. donate. to Common Crawl if you appreciate our free datasets! We're also seeking corporate sponsors to partner with Common Crawl for our non-profit work in big open data!

Common Crawl - Blog - Announcing the Common Crawl Index!

This is a guest post by Ilya Kreymer, a dedicated volunteer who has gifted large amounts of time, effort and talent to Common Crawl.

Common Crawl - Blog - Analyzing a Web graph with 129 billion edges using FlashGraph

This is a guest blog post by Da Zheng, the architect and main developer of the FlashGraph project.

Common Crawl - Blog - Evaluating graph computation systems

This is a guest blog post by Frank McSherry, a computer science researcher active in the area of large scale data analysis. While at Microsoft Research he co-invented differential privacy, and lead the Naiad streaming dataflow project.

Common Crawl - Blog - Twelve steps to running your Ruby code across five billion web pages

The following is a guest blog post by Pete Warden, a member of the Common Crawl Advisory Board. Pete is a British-born programmer living in San Francisco.

Common Crawl - Blog - Lexalytics Text Analysis Work with Common Crawl Data

This is a guest blog post by Oskar Singer, a Software Developer and Computer Science student at University of Massachusetts Amherst. He recently did some very interesting text analytics work during his internship at Lexalytics.

Common Crawl - Blog - Web Data Commons Extraction Framework for the Distributed Processing of CC Data

This is a guest blog post by Robert Meusel, a researcher at the University of Mannheim in the Data and Web Science Research Group and a key member of the Web Data Commons project.

Common Crawl - Blog - Navigating the WARC file format

This is a guest blog post by. Stephen Merity. , a Computational Science and Engineering master's candidate at Harvard University. His graduate work centers around machine learning and data analysis on large data sets.

Common Crawl - Blog - Web Image Size Prediction for Efficient Focused Image Crawling

This is a guest blog post by Katerina Andreadou, a research assistant at CERTH, specializing in multimedia analysis and web crawling.