Lawyers have told us that we need to have some rules about how you can use this site, our service and/or our data. Thanks to them, we have a finely crafted Terms of Use document that lays out all the ground rules between you and Common Crawl. Since we don’t like wading through pages of legal documents any more than you do, we have provided a summary of the ToU below.

Please note that use of the this site, our service and/or data constitutes your binding acceptance of the full Terms of Use and any not just of the summary. You can find the full, legally binding document here: Terms of Use.


  • Use of our site or services means you have agreed to be legally bound by the full ToU. (We know we already said that, but it is important enough to repeat).
  • We may amend the ToU by posting an updated version online. If we change it, you are bound by the most current version it even if you didn’t look at the most current version.
  • We grant you a non-assignable, non-transferable, non-sublicensable, limited license to use our site and data in accordance with the terms of the ToU.
  • Don’t break the law or do anything illegal with our site or data. This includes, but is not limited to, honoring the restrictions of robot.txt files and NOFOLLOW metatags. Other examples of illegal stuff you can’t do are:
        • Engage in abusive, harassing, hateful or otherwise offensive activities
        • Invade other people’s privacy
        • Harm minors
        • Violate other people’s rights (IP, proprietary, etc.)
        • Circumvent copy-protection
        • Interfere or disrupt our site, service or security
        • Spam people
        • Stalk people
        • Impersonate people or otherwise disguise your identity
        • Forge headers or otherwise disguise our content
        • Harvest personally identifiable information
        • Communicate for commercial solicitation
        • Break the law (Again, important enough to repeat)
  • We didn’t produce the crawled content, we just found it on the web. So we are not vouching for the content or liable if there is something wrong with it.
  • We have the right to delete, recategorize or otherwise change the contents of the crawl repository.
  • We will kick you off and ban you if you break the law.
  • This site is protected by copyright, trademark and service marks.
  • We will take take appropriate action if you let us know about a copyright infringement.
  • When you tell us about a copyright infringement, you have to: notify us in writing, sign the notification, describe the copyrighted work being infringed, and give us your contact information.
  • To report a copyright infringement, please contact us at:

CommonCrawl Foundation
9663 Santa Monica Blvd., #425
Beverly Hills, CA 90210

  • Our site, services and data are provided “as is” with no warranty.
  • If you violate the ToU or do something that is somehow illegal with our site, services or data and you get sued, we are not liable and it is your obligation to make it clear we are not liable.
  • Common Crawl is based in Los Angeles California, so any disputes will be settled in Los Angeles courts.
  • Even if we fail to enforce the ToU in some instances, that doesn’t mean we are giving up the right to enforce it in other instances.
  • The ToU is the only agreement between you, the user, and Common Crawl.


If you know of a violation of the ToU, you can report it to us at:

[email protected]
Common Crawl Foundation
9663 Santa Monica Blvd., #425
Beverly Hills, CA 90210