The crawl archive for July 2026 is now available.
The data was crawled between July 7th and July 25th, and contains 2.14 billion web pages (or 364.01 TiB of uncompressed content). Page captures are from 40.5 million hosts or 33.2 million registered domains and include 603 million new URLs, not visited in any of our prior crawls.
Archive Location & Download
The July 2026 crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2026-30/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively, please see Get Started for detailed instructions.
Improvements and Changes
This crawl comes with a series of improvements and changes.
WARC-Record-ID Now Uses Type 7 UUID
We switch from using Type 4 to Type 7 UUIDs for WARC record IDs.
Type 7 UUIDs are defined by RFC 9562 (published in 2024 and updating RFC 4122) and combine a Unix timestamp (epoch seconds) with a random component. One example of a Type 7 WARC record ID:
WARC/1.0
WARC-Type: response
WARC-Date: 2026-07-10T22:25:47Z
WARC-Record-ID: <urn:uuid:019f4e23-0ae1-7128-99c9-625a3048e3b1>
Content-Length: 27888
Content-Type: application/http; msgtype=response
WARC-Warcinfo-ID: <urn:uuid:019f4e20-cbc0-7088-8251-ecc9a008540a>
WARC-Concurrent-To: <urn:uuid:019f4e23-0ae1-71ae-a89b-b0992fcec9b8>
WARC-IP-Address: 2620:cb:2000::1
WARC-Target-URI: https://commoncrawl.org/
WARC-Protocol: h2
WARC-Protocol: tls/1.3
WARC-Cipher-Suite: TLS_AES_128_GCM_SHA256
WARC-Payload-Digest: sha1:BYSGGYPUHJIF4ZIOWKMPPLYUJ5U5YJQO
WARC-Block-Digest: sha1:5OM6GSLZYKMJJ5MPXWM5BUVWW43GGZZU
WARC-Identified-Payload-Type: text/htmlThe following Python snippets demonstrates how the UUID can be parsed to extract the timestamp:
import datetime as dt
import uuid
record_id = uuid.UUID('urn:uuid:019f4e23-0ae1-7128-99c9-625a3048e3b1')
dt.datetime.fromtimestamp(record_id.time / 1000).isoformat()
# returns '2026-07-11T00:25:47.233000'As you can see, the timestamp in the in the UUID mirrors the WARC-Date header. Of course, the UUIDs in the WARC response records follow the same pattern. For further details about this change, please see the corresponding issue on GitHub.
WARC-Record-ID Included in URL Indexes
The record ID of the WARC response records is now contained in our URL indexes without the decorating <urn:uuid:>:
- The CDXJ index stores the UUID in the field
recordid. - The URL index has a new column
warc_record_id.
In order to use the new column in the URL index with Amazon Athena, you need to use the updated table schema defined by the cc-index-create-table-flat.sql statement.
The column warc_record_id is a binary column. You can cast from and to the data type UUID:
select cast(warc_record_id as uuid)
from ccindex
where url = 'https://commoncrawl.org/'
and url_host_name_reversed = 'org.commoncrawl'
and crawl = 'CC-MAIN-2026-30';To look-up a record ID:
select url, fetch_time
from ccindex
where cast(warc_record_id as uuid) = uuid '019f4e23-0ae1-7128-99c9-625a3048e3b1'
and url_host_name_reversed = 'org.commoncrawl'
and crawl = 'CC-MAIN-2026-30';Also DuckDb provides a UUID data type. You can use the new column the exact same way as in Amazon Athena.
More details about the new column can be found in the pull request cc-index-table#42 on GitHub.
The URL Index Switches to ZStandard Parquet Compression
The Parquet files of the URL index, starting with the current crawl's partition (crawl=CC-MAIN-2026-30) are compressed using ZStandard compression.
Given that most Parquet readers and query engines support ZStandard nowadays, no changes are required to query the recent partitions of the URL index. Additional information can be found in this issue report on GitHub.
Statistics of IP Address and HTTP Protocol Versions
We started publishing statistics of IP address and HTTP protocol versions observed by our crawler CCBot. The new metrics are found on the Crawl Stats site along with other crawler metrics.
.svg)
