September 2020 crawl archive now available
The crawl archive for September 2020 is now available! The data was crawled between September 18th and October 2nd and contains 3.45 billion web pages or 345 TiB of uncompressed content. It includes page captures of 1.5 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The September crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-40/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-40/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-40/warc.paths.gz | 79600 | 81.8 |
WAT files | CC-MAIN-2020-40/wat.paths.gz | 79600 | 23.14 |
WET files | CC-MAIN-2020-40/wet.paths.gz | 79600 | 10.28 |
Robots.txt files | CC-MAIN-2020-40/robotstxt.paths.gz | 79600 | 0.22 |
Non-200 responses files | CC-MAIN-2020-40/non200responses.paths.gz | 79600 | 2.36 |
URL index files | CC-MAIN-2020-40/cc-index.paths.gz | 302 | 0.27 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-40/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
August 2020 crawl archive now available
The crawl archive for August 2020 is now available! It contains 2.45 billion web pages or 235 TiB of uncompressed content, crawled between August 2nd and 15th. It includes page captures of 940 million URLs unknown in any of our prior crawl archives.
Archive Location and Download
The August crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-34/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-34/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-34/warc.paths.gz | 60000 | 48.9 |
WAT files | CC-MAIN-2020-34/wat.paths.gz | 60000 | 16.9 |
WET files | CC-MAIN-2020-34/wet.paths.gz | 60000 | 7.56 |
Robots.txt files | CC-MAIN-2020-34/robotstxt.paths.gz | 60000 | 0.19 |
Non-200 responses files | CC-MAIN-2020-34/non200responses.paths.gz | 60000 | 1.94 |
URL index files | CC-MAIN-2020-34/cc-index.paths.gz | 302 | 0.19 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-34/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
July 2020 crawl archive now available
The crawl archive for July 2020 is now available! It contains 3.14 billion web pages or 300 TiB of uncompressed content, crawled between July 2nd and 16th. It includes page captures of 1.1 billion URLs unknown in any of our prior crawl archives.
Bug Fixes and Improvements
The URL index fields "redirect" and "mime" haven’t been filled if the corresponding HTTP headers Location
and Content-Type
are written in lower-case letters or any other variant not matching case. This bug has been detected during the crawl and was fixed for 90 out of 100 segments. It also affects the columnar index and the fields "fetch_redirect" resp. "content_mime_type". To a minor extend it may affect the detection of character set and content language as the value of the Content-Type
header is used as additional hint for the detection. Additional information about this bug fix is given in the corresponding issue report.
Archive Location and Download
The July crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-29/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-29/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-29/warc.paths.gz | 60000 | 62.64 |
WAT files | CC-MAIN-2020-29/wat.paths.gz | 60000 | 22.23 |
WET files | CC-MAIN-2020-29/wet.paths.gz | 60000 | 9.87 |
Robots.txt files | CC-MAIN-2020-29/robotstxt.paths.gz | 60000 | 0.21 |
Non-200 responses files | CC-MAIN-2020-29/non200responses.paths.gz | 60000 | 2.52 |
URL index files | CC-MAIN-2020-29/cc-index.paths.gz | 302 | 0.24 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-29/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
Host- and Domain-Level Web Graphs Feb/Mar/May 2020
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of February, March/April and May/June 2020. Additional information about the data formats, the processing pipeline, our objectives, and credits can be found in the announcements of prior webgraph releases (e.g., Nov/Dec/Jan 2017-2018 Webgraphs). You may also visit the projects cc-webgraph and cc-pyspark which host all scripts and tools required to construct the graphs.
What’s new?
The host-level graph now includes hosts visited by the crawler but not linking to any other host. Why is this possible – isn’t any host found via links the crawler is following? Yes, but some links were already detected in a prior crawl, not in one of the 3 crawls used to build the web graphs. More details about the issue are given in cc-pyspark#15. The impact of this fix on the graph size is minimal: the recent crawl now includes 1 million nodes (0.1% of all nodes) which are not connected to any other node.
Host-level graph
The graph consists of 927 million nodes and 3.88 billion edges and includes dangling nodes i.e. hosts that have not been crawled yet are pointed to from a link on a crawled page. There are 857 million dangling nodes (92.5%) and the largest strongly connected component contains 47 million (5.1%) nodes.
You can download the graph and the ranks of all 927 million hosts from AWS S3 on the path s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/host/
. Alternatively, you can use https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/host/
as prefix to access the files from everywhere.
Download files of the Common Crawl Feb/Mar/May 2020 host-level webgraph
Size | File | Description |
---|---|---|
5.67 GB | cc-main-2020-feb-mar-may-host-vertices.paths.gz | nodes 〈id, rev host〉, paths of 12 vertices files |
17.26 GB | cc-main-2020-feb-mar-may-host-edges.paths.gz | edges 〈from_id, to_id〉, paths of 24 edges files |
7.40 GB | cc-main-2020-feb-mar-may-host.graph | graph in BVGraph format |
2 kB | cc-main-2020-feb-mar-may-host.properties | |
8.57 GB | cc-main-2020-feb-mar-may-host-t.graph | transpose of the graph (outlinks inverted to inlinks) |
2 kB | cc-main-2020-feb-mar-may-host-t.properties | |
1 kB | cc-main-2020-feb-mar-may-host.stats | WebGraph statistics |
12.16 GB | cc-main-2020-feb-mar-may-host-ranks.txt.gz | harmonic centrality and pagerank |
Note that the host names are reversed and a leading www.
is stripped: www.subdomain.example.com
becomes com.example.subdomain
.
Domain-level graph
The domain graph was built by aggregating the host graph on the level of pay-level domains (PLDs) based on the public suffix list maintained on publicsuffix.org.
The domain-level graph has 91 million nodes and 1.96 billion edges. 51% or 46 million nodes are dangling nodes, the largest strongly connected component covers 36 million or 39% of the nodes.
All files related to the domain graph are available on AWS S3 under s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/domain/
resp. https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/domain/
.
Download files of the Common Crawl Feb/Mar/May 2020 domain-level webgraph
Size | File | Description |
---|---|---|
0.62 GB | cc-main-2020-feb-mar-may-domain-vertices.txt.gz | nodes 〈id, rev domain, num hosts〉 |
7.79 GB | cc-main-2020-feb-mar-may-domain-edges.txt.gz | edges 〈from_id, to_id〉 |
4.23 GB | cc-main-2020-feb-mar-may-domain.graph | graph in BVGraph format |
2 kB | cc-main-2020-feb-mar-may-domain.properties | |
4.16 GB | cc-main-2020-feb-mar-may-domain-t.graph | transpose of the graph |
2 kB | cc-main-2020-feb-mar-may-domain-t.properties | |
1 kB | cc-main-2020-feb-mar-may-domain.stats | WebGraph statistics |
1.96 GB | cc-main-2020-feb-mar-may-domain-ranks.txt.gz | harmonic centrality and pagerank |
Below you’ll find the top 1000 domains ranked by Harmonic Centrality or PageRank. The full list of all 91 million domain ranks is available for download.
Top 1000 domains ranked by harmonic centrality (Feb/Mar/May 2020)
harmonic centrality rank | hc value | page rank | page rank value | reversed hostname |
---|---|---|---|---|
1 | 32667618 | 1 | 0.018180 | com.googleapis |
2 | 30552772 | 3 | 0.011873 | com.facebook |
3 | 29569088 | 2 | 0.013789 | com.google |
4 | 26920460 | 4 | 0.007145 | com.twitter |
5 | 26883128 | 5 | 0.007106 | org.w |
6 | 26360448 | 6 | 0.006483 | com.youtube |
7 | 24719396 | 9 | 0.004210 | com.instagram |
8 | 24251942 | 8 | 0.005125 | org.gmpg |
9 | 23841332 | 7 | 0.005329 | com.googletagmanager |
10 | 23606890 | 13 | 0.002940 | com.linkedin |
11 | 22741292 | 10 | 0.003621 | com.cloudflare |
12 | 22732960 | 12 | 0.002974 | org.wordpress |
13 | 22661910 | 14 | 0.002515 | com.gravatar |
14 | 22577680 | 15 | 0.002438 | com.gstatic |
15 | 22378134 | 22 | 0.001529 | com.pinterest |
16 | 22196962 | 27 | 0.001192 | org.wikipedia |
17 | 22189650 | 19 | 0.001864 | com.wordpress |
18 | 22066028 | 16 | 0.002404 | com.bootstrapcdn |
19 | 21967760 | 18 | 0.001884 | com.apple |
20 | 21751768 | 20 | 0.001863 | com.jquery |
21 | 21589606 | 24 | 0.001461 | com.microsoft |
22 | 21568908 | 44 | 0.000785 | be.youtu |
23 | 21568474 | 43 | 0.000806 | com.blogspot |
24 | 21533280 | 31 | 0.001104 | com.vimeo |
25 | 21415938 | 46 | 0.000761 | gl.goo |
26 | 21399120 | 35 | 0.001040 | com.amazonaws |
27 | 21358048 | 53 | 0.000665 | com.amazon |
28 | 21331634 | 21 | 0.001737 | com.adobe |
29 | 21324666 | 23 | 0.001506 | com.wp |
30 | 21209012 | 70 | 0.000452 | com.tumblr |
31 | 21184360 | 17 | 0.001949 | com.github |
32 | 21150652 | 37 | 0.001008 | com.google-analytics |
33 | 21110976 | 30 | 0.001152 | com.baidu |
34 | 21096692 | 87 | 0.000387 | com.yahoo |
35 | 21081268 | 59 | 0.000547 | ly.bit |
36 | 21060360 | 33 | 0.001072 | com.macromedia |
37 | 21046916 | 36 | 0.001035 | net.cloudfront |
38 | 21036258 | 45 | 0.000763 | com.flickr |
39 | 20997926 | 32 | 0.001101 | com.googlesyndication |
40 | 20993476 | 26 | 0.001277 | me.wp |
41 | 20980462 | 97 | 0.000340 | com.googleusercontent |
42 | 20966446 | 56 | 0.000624 | eu.europa |
43 | 20960242 | 42 | 0.000807 | net.jsdelivr |
44 | 20959910 | 52 | 0.000677 | co.t |
45 | 20901872 | 29 | 0.001163 | ru.yandex |
46 | 20846092 | 50 | 0.000742 | net.doubleclick |
47 | 20843032 | 41 | 0.000869 | com.addthis |
48 | 20823518 | 69 | 0.000457 | io.github |
49 | 20817952 | 76 | 0.000433 | com.medium |
50 | 20810030 | 25 | 0.001287 | com.fontawesome |
51 | 20809120 | 139 | 0.000189 | com.forbes |
52 | 20796434 | 61 | 0.000510 | org.w3 |
53 | 20759102 | 55 | 0.000640 | com.paypal |
54 | 20757266 | 109 | 0.000282 | com.soundcloud |
55 | 20754514 | 90 | 0.000368 | org.creativecommons |
56 | 20747472 | 57 | 0.000619 | com.vk |
57 | 20711184 | 54 | 0.000658 | org.mozilla |
58 | 20710182 | 88 | 0.000382 | com.weebly |
59 | 20698442 | 84 | 0.000410 | com.wix |
60 | 20675372 | 102 | 0.000317 | com.weibo |
61 | 20663930 | 58 | 0.000604 | org.schema |
62 | 20650202 | 164 | 0.000151 | com.imgur |
63 | 20644452 | 147 | 0.000177 | org.apache |
64 | 20642282 | 178 | 0.000138 | uk.co.bbc |
65 | 20625560 | 129 | 0.000210 | org.archive |
66 | 20610354 | 274 | 0.000089 | com.ibm |
67 | 20609614 | 154 | 0.000169 | com.bing |
68 | 20602380 | 191 | 0.000125 | net.sourceforge |
69 | 20579012 | 130 | 0.000207 | com.nytimes |
70 | 20578626 | 150 | 0.000174 | int.who |
71 | 20571012 | 183 | 0.000131 | com.cnn |
72 | 20561674 | 174 | 0.000140 | net.slideshare |
73 | 20547634 | 158 | 0.000164 | gov.cdc |
74 | 20542546 | 202 | 0.000116 | com.android |
75 | 20527230 | 228 | 0.000104 | com.wsj |
76 | 20518548 | 194 | 0.000122 | edu.stanford |
77 | 20505546 | 205 | 0.000115 | com.businessinsider |
78 | 20495034 | 254 | 0.000095 | com.oracle |
79 | 20489434 | 34 | 0.001049 | net.fbcdn |
80 | 20488868 | 373 | 0.000067 | com.msn |
81 | 20488282 | 261 | 0.000093 | edu.harvard |
82 | 20483384 | 310 | 0.000080 | com.go |
83 | 20478152 | 99 | 0.000335 | com.shopify |
84 | 20471424 | 267 | 0.000093 | com.bbc |
85 | 20464434 | 297 | 0.000083 | edu.mit |
86 | 20461340 | 330 | 0.000076 | com.myspace |
87 | 20458776 | 62 | 0.000497 | com.whatsapp |
88 | 20457206 | 289 | 0.000085 | com.appspot |
89 | 20454466 | 307 | 0.000080 | com.wired |
90 | 20446300 | 292 | 0.000085 | com.reuters |
91 | 20442004 | 101 | 0.000323 | com.godaddy |
92 | 20435550 | 171 | 0.000147 | com.theguardian |
93 | 20417770 | 143 | 0.000182 | gov.nih |
94 | 20412536 | 196 | 0.000120 | org.ietf |
95 | 20401330 | 388 | 0.000065 | gov.nasa |
96 | 20397298 | 423 | 0.000061 | com.theverge |
97 | 20394736 | 149 | 0.000175 | com.giphy |
98 | 20394276 | 382 | 0.000066 | net.researchgate |
99 | 20384930 | 270 | 0.000092 | com.bloomberg |
100 | 20377778 | 108 | 0.000285 | com.unpkg |
101 | 20376394 | 114 | 0.000271 | com.reddit |
102 | 20373856 | 337 | 0.000075 | com.xinhuanet |
103 | 20366736 | 215 | 0.000108 | org.gnu |
104 | 20363506 | 318 | 0.000079 | com.usatoday |
105 | 20352660 | 813 | 0.000037 | org.chromium |
106 | 20344996 | 356 | 0.000071 | com.springer |
107 | 20343678 | 98 | 0.000335 | de.google |
108 | 20342420 | 28 | 0.001184 | com.qq |
109 | 20341824 | 345 | 0.000073 | com.example |
110 | 20336510 | 744 | 0.000041 | edu.psu |
111 | 20324536 | 468 | 0.000055 | edu.cornell |
112 | 20324378 | 184 | 0.000131 | com.blogger |
113 | 20314024 | 60 | 0.000516 | net.akamaihd |
114 | 20304242 | 375 | 0.000067 | org.hbr |
115 | 20302310 | 750 | 0.000040 | com.git-scm |
116 | 20300014 | 937 | 0.000032 | com.wikia |
117 | 20298546 | 137 | 0.000191 | com.spotify |
118 | 20296012 | 485 | 0.000053 | edu.yale |
119 | 20295516 | 113 | 0.000271 | com.jimdo |
120 | 20293140 | 554 | 0.000047 | com.cbsnews |
121 | 20291946 | 717 | 0.000043 | com.economist |
122 | 20290574 | 214 | 0.000109 | com.washingtonpost |
123 | 20288504 | 140 | 0.000188 | jp.co.yahoo |
124 | 20286470 | 285 | 0.000086 | com.huffingtonpost |
125 | 20284558 | 316 | 0.000080 | org.un |
126 | 20281874 | 410 | 0.000063 | fr.free |
127 | 20279946 | 473 | 0.000054 | edu.berkeley |
128 | 20275446 | 287 | 0.000086 | com.cnbc |
129 | 20273280 | 245 | 0.000099 | com.dribbble |
130 | 20271584 | 576 | 0.000046 | org.arxiv |
131 | 20269716 | 151 | 0.000172 | com.issuu |
132 | 20257038 | 545 | 0.000047 | com.mysql |
133 | 20256262 | 160 | 0.000157 | com.twimg |
134 | 20252532 | 107 | 0.000285 | com.statcounter |
135 | 20251682 | 338 | 0.000075 | uk.co.telegraph |
136 | 20247478 | 305 | 0.000081 | com.w3schools |
137 | 20246682 | 561 | 0.000047 | com.gitlab |
138 | 20242210 | 802 | 0.000038 | edu.columbia |
139 | 20240978 | 524 | 0.000049 | gov.noaa |
140 | 20230666 | 122 | 0.000230 | com.ytimg |
141 | 20229900 | 119 | 0.000233 | com.youtube-nocookie |
142 | 20227656 | 731 | 0.000042 | org.ieee |
143 | 20227126 | 333 | 0.000075 | org.npr |
144 | 20225528 | 729 | 0.000042 | io.readthedocs |
145 | 20225206 | 286 | 0.000086 | org.acm |
146 | 20222314 | 339 | 0.000074 | com.time |
147 | 20220430 | 1180 | 0.000025 | org.eclipse |
148 | 20220382 | 241 | 0.000100 | org.ampproject |
149 | 20218616 | 344 | 0.000074 | com.fc2 |
150 | 20215730 | 142 | 0.000185 | com.wixsite |
151 | 20213692 | 755 | 0.000040 | edu.washington |
152 | 20210122 | 421 | 0.000061 | com.force |
153 | 20209864 | 276 | 0.000089 | com.prnewswire |
154 | 20209130 | 500 | 0.000052 | com.buzzfeed |
155 | 20207136 | 434 | 0.000060 | com.nationalgeographic |
156 | 20206402 | 403 | 0.000063 | com.nature |
157 | 20203826 | 200 | 0.000118 | gle.forms |
158 | 20202490 | 799 | 0.000038 | org.sciencemag |
159 | 20201144 | 428 | 0.000061 | com.theatlantic |
160 | 20200104 | 871 | 0.000035 | com.stackexchange |
161 | 20198142 | 280 | 0.000088 | com.sciencedirect |
162 | 20185400 | 332 | 0.000075 | com.staticflickr |
163 | 20184528 | 495 | 0.000052 | uk.co.independent |
164 | 20182256 | 263 | 0.000093 | gov.ca |
165 | 20180972 | 687 | 0.000043 | org.worldbank |
166 | 20175994 | 435 | 0.000060 | com.mozilla |
167 | 20175400 | 734 | 0.000041 | com.marketwatch |
168 | 20168098 | 1087 | 0.000027 | com.hatenablog |
169 | 20167040 | 364 | 0.000069 | com.nypost |
170 | 20164016 | 646 | 0.000043 | org.bitbucket |
171 | 20161192 | 219 | 0.000107 | com.ft |
172 | 20151116 | 463 | 0.000056 | com.pixabay |
173 | 20143796 | 354 | 0.000071 | jp.co.rakuten |
174 | 20142652 | 743 | 0.000041 | edu.upenn |
175 | 20140126 | 277 | 0.000089 | org.doi |
176 | 20139376 | 966 | 0.000031 | jp.livedoor |
177 | 20136546 | 198 | 0.000120 | uk.co.google |
178 | 20134932 | 407 | 0.000063 | uk.co.dailymail |
179 | 20134404 | 724 | 0.000042 | org.pbs |
180 | 20133936 | 258 | 0.000094 | net.behance |
181 | 20132914 | 192 | 0.000124 | org.wikimedia |
182 | 20127860 | 917 | 0.000033 | edu.jhu |
183 | 20127828 | 454 | 0.000057 | gov.whitehouse |
184 | 20122352 | 856 | 0.000035 | org.weforum |
185 | 20122170 | 416 | 0.000062 | com.dailymotion |
186 | 20117054 | 1487 | 0.000020 | com.warnerbros |
187 | 20111898 | 326 | 0.000077 | org.opensource |
188 | 20110798 | 1091 | 0.000027 | cn.com.chinadaily |
189 | 20109916 | 548 | 0.000047 | me.about |
190 | 20109820 | 232 | 0.000103 | jp.ameblo |
191 | 20108940 | 558 | 0.000047 | com.oup |
192 | 20103428 | 325 | 0.000077 | com.digg |
193 | 20097418 | 455 | 0.000056 | com.entrepreneur |
194 | 20095108 | 631 | 0.000044 | com.vice |
195 | 20094142 | 749 | 0.000040 | com.qz |
196 | 20092692 | 1259 | 0.000024 | com.discovery |
197 | 20091154 | 444 | 0.000058 | com.goodreads |
198 | 20091052 | 447 | 0.000057 | gg.discord |
199 | 20082910 | 1109 | 0.000027 | com.sap |
200 | 20082186 | 353 | 0.000071 | com.scribd |
201 | 20079412 | 188 | 0.000128 | com.feedburner |
202 | 20076146 | 466 | 0.000055 | com.fortune |
203 | 20075556 | 580 | 0.000045 | com.gartner |
204 | 20072598 | 1012 | 0.000029 | com.500px |
205 | 20072136 | 458 | 0.000056 | jp.ne.sakura |
206 | 20067400 | 176 | 0.000139 | com.imdb |
207 | 20060950 | 732 | 0.000042 | uk.co.blogspot |
208 | 20059054 | 1735 | 0.000018 | com.amd |
209 | 20058228 | 947 | 0.000032 | edu.princeton |
210 | 20056666 | 890 | 0.000034 | org.cambridge |
211 | 20056572 | 51 | 0.000714 | com.fb |
212 | 20056272 | 848 | 0.000036 | com.evernote |
213 | 20054472 | 144 | 0.000180 | com.dropbox |
214 | 20053532 | 39 | 0.000951 | com.wixstatic |
215 | 20051662 | 617 | 0.000044 | org.unesco |
216 | 20050940 | 1461 | 0.000020 | com.fandom |
217 | 20048152 | 294 | 0.000084 | com.wiley |
218 | 20046134 | 768 | 0.000039 | com.withgoogle |
219 | 20039426 | 1015 | 0.000029 | org.altervista |
220 | 20039010 | 2337 | 0.000014 | com.wolfram |
221 | 20037920 | 798 | 0.000038 | com.slate |
222 | 20031484 | 1201 | 0.000025 | org.kernel |
223 | 20028164 | 1049 | 0.000028 | edu.purdue |
224 | 20025282 | 569 | 0.000046 | page.g |
225 | 20021340 | 786 | 0.000038 | com.trello |
226 | 20017018 | 230 | 0.000103 | com.disqus |
227 | 20012796 | 757 | 0.000040 | org.eff |
228 | 20010430 | 951 | 0.000031 | com.merriam-webster |
229 | 20004686 | 493 | 0.000052 | gov.usda |
230 | 20004240 | 981 | 0.000030 | com.netlify |
231 | 20003994 | 2179 | 0.000015 | com.diigo |
232 | 20002918 | 807 | 0.000038 | com.vox |
233 | 20002690 | 180 | 0.000135 | org.allaboutcookies |
234 | 20002220 | 1206 | 0.000025 | com.jetbrains |
235 | 19999418 | 1416 | 0.000021 | edu.arizona |
236 | 19994384 | 542 | 0.000047 | com.tandfonline |
237 | 19993030 | 844 | 0.000036 | com.foxnews |
238 | 19992184 | 291 | 0.000085 | com.live |
239 | 19991142 | 175 | 0.000140 | com.xing |
240 | 19989874 | 909 | 0.000033 | com.politico |
241 | 19988570 | 320 | 0.000079 | com.outlook |
242 | 19985036 | 1135 | 0.000026 | jp.ne.goo |
243 | 19983340 | 754 | 0.000040 | au.net.abc |
244 | 19982680 | 1945 | 0.000016 | com.wikidot |
245 | 19977934 | 793 | 0.000038 | com.investopedia |
246 | 19977574 | 1066 | 0.000028 | edu.uchicago |
247 | 19976820 | 1009 | 0.000029 | edu.wisc |
248 | 19975922 | 197 | 0.000120 | com.eepurl |
249 | 19972560 | 1039 | 0.000028 | com.bostonglobe |
250 | 19972096 | 775 | 0.000039 | org.semver |
251 | 19969594 | 619 | 0.000044 | com.sagepub |
252 | 19969182 | 497 | 0.000052 | gov.fda |
253 | 19968442 | 347 | 0.000073 | net.windows |
254 | 19968084 | 1568 | 0.000019 | edu.osu |
255 | 19965386 | 319 | 0.000079 | com.nbcnews |
256 | 19963946 | 244 | 0.000099 | com.myshopify |
257 | 19962892 | 585 | 0.000045 | cn.google |
258 | 19962530 | 608 | 0.000044 | site.business |
259 | 19961066 | 832 | 0.000036 | com.sciencedaily |
260 | 19960380 | 1044 | 0.000028 | com.strikingly |
261 | 19956366 | 1236 | 0.000024 | edu.unc |
262 | 19956268 | 1446 | 0.000021 | edu.virginia |
263 | 19956034 | 1204 | 0.000025 | co.elastic |
264 | 19952960 | 1194 | 0.000025 | com.nymag |
265 | 19950500 | 2206 | 0.000015 | com.renren |
266 | 19950490 | 742 | 0.000041 | gov.house |
267 | 19950448 | 2163 | 0.000015 | sg.edu.nus |
268 | 19947976 | 2285 | 0.000014 | org.wikibooks |
269 | 19947284 | 1961 | 0.000016 | com.googlesource |
270 | 19940598 | 235 | 0.000103 | com.wpengine |
271 | 19940158 | 323 | 0.000078 | com.googlecode |
272 | 19939212 | 761 | 0.000040 | gov.senate |
273 | 19938008 | 513 | 0.000051 | com.herokuapp |
274 | 19937738 | 452 | 0.000057 | org.pewresearch |
275 | 19937492 | 567 | 0.000046 | org.iana |
276 | 19936954 | 1093 | 0.000027 | com.podbean |
277 | 19935818 | 982 | 0.000030 | com.alexa |
278 | 19934742 | 1629 | 0.000019 | gd.is |
279 | 19933804 | 103 | 0.000301 | com.paypalobjects |
280 | 19932740 | 805 | 0.000038 | org.unicef |
281 | 19932416 | 718 | 0.000043 | com.newyorker |
282 | 19930858 | 969 | 0.000031 | uk.co.thetimes |
283 | 19929324 | 404 | 0.000063 | com.patreon |
284 | 19928266 | 1060 | 0.000028 | com.lifehacker |
285 | 19925940 | 381 | 0.000066 | com.criteo |
286 | 19924524 | 997 | 0.000030 | com.huffpost |
287 | 19922576 | 303 | 0.000081 | com.squareup |
288 | 19922510 | 839 | 0.000036 | ca.cbc |
289 | 19921808 | 1145 | 0.000026 | org.wiktionary |
290 | 19918844 | 146 | 0.000178 | com.addtoany |
291 | 19918174 | 201 | 0.000117 | com.optimizely |
292 | 19918052 | 1342 | 0.000022 | edu.msu |
293 | 19915986 | 1371 | 0.000022 | com.history |
294 | 19913384 | 418 | 0.000062 | com.calendly |
295 | 19905860 | 1181 | 0.000025 | com.udemy |
296 | 19903364 | 809 | 0.000037 | uk.ac.ox |
297 | 19902920 | 172 | 0.000145 | com.amazon-adsystem |
298 | 19899332 | 49 | 0.000743 | com.googleadservices |
299 | 19896924 | 155 | 0.000167 | com.opera |
300 | 19890970 | 887 | 0.000034 | org.fao |
301 | 19890832 | 1017 | 0.000029 | com.ecwid |
302 | 19890826 | 476 | 0.000054 | com.googleblog |
303 | 19887142 | 211 | 0.000110 | com.stackoverflow |
304 | 19886190 | 1419 | 0.000021 | uk.ac.lse |
305 | 19885312 | 360 | 0.000070 | com.getpocket |
306 | 19884456 | 1667 | 0.000018 | org.maven |
307 | 19883800 | 915 | 0.000033 | uk.co.guardian |
308 | 19883358 | 169 | 0.000148 | org.bbb |
309 | 19881084 | 1337 | 0.000022 | com.aljazeera |
310 | 19880790 | 255 | 0.000095 | com.aliyuncs |
311 | 19879938 | 2723 | 0.000013 | net.pixnet |
312 | 19874384 | 3180 | 0.000011 | net.hinet |
313 | 19869028 | 1170 | 0.000025 | com.smithsonianmag |
314 | 19868832 | 1347 | 0.000022 | edu.ucdavis |
315 | 19868258 | 894 | 0.000034 | gov.congress |
316 | 19867190 | 1320 | 0.000023 | edu.illinois |
317 | 19865168 | 1120 | 0.000026 | com.theglobeandmail |
318 | 19863306 | 1036 | 0.000029 | gov.archives |
319 | 19862414 | 492 | 0.000052 | it.placehold |
320 | 19861934 | 93 | 0.000359 | net.facebook |
321 | 19861376 | 1615 | 0.000019 | hk.com.google |
322 | 19860922 | 1473 | 0.000020 | ca.sfu |
323 | 19856352 | 1676 | 0.000018 | blog.home |
324 | 19855290 | 1073 | 0.000027 | com.apnews |
325 | 19854892 | 963 | 0.000031 | com.ssrn |
326 | 19853682 | 3383 | 0.000010 | com.wizards |
327 | 19851102 | 1997 | 0.000016 | com.nabble |
328 | 19851032 | 760 | 0.000040 | com.chinaz |
329 | 19850412 | 3667 | 0.000010 | cn.edu.sjtu |
330 | 19848140 | 1484 | 0.000020 | com.urbandictionary |
331 | 19844436 | 1136 | 0.000026 | com.scmp |
332 | 19842326 | 1489 | 0.000020 | ms.1drv |
333 | 19841796 | 4361 | 0.000008 | tw.com.gamer |
334 | 19838582 | 1392 | 0.000021 | com.flipboard |
335 | 19838166 | 919 | 0.000033 | co.g |
336 | 19837542 | 547 | 0.000047 | com.gofundme |
337 | 19836996 | 2097 | 0.000015 | com.france24 |
338 | 19835636 | 1405 | 0.000021 | jp.geocities |
339 | 19833654 | 1370 | 0.000022 | com.ibtimes |
340 | 19831362 | 581 | 0.000045 | com.biomedcentral |
341 | 19830056 | 1128 | 0.000026 | com.britannica |
342 | 19829420 | 2174 | 0.000015 | com.oregonlive |
343 | 19827062 | 412 | 0.000062 | com.kickstarter |
344 | 19826214 | 962 | 0.000031 | com.adjust |
345 | 19824188 | 867 | 0.000035 | gov.fcc |
346 | 19824048 | 715 | 0.000043 | uk.co.mirror |
347 | 19823266 | 589 | 0.000045 | us.icio |
348 | 19823172 | 1129 | 0.000026 | com.mediafire |
349 | 19821768 | 1432 | 0.000021 | edu.tamu |
350 | 19821310 | 587 | 0.000045 | com.usnews |
351 | 19820442 | 1314 | 0.000023 | org.greenpeace |
352 | 19820252 | 985 | 0.000030 | edu.academia |
353 | 19819486 | 1381 | 0.000021 | com.livescience |
354 | 19815972 | 1684 | 0.000018 | gov.cia |
355 | 19814564 | 1325 | 0.000023 | com.akamai |
356 | 19813266 | 930 | 0.000032 | com.chicagotribune |
357 | 19811538 | 156 | 0.000167 | com.npmjs |
358 | 19811100 | 1429 | 0.000021 | net.seesaa |
359 | 19810120 | 329 | 0.000076 | es.google |
360 | 19809710 | 1238 | 0.000024 | com.reverbnation |
361 | 19809490 | 550 | 0.000047 | com.quora |
362 | 19808314 | 3481 | 0.000010 | com.proboards |
363 | 19806268 | 1040 | 0.000028 | com.thehill |
364 | 19803840 | 321 | 0.000078 | org.python |
365 | 19801476 | 1132 | 0.000026 | org.jstor |
366 | 19801018 | 1722 | 0.000018 | ca.mcgill |
367 | 19799982 | 167 | 0.000149 | com.zendesk |
368 | 19792890 | 999 | 0.000030 | com.thelancet |
369 | 19792246 | 1094 | 0.000027 | com.jamanetwork |
370 | 19788594 | 1935 | 0.000016 | uk.ac.manchester |
371 | 19785214 | 540 | 0.000048 | com.udacity |
372 | 19783328 | 1372 | 0.000021 | ca.utoronto |
373 | 19783082 | 579 | 0.000046 | com.bigcartel |
374 | 19782230 | 2487 | 0.000013 | org.wikiquote |
375 | 19781186 | 1357 | 0.000022 | edu.rutgers |
376 | 19780028 | 896 | 0.000034 | org.apa |
377 | 19779718 | 439 | 0.000059 | com.newsweek |
378 | 19778538 | 920 | 0.000033 | com.healthline |
379 | 19777982 | 2204 | 0.000015 | com.knowyourmeme |
380 | 19775610 | 328 | 0.000077 | com.tinyurl |
381 | 19775558 | 726 | 0.000042 | gov.state |
382 | 19775092 | 216 | 0.000108 | com.unsplash |
383 | 19773702 | 1708 | 0.000018 | ca.ualberta |
384 | 19772378 | 406 | 0.000063 | com.githubusercontent |
385 | 19771900 | 1471 | 0.000020 | com.asahi |
386 | 19771220 | 259 | 0.000094 | org.nodejs |
387 | 19769436 | 475 | 0.000054 | com.latimes |
388 | 19769258 | 1027 | 0.000029 | com.timeanddate |
389 | 19768686 | 432 | 0.000060 | com.slack |
390 | 19768410 | 769 | 0.000039 | jp.shinobi |
391 | 19767976 | 1674 | 0.000018 | com.buzzfeednews |
392 | 19765038 | 415 | 0.000062 | com.elsevier |
393 | 19764722 | 1335 | 0.000022 | edu.gatech |
394 | 19764298 | 2861 | 0.000012 | com.youdao |
395 | 19761256 | 895 | 0.000034 | com.brightcove |
396 | 19759730 | 1774 | 0.000017 | com.bankofamerica |
397 | 19759530 | 2569 | 0.000013 | edu.byu |
398 | 19758760 | 1918 | 0.000016 | com.voanews |
399 | 19757586 | 3164 | 0.000011 | com.opendns |
400 | 19756816 | 1425 | 0.000021 | com.sky |
401 | 19755780 | 2336 | 0.000014 | com.slides |
402 | 19754462 | 1373 | 0.000021 | com.dw |
403 | 19754458 | 1158 | 0.000026 | com.nikkei |
404 | 19752590 | 904 | 0.000033 | com.cbslocal |
405 | 19748766 | 2236 | 0.000014 | net.earthlink |
406 | 19748678 | 391 | 0.000064 | com.cnet |
407 | 19748150 | 1642 | 0.000018 | com.xrea |
408 | 19747430 | 1354 | 0.000022 | uk.co.huffingtonpost |
409 | 19746424 | 182 | 0.000133 | com.eventbrite |
410 | 19746370 | 1071 | 0.000027 | com.nydailynews |
411 | 19744090 | 1305 | 0.000023 | me.vk |
412 | 19743194 | 918 | 0.000033 | gov.bls |
413 | 19741542 | 1458 | 0.000020 | org.ap |
414 | 19740936 | 384 | 0.000066 | net.imgix |
415 | 19739860 | 2414 | 0.000014 | org.aclweb |
416 | 19739750 | 1641 | 0.000018 | com.axios |
417 | 19738940 | 987 | 0.000030 | com.wattpad |
418 | 19737530 | 1713 | 0.000018 | com.straitstimes |
419 | 19737412 | 474 | 0.000054 | com.ted |
420 | 19736874 | 1294 | 0.000023 | edu.brookings |
421 | 19728634 | 967 | 0.000031 | int.coe |
422 | 19727580 | 212 | 0.000109 | com.etsy |
423 | 19727112 | 2392 | 0.000014 | com.biography |
424 | 19726080 | 865 | 0.000035 | gov.va |
425 | 19725710 | 217 | 0.000107 | com.typepad |
426 | 19724628 | 1932 | 0.000016 | com.cocolog-nifty |
427 | 19723580 | 1608 | 0.000019 | com.reference |
428 | 19720740 | 553 | 0.000047 | com.livejournal |
429 | 19717406 | 2096 | 0.000015 | ru.kremlin |
430 | 19716354 | 815 | 0.000037 | uk.gov.service |
431 | 19715378 | 298 | 0.000083 | com.techcrunch |
432 | 19712358 | 2462 | 0.000013 | org.wikisource |
433 | 19712296 | 1553 | 0.000019 | com.foxbusiness |
434 | 19711620 | 1281 | 0.000023 | mil.army |
435 | 19711244 | 1761 | 0.000017 | com.itv |
436 | 19710260 | 733 | 0.000041 | com.deviantart |
437 | 19705952 | 1311 | 0.000023 | de.mpg |
438 | 19705288 | 845 | 0.000036 | gov.justice |
439 | 19704574 | 1993 | 0.000016 | cn.people |
440 | 19703248 | 1262 | 0.000024 | au.com.smh |
441 | 19701656 | 1763 | 0.000017 | org.tensorflow |
442 | 19701634 | 1223 | 0.000024 | org.ohchr |
443 | 19701000 | 568 | 0.000046 | ru.gov |
444 | 19700136 | 400 | 0.000064 | com.technorati |
445 | 19699596 | 2134 | 0.000015 | jp.co.japantimes |
446 | 19697954 | 83 | 0.000413 | com.list-manage |
447 | 19697088 | 1068 | 0.000028 | com.thedrum |
448 | 19696754 | 1538 | 0.000019 | uk.co.standard |
449 | 19695430 | 185 | 0.000131 | com.rawgit |
450 | 19694216 | 2120 | 0.000015 | com.oxforddictionaries |
451 | 19693006 | 2241 | 0.000014 | com.shutterfly |
452 | 19692082 | 3147 | 0.000011 | tw.edu.ntu |
453 | 19691564 | 2550 | 0.000013 | com.smashwords |
454 | 19689862 | 1862 | 0.000016 | edu.unl |
455 | 19688768 | 2402 | 0.000014 | org.fas |
456 | 19688646 | 296 | 0.000084 | uk.org.ico |
457 | 19688138 | 2710 | 0.000013 | tv.blip |
458 | 19686066 | 957 | 0.000031 | com.bandsintown |
459 | 19684448 | 3516 | 0.000010 | cn.org.china |
460 | 19682960 | 1550 | 0.000019 | uk.co.express |
461 | 19679708 | 1082 | 0.000027 | jp.jugem |
462 | 19679158 | 3656 | 0.000010 | info.webry |
463 | 19678730 | 1403 | 0.000021 | gov.uscourts |
464 | 19677944 | 2157 | 0.000015 | au.edu.unimelb |
465 | 19675766 | 92 | 0.000363 | com.wsimg |
466 | 19674868 | 283 | 0.000086 | ru.rambler |
467 | 19673738 | 1921 | 0.000016 | com.washingtontimes |
468 | 19671754 | 351 | 0.000072 | com.proofpoint |
469 | 19669412 | 74 | 0.000441 | net.jsfiddle |
470 | 19668352 | 788 | 0.000038 | org.mediawiki |
471 | 19668158 | 2851 | 0.000012 | jp.blog |
472 | 19667740 | 1479 | 0.000020 | com.firebaseapp |
473 | 19667418 | 1618 | 0.000019 | com.webnode |
474 | 19665940 | 2173 | 0.000015 | com.pbworks |
475 | 19665748 | 3374 | 0.000011 | com.patheos |
476 | 19665684 | 3135 | 0.000011 | uk.co.timesonline |
477 | 19663980 | 2171 | 0.000015 | google.ai |
478 | 19663354 | 233 | 0.000103 | com.squarespace |
479 | 19662188 | 2904 | 0.000012 | fr.rfi |
480 | 19660984 | 1454 | 0.000020 | gov.supremecourt |
481 | 19659200 | 1889 | 0.000016 | int.unfccc |
482 | 19658534 | 331 | 0.000076 | com.office |
483 | 19656526 | 577 | 0.000046 | pl.google |
484 | 19654098 | 991 | 0.000030 | gov.wa |
485 | 19652796 | 804 | 0.000038 | gov.sba |
486 | 19652626 | 1267 | 0.000023 | com.cognitoforms |
487 | 19650066 | 2207 | 0.000015 | org.csis |
488 | 19649008 | 366 | 0.000068 | io.codepen |
489 | 19648750 | 2344 | 0.000014 | com.kobo |
490 | 19646512 | 110 | 0.000281 | com.mailchimp |
491 | 19643428 | 1671 | 0.000018 | edu.wustl |
492 | 19642572 | 2734 | 0.000013 | edu.kit |
493 | 19642334 | 1480 | 0.000020 | org.hrw |
494 | 19642276 | 953 | 0.000031 | edu.umich |
495 | 19641856 | 1389 | 0.000021 | com.dictionary |
496 | 19641544 | 836 | 0.000036 | com.mapquest |
497 | 19640836 | 1747 | 0.000017 | org.worldcat |
498 | 19640276 | 3621 | 0.000010 | net.aljazeera |
499 | 19640144 | 357 | 0.000071 | com.photobucket |
500 | 19639948 | 2046 | 0.000015 | net.cnki |
501 | 19638510 | 1705 | 0.000018 | com.secondlife |
502 | 19638416 | 2421 | 0.000014 | int.wmo |
503 | 19637888 | 1089 | 0.000027 | org.ilo |
504 | 19637450 | 1100 | 0.000027 | google.blog |
505 | 19636692 | 378 | 0.000067 | com.meetup |
506 | 19634634 | 995 | 0.000030 | uk.co.pinterest |
507 | 19633770 | 3397 | 0.000010 | com.freehostia |
508 | 19630412 | 3256 | 0.000011 | com.doodlekit |
509 | 19629746 | 936 | 0.000032 | com.arstechnica |
510 | 19628370 | 3730 | 0.000009 | com.colourlovers |
511 | 19628356 | 1696 | 0.000018 | ru.ucoz |
512 | 19628298 | 952 | 0.000031 | com.thenextweb |
513 | 19624458 | 2286 | 0.000014 | org.unep |
514 | 19622342 | 2252 | 0.000014 | org.icrc |
515 | 19621808 | 1424 | 0.000021 | com.findlaw |
516 | 19621134 | 2334 | 0.000014 | com.similarweb |
517 | 19620696 | 481 | 0.000054 | com.gmail |
518 | 19619304 | 3040 | 0.000012 | io.soup |
519 | 19616246 | 1437 | 0.000021 | com.imageshack |
520 | 19615956 | 2785 | 0.000013 | com.sputniknews |
521 | 19614078 | 3080 | 0.000012 | com.smore |
522 | 19613232 | 3246 | 0.000011 | org.iucnredlist |
523 | 19611766 | 3117 | 0.000011 | com.kinja |
524 | 19611760 | 1883 | 0.000016 | com.csmonitor |
525 | 19611604 | 145 | 0.000180 | ru.mail |
526 | 19610088 | 1339 | 0.000022 | gov.uscis |
527 | 19608554 | 446 | 0.000058 | net.secureservercdn |
528 | 19606314 | 3004 | 0.000012 | sh.now |
529 | 19605748 | 427 | 0.000061 | tv.twitch |
530 | 19604994 | 1580 | 0.000019 | link.app |
531 | 19600814 | 440 | 0.000059 | com.statista |
532 | 19599160 | 3676 | 0.000010 | jp.hatenablog |
533 | 19595550 | 4356 | 0.000008 | com.coroflot |
534 | 19595264 | 3177 | 0.000011 | org.jenkins-ci |
535 | 19595158 | 1757 | 0.000017 | gov.oregon |
536 | 19593130 | 3200 | 0.000011 | li.paper |
537 | 19593106 | 3847 | 0.000009 | com.pixar |
538 | 19589878 | 3095 | 0.000011 | com.shell |
539 | 19588194 | 4035 | 0.000009 | com.scienceblogs |
540 | 19586188 | 1625 | 0.000019 | org.amnesty |
541 | 19584824 | 892 | 0.000034 | com.thedailybeast |
542 | 19582464 | 1767 | 0.000017 | org.pypi |
543 | 19582346 | 2149 | 0.000015 | com.foreignpolicy |
544 | 19580310 | 2849 | 0.000012 | com.instapaper |
545 | 19579672 | 2910 | 0.000012 | org.accessnow |
546 | 19578614 | 1602 | 0.000019 | com.surveygizmo |
547 | 19577780 | 1733 | 0.000018 | ca.globalnews |
548 | 19576200 | 3175 | 0.000011 | de.uni-koeln |
549 | 19576198 | 239 | 0.000101 | io.shields |
550 | 19576184 | 3377 | 0.000011 | org.lds |
551 | 19575902 | 2238 | 0.000014 | org.rand |
552 | 19574790 | 207 | 0.000114 | com.salesforce |
553 | 19574544 | 3438 | 0.000010 | net.mootools |
554 | 19574428 | 2357 | 0.000014 | at.ac.univie |
555 | 19574182 | 4050 | 0.000009 | org.marxists |
556 | 19571664 | 2860 | 0.000012 | org.panda |
557 | 19571194 | 2806 | 0.000013 | com.oprah |
558 | 19568576 | 1874 | 0.000016 | com.justia |
559 | 19567970 | 3471 | 0.000010 | org.avaaz |
560 | 19567854 | 2880 | 0.000012 | com.openai |
561 | 19567764 | 3597 | 0.000010 | org.neocities |
562 | 19567260 | 3753 | 0.000009 | cn.edu.sdu |
563 | 19564960 | 762 | 0.000040 | com.netflix |
564 | 19564120 | 498 | 0.000052 | com.oreilly |
565 | 19563086 | 4405 | 0.000008 | com.yam |
566 | 19562248 | 227 | 0.000105 | uk.co.amazon |
567 | 19562204 | 866 | 0.000035 | com.zoho |
568 | 19560956 | 629 | 0.000044 | com.zdnet |
569 | 19559966 | 1298 | 0.000023 | ly.snip |
570 | 19558790 | 1790 | 0.000017 | ch.ipcc |
571 | 19558664 | 993 | 0.000030 | uk.parliament |
572 | 19558508 | 3787 | 0.000009 | com.nestle |
573 | 19556304 | 1254 | 0.000024 | se.google |
574 | 19556292 | 2997 | 0.000012 | com.treehugger |
575 | 19555184 | 1011 | 0.000029 | net.nocookie |
576 | 19555096 | 4644 | 0.000008 | com.x0 |
577 | 19553368 | 3631 | 0.000010 | org.tvtropes |
578 | 19550992 | 1141 | 0.000026 | org.sphinx-doc |
579 | 19549994 | 2122 | 0.000015 | ru.mos |
580 | 19548820 | 3044 | 0.000012 | es.csic |
581 | 19548530 | 2913 | 0.000012 | uk.gov.companieshouse |
582 | 19546576 | 1034 | 0.000029 | com.engadget |
583 | 19546230 | 1183 | 0.000025 | com.here |
584 | 19545492 | 5060 | 0.000007 | com.dbs |
585 | 19545438 | 4103 | 0.000009 | br.ufrj |
586 | 19544204 | 2159 | 0.000015 | edu.colostate |
587 | 19543398 | 2706 | 0.000013 | de.uni-heidelberg |
588 | 19540500 | 3059 | 0.000012 | com.pearltrees |
589 | 19539268 | 2176 | 0.000015 | net.openid |
590 | 19537880 | 2600 | 0.000013 | com.mystrikingly |
591 | 19537844 | 3880 | 0.000009 | com.chinatimes |
592 | 19535834 | 2400 | 0.000014 | link.page |
593 | 19534182 | 2354 | 0.000014 | com.real |
594 | 19533432 | 1836 | 0.000017 | org.ncsl |
595 | 19532288 | 301 | 0.000082 | com.surveymonkey |
596 | 19531930 | 362 | 0.000070 | com.hp |
597 | 19531412 | 1193 | 0.000025 | org.js |
598 | 19530700 | 2135 | 0.000015 | com.123formbuilder |
599 | 19528842 | 2426 | 0.000014 | org.vim |
600 | 19528104 | 3205 | 0.000011 | pl.wp |
601 | 19528018 | 2602 | 0.000013 | au.com.sbs |
602 | 19526780 | 170 | 0.000148 | com.yelp |
603 | 19526216 | 2499 | 0.000013 | uk.ac.kcl |
604 | 19524346 | 1338 | 0.000022 | org.aarp |
605 | 19523692 | 2621 | 0.000013 | th.co.google |
606 | 19523156 | 1006 | 0.000029 | uk.gov.legislation |
607 | 19523042 | 260 | 0.000094 | com.getbootstrap |
608 | 19522856 | 3663 | 0.000010 | com.magcloud |
609 | 19522274 | 3990 | 0.000009 | com.zynga |
610 | 19521942 | 1268 | 0.000023 | tw.com.google |
611 | 19521922 | 2829 | 0.000013 | com.kaggle |
612 | 19520130 | 948 | 0.000031 | gov.gpo |
613 | 19519742 | 946 | 0.000032 | com.about |
614 | 19519714 | 3273 | 0.000011 | org.rsf |
615 | 19518740 | 2976 | 0.000012 | org.tigris |
616 | 19518224 | 2727 | 0.000013 | uk.ac.leeds |
617 | 19515512 | 3535 | 0.000010 | de.dw |
618 | 19515434 | 3019 | 0.000012 | org.cfr |
619 | 19514574 | 3253 | 0.000011 | de.uni-freiburg |
620 | 19513570 | 3640 | 0.000010 | de.uni-konstanz |
621 | 19512714 | 3881 | 0.000009 | ua.at |
622 | 19511254 | 2117 | 0.000015 | info.worldometers |
623 | 19510314 | 4657 | 0.000008 | com.embarcadero |
624 | 19509370 | 2999 | 0.000012 | vn.zing |
625 | 19509134 | 3229 | 0.000011 | com.bangkokpost |
626 | 19508804 | 3615 | 0.000010 | ly.rebrand |
627 | 19508548 | 2008 | 0.000016 | gov.ky |
628 | 19508426 | 4009 | 0.000009 | org.wilsoncenter |
629 | 19506774 | 4059 | 0.000009 | jp.hatenadiary |
630 | 19506284 | 4374 | 0.000008 | com.musictoday |
631 | 19505388 | 3824 | 0.000009 | org.constitutioncenter |
632 | 19505186 | 372 | 0.000067 | com.booking |
633 | 19504402 | 2579 | 0.000013 | com.eiseverywhere |
634 | 19503800 | 4038 | 0.000009 | com.itsnicethat |
635 | 19503776 | 3331 | 0.000011 | il.ac.tau |
636 | 19502096 | 2359 | 0.000014 | mx.com.google |
637 | 19500806 | 3736 | 0.000009 | com.db |
638 | 19498928 | 312 | 0.000080 | com.ebay |
639 | 19498588 | 3578 | 0.000010 | jp.hateblo |
640 | 19498166 | 3348 | 0.000011 | org.democracynow |
641 | 19497296 | 3975 | 0.000009 | edu.odu |
642 | 19496812 | 2815 | 0.000013 | dk.au |
643 | 19496626 | 4220 | 0.000008 | com.etymonline |
644 | 19496184 | 2885 | 0.000012 | uk.gov.metoffice |
645 | 19495756 | 361 | 0.000070 | com.skype |
646 | 19495566 | 3570 | 0.000010 | com.hsbc |
647 | 19494844 | 2228 | 0.000015 | com.bankrate |
648 | 19494104 | 2240 | 0.000014 | gov.wi |
649 | 19493352 | 1815 | 0.000017 | fi.google |
650 | 19493306 | 4426 | 0.000008 | com.x10host |
651 | 19492136 | 3224 | 0.000011 | org.royalsociety |
652 | 19491096 | 817 | 0.000037 | com.pexels |
653 | 19490358 | 532 | 0.000048 | com.mashable |
654 | 19490282 | 4614 | 0.000008 | com.epochtimes |
655 | 19490018 | 1174 | 0.000025 | edu.ucla |
656 | 19489656 | 3226 | 0.000011 | cc.reurl |
657 | 19489414 | 3430 | 0.000010 | com.dailykos |
658 | 19489360 | 3742 | 0.000009 | uk.ac.uea |
659 | 19488050 | 3705 | 0.000010 | ca.shaw |
660 | 19486104 | 1968 | 0.000016 | uk.gov.tfl |
661 | 19485988 | 3434 | 0.000010 | uk.ac.nhm |
662 | 19485032 | 3060 | 0.000012 | com.ipage |
663 | 19484754 | 2498 | 0.000013 | com.prweek |
664 | 19484598 | 1819 | 0.000017 | gov.usembassy |
665 | 19483966 | 4861 | 0.000007 | am.do |
666 | 19483636 | 3086 | 0.000011 | com.viki |
667 | 19483518 | 3252 | 0.000011 | se.liu |
668 | 19482718 | 3066 | 0.000012 | com.coca-colacompany |
669 | 19482580 | 4232 | 0.000008 | br.ufrgs |
670 | 19482498 | 3639 | 0.000010 | de.uni-kiel |
671 | 19481340 | 1453 | 0.000020 | com.speakerdeck |
672 | 19480718 | 3077 | 0.000012 | net.openreview |
673 | 19480660 | 2208 | 0.000015 | de.auswaertiges-amt |
674 | 19480248 | 208 | 0.000113 | com.hubspot |
675 | 19479762 | 2026 | 0.000016 | com.lexisnexis |
676 | 19478700 | 2106 | 0.000015 | net.ucoz |
677 | 19477552 | 3494 | 0.000010 | com.iconarchive |
678 | 19477532 | 819 | 0.000037 | com.steampowered |
679 | 19477286 | 756 | 0.000040 | com.xiti |
680 | 19477132 | 2486 | 0.000013 | com.post-gazette |
681 | 19476898 | 3369 | 0.000011 | com.eklablog |
682 | 19476632 | 2937 | 0.000012 | uk.co.bbci |
683 | 19476378 | 1911 | 0.000016 | hu.google |
684 | 19476160 | 4399 | 0.000008 | com.jacobinmag |
685 | 19475974 | 3323 | 0.000011 | uk.ac.sussex |
686 | 19474368 | 3068 | 0.000012 | uk.ac.qmul |
687 | 19474212 | 3930 | 0.000009 | nf.co |
688 | 19473014 | 4114 | 0.000009 | com.collinsdictionary |
689 | 19472896 | 5215 | 0.000007 | com.evaair |
690 | 19472846 | 2572 | 0.000013 | com.marketwire |
691 | 19472580 | 3138 | 0.000011 | au.com.telstra |
692 | 19472114 | 3916 | 0.000009 | it.unitn |
693 | 19471646 | 898 | 0.000034 | com.visualstudio |
694 | 19471330 | 3807 | 0.000009 | in.ernet |
695 | 19470994 | 2906 | 0.000012 | nl.rug |
696 | 19468708 | 5297 | 0.000007 | org.arkive |
697 | 19468252 | 252 | 0.000096 | org.drupal |
698 | 19467050 | 3460 | 0.000010 | ca.dal |
699 | 19467046 | 3693 | 0.000010 | com.canada |
700 | 19465642 | 1451 | 0.000021 | com.tinypic |
701 | 19465304 | 3136 | 0.000011 | org.wri |
702 | 19465034 | 3698 | 0.000010 | com.la-croix |
703 | 19464108 | 4557 | 0.000008 | com.mitsubishielectric |
704 | 19463828 | 4748 | 0.000008 | com.gamejolt |
705 | 19462976 | 2789 | 0.000013 | gr.google |
706 | 19462882 | 4882 | 0.000007 | cz.webgarden |
707 | 19462404 | 3079 | 0.000012 | my.com.thestar |
708 | 19461830 | 269 | 0.000092 | net.php |
709 | 19461640 | 4329 | 0.000008 | au.gov.fairwork |
710 | 19460770 | 2279 | 0.000014 | co.pcdn |
711 | 19460176 | 3943 | 0.000009 | uk.ac.essex |
712 | 19459984 | 121 | 0.000231 | org.networkadvertising |
713 | 19459684 | 3396 | 0.000010 | org.rferl |
714 | 19459068 | 4211 | 0.000008 | com.sc |
715 | 19459020 | 3292 | 0.000011 | com.blogfa |
716 | 19458794 | 3382 | 0.000010 | ca.yelp |
717 | 19457580 | 4102 | 0.000009 | edu.utm |
718 | 19457248 | 5694 | 0.000007 | com.anghami |
719 | 19456532 | 5210 | 0.000007 | su.clan |
720 | 19456144 | 4095 | 0.000009 | it.justpaste |
721 | 19456006 | 414 | 0.000062 | com.sxsw |
722 | 19455914 | 3258 | 0.000011 | com.waterstones |
723 | 19454602 | 3960 | 0.000009 | com.jigsy |
724 | 19454516 | 838 | 0.000036 | com.intel |
725 | 19454394 | 4032 | 0.000009 | ee.ut |
726 | 19453242 | 916 | 0.000033 | com.docker |
727 | 19452988 | 738 | 0.000041 | com.samsung |
728 | 19451802 | 3422 | 0.000010 | es.ucm |
729 | 19450718 | 2503 | 0.000013 | com.washingtonexaminer |
730 | 19450342 | 3951 | 0.000009 | tl.page |
731 | 19450206 | 2209 | 0.000015 | org.wbur |
732 | 19449036 | 4112 | 0.000009 | site.negocio |
733 | 19448922 | 2773 | 0.000013 | com.yell |
734 | 19448516 | 3988 | 0.000009 | com.fatcow |
735 | 19448266 | 3282 | 0.000011 | pl.poznan |
736 | 19448198 | 135 | 0.000194 | com.youku |
737 | 19447930 | 2878 | 0.000012 | ae.thenational |
738 | 19447766 | 4705 | 0.000008 | id.co.kaskus |
739 | 19447668 | 3407 | 0.000010 | com.afp |
740 | 19447602 | 5336 | 0.000007 | net.manilatimes |
741 | 19446734 | 419 | 0.000062 | com.caniuse |
742 | 19446168 | 1470 | 0.000020 | com.pastebin |
743 | 19445910 | 3387 | 0.000010 | uk.org.rspb |
744 | 19445736 | 765 | 0.000039 | com.moz |
745 | 19444376 | 4027 | 0.000009 | lv.draugiem |
746 | 19441604 | 2508 | 0.000013 | gov.dni |
747 | 19440874 | 2593 | 0.000013 | ro.google |
748 | 19440144 | 2946 | 0.000012 | com.broadwayworld |
749 | 19439574 | 3750 | 0.000009 | ru.msu |
750 | 19439374 | 3766 | 0.000009 | pl.cba |
751 | 19439332 | 4137 | 0.000009 | org.rfa |
752 | 19439280 | 5562 | 0.000007 | org.bukkit |
753 | 19439086 | 2013 | 0.000016 | scot.gov |
754 | 19438868 | 133 | 0.000200 | com.constantcontact |
755 | 19438826 | 5638 | 0.000007 | org.adbusters |
756 | 19438094 | 4517 | 0.000008 | google.design |
757 | 19437654 | 4154 | 0.000008 | com.macobserver |
758 | 19437088 | 1649 | 0.000018 | fr.pagesjaunes |
759 | 19437020 | 2502 | 0.000013 | com.thenation |
760 | 19436776 | 3973 | 0.000009 | com.bbcamerica |
761 | 19434556 | 4857 | 0.000007 | com.orgfree |
762 | 19433810 | 2978 | 0.000012 | com.channelnewsasia |
763 | 19432506 | 735 | 0.000041 | gov.sec |
764 | 19432502 | 4008 | 0.000009 | com.teamspeak |
765 | 19432430 | 2800 | 0.000013 | org.gnupg |
766 | 19432260 | 3780 | 0.000009 | com.the-scientist |
767 | 19432252 | 3015 | 0.000012 | com.laweekly |
768 | 19431446 | 2921 | 0.000012 | au.edu.sydney |
769 | 19430084 | 3577 | 0.000010 | uk.co.yougov |
770 | 19430000 | 3140 | 0.000011 | vn.com.google |
771 | 19429942 | 4417 | 0.000008 | com.50webs |
772 | 19429004 | 3124 | 0.000011 | org.repec |
773 | 19428938 | 3215 | 0.000011 | org.ourworldindata |
774 | 19427890 | 3506 | 0.000010 | com.tradingeconomics |
775 | 19427352 | 3102 | 0.000011 | tw.com.pchome |
776 | 19426582 | 3332 | 0.000011 | com.monday |
777 | 19426556 | 3556 | 0.000010 | org.project-syndicate |
778 | 19425552 | 2331 | 0.000014 | com.amebaownd |
779 | 19424890 | 1596 | 0.000019 | org.whatbrowser |
780 | 19424750 | 1956 | 0.000016 | org.americanbar |
781 | 19424680 | 3739 | 0.000009 | ie.thejournal |
782 | 19424152 | 104 | 0.000298 | com.stripe |
783 | 19424140 | 4014 | 0.000009 | com.hatenadiary |
784 | 19424060 | 2933 | 0.000012 | org.thinkprogress |
785 | 19423712 | 3073 | 0.000012 | uk.gov.london |
786 | 19423054 | 3927 | 0.000009 | com.thesaurus |
787 | 19423006 | 3475 | 0.000010 | net.webself |
788 | 19422964 | 3432 | 0.000010 | io.pantheon |
789 | 19421712 | 3420 | 0.000010 | uk.ac.exeter |
790 | 19421508 | 4343 | 0.000008 | com.appledaily |
791 | 19421118 | 3528 | 0.000010 | com.bravesites |
792 | 19420816 | 5178 | 0.000007 | com.bambuser |
793 | 19420592 | 3379 | 0.000011 | com.foreignaffairs |
794 | 19419378 | 2432 | 0.000013 | com.instructables |
795 | 19416388 | 2185 | 0.000015 | vn.vietnamnet |
796 | 19414736 | 3994 | 0.000009 | com.webcindario |
797 | 19414328 | 2823 | 0.000013 | org.ewg |
798 | 19413934 | 4534 | 0.000008 | ws.nimb |
799 | 19413778 | 2833 | 0.000013 | org.fullfact |
800 | 19413352 | 256 | 0.000095 | us.zoom |
801 | 19412556 | 3685 | 0.000010 | com.encyclopedia |
802 | 19412474 | 3897 | 0.000009 | de.uni-erlangen |
803 | 19410822 | 5341 | 0.000007 | net.boards |
804 | 19409598 | 341 | 0.000074 | com.histats |
805 | 19409534 | 4201 | 0.000008 | is.pse |
806 | 19409436 | 748 | 0.000040 | fm.last |
807 | 19407808 | 3661 | 0.000010 | com.mongabay |
808 | 19407040 | 3220 | 0.000011 | me.site123 |
809 | 19406338 | 3436 | 0.000010 | com.seetickets |
810 | 19405550 | 5838 | 0.000007 | com.gamigo |
811 | 19404400 | 1666 | 0.000018 | com.materialdesignicons |
812 | 19404108 | 5140 | 0.000007 | bd.com.google |
813 | 19403242 | 790 | 0.000038 | com.venturebeat |
814 | 19401218 | 4601 | 0.000008 | uk.org.phrases |
815 | 19400780 | 3213 | 0.000011 | com.instructure |
816 | 19400298 | 2817 | 0.000013 | gov.arkansas |
817 | 19399890 | 72 | 0.000444 | com.livestream |
818 | 19399554 | 4081 | 0.000009 | cat.uab |
819 | 19399486 | 3546 | 0.000010 | org.lacity |
820 | 19399372 | 3612 | 0.000010 | com.heraldscotland |
821 | 19398370 | 1499 | 0.000020 | com.teachable |
822 | 19396672 | 2895 | 0.000012 | com.foodandwine |
823 | 19395752 | 1233 | 0.000024 | com.createjs |
824 | 19394274 | 2266 | 0.000014 | com.ajc |
825 | 19394172 | 3950 | 0.000009 | com.rappler |
826 | 19394030 | 2355 | 0.000014 | net.noscript |
827 | 19393982 | 4140 | 0.000009 | jp.doorblog |
828 | 19392882 | 2873 | 0.000012 | com.timeshighereducation |
829 | 19392238 | 275 | 0.000089 | com.bandcamp |
830 | 19389332 | 3969 | 0.000009 | jp.ne.hi-ho |
831 | 19388094 | 3629 | 0.000010 | net.inquirer |
832 | 19387882 | 552 | 0.000047 | com.cisco |
833 | 19387318 | 4076 | 0.000009 | pl.lublin |
834 | 19386370 | 1657 | 0.000018 | com.pcworld |
835 | 19383404 | 266 | 0.000093 | com.typeform |
836 | 19382886 | 203 | 0.000116 | com.naver |
837 | 19382698 | 3723 | 0.000010 | gov.bts |
838 | 19382192 | 1816 | 0.000017 | jp.makeshop |
839 | 19382102 | 4462 | 0.000008 | com.tor |
840 | 19382072 | 4513 | 0.000008 | com.weightwatchers |
841 | 19381346 | 1438 | 0.000021 | org.khanacademy |
842 | 19381274 | 954 | 0.000031 | com.thinkwithgoogle |
843 | 19381020 | 3385 | 0.000010 | uk.ac.jisc |
844 | 19380238 | 4088 | 0.000009 | ly.genial |
845 | 19379986 | 4007 | 0.000009 | com.themoscowtimes |
846 | 19378500 | 3272 | 0.000011 | com.nyt |
847 | 19378434 | 3760 | 0.000009 | com.springernature |
848 | 19378356 | 3390 | 0.000010 | int.cbd |
849 | 19377854 | 6045 | 0.000006 | es.xurl |
850 | 19376898 | 1756 | 0.000017 | com.netsolhost |
851 | 19376598 | 3852 | 0.000009 | au.edu.griffith |
852 | 19376054 | 4740 | 0.000008 | co.edu.unal |
853 | 19376040 | 4074 | 0.000009 | kr.co.koreatimes |
854 | 19374588 | 727 | 0.000042 | com.deloitte |
855 | 19374300 | 4986 | 0.000007 | org.edc |
856 | 19373940 | 4149 | 0.000008 | vn.tienphong |
857 | 19373476 | 3515 | 0.000010 | com.thediplomat |
858 | 19372932 | 4099 | 0.000009 | uk.ac.lancs |
859 | 19372798 | 5006 | 0.000007 | com.inoreader |
860 | 19372746 | 4922 | 0.000007 | com.ueuo |
861 | 19372594 | 1585 | 0.000019 | tv.ustream |
862 | 19372576 | 3234 | 0.000011 | com.tapatalk |
863 | 19372356 | 3416 | 0.000010 | nl.wur |
864 | 19372106 | 4848 | 0.000007 | net.hypermart |
865 | 19371636 | 2293 | 0.000014 | org.kff |
866 | 19369356 | 398 | 0.000064 | com.pubmatic |
867 | 19368982 | 3625 | 0.000010 | org.grist |
868 | 19368480 | 3088 | 0.000011 | tw.gov.cdc |
869 | 19368288 | 3389 | 0.000010 | com.gothamist |
870 | 19368130 | 1106 | 0.000027 | com.gizmodo |
871 | 19368116 | 4101 | 0.000009 | com.globalpost |
872 | 19367676 | 814 | 0.000037 | gov.nist |
873 | 19367536 | 4563 | 0.000008 | org.globalsecurity |
874 | 19366454 | 4547 | 0.000008 | build.bazel |
875 | 19366384 | 3782 | 0.000009 | us.ms.state |
876 | 19365878 | 4256 | 0.000008 | gr.ntua |
877 | 19365776 | 4444 | 0.000008 | se.thelocal |
878 | 19365372 | 2963 | 0.000012 | com.politifact |
879 | 19365128 | 1317 | 0.000023 | com.ensighten |
880 | 19363588 | 5097 | 0.000007 | ru.my1 |
881 | 19362680 | 3468 | 0.000010 | com.rabbitmq |
882 | 19359698 | 4138 | 0.000009 | com.elasticbeanstalk |
883 | 19359574 | 1364 | 0.000022 | com.billboard |
884 | 19359122 | 4766 | 0.000008 | cc.dict |
885 | 19358774 | 5687 | 0.000007 | fi.mbnet |
886 | 19357390 | 879 | 0.000035 | com.aliexpress |
887 | 19356918 | 210 | 0.000111 | to.amzn |
888 | 19355668 | 4275 | 0.000008 | edu.ohio |
889 | 19355546 | 3452 | 0.000010 | com.thejakartapost |
890 | 19355350 | 3277 | 0.000011 | vn.com.dantri |
891 | 19355080 | 5285 | 0.000007 | com.galvanize |
892 | 19354880 | 3484 | 0.000010 | jp.go.ndl |
893 | 19354790 | 4710 | 0.000008 | com.kiwibox |
894 | 19354514 | 2140 | 0.000015 | org.linuxfoundation |
895 | 19354500 | 4801 | 0.000007 | ru.nnov |
896 | 19353166 | 4288 | 0.000008 | gr.auth |
897 | 19352970 | 2257 | 0.000014 | net.vnexpress |
898 | 19351770 | 2900 | 0.000012 | com.crashlytics |
899 | 19351594 | 1045 | 0.000028 | com.dropboxusercontent |
900 | 19350828 | 3439 | 0.000010 | com.scotusblog |
901 | 19350712 | 4090 | 0.000009 | org.carnegieendowment |
902 | 19350278 | 395 | 0.000064 | com.atlassian |
903 | 19349726 | 3465 | 0.000010 | com.study |
904 | 19348724 | 350 | 0.000072 | com.mapbox |
905 | 19348532 | 1046 | 0.000028 | com.redhat |
906 | 19347886 | 1799 | 0.000017 | com.bravenet |
907 | 19347460 | 4284 | 0.000008 | uk.org.npg |
908 | 19347152 | 4463 | 0.000008 | com.btplc |
909 | 19347148 | 5289 | 0.000007 | ru.drom |
910 | 19346542 | 2430 | 0.000013 | com.vimeopro |
911 | 19345900 | 4419 | 0.000008 | edu.marquette |
912 | 19345644 | 426 | 0.000061 | com.adweek |
913 | 19345144 | 914 | 0.000033 | com.shutterstock |
914 | 19345090 | 1016 | 0.000029 | com.ubuntu |
915 | 19341960 | 5712 | 0.000007 | in.ac.nptel |
916 | 19341488 | 1227 | 0.000024 | com.msdn |
917 | 19340714 | 4707 | 0.000008 | com.vocabulary |
918 | 19340680 | 3929 | 0.000009 | edu.uaf |
919 | 19339658 | 3919 | 0.000009 | com.atavist |
920 | 19339456 | 3201 | 0.000011 | com.healthgrades |
921 | 19339092 | 2546 | 0.000013 | com.kinstacdn |
922 | 19338384 | 2345 | 0.000014 | com.gazhall |
923 | 19337938 | 5398 | 0.000007 | com.asmallorange |
924 | 19337800 | 3797 | 0.000009 | com.generalmills |
925 | 19336176 | 4585 | 0.000008 | vn.vtc |
926 | 19335908 | 1519 | 0.000020 | cn.gov.mofcom |
927 | 19333778 | 797 | 0.000038 | com.box |
928 | 19333606 | 3966 | 0.000009 | si.uni-lj |
929 | 19333322 | 4170 | 0.000008 | az.president |
930 | 19333194 | 1788 | 0.000017 | org.reactjs |
931 | 19332412 | 3605 | 0.000010 | com.postaffiliatepro |
932 | 19331922 | 5192 | 0.000007 | edu.uah |
933 | 19331280 | 3599 | 0.000010 | org.openedition |
934 | 19330696 | 4838 | 0.000007 | com.kapook |
935 | 19330382 | 4153 | 0.000008 | org.caringbridge |
936 | 19330374 | 483 | 0.000053 | com.aol |
937 | 19329614 | 2303 | 0.000014 | org.nfpa |
938 | 19329538 | 5956 | 0.000006 | com.glosbe |
939 | 19329194 | 4124 | 0.000009 | com.mcall |
940 | 19327622 | 4289 | 0.000008 | ru.tmweb |
941 | 19326876 | 4126 | 0.000009 | uk.co.liverpoolecho |
942 | 19326422 | 4244 | 0.000008 | com.atwebpages |
943 | 19325980 | 1067 | 0.000028 | com.freepik |
944 | 19324790 | 4085 | 0.000009 | org.specialolympics |
945 | 19323868 | 4845 | 0.000007 | net.freeforums |
946 | 19323676 | 4744 | 0.000008 | uk.ac.westminster |
947 | 19323532 | 4092 | 0.000009 | com.tok2 |
948 | 19323460 | 1025 | 0.000029 | com.elpais |
949 | 19323150 | 4946 | 0.000007 | tw.com.sina |
950 | 19322508 | 3296 | 0.000011 | com.wowza |
951 | 19322306 | 317 | 0.000079 | com.webs |
952 | 19322024 | 4697 | 0.000008 | com.warriorplus |
953 | 19321918 | 3414 | 0.000010 | com.cityam |
954 | 19321812 | 4482 | 0.000008 | org.fee |
955 | 19321520 | 4854 | 0.000007 | tw.edu.ntnu |
956 | 19321296 | 4962 | 0.000007 | com.sparknotes |
957 | 19320202 | 4516 | 0.000008 | com.newspapers |
958 | 19319634 | 2192 | 0.000015 | com.tutsplus |
959 | 19319600 | 5868 | 0.000007 | com.ananova |
960 | 19319274 | 3818 | 0.000009 | org.opensecrets |
961 | 19319134 | 633 | 0.000044 | gov.uspto |
962 | 19318722 | 5680 | 0.000007 | su.moy |
963 | 19318366 | 1013 | 0.000029 | com.uk |
964 | 19318266 | 4936 | 0.000007 | ru.pr-cy |
965 | 19318058 | 3827 | 0.000009 | cz.centrum |
966 | 19317780 | 4158 | 0.000008 | edu.niu |
967 | 19315320 | 1665 | 0.000018 | org.webkit |
968 | 19315014 | 4692 | 0.000008 | pl.edu.amu |
969 | 19314084 | 5186 | 0.000007 | com.artfire |
970 | 19313894 | 3800 | 0.000009 | org.ascd |
971 | 19312106 | 3801 | 0.000009 | edu.scu |
972 | 19311742 | 4307 | 0.000008 | com.taipeitimes |
973 | 19311568 | 4351 | 0.000008 | edu.whoi |
974 | 19310854 | 5949 | 0.000006 | com.voatiengviet |
975 | 19310748 | 3100 | 0.000011 | com.broadcastingcable |
976 | 19310720 | 4655 | 0.000008 | hk.rthk |
977 | 19310246 | 5703 | 0.000007 | com.enotes |
978 | 19309910 | 488 | 0.000053 | com.indiatimes |
979 | 19309660 | 860 | 0.000035 | com.playstation |
980 | 19309040 | 4866 | 0.000007 | com.brothersoft |
981 | 19308948 | 2708 | 0.000013 | uk.gov.defra |
982 | 19307606 | 231 | 0.000103 | org.whatwg |
983 | 19307178 | 4451 | 0.000008 | com.batchgeo |
984 | 19307118 | 751 | 0.000040 | com.psychologytoday |
985 | 19306368 | 4263 | 0.000008 | uk.co.lrb |
986 | 19306350 | 5034 | 0.000007 | ca.pe.gov |
987 | 19305884 | 4159 | 0.000008 | com.ecowatch |
988 | 19303820 | 4195 | 0.000008 | com.williamhill |
989 | 19303548 | 5767 | 0.000007 | pt.ipp |
990 | 19302972 | 4843 | 0.000007 | uk.org.38degrees |
991 | 19301624 | 1303 | 0.000023 | com.technologyreview |
992 | 19301464 | 4091 | 0.000009 | org.spie |
993 | 19301068 | 959 | 0.000031 | com.libsyn |
994 | 19300572 | 4795 | 0.000007 | com.storeboard |
995 | 19300548 | 3260 | 0.000011 | de.bmel |
996 | 19299448 | 4749 | 0.000008 | net.onlinewebshop |
997 | 19299274 | 3872 | 0.000009 | ru.1gb |
998 | 19298654 | 279 | 0.000088 | com.automattic |
999 | 19298502 | 3870 | 0.000009 | com.piie |
1000 | 19297440 | 5306 | 0.000007 | com.allthatsinteresting |
Credits
Thanks to the authors of the WebGraph framework, whose software made the computation of graph properties and ranks possible.
We hope the data will be useful for you to do any kind of research on ranking, graph analysis, link spam detection, etc. Let us know about your results via Common Crawl’s Google Group!
May/June 2020 crawl archive now available
The crawl archive for May/June 2020 is now available! It contains 2.75 billion web pages or 255 TiB of uncompressed content, crawled between May 24th and June 7th. It includes page captures of 1.2 billion URLs unknown in any of our prior crawl archives.
Starting with this crawl the WET files indicate the natural language(s) a text is written in. The language is detected using Compact Language Detector 2 (CLD2) and was made available since August 2018 only in WARC and WAT files and URL indexes. It is now also provided in WET files in the WARC header "WARC-Identified-Content-Language". Up to three language(s) are detected per document and given as comma-separated list of ISO-639-3 codes, here one example WET record fragment:
... WARC-Identified-Content-Language: isl,eng Content-Type: text/plain Content-Length: 10494 Bananabrauð með Nutella – Ljúfmeti og lekkerheit ...
Additional information about this improvement is given in the corresponding issue report.
Archive Location and Download
The May/June crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-24/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-24/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-24/warc.paths.gz | 60000 | 53.16 |
WAT files | CC-MAIN-2020-24/wat.paths.gz | 60000 | 19.02 |
WET files | CC-MAIN-2020-24/wet.paths.gz | 60000 | 8.42 |
Robots.txt files | CC-MAIN-2020-24/robotstxt.paths.gz | 60000 | 0.22 |
Non-200 responses files | CC-MAIN-2020-24/non200responses.paths.gz | 60000 | 2.77 |
URL index files | CC-MAIN-2020-24/cc-index.paths.gz | 302 | 0.22 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-24/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
March/April 2020 crawl archive now available
The crawl archive for March/April 2020 is now available! It contains 2.85 billion web pages or 280 TiB of uncompressed content, crawled between March 28th and April 10th. It includes page captures of 1 billion URLs unknown in any of our prior crawl archives.
Archive Location and Download
The March/April crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-16/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-16/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-16/warc.paths.gz | 56000 | 62.67 |
WAT files | CC-MAIN-2020-16/wat.paths.gz | 56000 | 20.37 |
WET files | CC-MAIN-2020-16/wet.paths.gz | 56000 | 8.97 |
Robots.txt files | CC-MAIN-2020-16/robotstxt.paths.gz | 56000 | 0.19 |
Non-200 responses files | CC-MAIN-2020-16/non200responses.paths.gz | 56000 | 1.39 |
URL index files | CC-MAIN-2020-16/cc-index.paths.gz | 302 | 0.21 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-16/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
February 2020 crawl archive now available
The crawl archive for February 2020 is now available! It contains 2.6 billion web pages or 240 TiB of uncompressed content, crawled between February 16th and 29th. It includes page captures of 1 billion URLs unknown in any of our prior crawl archives.
Improvements and Fixes
The HTTP headers in WARC response records have been fixed: the HTTP response status line now has a white space following the status code if the reason-phrase is empty. E.g., if a server sends an empty message (instead of “OK”), the status line will include a trailing space character: “HTTP/1.1 200
”. Following RFC 7230 the white space between status code and message is mandatory. Please refer to the bug report NUTCH-2763 for further details.
Archive Location and Download
The February crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-10/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-10/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-10/warc.paths.gz | 56000 | 49.28 |
WAT files | CC-MAIN-2020-10/wat.paths.gz | 56000 | 17.98 |
WET files | CC-MAIN-2020-10/wet.paths.gz | 56000 | 7.97 |
Robots.txt files | CC-MAIN-2020-10/robotstxt.paths.gz | 56000 | 0.22 |
Non-200 responses files | CC-MAIN-2020-10/non200responses.paths.gz | 56000 | 2.21 |
URL index files | CC-MAIN-2020-10/cc-index.paths.gz | 302 | 0.2 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-10/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
Host- and Domain-Level Web Graphs Nov/Dec/Jan 2019 – 2020
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of November, December 2019 and January 2020. Additional information about the data formats, the processing pipeline, our objectives, and credits can be found in the announcements of prior webgraph releases (e.g., Nov/Dec/Jan 2017-2018 Webgraphs). You may also visit the projects cc-webgraph and cc-pyspark which host all scripts and tools required to construct the graphs.
Host-level graph
The graph consists of 1.24 billion nodes and 4.54 billion edges and includes dangling nodes i.e. hosts that have not been crawled yet are pointed to from a link on a crawled page. There are 1.17 billion dangling nodes (95%) and the largest strongly connected component contains 45 million (3.6%) nodes.
You can download the graph and the ranks of all 1.24 billion hosts from AWS S3 on the path s3://commoncrawl/projects/hyperlinkgraph/cc-main-2019-20-nov-dec-jan/host/
. Alternatively, you can use https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2019-20-nov-dec-jan/host/
as prefix to access the files from everywhere.
Download files of the Common Crawl Nov/Dec/Jan 2019-20 host-level webgraph
Size | File | Description |
---|---|---|
7.23 GB | cc-main-2019-20-nov-dec-jan-host-vertices.paths.gz | nodes 〈id, rev host〉, paths of 12 vertices files |
20.16 GB | cc-main-2019-20-nov-dec-jan-host-edges.paths.gz | edges 〈from_id, to_id〉, paths of 24 edges files |
8.42 GB | cc-main-2019-20-nov-dec-jan-host.graph | graph in BVGraph format |
2 kB | cc-main-2019-20-nov-dec-jan-host.properties | |
10.80 GB | cc-main-2019-20-nov-dec-jan-host-t.graph | transpose of the graph (outlinks inverted to inlinks) |
2 kB | cc-main-2019-20-nov-dec-jan-host-t.properties | |
1 kB | cc-main-2019-20-nov-dec-jan-host.stats | WebGraph statistics |
16.32 GB | cc-main-2019-20-nov-dec-jan-host-ranks.txt.gz | harmonic centrality and pagerank |
Note that the host names are reversed and a leading www.
is stripped: www.subdomain.example.com
becomes com.example.subdomain
.
Domain-level graph
The domain graph was built by aggregating the host graph on the level of pay-level domains (PLDs) based on the public suffix list maintained on publicsuffix.org.
The domain-level graph has 85.8 million nodes and 1.9 billion edges. 51% or 44 million nodes are dangling nodes, the largest strongly connected component covers 34 million or 39% of the nodes.
All files related to the domain graph are available on AWS S3 under s3://commoncrawl/projects/hyperlinkgraph/cc-main-2019-20-nov-dec-jan/domain/
resp. https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2019-20-nov-dec-jan/domain/
.
Download files of the Common Crawl Nov/Dec/Jan 2019-20 domain-level webgraph
Size | File | Description |
---|---|---|
0.59 GB | cc-main-2019-20-nov-dec-jan-domain-vertices.txt.gz | nodes 〈id, rev domain, num hosts〉 |
7.65 GB | cc-main-2019-20-nov-dec-jan-domain-edges.txt.gz | edges 〈from_id, to_id〉 |
4.10 GB | cc-main-2019-20-nov-dec-jan-domain.graph | graph in BVGraph format |
2 kB | cc-main-2019-20-nov-dec-jan-domain.properties | |
4.13 GB | cc-main-2019-20-nov-dec-jan-domain-t.graph | transpose of the graph |
2 kB | cc-main-2019-20-nov-dec-jan-domain-t.properties | |
1 kB | cc-main-2019-20-nov-dec-jan-domain.stats | WebGraph statistics |
1.86 GB | cc-main-2019-20-nov-dec-jan-domain-ranks.txt.gz | harmonic centrality and pagerank |
Below you’ll find the top 1000 domains ranked by Harmonic Centrality or PageRank. The full list of all 86 million domain ranks is available for download.
Top 1000 domains ranked by harmonic centrality (Nov/Dec/Jan 2019-2020)
harmonic centrality rank | hc value | page rank | page rank value | reversed hostname |
---|---|---|---|---|
1 | 30598398 | 1 | 0.019072 | com.googleapis |
2 | 29113136 | 3 | 0.012214 | com.facebook |
3 | 27475138 | 2 | 0.013236 | com.google |
4 | 25610480 | 4 | 0.007452 | com.twitter |
5 | 24947126 | 5 | 0.007174 | org.w |
6 | 24904712 | 6 | 0.006611 | com.youtube |
7 | 23281504 | 9 | 0.004269 | com.instagram |
8 | 22446296 | 7 | 0.005561 | org.gmpg |
9 | 22154750 | 8 | 0.005033 | com.googletagmanager |
10 | 22107784 | 13 | 0.003001 | com.linkedin |
11 | 21307220 | 10 | 0.003433 | org.wordpress |
12 | 21290688 | 20 | 0.001717 | com.gravatar |
13 | 21096944 | 11 | 0.003266 | com.cloudflare |
14 | 21086168 | 23 | 0.001516 | com.pinterest |
15 | 20869868 | 16 | 0.002242 | com.gstatic |
16 | 20855286 | 15 | 0.002366 | com.wordpress |
17 | 20713268 | 25 | 0.001234 | org.wikipedia |
18 | 20641712 | 17 | 0.002130 | com.apple |
19 | 20584368 | 14 | 0.002460 | com.bootstrapcdn |
20 | 20371046 | 33 | 0.001102 | com.vimeo |
21 | 20336732 | 42 | 0.000833 | com.blogspot |
22 | 20237058 | 18 | 0.001787 | com.jquery |
23 | 20220572 | 50 | 0.000732 | be.youtu |
24 | 20218806 | 32 | 0.001130 | com.microsoft |
25 | 20119820 | 49 | 0.000737 | com.wp |
26 | 20059828 | 19 | 0.001764 | com.adobe |
27 | 20029544 | 52 | 0.000709 | com.amazon |
28 | 20013346 | 44 | 0.000784 | gl.goo |
29 | 19972608 | 35 | 0.001020 | com.amazonaws |
30 | 19941934 | 67 | 0.000471 | com.tumblr |
31 | 19937390 | 64 | 0.000518 | ly.bit |
32 | 19862906 | 29 | 0.001164 | com.macromedia |
33 | 19817494 | 30 | 0.001151 | com.baidu |
34 | 19816382 | 38 | 0.000958 | com.google-analytics |
35 | 19808066 | 31 | 0.001142 | com.googlesyndication |
36 | 19781460 | 34 | 0.001092 | net.cloudfront |
37 | 19770754 | 24 | 0.001254 | ru.yandex |
38 | 19749302 | 53 | 0.000693 | com.flickr |
39 | 19700112 | 22 | 0.001568 | com.github |
40 | 19698814 | 80 | 0.000368 | com.yahoo |
41 | 19676534 | 58 | 0.000644 | eu.europa |
42 | 19632506 | 115 | 0.000287 | com.reddit |
43 | 19603190 | 41 | 0.000918 | com.addthis |
44 | 19561006 | 72 | 0.000403 | com.weebly |
45 | 19559908 | 43 | 0.000823 | org.w3 |
46 | 19550116 | 63 | 0.000524 | me.wp |
47 | 19546614 | 108 | 0.000313 | com.googleusercontent |
48 | 19531876 | 45 | 0.000777 | io.github |
49 | 19523224 | 182 | 0.000140 | org.wikimedia |
50 | 19521602 | 70 | 0.000422 | com.medium |
51 | 19516098 | 47 | 0.000743 | org.schema |
52 | 19496222 | 46 | 0.000754 | net.jsdelivr |
53 | 19495662 | 76 | 0.000374 | org.creativecommons |
54 | 19472610 | 173 | 0.000153 | com.imgur |
55 | 19459848 | 36 | 0.000989 | net.doubleclick |
56 | 19451512 | 51 | 0.000711 | com.wix |
57 | 19416054 | 186 | 0.000138 | uk.co.bbc |
58 | 19408312 | 149 | 0.000181 | com.forbes |
59 | 19405426 | 93 | 0.000332 | com.weibo |
60 | 19404990 | 60 | 0.000601 | co.t |
61 | 19398622 | 28 | 0.001192 | com.fontawesome |
62 | 19382484 | 48 | 0.000741 | com.paypal |
63 | 19375914 | 210 | 0.000114 | com.cnn |
64 | 19372292 | 144 | 0.000190 | org.archive |
65 | 19368680 | 61 | 0.000583 | org.mozilla |
66 | 19348182 | 189 | 0.000137 | net.sourceforge |
67 | 19304774 | 315 | 0.000079 | edu.mit |
68 | 19302936 | 174 | 0.000152 | com.theguardian |
69 | 19293782 | 278 | 0.000089 | edu.harvard |
70 | 19283372 | 179 | 0.000144 | com.bing |
71 | 19271390 | 117 | 0.000286 | com.jimdo |
72 | 19270368 | 140 | 0.000202 | com.nytimes |
73 | 19260752 | 27 | 0.001195 | com.qq |
74 | 19259406 | 237 | 0.000103 | com.wsj |
75 | 19257754 | 21 | 0.001581 | org.apache |
76 | 19256254 | 56 | 0.000658 | com.googleadservices |
77 | 19241110 | 215 | 0.000111 | com.washingtonpost |
78 | 19237816 | 269 | 0.000092 | com.bloomberg |
79 | 19237164 | 259 | 0.000094 | com.techcrunch |
80 | 19210562 | 544 | 0.000049 | com.deviantart |
81 | 19205946 | 162 | 0.000162 | org.ietf |
82 | 19198346 | 247 | 0.000098 | com.oracle |
83 | 19194696 | 279 | 0.000089 | com.android |
84 | 19193448 | 69 | 0.000430 | com.list-manage |
85 | 19174968 | 397 | 0.000065 | com.ted |
86 | 19167820 | 321 | 0.000078 | com.reuters |
87 | 19161964 | 312 | 0.000080 | com.wired |
88 | 19158768 | 154 | 0.000173 | com.wixsite |
89 | 19154874 | 341 | 0.000073 | com.ft |
90 | 19152068 | 330 | 0.000076 | uk.co.telegraph |
91 | 19151666 | 427 | 0.000060 | com.theverge |
92 | 19145078 | 155 | 0.000172 | gov.nih |
93 | 19143886 | 272 | 0.000091 | com.myspace |
94 | 19142172 | 377 | 0.000068 | gov.nasa |
95 | 19136022 | 291 | 0.000085 | com.bbc |
96 | 19126138 | 339 | 0.000074 | com.example |
97 | 19119916 | 239 | 0.000103 | org.python |
98 | 19117868 | 82 | 0.000361 | com.whatsapp |
99 | 19111550 | 122 | 0.000267 | com.unpkg |
100 | 19110346 | 188 | 0.000138 | uk.co.google |
101 | 19089504 | 707 | 0.000041 | com.economist |
102 | 19088766 | 255 | 0.000095 | com.appspot |
103 | 19082300 | 384 | 0.000067 | uk.co.dailymail |
104 | 19081904 | 209 | 0.000115 | org.gnu |
105 | 19081708 | 262 | 0.000093 | com.githubusercontent |
106 | 19080182 | 131 | 0.000232 | com.ytimg |
107 | 19077156 | 320 | 0.000078 | org.un |
108 | 19075806 | 163 | 0.000162 | com.giphy |
109 | 19068860 | 398 | 0.000065 | com.latimes |
110 | 19067606 | 169 | 0.000157 | com.twimg |
111 | 19066404 | 431 | 0.000060 | com.googleblog |
112 | 19056162 | 176 | 0.000148 | com.blogger |
113 | 19054310 | 232 | 0.000104 | com.dribbble |
114 | 19052824 | 207 | 0.000115 | com.npmjs |
115 | 19050524 | 564 | 0.000047 | org.arxiv |
116 | 19045194 | 666 | 0.000042 | edu.upenn |
117 | 19042770 | 171 | 0.000154 | com.eventbrite |
118 | 19036612 | 379 | 0.000068 | com.springer |
119 | 19032422 | 277 | 0.000090 | org.ampproject |
120 | 19031354 | 557 | 0.000047 | com.gitlab |
121 | 19025616 | 596 | 0.000045 | com.vice |
122 | 19025236 | 206 | 0.000116 | com.disqus |
123 | 19023978 | 1036 | 0.000031 | com.hatenablog |
124 | 19023406 | 835 | 0.000039 | edu.columbia |
125 | 19018670 | 818 | 0.000040 | io.readthedocs |
126 | 19008280 | 205 | 0.000116 | me.t |
127 | 19005842 | 390 | 0.000066 | com.w3schools |
128 | 19004030 | 941 | 0.000034 | org.chromium |
129 | 19003900 | 418 | 0.000062 | com.nature |
130 | 19001452 | 716 | 0.000041 | com.slate |
131 | 19000862 | 157 | 0.000171 | jp.co.yahoo |
132 | 18997566 | 325 | 0.000077 | com.time |
133 | 18997164 | 430 | 0.000060 | com.statista |
134 | 18990336 | 744 | 0.000040 | com.ubuntu |
135 | 18985466 | 158 | 0.000167 | com.yelp |
136 | 18982550 | 632 | 0.000043 | org.worldbank |
137 | 18982428 | 143 | 0.000191 | com.spotify |
138 | 18981078 | 347 | 0.000072 | com.skype |
139 | 18978660 | 935 | 0.000034 | com.playstation |
140 | 18976524 | 306 | 0.000082 | com.fc2 |
141 | 18973312 | 1304 | 0.000024 | org.coursera |
142 | 18969438 | 121 | 0.000281 | com.stripe |
143 | 18968562 | 816 | 0.000040 | com.qz |
144 | 18968168 | 617 | 0.000044 | com.git-scm |
145 | 18966448 | 486 | 0.000053 | uk.co.independent |
146 | 18965810 | 199 | 0.000126 | com.eepurl |
147 | 18964444 | 961 | 0.000034 | com.500px |
148 | 18964342 | 405 | 0.000063 | net.researchgate |
149 | 18962654 | 241 | 0.000101 | com.bandcamp |
150 | 18959274 | 55 | 0.000669 | net.facebook |
151 | 18956478 | 389 | 0.000066 | com.outlook |
152 | 18955490 | 229 | 0.000105 | com.unsplash |
153 | 18950460 | 631 | 0.000043 | com.mysql |
154 | 18949330 | 419 | 0.000062 | com.theatlantic |
155 | 18948456 | 116 | 0.000286 | com.soundcloud |
156 | 18948364 | 180 | 0.000143 | com.amazon-adsystem |
157 | 18947670 | 123 | 0.000259 | org.networkadvertising |
158 | 18940768 | 571 | 0.000046 | org.bitbucket |
159 | 18940298 | 1163 | 0.000027 | com.jetbrains |
160 | 18936474 | 408 | 0.000063 | com.mozilla |
161 | 18936104 | 498 | 0.000052 | com.nationalgeographic |
162 | 18932906 | 316 | 0.000079 | com.usatoday |
163 | 18930890 | 439 | 0.000059 | com.criteo |
164 | 18927342 | 837 | 0.000039 | uk.ac.ox |
165 | 18925560 | 468 | 0.000054 | com.fortune |
166 | 18924956 | 466 | 0.000055 | com.pixabay |
167 | 18922224 | 1278 | 0.000024 | uk.co.thesun |
168 | 18921476 | 230 | 0.000104 | net.behance |
169 | 18916700 | 1547 | 0.000019 | com.amd |
170 | 18915574 | 822 | 0.000039 | com.evernote |
171 | 18909918 | 40 | 0.000932 | com.vk |
172 | 18909318 | 799 | 0.000040 | com.about |
173 | 18907910 | 505 | 0.000051 | uk.co.blogspot |
174 | 18904000 | 1193 | 0.000026 | se.haxx |
175 | 18903686 | 251 | 0.000097 | gle.forms |
176 | 18900132 | 719 | 0.000041 | com.docker |
177 | 18900020 | 967 | 0.000033 | uk.co.guardian |
178 | 18899576 | 303 | 0.000082 | org.doi |
179 | 18898330 | 497 | 0.000052 | me.about |
180 | 18896862 | 525 | 0.000050 | gg.discord |
181 | 18894432 | 1383 | 0.000022 | com.instructables |
182 | 18891582 | 147 | 0.000189 | com.dropbox |
183 | 18888432 | 1000 | 0.000032 | com.scientificamerican |
184 | 18885228 | 332 | 0.000076 | jp.co.rakuten |
185 | 18881548 | 903 | 0.000036 | google.blog |
186 | 18875704 | 194 | 0.000130 | com.feedburner |
187 | 18870758 | 1081 | 0.000030 | org.altervista |
188 | 18869068 | 634 | 0.000043 | org.unesco |
189 | 18868966 | 1082 | 0.000030 | org.eclipse |
190 | 18868488 | 275 | 0.000090 | gov.ca |
191 | 18866552 | 875 | 0.000037 | jp.livedoor |
192 | 18865164 | 1490 | 0.000020 | org.phys |
193 | 18864674 | 292 | 0.000085 | com.sciencedirect |
194 | 18860674 | 212 | 0.000113 | jp.ameblo |
195 | 18855700 | 572 | 0.000046 | gov.loc |
196 | 18854892 | 993 | 0.000033 | org.cambridge |
197 | 18845512 | 403 | 0.000064 | ca.google |
198 | 18842558 | 663 | 0.000042 | edu.washington |
199 | 18836168 | 71 | 0.000408 | net.slideshare |
200 | 18832808 | 543 | 0.000049 | com.cisco |
201 | 18829838 | 1364 | 0.000023 | edu.rutgers |
202 | 18828808 | 304 | 0.000082 | com.nbcnews |
203 | 18827294 | 243 | 0.000099 | ru.rambler |
204 | 18826594 | 823 | 0.000039 | au.net.abc |
205 | 18826034 | 998 | 0.000032 | uk.co.thetimes |
206 | 18825554 | 1472 | 0.000021 | com.bankofamerica |
207 | 18821912 | 62 | 0.000549 | com.fb |
208 | 18821670 | 863 | 0.000037 | org.sciencemag |
209 | 18813844 | 887 | 0.000036 | com.speakerdeck |
210 | 18810986 | 412 | 0.000063 | jp.ne.sakura |
211 | 18806748 | 220 | 0.000109 | org.iana |
212 | 18802112 | 1258 | 0.000025 | com.wikidot |
213 | 18799706 | 1650 | 0.000018 | pt.sapo |
214 | 18792880 | 905 | 0.000036 | uk.co.mirror |
215 | 18790620 | 218 | 0.000110 | edu.stanford |
216 | 18787734 | 1143 | 0.000028 | org.kernel |
217 | 18786526 | 450 | 0.000057 | com.elsevier |
218 | 18784420 | 1452 | 0.000021 | edu.osu |
219 | 18783446 | 1622 | 0.000019 | com.googlesource |
220 | 18782426 | 1395 | 0.000022 | com.vogue |
221 | 18781902 | 74 | 0.000401 | net.akamaihd |
222 | 18781258 | 842 | 0.000038 | gov.fcc |
223 | 18780926 | 1240 | 0.000025 | ms.1drv |
224 | 18779106 | 1377 | 0.000022 | edu.asu |
225 | 18778828 | 254 | 0.000095 | com.businessinsider |
226 | 18778534 | 844 | 0.000038 | co.ibb |
227 | 18778386 | 1843 | 0.000016 | com.wolfram |
228 | 18771090 | 721 | 0.000041 | com.trello |
229 | 18769148 | 106 | 0.000323 | com.paypalobjects |
230 | 18766298 | 301 | 0.000083 | net.windows |
231 | 18762238 | 1216 | 0.000026 | jp.geocities |
232 | 18762150 | 587 | 0.000045 | com.box |
233 | 18761840 | 876 | 0.000037 | com.sciencedaily |
234 | 18758802 | 227 | 0.000106 | com.wpengine |
235 | 18752986 | 493 | 0.000052 | com.herokuapp |
236 | 18752026 | 1030 | 0.000032 | edu.princeton |
237 | 18748814 | 915 | 0.000035 | edu.academia |
238 | 18748232 | 284 | 0.000087 | com.googlecode |
239 | 18746146 | 1177 | 0.000027 | com.asahi |
240 | 18740044 | 1375 | 0.000022 | com.newscientist |
241 | 18733718 | 1428 | 0.000021 | blog.home |
242 | 18732458 | 314 | 0.000080 | com.tinyurl |
243 | 18732010 | 516 | 0.000051 | com.udacity |
244 | 18731740 | 2103 | 0.000014 | com.wizards |
245 | 18729906 | 386 | 0.000067 | com.cnet |
246 | 18729644 | 1421 | 0.000022 | com.ndtv |
247 | 18729370 | 335 | 0.000075 | com.getpocket |
248 | 18729276 | 1349 | 0.000023 | com.fandom |
249 | 18726412 | 1241 | 0.000025 | net.seesaa |
250 | 18723376 | 167 | 0.000158 | com.imdb |
251 | 18718440 | 333 | 0.000076 | org.debian |
252 | 18715552 | 699 | 0.000041 | site.business |
253 | 18713638 | 261 | 0.000093 | com.live |
254 | 18711498 | 1059 | 0.000031 | jp.ne.goo |
255 | 18710778 | 1504 | 0.000020 | io.itch |
256 | 18704016 | 1155 | 0.000027 | org.greenpeace |
257 | 18702254 | 1047 | 0.000031 | com.netlify |
258 | 18701568 | 1808 | 0.000017 | net.pixnet |
259 | 18700552 | 402 | 0.000064 | com.squareup |
260 | 18699852 | 1145 | 0.000028 | co.elastic |
261 | 18699756 | 310 | 0.000081 | com.ibm |
262 | 18698784 | 203 | 0.000118 | com.stackoverflow |
263 | 18698146 | 494 | 0.000052 | com.indiatimes |
264 | 18696312 | 3420 | 0.000010 | com.armorgames |
265 | 18693148 | 265 | 0.000092 | com.aliyuncs |
266 | 18692378 | 226 | 0.000106 | com.optimizely |
267 | 18685944 | 1777 | 0.000017 | uk.co.timesonline |
268 | 18679570 | 975 | 0.000033 | com.mixcloud |
269 | 18677530 | 1618 | 0.000019 | com.itv |
270 | 18674460 | 198 | 0.000128 | org.bbb |
271 | 18674340 | 57 | 0.000648 | net.fbcdn |
272 | 18674148 | 2236 | 0.000014 | com.opendns |
273 | 18672682 | 3302 | 0.000010 | tw.com.gamer |
274 | 18672646 | 406 | 0.000063 | com.go |
275 | 18672486 | 437 | 0.000059 | com.msn |
276 | 18672432 | 37 | 0.000966 | com.wixstatic |
277 | 18667606 | 1485 | 0.000021 | org.archlinux |
278 | 18666546 | 448 | 0.000058 | org.pewresearch |
279 | 18664590 | 87 | 0.000345 | com.shopify |
280 | 18662914 | 838 | 0.000039 | jp.shinobi |
281 | 18662550 | 995 | 0.000032 | com.bmj |
282 | 18660516 | 1579 | 0.000019 | com.diigo |
283 | 18659688 | 160 | 0.000167 | com.opera |
284 | 18659332 | 1822 | 0.000017 | com.youdao |
285 | 18657686 | 1259 | 0.000025 | com.angelfire |
286 | 18657376 | 906 | 0.000036 | jp.naver |
287 | 18656934 | 1038 | 0.000031 | com.thelancet |
288 | 18653610 | 1585 | 0.000019 | uk.bl |
289 | 18652186 | 864 | 0.000037 | br.com.google |
290 | 18649914 | 490 | 0.000053 | com.bigcartel |
291 | 18648796 | 1007 | 0.000032 | com.sky |
292 | 18647596 | 1408 | 0.000022 | net.daringfireball |
293 | 18642592 | 1779 | 0.000017 | uk.ac.kcl |
294 | 18641168 | 1752 | 0.000017 | org.maven |
295 | 18639544 | 534 | 0.000049 | me.m |
296 | 18639298 | 1107 | 0.000029 | com.reverbnation |
297 | 18637290 | 1665 | 0.000018 | net.cnki |
298 | 18634102 | 1089 | 0.000030 | com.theconversation |
299 | 18630504 | 460 | 0.000056 | it.placehold |
300 | 18629064 | 1117 | 0.000029 | com.podbean |
301 | 18628660 | 881 | 0.000036 | org.fao |
302 | 18626956 | 1144 | 0.000028 | co.g |
303 | 18626264 | 1370 | 0.000023 | com.dw |
304 | 18623646 | 119 | 0.000283 | com.mailchimp |
305 | 18621260 | 1631 | 0.000019 | jp.ne.so-net |
306 | 18620264 | 1346 | 0.000023 | com.livescience |
307 | 18620110 | 2071 | 0.000015 | edu.kit |
308 | 18611900 | 1308 | 0.000024 | ca.utoronto |
309 | 18610618 | 1310 | 0.000024 | com.webnode |
310 | 18608388 | 957 | 0.000034 | au.gov.nsw |
311 | 18606964 | 1411 | 0.000022 | com.citrix |
312 | 18603704 | 1031 | 0.000032 | jp.jugem |
313 | 18602374 | 1115 | 0.000029 | gov.wa |
314 | 18600002 | 458 | 0.000056 | com.quora |
315 | 18598840 | 99 | 0.000326 | com.godaddy |
316 | 18597740 | 1334 | 0.000023 | com.bloglovin |
317 | 18596834 | 1194 | 0.000026 | com.serving-sys |
318 | 18595976 | 900 | 0.000036 | gov.dhs |
319 | 18595120 | 1630 | 0.000019 | org.edx |
320 | 18592696 | 244 | 0.000099 | me.wa |
321 | 18592582 | 1830 | 0.000017 | com.pearltrees |
322 | 18592022 | 1576 | 0.000019 | com.twitpic |
323 | 18591744 | 1776 | 0.000017 | cn.people |
324 | 18590350 | 1171 | 0.000027 | com.britannica |
325 | 18589380 | 1893 | 0.000016 | sg.edu.nus |
326 | 18588130 | 1791 | 0.000017 | com.kinja |
327 | 18587404 | 2296 | 0.000013 | com.authorstream |
328 | 18585922 | 1566 | 0.000019 | ca.mcgill |
329 | 18585132 | 380 | 0.000068 | com.kickstarter |
330 | 18581636 | 1237 | 0.000025 | com.lulu |
331 | 18576948 | 2591 | 0.000012 | com.colourlovers |
332 | 18575064 | 1919 | 0.000016 | com.hm |
333 | 18574242 | 337 | 0.000075 | com.rackcdn |
334 | 18567994 | 2261 | 0.000013 | uk.ac.sussex |
335 | 18563954 | 1945 | 0.000016 | org.vim |
336 | 18562946 | 1063 | 0.000031 | com.healthline |
337 | 18562860 | 1911 | 0.000016 | org.wikibooks |
338 | 18562650 | 1867 | 0.000016 | io.soup |
339 | 18560052 | 1704 | 0.000018 | nl.blogspot |
340 | 18555740 | 509 | 0.000051 | com.mashable |
341 | 18553730 | 264 | 0.000093 | com.typepad |
342 | 18553486 | 1071 | 0.000031 | com.adjust |
343 | 18551456 | 396 | 0.000065 | com.photobucket |
344 | 18544528 | 1741 | 0.000017 | org.bitcoin |
345 | 18542952 | 2227 | 0.000014 | tw.edu.ntu |
346 | 18541072 | 1100 | 0.000030 | com.ecwid |
347 | 18540798 | 1820 | 0.000017 | com.indianexpress |
348 | 18540288 | 1889 | 0.000016 | co.ello |
349 | 18538870 | 575 | 0.000046 | edu.berkeley |
350 | 18537372 | 2026 | 0.000015 | com.upi |
351 | 18537264 | 132 | 0.000232 | com.squarespace |
352 | 18535682 | 280 | 0.000089 | uk.org.ico |
353 | 18534924 | 1136 | 0.000028 | com.ssrn |
354 | 18534382 | 2152 | 0.000014 | com.viki |
355 | 18533818 | 1219 | 0.000025 | it.scoop |
356 | 18532606 | 270 | 0.000092 | com.surveymonkey |
357 | 18532016 | 1601 | 0.000019 | com.fastcodesign |
358 | 18530620 | 1782 | 0.000017 | org.unep |
359 | 18529588 | 1057 | 0.000031 | uk.parliament |
360 | 18527356 | 1966 | 0.000016 | org.haskell |
361 | 18527140 | 224 | 0.000107 | com.etsy |
362 | 18527064 | 1442 | 0.000021 | com.shutterfly |
363 | 18525388 | 1569 | 0.000019 | uk.org.tate |
364 | 18524530 | 2862 | 0.000011 | co.electrek |
365 | 18523134 | 2693 | 0.000011 | jp.doorblog |
366 | 18522838 | 156 | 0.000171 | com.issuu |
367 | 18519148 | 2018 | 0.000015 | com.dezeen |
368 | 18517910 | 2430 | 0.000013 | sh.now |
369 | 18517530 | 1157 | 0.000027 | com.tradedoubler |
370 | 18515028 | 1173 | 0.000027 | gov.weather |
371 | 18513616 | 1109 | 0.000029 | com.imageshack |
372 | 18512682 | 1693 | 0.000018 | com.channel4 |
373 | 18512134 | 1116 | 0.000029 | gov.dot |
374 | 18511018 | 2703 | 0.000011 | cn.edu.sdu |
375 | 18510554 | 1164 | 0.000027 | com.wikia |
376 | 18509244 | 282 | 0.000088 | com.huffingtonpost |
377 | 18509124 | 953 | 0.000034 | uk.co.pinterest |
378 | 18508676 | 924 | 0.000035 | com.arstechnica |
379 | 18507156 | 271 | 0.000091 | com.rawgit |
380 | 18505812 | 484 | 0.000053 | tv.twitch |
381 | 18505722 | 1917 | 0.000016 | th.co.google |
382 | 18503134 | 2390 | 0.000013 | uk.ac.nhm |
383 | 18502360 | 1764 | 0.000017 | com.netvibes |
384 | 18501156 | 1871 | 0.000016 | edu.emory |
385 | 18500964 | 918 | 0.000035 | in.amazon |
386 | 18500252 | 963 | 0.000034 | com.strikingly |
387 | 18499224 | 1773 | 0.000017 | net.bplaced |
388 | 18497786 | 3356 | 0.000010 | tw.edu.ntnu |
389 | 18495692 | 1811 | 0.000017 | edu.iu |
390 | 18494542 | 833 | 0.000039 | com.brightcove |
391 | 18491624 | 225 | 0.000107 | com.hubspot |
392 | 18491366 | 1470 | 0.000021 | com.wattpad |
393 | 18490914 | 1476 | 0.000021 | gov.michigan |
394 | 18489222 | 1916 | 0.000016 | nl.tudelft |
395 | 18488064 | 1436 | 0.000021 | org.c-span |
396 | 18487708 | 394 | 0.000065 | com.meetup |
397 | 18483412 | 2110 | 0.000014 | com.kaggle |
398 | 18481662 | 1299 | 0.000024 | edu.brookings |
399 | 18478490 | 86 | 0.000345 | net.jsfiddle |
400 | 18478420 | 2621 | 0.000012 | sh.surge |
401 | 18475530 | 2248 | 0.000014 | com.rsa |
402 | 18475220 | 1781 | 0.000017 | gov.ahrq |
403 | 18474128 | 825 | 0.000039 | org.mediawiki |
404 | 18473922 | 346 | 0.000072 | edu.yale |
405 | 18472664 | 826 | 0.000039 | com.intel |
406 | 18472288 | 1512 | 0.000020 | gov.faa |
407 | 18471926 | 1975 | 0.000015 | io.material |
408 | 18471732 | 1073 | 0.000031 | com.thenextweb |
409 | 18471706 | 1847 | 0.000016 | net.earthlink |
410 | 18469610 | 1908 | 0.000016 | jp.blog |
411 | 18469092 | 831 | 0.000039 | com.pexels |
412 | 18464760 | 1029 | 0.000032 | uk.gov.nationalarchives |
413 | 18460916 | 1973 | 0.000016 | com.smashwords |
414 | 18459088 | 939 | 0.000034 | org.ieee |
415 | 18457420 | 2185 | 0.000014 | com.smore |
416 | 18456724 | 345 | 0.000072 | com.livejournal |
417 | 18456360 | 3366 | 0.000010 | hk.edu.hkbu |
418 | 18453542 | 414 | 0.000063 | com.nypost |
419 | 18453464 | 1785 | 0.000017 | com.business-standard |
420 | 18453440 | 2907 | 0.000011 | com.yam |
421 | 18451248 | 1233 | 0.000025 | org.aarp |
422 | 18450408 | 1999 | 0.000015 | com.oprah |
423 | 18449942 | 2053 | 0.000015 | org.jpn |
424 | 18449880 | 1503 | 0.000020 | org.amnesty |
425 | 18449790 | 1651 | 0.000018 | com.avvo |
426 | 18449648 | 2540 | 0.000012 | com.cleantechnica |
427 | 18449504 | 522 | 0.000050 | edu.cornell |
428 | 18448124 | 2586 | 0.000012 | com.mysanantonio |
429 | 18447594 | 473 | 0.000054 | io.shields |
430 | 18447544 | 1444 | 0.000021 | org.hrw |
431 | 18444240 | 2487 | 0.000012 | org.neocities |
432 | 18442810 | 1943 | 0.000016 | com.care2 |
433 | 18440702 | 1714 | 0.000018 | com.snopes |
434 | 18440194 | 1148 | 0.000027 | com.gizmodo |
435 | 18440058 | 2474 | 0.000012 | com.googledrive |
436 | 18439272 | 2594 | 0.000012 | com.iflscience |
437 | 18437524 | 1594 | 0.000019 | org.pypi |
438 | 18437266 | 268 | 0.000092 | net.php |
439 | 18436902 | 1927 | 0.000016 | org.rsc |
440 | 18436288 | 1389 | 0.000022 | com.pbworks |
441 | 18435978 | 2735 | 0.000011 | com.itsnicethat |
442 | 18435362 | 2092 | 0.000015 | ae.thenational |
443 | 18435326 | 2461 | 0.000012 | com.hsbc |
444 | 18434642 | 338 | 0.000074 | com.hp |
445 | 18432636 | 1354 | 0.000023 | uk.co.standard |
446 | 18431764 | 1807 | 0.000017 | com.instapaper |
447 | 18431596 | 465 | 0.000055 | io.codepen |
448 | 18431390 | 553 | 0.000047 | com.buzzfeed |
449 | 18431020 | 1910 | 0.000016 | com.secondlife |
450 | 18430258 | 2425 | 0.000013 | jp.go.ndl |
451 | 18429756 | 2106 | 0.000014 | io.gitlab |
452 | 18428432 | 373 | 0.000069 | int.who |
453 | 18427128 | 2300 | 0.000013 | org.lds |
454 | 18426976 | 2471 | 0.000012 | uk.mod |
455 | 18426954 | 1711 | 0.000018 | google.ai |
456 | 18426290 | 96 | 0.000330 | de.google |
457 | 18423898 | 1427 | 0.000021 | com.thehindu |
458 | 18423724 | 1749 | 0.000017 | com.curbed |
459 | 18422902 | 1638 | 0.000019 | no.google |
460 | 18421738 | 340 | 0.000074 | com.cnbc |
461 | 18420686 | 1061 | 0.000031 | com.thedrum |
462 | 18419780 | 165 | 0.000160 | com.ebay |
463 | 18418790 | 627 | 0.000043 | com.zdnet |
464 | 18418454 | 2330 | 0.000013 | pl.cba |
465 | 18416392 | 2441 | 0.000013 | com.minds |
466 | 18413488 | 201 | 0.000125 | com.salesforce |
467 | 18413252 | 1574 | 0.000019 | com.moonfruit |
468 | 18412358 | 1156 | 0.000027 | com.mixpanel |
469 | 18411670 | 2818 | 0.000011 | tl.page |
470 | 18409196 | 1982 | 0.000015 | com.name |
471 | 18409080 | 2282 | 0.000013 | jp.hateblo |
472 | 18407830 | 2507 | 0.000012 | org.tvtropes |
473 | 18407318 | 2580 | 0.000012 | jp.hatenadiary |
474 | 18406348 | 2510 | 0.000012 | de.dw |
475 | 18405634 | 1854 | 0.000016 | com.googlegroups |
476 | 18405508 | 1876 | 0.000016 | mx.com.google |
477 | 18405134 | 2198 | 0.000014 | org.aiga |
478 | 18403404 | 2883 | 0.000011 | uk.co.birminghammail |
479 | 18403340 | 367 | 0.000069 | com.booking |
480 | 18401602 | 2314 | 0.000013 | vn.com.google |
481 | 18401550 | 1729 | 0.000018 | gov.pa |
482 | 18399972 | 1666 | 0.000018 | org.hrc |
483 | 18399618 | 882 | 0.000036 | gov.nist |
484 | 18398472 | 2742 | 0.000011 | com.exxonmobil |
485 | 18397376 | 1841 | 0.000016 | ar.com.google |
486 | 18396356 | 989 | 0.000033 | net.clickbank |
487 | 18395660 | 976 | 0.000033 | com.matterport |
488 | 18392402 | 2429 | 0.000013 | ua.at |
489 | 18390522 | 2011 | 0.000015 | uk.ac.leeds |
490 | 18387444 | 309 | 0.000081 | gov.cdc |
491 | 18386528 | 1558 | 0.000019 | int.unfccc |
492 | 18386408 | 2342 | 0.000013 | com.eklablog |
493 | 18385700 | 459 | 0.000056 | com.gmail |
494 | 18385598 | 401 | 0.000064 | org.npr |
495 | 18384832 | 1672 | 0.000018 | gov.maryland |
496 | 18384390 | 357 | 0.000070 | com.office |
497 | 18383950 | 2240 | 0.000014 | se.liu |
498 | 18383810 | 2067 | 0.000015 | com.discovermagazine |
499 | 18383400 | 2204 | 0.000014 | com.ipage |
500 | 18381626 | 1110 | 0.000029 | com.stackexchange |
501 | 18381594 | 2418 | 0.000013 | it.justpaste |
502 | 18380974 | 449 | 0.000058 | fr.free |
503 | 18380682 | 1718 | 0.000018 | sg.com.google |
504 | 18379672 | 1060 | 0.000031 | com.engadget |
505 | 18378238 | 2421 | 0.000013 | my.com.thestar |
506 | 18377282 | 1273 | 0.000024 | dk.google |
507 | 18377136 | 2210 | 0.000014 | org.biorxiv |
508 | 18377062 | 1861 | 0.000016 | com.weheartit |
509 | 18374194 | 1598 | 0.000019 | uk.gov.tfl |
510 | 18371274 | 508 | 0.000051 | gov.whitehouse |
511 | 18369330 | 1723 | 0.000018 | ly.snip |
512 | 18369006 | 1809 | 0.000017 | com.yourstory |
513 | 18366356 | 3154 | 0.000011 | com.bonanza |
514 | 18365650 | 2833 | 0.000011 | com.scienceblogs |
515 | 18365484 | 1431 | 0.000021 | com.ebayimg |
516 | 18365436 | 1774 | 0.000017 | gov.ky |
517 | 18363500 | 858 | 0.000038 | com.venturebeat |
518 | 18362924 | 1160 | 0.000027 | se.google |
519 | 18362454 | 1350 | 0.000023 | com.firebaseapp |
520 | 18362026 | 178 | 0.000147 | com.zendesk |
521 | 18360150 | 2004 | 0.000015 | uk.gov.metoffice |
522 | 18359990 | 928 | 0.000035 | com.windowsphone |
523 | 18359750 | 2336 | 0.000013 | com.rediff |
524 | 18358388 | 518 | 0.000051 | com.alibaba |
525 | 18355256 | 2225 | 0.000014 | com.blogfa |
526 | 18355232 | 415 | 0.000063 | com.fastcompany |
527 | 18353212 | 1426 | 0.000021 | com.surveygizmo |
528 | 18352352 | 2021 | 0.000015 | au.com.telstra |
529 | 18351454 | 1134 | 0.000028 | org.sphinx-doc |
530 | 18350502 | 2048 | 0.000015 | ro.google |
531 | 18350126 | 1904 | 0.000016 | org.tigris |
532 | 18349524 | 2835 | 0.000011 | be.lesoir |
533 | 18349430 | 2698 | 0.000011 | cz.centrum |
534 | 18349372 | 2047 | 0.000015 | link.page |
535 | 18349260 | 479 | 0.000054 | org.nodejs |
536 | 18349028 | 1960 | 0.000016 | com.marketwire |
537 | 18347672 | 2242 | 0.000014 | com.mystrikingly |
538 | 18347018 | 2260 | 0.000013 | ch.unige |
539 | 18346850 | 2753 | 0.000011 | cat.uab |
540 | 18346818 | 2889 | 0.000011 | com.zynga |
541 | 18345164 | 1510 | 0.000020 | us.mn.state |
542 | 18341622 | 2275 | 0.000013 | com.articulate |
543 | 18340012 | 991 | 0.000033 | edu.psu |
544 | 18339422 | 2141 | 0.000014 | com.thecvf |
545 | 18339020 | 2150 | 0.000014 | es.csic |
546 | 18338922 | 2880 | 0.000011 | co.carrd |
547 | 18337380 | 1611 | 0.000019 | gov.mo |
548 | 18337360 | 2297 | 0.000013 | com.newatlas |
549 | 18335690 | 3908 | 0.000009 | jp.rdy |
550 | 18334630 | 1990 | 0.000015 | org.iea |
551 | 18333598 | 2565 | 0.000012 | com.db |
552 | 18332716 | 2310 | 0.000013 | com.webstarts |
553 | 18332584 | 2488 | 0.000012 | jp.hatenablog |
554 | 18331976 | 2331 | 0.000013 | ly.rebrand |
555 | 18331344 | 370 | 0.000069 | com.mapbox |
556 | 18331228 | 485 | 0.000053 | com.livechatinc |
557 | 18325352 | 1998 | 0.000015 | org.mozillazine |
558 | 18324944 | 2271 | 0.000013 | de.uni-freiburg |
559 | 18324472 | 1372 | 0.000023 | com.tinypic |
560 | 18324252 | 883 | 0.000036 | com.steampowered |
561 | 18323842 | 2072 | 0.000015 | uk.ac.york |
562 | 18322186 | 1097 | 0.000030 | com.thinkwithgoogle |
563 | 18320582 | 2589 | 0.000012 | ru.msu |
564 | 18320156 | 2458 | 0.000012 | org.kotlinlang |
565 | 18319540 | 1629 | 0.000019 | gov.oregon |
566 | 18318914 | 3507 | 0.000010 | com.ingress |
567 | 18318120 | 1806 | 0.000017 | gov.wi |
568 | 18318056 | 541 | 0.000049 | com.aol |
569 | 18318040 | 1969 | 0.000016 | gr.google |
570 | 18317824 | 2741 | 0.000011 | lv.draugiem |
571 | 18316720 | 2305 | 0.000013 | org.iucnredlist |
572 | 18315822 | 2035 | 0.000015 | com.broadwayworld |
573 | 18314076 | 134 | 0.000221 | com.youtube-nocookie |
574 | 18313954 | 1511 | 0.000020 | net.openid |
575 | 18313704 | 168 | 0.000158 | com.tripadvisor |
576 | 18313516 | 435 | 0.000059 | com.dailymotion |
577 | 18313398 | 1548 | 0.000019 | net.leadpages |
578 | 18313360 | 3389 | 0.000010 | com.brother |
579 | 18313186 | 2755 | 0.000011 | com.webcindario |
580 | 18313116 | 3161 | 0.000011 | es.usal |
581 | 18312962 | 2338 | 0.000013 | bg.google |
582 | 18312728 | 907 | 0.000036 | com.xiti |
583 | 18312338 | 2273 | 0.000013 | us.oh.state |
584 | 18310760 | 720 | 0.000041 | fm.last |
585 | 18310700 | 1662 | 0.000018 | net.ucoz |
586 | 18306936 | 353 | 0.000071 | org.acm |
587 | 18304074 | 3931 | 0.000009 | com.worldlingo |
588 | 18303254 | 3379 | 0.000010 | com.embarcadero |
589 | 18303110 | 1789 | 0.000017 | com.eiseverywhere |
590 | 18302838 | 2230 | 0.000014 | org.wri |
591 | 18302746 | 395 | 0.000065 | com.pubmatic |
592 | 18302516 | 470 | 0.000054 | com.goodreads |
593 | 18300826 | 2212 | 0.000014 | com.thehindubusinessline |
594 | 18300196 | 2289 | 0.000013 | com.mihanblog |
595 | 18299988 | 2037 | 0.000015 | com.intensedebate |
596 | 18298184 | 3230 | 0.000011 | com.hellomagazine |
597 | 18297298 | 3474 | 0.000010 | net.hypermart |
598 | 18294990 | 235 | 0.000103 | uk.co.amazon |
599 | 18294428 | 2711 | 0.000011 | nf.co |
600 | 18294164 | 73 | 0.000401 | me.fb |
601 | 18294088 | 542 | 0.000049 | com.entrepreneur |
602 | 18293062 | 2543 | 0.000012 | com.futurelearn |
603 | 18292644 | 2389 | 0.000013 | com.iconarchive |
604 | 18292306 | 1275 | 0.000024 | com.cognitoforms |
605 | 18292138 | 1730 | 0.000018 | org.khanacademy |
606 | 18291072 | 2041 | 0.000015 | com.financialpost |
607 | 18291000 | 1744 | 0.000017 | us.pa.state |
608 | 18288698 | 2653 | 0.000012 | com.fatcow |
609 | 18288564 | 372 | 0.000069 | com.staticflickr |
610 | 18288272 | 1851 | 0.000016 | io.bower |
611 | 18287878 | 3657 | 0.000010 | nz.govt.tepapa |
612 | 18285000 | 2199 | 0.000014 | org.prlog |
613 | 18284980 | 2743 | 0.000011 | ca.shaw |
614 | 18284736 | 2104 | 0.000014 | com.bravesites |
615 | 18283030 | 2675 | 0.000012 | de.uni-erlangen |
616 | 18282568 | 2446 | 0.000012 | org.lacity |
617 | 18282024 | 1534 | 0.000020 | fi.google |
618 | 18282018 | 2267 | 0.000013 | de.uni-koeln |
619 | 18280510 | 2464 | 0.000012 | uk.co.spectator |
620 | 18279326 | 334 | 0.000076 | com.typeform |
621 | 18279324 | 2725 | 0.000011 | is.good |
622 | 18279290 | 3399 | 0.000010 | com.114la |
623 | 18278842 | 3277 | 0.000010 | net.freeforums |
624 | 18278384 | 920 | 0.000035 | com.zoho |
625 | 18273898 | 2465 | 0.000012 | uk.ac.jisc |
626 | 18273404 | 2489 | 0.000012 | com.mnn |
627 | 18273352 | 2393 | 0.000013 | ca.dal |
628 | 18272740 | 114 | 0.000290 | com.statcounter |
629 | 18272722 | 480 | 0.000054 | com.netflix |
630 | 18272318 | 1567 | 0.000019 | com.flashtalking |
631 | 18272212 | 1903 | 0.000016 | com.prweek |
632 | 18270806 | 3115 | 0.000011 | site.negocio |
633 | 18270794 | 2080 | 0.000015 | org.lung |
634 | 18270506 | 2727 | 0.000011 | com.mouser |
635 | 18270340 | 2569 | 0.000012 | uk.co.profilebusiness |
636 | 18269840 | 3616 | 0.000010 | uk.gov.number10 |
637 | 18268146 | 3806 | 0.000009 | net.dead |
638 | 18267734 | 3375 | 0.000010 | jp.ac.kobe-u |
639 | 18267628 | 1925 | 0.000016 | uk.org.nice |
640 | 18267512 | 88 | 0.000343 | com.oculus |
641 | 18267198 | 3196 | 0.000011 | build.bazel |
642 | 18266546 | 1878 | 0.000016 | org.gentoo |
643 | 18266166 | 2181 | 0.000014 | ie.thejournal |
644 | 18266148 | 109 | 0.000310 | com.sharethis |
645 | 18265976 | 1906 | 0.000016 | org.gnupg |
646 | 18264728 | 148 | 0.000186 | ru.mail |
647 | 18263376 | 1860 | 0.000016 | com.doodlekit |
648 | 18262238 | 1948 | 0.000016 | com.crashlytics |
649 | 18262156 | 1831 | 0.000017 | org.alz |
650 | 18261954 | 2549 | 0.000012 | us.ms.state |
651 | 18261162 | 2459 | 0.000012 | com.instructure |
652 | 18260540 | 820 | 0.000040 | com.cbsnews |
653 | 18259844 | 2877 | 0.000011 | ee.ut |
654 | 18259826 | 1211 | 0.000026 | com.msdn |
655 | 18259610 | 777 | 0.000040 | com.samsung |
656 | 18257004 | 1338 | 0.000023 | com.emailmeform |
657 | 18254934 | 549 | 0.000048 | edu.cmu |
658 | 18254822 | 2496 | 0.000012 | uk.co.osoo |
659 | 18254762 | 83 | 0.000354 | com.livestream |
660 | 18254656 | 2226 | 0.000014 | com.atavist |
661 | 18252876 | 2208 | 0.000014 | fr.archives-ouvertes |
662 | 18252282 | 2792 | 0.000011 | com.cnsnews |
663 | 18252018 | 2348 | 0.000013 | io.pantheon |
664 | 18251148 | 898 | 0.000036 | com.createjs |
665 | 18251026 | 1755 | 0.000017 | us.fl.state |
666 | 18250730 | 2321 | 0.000013 | com.rabbitmq |
667 | 18250628 | 2712 | 0.000011 | uk.co.newmedianow |
668 | 18248576 | 1422 | 0.000022 | com.123formbuilder |
669 | 18247032 | 2086 | 0.000015 | gov.nh |
670 | 18243504 | 2233 | 0.000014 | org.crossref |
671 | 18242314 | 2229 | 0.000014 | us.nm.state |
672 | 18242254 | 296 | 0.000084 | com.scribd |
673 | 18241366 | 3254 | 0.000010 | ca.qc.montreal |
674 | 18240908 | 3285 | 0.000010 | uk.co.lrb |
675 | 18240828 | 135 | 0.000215 | com.youku |
676 | 18239750 | 517 | 0.000051 | com.slack |
677 | 18239658 | 2677 | 0.000012 | com.hatenadiary |
678 | 18239656 | 2292 | 0.000013 | com.itsmyurls |
679 | 18237636 | 2671 | 0.000012 | uk.org.oxonaa |
680 | 18236902 | 246 | 0.000099 | com.constantcontact |
681 | 18236862 | 3348 | 0.000010 | com.outlookindia |
682 | 18235854 | 3893 | 0.000009 | in.ac.nptel |
683 | 18235540 | 2681 | 0.000012 | uk.org.oxfam |
684 | 18235236 | 2344 | 0.000013 | com.yext |
685 | 18233812 | 256 | 0.000094 | com.getbootstrap |
686 | 18233324 | 2107 | 0.000014 | org.jenkins-ci |
687 | 18230584 | 2055 | 0.000015 | com.broadcastingcable |
688 | 18230478 | 1686 | 0.000018 | uk.gov.direct |
689 | 18230416 | 2663 | 0.000012 | com.wmtransfer |
690 | 18230374 | 1977 | 0.000015 | gov.mt |
691 | 18230164 | 2821 | 0.000011 | uk.ac.stir |
692 | 18228540 | 1052 | 0.000031 | com.marketwatch |
693 | 18227744 | 2266 | 0.000013 | com.tmcnet |
694 | 18227440 | 3136 | 0.000011 | uk.co.hsbc |
695 | 18227086 | 1798 | 0.000017 | org.nfpa |
696 | 18226792 | 2939 | 0.000011 | com.batchgeo |
697 | 18225844 | 3275 | 0.000010 | com.weightwatchers |
698 | 18225636 | 234 | 0.000103 | to.amzn |
699 | 18224632 | 3574 | 0.000010 | com.orgfree |
700 | 18223778 | 1355 | 0.000023 | org.whatbrowser |
701 | 18221814 | 2843 | 0.000011 | com.adn |
702 | 18221276 | 1190 | 0.000026 | org.weforum |
703 | 18220506 | 481 | 0.000054 | org.hbr |
704 | 18219880 | 2820 | 0.000011 | au.edu.deakin |
705 | 18219734 | 1455 | 0.000021 | org.js |
706 | 18219118 | 2445 | 0.000013 | in.ernet |
707 | 18217962 | 2854 | 0.000011 | hu.elte |
708 | 18217516 | 3025 | 0.000011 | pl.edu.uw |
709 | 18217274 | 2367 | 0.000013 | uk.org.rspb |
710 | 18216528 | 2220 | 0.000014 | com.healthgrades |
711 | 18216264 | 2779 | 0.000011 | org.carbonbrief |
712 | 18214214 | 366 | 0.000069 | com.prnewswire |
713 | 18213956 | 2088 | 0.000015 | com.tapatalk |
714 | 18213180 | 2431 | 0.000013 | org.grist |
715 | 18212750 | 3423 | 0.000010 | id.co.kaskus |
716 | 18210638 | 456 | 0.000057 | com.oreilly |
717 | 18210106 | 3587 | 0.000010 | com.skepticalscience |
718 | 18209950 | 539 | 0.000049 | gov.sec |
719 | 18209922 | 3081 | 0.000011 | com.deccanherald |
720 | 18209668 | 1905 | 0.000016 | tl.we |
721 | 18208770 | 2311 | 0.000013 | us.ma.state |
722 | 18206860 | 1101 | 0.000030 | uk.ac.cam |
723 | 18205994 | 3630 | 0.000010 | ua.meta |
724 | 18205738 | 3526 | 0.000010 | app.web |
725 | 18204462 | 2398 | 0.000013 | uk.co.zoopla |
726 | 18201966 | 3210 | 0.000011 | org.oceanconservancy |
727 | 18199630 | 3421 | 0.000010 | org.atsjournals |
728 | 18198962 | 3532 | 0.000010 | ru.my1 |
729 | 18198444 | 3162 | 0.000011 | com.mozello |
730 | 18195600 | 1562 | 0.000019 | com.pastebin |
731 | 18194580 | 2867 | 0.000011 | de.freenet |
732 | 18193414 | 1137 | 0.000028 | edu.ucla |
733 | 18193100 | 3052 | 0.000011 | com.telegraphindia |
734 | 18193026 | 2857 | 0.000011 | com.chagasi |
735 | 18192758 | 937 | 0.000034 | br.com.uol |
736 | 18188994 | 2630 | 0.000012 | com.atwebpages |
737 | 18188626 | 3036 | 0.000011 | com.remind |
738 | 18187922 | 1132 | 0.000028 | com.redhat |
739 | 18187748 | 608 | 0.000044 | com.wikihow |
740 | 18187658 | 3377 | 0.000010 | edu.utep |
741 | 18187264 | 3455 | 0.000010 | ru.nnov |
742 | 18186834 | 1881 | 0.000016 | uk.gov.defra |
743 | 18186568 | 2359 | 0.000013 | net.portfoliobox |
744 | 18185624 | 2610 | 0.000012 | com.blogsky |
745 | 18185434 | 3856 | 0.000009 | uk.co.mailonsunday |
746 | 18185432 | 2723 | 0.000011 | jp.xxxxxxxx |
747 | 18184122 | 1425 | 0.000021 | edu.ucsd |
748 | 18183962 | 1449 | 0.000021 | com.digitaltrends |
749 | 18183738 | 196 | 0.000130 | jp.ne.hatena |
750 | 18182464 | 2563 | 0.000012 | uk.co.inews |
751 | 18181728 | 2313 | 0.000013 | gov.la |
752 | 18181656 | 1266 | 0.000024 | ly.ow |
753 | 18180360 | 3441 | 0.000010 | gr.sch |
754 | 18179802 | 3055 | 0.000011 | com.sc |
755 | 18178628 | 3373 | 0.000010 | com.cummins |
756 | 18177566 | 2363 | 0.000013 | com.activerain |
757 | 18176026 | 3801 | 0.000009 | com.kazeo |
758 | 18176002 | 2901 | 0.000011 | net.onlinewebshop |
759 | 18175422 | 3689 | 0.000010 | com.galvanize |
760 | 18174902 | 3473 | 0.000010 | ru.pr-cy |
761 | 18174826 | 503 | 0.000052 | com.dmca |
762 | 18173528 | 3328 | 0.000010 | com.kaywa |
763 | 18173348 | 821 | 0.000040 | com.psychologytoday |
764 | 18172118 | 2853 | 0.000011 | uk.co.heatall |
765 | 18171416 | 84 | 0.000350 | me.ogp |
766 | 18168128 | 2601 | 0.000012 | gov.ks |
767 | 18167782 | 1516 | 0.000020 | ca.blogspot |
768 | 18167558 | 2170 | 0.000014 | com.cityam |
769 | 18167284 | 3604 | 0.000010 | gov.cabq |
770 | 18166436 | 1813 | 0.000017 | org.reactjs |
771 | 18166052 | 3283 | 0.000010 | org.escardio |
772 | 18165734 | 1064 | 0.000031 | com.foxnews |
773 | 18165680 | 1897 | 0.000016 | com.fifa |
774 | 18164860 | 204 | 0.000117 | com.naver |
775 | 18164404 | 3761 | 0.000009 | com.carscoops |
776 | 18162680 | 2928 | 0.000011 | com.ecowatch |
777 | 18162390 | 1507 | 0.000020 | com.literatumonline |
778 | 18161998 | 535 | 0.000049 | net.2mdn |
779 | 18161800 | 476 | 0.000054 | com.force |
780 | 18160578 | 159 | 0.000167 | gov.privacyshield |
781 | 18160270 | 1896 | 0.000016 | com.pcworld |
782 | 18160192 | 2986 | 0.000011 | com.theyworkforyou |
783 | 18159730 | 81 | 0.000365 | com.messenger |
784 | 18159700 | 3939 | 0.000009 | com.anghami |
785 | 18159426 | 424 | 0.000061 | edu.nyu |
786 | 18157990 | 1294 | 0.000024 | com.indiegogo |
787 | 18157828 | 1869 | 0.000016 | kr.or.kisa |
788 | 18157816 | 364 | 0.000070 | com.discordapp |
789 | 18157014 | 3186 | 0.000011 | uk.org.38degrees |
790 | 18156850 | 3628 | 0.000010 | com.insideevs |
791 | 18155496 | 1488 | 0.000020 | com.placeholder |
792 | 18155072 | 3250 | 0.000010 | google.design |
793 | 18155044 | 3764 | 0.000009 | gle.goo |
794 | 18154462 | 454 | 0.000057 | com.walmart |
795 | 18153360 | 428 | 0.000060 | com.flipboard |
796 | 18152044 | 2902 | 0.000011 | pl.lublin |
797 | 18151952 | 422 | 0.000062 | com.wufoo |
798 | 18151198 | 1123 | 0.000029 | com.shutterstock |
799 | 18150684 | 2537 | 0.000012 | org.iihs |
800 | 18149446 | 2788 | 0.000011 | in.businessworld |
801 | 18148636 | 981 | 0.000033 | com.pinimg |
802 | 18147760 | 2407 | 0.000013 | jp.e-shops |
803 | 18147734 | 2250 | 0.000014 | com.codecademy |
804 | 18146340 | 2642 | 0.000012 | com.zx2c4 |
805 | 18146328 | 129 | 0.000243 | info.aboutads |
806 | 18145944 | 2138 | 0.000014 | ca.ubc |
807 | 18145538 | 2874 | 0.000011 | com.bnef |
808 | 18144354 | 3240 | 0.000011 | uk.ac.rcplondon |
809 | 18144254 | 3718 | 0.000009 | com.wsoctv |
810 | 18143902 | 3950 | 0.000009 | com.monbiot |
811 | 18143342 | 3463 | 0.000010 | com.droppages |
812 | 18143148 | 2366 | 0.000013 | gov.arts |
813 | 18142454 | 2644 | 0.000012 | us.wi.state |
814 | 18142046 | 3477 | 0.000010 | org.usatf |
815 | 18140878 | 1624 | 0.000019 | com.nvidia |
816 | 18138866 | 3636 | 0.000010 | com.elmercurio |
817 | 18138838 | 1538 | 0.000020 | com.businessweek |
818 | 18138462 | 2176 | 0.000014 | com.tutsplus |
819 | 18138382 | 554 | 0.000047 | com.atlassian |
820 | 18137356 | 1184 | 0.000026 | com.searchengineland |
821 | 18137278 | 3594 | 0.000010 | com.glu |
822 | 18137124 | 3645 | 0.000010 | es.consumer |
823 | 18135974 | 240 | 0.000102 | cn.com.sina |
824 | 18135596 | 3948 | 0.000009 | com.allmyfaves |
825 | 18135342 | 3446 | 0.000010 | com.businessgreen |
826 | 18133642 | 350 | 0.000072 | com.163 |
827 | 18133268 | 3292 | 0.000010 | org.jython |
828 | 18133230 | 471 | 0.000054 | com.smugmug |
829 | 18132816 | 3864 | 0.000009 | org.thechicagocouncil |
830 | 18132126 | 3576 | 0.000010 | gov.azdot |
831 | 18130470 | 1176 | 0.000027 | com.ycombinator |
832 | 18129838 | 3339 | 0.000010 | org.transportenvironment |
833 | 18128538 | 2993 | 0.000011 | gov.ferc |
834 | 18127910 | 936 | 0.000034 | com.aliexpress |
835 | 18126154 | 356 | 0.000070 | com.wiley |
836 | 18125790 | 696 | 0.000042 | com.moz |
837 | 18124996 | 2756 | 0.000011 | uk.gov.environment-agency |
838 | 18124886 | 3012 | 0.000011 | org.zsl |
839 | 18124136 | 3704 | 0.000009 | org.ssireview |
840 | 18123520 | 2378 | 0.000013 | uk.gov.scotland |
841 | 18122978 | 1595 | 0.000019 | tv.ustream |
842 | 18122522 | 3096 | 0.000011 | org.dailystrength |
843 | 18122038 | 598 | 0.000045 | com.caniuse |
844 | 18120996 | 2485 | 0.000012 | net.privacypolicytemplate |
845 | 18120866 | 768 | 0.000040 | gov.noaa |
846 | 18120818 | 1573 | 0.000019 | jp.makeshop |
847 | 18120518 | 3040 | 0.000011 | org.rspo |
848 | 18119946 | 2303 | 0.000013 | com.seetickets |
849 | 18119454 | 2183 | 0.000014 | com.ign |
850 | 18118896 | 404 | 0.000064 | mp.mailchi |
851 | 18118000 | 311 | 0.000081 | com.digg |
852 | 18118000 | 2855 | 0.000011 | gov.txdot |
853 | 18117366 | 3412 | 0.000010 | uk.ac.ceh |
854 | 18117164 | 1479 | 0.000021 | com.crunchbase |
855 | 18117074 | 1127 | 0.000029 | com.highcharts |
856 | 18115870 | 2645 | 0.000012 | com.9to5mac |
857 | 18114648 | 1090 | 0.000030 | com.withgoogle |
858 | 18114314 | 889 | 0.000036 | com.webs |
859 | 18114072 | 2481 | 0.000012 | uk.co.streetmap |
860 | 18112508 | 3865 | 0.000009 | com.pushwoosh |
861 | 18111708 | 3204 | 0.000011 | ca.uwaterloo |
862 | 18111130 | 817 | 0.000040 | com.shinystat |
863 | 18111078 | 305 | 0.000082 | fr.google |
864 | 18111050 | 3467 | 0.000010 | com.baomoi |
865 | 18110974 | 3916 | 0.000009 | uk.ac.tyndall |
866 | 18110396 | 1766 | 0.000017 | com.webmasterplan |
867 | 18110180 | 3686 | 0.000010 | dk.bloggersdelight |
868 | 18109908 | 3401 | 0.000010 | uk.gov.hm-treasury |
869 | 18109262 | 1793 | 0.000017 | uk.org.cqc |
870 | 18108948 | 1248 | 0.000025 | com.smashingmagazine |
871 | 18108138 | 331 | 0.000076 | com.automattic |
872 | 18107572 | 1530 | 0.000020 | com.ning |
873 | 18106984 | 2829 | 0.000011 | com.linkwithin |
874 | 18106522 | 3002 | 0.000011 | uk.org.greenpeace |
875 | 18103768 | 956 | 0.000034 | com.libsyn |
876 | 18103538 | 1239 | 0.000025 | com.sap |
877 | 18102956 | 2091 | 0.000015 | edu.uci |
878 | 18102564 | 628 | 0.000043 | com.patreon |
879 | 18102020 | 3503 | 0.000010 | com.climatechangenews |
880 | 18101758 | 409 | 0.000063 | com.xinhuanet |
881 | 18101336 | 3464 | 0.000010 | com.kapook |
882 | 18100618 | 885 | 0.000036 | com.newyorker |
883 | 18100474 | 3640 | 0.000010 | com.spruz |
884 | 18100196 | 478 | 0.000054 | com.inc |
885 | 18100062 | 2676 | 0.000012 | jp.aikotoba |
886 | 18099268 | 914 | 0.000035 | org.eff |
887 | 18098794 | 3662 | 0.000010 | com.platts |
888 | 18098556 | 3535 | 0.000010 | org.c2es |
889 | 18098550 | 2747 | 0.000011 | com.mykaratestore |
890 | 18096786 | 1770 | 0.000017 | com.ikea |
891 | 18096394 | 1423 | 0.000022 | com.billboard |
892 | 18095092 | 1070 | 0.000031 | com.hootsuite |
893 | 18094948 | 3525 | 0.000010 | com.jkp |
894 | 18093496 | 2824 | 0.000011 | org.mcsuk |
895 | 18092622 | 1254 | 0.000025 | es.agpd |
896 | 18092438 | 3349 | 0.000010 | net.edie |
897 | 18092358 | 533 | 0.000050 | com.ea |
898 | 18092112 | 376 | 0.000068 | org.opensource |
899 | 18091568 | 2903 | 0.000011 | ru.drom |
900 | 18090162 | 2639 | 0.000012 | com.yelloyello |
901 | 18089968 | 2544 | 0.000012 | uk.co.intersol |
902 | 18089740 | 139 | 0.000202 | com.alicdn |
903 | 18089422 | 4051 | 0.000009 | com.mforos |
904 | 18086990 | 1473 | 0.000021 | com.fiverr |
905 | 18086352 | 934 | 0.000035 | com.foursquare |
906 | 18085918 | 1737 | 0.000017 | org.freecsstemplates |
907 | 18084810 | 4142 | 0.000009 | uk.org.indymedia |
908 | 18084674 | 2049 | 0.000015 | uk.gov.education |
909 | 18083694 | 3843 | 0.000009 | com.thinkbroadband |
910 | 18082164 | 231 | 0.000104 | jp.co.amazon |
911 | 18080114 | 3868 | 0.000009 | org.sciencenewsforstudents |
912 | 18080034 | 221 | 0.000108 | org.drupal |
913 | 18079726 | 1096 | 0.000030 | com.variety |
914 | 18078666 | 290 | 0.000086 | com.stumbleupon |
915 | 18078038 | 3269 | 0.000010 | net.scienceontheweb |
916 | 18077582 | 1756 | 0.000017 | com.nba |
917 | 18077452 | 2561 | 0.000012 | org.webring |
918 | 18076502 | 1033 | 0.000031 | com.visualstudio |
919 | 18075958 | 4005 | 0.000009 | io.raindrop |
920 | 18074544 | 2744 | 0.000011 | jp.zouri |
921 | 18073766 | 3904 | 0.000009 | org.corporateeurope |
922 | 18072470 | 1402 | 0.000022 | com.storify |
923 | 18071436 | 375 | 0.000069 | gov.ftc |
924 | 18071372 | 1603 | 0.000019 | net.with2 |
925 | 18070926 | 1448 | 0.000021 | com.nike |
926 | 18070222 | 4048 | 0.000009 | io.dataquest |
927 | 18070066 | 1255 | 0.000025 | org.unicef |
928 | 18069672 | 3567 | 0.000010 | bnpparibas.group |
929 | 18069172 | 3685 | 0.000010 | com.thestatesman |
930 | 18068866 | 3427 | 0.000010 | uk.org.rya |
931 | 18067508 | 383 | 0.000068 | com.airbnb |
932 | 18067204 | 1635 | 0.000019 | de.zeit |
933 | 18067190 | 2555 | 0.000012 | com.hackernoon |
934 | 18066274 | 3451 | 0.000010 | ca.pe.gov |
935 | 18065266 | 4031 | 0.000009 | com.raamdev |
936 | 18064388 | 2438 | 0.000013 | io.postach |
937 | 18064126 | 1487 | 0.000020 | edu.purdue |
938 | 18063508 | 407 | 0.000063 | com.tripod |
939 | 18063228 | 1228 | 0.000025 | gov.fbi |
940 | 18063154 | 1369 | 0.000023 | com.lifehacker |
941 | 18063130 | 1069 | 0.000031 | com.uk |
942 | 18061878 | 3432 | 0.000010 | in.gov.mhrd |
943 | 18061130 | 3527 | 0.000010 | org.gmplib |
944 | 18060100 | 3879 | 0.000009 | com.gitimmersion |
945 | 18059578 | 2807 | 0.000011 | jp.at-ninja |
946 | 18059004 | 3010 | 0.000011 | com.shichihuku |
947 | 18058826 | 3629 | 0.000010 | com.h2database |
948 | 18057736 | 3482 | 0.000010 | uk.org.rcn |
949 | 18057640 | 3737 | 0.000009 | com.writetothem |
950 | 18056592 | 1366 | 0.000023 | com.parsiblog |
951 | 18056586 | 984 | 0.000033 | com.dropboxusercontent |
952 | 18055950 | 1306 | 0.000024 | com.prweb |
953 | 18055628 | 3695 | 0.000009 | com.websiteseguro |
954 | 18055104 | 1118 | 0.000029 | com.vox |
955 | 18054272 | 1397 | 0.000022 | us.imageshack |
956 | 18053964 | 2032 | 0.000015 | com.howstuffworks |
957 | 18052920 | 1531 | 0.000020 | com.yoast |
958 | 18052280 | 1298 | 0.000024 | com.pcmag |
959 | 18051398 | 3008 | 0.000011 | uk.org.woodlandtrust |
960 | 18050936 | 3523 | 0.000010 | gle.posts |
961 | 18050800 | 3838 | 0.000009 | org.priceofoil |
962 | 18049580 | 1614 | 0.000019 | com.ccbill |
963 | 18049066 | 3750 | 0.000009 | com.fourfour |
964 | 18047214 | 945 | 0.000034 | gov.census |
965 | 18046486 | 1328 | 0.000023 | edu.wisc |
966 | 18045876 | 151 | 0.000179 | jp.co.google |
967 | 18045710 | 1220 | 0.000025 | com.blackberry |
968 | 18045414 | 1103 | 0.000030 | edu.umich |
969 | 18045390 | 1952 | 0.000016 | com.w3layouts |
970 | 18043894 | 146 | 0.000190 | me.line |
971 | 18043816 | 1593 | 0.000019 | edu.usc |
972 | 18042356 | 2842 | 0.000011 | com.zatunen |
973 | 18042240 | 500 | 0.000052 | com.nasdaq |
974 | 18042130 | 567 | 0.000046 | net.daum |
975 | 18041570 | 3118 | 0.000011 | vn.tuoitre |
976 | 18040556 | 2573 | 0.000012 | com.hisupplier |
977 | 18039444 | 2023 | 0.000015 | com.nfl |
978 | 18039370 | 927 | 0.000035 | com.ggpht |
979 | 18039324 | 1549 | 0.000019 | com.vmware |
980 | 18039020 | 3827 | 0.000009 | com.realtytimes |
981 | 18038362 | 3261 | 0.000010 | net.batcave |
982 | 18038116 | 3341 | 0.000010 | org.mygamesonline |
983 | 18037866 | 734 | 0.000040 | com.mckinsey |
984 | 18037674 | 3983 | 0.000009 | org.eia-international |
985 | 18037604 | 258 | 0.000094 | com.sohu |
986 | 18037594 | 3700 | 0.000009 | io.dropwizard |
987 | 18037394 | 1026 | 0.000032 | gov.nps |
988 | 18037244 | 2131 | 0.000014 | au.com.news |
989 | 18036608 | 3652 | 0.000010 | de.epubli |
990 | 18034198 | 1381 | 0.000022 | com.unity3d |
991 | 18034072 | 2992 | 0.000011 | net.nend |
992 | 18033048 | 4098 | 0.000009 | com.easyhits4u |
993 | 18031890 | 1162 | 0.000027 | com.steamcommunity |
994 | 18031622 | 1451 | 0.000021 | edu.uchicago |
995 | 18031570 | 1086 | 0.000030 | com.uber |
996 | 18031470 | 5306 | 0.000007 | com.plurk |
997 | 18030490 | 597 | 0.000045 | com.adweek |
998 | 18030182 | 3635 | 0.000010 | com.jal |
999 | 18029670 | 1786 | 0.000017 | com.techradar |
1000 | 18029244 | 1271 | 0.000024 | com.ifttt |
Credits
Thanks to the authors of the WebGraph framework, whose software made the computation of graph properties and ranks possible.
We hope the data will be useful for you to do any kind of research on ranking, graph analysis, link spam detection, etc. Let us know about your results via Common Crawl’s Google Group!
January 2020 crawl archive now available
The crawl archive for January 2020 is now available! It contains 3.1 billion web pages or 300 TiB of uncompressed content, crawled between January 17th and 29th. It includes page captures of 960 million URLs not contained in any crawl archive before.
Improvements and Fixes
- date time values in the column "fetch_time" of the columnar index are now stored using the "int64" data type. For details and compatibility issues please see cc-index-table#7
- WARC request records now show the HTTP protocol version sent with the HTTP request which can be different from the version received in the HTTP response message, cf. NUTCH-2760
Archive Location and Download
The January crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-05/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List #Files Total Size
Compressed (TiB)
Segments CC-MAIN-2020-05/segment.paths.gz 100
WARC files CC-MAIN-2020-05/warc.paths.gz 56000 59.94
WAT files CC-MAIN-2020-05/wat.paths.gz 56000 22.3
WET files CC-MAIN-2020-05/wet.paths.gz 56000 10
Robots.txt files CC-MAIN-2020-05/robotstxt.paths.gz 56000 0.25
Non-200 responses files CC-MAIN-2020-05/non200responses.paths.gz 56000 2.28
URL index files CC-MAIN-2020-05/cc-index.paths.gz 302 0.23
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-05/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
December 2019 crawl archive now available
The crawl archive for December 2019 is now available! It contains 2.45 billion web pages or 234 TiB of uncompressed content, crawled between December 5th and 16th. It includes page captures of 850 million URLs not contained in any crawl archive before.
Archive Location and Download
The December crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2019-51/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List #Files Total Size
Compressed (TiB)
Segments CC-MAIN-2019-51/segment.paths.gz 100
WARC files CC-MAIN-2019-51/warc.paths.gz 56000 47.47
WAT files CC-MAIN-2019-51/wat.paths.gz 56000 17.6
WET files CC-MAIN-2019-51/wet.paths.gz 56000 8.06
Robots.txt files CC-MAIN-2019-51/robotstxt.paths.gz 56000 0.26
Non-200 responses files CC-MAIN-2019-51/non200responses.paths.gz 56000 3.5
URL index files CC-MAIN-2019-51/cc-index.paths.gz 302 0.19
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2019-51/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.