Host- and Domain-Level Web Graphs Jul/Aug/Sep 2020
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of July, August and September 2020. Additional information about the data formats, the processing pipeline, our objectives, and credits can be found in the announcements of prior webgraph releases (e.g., Nov/Dec/Jan 2017-2018 Webgraphs). You may also visit the projects cc-webgraph and cc-pyspark which host all scripts and tools required to construct the graphs.
Host-level graph
The graph consists of 539 million nodes and 3.02 billion edges and includes dangling nodes i.e. hosts that have not been crawled yet are pointed to from a link on a crawled page. There are 467 million dangling nodes (86.7%) and the largest strongly connected component contains 46 million (8.5%) nodes.
You can download the graph and the ranks of all 539 million hosts from AWS S3 on the path s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-jul-aug-sep/host/
. Alternatively, you can use https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2020-jul-aug-sep/host/
as prefix to access the files from everywhere.
Size | File | Description |
---|---|---|
3.32 GB | cc-main-2020-jul-aug-sep-host-vertices.paths.gz | nodes 〈id, rev host〉, paths of 12 vertices files |
13.7 GB | cc-main-2020-jul-aug-sep-host-edges.paths.gz | edges 〈from_id, to_id〉, paths of 24 edges files |
5.95 GB | cc-main-2020-jul-aug-sep-host.graph | graph in BVGraph format |
2 kB | cc-main-2020-jul-aug-sep-host.properties | |
6.76 GB | cc-main-2020-jul-aug-sep-host-t.graph | transpose of the graph (outlinks inverted to inlinks) |
2 kB | cc-main-2020-jul-aug-sep-host-t.properties | |
1 kB | cc-main-2020-jul-aug-sep-host.stats | WebGraph statistics |
7.77 GB | cc-main-2020-jul-aug-sep-host-ranks.txt.gz | harmonic centrality and pagerank |
Note that the host names are reversed and a leading www.
is stripped: www.subdomain.example.com
becomes com.example.subdomain
.
Domain-level graph
The domain graph was built by aggregating the host graph on the level of pay-level domains (PLDs) based on the public suffix list maintained on publicsuffix.org.
The domain-level graph has 89 million nodes and 1.71 billion edges. 51% or 45 million nodes are dangling nodes, the largest strongly connected component covers 35 million or 39% of the nodes.
All files related to the domain graph are available on AWS S3 under s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-jul-aug-sep/domain/
resp. https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2020-jul-aug-sep/domain/
.
Download files of the Common Crawl Jul/Aug/Sep 2020 domain-level webgraph
Size | File | Description |
---|---|---|
0.61 GB | cc-main-2020-jul-aug-sep-domain-vertices.txt.gz | nodes 〈id, rev domain, num hosts〉 |
6.80 GB | cc-main-2020-jul-aug-sep-domain-edges.txt.gz | edges 〈from_id, to_id〉 |
3.75 GB | cc-main-2020-jul-aug-sep-domain.graph | graph in BVGraph format |
2 kB | cc-main-2020-jul-aug-sep-domain.properties | |
3.69 GB | cc-main-2020-jul-aug-sep-domain-t.graph | transpose of the graph |
2 kB | cc-main-2020-jul-aug-sep-domain-t.properties | |
1 kB | cc-main-2020-jul-aug-sep-domain.stats | WebGraph statistics |
1.91 GB | cc-main-2020-jul-aug-sep-domain-ranks.txt.gz | harmonic centrality and pagerank |
Below you’ll find the top 1000 domains ranked by Harmonic Centrality or PageRank. The full list of all 89 million domain ranks is available for download.
Top 1000 domains ranked by harmonic centrality (Jul/Aug/Sep 2020)
harmonic centrality rank | hc value | page rank | page rank value | reversed hostname |
---|---|---|---|---|
1 | 32027928 | 1 | 0.018888 | com.googleapis |
2 | 30312944 | 3 | 0.012001 | com.facebook |
3 | 29025948 | 2 | 0.013237 | com.google |
4 | 26560472 | 4 | 0.007343 | org.w |
5 | 26516534 | 5 | 0.007172 | com.twitter |
6 | 26016464 | 6 | 0.006600 | com.youtube |
7 | 24614190 | 9 | 0.004795 | com.instagram |
8 | 24220712 | 8 | 0.005190 | org.gmpg |
9 | 23572970 | 7 | 0.005599 | com.googletagmanager |
10 | 23188190 | 11 | 0.003202 | com.linkedin |
11 | 22457894 | 15 | 0.002590 | com.gravatar |
12 | 22451350 | 10 | 0.003967 | com.cloudflare |
13 | 22364152 | 14 | 0.002726 | com.gstatic |
14 | 22350042 | 12 | 0.003105 | org.wordpress |
15 | 21926906 | 22 | 0.001505 | com.pinterest |
16 | 21699168 | 21 | 0.001752 | com.wordpress |
17 | 21599006 | 26 | 0.001181 | org.wikipedia |
18 | 21538264 | 16 | 0.002431 | com.bootstrapcdn |
19 | 21497526 | 18 | 0.001836 | com.apple |
20 | 21314410 | 30 | 0.001106 | com.vimeo |
21 | 21248994 | 41 | 0.000830 | be.youtu |
22 | 21186566 | 20 | 0.001794 | com.jquery |
23 | 21081822 | 23 | 0.001444 | com.microsoft |
24 | 21073240 | 45 | 0.000773 | com.blogspot |
25 | 20994964 | 39 | 0.000952 | com.amazonaws |
26 | 20975988 | 46 | 0.000732 | gl.goo |
27 | 20971574 | 25 | 0.001384 | com.wp |
28 | 20921220 | 47 | 0.000723 | com.amazon |
29 | 20788608 | 72 | 0.000439 | com.tumblr |
30 | 20716256 | 19 | 0.001804 | com.adobe |
31 | 20694562 | 67 | 0.000535 | ly.bit |
32 | 20675418 | 34 | 0.001018 | com.google-analytics |
33 | 20627694 | 53 | 0.000673 | org.mozilla |
34 | 20618998 | 17 | 0.001975 | com.github |
35 | 20617620 | 31 | 0.001059 | net.cloudfront |
36 | 20579928 | 71 | 0.000449 | com.yahoo |
37 | 20571130 | 29 | 0.001127 | com.googlesyndication |
38 | 20570586 | 60 | 0.000612 | eu.europa |
39 | 20562028 | 52 | 0.000679 | com.flickr |
40 | 20560188 | 42 | 0.000818 | net.jsdelivr |
41 | 20526264 | 97 | 0.000347 | com.googleusercontent |
42 | 20481758 | 62 | 0.000606 | co.t |
43 | 20480218 | 109 | 0.000313 | com.reddit |
44 | 20451670 | 24 | 0.001419 | com.fontawesome |
45 | 20436180 | 83 | 0.000389 | com.weebly |
46 | 20387228 | 56 | 0.000628 | com.paypal |
47 | 20375802 | 40 | 0.000910 | com.macromedia |
48 | 20372972 | 70 | 0.000450 | com.medium |
49 | 20370180 | 43 | 0.000808 | com.addthis |
50 | 20360678 | 28 | 0.001156 | ru.yandex |
51 | 20338498 | 27 | 0.001156 | me.wp |
52 | 20331252 | 64 | 0.000559 | org.w3 |
53 | 20326560 | 79 | 0.000411 | io.github |
54 | 20292836 | 138 | 0.000223 | com.nytimes |
55 | 20275824 | 76 | 0.000414 | org.creativecommons |
56 | 20274244 | 59 | 0.000615 | org.schema |
57 | 20255326 | 150 | 0.000192 | com.forbes |
58 | 20246068 | 173 | 0.000151 | com.imgur |
59 | 20227930 | 36 | 0.000979 | net.doubleclick |
60 | 20219612 | 194 | 0.000133 | uk.co.bbc |
61 | 20210924 | 114 | 0.000285 | com.soundcloud |
62 | 20171070 | 66 | 0.000548 | com.vk |
63 | 20155222 | 195 | 0.000133 | com.cnn |
64 | 20142696 | 44 | 0.000803 | org.apache |
65 | 20134806 | 63 | 0.000587 | com.whatsapp |
66 | 20129582 | 314 | 0.000082 | edu.mit |
67 | 20123032 | 180 | 0.000146 | com.imdb |
68 | 20118310 | 208 | 0.000124 | net.slideshare |
69 | 20116626 | 243 | 0.000101 | com.wsj |
70 | 20115768 | 197 | 0.000128 | org.wikimedia |
71 | 20089462 | 85 | 0.000388 | com.shopify |
72 | 20082204 | 215 | 0.000120 | edu.stanford |
73 | 20076684 | 154 | 0.000181 | gov.cdc |
74 | 20075632 | 328 | 0.000079 | com.wired |
75 | 20069724 | 268 | 0.000094 | com.techcrunch |
76 | 20057066 | 255 | 0.000096 | edu.harvard |
77 | 20051336 | 353 | 0.000076 | com.appspot |
78 | 20051292 | 207 | 0.000124 | net.sourceforge |
79 | 20051264 | 257 | 0.000096 | com.oracle |
80 | 20051250 | 155 | 0.000177 | int.who |
81 | 20050888 | 206 | 0.000124 | com.businessinsider |
82 | 20046050 | 137 | 0.000227 | org.archive |
83 | 20038198 | 230 | 0.000113 | com.washingtonpost |
84 | 20035810 | 250 | 0.000097 | com.live |
85 | 20029940 | 164 | 0.000163 | com.bing |
86 | 20028210 | 549 | 0.000054 | com.livejournal |
87 | 20027622 | 424 | 0.000069 | com.go |
88 | 20024666 | 456 | 0.000066 | com.msn |
89 | 20019992 | 407 | 0.000072 | uk.co.telegraph |
90 | 20009306 | 170 | 0.000154 | com.theguardian |
91 | 20002514 | 527 | 0.000056 | edu.cornell |
92 | 19997146 | 199 | 0.000128 | org.ietf |
93 | 19996714 | 486 | 0.000063 | gov.nasa |
94 | 19995476 | 259 | 0.000096 | com.android |
95 | 19986252 | 302 | 0.000084 | com.reuters |
96 | 19983946 | 51 | 0.000702 | net.fbcdn |
97 | 19974890 | 240 | 0.000102 | com.bloomberg |
98 | 19966464 | 162 | 0.000164 | com.giphy |
99 | 19960428 | 77 | 0.000414 | com.list-manage |
100 | 19959046 | 520 | 0.000057 | com.googleblog |
101 | 19956558 | 269 | 0.000093 | com.bbc |
102 | 19955204 | 409 | 0.000071 | com.slack |
103 | 19942056 | 143 | 0.000205 | com.spotify |
104 | 19938828 | 591 | 0.000049 | com.zdnet |
105 | 19936894 | 48 | 0.000721 | net.facebook |
106 | 19935010 | 586 | 0.000050 | com.quora |
107 | 19931072 | 126 | 0.000265 | com.ytimg |
108 | 19922774 | 444 | 0.000067 | com.myspace |
109 | 19922046 | 757 | 0.000038 | edu.umich |
110 | 19920178 | 715 | 0.000040 | edu.upenn |
111 | 19917482 | 151 | 0.000185 | gov.nih |
112 | 19907886 | 344 | 0.000077 | com.usatoday |
113 | 19903896 | 654 | 0.000045 | com.economist |
114 | 19903722 | 313 | 0.000082 | com.cnbc |
115 | 19902700 | 308 | 0.000083 | com.example |
116 | 19896552 | 525 | 0.000056 | com.pixabay |
117 | 19895014 | 418 | 0.000070 | net.researchgate |
118 | 19882790 | 449 | 0.000066 | com.latimes |
119 | 19881164 | 188 | 0.000138 | com.blogger |
120 | 19870046 | 387 | 0.000075 | org.python |
121 | 19864804 | 65 | 0.000555 | com.wix |
122 | 19860760 | 433 | 0.000068 | com.githubusercontent |
123 | 19858732 | 693 | 0.000042 | org.ieee |
124 | 19854254 | 499 | 0.000061 | com.mashable |
125 | 19850918 | 571 | 0.000052 | edu.berkeley |
126 | 19847554 | 135 | 0.000241 | com.youtube-nocookie |
127 | 19845130 | 160 | 0.000167 | com.issuu |
128 | 19843068 | 218 | 0.000118 | org.acm |
129 | 19839736 | 834 | 0.000036 | org.chromium |
130 | 19839550 | 235 | 0.000106 | uk.co.google |
131 | 19835790 | 551 | 0.000054 | org.arxiv |
132 | 19833020 | 246 | 0.000099 | net.behance |
133 | 19832682 | 291 | 0.000086 | org.npr |
134 | 19831994 | 108 | 0.000320 | com.unpkg |
135 | 19831136 | 884 | 0.000034 | com.arstechnica |
136 | 19826840 | 213 | 0.000121 | com.unsplash |
137 | 19822884 | 341 | 0.000078 | com.outlook |
138 | 19822670 | 110 | 0.000303 | de.google |
139 | 19812430 | 54 | 0.000654 | com.googleadservices |
140 | 19810872 | 347 | 0.000077 | com.prnewswire |
141 | 19806458 | 678 | 0.000043 | edu.columbia |
142 | 19805382 | 171 | 0.000153 | me.t |
143 | 19804886 | 297 | 0.000085 | com.dribbble |
144 | 19804142 | 256 | 0.000096 | com.squarespace |
145 | 19799032 | 139 | 0.000215 | gov.privacyshield |
146 | 19798806 | 306 | 0.000083 | com.huffingtonpost |
147 | 19797964 | 260 | 0.000096 | com.bandcamp |
148 | 19795112 | 398 | 0.000074 | com.time |
149 | 19793874 | 37 | 0.000975 | com.baidu |
150 | 19792082 | 616 | 0.000048 | com.gitlab |
151 | 19790406 | 334 | 0.000079 | com.nationalgeographic |
152 | 19788214 | 443 | 0.000067 | com.nature |
153 | 19785178 | 794 | 0.000037 | com.stackexchange |
154 | 19782114 | 179 | 0.000147 | gle.forms |
155 | 19781676 | 258 | 0.000096 | org.ampproject |
156 | 19778534 | 548 | 0.000054 | com.fortune |
157 | 19777902 | 813 | 0.000036 | com.git-scm |
158 | 19776608 | 33 | 0.001030 | com.wixstatic |
159 | 19774030 | 771 | 0.000038 | com.qz |
160 | 19772390 | 281 | 0.000089 | com.wiley |
161 | 19772268 | 646 | 0.000046 | au.net.abc |
162 | 19770930 | 638 | 0.000046 | edu.yale |
163 | 19769582 | 428 | 0.000068 | com.meetup |
164 | 19767876 | 468 | 0.000064 | com.ted |
165 | 19761386 | 1160 | 0.000026 | com.hatenablog |
166 | 19759052 | 448 | 0.000066 | com.patreon |
167 | 19757472 | 283 | 0.000089 | com.disqus |
168 | 19756748 | 936 | 0.000032 | edu.ucla |
169 | 19753998 | 147 | 0.000195 | com.dropbox |
170 | 19753380 | 168 | 0.000158 | com.yelp |
171 | 19750678 | 271 | 0.000093 | org.un |
172 | 19746384 | 212 | 0.000122 | com.twimg |
173 | 19743118 | 254 | 0.000096 | org.drupal |
174 | 19741474 | 689 | 0.000042 | org.bitbucket |
175 | 19736540 | 422 | 0.000069 | com.statista |
176 | 19735440 | 903 | 0.000033 | uk.ac.cam |
177 | 19731940 | 718 | 0.000040 | com.evernote |
178 | 19731916 | 682 | 0.000043 | com.newyorker |
179 | 19725638 | 603 | 0.000049 | com.buzzfeed |
180 | 19719544 | 606 | 0.000049 | me.about |
181 | 19718654 | 722 | 0.000040 | com.mysql |
182 | 19716804 | 850 | 0.000035 | com.thenextweb |
183 | 19715420 | 495 | 0.000061 | com.theatlantic |
184 | 19710920 | 279 | 0.000091 | com.sciencedirect |
185 | 19710826 | 403 | 0.000073 | com.getpocket |
186 | 19705326 | 669 | 0.000043 | uk.co.blogspot |
187 | 19702126 | 1293 | 0.000023 | com.tinypic |
188 | 19696730 | 450 | 0.000066 | com.booking |
189 | 19695652 | 514 | 0.000058 | com.xinhuanet |
190 | 19694904 | 743 | 0.000039 | org.weforum |
191 | 19694268 | 247 | 0.000098 | gov.ca |
192 | 19692322 | 602 | 0.000049 | gov.loc |
193 | 19690998 | 1282 | 0.000023 | org.postgresql |
194 | 19689908 | 828 | 0.000036 | edu.princeton |
195 | 19687954 | 239 | 0.000103 | uk.co.amazon |
196 | 19685942 | 480 | 0.000063 | com.dailymotion |
197 | 19679672 | 1452 | 0.000021 | ru.narod |
198 | 19678926 | 189 | 0.000138 | com.xing |
199 | 19675914 | 879 | 0.000034 | edu.jhu |
200 | 19673670 | 500 | 0.000060 | gov.whitehouse |
201 | 19671846 | 665 | 0.000044 | org.worldbank |
202 | 19668706 | 1365 | 0.000022 | org.eclipse |
203 | 19667770 | 400 | 0.000073 | com.springer |
204 | 19667684 | 445 | 0.000067 | com.nypost |
205 | 19665872 | 316 | 0.000081 | com.ft |
206 | 19660930 | 61 | 0.000606 | com.fb |
207 | 19658986 | 204 | 0.000125 | com.feedburner |
208 | 19658394 | 826 | 0.000036 | org.cambridge |
209 | 19654762 | 476 | 0.000063 | uk.co.dailymail |
210 | 19654386 | 766 | 0.000038 | edu.washington |
211 | 19654242 | 496 | 0.000061 | org.eff |
212 | 19653044 | 32 | 0.001054 | com.qq |
213 | 19650144 | 473 | 0.000064 | com.goodreads |
214 | 19649524 | 264 | 0.000095 | org.doi |
215 | 19649502 | 512 | 0.000058 | com.w3schools |
216 | 19641242 | 1311 | 0.000023 | edu.virginia |
217 | 19641212 | 440 | 0.000067 | com.googlecode |
218 | 19638348 | 633 | 0.000047 | com.vice |
219 | 19633128 | 506 | 0.000059 | com.force |
220 | 19632976 | 723 | 0.000040 | com.trello |
221 | 19632780 | 836 | 0.000035 | com.about |
222 | 19630562 | 523 | 0.000056 | com.inc |
223 | 19629482 | 453 | 0.000066 | com.scribd |
224 | 19629368 | 2053 | 0.000016 | com.wikidot |
225 | 19628436 | 619 | 0.000048 | org.semver |
226 | 19614496 | 607 | 0.000049 | com.cbsnews |
227 | 19607794 | 651 | 0.000045 | com.withgoogle |
228 | 19605512 | 146 | 0.000196 | me.line |
229 | 19603410 | 2089 | 0.000016 | com.googlesource |
230 | 19601476 | 219 | 0.000118 | org.iana |
231 | 19601452 | 546 | 0.000054 | gov.usda |
232 | 19599800 | 309 | 0.000083 | com.tinyurl |
233 | 19598290 | 1090 | 0.000027 | com.techradar |
234 | 19597674 | 858 | 0.000035 | com.dropboxusercontent |
235 | 19597446 | 384 | 0.000076 | com.ibm |
236 | 19595200 | 1284 | 0.000023 | co.elastic |
237 | 19594024 | 289 | 0.000087 | com.squareup |
238 | 19593336 | 1434 | 0.000021 | org.linuxfoundation |
239 | 19592388 | 1134 | 0.000026 | org.coursera |
240 | 19589830 | 1027 | 0.000029 | gov.fbi |
241 | 19588284 | 1158 | 0.000026 | edu.unc |
242 | 19586008 | 705 | 0.000041 | com.vox |
243 | 19583350 | 193 | 0.000134 | de.amazon |
244 | 19583096 | 550 | 0.000054 | uk.co.independent |
245 | 19580554 | 1423 | 0.000021 | ms.1drv |
246 | 19578950 | 383 | 0.000076 | com.digg |
247 | 19567612 | 1393 | 0.000022 | org.kernel |
248 | 19563948 | 113 | 0.000287 | com.sharethis |
249 | 19563468 | 751 | 0.000039 | org.d3js |
250 | 19557490 | 801 | 0.000037 | gov.fcc |
251 | 19557292 | 1026 | 0.000029 | com.hollywoodreporter |
252 | 19556258 | 1369 | 0.000022 | com.howstuffworks |
253 | 19553700 | 430 | 0.000068 | com.cnet |
254 | 19552068 | 804 | 0.000037 | com.foxnews |
255 | 19547134 | 152 | 0.000183 | com.addtoany |
256 | 19547006 | 644 | 0.000046 | com.indiatimes |
257 | 19546928 | 995 | 0.000029 | com.steamcommunity |
258 | 19546864 | 1105 | 0.000026 | cn.com.chinadaily |
259 | 19545628 | 584 | 0.000050 | com.psychologytoday |
260 | 19544130 | 823 | 0.000036 | uk.co.guardian |
261 | 19543920 | 1463 | 0.000021 | it.scoop |
262 | 19543754 | 133 | 0.000247 | com.mailchimp |
263 | 19542234 | 837 | 0.000035 | com.slate |
264 | 19542214 | 153 | 0.000182 | com.opera |
265 | 19538412 | 589 | 0.000050 | com.mckinsey |
266 | 19536816 | 1020 | 0.000029 | com.sap |
267 | 19536418 | 2605 | 0.000013 | org.wikiquote |
268 | 19534334 | 307 | 0.000083 | com.bitly |
269 | 19533308 | 627 | 0.000047 | com.mozilla |
270 | 19533054 | 262 | 0.000095 | jp.ameblo |
271 | 19531260 | 735 | 0.000039 | org.sciencemag |
272 | 19528246 | 116 | 0.000284 | com.paypalobjects |
273 | 19528108 | 2345 | 0.000014 | org.wikibooks |
274 | 19527104 | 176 | 0.000151 | com.amazon-adsystem |
275 | 19526948 | 688 | 0.000042 | gov.noaa |
276 | 19524868 | 305 | 0.000083 | com.netdna-ssl |
277 | 19524544 | 310 | 0.000083 | com.nbcnews |
278 | 19523330 | 989 | 0.000030 | com.target |
279 | 19522776 | 1523 | 0.000020 | com.instructables |
280 | 19517526 | 975 | 0.000030 | edu.umn |
281 | 19516530 | 965 | 0.000031 | com.merriam-webster |
282 | 19516260 | 1431 | 0.000021 | hk.com.google |
283 | 19514852 | 185 | 0.000140 | com.tripadvisor |
284 | 19514608 | 2377 | 0.000014 | com.diigo |
285 | 19503916 | 497 | 0.000061 | ca.google |
286 | 19499262 | 236 | 0.000106 | com.wpengine |
287 | 19499246 | 1029 | 0.000028 | com.sun |
288 | 19496562 | 1189 | 0.000025 | com.digitaltrends |
289 | 19496340 | 391 | 0.000075 | com.stumbleupon |
290 | 19491846 | 115 | 0.000284 | com.weibo |
291 | 19491638 | 1626 | 0.000019 | com.ign |
292 | 19491210 | 1314 | 0.000023 | com.mercurynews |
293 | 19490964 | 1352 | 0.000022 | de.zeit |
294 | 19490636 | 229 | 0.000114 | com.etsy |
295 | 19489106 | 797 | 0.000037 | uk.ac.ox |
296 | 19487454 | 284 | 0.000089 | com.optimizely |
297 | 19485106 | 73 | 0.000425 | net.akamaihd |
298 | 19484368 | 1207 | 0.000025 | net.speedtest |
299 | 19484284 | 1522 | 0.000020 | org.greenpeace |
300 | 19483622 | 1553 | 0.000020 | net.seesaa |
301 | 19479450 | 720 | 0.000040 | au.com.google |
302 | 19478604 | 904 | 0.000033 | de.spiegel |
303 | 19476336 | 1077 | 0.000027 | com.podbean |
304 | 19475142 | 628 | 0.000047 | org.pbs |
305 | 19474722 | 516 | 0.000058 | com.gofundme |
306 | 19474484 | 416 | 0.000070 | com.kickstarter |
307 | 19473590 | 1340 | 0.000022 | com.urbandictionary |
308 | 19472422 | 472 | 0.000064 | org.pewresearch |
309 | 19471320 | 519 | 0.000057 | com.bigcommerce |
310 | 19467912 | 2137 | 0.000015 | de.bild |
311 | 19467240 | 231 | 0.000112 | com.eepurl |
312 | 19465300 | 515 | 0.000058 | com.theverge |
313 | 19464792 | 273 | 0.000092 | com.stackoverflow |
314 | 19464598 | 926 | 0.000032 | com.politico |
315 | 19463036 | 811 | 0.000036 | co.ibb |
316 | 19462394 | 332 | 0.000079 | it.google |
317 | 19462162 | 2110 | 0.000016 | ly.visual |
318 | 19461840 | 955 | 0.000031 | org.unicef |
319 | 19460932 | 2020 | 0.000016 | org.tensorflow |
320 | 19457592 | 1688 | 0.000018 | com.itv |
321 | 19457150 | 1013 | 0.000029 | com.lifehacker |
322 | 19456512 | 106 | 0.000334 | com.stripe |
323 | 19456272 | 1349 | 0.000022 | edu.msu |
324 | 19455412 | 312 | 0.000083 | net.windows |
325 | 19453374 | 805 | 0.000037 | edu.academia |
326 | 19450284 | 1391 | 0.000022 | com.storify |
327 | 19449638 | 1257 | 0.000024 | com.crunchbase |
328 | 19449386 | 595 | 0.000049 | com.tandfonline |
329 | 19449132 | 1958 | 0.000017 | com.lego |
330 | 19444682 | 1187 | 0.000025 | com.jetbrains |
331 | 19443796 | 677 | 0.000043 | gov.senate |
332 | 19443664 | 855 | 0.000035 | com.chicagotribune |
333 | 19443234 | 2301 | 0.000014 | com.rottentomatoes |
334 | 19440224 | 770 | 0.000038 | ca.cbc |
335 | 19439934 | 205 | 0.000125 | com.eventbrite |
336 | 19439496 | 1273 | 0.000023 | hk.hku |
337 | 19436402 | 1035 | 0.000028 | edu.wisc |
338 | 19436104 | 691 | 0.000042 | com.libsyn |
339 | 19435742 | 1051 | 0.000028 | edu.northwestern |
340 | 19433212 | 944 | 0.000031 | com.scientificamerican |
341 | 19432798 | 1043 | 0.000028 | edu.uchicago |
342 | 19431182 | 1288 | 0.000023 | uk.co.wired |
343 | 19425546 | 190 | 0.000137 | jp.co.google |
344 | 19424346 | 2002 | 0.000016 | org.maven |
345 | 19423732 | 1030 | 0.000028 | com.mediafire |
346 | 19423350 | 415 | 0.000070 | me.telegram |
347 | 19418440 | 396 | 0.000074 | com.criteo |
348 | 19417208 | 357 | 0.000076 | fr.google |
349 | 19417038 | 664 | 0.000044 | us.icio |
350 | 19416402 | 1477 | 0.000020 | com.deadline |
351 | 19415808 | 640 | 0.000046 | com.sagepub |
352 | 19414256 | 730 | 0.000039 | com.ecwid |
353 | 19413466 | 1275 | 0.000023 | org.aclu |
354 | 19413258 | 576 | 0.000051 | com.typepad |
355 | 19412168 | 471 | 0.000064 | com.photobucket |
356 | 19407294 | 533 | 0.000055 | com.oup |
357 | 19407168 | 1199 | 0.000025 | com.reverbnation |
358 | 19406968 | 1514 | 0.000020 | de.mpg |
359 | 19405330 | 1389 | 0.000022 | edu.rutgers |
360 | 19404790 | 1067 | 0.000027 | com.scmp |
361 | 19403976 | 81 | 0.000392 | net.jsfiddle |
362 | 19403692 | 421 | 0.000069 | com.calendly |
363 | 19403618 | 844 | 0.000035 | com.sciencedaily |
364 | 19403468 | 727 | 0.000039 | gov.justice |
365 | 19400830 | 575 | 0.000051 | gov.hhs |
366 | 19398258 | 919 | 0.000032 | com.theconversation |
367 | 19397596 | 991 | 0.000030 | com.apnews |
368 | 19397442 | 938 | 0.000032 | com.huffpost |
369 | 19394934 | 1518 | 0.000020 | com.newscientist |
370 | 19394656 | 608 | 0.000049 | org.openstreetmap |
371 | 19393300 | 1287 | 0.000023 | com.aljazeera |
372 | 19393230 | 216 | 0.000119 | com.hubspot |
373 | 19390018 | 645 | 0.000046 | gov.house |
374 | 19388118 | 2682 | 0.000012 | uk.co.timesonline |
375 | 19388034 | 2564 | 0.000013 | com.space |
376 | 19383910 | 700 | 0.000041 | com.pinimg |
377 | 19383504 | 432 | 0.000068 | page.g |
378 | 19381990 | 1241 | 0.000024 | com.sky |
379 | 19381844 | 866 | 0.000035 | gov.congress |
380 | 19381026 | 912 | 0.000033 | com.500px |
381 | 19380632 | 1217 | 0.000024 | org.wiktionary |
382 | 19380340 | 958 | 0.000031 | com.ssrn |
383 | 19379742 | 1709 | 0.000018 | edu.bu |
384 | 19377640 | 1757 | 0.000018 | gov.cia |
385 | 19375740 | 214 | 0.000120 | org.bbb |
386 | 19375634 | 1438 | 0.000021 | com.foxbusiness |
387 | 19371814 | 624 | 0.000047 | ru.gov |
388 | 19371056 | 1598 | 0.000019 | ca.mcgill |
389 | 19367926 | 790 | 0.000037 | com.qualtrics |
390 | 19366054 | 1290 | 0.000023 | org.semanticscholar |
391 | 19365778 | 761 | 0.000038 | site.business |
392 | 19365760 | 267 | 0.000094 | ru.ok |
393 | 19363798 | 977 | 0.000030 | edu.si |
394 | 19363758 | 887 | 0.000034 | br.com.google |
395 | 19363688 | 847 | 0.000035 | co.g |
396 | 19363204 | 1021 | 0.000029 | uk.co.thetimes |
397 | 19362122 | 2663 | 0.000012 | com.discovermagazine |
398 | 19359920 | 182 | 0.000142 | us.zoom |
399 | 19359492 | 889 | 0.000034 | org.fao |
400 | 19359352 | 683 | 0.000043 | org.change |
401 | 19357866 | 1469 | 0.000020 | com.salon |
402 | 19356650 | 228 | 0.000114 | com.aliyuncs |
403 | 19356280 | 997 | 0.000029 | com.thehill |
404 | 19354818 | 973 | 0.000030 | gov.usgs |
405 | 19351584 | 298 | 0.000085 | com.ebay |
406 | 19350988 | 1222 | 0.000024 | com.nikkei |
407 | 19350142 | 338 | 0.000078 | com.rawgit |
408 | 19349660 | 578 | 0.000051 | it.placehold |
409 | 19348824 | 157 | 0.000173 | com.wixsite |
410 | 19348122 | 1238 | 0.000024 | com.smithsonianmag |
411 | 19346552 | 758 | 0.000038 | org.oecd |
412 | 19346514 | 1088 | 0.000027 | ee.linktr |
413 | 19345254 | 3312 | 0.000011 | com.openai |
414 | 19342288 | 1048 | 0.000028 | uk.co.mirror |
415 | 19341656 | 679 | 0.000043 | com.deviantart |
416 | 19341332 | 1576 | 0.000019 | org.phys |
417 | 19340598 | 413 | 0.000070 | tv.twitch |
418 | 19340138 | 404 | 0.000072 | com.mapbox |
419 | 19335246 | 1546 | 0.000020 | ca.sfu |
420 | 19332464 | 2754 | 0.000012 | com.instapaper |
421 | 19330656 | 244 | 0.000100 | org.gnu |
422 | 19330504 | 2115 | 0.000016 | au.edu.unimelb |
423 | 19328724 | 1044 | 0.000028 | int.coe |
424 | 19328320 | 2078 | 0.000016 | org.nobelprize |
425 | 19328286 | 667 | 0.000043 | pl.google |
426 | 19327680 | 1333 | 0.000022 | com.irishtimes |
427 | 19327578 | 293 | 0.000086 | com.office |
428 | 19327536 | 1962 | 0.000017 | org.torproject |
429 | 19324936 | 484 | 0.000063 | net.imgix |
430 | 19324628 | 1281 | 0.000023 | uk.ac.ucl |
431 | 19320926 | 1054 | 0.000028 | org.ohchr |
432 | 19318772 | 1213 | 0.000025 | com.strikingly |
433 | 19315502 | 509 | 0.000059 | org.hbr |
434 | 19315040 | 1411 | 0.000021 | uk.co.metro |
435 | 19314304 | 123 | 0.000270 | com.statcounter |
436 | 19313468 | 972 | 0.000030 | gov.dhs |
437 | 19313380 | 287 | 0.000088 | com.thedailybeast |
438 | 19313234 | 1811 | 0.000017 | com.bankofamerica |
439 | 19312534 | 1265 | 0.000024 | com.buzzsprout |
440 | 19311940 | 863 | 0.000035 | gov.nps |
441 | 19309868 | 2426 | 0.000014 | au.com.theage |
442 | 19307472 | 933 | 0.000032 | com.aweber |
443 | 19306766 | 1557 | 0.000020 | blog.home |
444 | 19305448 | 848 | 0.000035 | gov.bls |
445 | 19305296 | 490 | 0.000062 | edu.nyu |
446 | 19304346 | 2087 | 0.000016 | com.oxforddictionaries |
447 | 19304074 | 1162 | 0.000025 | gov.nyc |
448 | 19303568 | 93 | 0.000356 | org.reactjs |
449 | 19302778 | 1382 | 0.000022 | au.com.news |
450 | 19300882 | 2291 | 0.000014 | sg.edu.nus |
451 | 19299900 | 1429 | 0.000021 | com.flipboard |
452 | 19299896 | 481 | 0.000063 | com.scorecardresearch |
453 | 19298010 | 2517 | 0.000013 | com.dummies |
454 | 19295840 | 2465 | 0.000013 | org.rsc |
455 | 19295472 | 1010 | 0.000029 | com.britannica |
456 | 19294984 | 714 | 0.000040 | gov.state |
457 | 19294216 | 1700 | 0.000018 | org.gutenberg |
458 | 19292892 | 3565 | 0.000010 | fm.ask |
459 | 19290866 | 2970 | 0.000011 | com.pearltrees |
460 | 19289990 | 793 | 0.000037 | com.zapier |
461 | 19286494 | 2562 | 0.000013 | com.mystrikingly |
462 | 19284092 | 876 | 0.000034 | com.cctv |
463 | 19283500 | 816 | 0.000036 | com.healthline |
464 | 19283044 | 1955 | 0.000017 | com.chrome |
465 | 19282638 | 1484 | 0.000020 | com.rt |
466 | 19282550 | 967 | 0.000031 | com.newsweek |
467 | 19280538 | 2362 | 0.000014 | com.biography |
468 | 19279646 | 1005 | 0.000029 | ch.google |
469 | 19270504 | 1412 | 0.000021 | com.ifttt |
470 | 19270238 | 1584 | 0.000019 | com.axios |
471 | 19270042 | 466 | 0.000065 | es.google |
472 | 19269658 | 882 | 0.000034 | au.gov.nsw |
473 | 19267444 | 3483 | 0.000010 | hk.edu.cuhk |
474 | 19267150 | 862 | 0.000035 | com.stitcher |
475 | 19267000 | 2520 | 0.000013 | com.boredpanda |
476 | 19265582 | 1192 | 0.000025 | fr.lemonde |
477 | 19263992 | 554 | 0.000053 | com.steampowered |
478 | 19263878 | 1055 | 0.000028 | org.jstor |
479 | 19262150 | 1335 | 0.000022 | org.imf |
480 | 19261918 | 873 | 0.000034 | com.venturebeat |
481 | 19261196 | 825 | 0.000036 | org.poynter |
482 | 19259574 | 1684 | 0.000018 | com.straitstimes |
483 | 19259452 | 3390 | 0.000010 | com.chosun |
484 | 19259322 | 1502 | 0.000020 | edu.asu |
485 | 19258762 | 2351 | 0.000014 | io.gitlab |
486 | 19256810 | 956 | 0.000031 | ru.google |
487 | 19255996 | 952 | 0.000031 | sg.com.google |
488 | 19253798 | 1331 | 0.000022 | uk.co.standard |
489 | 19252906 | 612 | 0.000048 | de.gesetze-im-internet |
490 | 19251516 | 948 | 0.000031 | gov.archives |
491 | 19250270 | 2385 | 0.000014 | th.co.google |
492 | 19249730 | 423 | 0.000069 | io.codepen |
493 | 19248930 | 3033 | 0.000011 | com.nola |
494 | 19248894 | 2023 | 0.000016 | edu.gmu |
495 | 19245246 | 2836 | 0.000012 | app.netlify |
496 | 19245158 | 1116 | 0.000026 | com.wikia |
497 | 19242656 | 1353 | 0.000022 | com.history |
498 | 19242160 | 1007 | 0.000029 | com.thelancet |
499 | 19241830 | 2918 | 0.000011 | com.coca-colacompany |
500 | 19240640 | 2654 | 0.000012 | google.ai |
501 | 19240600 | 856 | 0.000035 | com.freepik |
502 | 19240430 | 1548 | 0.000020 | com.buzzfeednews |
503 | 19238648 | 2894 | 0.000012 | org.cato |
504 | 19237700 | 431 | 0.000068 | net.datatables |
505 | 19237456 | 501 | 0.000060 | com.rackcdn |
506 | 19236168 | 1590 | 0.000019 | gov.supremecourt |
507 | 19233302 | 2534 | 0.000013 | edu.byu |
508 | 19233268 | 642 | 0.000046 | fr.amazon |
509 | 19232920 | 2872 | 0.000012 | tw.blogspot |
510 | 19231944 | 803 | 0.000037 | in.co.google |
511 | 19231530 | 1977 | 0.000017 | org.edx |
512 | 19231228 | 1309 | 0.000023 | com.tunein |
513 | 19231156 | 1779 | 0.000018 | org.ocks |
514 | 19230478 | 522 | 0.000057 | nl.google |
515 | 19228370 | 555 | 0.000053 | com.gmail |
516 | 19227068 | 2398 | 0.000014 | com.nationalpost |
517 | 19226910 | 1867 | 0.000017 | edu.ucsb |
518 | 19226418 | 2383 | 0.000014 | edu.nd |
519 | 19226392 | 1372 | 0.000022 | com.dw |
520 | 19226256 | 127 | 0.000262 | com.jimdo |
521 | 19225860 | 2412 | 0.000014 | no.uio |
522 | 19225400 | 1006 | 0.000029 | google.blog |
523 | 19222398 | 1409 | 0.000021 | cn.cntv |
524 | 19222164 | 3285 | 0.000011 | cn.org.china |
525 | 19221136 | 1639 | 0.000019 | org.unwomen |
526 | 19218950 | 946 | 0.000031 | com.airtable |
527 | 19217788 | 2510 | 0.000013 | edu.uoregon |
528 | 19215376 | 2172 | 0.000015 | org.britishcouncil |
529 | 19214674 | 2668 | 0.000012 | org.icrc |
530 | 19214462 | 951 | 0.000031 | com.gallup |
531 | 19213378 | 2265 | 0.000015 | ru.kremlin |
532 | 19212894 | 1332 | 0.000022 | com.globalsign |
533 | 19210850 | 875 | 0.000034 | gov.uspto |
534 | 19210492 | 959 | 0.000031 | edu.psu |
535 | 19210022 | 1509 | 0.000020 | com.penguinrandomhouse |
536 | 19209318 | 1345 | 0.000022 | com.netdna-cdn |
537 | 19208686 | 3269 | 0.000011 | is.archive |
538 | 19208344 | 1531 | 0.000020 | uk.ac.lse |
539 | 19207952 | 2503 | 0.000013 | fi.helsinki |
540 | 19207620 | 2042 | 0.000016 | edu.pitt |
541 | 19207236 | 2170 | 0.000015 | net.openid |
542 | 19206256 | 1155 | 0.000026 | edu.brookings |
543 | 19205290 | 786 | 0.000037 | com.imageshack |
544 | 19204770 | 172 | 0.000152 | com.npmjs |
545 | 19204486 | 3290 | 0.000011 | de.diplo |
546 | 19204380 | 1956 | 0.000017 | edu.unl |
547 | 19203832 | 1544 | 0.000020 | edu.georgetown |
548 | 19203210 | 2125 | 0.000015 | org.metmuseum |
549 | 19202750 | 1240 | 0.000024 | org.nejm |
550 | 19202244 | 726 | 0.000040 | com.adage |
551 | 19200434 | 1990 | 0.000017 | com.channel4 |
552 | 19200290 | 1511 | 0.000020 | com.findlaw |
553 | 19200030 | 2224 | 0.000015 | com.france24 |
554 | 19198938 | 282 | 0.000089 | net.php |
555 | 19198698 | 1784 | 0.000017 | com.csmonitor |
556 | 19197866 | 419 | 0.000069 | com.proofpoint |
557 | 19195320 | 192 | 0.000135 | com.iubenda |
558 | 19194372 | 1011 | 0.000029 | gov.treasury |
559 | 19194028 | 1708 | 0.000018 | com.euronews |
560 | 19191446 | 2286 | 0.000014 | com.thoughtco |
561 | 19190136 | 3742 | 0.000009 | com.doodlekit |
562 | 19189862 | 107 | 0.000320 | com.godaddy |
563 | 19189334 | 1298 | 0.000023 | edu.duke |
564 | 19188652 | 2071 | 0.000016 | com.foreignpolicy |
565 | 19185118 | 1996 | 0.000017 | org.documentcloud |
566 | 19183756 | 1300 | 0.000023 | com.livescience |
567 | 19183706 | 2508 | 0.000013 | com.upi |
568 | 19183104 | 2085 | 0.000016 | com.gq |
569 | 19182260 | 178 | 0.000148 | com.zendesk |
570 | 19182074 | 3020 | 0.000011 | com.authorstream |
571 | 19182074 | 3915 | 0.000009 | com.mysanantonio |
572 | 19181694 | 4133 | 0.000008 | tw.edu.sinica |
573 | 19177894 | 2719 | 0.000012 | org.wikisource |
574 | 19177382 | 2220 | 0.000015 | com.insider |
575 | 19177180 | 851 | 0.000035 | gov.nist |
576 | 19177000 | 1625 | 0.000019 | com.thestar |
577 | 19176642 | 181 | 0.000145 | jp.co.yahoo |
578 | 19174546 | 1304 | 0.000023 | au.com.smh |
579 | 19174028 | 2025 | 0.000016 | org.ncsl |
580 | 19173800 | 4252 | 0.000008 | hk.edu.cityu |
581 | 19173744 | 3349 | 0.000010 | com.sina |
582 | 19173108 | 2197 | 0.000015 | ie.independent |
583 | 19172266 | 2156 | 0.000015 | edu.uky |
584 | 19171704 | 96 | 0.000349 | me.ogp |
585 | 19170936 | 3413 | 0.000010 | uk.ac.sussex |
586 | 19170792 | 1755 | 0.000018 | gov.doc |
587 | 19170704 | 131 | 0.000250 | org.networkadvertising |
588 | 19169566 | 320 | 0.000080 | io.shields |
589 | 19168058 | 649 | 0.000045 | gov.usa |
590 | 19166990 | 4291 | 0.000008 | org.china-embassy |
591 | 19166810 | 3137 | 0.000011 | com.udn |
592 | 19163774 | 161 | 0.000166 | ru.mail |
593 | 19163712 | 3474 | 0.000010 | com.worldatlas |
594 | 19163522 | 505 | 0.000060 | com.netflix |
595 | 19163254 | 857 | 0.000035 | com.thinkwithgoogle |
596 | 19162356 | 1441 | 0.000021 | gov.defense |
597 | 19161952 | 1318 | 0.000023 | tw.com.google |
598 | 19160826 | 1604 | 0.000019 | org.hrw |
599 | 19159812 | 1495 | 0.000020 | com.asahi |
600 | 19159570 | 785 | 0.000037 | io.readthedocs |
601 | 19158768 | 2688 | 0.000012 | org.freedomhouse |
602 | 19158654 | 1413 | 0.000021 | tv.ustream |
603 | 19157822 | 893 | 0.000034 | org.mediawiki |
604 | 19156446 | 1715 | 0.000018 | org.pypi |
605 | 19151800 | 3028 | 0.000011 | org.adb |
606 | 19151406 | 2099 | 0.000016 | fr.leparisien |
607 | 19151152 | 2615 | 0.000013 | com.abc7news |
608 | 19150650 | 2063 | 0.000016 | com.voanews |
609 | 19150048 | 1019 | 0.000029 | com.pcmag |
610 | 19148698 | 447 | 0.000067 | org.nodejs |
611 | 19148554 | 4288 | 0.000008 | com.theundefeated |
612 | 19147816 | 3860 | 0.000009 | org.gephi |
613 | 19147176 | 1327 | 0.000023 | org.undp |
614 | 19146462 | 3277 | 0.000011 | org.iucnredlist |
615 | 19146454 | 2583 | 0.000013 | com.sacbee |
616 | 19146204 | 1594 | 0.000019 | com.treehugger |
617 | 19145608 | 2292 | 0.000014 | no.google |
618 | 19144462 | 2471 | 0.000013 | co.ello |
619 | 19143354 | 1986 | 0.000017 | com.msnbc |
620 | 19143354 | 252 | 0.000097 | com.myshopify |
621 | 19142810 | 981 | 0.000030 | uk.parliament |
622 | 19142520 | 2287 | 0.000014 | co.pcdn |
623 | 19141942 | 1255 | 0.000024 | gov.uscourts |
624 | 19141896 | 1422 | 0.000021 | co.lpages |
625 | 19140780 | 2344 | 0.000014 | org.fas |
626 | 19139768 | 781 | 0.000037 | com.intel |
627 | 19138740 | 807 | 0.000036 | com.marketwatch |
628 | 19136914 | 2047 | 0.000016 | com.infogram |
629 | 19133848 | 2538 | 0.000013 | com.sputniknews |
630 | 19133704 | 2430 | 0.000014 | ie.google |
631 | 19132582 | 1344 | 0.000022 | se.google |
632 | 19131798 | 990 | 0.000030 | com.netlify |
633 | 19131000 | 925 | 0.000032 | com.jekyllrb |
634 | 19130612 | 3055 | 0.000011 | int.interpol |
635 | 19130308 | 524 | 0.000056 | fr.free |
636 | 19130180 | 1198 | 0.000025 | be.google |
637 | 19129750 | 1575 | 0.000019 | uk.co.huffingtonpost |
638 | 19129310 | 2323 | 0.000014 | ly.rebrand |
639 | 19129104 | 1504 | 0.000020 | link.page |
640 | 19128704 | 1794 | 0.000017 | com.sched |
641 | 19127724 | 2218 | 0.000015 | jp.co.japantimes |
642 | 19127254 | 2829 | 0.000012 | org.tigris |
643 | 19127152 | 2839 | 0.000012 | org.pri |
644 | 19127006 | 2319 | 0.000014 | nz.co.nzherald |
645 | 19125622 | 1204 | 0.000025 | at.google |
646 | 19125464 | 5292 | 0.000007 | org.arkive |
647 | 19125326 | 222 | 0.000116 | com.salesforce |
648 | 19123296 | 650 | 0.000045 | br.com.uol |
649 | 19121018 | 4242 | 0.000008 | kr.co.kbs |
650 | 19119374 | 1665 | 0.000018 | com.thebalance |
651 | 19119126 | 1455 | 0.000021 | org.oxfordjournals |
652 | 19118638 | 3738 | 0.000009 | com.encyclopedia |
653 | 19117262 | 2204 | 0.000015 | org.eji |
654 | 19116506 | 2818 | 0.000012 | org.heritage |
655 | 19116298 | 2371 | 0.000014 | com.popsci |
656 | 19114518 | 2199 | 0.000015 | com.snopes |
657 | 19114098 | 2601 | 0.000013 | org.oas |
658 | 19113348 | 156 | 0.000174 | com.aspnetcdn |
659 | 19112712 | 1031 | 0.000028 | org.ilo |
660 | 19109654 | 2263 | 0.000015 | com.insidehighered |
661 | 19108980 | 1587 | 0.000019 | gov.usembassy |
662 | 19108932 | 1622 | 0.000019 | dk.google |
663 | 19108040 | 3392 | 0.000010 | org.jenkins-ci |
664 | 19107388 | 2827 | 0.000012 | org.project-syndicate |
665 | 19106556 | 1963 | 0.000017 | com.justia |
666 | 19104120 | 1563 | 0.000019 | gov.govinfo |
667 | 19103152 | 1699 | 0.000018 | com.firebaseapp |
668 | 19102068 | 2093 | 0.000016 | edu.uga |
669 | 19102028 | 3678 | 0.000010 | edu.wm |
670 | 19101614 | 3284 | 0.000011 | com.cgtn |
671 | 19101596 | 1881 | 0.000017 | org.worldcat |
672 | 19101226 | 900 | 0.000033 | com.zoho |
673 | 19100590 | 392 | 0.000074 | com.atlassian |
674 | 19100290 | 2676 | 0.000012 | org.transparency |
675 | 19099776 | 1317 | 0.000023 | org.aarp |
676 | 19099686 | 1675 | 0.000018 | org.americanbar |
677 | 19099164 | 2239 | 0.000015 | com.timeshighereducation |
678 | 19097964 | 3270 | 0.000011 | com.pastemagazine |
679 | 19095902 | 2598 | 0.000013 | org.csis |
680 | 19094342 | 629 | 0.000047 | com.samsung |
681 | 19094058 | 774 | 0.000038 | com.pexels |
682 | 19093374 | 1964 | 0.000017 | com.washingtontimes |
683 | 19092714 | 2016 | 0.000016 | gov.usaid |
684 | 19090166 | 1334 | 0.000022 | org.heart |
685 | 19088764 | 191 | 0.000136 | com.automattic |
686 | 19088428 | 865 | 0.000035 | com.verisign |
687 | 19087660 | 2108 | 0.000016 | com.motherjones |
688 | 19087034 | 2944 | 0.000011 | org.vim |
689 | 19086498 | 2062 | 0.000016 | edu.nap |
690 | 19086172 | 924 | 0.000032 | com.webs |
691 | 19084778 | 1593 | 0.000019 | org.amnesty |
692 | 19084344 | 2101 | 0.000016 | ua.com.google |
693 | 19083552 | 3988 | 0.000009 | org.globalnetworkinitiative |
694 | 19083196 | 2546 | 0.000013 | org.globalcitizen |
695 | 19082500 | 1754 | 0.000018 | com.surveygizmo |
696 | 19082058 | 2262 | 0.000015 | org.wbur |
697 | 19081048 | 2353 | 0.000014 | uk.gov.companieshouse |
698 | 19080398 | 2468 | 0.000013 | jp.mainichi |
699 | 19080286 | 3181 | 0.000011 | com.podomatic |
700 | 19078116 | 1751 | 0.000018 | org.unhcr |
701 | 19076276 | 2118 | 0.000016 | ca.ctvnews |
702 | 19075310 | 2565 | 0.000013 | uk.co.bbci |
703 | 19073812 | 968 | 0.000031 | uk.gov.legislation |
704 | 19071522 | 2681 | 0.000012 | com.nationalreview |
705 | 19070832 | 2523 | 0.000013 | com.cleveland |
706 | 19070474 | 3814 | 0.000009 | org.neocities |
707 | 19069884 | 1073 | 0.000027 | ly.snip |
708 | 19068864 | 438 | 0.000067 | com.herokuapp |
709 | 19068510 | 656 | 0.000045 | com.oreilly |
710 | 19066730 | 1154 | 0.000026 | cz.google |
711 | 19066464 | 2164 | 0.000015 | org.nrdc |
712 | 19065768 | 2671 | 0.000012 | org.thinkprogress |
713 | 19065654 | 1795 | 0.000017 | ca.globalnews |
714 | 19065106 | 270 | 0.000093 | jp.co.amazon |
715 | 19062840 | 1328 | 0.000023 | org.altervista |
716 | 19061732 | 3119 | 0.000011 | uk.ac.nottingham |
717 | 19061168 | 1267 | 0.000024 | uk.gov.nationalarchives |
718 | 19060934 | 2106 | 0.000016 | au.edu.anu |
719 | 19060236 | 3035 | 0.000011 | com.intensedebate |
720 | 19060102 | 2734 | 0.000012 | de.hu-berlin |
721 | 19059802 | 736 | 0.000039 | com.airbnb |
722 | 19059800 | 2326 | 0.000014 | de.auswaertiges-amt |
723 | 19059376 | 2316 | 0.000014 | nz.co.google |
724 | 19059170 | 2672 | 0.000012 | org.unenvironment |
725 | 19058978 | 3132 | 0.000011 | org.rsf |
726 | 19057932 | 4110 | 0.000008 | com.koreaherald |
727 | 19057778 | 1960 | 0.000017 | org.pewtrusts |
728 | 19057678 | 2867 | 0.000012 | com.techinasia |
729 | 19057488 | 2276 | 0.000014 | com.thecut |
730 | 19056174 | 3700 | 0.000009 | com.viki |
731 | 19056068 | 2724 | 0.000012 | org.gnupg |
732 | 19054590 | 2469 | 0.000013 | ro.google |
733 | 19054394 | 2057 | 0.000016 | edu.gwu |
734 | 19054116 | 3057 | 0.000011 | com.bangkokpost |
735 | 19053626 | 2572 | 0.000013 | fr.rfi |
736 | 19052868 | 414 | 0.000070 | com.pubmatic |
737 | 19051906 | 2309 | 0.000014 | com.tutsplus |
738 | 19051648 | 1079 | 0.000027 | tr.com.google |
739 | 19051516 | 248 | 0.000098 | com.getbootstrap |
740 | 19050908 | 4424 | 0.000008 | com.wonderhowto |
741 | 19050626 | 3619 | 0.000010 | com.upworthy |
742 | 19050496 | 2883 | 0.000012 | org.sonatype |
743 | 19050382 | 288 | 0.000087 | com.typeform |
744 | 19049574 | 2806 | 0.000012 | il.co.google |
745 | 19049384 | 2739 | 0.000012 | uk.ac.leeds |
746 | 19048116 | 201 | 0.000127 | to.amzn |
747 | 19047986 | 2703 | 0.000012 | vn.com.google |
748 | 19047578 | 274 | 0.000092 | com.surveymonkey |
749 | 19047380 | 922 | 0.000032 | int.wipo |
750 | 19046288 | 1057 | 0.000028 | com.gizmodo |
751 | 19046144 | 874 | 0.000034 | com.box |
752 | 19045578 | 2298 | 0.000014 | com.oregonlive |
753 | 19044916 | 547 | 0.000054 | gg.discord |
754 | 19044444 | 3356 | 0.000010 | com.theepochtimes |
755 | 19044400 | 2480 | 0.000013 | ar.com.google |
756 | 19044144 | 2943 | 0.000011 | bg.google |
757 | 19043632 | 2061 | 0.000016 | com.squarespace-cdn |
758 | 19043400 | 3479 | 0.000010 | io.soup |
759 | 19042778 | 2545 | 0.000013 | com.webbyawards |
760 | 19042384 | 2744 | 0.000012 | io.fabric |
761 | 19042298 | 1588 | 0.000019 | com.speakerdeck |
762 | 19041684 | 136 | 0.000232 | info.aboutads |
763 | 19040606 | 907 | 0.000033 | com.docker |
764 | 19038814 | 1817 | 0.000017 | com.miamiherald |
765 | 19037924 | 3191 | 0.000011 | ph.com.google |
766 | 19037762 | 2463 | 0.000013 | com.channelnewsasia |
767 | 19037556 | 3198 | 0.000011 | uk.co.vogue |
768 | 19037554 | 2619 | 0.000013 | edu.fsu |
769 | 19035870 | 485 | 0.000063 | com.staticflickr |
770 | 19035284 | 2495 | 0.000013 | za.co.google |
771 | 19033678 | 2696 | 0.000012 | com.thejakartapost |
772 | 19032442 | 1236 | 0.000024 | edu.ucsd |
773 | 19032258 | 487 | 0.000062 | com.fc2 |
774 | 19032038 | 5415 | 0.000007 | com.armorgames |
775 | 19031944 | 2155 | 0.000015 | fi.google |
776 | 19031234 | 3885 | 0.000009 | com.alamy |
777 | 19030868 | 2221 | 0.000015 | id.co.google |
778 | 19030462 | 2794 | 0.000012 | com.rd |
779 | 19029712 | 2951 | 0.000011 | com.cartodb |
780 | 19029584 | 2092 | 0.000016 | com.newrepublic |
781 | 19029348 | 3436 | 0.000010 | com.benzinga |
782 | 19028364 | 661 | 0.000044 | com.entrepreneur |
783 | 19027960 | 5376 | 0.000007 | org.gwtproject |
784 | 19026660 | 2988 | 0.000011 | com.sciencealert |
785 | 19026538 | 2763 | 0.000012 | org.iaea |
786 | 19026402 | 2376 | 0.000014 | com.thenation |
787 | 19023692 | 3411 | 0.000010 | si.google |
788 | 19023046 | 2400 | 0.000014 | pt.google |
789 | 19020124 | 2965 | 0.000011 | au.gov.nla |
790 | 19019838 | 3513 | 0.000010 | com.dailykos |
791 | 19019756 | 494 | 0.000061 | com.aol |
792 | 19019128 | 2519 | 0.000013 | edu.emory |
793 | 19019012 | 3573 | 0.000010 | com.inhabitat |
794 | 19018956 | 3415 | 0.000010 | uk.ac.soas |
795 | 19018402 | 666 | 0.000044 | com.deloitte |
796 | 19018230 | 1185 | 0.000025 | com.today |
797 | 19016838 | 978 | 0.000030 | com.windowsphone |
798 | 19016186 | 3659 | 0.000010 | org.cpj |
799 | 19016164 | 2119 | 0.000016 | kr.co.google |
800 | 19015906 | 2981 | 0.000011 | se.lu |
801 | 19015780 | 2774 | 0.000012 | org.cfr |
802 | 19014856 | 429 | 0.000068 | me.fb |
803 | 19013678 | 3288 | 0.000011 | com.joins |
804 | 19012980 | 4264 | 0.000008 | sa.com.google |
805 | 19012878 | 2814 | 0.000012 | com.politifact |
806 | 19012292 | 964 | 0.000031 | com.alexa |
807 | 19011442 | 4131 | 0.000008 | edu.utm |
808 | 19011068 | 2735 | 0.000012 | com.law360 |
809 | 19010546 | 983 | 0.000030 | com.engadget |
810 | 19008662 | 3583 | 0.000010 | hr.google |
811 | 19008538 | 2146 | 0.000015 | hu.google |
812 | 19006860 | 631 | 0.000047 | fm.last |
813 | 19006540 | 2476 | 0.000013 | eu.politico |
814 | 19006248 | 4047 | 0.000009 | com.chinatimes |
815 | 19006116 | 2521 | 0.000013 | mx.com.google |
816 | 19006060 | 3141 | 0.000011 | com.jezebel |
817 | 19005942 | 3868 | 0.000009 | com.iconarchive |
818 | 19005318 | 3471 | 0.000010 | com.ogilvy |
819 | 19004866 | 2399 | 0.000014 | gr.google |
820 | 19004086 | 2816 | 0.000012 | com.monday |
821 | 19003252 | 2738 | 0.000012 | com.digitaljournal |
822 | 19003248 | 3149 | 0.000011 | com.nyt |
823 | 19003220 | 3300 | 0.000011 | audio.breaker |
824 | 19002640 | 2823 | 0.000012 | uk.co.guim |
825 | 19002384 | 625 | 0.000047 | com.cisco |
826 | 19002038 | 3391 | 0.000010 | cn.globaltimes |
827 | 19001808 | 2648 | 0.000012 | com.instructure |
828 | 19000646 | 3321 | 0.000011 | com.crashlytics |
829 | 18999720 | 2723 | 0.000012 | au.com.businessinsider |
830 | 18999338 | 3430 | 0.000010 | org.grist |
831 | 18998280 | 1209 | 0.000025 | com.pastebin |
832 | 18998118 | 315 | 0.000082 | ai.shortpixel |
833 | 18998078 | 3990 | 0.000009 | org.constitutioncenter |
834 | 18997960 | 4842 | 0.000007 | jp.hatenadiary |
835 | 18996780 | 3770 | 0.000009 | edu.ttu |
836 | 18996076 | 2997 | 0.000011 | uk.ac.york |
837 | 18995936 | 1671 | 0.000018 | com.eater |
838 | 18995084 | 90 | 0.000364 | com.livestream |
839 | 18995036 | 2772 | 0.000012 | com.bepress |
840 | 18994752 | 2898 | 0.000012 | org.wri |
841 | 18992262 | 2043 | 0.000016 | my.com.thestar |
842 | 18991122 | 3775 | 0.000009 | com.minds |
843 | 18990592 | 2352 | 0.000014 | mp.j |
844 | 18990570 | 3708 | 0.000009 | app.web |
845 | 18990062 | 3410 | 0.000010 | org.carnegieendowment |
846 | 18989786 | 3645 | 0.000010 | tr.com.aa |
847 | 18989418 | 711 | 0.000041 | gov.sec |
848 | 18987746 | 3812 | 0.000009 | com.hyperallergic |
849 | 18987282 | 3408 | 0.000010 | com.foreignaffairs |
850 | 18986640 | 3797 | 0.000009 | au.edu.uts |
851 | 18985392 | 470 | 0.000064 | com.fastcompany |
852 | 18985032 | 3560 | 0.000010 | org.hypotheses |
853 | 18984468 | 3896 | 0.000009 | com.japantoday |
854 | 18982752 | 3507 | 0.000010 | edu.wayne |
855 | 18982048 | 3713 | 0.000009 | uk.ac.kent |
856 | 18981988 | 3697 | 0.000009 | rs.google |
857 | 18980532 | 4071 | 0.000009 | org.sourcewatch |
858 | 18979366 | 832 | 0.000036 | com.symantec |
859 | 18978424 | 2539 | 0.000013 | fr.paris |
860 | 18977996 | 2942 | 0.000011 | com.prweek |
861 | 18977902 | 1765 | 0.000018 | ch.ipcc |
862 | 18976960 | 2217 | 0.000015 | com.kinstacdn |
863 | 18976262 | 1046 | 0.000028 | edu.cmu |
864 | 18975462 | 2039 | 0.000016 | int.unfccc |
865 | 18975062 | 4196 | 0.000008 | eg.com.google |
866 | 18974804 | 3180 | 0.000011 | org.nationalgeographic |
867 | 18974548 | 2643 | 0.000013 | gov.doi |
868 | 18973940 | 3406 | 0.000010 | de.uni-frankfurt |
869 | 18973494 | 4243 | 0.000008 | by.google |
870 | 18972022 | 5050 | 0.000007 | com.symbaloo |
871 | 18971010 | 3417 | 0.000010 | nl.wur |
872 | 18969950 | 2328 | 0.000014 | org.unodc |
873 | 18968430 | 1599 | 0.000019 | com.routledge |
874 | 18968412 | 4509 | 0.000008 | com.ipsos-mori |
875 | 18966962 | 3658 | 0.000010 | ae.google |
876 | 18966152 | 4482 | 0.000008 | com.etymonline |
877 | 18965888 | 4982 | 0.000007 | build.bazel |
878 | 18965566 | 3320 | 0.000011 | org.brainpickings |
879 | 18964544 | 3143 | 0.000011 | com.scotsman |
880 | 18963796 | 4295 | 0.000008 | com.oilprice |
881 | 18963380 | 3597 | 0.000010 | uk.ac.westminster |
882 | 18963266 | 4545 | 0.000008 | lk.google |
883 | 18962576 | 1260 | 0.000024 | fr.blogspot |
884 | 18961360 | 3412 | 0.000010 | org.rferl |
885 | 18961310 | 3173 | 0.000011 | org.epi |
886 | 18959900 | 4115 | 0.000008 | lv.google |
887 | 18959812 | 3909 | 0.000009 | au.edu.griffith |
888 | 18959422 | 4219 | 0.000008 | kr.ac.snu |
889 | 18957280 | 1312 | 0.000023 | com.upwork |
890 | 18957076 | 2436 | 0.000014 | com.html5rocks |
891 | 18956714 | 5493 | 0.000007 | me.nimbusweb |
892 | 18956502 | 2940 | 0.000011 | fr.archives-ouvertes |
893 | 18956398 | 4293 | 0.000008 | com.delawareonline |
894 | 18955462 | 1792 | 0.000017 | ru.rbc |
895 | 18954968 | 745 | 0.000039 | com.gartner |
896 | 18954930 | 1127 | 0.000026 | edu.utexas |
897 | 18953642 | 2526 | 0.000013 | net.noscript |
898 | 18953466 | 2717 | 0.000012 | ae.thenational |
899 | 18953336 | 3380 | 0.000010 | com.study |
900 | 18953092 | 427 | 0.000068 | com.hp |
901 | 18953074 | 3641 | 0.000010 | uk.co.spectator |
902 | 18952762 | 3869 | 0.000009 | com.cleantechnica |
903 | 18952208 | 2803 | 0.000012 | org.unctad |
904 | 18951200 | 4255 | 0.000008 | com.teslamotors |
905 | 18950118 | 1614 | 0.000019 | com.billboard |
906 | 18949366 | 3074 | 0.000011 | com.theculturetrip |
907 | 18947896 | 2454 | 0.000013 | com.multiscreensite |
908 | 18947738 | 704 | 0.000041 | com.visualstudio |
909 | 18947588 | 3985 | 0.000009 | uk.ac.plymouth |
910 | 18947454 | 2660 | 0.000012 | sk.google |
911 | 18947312 | 3811 | 0.000009 | net.aljazeera |
912 | 18947110 | 2413 | 0.000014 | com.theintercept |
913 | 18946556 | 3421 | 0.000010 | uk.ac.exeter |
914 | 18946494 | 3332 | 0.000010 | social.mastodon |
915 | 18945876 | 2828 | 0.000012 | com.euractiv |
916 | 18945864 | 3635 | 0.000010 | com.db |
917 | 18942736 | 4447 | 0.000008 | org.mises |
918 | 18942316 | 4680 | 0.000008 | ng.com.google |
919 | 18942016 | 2795 | 0.000012 | org.panda |
920 | 18941622 | 2466 | 0.000013 | uk.gov.justice |
921 | 18941430 | 5602 | 0.000007 | net.chinadialogue |
922 | 18940924 | 4118 | 0.000008 | cat.uab |
923 | 18940746 | 4227 | 0.000008 | com.spokesman |
924 | 18940082 | 3523 | 0.000010 | co.com.google |
925 | 18939230 | 4473 | 0.000008 | lu.google |
926 | 18938996 | 4189 | 0.000008 | pe.com.google |
927 | 18938618 | 3366 | 0.000010 | com.nybooks |
928 | 18938606 | 4381 | 0.000008 | uk.ac.core |
929 | 18938206 | 2228 | 0.000015 | com.termsfeed |
930 | 18938194 | 1669 | 0.000018 | com.pcworld |
931 | 18938112 | 3846 | 0.000009 | kr.co.yna |
932 | 18938002 | 4793 | 0.000007 | com.gust |
933 | 18937788 | 3880 | 0.000009 | org.cgiar |
934 | 18937300 | 4231 | 0.000008 | pk.com.google |
935 | 18936530 | 3575 | 0.000010 | net.inquirer |
936 | 18936008 | 3083 | 0.000011 | ru.lenta |
937 | 18934000 | 1468 | 0.000020 | com.nokia |
938 | 18933676 | 2932 | 0.000011 | tw.com.pchome |
939 | 18933496 | 1223 | 0.000024 | com.ycombinator |
940 | 18933350 | 2911 | 0.000011 | nl.volkskrant |
941 | 18933194 | 78 | 0.000411 | com.oculus |
942 | 18932612 | 3455 | 0.000010 | cl.google |
943 | 18931862 | 3949 | 0.000009 | org.polymer-project |
944 | 18930888 | 2637 | 0.000013 | com.washingtonexaminer |
945 | 18930622 | 3945 | 0.000009 | sk.sme |
946 | 18930534 | 3389 | 0.000010 | edu.monash |
947 | 18930086 | 918 | 0.000032 | com.canva |
948 | 18929552 | 454 | 0.000066 | org.opensource |
949 | 18929398 | 3977 | 0.000009 | com.rappler |
950 | 18928630 | 4000 | 0.000009 | org.plan-international |
951 | 18926518 | 4561 | 0.000008 | cr.co.google |
952 | 18926412 | 3587 | 0.000010 | lt.google |
953 | 18925832 | 3810 | 0.000009 | ca.macleans |
954 | 18925646 | 817 | 0.000036 | net.adform |
955 | 18925046 | 4873 | 0.000007 | com.blogto |
956 | 18924952 | 3508 | 0.000010 | uk.ac.nhm |
957 | 18924928 | 3211 | 0.000011 | edu.ua |
958 | 18923554 | 2815 | 0.000012 | com.articulate |
959 | 18923288 | 249 | 0.000098 | com.sxsw |
960 | 18922866 | 3993 | 0.000009 | org.wilsoncenter |
961 | 18922676 | 4082 | 0.000009 | edu.lehigh |
962 | 18922336 | 417 | 0.000070 | com.skype |
963 | 18921546 | 4699 | 0.000008 | com.out |
964 | 18920714 | 1085 | 0.000027 | com.redhat |
965 | 18920680 | 3266 | 0.000011 | my.com.google |
966 | 18919064 | 2031 | 0.000016 | gov.ecfr |
967 | 18918900 | 4585 | 0.000008 | org.nsidc |
968 | 18918778 | 412 | 0.000070 | net.secureservercdn |
969 | 18918112 | 4536 | 0.000008 | kz.google |
970 | 18917590 | 3295 | 0.000011 | org.osce |
971 | 18917562 | 557 | 0.000053 | org.whatwg |
972 | 18917418 | 4096 | 0.000009 | com.wsoctv |
973 | 18917380 | 2587 | 0.000013 | uk.org.nationaltrust |
974 | 18917220 | 3201 | 0.000011 | uk.gov.london |
975 | 18917048 | 1973 | 0.000017 | scot.gov |
976 | 18916982 | 3865 | 0.000009 | uk.ac.qub |
977 | 18916460 | 3807 | 0.000009 | com.governing |
978 | 18916430 | 528 | 0.000056 | com.businesswire |
979 | 18916300 | 2253 | 0.000015 | wales.gov |
980 | 18915066 | 3422 | 0.000010 | com.afp |
981 | 18914982 | 3080 | 0.000011 | uk.ac.qmul |
982 | 18914878 | 5154 | 0.000007 | com.ingress |
983 | 18914540 | 4596 | 0.000008 | com.webcindario |
984 | 18914316 | 3402 | 0.000010 | org.psychiatryonline |
985 | 18913230 | 4148 | 0.000008 | org.marxists |
986 | 18913096 | 4073 | 0.000009 | me.thinglink |
987 | 18912970 | 1660 | 0.000018 | com.css-tricks |
988 | 18912858 | 4732 | 0.000008 | ie.nuigalway |
989 | 18912514 | 4348 | 0.000008 | com.asiaone |
990 | 18912368 | 3354 | 0.000010 | com.kaspersky-labs |
991 | 18912110 | 1249 | 0.000024 | com.smashingmagazine |
992 | 18912064 | 3787 | 0.000009 | org.nationalinterest |
993 | 18911848 | 556 | 0.000053 | com.adweek |
994 | 18911436 | 4498 | 0.000008 | ec.com.google |
995 | 18911404 | 4722 | 0.000008 | bd.com.google |
996 | 18910006 | 4846 | 0.000007 | uy.com.google |
997 | 18909998 | 4233 | 0.000008 | com.match |
998 | 18909746 | 4021 | 0.000009 | ee.google |
999 | 18909688 | 3962 | 0.000009 | com.adn |
1000 | 18909474 | 4310 | 0.000008 | com.wnd |
Credits
Thanks to the authors of the WebGraph framework, whose software made the computation of graph properties and ranks possible.
We hope the data will be useful for you to do any kind of research on ranking, graph analysis, link spam detection, etc. Let us know about your results via Common Crawl’s Google Group!
September 2020 crawl archive now available
The crawl archive for September 2020 is now available! The data was crawled between September 18th and October 2nd and contains 3.45 billion web pages or 345 TiB of uncompressed content. It includes page captures of 1.5 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The September crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-40/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-40/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-40/warc.paths.gz | 79600 | 81.8 |
WAT files | CC-MAIN-2020-40/wat.paths.gz | 79600 | 23.14 |
WET files | CC-MAIN-2020-40/wet.paths.gz | 79600 | 10.28 |
Robots.txt files | CC-MAIN-2020-40/robotstxt.paths.gz | 79600 | 0.22 |
Non-200 responses files | CC-MAIN-2020-40/non200responses.paths.gz | 79600 | 2.36 |
URL index files | CC-MAIN-2020-40/cc-index.paths.gz | 302 | 0.27 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-40/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
August 2020 crawl archive now available
The crawl archive for August 2020 is now available! It contains 2.45 billion web pages or 235 TiB of uncompressed content, crawled between August 2nd and 15th. It includes page captures of 940 million URLs unknown in any of our prior crawl archives.
Archive Location and Download
The August crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-34/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-34/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-34/warc.paths.gz | 60000 | 48.9 |
WAT files | CC-MAIN-2020-34/wat.paths.gz | 60000 | 16.9 |
WET files | CC-MAIN-2020-34/wet.paths.gz | 60000 | 7.56 |
Robots.txt files | CC-MAIN-2020-34/robotstxt.paths.gz | 60000 | 0.19 |
Non-200 responses files | CC-MAIN-2020-34/non200responses.paths.gz | 60000 | 1.94 |
URL index files | CC-MAIN-2020-34/cc-index.paths.gz | 302 | 0.19 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-34/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
July 2020 crawl archive now available
The crawl archive for July 2020 is now available! It contains 3.14 billion web pages or 300 TiB of uncompressed content, crawled between July 2nd and 16th. It includes page captures of 1.1 billion URLs unknown in any of our prior crawl archives.
Bug Fixes and Improvements
The URL index fields "redirect" and "mime" haven’t been filled if the corresponding HTTP headers Location
and Content-Type
are written in lower-case letters or any other variant not matching case. This bug has been detected during the crawl and was fixed for 90 out of 100 segments. It also affects the columnar index and the fields "fetch_redirect" resp. "content_mime_type". To a minor extend it may affect the detection of character set and content language as the value of the Content-Type
header is used as additional hint for the detection. Additional information about this bug fix is given in the corresponding issue report.
Archive Location and Download
The July crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-29/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-29/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-29/warc.paths.gz | 60000 | 62.64 |
WAT files | CC-MAIN-2020-29/wat.paths.gz | 60000 | 22.23 |
WET files | CC-MAIN-2020-29/wet.paths.gz | 60000 | 9.87 |
Robots.txt files | CC-MAIN-2020-29/robotstxt.paths.gz | 60000 | 0.21 |
Non-200 responses files | CC-MAIN-2020-29/non200responses.paths.gz | 60000 | 2.52 |
URL index files | CC-MAIN-2020-29/cc-index.paths.gz | 302 | 0.24 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-29/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
Host- and Domain-Level Web Graphs Feb/Mar/May 2020
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of February, March/April and May/June 2020. Additional information about the data formats, the processing pipeline, our objectives, and credits can be found in the announcements of prior webgraph releases (e.g., Nov/Dec/Jan 2017-2018 Webgraphs). You may also visit the projects cc-webgraph and cc-pyspark which host all scripts and tools required to construct the graphs.
What’s new?
The host-level graph now includes hosts visited by the crawler but not linking to any other host. Why is this possible – isn’t any host found via links the crawler is following? Yes, but some links were already detected in a prior crawl, not in one of the 3 crawls used to build the web graphs. More details about the issue are given in cc-pyspark#15. The impact of this fix on the graph size is minimal: the recent crawl now includes 1 million nodes (0.1% of all nodes) which are not connected to any other node.
Host-level graph
The graph consists of 927 million nodes and 3.88 billion edges and includes dangling nodes i.e. hosts that have not been crawled yet are pointed to from a link on a crawled page. There are 857 million dangling nodes (92.5%) and the largest strongly connected component contains 47 million (5.1%) nodes.
You can download the graph and the ranks of all 927 million hosts from AWS S3 on the path s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/host/
. Alternatively, you can use https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/host/
as prefix to access the files from everywhere.
Download files of the Common Crawl Feb/Mar/May 2020 host-level webgraph
Size | File | Description |
---|---|---|
5.67 GB | cc-main-2020-feb-mar-may-host-vertices.paths.gz | nodes 〈id, rev host〉, paths of 12 vertices files |
17.26 GB | cc-main-2020-feb-mar-may-host-edges.paths.gz | edges 〈from_id, to_id〉, paths of 24 edges files |
7.40 GB | cc-main-2020-feb-mar-may-host.graph | graph in BVGraph format |
2 kB | cc-main-2020-feb-mar-may-host.properties | |
8.57 GB | cc-main-2020-feb-mar-may-host-t.graph | transpose of the graph (outlinks inverted to inlinks) |
2 kB | cc-main-2020-feb-mar-may-host-t.properties | |
1 kB | cc-main-2020-feb-mar-may-host.stats | WebGraph statistics |
12.16 GB | cc-main-2020-feb-mar-may-host-ranks.txt.gz | harmonic centrality and pagerank |
Note that the host names are reversed and a leading www.
is stripped: www.subdomain.example.com
becomes com.example.subdomain
.
Domain-level graph
The domain graph was built by aggregating the host graph on the level of pay-level domains (PLDs) based on the public suffix list maintained on publicsuffix.org.
The domain-level graph has 91 million nodes and 1.96 billion edges. 51% or 46 million nodes are dangling nodes, the largest strongly connected component covers 36 million or 39% of the nodes.
All files related to the domain graph are available on AWS S3 under s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/domain/
resp. https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/domain/
.
Download files of the Common Crawl Feb/Mar/May 2020 domain-level webgraph
Size | File | Description |
---|---|---|
0.62 GB | cc-main-2020-feb-mar-may-domain-vertices.txt.gz | nodes 〈id, rev domain, num hosts〉 |
7.79 GB | cc-main-2020-feb-mar-may-domain-edges.txt.gz | edges 〈from_id, to_id〉 |
4.23 GB | cc-main-2020-feb-mar-may-domain.graph | graph in BVGraph format |
2 kB | cc-main-2020-feb-mar-may-domain.properties | |
4.16 GB | cc-main-2020-feb-mar-may-domain-t.graph | transpose of the graph |
2 kB | cc-main-2020-feb-mar-may-domain-t.properties | |
1 kB | cc-main-2020-feb-mar-may-domain.stats | WebGraph statistics |
1.96 GB | cc-main-2020-feb-mar-may-domain-ranks.txt.gz | harmonic centrality and pagerank |
Below you’ll find the top 1000 domains ranked by Harmonic Centrality or PageRank. The full list of all 91 million domain ranks is available for download.
Top 1000 domains ranked by harmonic centrality (Feb/Mar/May 2020)
harmonic centrality rank | hc value | page rank | page rank value | reversed hostname |
---|---|---|---|---|
1 | 32667618 | 1 | 0.018180 | com.googleapis |
2 | 30552772 | 3 | 0.011873 | com.facebook |
3 | 29569088 | 2 | 0.013789 | com.google |
4 | 26920460 | 4 | 0.007145 | com.twitter |
5 | 26883128 | 5 | 0.007106 | org.w |
6 | 26360448 | 6 | 0.006483 | com.youtube |
7 | 24719396 | 9 | 0.004210 | com.instagram |
8 | 24251942 | 8 | 0.005125 | org.gmpg |
9 | 23841332 | 7 | 0.005329 | com.googletagmanager |
10 | 23606890 | 13 | 0.002940 | com.linkedin |
11 | 22741292 | 10 | 0.003621 | com.cloudflare |
12 | 22732960 | 12 | 0.002974 | org.wordpress |
13 | 22661910 | 14 | 0.002515 | com.gravatar |
14 | 22577680 | 15 | 0.002438 | com.gstatic |
15 | 22378134 | 22 | 0.001529 | com.pinterest |
16 | 22196962 | 27 | 0.001192 | org.wikipedia |
17 | 22189650 | 19 | 0.001864 | com.wordpress |
18 | 22066028 | 16 | 0.002404 | com.bootstrapcdn |
19 | 21967760 | 18 | 0.001884 | com.apple |
20 | 21751768 | 20 | 0.001863 | com.jquery |
21 | 21589606 | 24 | 0.001461 | com.microsoft |
22 | 21568908 | 44 | 0.000785 | be.youtu |
23 | 21568474 | 43 | 0.000806 | com.blogspot |
24 | 21533280 | 31 | 0.001104 | com.vimeo |
25 | 21415938 | 46 | 0.000761 | gl.goo |
26 | 21399120 | 35 | 0.001040 | com.amazonaws |
27 | 21358048 | 53 | 0.000665 | com.amazon |
28 | 21331634 | 21 | 0.001737 | com.adobe |
29 | 21324666 | 23 | 0.001506 | com.wp |
30 | 21209012 | 70 | 0.000452 | com.tumblr |
31 | 21184360 | 17 | 0.001949 | com.github |
32 | 21150652 | 37 | 0.001008 | com.google-analytics |
33 | 21110976 | 30 | 0.001152 | com.baidu |
34 | 21096692 | 87 | 0.000387 | com.yahoo |
35 | 21081268 | 59 | 0.000547 | ly.bit |
36 | 21060360 | 33 | 0.001072 | com.macromedia |
37 | 21046916 | 36 | 0.001035 | net.cloudfront |
38 | 21036258 | 45 | 0.000763 | com.flickr |
39 | 20997926 | 32 | 0.001101 | com.googlesyndication |
40 | 20993476 | 26 | 0.001277 | me.wp |
41 | 20980462 | 97 | 0.000340 | com.googleusercontent |
42 | 20966446 | 56 | 0.000624 | eu.europa |
43 | 20960242 | 42 | 0.000807 | net.jsdelivr |
44 | 20959910 | 52 | 0.000677 | co.t |
45 | 20901872 | 29 | 0.001163 | ru.yandex |
46 | 20846092 | 50 | 0.000742 | net.doubleclick |
47 | 20843032 | 41 | 0.000869 | com.addthis |
48 | 20823518 | 69 | 0.000457 | io.github |
49 | 20817952 | 76 | 0.000433 | com.medium |
50 | 20810030 | 25 | 0.001287 | com.fontawesome |
51 | 20809120 | 139 | 0.000189 | com.forbes |
52 | 20796434 | 61 | 0.000510 | org.w3 |
53 | 20759102 | 55 | 0.000640 | com.paypal |
54 | 20757266 | 109 | 0.000282 | com.soundcloud |
55 | 20754514 | 90 | 0.000368 | org.creativecommons |
56 | 20747472 | 57 | 0.000619 | com.vk |
57 | 20711184 | 54 | 0.000658 | org.mozilla |
58 | 20710182 | 88 | 0.000382 | com.weebly |
59 | 20698442 | 84 | 0.000410 | com.wix |
60 | 20675372 | 102 | 0.000317 | com.weibo |
61 | 20663930 | 58 | 0.000604 | org.schema |
62 | 20650202 | 164 | 0.000151 | com.imgur |
63 | 20644452 | 147 | 0.000177 | org.apache |
64 | 20642282 | 178 | 0.000138 | uk.co.bbc |
65 | 20625560 | 129 | 0.000210 | org.archive |
66 | 20610354 | 274 | 0.000089 | com.ibm |
67 | 20609614 | 154 | 0.000169 | com.bing |
68 | 20602380 | 191 | 0.000125 | net.sourceforge |
69 | 20579012 | 130 | 0.000207 | com.nytimes |
70 | 20578626 | 150 | 0.000174 | int.who |
71 | 20571012 | 183 | 0.000131 | com.cnn |
72 | 20561674 | 174 | 0.000140 | net.slideshare |
73 | 20547634 | 158 | 0.000164 | gov.cdc |
74 | 20542546 | 202 | 0.000116 | com.android |
75 | 20527230 | 228 | 0.000104 | com.wsj |
76 | 20518548 | 194 | 0.000122 | edu.stanford |
77 | 20505546 | 205 | 0.000115 | com.businessinsider |
78 | 20495034 | 254 | 0.000095 | com.oracle |
79 | 20489434 | 34 | 0.001049 | net.fbcdn |
80 | 20488868 | 373 | 0.000067 | com.msn |
81 | 20488282 | 261 | 0.000093 | edu.harvard |
82 | 20483384 | 310 | 0.000080 | com.go |
83 | 20478152 | 99 | 0.000335 | com.shopify |
84 | 20471424 | 267 | 0.000093 | com.bbc |
85 | 20464434 | 297 | 0.000083 | edu.mit |
86 | 20461340 | 330 | 0.000076 | com.myspace |
87 | 20458776 | 62 | 0.000497 | com.whatsapp |
88 | 20457206 | 289 | 0.000085 | com.appspot |
89 | 20454466 | 307 | 0.000080 | com.wired |
90 | 20446300 | 292 | 0.000085 | com.reuters |
91 | 20442004 | 101 | 0.000323 | com.godaddy |
92 | 20435550 | 171 | 0.000147 | com.theguardian |
93 | 20417770 | 143 | 0.000182 | gov.nih |
94 | 20412536 | 196 | 0.000120 | org.ietf |
95 | 20401330 | 388 | 0.000065 | gov.nasa |
96 | 20397298 | 423 | 0.000061 | com.theverge |
97 | 20394736 | 149 | 0.000175 | com.giphy |
98 | 20394276 | 382 | 0.000066 | net.researchgate |
99 | 20384930 | 270 | 0.000092 | com.bloomberg |
100 | 20377778 | 108 | 0.000285 | com.unpkg |
101 | 20376394 | 114 | 0.000271 | com.reddit |
102 | 20373856 | 337 | 0.000075 | com.xinhuanet |
103 | 20366736 | 215 | 0.000108 | org.gnu |
104 | 20363506 | 318 | 0.000079 | com.usatoday |
105 | 20352660 | 813 | 0.000037 | org.chromium |
106 | 20344996 | 356 | 0.000071 | com.springer |
107 | 20343678 | 98 | 0.000335 | de.google |
108 | 20342420 | 28 | 0.001184 | com.qq |
109 | 20341824 | 345 | 0.000073 | com.example |
110 | 20336510 | 744 | 0.000041 | edu.psu |
111 | 20324536 | 468 | 0.000055 | edu.cornell |
112 | 20324378 | 184 | 0.000131 | com.blogger |
113 | 20314024 | 60 | 0.000516 | net.akamaihd |
114 | 20304242 | 375 | 0.000067 | org.hbr |
115 | 20302310 | 750 | 0.000040 | com.git-scm |
116 | 20300014 | 937 | 0.000032 | com.wikia |
117 | 20298546 | 137 | 0.000191 | com.spotify |
118 | 20296012 | 485 | 0.000053 | edu.yale |
119 | 20295516 | 113 | 0.000271 | com.jimdo |
120 | 20293140 | 554 | 0.000047 | com.cbsnews |
121 | 20291946 | 717 | 0.000043 | com.economist |
122 | 20290574 | 214 | 0.000109 | com.washingtonpost |
123 | 20288504 | 140 | 0.000188 | jp.co.yahoo |
124 | 20286470 | 285 | 0.000086 | com.huffingtonpost |
125 | 20284558 | 316 | 0.000080 | org.un |
126 | 20281874 | 410 | 0.000063 | fr.free |
127 | 20279946 | 473 | 0.000054 | edu.berkeley |
128 | 20275446 | 287 | 0.000086 | com.cnbc |
129 | 20273280 | 245 | 0.000099 | com.dribbble |
130 | 20271584 | 576 | 0.000046 | org.arxiv |
131 | 20269716 | 151 | 0.000172 | com.issuu |
132 | 20257038 | 545 | 0.000047 | com.mysql |
133 | 20256262 | 160 | 0.000157 | com.twimg |
134 | 20252532 | 107 | 0.000285 | com.statcounter |
135 | 20251682 | 338 | 0.000075 | uk.co.telegraph |
136 | 20247478 | 305 | 0.000081 | com.w3schools |
137 | 20246682 | 561 | 0.000047 | com.gitlab |
138 | 20242210 | 802 | 0.000038 | edu.columbia |
139 | 20240978 | 524 | 0.000049 | gov.noaa |
140 | 20230666 | 122 | 0.000230 | com.ytimg |
141 | 20229900 | 119 | 0.000233 | com.youtube-nocookie |
142 | 20227656 | 731 | 0.000042 | org.ieee |
143 | 20227126 | 333 | 0.000075 | org.npr |
144 | 20225528 | 729 | 0.000042 | io.readthedocs |
145 | 20225206 | 286 | 0.000086 | org.acm |
146 | 20222314 | 339 | 0.000074 | com.time |
147 | 20220430 | 1180 | 0.000025 | org.eclipse |
148 | 20220382 | 241 | 0.000100 | org.ampproject |
149 | 20218616 | 344 | 0.000074 | com.fc2 |
150 | 20215730 | 142 | 0.000185 | com.wixsite |
151 | 20213692 | 755 | 0.000040 | edu.washington |
152 | 20210122 | 421 | 0.000061 | com.force |
153 | 20209864 | 276 | 0.000089 | com.prnewswire |
154 | 20209130 | 500 | 0.000052 | com.buzzfeed |
155 | 20207136 | 434 | 0.000060 | com.nationalgeographic |
156 | 20206402 | 403 | 0.000063 | com.nature |
157 | 20203826 | 200 | 0.000118 | gle.forms |
158 | 20202490 | 799 | 0.000038 | org.sciencemag |
159 | 20201144 | 428 | 0.000061 | com.theatlantic |
160 | 20200104 | 871 | 0.000035 | com.stackexchange |
161 | 20198142 | 280 | 0.000088 | com.sciencedirect |
162 | 20185400 | 332 | 0.000075 | com.staticflickr |
163 | 20184528 | 495 | 0.000052 | uk.co.independent |
164 | 20182256 | 263 | 0.000093 | gov.ca |
165 | 20180972 | 687 | 0.000043 | org.worldbank |
166 | 20175994 | 435 | 0.000060 | com.mozilla |
167 | 20175400 | 734 | 0.000041 | com.marketwatch |
168 | 20168098 | 1087 | 0.000027 | com.hatenablog |
169 | 20167040 | 364 | 0.000069 | com.nypost |
170 | 20164016 | 646 | 0.000043 | org.bitbucket |
171 | 20161192 | 219 | 0.000107 | com.ft |
172 | 20151116 | 463 | 0.000056 | com.pixabay |
173 | 20143796 | 354 | 0.000071 | jp.co.rakuten |
174 | 20142652 | 743 | 0.000041 | edu.upenn |
175 | 20140126 | 277 | 0.000089 | org.doi |
176 | 20139376 | 966 | 0.000031 | jp.livedoor |
177 | 20136546 | 198 | 0.000120 | uk.co.google |
178 | 20134932 | 407 | 0.000063 | uk.co.dailymail |
179 | 20134404 | 724 | 0.000042 | org.pbs |
180 | 20133936 | 258 | 0.000094 | net.behance |
181 | 20132914 | 192 | 0.000124 | org.wikimedia |
182 | 20127860 | 917 | 0.000033 | edu.jhu |
183 | 20127828 | 454 | 0.000057 | gov.whitehouse |
184 | 20122352 | 856 | 0.000035 | org.weforum |
185 | 20122170 | 416 | 0.000062 | com.dailymotion |
186 | 20117054 | 1487 | 0.000020 | com.warnerbros |
187 | 20111898 | 326 | 0.000077 | org.opensource |
188 | 20110798 | 1091 | 0.000027 | cn.com.chinadaily |
189 | 20109916 | 548 | 0.000047 | me.about |
190 | 20109820 | 232 | 0.000103 | jp.ameblo |
191 | 20108940 | 558 | 0.000047 | com.oup |
192 | 20103428 | 325 | 0.000077 | com.digg |
193 | 20097418 | 455 | 0.000056 | com.entrepreneur |
194 | 20095108 | 631 | 0.000044 | com.vice |
195 | 20094142 | 749 | 0.000040 | com.qz |
196 | 20092692 | 1259 | 0.000024 | com.discovery |
197 | 20091154 | 444 | 0.000058 | com.goodreads |
198 | 20091052 | 447 | 0.000057 | gg.discord |
199 | 20082910 | 1109 | 0.000027 | com.sap |
200 | 20082186 | 353 | 0.000071 | com.scribd |
201 | 20079412 | 188 | 0.000128 | com.feedburner |
202 | 20076146 | 466 | 0.000055 | com.fortune |
203 | 20075556 | 580 | 0.000045 | com.gartner |
204 | 20072598 | 1012 | 0.000029 | com.500px |
205 | 20072136 | 458 | 0.000056 | jp.ne.sakura |
206 | 20067400 | 176 | 0.000139 | com.imdb |
207 | 20060950 | 732 | 0.000042 | uk.co.blogspot |
208 | 20059054 | 1735 | 0.000018 | com.amd |
209 | 20058228 | 947 | 0.000032 | edu.princeton |
210 | 20056666 | 890 | 0.000034 | org.cambridge |
211 | 20056572 | 51 | 0.000714 | com.fb |
212 | 20056272 | 848 | 0.000036 | com.evernote |
213 | 20054472 | 144 | 0.000180 | com.dropbox |
214 | 20053532 | 39 | 0.000951 | com.wixstatic |
215 | 20051662 | 617 | 0.000044 | org.unesco |
216 | 20050940 | 1461 | 0.000020 | com.fandom |
217 | 20048152 | 294 | 0.000084 | com.wiley |
218 | 20046134 | 768 | 0.000039 | com.withgoogle |
219 | 20039426 | 1015 | 0.000029 | org.altervista |
220 | 20039010 | 2337 | 0.000014 | com.wolfram |
221 | 20037920 | 798 | 0.000038 | com.slate |
222 | 20031484 | 1201 | 0.000025 | org.kernel |
223 | 20028164 | 1049 | 0.000028 | edu.purdue |
224 | 20025282 | 569 | 0.000046 | page.g |
225 | 20021340 | 786 | 0.000038 | com.trello |
226 | 20017018 | 230 | 0.000103 | com.disqus |
227 | 20012796 | 757 | 0.000040 | org.eff |
228 | 20010430 | 951 | 0.000031 | com.merriam-webster |
229 | 20004686 | 493 | 0.000052 | gov.usda |
230 | 20004240 | 981 | 0.000030 | com.netlify |
231 | 20003994 | 2179 | 0.000015 | com.diigo |
232 | 20002918 | 807 | 0.000038 | com.vox |
233 | 20002690 | 180 | 0.000135 | org.allaboutcookies |
234 | 20002220 | 1206 | 0.000025 | com.jetbrains |
235 | 19999418 | 1416 | 0.000021 | edu.arizona |
236 | 19994384 | 542 | 0.000047 | com.tandfonline |
237 | 19993030 | 844 | 0.000036 | com.foxnews |
238 | 19992184 | 291 | 0.000085 | com.live |
239 | 19991142 | 175 | 0.000140 | com.xing |
240 | 19989874 | 909 | 0.000033 | com.politico |
241 | 19988570 | 320 | 0.000079 | com.outlook |
242 | 19985036 | 1135 | 0.000026 | jp.ne.goo |
243 | 19983340 | 754 | 0.000040 | au.net.abc |
244 | 19982680 | 1945 | 0.000016 | com.wikidot |
245 | 19977934 | 793 | 0.000038 | com.investopedia |
246 | 19977574 | 1066 | 0.000028 | edu.uchicago |
247 | 19976820 | 1009 | 0.000029 | edu.wisc |
248 | 19975922 | 197 | 0.000120 | com.eepurl |
249 | 19972560 | 1039 | 0.000028 | com.bostonglobe |
250 | 19972096 | 775 | 0.000039 | org.semver |
251 | 19969594 | 619 | 0.000044 | com.sagepub |
252 | 19969182 | 497 | 0.000052 | gov.fda |
253 | 19968442 | 347 | 0.000073 | net.windows |
254 | 19968084 | 1568 | 0.000019 | edu.osu |
255 | 19965386 | 319 | 0.000079 | com.nbcnews |
256 | 19963946 | 244 | 0.000099 | com.myshopify |
257 | 19962892 | 585 | 0.000045 | cn.google |
258 | 19962530 | 608 | 0.000044 | site.business |
259 | 19961066 | 832 | 0.000036 | com.sciencedaily |
260 | 19960380 | 1044 | 0.000028 | com.strikingly |
261 | 19956366 | 1236 | 0.000024 | edu.unc |
262 | 19956268 | 1446 | 0.000021 | edu.virginia |
263 | 19956034 | 1204 | 0.000025 | co.elastic |
264 | 19952960 | 1194 | 0.000025 | com.nymag |
265 | 19950500 | 2206 | 0.000015 | com.renren |
266 | 19950490 | 742 | 0.000041 | gov.house |
267 | 19950448 | 2163 | 0.000015 | sg.edu.nus |
268 | 19947976 | 2285 | 0.000014 | org.wikibooks |
269 | 19947284 | 1961 | 0.000016 | com.googlesource |
270 | 19940598 | 235 | 0.000103 | com.wpengine |
271 | 19940158 | 323 | 0.000078 | com.googlecode |
272 | 19939212 | 761 | 0.000040 | gov.senate |
273 | 19938008 | 513 | 0.000051 | com.herokuapp |
274 | 19937738 | 452 | 0.000057 | org.pewresearch |
275 | 19937492 | 567 | 0.000046 | org.iana |
276 | 19936954 | 1093 | 0.000027 | com.podbean |
277 | 19935818 | 982 | 0.000030 | com.alexa |
278 | 19934742 | 1629 | 0.000019 | gd.is |
279 | 19933804 | 103 | 0.000301 | com.paypalobjects |
280 | 19932740 | 805 | 0.000038 | org.unicef |
281 | 19932416 | 718 | 0.000043 | com.newyorker |
282 | 19930858 | 969 | 0.000031 | uk.co.thetimes |
283 | 19929324 | 404 | 0.000063 | com.patreon |
284 | 19928266 | 1060 | 0.000028 | com.lifehacker |
285 | 19925940 | 381 | 0.000066 | com.criteo |
286 | 19924524 | 997 | 0.000030 | com.huffpost |
287 | 19922576 | 303 | 0.000081 | com.squareup |
288 | 19922510 | 839 | 0.000036 | ca.cbc |
289 | 19921808 | 1145 | 0.000026 | org.wiktionary |
290 | 19918844 | 146 | 0.000178 | com.addtoany |
291 | 19918174 | 201 | 0.000117 | com.optimizely |
292 | 19918052 | 1342 | 0.000022 | edu.msu |
293 | 19915986 | 1371 | 0.000022 | com.history |
294 | 19913384 | 418 | 0.000062 | com.calendly |
295 | 19905860 | 1181 | 0.000025 | com.udemy |
296 | 19903364 | 809 | 0.000037 | uk.ac.ox |
297 | 19902920 | 172 | 0.000145 | com.amazon-adsystem |
298 | 19899332 | 49 | 0.000743 | com.googleadservices |
299 | 19896924 | 155 | 0.000167 | com.opera |
300 | 19890970 | 887 | 0.000034 | org.fao |
301 | 19890832 | 1017 | 0.000029 | com.ecwid |
302 | 19890826 | 476 | 0.000054 | com.googleblog |
303 | 19887142 | 211 | 0.000110 | com.stackoverflow |
304 | 19886190 | 1419 | 0.000021 | uk.ac.lse |
305 | 19885312 | 360 | 0.000070 | com.getpocket |
306 | 19884456 | 1667 | 0.000018 | org.maven |
307 | 19883800 | 915 | 0.000033 | uk.co.guardian |
308 | 19883358 | 169 | 0.000148 | org.bbb |
309 | 19881084 | 1337 | 0.000022 | com.aljazeera |
310 | 19880790 | 255 | 0.000095 | com.aliyuncs |
311 | 19879938 | 2723 | 0.000013 | net.pixnet |
312 | 19874384 | 3180 | 0.000011 | net.hinet |
313 | 19869028 | 1170 | 0.000025 | com.smithsonianmag |
314 | 19868832 | 1347 | 0.000022 | edu.ucdavis |
315 | 19868258 | 894 | 0.000034 | gov.congress |
316 | 19867190 | 1320 | 0.000023 | edu.illinois |
317 | 19865168 | 1120 | 0.000026 | com.theglobeandmail |
318 | 19863306 | 1036 | 0.000029 | gov.archives |
319 | 19862414 | 492 | 0.000052 | it.placehold |
320 | 19861934 | 93 | 0.000359 | net.facebook |
321 | 19861376 | 1615 | 0.000019 | hk.com.google |
322 | 19860922 | 1473 | 0.000020 | ca.sfu |
323 | 19856352 | 1676 | 0.000018 | blog.home |
324 | 19855290 | 1073 | 0.000027 | com.apnews |
325 | 19854892 | 963 | 0.000031 | com.ssrn |
326 | 19853682 | 3383 | 0.000010 | com.wizards |
327 | 19851102 | 1997 | 0.000016 | com.nabble |
328 | 19851032 | 760 | 0.000040 | com.chinaz |
329 | 19850412 | 3667 | 0.000010 | cn.edu.sjtu |
330 | 19848140 | 1484 | 0.000020 | com.urbandictionary |
331 | 19844436 | 1136 | 0.000026 | com.scmp |
332 | 19842326 | 1489 | 0.000020 | ms.1drv |
333 | 19841796 | 4361 | 0.000008 | tw.com.gamer |
334 | 19838582 | 1392 | 0.000021 | com.flipboard |
335 | 19838166 | 919 | 0.000033 | co.g |
336 | 19837542 | 547 | 0.000047 | com.gofundme |
337 | 19836996 | 2097 | 0.000015 | com.france24 |
338 | 19835636 | 1405 | 0.000021 | jp.geocities |
339 | 19833654 | 1370 | 0.000022 | com.ibtimes |
340 | 19831362 | 581 | 0.000045 | com.biomedcentral |
341 | 19830056 | 1128 | 0.000026 | com.britannica |
342 | 19829420 | 2174 | 0.000015 | com.oregonlive |
343 | 19827062 | 412 | 0.000062 | com.kickstarter |
344 | 19826214 | 962 | 0.000031 | com.adjust |
345 | 19824188 | 867 | 0.000035 | gov.fcc |
346 | 19824048 | 715 | 0.000043 | uk.co.mirror |
347 | 19823266 | 589 | 0.000045 | us.icio |
348 | 19823172 | 1129 | 0.000026 | com.mediafire |
349 | 19821768 | 1432 | 0.000021 | edu.tamu |
350 | 19821310 | 587 | 0.000045 | com.usnews |
351 | 19820442 | 1314 | 0.000023 | org.greenpeace |
352 | 19820252 | 985 | 0.000030 | edu.academia |
353 | 19819486 | 1381 | 0.000021 | com.livescience |
354 | 19815972 | 1684 | 0.000018 | gov.cia |
355 | 19814564 | 1325 | 0.000023 | com.akamai |
356 | 19813266 | 930 | 0.000032 | com.chicagotribune |
357 | 19811538 | 156 | 0.000167 | com.npmjs |
358 | 19811100 | 1429 | 0.000021 | net.seesaa |
359 | 19810120 | 329 | 0.000076 | es.google |
360 | 19809710 | 1238 | 0.000024 | com.reverbnation |
361 | 19809490 | 550 | 0.000047 | com.quora |
362 | 19808314 | 3481 | 0.000010 | com.proboards |
363 | 19806268 | 1040 | 0.000028 | com.thehill |
364 | 19803840 | 321 | 0.000078 | org.python |
365 | 19801476 | 1132 | 0.000026 | org.jstor |
366 | 19801018 | 1722 | 0.000018 | ca.mcgill |
367 | 19799982 | 167 | 0.000149 | com.zendesk |
368 | 19792890 | 999 | 0.000030 | com.thelancet |
369 | 19792246 | 1094 | 0.000027 | com.jamanetwork |
370 | 19788594 | 1935 | 0.000016 | uk.ac.manchester |
371 | 19785214 | 540 | 0.000048 | com.udacity |
372 | 19783328 | 1372 | 0.000021 | ca.utoronto |
373 | 19783082 | 579 | 0.000046 | com.bigcartel |
374 | 19782230 | 2487 | 0.000013 | org.wikiquote |
375 | 19781186 | 1357 | 0.000022 | edu.rutgers |
376 | 19780028 | 896 | 0.000034 | org.apa |
377 | 19779718 | 439 | 0.000059 | com.newsweek |
378 | 19778538 | 920 | 0.000033 | com.healthline |
379 | 19777982 | 2204 | 0.000015 | com.knowyourmeme |
380 | 19775610 | 328 | 0.000077 | com.tinyurl |
381 | 19775558 | 726 | 0.000042 | gov.state |
382 | 19775092 | 216 | 0.000108 | com.unsplash |
383 | 19773702 | 1708 | 0.000018 | ca.ualberta |
384 | 19772378 | 406 | 0.000063 | com.githubusercontent |
385 | 19771900 | 1471 | 0.000020 | com.asahi |
386 | 19771220 | 259 | 0.000094 | org.nodejs |
387 | 19769436 | 475 | 0.000054 | com.latimes |
388 | 19769258 | 1027 | 0.000029 | com.timeanddate |
389 | 19768686 | 432 | 0.000060 | com.slack |
390 | 19768410 | 769 | 0.000039 | jp.shinobi |
391 | 19767976 | 1674 | 0.000018 | com.buzzfeednews |
392 | 19765038 | 415 | 0.000062 | com.elsevier |
393 | 19764722 | 1335 | 0.000022 | edu.gatech |
394 | 19764298 | 2861 | 0.000012 | com.youdao |
395 | 19761256 | 895 | 0.000034 | com.brightcove |
396 | 19759730 | 1774 | 0.000017 | com.bankofamerica |
397 | 19759530 | 2569 | 0.000013 | edu.byu |
398 | 19758760 | 1918 | 0.000016 | com.voanews |
399 | 19757586 | 3164 | 0.000011 | com.opendns |
400 | 19756816 | 1425 | 0.000021 | com.sky |
401 | 19755780 | 2336 | 0.000014 | com.slides |
402 | 19754462 | 1373 | 0.000021 | com.dw |
403 | 19754458 | 1158 | 0.000026 | com.nikkei |
404 | 19752590 | 904 | 0.000033 | com.cbslocal |
405 | 19748766 | 2236 | 0.000014 | net.earthlink |
406 | 19748678 | 391 | 0.000064 | com.cnet |
407 | 19748150 | 1642 | 0.000018 | com.xrea |
408 | 19747430 | 1354 | 0.000022 | uk.co.huffingtonpost |
409 | 19746424 | 182 | 0.000133 | com.eventbrite |
410 | 19746370 | 1071 | 0.000027 | com.nydailynews |
411 | 19744090 | 1305 | 0.000023 | me.vk |
412 | 19743194 | 918 | 0.000033 | gov.bls |
413 | 19741542 | 1458 | 0.000020 | org.ap |
414 | 19740936 | 384 | 0.000066 | net.imgix |
415 | 19739860 | 2414 | 0.000014 | org.aclweb |
416 | 19739750 | 1641 | 0.000018 | com.axios |
417 | 19738940 | 987 | 0.000030 | com.wattpad |
418 | 19737530 | 1713 | 0.000018 | com.straitstimes |
419 | 19737412 | 474 | 0.000054 | com.ted |
420 | 19736874 | 1294 | 0.000023 | edu.brookings |
421 | 19728634 | 967 | 0.000031 | int.coe |
422 | 19727580 | 212 | 0.000109 | com.etsy |
423 | 19727112 | 2392 | 0.000014 | com.biography |
424 | 19726080 | 865 | 0.000035 | gov.va |
425 | 19725710 | 217 | 0.000107 | com.typepad |
426 | 19724628 | 1932 | 0.000016 | com.cocolog-nifty |
427 | 19723580 | 1608 | 0.000019 | com.reference |
428 | 19720740 | 553 | 0.000047 | com.livejournal |
429 | 19717406 | 2096 | 0.000015 | ru.kremlin |
430 | 19716354 | 815 | 0.000037 | uk.gov.service |
431 | 19715378 | 298 | 0.000083 | com.techcrunch |
432 | 19712358 | 2462 | 0.000013 | org.wikisource |
433 | 19712296 | 1553 | 0.000019 | com.foxbusiness |
434 | 19711620 | 1281 | 0.000023 | mil.army |
435 | 19711244 | 1761 | 0.000017 | com.itv |
436 | 19710260 | 733 | 0.000041 | com.deviantart |
437 | 19705952 | 1311 | 0.000023 | de.mpg |
438 | 19705288 | 845 | 0.000036 | gov.justice |
439 | 19704574 | 1993 | 0.000016 | cn.people |
440 | 19703248 | 1262 | 0.000024 | au.com.smh |
441 | 19701656 | 1763 | 0.000017 | org.tensorflow |
442 | 19701634 | 1223 | 0.000024 | org.ohchr |
443 | 19701000 | 568 | 0.000046 | ru.gov |
444 | 19700136 | 400 | 0.000064 | com.technorati |
445 | 19699596 | 2134 | 0.000015 | jp.co.japantimes |
446 | 19697954 | 83 | 0.000413 | com.list-manage |
447 | 19697088 | 1068 | 0.000028 | com.thedrum |
448 | 19696754 | 1538 | 0.000019 | uk.co.standard |
449 | 19695430 | 185 | 0.000131 | com.rawgit |
450 | 19694216 | 2120 | 0.000015 | com.oxforddictionaries |
451 | 19693006 | 2241 | 0.000014 | com.shutterfly |
452 | 19692082 | 3147 | 0.000011 | tw.edu.ntu |
453 | 19691564 | 2550 | 0.000013 | com.smashwords |
454 | 19689862 | 1862 | 0.000016 | edu.unl |
455 | 19688768 | 2402 | 0.000014 | org.fas |
456 | 19688646 | 296 | 0.000084 | uk.org.ico |
457 | 19688138 | 2710 | 0.000013 | tv.blip |
458 | 19686066 | 957 | 0.000031 | com.bandsintown |
459 | 19684448 | 3516 | 0.000010 | cn.org.china |
460 | 19682960 | 1550 | 0.000019 | uk.co.express |
461 | 19679708 | 1082 | 0.000027 | jp.jugem |
462 | 19679158 | 3656 | 0.000010 | info.webry |
463 | 19678730 | 1403 | 0.000021 | gov.uscourts |
464 | 19677944 | 2157 | 0.000015 | au.edu.unimelb |
465 | 19675766 | 92 | 0.000363 | com.wsimg |
466 | 19674868 | 283 | 0.000086 | ru.rambler |
467 | 19673738 | 1921 | 0.000016 | com.washingtontimes |
468 | 19671754 | 351 | 0.000072 | com.proofpoint |
469 | 19669412 | 74 | 0.000441 | net.jsfiddle |
470 | 19668352 | 788 | 0.000038 | org.mediawiki |
471 | 19668158 | 2851 | 0.000012 | jp.blog |
472 | 19667740 | 1479 | 0.000020 | com.firebaseapp |
473 | 19667418 | 1618 | 0.000019 | com.webnode |
474 | 19665940 | 2173 | 0.000015 | com.pbworks |
475 | 19665748 | 3374 | 0.000011 | com.patheos |
476 | 19665684 | 3135 | 0.000011 | uk.co.timesonline |
477 | 19663980 | 2171 | 0.000015 | google.ai |
478 | 19663354 | 233 | 0.000103 | com.squarespace |
479 | 19662188 | 2904 | 0.000012 | fr.rfi |
480 | 19660984 | 1454 | 0.000020 | gov.supremecourt |
481 | 19659200 | 1889 | 0.000016 | int.unfccc |
482 | 19658534 | 331 | 0.000076 | com.office |
483 | 19656526 | 577 | 0.000046 | pl.google |
484 | 19654098 | 991 | 0.000030 | gov.wa |
485 | 19652796 | 804 | 0.000038 | gov.sba |
486 | 19652626 | 1267 | 0.000023 | com.cognitoforms |
487 | 19650066 | 2207 | 0.000015 | org.csis |
488 | 19649008 | 366 | 0.000068 | io.codepen |
489 | 19648750 | 2344 | 0.000014 | com.kobo |
490 | 19646512 | 110 | 0.000281 | com.mailchimp |
491 | 19643428 | 1671 | 0.000018 | edu.wustl |
492 | 19642572 | 2734 | 0.000013 | edu.kit |
493 | 19642334 | 1480 | 0.000020 | org.hrw |
494 | 19642276 | 953 | 0.000031 | edu.umich |
495 | 19641856 | 1389 | 0.000021 | com.dictionary |
496 | 19641544 | 836 | 0.000036 | com.mapquest |
497 | 19640836 | 1747 | 0.000017 | org.worldcat |
498 | 19640276 | 3621 | 0.000010 | net.aljazeera |
499 | 19640144 | 357 | 0.000071 | com.photobucket |
500 | 19639948 | 2046 | 0.000015 | net.cnki |
501 | 19638510 | 1705 | 0.000018 | com.secondlife |
502 | 19638416 | 2421 | 0.000014 | int.wmo |
503 | 19637888 | 1089 | 0.000027 | org.ilo |
504 | 19637450 | 1100 | 0.000027 | google.blog |
505 | 19636692 | 378 | 0.000067 | com.meetup |
506 | 19634634 | 995 | 0.000030 | uk.co.pinterest |
507 | 19633770 | 3397 | 0.000010 | com.freehostia |
508 | 19630412 | 3256 | 0.000011 | com.doodlekit |
509 | 19629746 | 936 | 0.000032 | com.arstechnica |
510 | 19628370 | 3730 | 0.000009 | com.colourlovers |
511 | 19628356 | 1696 | 0.000018 | ru.ucoz |
512 | 19628298 | 952 | 0.000031 | com.thenextweb |
513 | 19624458 | 2286 | 0.000014 | org.unep |
514 | 19622342 | 2252 | 0.000014 | org.icrc |
515 | 19621808 | 1424 | 0.000021 | com.findlaw |
516 | 19621134 | 2334 | 0.000014 | com.similarweb |
517 | 19620696 | 481 | 0.000054 | com.gmail |
518 | 19619304 | 3040 | 0.000012 | io.soup |
519 | 19616246 | 1437 | 0.000021 | com.imageshack |
520 | 19615956 | 2785 | 0.000013 | com.sputniknews |
521 | 19614078 | 3080 | 0.000012 | com.smore |
522 | 19613232 | 3246 | 0.000011 | org.iucnredlist |
523 | 19611766 | 3117 | 0.000011 | com.kinja |
524 | 19611760 | 1883 | 0.000016 | com.csmonitor |
525 | 19611604 | 145 | 0.000180 | ru.mail |
526 | 19610088 | 1339 | 0.000022 | gov.uscis |
527 | 19608554 | 446 | 0.000058 | net.secureservercdn |
528 | 19606314 | 3004 | 0.000012 | sh.now |
529 | 19605748 | 427 | 0.000061 | tv.twitch |
530 | 19604994 | 1580 | 0.000019 | link.app |
531 | 19600814 | 440 | 0.000059 | com.statista |
532 | 19599160 | 3676 | 0.000010 | jp.hatenablog |
533 | 19595550 | 4356 | 0.000008 | com.coroflot |
534 | 19595264 | 3177 | 0.000011 | org.jenkins-ci |
535 | 19595158 | 1757 | 0.000017 | gov.oregon |
536 | 19593130 | 3200 | 0.000011 | li.paper |
537 | 19593106 | 3847 | 0.000009 | com.pixar |
538 | 19589878 | 3095 | 0.000011 | com.shell |
539 | 19588194 | 4035 | 0.000009 | com.scienceblogs |
540 | 19586188 | 1625 | 0.000019 | org.amnesty |
541 | 19584824 | 892 | 0.000034 | com.thedailybeast |
542 | 19582464 | 1767 | 0.000017 | org.pypi |
543 | 19582346 | 2149 | 0.000015 | com.foreignpolicy |
544 | 19580310 | 2849 | 0.000012 | com.instapaper |
545 | 19579672 | 2910 | 0.000012 | org.accessnow |
546 | 19578614 | 1602 | 0.000019 | com.surveygizmo |
547 | 19577780 | 1733 | 0.000018 | ca.globalnews |
548 | 19576200 | 3175 | 0.000011 | de.uni-koeln |
549 | 19576198 | 239 | 0.000101 | io.shields |
550 | 19576184 | 3377 | 0.000011 | org.lds |
551 | 19575902 | 2238 | 0.000014 | org.rand |
552 | 19574790 | 207 | 0.000114 | com.salesforce |
553 | 19574544 | 3438 | 0.000010 | net.mootools |
554 | 19574428 | 2357 | 0.000014 | at.ac.univie |
555 | 19574182 | 4050 | 0.000009 | org.marxists |
556 | 19571664 | 2860 | 0.000012 | org.panda |
557 | 19571194 | 2806 | 0.000013 | com.oprah |
558 | 19568576 | 1874 | 0.000016 | com.justia |
559 | 19567970 | 3471 | 0.000010 | org.avaaz |
560 | 19567854 | 2880 | 0.000012 | com.openai |
561 | 19567764 | 3597 | 0.000010 | org.neocities |
562 | 19567260 | 3753 | 0.000009 | cn.edu.sdu |
563 | 19564960 | 762 | 0.000040 | com.netflix |
564 | 19564120 | 498 | 0.000052 | com.oreilly |
565 | 19563086 | 4405 | 0.000008 | com.yam |
566 | 19562248 | 227 | 0.000105 | uk.co.amazon |
567 | 19562204 | 866 | 0.000035 | com.zoho |
568 | 19560956 | 629 | 0.000044 | com.zdnet |
569 | 19559966 | 1298 | 0.000023 | ly.snip |
570 | 19558790 | 1790 | 0.000017 | ch.ipcc |
571 | 19558664 | 993 | 0.000030 | uk.parliament |
572 | 19558508 | 3787 | 0.000009 | com.nestle |
573 | 19556304 | 1254 | 0.000024 | se.google |
574 | 19556292 | 2997 | 0.000012 | com.treehugger |
575 | 19555184 | 1011 | 0.000029 | net.nocookie |
576 | 19555096 | 4644 | 0.000008 | com.x0 |
577 | 19553368 | 3631 | 0.000010 | org.tvtropes |
578 | 19550992 | 1141 | 0.000026 | org.sphinx-doc |
579 | 19549994 | 2122 | 0.000015 | ru.mos |
580 | 19548820 | 3044 | 0.000012 | es.csic |
581 | 19548530 | 2913 | 0.000012 | uk.gov.companieshouse |
582 | 19546576 | 1034 | 0.000029 | com.engadget |
583 | 19546230 | 1183 | 0.000025 | com.here |
584 | 19545492 | 5060 | 0.000007 | com.dbs |
585 | 19545438 | 4103 | 0.000009 | br.ufrj |
586 | 19544204 | 2159 | 0.000015 | edu.colostate |
587 | 19543398 | 2706 | 0.000013 | de.uni-heidelberg |
588 | 19540500 | 3059 | 0.000012 | com.pearltrees |
589 | 19539268 | 2176 | 0.000015 | net.openid |
590 | 19537880 | 2600 | 0.000013 | com.mystrikingly |
591 | 19537844 | 3880 | 0.000009 | com.chinatimes |
592 | 19535834 | 2400 | 0.000014 | link.page |
593 | 19534182 | 2354 | 0.000014 | com.real |
594 | 19533432 | 1836 | 0.000017 | org.ncsl |
595 | 19532288 | 301 | 0.000082 | com.surveymonkey |
596 | 19531930 | 362 | 0.000070 | com.hp |
597 | 19531412 | 1193 | 0.000025 | org.js |
598 | 19530700 | 2135 | 0.000015 | com.123formbuilder |
599 | 19528842 | 2426 | 0.000014 | org.vim |
600 | 19528104 | 3205 | 0.000011 | pl.wp |
601 | 19528018 | 2602 | 0.000013 | au.com.sbs |
602 | 19526780 | 170 | 0.000148 | com.yelp |
603 | 19526216 | 2499 | 0.000013 | uk.ac.kcl |
604 | 19524346 | 1338 | 0.000022 | org.aarp |
605 | 19523692 | 2621 | 0.000013 | th.co.google |
606 | 19523156 | 1006 | 0.000029 | uk.gov.legislation |
607 | 19523042 | 260 | 0.000094 | com.getbootstrap |
608 | 19522856 | 3663 | 0.000010 | com.magcloud |
609 | 19522274 | 3990 | 0.000009 | com.zynga |
610 | 19521942 | 1268 | 0.000023 | tw.com.google |
611 | 19521922 | 2829 | 0.000013 | com.kaggle |
612 | 19520130 | 948 | 0.000031 | gov.gpo |
613 | 19519742 | 946 | 0.000032 | com.about |
614 | 19519714 | 3273 | 0.000011 | org.rsf |
615 | 19518740 | 2976 | 0.000012 | org.tigris |
616 | 19518224 | 2727 | 0.000013 | uk.ac.leeds |
617 | 19515512 | 3535 | 0.000010 | de.dw |
618 | 19515434 | 3019 | 0.000012 | org.cfr |
619 | 19514574 | 3253 | 0.000011 | de.uni-freiburg |
620 | 19513570 | 3640 | 0.000010 | de.uni-konstanz |
621 | 19512714 | 3881 | 0.000009 | ua.at |
622 | 19511254 | 2117 | 0.000015 | info.worldometers |
623 | 19510314 | 4657 | 0.000008 | com.embarcadero |
624 | 19509370 | 2999 | 0.000012 | vn.zing |
625 | 19509134 | 3229 | 0.000011 | com.bangkokpost |
626 | 19508804 | 3615 | 0.000010 | ly.rebrand |
627 | 19508548 | 2008 | 0.000016 | gov.ky |
628 | 19508426 | 4009 | 0.000009 | org.wilsoncenter |
629 | 19506774 | 4059 | 0.000009 | jp.hatenadiary |
630 | 19506284 | 4374 | 0.000008 | com.musictoday |
631 | 19505388 | 3824 | 0.000009 | org.constitutioncenter |
632 | 19505186 | 372 | 0.000067 | com.booking |
633 | 19504402 | 2579 | 0.000013 | com.eiseverywhere |
634 | 19503800 | 4038 | 0.000009 | com.itsnicethat |
635 | 19503776 | 3331 | 0.000011 | il.ac.tau |
636 | 19502096 | 2359 | 0.000014 | mx.com.google |
637 | 19500806 | 3736 | 0.000009 | com.db |
638 | 19498928 | 312 | 0.000080 | com.ebay |
639 | 19498588 | 3578 | 0.000010 | jp.hateblo |
640 | 19498166 | 3348 | 0.000011 | org.democracynow |
641 | 19497296 | 3975 | 0.000009 | edu.odu |
642 | 19496812 | 2815 | 0.000013 | dk.au |
643 | 19496626 | 4220 | 0.000008 | com.etymonline |
644 | 19496184 | 2885 | 0.000012 | uk.gov.metoffice |
645 | 19495756 | 361 | 0.000070 | com.skype |
646 | 19495566 | 3570 | 0.000010 | com.hsbc |
647 | 19494844 | 2228 | 0.000015 | com.bankrate |
648 | 19494104 | 2240 | 0.000014 | gov.wi |
649 | 19493352 | 1815 | 0.000017 | fi.google |
650 | 19493306 | 4426 | 0.000008 | com.x10host |
651 | 19492136 | 3224 | 0.000011 | org.royalsociety |
652 | 19491096 | 817 | 0.000037 | com.pexels |
653 | 19490358 | 532 | 0.000048 | com.mashable |
654 | 19490282 | 4614 | 0.000008 | com.epochtimes |
655 | 19490018 | 1174 | 0.000025 | edu.ucla |
656 | 19489656 | 3226 | 0.000011 | cc.reurl |
657 | 19489414 | 3430 | 0.000010 | com.dailykos |
658 | 19489360 | 3742 | 0.000009 | uk.ac.uea |
659 | 19488050 | 3705 | 0.000010 | ca.shaw |
660 | 19486104 | 1968 | 0.000016 | uk.gov.tfl |
661 | 19485988 | 3434 | 0.000010 | uk.ac.nhm |
662 | 19485032 | 3060 | 0.000012 | com.ipage |
663 | 19484754 | 2498 | 0.000013 | com.prweek |
664 | 19484598 | 1819 | 0.000017 | gov.usembassy |
665 | 19483966 | 4861 | 0.000007 | am.do |
666 | 19483636 | 3086 | 0.000011 | com.viki |
667 | 19483518 | 3252 | 0.000011 | se.liu |
668 | 19482718 | 3066 | 0.000012 | com.coca-colacompany |
669 | 19482580 | 4232 | 0.000008 | br.ufrgs |
670 | 19482498 | 3639 | 0.000010 | de.uni-kiel |
671 | 19481340 | 1453 | 0.000020 | com.speakerdeck |
672 | 19480718 | 3077 | 0.000012 | net.openreview |
673 | 19480660 | 2208 | 0.000015 | de.auswaertiges-amt |
674 | 19480248 | 208 | 0.000113 | com.hubspot |
675 | 19479762 | 2026 | 0.000016 | com.lexisnexis |
676 | 19478700 | 2106 | 0.000015 | net.ucoz |
677 | 19477552 | 3494 | 0.000010 | com.iconarchive |
678 | 19477532 | 819 | 0.000037 | com.steampowered |
679 | 19477286 | 756 | 0.000040 | com.xiti |
680 | 19477132 | 2486 | 0.000013 | com.post-gazette |
681 | 19476898 | 3369 | 0.000011 | com.eklablog |
682 | 19476632 | 2937 | 0.000012 | uk.co.bbci |
683 | 19476378 | 1911 | 0.000016 | hu.google |
684 | 19476160 | 4399 | 0.000008 | com.jacobinmag |
685 | 19475974 | 3323 | 0.000011 | uk.ac.sussex |
686 | 19474368 | 3068 | 0.000012 | uk.ac.qmul |
687 | 19474212 | 3930 | 0.000009 | nf.co |
688 | 19473014 | 4114 | 0.000009 | com.collinsdictionary |
689 | 19472896 | 5215 | 0.000007 | com.evaair |
690 | 19472846 | 2572 | 0.000013 | com.marketwire |
691 | 19472580 | 3138 | 0.000011 | au.com.telstra |
692 | 19472114 | 3916 | 0.000009 | it.unitn |
693 | 19471646 | 898 | 0.000034 | com.visualstudio |
694 | 19471330 | 3807 | 0.000009 | in.ernet |
695 | 19470994 | 2906 | 0.000012 | nl.rug |
696 | 19468708 | 5297 | 0.000007 | org.arkive |
697 | 19468252 | 252 | 0.000096 | org.drupal |
698 | 19467050 | 3460 | 0.000010 | ca.dal |
699 | 19467046 | 3693 | 0.000010 | com.canada |
700 | 19465642 | 1451 | 0.000021 | com.tinypic |
701 | 19465304 | 3136 | 0.000011 | org.wri |
702 | 19465034 | 3698 | 0.000010 | com.la-croix |
703 | 19464108 | 4557 | 0.000008 | com.mitsubishielectric |
704 | 19463828 | 4748 | 0.000008 | com.gamejolt |
705 | 19462976 | 2789 | 0.000013 | gr.google |
706 | 19462882 | 4882 | 0.000007 | cz.webgarden |
707 | 19462404 | 3079 | 0.000012 | my.com.thestar |
708 | 19461830 | 269 | 0.000092 | net.php |
709 | 19461640 | 4329 | 0.000008 | au.gov.fairwork |
710 | 19460770 | 2279 | 0.000014 | co.pcdn |
711 | 19460176 | 3943 | 0.000009 | uk.ac.essex |
712 | 19459984 | 121 | 0.000231 | org.networkadvertising |
713 | 19459684 | 3396 | 0.000010 | org.rferl |
714 | 19459068 | 4211 | 0.000008 | com.sc |
715 | 19459020 | 3292 | 0.000011 | com.blogfa |
716 | 19458794 | 3382 | 0.000010 | ca.yelp |
717 | 19457580 | 4102 | 0.000009 | edu.utm |
718 | 19457248 | 5694 | 0.000007 | com.anghami |
719 | 19456532 | 5210 | 0.000007 | su.clan |
720 | 19456144 | 4095 | 0.000009 | it.justpaste |
721 | 19456006 | 414 | 0.000062 | com.sxsw |
722 | 19455914 | 3258 | 0.000011 | com.waterstones |
723 | 19454602 | 3960 | 0.000009 | com.jigsy |
724 | 19454516 | 838 | 0.000036 | com.intel |
725 | 19454394 | 4032 | 0.000009 | ee.ut |
726 | 19453242 | 916 | 0.000033 | com.docker |
727 | 19452988 | 738 | 0.000041 | com.samsung |
728 | 19451802 | 3422 | 0.000010 | es.ucm |
729 | 19450718 | 2503 | 0.000013 | com.washingtonexaminer |
730 | 19450342 | 3951 | 0.000009 | tl.page |
731 | 19450206 | 2209 | 0.000015 | org.wbur |
732 | 19449036 | 4112 | 0.000009 | site.negocio |
733 | 19448922 | 2773 | 0.000013 | com.yell |
734 | 19448516 | 3988 | 0.000009 | com.fatcow |
735 | 19448266 | 3282 | 0.000011 | pl.poznan |
736 | 19448198 | 135 | 0.000194 | com.youku |
737 | 19447930 | 2878 | 0.000012 | ae.thenational |
738 | 19447766 | 4705 | 0.000008 | id.co.kaskus |
739 | 19447668 | 3407 | 0.000010 | com.afp |
740 | 19447602 | 5336 | 0.000007 | net.manilatimes |
741 | 19446734 | 419 | 0.000062 | com.caniuse |
742 | 19446168 | 1470 | 0.000020 | com.pastebin |
743 | 19445910 | 3387 | 0.000010 | uk.org.rspb |
744 | 19445736 | 765 | 0.000039 | com.moz |
745 | 19444376 | 4027 | 0.000009 | lv.draugiem |
746 | 19441604 | 2508 | 0.000013 | gov.dni |
747 | 19440874 | 2593 | 0.000013 | ro.google |
748 | 19440144 | 2946 | 0.000012 | com.broadwayworld |
749 | 19439574 | 3750 | 0.000009 | ru.msu |
750 | 19439374 | 3766 | 0.000009 | pl.cba |
751 | 19439332 | 4137 | 0.000009 | org.rfa |
752 | 19439280 | 5562 | 0.000007 | org.bukkit |
753 | 19439086 | 2013 | 0.000016 | scot.gov |
754 | 19438868 | 133 | 0.000200 | com.constantcontact |
755 | 19438826 | 5638 | 0.000007 | org.adbusters |
756 | 19438094 | 4517 | 0.000008 | google.design |
757 | 19437654 | 4154 | 0.000008 | com.macobserver |
758 | 19437088 | 1649 | 0.000018 | fr.pagesjaunes |
759 | 19437020 | 2502 | 0.000013 | com.thenation |
760 | 19436776 | 3973 | 0.000009 | com.bbcamerica |
761 | 19434556 | 4857 | 0.000007 | com.orgfree |
762 | 19433810 | 2978 | 0.000012 | com.channelnewsasia |
763 | 19432506 | 735 | 0.000041 | gov.sec |
764 | 19432502 | 4008 | 0.000009 | com.teamspeak |
765 | 19432430 | 2800 | 0.000013 | org.gnupg |
766 | 19432260 | 3780 | 0.000009 | com.the-scientist |
767 | 19432252 | 3015 | 0.000012 | com.laweekly |
768 | 19431446 | 2921 | 0.000012 | au.edu.sydney |
769 | 19430084 | 3577 | 0.000010 | uk.co.yougov |
770 | 19430000 | 3140 | 0.000011 | vn.com.google |
771 | 19429942 | 4417 | 0.000008 | com.50webs |
772 | 19429004 | 3124 | 0.000011 | org.repec |
773 | 19428938 | 3215 | 0.000011 | org.ourworldindata |
774 | 19427890 | 3506 | 0.000010 | com.tradingeconomics |
775 | 19427352 | 3102 | 0.000011 | tw.com.pchome |
776 | 19426582 | 3332 | 0.000011 | com.monday |
777 | 19426556 | 3556 | 0.000010 | org.project-syndicate |
778 | 19425552 | 2331 | 0.000014 | com.amebaownd |
779 | 19424890 | 1596 | 0.000019 | org.whatbrowser |
780 | 19424750 | 1956 | 0.000016 | org.americanbar |
781 | 19424680 | 3739 | 0.000009 | ie.thejournal |
782 | 19424152 | 104 | 0.000298 | com.stripe |
783 | 19424140 | 4014 | 0.000009 | com.hatenadiary |
784 | 19424060 | 2933 | 0.000012 | org.thinkprogress |
785 | 19423712 | 3073 | 0.000012 | uk.gov.london |
786 | 19423054 | 3927 | 0.000009 | com.thesaurus |
787 | 19423006 | 3475 | 0.000010 | net.webself |
788 | 19422964 | 3432 | 0.000010 | io.pantheon |
789 | 19421712 | 3420 | 0.000010 | uk.ac.exeter |
790 | 19421508 | 4343 | 0.000008 | com.appledaily |
791 | 19421118 | 3528 | 0.000010 | com.bravesites |
792 | 19420816 | 5178 | 0.000007 | com.bambuser |
793 | 19420592 | 3379 | 0.000011 | com.foreignaffairs |
794 | 19419378 | 2432 | 0.000013 | com.instructables |
795 | 19416388 | 2185 | 0.000015 | vn.vietnamnet |
796 | 19414736 | 3994 | 0.000009 | com.webcindario |
797 | 19414328 | 2823 | 0.000013 | org.ewg |
798 | 19413934 | 4534 | 0.000008 | ws.nimb |
799 | 19413778 | 2833 | 0.000013 | org.fullfact |
800 | 19413352 | 256 | 0.000095 | us.zoom |
801 | 19412556 | 3685 | 0.000010 | com.encyclopedia |
802 | 19412474 | 3897 | 0.000009 | de.uni-erlangen |
803 | 19410822 | 5341 | 0.000007 | net.boards |
804 | 19409598 | 341 | 0.000074 | com.histats |
805 | 19409534 | 4201 | 0.000008 | is.pse |
806 | 19409436 | 748 | 0.000040 | fm.last |
807 | 19407808 | 3661 | 0.000010 | com.mongabay |
808 | 19407040 | 3220 | 0.000011 | me.site123 |
809 | 19406338 | 3436 | 0.000010 | com.seetickets |
810 | 19405550 | 5838 | 0.000007 | com.gamigo |
811 | 19404400 | 1666 | 0.000018 | com.materialdesignicons |
812 | 19404108 | 5140 | 0.000007 | bd.com.google |
813 | 19403242 | 790 | 0.000038 | com.venturebeat |
814 | 19401218 | 4601 | 0.000008 | uk.org.phrases |
815 | 19400780 | 3213 | 0.000011 | com.instructure |
816 | 19400298 | 2817 | 0.000013 | gov.arkansas |
817 | 19399890 | 72 | 0.000444 | com.livestream |
818 | 19399554 | 4081 | 0.000009 | cat.uab |
819 | 19399486 | 3546 | 0.000010 | org.lacity |
820 | 19399372 | 3612 | 0.000010 | com.heraldscotland |
821 | 19398370 | 1499 | 0.000020 | com.teachable |
822 | 19396672 | 2895 | 0.000012 | com.foodandwine |
823 | 19395752 | 1233 | 0.000024 | com.createjs |
824 | 19394274 | 2266 | 0.000014 | com.ajc |
825 | 19394172 | 3950 | 0.000009 | com.rappler |
826 | 19394030 | 2355 | 0.000014 | net.noscript |
827 | 19393982 | 4140 | 0.000009 | jp.doorblog |
828 | 19392882 | 2873 | 0.000012 | com.timeshighereducation |
829 | 19392238 | 275 | 0.000089 | com.bandcamp |
830 | 19389332 | 3969 | 0.000009 | jp.ne.hi-ho |
831 | 19388094 | 3629 | 0.000010 | net.inquirer |
832 | 19387882 | 552 | 0.000047 | com.cisco |
833 | 19387318 | 4076 | 0.000009 | pl.lublin |
834 | 19386370 | 1657 | 0.000018 | com.pcworld |
835 | 19383404 | 266 | 0.000093 | com.typeform |
836 | 19382886 | 203 | 0.000116 | com.naver |
837 | 19382698 | 3723 | 0.000010 | gov.bts |
838 | 19382192 | 1816 | 0.000017 | jp.makeshop |
839 | 19382102 | 4462 | 0.000008 | com.tor |
840 | 19382072 | 4513 | 0.000008 | com.weightwatchers |
841 | 19381346 | 1438 | 0.000021 | org.khanacademy |
842 | 19381274 | 954 | 0.000031 | com.thinkwithgoogle |
843 | 19381020 | 3385 | 0.000010 | uk.ac.jisc |
844 | 19380238 | 4088 | 0.000009 | ly.genial |
845 | 19379986 | 4007 | 0.000009 | com.themoscowtimes |
846 | 19378500 | 3272 | 0.000011 | com.nyt |
847 | 19378434 | 3760 | 0.000009 | com.springernature |
848 | 19378356 | 3390 | 0.000010 | int.cbd |
849 | 19377854 | 6045 | 0.000006 | es.xurl |
850 | 19376898 | 1756 | 0.000017 | com.netsolhost |
851 | 19376598 | 3852 | 0.000009 | au.edu.griffith |
852 | 19376054 | 4740 | 0.000008 | co.edu.unal |
853 | 19376040 | 4074 | 0.000009 | kr.co.koreatimes |
854 | 19374588 | 727 | 0.000042 | com.deloitte |
855 | 19374300 | 4986 | 0.000007 | org.edc |
856 | 19373940 | 4149 | 0.000008 | vn.tienphong |
857 | 19373476 | 3515 | 0.000010 | com.thediplomat |
858 | 19372932 | 4099 | 0.000009 | uk.ac.lancs |
859 | 19372798 | 5006 | 0.000007 | com.inoreader |
860 | 19372746 | 4922 | 0.000007 | com.ueuo |
861 | 19372594 | 1585 | 0.000019 | tv.ustream |
862 | 19372576 | 3234 | 0.000011 | com.tapatalk |
863 | 19372356 | 3416 | 0.000010 | nl.wur |
864 | 19372106 | 4848 | 0.000007 | net.hypermart |
865 | 19371636 | 2293 | 0.000014 | org.kff |
866 | 19369356 | 398 | 0.000064 | com.pubmatic |
867 | 19368982 | 3625 | 0.000010 | org.grist |
868 | 19368480 | 3088 | 0.000011 | tw.gov.cdc |
869 | 19368288 | 3389 | 0.000010 | com.gothamist |
870 | 19368130 | 1106 | 0.000027 | com.gizmodo |
871 | 19368116 | 4101 | 0.000009 | com.globalpost |
872 | 19367676 | 814 | 0.000037 | gov.nist |
873 | 19367536 | 4563 | 0.000008 | org.globalsecurity |
874 | 19366454 | 4547 | 0.000008 | build.bazel |
875 | 19366384 | 3782 | 0.000009 | us.ms.state |
876 | 19365878 | 4256 | 0.000008 | gr.ntua |
877 | 19365776 | 4444 | 0.000008 | se.thelocal |
878 | 19365372 | 2963 | 0.000012 | com.politifact |
879 | 19365128 | 1317 | 0.000023 | com.ensighten |
880 | 19363588 | 5097 | 0.000007 | ru.my1 |
881 | 19362680 | 3468 | 0.000010 | com.rabbitmq |
882 | 19359698 | 4138 | 0.000009 | com.elasticbeanstalk |
883 | 19359574 | 1364 | 0.000022 | com.billboard |
884 | 19359122 | 4766 | 0.000008 | cc.dict |
885 | 19358774 | 5687 | 0.000007 | fi.mbnet |
886 | 19357390 | 879 | 0.000035 | com.aliexpress |
887 | 19356918 | 210 | 0.000111 | to.amzn |
888 | 19355668 | 4275 | 0.000008 | edu.ohio |
889 | 19355546 | 3452 | 0.000010 | com.thejakartapost |
890 | 19355350 | 3277 | 0.000011 | vn.com.dantri |
891 | 19355080 | 5285 | 0.000007 | com.galvanize |
892 | 19354880 | 3484 | 0.000010 | jp.go.ndl |
893 | 19354790 | 4710 | 0.000008 | com.kiwibox |
894 | 19354514 | 2140 | 0.000015 | org.linuxfoundation |
895 | 19354500 | 4801 | 0.000007 | ru.nnov |
896 | 19353166 | 4288 | 0.000008 | gr.auth |
897 | 19352970 | 2257 | 0.000014 | net.vnexpress |
898 | 19351770 | 2900 | 0.000012 | com.crashlytics |
899 | 19351594 | 1045 | 0.000028 | com.dropboxusercontent |
900 | 19350828 | 3439 | 0.000010 | com.scotusblog |
901 | 19350712 | 4090 | 0.000009 | org.carnegieendowment |
902 | 19350278 | 395 | 0.000064 | com.atlassian |
903 | 19349726 | 3465 | 0.000010 | com.study |
904 | 19348724 | 350 | 0.000072 | com.mapbox |
905 | 19348532 | 1046 | 0.000028 | com.redhat |
906 | 19347886 | 1799 | 0.000017 | com.bravenet |
907 | 19347460 | 4284 | 0.000008 | uk.org.npg |
908 | 19347152 | 4463 | 0.000008 | com.btplc |
909 | 19347148 | 5289 | 0.000007 | ru.drom |
910 | 19346542 | 2430 | 0.000013 | com.vimeopro |
911 | 19345900 | 4419 | 0.000008 | edu.marquette |
912 | 19345644 | 426 | 0.000061 | com.adweek |
913 | 19345144 | 914 | 0.000033 | com.shutterstock |
914 | 19345090 | 1016 | 0.000029 | com.ubuntu |
915 | 19341960 | 5712 | 0.000007 | in.ac.nptel |
916 | 19341488 | 1227 | 0.000024 | com.msdn |
917 | 19340714 | 4707 | 0.000008 | com.vocabulary |
918 | 19340680 | 3929 | 0.000009 | edu.uaf |
919 | 19339658 | 3919 | 0.000009 | com.atavist |
920 | 19339456 | 3201 | 0.000011 | com.healthgrades |
921 | 19339092 | 2546 | 0.000013 | com.kinstacdn |
922 | 19338384 | 2345 | 0.000014 | com.gazhall |
923 | 19337938 | 5398 | 0.000007 | com.asmallorange |
924 | 19337800 | 3797 | 0.000009 | com.generalmills |
925 | 19336176 | 4585 | 0.000008 | vn.vtc |
926 | 19335908 | 1519 | 0.000020 | cn.gov.mofcom |
927 | 19333778 | 797 | 0.000038 | com.box |
928 | 19333606 | 3966 | 0.000009 | si.uni-lj |
929 | 19333322 | 4170 | 0.000008 | az.president |
930 | 19333194 | 1788 | 0.000017 | org.reactjs |
931 | 19332412 | 3605 | 0.000010 | com.postaffiliatepro |
932 | 19331922 | 5192 | 0.000007 | edu.uah |
933 | 19331280 | 3599 | 0.000010 | org.openedition |
934 | 19330696 | 4838 | 0.000007 | com.kapook |
935 | 19330382 | 4153 | 0.000008 | org.caringbridge |
936 | 19330374 | 483 | 0.000053 | com.aol |
937 | 19329614 | 2303 | 0.000014 | org.nfpa |
938 | 19329538 | 5956 | 0.000006 | com.glosbe |
939 | 19329194 | 4124 | 0.000009 | com.mcall |
940 | 19327622 | 4289 | 0.000008 | ru.tmweb |
941 | 19326876 | 4126 | 0.000009 | uk.co.liverpoolecho |
942 | 19326422 | 4244 | 0.000008 | com.atwebpages |
943 | 19325980 | 1067 | 0.000028 | com.freepik |
944 | 19324790 | 4085 | 0.000009 | org.specialolympics |
945 | 19323868 | 4845 | 0.000007 | net.freeforums |
946 | 19323676 | 4744 | 0.000008 | uk.ac.westminster |
947 | 19323532 | 4092 | 0.000009 | com.tok2 |
948 | 19323460 | 1025 | 0.000029 | com.elpais |
949 | 19323150 | 4946 | 0.000007 | tw.com.sina |
950 | 19322508 | 3296 | 0.000011 | com.wowza |
951 | 19322306 | 317 | 0.000079 | com.webs |
952 | 19322024 | 4697 | 0.000008 | com.warriorplus |
953 | 19321918 | 3414 | 0.000010 | com.cityam |
954 | 19321812 | 4482 | 0.000008 | org.fee |
955 | 19321520 | 4854 | 0.000007 | tw.edu.ntnu |
956 | 19321296 | 4962 | 0.000007 | com.sparknotes |
957 | 19320202 | 4516 | 0.000008 | com.newspapers |
958 | 19319634 | 2192 | 0.000015 | com.tutsplus |
959 | 19319600 | 5868 | 0.000007 | com.ananova |
960 | 19319274 | 3818 | 0.000009 | org.opensecrets |
961 | 19319134 | 633 | 0.000044 | gov.uspto |
962 | 19318722 | 5680 | 0.000007 | su.moy |
963 | 19318366 | 1013 | 0.000029 | com.uk |
964 | 19318266 | 4936 | 0.000007 | ru.pr-cy |
965 | 19318058 | 3827 | 0.000009 | cz.centrum |
966 | 19317780 | 4158 | 0.000008 | edu.niu |
967 | 19315320 | 1665 | 0.000018 | org.webkit |
968 | 19315014 | 4692 | 0.000008 | pl.edu.amu |
969 | 19314084 | 5186 | 0.000007 | com.artfire |
970 | 19313894 | 3800 | 0.000009 | org.ascd |
971 | 19312106 | 3801 | 0.000009 | edu.scu |
972 | 19311742 | 4307 | 0.000008 | com.taipeitimes |
973 | 19311568 | 4351 | 0.000008 | edu.whoi |
974 | 19310854 | 5949 | 0.000006 | com.voatiengviet |
975 | 19310748 | 3100 | 0.000011 | com.broadcastingcable |
976 | 19310720 | 4655 | 0.000008 | hk.rthk |
977 | 19310246 | 5703 | 0.000007 | com.enotes |
978 | 19309910 | 488 | 0.000053 | com.indiatimes |
979 | 19309660 | 860 | 0.000035 | com.playstation |
980 | 19309040 | 4866 | 0.000007 | com.brothersoft |
981 | 19308948 | 2708 | 0.000013 | uk.gov.defra |
982 | 19307606 | 231 | 0.000103 | org.whatwg |
983 | 19307178 | 4451 | 0.000008 | com.batchgeo |
984 | 19307118 | 751 | 0.000040 | com.psychologytoday |
985 | 19306368 | 4263 | 0.000008 | uk.co.lrb |
986 | 19306350 | 5034 | 0.000007 | ca.pe.gov |
987 | 19305884 | 4159 | 0.000008 | com.ecowatch |
988 | 19303820 | 4195 | 0.000008 | com.williamhill |
989 | 19303548 | 5767 | 0.000007 | pt.ipp |
990 | 19302972 | 4843 | 0.000007 | uk.org.38degrees |
991 | 19301624 | 1303 | 0.000023 | com.technologyreview |
992 | 19301464 | 4091 | 0.000009 | org.spie |
993 | 19301068 | 959 | 0.000031 | com.libsyn |
994 | 19300572 | 4795 | 0.000007 | com.storeboard |
995 | 19300548 | 3260 | 0.000011 | de.bmel |
996 | 19299448 | 4749 | 0.000008 | net.onlinewebshop |
997 | 19299274 | 3872 | 0.000009 | ru.1gb |
998 | 19298654 | 279 | 0.000088 | com.automattic |
999 | 19298502 | 3870 | 0.000009 | com.piie |
1000 | 19297440 | 5306 | 0.000007 | com.allthatsinteresting |
Credits
Thanks to the authors of the WebGraph framework, whose software made the computation of graph properties and ranks possible.
We hope the data will be useful for you to do any kind of research on ranking, graph analysis, link spam detection, etc. Let us know about your results via Common Crawl’s Google Group!
May/June 2020 crawl archive now available
The crawl archive for May/June 2020 is now available! It contains 2.75 billion web pages or 255 TiB of uncompressed content, crawled between May 24th and June 7th. It includes page captures of 1.2 billion URLs unknown in any of our prior crawl archives.
Starting with this crawl the WET files indicate the natural language(s) a text is written in. The language is detected using Compact Language Detector 2 (CLD2) and was made available since August 2018 only in WARC and WAT files and URL indexes. It is now also provided in WET files in the WARC header "WARC-Identified-Content-Language". Up to three language(s) are detected per document and given as comma-separated list of ISO-639-3 codes, here one example WET record fragment:
... WARC-Identified-Content-Language: isl,eng Content-Type: text/plain Content-Length: 10494 Bananabrauð með Nutella – Ljúfmeti og lekkerheit ...
Additional information about this improvement is given in the corresponding issue report.
Archive Location and Download
The May/June crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-24/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-24/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-24/warc.paths.gz | 60000 | 53.16 |
WAT files | CC-MAIN-2020-24/wat.paths.gz | 60000 | 19.02 |
WET files | CC-MAIN-2020-24/wet.paths.gz | 60000 | 8.42 |
Robots.txt files | CC-MAIN-2020-24/robotstxt.paths.gz | 60000 | 0.22 |
Non-200 responses files | CC-MAIN-2020-24/non200responses.paths.gz | 60000 | 2.77 |
URL index files | CC-MAIN-2020-24/cc-index.paths.gz | 302 | 0.22 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-24/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
March/April 2020 crawl archive now available
The crawl archive for March/April 2020 is now available! It contains 2.85 billion web pages or 280 TiB of uncompressed content, crawled between March 28th and April 10th. It includes page captures of 1 billion URLs unknown in any of our prior crawl archives.
Archive Location and Download
The March/April crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-16/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-16/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-16/warc.paths.gz | 56000 | 62.67 |
WAT files | CC-MAIN-2020-16/wat.paths.gz | 56000 | 20.37 |
WET files | CC-MAIN-2020-16/wet.paths.gz | 56000 | 8.97 |
Robots.txt files | CC-MAIN-2020-16/robotstxt.paths.gz | 56000 | 0.19 |
Non-200 responses files | CC-MAIN-2020-16/non200responses.paths.gz | 56000 | 1.39 |
URL index files | CC-MAIN-2020-16/cc-index.paths.gz | 302 | 0.21 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-16/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
February 2020 crawl archive now available
The crawl archive for February 2020 is now available! It contains 2.6 billion web pages or 240 TiB of uncompressed content, crawled between February 16th and 29th. It includes page captures of 1 billion URLs unknown in any of our prior crawl archives.
Improvements and Fixes
The HTTP headers in WARC response records have been fixed: the HTTP response status line now has a white space following the status code if the reason-phrase is empty. E.g., if a server sends an empty message (instead of “OK”), the status line will include a trailing space character: “HTTP/1.1 200
”. Following RFC 7230 the white space between status code and message is mandatory. Please refer to the bug report NUTCH-2763 for further details.
Archive Location and Download
The February crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-10/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-10/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-10/warc.paths.gz | 56000 | 49.28 |
WAT files | CC-MAIN-2020-10/wat.paths.gz | 56000 | 17.98 |
WET files | CC-MAIN-2020-10/wet.paths.gz | 56000 | 7.97 |
Robots.txt files | CC-MAIN-2020-10/robotstxt.paths.gz | 56000 | 0.22 |
Non-200 responses files | CC-MAIN-2020-10/non200responses.paths.gz | 56000 | 2.21 |
URL index files | CC-MAIN-2020-10/cc-index.paths.gz | 302 | 0.2 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-10/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
Host- and Domain-Level Web Graphs Nov/Dec/Jan 2019 – 2020
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of November, December 2019 and January 2020. Additional information about the data formats, the processing pipeline, our objectives, and credits can be found in the announcements of prior webgraph releases (e.g., Nov/Dec/Jan 2017-2018 Webgraphs). You may also visit the projects cc-webgraph and cc-pyspark which host all scripts and tools required to construct the graphs.
Host-level graph
The graph consists of 1.24 billion nodes and 4.54 billion edges and includes dangling nodes i.e. hosts that have not been crawled yet are pointed to from a link on a crawled page. There are 1.17 billion dangling nodes (95%) and the largest strongly connected component contains 45 million (3.6%) nodes.
You can download the graph and the ranks of all 1.24 billion hosts from AWS S3 on the path s3://commoncrawl/projects/hyperlinkgraph/cc-main-2019-20-nov-dec-jan/host/
. Alternatively, you can use https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2019-20-nov-dec-jan/host/
as prefix to access the files from everywhere.
Download files of the Common Crawl Nov/Dec/Jan 2019-20 host-level webgraph
Size | File | Description |
---|---|---|
7.23 GB | cc-main-2019-20-nov-dec-jan-host-vertices.paths.gz | nodes 〈id, rev host〉, paths of 12 vertices files |
20.16 GB | cc-main-2019-20-nov-dec-jan-host-edges.paths.gz | edges 〈from_id, to_id〉, paths of 24 edges files |
8.42 GB | cc-main-2019-20-nov-dec-jan-host.graph | graph in BVGraph format |
2 kB | cc-main-2019-20-nov-dec-jan-host.properties | |
10.80 GB | cc-main-2019-20-nov-dec-jan-host-t.graph | transpose of the graph (outlinks inverted to inlinks) |
2 kB | cc-main-2019-20-nov-dec-jan-host-t.properties | |
1 kB | cc-main-2019-20-nov-dec-jan-host.stats | WebGraph statistics |
16.32 GB | cc-main-2019-20-nov-dec-jan-host-ranks.txt.gz | harmonic centrality and pagerank |
Note that the host names are reversed and a leading www.
is stripped: www.subdomain.example.com
becomes com.example.subdomain
.
Domain-level graph
The domain graph was built by aggregating the host graph on the level of pay-level domains (PLDs) based on the public suffix list maintained on publicsuffix.org.
The domain-level graph has 85.8 million nodes and 1.9 billion edges. 51% or 44 million nodes are dangling nodes, the largest strongly connected component covers 34 million or 39% of the nodes.
All files related to the domain graph are available on AWS S3 under s3://commoncrawl/projects/hyperlinkgraph/cc-main-2019-20-nov-dec-jan/domain/
resp. https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2019-20-nov-dec-jan/domain/
.
Download files of the Common Crawl Nov/Dec/Jan 2019-20 domain-level webgraph
Size | File | Description |
---|---|---|
0.59 GB | cc-main-2019-20-nov-dec-jan-domain-vertices.txt.gz | nodes 〈id, rev domain, num hosts〉 |
7.65 GB | cc-main-2019-20-nov-dec-jan-domain-edges.txt.gz | edges 〈from_id, to_id〉 |
4.10 GB | cc-main-2019-20-nov-dec-jan-domain.graph | graph in BVGraph format |
2 kB | cc-main-2019-20-nov-dec-jan-domain.properties | |
4.13 GB | cc-main-2019-20-nov-dec-jan-domain-t.graph | transpose of the graph |
2 kB | cc-main-2019-20-nov-dec-jan-domain-t.properties | |
1 kB | cc-main-2019-20-nov-dec-jan-domain.stats | WebGraph statistics |
1.86 GB | cc-main-2019-20-nov-dec-jan-domain-ranks.txt.gz | harmonic centrality and pagerank |
Below you’ll find the top 1000 domains ranked by Harmonic Centrality or PageRank. The full list of all 86 million domain ranks is available for download.
Top 1000 domains ranked by harmonic centrality (Nov/Dec/Jan 2019-2020)
harmonic centrality rank | hc value | page rank | page rank value | reversed hostname |
---|---|---|---|---|
1 | 30598398 | 1 | 0.019072 | com.googleapis |
2 | 29113136 | 3 | 0.012214 | com.facebook |
3 | 27475138 | 2 | 0.013236 | com.google |
4 | 25610480 | 4 | 0.007452 | com.twitter |
5 | 24947126 | 5 | 0.007174 | org.w |
6 | 24904712 | 6 | 0.006611 | com.youtube |
7 | 23281504 | 9 | 0.004269 | com.instagram |
8 | 22446296 | 7 | 0.005561 | org.gmpg |
9 | 22154750 | 8 | 0.005033 | com.googletagmanager |
10 | 22107784 | 13 | 0.003001 | com.linkedin |
11 | 21307220 | 10 | 0.003433 | org.wordpress |
12 | 21290688 | 20 | 0.001717 | com.gravatar |
13 | 21096944 | 11 | 0.003266 | com.cloudflare |
14 | 21086168 | 23 | 0.001516 | com.pinterest |
15 | 20869868 | 16 | 0.002242 | com.gstatic |
16 | 20855286 | 15 | 0.002366 | com.wordpress |
17 | 20713268 | 25 | 0.001234 | org.wikipedia |
18 | 20641712 | 17 | 0.002130 | com.apple |
19 | 20584368 | 14 | 0.002460 | com.bootstrapcdn |
20 | 20371046 | 33 | 0.001102 | com.vimeo |
21 | 20336732 | 42 | 0.000833 | com.blogspot |
22 | 20237058 | 18 | 0.001787 | com.jquery |
23 | 20220572 | 50 | 0.000732 | be.youtu |
24 | 20218806 | 32 | 0.001130 | com.microsoft |
25 | 20119820 | 49 | 0.000737 | com.wp |
26 | 20059828 | 19 | 0.001764 | com.adobe |
27 | 20029544 | 52 | 0.000709 | com.amazon |
28 | 20013346 | 44 | 0.000784 | gl.goo |
29 | 19972608 | 35 | 0.001020 | com.amazonaws |
30 | 19941934 | 67 | 0.000471 | com.tumblr |
31 | 19937390 | 64 | 0.000518 | ly.bit |
32 | 19862906 | 29 | 0.001164 | com.macromedia |
33 | 19817494 | 30 | 0.001151 | com.baidu |
34 | 19816382 | 38 | 0.000958 | com.google-analytics |
35 | 19808066 | 31 | 0.001142 | com.googlesyndication |
36 | 19781460 | 34 | 0.001092 | net.cloudfront |
37 | 19770754 | 24 | 0.001254 | ru.yandex |
38 | 19749302 | 53 | 0.000693 | com.flickr |
39 | 19700112 | 22 | 0.001568 | com.github |
40 | 19698814 | 80 | 0.000368 | com.yahoo |
41 | 19676534 | 58 | 0.000644 | eu.europa |
42 | 19632506 | 115 | 0.000287 | com.reddit |
43 | 19603190 | 41 | 0.000918 | com.addthis |
44 | 19561006 | 72 | 0.000403 | com.weebly |
45 | 19559908 | 43 | 0.000823 | org.w3 |
46 | 19550116 | 63 | 0.000524 | me.wp |
47 | 19546614 | 108 | 0.000313 | com.googleusercontent |
48 | 19531876 | 45 | 0.000777 | io.github |
49 | 19523224 | 182 | 0.000140 | org.wikimedia |
50 | 19521602 | 70 | 0.000422 | com.medium |
51 | 19516098 | 47 | 0.000743 | org.schema |
52 | 19496222 | 46 | 0.000754 | net.jsdelivr |
53 | 19495662 | 76 | 0.000374 | org.creativecommons |
54 | 19472610 | 173 | 0.000153 | com.imgur |
55 | 19459848 | 36 | 0.000989 | net.doubleclick |
56 | 19451512 | 51 | 0.000711 | com.wix |
57 | 19416054 | 186 | 0.000138 | uk.co.bbc |
58 | 19408312 | 149 | 0.000181 | com.forbes |
59 | 19405426 | 93 | 0.000332 | com.weibo |
60 | 19404990 | 60 | 0.000601 | co.t |
61 | 19398622 | 28 | 0.001192 | com.fontawesome |
62 | 19382484 | 48 | 0.000741 | com.paypal |
63 | 19375914 | 210 | 0.000114 | com.cnn |
64 | 19372292 | 144 | 0.000190 | org.archive |
65 | 19368680 | 61 | 0.000583 | org.mozilla |
66 | 19348182 | 189 | 0.000137 | net.sourceforge |
67 | 19304774 | 315 | 0.000079 | edu.mit |
68 | 19302936 | 174 | 0.000152 | com.theguardian |
69 | 19293782 | 278 | 0.000089 | edu.harvard |
70 | 19283372 | 179 | 0.000144 | com.bing |
71 | 19271390 | 117 | 0.000286 | com.jimdo |
72 | 19270368 | 140 | 0.000202 | com.nytimes |
73 | 19260752 | 27 | 0.001195 | com.qq |
74 | 19259406 | 237 | 0.000103 | com.wsj |
75 | 19257754 | 21 | 0.001581 | org.apache |
76 | 19256254 | 56 | 0.000658 | com.googleadservices |
77 | 19241110 | 215 | 0.000111 | com.washingtonpost |
78 | 19237816 | 269 | 0.000092 | com.bloomberg |
79 | 19237164 | 259 | 0.000094 | com.techcrunch |
80 | 19210562 | 544 | 0.000049 | com.deviantart |
81 | 19205946 | 162 | 0.000162 | org.ietf |
82 | 19198346 | 247 | 0.000098 | com.oracle |
83 | 19194696 | 279 | 0.000089 | com.android |
84 | 19193448 | 69 | 0.000430 | com.list-manage |
85 | 19174968 | 397 | 0.000065 | com.ted |
86 | 19167820 | 321 | 0.000078 | com.reuters |
87 | 19161964 | 312 | 0.000080 | com.wired |
88 | 19158768 | 154 | 0.000173 | com.wixsite |
89 | 19154874 | 341 | 0.000073 | com.ft |
90 | 19152068 | 330 | 0.000076 | uk.co.telegraph |
91 | 19151666 | 427 | 0.000060 | com.theverge |
92 | 19145078 | 155 | 0.000172 | gov.nih |
93 | 19143886 | 272 | 0.000091 | com.myspace |
94 | 19142172 | 377 | 0.000068 | gov.nasa |
95 | 19136022 | 291 | 0.000085 | com.bbc |
96 | 19126138 | 339 | 0.000074 | com.example |
97 | 19119916 | 239 | 0.000103 | org.python |
98 | 19117868 | 82 | 0.000361 | com.whatsapp |
99 | 19111550 | 122 | 0.000267 | com.unpkg |
100 | 19110346 | 188 | 0.000138 | uk.co.google |
101 | 19089504 | 707 | 0.000041 | com.economist |
102 | 19088766 | 255 | 0.000095 | com.appspot |
103 | 19082300 | 384 | 0.000067 | uk.co.dailymail |
104 | 19081904 | 209 | 0.000115 | org.gnu |
105 | 19081708 | 262 | 0.000093 | com.githubusercontent |
106 | 19080182 | 131 | 0.000232 | com.ytimg |
107 | 19077156 | 320 | 0.000078 | org.un |
108 | 19075806 | 163 | 0.000162 | com.giphy |
109 | 19068860 | 398 | 0.000065 | com.latimes |
110 | 19067606 | 169 | 0.000157 | com.twimg |
111 | 19066404 | 431 | 0.000060 | com.googleblog |
112 | 19056162 | 176 | 0.000148 | com.blogger |
113 | 19054310 | 232 | 0.000104 | com.dribbble |
114 | 19052824 | 207 | 0.000115 | com.npmjs |
115 | 19050524 | 564 | 0.000047 | org.arxiv |
116 | 19045194 | 666 | 0.000042 | edu.upenn |
117 | 19042770 | 171 | 0.000154 | com.eventbrite |
118 | 19036612 | 379 | 0.000068 | com.springer |
119 | 19032422 | 277 | 0.000090 | org.ampproject |
120 | 19031354 | 557 | 0.000047 | com.gitlab |
121 | 19025616 | 596 | 0.000045 | com.vice |
122 | 19025236 | 206 | 0.000116 | com.disqus |
123 | 19023978 | 1036 | 0.000031 | com.hatenablog |
124 | 19023406 | 835 | 0.000039 | edu.columbia |
125 | 19018670 | 818 | 0.000040 | io.readthedocs |
126 | 19008280 | 205 | 0.000116 | me.t |
127 | 19005842 | 390 | 0.000066 | com.w3schools |
128 | 19004030 | 941 | 0.000034 | org.chromium |
129 | 19003900 | 418 | 0.000062 | com.nature |
130 | 19001452 | 716 | 0.000041 | com.slate |
131 | 19000862 | 157 | 0.000171 | jp.co.yahoo |
132 | 18997566 | 325 | 0.000077 | com.time |
133 | 18997164 | 430 | 0.000060 | com.statista |
134 | 18990336 | 744 | 0.000040 | com.ubuntu |
135 | 18985466 | 158 | 0.000167 | com.yelp |
136 | 18982550 | 632 | 0.000043 | org.worldbank |
137 | 18982428 | 143 | 0.000191 | com.spotify |
138 | 18981078 | 347 | 0.000072 | com.skype |
139 | 18978660 | 935 | 0.000034 | com.playstation |
140 | 18976524 | 306 | 0.000082 | com.fc2 |
141 | 18973312 | 1304 | 0.000024 | org.coursera |
142 | 18969438 | 121 | 0.000281 | com.stripe |
143 | 18968562 | 816 | 0.000040 | com.qz |
144 | 18968168 | 617 | 0.000044 | com.git-scm |
145 | 18966448 | 486 | 0.000053 | uk.co.independent |
146 | 18965810 | 199 | 0.000126 | com.eepurl |
147 | 18964444 | 961 | 0.000034 | com.500px |
148 | 18964342 | 405 | 0.000063 | net.researchgate |
149 | 18962654 | 241 | 0.000101 | com.bandcamp |
150 | 18959274 | 55 | 0.000669 | net.facebook |
151 | 18956478 | 389 | 0.000066 | com.outlook |
152 | 18955490 | 229 | 0.000105 | com.unsplash |
153 | 18950460 | 631 | 0.000043 | com.mysql |
154 | 18949330 | 419 | 0.000062 | com.theatlantic |
155 | 18948456 | 116 | 0.000286 | com.soundcloud |
156 | 18948364 | 180 | 0.000143 | com.amazon-adsystem |
157 | 18947670 | 123 | 0.000259 | org.networkadvertising |
158 | 18940768 | 571 | 0.000046 | org.bitbucket |
159 | 18940298 | 1163 | 0.000027 | com.jetbrains |
160 | 18936474 | 408 | 0.000063 | com.mozilla |
161 | 18936104 | 498 | 0.000052 | com.nationalgeographic |
162 | 18932906 | 316 | 0.000079 | com.usatoday |
163 | 18930890 | 439 | 0.000059 | com.criteo |
164 | 18927342 | 837 | 0.000039 | uk.ac.ox |
165 | 18925560 | 468 | 0.000054 | com.fortune |
166 | 18924956 | 466 | 0.000055 | com.pixabay |
167 | 18922224 | 1278 | 0.000024 | uk.co.thesun |
168 | 18921476 | 230 | 0.000104 | net.behance |
169 | 18916700 | 1547 | 0.000019 | com.amd |
170 | 18915574 | 822 | 0.000039 | com.evernote |
171 | 18909918 | 40 | 0.000932 | com.vk |
172 | 18909318 | 799 | 0.000040 | com.about |
173 | 18907910 | 505 | 0.000051 | uk.co.blogspot |
174 | 18904000 | 1193 | 0.000026 | se.haxx |
175 | 18903686 | 251 | 0.000097 | gle.forms |
176 | 18900132 | 719 | 0.000041 | com.docker |
177 | 18900020 | 967 | 0.000033 | uk.co.guardian |
178 | 18899576 | 303 | 0.000082 | org.doi |
179 | 18898330 | 497 | 0.000052 | me.about |
180 | 18896862 | 525 | 0.000050 | gg.discord |
181 | 18894432 | 1383 | 0.000022 | com.instructables |
182 | 18891582 | 147 | 0.000189 | com.dropbox |
183 | 18888432 | 1000 | 0.000032 | com.scientificamerican |
184 | 18885228 | 332 | 0.000076 | jp.co.rakuten |
185 | 18881548 | 903 | 0.000036 | google.blog |
186 | 18875704 | 194 | 0.000130 | com.feedburner |
187 | 18870758 | 1081 | 0.000030 | org.altervista |
188 | 18869068 | 634 | 0.000043 | org.unesco |
189 | 18868966 | 1082 | 0.000030 | org.eclipse |
190 | 18868488 | 275 | 0.000090 | gov.ca |
191 | 18866552 | 875 | 0.000037 | jp.livedoor |
192 | 18865164 | 1490 | 0.000020 | org.phys |
193 | 18864674 | 292 | 0.000085 | com.sciencedirect |
194 | 18860674 | 212 | 0.000113 | jp.ameblo |
195 | 18855700 | 572 | 0.000046 | gov.loc |
196 | 18854892 | 993 | 0.000033 | org.cambridge |
197 | 18845512 | 403 | 0.000064 | ca.google |
198 | 18842558 | 663 | 0.000042 | edu.washington |
199 | 18836168 | 71 | 0.000408 | net.slideshare |
200 | 18832808 | 543 | 0.000049 | com.cisco |
201 | 18829838 | 1364 | 0.000023 | edu.rutgers |
202 | 18828808 | 304 | 0.000082 | com.nbcnews |
203 | 18827294 | 243 | 0.000099 | ru.rambler |
204 | 18826594 | 823 | 0.000039 | au.net.abc |
205 | 18826034 | 998 | 0.000032 | uk.co.thetimes |
206 | 18825554 | 1472 | 0.000021 | com.bankofamerica |
207 | 18821912 | 62 | 0.000549 | com.fb |
208 | 18821670 | 863 | 0.000037 | org.sciencemag |
209 | 18813844 | 887 | 0.000036 | com.speakerdeck |
210 | 18810986 | 412 | 0.000063 | jp.ne.sakura |
211 | 18806748 | 220 | 0.000109 | org.iana |
212 | 18802112 | 1258 | 0.000025 | com.wikidot |
213 | 18799706 | 1650 | 0.000018 | pt.sapo |
214 | 18792880 | 905 | 0.000036 | uk.co.mirror |
215 | 18790620 | 218 | 0.000110 | edu.stanford |
216 | 18787734 | 1143 | 0.000028 | org.kernel |
217 | 18786526 | 450 | 0.000057 | com.elsevier |
218 | 18784420 | 1452 | 0.000021 | edu.osu |
219 | 18783446 | 1622 | 0.000019 | com.googlesource |
220 | 18782426 | 1395 | 0.000022 | com.vogue |
221 | 18781902 | 74 | 0.000401 | net.akamaihd |
222 | 18781258 | 842 | 0.000038 | gov.fcc |
223 | 18780926 | 1240 | 0.000025 | ms.1drv |
224 | 18779106 | 1377 | 0.000022 | edu.asu |
225 | 18778828 | 254 | 0.000095 | com.businessinsider |
226 | 18778534 | 844 | 0.000038 | co.ibb |
227 | 18778386 | 1843 | 0.000016 | com.wolfram |
228 | 18771090 | 721 | 0.000041 | com.trello |
229 | 18769148 | 106 | 0.000323 | com.paypalobjects |
230 | 18766298 | 301 | 0.000083 | net.windows |
231 | 18762238 | 1216 | 0.000026 | jp.geocities |
232 | 18762150 | 587 | 0.000045 | com.box |
233 | 18761840 | 876 | 0.000037 | com.sciencedaily |
234 | 18758802 | 227 | 0.000106 | com.wpengine |
235 | 18752986 | 493 | 0.000052 | com.herokuapp |
236 | 18752026 | 1030 | 0.000032 | edu.princeton |
237 | 18748814 | 915 | 0.000035 | edu.academia |
238 | 18748232 | 284 | 0.000087 | com.googlecode |
239 | 18746146 | 1177 | 0.000027 | com.asahi |
240 | 18740044 | 1375 | 0.000022 | com.newscientist |
241 | 18733718 | 1428 | 0.000021 | blog.home |
242 | 18732458 | 314 | 0.000080 | com.tinyurl |
243 | 18732010 | 516 | 0.000051 | com.udacity |
244 | 18731740 | 2103 | 0.000014 | com.wizards |
245 | 18729906 | 386 | 0.000067 | com.cnet |
246 | 18729644 | 1421 | 0.000022 | com.ndtv |
247 | 18729370 | 335 | 0.000075 | com.getpocket |
248 | 18729276 | 1349 | 0.000023 | com.fandom |
249 | 18726412 | 1241 | 0.000025 | net.seesaa |
250 | 18723376 | 167 | 0.000158 | com.imdb |
251 | 18718440 | 333 | 0.000076 | org.debian |
252 | 18715552 | 699 | 0.000041 | site.business |
253 | 18713638 | 261 | 0.000093 | com.live |
254 | 18711498 | 1059 | 0.000031 | jp.ne.goo |
255 | 18710778 | 1504 | 0.000020 | io.itch |
256 | 18704016 | 1155 | 0.000027 | org.greenpeace |
257 | 18702254 | 1047 | 0.000031 | com.netlify |
258 | 18701568 | 1808 | 0.000017 | net.pixnet |
259 | 18700552 | 402 | 0.000064 | com.squareup |
260 | 18699852 | 1145 | 0.000028 | co.elastic |
261 | 18699756 | 310 | 0.000081 | com.ibm |
262 | 18698784 | 203 | 0.000118 | com.stackoverflow |
263 | 18698146 | 494 | 0.000052 | com.indiatimes |
264 | 18696312 | 3420 | 0.000010 | com.armorgames |
265 | 18693148 | 265 | 0.000092 | com.aliyuncs |
266 | 18692378 | 226 | 0.000106 | com.optimizely |
267 | 18685944 | 1777 | 0.000017 | uk.co.timesonline |
268 | 18679570 | 975 | 0.000033 | com.mixcloud |
269 | 18677530 | 1618 | 0.000019 | com.itv |
270 | 18674460 | 198 | 0.000128 | org.bbb |
271 | 18674340 | 57 | 0.000648 | net.fbcdn |
272 | 18674148 | 2236 | 0.000014 | com.opendns |
273 | 18672682 | 3302 | 0.000010 | tw.com.gamer |
274 | 18672646 | 406 | 0.000063 | com.go |
275 | 18672486 | 437 | 0.000059 | com.msn |
276 | 18672432 | 37 | 0.000966 | com.wixstatic |
277 | 18667606 | 1485 | 0.000021 | org.archlinux |
278 | 18666546 | 448 | 0.000058 | org.pewresearch |
279 | 18664590 | 87 | 0.000345 | com.shopify |
280 | 18662914 | 838 | 0.000039 | jp.shinobi |
281 | 18662550 | 995 | 0.000032 | com.bmj |
282 | 18660516 | 1579 | 0.000019 | com.diigo |
283 | 18659688 | 160 | 0.000167 | com.opera |
284 | 18659332 | 1822 | 0.000017 | com.youdao |
285 | 18657686 | 1259 | 0.000025 | com.angelfire |
286 | 18657376 | 906 | 0.000036 | jp.naver |
287 | 18656934 | 1038 | 0.000031 | com.thelancet |
288 | 18653610 | 1585 | 0.000019 | uk.bl |
289 | 18652186 | 864 | 0.000037 | br.com.google |
290 | 18649914 | 490 | 0.000053 | com.bigcartel |
291 | 18648796 | 1007 | 0.000032 | com.sky |
292 | 18647596 | 1408 | 0.000022 | net.daringfireball |
293 | 18642592 | 1779 | 0.000017 | uk.ac.kcl |
294 | 18641168 | 1752 | 0.000017 | org.maven |
295 | 18639544 | 534 | 0.000049 | me.m |
296 | 18639298 | 1107 | 0.000029 | com.reverbnation |
297 | 18637290 | 1665 | 0.000018 | net.cnki |
298 | 18634102 | 1089 | 0.000030 | com.theconversation |
299 | 18630504 | 460 | 0.000056 | it.placehold |
300 | 18629064 | 1117 | 0.000029 | com.podbean |
301 | 18628660 | 881 | 0.000036 | org.fao |
302 | 18626956 | 1144 | 0.000028 | co.g |
303 | 18626264 | 1370 | 0.000023 | com.dw |
304 | 18623646 | 119 | 0.000283 | com.mailchimp |
305 | 18621260 | 1631 | 0.000019 | jp.ne.so-net |
306 | 18620264 | 1346 | 0.000023 | com.livescience |
307 | 18620110 | 2071 | 0.000015 | edu.kit |
308 | 18611900 | 1308 | 0.000024 | ca.utoronto |
309 | 18610618 | 1310 | 0.000024 | com.webnode |
310 | 18608388 | 957 | 0.000034 | au.gov.nsw |
311 | 18606964 | 1411 | 0.000022 | com.citrix |
312 | 18603704 | 1031 | 0.000032 | jp.jugem |
313 | 18602374 | 1115 | 0.000029 | gov.wa |
314 | 18600002 | 458 | 0.000056 | com.quora |
315 | 18598840 | 99 | 0.000326 | com.godaddy |
316 | 18597740 | 1334 | 0.000023 | com.bloglovin |
317 | 18596834 | 1194 | 0.000026 | com.serving-sys |
318 | 18595976 | 900 | 0.000036 | gov.dhs |
319 | 18595120 | 1630 | 0.000019 | org.edx |
320 | 18592696 | 244 | 0.000099 | me.wa |
321 | 18592582 | 1830 | 0.000017 | com.pearltrees |
322 | 18592022 | 1576 | 0.000019 | com.twitpic |
323 | 18591744 | 1776 | 0.000017 | cn.people |
324 | 18590350 | 1171 | 0.000027 | com.britannica |
325 | 18589380 | 1893 | 0.000016 | sg.edu.nus |
326 | 18588130 | 1791 | 0.000017 | com.kinja |
327 | 18587404 | 2296 | 0.000013 | com.authorstream |
328 | 18585922 | 1566 | 0.000019 | ca.mcgill |
329 | 18585132 | 380 | 0.000068 | com.kickstarter |
330 | 18581636 | 1237 | 0.000025 | com.lulu |
331 | 18576948 | 2591 | 0.000012 | com.colourlovers |
332 | 18575064 | 1919 | 0.000016 | com.hm |
333 | 18574242 | 337 | 0.000075 | com.rackcdn |
334 | 18567994 | 2261 | 0.000013 | uk.ac.sussex |
335 | 18563954 | 1945 | 0.000016 | org.vim |
336 | 18562946 | 1063 | 0.000031 | com.healthline |
337 | 18562860 | 1911 | 0.000016 | org.wikibooks |
338 | 18562650 | 1867 | 0.000016 | io.soup |
339 | 18560052 | 1704 | 0.000018 | nl.blogspot |
340 | 18555740 | 509 | 0.000051 | com.mashable |
341 | 18553730 | 264 | 0.000093 | com.typepad |
342 | 18553486 | 1071 | 0.000031 | com.adjust |
343 | 18551456 | 396 | 0.000065 | com.photobucket |
344 | 18544528 | 1741 | 0.000017 | org.bitcoin |
345 | 18542952 | 2227 | 0.000014 | tw.edu.ntu |
346 | 18541072 | 1100 | 0.000030 | com.ecwid |
347 | 18540798 | 1820 | 0.000017 | com.indianexpress |
348 | 18540288 | 1889 | 0.000016 | co.ello |
349 | 18538870 | 575 | 0.000046 | edu.berkeley |
350 | 18537372 | 2026 | 0.000015 | com.upi |
351 | 18537264 | 132 | 0.000232 | com.squarespace |
352 | 18535682 | 280 | 0.000089 | uk.org.ico |
353 | 18534924 | 1136 | 0.000028 | com.ssrn |
354 | 18534382 | 2152 | 0.000014 | com.viki |
355 | 18533818 | 1219 | 0.000025 | it.scoop |
356 | 18532606 | 270 | 0.000092 | com.surveymonkey |
357 | 18532016 | 1601 | 0.000019 | com.fastcodesign |
358 | 18530620 | 1782 | 0.000017 | org.unep |
359 | 18529588 | 1057 | 0.000031 | uk.parliament |
360 | 18527356 | 1966 | 0.000016 | org.haskell |
361 | 18527140 | 224 | 0.000107 | com.etsy |
362 | 18527064 | 1442 | 0.000021 | com.shutterfly |
363 | 18525388 | 1569 | 0.000019 | uk.org.tate |
364 | 18524530 | 2862 | 0.000011 | co.electrek |
365 | 18523134 | 2693 | 0.000011 | jp.doorblog |
366 | 18522838 | 156 | 0.000171 | com.issuu |
367 | 18519148 | 2018 | 0.000015 | com.dezeen |
368 | 18517910 | 2430 | 0.000013 | sh.now |
369 | 18517530 | 1157 | 0.000027 | com.tradedoubler |
370 | 18515028 | 1173 | 0.000027 | gov.weather |
371 | 18513616 | 1109 | 0.000029 | com.imageshack |
372 | 18512682 | 1693 | 0.000018 | com.channel4 |
373 | 18512134 | 1116 | 0.000029 | gov.dot |
374 | 18511018 | 2703 | 0.000011 | cn.edu.sdu |
375 | 18510554 | 1164 | 0.000027 | com.wikia |
376 | 18509244 | 282 | 0.000088 | com.huffingtonpost |
377 | 18509124 | 953 | 0.000034 | uk.co.pinterest |
378 | 18508676 | 924 | 0.000035 | com.arstechnica |
379 | 18507156 | 271 | 0.000091 | com.rawgit |
380 | 18505812 | 484 | 0.000053 | tv.twitch |
381 | 18505722 | 1917 | 0.000016 | th.co.google |
382 | 18503134 | 2390 | 0.000013 | uk.ac.nhm |
383 | 18502360 | 1764 | 0.000017 | com.netvibes |
384 | 18501156 | 1871 | 0.000016 | edu.emory |
385 | 18500964 | 918 | 0.000035 | in.amazon |
386 | 18500252 | 963 | 0.000034 | com.strikingly |
387 | 18499224 | 1773 | 0.000017 | net.bplaced |
388 | 18497786 | 3356 | 0.000010 | tw.edu.ntnu |
389 | 18495692 | 1811 | 0.000017 | edu.iu |
390 | 18494542 | 833 | 0.000039 | com.brightcove |
391 | 18491624 | 225 | 0.000107 | com.hubspot |
392 | 18491366 | 1470 | 0.000021 | com.wattpad |
393 | 18490914 | 1476 | 0.000021 | gov.michigan |
394 | 18489222 | 1916 | 0.000016 | nl.tudelft |
395 | 18488064 | 1436 | 0.000021 | org.c-span |
396 | 18487708 | 394 | 0.000065 | com.meetup |
397 | 18483412 | 2110 | 0.000014 | com.kaggle |
398 | 18481662 | 1299 | 0.000024 | edu.brookings |
399 | 18478490 | 86 | 0.000345 | net.jsfiddle |
400 | 18478420 | 2621 | 0.000012 | sh.surge |
401 | 18475530 | 2248 | 0.000014 | com.rsa |
402 | 18475220 | 1781 | 0.000017 | gov.ahrq |
403 | 18474128 | 825 | 0.000039 | org.mediawiki |
404 | 18473922 | 346 | 0.000072 | edu.yale |
405 | 18472664 | 826 | 0.000039 | com.intel |
406 | 18472288 | 1512 | 0.000020 | gov.faa |
407 | 18471926 | 1975 | 0.000015 | io.material |
408 | 18471732 | 1073 | 0.000031 | com.thenextweb |
409 | 18471706 | 1847 | 0.000016 | net.earthlink |
410 | 18469610 | 1908 | 0.000016 | jp.blog |
411 | 18469092 | 831 | 0.000039 | com.pexels |
412 | 18464760 | 1029 | 0.000032 | uk.gov.nationalarchives |
413 | 18460916 | 1973 | 0.000016 | com.smashwords |
414 | 18459088 | 939 | 0.000034 | org.ieee |
415 | 18457420 | 2185 | 0.000014 | com.smore |
416 | 18456724 | 345 | 0.000072 | com.livejournal |
417 | 18456360 | 3366 | 0.000010 | hk.edu.hkbu |
418 | 18453542 | 414 | 0.000063 | com.nypost |
419 | 18453464 | 1785 | 0.000017 | com.business-standard |
420 | 18453440 | 2907 | 0.000011 | com.yam |
421 | 18451248 | 1233 | 0.000025 | org.aarp |
422 | 18450408 | 1999 | 0.000015 | com.oprah |
423 | 18449942 | 2053 | 0.000015 | org.jpn |
424 | 18449880 | 1503 | 0.000020 | org.amnesty |
425 | 18449790 | 1651 | 0.000018 | com.avvo |
426 | 18449648 | 2540 | 0.000012 | com.cleantechnica |
427 | 18449504 | 522 | 0.000050 | edu.cornell |
428 | 18448124 | 2586 | 0.000012 | com.mysanantonio |
429 | 18447594 | 473 | 0.000054 | io.shields |
430 | 18447544 | 1444 | 0.000021 | org.hrw |
431 | 18444240 | 2487 | 0.000012 | org.neocities |
432 | 18442810 | 1943 | 0.000016 | com.care2 |
433 | 18440702 | 1714 | 0.000018 | com.snopes |
434 | 18440194 | 1148 | 0.000027 | com.gizmodo |
435 | 18440058 | 2474 | 0.000012 | com.googledrive |
436 | 18439272 | 2594 | 0.000012 | com.iflscience |
437 | 18437524 | 1594 | 0.000019 | org.pypi |
438 | 18437266 | 268 | 0.000092 | net.php |
439 | 18436902 | 1927 | 0.000016 | org.rsc |
440 | 18436288 | 1389 | 0.000022 | com.pbworks |
441 | 18435978 | 2735 | 0.000011 | com.itsnicethat |
442 | 18435362 | 2092 | 0.000015 | ae.thenational |
443 | 18435326 | 2461 | 0.000012 | com.hsbc |
444 | 18434642 | 338 | 0.000074 | com.hp |
445 | 18432636 | 1354 | 0.000023 | uk.co.standard |
446 | 18431764 | 1807 | 0.000017 | com.instapaper |
447 | 18431596 | 465 | 0.000055 | io.codepen |
448 | 18431390 | 553 | 0.000047 | com.buzzfeed |
449 | 18431020 | 1910 | 0.000016 | com.secondlife |
450 | 18430258 | 2425 | 0.000013 | jp.go.ndl |
451 | 18429756 | 2106 | 0.000014 | io.gitlab |
452 | 18428432 | 373 | 0.000069 | int.who |
453 | 18427128 | 2300 | 0.000013 | org.lds |
454 | 18426976 | 2471 | 0.000012 | uk.mod |
455 | 18426954 | 1711 | 0.000018 | google.ai |
456 | 18426290 | 96 | 0.000330 | de.google |
457 | 18423898 | 1427 | 0.000021 | com.thehindu |
458 | 18423724 | 1749 | 0.000017 | com.curbed |
459 | 18422902 | 1638 | 0.000019 | no.google |
460 | 18421738 | 340 | 0.000074 | com.cnbc |
461 | 18420686 | 1061 | 0.000031 | com.thedrum |
462 | 18419780 | 165 | 0.000160 | com.ebay |
463 | 18418790 | 627 | 0.000043 | com.zdnet |
464 | 18418454 | 2330 | 0.000013 | pl.cba |
465 | 18416392 | 2441 | 0.000013 | com.minds |
466 | 18413488 | 201 | 0.000125 | com.salesforce |
467 | 18413252 | 1574 | 0.000019 | com.moonfruit |
468 | 18412358 | 1156 | 0.000027 | com.mixpanel |
469 | 18411670 | 2818 | 0.000011 | tl.page |
470 | 18409196 | 1982 | 0.000015 | com.name |
471 | 18409080 | 2282 | 0.000013 | jp.hateblo |
472 | 18407830 | 2507 | 0.000012 | org.tvtropes |
473 | 18407318 | 2580 | 0.000012 | jp.hatenadiary |
474 | 18406348 | 2510 | 0.000012 | de.dw |
475 | 18405634 | 1854 | 0.000016 | com.googlegroups |
476 | 18405508 | 1876 | 0.000016 | mx.com.google |
477 | 18405134 | 2198 | 0.000014 | org.aiga |
478 | 18403404 | 2883 | 0.000011 | uk.co.birminghammail |
479 | 18403340 | 367 | 0.000069 | com.booking |
480 | 18401602 | 2314 | 0.000013 | vn.com.google |
481 | 18401550 | 1729 | 0.000018 | gov.pa |
482 | 18399972 | 1666 | 0.000018 | org.hrc |
483 | 18399618 | 882 | 0.000036 | gov.nist |
484 | 18398472 | 2742 | 0.000011 | com.exxonmobil |
485 | 18397376 | 1841 | 0.000016 | ar.com.google |
486 | 18396356 | 989 | 0.000033 | net.clickbank |
487 | 18395660 | 976 | 0.000033 | com.matterport |
488 | 18392402 | 2429 | 0.000013 | ua.at |
489 | 18390522 | 2011 | 0.000015 | uk.ac.leeds |
490 | 18387444 | 309 | 0.000081 | gov.cdc |
491 | 18386528 | 1558 | 0.000019 | int.unfccc |
492 | 18386408 | 2342 | 0.000013 | com.eklablog |
493 | 18385700 | 459 | 0.000056 | com.gmail |
494 | 18385598 | 401 | 0.000064 | org.npr |
495 | 18384832 | 1672 | 0.000018 | gov.maryland |
496 | 18384390 | 357 | 0.000070 | com.office |
497 | 18383950 | 2240 | 0.000014 | se.liu |
498 | 18383810 | 2067 | 0.000015 | com.discovermagazine |
499 | 18383400 | 2204 | 0.000014 | com.ipage |
500 | 18381626 | 1110 | 0.000029 | com.stackexchange |
501 | 18381594 | 2418 | 0.000013 | it.justpaste |
502 | 18380974 | 449 | 0.000058 | fr.free |
503 | 18380682 | 1718 | 0.000018 | sg.com.google |
504 | 18379672 | 1060 | 0.000031 | com.engadget |
505 | 18378238 | 2421 | 0.000013 | my.com.thestar |
506 | 18377282 | 1273 | 0.000024 | dk.google |
507 | 18377136 | 2210 | 0.000014 | org.biorxiv |
508 | 18377062 | 1861 | 0.000016 | com.weheartit |
509 | 18374194 | 1598 | 0.000019 | uk.gov.tfl |
510 | 18371274 | 508 | 0.000051 | gov.whitehouse |
511 | 18369330 | 1723 | 0.000018 | ly.snip |
512 | 18369006 | 1809 | 0.000017 | com.yourstory |
513 | 18366356 | 3154 | 0.000011 | com.bonanza |
514 | 18365650 | 2833 | 0.000011 | com.scienceblogs |
515 | 18365484 | 1431 | 0.000021 | com.ebayimg |
516 | 18365436 | 1774 | 0.000017 | gov.ky |
517 | 18363500 | 858 | 0.000038 | com.venturebeat |
518 | 18362924 | 1160 | 0.000027 | se.google |
519 | 18362454 | 1350 | 0.000023 | com.firebaseapp |
520 | 18362026 | 178 | 0.000147 | com.zendesk |
521 | 18360150 | 2004 | 0.000015 | uk.gov.metoffice |
522 | 18359990 | 928 | 0.000035 | com.windowsphone |
523 | 18359750 | 2336 | 0.000013 | com.rediff |
524 | 18358388 | 518 | 0.000051 | com.alibaba |
525 | 18355256 | 2225 | 0.000014 | com.blogfa |
526 | 18355232 | 415 | 0.000063 | com.fastcompany |
527 | 18353212 | 1426 | 0.000021 | com.surveygizmo |
528 | 18352352 | 2021 | 0.000015 | au.com.telstra |
529 | 18351454 | 1134 | 0.000028 | org.sphinx-doc |
530 | 18350502 | 2048 | 0.000015 | ro.google |
531 | 18350126 | 1904 | 0.000016 | org.tigris |
532 | 18349524 | 2835 | 0.000011 | be.lesoir |
533 | 18349430 | 2698 | 0.000011 | cz.centrum |
534 | 18349372 | 2047 | 0.000015 | link.page |
535 | 18349260 | 479 | 0.000054 | org.nodejs |
536 | 18349028 | 1960 | 0.000016 | com.marketwire |
537 | 18347672 | 2242 | 0.000014 | com.mystrikingly |
538 | 18347018 | 2260 | 0.000013 | ch.unige |
539 | 18346850 | 2753 | 0.000011 | cat.uab |
540 | 18346818 | 2889 | 0.000011 | com.zynga |
541 | 18345164 | 1510 | 0.000020 | us.mn.state |
542 | 18341622 | 2275 | 0.000013 | com.articulate |
543 | 18340012 | 991 | 0.000033 | edu.psu |
544 | 18339422 | 2141 | 0.000014 | com.thecvf |
545 | 18339020 | 2150 | 0.000014 | es.csic |
546 | 18338922 | 2880 | 0.000011 | co.carrd |
547 | 18337380 | 1611 | 0.000019 | gov.mo |
548 | 18337360 | 2297 | 0.000013 | com.newatlas |
549 | 18335690 | 3908 | 0.000009 | jp.rdy |
550 | 18334630 | 1990 | 0.000015 | org.iea |
551 | 18333598 | 2565 | 0.000012 | com.db |
552 | 18332716 | 2310 | 0.000013 | com.webstarts |
553 | 18332584 | 2488 | 0.000012 | jp.hatenablog |
554 | 18331976 | 2331 | 0.000013 | ly.rebrand |
555 | 18331344 | 370 | 0.000069 | com.mapbox |
556 | 18331228 | 485 | 0.000053 | com.livechatinc |
557 | 18325352 | 1998 | 0.000015 | org.mozillazine |
558 | 18324944 | 2271 | 0.000013 | de.uni-freiburg |
559 | 18324472 | 1372 | 0.000023 | com.tinypic |
560 | 18324252 | 883 | 0.000036 | com.steampowered |
561 | 18323842 | 2072 | 0.000015 | uk.ac.york |
562 | 18322186 | 1097 | 0.000030 | com.thinkwithgoogle |
563 | 18320582 | 2589 | 0.000012 | ru.msu |
564 | 18320156 | 2458 | 0.000012 | org.kotlinlang |
565 | 18319540 | 1629 | 0.000019 | gov.oregon |
566 | 18318914 | 3507 | 0.000010 | com.ingress |
567 | 18318120 | 1806 | 0.000017 | gov.wi |
568 | 18318056 | 541 | 0.000049 | com.aol |
569 | 18318040 | 1969 | 0.000016 | gr.google |
570 | 18317824 | 2741 | 0.000011 | lv.draugiem |
571 | 18316720 | 2305 | 0.000013 | org.iucnredlist |
572 | 18315822 | 2035 | 0.000015 | com.broadwayworld |
573 | 18314076 | 134 | 0.000221 | com.youtube-nocookie |
574 | 18313954 | 1511 | 0.000020 | net.openid |
575 | 18313704 | 168 | 0.000158 | com.tripadvisor |
576 | 18313516 | 435 | 0.000059 | com.dailymotion |
577 | 18313398 | 1548 | 0.000019 | net.leadpages |
578 | 18313360 | 3389 | 0.000010 | com.brother |
579 | 18313186 | 2755 | 0.000011 | com.webcindario |
580 | 18313116 | 3161 | 0.000011 | es.usal |
581 | 18312962 | 2338 | 0.000013 | bg.google |
582 | 18312728 | 907 | 0.000036 | com.xiti |
583 | 18312338 | 2273 | 0.000013 | us.oh.state |
584 | 18310760 | 720 | 0.000041 | fm.last |
585 | 18310700 | 1662 | 0.000018 | net.ucoz |
586 | 18306936 | 353 | 0.000071 | org.acm |
587 | 18304074 | 3931 | 0.000009 | com.worldlingo |
588 | 18303254 | 3379 | 0.000010 | com.embarcadero |
589 | 18303110 | 1789 | 0.000017 | com.eiseverywhere |
590 | 18302838 | 2230 | 0.000014 | org.wri |
591 | 18302746 | 395 | 0.000065 | com.pubmatic |
592 | 18302516 | 470 | 0.000054 | com.goodreads |
593 | 18300826 | 2212 | 0.000014 | com.thehindubusinessline |
594 | 18300196 | 2289 | 0.000013 | com.mihanblog |
595 | 18299988 | 2037 | 0.000015 | com.intensedebate |
596 | 18298184 | 3230 | 0.000011 | com.hellomagazine |
597 | 18297298 | 3474 | 0.000010 | net.hypermart |
598 | 18294990 | 235 | 0.000103 | uk.co.amazon |
599 | 18294428 | 2711 | 0.000011 | nf.co |
600 | 18294164 | 73 | 0.000401 | me.fb |
601 | 18294088 | 542 | 0.000049 | com.entrepreneur |
602 | 18293062 | 2543 | 0.000012 | com.futurelearn |
603 | 18292644 | 2389 | 0.000013 | com.iconarchive |
604 | 18292306 | 1275 | 0.000024 | com.cognitoforms |
605 | 18292138 | 1730 | 0.000018 | org.khanacademy |
606 | 18291072 | 2041 | 0.000015 | com.financialpost |
607 | 18291000 | 1744 | 0.000017 | us.pa.state |
608 | 18288698 | 2653 | 0.000012 | com.fatcow |
609 | 18288564 | 372 | 0.000069 | com.staticflickr |
610 | 18288272 | 1851 | 0.000016 | io.bower |
611 | 18287878 | 3657 | 0.000010 | nz.govt.tepapa |
612 | 18285000 | 2199 | 0.000014 | org.prlog |
613 | 18284980 | 2743 | 0.000011 | ca.shaw |
614 | 18284736 | 2104 | 0.000014 | com.bravesites |
615 | 18283030 | 2675 | 0.000012 | de.uni-erlangen |
616 | 18282568 | 2446 | 0.000012 | org.lacity |
617 | 18282024 | 1534 | 0.000020 | fi.google |
618 | 18282018 | 2267 | 0.000013 | de.uni-koeln |
619 | 18280510 | 2464 | 0.000012 | uk.co.spectator |
620 | 18279326 | 334 | 0.000076 | com.typeform |
621 | 18279324 | 2725 | 0.000011 | is.good |
622 | 18279290 | 3399 | 0.000010 | com.114la |
623 | 18278842 | 3277 | 0.000010 | net.freeforums |
624 | 18278384 | 920 | 0.000035 | com.zoho |
625 | 18273898 | 2465 | 0.000012 | uk.ac.jisc |
626 | 18273404 | 2489 | 0.000012 | com.mnn |
627 | 18273352 | 2393 | 0.000013 | ca.dal |
628 | 18272740 | 114 | 0.000290 | com.statcounter |
629 | 18272722 | 480 | 0.000054 | com.netflix |
630 | 18272318 | 1567 | 0.000019 | com.flashtalking |
631 | 18272212 | 1903 | 0.000016 | com.prweek |
632 | 18270806 | 3115 | 0.000011 | site.negocio |
633 | 18270794 | 2080 | 0.000015 | org.lung |
634 | 18270506 | 2727 | 0.000011 | com.mouser |
635 | 18270340 | 2569 | 0.000012 | uk.co.profilebusiness |
636 | 18269840 | 3616 | 0.000010 | uk.gov.number10 |
637 | 18268146 | 3806 | 0.000009 | net.dead |
638 | 18267734 | 3375 | 0.000010 | jp.ac.kobe-u |
639 | 18267628 | 1925 | 0.000016 | uk.org.nice |
640 | 18267512 | 88 | 0.000343 | com.oculus |
641 | 18267198 | 3196 | 0.000011 | build.bazel |
642 | 18266546 | 1878 | 0.000016 | org.gentoo |
643 | 18266166 | 2181 | 0.000014 | ie.thejournal |
644 | 18266148 | 109 | 0.000310 | com.sharethis |
645 | 18265976 | 1906 | 0.000016 | org.gnupg |
646 | 18264728 | 148 | 0.000186 | ru.mail |
647 | 18263376 | 1860 | 0.000016 | com.doodlekit |
648 | 18262238 | 1948 | 0.000016 | com.crashlytics |
649 | 18262156 | 1831 | 0.000017 | org.alz |
650 | 18261954 | 2549 | 0.000012 | us.ms.state |
651 | 18261162 | 2459 | 0.000012 | com.instructure |
652 | 18260540 | 820 | 0.000040 | com.cbsnews |
653 | 18259844 | 2877 | 0.000011 | ee.ut |
654 | 18259826 | 1211 | 0.000026 | com.msdn |
655 | 18259610 | 777 | 0.000040 | com.samsung |
656 | 18257004 | 1338 | 0.000023 | com.emailmeform |
657 | 18254934 | 549 | 0.000048 | edu.cmu |
658 | 18254822 | 2496 | 0.000012 | uk.co.osoo |
659 | 18254762 | 83 | 0.000354 | com.livestream |
660 | 18254656 | 2226 | 0.000014 | com.atavist |
661 | 18252876 | 2208 | 0.000014 | fr.archives-ouvertes |
662 | 18252282 | 2792 | 0.000011 | com.cnsnews |
663 | 18252018 | 2348 | 0.000013 | io.pantheon |
664 | 18251148 | 898 | 0.000036 | com.createjs |
665 | 18251026 | 1755 | 0.000017 | us.fl.state |
666 | 18250730 | 2321 | 0.000013 | com.rabbitmq |
667 | 18250628 | 2712 | 0.000011 | uk.co.newmedianow |
668 | 18248576 | 1422 | 0.000022 | com.123formbuilder |
669 | 18247032 | 2086 | 0.000015 | gov.nh |
670 | 18243504 | 2233 | 0.000014 | org.crossref |
671 | 18242314 | 2229 | 0.000014 | us.nm.state |
672 | 18242254 | 296 | 0.000084 | com.scribd |
673 | 18241366 | 3254 | 0.000010 | ca.qc.montreal |
674 | 18240908 | 3285 | 0.000010 | uk.co.lrb |
675 | 18240828 | 135 | 0.000215 | com.youku |
676 | 18239750 | 517 | 0.000051 | com.slack |
677 | 18239658 | 2677 | 0.000012 | com.hatenadiary |
678 | 18239656 | 2292 | 0.000013 | com.itsmyurls |
679 | 18237636 | 2671 | 0.000012 | uk.org.oxonaa |
680 | 18236902 | 246 | 0.000099 | com.constantcontact |
681 | 18236862 | 3348 | 0.000010 | com.outlookindia |
682 | 18235854 | 3893 | 0.000009 | in.ac.nptel |
683 | 18235540 | 2681 | 0.000012 | uk.org.oxfam |
684 | 18235236 | 2344 | 0.000013 | com.yext |
685 | 18233812 | 256 | 0.000094 | com.getbootstrap |
686 | 18233324 | 2107 | 0.000014 | org.jenkins-ci |
687 | 18230584 | 2055 | 0.000015 | com.broadcastingcable |
688 | 18230478 | 1686 | 0.000018 | uk.gov.direct |
689 | 18230416 | 2663 | 0.000012 | com.wmtransfer |
690 | 18230374 | 1977 | 0.000015 | gov.mt |
691 | 18230164 | 2821 | 0.000011 | uk.ac.stir |
692 | 18228540 | 1052 | 0.000031 | com.marketwatch |
693 | 18227744 | 2266 | 0.000013 | com.tmcnet |
694 | 18227440 | 3136 | 0.000011 | uk.co.hsbc |
695 | 18227086 | 1798 | 0.000017 | org.nfpa |
696 | 18226792 | 2939 | 0.000011 | com.batchgeo |
697 | 18225844 | 3275 | 0.000010 | com.weightwatchers |
698 | 18225636 | 234 | 0.000103 | to.amzn |
699 | 18224632 | 3574 | 0.000010 | com.orgfree |
700 | 18223778 | 1355 | 0.000023 | org.whatbrowser |
701 | 18221814 | 2843 | 0.000011 | com.adn |
702 | 18221276 | 1190 | 0.000026 | org.weforum |
703 | 18220506 | 481 | 0.000054 | org.hbr |
704 | 18219880 | 2820 | 0.000011 | au.edu.deakin |
705 | 18219734 | 1455 | 0.000021 | org.js |
706 | 18219118 | 2445 | 0.000013 | in.ernet |
707 | 18217962 | 2854 | 0.000011 | hu.elte |
708 | 18217516 | 3025 | 0.000011 | pl.edu.uw |
709 | 18217274 | 2367 | 0.000013 | uk.org.rspb |
710 | 18216528 | 2220 | 0.000014 | com.healthgrades |
711 | 18216264 | 2779 | 0.000011 | org.carbonbrief |
712 | 18214214 | 366 | 0.000069 | com.prnewswire |
713 | 18213956 | 2088 | 0.000015 | com.tapatalk |
714 | 18213180 | 2431 | 0.000013 | org.grist |
715 | 18212750 | 3423 | 0.000010 | id.co.kaskus |
716 | 18210638 | 456 | 0.000057 | com.oreilly |
717 | 18210106 | 3587 | 0.000010 | com.skepticalscience |
718 | 18209950 | 539 | 0.000049 | gov.sec |
719 | 18209922 | 3081 | 0.000011 | com.deccanherald |
720 | 18209668 | 1905 | 0.000016 | tl.we |
721 | 18208770 | 2311 | 0.000013 | us.ma.state |
722 | 18206860 | 1101 | 0.000030 | uk.ac.cam |
723 | 18205994 | 3630 | 0.000010 | ua.meta |
724 | 18205738 | 3526 | 0.000010 | app.web |
725 | 18204462 | 2398 | 0.000013 | uk.co.zoopla |
726 | 18201966 | 3210 | 0.000011 | org.oceanconservancy |
727 | 18199630 | 3421 | 0.000010 | org.atsjournals |
728 | 18198962 | 3532 | 0.000010 | ru.my1 |
729 | 18198444 | 3162 | 0.000011 | com.mozello |
730 | 18195600 | 1562 | 0.000019 | com.pastebin |
731 | 18194580 | 2867 | 0.000011 | de.freenet |
732 | 18193414 | 1137 | 0.000028 | edu.ucla |
733 | 18193100 | 3052 | 0.000011 | com.telegraphindia |
734 | 18193026 | 2857 | 0.000011 | com.chagasi |
735 | 18192758 | 937 | 0.000034 | br.com.uol |
736 | 18188994 | 2630 | 0.000012 | com.atwebpages |
737 | 18188626 | 3036 | 0.000011 | com.remind |
738 | 18187922 | 1132 | 0.000028 | com.redhat |
739 | 18187748 | 608 | 0.000044 | com.wikihow |
740 | 18187658 | 3377 | 0.000010 | edu.utep |
741 | 18187264 | 3455 | 0.000010 | ru.nnov |
742 | 18186834 | 1881 | 0.000016 | uk.gov.defra |
743 | 18186568 | 2359 | 0.000013 | net.portfoliobox |
744 | 18185624 | 2610 | 0.000012 | com.blogsky |
745 | 18185434 | 3856 | 0.000009 | uk.co.mailonsunday |
746 | 18185432 | 2723 | 0.000011 | jp.xxxxxxxx |
747 | 18184122 | 1425 | 0.000021 | edu.ucsd |
748 | 18183962 | 1449 | 0.000021 | com.digitaltrends |
749 | 18183738 | 196 | 0.000130 | jp.ne.hatena |
750 | 18182464 | 2563 | 0.000012 | uk.co.inews |
751 | 18181728 | 2313 | 0.000013 | gov.la |
752 | 18181656 | 1266 | 0.000024 | ly.ow |
753 | 18180360 | 3441 | 0.000010 | gr.sch |
754 | 18179802 | 3055 | 0.000011 | com.sc |
755 | 18178628 | 3373 | 0.000010 | com.cummins |
756 | 18177566 | 2363 | 0.000013 | com.activerain |
757 | 18176026 | 3801 | 0.000009 | com.kazeo |
758 | 18176002 | 2901 | 0.000011 | net.onlinewebshop |
759 | 18175422 | 3689 | 0.000010 | com.galvanize |
760 | 18174902 | 3473 | 0.000010 | ru.pr-cy |
761 | 18174826 | 503 | 0.000052 | com.dmca |
762 | 18173528 | 3328 | 0.000010 | com.kaywa |
763 | 18173348 | 821 | 0.000040 | com.psychologytoday |
764 | 18172118 | 2853 | 0.000011 | uk.co.heatall |
765 | 18171416 | 84 | 0.000350 | me.ogp |
766 | 18168128 | 2601 | 0.000012 | gov.ks |
767 | 18167782 | 1516 | 0.000020 | ca.blogspot |
768 | 18167558 | 2170 | 0.000014 | com.cityam |
769 | 18167284 | 3604 | 0.000010 | gov.cabq |
770 | 18166436 | 1813 | 0.000017 | org.reactjs |
771 | 18166052 | 3283 | 0.000010 | org.escardio |
772 | 18165734 | 1064 | 0.000031 | com.foxnews |
773 | 18165680 | 1897 | 0.000016 | com.fifa |
774 | 18164860 | 204 | 0.000117 | com.naver |
775 | 18164404 | 3761 | 0.000009 | com.carscoops |
776 | 18162680 | 2928 | 0.000011 | com.ecowatch |
777 | 18162390 | 1507 | 0.000020 | com.literatumonline |
778 | 18161998 | 535 | 0.000049 | net.2mdn |
779 | 18161800 | 476 | 0.000054 | com.force |
780 | 18160578 | 159 | 0.000167 | gov.privacyshield |
781 | 18160270 | 1896 | 0.000016 | com.pcworld |
782 | 18160192 | 2986 | 0.000011 | com.theyworkforyou |
783 | 18159730 | 81 | 0.000365 | com.messenger |
784 | 18159700 | 3939 | 0.000009 | com.anghami |
785 | 18159426 | 424 | 0.000061 | edu.nyu |
786 | 18157990 | 1294 | 0.000024 | com.indiegogo |
787 | 18157828 | 1869 | 0.000016 | kr.or.kisa |
788 | 18157816 | 364 | 0.000070 | com.discordapp |
789 | 18157014 | 3186 | 0.000011 | uk.org.38degrees |
790 | 18156850 | 3628 | 0.000010 | com.insideevs |
791 | 18155496 | 1488 | 0.000020 | com.placeholder |
792 | 18155072 | 3250 | 0.000010 | google.design |
793 | 18155044 | 3764 | 0.000009 | gle.goo |
794 | 18154462 | 454 | 0.000057 | com.walmart |
795 | 18153360 | 428 | 0.000060 | com.flipboard |
796 | 18152044 | 2902 | 0.000011 | pl.lublin |
797 | 18151952 | 422 | 0.000062 | com.wufoo |
798 | 18151198 | 1123 | 0.000029 | com.shutterstock |
799 | 18150684 | 2537 | 0.000012 | org.iihs |
800 | 18149446 | 2788 | 0.000011 | in.businessworld |
801 | 18148636 | 981 | 0.000033 | com.pinimg |
802 | 18147760 | 2407 | 0.000013 | jp.e-shops |
803 | 18147734 | 2250 | 0.000014 | com.codecademy |
804 | 18146340 | 2642 | 0.000012 | com.zx2c4 |
805 | 18146328 | 129 | 0.000243 | info.aboutads |
806 | 18145944 | 2138 | 0.000014 | ca.ubc |
807 | 18145538 | 2874 | 0.000011 | com.bnef |
808 | 18144354 | 3240 | 0.000011 | uk.ac.rcplondon |
809 | 18144254 | 3718 | 0.000009 | com.wsoctv |
810 | 18143902 | 3950 | 0.000009 | com.monbiot |
811 | 18143342 | 3463 | 0.000010 | com.droppages |
812 | 18143148 | 2366 | 0.000013 | gov.arts |
813 | 18142454 | 2644 | 0.000012 | us.wi.state |
814 | 18142046 | 3477 | 0.000010 | org.usatf |
815 | 18140878 | 1624 | 0.000019 | com.nvidia |
816 | 18138866 | 3636 | 0.000010 | com.elmercurio |
817 | 18138838 | 1538 | 0.000020 | com.businessweek |
818 | 18138462 | 2176 | 0.000014 | com.tutsplus |
819 | 18138382 | 554 | 0.000047 | com.atlassian |
820 | 18137356 | 1184 | 0.000026 | com.searchengineland |
821 | 18137278 | 3594 | 0.000010 | com.glu |
822 | 18137124 | 3645 | 0.000010 | es.consumer |
823 | 18135974 | 240 | 0.000102 | cn.com.sina |
824 | 18135596 | 3948 | 0.000009 | com.allmyfaves |
825 | 18135342 | 3446 | 0.000010 | com.businessgreen |
826 | 18133642 | 350 | 0.000072 | com.163 |
827 | 18133268 | 3292 | 0.000010 | org.jython |
828 | 18133230 | 471 | 0.000054 | com.smugmug |
829 | 18132816 | 3864 | 0.000009 | org.thechicagocouncil |
830 | 18132126 | 3576 | 0.000010 | gov.azdot |
831 | 18130470 | 1176 | 0.000027 | com.ycombinator |
832 | 18129838 | 3339 | 0.000010 | org.transportenvironment |
833 | 18128538 | 2993 | 0.000011 | gov.ferc |
834 | 18127910 | 936 | 0.000034 | com.aliexpress |
835 | 18126154 | 356 | 0.000070 | com.wiley |
836 | 18125790 | 696 | 0.000042 | com.moz |
837 | 18124996 | 2756 | 0.000011 | uk.gov.environment-agency |
838 | 18124886 | 3012 | 0.000011 | org.zsl |
839 | 18124136 | 3704 | 0.000009 | org.ssireview |
840 | 18123520 | 2378 | 0.000013 | uk.gov.scotland |
841 | 18122978 | 1595 | 0.000019 | tv.ustream |
842 | 18122522 | 3096 | 0.000011 | org.dailystrength |
843 | 18122038 | 598 | 0.000045 | com.caniuse |
844 | 18120996 | 2485 | 0.000012 | net.privacypolicytemplate |
845 | 18120866 | 768 | 0.000040 | gov.noaa |
846 | 18120818 | 1573 | 0.000019 | jp.makeshop |
847 | 18120518 | 3040 | 0.000011 | org.rspo |
848 | 18119946 | 2303 | 0.000013 | com.seetickets |
849 | 18119454 | 2183 | 0.000014 | com.ign |
850 | 18118896 | 404 | 0.000064 | mp.mailchi |
851 | 18118000 | 311 | 0.000081 | com.digg |
852 | 18118000 | 2855 | 0.000011 | gov.txdot |
853 | 18117366 | 3412 | 0.000010 | uk.ac.ceh |
854 | 18117164 | 1479 | 0.000021 | com.crunchbase |
855 | 18117074 | 1127 | 0.000029 | com.highcharts |
856 | 18115870 | 2645 | 0.000012 | com.9to5mac |
857 | 18114648 | 1090 | 0.000030 | com.withgoogle |
858 | 18114314 | 889 | 0.000036 | com.webs |
859 | 18114072 | 2481 | 0.000012 | uk.co.streetmap |
860 | 18112508 | 3865 | 0.000009 | com.pushwoosh |
861 | 18111708 | 3204 | 0.000011 | ca.uwaterloo |
862 | 18111130 | 817 | 0.000040 | com.shinystat |
863 | 18111078 | 305 | 0.000082 | fr.google |
864 | 18111050 | 3467 | 0.000010 | com.baomoi |
865 | 18110974 | 3916 | 0.000009 | uk.ac.tyndall |
866 | 18110396 | 1766 | 0.000017 | com.webmasterplan |
867 | 18110180 | 3686 | 0.000010 | dk.bloggersdelight |
868 | 18109908 | 3401 | 0.000010 | uk.gov.hm-treasury |
869 | 18109262 | 1793 | 0.000017 | uk.org.cqc |
870 | 18108948 | 1248 | 0.000025 | com.smashingmagazine |
871 | 18108138 | 331 | 0.000076 | com.automattic |
872 | 18107572 | 1530 | 0.000020 | com.ning |
873 | 18106984 | 2829 | 0.000011 | com.linkwithin |
874 | 18106522 | 3002 | 0.000011 | uk.org.greenpeace |
875 | 18103768 | 956 | 0.000034 | com.libsyn |
876 | 18103538 | 1239 | 0.000025 | com.sap |
877 | 18102956 | 2091 | 0.000015 | edu.uci |
878 | 18102564 | 628 | 0.000043 | com.patreon |
879 | 18102020 | 3503 | 0.000010 | com.climatechangenews |
880 | 18101758 | 409 | 0.000063 | com.xinhuanet |
881 | 18101336 | 3464 | 0.000010 | com.kapook |
882 | 18100618 | 885 | 0.000036 | com.newyorker |
883 | 18100474 | 3640 | 0.000010 | com.spruz |
884 | 18100196 | 478 | 0.000054 | com.inc |
885 | 18100062 | 2676 | 0.000012 | jp.aikotoba |
886 | 18099268 | 914 | 0.000035 | org.eff |
887 | 18098794 | 3662 | 0.000010 | com.platts |
888 | 18098556 | 3535 | 0.000010 | org.c2es |
889 | 18098550 | 2747 | 0.000011 | com.mykaratestore |
890 | 18096786 | 1770 | 0.000017 | com.ikea |
891 | 18096394 | 1423 | 0.000022 | com.billboard |
892 | 18095092 | 1070 | 0.000031 | com.hootsuite |
893 | 18094948 | 3525 | 0.000010 | com.jkp |
894 | 18093496 | 2824 | 0.000011 | org.mcsuk |
895 | 18092622 | 1254 | 0.000025 | es.agpd |
896 | 18092438 | 3349 | 0.000010 | net.edie |
897 | 18092358 | 533 | 0.000050 | com.ea |
898 | 18092112 | 376 | 0.000068 | org.opensource |
899 | 18091568 | 2903 | 0.000011 | ru.drom |
900 | 18090162 | 2639 | 0.000012 | com.yelloyello |
901 | 18089968 | 2544 | 0.000012 | uk.co.intersol |
902 | 18089740 | 139 | 0.000202 | com.alicdn |
903 | 18089422 | 4051 | 0.000009 | com.mforos |
904 | 18086990 | 1473 | 0.000021 | com.fiverr |
905 | 18086352 | 934 | 0.000035 | com.foursquare |
906 | 18085918 | 1737 | 0.000017 | org.freecsstemplates |
907 | 18084810 | 4142 | 0.000009 | uk.org.indymedia |
908 | 18084674 | 2049 | 0.000015 | uk.gov.education |
909 | 18083694 | 3843 | 0.000009 | com.thinkbroadband |
910 | 18082164 | 231 | 0.000104 | jp.co.amazon |
911 | 18080114 | 3868 | 0.000009 | org.sciencenewsforstudents |
912 | 18080034 | 221 | 0.000108 | org.drupal |
913 | 18079726 | 1096 | 0.000030 | com.variety |
914 | 18078666 | 290 | 0.000086 | com.stumbleupon |
915 | 18078038 | 3269 | 0.000010 | net.scienceontheweb |
916 | 18077582 | 1756 | 0.000017 | com.nba |
917 | 18077452 | 2561 | 0.000012 | org.webring |
918 | 18076502 | 1033 | 0.000031 | com.visualstudio |
919 | 18075958 | 4005 | 0.000009 | io.raindrop |
920 | 18074544 | 2744 | 0.000011 | jp.zouri |
921 | 18073766 | 3904 | 0.000009 | org.corporateeurope |
922 | 18072470 | 1402 | 0.000022 | com.storify |
923 | 18071436 | 375 | 0.000069 | gov.ftc |
924 | 18071372 | 1603 | 0.000019 | net.with2 |
925 | 18070926 | 1448 | 0.000021 | com.nike |
926 | 18070222 | 4048 | 0.000009 | io.dataquest |
927 | 18070066 | 1255 | 0.000025 | org.unicef |
928 | 18069672 | 3567 | 0.000010 | bnpparibas.group |
929 | 18069172 | 3685 | 0.000010 | com.thestatesman |
930 | 18068866 | 3427 | 0.000010 | uk.org.rya |
931 | 18067508 | 383 | 0.000068 | com.airbnb |
932 | 18067204 | 1635 | 0.000019 | de.zeit |
933 | 18067190 | 2555 | 0.000012 | com.hackernoon |
934 | 18066274 | 3451 | 0.000010 | ca.pe.gov |
935 | 18065266 | 4031 | 0.000009 | com.raamdev |
936 | 18064388 | 2438 | 0.000013 | io.postach |
937 | 18064126 | 1487 | 0.000020 | edu.purdue |
938 | 18063508 | 407 | 0.000063 | com.tripod |
939 | 18063228 | 1228 | 0.000025 | gov.fbi |
940 | 18063154 | 1369 | 0.000023 | com.lifehacker |
941 | 18063130 | 1069 | 0.000031 | com.uk |
942 | 18061878 | 3432 | 0.000010 | in.gov.mhrd |
943 | 18061130 | 3527 | 0.000010 | org.gmplib |
944 | 18060100 | 3879 | 0.000009 | com.gitimmersion |
945 | 18059578 | 2807 | 0.000011 | jp.at-ninja |
946 | 18059004 | 3010 | 0.000011 | com.shichihuku |
947 | 18058826 | 3629 | 0.000010 | com.h2database |
948 | 18057736 | 3482 | 0.000010 | uk.org.rcn |
949 | 18057640 | 3737 | 0.000009 | com.writetothem |
950 | 18056592 | 1366 | 0.000023 | com.parsiblog |
951 | 18056586 | 984 | 0.000033 | com.dropboxusercontent |
952 | 18055950 | 1306 | 0.000024 | com.prweb |
953 | 18055628 | 3695 | 0.000009 | com.websiteseguro |
954 | 18055104 | 1118 | 0.000029 | com.vox |
955 | 18054272 | 1397 | 0.000022 | us.imageshack |
956 | 18053964 | 2032 | 0.000015 | com.howstuffworks |
957 | 18052920 | 1531 | 0.000020 | com.yoast |
958 | 18052280 | 1298 | 0.000024 | com.pcmag |
959 | 18051398 | 3008 | 0.000011 | uk.org.woodlandtrust |
960 | 18050936 | 3523 | 0.000010 | gle.posts |
961 | 18050800 | 3838 | 0.000009 | org.priceofoil |
962 | 18049580 | 1614 | 0.000019 | com.ccbill |
963 | 18049066 | 3750 | 0.000009 | com.fourfour |
964 | 18047214 | 945 | 0.000034 | gov.census |
965 | 18046486 | 1328 | 0.000023 | edu.wisc |
966 | 18045876 | 151 | 0.000179 | jp.co.google |
967 | 18045710 | 1220 | 0.000025 | com.blackberry |
968 | 18045414 | 1103 | 0.000030 | edu.umich |
969 | 18045390 | 1952 | 0.000016 | com.w3layouts |
970 | 18043894 | 146 | 0.000190 | me.line |
971 | 18043816 | 1593 | 0.000019 | edu.usc |
972 | 18042356 | 2842 | 0.000011 | com.zatunen |
973 | 18042240 | 500 | 0.000052 | com.nasdaq |
974 | 18042130 | 567 | 0.000046 | net.daum |
975 | 18041570 | 3118 | 0.000011 | vn.tuoitre |
976 | 18040556 | 2573 | 0.000012 | com.hisupplier |
977 | 18039444 | 2023 | 0.000015 | com.nfl |
978 | 18039370 | 927 | 0.000035 | com.ggpht |
979 | 18039324 | 1549 | 0.000019 | com.vmware |
980 | 18039020 | 3827 | 0.000009 | com.realtytimes |
981 | 18038362 | 3261 | 0.000010 | net.batcave |
982 | 18038116 | 3341 | 0.000010 | org.mygamesonline |
983 | 18037866 | 734 | 0.000040 | com.mckinsey |
984 | 18037674 | 3983 | 0.000009 | org.eia-international |
985 | 18037604 | 258 | 0.000094 | com.sohu |
986 | 18037594 | 3700 | 0.000009 | io.dropwizard |
987 | 18037394 | 1026 | 0.000032 | gov.nps |
988 | 18037244 | 2131 | 0.000014 | au.com.news |
989 | 18036608 | 3652 | 0.000010 | de.epubli |
990 | 18034198 | 1381 | 0.000022 | com.unity3d |
991 | 18034072 | 2992 | 0.000011 | net.nend |
992 | 18033048 | 4098 | 0.000009 | com.easyhits4u |
993 | 18031890 | 1162 | 0.000027 | com.steamcommunity |
994 | 18031622 | 1451 | 0.000021 | edu.uchicago |
995 | 18031570 | 1086 | 0.000030 | com.uber |
996 | 18031470 | 5306 | 0.000007 | com.plurk |
997 | 18030490 | 597 | 0.000045 | com.adweek |
998 | 18030182 | 3635 | 0.000010 | com.jal |
999 | 18029670 | 1786 | 0.000017 | com.techradar |
1000 | 18029244 | 1271 | 0.000024 | com.ifttt |
Credits
Thanks to the authors of the WebGraph framework, whose software made the computation of graph properties and ranks possible.
We hope the data will be useful for you to do any kind of research on ranking, graph analysis, link spam detection, etc. Let us know about your results via Common Crawl’s Google Group!
January 2020 crawl archive now available
The crawl archive for January 2020 is now available! It contains 3.1 billion web pages or 300 TiB of uncompressed content, crawled between January 17th and 29th. It includes page captures of 960 million URLs not contained in any crawl archive before.
Improvements and Fixes
- date time values in the column "fetch_time" of the columnar index are now stored using the "int64" data type. For details and compatibility issues please see cc-index-table#7
- WARC request records now show the HTTP protocol version sent with the HTTP request which can be different from the version received in the HTTP response message, cf. NUTCH-2760
Archive Location and Download
The January crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-05/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List #Files Total Size
Compressed (TiB)
Segments CC-MAIN-2020-05/segment.paths.gz 100
WARC files CC-MAIN-2020-05/warc.paths.gz 56000 59.94
WAT files CC-MAIN-2020-05/wat.paths.gz 56000 22.3
WET files CC-MAIN-2020-05/wet.paths.gz 56000 10
Robots.txt files CC-MAIN-2020-05/robotstxt.paths.gz 56000 0.25
Non-200 responses files CC-MAIN-2020-05/non200responses.paths.gz 56000 2.28
URL index files CC-MAIN-2020-05/cc-index.paths.gz 302 0.23
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-05/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.