June 2021 crawl archive now available
The crawl archive for June 2021 is now available! The data was crawled June 12 – 25 and contains 2.45 billion web pages or 260 TiB of uncompressed content. It includes page captures of 1.3 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The June crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2021-25/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2021-25/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2021-25/warc.paths.gz | 64000 | 57.9 |
WAT files | CC-MAIN-2021-25/wat.paths.gz | 64000 | 16.54 |
WET files | CC-MAIN-2021-25/wet.paths.gz | 64000 | 7.24 |
Robots.txt files | CC-MAIN-2021-25/robotstxt.paths.gz | 64000 | 0.16 |
Non-200 responses files | CC-MAIN-2021-25/non200responses.paths.gz | 64000 | 1.76 |
URL index files | CC-MAIN-2021-25/cc-index.paths.gz | 302 | 0.19 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2021-25/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
Host- and Domain-Level Web Graphs February/March, April and May 2021
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of February/March, April and May 2021. Additional information about the data formats, the processing pipeline, our objectives, and credits can be found in the announcements of prior webgraph releases. You may also visit the projects cc-webgraph and cc-pyspark which include all scripts and tools required to construct the graphs. Instructions to explore the graphs in the webgraph format are given in our collection of webgraph notebooks.
What’s new?
The host-level graph now includes all hosts visited by the crawler even if there is no link pointing to the host and all visited URLs of a host failed (HTTP 404 and other error codes) or the host’s robots.txt does not allow crawling. Note that the links leading to these hosts may have been found in a prior crawl, not in one of the 3 crawls used to build this web graph.
Host-level graph
The graph consists of 515 million nodes and 2.82 billion edges. Both hyperlinks and HTTP redirects and link headers are used as edges to span up the graph. All types of links are included, including pure “technical” ones pointing to images, JavaScript libraries, web fonts, etc. However, only host names with a valid IANA TLD are used. Consequently, URLs with an IP address as host component are not taken into account for building the host-level graph.
There are 452 million dangling nodes (87.9%) and the largest strongly connected component contains 45.2 million (8.8%) nodes. Dangling nodes stem from
- hosts that have not been crawled, yet are pointed to from a link on a crawled page
- hosts without any links pointing to a different host name
- or hosts which did only return an error page (eg. HTTP 404)
Host names in the graph are in reverse domain name notation and a leading www.
is stripped: www.subdomain.example.com
becomes com.example.subdomain
.
You can download the graph and the ranks of all 515 million hosts from AWS S3 on the path s3://commoncrawl/projects/hyperlinkgraph/cc-main-2021-feb-apr-may/host/
. Alternatively, you can use https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2021-feb-apr-may/host/
as prefix to access the files from everywhere.
Please note that the text representation of the host-level graph is shipped in 72 gzip-compressed files listed in two path listings – one for the nodes (vertices), one for the edges (arcs). First, download the paths listing and decompress it using “gzip”. By adding the prefix s3://commoncrawl/
or https://data.commoncrawl.org/
to each line in the path listing you get the list of URLs to download the entire graph.
Download files of the Common Crawl Feb/Apr/May 2021 host-level webgraph
Size | File | Description |
---|---|---|
3.31 GB | cc-main-2021-feb-apr-may-host-vertices.paths.gz | nodes 〈id, rev host〉, paths of 24 vertices files |
12.94 GB | cc-main-2021-feb-apr-may-host-edges.paths.gz | edges 〈from_id, to_id〉, paths of 48 edges files |
5.57 GB | cc-main-2021-feb-apr-may-host.graph | graph in BVGraph format |
2 kB | cc-main-2021-feb-apr-may-host.properties | |
6.22 GB | cc-main-2021-feb-apr-may-host-t.graph | transpose of the graph (outlinks inverted to inlinks) |
2 kB | cc-main-2021-feb-apr-may-host-t.properties | |
1 kB | cc-main-2021-feb-apr-may-host.stats | WebGraph statistics |
7.69 GB | cc-main-2021-feb-apr-may-host-ranks.txt.gz | harmonic centrality and pagerank |
Domain-level graph
The domain graph is built by aggregating the host graph on the level of pay-level domains (PLDs) based on the public suffix list maintained on publicsuffix.org.
The domain-level graph has 88 million nodes and 1.58 billion edges. 50% or 44 million nodes are dangling nodes, the largest strongly connected component covers 34 million or 39% of the nodes.
All files related to the domain graph are available on AWS S3 under s3://commoncrawl/projects/hyperlinkgraph/cc-main-2021-feb-apr-may/domain/
resp. https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2021-feb-apr-may/domain/
.
Download files of the Common Crawl Feb/Apr/May 2021 domain-level webgraph
Size | File | Description |
---|---|---|
0.61 GB | cc-main-2021-feb-apr-may-domain-vertices.txt.gz | nodes 〈id, rev domain, num hosts〉 |
6.37 GB | cc-main-2021-feb-apr-may-domain-edges.txt.gz | edges 〈from_id, to_id〉 |
3.58 GB | cc-main-2021-feb-apr-may-domain.graph | graph in BVGraph format |
2 kB | cc-main-2021-feb-apr-may-domain.properties | |
3.42 GB | cc-main-2021-feb-apr-may-domain-t.graph | transpose of the graph |
2 kB | cc-main-2021-feb-apr-may-domain-t.properties | |
1 kB | cc-main-2021-feb-apr-may-domain.stats | WebGraph statistics |
1.89 GB | cc-main-2021-feb-apr-may-domain-ranks.txt.gz | harmonic centrality and pagerank |
Below you’ll find the top 1000 domains ranked by Harmonic Centrality or PageRank. The full list of all 88 million domain ranks is available for download.
Top 1000 domains ranked by harmonic centrality (Feb/Apr/May 2021)
harmonic centrality rank | hc value | page rank | page rank value | reversed domain name |
---|---|---|---|---|
1 | 31920934 | 1 | 0.017627 | com.googleapis |
2 | 31032784 | 3 | 0.013762 | com.facebook |
3 | 29681304 | 2 | 0.013832 | com.google |
4 | 27101692 | 4 | 0.007844 | com.twitter |
5 | 26954660 | 5 | 0.007519 | org.w |
6 | 26886624 | 7 | 0.006967 | com.youtube |
7 | 25515850 | 8 | 0.005718 | com.instagram |
8 | 25031490 | 6 | 0.007143 | com.googletagmanager |
9 | 24396116 | 9 | 0.005506 | org.gmpg |
10 | 23807122 | 12 | 0.003347 | com.linkedin |
11 | 22970992 | 13 | 0.003048 | com.gstatic |
12 | 22854052 | 10 | 0.003951 | com.cloudflare |
13 | 22698594 | 19 | 0.001914 | com.gravatar |
14 | 22504168 | 14 | 0.002908 | org.wordpress |
15 | 22434542 | 22 | 0.001564 | com.pinterest |
16 | 22100870 | 25 | 0.001270 | org.wikipedia |
17 | 21950578 | 17 | 0.002031 | com.wordpress |
18 | 21940826 | 18 | 0.001958 | com.apple |
19 | 21766696 | 15 | 0.002258 | com.bootstrapcdn |
20 | 21762964 | 30 | 0.001174 | com.vimeo |
21 | 21722198 | 38 | 0.000914 | be.youtu |
22 | 21556142 | 21 | 0.001842 | com.jquery |
23 | 21478118 | 29 | 0.001182 | com.microsoft |
24 | 21432212 | 53 | 0.000703 | com.blogspot |
25 | 21354260 | 35 | 0.001025 | com.amazonaws |
26 | 21337432 | 44 | 0.000765 | com.amazon |
27 | 21320702 | 43 | 0.000789 | gl.goo |
28 | 21170722 | 62 | 0.000600 | ly.bit |
29 | 21149628 | 99 | 0.000409 | com.tumblr |
30 | 21148242 | 50 | 0.000739 | com.wp |
31 | 21136818 | 45 | 0.000758 | org.mozilla |
32 | 21110018 | 57 | 0.000689 | eu.europa |
33 | 21104262 | 20 | 0.001894 | com.adobe |
34 | 21048760 | 16 | 0.002200 | com.github |
35 | 21040284 | 34 | 0.001026 | com.google-analytics |
36 | 21027350 | 36 | 0.001015 | net.jsdelivr |
37 | 20998320 | 27 | 0.001218 | com.wixstatic |
38 | 20995232 | 31 | 0.001119 | net.cloudfront |
39 | 20946148 | 47 | 0.000744 | com.flickr |
40 | 20913104 | 107 | 0.000338 | com.yahoo |
41 | 20851316 | 83 | 0.000436 | com.googleusercontent |
42 | 20843068 | 37 | 0.000929 | io.github |
43 | 20840670 | 111 | 0.000317 | com.reddit |
44 | 20834398 | 58 | 0.000677 | com.paypal |
45 | 20816886 | 23 | 0.001554 | com.fontawesome |
46 | 20773582 | 103 | 0.000368 | com.weebly |
47 | 20764576 | 79 | 0.000455 | com.medium |
48 | 20764512 | 33 | 0.001035 | com.googlesyndication |
49 | 20757582 | 32 | 0.001118 | ru.yandex |
50 | 20741944 | 48 | 0.000743 | com.whatsapp |
51 | 20708152 | 68 | 0.000520 | org.w3 |
52 | 20705826 | 132 | 0.000240 | com.nytimes |
53 | 20696906 | 59 | 0.000673 | co.t |
54 | 20678088 | 102 | 0.000375 | org.creativecommons |
55 | 20675822 | 115 | 0.000290 | com.soundcloud |
56 | 20644978 | 60 | 0.000624 | org.schema |
57 | 20627114 | 74 | 0.000479 | com.shopify |
58 | 20621162 | 66 | 0.000543 | com.vk |
59 | 20604726 | 181 | 0.000149 | org.wikimedia |
60 | 20604724 | 147 | 0.000204 | com.dropbox |
61 | 20579720 | 55 | 0.000702 | com.addthis |
62 | 20572950 | 138 | 0.000211 | org.archive |
63 | 20570610 | 198 | 0.000133 | com.cnn |
64 | 20558114 | 152 | 0.000187 | gov.cdc |
65 | 20550306 | 80 | 0.000446 | me.wp |
66 | 20538816 | 193 | 0.000136 | com.imgur |
67 | 20530078 | 49 | 0.000740 | net.doubleclick |
68 | 20512294 | 199 | 0.000133 | uk.co.bbc |
69 | 20505964 | 200 | 0.000133 | net.slideshare |
70 | 20499864 | 171 | 0.000155 | com.theguardian |
71 | 20489756 | 158 | 0.000175 | int.who |
72 | 20482256 | 120 | 0.000263 | com.spotify |
73 | 20481118 | 175 | 0.000151 | com.bing |
74 | 20478320 | 213 | 0.000124 | com.businessinsider |
75 | 20477478 | 253 | 0.000104 | com.bloomberg |
76 | 20477300 | 144 | 0.000206 | gov.nih |
77 | 20473648 | 46 | 0.000748 | com.macromedia |
78 | 20440520 | 254 | 0.000103 | com.wsj |
79 | 20434320 | 224 | 0.000118 | edu.stanford |
80 | 20419762 | 41 | 0.000847 | net.fbcdn |
81 | 20417930 | 39 | 0.000885 | org.apache |
82 | 20409636 | 157 | 0.000175 | org.ietf |
83 | 20397792 | 90 | 0.000420 | com.list-manage |
84 | 20395594 | 368 | 0.000071 | com.googleblog |
85 | 20395350 | 217 | 0.000123 | com.stackoverflow |
86 | 20393172 | 170 | 0.000155 | com.giphy |
87 | 20391226 | 314 | 0.000085 | edu.mit |
88 | 20381948 | 223 | 0.000118 | com.washingtonpost |
89 | 20372602 | 134 | 0.000232 | com.ytimg |
90 | 20363592 | 362 | 0.000073 | com.appspot |
91 | 20360236 | 351 | 0.000076 | com.theverge |
92 | 20359610 | 286 | 0.000093 | com.bbc |
93 | 20358870 | 396 | 0.000067 | uk.co.telegraph |
94 | 20356036 | 499 | 0.000056 | edu.berkeley |
95 | 20348048 | 266 | 0.000101 | edu.harvard |
96 | 20346012 | 330 | 0.000080 | com.go |
97 | 20341676 | 237 | 0.000112 | com.office |
98 | 20338710 | 145 | 0.000206 | us.zoom |
99 | 20335782 | 247 | 0.000109 | com.android |
100 | 20335366 | 327 | 0.000082 | com.wired |
101 | 20334160 | 288 | 0.000092 | com.techcrunch |
102 | 20331782 | 238 | 0.000111 | com.oracle |
103 | 20323638 | 547 | 0.000051 | com.livejournal |
104 | 20296670 | 164 | 0.000170 | com.issuu |
105 | 20295840 | 296 | 0.000090 | com.cnbc |
106 | 20292146 | 211 | 0.000124 | gov.ca |
107 | 20291754 | 402 | 0.000066 | com.ted |
108 | 20288380 | 379 | 0.000069 | gov.nasa |
109 | 20283426 | 149 | 0.000195 | com.forbes |
110 | 20283050 | 148 | 0.000199 | com.wixsite |
111 | 20282972 | 151 | 0.000192 | com.npmjs |
112 | 20282524 | 518 | 0.000054 | com.zdnet |
113 | 20279656 | 447 | 0.000062 | com.msn |
114 | 20277752 | 292 | 0.000091 | com.reuters |
115 | 20275540 | 350 | 0.000076 | com.nature |
116 | 20273474 | 78 | 0.000459 | com.godaddy |
117 | 20271718 | 371 | 0.000070 | com.myspace |
118 | 20270494 | 222 | 0.000119 | com.etsy |
119 | 20268832 | 321 | 0.000084 | com.prnewswire |
120 | 20255726 | 209 | 0.000125 | org.ampproject |
121 | 20252386 | 407 | 0.000065 | org.arxiv |
122 | 20252292 | 312 | 0.000085 | org.npr |
123 | 20252218 | 263 | 0.000101 | com.sciencedirect |
124 | 20248804 | 98 | 0.000410 | com.unpkg |
125 | 20246402 | 265 | 0.000101 | com.example |
126 | 20245616 | 67 | 0.000524 | net.akamaihd |
127 | 20237056 | 215 | 0.000123 | com.eventbrite |
128 | 20234532 | 367 | 0.000072 | org.hbr |
129 | 20232338 | 176 | 0.000151 | com.blogger |
130 | 20231658 | 127 | 0.000247 | org.networkadvertising |
131 | 20231552 | 399 | 0.000066 | com.latimes |
132 | 20228690 | 268 | 0.000101 | org.acm |
133 | 20223242 | 338 | 0.000079 | com.statista |
134 | 20209434 | 389 | 0.000068 | com.fastcompany |
135 | 20205848 | 660 | 0.000043 | com.economist |
136 | 20202482 | 343 | 0.000078 | com.time |
137 | 20202452 | 226 | 0.000117 | com.twimg |
138 | 20201902 | 679 | 0.000042 | edu.upenn |
139 | 20201530 | 550 | 0.000050 | edu.yale |
140 | 20200842 | 258 | 0.000102 | com.githubusercontent |
141 | 20191272 | 474 | 0.000060 | com.steampowered |
142 | 20189824 | 143 | 0.000206 | com.opera |
143 | 20188620 | 444 | 0.000062 | uk.co.dailymail |
144 | 20188486 | 353 | 0.000076 | com.springer |
145 | 20186806 | 576 | 0.000047 | com.scribd |
146 | 20184784 | 780 | 0.000041 | edu.columbia |
147 | 20180100 | 535 | 0.000052 | org.chromium |
148 | 20175876 | 591 | 0.000046 | me.about |
149 | 20175732 | 604 | 0.000046 | google.blog |
150 | 20175284 | 285 | 0.000094 | com.squarespace |
151 | 20174050 | 335 | 0.000079 | com.huffingtonpost |
152 | 20171356 | 431 | 0.000063 | com.nationalgeographic |
153 | 20168788 | 221 | 0.000119 | uk.co.google |
154 | 20165372 | 208 | 0.000125 | com.unsplash |
155 | 20163580 | 388 | 0.000068 | com.w3schools |
156 | 20158956 | 339 | 0.000079 | com.dribbble |
157 | 20154786 | 340 | 0.000079 | com.tiktok |
158 | 20153356 | 293 | 0.000091 | org.un |
159 | 20137924 | 794 | 0.000040 | com.qz |
160 | 20133814 | 248 | 0.000108 | com.bandcamp |
161 | 20129598 | 485 | 0.000058 | edu.cornell |
162 | 20125954 | 821 | 0.000039 | edu.umich |
163 | 20121120 | 119 | 0.000267 | com.ft |
164 | 20115342 | 435 | 0.000063 | com.theatlantic |
165 | 20111028 | 966 | 0.000033 | edu.princeton |
166 | 20110808 | 341 | 0.000078 | com.usatoday |
167 | 20105556 | 786 | 0.000040 | com.evernote |
168 | 20105482 | 133 | 0.000235 | info.aboutads |
169 | 20104810 | 408 | 0.000065 | com.meetup |
170 | 20102638 | 438 | 0.000062 | com.goodreads |
171 | 20100894 | 625 | 0.000045 | org.ieee |
172 | 20098972 | 878 | 0.000036 | com.slate |
173 | 20097870 | 677 | 0.000042 | com.mysql |
174 | 20097656 | 453 | 0.000061 | com.patreon |
175 | 20097530 | 137 | 0.000216 | me.t |
176 | 20095600 | 515 | 0.000055 | com.cbsnews |
177 | 20084204 | 656 | 0.000043 | com.docker |
178 | 20083336 | 291 | 0.000092 | com.wiley |
179 | 20082520 | 480 | 0.000059 | gov.usda |
180 | 20080664 | 454 | 0.000061 | com.dailymotion |
181 | 20078818 | 817 | 0.000039 | edu.washington |
182 | 20077160 | 493 | 0.000057 | com.withgoogle |
183 | 20075064 | 523 | 0.000054 | io.readthedocs |
184 | 20071014 | 644 | 0.000044 | com.marketwatch |
185 | 20065010 | 650 | 0.000043 | uk.co.blogspot |
186 | 20062734 | 868 | 0.000037 | com.shutterstock |
187 | 20062652 | 54 | 0.000703 | com.fb |
188 | 20059664 | 497 | 0.000056 | uk.co.independent |
189 | 20056344 | 76 | 0.000467 | com.wix |
190 | 20055932 | 811 | 0.000039 | org.cambridge |
191 | 20051844 | 559 | 0.000049 | com.pexels |
192 | 20048576 | 779 | 0.000041 | org.sciencemag |
193 | 20048004 | 592 | 0.000046 | com.buzzfeed |
194 | 20044248 | 819 | 0.000039 | com.stackexchange |
195 | 20043466 | 179 | 0.000149 | ru.mail |
196 | 20043446 | 844 | 0.000038 | com.webs |
197 | 20043074 | 573 | 0.000048 | com.git-scm |
198 | 20040208 | 464 | 0.000060 | com.inc |
199 | 20037354 | 272 | 0.000100 | net.behance |
200 | 20029744 | 425 | 0.000063 | gov.whitehouse |
201 | 20025342 | 832 | 0.000038 | com.apnews |
202 | 20023518 | 769 | 0.000041 | com.vox |
203 | 20022030 | 1365 | 0.000024 | uk.co.thesun |
204 | 20018548 | 274 | 0.000098 | com.outlook |
205 | 20018318 | 772 | 0.000041 | org.bitbucket |
206 | 20017276 | 40 | 0.000871 | com.qq |
207 | 20014872 | 244 | 0.000110 | org.doi |
208 | 20012082 | 812 | 0.000039 | uk.ac.cam |
209 | 20011998 | 255 | 0.000103 | com.disqus |
210 | 20007312 | 236 | 0.000112 | com.feedburner |
211 | 20005630 | 670 | 0.000043 | org.worldbank |
212 | 20001230 | 584 | 0.000047 | org.unicef |
213 | 20000932 | 419 | 0.000064 | com.mozilla |
214 | 19999740 | 593 | 0.000046 | co.ibb |
215 | 19999080 | 26 | 0.001261 | io.polyfill |
216 | 19997928 | 525 | 0.000054 | com.booking |
217 | 19993488 | 42 | 0.000808 | com.baidu |
218 | 19989784 | 260 | 0.000101 | com.cloudinary |
219 | 19985856 | 289 | 0.000092 | com.tinyurl |
220 | 19983980 | 345 | 0.000077 | com.ibm |
221 | 19983022 | 1163 | 0.000027 | com.speakerdeck |
222 | 19982506 | 597 | 0.000046 | gov.noaa |
223 | 19978206 | 612 | 0.000045 | ee.linktr |
224 | 19977310 | 569 | 0.000048 | com.psychologytoday |
225 | 19973710 | 531 | 0.000053 | gov.loc |
226 | 19972920 | 400 | 0.000066 | com.getpocket |
227 | 19972760 | 1041 | 0.000031 | edu.utexas |
228 | 19971794 | 320 | 0.000084 | org.pewresearch |
229 | 19971310 | 1366 | 0.000024 | edu.rutgers |
230 | 19970894 | 551 | 0.000050 | com.sagepub |
231 | 19970200 | 309 | 0.000087 | com.nbcnews |
232 | 19967962 | 1134 | 0.000028 | org.eclipse |
233 | 19965586 | 648 | 0.000043 | com.trello |
234 | 19964280 | 326 | 0.000082 | net.windows |
235 | 19964194 | 384 | 0.000068 | com.quora |
236 | 19961430 | 600 | 0.000046 | net.azurewebsites |
237 | 19959910 | 275 | 0.000098 | gov.ftc |
238 | 19955938 | 1057 | 0.000030 | edu.uchicago |
239 | 19953308 | 311 | 0.000086 | com.netdna-ssl |
240 | 19951960 | 782 | 0.000041 | org.semver |
241 | 19951286 | 124 | 0.000252 | com.mailchimp |
242 | 19950294 | 436 | 0.000063 | com.nypost |
243 | 19949296 | 1195 | 0.000027 | com.hatenablog |
244 | 19947142 | 652 | 0.000043 | com.newyorker |
245 | 19943938 | 985 | 0.000033 | uk.co.guardian |
246 | 19943564 | 590 | 0.000046 | com.usnews |
247 | 19940498 | 220 | 0.000119 | tv.twitch |
248 | 19939738 | 784 | 0.000041 | au.net.abc |
249 | 19938820 | 166 | 0.000167 | com.amazon-adsystem |
250 | 19936308 | 1278 | 0.000025 | com.vogue |
251 | 19935466 | 230 | 0.000113 | com.wpengine |
252 | 19934098 | 106 | 0.000338 | com.stripe |
253 | 19933266 | 1261 | 0.000025 | org.kernel |
254 | 19929738 | 941 | 0.000034 | com.politico |
255 | 19926416 | 1193 | 0.000027 | org.unicode |
256 | 19925602 | 580 | 0.000047 | org.eff |
257 | 19925174 | 541 | 0.000051 | br.com.uol |
258 | 19924806 | 852 | 0.000037 | com.about |
259 | 19923644 | 1358 | 0.000024 | edu.hbs |
260 | 19923600 | 954 | 0.000034 | com.dropboxusercontent |
261 | 19923464 | 911 | 0.000035 | edu.jhu |
262 | 19922062 | 993 | 0.000032 | co.elastic |
263 | 19921888 | 913 | 0.000035 | com.steamcommunity |
264 | 19920150 | 1971 | 0.000018 | com.googlesource |
265 | 19919760 | 522 | 0.000054 | com.tandfonline |
266 | 19918010 | 277 | 0.000097 | com.criteo |
267 | 19915708 | 552 | 0.000050 | org.pbs |
268 | 19912986 | 1106 | 0.000029 | edu.umd |
269 | 19912224 | 64 | 0.000549 | co.g |
270 | 19908340 | 865 | 0.000037 | com.foxnews |
271 | 19907456 | 123 | 0.000261 | com.sharethis |
272 | 19904178 | 1027 | 0.000031 | com.rollingstone |
273 | 19903082 | 228 | 0.000115 | com.imdb |
274 | 19902774 | 977 | 0.000033 | com.scientificamerican |
275 | 19901940 | 1392 | 0.000023 | com.urbandictionary |
276 | 19900876 | 775 | 0.000041 | uk.ac.ox |
277 | 19900406 | 391 | 0.000067 | com.arcgis |
278 | 19898520 | 2016 | 0.000018 | com.lego |
279 | 19898420 | 251 | 0.000107 | page.g |
280 | 19898318 | 631 | 0.000044 | gov.census |
281 | 19890056 | 530 | 0.000053 | com.oup |
282 | 19887968 | 346 | 0.000077 | com.optimizely |
283 | 19887424 | 582 | 0.000047 | com.indiatimes |
284 | 19887194 | 376 | 0.000069 | com.cnet |
285 | 19884024 | 422 | 0.000064 | com.wufoo |
286 | 19882930 | 704 | 0.000042 | uk.co.eventbrite |
287 | 19882806 | 421 | 0.000064 | com.bigcommerce |
288 | 19880306 | 1350 | 0.000024 | ca.blogspot |
289 | 19879016 | 833 | 0.000038 | org.fao |
290 | 19878732 | 908 | 0.000035 | com.jetbrains |
291 | 19871044 | 1467 | 0.000022 | ca.ubc |
292 | 19867650 | 1938 | 0.000018 | com.warnerbros |
293 | 19866012 | 446 | 0.000062 | org.d3js |
294 | 19865518 | 946 | 0.000034 | org.greenpeace |
295 | 19864632 | 206 | 0.000127 | net.sourceforge |
296 | 19863450 | 323 | 0.000083 | fr.google |
297 | 19862916 | 1279 | 0.000025 | com.history |
298 | 19861806 | 851 | 0.000038 | com.gumroad |
299 | 19861750 | 919 | 0.000035 | com.chicagotribune |
300 | 19859844 | 636 | 0.000044 | gov.archives |
301 | 19858902 | 284 | 0.000095 | com.googlecode |
302 | 19853502 | 342 | 0.000078 | com.slack |
303 | 19851932 | 229 | 0.000114 | com.eepurl |
304 | 19845626 | 114 | 0.000292 | com.paypalobjects |
305 | 19841702 | 927 | 0.000035 | com.sap |
306 | 19839830 | 153 | 0.000180 | com.addtoany |
307 | 19837466 | 290 | 0.000092 | com.typepad |
308 | 19834082 | 1562 | 0.000021 | de.mpg |
309 | 19830054 | 664 | 0.000043 | com.pinimg |
310 | 19828148 | 282 | 0.000095 | com.calendly |
311 | 19827530 | 491 | 0.000057 | gov.epa |
312 | 19825756 | 354 | 0.000076 | com.proofpoint |
313 | 19821128 | 1430 | 0.000023 | ch.ethz |
314 | 19821094 | 1028 | 0.000031 | com.500px |
315 | 19820554 | 1732 | 0.000019 | com.diigo |
316 | 19820398 | 334 | 0.000079 | com.live |
317 | 19820034 | 1277 | 0.000025 | org.postgresql |
318 | 19818544 | 1257 | 0.000025 | org.wiktionary |
319 | 19817910 | 1274 | 0.000025 | org.aclu |
320 | 19817698 | 981 | 0.000033 | edu.si |
321 | 19816586 | 1394 | 0.000023 | edu.msu |
322 | 19816210 | 1029 | 0.000031 | com.thehill |
323 | 19814936 | 890 | 0.000036 | de.spiegel |
324 | 19813172 | 916 | 0.000035 | com.huffpost |
325 | 19811282 | 472 | 0.000060 | gov.hhs |
326 | 19809240 | 1114 | 0.000028 | com.scmp |
327 | 19806650 | 73 | 0.000484 | me.fb |
328 | 19806306 | 764 | 0.000042 | org.change |
329 | 19805070 | 378 | 0.000069 | com.sohu |
330 | 19804336 | 1329 | 0.000024 | edu.illinois |
331 | 19804164 | 185 | 0.000147 | com.xing |
332 | 19801192 | 1323 | 0.000024 | org.tensorflow |
333 | 19801086 | 1008 | 0.000032 | com.ssrn |
334 | 19800184 | 162 | 0.000171 | com.zendesk |
335 | 19798428 | 904 | 0.000035 | com.netlify |
336 | 19797294 | 508 | 0.000056 | com.squareup |
337 | 19797020 | 1352 | 0.000024 | com.sky |
338 | 19794400 | 196 | 0.000134 | org.iana |
339 | 19792714 | 1078 | 0.000029 | uk.co.thetimes |
340 | 19792494 | 847 | 0.000038 | gov.congress |
341 | 19788704 | 809 | 0.000039 | org.pypi |
342 | 19783878 | 1422 | 0.000023 | cn.com.chinadaily |
343 | 19781142 | 972 | 0.000033 | edu.academia |
344 | 19780974 | 456 | 0.000061 | com.kickstarter |
345 | 19780084 | 802 | 0.000040 | gov.senate |
346 | 19779128 | 2415 | 0.000015 | org.pydata |
347 | 19778124 | 1140 | 0.000027 | org.semanticscholar |
348 | 19775716 | 620 | 0.000045 | site.business |
349 | 19775012 | 1275 | 0.000025 | com.over-blog |
350 | 19774866 | 792 | 0.000040 | org.oecd |
351 | 19774846 | 1660 | 0.000020 | org.phys |
352 | 19774334 | 999 | 0.000032 | com.yarnpkg |
353 | 19772248 | 816 | 0.000039 | com.deviantart |
354 | 19770936 | 1084 | 0.000029 | uk.co.mirror |
355 | 19770522 | 187 | 0.000145 | com.rawgit |
356 | 19770114 | 1315 | 0.000024 | com.axios |
357 | 19769700 | 623 | 0.000045 | gov.house |
358 | 19768998 | 894 | 0.000036 | com.discordapp |
359 | 19768866 | 880 | 0.000036 | com.sciencedaily |
360 | 19766292 | 511 | 0.000055 | com.gmail |
361 | 19765678 | 423 | 0.000064 | com.technorati |
362 | 19763944 | 216 | 0.000123 | com.hubspot |
363 | 19761638 | 1433 | 0.000023 | com.unity3d |
364 | 19760768 | 2137 | 0.000017 | org.threejs |
365 | 19760238 | 1364 | 0.000024 | com.aljazeera |
366 | 19759580 | 245 | 0.000109 | org.nodejs |
367 | 19758846 | 896 | 0.000036 | com.bmj |
368 | 19755564 | 261 | 0.000101 | com.ebay |
369 | 19755198 | 1197 | 0.000026 | au.com.smh |
370 | 19753628 | 234 | 0.000113 | org.gnu |
371 | 19751964 | 1516 | 0.000021 | edu.osu |
372 | 19751362 | 1025 | 0.000031 | int.coe |
373 | 19750302 | 994 | 0.000032 | com.britannica |
374 | 19748408 | 1312 | 0.000024 | edu.gatech |
375 | 19746818 | 2691 | 0.000013 | com.openai |
376 | 19744370 | 495 | 0.000056 | org.openstreetmap |
377 | 19743086 | 437 | 0.000062 | com.ssl-images-amazon |
378 | 19741582 | 791 | 0.000040 | br.com.google |
379 | 19741030 | 855 | 0.000037 | ca.cbc |
380 | 19740484 | 869 | 0.000037 | com.theconversation |
381 | 19739852 | 2582 | 0.000014 | edu.toronto |
382 | 19738652 | 1044 | 0.000031 | gov.usgs |
383 | 19738306 | 1556 | 0.000021 | com.newscientist |
384 | 19736226 | 301 | 0.000088 | net.themeforest |
385 | 19735698 | 605 | 0.000046 | com.udacity |
386 | 19735668 | 473 | 0.000060 | edu.nyu |
387 | 19734084 | 1716 | 0.000019 | edu.ucsc |
388 | 19723708 | 1700 | 0.000020 | org.emojipedia |
389 | 19722194 | 2068 | 0.000017 | it.scoop |
390 | 19722024 | 2754 | 0.000013 | com.slides |
391 | 19721872 | 1459 | 0.000022 | ca.sfu |
392 | 19720004 | 845 | 0.000038 | au.gov.nsw |
393 | 19717908 | 1903 | 0.000019 | org.propublica |
394 | 19717586 | 1386 | 0.000023 | com.firebaseapp |
395 | 19716094 | 2247 | 0.000016 | com.skyrock |
396 | 19710516 | 776 | 0.000041 | com.freepik |
397 | 19707962 | 97 | 0.000412 | net.facebook |
398 | 19704900 | 1454 | 0.000022 | com.penguinrandomhouse |
399 | 19703572 | 195 | 0.000135 | org.bbb |
400 | 19703432 | 1934 | 0.000018 | jp.co.japantimes |
401 | 19701030 | 1762 | 0.000019 | com.itv |
402 | 19700818 | 82 | 0.000437 | net.jsfiddle |
403 | 19700616 | 1985 | 0.000018 | org.maven |
404 | 19699746 | 2370 | 0.000015 | com.deepmind |
405 | 19697844 | 617 | 0.000045 | com.healthline |
406 | 19695324 | 506 | 0.000056 | de.gesetze-im-internet |
407 | 19694720 | 465 | 0.000060 | org.python |
408 | 19694428 | 2331 | 0.000015 | com.mystrikingly |
409 | 19691536 | 884 | 0.000036 | gov.dhs |
410 | 19688238 | 1233 | 0.000026 | com.wikia |
411 | 19685986 | 2090 | 0.000017 | org.sqlite |
412 | 19682976 | 1544 | 0.000021 | ms.1drv |
413 | 19682094 | 178 | 0.000150 | com.salesforce |
414 | 19679914 | 322 | 0.000084 | net.php |
415 | 19671484 | 324 | 0.000083 | com.surveymonkey |
416 | 19670962 | 634 | 0.000044 | com.mashable |
417 | 19670338 | 1628 | 0.000020 | com.motherjones |
418 | 19668724 | 139 | 0.000211 | com.weibo |
419 | 19668554 | 2453 | 0.000014 | com.fastcodesign |
420 | 19667444 | 1506 | 0.000021 | com.flipboard |
421 | 19666746 | 2435 | 0.000015 | edu.byu |
422 | 19665482 | 1748 | 0.000019 | edu.cuny |
423 | 19664886 | 317 | 0.000085 | ru.ok |
424 | 19662618 | 287 | 0.000092 | net.azureedge |
425 | 19662108 | 1339 | 0.000024 | com.thedailybeast |
426 | 19659672 | 246 | 0.000109 | org.aboutcookies |
427 | 19658838 | 2283 | 0.000015 | com.shutterfly |
428 | 19656108 | 1413 | 0.000023 | com.reverbnation |
429 | 19655722 | 2666 | 0.000013 | io.material |
430 | 19655254 | 537 | 0.000052 | io.codepen |
431 | 19652776 | 1296 | 0.000025 | com.dw |
432 | 19651986 | 125 | 0.000250 | com.youtube-nocookie |
433 | 19650416 | 1724 | 0.000019 | com.esri |
434 | 19650188 | 490 | 0.000057 | fr.free |
435 | 19648416 | 1509 | 0.000021 | com.substack |
436 | 19647438 | 561 | 0.000049 | com.matterport |
437 | 19646584 | 1956 | 0.000018 | com.hindustantimes |
438 | 19645830 | 1909 | 0.000019 | com.insider |
439 | 19642342 | 2110 | 0.000017 | edu.oregonstate |
440 | 19641814 | 2390 | 0.000015 | org.wikibooks |
441 | 19640838 | 891 | 0.000036 | int.wipo |
442 | 19640244 | 2820 | 0.000013 | org.aclweb |
443 | 19639226 | 607 | 0.000045 | gov.state |
444 | 19638894 | 2366 | 0.000015 | com.wattpad |
445 | 19638652 | 160 | 0.000172 | gle.forms |
446 | 19636692 | 1052 | 0.000030 | org.jstor |
447 | 19636398 | 1951 | 0.000018 | com.channel4 |
448 | 19636126 | 1752 | 0.000019 | edu.ucsb |
449 | 19635942 | 1320 | 0.000024 | gov.supremecourt |
450 | 19633994 | 56 | 0.000697 | com.googleadservices |
451 | 19631760 | 2441 | 0.000015 | at.ac.univie |
452 | 19629096 | 2924 | 0.000013 | com.pbase |
453 | 19626572 | 278 | 0.000097 | uk.org.ico |
454 | 19624802 | 639 | 0.000044 | com.licdn |
455 | 19623422 | 1518 | 0.000021 | ch.ipcc |
456 | 19621874 | 937 | 0.000034 | com.gallup |
457 | 19621780 | 496 | 0.000056 | com.herokuapp |
458 | 19618584 | 1141 | 0.000027 | edu.brookings |
459 | 19617388 | 963 | 0.000033 | edu.psu |
460 | 19616790 | 1333 | 0.000024 | mil.army |
461 | 19616626 | 434 | 0.000063 | com.rackcdn |
462 | 19614484 | 385 | 0.000068 | com.atlassian |
463 | 19611760 | 1226 | 0.000026 | com.smashingmagazine |
464 | 19609634 | 2227 | 0.000016 | blog.home |
465 | 19608450 | 1362 | 0.000024 | gov.defense |
466 | 19607698 | 1131 | 0.000028 | com.photoshelter |
467 | 19607464 | 483 | 0.000058 | net.imgix |
468 | 19607012 | 182 | 0.000149 | jp.co.yahoo |
469 | 19605316 | 2284 | 0.000015 | com.contently |
470 | 19602040 | 826 | 0.000039 | com.oreilly |
471 | 19597708 | 1174 | 0.000027 | com.mediafire |
472 | 19595596 | 2117 | 0.000017 | com.thecut |
473 | 19594604 | 1960 | 0.000018 | google.ai |
474 | 19594568 | 3151 | 0.000012 | cc.uxdesign |
475 | 19594280 | 3161 | 0.000012 | edu.uvm |
476 | 19594100 | 520 | 0.000054 | edu.cmu |
477 | 19593086 | 3137 | 0.000012 | com.instapaper |
478 | 19591090 | 1591 | 0.000020 | com.thestar |
479 | 19588378 | 369 | 0.000071 | net.researchgate |
480 | 19587214 | 3502 | 0.000011 | com.raywenderlich |
481 | 19587008 | 527 | 0.000053 | com.thinkwithgoogle |
482 | 19584868 | 2149 | 0.000016 | fr.liberation |
483 | 19582230 | 109 | 0.000336 | de.google |
484 | 19581418 | 1574 | 0.000021 | com.buzzfeednews |
485 | 19577648 | 767 | 0.000041 | org.worldwildlife |
486 | 19576662 | 1013 | 0.000032 | com.ecwid |
487 | 19576118 | 1477 | 0.000022 | com.findlaw |
488 | 19574804 | 1012 | 0.000032 | com.thelancet |
489 | 19573936 | 774 | 0.000041 | com.vice |
490 | 19573506 | 813 | 0.000039 | gov.nist |
491 | 19572872 | 1964 | 0.000018 | org.google |
492 | 19572508 | 1531 | 0.000021 | org.hrw |
493 | 19570410 | 765 | 0.000042 | com.intel |
494 | 19568238 | 2695 | 0.000013 | uk.co.ibtimes |
495 | 19567790 | 2372 | 0.000015 | com.oprah |
496 | 19567558 | 87 | 0.000428 | com.workplace |
497 | 19567194 | 3329 | 0.000011 | com.pearltrees |
498 | 19567174 | 2103 | 0.000017 | com.voanews |
499 | 19566762 | 965 | 0.000033 | com.engadget |
500 | 19566188 | 126 | 0.000247 | com.statcounter |
501 | 19564772 | 3365 | 0.000011 | org.edublogs |
502 | 19563980 | 1260 | 0.000025 | org.aiga |
503 | 19562828 | 1031 | 0.000031 | de.stern |
504 | 19562068 | 1583 | 0.000020 | fr.francetvinfo |
505 | 19560196 | 2620 | 0.000014 | com.hm |
506 | 19559342 | 315 | 0.000085 | org.drupal |
507 | 19559132 | 3736 | 0.000010 | fr.unblog |
508 | 19558786 | 747 | 0.000042 | com.canva |
509 | 19558362 | 2870 | 0.000013 | edu.ucf |
510 | 19558064 | 3204 | 0.000012 | ph.telegra |
511 | 19557534 | 926 | 0.000035 | uk.co.pinterest |
512 | 19557072 | 2402 | 0.000015 | edu.kit |
513 | 19556358 | 544 | 0.000051 | it.placehold |
514 | 19555528 | 2219 | 0.000016 | net.corporate-ir |
515 | 19553910 | 2768 | 0.000013 | co.ello |
516 | 19553426 | 881 | 0.000036 | com.arstechnica |
517 | 19553018 | 1449 | 0.000022 | com.livescience |
518 | 19550968 | 2150 | 0.000016 | com.gq |
519 | 19550836 | 1953 | 0.000018 | uk.gov.tfl |
520 | 19550254 | 210 | 0.000125 | com.iubenda |
521 | 19550042 | 533 | 0.000053 | com.pixabay |
522 | 19548328 | 1408 | 0.000023 | org.undp |
523 | 19547668 | 807 | 0.000039 | ca.amazon |
524 | 19547226 | 1020 | 0.000031 | it.smarturl |
525 | 19547032 | 2645 | 0.000014 | org.icrc |
526 | 19546934 | 2447 | 0.000015 | com.webbyawards |
527 | 19545564 | 2423 | 0.000015 | uk.ac.kcl |
528 | 19545554 | 949 | 0.000034 | edu.ucla |
529 | 19544462 | 1444 | 0.000022 | link.page |
530 | 19543968 | 2861 | 0.000013 | com.dummies |
531 | 19541366 | 1581 | 0.000021 | org.ocks |
532 | 19540748 | 65 | 0.000544 | net.typekit |
533 | 19540022 | 1122 | 0.000028 | org.ilo |
534 | 19538882 | 2564 | 0.000014 | com.depositphotos |
535 | 19538866 | 2502 | 0.000014 | com.unilever |
536 | 19536950 | 1348 | 0.000024 | org.acs |
537 | 19536262 | 81 | 0.000440 | com.livestream |
538 | 19535098 | 2672 | 0.000013 | org.rsf |
539 | 19535076 | 489 | 0.000057 | com.adweek |
540 | 19534044 | 2050 | 0.000017 | com.msnbc |
541 | 19530220 | 2509 | 0.000014 | com.slidesharecdn |
542 | 19530084 | 2035 | 0.000018 | com.chronicle |
543 | 19529836 | 3088 | 0.000012 | com.bepress |
544 | 19529580 | 2571 | 0.000014 | com.biography |
545 | 19529322 | 3384 | 0.000011 | tl.de |
546 | 19527886 | 332 | 0.000079 | com.typeform |
547 | 19526428 | 2185 | 0.000016 | com.newrepublic |
548 | 19525400 | 2303 | 0.000015 | com.thoughtco |
549 | 19523856 | 606 | 0.000045 | com.samsung |
550 | 19523112 | 1100 | 0.000029 | org.ohchr |
551 | 19522668 | 790 | 0.000040 | com.fiverr |
552 | 19521518 | 1743 | 0.000019 | io.gitlab |
553 | 19521240 | 121 | 0.000262 | com.jimdo |
554 | 19520292 | 1157 | 0.000027 | com.thenextweb |
555 | 19520070 | 2009 | 0.000018 | fr.orange |
556 | 19519618 | 3272 | 0.000012 | net.openreview |
557 | 19518936 | 2294 | 0.000015 | com.channelnewsasia |
558 | 19517090 | 1283 | 0.000025 | org.aarp |
559 | 19516918 | 2634 | 0.000014 | org.pewsocialtrends |
560 | 19516476 | 1998 | 0.000018 | com.straitstimes |
561 | 19513936 | 2310 | 0.000015 | edu.nd |
562 | 19510956 | 2099 | 0.000017 | com.dallasnews |
563 | 19510732 | 2130 | 0.000017 | de.br |
564 | 19508818 | 2278 | 0.000015 | org.fas |
565 | 19508000 | 1297 | 0.000024 | org.altervista |
566 | 19507978 | 256 | 0.000103 | uk.co.amazon |
567 | 19507290 | 219 | 0.000121 | to.amzn |
568 | 19506624 | 2835 | 0.000013 | com.thejakartapost |
569 | 19505128 | 2211 | 0.000016 | gov.lbl |
570 | 19504556 | 1610 | 0.000020 | de.berlin |
571 | 19504362 | 1086 | 0.000029 | com.popularmechanics |
572 | 19503706 | 2743 | 0.000013 | uk.ac.leeds |
573 | 19503644 | 459 | 0.000061 | com.staticflickr |
574 | 19503210 | 3397 | 0.000011 | org.neocities |
575 | 19502358 | 2996 | 0.000012 | org.vim |
576 | 19502186 | 2883 | 0.000013 | org.globalcitizen |
577 | 19499450 | 572 | 0.000048 | com.deloitte |
578 | 19499392 | 922 | 0.000035 | com.zoho |
579 | 19498964 | 233 | 0.000113 | io.shields |
580 | 19498936 | 2328 | 0.000015 | com.indianexpress |
581 | 19498902 | 3889 | 0.000010 | com.stratechery |
582 | 19497728 | 2819 | 0.000013 | app.web |
583 | 19496358 | 3386 | 0.000011 | org.zotero |
584 | 19493624 | 2939 | 0.000013 | uk.gov.scotland |
585 | 19493314 | 567 | 0.000048 | com.photobucket |
586 | 19491524 | 3756 | 0.000010 | com.bravesites |
587 | 19490552 | 1464 | 0.000022 | org.iea |
588 | 19489976 | 432 | 0.000063 | com.hp |
589 | 19489954 | 2713 | 0.000013 | uk.co.timesonline |
590 | 19489478 | 365 | 0.000073 | com.quantserve |
591 | 19489336 | 404 | 0.000066 | com.digg |
592 | 19486660 | 560 | 0.000049 | com.cisco |
593 | 19486618 | 1155 | 0.000027 | uk.parliament |
594 | 19485014 | 2914 | 0.000013 | com.nwsource |
595 | 19485012 | 2362 | 0.000015 | com.fineartamerica |
596 | 19484598 | 267 | 0.000101 | com.onesignal |
597 | 19484238 | 2234 | 0.000016 | com.foreignpolicy |
598 | 19484200 | 798 | 0.000040 | org.weforum |
599 | 19483398 | 2990 | 0.000012 | com.thoughtworks |
600 | 19483202 | 1548 | 0.000021 | com.treehugger |
601 | 19482398 | 307 | 0.000087 | com.aliyuncs |
602 | 19482224 | 602 | 0.000046 | org.js |
603 | 19480232 | 1527 | 0.000021 | gov.uscis |
604 | 19479040 | 3256 | 0.000012 | uk.ac.city |
605 | 19477476 | 2077 | 0.000017 | com.washingtontimes |
606 | 19477198 | 3504 | 0.000011 | com.mariadb |
607 | 19476316 | 2565 | 0.000014 | org.oas |
608 | 19475236 | 417 | 0.000065 | com.gitlab |
609 | 19472258 | 2584 | 0.000014 | com.mathworks |
610 | 19471752 | 2830 | 0.000013 | com.dezeen |
611 | 19471284 | 835 | 0.000038 | com.investopedia |
612 | 19470638 | 2497 | 0.000014 | uk.co.yougov |
613 | 19469316 | 2934 | 0.000013 | org.heritage |
614 | 19469308 | 614 | 0.000045 | com.netflix |
615 | 19466252 | 3281 | 0.000011 | com.shell |
616 | 19465388 | 2540 | 0.000014 | fr.paris |
617 | 19464956 | 448 | 0.000061 | gov.irs |
618 | 19462732 | 4088 | 0.000009 | tl.page |
619 | 19461330 | 1361 | 0.000024 | com.upwork |
620 | 19461170 | 462 | 0.000061 | com.sxsw |
621 | 19460914 | 1255 | 0.000025 | com.digitaloceanspaces |
622 | 19460548 | 4091 | 0.000009 | com.jigsy |
623 | 19460066 | 861 | 0.000037 | com.venturebeat |
624 | 19458418 | 1215 | 0.000026 | com.dell |
625 | 19457348 | 1016 | 0.000031 | gov.fcc |
626 | 19456828 | 3229 | 0.000012 | uk.co.walesonline |
627 | 19456346 | 2961 | 0.000013 | org.project-syndicate |
628 | 19455696 | 2024 | 0.000018 | com.fivethirtyeight |
629 | 19455242 | 920 | 0.000035 | fm.last |
630 | 19455056 | 2086 | 0.000017 | info.worldometers |
631 | 19454252 | 931 | 0.000034 | org.mediawiki |
632 | 19453670 | 2377 | 0.000015 | ly.rebrand |
633 | 19453158 | 4077 | 0.000009 | net.myanimelist |
634 | 19452824 | 2075 | 0.000017 | cn.gov.fmprc |
635 | 19452012 | 1502 | 0.000021 | org.amnesty |
636 | 19450548 | 349 | 0.000077 | com.adnxs |
637 | 19449350 | 1945 | 0.000018 | com.justia |
638 | 19448712 | 4019 | 0.000009 | edu.usfca |
639 | 19448298 | 2705 | 0.000013 | com.monday |
640 | 19446576 | 1515 | 0.000021 | ca.bc.gov |
641 | 19446486 | 943 | 0.000034 | org.reactjs |
642 | 19446126 | 2285 | 0.000015 | net.openid |
643 | 19445904 | 383 | 0.000068 | com.newrelic |
644 | 19445366 | 1363 | 0.000024 | com.imageshack |
645 | 19445144 | 3568 | 0.000010 | org.globalnetworkinitiative |
646 | 19443940 | 2549 | 0.000014 | com.kaggle |
647 | 19443562 | 3693 | 0.000010 | com.doodlekit |
648 | 19439792 | 259 | 0.000102 | com.getbootstrap |
649 | 19438670 | 2831 | 0.000013 | uk.co.inews |
650 | 19438312 | 3129 | 0.000012 | com.bangkokpost |
651 | 19438230 | 409 | 0.000065 | com.force |
652 | 19437908 | 2107 | 0.000017 | uk.ac.imperial |
653 | 19435434 | 4629 | 0.000008 | net.vingle |
654 | 19434150 | 1982 | 0.000018 | be.kuleuven |
655 | 19434066 | 3530 | 0.000011 | com.intensedebate |
656 | 19432926 | 568 | 0.000048 | com.entrepreneur |
657 | 19432350 | 3518 | 0.000011 | be.blogspot |
658 | 19429740 | 3166 | 0.000012 | se.blogspot |
659 | 19429712 | 1318 | 0.000024 | co.lpages |
660 | 19428992 | 3266 | 0.000012 | org.carnegieendowment |
661 | 19428674 | 837 | 0.000038 | com.globenewswire |
662 | 19428662 | 3175 | 0.000012 | is.good |
663 | 19428098 | 2246 | 0.000016 | com.instructure |
664 | 19427698 | 2965 | 0.000012 | net.alarabiya |
665 | 19427204 | 4090 | 0.000009 | com.kongregate |
666 | 19426514 | 2795 | 0.000013 | com.discovermagazine |
667 | 19425746 | 2613 | 0.000014 | org.gnupg |
668 | 19425518 | 556 | 0.000049 | com.visualstudio |
669 | 19424130 | 191 | 0.000139 | com.atdmt |
670 | 19423528 | 3773 | 0.000010 | com.openlearning |
671 | 19423230 | 3794 | 0.000010 | ch.swissinfo |
672 | 19421982 | 3547 | 0.000010 | com.pixar |
673 | 19420080 | 2154 | 0.000016 | com.livemint |
674 | 19419708 | 957 | 0.000033 | com.variety |
675 | 19417142 | 2816 | 0.000013 | uk.gov.metoffice |
676 | 19414346 | 2004 | 0.000018 | com.surveygizmo |
677 | 19412994 | 3337 | 0.000011 | cn.globaltimes |
678 | 19411212 | 929 | 0.000035 | uk.gov.legislation |
679 | 19411070 | 2639 | 0.000014 | org.ballotpedia |
680 | 19409736 | 243 | 0.000110 | org.whatwg |
681 | 19408620 | 3148 | 0.000012 | com.coca-colacompany |
682 | 19408342 | 1343 | 0.000024 | uk.gov.nationalarchives |
683 | 19406168 | 2326 | 0.000015 | com.thebalancesmb |
684 | 19404822 | 3145 | 0.000012 | uk.gov.companieshouse |
685 | 19403088 | 3532 | 0.000011 | com.dailykos |
686 | 19401008 | 165 | 0.000170 | com.yelp |
687 | 19400512 | 257 | 0.000103 | com.automattic |
688 | 19400270 | 4169 | 0.000009 | com.penzu |
689 | 19399686 | 2489 | 0.000014 | com.bloomberglaw |
690 | 19399662 | 412 | 0.000065 | org.opensource |
691 | 19398126 | 1547 | 0.000021 | org.khanacademy |
692 | 19397376 | 3834 | 0.000010 | com.sfweekly |
693 | 19395236 | 2779 | 0.000013 | com.thumbtack |
694 | 19394202 | 2880 | 0.000013 | org.royalsociety |
695 | 19393684 | 1674 | 0.000020 | kr.co.google |
696 | 19393678 | 2531 | 0.000014 | com.post-gazette |
697 | 19393520 | 2800 | 0.000013 | org.panda |
698 | 19390648 | 2421 | 0.000015 | com.thenation |
699 | 19389714 | 2823 | 0.000013 | io.fabric |
700 | 19388974 | 4936 | 0.000008 | org.arkive |
701 | 19388756 | 2689 | 0.000013 | uk.co.bbci |
702 | 19387624 | 4042 | 0.000009 | hk.edu.cityu |
703 | 19387406 | 3194 | 0.000012 | com.scribblelive |
704 | 19386352 | 3553 | 0.000010 | com.gimletmedia |
705 | 19385872 | 3489 | 0.000011 | com.tweetmeme |
706 | 19384830 | 2541 | 0.000014 | de.uni-heidelberg |
707 | 19384284 | 298 | 0.000089 | ai.shortpixel |
708 | 19383872 | 1920 | 0.000019 | gov.gao |
709 | 19382974 | 4425 | 0.000008 | com.storeboard |
710 | 19381650 | 2814 | 0.000013 | com.politifact |
711 | 19380202 | 3349 | 0.000011 | org.cato |
712 | 19379282 | 4889 | 0.000008 | com.uberant |
713 | 19377306 | 3183 | 0.000012 | fr.lepoint |
714 | 19377194 | 3809 | 0.000010 | edu.depaul |
715 | 19376126 | 3844 | 0.000010 | net.thedailystar |
716 | 19375590 | 406 | 0.000066 | com.aol |
717 | 19375570 | 4046 | 0.000009 | edu.umt |
718 | 19372794 | 1948 | 0.000018 | tv.ustream |
719 | 19372628 | 1034 | 0.000031 | com.verisign |
720 | 19369588 | 3279 | 0.000011 | com.theweek |
721 | 19367934 | 905 | 0.000035 | com.box |
722 | 19367170 | 3724 | 0.000010 | com.eklablog |
723 | 19365850 | 3488 | 0.000011 | com.militarytimes |
724 | 19365832 | 866 | 0.000037 | gov.uspto |
725 | 19365580 | 3483 | 0.000011 | com.multiscreensite |
726 | 19364098 | 3103 | 0.000012 | uk.ac.york |
727 | 19359488 | 3165 | 0.000012 | org.openweathermap |
728 | 19358574 | 1526 | 0.000021 | com.techrepublic |
729 | 19358070 | 3315 | 0.000011 | org.jenkins-ci |
730 | 19357968 | 2815 | 0.000013 | org.wnyc |
731 | 19357458 | 638 | 0.000044 | gov.copyright |
732 | 19356834 | 3433 | 0.000011 | com.lawfareblog |
733 | 19354610 | 2357 | 0.000015 | co.pcdn |
734 | 19353004 | 3263 | 0.000012 | com.nyt |
735 | 19352766 | 3101 | 0.000012 | se.svt |
736 | 19351866 | 1048 | 0.000030 | net.clickbank |
737 | 19351546 | 3121 | 0.000012 | com.scotsman |
738 | 19348720 | 1182 | 0.000027 | com.foursquare |
739 | 19348660 | 1239 | 0.000026 | com.pingdom |
740 | 19348048 | 2475 | 0.000014 | com.squarespace-cdn |
741 | 19346678 | 2323 | 0.000015 | com.natlawreview |
742 | 19346350 | 2769 | 0.000013 | org.wri |
743 | 19345800 | 3430 | 0.000011 | com.bigthink |
744 | 19345054 | 4132 | 0.000009 | com.newgrounds |
745 | 19344692 | 3862 | 0.000010 | org.sourcewatch |
746 | 19342356 | 3720 | 0.000010 | re.cli |
747 | 19341788 | 3156 | 0.000012 | gov.ncjrs |
748 | 19341458 | 3087 | 0.000012 | my.com.thestar |
749 | 19340698 | 3307 | 0.000011 | gov.anl |
750 | 19339932 | 3117 | 0.000012 | com.nationalreview |
751 | 19339132 | 2597 | 0.000014 | ca.newswire |
752 | 19338090 | 1603 | 0.000020 | org.webkit |
753 | 19337402 | 3700 | 0.000010 | org.elasticsearch |
754 | 19335276 | 928 | 0.000035 | com.hootsuite |
755 | 19334936 | 300 | 0.000088 | com.caniuse |
756 | 19334252 | 3236 | 0.000012 | gov.fec |
757 | 19333910 | 2327 | 0.000015 | ru.rg |
758 | 19333124 | 3741 | 0.000010 | org.constitutioncenter |
759 | 19332102 | 1602 | 0.000020 | com.jwplayer |
760 | 19331754 | 4253 | 0.000009 | com.etymonline |
761 | 19331678 | 3620 | 0.000010 | it.eventbrite |
762 | 19331510 | 2960 | 0.000013 | com.madmimi |
763 | 19331460 | 3491 | 0.000011 | com.afp |
764 | 19330192 | 1907 | 0.000019 | com.kinstacdn |
765 | 19328136 | 3163 | 0.000012 | gov.ornl |
766 | 19327042 | 461 | 0.000061 | com.pubmatic |
767 | 19325866 | 401 | 0.000066 | gg.discord |
768 | 19325518 | 1289 | 0.000025 | com.intuit |
769 | 19325482 | 1168 | 0.000027 | com.ycombinator |
770 | 19325258 | 3292 | 0.000011 | com.crashlytics |
771 | 19324302 | 4270 | 0.000009 | com.underconsideration |
772 | 19322856 | 2599 | 0.000014 | com.articulate |
773 | 19322230 | 3246 | 0.000012 | de.uni-frankfurt |
774 | 19321496 | 3692 | 0.000010 | uk.co.spectator |
775 | 19321096 | 867 | 0.000037 | com.wikihow |
776 | 19321010 | 4275 | 0.000009 | to.gplus |
777 | 19320802 | 4920 | 0.000008 | pl.pastebin |
778 | 19320622 | 3791 | 0.000010 | uk.co.manchestereveningnews |
779 | 19319854 | 2938 | 0.000013 | edu.unh |
780 | 19318976 | 2553 | 0.000014 | de.tagesschau |
781 | 19318802 | 2116 | 0.000017 | gov.energystar |
782 | 19318372 | 429 | 0.000063 | com.businesswire |
783 | 19318050 | 829 | 0.000038 | com.moz |
784 | 19314848 | 3550 | 0.000010 | org.avaaz |
785 | 19314554 | 3683 | 0.000010 | com.mnn |
786 | 19314476 | 1172 | 0.000027 | com.alexa |
787 | 19314150 | 2332 | 0.000015 | net.vnexpress |
788 | 19313268 | 348 | 0.000077 | com.constantcontact |
789 | 19312732 | 3600 | 0.000010 | com.heraldscotland |
790 | 19312326 | 3843 | 0.000010 | fm.audioboo |
791 | 19311750 | 4481 | 0.000008 | tv.eurovision |
792 | 19311646 | 974 | 0.000033 | com.fandom |
793 | 19311256 | 3717 | 0.000010 | uk.ac.uea |
794 | 19311174 | 3697 | 0.000010 | uk.ac.core |
795 | 19310268 | 3514 | 0.000011 | com.hsbc |
796 | 19310254 | 3492 | 0.000011 | org.sciencenews |
797 | 19310242 | 4916 | 0.000008 | com.blackplanet |
798 | 19310096 | 3289 | 0.000011 | com.realclearpolitics |
799 | 19309366 | 1698 | 0.000020 | com.pastebin |
800 | 19309196 | 3190 | 0.000012 | uk.org.rspb |
801 | 19308322 | 1377 | 0.000023 | com.techradar |
802 | 19308094 | 529 | 0.000053 | com.indeed |
803 | 19307548 | 4985 | 0.000007 | dk.bloggersdelight |
804 | 19307144 | 4491 | 0.000008 | com.xtgem |
805 | 19306108 | 2073 | 0.000017 | ca.on.gov |
806 | 19305500 | 3536 | 0.000011 | uk.co.thisismoney |
807 | 19304908 | 797 | 0.000040 | gov.sec |
808 | 19302330 | 1128 | 0.000028 | net.atlassian |
809 | 19302240 | 3937 | 0.000009 | com.collinsdictionary |
810 | 19299944 | 1479 | 0.000022 | edu.purdue |
811 | 19299020 | 3179 | 0.000012 | com.wayfair |
812 | 19298908 | 3611 | 0.000010 | org.chathamhouse |
813 | 19297900 | 3218 | 0.000012 | org.rferl |
814 | 19297216 | 397 | 0.000066 | com.skype |
815 | 19296536 | 4738 | 0.000008 | edu.ualr |
816 | 19296016 | 3523 | 0.000011 | org.diva-portal |
817 | 19295672 | 2785 | 0.000013 | org.cfr |
818 | 19294806 | 1249 | 0.000025 | com.merriam-webster |
819 | 19292968 | 4835 | 0.000008 | com.designobserver |
820 | 19292734 | 3399 | 0.000011 | org.pewforum |
821 | 19292200 | 270 | 0.000100 | jp.co.amazon |
822 | 19291468 | 3994 | 0.000009 | uk.co.dailyrecord |
823 | 19290936 | 3951 | 0.000009 | edu.swarthmore |
824 | 19290570 | 3339 | 0.000011 | com.ubs |
825 | 19289748 | 1075 | 0.000030 | so.notion |
826 | 19289742 | 2847 | 0.000013 | us.govtrack |
827 | 19289236 | 1256 | 0.000025 | com.udemy |
828 | 19289040 | 333 | 0.000079 | com.hackerone |
829 | 19288716 | 3787 | 0.000010 | org.nationalinterest |
830 | 19288626 | 3138 | 0.000012 | com.doubleclickbygoogle |
831 | 19288000 | 279 | 0.000097 | de.amazon |
832 | 19287244 | 2036 | 0.000018 | org.doxygen |
833 | 19286840 | 1661 | 0.000020 | scot.gov |
834 | 19286652 | 3933 | 0.000009 | de.berliner-zeitung |
835 | 19285868 | 1519 | 0.000021 | com.billboard |
836 | 19283910 | 681 | 0.000042 | com.gartner |
837 | 19283390 | 4698 | 0.000008 | net.writeablog |
838 | 19282688 | 2465 | 0.000014 | com.infoworld |
839 | 19282084 | 823 | 0.000039 | com.sedo |
840 | 19281700 | 3200 | 0.000012 | org.aei |
841 | 19280820 | 71 | 0.000502 | com.oculus |
842 | 19280652 | 1580 | 0.000021 | edu.ucsd |
843 | 19280396 | 329 | 0.000081 | mp.mailchi |
844 | 19280288 | 3917 | 0.000009 | edu.umaine |
845 | 19279222 | 3262 | 0.000012 | org.iucnredlist |
846 | 19279130 | 2827 | 0.000013 | com.lexology |
847 | 19278304 | 4851 | 0.000008 | com.nation2 |
848 | 19278156 | 5290 | 0.000007 | com.anotepad |
849 | 19278056 | 4128 | 0.000009 | za.co.mg |
850 | 19276824 | 77 | 0.000467 | com.messenger |
851 | 19276460 | 2083 | 0.000017 | org.dejure |
852 | 19276002 | 4494 | 0.000008 | net.blogfreely |
853 | 19275630 | 1302 | 0.000024 | org.owasp |
854 | 19275142 | 3309 | 0.000011 | com.foreignaffairs |
855 | 19275092 | 4067 | 0.000009 | tw.com.books |
856 | 19274916 | 4267 | 0.000009 | ca.nfb |
857 | 19274822 | 364 | 0.000073 | com.bitly |
858 | 19274560 | 3225 | 0.000012 | org.osce |
859 | 19274028 | 3726 | 0.000010 | uk.org.wwf |
860 | 19274006 | 3971 | 0.000009 | org.truthout |
861 | 19273104 | 155 | 0.000178 | gov.privacyshield |
862 | 19272708 | 1981 | 0.000018 | edu.uci |
863 | 19272368 | 2044 | 0.000017 | se.haxx |
864 | 19272288 | 897 | 0.000036 | com.emarketer |
865 | 19272110 | 4532 | 0.000008 | com.symbaloo |
866 | 19271508 | 1004 | 0.000032 | com.playstation |
867 | 19271338 | 2196 | 0.000016 | org.sundance |
868 | 19271216 | 363 | 0.000073 | eu.youronlinechoices |
869 | 19271196 | 3496 | 0.000011 | com.rev |
870 | 19271080 | 4071 | 0.000009 | in.thewire |
871 | 19270976 | 159 | 0.000174 | org.nginx |
872 | 19270528 | 903 | 0.000036 | com.libsyn |
873 | 19268650 | 2400 | 0.000015 | us.pa.state |
874 | 19267610 | 146 | 0.000205 | me.line |
875 | 19267478 | 5202 | 0.000007 | net.bravejournal |
876 | 19267386 | 3140 | 0.000012 | ru.kp |
877 | 19267334 | 4014 | 0.000009 | com.ecowatch |
878 | 19266700 | 514 | 0.000055 | org.debian |
879 | 19266302 | 539 | 0.000052 | com.gofundme |
880 | 19266194 | 976 | 0.000033 | com.pcmag |
881 | 19264914 | 4151 | 0.000009 | com.theoutline |
882 | 19264512 | 4316 | 0.000009 | org.icj-cij |
883 | 19263626 | 1470 | 0.000022 | org.coursera |
884 | 19261610 | 2076 | 0.000017 | gov.healthcare |
885 | 19260626 | 3721 | 0.000010 | com.iconarchive |
886 | 19259734 | 1657 | 0.000020 | net.leadpages |
887 | 19259034 | 1486 | 0.000022 | com.technologyreview |
888 | 19258032 | 2367 | 0.000015 | ca.citizenlab |
889 | 19257884 | 3690 | 0.000010 | com.governing |
890 | 19257782 | 3322 | 0.000011 | com.wikidot |
891 | 19257260 | 2385 | 0.000015 | org.raspberrypi |
892 | 19256452 | 4621 | 0.000008 | jp.ac.kobe-u |
893 | 19255454 | 1073 | 0.000030 | com.timeanddate |
894 | 19254836 | 1096 | 0.000029 | com.buffer |
895 | 19254032 | 3978 | 0.000009 | com.ogilvy |
896 | 19251530 | 940 | 0.000034 | com.css-tricks |
897 | 19251096 | 1501 | 0.000021 | com.msdn |
898 | 19250138 | 3958 | 0.000009 | com.gab |
899 | 19249994 | 3673 | 0.000010 | com.what3words |
900 | 19249260 | 1241 | 0.000026 | com.tableau |
901 | 19248316 | 1319 | 0.000024 | com.xkcd |
902 | 19248224 | 3695 | 0.000010 | com.nestle |
903 | 19247678 | 4982 | 0.000007 | net.postheaven |
904 | 19246428 | 470 | 0.000060 | com.fc2 |
905 | 19246238 | 1795 | 0.000019 | com.pcworld |
906 | 19246028 | 2589 | 0.000014 | mp.j |
907 | 19245754 | 4318 | 0.000009 | org.kuow |
908 | 19245300 | 3906 | 0.000009 | org.migrationpolicy |
909 | 19245282 | 585 | 0.000047 | com.fortune |
910 | 19244324 | 3769 | 0.000010 | de.morgenpost |
911 | 19244120 | 3282 | 0.000011 | uk.gov.data |
912 | 19243558 | 4952 | 0.000007 | cz.webgarden |
913 | 19243100 | 2118 | 0.000017 | org.donorbox |
914 | 19242192 | 3909 | 0.000009 | de.uni-konstanz |
915 | 19241684 | 4218 | 0.000009 | org.birdlife |
916 | 19240982 | 3875 | 0.000010 | org.people-press |
917 | 19240778 | 2132 | 0.000017 | to.dev |
918 | 19239846 | 906 | 0.000035 | org.golang |
919 | 19238732 | 2425 | 0.000015 | net.noscript |
920 | 19237742 | 1223 | 0.000026 | com.podbean |
921 | 19235906 | 4130 | 0.000009 | com.scienceblogs |
922 | 19235706 | 4948 | 0.000007 | it.clyp |
923 | 19235498 | 3355 | 0.000011 | edu.fordham |
924 | 19231696 | 4076 | 0.000009 | org.oyez |
925 | 19230656 | 3441 | 0.000011 | com.joebiden |
926 | 19229960 | 2867 | 0.000013 | com.washingtonexaminer |
927 | 19229728 | 1115 | 0.000028 | com.gizmodo |
928 | 19229112 | 2757 | 0.000013 | org.healthaffairs |
929 | 19228910 | 1232 | 0.000026 | com.searchengineland |
930 | 19228678 | 854 | 0.000037 | fm.anchor |
931 | 19227412 | 5084 | 0.000007 | com.zcubes |
932 | 19227258 | 1995 | 0.000018 | com.ssllabs |
933 | 19225964 | 1072 | 0.000030 | org.poynter |
934 | 19224436 | 1644 | 0.000020 | net.java |
935 | 19223632 | 1514 | 0.000021 | edu.usc |
936 | 19223252 | 3680 | 0.000010 | org.carbonbrief |
937 | 19221502 | 5165 | 0.000007 | org.csgrid |
938 | 19221286 | 308 | 0.000087 | jp.ameblo |
939 | 19220064 | 1578 | 0.000021 | com.sun |
940 | 19220010 | 3959 | 0.000009 | org.rfa |
941 | 19218588 | 2616 | 0.000014 | uk.gov.defra |
942 | 19218556 | 3912 | 0.000009 | com.exxonmobil |
943 | 19218102 | 5249 | 0.000007 | com.topsitenet |
944 | 19217732 | 3012 | 0.000012 | com.html5rocks |
945 | 19217494 | 3660 | 0.000010 | ca.yelp |
946 | 19216576 | 2940 | 0.000013 | com.instructables |
947 | 19215582 | 2212 | 0.000016 | org.linuxfoundation |
948 | 19215410 | 4069 | 0.000009 | uk.org.woodlandtrust |
949 | 19213854 | 2058 | 0.000017 | org.json |
950 | 19213790 | 214 | 0.000124 | com.tripadvisor |
951 | 19212490 | 5233 | 0.000007 | net.squareblogs |
952 | 19212378 | 3864 | 0.000010 | ru.mid |
953 | 19212170 | 231 | 0.000113 | com.myshopify |
954 | 19211108 | 3310 | 0.000011 | com.flippa |
955 | 19211092 | 3850 | 0.000010 | com.townandcountrymag |
956 | 19210938 | 1292 | 0.000025 | build.bazel |
957 | 19210816 | 5295 | 0.000007 | net.werite |
958 | 19210212 | 1240 | 0.000026 | com.uk |
959 | 19209742 | 2354 | 0.000015 | com.storify |
960 | 19209508 | 3280 | 0.000011 | org.cjr |
961 | 19208854 | 3158 | 0.000012 | org.acog |
962 | 19208448 | 3921 | 0.000009 | br.com.sebrae |
963 | 19208380 | 250 | 0.000107 | org.icann |
964 | 19207876 | 1635 | 0.000020 | fr.blogspot |
965 | 19207582 | 122 | 0.000262 | com.bizjournals |
966 | 19207280 | 3406 | 0.000011 | org.cites |
967 | 19207102 | 2687 | 0.000013 | com.tutsplus |
968 | 19207058 | 3409 | 0.000011 | tr.com.aa |
969 | 19206380 | 1109 | 0.000028 | org.whatbrowser |
970 | 19205750 | 4680 | 0.000008 | org.learner |
971 | 19205144 | 3424 | 0.000011 | no.yr |
972 | 19203738 | 4271 | 0.000009 | com.s-nbcnews |
973 | 19203150 | 4166 | 0.000009 | org.spie |
974 | 19203082 | 1335 | 0.000024 | com.indiegogo |
975 | 19202634 | 708 | 0.000042 | com.airbnb |
976 | 19202288 | 4217 | 0.000009 | com.revolut |
977 | 19201514 | 4339 | 0.000009 | org.atsjournals |
978 | 19201386 | 1033 | 0.000031 | com.redhat |
979 | 19200760 | 4066 | 0.000009 | uk.co.zoopla |
980 | 19199826 | 318 | 0.000084 | it.google |
981 | 19199246 | 1137 | 0.000028 | com.windowsphone |
982 | 19198666 | 1485 | 0.000022 | edu.unc |
983 | 19198508 | 466 | 0.000060 | gov.fda |
984 | 19198408 | 653 | 0.000043 | com.zapier |
985 | 19198272 | 2161 | 0.000016 | com.gigaom |
986 | 19197316 | 4457 | 0.000008 | ru.novayagazeta |
987 | 19196504 | 1936 | 0.000018 | br.com.correios |
988 | 19196468 | 4101 | 0.000009 | google.design |
989 | 19195350 | 2194 | 0.000016 | org.eu |
990 | 19192238 | 3758 | 0.000010 | com.mail-archive |
991 | 19191310 | 4437 | 0.000008 | com.out |
992 | 19191000 | 4759 | 0.000008 | tw.focustaiwan |
993 | 19190946 | 4235 | 0.000009 | org.insideclimatenews |
994 | 19190774 | 2038 | 0.000017 | com.freeprivacypolicy |
995 | 19190442 | 4265 | 0.000009 | org.escardio |
996 | 19190354 | 4663 | 0.000008 | com.theschooloflife |
997 | 19189766 | 241 | 0.000111 | com.naver |
998 | 19188362 | 4711 | 0.000008 | edu.uah |
999 | 19188230 | 1611 | 0.000020 | com.nike |
1000 | 19187360 | 4319 | 0.000009 | edu.mtsu |
Credits
Thanks to the authors of the WebGraph framework, whose software made the computation of graph properties and ranks possible.
We hope the data will be useful for you to do any kind of research on ranking, graph analysis, link spam detection, etc. Let us know about your results via Common Crawl’s Google Group!
May 2021 crawl archive now available
The crawl archive for May 2021 is now available! The data was crawled May 5 – 19 and contains 2.6 billion web pages or 280 TiB of uncompressed content. It includes page captures of 1.28 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The May crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2021-21/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2021-21/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2021-21/warc.paths.gz | 64000 | 66.17 |
WAT files | CC-MAIN-2021-21/wat.paths.gz | 64000 | 17.61 |
WET files | CC-MAIN-2021-21/wet.paths.gz | 64000 | 7.65 |
Robots.txt files | CC-MAIN-2021-21/robotstxt.paths.gz | 64000 | 0.17 |
Non-200 responses files | CC-MAIN-2021-21/non200responses.paths.gz | 64000 | 1.86 |
URL index files | CC-MAIN-2021-21/cc-index.paths.gz | 302 | 0.2 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2021-21/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
April 2021 crawl archive now available
The crawl archive for April 2021 is now available! The data was crawled April 10 – 23 and contains 3.1 billion web pages or 320 TiB of uncompressed content. It includes page captures of 1.35 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The April crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2021-17/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2021-17/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2021-17/warc.paths.gz | 64000 | 69.78 |
WAT files | CC-MAIN-2021-17/wat.paths.gz | 64000 | 21.05 |
WET files | CC-MAIN-2021-17/wet.paths.gz | 64000 | 9.25 |
Robots.txt files | CC-MAIN-2021-17/robotstxt.paths.gz | 64000 | 0.16 |
Non-200 responses files | CC-MAIN-2021-17/non200responses.paths.gz | 64000 | 1.76 |
URL index files | CC-MAIN-2021-17/cc-index.paths.gz | 302 | 0.23 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2021-17/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
February/March 2021 crawl archive now available
The crawl archive for February/March 2021 is now available! The data was crawled between February 24th and March 9th and contains 2.7 billion web pages or 280 TiB of uncompressed content. It includes page captures of 1.2 billion new URLs, not visited in any of our prior crawls.
Improvements and Fixes
The ISO639-3 code for the Hmong language was updated to "hmn" – the code "blu" used so far was already deprecated in 2008. Crawl archives prior to this crawl will still use the code "blu". More details about this update are found here.
Archive Location and Download
The February/March crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2021-10/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2021-10/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2021-10/warc.paths.gz | 64000 | 62.51 |
WAT files | CC-MAIN-2021-10/wat.paths.gz | 64000 | 18.44 |
WET files | CC-MAIN-2021-10/wet.paths.gz | 64000 | 8.06 |
Robots.txt files | CC-MAIN-2021-10/robotstxt.paths.gz | 64000 | 0.2 |
Non-200 responses files | CC-MAIN-2021-10/non200responses.paths.gz | 64000 | 1.58 |
URL index files | CC-MAIN-2021-10/cc-index.paths.gz | 302 | 0.21 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2021-10/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
Host- and Domain-Level Web Graphs October, November/December 2020 and January 2021
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of October, November/December 2020 and January 2021. Additional information about the data formats, the processing pipeline, our objectives, and credits can be found in the announcements of prior webgraph releases (e.g., Nov/Dec/Jan 2017-2018 Webgraphs). You may also visit the projects cc-webgraph and cc-pyspark which include all scripts and tools required to construct the graphs. Instructions to explore the graphs in the webgraph format are given in our collection of webgraph notebooks.
Host-level graph
The graph consists of 490 million nodes and 2.57 billion edges and includes dangling nodes i.e. hosts that have not been crawled yet are pointed to from a link on a crawled page. There are 414 million dangling nodes (84.4%) and the largest strongly connected component contains 42.6 million (8.7%) nodes.
Host names in the graph are in reverse domain name notation and a leading www.
is stripped: www.subdomain.example.com
becomes com.example.subdomain
.
You can download the graph and the ranks of all 490 million hosts from AWS S3 on the path s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-21-oct-nov-jan/host/
. Alternatively, you can use https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2020-21-oct-nov-jan/host/
as prefix to access the files from everywhere.
Please note that the text representation of the host-level graph is shipped in 36 gzip-compressed files listed in two path listings – one for the nodes, one for the edges. First, download the paths listing and uncompress it using “gzip”. By adding the prefix s3://commoncrawl/
or https://data.commoncrawl.org/
to each line in the path listing you get the list of URLs to download the entire graph.
Download files of the Common Crawl Oct/Nov/Jan 2020-2021 host-level webgraph
Size | File | Description |
---|---|---|
3.08 GB | cc-main-2020-21-oct-nov-jan-host-vertices.paths.gz | nodes 〈id, rev host〉, paths of 12 vertices files |
11.76 GB | cc-main-2020-21-oct-nov-jan-host-edges.paths.gz | edges 〈from_id, to_id〉, paths of 24 edges files |
5.18 GB | cc-main-2020-21-oct-nov-jan-host.graph | graph in BVGraph format |
2 kB | cc-main-2020-21-oct-nov-jan-host.properties | |
5.63 GB | cc-main-2020-21-oct-nov-jan-host-t.graph | transpose of the graph (outlinks inverted to inlinks) |
2 kB | cc-main-2020-21-oct-nov-jan-host-t.properties | |
1 kB | cc-main-2020-21-oct-nov-jan-host.stats | WebGraph statistics |
7.04 GB | cc-main-2020-21-oct-nov-jan-host-ranks.txt.gz | harmonic centrality and pagerank |
Domain-level graph
The domain graph was built by aggregating the host graph on the level of pay-level domains (PLDs) based on the public suffix list maintained on publicsuffix.org.
The domain-level graph has 86 million nodes and 1.47 billion edges. 50% or 43 million nodes are dangling nodes, the largest strongly connected component covers 34 million or 39% of the nodes.
All files related to the domain graph are available on AWS S3 under s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-21-oct-nov-jan/domain/
resp. https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2020-21-oct-nov-jan/domain/
.
Download files of the Common Crawl Oct/Nov/Jan 2020-2021 domain-level webgraph
Size | File | Description |
---|---|---|
0.59 GB | cc-main-2020-21-oct-nov-jan-domain-vertices.txt.gz | nodes 〈id, rev domain, num hosts〉 |
6.00 GB | cc-main-2020-21-oct-nov-jan-domain-edges.txt.gz | edges 〈from_id, to_id〉 |
3.40 GB | cc-main-2020-21-oct-nov-jan-domain.graph | graph in BVGraph format |
2 kB | cc-main-2020-21-oct-nov-jan-domain.properties | |
3.26 GB | cc-main-2020-21-oct-nov-jan-domain-t.graph | transpose of the graph |
2 kB | cc-main-2020-21-oct-nov-jan-domain-t.properties | |
1 kB | cc-main-2020-21-oct-nov-jan-domain.stats | WebGraph statistics |
1.85 GB | cc-main-2020-21-oct-nov-jan-domain-ranks.txt.gz | harmonic centrality and pagerank |
Below you’ll find the top 1000 domains ranked by Harmonic Centrality or PageRank. The full list of all 86 million domain ranks is available for download.
Top 1000 domains ranked by harmonic centrality (Oct/Nov/Jan 2020-2021)
harmonic centrality rank | hc value | page rank | page rank value | reversed domain name |
---|---|---|---|---|
1 | 30355566 | 1 | 0.017956 | com.googleapis |
2 | 29427164 | 3 | 0.012871 | com.facebook |
3 | 28173562 | 2 | 0.012899 | com.google |
4 | 25702812 | 5 | 0.007348 | com.twitter |
5 | 25628314 | 4 | 0.007628 | org.w |
6 | 25297808 | 6 | 0.007231 | com.youtube |
7 | 24195466 | 9 | 0.005352 | com.instagram |
8 | 23355356 | 8 | 0.005532 | org.gmpg |
9 | 23233674 | 7 | 0.006500 | com.googletagmanager |
10 | 22492432 | 11 | 0.003277 | com.linkedin |
11 | 21576402 | 10 | 0.004076 | com.cloudflare |
12 | 21468510 | 14 | 0.002649 | com.gravatar |
13 | 21395642 | 13 | 0.003020 | org.wordpress |
14 | 21353798 | 22 | 0.001726 | com.pinterest |
15 | 20946722 | 30 | 0.001242 | org.wikipedia |
16 | 20926308 | 19 | 0.001834 | com.wordpress |
17 | 20877776 | 16 | 0.002056 | com.gstatic |
18 | 20799472 | 15 | 0.002451 | com.bootstrapcdn |
19 | 20795402 | 18 | 0.001943 | com.apple |
20 | 20626472 | 32 | 0.001165 | com.vimeo |
21 | 20527986 | 41 | 0.000886 | be.youtu |
22 | 20419038 | 21 | 0.001769 | com.jquery |
23 | 20391686 | 28 | 0.001246 | com.microsoft |
24 | 20327544 | 24 | 0.001500 | com.wp |
25 | 20314602 | 45 | 0.000769 | com.blogspot |
26 | 20231490 | 37 | 0.001025 | com.amazonaws |
27 | 20208912 | 51 | 0.000691 | com.amazon |
28 | 20199388 | 47 | 0.000740 | gl.goo |
29 | 20093688 | 71 | 0.000448 | com.tumblr |
30 | 20070176 | 35 | 0.001070 | com.google-analytics |
31 | 20050256 | 61 | 0.000598 | ly.bit |
32 | 20030452 | 20 | 0.001794 | com.adobe |
33 | 19998314 | 17 | 0.002005 | com.github |
34 | 19989010 | 50 | 0.000715 | org.mozilla |
35 | 19962834 | 58 | 0.000639 | eu.europa |
36 | 19945306 | 34 | 0.001103 | net.cloudfront |
37 | 19849112 | 52 | 0.000682 | com.flickr |
38 | 19843288 | 40 | 0.000909 | net.jsdelivr |
39 | 19833032 | 91 | 0.000369 | com.googleusercontent |
40 | 19823560 | 105 | 0.000347 | com.yahoo |
41 | 19752300 | 56 | 0.000650 | co.t |
42 | 19722088 | 33 | 0.001114 | com.googlesyndication |
43 | 19712406 | 23 | 0.001517 | com.fontawesome |
44 | 19708354 | 81 | 0.000392 | com.weebly |
45 | 19706054 | 55 | 0.000653 | com.paypal |
46 | 19695288 | 109 | 0.000308 | com.reddit |
47 | 19641534 | 31 | 0.001231 | me.wp |
48 | 19640398 | 73 | 0.000435 | com.medium |
49 | 19635162 | 67 | 0.000491 | io.github |
50 | 19590444 | 137 | 0.000225 | com.nytimes |
51 | 19587880 | 121 | 0.000280 | com.soundcloud |
52 | 19585192 | 27 | 0.001262 | ru.yandex |
53 | 19583494 | 43 | 0.000786 | com.addthis |
54 | 19582250 | 44 | 0.000776 | com.macromedia |
55 | 19560416 | 66 | 0.000504 | org.w3 |
56 | 19549714 | 70 | 0.000451 | com.shopify |
57 | 19518672 | 146 | 0.000201 | com.forbes |
58 | 19502448 | 144 | 0.000205 | org.archive |
59 | 19496300 | 90 | 0.000371 | org.creativecommons |
60 | 19490348 | 194 | 0.000131 | uk.co.bbc |
61 | 19482926 | 59 | 0.000630 | org.schema |
62 | 19479528 | 39 | 0.000910 | com.baidu |
63 | 19464572 | 36 | 0.001035 | net.doubleclick |
64 | 19459966 | 200 | 0.000129 | com.cnn |
65 | 19451100 | 53 | 0.000677 | com.whatsapp |
66 | 19449068 | 60 | 0.000611 | com.vk |
67 | 19444966 | 206 | 0.000126 | net.slideshare |
68 | 19443956 | 158 | 0.000169 | com.bing |
69 | 19419878 | 174 | 0.000152 | com.imdb |
70 | 19385956 | 186 | 0.000140 | com.imgur |
71 | 19372520 | 236 | 0.000112 | com.washingtonpost |
72 | 19371076 | 176 | 0.000150 | com.theguardian |
73 | 19356952 | 254 | 0.000102 | com.wsj |
74 | 19356474 | 210 | 0.000123 | org.wikimedia |
75 | 19352128 | 219 | 0.000117 | com.businessinsider |
76 | 19347698 | 209 | 0.000123 | com.stackoverflow |
77 | 19342712 | 409 | 0.000065 | com.msn |
78 | 19326654 | 327 | 0.000079 | com.appspot |
79 | 19324334 | 157 | 0.000172 | int.who |
80 | 19321112 | 216 | 0.000119 | edu.stanford |
81 | 19316796 | 179 | 0.000148 | org.apache |
82 | 19310390 | 333 | 0.000078 | com.ibm |
83 | 19309354 | 337 | 0.000077 | edu.mit |
84 | 19304938 | 225 | 0.000116 | net.sourceforge |
85 | 19292932 | 116 | 0.000288 | com.ytimg |
86 | 19287812 | 57 | 0.000649 | net.fbcdn |
87 | 19282486 | 285 | 0.000091 | com.techcrunch |
88 | 19276500 | 269 | 0.000094 | com.bbc |
89 | 19275480 | 155 | 0.000181 | com.wixsite |
90 | 19275222 | 152 | 0.000189 | gov.nih |
91 | 19275200 | 220 | 0.000117 | com.livejournal |
92 | 19270650 | 233 | 0.000113 | uk.co.google |
93 | 19270610 | 440 | 0.000062 | gov.nasa |
94 | 19263354 | 54 | 0.000666 | com.googleadservices |
95 | 19243404 | 262 | 0.000097 | edu.harvard |
96 | 19243154 | 270 | 0.000094 | com.oracle |
97 | 19243126 | 276 | 0.000093 | org.acm |
98 | 19238650 | 218 | 0.000117 | org.ietf |
99 | 19238450 | 185 | 0.000142 | com.blogger |
100 | 19238426 | 223 | 0.000116 | gov.ca |
101 | 19234630 | 465 | 0.000059 | fr.free |
102 | 19232058 | 259 | 0.000098 | com.bloomberg |
103 | 19221844 | 275 | 0.000093 | com.android |
104 | 19218636 | 304 | 0.000085 | com.live |
105 | 19210812 | 126 | 0.000271 | com.jimdo |
106 | 19208896 | 169 | 0.000159 | com.issuu |
107 | 19205802 | 166 | 0.000162 | com.giphy |
108 | 19194156 | 438 | 0.000062 | com.ted |
109 | 19190178 | 348 | 0.000075 | com.huffingtonpost |
110 | 19187782 | 130 | 0.000254 | com.weibo |
111 | 19186862 | 154 | 0.000186 | us.zoom |
112 | 19185794 | 252 | 0.000103 | org.gnu |
113 | 19176332 | 403 | 0.000066 | com.myspace |
114 | 19162122 | 1039 | 0.000030 | com.wikia |
115 | 19152582 | 373 | 0.000071 | net.researchgate |
116 | 19150058 | 343 | 0.000075 | com.usatoday |
117 | 19148332 | 309 | 0.000084 | com.reuters |
118 | 19143988 | 400 | 0.000067 | uk.co.telegraph |
119 | 19141202 | 446 | 0.000061 | com.latimes |
120 | 19130976 | 372 | 0.000071 | com.example |
121 | 19129552 | 345 | 0.000075 | com.githubusercontent |
122 | 19127344 | 93 | 0.000366 | com.unpkg |
123 | 19127116 | 384 | 0.000069 | com.nature |
124 | 19125396 | 336 | 0.000077 | com.wired |
125 | 19124320 | 25 | 0.001485 | com.wixstatic |
126 | 19114842 | 299 | 0.000087 | org.npr |
127 | 19111018 | 308 | 0.000084 | com.cnbc |
128 | 19107772 | 328 | 0.000079 | com.ebay |
129 | 19103704 | 293 | 0.000088 | com.wiley |
130 | 19102814 | 111 | 0.000299 | de.google |
131 | 19097732 | 191 | 0.000135 | com.npmjs |
132 | 19095454 | 344 | 0.000075 | com.hp |
133 | 19088550 | 539 | 0.000050 | com.cisco |
134 | 19084048 | 932 | 0.000034 | com.stackexchange |
135 | 19081736 | 132 | 0.000251 | com.youtube-nocookie |
136 | 19080638 | 134 | 0.000250 | com.ft |
137 | 19078814 | 213 | 0.000120 | org.ampproject |
138 | 19077232 | 532 | 0.000051 | com.steampowered |
139 | 19074638 | 365 | 0.000072 | com.patreon |
140 | 19072918 | 455 | 0.000061 | com.theatlantic |
141 | 19072880 | 476 | 0.000057 | com.gitlab |
142 | 19072344 | 890 | 0.000035 | com.pcmag |
143 | 19068436 | 195 | 0.000131 | com.unsplash |
144 | 19065494 | 877 | 0.000036 | edu.psu |
145 | 19063926 | 376 | 0.000070 | com.time |
146 | 19061142 | 208 | 0.000125 | com.twimg |
147 | 19061064 | 164 | 0.000165 | com.yelp |
148 | 19059332 | 873 | 0.000036 | edu.washington |
149 | 19057196 | 533 | 0.000051 | edu.cornell |
150 | 19054152 | 148 | 0.000197 | com.dropbox |
151 | 19051738 | 603 | 0.000046 | org.arxiv |
152 | 19047626 | 379 | 0.000070 | com.statista |
153 | 19043050 | 324 | 0.000080 | org.un |
154 | 19042602 | 249 | 0.000104 | com.bandcamp |
155 | 19040914 | 824 | 0.000038 | com.venturebeat |
156 | 19040684 | 75 | 0.000432 | me.fb |
157 | 19039882 | 841 | 0.000037 | org.chromium |
158 | 19033464 | 65 | 0.000519 | com.wix |
159 | 19026244 | 284 | 0.000092 | com.sciencedirect |
160 | 19019766 | 629 | 0.000045 | edu.yale |
161 | 19016326 | 584 | 0.000047 | com.pexels |
162 | 19015230 | 826 | 0.000038 | org.bitbucket |
163 | 19010452 | 832 | 0.000038 | org.ieee |
164 | 19007636 | 388 | 0.000068 | com.springer |
165 | 19001810 | 765 | 0.000041 | com.evernote |
166 | 18997506 | 855 | 0.000037 | edu.upenn |
167 | 18994926 | 258 | 0.000098 | jp.ameblo |
168 | 18993772 | 149 | 0.000195 | me.t |
169 | 18992834 | 416 | 0.000065 | org.hbr |
170 | 18992028 | 296 | 0.000088 | com.outlook |
171 | 18985954 | 168 | 0.000160 | jp.co.yahoo |
172 | 18983238 | 577 | 0.000048 | com.cbsnews |
173 | 18982546 | 792 | 0.000040 | me.about |
174 | 18981228 | 891 | 0.000035 | com.git-scm |
175 | 18980336 | 829 | 0.000038 | com.economist |
176 | 18980328 | 150 | 0.000193 | com.opera |
177 | 18978056 | 138 | 0.000223 | me.line |
178 | 18974996 | 450 | 0.000061 | com.goodreads |
179 | 18973364 | 645 | 0.000044 | com.mysql |
180 | 18973114 | 842 | 0.000037 | com.docker |
181 | 18969708 | 562 | 0.000048 | com.buzzfeed |
182 | 18969566 | 565 | 0.000048 | com.mashable |
183 | 18968398 | 587 | 0.000047 | com.mozilla |
184 | 18964540 | 951 | 0.000034 | com.about |
185 | 18962632 | 797 | 0.000040 | org.worldbank |
186 | 18956128 | 815 | 0.000039 | com.newyorker |
187 | 18954668 | 342 | 0.000076 | com.dribbble |
188 | 18954236 | 265 | 0.000096 | net.behance |
189 | 18951876 | 390 | 0.000068 | com.theverge |
190 | 18951838 | 501 | 0.000054 | gov.whitehouse |
191 | 18950142 | 456 | 0.000061 | uk.co.dailymail |
192 | 18943890 | 347 | 0.000075 | com.xinhuanet |
193 | 18942812 | 320 | 0.000080 | com.w3schools |
194 | 18941124 | 378 | 0.000070 | com.fc2 |
195 | 18936488 | 1151 | 0.000027 | edu.wisc |
196 | 18935074 | 764 | 0.000041 | gov.noaa |
197 | 18932396 | 294 | 0.000088 | com.disqus |
198 | 18931228 | 1337 | 0.000023 | co.elastic |
199 | 18927646 | 38 | 0.000956 | com.qq |
200 | 18926694 | 448 | 0.000061 | com.bigcommerce |
201 | 18926442 | 624 | 0.000045 | gov.loc |
202 | 18925620 | 156 | 0.000179 | gov.cdc |
203 | 18924632 | 929 | 0.000035 | gov.fcc |
204 | 18922816 | 136 | 0.000228 | info.aboutads |
205 | 18921630 | 821 | 0.000039 | com.qz |
206 | 18921308 | 2295 | 0.000015 | com.wikidot |
207 | 18919240 | 385 | 0.000069 | com.scribd |
208 | 18915104 | 748 | 0.000042 | org.unesco |
209 | 18914418 | 959 | 0.000033 | com.apnews |
210 | 18912426 | 375 | 0.000070 | com.digg |
211 | 18911082 | 779 | 0.000040 | com.vox |
212 | 18910370 | 180 | 0.000147 | com.amazon-adsystem |
213 | 18910110 | 272 | 0.000094 | com.squareup |
214 | 18907410 | 495 | 0.000054 | uk.co.independent |
215 | 18906224 | 256 | 0.000100 | org.iana |
216 | 18905608 | 1251 | 0.000025 | edu.uchicago |
217 | 18901398 | 420 | 0.000064 | com.force |
218 | 18898702 | 646 | 0.000044 | com.usnews |
219 | 18898108 | 647 | 0.000044 | com.gartner |
220 | 18894918 | 295 | 0.000088 | com.nbcnews |
221 | 18890160 | 470 | 0.000058 | com.dailymotion |
222 | 18883488 | 1004 | 0.000031 | com.dropboxusercontent |
223 | 18878276 | 617 | 0.000045 | org.pbs |
224 | 18876454 | 181 | 0.000147 | jp.co.google |
225 | 18876164 | 113 | 0.000292 | com.sharethis |
226 | 18875824 | 467 | 0.000059 | com.nationalgeographic |
227 | 18874112 | 811 | 0.000039 | uk.co.blogspot |
228 | 18873340 | 844 | 0.000037 | au.net.abc |
229 | 18868000 | 934 | 0.000034 | com.foxnews |
230 | 18865322 | 1559 | 0.000020 | org.eclipse |
231 | 18859464 | 399 | 0.000067 | com.getpocket |
232 | 18859228 | 947 | 0.000034 | com.slate |
233 | 18859062 | 266 | 0.000095 | org.doi |
234 | 18858866 | 63 | 0.000541 | com.fb |
235 | 18856638 | 968 | 0.000033 | com.politico |
236 | 18849992 | 907 | 0.000035 | com.playstation |
237 | 18849334 | 600 | 0.000046 | org.semver |
238 | 18848468 | 1565 | 0.000020 | gd.is |
239 | 18847004 | 1311 | 0.000024 | edu.unc |
240 | 18846758 | 1523 | 0.000021 | org.kernel |
241 | 18846310 | 839 | 0.000037 | org.sciencemag |
242 | 18846038 | 257 | 0.000099 | com.typepad |
243 | 18844998 | 1152 | 0.000027 | com.hatenablog |
244 | 18844004 | 1981 | 0.000018 | com.googlesource |
245 | 18842180 | 202 | 0.000128 | com.naver |
246 | 18840548 | 248 | 0.000104 | com.feedburner |
247 | 18839830 | 1028 | 0.000030 | edu.umn |
248 | 18837518 | 421 | 0.000064 | com.ecwid |
249 | 18833048 | 332 | 0.000078 | net.windows |
250 | 18831042 | 914 | 0.000035 | com.trello |
251 | 18829176 | 554 | 0.000049 | com.tandfonline |
252 | 18829172 | 1369 | 0.000023 | cn.com.chinadaily |
253 | 18828382 | 189 | 0.000138 | org.allaboutcookies |
254 | 18825844 | 746 | 0.000042 | gov.senate |
255 | 18823946 | 119 | 0.000286 | com.paypalobjects |
256 | 18819980 | 1005 | 0.000031 | ly.ow |
257 | 18818724 | 2014 | 0.000017 | org.tensorflow |
258 | 18818710 | 901 | 0.000035 | edu.umich |
259 | 18817936 | 291 | 0.000089 | com.tinyurl |
260 | 18817212 | 479 | 0.000056 | org.pewresearch |
261 | 18815000 | 76 | 0.000423 | com.list-manage |
262 | 18811132 | 239 | 0.000111 | com.wpengine |
263 | 18806908 | 834 | 0.000038 | ca.cbc |
264 | 18805144 | 740 | 0.000043 | co.ibb |
265 | 18804044 | 477 | 0.000057 | gov.fda |
266 | 18802934 | 222 | 0.000117 | com.eepurl |
267 | 18802462 | 318 | 0.000081 | it.google |
268 | 18798744 | 79 | 0.000413 | net.facebook |
269 | 18797046 | 2019 | 0.000017 | com.instructables |
270 | 18795562 | 1200 | 0.000026 | edu.northwestern |
271 | 18794710 | 752 | 0.000042 | org.change |
272 | 18793610 | 394 | 0.000068 | es.google |
273 | 18793484 | 893 | 0.000035 | org.cambridge |
274 | 18790202 | 251 | 0.000103 | com.calendly |
275 | 18784862 | 962 | 0.000033 | gov.congress |
276 | 18784862 | 1022 | 0.000030 | uk.co.guardian |
277 | 18782014 | 555 | 0.000049 | com.bigcartel |
278 | 18777808 | 1348 | 0.000023 | org.semanticscholar |
279 | 18776340 | 1006 | 0.000031 | com.gumroad |
280 | 18775690 | 637 | 0.000044 | org.plos |
281 | 18774956 | 1341 | 0.000023 | com.nikkei |
282 | 18773712 | 313 | 0.000083 | com.optimizely |
283 | 18772988 | 405 | 0.000066 | com.googlecode |
284 | 18766674 | 896 | 0.000035 | gov.justice |
285 | 18764788 | 1044 | 0.000029 | com.huffpost |
286 | 18764312 | 153 | 0.000186 | com.addtoany |
287 | 18763408 | 398 | 0.000067 | me.m |
288 | 18761658 | 80 | 0.000403 | com.wsimg |
289 | 18760046 | 411 | 0.000065 | com.tripod |
290 | 18754884 | 957 | 0.000033 | ee.linktr |
291 | 18754526 | 1021 | 0.000030 | gov.usgs |
292 | 18753164 | 1459 | 0.000021 | uk.co.wired |
293 | 18752728 | 338 | 0.000077 | fr.google |
294 | 18751846 | 1059 | 0.000029 | com.500px |
295 | 18751636 | 452 | 0.000061 | ca.google |
296 | 18749418 | 1996 | 0.000017 | com.amd |
297 | 18744444 | 1944 | 0.000018 | com.azure |
298 | 18742964 | 777 | 0.000040 | au.com.google |
299 | 18742506 | 481 | 0.000056 | com.163 |
300 | 18741292 | 1091 | 0.000028 | com.ssrn |
301 | 18740758 | 1065 | 0.000029 | com.newsweek |
302 | 18734910 | 1688 | 0.000019 | ca.utoronto |
303 | 18734620 | 139 | 0.000218 | com.spotify |
304 | 18731112 | 744 | 0.000042 | cn.com.people |
305 | 18730384 | 334 | 0.000078 | page.g |
306 | 18730074 | 2751 | 0.000012 | com.nabble |
307 | 18728400 | 1454 | 0.000021 | com.howstuffworks |
308 | 18722938 | 2107 | 0.000016 | com.lego |
309 | 18719762 | 1675 | 0.000019 | com.storify |
310 | 18719332 | 1140 | 0.000027 | uk.co.thetimes |
311 | 18717930 | 801 | 0.000039 | site.business |
312 | 18717726 | 884 | 0.000036 | uk.ac.ox |
313 | 18716206 | 311 | 0.000083 | com.bitly |
314 | 18715060 | 1218 | 0.000026 | com.scmp |
315 | 18713618 | 798 | 0.000040 | com.adage |
316 | 18713552 | 654 | 0.000044 | com.indiatimes |
317 | 18712564 | 1908 | 0.000018 | de.mpg |
318 | 18712368 | 1057 | 0.000029 | com.thehill |
319 | 18705466 | 519 | 0.000052 | com.criteo |
320 | 18704754 | 1078 | 0.000028 | org.ohchr |
321 | 18704474 | 1531 | 0.000020 | com.aljazeera |
322 | 18703348 | 802 | 0.000039 | uk.gov.service |
323 | 18701482 | 1545 | 0.000020 | org.greenpeace |
324 | 18699064 | 331 | 0.000078 | com.netdna-ssl |
325 | 18698378 | 967 | 0.000033 | ch.google |
326 | 18693994 | 784 | 0.000040 | us.icio |
327 | 18693690 | 1153 | 0.000027 | int.coe |
328 | 18692556 | 933 | 0.000034 | org.d3js |
329 | 18690456 | 1499 | 0.000021 | com.history |
330 | 18689794 | 1018 | 0.000030 | com.netlify |
331 | 18688064 | 1320 | 0.000023 | com.nymag |
332 | 18687064 | 1363 | 0.000023 | org.wiktionary |
333 | 18684868 | 287 | 0.000091 | ru.ok |
334 | 18683792 | 1293 | 0.000024 | com.intuit |
335 | 18682796 | 1419 | 0.000022 | uk.co.standard |
336 | 18681388 | 1995 | 0.000017 | edu.arizona |
337 | 18679058 | 944 | 0.000034 | gov.archives |
338 | 18678794 | 953 | 0.000034 | ru.google |
339 | 18677084 | 1054 | 0.000029 | sg.com.google |
340 | 18675890 | 900 | 0.000035 | br.com.google |
341 | 18674402 | 85 | 0.000385 | co.g |
342 | 18674068 | 1975 | 0.000018 | com.wattpad |
343 | 18673754 | 526 | 0.000051 | ru.gov |
344 | 18673370 | 1351 | 0.000023 | com.ikea |
345 | 18668598 | 1461 | 0.000021 | com.reverbnation |
346 | 18668444 | 2681 | 0.000013 | edu.drexel |
347 | 18668276 | 1121 | 0.000027 | edu.si |
348 | 18666994 | 1174 | 0.000027 | uk.co.mirror |
349 | 18666846 | 2572 | 0.000013 | org.maven |
350 | 18666724 | 412 | 0.000065 | com.cnet |
351 | 18664542 | 580 | 0.000048 | org.openstreetmap |
352 | 18663710 | 1373 | 0.000023 | com.jetbrains |
353 | 18663688 | 1032 | 0.000030 | com.theconversation |
354 | 18663542 | 1921 | 0.000018 | com.newscientist |
355 | 18661472 | 847 | 0.000037 | gov.state |
356 | 18661146 | 1572 | 0.000020 | ms.1drv |
357 | 18660152 | 2644 | 0.000013 | com.mystrikingly |
358 | 18655360 | 973 | 0.000032 | org.fao |
359 | 18654458 | 590 | 0.000047 | cn.google |
360 | 18653472 | 235 | 0.000112 | com.etsy |
361 | 18652232 | 1485 | 0.000021 | com.flipboard |
362 | 18651820 | 767 | 0.000041 | com.deviantart |
363 | 18651514 | 1375 | 0.000023 | com.thedailybeast |
364 | 18651404 | 1220 | 0.000026 | org.jstor |
365 | 18649024 | 1270 | 0.000024 | com.strikingly |
366 | 18647422 | 2045 | 0.000017 | blog.home |
367 | 18646812 | 634 | 0.000044 | com.zdnet |
368 | 18644828 | 325 | 0.000079 | tv.twitch |
369 | 18642272 | 2781 | 0.000012 | com.diigo |
370 | 18640482 | 1123 | 0.000027 | com.britannica |
371 | 18639254 | 1904 | 0.000018 | ca.ubc |
372 | 18638840 | 367 | 0.000072 | com.jotform |
373 | 18635188 | 1959 | 0.000018 | com.gettyimages |
374 | 18634254 | 1685 | 0.000019 | com.channel4 |
375 | 18631278 | 1494 | 0.000021 | org.pypi |
376 | 18630386 | 813 | 0.000039 | in.co.google |
377 | 18627814 | 417 | 0.000064 | com.ssl-images-amazon |
378 | 18626978 | 161 | 0.000166 | gle.forms |
379 | 18623310 | 1982 | 0.000018 | org.hrw |
380 | 18623132 | 281 | 0.000092 | com.cloudinary |
381 | 18618612 | 1382 | 0.000022 | au.com.smh |
382 | 18617234 | 1566 | 0.000020 | uk.co.metro |
383 | 18617180 | 2031 | 0.000017 | hk.com.google |
384 | 18617072 | 1599 | 0.000020 | edu.ufl |
385 | 18613590 | 2332 | 0.000015 | ly.rebrand |
386 | 18612786 | 457 | 0.000061 | net.imgix |
387 | 18609746 | 418 | 0.000064 | com.webflow |
388 | 18609050 | 2311 | 0.000015 | com.shutterfly |
389 | 18607782 | 568 | 0.000048 | com.feedly |
390 | 18603850 | 538 | 0.000050 | gov.epa |
391 | 18602470 | 104 | 0.000348 | com.stripe |
392 | 18601118 | 83 | 0.000391 | net.jsfiddle |
393 | 18599796 | 3423 | 0.000010 | org.aclweb |
394 | 18597166 | 2348 | 0.000014 | com.yarnpkg |
395 | 18596278 | 69 | 0.000461 | net.akamaihd |
396 | 18596202 | 1907 | 0.000018 | gov.supremecourt |
397 | 18595244 | 2344 | 0.000014 | com.thefreedictionary |
398 | 18593816 | 468 | 0.000058 | nl.google |
399 | 18592072 | 1578 | 0.000020 | com.dw |
400 | 18588294 | 2955 | 0.000012 | com.upi |
401 | 18587932 | 981 | 0.000032 | com.thelancet |
402 | 18587926 | 425 | 0.000064 | com.slack |
403 | 18587680 | 396 | 0.000067 | com.kickstarter |
404 | 18587378 | 787 | 0.000040 | com.urldefense |
405 | 18585950 | 1713 | 0.000019 | ca.sfu |
406 | 18583582 | 460 | 0.000060 | com.livechatinc |
407 | 18581082 | 623 | 0.000045 | com.quora |
408 | 18580964 | 428 | 0.000063 | com.rackcdn |
409 | 18580620 | 1967 | 0.000018 | com.euronews |
410 | 18580552 | 451 | 0.000061 | com.go |
411 | 18580130 | 1368 | 0.000023 | com.tunein |
412 | 18578076 | 594 | 0.000046 | ru.liveinternet |
413 | 18576712 | 475 | 0.000057 | com.googleblog |
414 | 18571776 | 2597 | 0.000013 | pt.sapo |
415 | 18571212 | 2109 | 0.000016 | com.itv |
416 | 18570630 | 1945 | 0.000018 | uk.co.huffingtonpost |
417 | 18570542 | 1286 | 0.000024 | edu.brookings |
418 | 18570528 | 4423 | 0.000008 | tl.page |
419 | 18570058 | 2369 | 0.000014 | com.angelfire |
420 | 18568882 | 2614 | 0.000013 | org.wikibooks |
421 | 18567302 | 1692 | 0.000019 | com.ifttt |
422 | 18564134 | 861 | 0.000036 | com.freepik |
423 | 18563246 | 2244 | 0.000015 | com.netvibes |
424 | 18562602 | 133 | 0.000251 | com.mailchimp |
425 | 18562564 | 364 | 0.000072 | me.telegram |
426 | 18562400 | 561 | 0.000048 | com.microsoftonline |
427 | 18562224 | 1976 | 0.000018 | uk.co.express |
428 | 18559206 | 2888 | 0.000012 | sg.edu.nus |
429 | 18559092 | 1928 | 0.000018 | io.webflow |
430 | 18557292 | 772 | 0.000041 | pl.google |
431 | 18555900 | 480 | 0.000056 | com.meetup |
432 | 18555482 | 4752 | 0.000007 | com.newgrounds |
433 | 18554944 | 2397 | 0.000014 | google.ai |
434 | 18554512 | 2439 | 0.000014 | com.yolasite |
435 | 18553912 | 2124 | 0.000016 | jp.geocities |
436 | 18552986 | 3394 | 0.000011 | com.instapaper |
437 | 18551338 | 362 | 0.000072 | com.proofpoint |
438 | 18548844 | 1358 | 0.000023 | com.people |
439 | 18546296 | 64 | 0.000531 | net.typekit |
440 | 18543694 | 2104 | 0.000016 | org.c-span |
441 | 18541918 | 159 | 0.000169 | ru.mail |
442 | 18541834 | 2043 | 0.000017 | com.avg |
443 | 18540650 | 2249 | 0.000015 | app.netlify |
444 | 18539394 | 3004 | 0.000011 | com.000webhostapp |
445 | 18539316 | 485 | 0.000055 | com.elsevier |
446 | 18538008 | 3494 | 0.000010 | cn.edu.pku |
447 | 18536872 | 1609 | 0.000020 | com.asahi |
448 | 18535422 | 876 | 0.000036 | org.worldwildlife |
449 | 18535204 | 1127 | 0.000027 | uk.parliament |
450 | 18534822 | 1956 | 0.000018 | uk.gov.ons |
451 | 18533694 | 188 | 0.000138 | com.iubenda |
452 | 18532790 | 2113 | 0.000016 | org.documentcloud |
453 | 18532338 | 3074 | 0.000011 | uk.co.timesonline |
454 | 18531118 | 264 | 0.000096 | com.office |
455 | 18527764 | 237 | 0.000112 | com.eventbrite |
456 | 18527012 | 2699 | 0.000013 | com.self |
457 | 18526172 | 2511 | 0.000013 | com.foreignpolicy |
458 | 18524804 | 2421 | 0.000014 | org.sundance |
459 | 18524702 | 214 | 0.000120 | com.aliyuncs |
460 | 18524140 | 1213 | 0.000026 | be.google |
461 | 18523242 | 2200 | 0.000016 | ie.google |
462 | 18523000 | 1432 | 0.000022 | gov.weather |
463 | 18522694 | 3136 | 0.000011 | com.openai |
464 | 18522588 | 879 | 0.000036 | org.mediawiki |
465 | 18521124 | 2806 | 0.000012 | com.pearltrees |
466 | 18520306 | 1704 | 0.000019 | com.firebaseapp |
467 | 18516520 | 3620 | 0.000010 | com.dailycaller |
468 | 18514512 | 498 | 0.000054 | it.placehold |
469 | 18514168 | 2695 | 0.000013 | com.france24 |
470 | 18513026 | 644 | 0.000044 | edu.berkeley |
471 | 18512138 | 492 | 0.000055 | cn.360 |
472 | 18511428 | 2296 | 0.000015 | com.msnbc |
473 | 18510986 | 2089 | 0.000017 | com.thestar |
474 | 18510258 | 3732 | 0.000009 | me.site123 |
475 | 18509392 | 2133 | 0.000016 | com.gfycat |
476 | 18508906 | 341 | 0.000076 | com.rawgit |
477 | 18507920 | 521 | 0.000052 | com.gmail |
478 | 18507686 | 1952 | 0.000018 | org.ocks |
479 | 18506872 | 2739 | 0.000012 | org.rsc |
480 | 18504310 | 2486 | 0.000014 | edu.hawaii |
481 | 18503766 | 2366 | 0.000014 | de.br |
482 | 18503250 | 2447 | 0.000014 | edu.colostate |
483 | 18502578 | 171 | 0.000154 | com.zendesk |
484 | 18501424 | 2222 | 0.000015 | org.nobelprize |
485 | 18501096 | 3293 | 0.000011 | net.pixnet |
486 | 18500188 | 1528 | 0.000020 | net.seesaa |
487 | 18500164 | 2471 | 0.000014 | com.motherjones |
488 | 18499720 | 756 | 0.000042 | com.vice |
489 | 18499378 | 4234 | 0.000008 | com.masslive |
490 | 18496634 | 2355 | 0.000014 | com.cision |
491 | 18495058 | 101 | 0.000361 | com.godaddy |
492 | 18492104 | 886 | 0.000036 | gov.nist |
493 | 18491956 | 1249 | 0.000025 | org.ilo |
494 | 18490654 | 2070 | 0.000017 | com.surveygizmo |
495 | 18490628 | 3378 | 0.000011 | com.minds |
496 | 18490576 | 635 | 0.000044 | com.matterport |
497 | 18489858 | 2656 | 0.000013 | ph.com.google |
498 | 18488106 | 369 | 0.000071 | org.python |
499 | 18487032 | 980 | 0.000032 | gov.va |
500 | 18485800 | 1166 | 0.000027 | at.google |
501 | 18485152 | 1318 | 0.000023 | se.google |
502 | 18483644 | 1961 | 0.000018 | ru.ucoz |
503 | 18482996 | 2401 | 0.000014 | com.freep |
504 | 18482190 | 3874 | 0.000009 | com.wizards |
505 | 18481738 | 3583 | 0.000010 | edu.uvm |
506 | 18478142 | 3711 | 0.000010 | org.tvtropes |
507 | 18476988 | 1506 | 0.000021 | com.cognitoforms |
508 | 18476516 | 1493 | 0.000021 | gov.uscourts |
509 | 18476024 | 3530 | 0.000010 | org.oxfam |
510 | 18473992 | 2235 | 0.000015 | cn.t |
511 | 18473054 | 4331 | 0.000008 | fm.ask |
512 | 18473034 | 1708 | 0.000019 | dk.google |
513 | 18470526 | 3122 | 0.000011 | de.dw |
514 | 18467204 | 2009 | 0.000017 | ua.com.google |
515 | 18467126 | 3935 | 0.000009 | com.youdao |
516 | 18464016 | 128 | 0.000262 | org.networkadvertising |
517 | 18462968 | 1031 | 0.000030 | com.arstechnica |
518 | 18462674 | 2310 | 0.000015 | int.unfccc |
519 | 18461844 | 3323 | 0.000011 | ch.nzz |
520 | 18460156 | 123 | 0.000276 | com.statcounter |
521 | 18460126 | 3757 | 0.000009 | net.hinet |
522 | 18460018 | 2484 | 0.000014 | com.washingtontimes |
523 | 18459778 | 3391 | 0.000011 | edu.miami |
524 | 18459648 | 5025 | 0.000007 | tw.com.gamer |
525 | 18459120 | 4313 | 0.000008 | ch.qos |
526 | 18458774 | 788 | 0.000040 | com.intel |
527 | 18456584 | 2220 | 0.000015 | mx.com.google |
528 | 18455734 | 2241 | 0.000015 | gov.ky |
529 | 18455504 | 3426 | 0.000010 | com.nwsource |
530 | 18454948 | 856 | 0.000037 | io.readthedocs |
531 | 18453730 | 2187 | 0.000016 | gov.cisa |
532 | 18451988 | 2256 | 0.000015 | com.straitstimes |
533 | 18449466 | 371 | 0.000071 | io.codepen |
534 | 18447006 | 361 | 0.000072 | com.prnewswire |
535 | 18446224 | 4097 | 0.000009 | com.smore |
536 | 18446132 | 2188 | 0.000016 | pt.google |
537 | 18445920 | 2719 | 0.000012 | net.bplaced |
538 | 18445802 | 5349 | 0.000007 | net.wargaming |
539 | 18445232 | 3272 | 0.000011 | org.csis |
540 | 18444732 | 1435 | 0.000022 | org.aarp |
541 | 18444080 | 289 | 0.000090 | net.php |
542 | 18443758 | 2282 | 0.000015 | no.google |
543 | 18443228 | 3924 | 0.000009 | com.steemit |
544 | 18443146 | 1304 | 0.000024 | tw.com.google |
545 | 18442018 | 314 | 0.000083 | com.squarespace |
546 | 18440872 | 743 | 0.000043 | com.oreilly |
547 | 18440596 | 199 | 0.000130 | com.hubspot |
548 | 18439354 | 4877 | 0.000007 | com.bonanza |
549 | 18438802 | 2020 | 0.000017 | co.lpages |
550 | 18438606 | 1079 | 0.000028 | net.ovh |
551 | 18438208 | 835 | 0.000037 | com.imageshack |
552 | 18437874 | 4023 | 0.000009 | com.doodlekit |
553 | 18436818 | 2425 | 0.000014 | com.voanews |
554 | 18436680 | 358 | 0.000073 | ru.rambler |
555 | 18436048 | 2805 | 0.000012 | com.nationalpost |
556 | 18435420 | 4534 | 0.000008 | by.google |
557 | 18435256 | 614 | 0.000045 | org.nodejs |
558 | 18435200 | 397 | 0.000067 | com.onesignal |
559 | 18434470 | 3374 | 0.000011 | fr.rfi |
560 | 18434466 | 463 | 0.000060 | gov.irs |
561 | 18434444 | 2584 | 0.000013 | com.snopes |
562 | 18434230 | 1899 | 0.000018 | link.page |
563 | 18434190 | 3637 | 0.000010 | org.vim |
564 | 18434018 | 2240 | 0.000015 | th.co.google |
565 | 18433782 | 3395 | 0.000010 | org.scala-lang |
566 | 18432434 | 3142 | 0.000011 | com.inquirer |
567 | 18430898 | 2887 | 0.000012 | org.ballotpedia |
568 | 18430888 | 3324 | 0.000011 | com.real |
569 | 18428600 | 649 | 0.000044 | br.com.uol |
570 | 18428004 | 513 | 0.000052 | com.pixabay |
571 | 18426658 | 2142 | 0.000016 | uk.co.which |
572 | 18426634 | 4070 | 0.000009 | com.viki |
573 | 18425674 | 1038 | 0.000030 | com.thenextweb |
574 | 18424302 | 3146 | 0.000011 | org.aps |
575 | 18424050 | 2764 | 0.000012 | com.post-gazette |
576 | 18423516 | 2499 | 0.000014 | net.openid |
577 | 18422702 | 2627 | 0.000013 | edu.usf |
578 | 18421138 | 82 | 0.000391 | com.livestream |
579 | 18420414 | 961 | 0.000033 | jp.shinobi |
580 | 18420272 | 956 | 0.000033 | int.wipo |
581 | 18417146 | 4450 | 0.000008 | com.bravesites |
582 | 18415542 | 2881 | 0.000012 | ru.aif |
583 | 18414574 | 2906 | 0.000012 | io.gitlab |
584 | 18414284 | 3387 | 0.000011 | org.pri |
585 | 18414276 | 1932 | 0.000018 | gov.ct |
586 | 18413984 | 2602 | 0.000013 | il.co.google |
587 | 18413906 | 1910 | 0.000018 | org.oxfordjournals |
588 | 18413218 | 4664 | 0.000008 | com.ucoz |
589 | 18412422 | 566 | 0.000048 | com.photobucket |
590 | 18412344 | 2191 | 0.000016 | com.xrea |
591 | 18412198 | 2234 | 0.000015 | nz.co.google |
592 | 18410920 | 2088 | 0.000017 | net.cnki |
593 | 18410828 | 2847 | 0.000012 | com.webbyawards |
594 | 18410164 | 433 | 0.000063 | com.staticflickr |
595 | 18409934 | 3675 | 0.000010 | org.heritage |
596 | 18408908 | 1993 | 0.000018 | tr.com.google |
597 | 18408574 | 2053 | 0.000017 | com.treehugger |
598 | 18406062 | 1695 | 0.000019 | net.leadpages |
599 | 18405282 | 2112 | 0.000016 | fi.google |
600 | 18402764 | 5153 | 0.000007 | kz.google |
601 | 18402708 | 211 | 0.000121 | to.amzn |
602 | 18402670 | 569 | 0.000048 | com.deloitte |
603 | 18402662 | 1100 | 0.000028 | cz.google |
604 | 18402526 | 4562 | 0.000008 | com.freehostia |
605 | 18402334 | 2156 | 0.000016 | gov.faa |
606 | 18402326 | 2724 | 0.000012 | com.detroitnews |
607 | 18402220 | 2774 | 0.000012 | com.slidesharecdn |
608 | 18402102 | 346 | 0.000075 | com.adnxs |
609 | 18396726 | 812 | 0.000039 | com.thinkwithgoogle |
610 | 18392816 | 1471 | 0.000021 | com.trustwave |
611 | 18392376 | 2640 | 0.000013 | org.iea |
612 | 18392262 | 2883 | 0.000012 | jp.blog |
613 | 18391148 | 4426 | 0.000008 | com.goal |
614 | 18390184 | 3284 | 0.000011 | com.financialpost |
615 | 18389140 | 3636 | 0.000010 | net.alarabiya |
616 | 18389082 | 3570 | 0.000010 | org.neocities |
617 | 18388580 | 3784 | 0.000009 | co.ello |
618 | 18388256 | 207 | 0.000126 | com.salesforce |
619 | 18386478 | 3500 | 0.000010 | com.archdaily |
620 | 18385984 | 4517 | 0.000008 | com.alamy |
621 | 18385924 | 2297 | 0.000015 | gr.google |
622 | 18385398 | 160 | 0.000168 | gov.privacyshield |
623 | 18385020 | 2569 | 0.000013 | org.kqed |
624 | 18383196 | 277 | 0.000093 | org.drupal |
625 | 18382110 | 354 | 0.000074 | com.snapchat |
626 | 18381496 | 2338 | 0.000015 | ro.google |
627 | 18381392 | 3367 | 0.000011 | uk.ac.leeds |
628 | 18381316 | 271 | 0.000094 | com.mapbox |
629 | 18380144 | 3907 | 0.000009 | uk.gov.scotland |
630 | 18379620 | 1946 | 0.000018 | hu.google |
631 | 18378224 | 4399 | 0.000008 | co.aeon |
632 | 18377446 | 374 | 0.000070 | com.cdninstagram |
633 | 18376062 | 3545 | 0.000010 | gov.fec |
634 | 18376022 | 3312 | 0.000011 | com.virgin |
635 | 18375628 | 2219 | 0.000015 | ar.com.google |
636 | 18375060 | 4128 | 0.000009 | cn.globaltimes |
637 | 18374688 | 4333 | 0.000008 | com.corel |
638 | 18374066 | 464 | 0.000059 | com.herokuapp |
639 | 18373200 | 4062 | 0.000009 | jp.go.ndl |
640 | 18373110 | 791 | 0.000040 | google.blog |
641 | 18372316 | 2208 | 0.000016 | com.justia |
642 | 18372216 | 2320 | 0.000015 | za.co.google |
643 | 18370616 | 2216 | 0.000016 | ru.ria |
644 | 18370232 | 3694 | 0.000010 | com.intensedebate |
645 | 18369794 | 3793 | 0.000009 | com.visualcapitalist |
646 | 18369094 | 2722 | 0.000012 | si.google |
647 | 18368512 | 4182 | 0.000008 | com.rediff |
648 | 18367604 | 3834 | 0.000009 | ca.uvic |
649 | 18367236 | 2577 | 0.000013 | ru.rosminzdrav |
650 | 18365918 | 439 | 0.000062 | com.nypost |
651 | 18365880 | 4678 | 0.000008 | org.wikimapia |
652 | 18365350 | 3439 | 0.000010 | com.nationalreview |
653 | 18364962 | 2134 | 0.000016 | uk.org.asa |
654 | 18364282 | 3850 | 0.000009 | tw.edu.ntu |
655 | 18363974 | 598 | 0.000046 | com.samsung |
656 | 18363190 | 2703 | 0.000012 | is.google |
657 | 18362598 | 3869 | 0.000009 | com.podomatic |
658 | 18361242 | 316 | 0.000082 | cn.bshare |
659 | 18360424 | 3484 | 0.000010 | org.wri |
660 | 18360028 | 4160 | 0.000009 | uk.co.spectator |
661 | 18359858 | 1711 | 0.000019 | ly.cutt |
662 | 18358316 | 4989 | 0.000007 | to.gplus |
663 | 18358086 | 4908 | 0.000007 | com.atwebpages |
664 | 18357826 | 177 | 0.000150 | com.tripadvisor |
665 | 18357438 | 5003 | 0.000007 | org.scala-sbt |
666 | 18356488 | 4276 | 0.000008 | ru.msu |
667 | 18356450 | 1161 | 0.000027 | com.udemy |
668 | 18355358 | 2973 | 0.000011 | com.timesofisrael |
669 | 18352506 | 5213 | 0.000007 | edu.csulb |
670 | 18351622 | 4744 | 0.000007 | com.authorstream |
671 | 18350944 | 4127 | 0.000009 | gy.rb |
672 | 18350110 | 3204 | 0.000011 | us.ny.state |
673 | 18349876 | 3644 | 0.000010 | com.linuxquota |
674 | 18349798 | 3563 | 0.000010 | com.udn |
675 | 18349578 | 3845 | 0.000009 | org.jenkins-ci |
676 | 18349508 | 1686 | 0.000019 | com.pcworld |
677 | 18349104 | 2481 | 0.000014 | uk.ac.imperial |
678 | 18348784 | 5238 | 0.000007 | com.etymonline |
679 | 18348026 | 3492 | 0.000010 | eg.com.google |
680 | 18347774 | 3363 | 0.000011 | uk.co.bbci |
681 | 18347338 | 2386 | 0.000014 | com.name |
682 | 18346938 | 3745 | 0.000009 | com.novell |
683 | 18345924 | 1487 | 0.000021 | com.digitaloceanspaces |
684 | 18345376 | 6040 | 0.000006 | net.vingle |
685 | 18345350 | 2615 | 0.000013 | us.pa.state |
686 | 18345040 | 642 | 0.000044 | com.xiti |
687 | 18345006 | 2302 | 0.000015 | fr.pagesjaunes |
688 | 18344246 | 4604 | 0.000008 | by.tut |
689 | 18341982 | 78 | 0.000417 | com.messenger |
690 | 18341502 | 1672 | 0.000019 | id.co.google |
691 | 18341492 | 4012 | 0.000009 | com.donaldjtrump |
692 | 18339724 | 2359 | 0.000014 | co.pcdn |
693 | 18338674 | 606 | 0.000046 | com.indeed |
694 | 18338446 | 459 | 0.000060 | com.sxsw |
695 | 18337870 | 2379 | 0.000014 | sk.google |
696 | 18337126 | 246 | 0.000105 | uk.co.amazon |
697 | 18336826 | 351 | 0.000074 | com.atlassian |
698 | 18336810 | 1225 | 0.000025 | com.dell |
699 | 18336442 | 4947 | 0.000007 | fr.online |
700 | 18336226 | 1933 | 0.000018 | com.law |
701 | 18335648 | 3783 | 0.000009 | com.wmtransfer |
702 | 18335422 | 2242 | 0.000015 | kr.co.google |
703 | 18335402 | 4709 | 0.000008 | edu.odu |
704 | 18335130 | 2971 | 0.000011 | cl.google |
705 | 18335024 | 4300 | 0.000008 | il.ac.huji |
706 | 18334782 | 4271 | 0.000008 | tw.gov.cdc |
707 | 18333794 | 2886 | 0.000012 | my.com.google |
708 | 18333014 | 3385 | 0.000011 | com.scotsman |
709 | 18332864 | 3322 | 0.000011 | com.instructure |
710 | 18332832 | 4563 | 0.000008 | com.hackaday |
711 | 18332194 | 2131 | 0.000016 | gov.pa |
712 | 18332054 | 627 | 0.000045 | com.withgoogle |
713 | 18331108 | 1997 | 0.000017 | scot.gov |
714 | 18330912 | 3178 | 0.000011 | com.broadwayworld |
715 | 18330804 | 858 | 0.000036 | com.canva |
716 | 18330694 | 4525 | 0.000008 | com.mongabay |
717 | 18329802 | 4508 | 0.000008 | com.macobserver |
718 | 18329686 | 3725 | 0.000010 | org.sonatype |
719 | 18328118 | 2391 | 0.000014 | gov.wi |
720 | 18327736 | 2683 | 0.000013 | org.usgbc |
721 | 18327662 | 4113 | 0.000009 | gov.peacecorps |
722 | 18327624 | 4652 | 0.000008 | cn.tianya |
723 | 18326710 | 3495 | 0.000010 | pk.com.google |
724 | 18326302 | 870 | 0.000036 | com.marketwatch |
725 | 18326164 | 1490 | 0.000021 | com.billboard |
726 | 18324976 | 107 | 0.000316 | net.gandi |
727 | 18324878 | 2845 | 0.000012 | com.thecut |
728 | 18324686 | 89 | 0.000372 | me.ogp |
729 | 18323980 | 4585 | 0.000008 | io.meduza |
730 | 18323898 | 2827 | 0.000012 | uk.org.nationaltrust |
731 | 18323758 | 3911 | 0.000009 | au.edu.adelaide |
732 | 18323398 | 4766 | 0.000007 | de.uni-erlangen |
733 | 18322482 | 3759 | 0.000009 | uk.org.rspb |
734 | 18322376 | 3773 | 0.000009 | cv.google |
735 | 18321256 | 5135 | 0.000007 | cat.bcn |
736 | 18319736 | 3728 | 0.000009 | com.ipage |
737 | 18319726 | 5311 | 0.000007 | com.brother |
738 | 18318148 | 2410 | 0.000014 | my.com.thestar |
739 | 18317872 | 3401 | 0.000010 | uk.ac.york |
740 | 18317504 | 3315 | 0.000011 | com.politifact |
741 | 18317408 | 3128 | 0.000011 | ee.google |
742 | 18317178 | 3326 | 0.000011 | org.thinkprogress |
743 | 18317034 | 2102 | 0.000016 | se.haxx |
744 | 18316764 | 4554 | 0.000008 | au.edu.rmit |
745 | 18316272 | 2959 | 0.000011 | hr.google |
746 | 18315296 | 5212 | 0.000007 | com.selfridges |
747 | 18315244 | 3772 | 0.000009 | au.com.telstra |
748 | 18313746 | 1436 | 0.000022 | com.fiverr |
749 | 18313044 | 3420 | 0.000010 | de.hu-berlin |
750 | 18311516 | 3572 | 0.000010 | com.nola |
751 | 18311094 | 3458 | 0.000010 | sa.com.google |
752 | 18310436 | 4145 | 0.000009 | ca.dal |
753 | 18310126 | 6237 | 0.000006 | org.arkive |
754 | 18309422 | 2759 | 0.000012 | bg.google |
755 | 18308696 | 3429 | 0.000010 | com.monday |
756 | 18308664 | 4635 | 0.000008 | at.tugraz |
757 | 18308432 | 3508 | 0.000010 | com.eiseverywhere |
758 | 18308298 | 3764 | 0.000009 | uk.co.cfdr |
759 | 18308102 | 3298 | 0.000011 | org.iucn |
760 | 18307444 | 3571 | 0.000010 | app.web |
761 | 18306932 | 3702 | 0.000010 | org.iucnredlist |
762 | 18306908 | 292 | 0.000088 | com.surveymonkey |
763 | 18306390 | 3806 | 0.000009 | gi.com.google |
764 | 18306038 | 5056 | 0.000007 | ec.com.google |
765 | 18305962 | 3875 | 0.000009 | de.uni-freiburg |
766 | 18305528 | 4244 | 0.000008 | au.com.heraldsun |
767 | 18305222 | 515 | 0.000052 | io.shields |
768 | 18304914 | 610 | 0.000046 | org.eff |
769 | 18304878 | 3829 | 0.000009 | com.psmag |
770 | 18304506 | 4721 | 0.000007 | ua.at |
771 | 18302798 | 930 | 0.000034 | gov.uspto |
772 | 18302648 | 190 | 0.000137 | com.automattic |
773 | 18301286 | 3948 | 0.000009 | com.mozello |
774 | 18300612 | 1108 | 0.000028 | com.gizmodo |
775 | 18300418 | 3596 | 0.000010 | pl.wp |
776 | 18300322 | 3471 | 0.000010 | org.royalsociety |
777 | 18299622 | 2819 | 0.000012 | org.unep |
778 | 18299452 | 3606 | 0.000010 | com.realclearpolitics |
779 | 18298298 | 3531 | 0.000010 | jp.coocan |
780 | 18298296 | 2613 | 0.000013 | vn.com.google |
781 | 18298218 | 4434 | 0.000008 | jp.hatenablog |
782 | 18297896 | 4281 | 0.000008 | com.waitrose |
783 | 18297876 | 4676 | 0.000008 | info.webry |
784 | 18297852 | 4427 | 0.000008 | net.inquirer |
785 | 18297704 | 4274 | 0.000008 | jp.gree |
786 | 18297178 | 4611 | 0.000008 | org.nationalinterest |
787 | 18296330 | 2981 | 0.000011 | edu.uconn |
788 | 18295610 | 946 | 0.000034 | edu.columbia |
789 | 18295554 | 5531 | 0.000006 | org.mises |
790 | 18295452 | 1274 | 0.000024 | com.smashingmagazine |
791 | 18295224 | 3303 | 0.000011 | uk.gov.companieshouse |
792 | 18294866 | 4442 | 0.000008 | gov.ourdocuments |
793 | 18294666 | 3894 | 0.000009 | sl.com.google |
794 | 18292912 | 6218 | 0.000006 | com.rhino3d |
795 | 18292842 | 3435 | 0.000010 | org.cfr |
796 | 18292780 | 790 | 0.000040 | com.airbnb |
797 | 18292712 | 283 | 0.000092 | jp.co.amazon |
798 | 18291570 | 413 | 0.000065 | com.pubmatic |
799 | 18290920 | 878 | 0.000036 | com.box |
800 | 18290426 | 5610 | 0.000006 | com.coroflot |
801 | 18290346 | 4348 | 0.000008 | com.thediplomat |
802 | 18286902 | 4066 | 0.000009 | com.inhabitat |
803 | 18286668 | 3277 | 0.000011 | com.bp |
804 | 18286522 | 4592 | 0.000008 | cat.uab |
805 | 18283480 | 3827 | 0.000009 | uk.co.villiers-london |
806 | 18283014 | 4140 | 0.000009 | org.grist |
807 | 18282452 | 4016 | 0.000009 | com.foreignaffairs |
808 | 18281324 | 1081 | 0.000028 | com.tapad |
809 | 18280378 | 1347 | 0.000023 | org.altervista |
810 | 18280358 | 382 | 0.000069 | com.skype |
811 | 18280324 | 4349 | 0.000008 | com.worldsecuresystems |
812 | 18279680 | 2409 | 0.000014 | com.volusion |
813 | 18279516 | 2907 | 0.000012 | ru.nethouse |
814 | 18279480 | 3527 | 0.000010 | pe.com.google |
815 | 18279438 | 4779 | 0.000007 | be.lesoir |
816 | 18278874 | 3288 | 0.000011 | co.com.google |
817 | 18278816 | 3885 | 0.000009 | de.uni-koeln |
818 | 18278778 | 2910 | 0.000012 | org.gnupg |
819 | 18278022 | 4656 | 0.000008 | com.mihanblog |
820 | 18277554 | 3360 | 0.000011 | org.panda |
821 | 18277186 | 3440 | 0.000010 | lv.google |
822 | 18276674 | 5300 | 0.000007 | lu.google |
823 | 18276442 | 484 | 0.000055 | com.inc |
824 | 18275676 | 5103 | 0.000007 | cn.com.caijing |
825 | 18275134 | 3331 | 0.000011 | uk.gov.metoffice |
826 | 18274258 | 68 | 0.000471 | com.oculus |
827 | 18273732 | 2364 | 0.000014 | org.donorbox |
828 | 18273312 | 3038 | 0.000011 | rs.google |
829 | 18273256 | 1197 | 0.000026 | com.merriam-webster |
830 | 18271448 | 5051 | 0.000007 | ee.ut |
831 | 18271060 | 2519 | 0.000013 | com.amebaownd |
832 | 18270922 | 4482 | 0.000008 | com.marksandspencer |
833 | 18270780 | 6447 | 0.000006 | su.clan |
834 | 18269948 | 4096 | 0.000009 | ru.interfax |
835 | 18269620 | 3852 | 0.000009 | org.rferl |
836 | 18268756 | 2904 | 0.000012 | gov.nd |
837 | 18267994 | 548 | 0.000049 | com.fortune |
838 | 18267776 | 4693 | 0.000008 | it.unitn |
839 | 18267714 | 5665 | 0.000006 | am.google |
840 | 18266762 | 3502 | 0.000010 | org.iaea |
841 | 18263748 | 3893 | 0.000009 | pr.com.google |
842 | 18262158 | 5045 | 0.000007 | com.tok2 |
843 | 18261938 | 1901 | 0.000018 | ch.ethz |
844 | 18261922 | 3342 | 0.000011 | gov.la |
845 | 18261182 | 4507 | 0.000008 | org.democracynow |
846 | 18261176 | 2593 | 0.000013 | net.noscript |
847 | 18260216 | 836 | 0.000037 | com.mix |
848 | 18259862 | 408 | 0.000066 | net.adform |
849 | 18259608 | 5208 | 0.000007 | tn.google |
850 | 18257978 | 4212 | 0.000008 | jp.hateblo |
851 | 18257888 | 6029 | 0.000006 | hk.edu.hkbu |
852 | 18257680 | 3884 | 0.000009 | nl.wur |
853 | 18257594 | 5009 | 0.000007 | gr.auth |
854 | 18257406 | 997 | 0.000031 | com.webs |
855 | 18256760 | 4512 | 0.000008 | com.mnn |
856 | 18256702 | 5759 | 0.000006 | ru.nnov |
857 | 18256238 | 3954 | 0.000009 | com.afp |
858 | 18255744 | 1365 | 0.000023 | com.format |
859 | 18255662 | 5209 | 0.000007 | nf.co |
860 | 18253954 | 329 | 0.000079 | com.getbootstrap |
861 | 18252988 | 4961 | 0.000007 | jp.hatenadiary |
862 | 18252154 | 4728 | 0.000007 | hk.com.hkex |
863 | 18251258 | 1193 | 0.000026 | com.redhat |
864 | 18250974 | 5600 | 0.000006 | com.gust |
865 | 18250088 | 1067 | 0.000029 | com.symantec |
866 | 18249466 | 2562 | 0.000013 | net.ucoz |
867 | 18249320 | 268 | 0.000095 | com.typeform |
868 | 18248694 | 6327 | 0.000006 | com.x10host |
869 | 18248332 | 3547 | 0.000010 | uk.co.saveourschools |
870 | 18247898 | 2934 | 0.000012 | com.squarespace-cdn |
871 | 18247292 | 2977 | 0.000011 | lt.google |
872 | 18246872 | 525 | 0.000051 | com.adweek |
873 | 18246844 | 4295 | 0.000008 | com.scienceblogs |
874 | 18246472 | 4848 | 0.000007 | de.uni-konstanz |
875 | 18245562 | 6362 | 0.000006 | com.ueuo |
876 | 18245048 | 3856 | 0.000009 | uk.gov.data |
877 | 18244756 | 4005 | 0.000009 | tr.com.hurriyet |
878 | 18243652 | 3070 | 0.000011 | ae.google |
879 | 18243570 | 1891 | 0.000019 | com.speakerdeck |
880 | 18243330 | 5079 | 0.000007 | com.blogsky |
881 | 18243134 | 2044 | 0.000017 | tv.ustream |
882 | 18240374 | 6711 | 0.000006 | su.moy |
883 | 18239298 | 761 | 0.000041 | gov.copyright |
884 | 18239096 | 5292 | 0.000007 | ru.novayagazeta |
885 | 18239044 | 2789 | 0.000012 | gov.nh |
886 | 18238990 | 4057 | 0.000009 | org.hathitrust |
887 | 18238948 | 3648 | 0.000010 | org.annualreviews |
888 | 18238932 | 1154 | 0.000027 | pl.home |
889 | 18238882 | 3815 | 0.000009 | com.businesscatalyst |
890 | 18237740 | 472 | 0.000058 | com.ea |
891 | 18237726 | 3087 | 0.000011 | uk.gov.hmrc |
892 | 18236940 | 3930 | 0.000009 | cc.uxdesign |
893 | 18236894 | 6015 | 0.000006 | com.artfire |
894 | 18236704 | 366 | 0.000072 | org.opensource |
895 | 18236530 | 3467 | 0.000010 | it.beniculturali |
896 | 18236132 | 2507 | 0.000014 | gov.mn |
897 | 18236076 | 1019 | 0.000030 | com.engadget |
898 | 18235902 | 3682 | 0.000010 | ve.co.google |
899 | 18235452 | 4973 | 0.000007 | com.teslamotors |
900 | 18234038 | 7475 | 0.000005 | com.hangame |
901 | 18233966 | 427 | 0.000063 | com.fastcompany |
902 | 18233600 | 4263 | 0.000008 | com.hsbc |
903 | 18233074 | 2462 | 0.000014 | com.netsolhost |
904 | 18232582 | 5556 | 0.000006 | me.google |
905 | 18232344 | 5643 | 0.000006 | mu.google |
906 | 18231570 | 5529 | 0.000006 | com.yam |
907 | 18231242 | 3969 | 0.000009 | tz.co.google |
908 | 18230998 | 974 | 0.000032 | com.verisign |
909 | 18230916 | 3364 | 0.000011 | tw.com.pchome |
910 | 18230662 | 7293 | 0.000005 | com.addr |
911 | 18230628 | 2636 | 0.000013 | com.shell |
912 | 18230602 | 6599 | 0.000006 | com.dropmark |
913 | 18229708 | 5635 | 0.000006 | li.google |
914 | 18229116 | 5002 | 0.000007 | com.gab |
915 | 18229106 | 4493 | 0.000008 | com.tapatalk |
916 | 18228194 | 1325 | 0.000023 | edu.ucla |
917 | 18227958 | 3557 | 0.000010 | uk.co.newmedianow |
918 | 18227938 | 4988 | 0.000007 | edu.whoi |
919 | 18227810 | 3738 | 0.000009 | ng.com.google |
920 | 18227630 | 5444 | 0.000007 | ni.com.google |
921 | 18226076 | 4110 | 0.000009 | uk.co.sainsburys |
922 | 18225458 | 4412 | 0.000008 | com.iconarchive |
923 | 18225080 | 5380 | 0.000007 | gr.ntua |
924 | 18224924 | 6152 | 0.000006 | com.epochtimes |
925 | 18224716 | 5198 | 0.000007 | org.birdlife |
926 | 18224610 | 3532 | 0.000010 | uk.co.intersol |
927 | 18224178 | 5615 | 0.000006 | id.co.kaskus |
928 | 18223762 | 950 | 0.000034 | com.zoho |
929 | 18223166 | 5403 | 0.000007 | cr.co.google |
930 | 18223046 | 5695 | 0.000006 | sv.com.google |
931 | 18222882 | 4074 | 0.000009 | vn.zing |
932 | 18222714 | 4537 | 0.000008 | uk.co.zoopla |
933 | 18222480 | 4039 | 0.000009 | uk.ac.jisc |
934 | 18221034 | 3836 | 0.000009 | com.prweek |
935 | 18220422 | 3098 | 0.000011 | int.wmo |
936 | 18220410 | 5466 | 0.000006 | mz.co.google |
937 | 18220202 | 4966 | 0.000007 | edu.umb |
938 | 18220196 | 1290 | 0.000024 | uk.co.freeukbusinessdirectory |
939 | 18220068 | 1476 | 0.000021 | org.owasp |
940 | 18219726 | 6669 | 0.000006 | net.comunidades |
941 | 18218976 | 4141 | 0.000009 | com.scotusblog |
942 | 18218840 | 5636 | 0.000006 | com.cyberlink |
943 | 18218738 | 3828 | 0.000009 | do.com.google |
944 | 18218672 | 2966 | 0.000011 | io.termly |
945 | 18218262 | 4735 | 0.000007 | com.fatcow |
946 | 18218172 | 3851 | 0.000009 | mt.com.google |
947 | 18218110 | 3589 | 0.000010 | uk.org.oxonaa |
948 | 18217958 | 3774 | 0.000009 | gt.com.google |
949 | 18216908 | 3737 | 0.000009 | com.solidworks |
950 | 18216782 | 3641 | 0.000010 | uk.co.profilebusiness |
951 | 18216270 | 3625 | 0.000010 | uk.co.heatall |
952 | 18216034 | 4506 | 0.000008 | com.theringer |
953 | 18215388 | 2558 | 0.000013 | nl.jouwweb |
954 | 18215320 | 800 | 0.000039 | com.wikihow |
955 | 18215060 | 5953 | 0.000006 | com.symbaloo |
956 | 18214768 | 5171 | 0.000007 | pl.cba |
957 | 18214162 | 5740 | 0.000006 | kg.google |
958 | 18213594 | 2321 | 0.000015 | com.freeprivacypolicy |
959 | 18212850 | 1222 | 0.000026 | com.att |
960 | 18212680 | 5203 | 0.000007 | pl.lublin |
961 | 18212672 | 1541 | 0.000020 | edu.umd |
962 | 18212174 | 5485 | 0.000006 | uk.org.labour |
963 | 18212074 | 4288 | 0.000008 | us.ms.state |
964 | 18211828 | 3449 | 0.000010 | com.wantedly |
965 | 18211570 | 4396 | 0.000008 | org.ametsoc |
966 | 18211542 | 3701 | 0.000010 | uy.com.google |
967 | 18211486 | 5553 | 0.000006 | jp.ifdef |
968 | 18211438 | 5218 | 0.000007 | es.usal |
969 | 18211398 | 769 | 0.000041 | com.netflix |
970 | 18211196 | 6329 | 0.000006 | org.cgsociety |
971 | 18210854 | 3897 | 0.000009 | hn.google |
972 | 18210544 | 5602 | 0.000006 | org.svoboda |
973 | 18207828 | 4432 | 0.000008 | org.ascd |
974 | 18207784 | 4500 | 0.000008 | uk.co.dailystar |
975 | 18207712 | 3651 | 0.000010 | uk.co.articlelistings |
976 | 18207370 | 503 | 0.000054 | com.dmca |
977 | 18207114 | 916 | 0.000035 | com.ggpht |
978 | 18207032 | 5199 | 0.000007 | com.curseforge |
979 | 18206432 | 5265 | 0.000007 | org.nsidc |
980 | 18206340 | 1520 | 0.000021 | com.technologyreview |
981 | 18205908 | 5668 | 0.000006 | ug.co.google |
982 | 18205822 | 4030 | 0.000009 | org.lacity |
983 | 18205348 | 4843 | 0.000007 | com.cbn |
984 | 18204716 | 434 | 0.000063 | com.businesswire |
985 | 18204712 | 5860 | 0.000006 | mn.google |
986 | 18204394 | 6868 | 0.000005 | kr.ac.postech |
987 | 18204332 | 5613 | 0.000006 | it.unige |
988 | 18203526 | 3314 | 0.000011 | uk.gov.food |
989 | 18203314 | 6353 | 0.000006 | com.skepticalscience |
990 | 18203052 | 909 | 0.000035 | org.weforum |
991 | 18202434 | 4907 | 0.000007 | com.globalpost |
992 | 18202416 | 5172 | 0.000007 | com.weightwatchers |
993 | 18202000 | 3403 | 0.000010 | com.lexology |
994 | 18200738 | 5944 | 0.000006 | tt.google |
995 | 18200210 | 5282 | 0.000007 | com.betfair |
996 | 18199968 | 5428 | 0.000007 | py.com.google |
997 | 18198928 | 4815 | 0.000007 | com.abcnews |
998 | 18198698 | 763 | 0.000041 | com.psychologytoday |
999 | 18198512 | 6974 | 0.000005 | org.toile-libre |
1000 | 18198414 | 3291 | 0.000011 | net.vnexpress |
Credits
Thanks to the authors of the WebGraph framework, whose software made the computation of graph properties and ranks possible.
We hope the data will be useful for you to do any kind of research on ranking, graph analysis, link spam detection, etc. Let us know about your results via Common Crawl’s Google Group!
January 2021 crawl archive now available
The crawl archive for January 2021 is now available! The data was crawled between January 15th and 28th and contains 3.4 billion web pages or 350 TiB of uncompressed content. It includes page captures of 1.15 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The January crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2021-04/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2021-04/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2021-04/warc.paths.gz | 79840 | 78.98 |
WAT files | CC-MAIN-2021-04/wat.paths.gz | 79840 | 22.92 |
WET files | CC-MAIN-2021-04/wet.paths.gz | 79840 | 10.04 |
Robots.txt files | CC-MAIN-2021-04/robotstxt.paths.gz | 79840 | 0.23 |
Non-200 responses files | CC-MAIN-2021-04/non200responses.paths.gz | 79840 | 2.11 |
URL index files | CC-MAIN-2021-04/cc-index.paths.gz | 302 | 0.26 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2021-04/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
November/December 2020 crawl archive now available
The crawl archive for November/December 2020 is now available! The data was crawled between November 23 and December 6 and contains 2.64 billion web pages or 270 TiB of uncompressed content. It includes page captures of 1.4 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The November/December crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-50/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-50/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-50/warc.paths.gz | 72000 | 59.95 |
WAT files | CC-MAIN-2020-50/wat.paths.gz | 72000 | 17.82 |
WET files | CC-MAIN-2020-50/wet.paths.gz | 72000 | 7.89 |
Robots.txt files | CC-MAIN-2020-50/robotstxt.paths.gz | 72000 | 0.2 |
Non-200 responses files | CC-MAIN-2020-50/non200responses.paths.gz | 72000 | 1.71 |
URL index files | CC-MAIN-2020-50/cc-index.paths.gz | 302 | 0.2 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-50/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
October 2020 crawl archive now available
The crawl archive for October 2020 is now available! The data was crawled between October 19th and November 1st and contains 2.71 billion web pages or 280 TiB of uncompressed content. It includes page captures of 1.5 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The October crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-45/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-45/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-45/warc.paths.gz | 72000 | 63.79 |
WAT files | CC-MAIN-2020-45/wat.paths.gz | 72000 | 18.39 |
WET files | CC-MAIN-2020-45/wet.paths.gz | 72000 | 8.23 |
Robots.txt files | CC-MAIN-2020-45/robotstxt.paths.gz | 72000 | 0.2 |
Non-200 responses files | CC-MAIN-2020-45/non200responses.paths.gz | 72000 | 1.75 |
URL index files | CC-MAIN-2020-45/cc-index.paths.gz | 302 | 0.21 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-45/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
Interactive Webgraph Statistics Notebook Released
We are pleased to announce the release of an interactive Jupyter notebook that is used to provide:
- Visualization of web graph statistics
- An interface for interacting with the webgraph
The visualization of the web graph statistics is done by leveraging the WebGraph framework, which provides means of gathering many interesting data points of a web graph, such as the frequency distribution of indegrees/outdegrees in the graph, or size distributions of the connected components. We then are able to use pandas and matplotlib to provide a visualization for the data provided by WebGraph. This effort was largely inspired by the Topology of the 2012 WDC Hyperlink Graph document. Further details of WebGraph tool installation/usage, and the data visualization may be found in the cc-notebooks repository.
The interface for interacting with the webgraph is done by using pyWebGraph, a front end that interfaces Jython with WebGraph. First, before using this interface we must re-build the string maps, in order to create a mapping between the node ID (a numerical value), to domain name (and vice versa). Once this is established we are able to simply load up the graph into pyWebGraph, and you will be able to traverse the graph interactively.
Further details of pyWebGraph installation/usage, and how to rebuild the string maps may be found in interactive webgraph README of the cc-notebooks repository.
The Jupyter notebook is available on Github in the same repository. More details about how to navigate the repository can be found in the notebook itself, as well as in the README.
We hope that users will be able to use these notebooks to gain more insight into the web graph in a numerical and practical sense.
We are grateful for WebGraph for providing extremely useful tools for processing the web graph itself, and Massimo Santini for developing pyWebGraph.