November/December 2020 crawl archive now available
The crawl archive for November/December 2020 is now available! The data was crawled between November 23 and December 6 and contains 2.64 billion web pages or 270 TiB of uncompressed content. It includes page captures of 1.4 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The November/December crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-50/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://commoncrawl.s3.amazonaws.com/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-50/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-50/warc.paths.gz | 72000 | 59.95 |
WAT files | CC-MAIN-2020-50/wat.paths.gz | 72000 | 17.82 |
WET files | CC-MAIN-2020-50/wet.paths.gz | 72000 | 7.89 |
Robots.txt files | CC-MAIN-2020-50/robotstxt.paths.gz | 72000 | 0.2 |
Non-200 responses files | CC-MAIN-2020-50/non200responses.paths.gz | 72000 | 1.71 |
URL index files | CC-MAIN-2020-50/cc-index.paths.gz | 302 | 0.2 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-50/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
October 2020 crawl archive now available
The crawl archive for October 2020 is now available! The data was crawled between October 19th and November 1st and contains 2.71 billion web pages or 280 TiB of uncompressed content. It includes page captures of 1.5 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The October crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-45/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://commoncrawl.s3.amazonaws.com/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-45/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-45/warc.paths.gz | 72000 | 63.79 |
WAT files | CC-MAIN-2020-45/wat.paths.gz | 72000 | 18.39 |
WET files | CC-MAIN-2020-45/wet.paths.gz | 72000 | 8.23 |
Robots.txt files | CC-MAIN-2020-45/robotstxt.paths.gz | 72000 | 0.2 |
Non-200 responses files | CC-MAIN-2020-45/non200responses.paths.gz | 72000 | 1.75 |
URL index files | CC-MAIN-2020-45/cc-index.paths.gz | 302 | 0.21 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-45/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
Interactive Webgraph Statistics Notebook Released
We are pleased to announce the release of an interactive Jupyter notebook that is used to provide:
- Visualization of web graph statistics
- An interface for interacting with the webgraph
The visualization of the web graph statistics is done by leveraging the WebGraph framework, which provides means of gathering many interesting data points of a web graph, such as the frequency distribution of indegrees/outdegrees in the graph, or size distributions of the connected components. We then are able to use pandas and matplotlib to provide a visualization for the data provided by WebGraph. This effort was largely inspired by the Topology of the 2012 WDC Hyperlink Graph document. Further details of WebGraph tool installation/usage, and the data visualization may be found in the cc-notebooks repository.
The interface for interacting with the webgraph is done by using pyWebGraph, a front end that interfaces Jython with WebGraph. First, before using this interface we must re-build the string maps, in order to create a mapping between the node ID (a numerical value), to domain name (and vice versa). Once this is established we are able to simply load up the graph into pyWebGraph, and you will be able to traverse the graph interactively.
Further details of pyWebGraph installation/usage, and how to rebuild the string maps may be found in interactive webgraph README of the cc-notebooks repository.
The Jupyter notebook is available on Github in the same repository. More details about how to navigate the repository can be found in the notebook itself, as well as in the README.
We hope that users will be able to use these notebooks to gain more insight into the web graph in a numerical and practical sense.
We are grateful for WebGraph for providing extremely useful tools for processing the web graph itself, and Massimo Santini for developing pyWebGraph.
Host- and Domain-Level Web Graphs Jul/Aug/Sep 2020
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of July, August and September 2020. Additional information about the data formats, the processing pipeline, our objectives, and credits can be found in the announcements of prior webgraph releases (e.g., Nov/Dec/Jan 2017-2018 Webgraphs). You may also visit the projects cc-webgraph and cc-pyspark which host all scripts and tools required to construct the graphs.
Host-level graph
The graph consists of 539 million nodes and 3.02 billion edges and includes dangling nodes i.e. hosts that have not been crawled yet are pointed to from a link on a crawled page. There are 467 million dangling nodes (86.7%) and the largest strongly connected component contains 46 million (8.5%) nodes.
You can download the graph and the ranks of all 539 million hosts from AWS S3 on the path s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-jul-aug-sep/host/
. Alternatively, you can use https://commoncrawl.s3.amazonaws.com/projects/hyperlinkgraph/cc-main-2020-jul-aug-sep/host/
as prefix to access the files from everywhere.
Size | File | Description |
---|---|---|
3.32 GB | cc-main-2020-jul-aug-sep-host-vertices.paths.gz | nodes 〈id, rev host〉, paths of 12 vertices files |
13.7 GB | cc-main-2020-jul-aug-sep-host-edges.paths.gz | edges 〈from_id, to_id〉, paths of 24 edges files |
5.95 GB | cc-main-2020-jul-aug-sep-host.graph | graph in BVGraph format |
2 kB | cc-main-2020-jul-aug-sep-host.properties | |
6.76 GB | cc-main-2020-jul-aug-sep-host-t.graph | transpose of the graph (outlinks inverted to inlinks) |
2 kB | cc-main-2020-jul-aug-sep-host-t.properties | |
1 kB | cc-main-2020-jul-aug-sep-host.stats | WebGraph statistics |
7.77 GB | cc-main-2020-jul-aug-sep-host-ranks.txt.gz | harmonic centrality and pagerank |
Note that the host names are reversed and a leading www.
is stripped: www.subdomain.example.com
becomes com.example.subdomain
.
Domain-level graph
The domain graph was built by aggregating the host graph on the level of pay-level domains (PLDs) based on the public suffix list maintained on publicsuffix.org.
The domain-level graph has 89 million nodes and 1.71 billion edges. 51% or 45 million nodes are dangling nodes, the largest strongly connected component covers 35 million or 39% of the nodes.
All files related to the domain graph are available on AWS S3 under s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-jul-aug-sep/domain/
resp. https://commoncrawl.s3.amazonaws.com/projects/hyperlinkgraph/cc-main-2020-jul-aug-sep/domain/
.
Download files of the Common Crawl Jul/Aug/Sep 2020 domain-level webgraph
Size | File | Description |
---|---|---|
0.61 GB | cc-main-2020-jul-aug-sep-domain-vertices.txt.gz | nodes 〈id, rev domain, num hosts〉 |
6.80 GB | cc-main-2020-jul-aug-sep-domain-edges.txt.gz | edges 〈from_id, to_id〉 |
3.75 GB | cc-main-2020-jul-aug-sep-domain.graph | graph in BVGraph format |
2 kB | cc-main-2020-jul-aug-sep-domain.properties | |
3.69 GB | cc-main-2020-jul-aug-sep-domain-t.graph | transpose of the graph |
2 kB | cc-main-2020-jul-aug-sep-domain-t.properties | |
1 kB | cc-main-2020-jul-aug-sep-domain.stats | WebGraph statistics |
1.91 GB | cc-main-2020-jul-aug-sep-domain-ranks.txt.gz | harmonic centrality and pagerank |
Below you’ll find the top 1000 domains ranked by Harmonic Centrality or PageRank. The full list of all 89 million domain ranks is available for download.
Top 1000 domains ranked by harmonic centrality (Jul/Aug/Sep 2020)
harmonic centrality rank | hc value | page rank | page rank value | reversed hostname |
---|---|---|---|---|
1 | 32027928 | 1 | 0.018888 | com.googleapis |
2 | 30312944 | 3 | 0.012001 | com.facebook |
3 | 29025948 | 2 | 0.013237 | com.google |
4 | 26560472 | 4 | 0.007343 | org.w |
5 | 26516534 | 5 | 0.007172 | com.twitter |
6 | 26016464 | 6 | 0.006600 | com.youtube |
7 | 24614190 | 9 | 0.004795 | com.instagram |
8 | 24220712 | 8 | 0.005190 | org.gmpg |
9 | 23572970 | 7 | 0.005599 | com.googletagmanager |
10 | 23188190 | 11 | 0.003202 | com.linkedin |
11 | 22457894 | 15 | 0.002590 | com.gravatar |
12 | 22451350 | 10 | 0.003967 | com.cloudflare |
13 | 22364152 | 14 | 0.002726 | com.gstatic |
14 | 22350042 | 12 | 0.003105 | org.wordpress |
15 | 21926906 | 22 | 0.001505 | com.pinterest |
16 | 21699168 | 21 | 0.001752 | com.wordpress |
17 | 21599006 | 26 | 0.001181 | org.wikipedia |
18 | 21538264 | 16 | 0.002431 | com.bootstrapcdn |
19 | 21497526 | 18 | 0.001836 | com.apple |
20 | 21314410 | 30 | 0.001106 | com.vimeo |
21 | 21248994 | 41 | 0.000830 | be.youtu |
22 | 21186566 | 20 | 0.001794 | com.jquery |
23 | 21081822 | 23 | 0.001444 | com.microsoft |
24 | 21073240 | 45 | 0.000773 | com.blogspot |
25 | 20994964 | 39 | 0.000952 | com.amazonaws |
26 | 20975988 | 46 | 0.000732 | gl.goo |
27 | 20971574 | 25 | 0.001384 | com.wp |
28 | 20921220 | 47 | 0.000723 | com.amazon |
29 | 20788608 | 72 | 0.000439 | com.tumblr |
30 | 20716256 | 19 | 0.001804 | com.adobe |
31 | 20694562 | 67 | 0.000535 | ly.bit |
32 | 20675418 | 34 | 0.001018 | com.google-analytics |
33 | 20627694 | 53 | 0.000673 | org.mozilla |
34 | 20618998 | 17 | 0.001975 | com.github |
35 | 20617620 | 31 | 0.001059 | net.cloudfront |
36 | 20579928 | 71 | 0.000449 | com.yahoo |
37 | 20571130 | 29 | 0.001127 | com.googlesyndication |
38 | 20570586 | 60 | 0.000612 | eu.europa |
39 | 20562028 | 52 | 0.000679 | com.flickr |
40 | 20560188 | 42 | 0.000818 | net.jsdelivr |
41 | 20526264 | 97 | 0.000347 | com.googleusercontent |
42 | 20481758 | 62 | 0.000606 | co.t |
43 | 20480218 | 109 | 0.000313 | com.reddit |
44 | 20451670 | 24 | 0.001419 | com.fontawesome |
45 | 20436180 | 83 | 0.000389 | com.weebly |
46 | 20387228 | 56 | 0.000628 | com.paypal |
47 | 20375802 | 40 | 0.000910 | com.macromedia |
48 | 20372972 | 70 | 0.000450 | com.medium |
49 | 20370180 | 43 | 0.000808 | com.addthis |
50 | 20360678 | 28 | 0.001156 | ru.yandex |
51 | 20338498 | 27 | 0.001156 | me.wp |
52 | 20331252 | 64 | 0.000559 | org.w3 |
53 | 20326560 | 79 | 0.000411 | io.github |
54 | 20292836 | 138 | 0.000223 | com.nytimes |
55 | 20275824 | 76 | 0.000414 | org.creativecommons |
56 | 20274244 | 59 | 0.000615 | org.schema |
57 | 20255326 | 150 | 0.000192 | com.forbes |
58 | 20246068 | 173 | 0.000151 | com.imgur |
59 | 20227930 | 36 | 0.000979 | net.doubleclick |
60 | 20219612 | 194 | 0.000133 | uk.co.bbc |
61 | 20210924 | 114 | 0.000285 | com.soundcloud |
62 | 20171070 | 66 | 0.000548 | com.vk |
63 | 20155222 | 195 | 0.000133 | com.cnn |
64 | 20142696 | 44 | 0.000803 | org.apache |
65 | 20134806 | 63 | 0.000587 | com.whatsapp |
66 | 20129582 | 314 | 0.000082 | edu.mit |
67 | 20123032 | 180 | 0.000146 | com.imdb |
68 | 20118310 | 208 | 0.000124 | net.slideshare |
69 | 20116626 | 243 | 0.000101 | com.wsj |
70 | 20115768 | 197 | 0.000128 | org.wikimedia |
71 | 20089462 | 85 | 0.000388 | com.shopify |
72 | 20082204 | 215 | 0.000120 | edu.stanford |
73 | 20076684 | 154 | 0.000181 | gov.cdc |
74 | 20075632 | 328 | 0.000079 | com.wired |
75 | 20069724 | 268 | 0.000094 | com.techcrunch |
76 | 20057066 | 255 | 0.000096 | edu.harvard |
77 | 20051336 | 353 | 0.000076 | com.appspot |
78 | 20051292 | 207 | 0.000124 | net.sourceforge |
79 | 20051264 | 257 | 0.000096 | com.oracle |
80 | 20051250 | 155 | 0.000177 | int.who |
81 | 20050888 | 206 | 0.000124 | com.businessinsider |
82 | 20046050 | 137 | 0.000227 | org.archive |
83 | 20038198 | 230 | 0.000113 | com.washingtonpost |
84 | 20035810 | 250 | 0.000097 | com.live |
85 | 20029940 | 164 | 0.000163 | com.bing |
86 | 20028210 | 549 | 0.000054 | com.livejournal |
87 | 20027622 | 424 | 0.000069 | com.go |
88 | 20024666 | 456 | 0.000066 | com.msn |
89 | 20019992 | 407 | 0.000072 | uk.co.telegraph |
90 | 20009306 | 170 | 0.000154 | com.theguardian |
91 | 20002514 | 527 | 0.000056 | edu.cornell |
92 | 19997146 | 199 | 0.000128 | org.ietf |
93 | 19996714 | 486 | 0.000063 | gov.nasa |
94 | 19995476 | 259 | 0.000096 | com.android |
95 | 19986252 | 302 | 0.000084 | com.reuters |
96 | 19983946 | 51 | 0.000702 | net.fbcdn |
97 | 19974890 | 240 | 0.000102 | com.bloomberg |
98 | 19966464 | 162 | 0.000164 | com.giphy |
99 | 19960428 | 77 | 0.000414 | com.list-manage |
100 | 19959046 | 520 | 0.000057 | com.googleblog |
101 | 19956558 | 269 | 0.000093 | com.bbc |
102 | 19955204 | 409 | 0.000071 | com.slack |
103 | 19942056 | 143 | 0.000205 | com.spotify |
104 | 19938828 | 591 | 0.000049 | com.zdnet |
105 | 19936894 | 48 | 0.000721 | net.facebook |
106 | 19935010 | 586 | 0.000050 | com.quora |
107 | 19931072 | 126 | 0.000265 | com.ytimg |
108 | 19922774 | 444 | 0.000067 | com.myspace |
109 | 19922046 | 757 | 0.000038 | edu.umich |
110 | 19920178 | 715 | 0.000040 | edu.upenn |
111 | 19917482 | 151 | 0.000185 | gov.nih |
112 | 19907886 | 344 | 0.000077 | com.usatoday |
113 | 19903896 | 654 | 0.000045 | com.economist |
114 | 19903722 | 313 | 0.000082 | com.cnbc |
115 | 19902700 | 308 | 0.000083 | com.example |
116 | 19896552 | 525 | 0.000056 | com.pixabay |
117 | 19895014 | 418 | 0.000070 | net.researchgate |
118 | 19882790 | 449 | 0.000066 | com.latimes |
119 | 19881164 | 188 | 0.000138 | com.blogger |
120 | 19870046 | 387 | 0.000075 | org.python |
121 | 19864804 | 65 | 0.000555 | com.wix |
122 | 19860760 | 433 | 0.000068 | com.githubusercontent |
123 | 19858732 | 693 | 0.000042 | org.ieee |
124 | 19854254 | 499 | 0.000061 | com.mashable |
125 | 19850918 | 571 | 0.000052 | edu.berkeley |
126 | 19847554 | 135 | 0.000241 | com.youtube-nocookie |
127 | 19845130 | 160 | 0.000167 | com.issuu |
128 | 19843068 | 218 | 0.000118 | org.acm |
129 | 19839736 | 834 | 0.000036 | org.chromium |
130 | 19839550 | 235 | 0.000106 | uk.co.google |
131 | 19835790 | 551 | 0.000054 | org.arxiv |
132 | 19833020 | 246 | 0.000099 | net.behance |
133 | 19832682 | 291 | 0.000086 | org.npr |
134 | 19831994 | 108 | 0.000320 | com.unpkg |
135 | 19831136 | 884 | 0.000034 | com.arstechnica |
136 | 19826840 | 213 | 0.000121 | com.unsplash |
137 | 19822884 | 341 | 0.000078 | com.outlook |
138 | 19822670 | 110 | 0.000303 | de.google |
139 | 19812430 | 54 | 0.000654 | com.googleadservices |
140 | 19810872 | 347 | 0.000077 | com.prnewswire |
141 | 19806458 | 678 | 0.000043 | edu.columbia |
142 | 19805382 | 171 | 0.000153 | me.t |
143 | 19804886 | 297 | 0.000085 | com.dribbble |
144 | 19804142 | 256 | 0.000096 | com.squarespace |
145 | 19799032 | 139 | 0.000215 | gov.privacyshield |
146 | 19798806 | 306 | 0.000083 | com.huffingtonpost |
147 | 19797964 | 260 | 0.000096 | com.bandcamp |
148 | 19795112 | 398 | 0.000074 | com.time |
149 | 19793874 | 37 | 0.000975 | com.baidu |
150 | 19792082 | 616 | 0.000048 | com.gitlab |
151 | 19790406 | 334 | 0.000079 | com.nationalgeographic |
152 | 19788214 | 443 | 0.000067 | com.nature |
153 | 19785178 | 794 | 0.000037 | com.stackexchange |
154 | 19782114 | 179 | 0.000147 | gle.forms |
155 | 19781676 | 258 | 0.000096 | org.ampproject |
156 | 19778534 | 548 | 0.000054 | com.fortune |
157 | 19777902 | 813 | 0.000036 | com.git-scm |
158 | 19776608 | 33 | 0.001030 | com.wixstatic |
159 | 19774030 | 771 | 0.000038 | com.qz |
160 | 19772390 | 281 | 0.000089 | com.wiley |
161 | 19772268 | 646 | 0.000046 | au.net.abc |
162 | 19770930 | 638 | 0.000046 | edu.yale |
163 | 19769582 | 428 | 0.000068 | com.meetup |
164 | 19767876 | 468 | 0.000064 | com.ted |
165 | 19761386 | 1160 | 0.000026 | com.hatenablog |
166 | 19759052 | 448 | 0.000066 | com.patreon |
167 | 19757472 | 283 | 0.000089 | com.disqus |
168 | 19756748 | 936 | 0.000032 | edu.ucla |
169 | 19753998 | 147 | 0.000195 | com.dropbox |
170 | 19753380 | 168 | 0.000158 | com.yelp |
171 | 19750678 | 271 | 0.000093 | org.un |
172 | 19746384 | 212 | 0.000122 | com.twimg |
173 | 19743118 | 254 | 0.000096 | org.drupal |
174 | 19741474 | 689 | 0.000042 | org.bitbucket |
175 | 19736540 | 422 | 0.000069 | com.statista |
176 | 19735440 | 903 | 0.000033 | uk.ac.cam |
177 | 19731940 | 718 | 0.000040 | com.evernote |
178 | 19731916 | 682 | 0.000043 | com.newyorker |
179 | 19725638 | 603 | 0.000049 | com.buzzfeed |
180 | 19719544 | 606 | 0.000049 | me.about |
181 | 19718654 | 722 | 0.000040 | com.mysql |
182 | 19716804 | 850 | 0.000035 | com.thenextweb |
183 | 19715420 | 495 | 0.000061 | com.theatlantic |
184 | 19710920 | 279 | 0.000091 | com.sciencedirect |
185 | 19710826 | 403 | 0.000073 | com.getpocket |
186 | 19705326 | 669 | 0.000043 | uk.co.blogspot |
187 | 19702126 | 1293 | 0.000023 | com.tinypic |
188 | 19696730 | 450 | 0.000066 | com.booking |
189 | 19695652 | 514 | 0.000058 | com.xinhuanet |
190 | 19694904 | 743 | 0.000039 | org.weforum |
191 | 19694268 | 247 | 0.000098 | gov.ca |
192 | 19692322 | 602 | 0.000049 | gov.loc |
193 | 19690998 | 1282 | 0.000023 | org.postgresql |
194 | 19689908 | 828 | 0.000036 | edu.princeton |
195 | 19687954 | 239 | 0.000103 | uk.co.amazon |
196 | 19685942 | 480 | 0.000063 | com.dailymotion |
197 | 19679672 | 1452 | 0.000021 | ru.narod |
198 | 19678926 | 189 | 0.000138 | com.xing |
199 | 19675914 | 879 | 0.000034 | edu.jhu |
200 | 19673670 | 500 | 0.000060 | gov.whitehouse |
201 | 19671846 | 665 | 0.000044 | org.worldbank |
202 | 19668706 | 1365 | 0.000022 | org.eclipse |
203 | 19667770 | 400 | 0.000073 | com.springer |
204 | 19667684 | 445 | 0.000067 | com.nypost |
205 | 19665872 | 316 | 0.000081 | com.ft |
206 | 19660930 | 61 | 0.000606 | com.fb |
207 | 19658986 | 204 | 0.000125 | com.feedburner |
208 | 19658394 | 826 | 0.000036 | org.cambridge |
209 | 19654762 | 476 | 0.000063 | uk.co.dailymail |
210 | 19654386 | 766 | 0.000038 | edu.washington |
211 | 19654242 | 496 | 0.000061 | org.eff |
212 | 19653044 | 32 | 0.001054 | com.qq |
213 | 19650144 | 473 | 0.000064 | com.goodreads |
214 | 19649524 | 264 | 0.000095 | org.doi |
215 | 19649502 | 512 | 0.000058 | com.w3schools |
216 | 19641242 | 1311 | 0.000023 | edu.virginia |
217 | 19641212 | 440 | 0.000067 | com.googlecode |
218 | 19638348 | 633 | 0.000047 | com.vice |
219 | 19633128 | 506 | 0.000059 | com.force |
220 | 19632976 | 723 | 0.000040 | com.trello |
221 | 19632780 | 836 | 0.000035 | com.about |
222 | 19630562 | 523 | 0.000056 | com.inc |
223 | 19629482 | 453 | 0.000066 | com.scribd |
224 | 19629368 | 2053 | 0.000016 | com.wikidot |
225 | 19628436 | 619 | 0.000048 | org.semver |
226 | 19614496 | 607 | 0.000049 | com.cbsnews |
227 | 19607794 | 651 | 0.000045 | com.withgoogle |
228 | 19605512 | 146 | 0.000196 | me.line |
229 | 19603410 | 2089 | 0.000016 | com.googlesource |
230 | 19601476 | 219 | 0.000118 | org.iana |
231 | 19601452 | 546 | 0.000054 | gov.usda |
232 | 19599800 | 309 | 0.000083 | com.tinyurl |
233 | 19598290 | 1090 | 0.000027 | com.techradar |
234 | 19597674 | 858 | 0.000035 | com.dropboxusercontent |
235 | 19597446 | 384 | 0.000076 | com.ibm |
236 | 19595200 | 1284 | 0.000023 | co.elastic |
237 | 19594024 | 289 | 0.000087 | com.squareup |
238 | 19593336 | 1434 | 0.000021 | org.linuxfoundation |
239 | 19592388 | 1134 | 0.000026 | org.coursera |
240 | 19589830 | 1027 | 0.000029 | gov.fbi |
241 | 19588284 | 1158 | 0.000026 | edu.unc |
242 | 19586008 | 705 | 0.000041 | com.vox |
243 | 19583350 | 193 | 0.000134 | de.amazon |
244 | 19583096 | 550 | 0.000054 | uk.co.independent |
245 | 19580554 | 1423 | 0.000021 | ms.1drv |
246 | 19578950 | 383 | 0.000076 | com.digg |
247 | 19567612 | 1393 | 0.000022 | org.kernel |
248 | 19563948 | 113 | 0.000287 | com.sharethis |
249 | 19563468 | 751 | 0.000039 | org.d3js |
250 | 19557490 | 801 | 0.000037 | gov.fcc |
251 | 19557292 | 1026 | 0.000029 | com.hollywoodreporter |
252 | 19556258 | 1369 | 0.000022 | com.howstuffworks |
253 | 19553700 | 430 | 0.000068 | com.cnet |
254 | 19552068 | 804 | 0.000037 | com.foxnews |
255 | 19547134 | 152 | 0.000183 | com.addtoany |
256 | 19547006 | 644 | 0.000046 | com.indiatimes |
257 | 19546928 | 995 | 0.000029 | com.steamcommunity |
258 | 19546864 | 1105 | 0.000026 | cn.com.chinadaily |
259 | 19545628 | 584 | 0.000050 | com.psychologytoday |
260 | 19544130 | 823 | 0.000036 | uk.co.guardian |
261 | 19543920 | 1463 | 0.000021 | it.scoop |
262 | 19543754 | 133 | 0.000247 | com.mailchimp |
263 | 19542234 | 837 | 0.000035 | com.slate |
264 | 19542214 | 153 | 0.000182 | com.opera |
265 | 19538412 | 589 | 0.000050 | com.mckinsey |
266 | 19536816 | 1020 | 0.000029 | com.sap |
267 | 19536418 | 2605 | 0.000013 | org.wikiquote |
268 | 19534334 | 307 | 0.000083 | com.bitly |
269 | 19533308 | 627 | 0.000047 | com.mozilla |
270 | 19533054 | 262 | 0.000095 | jp.ameblo |
271 | 19531260 | 735 | 0.000039 | org.sciencemag |
272 | 19528246 | 116 | 0.000284 | com.paypalobjects |
273 | 19528108 | 2345 | 0.000014 | org.wikibooks |
274 | 19527104 | 176 | 0.000151 | com.amazon-adsystem |
275 | 19526948 | 688 | 0.000042 | gov.noaa |
276 | 19524868 | 305 | 0.000083 | com.netdna-ssl |
277 | 19524544 | 310 | 0.000083 | com.nbcnews |
278 | 19523330 | 989 | 0.000030 | com.target |
279 | 19522776 | 1523 | 0.000020 | com.instructables |
280 | 19517526 | 975 | 0.000030 | edu.umn |
281 | 19516530 | 965 | 0.000031 | com.merriam-webster |
282 | 19516260 | 1431 | 0.000021 | hk.com.google |
283 | 19514852 | 185 | 0.000140 | com.tripadvisor |
284 | 19514608 | 2377 | 0.000014 | com.diigo |
285 | 19503916 | 497 | 0.000061 | ca.google |
286 | 19499262 | 236 | 0.000106 | com.wpengine |
287 | 19499246 | 1029 | 0.000028 | com.sun |
288 | 19496562 | 1189 | 0.000025 | com.digitaltrends |
289 | 19496340 | 391 | 0.000075 | com.stumbleupon |
290 | 19491846 | 115 | 0.000284 | com.weibo |
291 | 19491638 | 1626 | 0.000019 | com.ign |
292 | 19491210 | 1314 | 0.000023 | com.mercurynews |
293 | 19490964 | 1352 | 0.000022 | de.zeit |
294 | 19490636 | 229 | 0.000114 | com.etsy |
295 | 19489106 | 797 | 0.000037 | uk.ac.ox |
296 | 19487454 | 284 | 0.000089 | com.optimizely |
297 | 19485106 | 73 | 0.000425 | net.akamaihd |
298 | 19484368 | 1207 | 0.000025 | net.speedtest |
299 | 19484284 | 1522 | 0.000020 | org.greenpeace |
300 | 19483622 | 1553 | 0.000020 | net.seesaa |
301 | 19479450 | 720 | 0.000040 | au.com.google |
302 | 19478604 | 904 | 0.000033 | de.spiegel |
303 | 19476336 | 1077 | 0.000027 | com.podbean |
304 | 19475142 | 628 | 0.000047 | org.pbs |
305 | 19474722 | 516 | 0.000058 | com.gofundme |
306 | 19474484 | 416 | 0.000070 | com.kickstarter |
307 | 19473590 | 1340 | 0.000022 | com.urbandictionary |
308 | 19472422 | 472 | 0.000064 | org.pewresearch |
309 | 19471320 | 519 | 0.000057 | com.bigcommerce |
310 | 19467912 | 2137 | 0.000015 | de.bild |
311 | 19467240 | 231 | 0.000112 | com.eepurl |
312 | 19465300 | 515 | 0.000058 | com.theverge |
313 | 19464792 | 273 | 0.000092 | com.stackoverflow |
314 | 19464598 | 926 | 0.000032 | com.politico |
315 | 19463036 | 811 | 0.000036 | co.ibb |
316 | 19462394 | 332 | 0.000079 | it.google |
317 | 19462162 | 2110 | 0.000016 | ly.visual |
318 | 19461840 | 955 | 0.000031 | org.unicef |
319 | 19460932 | 2020 | 0.000016 | org.tensorflow |
320 | 19457592 | 1688 | 0.000018 | com.itv |
321 | 19457150 | 1013 | 0.000029 | com.lifehacker |
322 | 19456512 | 106 | 0.000334 | com.stripe |
323 | 19456272 | 1349 | 0.000022 | edu.msu |
324 | 19455412 | 312 | 0.000083 | net.windows |
325 | 19453374 | 805 | 0.000037 | edu.academia |
326 | 19450284 | 1391 | 0.000022 | com.storify |
327 | 19449638 | 1257 | 0.000024 | com.crunchbase |
328 | 19449386 | 595 | 0.000049 | com.tandfonline |
329 | 19449132 | 1958 | 0.000017 | com.lego |
330 | 19444682 | 1187 | 0.000025 | com.jetbrains |
331 | 19443796 | 677 | 0.000043 | gov.senate |
332 | 19443664 | 855 | 0.000035 | com.chicagotribune |
333 | 19443234 | 2301 | 0.000014 | com.rottentomatoes |
334 | 19440224 | 770 | 0.000038 | ca.cbc |
335 | 19439934 | 205 | 0.000125 | com.eventbrite |
336 | 19439496 | 1273 | 0.000023 | hk.hku |
337 | 19436402 | 1035 | 0.000028 | edu.wisc |
338 | 19436104 | 691 | 0.000042 | com.libsyn |
339 | 19435742 | 1051 | 0.000028 | edu.northwestern |
340 | 19433212 | 944 | 0.000031 | com.scientificamerican |
341 | 19432798 | 1043 | 0.000028 | edu.uchicago |
342 | 19431182 | 1288 | 0.000023 | uk.co.wired |
343 | 19425546 | 190 | 0.000137 | jp.co.google |
344 | 19424346 | 2002 | 0.000016 | org.maven |
345 | 19423732 | 1030 | 0.000028 | com.mediafire |
346 | 19423350 | 415 | 0.000070 | me.telegram |
347 | 19418440 | 396 | 0.000074 | com.criteo |
348 | 19417208 | 357 | 0.000076 | fr.google |
349 | 19417038 | 664 | 0.000044 | us.icio |
350 | 19416402 | 1477 | 0.000020 | com.deadline |
351 | 19415808 | 640 | 0.000046 | com.sagepub |
352 | 19414256 | 730 | 0.000039 | com.ecwid |
353 | 19413466 | 1275 | 0.000023 | org.aclu |
354 | 19413258 | 576 | 0.000051 | com.typepad |
355 | 19412168 | 471 | 0.000064 | com.photobucket |
356 | 19407294 | 533 | 0.000055 | com.oup |
357 | 19407168 | 1199 | 0.000025 | com.reverbnation |
358 | 19406968 | 1514 | 0.000020 | de.mpg |
359 | 19405330 | 1389 | 0.000022 | edu.rutgers |
360 | 19404790 | 1067 | 0.000027 | com.scmp |
361 | 19403976 | 81 | 0.000392 | net.jsfiddle |
362 | 19403692 | 421 | 0.000069 | com.calendly |
363 | 19403618 | 844 | 0.000035 | com.sciencedaily |
364 | 19403468 | 727 | 0.000039 | gov.justice |
365 | 19400830 | 575 | 0.000051 | gov.hhs |
366 | 19398258 | 919 | 0.000032 | com.theconversation |
367 | 19397596 | 991 | 0.000030 | com.apnews |
368 | 19397442 | 938 | 0.000032 | com.huffpost |
369 | 19394934 | 1518 | 0.000020 | com.newscientist |
370 | 19394656 | 608 | 0.000049 | org.openstreetmap |
371 | 19393300 | 1287 | 0.000023 | com.aljazeera |
372 | 19393230 | 216 | 0.000119 | com.hubspot |
373 | 19390018 | 645 | 0.000046 | gov.house |
374 | 19388118 | 2682 | 0.000012 | uk.co.timesonline |
375 | 19388034 | 2564 | 0.000013 | com.space |
376 | 19383910 | 700 | 0.000041 | com.pinimg |
377 | 19383504 | 432 | 0.000068 | page.g |
378 | 19381990 | 1241 | 0.000024 | com.sky |
379 | 19381844 | 866 | 0.000035 | gov.congress |
380 | 19381026 | 912 | 0.000033 | com.500px |
381 | 19380632 | 1217 | 0.000024 | org.wiktionary |
382 | 19380340 | 958 | 0.000031 | com.ssrn |
383 | 19379742 | 1709 | 0.000018 | edu.bu |
384 | 19377640 | 1757 | 0.000018 | gov.cia |
385 | 19375740 | 214 | 0.000120 | org.bbb |
386 | 19375634 | 1438 | 0.000021 | com.foxbusiness |
387 | 19371814 | 624 | 0.000047 | ru.gov |
388 | 19371056 | 1598 | 0.000019 | ca.mcgill |
389 | 19367926 | 790 | 0.000037 | com.qualtrics |
390 | 19366054 | 1290 | 0.000023 | org.semanticscholar |
391 | 19365778 | 761 | 0.000038 | site.business |
392 | 19365760 | 267 | 0.000094 | ru.ok |
393 | 19363798 | 977 | 0.000030 | edu.si |
394 | 19363758 | 887 | 0.000034 | br.com.google |
395 | 19363688 | 847 | 0.000035 | co.g |
396 | 19363204 | 1021 | 0.000029 | uk.co.thetimes |
397 | 19362122 | 2663 | 0.000012 | com.discovermagazine |
398 | 19359920 | 182 | 0.000142 | us.zoom |
399 | 19359492 | 889 | 0.000034 | org.fao |
400 | 19359352 | 683 | 0.000043 | org.change |
401 | 19357866 | 1469 | 0.000020 | com.salon |
402 | 19356650 | 228 | 0.000114 | com.aliyuncs |
403 | 19356280 | 997 | 0.000029 | com.thehill |
404 | 19354818 | 973 | 0.000030 | gov.usgs |
405 | 19351584 | 298 | 0.000085 | com.ebay |
406 | 19350988 | 1222 | 0.000024 | com.nikkei |
407 | 19350142 | 338 | 0.000078 | com.rawgit |
408 | 19349660 | 578 | 0.000051 | it.placehold |
409 | 19348824 | 157 | 0.000173 | com.wixsite |
410 | 19348122 | 1238 | 0.000024 | com.smithsonianmag |
411 | 19346552 | 758 | 0.000038 | org.oecd |
412 | 19346514 | 1088 | 0.000027 | ee.linktr |
413 | 19345254 | 3312 | 0.000011 | com.openai |
414 | 19342288 | 1048 | 0.000028 | uk.co.mirror |
415 | 19341656 | 679 | 0.000043 | com.deviantart |
416 | 19341332 | 1576 | 0.000019 | org.phys |
417 | 19340598 | 413 | 0.000070 | tv.twitch |
418 | 19340138 | 404 | 0.000072 | com.mapbox |
419 | 19335246 | 1546 | 0.000020 | ca.sfu |
420 | 19332464 | 2754 | 0.000012 | com.instapaper |
421 | 19330656 | 244 | 0.000100 | org.gnu |
422 | 19330504 | 2115 | 0.000016 | au.edu.unimelb |
423 | 19328724 | 1044 | 0.000028 | int.coe |
424 | 19328320 | 2078 | 0.000016 | org.nobelprize |
425 | 19328286 | 667 | 0.000043 | pl.google |
426 | 19327680 | 1333 | 0.000022 | com.irishtimes |
427 | 19327578 | 293 | 0.000086 | com.office |
428 | 19327536 | 1962 | 0.000017 | org.torproject |
429 | 19324936 | 484 | 0.000063 | net.imgix |
430 | 19324628 | 1281 | 0.000023 | uk.ac.ucl |
431 | 19320926 | 1054 | 0.000028 | org.ohchr |
432 | 19318772 | 1213 | 0.000025 | com.strikingly |
433 | 19315502 | 509 | 0.000059 | org.hbr |
434 | 19315040 | 1411 | 0.000021 | uk.co.metro |
435 | 19314304 | 123 | 0.000270 | com.statcounter |
436 | 19313468 | 972 | 0.000030 | gov.dhs |
437 | 19313380 | 287 | 0.000088 | com.thedailybeast |
438 | 19313234 | 1811 | 0.000017 | com.bankofamerica |
439 | 19312534 | 1265 | 0.000024 | com.buzzsprout |
440 | 19311940 | 863 | 0.000035 | gov.nps |
441 | 19309868 | 2426 | 0.000014 | au.com.theage |
442 | 19307472 | 933 | 0.000032 | com.aweber |
443 | 19306766 | 1557 | 0.000020 | blog.home |
444 | 19305448 | 848 | 0.000035 | gov.bls |
445 | 19305296 | 490 | 0.000062 | edu.nyu |
446 | 19304346 | 2087 | 0.000016 | com.oxforddictionaries |
447 | 19304074 | 1162 | 0.000025 | gov.nyc |
448 | 19303568 | 93 | 0.000356 | org.reactjs |
449 | 19302778 | 1382 | 0.000022 | au.com.news |
450 | 19300882 | 2291 | 0.000014 | sg.edu.nus |
451 | 19299900 | 1429 | 0.000021 | com.flipboard |
452 | 19299896 | 481 | 0.000063 | com.scorecardresearch |
453 | 19298010 | 2517 | 0.000013 | com.dummies |
454 | 19295840 | 2465 | 0.000013 | org.rsc |
455 | 19295472 | 1010 | 0.000029 | com.britannica |
456 | 19294984 | 714 | 0.000040 | gov.state |
457 | 19294216 | 1700 | 0.000018 | org.gutenberg |
458 | 19292892 | 3565 | 0.000010 | fm.ask |
459 | 19290866 | 2970 | 0.000011 | com.pearltrees |
460 | 19289990 | 793 | 0.000037 | com.zapier |
461 | 19286494 | 2562 | 0.000013 | com.mystrikingly |
462 | 19284092 | 876 | 0.000034 | com.cctv |
463 | 19283500 | 816 | 0.000036 | com.healthline |
464 | 19283044 | 1955 | 0.000017 | com.chrome |
465 | 19282638 | 1484 | 0.000020 | com.rt |
466 | 19282550 | 967 | 0.000031 | com.newsweek |
467 | 19280538 | 2362 | 0.000014 | com.biography |
468 | 19279646 | 1005 | 0.000029 | ch.google |
469 | 19270504 | 1412 | 0.000021 | com.ifttt |
470 | 19270238 | 1584 | 0.000019 | com.axios |
471 | 19270042 | 466 | 0.000065 | es.google |
472 | 19269658 | 882 | 0.000034 | au.gov.nsw |
473 | 19267444 | 3483 | 0.000010 | hk.edu.cuhk |
474 | 19267150 | 862 | 0.000035 | com.stitcher |
475 | 19267000 | 2520 | 0.000013 | com.boredpanda |
476 | 19265582 | 1192 | 0.000025 | fr.lemonde |
477 | 19263992 | 554 | 0.000053 | com.steampowered |
478 | 19263878 | 1055 | 0.000028 | org.jstor |
479 | 19262150 | 1335 | 0.000022 | org.imf |
480 | 19261918 | 873 | 0.000034 | com.venturebeat |
481 | 19261196 | 825 | 0.000036 | org.poynter |
482 | 19259574 | 1684 | 0.000018 | com.straitstimes |
483 | 19259452 | 3390 | 0.000010 | com.chosun |
484 | 19259322 | 1502 | 0.000020 | edu.asu |
485 | 19258762 | 2351 | 0.000014 | io.gitlab |
486 | 19256810 | 956 | 0.000031 | ru.google |
487 | 19255996 | 952 | 0.000031 | sg.com.google |
488 | 19253798 | 1331 | 0.000022 | uk.co.standard |
489 | 19252906 | 612 | 0.000048 | de.gesetze-im-internet |
490 | 19251516 | 948 | 0.000031 | gov.archives |
491 | 19250270 | 2385 | 0.000014 | th.co.google |
492 | 19249730 | 423 | 0.000069 | io.codepen |
493 | 19248930 | 3033 | 0.000011 | com.nola |
494 | 19248894 | 2023 | 0.000016 | edu.gmu |
495 | 19245246 | 2836 | 0.000012 | app.netlify |
496 | 19245158 | 1116 | 0.000026 | com.wikia |
497 | 19242656 | 1353 | 0.000022 | com.history |
498 | 19242160 | 1007 | 0.000029 | com.thelancet |
499 | 19241830 | 2918 | 0.000011 | com.coca-colacompany |
500 | 19240640 | 2654 | 0.000012 | google.ai |
501 | 19240600 | 856 | 0.000035 | com.freepik |
502 | 19240430 | 1548 | 0.000020 | com.buzzfeednews |
503 | 19238648 | 2894 | 0.000012 | org.cato |
504 | 19237700 | 431 | 0.000068 | net.datatables |
505 | 19237456 | 501 | 0.000060 | com.rackcdn |
506 | 19236168 | 1590 | 0.000019 | gov.supremecourt |
507 | 19233302 | 2534 | 0.000013 | edu.byu |
508 | 19233268 | 642 | 0.000046 | fr.amazon |
509 | 19232920 | 2872 | 0.000012 | tw.blogspot |
510 | 19231944 | 803 | 0.000037 | in.co.google |
511 | 19231530 | 1977 | 0.000017 | org.edx |
512 | 19231228 | 1309 | 0.000023 | com.tunein |
513 | 19231156 | 1779 | 0.000018 | org.ocks |
514 | 19230478 | 522 | 0.000057 | nl.google |
515 | 19228370 | 555 | 0.000053 | com.gmail |
516 | 19227068 | 2398 | 0.000014 | com.nationalpost |
517 | 19226910 | 1867 | 0.000017 | edu.ucsb |
518 | 19226418 | 2383 | 0.000014 | edu.nd |
519 | 19226392 | 1372 | 0.000022 | com.dw |
520 | 19226256 | 127 | 0.000262 | com.jimdo |
521 | 19225860 | 2412 | 0.000014 | no.uio |
522 | 19225400 | 1006 | 0.000029 | google.blog |
523 | 19222398 | 1409 | 0.000021 | cn.cntv |
524 | 19222164 | 3285 | 0.000011 | cn.org.china |
525 | 19221136 | 1639 | 0.000019 | org.unwomen |
526 | 19218950 | 946 | 0.000031 | com.airtable |
527 | 19217788 | 2510 | 0.000013 | edu.uoregon |
528 | 19215376 | 2172 | 0.000015 | org.britishcouncil |
529 | 19214674 | 2668 | 0.000012 | org.icrc |
530 | 19214462 | 951 | 0.000031 | com.gallup |
531 | 19213378 | 2265 | 0.000015 | ru.kremlin |
532 | 19212894 | 1332 | 0.000022 | com.globalsign |
533 | 19210850 | 875 | 0.000034 | gov.uspto |
534 | 19210492 | 959 | 0.000031 | edu.psu |
535 | 19210022 | 1509 | 0.000020 | com.penguinrandomhouse |
536 | 19209318 | 1345 | 0.000022 | com.netdna-cdn |
537 | 19208686 | 3269 | 0.000011 | is.archive |
538 | 19208344 | 1531 | 0.000020 | uk.ac.lse |
539 | 19207952 | 2503 | 0.000013 | fi.helsinki |
540 | 19207620 | 2042 | 0.000016 | edu.pitt |
541 | 19207236 | 2170 | 0.000015 | net.openid |
542 | 19206256 | 1155 | 0.000026 | edu.brookings |
543 | 19205290 | 786 | 0.000037 | com.imageshack |
544 | 19204770 | 172 | 0.000152 | com.npmjs |
545 | 19204486 | 3290 | 0.000011 | de.diplo |
546 | 19204380 | 1956 | 0.000017 | edu.unl |
547 | 19203832 | 1544 | 0.000020 | edu.georgetown |
548 | 19203210 | 2125 | 0.000015 | org.metmuseum |
549 | 19202750 | 1240 | 0.000024 | org.nejm |
550 | 19202244 | 726 | 0.000040 | com.adage |
551 | 19200434 | 1990 | 0.000017 | com.channel4 |
552 | 19200290 | 1511 | 0.000020 | com.findlaw |
553 | 19200030 | 2224 | 0.000015 | com.france24 |
554 | 19198938 | 282 | 0.000089 | net.php |
555 | 19198698 | 1784 | 0.000017 | com.csmonitor |
556 | 19197866 | 419 | 0.000069 | com.proofpoint |
557 | 19195320 | 192 | 0.000135 | com.iubenda |
558 | 19194372 | 1011 | 0.000029 | gov.treasury |
559 | 19194028 | 1708 | 0.000018 | com.euronews |
560 | 19191446 | 2286 | 0.000014 | com.thoughtco |
561 | 19190136 | 3742 | 0.000009 | com.doodlekit |
562 | 19189862 | 107 | 0.000320 | com.godaddy |
563 | 19189334 | 1298 | 0.000023 | edu.duke |
564 | 19188652 | 2071 | 0.000016 | com.foreignpolicy |
565 | 19185118 | 1996 | 0.000017 | org.documentcloud |
566 | 19183756 | 1300 | 0.000023 | com.livescience |
567 | 19183706 | 2508 | 0.000013 | com.upi |
568 | 19183104 | 2085 | 0.000016 | com.gq |
569 | 19182260 | 178 | 0.000148 | com.zendesk |
570 | 19182074 | 3020 | 0.000011 | com.authorstream |
571 | 19182074 | 3915 | 0.000009 | com.mysanantonio |
572 | 19181694 | 4133 | 0.000008 | tw.edu.sinica |
573 | 19177894 | 2719 | 0.000012 | org.wikisource |
574 | 19177382 | 2220 | 0.000015 | com.insider |
575 | 19177180 | 851 | 0.000035 | gov.nist |
576 | 19177000 | 1625 | 0.000019 | com.thestar |
577 | 19176642 | 181 | 0.000145 | jp.co.yahoo |
578 | 19174546 | 1304 | 0.000023 | au.com.smh |
579 | 19174028 | 2025 | 0.000016 | org.ncsl |
580 | 19173800 | 4252 | 0.000008 | hk.edu.cityu |
581 | 19173744 | 3349 | 0.000010 | com.sina |
582 | 19173108 | 2197 | 0.000015 | ie.independent |
583 | 19172266 | 2156 | 0.000015 | edu.uky |
584 | 19171704 | 96 | 0.000349 | me.ogp |
585 | 19170936 | 3413 | 0.000010 | uk.ac.sussex |
586 | 19170792 | 1755 | 0.000018 | gov.doc |
587 | 19170704 | 131 | 0.000250 | org.networkadvertising |
588 | 19169566 | 320 | 0.000080 | io.shields |
589 | 19168058 | 649 | 0.000045 | gov.usa |
590 | 19166990 | 4291 | 0.000008 | org.china-embassy |
591 | 19166810 | 3137 | 0.000011 | com.udn |
592 | 19163774 | 161 | 0.000166 | ru.mail |
593 | 19163712 | 3474 | 0.000010 | com.worldatlas |
594 | 19163522 | 505 | 0.000060 | com.netflix |
595 | 19163254 | 857 | 0.000035 | com.thinkwithgoogle |
596 | 19162356 | 1441 | 0.000021 | gov.defense |
597 | 19161952 | 1318 | 0.000023 | tw.com.google |
598 | 19160826 | 1604 | 0.000019 | org.hrw |
599 | 19159812 | 1495 | 0.000020 | com.asahi |
600 | 19159570 | 785 | 0.000037 | io.readthedocs |
601 | 19158768 | 2688 | 0.000012 | org.freedomhouse |
602 | 19158654 | 1413 | 0.000021 | tv.ustream |
603 | 19157822 | 893 | 0.000034 | org.mediawiki |
604 | 19156446 | 1715 | 0.000018 | org.pypi |
605 | 19151800 | 3028 | 0.000011 | org.adb |
606 | 19151406 | 2099 | 0.000016 | fr.leparisien |
607 | 19151152 | 2615 | 0.000013 | com.abc7news |
608 | 19150650 | 2063 | 0.000016 | com.voanews |
609 | 19150048 | 1019 | 0.000029 | com.pcmag |
610 | 19148698 | 447 | 0.000067 | org.nodejs |
611 | 19148554 | 4288 | 0.000008 | com.theundefeated |
612 | 19147816 | 3860 | 0.000009 | org.gephi |
613 | 19147176 | 1327 | 0.000023 | org.undp |
614 | 19146462 | 3277 | 0.000011 | org.iucnredlist |
615 | 19146454 | 2583 | 0.000013 | com.sacbee |
616 | 19146204 | 1594 | 0.000019 | com.treehugger |
617 | 19145608 | 2292 | 0.000014 | no.google |
618 | 19144462 | 2471 | 0.000013 | co.ello |
619 | 19143354 | 1986 | 0.000017 | com.msnbc |
620 | 19143354 | 252 | 0.000097 | com.myshopify |
621 | 19142810 | 981 | 0.000030 | uk.parliament |
622 | 19142520 | 2287 | 0.000014 | co.pcdn |
623 | 19141942 | 1255 | 0.000024 | gov.uscourts |
624 | 19141896 | 1422 | 0.000021 | co.lpages |
625 | 19140780 | 2344 | 0.000014 | org.fas |
626 | 19139768 | 781 | 0.000037 | com.intel |
627 | 19138740 | 807 | 0.000036 | com.marketwatch |
628 | 19136914 | 2047 | 0.000016 | com.infogram |
629 | 19133848 | 2538 | 0.000013 | com.sputniknews |
630 | 19133704 | 2430 | 0.000014 | ie.google |
631 | 19132582 | 1344 | 0.000022 | se.google |
632 | 19131798 | 990 | 0.000030 | com.netlify |
633 | 19131000 | 925 | 0.000032 | com.jekyllrb |
634 | 19130612 | 3055 | 0.000011 | int.interpol |
635 | 19130308 | 524 | 0.000056 | fr.free |
636 | 19130180 | 1198 | 0.000025 | be.google |
637 | 19129750 | 1575 | 0.000019 | uk.co.huffingtonpost |
638 | 19129310 | 2323 | 0.000014 | ly.rebrand |
639 | 19129104 | 1504 | 0.000020 | link.page |
640 | 19128704 | 1794 | 0.000017 | com.sched |
641 | 19127724 | 2218 | 0.000015 | jp.co.japantimes |
642 | 19127254 | 2829 | 0.000012 | org.tigris |
643 | 19127152 | 2839 | 0.000012 | org.pri |
644 | 19127006 | 2319 | 0.000014 | nz.co.nzherald |
645 | 19125622 | 1204 | 0.000025 | at.google |
646 | 19125464 | 5292 | 0.000007 | org.arkive |
647 | 19125326 | 222 | 0.000116 | com.salesforce |
648 | 19123296 | 650 | 0.000045 | br.com.uol |
649 | 19121018 | 4242 | 0.000008 | kr.co.kbs |
650 | 19119374 | 1665 | 0.000018 | com.thebalance |
651 | 19119126 | 1455 | 0.000021 | org.oxfordjournals |
652 | 19118638 | 3738 | 0.000009 | com.encyclopedia |
653 | 19117262 | 2204 | 0.000015 | org.eji |
654 | 19116506 | 2818 | 0.000012 | org.heritage |
655 | 19116298 | 2371 | 0.000014 | com.popsci |
656 | 19114518 | 2199 | 0.000015 | com.snopes |
657 | 19114098 | 2601 | 0.000013 | org.oas |
658 | 19113348 | 156 | 0.000174 | com.aspnetcdn |
659 | 19112712 | 1031 | 0.000028 | org.ilo |
660 | 19109654 | 2263 | 0.000015 | com.insidehighered |
661 | 19108980 | 1587 | 0.000019 | gov.usembassy |
662 | 19108932 | 1622 | 0.000019 | dk.google |
663 | 19108040 | 3392 | 0.000010 | org.jenkins-ci |
664 | 19107388 | 2827 | 0.000012 | org.project-syndicate |
665 | 19106556 | 1963 | 0.000017 | com.justia |
666 | 19104120 | 1563 | 0.000019 | gov.govinfo |
667 | 19103152 | 1699 | 0.000018 | com.firebaseapp |
668 | 19102068 | 2093 | 0.000016 | edu.uga |
669 | 19102028 | 3678 | 0.000010 | edu.wm |
670 | 19101614 | 3284 | 0.000011 | com.cgtn |
671 | 19101596 | 1881 | 0.000017 | org.worldcat |
672 | 19101226 | 900 | 0.000033 | com.zoho |
673 | 19100590 | 392 | 0.000074 | com.atlassian |
674 | 19100290 | 2676 | 0.000012 | org.transparency |
675 | 19099776 | 1317 | 0.000023 | org.aarp |
676 | 19099686 | 1675 | 0.000018 | org.americanbar |
677 | 19099164 | 2239 | 0.000015 | com.timeshighereducation |
678 | 19097964 | 3270 | 0.000011 | com.pastemagazine |
679 | 19095902 | 2598 | 0.000013 | org.csis |
680 | 19094342 | 629 | 0.000047 | com.samsung |
681 | 19094058 | 774 | 0.000038 | com.pexels |
682 | 19093374 | 1964 | 0.000017 | com.washingtontimes |
683 | 19092714 | 2016 | 0.000016 | gov.usaid |
684 | 19090166 | 1334 | 0.000022 | org.heart |
685 | 19088764 | 191 | 0.000136 | com.automattic |
686 | 19088428 | 865 | 0.000035 | com.verisign |
687 | 19087660 | 2108 | 0.000016 | com.motherjones |
688 | 19087034 | 2944 | 0.000011 | org.vim |
689 | 19086498 | 2062 | 0.000016 | edu.nap |
690 | 19086172 | 924 | 0.000032 | com.webs |
691 | 19084778 | 1593 | 0.000019 | org.amnesty |
692 | 19084344 | 2101 | 0.000016 | ua.com.google |
693 | 19083552 | 3988 | 0.000009 | org.globalnetworkinitiative |
694 | 19083196 | 2546 | 0.000013 | org.globalcitizen |
695 | 19082500 | 1754 | 0.000018 | com.surveygizmo |
696 | 19082058 | 2262 | 0.000015 | org.wbur |
697 | 19081048 | 2353 | 0.000014 | uk.gov.companieshouse |
698 | 19080398 | 2468 | 0.000013 | jp.mainichi |
699 | 19080286 | 3181 | 0.000011 | com.podomatic |
700 | 19078116 | 1751 | 0.000018 | org.unhcr |
701 | 19076276 | 2118 | 0.000016 | ca.ctvnews |
702 | 19075310 | 2565 | 0.000013 | uk.co.bbci |
703 | 19073812 | 968 | 0.000031 | uk.gov.legislation |
704 | 19071522 | 2681 | 0.000012 | com.nationalreview |
705 | 19070832 | 2523 | 0.000013 | com.cleveland |
706 | 19070474 | 3814 | 0.000009 | org.neocities |
707 | 19069884 | 1073 | 0.000027 | ly.snip |
708 | 19068864 | 438 | 0.000067 | com.herokuapp |
709 | 19068510 | 656 | 0.000045 | com.oreilly |
710 | 19066730 | 1154 | 0.000026 | cz.google |
711 | 19066464 | 2164 | 0.000015 | org.nrdc |
712 | 19065768 | 2671 | 0.000012 | org.thinkprogress |
713 | 19065654 | 1795 | 0.000017 | ca.globalnews |
714 | 19065106 | 270 | 0.000093 | jp.co.amazon |
715 | 19062840 | 1328 | 0.000023 | org.altervista |
716 | 19061732 | 3119 | 0.000011 | uk.ac.nottingham |
717 | 19061168 | 1267 | 0.000024 | uk.gov.nationalarchives |
718 | 19060934 | 2106 | 0.000016 | au.edu.anu |
719 | 19060236 | 3035 | 0.000011 | com.intensedebate |
720 | 19060102 | 2734 | 0.000012 | de.hu-berlin |
721 | 19059802 | 736 | 0.000039 | com.airbnb |
722 | 19059800 | 2326 | 0.000014 | de.auswaertiges-amt |
723 | 19059376 | 2316 | 0.000014 | nz.co.google |
724 | 19059170 | 2672 | 0.000012 | org.unenvironment |
725 | 19058978 | 3132 | 0.000011 | org.rsf |
726 | 19057932 | 4110 | 0.000008 | com.koreaherald |
727 | 19057778 | 1960 | 0.000017 | org.pewtrusts |
728 | 19057678 | 2867 | 0.000012 | com.techinasia |
729 | 19057488 | 2276 | 0.000014 | com.thecut |
730 | 19056174 | 3700 | 0.000009 | com.viki |
731 | 19056068 | 2724 | 0.000012 | org.gnupg |
732 | 19054590 | 2469 | 0.000013 | ro.google |
733 | 19054394 | 2057 | 0.000016 | edu.gwu |
734 | 19054116 | 3057 | 0.000011 | com.bangkokpost |
735 | 19053626 | 2572 | 0.000013 | fr.rfi |
736 | 19052868 | 414 | 0.000070 | com.pubmatic |
737 | 19051906 | 2309 | 0.000014 | com.tutsplus |
738 | 19051648 | 1079 | 0.000027 | tr.com.google |
739 | 19051516 | 248 | 0.000098 | com.getbootstrap |
740 | 19050908 | 4424 | 0.000008 | com.wonderhowto |
741 | 19050626 | 3619 | 0.000010 | com.upworthy |
742 | 19050496 | 2883 | 0.000012 | org.sonatype |
743 | 19050382 | 288 | 0.000087 | com.typeform |
744 | 19049574 | 2806 | 0.000012 | il.co.google |
745 | 19049384 | 2739 | 0.000012 | uk.ac.leeds |
746 | 19048116 | 201 | 0.000127 | to.amzn |
747 | 19047986 | 2703 | 0.000012 | vn.com.google |
748 | 19047578 | 274 | 0.000092 | com.surveymonkey |
749 | 19047380 | 922 | 0.000032 | int.wipo |
750 | 19046288 | 1057 | 0.000028 | com.gizmodo |
751 | 19046144 | 874 | 0.000034 | com.box |
752 | 19045578 | 2298 | 0.000014 | com.oregonlive |
753 | 19044916 | 547 | 0.000054 | gg.discord |
754 | 19044444 | 3356 | 0.000010 | com.theepochtimes |
755 | 19044400 | 2480 | 0.000013 | ar.com.google |
756 | 19044144 | 2943 | 0.000011 | bg.google |
757 | 19043632 | 2061 | 0.000016 | com.squarespace-cdn |
758 | 19043400 | 3479 | 0.000010 | io.soup |
759 | 19042778 | 2545 | 0.000013 | com.webbyawards |
760 | 19042384 | 2744 | 0.000012 | io.fabric |
761 | 19042298 | 1588 | 0.000019 | com.speakerdeck |
762 | 19041684 | 136 | 0.000232 | info.aboutads |
763 | 19040606 | 907 | 0.000033 | com.docker |
764 | 19038814 | 1817 | 0.000017 | com.miamiherald |
765 | 19037924 | 3191 | 0.000011 | ph.com.google |
766 | 19037762 | 2463 | 0.000013 | com.channelnewsasia |
767 | 19037556 | 3198 | 0.000011 | uk.co.vogue |
768 | 19037554 | 2619 | 0.000013 | edu.fsu |
769 | 19035870 | 485 | 0.000063 | com.staticflickr |
770 | 19035284 | 2495 | 0.000013 | za.co.google |
771 | 19033678 | 2696 | 0.000012 | com.thejakartapost |
772 | 19032442 | 1236 | 0.000024 | edu.ucsd |
773 | 19032258 | 487 | 0.000062 | com.fc2 |
774 | 19032038 | 5415 | 0.000007 | com.armorgames |
775 | 19031944 | 2155 | 0.000015 | fi.google |
776 | 19031234 | 3885 | 0.000009 | com.alamy |
777 | 19030868 | 2221 | 0.000015 | id.co.google |
778 | 19030462 | 2794 | 0.000012 | com.rd |
779 | 19029712 | 2951 | 0.000011 | com.cartodb |
780 | 19029584 | 2092 | 0.000016 | com.newrepublic |
781 | 19029348 | 3436 | 0.000010 | com.benzinga |
782 | 19028364 | 661 | 0.000044 | com.entrepreneur |
783 | 19027960 | 5376 | 0.000007 | org.gwtproject |
784 | 19026660 | 2988 | 0.000011 | com.sciencealert |
785 | 19026538 | 2763 | 0.000012 | org.iaea |
786 | 19026402 | 2376 | 0.000014 | com.thenation |
787 | 19023692 | 3411 | 0.000010 | si.google |
788 | 19023046 | 2400 | 0.000014 | pt.google |
789 | 19020124 | 2965 | 0.000011 | au.gov.nla |
790 | 19019838 | 3513 | 0.000010 | com.dailykos |
791 | 19019756 | 494 | 0.000061 | com.aol |
792 | 19019128 | 2519 | 0.000013 | edu.emory |
793 | 19019012 | 3573 | 0.000010 | com.inhabitat |
794 | 19018956 | 3415 | 0.000010 | uk.ac.soas |
795 | 19018402 | 666 | 0.000044 | com.deloitte |
796 | 19018230 | 1185 | 0.000025 | com.today |
797 | 19016838 | 978 | 0.000030 | com.windowsphone |
798 | 19016186 | 3659 | 0.000010 | org.cpj |
799 | 19016164 | 2119 | 0.000016 | kr.co.google |
800 | 19015906 | 2981 | 0.000011 | se.lu |
801 | 19015780 | 2774 | 0.000012 | org.cfr |
802 | 19014856 | 429 | 0.000068 | me.fb |
803 | 19013678 | 3288 | 0.000011 | com.joins |
804 | 19012980 | 4264 | 0.000008 | sa.com.google |
805 | 19012878 | 2814 | 0.000012 | com.politifact |
806 | 19012292 | 964 | 0.000031 | com.alexa |
807 | 19011442 | 4131 | 0.000008 | edu.utm |
808 | 19011068 | 2735 | 0.000012 | com.law360 |
809 | 19010546 | 983 | 0.000030 | com.engadget |
810 | 19008662 | 3583 | 0.000010 | hr.google |
811 | 19008538 | 2146 | 0.000015 | hu.google |
812 | 19006860 | 631 | 0.000047 | fm.last |
813 | 19006540 | 2476 | 0.000013 | eu.politico |
814 | 19006248 | 4047 | 0.000009 | com.chinatimes |
815 | 19006116 | 2521 | 0.000013 | mx.com.google |
816 | 19006060 | 3141 | 0.000011 | com.jezebel |
817 | 19005942 | 3868 | 0.000009 | com.iconarchive |
818 | 19005318 | 3471 | 0.000010 | com.ogilvy |
819 | 19004866 | 2399 | 0.000014 | gr.google |
820 | 19004086 | 2816 | 0.000012 | com.monday |
821 | 19003252 | 2738 | 0.000012 | com.digitaljournal |
822 | 19003248 | 3149 | 0.000011 | com.nyt |
823 | 19003220 | 3300 | 0.000011 | audio.breaker |
824 | 19002640 | 2823 | 0.000012 | uk.co.guim |
825 | 19002384 | 625 | 0.000047 | com.cisco |
826 | 19002038 | 3391 | 0.000010 | cn.globaltimes |
827 | 19001808 | 2648 | 0.000012 | com.instructure |
828 | 19000646 | 3321 | 0.000011 | com.crashlytics |
829 | 18999720 | 2723 | 0.000012 | au.com.businessinsider |
830 | 18999338 | 3430 | 0.000010 | org.grist |
831 | 18998280 | 1209 | 0.000025 | com.pastebin |
832 | 18998118 | 315 | 0.000082 | ai.shortpixel |
833 | 18998078 | 3990 | 0.000009 | org.constitutioncenter |
834 | 18997960 | 4842 | 0.000007 | jp.hatenadiary |
835 | 18996780 | 3770 | 0.000009 | edu.ttu |
836 | 18996076 | 2997 | 0.000011 | uk.ac.york |
837 | 18995936 | 1671 | 0.000018 | com.eater |
838 | 18995084 | 90 | 0.000364 | com.livestream |
839 | 18995036 | 2772 | 0.000012 | com.bepress |
840 | 18994752 | 2898 | 0.000012 | org.wri |
841 | 18992262 | 2043 | 0.000016 | my.com.thestar |
842 | 18991122 | 3775 | 0.000009 | com.minds |
843 | 18990592 | 2352 | 0.000014 | mp.j |
844 | 18990570 | 3708 | 0.000009 | app.web |
845 | 18990062 | 3410 | 0.000010 | org.carnegieendowment |
846 | 18989786 | 3645 | 0.000010 | tr.com.aa |
847 | 18989418 | 711 | 0.000041 | gov.sec |
848 | 18987746 | 3812 | 0.000009 | com.hyperallergic |
849 | 18987282 | 3408 | 0.000010 | com.foreignaffairs |
850 | 18986640 | 3797 | 0.000009 | au.edu.uts |
851 | 18985392 | 470 | 0.000064 | com.fastcompany |
852 | 18985032 | 3560 | 0.000010 | org.hypotheses |
853 | 18984468 | 3896 | 0.000009 | com.japantoday |
854 | 18982752 | 3507 | 0.000010 | edu.wayne |
855 | 18982048 | 3713 | 0.000009 | uk.ac.kent |
856 | 18981988 | 3697 | 0.000009 | rs.google |
857 | 18980532 | 4071 | 0.000009 | org.sourcewatch |
858 | 18979366 | 832 | 0.000036 | com.symantec |
859 | 18978424 | 2539 | 0.000013 | fr.paris |
860 | 18977996 | 2942 | 0.000011 | com.prweek |
861 | 18977902 | 1765 | 0.000018 | ch.ipcc |
862 | 18976960 | 2217 | 0.000015 | com.kinstacdn |
863 | 18976262 | 1046 | 0.000028 | edu.cmu |
864 | 18975462 | 2039 | 0.000016 | int.unfccc |
865 | 18975062 | 4196 | 0.000008 | eg.com.google |
866 | 18974804 | 3180 | 0.000011 | org.nationalgeographic |
867 | 18974548 | 2643 | 0.000013 | gov.doi |
868 | 18973940 | 3406 | 0.000010 | de.uni-frankfurt |
869 | 18973494 | 4243 | 0.000008 | by.google |
870 | 18972022 | 5050 | 0.000007 | com.symbaloo |
871 | 18971010 | 3417 | 0.000010 | nl.wur |
872 | 18969950 | 2328 | 0.000014 | org.unodc |
873 | 18968430 | 1599 | 0.000019 | com.routledge |
874 | 18968412 | 4509 | 0.000008 | com.ipsos-mori |
875 | 18966962 | 3658 | 0.000010 | ae.google |
876 | 18966152 | 4482 | 0.000008 | com.etymonline |
877 | 18965888 | 4982 | 0.000007 | build.bazel |
878 | 18965566 | 3320 | 0.000011 | org.brainpickings |
879 | 18964544 | 3143 | 0.000011 | com.scotsman |
880 | 18963796 | 4295 | 0.000008 | com.oilprice |
881 | 18963380 | 3597 | 0.000010 | uk.ac.westminster |
882 | 18963266 | 4545 | 0.000008 | lk.google |
883 | 18962576 | 1260 | 0.000024 | fr.blogspot |
884 | 18961360 | 3412 | 0.000010 | org.rferl |
885 | 18961310 | 3173 | 0.000011 | org.epi |
886 | 18959900 | 4115 | 0.000008 | lv.google |
887 | 18959812 | 3909 | 0.000009 | au.edu.griffith |
888 | 18959422 | 4219 | 0.000008 | kr.ac.snu |
889 | 18957280 | 1312 | 0.000023 | com.upwork |
890 | 18957076 | 2436 | 0.000014 | com.html5rocks |
891 | 18956714 | 5493 | 0.000007 | me.nimbusweb |
892 | 18956502 | 2940 | 0.000011 | fr.archives-ouvertes |
893 | 18956398 | 4293 | 0.000008 | com.delawareonline |
894 | 18955462 | 1792 | 0.000017 | ru.rbc |
895 | 18954968 | 745 | 0.000039 | com.gartner |
896 | 18954930 | 1127 | 0.000026 | edu.utexas |
897 | 18953642 | 2526 | 0.000013 | net.noscript |
898 | 18953466 | 2717 | 0.000012 | ae.thenational |
899 | 18953336 | 3380 | 0.000010 | com.study |
900 | 18953092 | 427 | 0.000068 | com.hp |
901 | 18953074 | 3641 | 0.000010 | uk.co.spectator |
902 | 18952762 | 3869 | 0.000009 | com.cleantechnica |
903 | 18952208 | 2803 | 0.000012 | org.unctad |
904 | 18951200 | 4255 | 0.000008 | com.teslamotors |
905 | 18950118 | 1614 | 0.000019 | com.billboard |
906 | 18949366 | 3074 | 0.000011 | com.theculturetrip |
907 | 18947896 | 2454 | 0.000013 | com.multiscreensite |
908 | 18947738 | 704 | 0.000041 | com.visualstudio |
909 | 18947588 | 3985 | 0.000009 | uk.ac.plymouth |
910 | 18947454 | 2660 | 0.000012 | sk.google |
911 | 18947312 | 3811 | 0.000009 | net.aljazeera |
912 | 18947110 | 2413 | 0.000014 | com.theintercept |
913 | 18946556 | 3421 | 0.000010 | uk.ac.exeter |
914 | 18946494 | 3332 | 0.000010 | social.mastodon |
915 | 18945876 | 2828 | 0.000012 | com.euractiv |
916 | 18945864 | 3635 | 0.000010 | com.db |
917 | 18942736 | 4447 | 0.000008 | org.mises |
918 | 18942316 | 4680 | 0.000008 | ng.com.google |
919 | 18942016 | 2795 | 0.000012 | org.panda |
920 | 18941622 | 2466 | 0.000013 | uk.gov.justice |
921 | 18941430 | 5602 | 0.000007 | net.chinadialogue |
922 | 18940924 | 4118 | 0.000008 | cat.uab |
923 | 18940746 | 4227 | 0.000008 | com.spokesman |
924 | 18940082 | 3523 | 0.000010 | co.com.google |
925 | 18939230 | 4473 | 0.000008 | lu.google |
926 | 18938996 | 4189 | 0.000008 | pe.com.google |
927 | 18938618 | 3366 | 0.000010 | com.nybooks |
928 | 18938606 | 4381 | 0.000008 | uk.ac.core |
929 | 18938206 | 2228 | 0.000015 | com.termsfeed |
930 | 18938194 | 1669 | 0.000018 | com.pcworld |
931 | 18938112 | 3846 | 0.000009 | kr.co.yna |
932 | 18938002 | 4793 | 0.000007 | com.gust |
933 | 18937788 | 3880 | 0.000009 | org.cgiar |
934 | 18937300 | 4231 | 0.000008 | pk.com.google |
935 | 18936530 | 3575 | 0.000010 | net.inquirer |
936 | 18936008 | 3083 | 0.000011 | ru.lenta |
937 | 18934000 | 1468 | 0.000020 | com.nokia |
938 | 18933676 | 2932 | 0.000011 | tw.com.pchome |
939 | 18933496 | 1223 | 0.000024 | com.ycombinator |
940 | 18933350 | 2911 | 0.000011 | nl.volkskrant |
941 | 18933194 | 78 | 0.000411 | com.oculus |
942 | 18932612 | 3455 | 0.000010 | cl.google |
943 | 18931862 | 3949 | 0.000009 | org.polymer-project |
944 | 18930888 | 2637 | 0.000013 | com.washingtonexaminer |
945 | 18930622 | 3945 | 0.000009 | sk.sme |
946 | 18930534 | 3389 | 0.000010 | edu.monash |
947 | 18930086 | 918 | 0.000032 | com.canva |
948 | 18929552 | 454 | 0.000066 | org.opensource |
949 | 18929398 | 3977 | 0.000009 | com.rappler |
950 | 18928630 | 4000 | 0.000009 | org.plan-international |
951 | 18926518 | 4561 | 0.000008 | cr.co.google |
952 | 18926412 | 3587 | 0.000010 | lt.google |
953 | 18925832 | 3810 | 0.000009 | ca.macleans |
954 | 18925646 | 817 | 0.000036 | net.adform |
955 | 18925046 | 4873 | 0.000007 | com.blogto |
956 | 18924952 | 3508 | 0.000010 | uk.ac.nhm |
957 | 18924928 | 3211 | 0.000011 | edu.ua |
958 | 18923554 | 2815 | 0.000012 | com.articulate |
959 | 18923288 | 249 | 0.000098 | com.sxsw |
960 | 18922866 | 3993 | 0.000009 | org.wilsoncenter |
961 | 18922676 | 4082 | 0.000009 | edu.lehigh |
962 | 18922336 | 417 | 0.000070 | com.skype |
963 | 18921546 | 4699 | 0.000008 | com.out |
964 | 18920714 | 1085 | 0.000027 | com.redhat |
965 | 18920680 | 3266 | 0.000011 | my.com.google |
966 | 18919064 | 2031 | 0.000016 | gov.ecfr |
967 | 18918900 | 4585 | 0.000008 | org.nsidc |
968 | 18918778 | 412 | 0.000070 | net.secureservercdn |
969 | 18918112 | 4536 | 0.000008 | kz.google |
970 | 18917590 | 3295 | 0.000011 | org.osce |
971 | 18917562 | 557 | 0.000053 | org.whatwg |
972 | 18917418 | 4096 | 0.000009 | com.wsoctv |
973 | 18917380 | 2587 | 0.000013 | uk.org.nationaltrust |
974 | 18917220 | 3201 | 0.000011 | uk.gov.london |
975 | 18917048 | 1973 | 0.000017 | scot.gov |
976 | 18916982 | 3865 | 0.000009 | uk.ac.qub |
977 | 18916460 | 3807 | 0.000009 | com.governing |
978 | 18916430 | 528 | 0.000056 | com.businesswire |
979 | 18916300 | 2253 | 0.000015 | wales.gov |
980 | 18915066 | 3422 | 0.000010 | com.afp |
981 | 18914982 | 3080 | 0.000011 | uk.ac.qmul |
982 | 18914878 | 5154 | 0.000007 | com.ingress |
983 | 18914540 | 4596 | 0.000008 | com.webcindario |
984 | 18914316 | 3402 | 0.000010 | org.psychiatryonline |
985 | 18913230 | 4148 | 0.000008 | org.marxists |
986 | 18913096 | 4073 | 0.000009 | me.thinglink |
987 | 18912970 | 1660 | 0.000018 | com.css-tricks |
988 | 18912858 | 4732 | 0.000008 | ie.nuigalway |
989 | 18912514 | 4348 | 0.000008 | com.asiaone |
990 | 18912368 | 3354 | 0.000010 | com.kaspersky-labs |
991 | 18912110 | 1249 | 0.000024 | com.smashingmagazine |
992 | 18912064 | 3787 | 0.000009 | org.nationalinterest |
993 | 18911848 | 556 | 0.000053 | com.adweek |
994 | 18911436 | 4498 | 0.000008 | ec.com.google |
995 | 18911404 | 4722 | 0.000008 | bd.com.google |
996 | 18910006 | 4846 | 0.000007 | uy.com.google |
997 | 18909998 | 4233 | 0.000008 | com.match |
998 | 18909746 | 4021 | 0.000009 | ee.google |
999 | 18909688 | 3962 | 0.000009 | com.adn |
1000 | 18909474 | 4310 | 0.000008 | com.wnd |
Credits
Thanks to the authors of the WebGraph framework, whose software made the computation of graph properties and ranks possible.
We hope the data will be useful for you to do any kind of research on ranking, graph analysis, link spam detection, etc. Let us know about your results via Common Crawl’s Google Group!
September 2020 crawl archive now available
The crawl archive for September 2020 is now available! The data was crawled between September 18th and October 2nd and contains 3.45 billion web pages or 345 TiB of uncompressed content. It includes page captures of 1.5 billion new URLs, not visited in any of our prior crawls.
Archive Location and Download
The September crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-40/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://commoncrawl.s3.amazonaws.com/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-40/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-40/warc.paths.gz | 79600 | 81.8 |
WAT files | CC-MAIN-2020-40/wat.paths.gz | 79600 | 23.14 |
WET files | CC-MAIN-2020-40/wet.paths.gz | 79600 | 10.28 |
Robots.txt files | CC-MAIN-2020-40/robotstxt.paths.gz | 79600 | 0.22 |
Non-200 responses files | CC-MAIN-2020-40/non200responses.paths.gz | 79600 | 2.36 |
URL index files | CC-MAIN-2020-40/cc-index.paths.gz | 302 | 0.27 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-40/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
August 2020 crawl archive now available
The crawl archive for August 2020 is now available! It contains 2.45 billion web pages or 235 TiB of uncompressed content, crawled between August 2nd and 15th. It includes page captures of 940 million URLs unknown in any of our prior crawl archives.
Archive Location and Download
The August crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-34/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://commoncrawl.s3.amazonaws.com/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-34/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-34/warc.paths.gz | 60000 | 48.9 |
WAT files | CC-MAIN-2020-34/wat.paths.gz | 60000 | 16.9 |
WET files | CC-MAIN-2020-34/wet.paths.gz | 60000 | 7.56 |
Robots.txt files | CC-MAIN-2020-34/robotstxt.paths.gz | 60000 | 0.19 |
Non-200 responses files | CC-MAIN-2020-34/non200responses.paths.gz | 60000 | 1.94 |
URL index files | CC-MAIN-2020-34/cc-index.paths.gz | 302 | 0.19 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-34/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
July 2020 crawl archive now available
The crawl archive for July 2020 is now available! It contains 3.14 billion web pages or 300 TiB of uncompressed content, crawled between July 2nd and 16th. It includes page captures of 1.1 billion URLs unknown in any of our prior crawl archives.
Bug Fixes and Improvements
The URL index fields "redirect" and "mime" haven’t been filled if the corresponding HTTP headers Location
and Content-Type
are written in lower-case letters or any other variant not matching case. This bug has been detected during the crawl and was fixed for 90 out of 100 segments. It also affects the columnar index and the fields "fetch_redirect" resp. "content_mime_type". To a minor extend it may affect the detection of character set and content language as the value of the Content-Type
header is used as additional hint for the detection. Additional information about this bug fix is given in the corresponding issue report.
Archive Location and Download
The July crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-29/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://commoncrawl.s3.amazonaws.com/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-29/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-29/warc.paths.gz | 60000 | 62.64 |
WAT files | CC-MAIN-2020-29/wat.paths.gz | 60000 | 22.23 |
WET files | CC-MAIN-2020-29/wet.paths.gz | 60000 | 9.87 |
Robots.txt files | CC-MAIN-2020-29/robotstxt.paths.gz | 60000 | 0.21 |
Non-200 responses files | CC-MAIN-2020-29/non200responses.paths.gz | 60000 | 2.52 |
URL index files | CC-MAIN-2020-29/cc-index.paths.gz | 302 | 0.24 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-29/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
Host- and Domain-Level Web Graphs Feb/Mar/May 2020
We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of February, March/April and May/June 2020. Additional information about the data formats, the processing pipeline, our objectives, and credits can be found in the announcements of prior webgraph releases (e.g., Nov/Dec/Jan 2017-2018 Webgraphs). You may also visit the projects cc-webgraph and cc-pyspark which host all scripts and tools required to construct the graphs.
What’s new?
The host-level graph now includes hosts visited by the crawler but not linking to any other host. Why is this possible – isn’t any host found via links the crawler is following? Yes, but some links were already detected in a prior crawl, not in one of the 3 crawls used to build the web graphs. More details about the issue are given in cc-pyspark#15. The impact of this fix on the graph size is minimal: the recent crawl now includes 1 million nodes (0.1% of all nodes) which are not connected to any other node.
Host-level graph
The graph consists of 927 million nodes and 3.88 billion edges and includes dangling nodes i.e. hosts that have not been crawled yet are pointed to from a link on a crawled page. There are 857 million dangling nodes (92.5%) and the largest strongly connected component contains 47 million (5.1%) nodes.
You can download the graph and the ranks of all 927 million hosts from AWS S3 on the path s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/host/
. Alternatively, you can use https://commoncrawl.s3.amazonaws.com/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/host/
as prefix to access the files from everywhere.
Download files of the Common Crawl Feb/Mar/May 2020 host-level webgraph
Size | File | Description |
---|---|---|
5.67 GB | cc-main-2020-feb-mar-may-host-vertices.paths.gz | nodes 〈id, rev host〉, paths of 12 vertices files |
17.26 GB | cc-main-2020-feb-mar-may-host-edges.paths.gz | edges 〈from_id, to_id〉, paths of 24 edges files |
7.40 GB | cc-main-2020-feb-mar-may-host.graph | graph in BVGraph format |
2 kB | cc-main-2020-feb-mar-may-host.properties | |
8.57 GB | cc-main-2020-feb-mar-may-host-t.graph | transpose of the graph (outlinks inverted to inlinks) |
2 kB | cc-main-2020-feb-mar-may-host-t.properties | |
1 kB | cc-main-2020-feb-mar-may-host.stats | WebGraph statistics |
12.16 GB | cc-main-2020-feb-mar-may-host-ranks.txt.gz | harmonic centrality and pagerank |
Note that the host names are reversed and a leading www.
is stripped: www.subdomain.example.com
becomes com.example.subdomain
.
Domain-level graph
The domain graph was built by aggregating the host graph on the level of pay-level domains (PLDs) based on the public suffix list maintained on publicsuffix.org.
The domain-level graph has 91 million nodes and 1.96 billion edges. 51% or 46 million nodes are dangling nodes, the largest strongly connected component covers 36 million or 39% of the nodes.
All files related to the domain graph are available on AWS S3 under s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/domain/
resp. https://commoncrawl.s3.amazonaws.com/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/domain/
.
Download files of the Common Crawl Feb/Mar/May 2020 domain-level webgraph
Size | File | Description |
---|---|---|
0.62 GB | cc-main-2020-feb-mar-may-domain-vertices.txt.gz | nodes 〈id, rev domain, num hosts〉 |
7.79 GB | cc-main-2020-feb-mar-may-domain-edges.txt.gz | edges 〈from_id, to_id〉 |
4.23 GB | cc-main-2020-feb-mar-may-domain.graph | graph in BVGraph format |
2 kB | cc-main-2020-feb-mar-may-domain.properties | |
4.16 GB | cc-main-2020-feb-mar-may-domain-t.graph | transpose of the graph |
2 kB | cc-main-2020-feb-mar-may-domain-t.properties | |
1 kB | cc-main-2020-feb-mar-may-domain.stats | WebGraph statistics |
1.96 GB | cc-main-2020-feb-mar-may-domain-ranks.txt.gz | harmonic centrality and pagerank |
Below you’ll find the top 1000 domains ranked by Harmonic Centrality or PageRank. The full list of all 91 million domain ranks is available for download.
Top 1000 domains ranked by harmonic centrality (Feb/Mar/May 2020)
harmonic centrality rank | hc value | page rank | page rank value | reversed hostname |
---|---|---|---|---|
1 | 32667618 | 1 | 0.018180 | com.googleapis |
2 | 30552772 | 3 | 0.011873 | com.facebook |
3 | 29569088 | 2 | 0.013789 | com.google |
4 | 26920460 | 4 | 0.007145 | com.twitter |
5 | 26883128 | 5 | 0.007106 | org.w |
6 | 26360448 | 6 | 0.006483 | com.youtube |
7 | 24719396 | 9 | 0.004210 | com.instagram |
8 | 24251942 | 8 | 0.005125 | org.gmpg |
9 | 23841332 | 7 | 0.005329 | com.googletagmanager |
10 | 23606890 | 13 | 0.002940 | com.linkedin |
11 | 22741292 | 10 | 0.003621 | com.cloudflare |
12 | 22732960 | 12 | 0.002974 | org.wordpress |
13 | 22661910 | 14 | 0.002515 | com.gravatar |
14 | 22577680 | 15 | 0.002438 | com.gstatic |
15 | 22378134 | 22 | 0.001529 | com.pinterest |
16 | 22196962 | 27 | 0.001192 | org.wikipedia |
17 | 22189650 | 19 | 0.001864 | com.wordpress |
18 | 22066028 | 16 | 0.002404 | com.bootstrapcdn |
19 | 21967760 | 18 | 0.001884 | com.apple |
20 | 21751768 | 20 | 0.001863 | com.jquery |
21 | 21589606 | 24 | 0.001461 | com.microsoft |
22 | 21568908 | 44 | 0.000785 | be.youtu |
23 | 21568474 | 43 | 0.000806 | com.blogspot |
24 | 21533280 | 31 | 0.001104 | com.vimeo |
25 | 21415938 | 46 | 0.000761 | gl.goo |
26 | 21399120 | 35 | 0.001040 | com.amazonaws |
27 | 21358048 | 53 | 0.000665 | com.amazon |
28 | 21331634 | 21 | 0.001737 | com.adobe |
29 | 21324666 | 23 | 0.001506 | com.wp |
30 | 21209012 | 70 | 0.000452 | com.tumblr |
31 | 21184360 | 17 | 0.001949 | com.github |
32 | 21150652 | 37 | 0.001008 | com.google-analytics |
33 | 21110976 | 30 | 0.001152 | com.baidu |
34 | 21096692 | 87 | 0.000387 | com.yahoo |
35 | 21081268 | 59 | 0.000547 | ly.bit |
36 | 21060360 | 33 | 0.001072 | com.macromedia |
37 | 21046916 | 36 | 0.001035 | net.cloudfront |
38 | 21036258 | 45 | 0.000763 | com.flickr |
39 | 20997926 | 32 | 0.001101 | com.googlesyndication |
40 | 20993476 | 26 | 0.001277 | me.wp |
41 | 20980462 | 97 | 0.000340 | com.googleusercontent |
42 | 20966446 | 56 | 0.000624 | eu.europa |
43 | 20960242 | 42 | 0.000807 | net.jsdelivr |
44 | 20959910 | 52 | 0.000677 | co.t |
45 | 20901872 | 29 | 0.001163 | ru.yandex |
46 | 20846092 | 50 | 0.000742 | net.doubleclick |
47 | 20843032 | 41 | 0.000869 | com.addthis |
48 | 20823518 | 69 | 0.000457 | io.github |
49 | 20817952 | 76 | 0.000433 | com.medium |
50 | 20810030 | 25 | 0.001287 | com.fontawesome |
51 | 20809120 | 139 | 0.000189 | com.forbes |
52 | 20796434 | 61 | 0.000510 | org.w3 |
53 | 20759102 | 55 | 0.000640 | com.paypal |
54 | 20757266 | 109 | 0.000282 | com.soundcloud |
55 | 20754514 | 90 | 0.000368 | org.creativecommons |
56 | 20747472 | 57 | 0.000619 | com.vk |
57 | 20711184 | 54 | 0.000658 | org.mozilla |
58 | 20710182 | 88 | 0.000382 | com.weebly |
59 | 20698442 | 84 | 0.000410 | com.wix |
60 | 20675372 | 102 | 0.000317 | com.weibo |
61 | 20663930 | 58 | 0.000604 | org.schema |
62 | 20650202 | 164 | 0.000151 | com.imgur |
63 | 20644452 | 147 | 0.000177 | org.apache |
64 | 20642282 | 178 | 0.000138 | uk.co.bbc |
65 | 20625560 | 129 | 0.000210 | org.archive |
66 | 20610354 | 274 | 0.000089 | com.ibm |
67 | 20609614 | 154 | 0.000169 | com.bing |
68 | 20602380 | 191 | 0.000125 | net.sourceforge |
69 | 20579012 | 130 | 0.000207 | com.nytimes |
70 | 20578626 | 150 | 0.000174 | int.who |
71 | 20571012 | 183 | 0.000131 | com.cnn |
72 | 20561674 | 174 | 0.000140 | net.slideshare |
73 | 20547634 | 158 | 0.000164 | gov.cdc |
74 | 20542546 | 202 | 0.000116 | com.android |
75 | 20527230 | 228 | 0.000104 | com.wsj |
76 | 20518548 | 194 | 0.000122 | edu.stanford |
77 | 20505546 | 205 | 0.000115 | com.businessinsider |
78 | 20495034 | 254 | 0.000095 | com.oracle |
79 | 20489434 | 34 | 0.001049 | net.fbcdn |
80 | 20488868 | 373 | 0.000067 | com.msn |
81 | 20488282 | 261 | 0.000093 | edu.harvard |
82 | 20483384 | 310 | 0.000080 | com.go |
83 | 20478152 | 99 | 0.000335 | com.shopify |
84 | 20471424 | 267 | 0.000093 | com.bbc |
85 | 20464434 | 297 | 0.000083 | edu.mit |
86 | 20461340 | 330 | 0.000076 | com.myspace |
87 | 20458776 | 62 | 0.000497 | com.whatsapp |
88 | 20457206 | 289 | 0.000085 | com.appspot |
89 | 20454466 | 307 | 0.000080 | com.wired |
90 | 20446300 | 292 | 0.000085 | com.reuters |
91 | 20442004 | 101 | 0.000323 | com.godaddy |
92 | 20435550 | 171 | 0.000147 | com.theguardian |
93 | 20417770 | 143 | 0.000182 | gov.nih |
94 | 20412536 | 196 | 0.000120 | org.ietf |
95 | 20401330 | 388 | 0.000065 | gov.nasa |
96 | 20397298 | 423 | 0.000061 | com.theverge |
97 | 20394736 | 149 | 0.000175 | com.giphy |
98 | 20394276 | 382 | 0.000066 | net.researchgate |
99 | 20384930 | 270 | 0.000092 | com.bloomberg |
100 | 20377778 | 108 | 0.000285 | com.unpkg |
101 | 20376394 | 114 | 0.000271 | com.reddit |
102 | 20373856 | 337 | 0.000075 | com.xinhuanet |
103 | 20366736 | 215 | 0.000108 | org.gnu |
104 | 20363506 | 318 | 0.000079 | com.usatoday |
105 | 20352660 | 813 | 0.000037 | org.chromium |
106 | 20344996 | 356 | 0.000071 | com.springer |
107 | 20343678 | 98 | 0.000335 | de.google |
108 | 20342420 | 28 | 0.001184 | com.qq |
109 | 20341824 | 345 | 0.000073 | com.example |
110 | 20336510 | 744 | 0.000041 | edu.psu |
111 | 20324536 | 468 | 0.000055 | edu.cornell |
112 | 20324378 | 184 | 0.000131 | com.blogger |
113 | 20314024 | 60 | 0.000516 | net.akamaihd |
114 | 20304242 | 375 | 0.000067 | org.hbr |
115 | 20302310 | 750 | 0.000040 | com.git-scm |
116 | 20300014 | 937 | 0.000032 | com.wikia |
117 | 20298546 | 137 | 0.000191 | com.spotify |
118 | 20296012 | 485 | 0.000053 | edu.yale |
119 | 20295516 | 113 | 0.000271 | com.jimdo |
120 | 20293140 | 554 | 0.000047 | com.cbsnews |
121 | 20291946 | 717 | 0.000043 | com.economist |
122 | 20290574 | 214 | 0.000109 | com.washingtonpost |
123 | 20288504 | 140 | 0.000188 | jp.co.yahoo |
124 | 20286470 | 285 | 0.000086 | com.huffingtonpost |
125 | 20284558 | 316 | 0.000080 | org.un |
126 | 20281874 | 410 | 0.000063 | fr.free |
127 | 20279946 | 473 | 0.000054 | edu.berkeley |
128 | 20275446 | 287 | 0.000086 | com.cnbc |
129 | 20273280 | 245 | 0.000099 | com.dribbble |
130 | 20271584 | 576 | 0.000046 | org.arxiv |
131 | 20269716 | 151 | 0.000172 | com.issuu |
132 | 20257038 | 545 | 0.000047 | com.mysql |
133 | 20256262 | 160 | 0.000157 | com.twimg |
134 | 20252532 | 107 | 0.000285 | com.statcounter |
135 | 20251682 | 338 | 0.000075 | uk.co.telegraph |
136 | 20247478 | 305 | 0.000081 | com.w3schools |
137 | 20246682 | 561 | 0.000047 | com.gitlab |
138 | 20242210 | 802 | 0.000038 | edu.columbia |
139 | 20240978 | 524 | 0.000049 | gov.noaa |
140 | 20230666 | 122 | 0.000230 | com.ytimg |
141 | 20229900 | 119 | 0.000233 | com.youtube-nocookie |
142 | 20227656 | 731 | 0.000042 | org.ieee |
143 | 20227126 | 333 | 0.000075 | org.npr |
144 | 20225528 | 729 | 0.000042 | io.readthedocs |
145 | 20225206 | 286 | 0.000086 | org.acm |
146 | 20222314 | 339 | 0.000074 | com.time |
147 | 20220430 | 1180 | 0.000025 | org.eclipse |
148 | 20220382 | 241 | 0.000100 | org.ampproject |
149 | 20218616 | 344 | 0.000074 | com.fc2 |
150 | 20215730 | 142 | 0.000185 | com.wixsite |
151 | 20213692 | 755 | 0.000040 | edu.washington |
152 | 20210122 | 421 | 0.000061 | com.force |
153 | 20209864 | 276 | 0.000089 | com.prnewswire |
154 | 20209130 | 500 | 0.000052 | com.buzzfeed |
155 | 20207136 | 434 | 0.000060 | com.nationalgeographic |
156 | 20206402 | 403 | 0.000063 | com.nature |
157 | 20203826 | 200 | 0.000118 | gle.forms |
158 | 20202490 | 799 | 0.000038 | org.sciencemag |
159 | 20201144 | 428 | 0.000061 | com.theatlantic |
160 | 20200104 | 871 | 0.000035 | com.stackexchange |
161 | 20198142 | 280 | 0.000088 | com.sciencedirect |
162 | 20185400 | 332 | 0.000075 | com.staticflickr |
163 | 20184528 | 495 | 0.000052 | uk.co.independent |
164 | 20182256 | 263 | 0.000093 | gov.ca |
165 | 20180972 | 687 | 0.000043 | org.worldbank |
166 | 20175994 | 435 | 0.000060 | com.mozilla |
167 | 20175400 | 734 | 0.000041 | com.marketwatch |
168 | 20168098 | 1087 | 0.000027 | com.hatenablog |
169 | 20167040 | 364 | 0.000069 | com.nypost |
170 | 20164016 | 646 | 0.000043 | org.bitbucket |
171 | 20161192 | 219 | 0.000107 | com.ft |
172 | 20151116 | 463 | 0.000056 | com.pixabay |
173 | 20143796 | 354 | 0.000071 | jp.co.rakuten |
174 | 20142652 | 743 | 0.000041 | edu.upenn |
175 | 20140126 | 277 | 0.000089 | org.doi |
176 | 20139376 | 966 | 0.000031 | jp.livedoor |
177 | 20136546 | 198 | 0.000120 | uk.co.google |
178 | 20134932 | 407 | 0.000063 | uk.co.dailymail |
179 | 20134404 | 724 | 0.000042 | org.pbs |
180 | 20133936 | 258 | 0.000094 | net.behance |
181 | 20132914 | 192 | 0.000124 | org.wikimedia |
182 | 20127860 | 917 | 0.000033 | edu.jhu |
183 | 20127828 | 454 | 0.000057 | gov.whitehouse |
184 | 20122352 | 856 | 0.000035 | org.weforum |
185 | 20122170 | 416 | 0.000062 | com.dailymotion |
186 | 20117054 | 1487 | 0.000020 | com.warnerbros |
187 | 20111898 | 326 | 0.000077 | org.opensource |
188 | 20110798 | 1091 | 0.000027 | cn.com.chinadaily |
189 | 20109916 | 548 | 0.000047 | me.about |
190 | 20109820 | 232 | 0.000103 | jp.ameblo |
191 | 20108940 | 558 | 0.000047 | com.oup |
192 | 20103428 | 325 | 0.000077 | com.digg |
193 | 20097418 | 455 | 0.000056 | com.entrepreneur |
194 | 20095108 | 631 | 0.000044 | com.vice |
195 | 20094142 | 749 | 0.000040 | com.qz |
196 | 20092692 | 1259 | 0.000024 | com.discovery |
197 | 20091154 | 444 | 0.000058 | com.goodreads |
198 | 20091052 | 447 | 0.000057 | gg.discord |
199 | 20082910 | 1109 | 0.000027 | com.sap |
200 | 20082186 | 353 | 0.000071 | com.scribd |
201 | 20079412 | 188 | 0.000128 | com.feedburner |
202 | 20076146 | 466 | 0.000055 | com.fortune |
203 | 20075556 | 580 | 0.000045 | com.gartner |
204 | 20072598 | 1012 | 0.000029 | com.500px |
205 | 20072136 | 458 | 0.000056 | jp.ne.sakura |
206 | 20067400 | 176 | 0.000139 | com.imdb |
207 | 20060950 | 732 | 0.000042 | uk.co.blogspot |
208 | 20059054 | 1735 | 0.000018 | com.amd |
209 | 20058228 | 947 | 0.000032 | edu.princeton |
210 | 20056666 | 890 | 0.000034 | org.cambridge |
211 | 20056572 | 51 | 0.000714 | com.fb |
212 | 20056272 | 848 | 0.000036 | com.evernote |
213 | 20054472 | 144 | 0.000180 | com.dropbox |
214 | 20053532 | 39 | 0.000951 | com.wixstatic |
215 | 20051662 | 617 | 0.000044 | org.unesco |
216 | 20050940 | 1461 | 0.000020 | com.fandom |
217 | 20048152 | 294 | 0.000084 | com.wiley |
218 | 20046134 | 768 | 0.000039 | com.withgoogle |
219 | 20039426 | 1015 | 0.000029 | org.altervista |
220 | 20039010 | 2337 | 0.000014 | com.wolfram |
221 | 20037920 | 798 | 0.000038 | com.slate |
222 | 20031484 | 1201 | 0.000025 | org.kernel |
223 | 20028164 | 1049 | 0.000028 | edu.purdue |
224 | 20025282 | 569 | 0.000046 | page.g |
225 | 20021340 | 786 | 0.000038 | com.trello |
226 | 20017018 | 230 | 0.000103 | com.disqus |
227 | 20012796 | 757 | 0.000040 | org.eff |
228 | 20010430 | 951 | 0.000031 | com.merriam-webster |
229 | 20004686 | 493 | 0.000052 | gov.usda |
230 | 20004240 | 981 | 0.000030 | com.netlify |
231 | 20003994 | 2179 | 0.000015 | com.diigo |
232 | 20002918 | 807 | 0.000038 | com.vox |
233 | 20002690 | 180 | 0.000135 | org.allaboutcookies |
234 | 20002220 | 1206 | 0.000025 | com.jetbrains |
235 | 19999418 | 1416 | 0.000021 | edu.arizona |
236 | 19994384 | 542 | 0.000047 | com.tandfonline |
237 | 19993030 | 844 | 0.000036 | com.foxnews |
238 | 19992184 | 291 | 0.000085 | com.live |
239 | 19991142 | 175 | 0.000140 | com.xing |
240 | 19989874 | 909 | 0.000033 | com.politico |
241 | 19988570 | 320 | 0.000079 | com.outlook |
242 | 19985036 | 1135 | 0.000026 | jp.ne.goo |
243 | 19983340 | 754 | 0.000040 | au.net.abc |
244 | 19982680 | 1945 | 0.000016 | com.wikidot |
245 | 19977934 | 793 | 0.000038 | com.investopedia |
246 | 19977574 | 1066 | 0.000028 | edu.uchicago |
247 | 19976820 | 1009 | 0.000029 | edu.wisc |
248 | 19975922 | 197 | 0.000120 | com.eepurl |
249 | 19972560 | 1039 | 0.000028 | com.bostonglobe |
250 | 19972096 | 775 | 0.000039 | org.semver |
251 | 19969594 | 619 | 0.000044 | com.sagepub |
252 | 19969182 | 497 | 0.000052 | gov.fda |
253 | 19968442 | 347 | 0.000073 | net.windows |
254 | 19968084 | 1568 | 0.000019 | edu.osu |
255 | 19965386 | 319 | 0.000079 | com.nbcnews |
256 | 19963946 | 244 | 0.000099 | com.myshopify |
257 | 19962892 | 585 | 0.000045 | cn.google |
258 | 19962530 | 608 | 0.000044 | site.business |
259 | 19961066 | 832 | 0.000036 | com.sciencedaily |
260 | 19960380 | 1044 | 0.000028 | com.strikingly |
261 | 19956366 | 1236 | 0.000024 | edu.unc |
262 | 19956268 | 1446 | 0.000021 | edu.virginia |
263 | 19956034 | 1204 | 0.000025 | co.elastic |
264 | 19952960 | 1194 | 0.000025 | com.nymag |
265 | 19950500 | 2206 | 0.000015 | com.renren |
266 | 19950490 | 742 | 0.000041 | gov.house |
267 | 19950448 | 2163 | 0.000015 | sg.edu.nus |
268 | 19947976 | 2285 | 0.000014 | org.wikibooks |
269 | 19947284 | 1961 | 0.000016 | com.googlesource |
270 | 19940598 | 235 | 0.000103 | com.wpengine |
271 | 19940158 | 323 | 0.000078 | com.googlecode |
272 | 19939212 | 761 | 0.000040 | gov.senate |
273 | 19938008 | 513 | 0.000051 | com.herokuapp |
274 | 19937738 | 452 | 0.000057 | org.pewresearch |
275 | 19937492 | 567 | 0.000046 | org.iana |
276 | 19936954 | 1093 | 0.000027 | com.podbean |
277 | 19935818 | 982 | 0.000030 | com.alexa |
278 | 19934742 | 1629 | 0.000019 | gd.is |
279 | 19933804 | 103 | 0.000301 | com.paypalobjects |
280 | 19932740 | 805 | 0.000038 | org.unicef |
281 | 19932416 | 718 | 0.000043 | com.newyorker |
282 | 19930858 | 969 | 0.000031 | uk.co.thetimes |
283 | 19929324 | 404 | 0.000063 | com.patreon |
284 | 19928266 | 1060 | 0.000028 | com.lifehacker |
285 | 19925940 | 381 | 0.000066 | com.criteo |
286 | 19924524 | 997 | 0.000030 | com.huffpost |
287 | 19922576 | 303 | 0.000081 | com.squareup |
288 | 19922510 | 839 | 0.000036 | ca.cbc |
289 | 19921808 | 1145 | 0.000026 | org.wiktionary |
290 | 19918844 | 146 | 0.000178 | com.addtoany |
291 | 19918174 | 201 | 0.000117 | com.optimizely |
292 | 19918052 | 1342 | 0.000022 | edu.msu |
293 | 19915986 | 1371 | 0.000022 | com.history |
294 | 19913384 | 418 | 0.000062 | com.calendly |
295 | 19905860 | 1181 | 0.000025 | com.udemy |
296 | 19903364 | 809 | 0.000037 | uk.ac.ox |
297 | 19902920 | 172 | 0.000145 | com.amazon-adsystem |
298 | 19899332 | 49 | 0.000743 | com.googleadservices |
299 | 19896924 | 155 | 0.000167 | com.opera |
300 | 19890970 | 887 | 0.000034 | org.fao |
301 | 19890832 | 1017 | 0.000029 | com.ecwid |
302 | 19890826 | 476 | 0.000054 | com.googleblog |
303 | 19887142 | 211 | 0.000110 | com.stackoverflow |
304 | 19886190 | 1419 | 0.000021 | uk.ac.lse |
305 | 19885312 | 360 | 0.000070 | com.getpocket |
306 | 19884456 | 1667 | 0.000018 | org.maven |
307 | 19883800 | 915 | 0.000033 | uk.co.guardian |
308 | 19883358 | 169 | 0.000148 | org.bbb |
309 | 19881084 | 1337 | 0.000022 | com.aljazeera |
310 | 19880790 | 255 | 0.000095 | com.aliyuncs |
311 | 19879938 | 2723 | 0.000013 | net.pixnet |
312 | 19874384 | 3180 | 0.000011 | net.hinet |
313 | 19869028 | 1170 | 0.000025 | com.smithsonianmag |
314 | 19868832 | 1347 | 0.000022 | edu.ucdavis |
315 | 19868258 | 894 | 0.000034 | gov.congress |
316 | 19867190 | 1320 | 0.000023 | edu.illinois |
317 | 19865168 | 1120 | 0.000026 | com.theglobeandmail |
318 | 19863306 | 1036 | 0.000029 | gov.archives |
319 | 19862414 | 492 | 0.000052 | it.placehold |
320 | 19861934 | 93 | 0.000359 | net.facebook |
321 | 19861376 | 1615 | 0.000019 | hk.com.google |
322 | 19860922 | 1473 | 0.000020 | ca.sfu |
323 | 19856352 | 1676 | 0.000018 | blog.home |
324 | 19855290 | 1073 | 0.000027 | com.apnews |
325 | 19854892 | 963 | 0.000031 | com.ssrn |
326 | 19853682 | 3383 | 0.000010 | com.wizards |
327 | 19851102 | 1997 | 0.000016 | com.nabble |
328 | 19851032 | 760 | 0.000040 | com.chinaz |
329 | 19850412 | 3667 | 0.000010 | cn.edu.sjtu |
330 | 19848140 | 1484 | 0.000020 | com.urbandictionary |
331 | 19844436 | 1136 | 0.000026 | com.scmp |
332 | 19842326 | 1489 | 0.000020 | ms.1drv |
333 | 19841796 | 4361 | 0.000008 | tw.com.gamer |
334 | 19838582 | 1392 | 0.000021 | com.flipboard |
335 | 19838166 | 919 | 0.000033 | co.g |
336 | 19837542 | 547 | 0.000047 | com.gofundme |
337 | 19836996 | 2097 | 0.000015 | com.france24 |
338 | 19835636 | 1405 | 0.000021 | jp.geocities |
339 | 19833654 | 1370 | 0.000022 | com.ibtimes |
340 | 19831362 | 581 | 0.000045 | com.biomedcentral |
341 | 19830056 | 1128 | 0.000026 | com.britannica |
342 | 19829420 | 2174 | 0.000015 | com.oregonlive |
343 | 19827062 | 412 | 0.000062 | com.kickstarter |
344 | 19826214 | 962 | 0.000031 | com.adjust |
345 | 19824188 | 867 | 0.000035 | gov.fcc |
346 | 19824048 | 715 | 0.000043 | uk.co.mirror |
347 | 19823266 | 589 | 0.000045 | us.icio |
348 | 19823172 | 1129 | 0.000026 | com.mediafire |
349 | 19821768 | 1432 | 0.000021 | edu.tamu |
350 | 19821310 | 587 | 0.000045 | com.usnews |
351 | 19820442 | 1314 | 0.000023 | org.greenpeace |
352 | 19820252 | 985 | 0.000030 | edu.academia |
353 | 19819486 | 1381 | 0.000021 | com.livescience |
354 | 19815972 | 1684 | 0.000018 | gov.cia |
355 | 19814564 | 1325 | 0.000023 | com.akamai |
356 | 19813266 | 930 | 0.000032 | com.chicagotribune |
357 | 19811538 | 156 | 0.000167 | com.npmjs |
358 | 19811100 | 1429 | 0.000021 | net.seesaa |
359 | 19810120 | 329 | 0.000076 | es.google |
360 | 19809710 | 1238 | 0.000024 | com.reverbnation |
361 | 19809490 | 550 | 0.000047 | com.quora |
362 | 19808314 | 3481 | 0.000010 | com.proboards |
363 | 19806268 | 1040 | 0.000028 | com.thehill |
364 | 19803840 | 321 | 0.000078 | org.python |
365 | 19801476 | 1132 | 0.000026 | org.jstor |
366 | 19801018 | 1722 | 0.000018 | ca.mcgill |
367 | 19799982 | 167 | 0.000149 | com.zendesk |
368 | 19792890 | 999 | 0.000030 | com.thelancet |
369 | 19792246 | 1094 | 0.000027 | com.jamanetwork |
370 | 19788594 | 1935 | 0.000016 | uk.ac.manchester |
371 | 19785214 | 540 | 0.000048 | com.udacity |
372 | 19783328 | 1372 | 0.000021 | ca.utoronto |
373 | 19783082 | 579 | 0.000046 | com.bigcartel |
374 | 19782230 | 2487 | 0.000013 | org.wikiquote |
375 | 19781186 | 1357 | 0.000022 | edu.rutgers |
376 | 19780028 | 896 | 0.000034 | org.apa |
377 | 19779718 | 439 | 0.000059 | com.newsweek |
378 | 19778538 | 920 | 0.000033 | com.healthline |
379 | 19777982 | 2204 | 0.000015 | com.knowyourmeme |
380 | 19775610 | 328 | 0.000077 | com.tinyurl |
381 | 19775558 | 726 | 0.000042 | gov.state |
382 | 19775092 | 216 | 0.000108 | com.unsplash |
383 | 19773702 | 1708 | 0.000018 | ca.ualberta |
384 | 19772378 | 406 | 0.000063 | com.githubusercontent |
385 | 19771900 | 1471 | 0.000020 | com.asahi |
386 | 19771220 | 259 | 0.000094 | org.nodejs |
387 | 19769436 | 475 | 0.000054 | com.latimes |
388 | 19769258 | 1027 | 0.000029 | com.timeanddate |
389 | 19768686 | 432 | 0.000060 | com.slack |
390 | 19768410 | 769 | 0.000039 | jp.shinobi |
391 | 19767976 | 1674 | 0.000018 | com.buzzfeednews |
392 | 19765038 | 415 | 0.000062 | com.elsevier |
393 | 19764722 | 1335 | 0.000022 | edu.gatech |
394 | 19764298 | 2861 | 0.000012 | com.youdao |
395 | 19761256 | 895 | 0.000034 | com.brightcove |
396 | 19759730 | 1774 | 0.000017 | com.bankofamerica |
397 | 19759530 | 2569 | 0.000013 | edu.byu |
398 | 19758760 | 1918 | 0.000016 | com.voanews |
399 | 19757586 | 3164 | 0.000011 | com.opendns |
400 | 19756816 | 1425 | 0.000021 | com.sky |
401 | 19755780 | 2336 | 0.000014 | com.slides |
402 | 19754462 | 1373 | 0.000021 | com.dw |
403 | 19754458 | 1158 | 0.000026 | com.nikkei |
404 | 19752590 | 904 | 0.000033 | com.cbslocal |
405 | 19748766 | 2236 | 0.000014 | net.earthlink |
406 | 19748678 | 391 | 0.000064 | com.cnet |
407 | 19748150 | 1642 | 0.000018 | com.xrea |
408 | 19747430 | 1354 | 0.000022 | uk.co.huffingtonpost |
409 | 19746424 | 182 | 0.000133 | com.eventbrite |
410 | 19746370 | 1071 | 0.000027 | com.nydailynews |
411 | 19744090 | 1305 | 0.000023 | me.vk |
412 | 19743194 | 918 | 0.000033 | gov.bls |
413 | 19741542 | 1458 | 0.000020 | org.ap |
414 | 19740936 | 384 | 0.000066 | net.imgix |
415 | 19739860 | 2414 | 0.000014 | org.aclweb |
416 | 19739750 | 1641 | 0.000018 | com.axios |
417 | 19738940 | 987 | 0.000030 | com.wattpad |
418 | 19737530 | 1713 | 0.000018 | com.straitstimes |
419 | 19737412 | 474 | 0.000054 | com.ted |
420 | 19736874 | 1294 | 0.000023 | edu.brookings |
421 | 19728634 | 967 | 0.000031 | int.coe |
422 | 19727580 | 212 | 0.000109 | com.etsy |
423 | 19727112 | 2392 | 0.000014 | com.biography |
424 | 19726080 | 865 | 0.000035 | gov.va |
425 | 19725710 | 217 | 0.000107 | com.typepad |
426 | 19724628 | 1932 | 0.000016 | com.cocolog-nifty |
427 | 19723580 | 1608 | 0.000019 | com.reference |
428 | 19720740 | 553 | 0.000047 | com.livejournal |
429 | 19717406 | 2096 | 0.000015 | ru.kremlin |
430 | 19716354 | 815 | 0.000037 | uk.gov.service |
431 | 19715378 | 298 | 0.000083 | com.techcrunch |
432 | 19712358 | 2462 | 0.000013 | org.wikisource |
433 | 19712296 | 1553 | 0.000019 | com.foxbusiness |
434 | 19711620 | 1281 | 0.000023 | mil.army |
435 | 19711244 | 1761 | 0.000017 | com.itv |
436 | 19710260 | 733 | 0.000041 | com.deviantart |
437 | 19705952 | 1311 | 0.000023 | de.mpg |
438 | 19705288 | 845 | 0.000036 | gov.justice |
439 | 19704574 | 1993 | 0.000016 | cn.people |
440 | 19703248 | 1262 | 0.000024 | au.com.smh |
441 | 19701656 | 1763 | 0.000017 | org.tensorflow |
442 | 19701634 | 1223 | 0.000024 | org.ohchr |
443 | 19701000 | 568 | 0.000046 | ru.gov |
444 | 19700136 | 400 | 0.000064 | com.technorati |
445 | 19699596 | 2134 | 0.000015 | jp.co.japantimes |
446 | 19697954 | 83 | 0.000413 | com.list-manage |
447 | 19697088 | 1068 | 0.000028 | com.thedrum |
448 | 19696754 | 1538 | 0.000019 | uk.co.standard |
449 | 19695430 | 185 | 0.000131 | com.rawgit |
450 | 19694216 | 2120 | 0.000015 | com.oxforddictionaries |
451 | 19693006 | 2241 | 0.000014 | com.shutterfly |
452 | 19692082 | 3147 | 0.000011 | tw.edu.ntu |
453 | 19691564 | 2550 | 0.000013 | com.smashwords |
454 | 19689862 | 1862 | 0.000016 | edu.unl |
455 | 19688768 | 2402 | 0.000014 | org.fas |
456 | 19688646 | 296 | 0.000084 | uk.org.ico |
457 | 19688138 | 2710 | 0.000013 | tv.blip |
458 | 19686066 | 957 | 0.000031 | com.bandsintown |
459 | 19684448 | 3516 | 0.000010 | cn.org.china |
460 | 19682960 | 1550 | 0.000019 | uk.co.express |
461 | 19679708 | 1082 | 0.000027 | jp.jugem |
462 | 19679158 | 3656 | 0.000010 | info.webry |
463 | 19678730 | 1403 | 0.000021 | gov.uscourts |
464 | 19677944 | 2157 | 0.000015 | au.edu.unimelb |
465 | 19675766 | 92 | 0.000363 | com.wsimg |
466 | 19674868 | 283 | 0.000086 | ru.rambler |
467 | 19673738 | 1921 | 0.000016 | com.washingtontimes |
468 | 19671754 | 351 | 0.000072 | com.proofpoint |
469 | 19669412 | 74 | 0.000441 | net.jsfiddle |
470 | 19668352 | 788 | 0.000038 | org.mediawiki |
471 | 19668158 | 2851 | 0.000012 | jp.blog |
472 | 19667740 | 1479 | 0.000020 | com.firebaseapp |
473 | 19667418 | 1618 | 0.000019 | com.webnode |
474 | 19665940 | 2173 | 0.000015 | com.pbworks |
475 | 19665748 | 3374 | 0.000011 | com.patheos |
476 | 19665684 | 3135 | 0.000011 | uk.co.timesonline |
477 | 19663980 | 2171 | 0.000015 | google.ai |
478 | 19663354 | 233 | 0.000103 | com.squarespace |
479 | 19662188 | 2904 | 0.000012 | fr.rfi |
480 | 19660984 | 1454 | 0.000020 | gov.supremecourt |
481 | 19659200 | 1889 | 0.000016 | int.unfccc |
482 | 19658534 | 331 | 0.000076 | com.office |
483 | 19656526 | 577 | 0.000046 | pl.google |
484 | 19654098 | 991 | 0.000030 | gov.wa |
485 | 19652796 | 804 | 0.000038 | gov.sba |
486 | 19652626 | 1267 | 0.000023 | com.cognitoforms |
487 | 19650066 | 2207 | 0.000015 | org.csis |
488 | 19649008 | 366 | 0.000068 | io.codepen |
489 | 19648750 | 2344 | 0.000014 | com.kobo |
490 | 19646512 | 110 | 0.000281 | com.mailchimp |
491 | 19643428 | 1671 | 0.000018 | edu.wustl |
492 | 19642572 | 2734 | 0.000013 | edu.kit |
493 | 19642334 | 1480 | 0.000020 | org.hrw |
494 | 19642276 | 953 | 0.000031 | edu.umich |
495 | 19641856 | 1389 | 0.000021 | com.dictionary |
496 | 19641544 | 836 | 0.000036 | com.mapquest |
497 | 19640836 | 1747 | 0.000017 | org.worldcat |
498 | 19640276 | 3621 | 0.000010 | net.aljazeera |
499 | 19640144 | 357 | 0.000071 | com.photobucket |
500 | 19639948 | 2046 | 0.000015 | net.cnki |
501 | 19638510 | 1705 | 0.000018 | com.secondlife |
502 | 19638416 | 2421 | 0.000014 | int.wmo |
503 | 19637888 | 1089 | 0.000027 | org.ilo |
504 | 19637450 | 1100 | 0.000027 | google.blog |
505 | 19636692 | 378 | 0.000067 | com.meetup |
506 | 19634634 | 995 | 0.000030 | uk.co.pinterest |
507 | 19633770 | 3397 | 0.000010 | com.freehostia |
508 | 19630412 | 3256 | 0.000011 | com.doodlekit |
509 | 19629746 | 936 | 0.000032 | com.arstechnica |
510 | 19628370 | 3730 | 0.000009 | com.colourlovers |
511 | 19628356 | 1696 | 0.000018 | ru.ucoz |
512 | 19628298 | 952 | 0.000031 | com.thenextweb |
513 | 19624458 | 2286 | 0.000014 | org.unep |
514 | 19622342 | 2252 | 0.000014 | org.icrc |
515 | 19621808 | 1424 | 0.000021 | com.findlaw |
516 | 19621134 | 2334 | 0.000014 | com.similarweb |
517 | 19620696 | 481 | 0.000054 | com.gmail |
518 | 19619304 | 3040 | 0.000012 | io.soup |
519 | 19616246 | 1437 | 0.000021 | com.imageshack |
520 | 19615956 | 2785 | 0.000013 | com.sputniknews |
521 | 19614078 | 3080 | 0.000012 | com.smore |
522 | 19613232 | 3246 | 0.000011 | org.iucnredlist |
523 | 19611766 | 3117 | 0.000011 | com.kinja |
524 | 19611760 | 1883 | 0.000016 | com.csmonitor |
525 | 19611604 | 145 | 0.000180 | ru.mail |
526 | 19610088 | 1339 | 0.000022 | gov.uscis |
527 | 19608554 | 446 | 0.000058 | net.secureservercdn |
528 | 19606314 | 3004 | 0.000012 | sh.now |
529 | 19605748 | 427 | 0.000061 | tv.twitch |
530 | 19604994 | 1580 | 0.000019 | link.app |
531 | 19600814 | 440 | 0.000059 | com.statista |
532 | 19599160 | 3676 | 0.000010 | jp.hatenablog |
533 | 19595550 | 4356 | 0.000008 | com.coroflot |
534 | 19595264 | 3177 | 0.000011 | org.jenkins-ci |
535 | 19595158 | 1757 | 0.000017 | gov.oregon |
536 | 19593130 | 3200 | 0.000011 | li.paper |
537 | 19593106 | 3847 | 0.000009 | com.pixar |
538 | 19589878 | 3095 | 0.000011 | com.shell |
539 | 19588194 | 4035 | 0.000009 | com.scienceblogs |
540 | 19586188 | 1625 | 0.000019 | org.amnesty |
541 | 19584824 | 892 | 0.000034 | com.thedailybeast |
542 | 19582464 | 1767 | 0.000017 | org.pypi |
543 | 19582346 | 2149 | 0.000015 | com.foreignpolicy |
544 | 19580310 | 2849 | 0.000012 | com.instapaper |
545 | 19579672 | 2910 | 0.000012 | org.accessnow |
546 | 19578614 | 1602 | 0.000019 | com.surveygizmo |
547 | 19577780 | 1733 | 0.000018 | ca.globalnews |
548 | 19576200 | 3175 | 0.000011 | de.uni-koeln |
549 | 19576198 | 239 | 0.000101 | io.shields |
550 | 19576184 | 3377 | 0.000011 | org.lds |
551 | 19575902 | 2238 | 0.000014 | org.rand |
552 | 19574790 | 207 | 0.000114 | com.salesforce |
553 | 19574544 | 3438 | 0.000010 | net.mootools |
554 | 19574428 | 2357 | 0.000014 | at.ac.univie |
555 | 19574182 | 4050 | 0.000009 | org.marxists |
556 | 19571664 | 2860 | 0.000012 | org.panda |
557 | 19571194 | 2806 | 0.000013 | com.oprah |
558 | 19568576 | 1874 | 0.000016 | com.justia |
559 | 19567970 | 3471 | 0.000010 | org.avaaz |
560 | 19567854 | 2880 | 0.000012 | com.openai |
561 | 19567764 | 3597 | 0.000010 | org.neocities |
562 | 19567260 | 3753 | 0.000009 | cn.edu.sdu |
563 | 19564960 | 762 | 0.000040 | com.netflix |
564 | 19564120 | 498 | 0.000052 | com.oreilly |
565 | 19563086 | 4405 | 0.000008 | com.yam |
566 | 19562248 | 227 | 0.000105 | uk.co.amazon |
567 | 19562204 | 866 | 0.000035 | com.zoho |
568 | 19560956 | 629 | 0.000044 | com.zdnet |
569 | 19559966 | 1298 | 0.000023 | ly.snip |
570 | 19558790 | 1790 | 0.000017 | ch.ipcc |
571 | 19558664 | 993 | 0.000030 | uk.parliament |
572 | 19558508 | 3787 | 0.000009 | com.nestle |
573 | 19556304 | 1254 | 0.000024 | se.google |
574 | 19556292 | 2997 | 0.000012 | com.treehugger |
575 | 19555184 | 1011 | 0.000029 | net.nocookie |
576 | 19555096 | 4644 | 0.000008 | com.x0 |
577 | 19553368 | 3631 | 0.000010 | org.tvtropes |
578 | 19550992 | 1141 | 0.000026 | org.sphinx-doc |
579 | 19549994 | 2122 | 0.000015 | ru.mos |
580 | 19548820 | 3044 | 0.000012 | es.csic |
581 | 19548530 | 2913 | 0.000012 | uk.gov.companieshouse |
582 | 19546576 | 1034 | 0.000029 | com.engadget |
583 | 19546230 | 1183 | 0.000025 | com.here |
584 | 19545492 | 5060 | 0.000007 | com.dbs |
585 | 19545438 | 4103 | 0.000009 | br.ufrj |
586 | 19544204 | 2159 | 0.000015 | edu.colostate |
587 | 19543398 | 2706 | 0.000013 | de.uni-heidelberg |
588 | 19540500 | 3059 | 0.000012 | com.pearltrees |
589 | 19539268 | 2176 | 0.000015 | net.openid |
590 | 19537880 | 2600 | 0.000013 | com.mystrikingly |
591 | 19537844 | 3880 | 0.000009 | com.chinatimes |
592 | 19535834 | 2400 | 0.000014 | link.page |
593 | 19534182 | 2354 | 0.000014 | com.real |
594 | 19533432 | 1836 | 0.000017 | org.ncsl |
595 | 19532288 | 301 | 0.000082 | com.surveymonkey |
596 | 19531930 | 362 | 0.000070 | com.hp |
597 | 19531412 | 1193 | 0.000025 | org.js |
598 | 19530700 | 2135 | 0.000015 | com.123formbuilder |
599 | 19528842 | 2426 | 0.000014 | org.vim |
600 | 19528104 | 3205 | 0.000011 | pl.wp |
601 | 19528018 | 2602 | 0.000013 | au.com.sbs |
602 | 19526780 | 170 | 0.000148 | com.yelp |
603 | 19526216 | 2499 | 0.000013 | uk.ac.kcl |
604 | 19524346 | 1338 | 0.000022 | org.aarp |
605 | 19523692 | 2621 | 0.000013 | th.co.google |
606 | 19523156 | 1006 | 0.000029 | uk.gov.legislation |
607 | 19523042 | 260 | 0.000094 | com.getbootstrap |
608 | 19522856 | 3663 | 0.000010 | com.magcloud |
609 | 19522274 | 3990 | 0.000009 | com.zynga |
610 | 19521942 | 1268 | 0.000023 | tw.com.google |
611 | 19521922 | 2829 | 0.000013 | com.kaggle |
612 | 19520130 | 948 | 0.000031 | gov.gpo |
613 | 19519742 | 946 | 0.000032 | com.about |
614 | 19519714 | 3273 | 0.000011 | org.rsf |
615 | 19518740 | 2976 | 0.000012 | org.tigris |
616 | 19518224 | 2727 | 0.000013 | uk.ac.leeds |
617 | 19515512 | 3535 | 0.000010 | de.dw |
618 | 19515434 | 3019 | 0.000012 | org.cfr |
619 | 19514574 | 3253 | 0.000011 | de.uni-freiburg |
620 | 19513570 | 3640 | 0.000010 | de.uni-konstanz |
621 | 19512714 | 3881 | 0.000009 | ua.at |
622 | 19511254 | 2117 | 0.000015 | info.worldometers |
623 | 19510314 | 4657 | 0.000008 | com.embarcadero |
624 | 19509370 | 2999 | 0.000012 | vn.zing |
625 | 19509134 | 3229 | 0.000011 | com.bangkokpost |
626 | 19508804 | 3615 | 0.000010 | ly.rebrand |
627 | 19508548 | 2008 | 0.000016 | gov.ky |
628 | 19508426 | 4009 | 0.000009 | org.wilsoncenter |
629 | 19506774 | 4059 | 0.000009 | jp.hatenadiary |
630 | 19506284 | 4374 | 0.000008 | com.musictoday |
631 | 19505388 | 3824 | 0.000009 | org.constitutioncenter |
632 | 19505186 | 372 | 0.000067 | com.booking |
633 | 19504402 | 2579 | 0.000013 | com.eiseverywhere |
634 | 19503800 | 4038 | 0.000009 | com.itsnicethat |
635 | 19503776 | 3331 | 0.000011 | il.ac.tau |
636 | 19502096 | 2359 | 0.000014 | mx.com.google |
637 | 19500806 | 3736 | 0.000009 | com.db |
638 | 19498928 | 312 | 0.000080 | com.ebay |
639 | 19498588 | 3578 | 0.000010 | jp.hateblo |
640 | 19498166 | 3348 | 0.000011 | org.democracynow |
641 | 19497296 | 3975 | 0.000009 | edu.odu |
642 | 19496812 | 2815 | 0.000013 | dk.au |
643 | 19496626 | 4220 | 0.000008 | com.etymonline |
644 | 19496184 | 2885 | 0.000012 | uk.gov.metoffice |
645 | 19495756 | 361 | 0.000070 | com.skype |
646 | 19495566 | 3570 | 0.000010 | com.hsbc |
647 | 19494844 | 2228 | 0.000015 | com.bankrate |
648 | 19494104 | 2240 | 0.000014 | gov.wi |
649 | 19493352 | 1815 | 0.000017 | fi.google |
650 | 19493306 | 4426 | 0.000008 | com.x10host |
651 | 19492136 | 3224 | 0.000011 | org.royalsociety |
652 | 19491096 | 817 | 0.000037 | com.pexels |
653 | 19490358 | 532 | 0.000048 | com.mashable |
654 | 19490282 | 4614 | 0.000008 | com.epochtimes |
655 | 19490018 | 1174 | 0.000025 | edu.ucla |
656 | 19489656 | 3226 | 0.000011 | cc.reurl |
657 | 19489414 | 3430 | 0.000010 | com.dailykos |
658 | 19489360 | 3742 | 0.000009 | uk.ac.uea |
659 | 19488050 | 3705 | 0.000010 | ca.shaw |
660 | 19486104 | 1968 | 0.000016 | uk.gov.tfl |
661 | 19485988 | 3434 | 0.000010 | uk.ac.nhm |
662 | 19485032 | 3060 | 0.000012 | com.ipage |
663 | 19484754 | 2498 | 0.000013 | com.prweek |
664 | 19484598 | 1819 | 0.000017 | gov.usembassy |
665 | 19483966 | 4861 | 0.000007 | am.do |
666 | 19483636 | 3086 | 0.000011 | com.viki |
667 | 19483518 | 3252 | 0.000011 | se.liu |
668 | 19482718 | 3066 | 0.000012 | com.coca-colacompany |
669 | 19482580 | 4232 | 0.000008 | br.ufrgs |
670 | 19482498 | 3639 | 0.000010 | de.uni-kiel |
671 | 19481340 | 1453 | 0.000020 | com.speakerdeck |
672 | 19480718 | 3077 | 0.000012 | net.openreview |
673 | 19480660 | 2208 | 0.000015 | de.auswaertiges-amt |
674 | 19480248 | 208 | 0.000113 | com.hubspot |
675 | 19479762 | 2026 | 0.000016 | com.lexisnexis |
676 | 19478700 | 2106 | 0.000015 | net.ucoz |
677 | 19477552 | 3494 | 0.000010 | com.iconarchive |
678 | 19477532 | 819 | 0.000037 | com.steampowered |
679 | 19477286 | 756 | 0.000040 | com.xiti |
680 | 19477132 | 2486 | 0.000013 | com.post-gazette |
681 | 19476898 | 3369 | 0.000011 | com.eklablog |
682 | 19476632 | 2937 | 0.000012 | uk.co.bbci |
683 | 19476378 | 1911 | 0.000016 | hu.google |
684 | 19476160 | 4399 | 0.000008 | com.jacobinmag |
685 | 19475974 | 3323 | 0.000011 | uk.ac.sussex |
686 | 19474368 | 3068 | 0.000012 | uk.ac.qmul |
687 | 19474212 | 3930 | 0.000009 | nf.co |
688 | 19473014 | 4114 | 0.000009 | com.collinsdictionary |
689 | 19472896 | 5215 | 0.000007 | com.evaair |
690 | 19472846 | 2572 | 0.000013 | com.marketwire |
691 | 19472580 | 3138 | 0.000011 | au.com.telstra |
692 | 19472114 | 3916 | 0.000009 | it.unitn |
693 | 19471646 | 898 | 0.000034 | com.visualstudio |
694 | 19471330 | 3807 | 0.000009 | in.ernet |
695 | 19470994 | 2906 | 0.000012 | nl.rug |
696 | 19468708 | 5297 | 0.000007 | org.arkive |
697 | 19468252 | 252 | 0.000096 | org.drupal |
698 | 19467050 | 3460 | 0.000010 | ca.dal |
699 | 19467046 | 3693 | 0.000010 | com.canada |
700 | 19465642 | 1451 | 0.000021 | com.tinypic |
701 | 19465304 | 3136 | 0.000011 | org.wri |
702 | 19465034 | 3698 | 0.000010 | com.la-croix |
703 | 19464108 | 4557 | 0.000008 | com.mitsubishielectric |
704 | 19463828 | 4748 | 0.000008 | com.gamejolt |
705 | 19462976 | 2789 | 0.000013 | gr.google |
706 | 19462882 | 4882 | 0.000007 | cz.webgarden |
707 | 19462404 | 3079 | 0.000012 | my.com.thestar |
708 | 19461830 | 269 | 0.000092 | net.php |
709 | 19461640 | 4329 | 0.000008 | au.gov.fairwork |
710 | 19460770 | 2279 | 0.000014 | co.pcdn |
711 | 19460176 | 3943 | 0.000009 | uk.ac.essex |
712 | 19459984 | 121 | 0.000231 | org.networkadvertising |
713 | 19459684 | 3396 | 0.000010 | org.rferl |
714 | 19459068 | 4211 | 0.000008 | com.sc |
715 | 19459020 | 3292 | 0.000011 | com.blogfa |
716 | 19458794 | 3382 | 0.000010 | ca.yelp |
717 | 19457580 | 4102 | 0.000009 | edu.utm |
718 | 19457248 | 5694 | 0.000007 | com.anghami |
719 | 19456532 | 5210 | 0.000007 | su.clan |
720 | 19456144 | 4095 | 0.000009 | it.justpaste |
721 | 19456006 | 414 | 0.000062 | com.sxsw |
722 | 19455914 | 3258 | 0.000011 | com.waterstones |
723 | 19454602 | 3960 | 0.000009 | com.jigsy |
724 | 19454516 | 838 | 0.000036 | com.intel |
725 | 19454394 | 4032 | 0.000009 | ee.ut |
726 | 19453242 | 916 | 0.000033 | com.docker |
727 | 19452988 | 738 | 0.000041 | com.samsung |
728 | 19451802 | 3422 | 0.000010 | es.ucm |
729 | 19450718 | 2503 | 0.000013 | com.washingtonexaminer |
730 | 19450342 | 3951 | 0.000009 | tl.page |
731 | 19450206 | 2209 | 0.000015 | org.wbur |
732 | 19449036 | 4112 | 0.000009 | site.negocio |
733 | 19448922 | 2773 | 0.000013 | com.yell |
734 | 19448516 | 3988 | 0.000009 | com.fatcow |
735 | 19448266 | 3282 | 0.000011 | pl.poznan |
736 | 19448198 | 135 | 0.000194 | com.youku |
737 | 19447930 | 2878 | 0.000012 | ae.thenational |
738 | 19447766 | 4705 | 0.000008 | id.co.kaskus |
739 | 19447668 | 3407 | 0.000010 | com.afp |
740 | 19447602 | 5336 | 0.000007 | net.manilatimes |
741 | 19446734 | 419 | 0.000062 | com.caniuse |
742 | 19446168 | 1470 | 0.000020 | com.pastebin |
743 | 19445910 | 3387 | 0.000010 | uk.org.rspb |
744 | 19445736 | 765 | 0.000039 | com.moz |
745 | 19444376 | 4027 | 0.000009 | lv.draugiem |
746 | 19441604 | 2508 | 0.000013 | gov.dni |
747 | 19440874 | 2593 | 0.000013 | ro.google |
748 | 19440144 | 2946 | 0.000012 | com.broadwayworld |
749 | 19439574 | 3750 | 0.000009 | ru.msu |
750 | 19439374 | 3766 | 0.000009 | pl.cba |
751 | 19439332 | 4137 | 0.000009 | org.rfa |
752 | 19439280 | 5562 | 0.000007 | org.bukkit |
753 | 19439086 | 2013 | 0.000016 | scot.gov |
754 | 19438868 | 133 | 0.000200 | com.constantcontact |
755 | 19438826 | 5638 | 0.000007 | org.adbusters |
756 | 19438094 | 4517 | 0.000008 | google.design |
757 | 19437654 | 4154 | 0.000008 | com.macobserver |
758 | 19437088 | 1649 | 0.000018 | fr.pagesjaunes |
759 | 19437020 | 2502 | 0.000013 | com.thenation |
760 | 19436776 | 3973 | 0.000009 | com.bbcamerica |
761 | 19434556 | 4857 | 0.000007 | com.orgfree |
762 | 19433810 | 2978 | 0.000012 | com.channelnewsasia |
763 | 19432506 | 735 | 0.000041 | gov.sec |
764 | 19432502 | 4008 | 0.000009 | com.teamspeak |
765 | 19432430 | 2800 | 0.000013 | org.gnupg |
766 | 19432260 | 3780 | 0.000009 | com.the-scientist |
767 | 19432252 | 3015 | 0.000012 | com.laweekly |
768 | 19431446 | 2921 | 0.000012 | au.edu.sydney |
769 | 19430084 | 3577 | 0.000010 | uk.co.yougov |
770 | 19430000 | 3140 | 0.000011 | vn.com.google |
771 | 19429942 | 4417 | 0.000008 | com.50webs |
772 | 19429004 | 3124 | 0.000011 | org.repec |
773 | 19428938 | 3215 | 0.000011 | org.ourworldindata |
774 | 19427890 | 3506 | 0.000010 | com.tradingeconomics |
775 | 19427352 | 3102 | 0.000011 | tw.com.pchome |
776 | 19426582 | 3332 | 0.000011 | com.monday |
777 | 19426556 | 3556 | 0.000010 | org.project-syndicate |
778 | 19425552 | 2331 | 0.000014 | com.amebaownd |
779 | 19424890 | 1596 | 0.000019 | org.whatbrowser |
780 | 19424750 | 1956 | 0.000016 | org.americanbar |
781 | 19424680 | 3739 | 0.000009 | ie.thejournal |
782 | 19424152 | 104 | 0.000298 | com.stripe |
783 | 19424140 | 4014 | 0.000009 | com.hatenadiary |
784 | 19424060 | 2933 | 0.000012 | org.thinkprogress |
785 | 19423712 | 3073 | 0.000012 | uk.gov.london |
786 | 19423054 | 3927 | 0.000009 | com.thesaurus |
787 | 19423006 | 3475 | 0.000010 | net.webself |
788 | 19422964 | 3432 | 0.000010 | io.pantheon |
789 | 19421712 | 3420 | 0.000010 | uk.ac.exeter |
790 | 19421508 | 4343 | 0.000008 | com.appledaily |
791 | 19421118 | 3528 | 0.000010 | com.bravesites |
792 | 19420816 | 5178 | 0.000007 | com.bambuser |
793 | 19420592 | 3379 | 0.000011 | com.foreignaffairs |
794 | 19419378 | 2432 | 0.000013 | com.instructables |
795 | 19416388 | 2185 | 0.000015 | vn.vietnamnet |
796 | 19414736 | 3994 | 0.000009 | com.webcindario |
797 | 19414328 | 2823 | 0.000013 | org.ewg |
798 | 19413934 | 4534 | 0.000008 | ws.nimb |
799 | 19413778 | 2833 | 0.000013 | org.fullfact |
800 | 19413352 | 256 | 0.000095 | us.zoom |
801 | 19412556 | 3685 | 0.000010 | com.encyclopedia |
802 | 19412474 | 3897 | 0.000009 | de.uni-erlangen |
803 | 19410822 | 5341 | 0.000007 | net.boards |
804 | 19409598 | 341 | 0.000074 | com.histats |
805 | 19409534 | 4201 | 0.000008 | is.pse |
806 | 19409436 | 748 | 0.000040 | fm.last |
807 | 19407808 | 3661 | 0.000010 | com.mongabay |
808 | 19407040 | 3220 | 0.000011 | me.site123 |
809 | 19406338 | 3436 | 0.000010 | com.seetickets |
810 | 19405550 | 5838 | 0.000007 | com.gamigo |
811 | 19404400 | 1666 | 0.000018 | com.materialdesignicons |
812 | 19404108 | 5140 | 0.000007 | bd.com.google |
813 | 19403242 | 790 | 0.000038 | com.venturebeat |
814 | 19401218 | 4601 | 0.000008 | uk.org.phrases |
815 | 19400780 | 3213 | 0.000011 | com.instructure |
816 | 19400298 | 2817 | 0.000013 | gov.arkansas |
817 | 19399890 | 72 | 0.000444 | com.livestream |
818 | 19399554 | 4081 | 0.000009 | cat.uab |
819 | 19399486 | 3546 | 0.000010 | org.lacity |
820 | 19399372 | 3612 | 0.000010 | com.heraldscotland |
821 | 19398370 | 1499 | 0.000020 | com.teachable |
822 | 19396672 | 2895 | 0.000012 | com.foodandwine |
823 | 19395752 | 1233 | 0.000024 | com.createjs |
824 | 19394274 | 2266 | 0.000014 | com.ajc |
825 | 19394172 | 3950 | 0.000009 | com.rappler |
826 | 19394030 | 2355 | 0.000014 | net.noscript |
827 | 19393982 | 4140 | 0.000009 | jp.doorblog |
828 | 19392882 | 2873 | 0.000012 | com.timeshighereducation |
829 | 19392238 | 275 | 0.000089 | com.bandcamp |
830 | 19389332 | 3969 | 0.000009 | jp.ne.hi-ho |
831 | 19388094 | 3629 | 0.000010 | net.inquirer |
832 | 19387882 | 552 | 0.000047 | com.cisco |
833 | 19387318 | 4076 | 0.000009 | pl.lublin |
834 | 19386370 | 1657 | 0.000018 | com.pcworld |
835 | 19383404 | 266 | 0.000093 | com.typeform |
836 | 19382886 | 203 | 0.000116 | com.naver |
837 | 19382698 | 3723 | 0.000010 | gov.bts |
838 | 19382192 | 1816 | 0.000017 | jp.makeshop |
839 | 19382102 | 4462 | 0.000008 | com.tor |
840 | 19382072 | 4513 | 0.000008 | com.weightwatchers |
841 | 19381346 | 1438 | 0.000021 | org.khanacademy |
842 | 19381274 | 954 | 0.000031 | com.thinkwithgoogle |
843 | 19381020 | 3385 | 0.000010 | uk.ac.jisc |
844 | 19380238 | 4088 | 0.000009 | ly.genial |
845 | 19379986 | 4007 | 0.000009 | com.themoscowtimes |
846 | 19378500 | 3272 | 0.000011 | com.nyt |
847 | 19378434 | 3760 | 0.000009 | com.springernature |
848 | 19378356 | 3390 | 0.000010 | int.cbd |
849 | 19377854 | 6045 | 0.000006 | es.xurl |
850 | 19376898 | 1756 | 0.000017 | com.netsolhost |
851 | 19376598 | 3852 | 0.000009 | au.edu.griffith |
852 | 19376054 | 4740 | 0.000008 | co.edu.unal |
853 | 19376040 | 4074 | 0.000009 | kr.co.koreatimes |
854 | 19374588 | 727 | 0.000042 | com.deloitte |
855 | 19374300 | 4986 | 0.000007 | org.edc |
856 | 19373940 | 4149 | 0.000008 | vn.tienphong |
857 | 19373476 | 3515 | 0.000010 | com.thediplomat |
858 | 19372932 | 4099 | 0.000009 | uk.ac.lancs |
859 | 19372798 | 5006 | 0.000007 | com.inoreader |
860 | 19372746 | 4922 | 0.000007 | com.ueuo |
861 | 19372594 | 1585 | 0.000019 | tv.ustream |
862 | 19372576 | 3234 | 0.000011 | com.tapatalk |
863 | 19372356 | 3416 | 0.000010 | nl.wur |
864 | 19372106 | 4848 | 0.000007 | net.hypermart |
865 | 19371636 | 2293 | 0.000014 | org.kff |
866 | 19369356 | 398 | 0.000064 | com.pubmatic |
867 | 19368982 | 3625 | 0.000010 | org.grist |
868 | 19368480 | 3088 | 0.000011 | tw.gov.cdc |
869 | 19368288 | 3389 | 0.000010 | com.gothamist |
870 | 19368130 | 1106 | 0.000027 | com.gizmodo |
871 | 19368116 | 4101 | 0.000009 | com.globalpost |
872 | 19367676 | 814 | 0.000037 | gov.nist |
873 | 19367536 | 4563 | 0.000008 | org.globalsecurity |
874 | 19366454 | 4547 | 0.000008 | build.bazel |
875 | 19366384 | 3782 | 0.000009 | us.ms.state |
876 | 19365878 | 4256 | 0.000008 | gr.ntua |
877 | 19365776 | 4444 | 0.000008 | se.thelocal |
878 | 19365372 | 2963 | 0.000012 | com.politifact |
879 | 19365128 | 1317 | 0.000023 | com.ensighten |
880 | 19363588 | 5097 | 0.000007 | ru.my1 |
881 | 19362680 | 3468 | 0.000010 | com.rabbitmq |
882 | 19359698 | 4138 | 0.000009 | com.elasticbeanstalk |
883 | 19359574 | 1364 | 0.000022 | com.billboard |
884 | 19359122 | 4766 | 0.000008 | cc.dict |
885 | 19358774 | 5687 | 0.000007 | fi.mbnet |
886 | 19357390 | 879 | 0.000035 | com.aliexpress |
887 | 19356918 | 210 | 0.000111 | to.amzn |
888 | 19355668 | 4275 | 0.000008 | edu.ohio |
889 | 19355546 | 3452 | 0.000010 | com.thejakartapost |
890 | 19355350 | 3277 | 0.000011 | vn.com.dantri |
891 | 19355080 | 5285 | 0.000007 | com.galvanize |
892 | 19354880 | 3484 | 0.000010 | jp.go.ndl |
893 | 19354790 | 4710 | 0.000008 | com.kiwibox |
894 | 19354514 | 2140 | 0.000015 | org.linuxfoundation |
895 | 19354500 | 4801 | 0.000007 | ru.nnov |
896 | 19353166 | 4288 | 0.000008 | gr.auth |
897 | 19352970 | 2257 | 0.000014 | net.vnexpress |
898 | 19351770 | 2900 | 0.000012 | com.crashlytics |
899 | 19351594 | 1045 | 0.000028 | com.dropboxusercontent |
900 | 19350828 | 3439 | 0.000010 | com.scotusblog |
901 | 19350712 | 4090 | 0.000009 | org.carnegieendowment |
902 | 19350278 | 395 | 0.000064 | com.atlassian |
903 | 19349726 | 3465 | 0.000010 | com.study |
904 | 19348724 | 350 | 0.000072 | com.mapbox |
905 | 19348532 | 1046 | 0.000028 | com.redhat |
906 | 19347886 | 1799 | 0.000017 | com.bravenet |
907 | 19347460 | 4284 | 0.000008 | uk.org.npg |
908 | 19347152 | 4463 | 0.000008 | com.btplc |
909 | 19347148 | 5289 | 0.000007 | ru.drom |
910 | 19346542 | 2430 | 0.000013 | com.vimeopro |
911 | 19345900 | 4419 | 0.000008 | edu.marquette |
912 | 19345644 | 426 | 0.000061 | com.adweek |
913 | 19345144 | 914 | 0.000033 | com.shutterstock |
914 | 19345090 | 1016 | 0.000029 | com.ubuntu |
915 | 19341960 | 5712 | 0.000007 | in.ac.nptel |
916 | 19341488 | 1227 | 0.000024 | com.msdn |
917 | 19340714 | 4707 | 0.000008 | com.vocabulary |
918 | 19340680 | 3929 | 0.000009 | edu.uaf |
919 | 19339658 | 3919 | 0.000009 | com.atavist |
920 | 19339456 | 3201 | 0.000011 | com.healthgrades |
921 | 19339092 | 2546 | 0.000013 | com.kinstacdn |
922 | 19338384 | 2345 | 0.000014 | com.gazhall |
923 | 19337938 | 5398 | 0.000007 | com.asmallorange |
924 | 19337800 | 3797 | 0.000009 | com.generalmills |
925 | 19336176 | 4585 | 0.000008 | vn.vtc |
926 | 19335908 | 1519 | 0.000020 | cn.gov.mofcom |
927 | 19333778 | 797 | 0.000038 | com.box |
928 | 19333606 | 3966 | 0.000009 | si.uni-lj |
929 | 19333322 | 4170 | 0.000008 | az.president |
930 | 19333194 | 1788 | 0.000017 | org.reactjs |
931 | 19332412 | 3605 | 0.000010 | com.postaffiliatepro |
932 | 19331922 | 5192 | 0.000007 | edu.uah |
933 | 19331280 | 3599 | 0.000010 | org.openedition |
934 | 19330696 | 4838 | 0.000007 | com.kapook |
935 | 19330382 | 4153 | 0.000008 | org.caringbridge |
936 | 19330374 | 483 | 0.000053 | com.aol |
937 | 19329614 | 2303 | 0.000014 | org.nfpa |
938 | 19329538 | 5956 | 0.000006 | com.glosbe |
939 | 19329194 | 4124 | 0.000009 | com.mcall |
940 | 19327622 | 4289 | 0.000008 | ru.tmweb |
941 | 19326876 | 4126 | 0.000009 | uk.co.liverpoolecho |
942 | 19326422 | 4244 | 0.000008 | com.atwebpages |
943 | 19325980 | 1067 | 0.000028 | com.freepik |
944 | 19324790 | 4085 | 0.000009 | org.specialolympics |
945 | 19323868 | 4845 | 0.000007 | net.freeforums |
946 | 19323676 | 4744 | 0.000008 | uk.ac.westminster |
947 | 19323532 | 4092 | 0.000009 | com.tok2 |
948 | 19323460 | 1025 | 0.000029 | com.elpais |
949 | 19323150 | 4946 | 0.000007 | tw.com.sina |
950 | 19322508 | 3296 | 0.000011 | com.wowza |
951 | 19322306 | 317 | 0.000079 | com.webs |
952 | 19322024 | 4697 | 0.000008 | com.warriorplus |
953 | 19321918 | 3414 | 0.000010 | com.cityam |
954 | 19321812 | 4482 | 0.000008 | org.fee |
955 | 19321520 | 4854 | 0.000007 | tw.edu.ntnu |
956 | 19321296 | 4962 | 0.000007 | com.sparknotes |
957 | 19320202 | 4516 | 0.000008 | com.newspapers |
958 | 19319634 | 2192 | 0.000015 | com.tutsplus |
959 | 19319600 | 5868 | 0.000007 | com.ananova |
960 | 19319274 | 3818 | 0.000009 | org.opensecrets |
961 | 19319134 | 633 | 0.000044 | gov.uspto |
962 | 19318722 | 5680 | 0.000007 | su.moy |
963 | 19318366 | 1013 | 0.000029 | com.uk |
964 | 19318266 | 4936 | 0.000007 | ru.pr-cy |
965 | 19318058 | 3827 | 0.000009 | cz.centrum |
966 | 19317780 | 4158 | 0.000008 | edu.niu |
967 | 19315320 | 1665 | 0.000018 | org.webkit |
968 | 19315014 | 4692 | 0.000008 | pl.edu.amu |
969 | 19314084 | 5186 | 0.000007 | com.artfire |
970 | 19313894 | 3800 | 0.000009 | org.ascd |
971 | 19312106 | 3801 | 0.000009 | edu.scu |
972 | 19311742 | 4307 | 0.000008 | com.taipeitimes |
973 | 19311568 | 4351 | 0.000008 | edu.whoi |
974 | 19310854 | 5949 | 0.000006 | com.voatiengviet |
975 | 19310748 | 3100 | 0.000011 | com.broadcastingcable |
976 | 19310720 | 4655 | 0.000008 | hk.rthk |
977 | 19310246 | 5703 | 0.000007 | com.enotes |
978 | 19309910 | 488 | 0.000053 | com.indiatimes |
979 | 19309660 | 860 | 0.000035 | com.playstation |
980 | 19309040 | 4866 | 0.000007 | com.brothersoft |
981 | 19308948 | 2708 | 0.000013 | uk.gov.defra |
982 | 19307606 | 231 | 0.000103 | org.whatwg |
983 | 19307178 | 4451 | 0.000008 | com.batchgeo |
984 | 19307118 | 751 | 0.000040 | com.psychologytoday |
985 | 19306368 | 4263 | 0.000008 | uk.co.lrb |
986 | 19306350 | 5034 | 0.000007 | ca.pe.gov |
987 | 19305884 | 4159 | 0.000008 | com.ecowatch |
988 | 19303820 | 4195 | 0.000008 | com.williamhill |
989 | 19303548 | 5767 | 0.000007 | pt.ipp |
990 | 19302972 | 4843 | 0.000007 | uk.org.38degrees |
991 | 19301624 | 1303 | 0.000023 | com.technologyreview |
992 | 19301464 | 4091 | 0.000009 | org.spie |
993 | 19301068 | 959 | 0.000031 | com.libsyn |
994 | 19300572 | 4795 | 0.000007 | com.storeboard |
995 | 19300548 | 3260 | 0.000011 | de.bmel |
996 | 19299448 | 4749 | 0.000008 | net.onlinewebshop |
997 | 19299274 | 3872 | 0.000009 | ru.1gb |
998 | 19298654 | 279 | 0.000088 | com.automattic |
999 | 19298502 | 3870 | 0.000009 | com.piie |
1000 | 19297440 | 5306 | 0.000007 | com.allthatsinteresting |
Credits
Thanks to the authors of the WebGraph framework, whose software made the computation of graph properties and ranks possible.
We hope the data will be useful for you to do any kind of research on ranking, graph analysis, link spam detection, etc. Let us know about your results via Common Crawl’s Google Group!
May/June 2020 crawl archive now available
The crawl archive for May/June 2020 is now available! It contains 2.75 billion web pages or 255 TiB of uncompressed content, crawled between May 24th and June 7th. It includes page captures of 1.2 billion URLs unknown in any of our prior crawl archives.
Starting with this crawl the WET files indicate the natural language(s) a text is written in. The language is detected using Compact Language Detector 2 (CLD2) and was made available since August 2018 only in WARC and WAT files and URL indexes. It is now also provided in WET files in the WARC header "WARC-Identified-Content-Language". Up to three language(s) are detected per document and given as comma-separated list of ISO-639-3 codes, here one example WET record fragment:
... WARC-Identified-Content-Language: isl,eng Content-Type: text/plain Content-Length: 10494 Bananabrauð með Nutella – Ljúfmeti og lekkerheit ...
Additional information about this improvement is given in the corresponding issue report.
Archive Location and Download
The May/June crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-24/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://commoncrawl.s3.amazonaws.com/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-24/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-24/warc.paths.gz | 60000 | 53.16 |
WAT files | CC-MAIN-2020-24/wat.paths.gz | 60000 | 19.02 |
WET files | CC-MAIN-2020-24/wet.paths.gz | 60000 | 8.42 |
Robots.txt files | CC-MAIN-2020-24/robotstxt.paths.gz | 60000 | 0.22 |
Non-200 responses files | CC-MAIN-2020-24/non200responses.paths.gz | 60000 | 2.77 |
URL index files | CC-MAIN-2020-24/cc-index.paths.gz | 302 | 0.22 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-24/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.
March/April 2020 crawl archive now available
The crawl archive for March/April 2020 is now available! It contains 2.85 billion web pages or 280 TiB of uncompressed content, crawled between March 28th and April 10th. It includes page captures of 1 billion URLs unknown in any of our prior crawl archives.
Archive Location and Download
The March/April crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2020-16/.
To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files.
By simply adding either s3://commoncrawl/ or https://commoncrawl.s3.amazonaws.com/ to each line, you end up with the S3 and HTTP paths respectively.
File List | #Files | Total Size Compressed (TiB) |
|
---|---|---|---|
Segments | CC-MAIN-2020-16/segment.paths.gz | 100 | |
WARC files | CC-MAIN-2020-16/warc.paths.gz | 56000 | 62.67 |
WAT files | CC-MAIN-2020-16/wat.paths.gz | 56000 | 20.37 |
WET files | CC-MAIN-2020-16/wet.paths.gz | 56000 | 8.97 |
Robots.txt files | CC-MAIN-2020-16/robotstxt.paths.gz | 56000 | 0.19 |
Non-200 responses files | CC-MAIN-2020-16/non200responses.paths.gz | 56000 | 1.39 |
URL index files | CC-MAIN-2020-16/cc-index.paths.gz | 302 | 0.21 |
The Common Crawl URL Index for this crawl is available at: https://index.commoncrawl.org/CC-MAIN-2020-16/. Also the columnar index has been updated to contain this crawl.
Please donate to Common Crawl if you appreciate our free datasets! We’re also seeking corporate sponsors to partner with Common Crawl for our non-profit work in open data. Please contact [email protected] for sponsorship information.