We are pleased to announce a new release of host-level and domain-level web graphs based on the crawls of February, March/April and May/June 2020. Additional information about the data formats, the processing pipeline, our objectives, and credits can be found in the announcements of prior webgraph releases (e.g., Nov/Dec/Jan 2017-2018 Webgraphs). You may also visit the projects cc-webgraph and cc-pyspark which host all scripts and tools required to construct the graphs.

What’s new?

The host-level graph now includes hosts visited by the crawler but not linking to any other host. Why is this possible – isn’t any host found via links the crawler is following? Yes, but some links were already detected in a prior crawl, not in one of the 3 crawls used to build the web graphs. More details about the issue are given in cc-pyspark#15. The impact of this fix on the graph size is minimal: the recent crawl now includes 1 million nodes (0.1% of all nodes) which are not connected to any other node.

Host-level graph

The graph consists of 927 million nodes and 3.88 billion edges and includes dangling nodes i.e. hosts that have not been crawled yet are pointed to from a link on a crawled page. There are 857 million dangling nodes (92.5%) and the largest strongly connected component contains 47 million (5.1%) nodes.

You can download the graph and the ranks of all 927 million hosts from AWS S3 on the path s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/host/. Alternatively, you can use https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/host/ as prefix to access the files from everywhere.

Download files of the Common Crawl Feb/Mar/May 2020 host-level webgraph

SizeFileDescription
5.67 GBcc-main-2020-feb-mar-may-host-vertices.paths.gznodes ⟨id, rev host⟩, paths of 12 vertices files
17.26 GBcc-main-2020-feb-mar-may-host-edges.paths.gzedges ⟨from_id, to_id⟩, paths of 24 edges files
7.40 GBcc-main-2020-feb-mar-may-host.graphgraph in BVGraph format
2 kBcc-main-2020-feb-mar-may-host.properties
8.57 GBcc-main-2020-feb-mar-may-host-t.graphtranspose of the graph (outlinks inverted to inlinks)
2 kBcc-main-2020-feb-mar-may-host-t.properties
1 kBcc-main-2020-feb-mar-may-host.statsWebGraph statistics
12.16 GBcc-main-2020-feb-mar-may-host-ranks.txt.gzharmonic centrality and pagerank

Note that the host names are reversed and a leading www. is stripped: www.subdomain.example.com becomes com.example.subdomain.

Domain-level graph

The domain graph was built by aggregating the host graph on the level of pay-level domains (PLDs) based on the public suffix list maintained on publicsuffix.org.

The domain-level graph has 91 million nodes and 1.96 billion edges. 51% or 46 million nodes are dangling nodes, the largest strongly connected component covers 36 million or 39% of the nodes.

All files related to the domain graph are available on AWS S3 under s3://commoncrawl/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/domain/ resp. https://data.commoncrawl.org/projects/hyperlinkgraph/cc-main-2020-feb-mar-may/domain/.

Download files of the Common Crawl Feb/Mar/May 2020 domain-level webgraph

SizeFileDescription
0.62 GBcc-main-2020-feb-mar-may-domain-vertices.txt.gznodes ⟨id, rev domain, num hosts⟩
7.79 GBcc-main-2020-feb-mar-may-domain-edges.txt.gzedges ⟨from_id, to_id⟩
4.23 GBcc-main-2020-feb-mar-may-domain.graphgraph in BVGraph format
2 kBcc-main-2020-feb-mar-may-domain.properties
4.16 GBcc-main-2020-feb-mar-may-domain-t.graphtranspose of the graph
2 kBcc-main-2020-feb-mar-may-domain-t.properties
1 kBcc-main-2020-feb-mar-may-domain.statsWebGraph statistics
1.96 GBcc-main-2020-feb-mar-may-domain-ranks.txt.gzharmonic centrality and pagerank

Below you’ll find the top 1000 domains ranked by Harmonic Centrality or PageRank. The full list of all 91 million domain ranks is available for download.

Top 1000 domains ranked by harmonic centrality (Feb/Mar/May 2020)

harmonic
centrality
rank
hc valuepage rankpage rank
value
reversed hostname
13266761810.018180com.googleapis
23055277230.011873com.facebook
32956908820.013789com.google
42692046040.007145com.twitter
52688312850.007106org.w
62636044860.006483com.youtube
72471939690.004210com.instagram
82425194280.005125org.gmpg
92384133270.005329com.googletagmanager
1023606890130.002940com.linkedin
1122741292100.003621com.cloudflare
1222732960120.002974org.wordpress
1322661910140.002515com.gravatar
1422577680150.002438com.gstatic
1522378134220.001529com.pinterest
1622196962270.001192org.wikipedia
1722189650190.001864com.wordpress
1822066028160.002404com.bootstrapcdn
1921967760180.001884com.apple
2021751768200.001863com.jquery
2121589606240.001461com.microsoft
2221568908440.000785be.youtu
2321568474430.000806com.blogspot
2421533280310.001104com.vimeo
2521415938460.000761gl.goo
2621399120350.001040com.amazonaws
2721358048530.000665com.amazon
2821331634210.001737com.adobe
2921324666230.001506com.wp
3021209012700.000452com.tumblr
3121184360170.001949com.github
3221150652370.001008com.google-analytics
3321110976300.001152com.baidu
3421096692870.000387com.yahoo
3521081268590.000547ly.bit
3621060360330.001072com.macromedia
3721046916360.001035net.cloudfront
3821036258450.000763com.flickr
3920997926320.001101com.googlesyndication
4020993476260.001277me.wp
4120980462970.000340com.googleusercontent
4220966446560.000624eu.europa
4320960242420.000807net.jsdelivr
4420959910520.000677co.t
4520901872290.001163ru.yandex
4620846092500.000742net.doubleclick
4720843032410.000869com.addthis
4820823518690.000457io.github
4920817952760.000433com.medium
5020810030250.001287com.fontawesome
51208091201390.000189com.forbes
5220796434610.000510org.w3
5320759102550.000640com.paypal
54207572661090.000282com.soundcloud
5520754514900.000368org.creativecommons
5620747472570.000619com.vk
5720711184540.000658org.mozilla
5820710182880.000382com.weebly
5920698442840.000410com.wix
60206753721020.000317com.weibo
6120663930580.000604org.schema
62206502021640.000151com.imgur
63206444521470.000177org.apache
64206422821780.000138uk.co.bbc
65206255601290.000210org.archive
66206103542740.000089com.ibm
67206096141540.000169com.bing
68206023801910.000125net.sourceforge
69205790121300.000207com.nytimes
70205786261500.000174int.who
71205710121830.000131com.cnn
72205616741740.000140net.slideshare
73205476341580.000164gov.cdc
74205425462020.000116com.android
75205272302280.000104com.wsj
76205185481940.000122edu.stanford
77205055462050.000115com.businessinsider
78204950342540.000095com.oracle
7920489434340.001049net.fbcdn
80204888683730.000067com.msn
81204882822610.000093edu.harvard
82204833843100.000080com.go
8320478152990.000335com.shopify
84204714242670.000093com.bbc
85204644342970.000083edu.mit
86204613403300.000076com.myspace
8720458776620.000497com.whatsapp
88204572062890.000085com.appspot
89204544663070.000080com.wired
90204463002920.000085com.reuters
91204420041010.000323com.godaddy
92204355501710.000147com.theguardian
93204177701430.000182gov.nih
94204125361960.000120org.ietf
95204013303880.000065gov.nasa
96203972984230.000061com.theverge
97203947361490.000175com.giphy
98203942763820.000066net.researchgate
99203849302700.000092com.bloomberg
100203777781080.000285com.unpkg
101203763941140.000271com.reddit
102203738563370.000075com.xinhuanet
103203667362150.000108org.gnu
104203635063180.000079com.usatoday
105203526608130.000037org.chromium
106203449963560.000071com.springer
10720343678980.000335de.google
10820342420280.001184com.qq
109203418243450.000073com.example
110203365107440.000041edu.psu
111203245364680.000055edu.cornell
112203243781840.000131com.blogger
11320314024600.000516net.akamaihd
114203042423750.000067org.hbr
115203023107500.000040com.git-scm
116203000149370.000032com.wikia
117202985461370.000191com.spotify
118202960124850.000053edu.yale
119202955161130.000271com.jimdo
120202931405540.000047com.cbsnews
121202919467170.000043com.economist
122202905742140.000109com.washingtonpost
123202885041400.000188jp.co.yahoo
124202864702850.000086com.huffingtonpost
125202845583160.000080org.un
126202818744100.000063fr.free
127202799464730.000054edu.berkeley
128202754462870.000086com.cnbc
129202732802450.000099com.dribbble
130202715845760.000046org.arxiv
131202697161510.000172com.issuu
132202570385450.000047com.mysql
133202562621600.000157com.twimg
134202525321070.000285com.statcounter
135202516823380.000075uk.co.telegraph
136202474783050.000081com.w3schools
137202466825610.000047com.gitlab
138202422108020.000038edu.columbia
139202409785240.000049gov.noaa
140202306661220.000230com.ytimg
141202299001190.000233com.youtube-nocookie
142202276567310.000042org.ieee
143202271263330.000075org.npr
144202255287290.000042io.readthedocs
145202252062860.000086org.acm
146202223143390.000074com.time
1472022043011800.000025org.eclipse
148202203822410.000100org.ampproject
149202186163440.000074com.fc2
150202157301420.000185com.wixsite
151202136927550.000040edu.washington
152202101224210.000061com.force
153202098642760.000089com.prnewswire
154202091305000.000052com.buzzfeed
155202071364340.000060com.nationalgeographic
156202064024030.000063com.nature
157202038262000.000118gle.forms
158202024907990.000038org.sciencemag
159202011444280.000061com.theatlantic
160202001048710.000035com.stackexchange
161201981422800.000088com.sciencedirect
162201854003320.000075com.staticflickr
163201845284950.000052uk.co.independent
164201822562630.000093gov.ca
165201809726870.000043org.worldbank
166201759944350.000060com.mozilla
167201754007340.000041com.marketwatch
1682016809810870.000027com.hatenablog
169201670403640.000069com.nypost
170201640166460.000043org.bitbucket
171201611922190.000107com.ft
172201511164630.000056com.pixabay
173201437963540.000071jp.co.rakuten
174201426527430.000041edu.upenn
175201401262770.000089org.doi
176201393769660.000031jp.livedoor
177201365461980.000120uk.co.google
178201349324070.000063uk.co.dailymail
179201344047240.000042org.pbs
180201339362580.000094net.behance
181201329141920.000124org.wikimedia
182201278609170.000033edu.jhu
183201278284540.000057gov.whitehouse
184201223528560.000035org.weforum
185201221704160.000062com.dailymotion
1862011705414870.000020com.warnerbros
187201118983260.000077org.opensource
1882011079810910.000027cn.com.chinadaily
189201099165480.000047me.about
190201098202320.000103jp.ameblo
191201089405580.000047com.oup
192201034283250.000077com.digg
193200974184550.000056com.entrepreneur
194200951086310.000044com.vice
195200941427490.000040com.qz
1962009269212590.000024com.discovery
197200911544440.000058com.goodreads
198200910524470.000057gg.discord
1992008291011090.000027com.sap
200200821863530.000071com.scribd
201200794121880.000128com.feedburner
202200761464660.000055com.fortune
203200755565800.000045com.gartner
2042007259810120.000029com.500px
205200721364580.000056jp.ne.sakura
206200674001760.000139com.imdb
207200609507320.000042uk.co.blogspot
2082005905417350.000018com.amd
209200582289470.000032edu.princeton
210200566668900.000034org.cambridge
21120056572510.000714com.fb
212200562728480.000036com.evernote
213200544721440.000180com.dropbox
21420053532390.000951com.wixstatic
215200516626170.000044org.unesco
2162005094014610.000020com.fandom
217200481522940.000084com.wiley
218200461347680.000039com.withgoogle
2192003942610150.000029org.altervista
2202003901023370.000014com.wolfram
221200379207980.000038com.slate
2222003148412010.000025org.kernel
2232002816410490.000028edu.purdue
224200252825690.000046page.g
225200213407860.000038com.trello
226200170182300.000103com.disqus
227200127967570.000040org.eff
228200104309510.000031com.merriam-webster
229200046864930.000052gov.usda
230200042409810.000030com.netlify
2312000399421790.000015com.diigo
232200029188070.000038com.vox
233200026901800.000135org.allaboutcookies
2342000222012060.000025com.jetbrains
2351999941814160.000021edu.arizona
236199943845420.000047com.tandfonline
237199930308440.000036com.foxnews
238199921842910.000085com.live
239199911421750.000140com.xing
240199898749090.000033com.politico
241199885703200.000079com.outlook
2421998503611350.000026jp.ne.goo
243199833407540.000040au.net.abc
2441998268019450.000016com.wikidot
245199779347930.000038com.investopedia
2461997757410660.000028edu.uchicago
2471997682010090.000029edu.wisc
248199759221970.000120com.eepurl
2491997256010390.000028com.bostonglobe
250199720967750.000039org.semver
251199695946190.000044com.sagepub
252199691824970.000052gov.fda
253199684423470.000073net.windows
2541996808415680.000019edu.osu
255199653863190.000079com.nbcnews
256199639462440.000099com.myshopify
257199628925850.000045cn.google
258199625306080.000044site.business
259199610668320.000036com.sciencedaily
2601996038010440.000028com.strikingly
2611995636612360.000024edu.unc
2621995626814460.000021edu.virginia
2631995603412040.000025co.elastic
2641995296011940.000025com.nymag
2651995050022060.000015com.renren
266199504907420.000041gov.house
2671995044821630.000015sg.edu.nus
2681994797622850.000014org.wikibooks
2691994728419610.000016com.googlesource
270199405982350.000103com.wpengine
271199401583230.000078com.googlecode
272199392127610.000040gov.senate
273199380085130.000051com.herokuapp
274199377384520.000057org.pewresearch
275199374925670.000046org.iana
2761993695410930.000027com.podbean
277199358189820.000030com.alexa
2781993474216290.000019gd.is
279199338041030.000301com.paypalobjects
280199327408050.000038org.unicef
281199324167180.000043com.newyorker
282199308589690.000031uk.co.thetimes
283199293244040.000063com.patreon
2841992826610600.000028com.lifehacker
285199259403810.000066com.criteo
286199245249970.000030com.huffpost
287199225763030.000081com.squareup
288199225108390.000036ca.cbc
2891992180811450.000026org.wiktionary
290199188441460.000178com.addtoany
291199181742010.000117com.optimizely
2921991805213420.000022edu.msu
2931991598613710.000022com.history
294199133844180.000062com.calendly
2951990586011810.000025com.udemy
296199033648090.000037uk.ac.ox
297199029201720.000145com.amazon-adsystem
29819899332490.000743com.googleadservices
299198969241550.000167com.opera
300198909708870.000034org.fao
3011989083210170.000029com.ecwid
302198908264760.000054com.googleblog
303198871422110.000110com.stackoverflow
3041988619014190.000021uk.ac.lse
305198853123600.000070com.getpocket
3061988445616670.000018org.maven
307198838009150.000033uk.co.guardian
308198833581690.000148org.bbb
3091988108413370.000022com.aljazeera
310198807902550.000095com.aliyuncs
3111987993827230.000013net.pixnet
3121987438431800.000011net.hinet
3131986902811700.000025com.smithsonianmag
3141986883213470.000022edu.ucdavis
315198682588940.000034gov.congress
3161986719013200.000023edu.illinois
3171986516811200.000026com.theglobeandmail
3181986330610360.000029gov.archives
319198624144920.000052it.placehold
32019861934930.000359net.facebook
3211986137616150.000019hk.com.google
3221986092214730.000020ca.sfu
3231985635216760.000018blog.home
3241985529010730.000027com.apnews
325198548929630.000031com.ssrn
3261985368233830.000010com.wizards
3271985110219970.000016com.nabble
328198510327600.000040com.chinaz
3291985041236670.000010cn.edu.sjtu
3301984814014840.000020com.urbandictionary
3311984443611360.000026com.scmp
3321984232614890.000020ms.1drv
3331984179643610.000008tw.com.gamer
3341983858213920.000021com.flipboard
335198381669190.000033co.g
336198375425470.000047com.gofundme
3371983699620970.000015com.france24
3381983563614050.000021jp.geocities
3391983365413700.000022com.ibtimes
340198313625810.000045com.biomedcentral
3411983005611280.000026com.britannica
3421982942021740.000015com.oregonlive
343198270624120.000062com.kickstarter
344198262149620.000031com.adjust
345198241888670.000035gov.fcc
346198240487150.000043uk.co.mirror
347198232665890.000045us.icio
3481982317211290.000026com.mediafire
3491982176814320.000021edu.tamu
350198213105870.000045com.usnews
3511982044213140.000023org.greenpeace
352198202529850.000030edu.academia
3531981948613810.000021com.livescience
3541981597216840.000018gov.cia
3551981456413250.000023com.akamai
356198132669300.000032com.chicagotribune
357198115381560.000167com.npmjs
3581981110014290.000021net.seesaa
359198101203290.000076es.google
3601980971012380.000024com.reverbnation
361198094905500.000047com.quora
3621980831434810.000010com.proboards
3631980626810400.000028com.thehill
364198038403210.000078org.python
3651980147611320.000026org.jstor
3661980101817220.000018ca.mcgill
367197999821670.000149com.zendesk
368197928909990.000030com.thelancet
3691979224610940.000027com.jamanetwork
3701978859419350.000016uk.ac.manchester
371197852145400.000048com.udacity
3721978332813720.000021ca.utoronto
373197830825790.000046com.bigcartel
3741978223024870.000013org.wikiquote
3751978118613570.000022edu.rutgers
376197800288960.000034org.apa
377197797184390.000059com.newsweek
378197785389200.000033com.healthline
3791977798222040.000015com.knowyourmeme
380197756103280.000077com.tinyurl
381197755587260.000042gov.state
382197750922160.000108com.unsplash
3831977370217080.000018ca.ualberta
384197723784060.000063com.githubusercontent
3851977190014710.000020com.asahi
386197712202590.000094org.nodejs
387197694364750.000054com.latimes
3881976925810270.000029com.timeanddate
389197686864320.000060com.slack
390197684107690.000039jp.shinobi
3911976797616740.000018com.buzzfeednews
392197650384150.000062com.elsevier
3931976472213350.000022edu.gatech
3941976429828610.000012com.youdao
395197612568950.000034com.brightcove
3961975973017740.000017com.bankofamerica
3971975953025690.000013edu.byu
3981975876019180.000016com.voanews
3991975758631640.000011com.opendns
4001975681614250.000021com.sky
4011975578023360.000014com.slides
4021975446213730.000021com.dw
4031975445811580.000026com.nikkei
404197525909040.000033com.cbslocal
4051974876622360.000014net.earthlink
406197486783910.000064com.cnet
4071974815016420.000018com.xrea
4081974743013540.000022uk.co.huffingtonpost
409197464241820.000133com.eventbrite
4101974637010710.000027com.nydailynews
4111974409013050.000023me.vk
412197431949180.000033gov.bls
4131974154214580.000020org.ap
414197409363840.000066net.imgix
4151973986024140.000014org.aclweb
4161973975016410.000018com.axios
417197389409870.000030com.wattpad
4181973753017130.000018com.straitstimes
419197374124740.000054com.ted
4201973687412940.000023edu.brookings
421197286349670.000031int.coe
422197275802120.000109com.etsy
4231972711223920.000014com.biography
424197260808650.000035gov.va
425197257102170.000107com.typepad
4261972462819320.000016com.cocolog-nifty
4271972358016080.000019com.reference
428197207405530.000047com.livejournal
4291971740620960.000015ru.kremlin
430197163548150.000037uk.gov.service
431197153782980.000083com.techcrunch
4321971235824620.000013org.wikisource
4331971229615530.000019com.foxbusiness
4341971162012810.000023mil.army
4351971124417610.000017com.itv
436197102607330.000041com.deviantart
4371970595213110.000023de.mpg
438197052888450.000036gov.justice
4391970457419930.000016cn.people
4401970324812620.000024au.com.smh
4411970165617630.000017org.tensorflow
4421970163412230.000024org.ohchr
443197010005680.000046ru.gov
444197001364000.000064com.technorati
4451969959621340.000015jp.co.japantimes
44619697954830.000413com.list-manage
4471969708810680.000028com.thedrum
4481969675415380.000019uk.co.standard
449196954301850.000131com.rawgit
4501969421621200.000015com.oxforddictionaries
4511969300622410.000014com.shutterfly
4521969208231470.000011tw.edu.ntu
4531969156425500.000013com.smashwords
4541968986218620.000016edu.unl
4551968876824020.000014org.fas
456196886462960.000084uk.org.ico
4571968813827100.000013tv.blip
458196860669570.000031com.bandsintown
4591968444835160.000010cn.org.china
4601968296015500.000019uk.co.express
4611967970810820.000027jp.jugem
4621967915836560.000010info.webry
4631967873014030.000021gov.uscourts
4641967794421570.000015au.edu.unimelb
46519675766920.000363com.wsimg
466196748682830.000086ru.rambler
4671967373819210.000016com.washingtontimes
468196717543510.000072com.proofpoint
46919669412740.000441net.jsfiddle
470196683527880.000038org.mediawiki
4711966815828510.000012jp.blog
4721966774014790.000020com.firebaseapp
4731966741816180.000019com.webnode
4741966594021730.000015com.pbworks
4751966574833740.000011com.patheos
4761966568431350.000011uk.co.timesonline
4771966398021710.000015google.ai
478196633542330.000103com.squarespace
4791966218829040.000012fr.rfi
4801966098414540.000020gov.supremecourt
4811965920018890.000016int.unfccc
482196585343310.000076com.office
483196565265770.000046pl.google
484196540989910.000030gov.wa
485196527968040.000038gov.sba
4861965262612670.000023com.cognitoforms
4871965006622070.000015org.csis
488196490083660.000068io.codepen
4891964875023440.000014com.kobo
490196465121100.000281com.mailchimp
4911964342816710.000018edu.wustl
4921964257227340.000013edu.kit
4931964233414800.000020org.hrw
494196422769530.000031edu.umich
4951964185613890.000021com.dictionary
496196415448360.000036com.mapquest
4971964083617470.000017org.worldcat
4981964027636210.000010net.aljazeera
499196401443570.000071com.photobucket
5001963994820460.000015net.cnki
5011963851017050.000018com.secondlife
5021963841624210.000014int.wmo
5031963788810890.000027org.ilo
5041963745011000.000027google.blog
505196366923780.000067com.meetup
506196346349950.000030uk.co.pinterest
5071963377033970.000010com.freehostia
5081963041232560.000011com.doodlekit
509196297469360.000032com.arstechnica
5101962837037300.000009com.colourlovers
5111962835616960.000018ru.ucoz
512196282989520.000031com.thenextweb
5131962445822860.000014org.unep
5141962234222520.000014org.icrc
5151962180814240.000021com.findlaw
5161962113423340.000014com.similarweb
517196206964810.000054com.gmail
5181961930430400.000012io.soup
5191961624614370.000021com.imageshack
5201961595627850.000013com.sputniknews
5211961407830800.000012com.smore
5221961323232460.000011org.iucnredlist
5231961176631170.000011com.kinja
5241961176018830.000016com.csmonitor
525196116041450.000180ru.mail
5261961008813390.000022gov.uscis
527196085544460.000058net.secureservercdn
5281960631430040.000012sh.now
529196057484270.000061tv.twitch
5301960499415800.000019link.app
531196008144400.000059com.statista
5321959916036760.000010jp.hatenablog
5331959555043560.000008com.coroflot
5341959526431770.000011org.jenkins-ci
5351959515817570.000017gov.oregon
5361959313032000.000011li.paper
5371959310638470.000009com.pixar
5381958987830950.000011com.shell
5391958819440350.000009com.scienceblogs
5401958618816250.000019org.amnesty
541195848248920.000034com.thedailybeast
5421958246417670.000017org.pypi
5431958234621490.000015com.foreignpolicy
5441958031028490.000012com.instapaper
5451957967229100.000012org.accessnow
5461957861416020.000019com.surveygizmo
5471957778017330.000018ca.globalnews
5481957620031750.000011de.uni-koeln
549195761982390.000101io.shields
5501957618433770.000011org.lds
5511957590222380.000014org.rand
552195747902070.000114com.salesforce
5531957454434380.000010net.mootools
5541957442823570.000014at.ac.univie
5551957418240500.000009org.marxists
5561957166428600.000012org.panda
5571957119428060.000013com.oprah
5581956857618740.000016com.justia
5591956797034710.000010org.avaaz
5601956785428800.000012com.openai
5611956776435970.000010org.neocities
5621956726037530.000009cn.edu.sdu
563195649607620.000040com.netflix
564195641204980.000052com.oreilly
5651956308644050.000008com.yam
566195622482270.000105uk.co.amazon
567195622048660.000035com.zoho
568195609566290.000044com.zdnet
5691955996612980.000023ly.snip
5701955879017900.000017ch.ipcc
571195586649930.000030uk.parliament
5721955850837870.000009com.nestle
5731955630412540.000024se.google
5741955629229970.000012com.treehugger
5751955518410110.000029net.nocookie
5761955509646440.000008com.x0
5771955336836310.000010org.tvtropes
5781955099211410.000026org.sphinx-doc
5791954999421220.000015ru.mos
5801954882030440.000012es.csic
5811954853029130.000012uk.gov.companieshouse
5821954657610340.000029com.engadget
5831954623011830.000025com.here
5841954549250600.000007com.dbs
5851954543841030.000009br.ufrj
5861954420421590.000015edu.colostate
5871954339827060.000013de.uni-heidelberg
5881954050030590.000012com.pearltrees
5891953926821760.000015net.openid
5901953788026000.000013com.mystrikingly
5911953784438800.000009com.chinatimes
5921953583424000.000014link.page
5931953418223540.000014com.real
5941953343218360.000017org.ncsl
595195322883010.000082com.surveymonkey
596195319303620.000070com.hp
5971953141211930.000025org.js
5981953070021350.000015com.123formbuilder
5991952884224260.000014org.vim
6001952810432050.000011pl.wp
6011952801826020.000013au.com.sbs
602195267801700.000148com.yelp
6031952621624990.000013uk.ac.kcl
6041952434613380.000022org.aarp
6051952369226210.000013th.co.google
6061952315610060.000029uk.gov.legislation
607195230422600.000094com.getbootstrap
6081952285636630.000010com.magcloud
6091952227439900.000009com.zynga
6101952194212680.000023tw.com.google
6111952192228290.000013com.kaggle
612195201309480.000031gov.gpo
613195197429460.000032com.about
6141951971432730.000011org.rsf
6151951874029760.000012org.tigris
6161951822427270.000013uk.ac.leeds
6171951551235350.000010de.dw
6181951543430190.000012org.cfr
6191951457432530.000011de.uni-freiburg
6201951357036400.000010de.uni-konstanz
6211951271438810.000009ua.at
6221951125421170.000015info.worldometers
6231951031446570.000008com.embarcadero
6241950937029990.000012vn.zing
6251950913432290.000011com.bangkokpost
6261950880436150.000010ly.rebrand
6271950854820080.000016gov.ky
6281950842640090.000009org.wilsoncenter
6291950677440590.000009jp.hatenadiary
6301950628443740.000008com.musictoday
6311950538838240.000009org.constitutioncenter
632195051863720.000067com.booking
6331950440225790.000013com.eiseverywhere
6341950380040380.000009com.itsnicethat
6351950377633310.000011il.ac.tau
6361950209623590.000014mx.com.google
6371950080637360.000009com.db
638194989283120.000080com.ebay
6391949858835780.000010jp.hateblo
6401949816633480.000011org.democracynow
6411949729639750.000009edu.odu
6421949681228150.000013dk.au
6431949662642200.000008com.etymonline
6441949618428850.000012uk.gov.metoffice
645194957563610.000070com.skype
6461949556635700.000010com.hsbc
6471949484422280.000015com.bankrate
6481949410422400.000014gov.wi
6491949335218150.000017fi.google
6501949330644260.000008com.x10host
6511949213632240.000011org.royalsociety
652194910968170.000037com.pexels
653194903585320.000048com.mashable
6541949028246140.000008com.epochtimes
6551949001811740.000025edu.ucla
6561948965632260.000011cc.reurl
6571948941434300.000010com.dailykos
6581948936037420.000009uk.ac.uea
6591948805037050.000010ca.shaw
6601948610419680.000016uk.gov.tfl
6611948598834340.000010uk.ac.nhm
6621948503230600.000012com.ipage
6631948475424980.000013com.prweek
6641948459818190.000017gov.usembassy
6651948396648610.000007am.do
6661948363630860.000011com.viki
6671948351832520.000011se.liu
6681948271830660.000012com.coca-colacompany
6691948258042320.000008br.ufrgs
6701948249836390.000010de.uni-kiel
6711948134014530.000020com.speakerdeck
6721948071830770.000012net.openreview
6731948066022080.000015de.auswaertiges-amt
674194802482080.000113com.hubspot
6751947976220260.000016com.lexisnexis
6761947870021060.000015net.ucoz
6771947755234940.000010com.iconarchive
678194775328190.000037com.steampowered
679194772867560.000040com.xiti
6801947713224860.000013com.post-gazette
6811947689833690.000011com.eklablog
6821947663229370.000012uk.co.bbci
6831947637819110.000016hu.google
6841947616043990.000008com.jacobinmag
6851947597433230.000011uk.ac.sussex
6861947436830680.000012uk.ac.qmul
6871947421239300.000009nf.co
6881947301441140.000009com.collinsdictionary
6891947289652150.000007com.evaair
6901947284625720.000013com.marketwire
6911947258031380.000011au.com.telstra
6921947211439160.000009it.unitn
693194716468980.000034com.visualstudio
6941947133038070.000009in.ernet
6951947099429060.000012nl.rug
6961946870852970.000007org.arkive
697194682522520.000096org.drupal
6981946705034600.000010ca.dal
6991946704636930.000010com.canada
7001946564214510.000021com.tinypic
7011946530431360.000011org.wri
7021946503436980.000010com.la-croix
7031946410845570.000008com.mitsubishielectric
7041946382847480.000008com.gamejolt
7051946297627890.000013gr.google
7061946288248820.000007cz.webgarden
7071946240430790.000012my.com.thestar
708194618302690.000092net.php
7091946164043290.000008au.gov.fairwork
7101946077022790.000014co.pcdn
7111946017639430.000009uk.ac.essex
712194599841210.000231org.networkadvertising
7131945968433960.000010org.rferl
7141945906842110.000008com.sc
7151945902032920.000011com.blogfa
7161945879433820.000010ca.yelp
7171945758041020.000009edu.utm
7181945724856940.000007com.anghami
7191945653252100.000007su.clan
7201945614440950.000009it.justpaste
721194560064140.000062com.sxsw
7221945591432580.000011com.waterstones
7231945460239600.000009com.jigsy
724194545168380.000036com.intel
7251945439440320.000009ee.ut
726194532429160.000033com.docker
727194529887380.000041com.samsung
7281945180234220.000010es.ucm
7291945071825030.000013com.washingtonexaminer
7301945034239510.000009tl.page
7311945020622090.000015org.wbur
7321944903641120.000009site.negocio
7331944892227730.000013com.yell
7341944851639880.000009com.fatcow
7351944826632820.000011pl.poznan
736194481981350.000194com.youku
7371944793028780.000012ae.thenational
7381944776647050.000008id.co.kaskus
7391944766834070.000010com.afp
7401944760253360.000007net.manilatimes
741194467344190.000062com.caniuse
7421944616814700.000020com.pastebin
7431944591033870.000010uk.org.rspb
744194457367650.000039com.moz
7451944437640270.000009lv.draugiem
7461944160425080.000013gov.dni
7471944087425930.000013ro.google
7481944014429460.000012com.broadwayworld
7491943957437500.000009ru.msu
7501943937437660.000009pl.cba
7511943933241370.000009org.rfa
7521943928055620.000007org.bukkit
7531943908620130.000016scot.gov
754194388681330.000200com.constantcontact
7551943882656380.000007org.adbusters
7561943809445170.000008google.design
7571943765441540.000008com.macobserver
7581943708816490.000018fr.pagesjaunes
7591943702025020.000013com.thenation
7601943677639730.000009com.bbcamerica
7611943455648570.000007com.orgfree
7621943381029780.000012com.channelnewsasia
763194325067350.000041gov.sec
7641943250240080.000009com.teamspeak
7651943243028000.000013org.gnupg
7661943226037800.000009com.the-scientist
7671943225230150.000012com.laweekly
7681943144629210.000012au.edu.sydney
7691943008435770.000010uk.co.yougov
7701943000031400.000011vn.com.google
7711942994244170.000008com.50webs
7721942900431240.000011org.repec
7731942893832150.000011org.ourworldindata
7741942789035060.000010com.tradingeconomics
7751942735231020.000011tw.com.pchome
7761942658233320.000011com.monday
7771942655635560.000010org.project-syndicate
7781942555223310.000014com.amebaownd
7791942489015960.000019org.whatbrowser
7801942475019560.000016org.americanbar
7811942468037390.000009ie.thejournal
782194241521040.000298com.stripe
7831942414040140.000009com.hatenadiary
7841942406029330.000012org.thinkprogress
7851942371230730.000012uk.gov.london
7861942305439270.000009com.thesaurus
7871942300634750.000010net.webself
7881942296434320.000010io.pantheon
7891942171234200.000010uk.ac.exeter
7901942150843430.000008com.appledaily
7911942111835280.000010com.bravesites
7921942081651780.000007com.bambuser
7931942059233790.000011com.foreignaffairs
7941941937824320.000013com.instructables
7951941638821850.000015vn.vietnamnet
7961941473639940.000009com.webcindario
7971941432828230.000013org.ewg
7981941393445340.000008ws.nimb
7991941377828330.000013org.fullfact
800194133522560.000095us.zoom
8011941255636850.000010com.encyclopedia
8021941247438970.000009de.uni-erlangen
8031941082253410.000007net.boards
804194095983410.000074com.histats
8051940953442010.000008is.pse
806194094367480.000040fm.last
8071940780836610.000010com.mongabay
8081940704032200.000011me.site123
8091940633834360.000010com.seetickets
8101940555058380.000007com.gamigo
8111940440016660.000018com.materialdesignicons
8121940410851400.000007bd.com.google
813194032427900.000038com.venturebeat
8141940121846010.000008uk.org.phrases
8151940078032130.000011com.instructure
8161940029828170.000013gov.arkansas
81719399890720.000444com.livestream
8181939955440810.000009cat.uab
8191939948635460.000010org.lacity
8201939937236120.000010com.heraldscotland
8211939837014990.000020com.teachable
8221939667228950.000012com.foodandwine
8231939575212330.000024com.createjs
8241939427422660.000014com.ajc
8251939417239500.000009com.rappler
8261939403023550.000014net.noscript
8271939398241400.000009jp.doorblog
8281939288228730.000012com.timeshighereducation
829193922382750.000089com.bandcamp
8301938933239690.000009jp.ne.hi-ho
8311938809436290.000010net.inquirer
832193878825520.000047com.cisco
8331938731840760.000009pl.lublin
8341938637016570.000018com.pcworld
835193834042660.000093com.typeform
836193828862030.000116com.naver
8371938269837230.000010gov.bts
8381938219218160.000017jp.makeshop
8391938210244620.000008com.tor
8401938207245130.000008com.weightwatchers
8411938134614380.000021org.khanacademy
842193812749540.000031com.thinkwithgoogle
8431938102033850.000010uk.ac.jisc
8441938023840880.000009ly.genial
8451937998640070.000009com.themoscowtimes
8461937850032720.000011com.nyt
8471937843437600.000009com.springernature
8481937835633900.000010int.cbd
8491937785460450.000006es.xurl
8501937689817560.000017com.netsolhost
8511937659838520.000009au.edu.griffith
8521937605447400.000008co.edu.unal
8531937604040740.000009kr.co.koreatimes
854193745887270.000042com.deloitte
8551937430049860.000007org.edc
8561937394041490.000008vn.tienphong
8571937347635150.000010com.thediplomat
8581937293240990.000009uk.ac.lancs
8591937279850060.000007com.inoreader
8601937274649220.000007com.ueuo
8611937259415850.000019tv.ustream
8621937257632340.000011com.tapatalk
8631937235634160.000010nl.wur
8641937210648480.000007net.hypermart
8651937163622930.000014org.kff
866193693563980.000064com.pubmatic
8671936898236250.000010org.grist
8681936848030880.000011tw.gov.cdc
8691936828833890.000010com.gothamist
8701936813011060.000027com.gizmodo
8711936811641010.000009com.globalpost
872193676768140.000037gov.nist
8731936753645630.000008org.globalsecurity
8741936645445470.000008build.bazel
8751936638437820.000009us.ms.state
8761936587842560.000008gr.ntua
8771936577644440.000008se.thelocal
8781936537229630.000012com.politifact
8791936512813170.000023com.ensighten
8801936358850970.000007ru.my1
8811936268034680.000010com.rabbitmq
8821935969841380.000009com.elasticbeanstalk
8831935957413640.000022com.billboard
8841935912247660.000008cc.dict
8851935877456870.000007fi.mbnet
886193573908790.000035com.aliexpress
887193569182100.000111to.amzn
8881935566842750.000008edu.ohio
8891935554634520.000010com.thejakartapost
8901935535032770.000011vn.com.dantri
8911935508052850.000007com.galvanize
8921935488034840.000010jp.go.ndl
8931935479047100.000008com.kiwibox
8941935451421400.000015org.linuxfoundation
8951935450048010.000007ru.nnov
8961935316642880.000008gr.auth
8971935297022570.000014net.vnexpress
8981935177029000.000012com.crashlytics
8991935159410450.000028com.dropboxusercontent
9001935082834390.000010com.scotusblog
9011935071240900.000009org.carnegieendowment
902193502783950.000064com.atlassian
9031934972634650.000010com.study
904193487243500.000072com.mapbox
9051934853210460.000028com.redhat
9061934788617990.000017com.bravenet
9071934746042840.000008uk.org.npg
9081934715244630.000008com.btplc
9091934714852890.000007ru.drom
9101934654224300.000013com.vimeopro
9111934590044190.000008edu.marquette
912193456444260.000061com.adweek
913193451449140.000033com.shutterstock
9141934509010160.000029com.ubuntu
9151934196057120.000007in.ac.nptel
9161934148812270.000024com.msdn
9171934071447070.000008com.vocabulary
9181934068039290.000009edu.uaf
9191933965839190.000009com.atavist
9201933945632010.000011com.healthgrades
9211933909225460.000013com.kinstacdn
9221933838423450.000014com.gazhall
9231933793853980.000007com.asmallorange
9241933780037970.000009com.generalmills
9251933617645850.000008vn.vtc
9261933590815190.000020cn.gov.mofcom
927193337787970.000038com.box
9281933360639660.000009si.uni-lj
9291933332241700.000008az.president
9301933319417880.000017org.reactjs
9311933241236050.000010com.postaffiliatepro
9321933192251920.000007edu.uah
9331933128035990.000010org.openedition
9341933069648380.000007com.kapook
9351933038241530.000008org.caringbridge
936193303744830.000053com.aol
9371932961423030.000014org.nfpa
9381932953859560.000006com.glosbe
9391932919441240.000009com.mcall
9401932762242890.000008ru.tmweb
9411932687641260.000009uk.co.liverpoolecho
9421932642242440.000008com.atwebpages
9431932598010670.000028com.freepik
9441932479040850.000009org.specialolympics
9451932386848450.000007net.freeforums
9461932367647440.000008uk.ac.westminster
9471932353240920.000009com.tok2
9481932346010250.000029com.elpais
9491932315049460.000007tw.com.sina
9501932250832960.000011com.wowza
951193223063170.000079com.webs
9521932202446970.000008com.warriorplus
9531932191834140.000010com.cityam
9541932181244820.000008org.fee
9551932152048540.000007tw.edu.ntnu
9561932129649620.000007com.sparknotes
9571932020245160.000008com.newspapers
9581931963421920.000015com.tutsplus
9591931960058680.000007com.ananova
9601931927438180.000009org.opensecrets
961193191346330.000044gov.uspto
9621931872256800.000007su.moy
9631931836610130.000029com.uk
9641931826649360.000007ru.pr-cy
9651931805838270.000009cz.centrum
9661931778041580.000008edu.niu
9671931532016650.000018org.webkit
9681931501446920.000008pl.edu.amu
9691931408451860.000007com.artfire
9701931389438000.000009org.ascd
9711931210638010.000009edu.scu
9721931174243070.000008com.taipeitimes
9731931156843510.000008edu.whoi
9741931085459490.000006com.voatiengviet
9751931074831000.000011com.broadcastingcable
9761931072046550.000008hk.rthk
9771931024657030.000007com.enotes
978193099104880.000053com.indiatimes
979193096608600.000035com.playstation
9801930904048660.000007com.brothersoft
9811930894827080.000013uk.gov.defra
982193076062310.000103org.whatwg
9831930717844510.000008com.batchgeo
984193071187510.000040com.psychologytoday
9851930636842630.000008uk.co.lrb
9861930635050340.000007ca.pe.gov
9871930588441590.000008com.ecowatch
9881930382041950.000008com.williamhill
9891930354857670.000007pt.ipp
9901930297248430.000007uk.org.38degrees
9911930162413030.000023com.technologyreview
9921930146440910.000009org.spie
993193010689590.000031com.libsyn
9941930057247950.000007com.storeboard
9951930054832600.000011de.bmel
9961929944847490.000008net.onlinewebshop
9971929927438720.000009ru.1gb
998192986542790.000088com.automattic
9991929850238700.000009com.piie
10001929744053060.000007com.allthatsinteresting

Credits

Thanks to the authors of the WebGraph framework, whose software made the computation of graph properties and ranks possible.

We hope the data will be useful for you to do any kind of research on ranking, graph analysis, link spam detection, etc. Let us know about your results via Common Crawl’s Google Group!