12
SAM-SIG, 23rd August 200612 Cost vs Cost But sometimes vertical scaling is right Buying a bigger box is quick (ish) Redesigning software is not Running out of MySQL performance? –Spend months on data federation –Or, Just buy a ton more RAM

13
SAM-SIG, 23rd August 200613 Cost vs Cost But lets talk horizontal –Else this is going to be boring

14
SAM-SIG, 23rd August 200614 Architectures then? The way the bits fit together What grows where The trade offs between good/fast/cheap

15
SAM-SIG, 23rd August 200615 LAMP Were talking about LAMP –Linux –Apache (or LightHTTPd) –MySQL (or Postgres) –PHP (or Perl, Python, Ruby) All open source All well supported All used in large operations

22
SAM-SIG, 23rd August 200622 Mobile local sessions Custom built –Store last session location in cookie –If we hit a different server, pull our session information across If your load balancer has sticky sessions, you can still get hotspots –Depends on volume – fewer heavier users hurt more

23
SAM-SIG, 23rd August 200623 Remote centralized sessions Store in a central database –Or an in-memory cache No porting around of session data No need for sticky sessions No hot spots Need to be able to scale the data store –But weve pushed the issue down the stack

25
SAM-SIG, 23rd August 200625 Super slim sessions If you need more than the cookie (login status, user id, username), then pull their account row from the DB –Or from the account cache None of the drawbacks of sessions Avoids the overhead of a query per page –Great for high-volume pages which need little personalization –Turns out you can stick quite a lot in a cookie too –Pack with base64 and its easy to delimit fields

26
SAM-SIG, 23rd August 200626 App servers The Rasmus way –App server has shared nothing –Responsibility pushed down the stack Ooh, the stack

42
SAM-SIG, 23rd August 200642 The big three Lets talk about the big three then… –Databases –Serving lots of static content –Storing lots of static content

43
SAM-SIG, 23rd August 200643 Databases Unless were doing a lot of file serving, the database is the toughest part to scale If we can, best to avoid the issue altogether and just buy bigger hardware Dual Opteron/Intel64 systems with 16GB of RAM can get you a long way

44
SAM-SIG, 23rd August 200644 More read power Web apps typically have a read/write ratio of somewhere between 80/20 and 90/10 If we can scale read capacity, we can solve a lot of situations MySQL replication!

66
SAM-SIG, 23rd August 200666 Rings Master-master is just a small ring –With 2 members Bigger rings are possible –But not a mesh! –Each slave may only have a single master –Unless you build some kind of manual replication

71
SAM-SIG, 23rd August 200671 Data federation At some point, you need more writes –This is tough –Each cluster of servers has limited write capacity Just add more clusters!

72
SAM-SIG, 23rd August 200672 Data federation Split up large tables, organized by some primary object –Usually users Put all of a users data on one cluster –Or shard, or cell Have one central cluster for lookups

74
SAM-SIG, 23rd August 200674 Data federation Need more capacity? –Just add shards! –Dont assign to shards based on user_id! For resource leveling as time goes on, we want to be able to move objects between shards –Lockable objects

76
SAM-SIG, 23rd August 200676 Downsides Need to keep stuff in the right place App logic gets more complicated More clusters to manage –Backups, etc More database connections needed per page The dual table issue –Avoid walking the shards!

77
SAM-SIG, 23rd August 200677 Bottom line Data federation is how large applications are scaled

78
SAM-SIG, 23rd August 200678 Bottom line Its hard, but not impossible Good software design makes it easier –Abstraction! Master-master pairs for shards give us HA Master-master trees work for central cluster (many reads, few writes)

83
SAM-SIG, 23rd August 200683 Reverse proxy Serving out of memory is fast! –And our caching proxies can have disks too –Fast or otherwise More spindles is better We stay in sync automatically We can parallelize it! –50 cache servers gives us 50 times the serving rate of the origin server –Assuming the working set is small enough to fit in memory in the cache cluster

84
SAM-SIG, 23rd August 200684 Invalidation Dealing with invalidation is tricky We can prod the cache servers directly to clear stuff out –Scales badly – need to clear asset from every server – doesnt work well for 100 caches

91
SAM-SIG, 23rd August 200691 Permission URLs But why bother!? If we bake the auth into the URL then it saves the auth step We can do the auth on the web app servers when creating HTML Just need some magic to translate to paths We dont want paths to be guessable

97
SAM-SIG, 23rd August 200697 Multiple hosts Further down the road, a single host will be too small Total throughput of machine becomes an issue Even physical space can start to matter So we need to be able to use multiple hosts

99
SAM-SIG, 23rd August 200699 HA Storage HA is important for assets too –We can back stuff up –But we want it hot redundant RAID is good –RAID5 is cheap, RAID 10 is fast

100
SAM-SIG, 23rd August 2006100 HA Storage But whole machines can fail So we stick assets on multiple machines In this case, we can ignore RAID –In failure case, we serve from alternative source –But need to weigh up the rebuild time and effort against the risk –Store more than 2 copies?