WordPress Shortcode

Link

Global Netflix Platform

Slides from QConSF Nov 19th, 2011 focusing this time on describing the globally distributed and scaled industrial strength Java Platform as a Service that Netflix has built and run on top of AWS and
…

Slides from QConSF Nov 19th, 2011 focusing this time on describing the globally distributed and scaled industrial strength Java Platform as a Service that Netflix has built and run on top of AWS and Cassandra. Parts of that platform are being released as open source - Curator, Priam and Astyanax.

Transcript

2.
Ne9lix Inc. With more than 20 million streaming members in the United States, Canada and La8n America, Ne<lix, Inc. is the worlds leading Internet subscrip8on service for enjoying movies and TV shows. Interna8onal Expansion Ne<lix, Inc., the leading global Internet movie subscrip8on service… announced it will expand to the United Kingdom and Ireland in early 2012. Source: h=p://ir.ne9lix.com

9.
Ne9lix.com is now ~100% Cloud A few small back end data sources sLll in progress All internaLonal product is cloud based USA speciﬁc logisLcs remains in the Datacenter Working aggressively on billing, PCI compliance on AWS

10.
Ne9lix Choice was AWS with our own pla9orm and tools Unique pla9orm requirements and extreme scale, agility and ﬂexibility

11.
Leverage AWS Scale “the biggest public cloud” AWS investment in features and automaLon Use AWS zones and regions for high availability, scalability and global deployment

12.
But isn’t Amazon a compeLtor? Many products that compete with Amazon run on AWS We are a “poster child” for the AWS Architecture Ne9lix is one of the biggest AWS customers Strategy – turn compeLtors into partners

13.
Could Ne9lix use another cloud? Would be nice, we use three interchangeable CDN Vendors But no-­‐one else has the scale and features of AWS You have to be this tall to ride this ride… Maybe in 2-­‐3 years?

14.
We want to use clouds, we don’t have Lme to build them Public cloud for agility and scale We use electricity too, but don’t want to build our own power staLon… AWS because they are big enough to allocate thousands of instances per hour when we need to

30.
Entrypoints Queries MongoDB is good for low traﬃc complex queries against complex objects DescripAon Range expression Find all acLve instances. all() Find all instances associated with a group %(cloudmonkey) name. Find all instances associated with a /^cloudmonkey$/discovery() discovery group. Find all auto scale groups with no instances. asg(),-­‐has(INSTANCES;asg()) How many instances are not in an auto count(all(),-­‐info(eval(INSTANCES;asg()))) scale group? What groups include an instance? *(i-­‐4e108521) What auto scale groups and elasLc load ﬁlter(TYPE;asg,elb;*(i-­‐4e108521)) balancers include an instance? What instance has a given public ip? ﬁlter(PUBLIC_IP;174.129.188.{0..255};all())

37.
Aside: Adrian’s Rant on CAP Theorem • Instances and Networks will fail • Network failure = ParLLon “P” is a given • Distributed Systems: two choices – CP or AP • “Vendor claims CA” – Usually they mean available when instances fail • Master-­‐Slave = Consistent when ParLLoned – You can’t write unless you can see the master • Quorum = Available when ParLLoned – Writes proceed, conﬂicts will be patched up later

38.
Why Cassandra? • We value Availability over Consistency – AP – Cassandra is a Java distributed systems toolkit • We have a building full of Java engineers – Riak is in Erlang – a blessing and a curse… • We want FOSS + Support – Voldemort doesn’t have a support model • Writes are intrinsically harder than reads – Hbase is opLmized for reads, Cassandra for writes • We tested Cassandra and it works for us – Step by step into full producLon over the last year

71.
Takeaway Ne<lix has built and deployed a scalable global Pla<orm as a Service. Also, benchmarking in the cloud is fast, cheap and scalable h=p://www.linkedin.com/in/adriancockcro6 @adrianco #ne9lixcloud acockcro6@ne9lix.com