Austin Scales - Nexus - Bazaarvoice's Cloud Infrastructure

Nexus is Bazaarvoice's next generation cloud infrastructure built on top of Amazon Web Services. Nexus is highly available and resilient, built with best practices on top of services such as VPC, Autoscaling, ELB, Cloudformation, and more.

3.
BazaarvoiceNot bizarre boysAustin-based company founded in 2005 Basic stats: Thousands of clientsSaaS serving software that collectsand displays user generated content, Hundreds of millions of pieces ofcrunches analytics, and extracts contentdata. Hundreds of millions of uniqueEngineering offices in Austin, NYC, visitors per monthLondon, and San Francisco Tens of billions of page-views per month

6.
VPC & Subnets VPC allows us to choose our internal IP space. Public: Default route via IGW Default Route for All Subnets to IGW ● Lets call these subnets all "Public" ● Requires all instances to have EIPs before talking to the internet ● EIPs are a limited resource Private: Default route via instance(s) in Public Subnets Advantage: Most instances in the private subnet can talk to the internet without dealing with an EIP.

10.
{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "A text description for the template usage", "Parameters": { // A set of inputs used to customize the template perdeployment }, "Mappings": { // Mappings match a key to a corresponding set of named values }, "Resources" : { // The set of AWS resources and relationships between them }, "Outputs" : { // A set of values to be made visible to the stack creator }}

13.
In the beginning...A Java application server + a MySQLDatabaseScaled by adding in anotherapplication server.Then we just duplicated this entirestack, giving us two "clusters".Scaled more by adding more and moreclusters.

18.
Multiple AccountsGood Bad● Provides for full resource ● Inter-team network control with direct API communications can become access very complicated, relying● Protects teams from one on VPN between VPCs -> another Reduced Reliability● Allows for easy accounting ● Management of networking is on a per-team basis a possible bottleneck● May make it easier for ● Shared resources may need external auditors to to be redundantly built in determine which teams have every VPC: LDAP, DNS, "production" access Monitoring

19.
Single Shared AccountGood Bad● Sharing of resources will ● No built-in protections be simple - just open between teams, even with access via security groups IAM between teams ● Creates a centralized● Reliable networking between resource that someone has teams without need for VPN to maintain● Possibly better performance ● Requires us to build tools due to fewer hops to use long-term● Certain resources can be shared: LDAP, DNS, Monitoring, etc.

21.
In more detail...Nexus is:● AWS Infrastructure designed with best practices: ○ secure ○ highly available ○ multi-region ○ repeatable● Cloud building blocks and recipes for all of Engineering● A Single Account SolutionPhilosophy: Engineering teams at Bazaarvoice are free to choosetheir own stack, but we want to make Nexus so compelling that itis the default choice.

32.
Limitations & Risks● Danger! Single Shared Account ○ You can wipe out all of a region with a bad script.● Single NAT per AZ ○ Someone else downloading lots of data from the internet will affect all other instances sharing your private subnet.● Single VPN Instance per VPN Destination ○ Similar to NAT problem, but worse. ○ Avoid VPN when possible ○ If not possible, make your VPN dependency resilient to lack of bandwidth and network blips