Maybe you're a Dropbox devotee. Or perhaps you really like streaming Sherlock on Netflix. For that, you can thank the cloud.

In fact, it's safe to say that Amazon Web Services (AWS) has become synonymous with cloud computing; it's the platform on which some of the Internet's most popular sites and services are built. But just as cloud computing is used as a simplistic catchall term for a variety of online services, the same can be said for AWS—there's a lot more going on behind the scenes than you might think.

If you've ever wanted to drop terms like EC2 and S3 into casual conversation (and really, who doesn't?) we're going to demystify the most important parts of AWS and show you how Amazon's cloud really works.

Elastic Cloud Compute (EC2)

Think of EC2 as the computational brain behind an online application or service. EC2 is made up of myriad instances, which is really just Amazon's way of saying virtual machines. Each server can run multiple instances at a time, in either Linux or Windows configurations, and developers can harness multiple instances—hundreds, even thousands—to handle computational tasks of varying degrees. This is what the elastic in Elastic Cloud Compute refers to; EC2 will scale based on a user's unique needs.

Instances can be configured as either Windows machines, or with various flavors of Linux. Again, each instance comes in different sizes, depending on a developer's needs. Micro instances, for example, only come with 613 MB of RAM, while Extra Large instances can go up to 15GB. There are also other configurations for various CPU or GPU processing needs.

Finally, EC2 instances can be deployed across multiple regions—which is really just a fancy way of referring to the geographic location of Amazon's data centers. Multiple instances can be deployed within the same region (on separate blocks of infrastructure called availability zones, such as US East-1, US East-2, etc.), or across more than one region if increased redundancy and reduced latency is desired

Elastic Load Balance (ELB)

Another reason why a developer might deploy EC2 instances across multiple availability zones and regions is for the purpose of load balancing. Netflix, for example, uses a number of EC2 instances across multiple geographic location. If there was a problem with Amazon's US East center, for example, users would hopefully be able to connect to Netflix via the service's US West instances instead.

But what if there is no problem, and a higher number of users are connecting via instances on the East Coast than on the West? Or what if something goes wrong with a particular instance in a given availability zone? Amazon's Elastic Load Balance allows developers to create multiple EC2 instances and set rules that allow traffic to be distributed between them. That way, no one instance is needlessly burdened while others idle—and when combined with the ability for EC2 to scale, more instances can also be added for balance where required.

Elastic Block Storage (EBS)

Think of EBS as a hard drive in your computer—it's where an EC2 instance stores persistent files and applications that can be accessed again over time. An EBS volume can only be attached to one EC2 instance at a time, but multiple volumes can be attached to the same instance. An EBS volume can range from 1GB to 1TB in size, but must be located in the same availability zone as the instance you'd like to attach to.

Because EC2 instances by default don't include a great deal of local storage, it's possible to boot from an EBS volume instead. That way, when you shut down an EC2 instance and want to re-launch it at a later date, it's not just files and application data that persist, but the operating system itself.

Simple Storage Service (S3)

Unlike EBS volumes, which are used to store operating system and application data for use with an EC2 instance, Amazon's Simple Storage Service is where publicly facing data is usually stored instead. In other words, when you upload a new profile picture to Twitter, it's not being stored on an EBS volume, but with S3.

S3 is often used for static content, such as videos, images or music, though virtually anything can be uploaded and stored. Files uploaded to S3 are referred to as objects, which are then stored in buckets. As with EC2, S3 storage is scalable, which means that the only limit on storage is the amount of money you have to pay for it.

Buckets are also stored in regions, and within that region “are redundantly stored on multiple devices across multiple facilities.” However, this can cause latency issues if a user in Europe is trying to access files stored in a bucket within the US West region, for example. As a result, Amazon also offers a service called CloudFront, which allows objects to be mirrored across other regions.

While these are the core features that make up Amazon Web Services, this is far from a comprehensive list. For example, on the AWS landing page alone, you'll find things such as DynamoDB, Route53, Elastic Beanstalk, and other features that would take much longer to detail here.

However, if you've ever been confused about how the basics of AWS work—specifically, how computational data and storage is provisioned and scaled—we hope this gives you a better sense of how Amazon's brand of cloud works.

Correction: Initially, we confused regions in AWS with availability zones. As Mhj.work explains in the comments of this article, "availability Zones are actually "discrete" blocks of infrastructure ... at a single geographical location, whereas the geographical units are called Regions. So for example, EU-West is the Region, whilst EU-West-1, EU-West-2, and EU-West-3 are Availability Zones in that Region." We have updated the text to make this point clearer.

40 Reader Comments

Instances can be configured as either Windows machines, or with various flavors of Linux. Again, each instance comes in different sizes, depending on a developer's needs. Micro instances, for example, only come with 613 MB of RAM, while Extra Large instances can go up to 15GB. There are also other configurations for various CPU or GPU processing needs.

Instances can be configured as either Windows machines, or with various flavors of Linux. Again, each instance comes in different sizes, depending on a developer's needs. Micro instances, for example, only come with 613 MB of RAM, while Extra Large instances can go up to 15GB. There are also other configurations for various CPU or GPU processing needs.

Interesting how a shopping empire developed a need for GPU computing.

I think they have GPU instances because they found that customers want them, not because the retail side of the business needs them.

I have been using EC2 for awhile now, and I am still a total EC2 newb. But yes, it would be wonderful if a series of articles were run on the common 'gotchas' involving persistent storage, backing up, AMI/instance management, key management, etc. it was a non-trivial learning curve, so i think this audience would appreciate more detail.

There are a number of companies that will handle the low-level details of AWS. It's fantastic to be able to worry about a narrow portion of the stack, instead of the entire (and complicated) stack required on AWS to be used effectively.

I like to think that this amazing collection of computing resources is all going to be so much scrap when the real computer revolution comes along

Of course, that's hyperbole. Nobody expects much else these days.

I'm astounded that cloud computing has taken off as quickly as it has. I love my Dropbox and Crashplan instances.

Hmm...what is this CrashPlan you speak of. For years I've used JungleDisk to backup to S3 directly, but lately it's been sucking and abandoned. I've considered writing my own backup client using the S3 APIs but I'd rather not spend all the time on that. Other options however have not been as appealing since most cost $5-10/mo, while using S3 directly I'm usually billed less than a buck a month. CrashPlan however appears to only cost $2.50/mo when used month-to-month. That's about the price level I would like. Is it good/reliable/easy?

Hmm...what is this CrashPlan you speak of. ... Is it good/reliable/easy?

I have to admit, I don't pay for it.

I use the software to 'crossbackup' my partners and my computer, no internet involved. Australia has quite abysmal data tariffs, so 'backing up over the internet' isn't very attractive.

So I can't in all honesty answer your question. I'm pretty sure that there are people actively using Crashplan for their backup strategy. I have heard nothing but good reports, and their policies and offerings seem sensible, reasonable, and affordable.

Instances can be configured as either Windows machines, or with various flavors of Linux. Again, each instance comes in different sizes, depending on a developer's needs. Micro instances, for example, only come with 613 MB of RAM, while Extra Large instances can go up to 15GB. There are also other configurations for various CPU or GPU processing needs.

Thanks for the article! I believe it's important for as many people as possible to know how Amazon's cloud works so that we'll have a fighting chance against the ShopNet AI when it spontaneously becomes self-aware.

As much as a good primer on AWS and its intricacies is needed, I don't think this article is doing a good job at it.It is too heavy on the buzzwords, yet doesn't really develop what they entail. Concepts like AMIs, instance sizes (FYI, m2.4xlarge instances go up to 64G of RAM and 1.6Tb of storage), the differences between EBS and S3 (storage both, with different characteristics... but also are SimpleDB or Dynamo), or really what availability zones means are not accurately explained, while the actual benefits or hindrances for anybody willing to move to AWS (from a datacenter or an other CaaS platform) are not touched on.You know what would be great? One of those multi-page, detailed feature that Ars does once in a while on the topic... because I might have learnt by now about the intricacies of setting up EBS-backed instances with local storage, but I would bet a lot of people think the learning curve is too steep for them to bother with this.

Amazon might be better placed to host such technologies (Cloud services in its variety) in regions like Europe and Asia for customers who are phobic to US data privacy laws. That stances has been a big concern for business customers and increasingly general consumers due to the increasing "police state" legislation that the US seem to erect on a monthly basis. It is difficult to determine the security unless there are ways to have third-party audits as to how the data was managed and viewed/used if at all.In all, it would call for another level of application-level encryption to "block" any enforcement-snooping of data. There is much abuse if many parties are privy to accessing the data other than the registered owner. The old case of needing a court order in order to perform data searches seemed so rare, I do not think it exists anymore.

It would be a shame to waste all this technological capability to one region of the world which tends to cow-tow to enforcement agencies without the slightest hint of challenging the legality and/or informing the owner of the data.

The term "innocent until proven guilty" must always be fresh on people's mind to prevent mass privacy erosion that tends to end up in nefarious purposes.

As much as a good primer on AWS and its intricacies is needed, I don't think this article is doing a good job at it.It is too heavy on the buzzwords, yet doesn't really develop what they entail. Concepts like AMIs, instance sizes (FYI, m2.4xlarge instances go up to 64G of RAM and 1.6Tb of storage), the differences between EBS and S3 (storage both, with different characteristics... but also are SimpleDB or Dynamo), or really what availability zones means are not accurately explained, while the actual benefits or hindrances for anybody willing to move to AWS (from a datacenter or an other CaaS platform) are not touched on.You know what would be great? One of those multi-page, detailed feature that Ars does once in a while on the topic... because I might have learnt by now about the intricacies of setting up EBS-backed instances with local storage, but I would bet a lot of people think the learning curve is too steep for them to bother with this.

Good points you're raising with your comment. I also thought that this primer is lacking a bit of substance. For example some illustration could be very helpful to better describe the relationships between EC2, S3 and EBS.

Good article - but worth pointing out that Availability Zones are actually "discrete" blocks of infrastructure (although how discrete is open to debate following the recent AWS outage) at a single geographical location, whereas the geographical units are called Regions. So for example, EU-West is the Region, whilst EU-West-1, EU-West-2, and EU-West-3 are Availability Zones in that Region.

In general, designing high-availability systems (at least using AWS tools) is easier (and cheaper) across Zones at a single Region, than across Regions, since you don't pay for data transfer between Zones, only Regions.

I agree with other people. I think a more detailed feature article would be great. For example, I have no idea why you would use EBS for the sizes it is supported to, if S3 exists to store media content. I can understand wanting to store the operating system and everything, but why need up to 1TB of storage, if you don't plan on storing some sort of media content.

Hmm...what is this CrashPlan you speak of. For years I've used JungleDisk to backup to S3 directly, but lately it's been sucking and abandoned. I've considered writing my own backup client using the S3 APIs but I'd rather not spend all the time on that. Other options however have not been as appealing since most cost $5-10/mo, while using S3 directly I'm usually billed less than a buck a month. CrashPlan however appears to only cost $2.50/mo when used month-to-month. That's about the price level I would like. Is it good/reliable/easy?

If you use OS X, you could give Arq a go. It uses S3, and lets you select what to send to the cloud, rather then sending everything to the cloud.

As much as a good primer on AWS and its intricacies is needed, I don't think this article is doing a good job at it.It is too heavy on the buzzwords, yet doesn't really develop what they entail. Concepts like AMIs, instance sizes (FYI, m2.4xlarge instances go up to 64G of RAM and 1.6Tb of storage), the differences between EBS and S3 (storage both, with different characteristics... but also are SimpleDB or Dynamo), or really what availability zones means are not accurately explained, while the actual benefits or hindrances for anybody willing to move to AWS (from a datacenter or an other CaaS platform) are not touched on.You know what would be great? One of those multi-page, detailed feature that Ars does once in a while on the topic... because I might have learnt by now about the intricacies of setting up EBS-backed instances with local storage, but I would bet a lot of people think the learning curve is too steep for them to bother with this.

This is good feedback. Seriously. With this particular piece, I wanted to try and strike a good balance between highlighting AWS buzzwords that are often mentioned in other articles and places online, and giving a basic, understandable explainer of what they mean—though it seems people are definitely split between whether I achieved this or not.

I'll see if a more detailed, multi-page feature is doable, and perhaps interview some people who actually know things for case study purposes. While I've dabbled with AWS, I'm certainly no expert, so this would take a bit more time.

And for those who say they want more of this sort of thing, I'm totally open to suggestions.

For those that want a more on-hands technical primer, I recommend the OReilly Short (76 pgs) "Python and AWS Cookbook". If you're too cheap to spend a few bucks, just download the sample code, it's not too difficult to figure out what it's doing yourself. (I have no connection with OReilly or the author.)

By the way, AWS has a "free tier" that gives you basically one year free with a t1.micro and a public IP which is more than enough for learning the ropes and practicing with a simple personal website. And if you're sure you want to keep it, pay upfront for a 3-yr reservation to save some serious money (about $100 for 3 years of micro last I looked).

As much as a good primer on AWS and its intricacies is needed, I don't think this article is doing a good job at it.It is too heavy on the buzzwords, yet doesn't really develop what they entail. Concepts like AMIs, instance sizes (FYI, m2.4xlarge instances go up to 64G of RAM and 1.6Tb of storage), the differences between EBS and S3 (storage both, with different characteristics... but also are SimpleDB or Dynamo), or really what availability zones means are not accurately explained, while the actual benefits or hindrances for anybody willing to move to AWS (from a datacenter or an other CaaS platform) are not touched on.You know what would be great? One of those multi-page, detailed feature that Ars does once in a while on the topic... because I might have learnt by now about the intricacies of setting up EBS-backed instances with local storage, but I would bet a lot of people think the learning curve is too steep for them to bother with this.

I have a lot of instances in two regions in AWS, ranging from t1.micro to 37GB RAM commercial RDBMS. The article reviews the Amazon feature list and neglects the realities:

Complications of instance sizing between 64 and 32 bit VMs. All small instances are 32-bit and t1.micro instances can be throttled, forcing many workloads into t1.large instances by default.

AMI sourcing is needlessly complicated if you want something other than Amazon Linux (RHEL beta with irregular updates) or Windows.

Coping with never getting the same internal IPs assigned from DHCP, even if external IPs are all expensive ElasticIPs (static IP at the NAT boundary).

The limitations of firewalling with security groups and IP ACLs.

Not being able to get contiguous netblocks of ElasticIPs.

Occasional reliability and performance problems with hosts.

Elastic Load Balancer balances equally not between instances but between Availability Zones. Hosts can't security ACL themselves from foreign instances without cutting off the ELB probes. Amazon presumes that applications are public by default.

Regions are independent EC2 clouds and don't share, for instance, EBS images, so EBS-backed instances can't be almost-trivially shifted between Regions as they can between availability zones.

Amazon will occasionally require customers to shutdown and restart instances to move them between hosts when hosts need maintenance, despite Xen virtualization being capable of Live Migration. This has availability impact on systems not designed to span many instances in a cloud.

The legacy reasons why Amazon seems S3-centric even though conventional virtualization/VPS and enterprise-migrating customers should head straight for EBS for initial implementations.

These aren't showstoppers, especially if your EC2 implementation is actually a private cloud using Eucalyptus or equivalent, but they're just a few of things you won't see spelled out between hollow exhortations about cloud architecture and numerous proprietary and opaque ElasticUnderpants service names. Amazon does make continuous improvements but one must keep up with their service releases.

A nice start. I agree with the comments above requesting more in-depth articles to follow this one. However, if I can make three small complaints:

First, I do NOT think it is safe to say that Amazon Web Services "is synonymous with" cloud computing. I think AWS is the dominant player in providing some of the aspects of cloud computing (IaaS, and to a lesser extent PaaS). They are not a SaaS provider, and that is a huge chunk of this space. And you neglect to consider that many companies are deploying widescale on-premises virtualization, and are thus also engaging in cloud computing. If you include those in the picture, AWS is just one piece of a much larger puzzle, and oftentimes (but not always) in no way related to that.

Second, I agree that you can't include everything AWS advertises, but to round out the services, I would have included VPC. Continuing on the latter point, I think it's important to state how Amazon's cloud can exist on its own, or as an extension of an existing company's network.

Good short article, although I agree with many here that you should do a more in depth look at cloud services, and not just AWS. It would be great to see a comparison of a bunch of providers: AWS, joyent, rackspace, heroku, linode, etc.

You know what would be great? One of those multi-page, detailed feature that Ars does once in a while on the topic...

I have a lot of instances in two regions in AWS, ranging from t1.micro to 37GB RAM commercial RDBMS. The article reviews the Amazon feature list and neglects the realities:

Complications of instance sizing between 64 and 32 bit VMs. All small instances are 32-bit and t1.micro instances can be throttled, forcing many workloads into t1.large instances by default.

AMI sourcing is needlessly complicated if you want something other than Amazon Linux (RHEL beta with irregular updates) or Windows.

These aren't showstoppers, especially if your EC2 implementation is actually a private cloud using Eucalyptus or equivalent, but they're just a few of things you won't see spelled out between hollow exhortations about cloud architecture and numerous proprietary and opaque ElasticUnderpants service names.

#1 -- All sizes now support 64bit images, as of a few weeks ago.#2 -- I have no idea what you're talking about. A number of linux distros (e.g. Ubuntu) have official cloud images that are up-to-date. I can start an up-to-date stable ubuntu image in less than 3 minutes, and customize it from a script in another 2.

I can't vouch for the rest of your complaints (why exactly do you need contiguous blocks of IPs, for example, and when is ElasticIP not good enough? Just curious...) The documentation is still pretty thin, and I generally agree with your ElasticUnderpants assessment, but AWS has come a long way and seems pretty mature now.

This is good feedback. Seriously. With this particular piece, I wanted to try and strike a good balance between highlighting AWS buzzwords that are often mentioned in other articles and places online, and giving a basic, understandable explainer of what they mean—though it seems people are definitely split between whether I achieved this or not.

I'll see if a more detailed, multi-page feature is doable, and perhaps interview some people who actually know things for case study purposes. While I've dabbled with AWS, I'm certainly no expert, so this would take a bit more time.

And for those who say they want more of this sort of thing, I'm totally open to suggestions.

It's nice to see that you're looking for feedback. I spent a few hours with AWS this month, and I think there's a lot going on, and it's a little hard to pick out what's most important, or how to approach it. As an Ars reader, I'm conflicted on what to ask for.

If I have a specific question I need answered, I'll use google and, as often as not, find a blog post where an individual did exactly what I needed, documented it, etc. I read Ars to get perspective, history, and expert opinion (and to watch comment troll-baiting, but that's another story). In other words, things that google can't easily tell me. Sooo:

* Interviews! Yes!* How the hell does all this hardware work on the backend? What are the economies of it? Is it super-secret, on a remote fusion-powered trans-dimensional island connected to the 'net via ansibles?* Networking: Hardware IPs vs. elastic IPs?* Competition (as others have mentioned)* History -- where did all of this come from? To quote David Byrne, "This is not my beautiful 'nets!"

Sounds like the developer has to worry about & configure load balancing, data mirroring, O/S platform, etc. Google App Engine does all that for you transparently to the developer. Sounds like AWS is just costing me more work, what is it buying me over GAE?

Complications of instance sizing between 64 and 32 bit VMs. All small instances are 32-bit and t1.micro instances can be throttled, forcing many workloads into t1.large instances by default.

AMI sourcing is needlessly complicated if you want something other than Amazon Linux (RHEL beta with irregular updates) or Windows.

These aren't showstoppers, especially if your EC2 implementation is actually a private cloud using Eucalyptus or equivalent, but they're just a few of things you won't see spelled out between hollow exhortations about cloud architecture and numerous proprietary and opaque ElasticUnderpants service names.

#1 -- All sizes now support 64bit images, as of a few weeks ago.

Thanks! However, you deliberately cut my quote where I said "Amazon does make continuous improvements but one must keep up with their service releases." I infer they've now refreshed a sufficient amount of their hardware fleet not to force any image sizes into 32-bit.

Quote:

#2 -- I have no idea what you're talking about. A number of linux distros (e.g. Ubuntu) have official cloud images that are up-to-date. I can start an up-to-date stable ubuntu image in less than 3 minutes, and customize it from a script in another 2.

So can I. However you and I benefit from experience. When a user uses the AWS web-UI to create a new instance, it's difficult to find any official AMIs except those of Amazon, and Amazon seems only to supply Amazon Linux, which is effectively a beta of RHEL with different repos and infuriatingly inconsistent updates. To use an official Ubuntu -- official from Canonical, not Amazon -- AMI you need to find the AMI number from the Ubuntu release page, not from Amazon. There is no longer an official Debian AMI, unfortunately, because the maintainer decided to supply only AMIs for Ubuntu. Unless this has changed in the last few weeks, in which case see my original comment about keeping up with AWS improvements.

Quote:

I can't vouch for the rest of your complaints (why exactly do you need contiguous blocks of IPs, for example, and when is ElasticIP not good enough? Just curious...)

Don't be defensive. ACLs for non-AWS systems and services, obviously. Let's say you want to allow all of your AWS servers to access a private git repository hosted elsewhere to do distributed builds or infrastructure version control. Currently you have no Layer 3 ACL choices except to use individual ElasticIPs in routers and Layer 7 ACLs because all your instances' public IPs cannot be segregated into netblocks. This is a very conscious decision on the part of Amazon to stringently conserve IP address space, but it's an unexpected issue for many implementers and must be architected around.

Quote:

The documentation is still pretty thin, and I generally agree with your ElasticUnderpants assessment, but AWS has come a long way and seems pretty mature now.

This is true, but they also have a long way to go. Amazon chose to go with the fully dynamic and automateable IaaS cloud paradigm instead of a more-conventional offsite-virtualization-as-a-service, and they're consequently needing to create methodologies and standards as they go.

The problem isn't documentation per se, it's just that Amazon themselves, and no single source I've seen, will tell you things you need to know about the infrastructure works and what to do about it in order to accomplish your business goals. Amazon's architecture whitepapers are mostly touts for the radically-dynamic services model and distributing workloads across outsourced SOA and don't actually tell you things like:

All instances get RFC1918 addresses, and those addresses are used to communicate intra-region. Public addresses (designated ElasticIPs and generic dynamically-assigned public addresses) are assigned at border gateways and not visible to the instance in conventional ways.

Amazon SOAP APIs can be used to pull such information, and the instance can pull its own information in clever RESTful ways, but relatively few standard tools yet exist to use these APIs. Customers can assign their own private fields, but Amazon is evolving this rapidly and it's not uncommon that they assign a standard field to meet an obvious need after you've started using a custom field.

Amazon won't assign ElasticIPs in contiguous netblocks unless you're a very large customer. Netflix is able to do this, with an EC2 infrastructure on the order of 10,000 instances (probably peak).

Maintaining a /24 set-aside of ElasticIPs would be expensive at list prices in any event.

Security groups control Layer 3 ACLs and can thankfully refer to an account's own security groups, but has no objects for AWS infrastructure itself, nor can arbitrary objects or foreign objects be defined.

Security groups have no comment fields and no configuration management. Configuration management is weak all around, especially when multiple admins are using one AWS account.

Regions are separate islands of EC2 that are accessed through a unified UI and billing. Referring to instances in foreign regions can only be done through public IPs. EBS volumes must be copied manually as AMIs. S3 is available globally through one namespace, however.

Using existing AMIs to build EBS-backed VPSes isn't Amazon's favorite use-case, of course. It was originally designed for each customer to build AMIs and spin them up and destroy them in automated fashion, using only S3 for persistent storage. EBS came later because a lot of customers want to use EC2 in a vaguely-conventional virtualization fashion instead of OS-images-as-code/workload.

Instances which are rebooted will not move between hardware hosts and will retain their internal IP address across reboots. Instances which are stopped and restarted (applies to EBS-backed instances obviously) will get new private IPs and very likely instantiate on new hardware hosts.

Elastic IPs used to not be assigned automatically to instances when those instances started. You had to script this on the host. Then Amazon changed this so they were automatically assigned, which is a big improvement except that this can easily blow-up pre-existing automation.

Unlike VMware, explicit timekeeping isn't necessary because Xen is paravirtual by default so there's no pseudo-hardware RTC to deal with.

There are many, many piece of information like this that aren't documented in any one place. I really should write it all up, but keeping it up to date would be time-intensive, and out of date information is counterproductive. There are some things I haven't looked at yet:

Does the Route53 DNS service fully accommodate dynamic updates in the internal RFC1918 and external public space in the same way as a custom DNS solution?

Is it practical to use the SOAP API to dynamically pull instance IPs for ACLs on external services? Heavy scripting with outside services that support scripting could work as glue here.

IPv6 support internally and public.

Does the IAM support any configuration management beyond the AAA so that a distributed team can track and label changes in-band?

I'm choosing to keep a few other caveats to myself, particularly those involving the Enterprise Load Balancer.

Sounds like the developer has to worry about & configure load balancing, data mirroring, O/S platform, etc. Google App Engine does all that for you transparently to the developer. Sounds like AWS is just costing me more work, what is it buying me over GAE?

IaaS rarely buys you anything over PaaS, if PaaS can work for you. Until very recently, few if any PaaS players supported PHP, and I still haven't dug into the APIs and limitations of those who recently announced (like Elastic Beanstalk). I'm a big customer of PaaS and will be the last person to suggest IaaS ('virtualization servers you don't own') over PaaS if PaaS will service your workload.

helmingstay wrote:

* How the hell does all this hardware work on the backend? What are the economies of it? Is it super-secret, on a remote fusion-powered trans-dimensional island connected to the 'net via ansibles?

There is some public information on Amazon servers. They've now got economies of scale sufficient to start to do custom-contract hardware similar to Google. A combination of Taiwanese manufacturer-integrators and stateside VARs, according to the Wired articles on the subject linked from Ars. More interesting by far is their storage architecture. S3 AMIs are presumably served internally over HTTP, and EBS from conventional SAN or NAS. If COTS SAN there was probably a lot of custom glue with vendor APIs, and Amazon needs commodification and leverage, here. It's an in-house system leveraging open source in all likelihood, and more likely NAS than SAN, but that's only my educated guess.

Quote:

* Networking: Hardware IPs vs. elastic IPs?

I explained that in in some detail in my previous post. Is there anything else you want to know?

Quote:

* Competition (as others have mentioned)

Linode is a 'conventional' Xen VPS and has a nice AJAX interface, including console access. Which reminds me of a huge limitation I previously forgot, that console access is ready-only in EC2, a big operational problem. Make sure your instance will never sit and ask for an fsck (y/n) or you'll be remediating by hand. Debian/Ubuntu has a nice fix for this in /etc/default/rcS:

Code:

# grep FSCK /etc/default/rcSFSCKFIX=yes

OEL5/RHEL5/CentOS 5 don't have this. Haven't checked 6 yet.

I have some Rackspace instances also but they're also fairly conventional in comparison and I don't have nearly the size of operation there as on EC2.

* How the hell does all this hardware work on the backend? What are the economies of it? Is it super-secret, on a remote fusion-powered trans-dimensional island connected to the 'net via ansibles?

There is some public information on Amazon servers. They've now got economies of scale sufficient to start to do custom-contract hardware similar to Google. A combination of Taiwanese manufacturer-integrators and stateside VARs, according to the Wired articles on the subject linked from Ars. More interesting by far is their storage architecture. S3 AMIs are presumably served internally over HTTP, and EBS from conventional SAN or NAS. If COTS SAN there was probably a lot of custom glue with vendor APIs, and Amazon needs commodification and leverage, here. It's an in-house system leveraging open source in all likelihood, and more likely NAS than SAN, but that's only my educated guess.

Quote:

The amazing thing is how much of the "Cloud as a service" wouldn't exist or be as big if it wasn't for open source.

The amazing thing is how much of the "Cloud as a service" wouldn't exist or be as big if it wasn't for open source.

Not only the infrastructure itself, but proprietary code and licenses adapt poorly to cloud services. Oracle is license-inefficient in any virtualized environment. Amazon only makes Windows instances practical in EC2 by having a special deal with Microsoft, and probably custom infrastructure for sysid/sysprep. Any software that uses hostids, MAC-keying, or dongles is extremely painful to run in any virtualized environment (VMware tacitly caters to ISVs by disallowing arbitrary MACs in vSphere) and is a total non-starter in IaaS. Cloud is built for more-liberal licenses, open-source especially.

Microsoft is trying to turn lemons into lemonade by positioning the Azure IaaS as the best source for Windows on public cloud infrastructure without necessarily risking channel issues of giving the same liberal deal to other cloud vendors or making Software Assurance customers feel cheated by paying full retail prices for OSes they're already virtualizing.

Thanks! However, you deliberately cut my quote where I said "Amazon does make continuous improvements but one must keep up with their service releases."

Quote:

#2 -- I have no idea what you're talking about. A number of linux distros (e.g. Ubuntu) have official cloud images that are up-to-date. I can start an up-to-date stable ubuntu image in less than 3 minutes, and customize it from a script in another 2.

So can I. However you and I benefit from experience. ... Unless this has changed in the last few weeks, in which case see my original comment about keeping up with AWS improvements.

Don't be defensive. ACLs for non-AWS systems and services, obviously. Let's say you want to allow all of your AWS servers to access a private git repository hosted elsewhere to do distributed builds or infrastructure version control. Currently you have no Layer 3 ACL choices except to use individual ElasticIPs in routers and Layer 7 ACLs because all your instances' public IPs cannot be segregated into netblocks. This is a very conscious decision on the part of Amazon to stringently conserve IP address space, but it's an unexpected issue for many implementers and must be architected around.

Quote:

The documentation is still pretty thin, and I generally agree with your ElasticUnderpants assessment, but AWS has come a long way and seems pretty mature now.

This is true, but they also have a long way to go. Amazon chose to go with the fully dynamic and automateable IaaS cloud paradigm instead of a more-conventional offsite-virtualization-as-a-service, and they're consequently needing to create methodologies and standards as they go.

The problem isn't documentation per se, it's just that Amazon themselves, and no single source I've seen, will tell you things you need to know about the infrastructure works and what to do about it in order to accomplish your business goals. I'm choosing to keep a few other caveats to myself, particularly those involving the Enterprise Load Balancer.

Thanks. That was pretty epic.

#1 -- Sorry, wasn't trying to be an ass or selectively quote, etc. Was genuinely curious, and was just trying to edit for clarity.

#2 -- FWIW, the AWS "Request Instances wizard" now shows official images from RHEL, SUSE, and Ubuntu in the default, visible quickstart list. I must admit, I recall having been annoyed years ago at the abject lack of metadata in the "Community AMIs" -- here, it's basically necessary to search by ID.

#3 -- Interesting comments via OSS, leveraging, and licensing. I do really feel that docs for basic tools are annoyingly thin and non-standard -- one example -- I still don't understand the device naming scheme behind the -d arg of ec2-attach-volume...I mention this because in many ways, OSS lives and dies on the quality of its docs (and screenshots...). The fact that for-profit corps can throw a bunch of shit together and profit is all well and good, but profit is step 3, immediately *after* writing usable documentation!

#4 -- I'm not up on ACLs in general (and the wikipedia article is a little thin). I tend to think of PKI as the only real security. Is this more of a convenience issue, or is are IP-based ACLs actually secure?

Thanks for all the details. I didn't catch all of the details, but I now have a *way* better idea of the magic behind the curtain!

In general, designing high-availability systems (at least using AWS tools) is easier (and cheaper) across Zones at a single Region, than across Regions, since you don't pay for data transfer between Zones, only Regions.

This deserves to be emphasized again. Because AWS Regions are independent 'islands' of EC2 connected only by a unified UI and billing, the economies of scale across Regions are not the same as across Availability Zones. Therefore, you don't want to put assets in different regions unless you absolutely need it. I'd go so far as to say that as things exist now, you might as well implement in a second EC2 provider as in multiple Regions.

helmingstay wrote:

Thanks. That was pretty epic.

#1 -- Sorry, wasn't trying to be an ass or selectively quote, etc. Was genuinely curious, and was just trying to edit for clarity.

My apologies for being curt.

Quote:

#2 -- FWIW, the AWS "Request Instances wizard" now shows official images from RHEL, SUSE, and Ubuntu in the default, visible quickstart list. I must admit, I recall having been annoyed years ago at the abject lack of metadata in the "Community AMIs" -- here, it's basically necessary to search by ID.

Yes. We haven't even had the discussion about the amoung of trust to invest in the community AMIs -- I'm quite appalled how Amazon treats the subject. There's already been one big incident where an AMI was made public already containing root SSH keys. This seems like an obvious, if huge, security oversight but it's also an easy path to plausible deniability for an attacker of opportunity.

For those who don't know, AMIs can be made public within AWS with one simple checkbox, but it's very hard to find and guarantee the authenticity of even 'officially' published AMIs. Because AMIs are made from existing EBS-backed instances and not merely fresh installs, there's also a major risk of security breach if someone mistakenly or accidentally selects the option to make an AMI public.

Quote:

#3 -- Interesting comments via OSS, leveraging, and licensing.

Read between the lines on all of the startups who use clouds or build clouds (IaaS, PaaS, SaaS). They're not licensing a bunch of different components and paying as they scale up or out; they're using open source for almost everything. They have little choice, really. The only people leveraging restricted-license applications are those that have struck special deals to make the licensing costs viable.

Quote:

#4 -- I'm not up on ACLs in general (and the wikipedia article is a little thin). I tend to think of PKI as the only real security. Is this more of a convenience issue, or is are IP-based ACLs actually secure?

Do you use network or host-based firewalling at all? They are ACLs, usually by IP. "Allow from 192.0.2.0/24" in Apache configuration is a Layer 5 ACL by source address.

Quote:

Thanks for all the details. I didn't catch all of the details, but I now have a *way* better idea of the magic behind the curtain!

Ars can consider these posts an indication of my willingness to write an article on the topic. I know of no decent unified sources for this information, but then I haven't read the "Python and AWS Cookbook".