Crafting a Secure and Maintainable AWS Multi-Account Setup #1

AWS and other public clouds give developers huge flexibility and freedom to innovate so they can act faster on business requirements. However, such power doesn't come without a cost.

How do you implement a consistent least privilege access policy in a big and complex single AWS account or in a multi-account setup? How do you limit the blast radius in case of a security breach?

Securing AWS resources the old-school way

AWS API authentication is something that is managed on a per-account level, and in case of a security compromise, someone can gain access to all resources in an account. A best practice is to limit each individual’s access to just the APIs and the API calls that they require to perform their daily work. This is all good and sounds perfectly logical until your team gets big and you need to implement more complex IAM policies.

Tags to the rescue! One way to limit access to one service in a shared account between several teams is to use resource tags and conditions in the IAM policies.

This approach is recommended by AWS, but speaking from personal experience this is not a silver bullet.

One problem with this approach is that not all resources support tags.

Which is weird since AWS tell you that you should tag everything.

As your environment gets more and more complex, so do your IAM policies which also increases the risk of errors in them. At this point, it’s a good idea to start thinking about limiting the scope of your credentials on an account level.

Introducing AWS Organizations

To achieve this you basically create more accounts to separate different teams, projects or departments. This approach has several advantages in addition to limiting the scope.

One big plus is that AWS cost can be separated by environment, project or business unit for example. When you use a single AWS account, some cost factors can’t be allocated to the corresponding cost center, for example, network traffic. Another advantage is that now you can’t have two teams interfering with each other due to something like resource limit exhaustion.

As a negative, networking can become complex, but it depends on the use case. Managing several accounts doesn’t seem troublesome, but with the increase in the number of accounts, management can become a problem. This is where you might benefit from AWS Organizations.

Additional pros are that volume discounts are shared between all the accounts in an organization and that you can use the so-called Service Control Policies (SCP) to limit certain services and regions from being used in an individual account or a group of accounts. SCPs act more like a boundary, meaning they limit what you can use in a specific account. SCPs help you with security policy enforcement as they can’t be bypassed even by the root user of an AWS account.

An SCP can be assigned on the organization root. This means that it’s inherited by all accounts in the organization. It can be assigned on an Organizational Unit (a group of accounts) or on an individual account.

Service Control Policies Scope

SCPs use the same syntax as IAM policies so it’s nothing that a typical engineer with some knowledge in IAM can’t understand. SCPs can be whitelists or blacklists. A best practice is to choose one type and make sure you stick with it as mixing them can lead to confusion.

Another benefit of AWS organizations is that the account creation/management can be implemented entirely through code. Thus you are able to create as many accounts as needed (some companies have gone berserk and have thousands of accounts). This means that you can separate your infrastructure and thus limit the blast radius. More on this topic will be covered in a future blog post.

Final thoughts

The engineers at AWS have worked hard on making it easy to build and manage an organization. Currently, there are two major AWS products that can help you with this. The first one is AWS Landing Zone and the second is the brand new AWS Control Tower.

AWS Landing Zone has been available for a while. It provides standard multi-account patterns, the ability to apply a baseline to each account and some basic networking like VPC peering. While this solution works there are some cons to it. It’s not trivial to make changes and troubleshooting can be frustrating as there are many moving parts involved.

At the time of writing this post (summer of 2019), AWS Control Tower was just released and it’s not a mature service. If you already have an AWS organization then you can’t use Control Tower and also there is still no API available. I’m sure that, in the future, AWS will polish Control Tower and it will become the standard.

Stay tuned for our next post when we will share how we implemented AWS Organizations at ST6.