Tag Archives: CloudFormation

For our Serverless project running on AWS infrastructure we needed an outbound Lambda API call to a SaaS platform which demands a whitelist of the source IP addresses. Which is pretty hard since AWS has a whole range. Luckily there is a trick to target your Lambda function in a VPC which can be configured to use a elastic IP for outbound communication.

High Level Design

Steps to configure the VPC / Network

Step 1 – create/use a VPC

Our AWS Network configuration always start with an AWS VPC (Virtual Private Cloud). You can use an existing VPC and configure your subnets there, or create a new one. Network configuration on AWS can look simple with the GUI wizard, but the setup of your VPC had major impact on all your resources so for real production a thing to remember. In the example I use CIDR block 10.0.0.0/16 which gives me (way to much) options.

Step 2 – create a public and 1-n private subnets

We will need 1 public subnet to attach to a NAT Gateway and route our traffic to the Internet. The example can work with 1 private subnet which hosts your Lambda function. However for availability purposes you would want to have multiple private subnets in different availability zones for your lambda to run. In the example I use CIDR block 10.0.11.0/24 for the public subnet and 10.0.21.0/24, 10.0.22.0/24 and 10.0.23.0/24 for the 3 private subnets on each eu-west-1 availability zone.

Step 3 – Configure the Internet Gateway and the public subnet configuration

We need an Internet Gateway for our VPC, attach to our public subnet and a public route table configuration which targets all (0.0.0.0/0) outbound traffic from the public subnet.

We need a NAT Gateway which uses an elastic IP address. By adding a private route table which we attach to our private subnet(s) we make sure that all functions in the VPC will use our elastic IP for outbound communication. At lease, if we make sure the private route table contains a (0.0.0.0/0) target to the NAT Gateway

Configure your Lambda function

Make sure your Lambda function(s) use the configured VPC by selecting it in the VPC pulldown and then select all the private subnet(s).

Notes:

Make sure your Lambda runs with the IAM AWSLambdaVPCAccessExecutionRole

In other Serverless projects we use a single resource definition to include AWS CloudFormation scripts in our build/deploy. Where the resource file contained component definitions like DDBTable: in the root.

resources:
Resources: ${file(resources.yml)}

However a new project contains a lot of AWS configurations so I decided to split it up. You can do that using the next syntax. However make sure your files start with the Resources: data type.

We needed a public network load balancer with SSL (through AWS Certificate Manager) and took me some retry’s to get it right since most examples are based upon the classic or application load balancer so here to share:

In AWS CloudFormation templates you often have the need to make a reference to an earlier created component. For instance the unique ID of a vpc, subnet, security group or instance. You have two choices: continue with separated stacks or combine them to create a nested stack. In our case we wanted to keep seperated stacks so needed a way to export/import settings from an earlier network related CloudFormation stack holding our VPC and Subnet Identifier which the new stack uses. To share information between stacks we can export output values from the first stack and import these values in new stacks. Other stacks that are in the same AWS account and region can import the exported values.

Example of an VPC ID which we export in our network stack (named t10-cf-vpc):

While spinning up EC2 instances you can bootstrap them with packages, files, etc in different ways. For our stack we wanted to pull scripts from an AWS CodeCommit to make life easier.

The (bash) scripts are stored in our CodeCommit so first we need to make sure the EC2 instances, while spinning up, are allowed to access the repository. So we created an IAM Policy with these sufficient rights and attach the policy to a IAM role which we can use to attach to our EC2 instances.

We make sure the EC2 instances uses the new IAM Role by defining IamInstanceProfile with our example IAM Role t10-ec2-role in the CloudFormation template. Further on by using the UserData segment we can execute scripts during bootstrap of the server. Install the AWSCLI is required for the credential helper

So the error ended in a Google search with many hits, many questions, many suggestions, but very few real answers.

Until I saw this answer from johnhunsley:I believe you have created a Security Group without specifying a VPC ID. You have then attempted to create a launch config which launches instances into a subnet within a VPC. Therefore, when It attempts to assign the security group to those instances it fails because it expects the security group ID rather than the name.

So I think the response from AWS is in the running for the “Worst Error Message Ever” but the solution is very simple. Don’t make the mistake of not specifying your custom VPC ID when creating a new security group.