Set your VPC [enableDnsSupport] setting to true to ensure that queries to the Amazon provided DNS server at the 169.254.169.253 IP address, or the reserved IP address at the base of the VPC IPv4 network range plus two will succeed. See the Using DNS with Your VPC Amazon Web Servies documentation for additional details.

Private Subnet Requirements

The following additional settings are required to support using private subnets on AWS with CircleCI:

The private subnet for builder boxes must be a network address translation (NAT) instance or an internet gateway configured for the outbound traffic to the internet.

The VPC Endpoint for S3 should be enabled. Enabling the VPC endpoint for S3 should significantly improve S3 operations for CircleCI and other nodes within your subnet.

Adequately power the NAT instance for heavy network operations. Depending on the specifics of your deployment, it is possible for NAT instances to become constrained by highly parallel builds using Docker and external network resources. A NAT that is inadequate could cause slowness in network and cache operations.

If you are integrating with https://github.com/, ensure that your network access control list (ACL) whitelists github.com webhooks. When integrating with GitHub, either set up CircleCI in a public subnet, or set up a public load balancer to forward github.com traffic.

Sizing AWS Instances

The following table describes the number of containers or concurrent builds supported by the M3, C3, and R3 machines using the the default CircleCI container size of 2CPU/4GB RAM and 2CPU overhead.

Type

Supported Containers

Model

Notes

M3

maximum of 3

m3.2xlarge

Largest M3 instance

C3

maximum of 6

c3.4xlarge

Limited memory allocation

C3

maximum of 14

c3.8xlarge

Limited memory allocation

R3

maximum of 3

r3.2slarge

High memory, limited CPU

R3

maximum of 7

r3.4xlarge

High memory, limited CPU

R3

maximum of 15

r3.8xlarge

High memory, limited CPU

The R3 instances are a good choice for memory-intensive builds or if you plan to increase the default memory allocation for each container. If you have noisy neighbor problems or contention for resources, it is possible that the additional memory of the R3 family may speed up your builds without changing the default container allocation.

Steps for Installation

This procedure uses Terraform to automate infrastructure provisioning and management of the CircleCI Services and Builders.

Open terraform.tfvars in your favorite text editor and input the
prerequisites stated above in the required fields.

Launch the Terraform stack:

Linux or Windows: Run the terraform apply command from the repo root.

OSX: Run the bin/terraform apply command and follow the prompts.

Note: you may need to run terraform init first to install the Terraform plugin for AWS.

Go to the URL output from Terraform and click Get Started.

Trust the temporary SSL cert.

Use a new or exisiting self-signed certificate. By default, all machines in a CircleCI Enterprise installation verify SSL certificates for the GitHub Enterprise instance. If you’re using a self-signed cert,
or using a custom CA root, select the “HTTPS (with self-signed certificate)” option in the System Console at port 8800.
You also need to export CIRCLE_IGNORE_CERT_HOST=insecure-ghe.example.com on builder machines replacing insecure-ghe.example.com with the host of your GitHub Enterprise instance. See this doc for details on setting builder machine environment variables.

Upload your CircleCI license.

Secure the console with a password.

Register CircleCI as a new OAuth application in GitHub at https://github.com/settings/applications/new using the IP address of the AWS instance for the Homepage URL and using http(s)://AWS instance IP address/auth/github as the Authorization callback URL. Click the Register Application button.

Copy the Client ID from GitHub and paste it into the entry field for GitHub Application Client ID.

Copy the Secret from GitHub and paste it into the entry field for GitHub Application Client Secret and click Test Authentication.

Agree to the license agreement.

Save and start the CircleCI application. The System Console Dashboard appears with an indication that the app has started as shown in the following image.

Next Steps for Getting Started

Click the Open link in the dashboard to go to the CircleCI Enterprise app. The Starting page appears for a few minutes as the CircleCI application is booting up, then automatically redirects to the homepage.

Sign up or sign in by clicking the Get Started button. Because you are the first user to log in, you become the Administrator.