Automating Elastic Beanstalk

Good practice and scalability

When I was figuring out to use Docker containers on Amazon's Elastic Beanstalk, I seemed to find plenty of articles detailing how to manually setup Elastic Beanstalk using the AWS dashboard, but I really wanted a solution that was automatable / repeatable.

Specifically, with this project we want to be able to:

Install and use Docker on Mac OSX (though other host platforms are fine)

Build a custom apache-php5 Docker image using a Dockerfile, fully provisioned and ready for our app

Build a php-app Docker image containing our custom app on top of this apache-php5 Docker image

We want to follow good practice, and this involves splitting our database and application into separate containers. This will then alow us to scale up application containers and database containers independently.

For the local install, we'll deploy both the application and database container on the same host, but when deploying for production, we'll use RDS to serve our database, and an EC2 instance dedicated to each application container.

Presuming that our application is PHP heavy but database light (say, because it only uses the database for login / personal profile storage), we can probably get away with a single
database container / RDS instance, while scaling up more EC2 instances of our application container when demand increases.

Our app will do nothing more arduous than displaying the information generated by phpinfo() (on the php-app container) and show that it can connect to a MySQL database (on the mysql container if local or the using RDS if deployed on AWS).

However, you should easily be able to build out a much more complicated PHP application after
reading through these notes.

Note that we build an apache-php5 Docker image first, and then another php-app image containing our app on top of this image. This allows us to re-use the existing apache-php5 Docker image whenever our app changes, and thus speeds up our builds.

We're also presuming that you're reasonably comfortable with the principles behind Docker, and have probably already had some exposure to AWS.

Configuration

We'll use a local shell script, .env_local, not versiond in git, to store our custom / sensitive details and implement them as environment variables.
This should be the only place we have to regularly make changes, and is used by our Makefile when running build steps:

Build the docker images

Commit and tag our changes, e.g.:

git tag 2.3.0

The use of git describe --tags in the Makefile means that we'll be using the latest git tag to tag our Docker images.
Note that, if you've made commits since your last git tag, we'll end up using a tag value which is a combination of the tag and the last commit hash:

git describe --tags
2.4.0-9-g8215032

Build our mebooks/apache-php5 base Docker image

# Create the `mebooks/apache-php5` Docker image
make base
# Check that the image was created
docker images

Edit docker/app/Dockerfile and ensure that our php-app is refering to the same version as that that we just built:

FROM mebooks/apache-php5:2.3.0

Our mebooks/apache-php5 Docker image should only need updating when we want to update the packages in the distribution, such as when there are security vulnerabilities.

Build our app docker image

Note that our app docker image wil be tagged with the current version of the repo and our Dockerrun.aws.json file will be updated accordingly.

# Create our php-app image and push it to hub.docker.com
make app
# Check that we updated the image version in our `Dockerrun.aws.json`
cat Dockerrun.aws.json | grep 'Name'
"Name": "mebooks/php-app:2.4.0-8-gb0eef33",

As we've just tagged our Dockerrun.aws.json file with the current version of the repo,
we need to commit changes, otherwise the eb create / eb deploy (which makes use of git archive) will use the previous version of the Dockerrun.aws.json file.

We should now be able to see the PHP Info details at the address reported by docker-machine ip, e.g.:

docker-machine ip
http://192.168.99.101/

If we've started the mysql container, we should also be able to see a simple example of connecting to our MySQL database at http://192.168.99.101/mysql.php

Push our images to hub.docker

Although our Makefile handles the pushing of our containers to hub.docker, we'll cover it here as we need to know about AWS using the associated config file to gain access to pull our images from hub.docker.

Login to our docker account:

# Enter the username, password and email when prompted
docker login
# Alternatively, specify username, email and password
# If any of these parameters are not supplied, you'll be prompted for them
docker login --username=mebooks --email=jcdarwin@gmail.com --password=WHATEVER

The login will create a config file at ~/.docker/config.json.

AWS currently uses an older format of the docker config for authentication to hub.docker,
so we need to change our current ~/.docker/config.json to the required format by removing the auths wrapper:

Note that https://<aws_account_id>.dkr.ecr.us-west-2.amazonaws.com is the URL for our container registry.

Copy and paste the docker login command into a terminal to authenticate your Docker CLI to the registry. This command provides an authorization token that is valid for the specified registry for 12 hours.

Create our Dockerrun.aws.json (single container version)

Although the multi-container version above is useful for testing locally, in production we'll be using RDS to host MySQL, so we'll use a single container Dockerrun.aws.json to deploy our php-app container, and use an Amazon RDS instance for our database.

If we want to reference an existing config, we can do this during creation:

eb create -v --cfg local-elasticbeanstalk

Note that eb create will use the settings from .gitattributes to export-ignore files in the zip file that it creates and uploads to s3.
As such, we have to be careful that the Dockerrun.aws.json file is included in the root level of our zip file.

Creating application version archive "app-4bbe-160402_140739".
Uploading local-elasticbeanstalk-php-demo/app-4bbe-160402_140739.zip to S3. This may take a while.

If we wish, we can easily download this zip file and inspect the contents:

s3cmd get s3://elasticbeanstalk-ap-southeast-2-<aws_account_id>/local-elasticbeanstalk-php-demo/app-4bbe-160402_140739.zip

RDS

Using Multi-AZ ensures that we mitigate the chance of failure by having our main RDS instance in one availability zone, and a RDS failover in another availability zone.
Ensure that the security group used for the RDS instance allows connections to and from port 3306, and that the EC2 instances will be also using this security group.

Importantly, we should ensure that our RDS instance is private, but set to use the same VPC as our EC2 instances.

Once setup, we should be able to connect from an EC2 instance in the same group as follows (depending on the actual assigned RDS endpoint):

As per this answer, there's currently no easy way to automate the attachment of an RDS database existing outside of an Elastic Beanstalk environment to the environment during creation.
Amazon seem to assume that you'll be wanting to create an RDS instance inside the Elastic Beanstalk environment, which means that your database becomes less independent, and may disappear should anything go wrong with your Elastic Beanstalk environment.

Instead, if you want the RDS instance to exist outside of the environment you can simply provide the connection parameters as environment variables via the EB Console: Configuration -> Web Layer -> Software Configuration:

This does mean that our application will not be able to connect to our database on the initial deployment during the eb create, as we won't have yet had the chance to set these RDS
environment variables in the EB dashboard.

Note that with this method, as the RDS_PASSWORD will be in plain sight on the AWS dashboard, it's probably wise to change this regularly. When you make changes to your environment variables on the EB dashboard and click apply, Elastic Beanstalk will then re-provision your existing EC2 instances with the new values.

Accessing our application

We should see our PHP Info page, and be able to see our database connection details at /mysql.php.

Once our Elastic Beanstalk cluster is running , we can ssh into our load balancer, and then telnet to our instances:

# once sshed into the load balancer, you'll be presented with a choice of the
# EC2 instances if there are more than one.
eb ssh
sudo -s
yum install telnet
telnet mebooks-mysql-dbinstance.criieggarwwz.ap-southeast-2.rds.amazonaws.com

Once we know the public DNS of our ec2 instance, we can also directly ssh in as ec2-user:

ssh -i ~/.ssh/aws-eb ec2-user@ec2-52-62-4-141.ap-southeast-2.compute.amazonaws.com
# Once logged in, elevate to root to be able to acces docker:
sudo -s

Redeploying

Presuming that we're only updating our custom php-app code, successive deployments should be a matter of the following:

# create the image and push it to hub.docker
make app

As make app tagged our Dockerrun.aws.json file with the current version of the repo, we need to commit the changes, otherwise the eb create / eb deploy (which uses git archive) will use the previously-committed version of the Dockerrun.aws.json file.

Terminating

Once we've finished with a particular environment, we can terminate it:

eb terminate local-elasticbeanstalk

Cleaning up

Once we're finished, we can remove our containers locally, either by id or by name

docker rm 832af3ff45d8 fc6a9553583f

If we're finished with our image, we can delete it:

docker rmi mebooks/php-app
docker rmi mebooks/apache-php5

Summary

Hopefuly the above is well-enough detailed to help you get to grips with Elastic Beanstalk.

It's a great mechanism for easily deploying a scalable Docker-based application, and all the better when we can automate the creation and deployment of our environments and applications.

In case using a Makefile as we did here becomes a bit limiting, you may want to look at using something like Ansible to automate the creation and deployment tasks, and there's
a great example of this at https://github.com/hsingh/ansible-elastic-beanstalk.

Elastic Beankstalk is a very handy service, but should you want to create a more enterprise-level cloud application, you might also want to look at something like RedHat's OpenShift Origin.

However, being a developer rather than operations, I found the method outlined above
suited me, as it was more Dev-ops (with a big 'D') rather than dev-Ops (with a big 'O').