Post navigation

Using ELB to Serve Multiple Domains Over SSL on EC2 for Giggles and Unicorns

One of the complaints about EC2 is that you only have one IP address allocated per instance which makes it difficult to host, in a clean manner, multiple domains that require SSL certs. Where you have control over IP allocation you could punch down a couple for one server and then set up your domains and SSL certs by IPs. That method is a no go so you are left with the ugly method of allocating those certs by port, something that Joe and Jane public are a little skittish about (https://superawesomefuntime.com is cool but https://superawesomefuntime.com:8443 smells phishy). Thankfully, ELB makes for a great proxy to hide the hideousness of the port-based workaround.

Essentially, as solutions go, this one is stupid easy; for each domain that you need to handle SSL for create a load balancer. Many balancers to one or a pool of instances. That’s it. Here’s a sloppy diagram…

For this example, we’ll be serving superawesomefuntime.com and unicorns-unlimited.com:

Take note of the address which are returned to you as need that for the cname portion of the game. They’ll look something like this: superawesome-123456789.us-east-1.elb.amazonaws.com and unicorns-987654321.us-east-1.elb.amazonaws.com.

The next step would be to add your instance to those load balancers:

elb-register-instances-with-lb superawesome --instances i-12ab3c45

elb-register-instances-with-lb unicorns --instances i-12ab3c45

Over on the server set up your virtual hosts in a way that hopefully is as hasty as mine…

Then all you need to do is set up the cname for each domain to point at the load balancer address, in this example the cnames would be for *.superawesomefuntime.com = superawesome-123456789.us-east-1.elb.amazonaws.com and *.unicorns-unlimited = unicorns-987654321.us-east-1.elb.amazonaws.com.

The only loose end is configuring your healthcheck but that is a deeply personal decision best left for you and your app to work out.

Actually the vast majority of punters probably don’t even notice that (even though it might smell a bit to you or I). The real drawback is that a lot of corporate firewalls block anything but port 80 and 443 for web access and you find that significant numbers of people can’t get to the SSL part of your site.

I documented doing the port-based approach in detail on my blog but pulled the post when I found we were getting people complaining about not getting to our site. I ended up just going with a multi domain cert but the technique you describe looks like the best of both worlds.

Hmm, well as I understand it ELB is acting as a reverse-proxy much as if you dropped Nginx or HAProxy in front of your web servers to act as a balancer, all the traffic flows through the balancer on it’s foward facing ports. The whole process should be transparent to the end user. Were the people complaining that they could reach your site able to prior to implementing ELB? It has been several years since I was managing a corporate firewall but we implemented also a content filter that had white/black list domains and that changed on a fairly regular basis.

To see how all this was operating I performed a quick test using ufw/iptables locally and operating in a restricted mode:

Unsurprisingly, ports 8443 and 8445 on the raw instance address failed (browser timed out) but hitting them on 443 using the cname or the elb address worked. Granted this is a terribly unscientific method but it bears out the assumption that ELB is acting as a reverse proxy though it is worth digging up some testers behind corporate firewalls.

Good article, though not exactly what i was looking for, but something i have pondered and will be implementing and linking to in the near future.

I should mention, if you came across this article while looking for a way to actually enable 443 or any other port, other then 80 and 22 on your instance (and probably got completely lost, because you are just getting started), hit up for a quick amazon tools setup tutorial amongst which lingers the command to enable 443 on your instance (or 8443 or 8445, or whatever port) http://paulstamatiou.com/how-to-getting-started-with-amazon-ec2

You should be able to secure those ports through your amazon tools with ec2 tools too, no?

ec2-revoke default -p 8443

this way if you have a group of servers you are doing this with, you could manage them all at the same time…

Well, ELB operates outside the EC2 firewall so port 8443 needs to be open for it to communicate to it. It is likely possible to limit it to the ELB firewall IP range but I have seen mine shift a bit so I’m not too comfortable closing it that tight.

Ha! Well, glancing over the Unicorn docs it seems that it should be fairly straight forward though it would seem that you need to drop in something like Nginx or Apache to handle the SSL portion of things, so if that’s the case it’s just a matter of changing the listener for each domain you’d want to serve.

That sounds like a great approach. I have a question about how you would handle DNS for this though. I normally like to redirect example.com to http://www.example.com in Apache so that both are usable. This would require that I pass example.com to ELB but if I remember correctly your root DNS record cannot point to a CNAME. Thanks for any insights!

Thanks for the post James. One issue I’m running into is that I cannot assign a CNAME record for my root domain to point to the ELB host. In the context of your tutorial, I want my DNS to point superawesomefuntime.com to the CNAME superawesome-123456789.us-east-1.elb.amazonaws.com . This is possible for the subdomain “www” but as far as i know it’s not possible for the root of the domain. This means that https://www.superawesomefuntime.com would work but https://superawesomefuntime.com would not work. Got any ideas?

Correct, just like in the example we are using multiple ELBs and CNames in front of several instances running apache. Each instance is joined to each ELB with port 443 mapping to the corresponding port for that wildcard cert (ie superawesomefuntime.com has an ELB listener of 443 mapping to 8443 and unicorns-unlimited.com has an ELB listener of 443 mapping to 8445).

To handle failure we scripted an up or down state for port 80 which ELB is listening to for the health check, if our app fails then that health check will fail and the server is pulled out of all the balancers. For example:

That will have ELB check http://ec2-public-address:80/health and if it returns anything other than 200 it stops serving traffic from it. We also use this method to do rolling deploys through the stack while still keeping the site online.

We definitely use the command line tools for a lot of things, but I never tried to create an ELB from the CLI tools. We were using the Amazon Control Panel web app to create our load balancers. In the AWS control panel, when you try to create an ELB it will only show you instances that are not already assigned to a load balancer. So we just assumed it couldn’t be done.

Thanks to your post, I created a new ELB using the CLI tools and was able to do just that!

This worked on Windows Server 2008 and my ssl was on the server. Use http 80 -> 80, tcp 443 -> something else (4431) for the load balancer settings, and the health check should use tcp 80. Once you update your DNS to point to the load balancer it should work.

Checks: be sure to turn off or modify firewall, check the amazon firewall, and make sure it is running by checking the Health Check.

I had the Health Check set to HTTP and it was causing an error due to my website setup not using the default website in IIS so no /index.html file could be found.

Good job – Thanks for great information on how to solve my mulitple SSL problem!

Why do the certs need to be on the apache instances? When you create the ELB cant you tell associate it with a signed certificate you have already uploaded via:
iam-servercertupload -b public_key_certificate_file -c certificate_chain_file -k privatekey.pem -s certificate_object_name

This way the ELB will handle the overhead associated with SSL, and the apache servers can accept cleartext http to a non secure listening vhost? This will allow the apache server instances to perform better (dont have the mod_ssl overhead)…

So example:
I upload the *.superawesomefurntime.com and *.unicorns-unlimited SSL certs via the cmd above.

the superawesome ELB (using the superawesome cert) will listen on 443 and route to http(unsecure) port 8440 (*.superawesomefuntime.com vhost)
the unicorns ELB (using the unicorns cert) will listen on 443 and route to http(unsecure) port 8441 (*.unicorns-unlimited.com vhost)

When I first wrote this ELB didn’t offer cert hosting and that is a welcome addition. So thinking this out (bear with me, riding out a NyQuil hangover)…

For myself, the challenge is that all traffic for the app must be served over HTTPS and since the loadbalancer cannot (as far as I know) be locked down via the security groups it creates a potential hole where plaintext traffic exists. The way I have things set up at the moment is that all HTTP traffic, outside the healthcheck, is intercepted by mod-rewrite and pushed to HTTPS, additionally the application displays custom errors if people try and hit the instance directly, either by its public DNS or the FQDN we punched down for it.

For my own piece of mind and compliance with our security practices I’d prefer that the necessary ports only be open to the ELB so that the plaintext traffic becomes less of an issue. That said, if it is not as much of a pressing issue for yourself I cannot thing of a reason why moving the certs into the balancer would not work.