Becoming more flexible

Our friends at Engine Yard have just launched the beta of their new cloud hosting product, Flex. If you’re familiar with their Solo product you’ll find Flex to be pretty similar, just more… flexible. Where solo lets you run your Ruby on Rails application on an Engine Yard stack on an Amazon EC2 instance, Flex lets you run it on a cluster of EC2 instances.

In the last month I’ve put a handful of applications up on Solo, mostly demo or staging servers for doing story acceptance and release testing. Solo is great for that. After you’ve gone through the setup process once, you can easily spool up a server for a few hours when you need it on just a few minutes notice, then turn it off when it’s not needed anymore. Flex gives you that same kind of adaptability, allowing you to add instances to your cluster to match traffic demands as needed.

Last week I got my first production application running on Flex. The Pivotal Labs company website is now hosted on Flex, and it’s humming along quite nicely. There were a few rough spots to work through since we were working with a pre-release product, but I’m pretty happy with our setup now. Since Flex is new, I thought it might be useful to share some of the things my fellow pivots and I learned getting things running there.

It turns out the trickiest bit for us was converting some rather intricate URL rewrite rules from Apache to Nginx. Yes, Flex and Solo support Apache, but we’ve been moving to standardize on Nginx so it was worth a bit of fiddling. My point is, the hardest part had nothing to do with Flex itself.

Now, about Flex. If you haven’t checked out Solo yet, there are some things that are different from your standard Engine Yard deployment, or most other deployments even. For starters, your server config has to be something that can be started and stopped and restarted. There are two or three parts to how that works. The base system (OS + packages) is created from an image and configured using Chef. The database has its own storage and uses an Amazon EBS volume for storing snapshots. The Rails application and some system config files (everything under /data) are on a separate EBS volume. Keeping snapshots on EBS means it’s easy to restore an instance.

Setting things up is fairly easy but takes a few steps. Each instance is assembled from an environment, application, IP address, SSL certs. I don’t want to do a full tutorial here, but just want to give some context and an idea of the level of complexity. Going through all the steps in the control panel to configure an instance takes only a few minutes. I can set up a new application now in about 10 minutes if I’m not doing anything tricky with multiple virtual hosts. If you want to see how it works in detail, check out the Getting Started page for EY Cloud.

There are a few configuration options where you’ll probably end up wondering how to decide which to pick. Some of these will affect how much you pay so you should make sure you are working within your budget, but here’s what I’d recommend.

Availability Zone: the default is probably fine. If you care about this you probably know enough for how to choose.

Instance Size: 1.7GB RAM / 5ECU / 32bit – this gives good performance and the best value for your dollar. This is similar to a 2 core 2.3 GHz server.

Storage Volume: 5GB – the default size is probably fine. All you’re storing here is your app code and static assets. You probably want to put any uploaded files on S3 instead of storing on EBS.

Database Volume: ??? – this is the MySQL file system and needs space for the storage for your database. Pick a size that’s large enough for your requirements. 5GB is probably fine for small apps, but I’ve seen much, much larger.

By the way, EBS volume snapshots are stored on S3, so you’ll be paying for storing them as well.

As I said, converting the web server config from Apache to Nginx was a pain, but that’s not typical. Aside from that, there are a few things to adjust to. The biggest is that in many cases you may not need to use Capistrano to automate your application deployment. If your application code is in a git repo, all you need to do is supply a URL and set up the deploy key. The Chef deploy scripts will take care of pulling down the code, creating symlinks, running migrations, and managing restarts of Mongrel or Passenger. If you require extra steps, you can add ruby files to your project that run at several hooks (before migration, etc). Or you can use Capistrano, which is what we did for this site. We had a fairly standard eycap based deploy setup, and it only took a few minutes to convert it to deploy to Flex. If you do opt to use Capistrano, you want to make sure when you click the deploy link in the control panel to uncheck the options to deploy the code and run migrations.

You won’t be able to run sendmail on your Flex server, so you’ll need to find a SMTP service to use to send email from your application. You can run a SMTP server on your own box somewhere else, or you can use a 3rd party service. We’re using authsmtp.com and that’s working fine, but I wasn’t involved in that choice so I can’t say much about why we chose them.

Flex/Solo let you specify the rubygems your app needs in the control panel, but I don’t like that solution. It’s hard to manage and keep up to date. We have our apps set up using geminstaller. Once you have your gems listed in config/geminstaller.yml you only need to add the geminstaller gem in the control panel. We added a hook in deploy/before_migration.rb to run geminstaller and everything works fine now. I suppose you could do something similar using the config.gem and rake gems:install, but I’ve found that pretty fragile for installing gems (go ahead, mention a class defined in a gem in your environment and see what happens).

New products always have a few rough edges and Flex is still getting polished. There are a couple areas I’m eager to see improvement on. One is how system configuration files are managed – the current approach for keeping modified files by tweaking their names is an interim solution and isn’t very user friendly. I’m also looking forward to more flexibility in customizing the chef scripts, and API access for configuring instances instead of going through the GUI for everything. It would also be nice to have a way to manage sensitive files you don’t want to store in your code repo, like S3 key files.

Flex is a nice solution for easy scalable deployment, even considering it’s a beta release. Engine Yard is already aggressively working on addressing all of the issues I’ve encountered with the system, and then some. Congratulations to them on a compelling new product.