Friday, April 22, 2016

Setting it all up and how to avoid some stupid pitfalls..

I created a rails app to server users with the ability for them to upload images and files.
To improve the performance of the site, I serve all app assets (JavaScript, Images, CSS files, etc.) from a CloudFront distribution. I then decided to also add a CloudFront in front of the bucket I use for user uploaded files.
Using carrier wave gem is quite straightforward and even adding the CloudFront CDN was quite easy. Just in case you have no experience, here are the basics:

1. add the carrierwave and carrierwave-aws gems - this is much better than using fog which bloats your app with multiple unneeded gems. It also supports more of the AWS API.

2. add a field to your relevant table to store the uploaded image or asset, like:

add_column :users, :avatar, :string

3. In your model class, mount an uploader for this field:

mount_uploader :avatar, AvatarUploader

4. Implement an uploader class (AvatarUploader in this exmple):

class AvatarUploader &lt; CarrierWave::Uploader::Base
include CarrierWave::MiniMagick
storage :aws
# Override the directory where uploaded files will be stored.
# This is a sensible default for uploaders that are meant to be mounted:
def store_dir
"uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}"
end
# Create different versions of your uploaded files:
# version :thumb do
# process :resize_to_fit =&gt; [50, 50]
# end
# Add a white list of extensions which are allowed to be uploaded.
# For images you might use something like this:
def extension_white_list
%w(jpg jpeg gif png)
end
def fix_exif_rotation #this is my attempted solution
manipulate! do |img|
img.tap(&amp;:auto_orient)
end
end
process :fix_exif_rotation
process resize_to_limit: [200, 200]
end

In the above code, I am including MiniMagick to manipulate the uploaded image (resize and fix orientation - the orientation was to fix issue with landscape oriented images uploaded from iPhone).
If you want to use MiniMagick, you will have to add the mini_magick gem.

5. You will then need to configure carrierwave with AWS credentials, bucket name and region like that:

BTW, you could use local file storage for development by changing both carrier_wave initializer to use storage of :file as well as your uploader class.

6. To serve app assets from AWS CloudFront (for much improved performance and less load on your poor web server...), you can add an assets CDN host in your config/environments/production.rb. That's what I have:

if ENV['APP_CDN_HOST'].present?
config.action_controller.asset_host = ENV['APP_CDN_HOST']
end

7. To server those app assets from CloudFront, you will need to create a distribution pointing to your domain as origin. To do that, go to your AWS console, select the CloudFront service, and click "Create Distribution" at the top. In the following screen, click "Get Started" under the "Web" section ("RTMP" is used for streaming media). In the Origin Domain Name field, type in your domain (where your application is served from). If you want to support both http and https request you should pick "Match Viewer".

Once created, set the APP_CDN_HOST to point to the CloudFront distribution URL.

8. To place CloudFront in front of your user uploaded files, you need to add the following to your carrier_wave configuration file:

config.asset_host = ENV['CDN_HOST'] || 'http://.cloudfront.net'

I recommend of course to store the distribution in an environment variable, but sometimes it is convenient to have it as above for your development environment. You will also need to create a cloud front distribution where the origin is the bucket you use for carrier wave (as defined above in the carrier_wave.rb configuration file).

9. Now all is smooth and performant. There is one small issue (which may or may not be relevant to you): anyone with a link to a user uploaded file, can share that link and so others can access the files. If you care about privacy and security of those files, you may want to protect them with a CloudFront signed URLs. Signed URLs are signed with a trusted private key, and can encode an expiration time for the link. This means that anyone trying to access a cloud front signed URL after the expiration time, will get an access denied result. To add this extra security step, you should follow the following steps...

10. Create an IAM user identity that will be used as a trusted signer. This can be the user creating the bucket, or another user. I recommend NOT using your root identity for either of those operations. What I did, is I created a user for both creating the bucket, as well as the cloud front distribution and gave that user the trusted signer role. I automated all those actions in a ruby script (assuming the aws_boto user is an admin user I have with the given credentials):

One last issue that caused me a lot of confusion and few hours of wasted time is that if you find that some of the uploaded assets "disappear" - for example, an uploaded image shows as a broken image. The reason might be (and was on my end) the rails cache kicking in, and serving the fragment (including the image URL) beyond the signed URL expiration time. You better not cache such fragments or else they will be broken once the signed URL expiration time arrives.