Signup for news on OpenSolaris AMIsList of OpenSolaris AMIs Technical Support Team

ZFS snapshots to and from S3

We can use ZFS snapshots to save and restore filesystems from one
Solaris EC2 instance to another. This functionality is very useful, for
example, for saving user home directories, web server documents, MySQL
databases, etc., terminating a EC2 instance, and then restoring these
filesystems on a new EC2 instance created at a later date.

Amazon Simple Storage Service (S3) provides a simple web services
interface that can be used to store and retrieve any amount of data, at
any time, from anywhere on the web. It gives any developer access to
the same highly scalable, reliable, fast, inexpensive data storage
infrastructure that Amazon uses to run its own global network of web
sites.

Getting StartedIn our example, we use the second EC2
ephemeral storage device, c0d1p0 (c4d1p0 for OpenSolaris 2008.05, c3d0s0 for OpenSolaris 2008.11),
to create a ZFS pool. We create filesystems within this pool for data
that
we want to restore for each new EC2 instance, and then save and restore
the data using S3.

We use the S3 Sync command line tools listed in the references section
to interface with S3. We assume the that reader has an AWS account and
has a basic understanding of EC2, S3 and Solaris ZFS.

In this section we describe the steps necessary to get started using
ZFS within the EC2 environment. For our example, we use the SXCE Build 79 32 bit
image AMI: ami-eb7a9f82.
This example has been tested and works with any of the Sun provided
Solaris AMIs, including OpenSolaris 2008.05.

Please note that there is an additional step required if you are using
the OpenSolaris 2008.05 AMI. This AMI does not have ruby installed.
Since the S3 Sync tools require ruby, we need to install ruby if using
the OpenSolaris 22008.05 AMI.

bash# which rubyno ruby in /usr/sbin /usr/bin /usr/local/aws/s3sync

bash# pkg install SUNWruby18
bash# which ruby/usr/bin/ruby

Set the correct time, set your AWS
environment variables, and create a S3 bucket to store your saved ZFS
snapshots.

We are now ready to create our ZFS pool and a few file systems. Logged
in to the instance created and setup with the steps above, do the
following.

Note: the extra disk for the OpenSolaris 2008.05 image is: c4d1p0. From
this point forward, we use the SXCE Build 79 32 bit image in our
examples. If using OpenSolaris 2008.05, substitute c4d1p0 for c0d1p0.

The s3 pool mount points that you list appear to be capable of storing 147GB of data, however, I was under the impression that the maximum size of an S3 object is 5GB. How does your solution work around this limitation? I.E. a fully populated s3pool/home would result in a s3-home.bz2 file larger than 5GB which exceeds the S3 object size limitation... does your solution some how slit the s3-home.bz2 over multiple S3 objects within a given bucket?

Thank you for your interest and question. I did not consider the 5Gb object limit in S3, but I think we can solve with an enhanced process to upload the files to S3, described in the blog entry in section "Upload the ZFS snapshot streams to S3".

Included below is a sample script that can be used. At this point we have created the ZFS snapshots and saved to a directory, for example /var/tmp/snapshots. We then cd to this directory and run the script shown below. Not sure of the formatting of this comment, so please feel free to email me at sean.odell @ sun.com if you would like me to send the example script file to you.

One thing to note is that you will need to modify the "restore from S3" process to retrieve the stream piece parts and then use cat to assemble back into a bz2 file. Also, I did not have a file larger then 5Gb, so I used 500Mb to test.

#===============================================
#!/usr/bin/bash

# script to split files and then upload to S3 bucket

# 500 Mb
s3objmazsize=$((500\*1024\*1024))

# 5 Gb
#s3objmazsize=$((5\*1024\*1024\*1024))

# set the S3 bucket name to store the objects
s3bucketname=skodell.ec2.zfs.snapshots

We describe a process to add a script to a Solaris AMI which is then executed every time the instance is first launched. In the example, we setup a NFS server, which when first launched, downloads ZFS snapshots that were saved to S3, and then restores the ZFS snapshots.