A fascination with internet maps

Main menu

Monthly Archives: November 2008

Elastic Block Store, EBS, is a useful extension of the AWS services offered by Amazon’s cloud platforms. EBS provides a way to add storage in an EC2 instance independent mode. In other words, storage doesn’t have to be tied to the instance storage, but can exist independently as an external volume. The big deal here is the persistence of your data, even if an instance happens to get killed for some reason. An additional backup security is the ability to make snapshots of an EBS volume and store on the S3 service in S3 buckets.

The cost is not a great burden:

EBSVolumes

·$0.15 per GB-month of provisioned storage

·$0.10 per 1 million I/O requests

Amazon EBS Snapshots to Amazon S3

·$0.15 per GB-month of data stored

·$0.01 per 1,000 PUT requests (when saving a snapshot)

·$0.01 per 10,000 GET requests (when loading a snapshot)

The basic approach for creating an EBS volume:

1.start an instance

2.create an EBS volume

3.attach the volume to the instance

4.partition and format the volume

5.add data and services to the instance and its attached volume

6.bundle and register the instance as an AMI stored in S3

7.create a snapshot of the EBS volume

After this is complete there is a peace of mind knowing that the instance can be reconstructed from backup services.

Restore follows this path if the EBS volume is intact:

1.start a new instance from the AMI bundled previously

2.just attach the volume to the new instance

3.repoint the DNS to this new instance server

Restore follows this path if the EBS volume is also trashed:

1.start a new instance from the AMI bundled previously

2.create a new volume from the S3 snapshot

3.attach this new volume to the new instance

4.repoint the DNS to this new instance server

I have an Open Source GIS stack loaded on a windows ec2 instance and decided it was time to make a conversion to the security of an EBS volume.

Once you have a useable EBS, how would you go about making it useful to the GIS stack?

In my stack I am using:

PostgreSQL/PostGIS

Tomcat

Geoserver

This means I would like to move all of the PostgreSQL data, tomcat webapps, and the geoserver data to the new EBS volume. Then it will be available for snapshot backup.

Postgresql data:

Changing Postgresql data to a new location involves a change to the registry. Stop postgresql service, then change registry ImagePath, move the C:\Program Files\PostgreSQL\8.3\data subdirectory to its new EBS location, E:\postgresql_data, and finally restart the service.

Run regedit:

“HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\pgsql-some version” and on the ImagePath change the –D option to point to a location on your new EBS volume. Here is the wicki entry with details:Postgresql Wicki for changing PGDATA.

Tomcat webapps:

Open %TOMCAT_HOME%/config/server.xml with a text editor. There should be an entry similar to this:

<Host name=”localhost” appBase=”E://tomcat_webapp”

unpackWARs=”true” autoDeploy=”true”

xmlValidation=”false” xmlNamespaceAware=”false”>

Here you can see that I’ve changed the appBase to point to a subdirectory on my E: drive, the EBS volume. Copy the existing webapp subdirectory to this EBS subdirectory.

Geoserver data:

Go to the geoserver webapp’s WEB-INF/web.xml and make sure that the GEOSERVER_DATA_DIR points at a location on the EBS volume. Remember to make the change to the web.xml found in the tomcat webapp directory on the new EBS volume. Copy the geoserver data to its new EBS subdirectory.

<context-param>

<param-name>GEOSERVER_DATA_DIR</param-name>

<param-value>E:\geoserver_data</param-value>

</context-param>

Now the data for PostgreSQL/PostGIS, Apache Tomcat, and Geoserver will be accruing on an external volume, safe from sudden EC2 instance death. Of course now that EC2 is no longer beta and the SLA agreement is available this should be a rare occurrence.

Now to make things even safer lets run a snapshot:

C:\EC2>ec2-create-snapshot vol-********

SNAPSHOTsnap-********vol-********pending 2008-11-04T22:14:30+0000

C:\EC2>ec2-describe-snapshots snap-********

SNAPSHOTsnap-********vol-********completed2008-11-04T22:14:30+0000

At this point a snapshot of my volume is stored to S3 where I can use it to create a new volume for use in another instance. I can use the snapshot if I’m creating multi instance clusters or if I need to restore my instance.

Of course it would also be wise to make an AMI bundle to reflect the changes made to the basic instance, directory pointers regedits etc. Here is the ami bundle guide for windows instances: AMI Bundle for windows info

You will first need to prepare a bucket on S3 to receive the AMI bundle. S3 Info

Amazon cloud is now out of beta and comes with independent storage volumes and snapshot capability useful for backup and scaling functions. GIS open source stacks can make use of these options without a huge effort.