Details

In this article, you develop a Grails-based Web application with which you can browse, add, delete, upload, and download your Amazon Simple Storage Service (Amazon S3) objects. Along the way, you get an introduction to Grails and the Amazon Web Services (AWS) software development kit (SDK) for Java. In 20 or 30 minutes, you'll have the browser for Amazon S3 running on your system:

Generating the Browser for Amazon S3 with Grails

Grails is an amazing framework that provides rapid application development (RAD) for Web applications. One of the cool features of Grails is its ability to generate the infrastructure for an entire application in seconds. Then, after creating a simple domain class, Grails can generate the artifacts required to provide Web-based maintenance for that domain.

You can download the latest Grails release from Grails - Downloads, then extract the .zip file to a local directory (I typically use /opt/grails on UNIX and C:\opt\grails on Windows). Grails requires Java SDK 5.0 or later. I usually extract my Java downloads to /opt/java on UNIX and C:\opt\Java on Windows.

With Grails and Java installed, you set up set up GRAILS_HOME and JAVA_HOME environment variables so that they point to their prospective installation directory. Then, modify the PATH environment variable to reference the Grails and Java bin directories. (See Grails - Installation for more information on installation.)

To create the browser for Amazon S3, you'll cheat a bit by asking the Grails framework to generate the application infrastructure complete with controller classes and fully styled Web pages. To do that, perform the following steps:

Use Grails to generate the maintenance infrastructure, including controller classes and HTML pages.

Generate the Infrastructure

Begin by asking Grails to generate the infrastructure for the application. Open a shell (Windows command window or UNIX shell), and navigate to a directory in which you want the application's root directory to be generated. Type the following Grails command:

grails create-app s3browser

Grails creates the complete infrastructure for a Spring-based Web application. At this point, you could test the application, but it does little more than throw up an index page.

Create the Domain Classes

Your next step is to add a couple of domain classes. You could create the files yourself, but let Grails do the work, instead. In your shell window, change your directory to the newly generated s3browser directory, and use the Grails create command to generate domain objects called Bucket and BucketObject.

Note: The Grails project is Eclipse ready in that you can import it directly into an Eclipse workspace. NetBeans and IntelliJ also have excellent support for Grails development.

Now, you can test the application. Again, from the command line, run the Grails test server:

grails run-app

Then, test the generated application from your browser:

Click the BucketController link to access your bucket list:

In the ID column, click one to display a list of bucket objects:

Working with the Generated Code

Now that you've let Grails set up the infrastructure for your browser for Amazon S3, you can make a few changes to get the application to work with live Amazon S3 data. But, before getting into that, let's review the artifacts that Grails generated to manage Bucket objects:

The important artifacts are the two Groovy classes under /grails-app/controller, the two classes under /grails-app/domain, and the Groovy Server Pages (GSP) under /grails-app/views/bucket and /bucketObject.

REST Easy with Grails

Grails uses a RESTful ModelViewController (MVC) strategy. For example, take the URL http://localhost:8080/s3browser/bucket/list. The component parts of the URL are interpreted as follows:

/s3browser. The good old context root

/bucket. By Grails convention, directs the request to BucketController for handling

/list. By Grails convention, specifies the name of the action BucketController is to invoke

The BucketController consists of a set of actions that respond to HTML form actions. The actions are implemented as Groovy closures. If you are not familiar with closures, you can think of them as methods.

The following table describes the actions of BucketController.groovy:

Action

Description

list

Build a list of Bucket objects from the relational database with built-in pagination.

create

Instance a Bucket object from the relational database to be displayed for update.

save

Perform validation on Bucket object attributes, and persist the changes to the database.

show

Retrieve a Bucket object from the relational database for display.

edit

Retrieve a Bucket object from the relational database for update.

update

Update a Bucket row in the relational database from HTTP request parameters.

delete

Delete a Bucket row in the relational database.

The following table describes the GSP that Grails generates:

File

Description

grails-app/controllers/BucketController.groovy

Handles create, read, update, and delete operations as well as list generation on Bucket objects

grails-app/views/bucket/create.gsp

Displays a new Bucket object in edit mode

grails-app/views/bucket/edit.gsp

Displays an existing Bucket object in edit mode

grails-app/views/bucket/list.gsp

Displays a paginated list of Bucket objects

grails-app/views/bucket/show.gsp

Displays a Bucket object with options to edit or delete

You may have noticed that the GSP file names matched the names of the controller's actions. That's Convention-Over-Configuration (CoF) at work. A Grails controller action, unless explicitly specified with render(view:'gspName'), will build the page whose name matches the controller's action name.

Grails also generated, for maintenance of BucketObject objects, a BucketObjectController.groovy file and its associated create, edit, list, and show GSP elements.

The AWS SDK for Java

The first step in modifying the generated code to work with Amazon S3 objects is to download the AWS SDK for Java. After extracting the AWS SDK, copy each of the following Java archive (JAR) files to your s3browser/lib directory:

The BucketController

The Grails-generated actions of the BucketController use Grails Object Relational Mapping (GORM) calls to create, read, update, and delete rows from a relational database. You'll continue to use the same actions, but you'll replace the use of the GORM API with the AWS SDK for Java.

Replace the entire contents of grails-app/controllers/BucketController.groovy with the following code:

The private static AmazonS3 s3 object that is instanced in beforeInterceptor is the controller's hook to your Amazon S3 objects. Amazon knows which Amazon S3 buckets you own from the access and secret keys you placed in the AwsCredentials.properties file. The beforeInterceptor closure does what you might guess: Before any of the action closures of the BucketObject are invoked, the beforeInterceptor's code is executed.

The following table describes the Amazon S3 functionality that replaces the Grails-generated GORM code in the BucketController's actions:

Action

Description

list

This terse closure simply creates a hash map with a value retrieved from a call to AmazonS3.listBuckets(). That map is then automatically passed to list.gsp for rendering.

create

This noop closure is simply a placeholder. RESTful requests to the create action are forwarded to create.gsp. Subsequent HTTP posts from create.gsp are handled by the save action.

save

This action double-checks for the bucket's existence before creating a new bucket. An HTTP redirect is made to the list action to rebuild the list of buckets. Note the flash.message string, if it exists, will be displayed on the list page.

show

The contents of an Amazon S3 bucket is the list of bucket objects that it contains. The BucketObjectController knows how to do that, so a redirect is made to its list action.

edit

This action is not implemented. Amazon S3 objects are not edited, they are replaced.

update

This action is not implemented. Amazon S3 objects are not updated, they are replaced.

delete

Delete a Bucket object using the AmazonS3.deleteBucket method.

In your grails-app/view/bucket directory, delete the edit.gsp and show.gsp files, and then, in grails-app/views/bucket/list.gsp, replace the contents of the <div class="list"> division with the following:

You will be modifying the create, list, and show GSPs, but go ahead and delete edit.gsp. You won't need the edit page, as you don't edit Amazon S3 objects: You can only replace the contents of an existing Amazon S3 Bucket object.

Amazon S3 Bucket Object List Processing

The list action of BucketObjectController uses several of the overloaded Amazon S3 listObjects methods. But because a bucket could contain innumerable objects, pagination is required. The Amazon S3 version of the list action could not easily use the standard Grails pagination facilities. The ListObjectsRequest parameter to the Amazon S3 listObjects(ListObjectsRequest listObjectsRequest) method has attributes for bucket name and the maximum number of keys (of bucket objects) you want returned. BucketObjectController's list closure sets the maximum key value to the default of 5 (you probably want to change that to a more reasonable number, like 10 or 20). The listObjects method returns an ObjectListing object that has an isTruncated() method. With BucketObjectController's list action, if the returned ObjectListingisTruncated() is true, the list action stores the ObjectListing to the Grails flash scope.

The existance of flash.bucketObjects is then used to predicate the Next button in list.gsp. And, when the list action responds to a Next or First button click, flash.bucketObjects is used to predicate the use of listNextBatchOfObjects(ObjectListing previousObjectListing). Also, so users know where they are in the list, a page number is kept in flash and displayed on the pagination line of list.gsp.

Note: Objects with the Grails' flash scope are retained for two HTTP request cycles.

The Bucket Object's List Page

In grails-app/views/bucketObject/list.gsp, replace the <div class="list"> and <div class="paginateButtons"> divisions with the following:

One last change to list.gsp: So that the BucketObjectController's list action knows the bucket name to list, add the following id attribute setting to the <g:link class="list" action="list"> tag that is inside <div class="nav">:

Uploads to Amazon S3 Using HTML Post

The BucketController's create action builds the information necessary for Amazon S3 to accept an HTML post. Amazon requires the post to contain, among a few other things, policy and signature parameters. The policy parameter is a JavaScript Object Notation (JSON) string that must be Base64, and the signature needs to be encrypted using your Amazon secret key. The more important values in the policy JSON string to pay particular attention to are:

bucket. This is the name of the bucket.

content-length-range. Use this value to limit the upload bytes.

expiration. This is the time at which this HTTP post request would no longer be valid. This application sets it to a day later, but you might set it to 5 or 10 minutes later for a tighter security measure.

succes_action_redirect. The request goes to Amazon, not this Web application. Amazon needs the URL of a page to display when the upload is complete.

The succes_action_redirect setting bears extra mention, as well. The Grails grails-app/conf/Config.groovy file has a setting called grails.serverURL. When in development mode, it is set to http://localhost:8080/<Your_App>. But, for production, you will have to set the proper domain and context root in Config.groovy.

The Amazon HTTP Post strategy is described in great detail in Browser Uploads to S3 using HTML POST Forms. But whereas the article provides Ruby, Java, and Python code examples, BucketController's create action gives you a groovier example.

The Bucket Object Upload Page

In your grails-app/view/bucketObject directory, delete edit.gsp. Then, replace the entire contents of create.gsp with the following:

Displaying a Bucket Object

Bucket objects can contain huge amounts of data, and that data can be in any format. But the browser for Amazon S3 blatantly displays any Amazon S3 object data in a <pre> tag of show.gsp. So, the browser for Amazon S3 has to provide pagination when showing the contents of a bucket object. Luckily, unlike the BucketObjectController's list action, the show action is able to use standard Grails pagination. Grails uses two HTTP request parametersmax and offsetto control the number of objects to display and keep track of where to start retrieving the next set of objects. With GORM/Java Database Components (JDBC), max stored the count of domain objects to display, and offset stored a row number. But with BucketObjectController's show action, max is the number of bytes to display, and offset is the index of the Amazon S3 bucket objects' byte contents. You certainly don't want to retrieve the complete contents of a 1-GB MySQL backup for display!

Among the AmazonS3.getObject()'s overloaded methods is one that takes a GetObjectRequest object. GetObjectRequest has a withRange method, which lets you set the byte range to retrieve. Here's the complete code for BucketObjectController's show action:

Then, to the <g:link class="list" action="list"> and <g:link class="create" action="create"> tags that are inside the <div class="nav"> near the top of the GSP, add an id attribute setting so that the BucketObjectController's list action knows which bucket name to list or create:

id="${params.bucketName}"

Odds and Ends

Here are a few odds and ends to finish up the browser.

Changing the Default Page

If the strange Hello Worldstyle default Grails page doesn't show up, edit grails-app/conf/UrlMappings.groovy to replace "/"(view:"/index") with "/"(controller:'bucket',action:"/list"). Now, whenever you go to the root URL or click Home, the bucket list will be displayed.

The Download Option

You may have noticed the download link in the Bucket Object list. That download is implemented with two Groovy statements in a closure called download in BucketObjectControoler:

Using the combined magic of Groovy and Grails, the InputStream returned from the AmazonS3Client.getObject() method is left-shifted << into the HTTP output response stream. The browser then prompts the user to open or save the file. Between the upload option (invoked by clicking the New Bucket Object button) or the download links, you have a simple way to back up (and version) important files.

Note: If you would like to download a file locally, use the following one-liner:

new File('<your_local_file_name>').append(s3.getObject('<Bucket_Name>', '<Bucket_Key>').getObjectContent())

Deleting Buckets and Bucket Objects

Deletion of buckets and bucket objects is enabled with Delete links that point to the delete action of BucketController or BucketObjectController. Those links pass the bucket name and (if the deletion is for a Bucket object) the object's key. The Controller code is trivial. BucketObjectController's delete action simply executes the following:

s3.deleteObject(params.bucketName, params.key)

The delete action then puts an informational message in the flash scope and redirects to the list action. Subsequently, the list.gsp page displays the deletion message.

BucketController's delete action does pretty much the same thing as BucketObjectController, except that it wraps s3.deleteBucket(params.bucketName) in a try/catch block. I found that I had to add the try/catch to monitor for the Amazon "The bucket you tried to delete is not empty" error. If an error occurs, the delete action stuffs the message in the flash scope and redirects to the list action.

Improvements

There are a couple of areas where the browser for Amazon S3 application could be improved:

The Browser for Amazon S3's Bucket List Home Page

The Amazon S3 Bucket List displays all the buckets you own. If you click New Bucket, you will be prompted for the name of the bucket you want to create. The browser for Amazon S3 application will add the bucket (or display an error, such as "The bucket already exists") and rebuild the Bucket Object List to show the newly added bucket.

Create a Bucket Prompt

On the home page (the Amazon S3 Bucket List,) if you click one of the Bucket Object links, a paginated list of the objects in that bucket is displayed.

The Amazon S3 Bucket Object List

Click one of the bucket object links to see a paginated list (where each page is 10,000 bytes) of the contents of that object.

The Amazon S3 Bucket Object Display

The Bucket Object page, in addition to pagination buttons, has a New BucketObject button and a Delete button. A click of Delete deletes the object for the Amazon S3 bucket and redisplays the Amazon S3 Bucket Object List. A click of New BucketObject prompts you with the Amazon S3 Bucket Object Create page.

The Amazon S3 Bucket Object Create Page

The Create page's HTTP post goes directly to Amazon, so the page is built with the Amazon S3required encryption policy and signature variables. When you have selected a file to upload and click Upload, Amazon adds the bucket and redirects your browser to the browser for Amazon S3's Bucket Object List.

If, on the Amazon S3 Bucket Object List, you click one of the download links, the browser for Amazon S3 application pulls the object from Amazon and passes it to the browser, where you will be prompted to save the file.

The S3 Bucket Object Download Page

The Download page is self-explanatory.

Stats and Usage

The browser for Amazon S3 application gives you a simple interface with which to manage your Amazon S3 objects. You pay Amazon to store those objects, but without a browser, the number and size of your Amazon S3 objects can easily get out of control. It's also nice to have a simple facility to quickly back up, restore, or transfer files. For example, I've been using the browser for Amazon S3 to save some of the oddball files I have from spread sheets to MySQL dumps.

What is impressive about Groovy and Grails is that this application contained a mere 127 lines of code (not counting the HTML.) Grails generated the application infrastructure, mananged the REST-based MVC architecture, and provided the simplified user interface of GSP. Groovy made the interface with the AWS SDK for Java trivial.

Regardless of whether you use the browser for Amazon S3, a benefit of this article is that it provides a tutorial on the use of the AWS SDK for Java. You now have a kit-bag of 13 liners of Groovy code that gives you easy access to maintenance operations on your Amazon S3 objects.