Stats

Exploring Amazon S3 with F#

The article demonstrates how to explore the Amazon S3 API using F# and achieve great results just in 45 minutes.

Introduction

Traditional imperative languages teach developers that program code requires ceremony. You can't just write a line and expect it to do something. When programming book writers explain how easy it is to display the "Hello world" string in their favorite language, they actually start with an explanation that every call should be wrapped in a method, and the method typically takes some arguments etc. And of course, they have to let beginners know how to compile the resulting program and how to execute it. Without all these steps, the world won't get its greeting.

All these steps have valid reasons, but as a result, development environment based on major imperative languages do not really encourage exploration. It's best suited towards large scale development, but if a developer wants to have a quick look on something new, and he can't proceed without creating a new project, then he will often postpone this work until he has enough time or enough reasons to go ahead.

Luckily, there have always been alternatives to languages like C# or Java that worked better for rapid prototyping or technology exploration. One of the late additions is F#, and when I recently introduced myself to Amazon Simple Storage Service, I realized that if I need to get a quick overview of its programming facilities, using F# would be much more efficient than writing a test program in C#.

The purpose of this article is to show how fast you can dig into the low level details of an unknown technology using F#, and how little code you need to write. Therefore, I keep the text of the article short, focusing just on the required steps. If you need more information about F# or Amazon S3, there are plenty of books and articles available. Just Google for them. Instead of numbering steps that I've gone through, I'll provide timing information: I started the exercise at about 17:00 CET on July 15th. It took me about 45 minutes to display a picture extracted from my S3 bucket. And all code was written in F#.

17:00. Getting the Amazon Web Services SDK

Amazon Web Services SDK, also known as AWS SDK, is available at http://aws.amazon.com/sdkfornet. Once downloaded, its installer copies binaries and samples, and optionally registers AWSSDK.dll in the GAC.

17:10. You can't go wrong with Reflector

Before firing a Visual Studio F# session, I checked the contents of the installed binaries and loaded into Red Gate .NET Reflector the only .NET assembly that I found: AWSSDK.dll. I could probably have managed without it, but it would have taken longer. After I created an S3 client session, I mostly relied on Intellisense support for F# in Visual Studio, but spending a few minutes looking at namespaces and classes hinted me about entry points. For example, I assumed that I would need to create an instance of AmazonS3Client and probably use classes from Amazon.S3.Model.

17:15. Opening an F# Interactive session

I started Visual Studio and immediately was able to use the comparative advantage of the F# environment. I didn't have to create a project - in fact, I didn't know what kind of project I would need to create if I had to: Should it be a console application? Or a Windows Forms program? I didn't know yet what to expect, all I wanted to get from an IDE is a Notepad where I could write code lines and execute them one by one. So F# was a very good fit for my intention: I created a new F# script file and started sending individual lines to an FSI window. The first lines referenced AWSSDK and imported the Amazon.S3 namespace that I expected to be useful (thanks Reflector):

#r "AWSSDK"open Amazon.S3

Later I imported another namespace (Amazon.S3.Model), but these two lines above were enough to connect to the Amazon S3 storage.

17:20. Connecting to Amazon S3 and accessing a bucket

The fun part began. My guess was that since the AmazonS3Client class has a constructor with two string parameters, this is the one I should use, and assign parameters to the AWS key ID and secret key. F# Intellisense confirmed it. So here's my first call to Amazon S3 in F# (I removed the actual access key values).

let s3Client = new AmazonS3Client("KEY-ID", "SECRET-KEY")

After a few seconds, F# Interactive came with a response:

val s3Client : AmazonS3Client

Now I had a mighty client with many interesting methods to try. The next logical step would be to list the available buckets. I only had one, so I could retrieve only the first element of the list.

Great! This looked like a real thing. Encouraged with a quick success, I wanted to start retrieving the bucket contents.

17:25. Failed attempt to retrieve bucket objects

The bucket collection was wrapped in a sequence, so I only needed to get its head to obtain the only bucket I had at S3:

let bucket = Seq.head response.Buckets

I thought in order to retrieve bucket objects, I had to send the obtained bucket to some method, but I was wrong. Actually, I didn't need to obtain a bucket at all - since I knew the name of the bucket, I could construct a ListObjectRequest from the name and send it to a ListObjects. But I didn't know about it 5 minutes ago. Anyway, here're new calls, to retrieve bucket objects - data files with images from our home photo gallery:

Now when I am writing this article, I am in fact glad that I had this problem. If everything went smooth, you might look at my timing information with a grain of disbelief: the guy probably knew what he was doing. No I didn't! So I had to check around. Since the error came after some basic steps, I hoped that I wasn't alone in experiencing it, and luckily, I was not. Several people complained at Amazon forum, and they were recommended a simple workaround in case they would accept using HTTP instead of HTTPS. And for the purpose of exploration, it was just fine.

17:30. Connection to Amazon S3 revisited

Here's an updated version of the connection code, taking now two lines instead of one.

17:35. Retrieving object data

I inspected the response and found the property that I needed: S3Objects. It was a collection of bucket objects representing the photo gallery files I uploaded to S3. Taking a further look at the available AmazonS3Client methods, I found a method that I should use to retrieve an individual object: GetObject. It took an instance of GetObjectRequest. I first created a GetObjectRequest instance without assigning a Key property. But after receiving an exception with an error message about missing a key, I corrected my mistake. Here's a code that worked:

One of the properties was called ReponseStream. This sounded very promising - perhaps I could try to create an image out of this stream and display it in a Windows Form?

17:40. Displaying an image

What should you do if you want to display an image in a Windows Form in C#? You have to start from creating a new project using the Windows Forms Application template. Then you can edit the Form class and add an Image control to it in the Form designer. How do you do this in F#? You simply create a form with an image, right from the script. Here's the code:

Note that the first four lines are just to reference the required DLLs and import namespaces. The rest of the code is equally short and completes the task. Look at the Image.FromStream call: I wasn't quite sure I could do this, but the property name "ResponseStream" was too tempting not to give it a try. And suddenly I saw this window:

This is our house cat Figaro (unfortunately, not with us anymore). And no, I didn't specifically select this photo for my test. This was a random choice, perhaps hinting about what kind of pictures some families tend to take.

17:45. Done!

Yes, we are! Perhaps just to show my environmentalist attitude, I'll make one last call:

s3Client.Dispose()

Conclusion

So it was three quarters of exploration. Completely new API, combined with my relatively poor experience with Amazon S3, but a picture from a family photo gallery retrieved from an S3 bucket and shown in a Windows Form proves that we managed the full stack of operations. All done in F#. This experiment does not expose other (and far more important) language qualities, but I deliberately wanted to focus on a different topic: language efficiency for the purpose of technology exploration and rapid prototyping. Being a .NET language, F# can access any .NET area or feature, and its REPL support ensures that the developer's time is spent in a most optimal way.

Share

About the Author

Vagif Abilov is a Russian/Norwegian software developer and architect working for Miles. He has more than twenty years of programming experience that includes various programming languages, currently using mostly C# and F#.

Vagif writes articles and speaks at user group sessions and conferences. He is a contributor and maintainer of several open source projects, including Simple.Data OData adapter, Simple.OData.Client and MongOData.

Yes, much of the code I wrote here would look structuraly similar if it was implemented in C#. Pure F# beauties were covered in my other article about symbolic calculations (Symbolic Calculation in F#[^]. I had two reasons to use F# for this work:

1. F# is a great tool for explorative scripting, something that C# lacks. You don't need to create an application to begin exploring new things - just create a new script and start experimenting.2. Developers often are skeptical about trying new languages because they are afraid of steep learning curve and that they can stay more efficient with the languages they already know. And in case of functional languages we hear voices that there must be some special reasons to start using them - it's only niche tasks that will fit functional programming. I tried to address this kind of arguments with my article - that you can be as efficient as you would be with imperative languages even when implementing traditional tasks - like integrating 3rd party API or calling Web services.

Вагиф АбиловMCPD (Enterprise Application Development)Oslo, Norway

If you're in a war, instead of throwing a hand grenade at the enemy, throw one of those small pumpkins. Maybe it'll make everyone think how stupid war is, and while they are thinking, you can throw a real grenade at them.Jack Handey.

Thanks, I hadn't read the inro (unless it was changed) .Even in c# you have the immediate window for some minor iterative things , but it's true it's less strong than F# for this purpose.I'll change my vote to a 5

Well you didn't have to reconsider your vote. But thanks! It's motivating

Вагиф АбиловMCPD (Enterprise Application Development)Oslo, Norway

If you're in a war, instead of throwing a hand grenade at the enemy, throw one of those small pumpkins. Maybe it'll make everyone think how stupid war is, and while they are thinking, you can throw a real grenade at them.Jack Handey.

If you're in a war, instead of throwing a hand grenade at the enemy, throw one of those small pumpkins. Maybe it'll make everyone think how stupid war is, and while they are thinking, you can throw a real grenade at them.Jack Handey.

If you're in a war, instead of throwing a hand grenade at the enemy, throw one of those small pumpkins. Maybe it'll make everyone think how stupid war is, and while they are thinking, you can throw a real grenade at them.Jack Handey.

I always enjoy learning how other people employ Amazon S3 online storage. I am wondering if you can check out my very own tool CloudBerry Explorer that helps to manage S3 on Windows . It is a freeware.

Of course I checked it and even bought yesterday Pro version. Excellent stuff!

Вагиф АбиловMCPD (Enterprise Application Development)Oslo, Norway

If you're in a war, instead of throwing a hand grenade at the enemy, throw one of those small pumpkins. Maybe it'll make everyone think how stupid war is, and while they are thinking, you can throw a real grenade at them.Jack Handey.

If you're in a war, instead of throwing a hand grenade at the enemy, throw one of those small pumpkins. Maybe it'll make everyone think how stupid war is, and while they are thinking, you can throw a real grenade at them.Jack Handey.