Downloading Large Files from Amazon S3 with the AWS SDK for iOS

Version 2 of the AWS Mobile SDK

This article and sample apply to Version 1 of the AWS Mobile SDK. If you are building new apps, we recommend you use Version 2. For details, please visit the AWS Mobile SDK page.

This content is being maintained for historical reference.

In a previous post, we discussed the S3TransferManager and how it can be used for uploading larger files to Amazon S3 with the AWS SDK for iOS. But what about downloading larger files? We are currently looking into extending the S3TransferManager to also handle downloads, but in the meantime, the SDK does offer ways for handling larger files with the lower level interface.

Streaming to a file

The most straightforward method to download larger files with the AWS SDK for iOS is to use the outputStream property of the S3GetObjectRequest. Setting this property to an already opened stream will prevent the SDK from caching the entire file in memory.

This example code runs synchronously (see our previous post about synchronous vs. asynchronous requests). If you choose to also assign an AmazonServiceRequestDelegate to track progress of the download, be aware that the data is passed to the request:didReceiveData: method and should not be retained or appended to a buffer if you want to avoid an out of memory exception. Additionally, we will need to close our stream in the delegate callbacks on failure or success instead of the code block that initiated the request.

So we now can get the contents of a large file from a single request and stream it to a file, but what happens if we aren’t on the most reliable connection? Thankfully, if we do timeout or otherwise fail to download the whole file, we don’t necessarily need to restart the whole request.

S3 Ranged Gets

S3GetObjectRequest supports fetching a portion of the file through the use of two properties, rangeStart and rangeEnd. These values are 0-indexed and inclusive, meaning that if we have a file that is 2000 bytes, the valid range values would be 0–1999. In order to make use of ranged gets, our app will need to know the size of the file before we start the download. We can easily get this value by using the S3GetObjectMetadataRequest object, which among other things will return a response with the file’s contentLength.

The previous code is not quite complete; you will likely want to add a maximum number of retries and also break out of the retry loop if you detect there is no network connection at all. Additionally, you may want to verify the integrity of the download by calculating the MD5 checksum and comparing it to the value in S3. Hopefully, it gives you the building blocks you need to allow your app to download larger files, even on slower or unreliable connections. We are eager for feedback, and we want to know what other challenges developers face when building cloud-backed mobile apps. Please feel free to leave a comment below, or visit our forums to post feedback and questions.