Package files

Internal call graph ▹

Internal call graph ▾

In the call graph viewer below, each node
is a function belonging to this package
and its children are the functions it
calls—perhaps dynamically.

The root nodes are the entry points of the
package: functions that may be called from
outside the package.
There may be non-exported or anonymous
functions among them if they are called
dynamically from another package.

Click a node to visit that function's source code.
From there you can visit its callers by
clicking its declaring func
token.

Functions may be omitted if they were
determined to be unreachable in the
particular programs or tests that were
analyzed.

const (
// DefaultBatchSize is the batch size we initialize when constructing a batch delete client.// This value is used when calling DeleteObjects. This represents how many objects to delete// per DeleteObjects call.DefaultBatchSize = 100
)

const DefaultDownloadConcurrency = 5

DefaultDownloadConcurrency is the default number of goroutines to spin up
when using Download().

const DefaultDownloadPartSize = 1024 * 1024 * 5

DefaultDownloadPartSize is the default range of bytes to get at a time when
using Download().

const DefaultUploadConcurrency = 5

DefaultUploadConcurrency is the default number of goroutines to spin up when
using Upload().

DefaultUploadPartSize is the default part size to buffer chunks of a
payload into.

const (
// ErrDeleteBatchFailCode represents an error code which will be returned// only when DeleteObjects.Errors has an error that does not contain a code.ErrDeleteBatchFailCode = "DeleteBatchError"
)

const MaxUploadParts = 10000

MaxUploadParts is the maximum allowed number of parts in a multi-part upload
on Amazon S3.

const MinUploadPartSize int64 = 1024 * 1024 * 5

MinUploadPartSize is the minimum allowed part size when uploading a part to
Amazon S3.

GetBucketRegion will attempt to get the region for a bucket using the
regionHint to determine which AWS partition to perform the query on.

The request will not be signed, and will not use your AWS credentials.

A "NotFound" error code will be returned if the bucket does not exist in the
AWS partition the regionHint belongs to. If the regionHint parameter is an
empty string GetBucketRegion will fallback to the ConfigProvider's region
config. If the regionHint is empty, and the ConfigProvider does not have a
region value, an error will be returned..

For example to get the region of a bucket which exists in "eu-central-1"
you could provide a region hint of "us-west-2".

GetBucketRegionWithClient is the same as GetBucketRegion with the exception
that it takes a S3 service client instead of a Session. The regionHint is
derived from the region the S3 service client was created in.

type BatchDeleteObject struct {
Object *s3.DeleteObjectInput// After will run after each iteration during the batch process. This function will// be executed whether or not the request was successful.
After func() error
}

BatchDeleteObject is a wrapper object for calling the batch delete operation.

type BatchDownloadObject struct {
Object *s3.GetObjectInput
Writer io.WriterAt
// After will run after each iteration during the batch process. This function will// be executed whether or not the request was successful.
After func() error
}

BatchDownloadObject contains all necessary information to run a batch operation once.

type BatchUploadObject struct {
Object *UploadInput// After will run after each iteration during the batch process. This function will// be executed whether or not the request was successful.
After func() error
}

BatchUploadObject contains all necessary information to run a batch operation once.

type Downloader struct {
// The buffer size (in bytes) to use when buffering data into chunks and// sending them as parts to S3. The minimum allowed part size is 5MB, and// if this value is set to zero, the DefaultDownloadPartSize value will be used.//// PartSize is ignored if the Range input parameter is provided.
PartSize int64
// The number of goroutines to spin up in parallel when sending parts.// If this is set to zero, the DefaultDownloadConcurrency value will be used.//// Concurrency is ignored if the Range input parameter is provided.
Concurrency int
// An S3 client to use when performing downloads.
S3 s3iface.S3API
// List of request options that will be passed down to individual API// operation requests made by the downloader.
RequestOptions []request.Option
}

The Downloader structure that calls Download(). It is safe to call Download()
on this structure for multiple objects and across concurrent goroutines.
Mutating the Downloader's properties is not safe to be done concurrently.

NewDownloader creates a new Downloader instance to downloads objects from
S3 in concurrent chunks. Pass in additional functional options to customize
the downloader behavior. Requires a client.ConfigProvider in order to create
a S3 service client. The session.Session satisfies the client.ConfigProvider
interface.

NewDownloaderWithClient creates a new Downloader instance to downloads
objects from S3 in concurrent chunks. Pass in additional functional
options to customize the downloader behavior. Requires a S3 service client
to make S3 API calls.

Download downloads an object in S3 and writes the payload into w using
concurrent GET requests.

Additional functional options can be provided to configure the individual
download. These options are copies of the Downloader instance Download is called from.
Modifying the options will not impact the original Downloader instance.

It is safe to call this method concurrently across goroutines.

The w io.WriterAt can be satisfied by an os.File to do multipart concurrent
downloads, or in memory []byte wrapper using aws.WriteAtBuffer.

If the GetObjectInput's Range value is provided that will cause the downloader
to perform a single GetObjectInput request for that object's range. This will
caused the part size, and concurrency configurations to be ignored.

DownloadWithContext downloads an object in S3 and writes the payload into w
using concurrent GET requests.

DownloadWithContext is the same as Download with the additional support for
Context input parameters. The Context must not be nil. A nil Context will
cause a panic. Use the Context to add deadlining, timeouts, ect. The
DownloadWithContext may create sub-contexts for individual underlying
requests.

Additional functional options can be provided to configure the individual
download. These options are copies of the Downloader instance Download is
called from. Modifying the options will not impact the original Downloader
instance. Use the WithDownloaderRequestOptions helper function to pass in request
options that will be applied to all API operations made with this downloader.

The w io.WriterAt can be satisfied by an os.File to do multipart concurrent
downloads, or in memory []byte wrapper using aws.WriteAtBuffer.

It is safe to call this method concurrently across goroutines.

If the GetObjectInput's Range value is provided that will cause the downloader
to perform a single GetObjectInput request for that object's range. This will
caused the part size, and concurrency configurations to be ignored.

A MultiUploadFailure wraps a failed S3 multipart upload. An error returned
will satisfy this interface when a multi part upload failed to upload all
chucks to S3. In the case of a failure the UploadID is needed to operate on
the chunks, if any, which were uploaded.

type UploadInput struct {
// The canned ACL to apply to the object.
ACL *string `location:"header" locationName:"x-amz-acl" type:"string"`
Bucket *string `location:"uri" locationName:"Bucket" type:"string" required:"true"`
// Specifies caching behavior along the request/reply chain.
CacheControl *string `location:"header" locationName:"Cache-Control" type:"string"`
// Specifies presentational information for the object.
ContentDisposition *string `location:"header" locationName:"Content-Disposition" type:"string"`
// Specifies what content encodings have been applied to the object and thus// what decoding mechanisms must be applied to obtain the media-type referenced// by the Content-Type header field.
ContentEncoding *string `location:"header" locationName:"Content-Encoding" type:"string"`
// The language the content is in.
ContentLanguage *string `location:"header" locationName:"Content-Language" type:"string"`
// The base64-encoded 128-bit MD5 digest of the part data.
ContentMD5 *string `location:"header" locationName:"Content-MD5" type:"string"`
// A standard MIME type describing the format of the object data.
ContentType *string `location:"header" locationName:"Content-Type" type:"string"`
// The date and time at which the object is no longer cacheable.
Expires *time.Time `location:"header" locationName:"Expires" type:"timestamp" timestampFormat:"rfc822"`
// Gives the grantee READ, READ_ACP, and WRITE_ACP permissions on the object.
GrantFullControl *string `location:"header" locationName:"x-amz-grant-full-control" type:"string"`
// Allows grantee to read the object data and its metadata.
GrantRead *string `location:"header" locationName:"x-amz-grant-read" type:"string"`
// Allows grantee to read the object ACL.
GrantReadACP *string `location:"header" locationName:"x-amz-grant-read-acp" type:"string"`
// Allows grantee to write the ACL for the applicable object.
GrantWriteACP *string `location:"header" locationName:"x-amz-grant-write-acp" type:"string"`
Key *string `location:"uri" locationName:"Key" type:"string" required:"true"`
// A map of metadata to store with the object in S3.
Metadata map[string]*string `location:"headers" locationName:"x-amz-meta-" type:"map"`
// Confirms that the requester knows that she or he will be charged for the// request. Bucket owners need not specify this parameter in their requests.// Documentation on downloading objects from requester pays buckets can be found// at http://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html
RequestPayer *string `location:"header" locationName:"x-amz-request-payer" type:"string"`
// Specifies the algorithm to use to when encrypting the object (e.g., AES256,// aws:kms).
SSECustomerAlgorithm *string `location:"header" locationName:"x-amz-server-side-encryption-customer-algorithm" type:"string"`
// Specifies the customer-provided encryption key for Amazon S3 to use in encrypting// data. This value is used to store the object and then it is discarded; Amazon// does not store the encryption key. The key must be appropriate for use with// the algorithm specified in the x-amz-server-side​-encryption​-customer-algorithm// header.
SSECustomerKey *string `location:"header" locationName:"x-amz-server-side-encryption-customer-key" type:"string"`
// Specifies the 128-bit MD5 digest of the encryption key according to RFC 1321.// Amazon S3 uses this header for a message integrity check to ensure the encryption// key was transmitted without error.
SSECustomerKeyMD5 *string `location:"header" locationName:"x-amz-server-side-encryption-customer-key-MD5" type:"string"`
// Specifies the AWS KMS key ID to use for object encryption. All GET and PUT// requests for an object protected by AWS KMS will fail if not made via SSL// or using SigV4. Documentation on configuring any of the officially supported// AWS SDKs and CLI can be found at http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingAWSSDK.html#specify-signature-version
SSEKMSKeyId *string `location:"header" locationName:"x-amz-server-side-encryption-aws-kms-key-id" type:"string"`
// The Server-side encryption algorithm used when storing this object in S3// (e.g., AES256, aws:kms).
ServerSideEncryption *string `location:"header" locationName:"x-amz-server-side-encryption" type:"string"`
// The type of storage to use for the object. Defaults to 'STANDARD'.
StorageClass *string `location:"header" locationName:"x-amz-storage-class" type:"string"`
// The tag-set for the object. The tag-set must be encoded as URL Query parameters
Tagging *string `location:"header" locationName:"x-amz-tagging" type:"string"`
// If the bucket is configured as a website, redirects requests for this object// to another object in the same bucket or to an external URL. Amazon S3 stores// the value of this header in the object metadata.
WebsiteRedirectLocation *string `location:"header" locationName:"x-amz-website-redirect-location" type:"string"`
// The readable body payload to send to S3.
Body io.Reader
}

type UploadOutput struct {
// The URL where the object was uploaded to.
Location string
// The version of the object that was uploaded. Will only be populated if// the S3 Bucket is versioned. If the bucket is not versioned this field// will not be set.
VersionID *string
// The ID for a multipart upload to S3. In the case of an error the error// can be cast to the MultiUploadFailure interface to extract the upload ID.
UploadID string
}

type Uploader struct {
// The buffer size (in bytes) to use when buffering data into chunks and// sending them as parts to S3. The minimum allowed part size is 5MB, and// if this value is set to zero, the DefaultUploadPartSize value will be used.
PartSize int64
// The number of goroutines to spin up in parallel per call to Upload when// sending parts. If this is set to zero, the DefaultUploadConcurrency value// will be used.//// The concurrency pool is not shared between calls to Upload.
Concurrency int
// Setting this value to true will cause the SDK to avoid calling// AbortMultipartUpload on a failure, leaving all successfully uploaded// parts on S3 for manual recovery.//// Note that storing parts of an incomplete multipart upload counts towards// space usage on S3 and will add additional costs if not cleaned up.
LeavePartsOnError bool
// MaxUploadParts is the max number of parts which will be uploaded to S3.// Will be used to calculate the partsize of the object to be uploaded.// E.g: 5GB file, with MaxUploadParts set to 100, will upload the file// as 100, 50MB parts.// With a limited of s3.MaxUploadParts (10,000 parts).
MaxUploadParts int
// The client to use when uploading to S3.
S3 s3iface.S3API
// List of request options that will be passed down to individual API// operation requests made by the uploader.
RequestOptions []request.Option
}

The Uploader structure that calls Upload(). It is safe to call Upload()
on this structure for multiple objects and across concurrent goroutines.
Mutating the Uploader's properties is not safe to be done concurrently.

NewUploader creates a new Uploader instance to upload objects to S3. Pass In
additional functional options to customize the uploader's behavior. Requires a
client.ConfigProvider in order to create a S3 service client. The session.Session
satisfies the client.ConfigProvider interface.

NewUploaderWithClient creates a new Uploader instance to upload objects to S3. Pass in
additional functional options to customize the uploader's behavior. Requires
a S3 service client to make S3 API calls.

Upload uploads an object to S3, intelligently buffering large files into
smaller chunks and sending them in parallel across multiple goroutines. You
can configure the buffer size and concurrency through the Uploader's parameters.

Additional functional options can be provided to configure the individual
upload. These options are copies of the Uploader instance Upload is called from.
Modifying the options will not impact the original Uploader instance.

Use the WithUploaderRequestOptions helper function to pass in request
options that will be applied to all API operations made with this uploader.

UploadWithContext uploads an object to S3, intelligently buffering large
files into smaller chunks and sending them in parallel across multiple
goroutines. You can configure the buffer size and concurrency through the
Uploader's parameters.

UploadWithContext is the same as Upload with the additional support for
Context input parameters. The Context must not be nil. A nil Context will
cause a panic. Use the context to add deadlining, timeouts, ect. The
UploadWithContext may create sub-contexts for individual underlying requests.

Additional functional options can be provided to configure the individual
upload. These options are copies of the Uploader instance Upload is called from.
Modifying the options will not impact the original Uploader instance.

Use the WithUploaderRequestOptions helper function to pass in request
options that will be applied to all API operations made with this uploader.

UploadWithIterator will upload a batched amount of objects to S3. This operation uses
the iterator pattern to know which object to upload next. Since this is an interface this
allows for custom defined functionality.