NAME

Amazon::S3::Bucket - An Amazon S3 bucket and object interface

SYNOPSIS

use Amazon::S3;

my $s3 = Amazon::S3->new(
  { credentials => $credentials,
    region      => 'us-east-1',
  }
);

my $bucket = $s3->bucket('example-bucket');

$bucket->add_key(
  'example.txt',
  'hello world',
  { content_type => 'text/plain',
  }
);

my $object = $bucket->get_key('example.txt');

my $response = $bucket->list_v2(
  { prefix => 'logs/',
  }
);

$bucket->delete_key('example.txt');

DESCRIPTION

Amazon::S3::Bucket represents an Amazon S3 bucket and provides bucket-scoped object operations.

Instances are normally created by "bucket" in Amazon::S3 or "bucketv2" in Amazon::S3 rather than by calling new() directly.

This document is primarily a method reference. For broader discussion of credentials, checksums, object listing, multipart uploads, error handling, and directory buckets, see Amazon::S3.

METHODS AND SUBROUTINES

CONSTRUCTOR

new

my $bucket = Amazon::S3::Bucket->new(%options);

my $bucket = Amazon::S3::Bucket->new(\%options);

Creates and returns a bucket object.

The constructor accepts either a list of key/value pairs or a hash reference.

The following options are supported:

account

Required. The Amazon::S3 object associated with this bucket.

bucket

Required. Bucket name.

buffer_size

Buffer size used when streaming object data.

The default is 4096 bytes.

logger

Logger used by the bucket object.

When omitted, the logger from account is used.

region

Region containing the bucket.

When omitted and verify_region is false, the region configured on the associated Amazon::S3 object is used.

verify_region

When true and no region is supplied, determine the bucket region by calling get_location_constraint().

The default is false.

The constructor throws an exception when bucket or account is not supplied.

On success, returns the new Amazon::S3::Bucket object.

ACCESSORS

account

my $s3 = $bucket->account;

Gets or sets the associated Amazon::S3 object.

bucket

my $name = $bucket->bucket;

Gets or sets the bucket name.

buffer_size

my $buffer_size = $bucket->buffer_size;

$bucket->buffer_size($bytes);

Gets or sets the buffer size used when streaming object data.

creation_date

my $creation_date = $bucket->creation_date;

Gets or sets the creation date associated with the bucket object.

Bucket objects returned by "buckets" in Amazon::S3 may have this value populated from the ListBuckets response.

logger

my $logger = $bucket->logger;

$bucket->logger($logger);

Gets or sets the logger used by the bucket object.

region

my $region = $bucket->region;

$bucket->region($region);

Gets or sets the region containing the bucket.

verify_region

my $verify_region = $bucket->verify_region;

$bucket->verify_region($boolean);

Gets or sets whether the bucket constructor should determine the bucket region when no region is supplied.

OBJECT OPERATIONS

add_key

my $ok = $bucket->add_key( $key, $value );

my $ok = $bucket->add_key( $key, $value, %configuration, );

my $ok = $bucket->add_key(
  { key     => $key,
    data    => $data,
    headers => %headers,
  }
);

my $ok = $bucket->add_key(
  { key      => $key,
    filename => $filename,
    headers  => %headers,
  }
);

my $ok = $bucket->add_key(
  { key     => $key,
    fh      => $fh,
    headers => %headers,
  }
);

my $ok = $bucket->add_key(
  { key      => $key,
    callback => $callback,
    headers  => %headers,
  }
);

Creates or replaces an object.

The traditional positional interface and the hash-reference interface are both supported.

Positional interface

The positional interface accepts:

key

Required. Object key.

value

Required. Object content.

A scalar value is uploaded directly.

A scalar reference is interpreted as a filename and the referenced file is streamed to S3.

Use add_key_filename() when uploading a file by name.

configuration

Optional hash reference containing request headers and object configuration.

Entries are added to the request headers. A nested headers hash reference may also be supplied; entries in headers take precedence over duplicate top-level configuration entries.

The special acl_short entry sets x-amz-acl after validating the canned ACL value.

Hash-reference interface

The hash-reference interface requires key and exactly one upload source.

Supported upload sources are:

data

Scalar object content.

filename

Name of a local file containing the object data.

The supplied file is used directly and is not copied to temporary storage.

fh

Filehandle from which object data is read.

The complete contents of the filehandle are staged in a temporary file before the request is sent.

callback

Code reference that supplies object data.

The callback is repeatedly invoked until it returns undef. Each successful invocation must return a scalar reference containing the next chunk of object data.

For example:

my $callback = sub {
  return
    if !@chunks;

  my $chunk = shift @chunks;

  return \$chunk;
};

The complete contents produced by the callback are staged in a temporary file before the request is sent.

Only one of data, filename, fh, or callback may be supplied.

Other entries in the hash reference are treated in the same manner as the positional configuration hash.

Temporary storage

Uploads using fh or callback are staged completely in a temporary file before the S3 request begins.

Staging allows C[Amazon::S3](Amazon::S3) to determine the complete object size, calculate request checksums, and replay the upload if necessary.

The system temporary directory must therefore be writable and must have sufficient free space to hold the complete object. An upload using fh or callback may fail before contacting S3 if temporary storage cannot be created or written.

Uploads using data do not require temporary storage.

Uploads using filename use the supplied file directly and do not create an additional temporary copy.

Temporary files created for fh or callback uploads are removed after the upload completes or fails.

Checksums

Content-MD5 is added automatically.

C[Amazon::S3](Amazon::S3) also calculates and sends an S3 checksum automatically. By default, CRC64NVME is used. Most applications do not need to select or calculate a checksum explicitly.

The checksum algorithm can be changed using the associated L[Amazon::S3](Amazon::S3) object's checksum_algorithm setting.

See L[Amazon::S3/CHECKSUMS](Amazon::S3/CHECKSUMS).

Return Value

On success, returns a true value.

On failure, returns undef or throws an exception depending on the underlying request failure.

add_key_filename

my $ok = $bucket->add_key_filename($key, $filename, %configuration,);

Creates or replaces an object using the contents of a local file.

This is a convenience wrapper around the filename form of add_key().

The supplied file is streamed directly and is not copied to temporary storage.

Returns the same value as add_key().

See "add_key".

copy_object

my $result = $bucket->copy_object(
  { key    => $destination_key,
    source => $source_key,
    bucket => $source_bucket,
    headers => \%headers,
  }
);

my $result = $bucket->copy_object(
  key    => $destination_key,
  source => $source_key,
  bucket => $source_bucket,
);

Copies an S3 object.

The bucket represented by this object is the destination bucket.

The following parameters are supported:

bucket

Optional source bucket name.

The default is the destination bucket.

headers

Optional hash or array reference containing request headers.

x-amz-copy-source may be supplied directly in these headers.

x-amz-tagging-directive defaults to COPY.

key

Required destination object key.

source

Source object key.

Either source or the x-amz-copy-source request header is required.

When x-amz-copy-source is not supplied explicitly, it is generated from bucket and source.

On success, returns the parsed CopyObjectResult response.

On a non-2xx response, records the S3 error and throws an exception.

The HTTP response is available through last_response().

See https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html.

delete_key

my $ok = $bucket->delete_key($key);

my $ok = $bucket->delete_key($key, $version_id);

Deletes an object.

key

Required object key.

version_id

Optional version ID.

When supplied, the specified object version is deleted.

Returns a true value on success.

delete_keys

my $response = $bucket->delete_keys(@keys);

my $response = $bucket->delete_keys(\@keys);

my $response = $bucket->delete_keys(\@objects);

my $response = $bucket->delete_keys($callback);

my $response = $bucket->delete_keys(
  { keys    => \@objects,
    quiet   => 1,
    headers => \%headers,
  }
);

Deletes multiple objects using the S3 DeleteObjects API.

The following input forms are supported:

list of keys
$bucket->delete_keys(qw(foo bar baz));
array reference of keys
$bucket->delete_keys([qw(foo bar baz)]);
array reference of object hashes

Each hash contains Key and may contain VersionId.

$bucket->delete_keys(
  [ { Key => 'foo', VersionId => '1' },
    { Key => 'bar' },
  ]
);
callback

The callback is repeatedly invoked and should return a key and, optionally, a version ID.

Iteration ends when the callback returns no key.

$bucket->delete_keys(
  sub {
    return ( $key, $version_id );
  }
);
configuration hash reference

The hash reference supports:

headers

Optional request headers.

keys

Required key specification in one of the supported forms.

quiet

Optional boolean controlling DeleteObjects quiet mode.

The default is false.

A maximum of 1000 objects may be supplied in one call.

The request Content-MD5 header is generated automatically.

Returns the response from the DeleteObjects request.

Invalid input or more than 1000 objects causes an exception.

get_key

my $object = $bucket->get_key($key);

my $object = $bucket->get_key(
  $key,
  $method,
  $headers,
  $uri_params,
);

my $object = $bucket->get_key(
  { key              => $key,
    method           => 'GET',
    headers          => \%headers,
    uri_params       => \%uri_params,
    verify_checksums => 1,
  }
);

Retrieves object data and metadata.

The positional and hash-reference forms are both supported.

headers

Optional hash reference containing request headers.

key

Required object key.

method

Optional HTTP method.

The default is GET.

HEAD may be used to retrieve metadata without the object body.

uri_params

Optional hash reference containing GetObject URI parameters.

Examples include:

partNumber
response-cache-control
response-content-disposition
response-content-encoding
response-content-language
response-content-type
response-expires
versionId
verify_checksums

Optional per-request override for checksum verification.

When omitted, the value of "verify_checksums" in Amazon::S3 is used.

This option is available only in the hash-reference form.

When checksum verification is enabled for a GET, the request asks S3 to return checksum metadata.

On success, returns a hash reference containing:

checksum_type

The value of x-amz-checksum-type, when returned by S3.

checksums

Hash reference containing checksum values returned in x-amz-checksum-* response headers.

Keys are lowercase algorithm names.

content_length

Object content length.

content_range

Content-Range header value, when present.

content_type

Object content type.

etag

ETag returned by S3.

The ETag must not be assumed to be a checksum of the complete object.

last_modified

Last-Modified response header.

value

Object content.

x-amz-meta-*

User metadata headers are also added to the returned hash using lowercase header names.

Returns undef when the object does not exist.

Other request errors throw an exception.

Supported FULL_OBJECT checksums are verified when verification is enabled. Partial-content responses and COMPOSITE checksums are not verified.

A checksum mismatch throws an exception.

See "CHECKSUMS" in Amazon::S3 and https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html.

get_key_filename

my $object = $bucket->get_key_filename(
  $key,
  $method,
  $filename,
  $headers,
  $uri_params,
);

my $object = $bucket->get_key_filename(
  { key              => $key,
    filename         => $filename,
    method           => 'GET',
    headers          => \%headers,
    uri_params       => \%uri_params,
    verify_checksums => 1,
  }
);

Retrieves an object and writes the body to a local file.

The accepted parameters and return metadata are the same as for get_key(), with the addition of:

filename

Destination filename.

When omitted, the object key is used as the filename.

Checksum verification, when enabled, is performed against the downloaded file.

Returns undef when the object does not exist.

Other request errors or checksum mismatches throw an exception.

See get_key().

get_key_v2

my $object = $bucket->get_key_v2(
  $key,
  $method,
  $headers,
);

Compatibility wrapper around the object retrieval implementation.

It accepts an object key, optional HTTP method, and optional headers hash reference.

For new code, use get_key().

head_key

my $metadata = $bucket->head_key($key);

Retrieves object metadata using an HTTP HEAD request.

This is equivalent to:

$bucket->get_key($key, 'HEAD');

Returns the same metadata structure as get_key().

The value entry is empty because no object body is returned.

Returns undef when the object does not exist.

Other request errors throw an exception.

LISTING METHODS

list

my $response = $bucket->list;

my $response = $bucket->list(%parameters);

Lists one page of objects in this bucket using the original S3 ListObjects API.

The bucket name is supplied automatically. %parameters may contain the same listing options accepted by L[Amazon::S3/list_bucket](Amazon::S3/list_bucket), including:

delimiter

Optional delimiter used to group keys into common prefixes.

headers

Optional hash reference containing additional HTTP request headers.

marker

Optional key from which listing should continue.

max-keys

Optional maximum number of objects returned by this request.

prefix

Optional prefix used to restrict the returned keys.

This method retrieves a single page of results. If S3 indicates that additional objects are available, the returned structure contains the marker needed to request the next page.

Use list_all() when all matching objects should be retrieved automatically.

Returns the same normalized result as L[Amazon::S3/list_bucket](Amazon::S3/list_bucket).

See "LISTING OBJECTS" in Amazon::S3.

list_v2

my $response = $bucket->list_v2;

my $response = $bucket->list_v2(%parameters);

Lists one page of objects in this bucket using the S3 ListObjectsV2 API.

The bucket name is supplied automatically. %parameters may contain the same listing options accepted by L[Amazon::S3/list_bucket_v2](Amazon::S3/list_bucket_v2), including:

continuation-token

Optional continuation token returned by a previous ListObjectsV2 request.

delimiter

Optional delimiter used to group keys into common prefixes.

encoding-type

Optional encoding type requested for returned keys.

fetch-owner

Optional value controlling whether owner information is returned.

headers

Optional hash reference containing additional HTTP request headers.

marker

Compatibility alias for continuation-token.

max-keys

Optional maximum number of objects returned by this request.

prefix

Optional prefix used to restrict the returned keys.

start-after

Optional key after which S3 should begin the listing.

This method retrieves a single page of results. C[Amazon::S3](Amazon::S3) presents the ListObjectsV2 continuation value through its normalized marker interface so that callers can paginate consistently with the original listing API.

Use list_all_v2() when all matching objects should be retrieved automatically.

Returns the same normalized result as L[Amazon::S3/list_bucket_v2](Amazon::S3/list_bucket_v2).

See "LISTING OBJECTS" in Amazon::S3.

list_all

my $response = $bucket->list_all;

my $response = $bucket->list_all(%parameters);

Lists all matching objects in this bucket using the original S3 ListObjects API.

The bucket name is supplied automatically. %parameters may contain the same listing options accepted by L[Amazon::S3/list_bucket_all](Amazon::S3/list_bucket_all), including:

delimiter

Optional delimiter used to group keys into common prefixes.

headers

Optional hash reference containing additional HTTP request headers.

marker

Optional key from which listing should continue.

max-keys

Optional maximum number of objects requested from S3 per request.

Because this method follows pagination automatically, max-keys limits the size of each request rather than the total number of objects returned.

prefix

Optional prefix used to restrict the returned keys.

Pagination is followed automatically until all matching objects have been retrieved.

Returns the same normalized result as L[Amazon::S3/list_bucket_all](Amazon::S3/list_bucket_all).

See "LISTING OBJECTS" in Amazon::S3.

list_all_v2

my $response = $bucket->list_all_v2;

my $response = $bucket->list_all_v2(%parameters);

Lists all matching objects in this bucket using the S3 ListObjectsV2 API.

The bucket name is supplied automatically. %parameters may contain the same listing options accepted by L[Amazon::S3/list_bucket_all_v2](Amazon::S3/list_bucket_all_v2), including:

continuation-token

Optional continuation token returned by S3.

delimiter

Optional delimiter used to group keys into common prefixes.

encoding-type

Optional encoding type requested for returned keys.

fetch-owner

Optional value controlling whether owner information is returned.

headers

Optional hash reference containing additional HTTP request headers.

marker

Compatibility alias for continuation-token.

max-keys

Optional maximum number of objects requested from S3 per request.

Because this method follows pagination automatically, max-keys limits the size of each request rather than the total number of objects returned.

prefix

Optional prefix used to restrict the returned keys.

start-after

Optional key after which S3 should begin the listing.

Pagination is followed automatically until all matching objects have been retrieved.

Returns the same normalized result as L[Amazon::S3/list_bucket_all_v2](Amazon::S3/list_bucket_all_v2).

See "LISTING OBJECTS" in Amazon::S3.

ACCESS CONTROL AND BUCKET METADATA

delete_bucket

my $ok = $bucket->delete_bucket;

Deletes this bucket.

This is equivalent to:

$bucket->account->delete_bucket($bucket);

The bucket must be empty before S3 will delete it.

Returns the value returned by "delete_bucket" in Amazon::S3.

get_acl

my $xml = $bucket->get_acl;

my $xml = $bucket->get_acl($key);

my $xml = $bucket->get_acl($key, \%headers);

Retrieves the access control list for the bucket or an object.

headers

Optional request headers.

key

Optional object key.

When omitted, retrieves the ACL for the bucket.

On success, returns the ACL XML document as a scalar.

Returns undef when S3 returns 404 Not Found.

Other request errors throw an exception.

get_location_constraint

my $location = $bucket->get_location_constraint;

my $location = $bucket->get_location_constraint(
  { bucket  => $bucket_name,
    headers => \%headers,
    region  => $region,
  }
);

Returns the S3 location constraint for a bucket.

The optional parameters are:

bucket

Bucket name.

The default is the current bucket.

headers

Optional request headers.

region

Region used to sign the request.

The default is the bucket region.

For us-east-1, S3 may return no location constraint.

Callers that require a normalized region name can use "get_bucket_location" in Amazon::S3.

See https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketLocation.html.

set_acl

my $ok = $bucket->set_acl(
  { acl_short => 'private',
    key       => $key,
    headers   => \%headers,
  }
);

my $ok = $bucket->set_acl(
  { acl_xml => $xml,
    key     => $key,
    headers => \%headers,
  }
);

Sets the access control list for the bucket or an object.

Exactly one of acl_short or acl_xml is required.

acl_short

Canned ACL value sent using x-amz-acl.

acl_xml

ACL XML document.

headers

Request headers.

key

Optional object key.

When omitted, sets the ACL for the bucket.

Returns a true value on success.

Invalid ACL configuration throws an exception.

ERROR AND RESPONSE ACCESSORS

err

Returns the most recent error code from the associated Amazon::S3 object.

See "ERROR HANDLING" in Amazon::S3.

error

Returns the most recent parsed structured error from the associated Amazon::S3 object.

See "ERROR HANDLING" in Amazon::S3.

errstr

Returns the most recent human-readable error message from the associated Amazon::S3 object.

See "ERROR HANDLING" in Amazon::S3.

last_response

Returns the most recent HTTP::Response from the associated Amazon::S3 object.

See "ERROR HANDLING" in Amazon::S3.

MULTIPART UPLOAD METHODS

For normal multipart uploads, upload_multipart_object() is the preferred interface.

The remaining multipart methods expose the lower-level multipart lifecycle for callers that need to manage initiation, individual parts, completion, or abort behavior themselves.

See "MULTIPART UPLOADS" in Amazon::S3.

abort_multipart_upload

my $ok = $bucket->abort_multipart_upload(
  $key,
  $upload_id,
);

Aborts an existing multipart upload.

key

Required object key.

upload_id

Required multipart upload ID.

Returns a true value on success.

Request errors throw an exception.

complete_multipart_upload

my $ok = $bucket->complete_multipart_upload(
  $key,
  $upload_id,
  \%parts,
);

my $ok = $bucket->complete_multipart_upload(
  $key,
  $upload_id,
  \%parts,
  $algorithm,
);

Completes an existing multipart upload.

algorithm

Optional checksum algorithm associated with the multipart upload.

key

Required object key.

parts

Required hash reference keyed by part number.

For the historical interface, each value is the ETag returned for the part:

{
  1 => $etag_1,
  2 => $etag_2,
}

When checksum information is needed, each value may instead be a hash reference:

{
  1 => {
    etag     => $etag_1,
    checksum => $checksum_1,
  },
}
upload_id

Required multipart upload ID.

Returns a true value on success.

Invalid arguments or request errors throw an exception.

initiate_multipart_upload

my $upload_id = $bucket->initiate_multipart_upload(
  $key,
  \%headers,
);

my ( $upload_id, $algorithm )
  = $bucket->initiate_multipart_upload(
    $key,
    \%headers,
  );

Initiates a multipart upload.

headers

Optional request headers.

When x-amz-checksum-algorithm is supplied, the selected algorithm is carried through the multipart workflow.

For CRC64NVME, CRC32, and CRC32C, x-amz-checksum-type is set to FULL_OBJECT.

key

Required object key.

In scalar context, returns the upload ID assigned by S3.

In list context, returns the upload ID and the lowercase checksum algorithm selected for the upload.

Invalid arguments, unsupported explicitly requested checksum algorithms, or request errors throw an exception.

list_multipart_upload_parts

my $xml = $bucket->list_multipart_upload_parts(
  $key,
  $upload_id,
  \%headers,
);

Lists parts already uploaded for an existing multipart upload.

headers

Optional request headers.

key

Required object key.

upload_id

Required multipart upload ID.

Returns the XML response body returned by S3.

Request errors throw an exception.

list_multipart_uploads

my $xml = $bucket->list_multipart_uploads;

my $xml = $bucket->list_multipart_uploads(\%headers);

Lists active multipart uploads for this bucket.

The optional argument is a request-headers hash reference.

Returns the XML response body returned by S3.

Request errors throw an exception.

upload_multipart_object

my $parts = $bucket->upload_multipart_object(
  { key  => $key,
    data => $data,
  }
);

Uploads an object using the multipart upload API and manages the multipart lifecycle.

The method accepts a hash reference or a list of key/value pairs.

Exactly one usable data source must be supplied using data, callback, or fh.

The following parameters are supported:

abort_on_error

When true, attempt to abort the multipart upload if an error occurs.

The default is true.

callback

Coderef used to provide object data.

The callback receives no arguments and should return:

( \$buffer, $length )

Returning no buffer ends the upload.

chunk_size

Requested multipart chunk size.

For file-handle uploads, values smaller than the S3 minimum multipart part size are raised to that minimum.

data

Scalar or scalar reference containing object data.

When neither callback nor fh is supplied, the data is read through an in-memory file handle.

fh

Open file handle containing the object data.

The file must be at least the minimum multipart upload size.

headers

Optional headers supplied when initiating the multipart upload.

When no x-amz-checksum-algorithm header is supplied, the checksum algorithm configured on the associated Amazon::S3 object is used.

key

Required destination object key.

On success, returns a hash reference mapping part numbers to the ETags returned by S3:

{
  1 => $etag_1,
  2 => $etag_2,
}

The method automatically initiates the upload, uploads each part, and completes the upload.

When abort_on_error is true, an error during the managed workflow causes an abort attempt and the returned part hash is empty.

See "MULTIPART UPLOADS" in Amazon::S3 and "CHECKSUMS" in Amazon::S3.

upload_part_of_multipart_upload

my $etag = $bucket->upload_part_of_multipart_upload(
  $key,
  $upload_id,
  $part_number,
  $data,
  $length,
  $algorithm,
);

my ( $etag, $checksum )
  = $bucket->upload_part_of_multipart_upload(
    { key       => $key,
      id        => $upload_id,
      part      => $part_number,
      data      => $data,
      length    => $length,
      algorithm => $algorithm,
    }
  );

Uploads one part of an existing multipart upload.

The method accepts positional arguments, a hash reference, or an array reference.

algorithm

Optional checksum algorithm associated with the multipart upload.

When a local implementation is available, the checksum is calculated and sent with the UploadPart request.

data

Required part data.

A scalar or scalar reference may be supplied.

id

Required multipart upload ID.

key

Required object key.

length

Optional data length.

When omitted, the length is calculated from data.

part

Required part number.

In scalar context, returns the ETag returned by S3.

In list context, returns the ETag and the Base64-encoded checksum calculated for the part, when one was calculated.

Invalid arguments or request errors throw an exception.

SEE ALSO

Amazon::S3

Amazon::S3::BucketV2

AUTHOR

Please see Amazon::S3 for author, copyright, and license information.

CONTRIBUTORS

Rob Lauer

Jojess Fournier

Tim Mullin

Todd Rinaldo

luiserd97