NAME

Amazon::API - A generic base class for AWS Services

SYNOPSIS

use Amazon::API;

my $service = Amazon::API->new( service => 'events', api => 'AWSEvents');
my $rules = $service->invoke_api('ListRules');

DESCRIPTION

Generic class for constructing AWS API interfaces. Typically used as a parent class, but can be used directly. This package can also generate stubs for Amazon APIs using the Botocore project metadata. (See "BOTOCORE SUPPORT").

The typical use of this API is through the classes you build with the included tool (amzn-api). The tool leverages the Botocore project's metadata to build classes that are specific to each API (and are documented via the separate Amazon::API::Help tool). Using Amazon::API directly may not work in all circumstances unless you are very familiar with the API you are calling. If you decide to take the Luddite approaches, read the documentation carefully before using Amazon::API.

See "IMPLEMENTATION NOTES" for using Amazon::API directly to call AWS services.
See Amazon::CloudWatchEvents for an example of how to use this module as a parent class.
See amzn-api -h for information regarding how to automatically create Perl classes for AWS services using Botocore metadata.

Try It Using Docker

If you just want to try Amazon::API without installing anything locally, a pre-built Docker image is available from DockerHub. Assuming you have AWS credentials that can be discovered from your AWS CLI configuration in $HOME/.aws:

docker run --rm \
  -v "$HOME/.aws:/root/.aws:ro" \
  -e AWS_PROFILE=my-profile \
  rlauer/amazon-api:latest \
  perl -MAmazon::API::STS -MData::Dumper \
       -e 'print Dumper(Amazon::API::STS->new->GetCallerIdentity)'

BACKGROUND AND MOTIVATION

A comprehensive Perl interface to AWS services similar to the Botocore library for Python has been a long time in coming. The Paws project has been creating an always up-to-date AWS interface with community support. If you are looking for an extensible method of installing and invoking a subset of services you might want to consider Amazon::API.

Think of this class as a DIY kit for installing only the APIs and methods you need for your AWS project. Using the included amzn-api utility you can also roll your own complete Amazon API classes that include support for serializing requests and responses based on metadata provided by the Botocore project. Detailed service and API documentation is provided separately by Amazon::API::Help. See "BOTOCORE SUPPORT" for more details.

NOTE: The original Amazon::API was written in 2017 as a very lightweight way to call a handful of APIs. The evolution of the module was based on discovering, without much documentation or help, the nature of Amazon APIs. In retrospect, even back then, it would have been easier to consult the Botocore project and decipher how that project managed to create a library from the metadata. Fast forward to 2022 and Amazon::API began using the Botocore metadata in order to, in most cases, correctly call any AWS service. The Amazon::API module can still be used without the assistance of Botocore metadata, but it works a heckuva lot better with it.

You can use Amazon::API in several different ways:

Take the Luddite approach

my $queues = Amazon::API->new(
 {
  service     => 'sqs',
  http_method => 'GET'
 })->invoke_api('ListQueues');

Build your own API classes with just what you need

package Amazon::API::SQS;

use strict;
use warnings;

use parent qw( Amazon::API );

our @API_METHODS = qw(
  ListQueues
  PurgeQueue
  ReceiveMessage
  SendMessage
);

sub new {
  my ( $class, @options ) = @_;
  $class = ref($class) || $class;

  my %options = ref( $options[0] ) ? %{ $options[0] } : @options;

  return $class->SUPER::new(
    { service       => 'sqs',
      http_method   => 'GET',
      api_methods   => \@API_METHODS,
      decode_always => 1,
      %options
    }
  );
}

1;

use Amazon::API::SQS;
use Data::Dumper;

my $sqs = Amazon::API::SQS->new;

print {*STDERR} Dumper($sqs->ListQueues);

Amazon::API DarkPAN

Many of the Amazon API classes have already been pre-built and are available in the Amazon::API DarkPAN:

https://cpan.openbedrock.net/orepan2

Each module that is generated and installed in the DarkPAN is signed. You can find instructions on how to verify the signatures by visiting https://cpan.openbedrock.net/signature/index.html.

Modules not currently available can be requested on the same site.

Build from the cloned https://github.com/rlauer6/Amazon-API.git project

You can build a complete CPAN distribution for an API class by cloning the project from GitHub. Follow the instructions below.

1. If you need to...install these dependencies

ca-certificates
curl
gcc
git
libexpat-dev
libssl-dev
libzip-dev
make
perl

2. Install cpm

curl -fsSL https://raw.githubusercontent.com/skaji/cpm/main/cpm | perl - install -g App::cpm

3. Clone the Amazon-API repository

git clone --depth 1 --branch master https://github.com/rlauer6/Amazon-API.git

4. Install some build dependencies

cd Amazon-API
cat build-requires | cpm install -

5. Clone the Botocore repository

git clone --depth 1 --branch master https://github.com/boto/botocore.git

6. Export some environment variables

export PERL5LIB=/Amazon-API/local/lib/perl5
export PATH=/Amazon-API/local/bin:$PATH
export CPAN_MAKER_CONFIG=$(pwd)/.cpan-makerrc

7. Build the project

make

8. Build a service

make cpan-dist SERVICE=sts

9. Test the service

cpm install Amazon-STS*.tar.gz

AWS_PROFILE=sandbox perl -I local/lib/perl5 -MData::Dumper -MAmazon::API::STS \
  -e 'print Dumper(Amazon::API::STS->new->GetCallerIdentity);'

Use the Botocore metadata to build classes locally

In order to use Botocore metadata you must clone the Botocore repository yourself and point the utility at the repository directory.

This method is recommended solely for experimenting with the Amazon::API.

Install Amazon::API
```
cpanm Amazon::API
```

Clone the Botocore project from GitHub:

git clone --depth=1 --branch=master https://github.com/boto/botocore.git

Generate a stub class for the API:

amzn-api -b botocore -s sts -o lib create-stub

Test

Make sure your credentials are available before running your test. Credentials in your environment, instance profile or your credentials file will be found automatically by Amazon::Credentials.
```
AWS_PROFILE=sandbox perl -I lib -MAmazon::API::STS -MData::Dumper \
  -e 'Amazon::API::STS->new->GetCallerIdentity;'
```

THE APPROACH

Essentially, most AWS APIs are RESTful services that adhere to a common protocol, but differences in services make a single solution difficult. All services more or less adhere to this framework:

1. Set HTTP headers (or query string) to indicate the API and method to be invoked
2. Set credentials in the header
3. Set API specific headers
4. Sign the request and set the signature in the header
5. Optionally send a payload of parameters for the method being invoked

Specific details of the more recent AWS services are well documented, however early services were usually implemented as simple HTTP services that accepted a query string. This module attempts to account for most of the nuances involved in invoking AWS services and provide a fairly generic way of invoking these APIs in the most lightweight way possible.

Using Amazon::API as a generic, lightweight module, naturally does not provide nuanced support for individual AWS services. To use this class in that manner for invoking the AWS APIs, you need to be very familiar with the specific API requirements and responses and be willing to invest time reading the documentation on Amazon's website. The payoff is that you can probably use this class to call any AWS API without installing a large number of dependencies.

If you don't mind a few extra dependencies and overhead, you should generate the stub APIs using the amzn-api utility or download the classes from the Amazon::API website. The stubs produced by the utility will serialize and deserialize requests and responses correctly by using the Botocore metadata. The generated classes use the Botocore service models together with Amazon::API's protocol implementation to serialize requests and deserialize responses for the supported Botocore protocols.

Invoking some of the APIs can potentially be as easy as:

Amazon::API->new(
  service     => 'sqs',
  http_method => 'GET'
}
)->invoke_api('ListQueues');

...again using this approach requires that you understand the requirements for each API.

BOTOCORE SUPPORT

Using Botocore metadata and the utilities in this project, you can create Perl classes that simplify calling AWS services. After creating service classes from the Botocore metadata calling AWS APIs will look something like this:

use Amazon::API::SQS;

my $sqs = Amazon::API::SQS->new;
my $rsp = $sqs->ListQueues();

The Amazon::API::Role::Botocore module augments Amazon::API by using Botocore metadata for determining how to call individual services and serialize parameters passed to its API methods. amzn-api is provided to help you generate Perl classes for all AWS services using the Botocore metadata.

Perl classes that represent AWS data structures (aka shapes) that are passed to or returned from services are generated dynamically from the Botocore metadata. These classes use the Botocore shape and protocol metadata to serialize requests and deserialize responses for each method.

Service classes are subclassed from Amazon::API so their new() constructor takes the same arguments as Amazon::API::new().

my $credentials = Amazon::Credentials->new();

my $sqs = Amazon::API::SQS->new( credentials => $credentials );

Response Serialization

Botocore responses are decoded and serialized into Perl data structures using the service shape and protocol metadata.

Starting with version 1.4.5, an exception encountered while decoding or serializing a response causes the raw response to be returned to the caller rather than making an otherwise successful API response unavailable.

If you need serialization failures to be fatal -- for example while debugging or developing protocol support -- set raise_serialization_errors to a true value. The default is false.

Protocol Compatibility Testing

The request and response serialization code is tested against the protocol test fixtures distributed with Botocore. These tests exercise the protocol behavior independently of any particular AWS service or live AWS endpoint.

The test suite covers all five Botocore protocol families currently supported by Amazon::API:

query
ec2
json
rest-json
rest-xml

The protocol tests exercise request serialization, successful response deserialization, and AWS error responses. This includes behavior such as URI and query-string parameters, request and response headers, payloads, XML namespaces and wrappers, timestamps, blobs, empty structures, and protocol-specific result wrappers.

At the time this documentation was updated, the Botocore protocol corpus comprised over 600 request and successful-response tests and an additional 30+ error-response tests, all of which pass.

Passing the Botocore protocol corpus does not mean that every operation of every AWS service has been exercised against AWS. Botocore metadata describes the service models, but some protocol behavior is implemented by Botocore itself rather than expressed completely by that metadata. There may therefore still be service-specific exceptions or behavior not represented by the protocol fixtures.

In particular, values returned by Amazon::API are intended to match the values modeled by Botocore, not necessarily the literal values seen on the HTTP wire or printed by the AWS CLI. Timestamps, booleans, blobs, empty collections, wrapper elements, and header-derived values are examples where normalization may occur.

Request Validation

Botocore shape definitions carry more than structure -- many members declare value constraints: an enum of permitted strings, a min and max (length for strings, magnitude for numbers, element count for lists), and a regular-expression pattern. Starting with version 2.4.6, a Botocore request validates the values you supply against these constraints before the request is serialized and sent, so a bad value is caught locally -- with the offending parameter named -- rather than surfacing as an opaque InvalidParameterValue from AWS several layers away.

Validation is controlled by the package variable $Amazon::API::VALIDATE_MODE, which takes one of three values:

strict (the default)

An invalid value throws an exception naming the parameter, the value, and -- for an enum -- the list of permitted values.
warn

An invalid value emits a warning and the request proceeds unchanged.
off

Constraints are not checked.

Because the constraint data is only as current as the Botocore metadata this client was generated from, strict is deliberately safe against staleness: the exception it raises tells you how to relax the mode. If AWS has added an enum value (or loosened a bound) since your metadata was built, strict will reject a value that is in fact now valid -- so the message points you at warn and off as the escape hatch. This is the same posture the scanner takes toward unrecognized members: act on the metadata you have, but never let a point-in-time snapshot become a hard wall.

The mode is a package global, so set it once to change the default for every request:

$Amazon::API::VALIDATE_MODE = 'warn';

or localize it to relax validation for a single call while leaving the global default intact:

{
  local $Amazon::API::VALIDATE_MODE = 'off';
  $ec2->RunInstances( \%params );
}

Validation applies only to Botocore requests -- requests built from generated service classes, where the shape constraints are available. A request you construct by hand (the Luddite approach) is sent as-is.

ERRORS

When an error is returned from an API request, an exception class (Amazon::API::Error) will be raised if raise_error has been set to a true value (the default). If you set print_error to true AND raise_error is false, then errors will be printed to STDERR.

See Amazon::API::Error for more details.

METHODS AND SUBROUTINES

Reminder: You can mostly ignore this part of the documentation when you are leveraging Botocore to generate your API classes.

new

new(options)

All options are described below. options can be a list of key/values or hash reference.

action

The API method. Normally, you would not set action when you construct your object. It is set when you call the invoke_api method or automatically set when you call one of the API stubs created for you.

Example: 'PutEvents'

api

The name of the AWS service. See "IMPLEMENTATION NOTES" for a detailed explanation of when to set this value.

Example: 'AWSEvents'

api_methods

A reference to an array of method names for the API. The new constructor will automatically create methods for each of the method names listed in the array.

The methods that are created for you are nothing more than stubs that call invoke_api. The stub is a convenience for calling the invoke_api method as shown below.

my $api = Amazon::CloudWatch->new;

$api->PutEvents($events);

...is equivalent to:

$api->invoke_api->('PutEvents', $events);

Consult the Amazon API documentation for the service to determine what parameters each action requires.

aws_access_key_id

Your AWS access key. Both the access key and secret access key are required if either is passed. If no credentials are passed, an attempt will be made to find credentials using Amazon::Credentials. Note that you may need to pass token as well if you are using temporary credentials.

aws_secret_access_key

Your AWS secret access key.

content_type

Default content for parameters passed to the invoke_api() method. If you do not provide this value, a default content type will be selected based on the service's protocol.

query     => application/x-www-form-urlencoded
rest-json => application/x-amz-json-1.1
json      => application/json
rest-xml  => application/xml

credentials (optional)

Accessing AWS services requires credentials with sufficient privileges to make programmatic calls to the APIs that support a service. This module supports three ways that you can provide those credentials.

1. Pass the credentials directly.

Pass the values for the credentials (aws_access_key_id, aws_secaret_access_key, token) when you call the new method. A session token is typically required when you have assumed a role, you are using the EC2's instance role or a container's role.

2. Pass a class that will provide the credential keys.

Pass a reference to a class that has getters for the credential keys. The class should supply getters for all three credential keys.

Pass the reference to the class as credentials in the constructor as shown here:

my $api = Amazon::API->new(credentials => $credentials_class, ... );

3. Use the default Amazon::Credentials class.

If you do not explicitly pass credentials or do not pass a class that will supply credentials, the module will use the Amazon::Credentials class that attempts to find credentials in the environment, your credentials file(s), or the container or instance role. See Amazon::Credentials for more details.

NOTE: The latter method of obtaining credentials is probably the easiest to use and provides the most succinct and secure way of obtaining credentials.

decode_always

Set decode_always to a true value to return Perl objects from API method calls. Typically, API calls return either XML or JSON encoded responses which will be automatically deserialized. Setting decode_always to false returns the raw content instead.

default: true

error

The most recent result of an API call. undef indicates no error was encountered the last time invoke_api was called.

http_method

Sets the HTTP method used to invoke the API. Consult the AWS documentation for each service to determine the method utilized. Most of the more recent services utilize the POST method, however older services like SQS or S3 utilize GET or a combination of methods depending on the specific method being invoked.

default: POST

last_action

The last method call invoked.

log_level

Sets the diagnostic logging level. The default is info, which suppresses the debug and trace diagnostic messages emitted by Amazon::API.

Set this to debug for operational diagnostics or trace for low-level protocol and serialization diagnostics. Trace-level logging may contain sensitive application data; see "Trace logging and sensitive data".

The logging level may also be changed after construction with set_log_level.

default: info

no_logger

Set no_logger to a true value to disable the internal logger. This is useful when Amazon::API is used inside a logging appender such as Log::Log4perl::Appender::CloudWatch where internal logging would cause re-entrant behavior.

default: false

no_passkey_warning

Prevent passkey warning. This is an option to Amazon::Credentials.

print_error

Setting this value to a true value will print a detailed error message containing the error code and any messages returned by the API to STDERR when an error occurs. Errors will NOT be printed if raise_error is also true.

default: true

protocol

One of 'http' or 'https'. Some Amazon services do not support https (yet).

default: https

raise_error

Setting this value to a true value will raise an exception when errors occur. If you set this value to false you can inspect the error attribute to determine the success or failure of the last method call.

$api->invoke_api('ListQueues');

if ( $api->get_error ) {
  ...
}

default: true

region

The AWS region. Pass an empty string if the service is a global service that does not require or want a region.

default: $ENV{'AWS_REGION'}, $ENV{'AWS_DEFAULT_REGION'}, 'us-east-1'

response

The HTTP response from the last API call.

service

The AWS service name. Example: sqs. This value is used as a prefix when constructing the the service URL (if not url attribute is set).

service_url_base

Deprecated, use service

token

Session token for assumed roles.

url

The service url. Example: https://events.us-east-1.amazonaws.com

Typically this will be constructed for you based on the region and the service being invoked. However, you may want to set this manually if for example you are using a service like LocalStack that mocks AWS API calls.

my $api = Amazon::API->new(service => 's3', url => 'http://localhost:4566/');

user_agent

Set user_agent if you want to use a different user agent. The user agent must implement a single request method that accepts an HTTP::Request object and returns a response object providing the following methods:

is_success - returns true if the request succeeded
code - returns the HTTP status code
content - returns the response body
content_type - returns the content type of the response

By default, Amazon::API uses Amazon::API::HTTP::UserAgent, a lightweight wrapper around HTTP::Tiny that satisfies this interface.

default: Amazon::API::HTTP::UserAgent

version

Sets the API version. Some APIs require a version. Consult the documentation for individual services.

invoke_api

invoke_api(action, [parameters], [content-type], [headers]);

or using named parameters...

invoke_api({ action => args, ... } )

Invokes the API with the provided parameters.

action

API name.

parameters

Parameters to send to the API. parameters can be a scalar, a hash reference or an array reference. See the discussion below regarding content-type and how invoke_api() formats parameters before sending them as a payload to the API.

You can use the param_n() method to format query string arguments that are required to be in the param.n notation. This is about the best documentation I have seen for that format. From the AWS documentation...

Some actions take lists of parameters. These lists are specified using the param.n notation. Values of n are integers starting from 1. For example, a parameter list with two elements looks like this:

&AttributeName.1=first

&AttributeName.2=second

An example of using this notation is to set queue attributes when creating an SQS queue.

my $attributes = { Attributes => [ { Name => 'VisibilityTimeout', Value => '100' } ] };
my @sqs_attributes= Amazon::API::param_n($attributes);

eval {
  $sqs->CreateQueue([ 'QueueName=foo', @sqs_attributes ]);
};

See "param_n" for more details.

content-type

If you pass the content-type parameter, it is assumed that the parameters are the actual payload to be sent in the request (unless the parameter is a reference).

The parameters will be converted to a JSON string if the parameters value is a hash reference. If the parameters value is an array reference it will be converted to a query string (Name=Value&...).

To pass a query string, you should send an array of key/value pairs, or an array of scalars of the form Name=Value.

[ { Action => 'DescribeInstances' } ]
[ 'Action=DescribeInstances' ]

headers

Array reference of key/value pairs representing additional headers to send with the request.

Note: This method consults the raise_error and print_error options to determine how errors are handled.

decode_response

Boolean that indicates whether or not to deserialize the most recent response from an invoked API based on the Content-Type header returned. If there is no Content-Type header, then the method will try to decode it first as a JSON string and then as an XML string. If both of those fail, the raw content is returned.

You can enable or disable deserializing responses globally by setting the decode_always attribute when you call the new constructor.

default: true

By default, Amazon::API will retrieve all results for Botocore based API calls that require pagination. To turn this behavior off, set use_paginator to a false value when you instantiate the API service.

my $ec2 = Amazon::API->new(use_paginator => 0);

print_error

Prints a formatted version of the last error encountered to STDERR.

submit

submit(options)

This method is used internally by invoke_api and normally should not be called by your applications.

options is a reference to a hash of options:

content

Payload to send.

content_type

Content types we have seen used to send values to AWS APIs:

application/json
application/x-amz-json-1.0
application/x-amz-json-1.1
application/x-www-form-urlencoded

Check the documentation for the individual APIs for the correct content type.

headers

Array reference of key/value pairs that represent additional headers to send with the request.

generate_xml

generate_xml(object)

Generates XML from a Perl object (uses XML::Twig).

EXPORTED METHODS

get_api_service

get_api_service(api, options)

Convenience routine that will return an API instance.

my $sqs = get_api_service 'sqs';

Equivalent to:

require Amazon::API::SQS;

my $sqs = Amazon::API::SQS->new(%options);

api: The service name. Example: route53, sqs, sns
options: list of key/value pairs passed to the new constructor as options

create_urlencoded_content

create_urlencoded_content(parameters, action, version)

Returns a URL encoded query string. parameters can be any of SCALAR, ARRAY, or HASH. See below.

parameters

SCALAR

Query string to encode (x=y&w=z..)

ARRAY

Can be one of:

Array of hashes where the keys are the query string variable and the value is the value of that variable
Array of strings of the form "x=y"
An array of key/value pairs - qw( x y w z )

HASH

Key/value pairs. If value is an array it is assumed to be a list of hashes

action

The method being called. For some query type APIs an Action query variable is required.

version

The WSDL version for the API. Some query type APIs require a Version query variable.

param_n

param_n(parameters)

Format parameters in the "param.n" notation.

parameters should be a hash or array reference.

A good example of a service that uses this notation is the SendMessageBatch SQS API call.

The sample request can be found here:

SendMessageBatch

https://sqs.us-east-2.amazonaws.com/123456789012/MyQueue/
?Action=SendMessageBatch
&SendMessageBatchRequestEntry.1.Id=test_msg_001
&SendMessageBatchRequestEntry.1.MessageBody=test%20message%20body%201
&SendMessageBatchRequestEntry.2.Id=test_msg_002
&SendMessageBatchRequestEntry.2.MessageBody=test%20message%20body%202
&SendMessageBatchRequestEntry.2.DelaySeconds=60
&SendMessageBatchRequestEntry.2.MessageAttribute.1.Name=test_attribute_name_1
&SendMessageBatchRequestEntry.2.MessageAttribute.1.Value.StringValue=test_attribute_value_1
&SendMessageBatchRequestEntry.2.MessageAttribute.1.Value.DataType=String
&Expires=2020-05-05T22%3A52%3A43PST
&Version=2012-11-05
&AUTHPARAMS

To produce this message you would pass the Perl object below to param_n():

my $message = {
  SendMessageBatchRequestEntry => [
    { Id          => 'test_msg_001',
      MessageBody => 'test message body 1'
    },
    { Id               => 'test_msg_002',
      MessageBody      => 'test message body 2',
      DelaySeconds     => 60,
      MessageAttribute => [
        { Name  => 'test_attribute_name_1',
          Value =>
            { StringValue => 'test_attribute_value_1', DataType => 'String' }
        }
      ]
    }
  ]
};

service

See "get_api_service"

CAVEATS

If you are calling an API that does not expect parameters (or all of them are optional and you do not pass a parameter) the default is to pass an empty hash..

$cwe->ListRules();

would be equivalent to...

$cwe->ListRules({});

CAUTION! This may not be what the API expects! Always consult the AWS API for the service you are are calling.

Request-side value validation (enum, min/max, pattern) is enforced against the Botocore metadata this client was built from, which is a point-in-time snapshot. See "Request Validation" for how $Amazon::API::VALIDATE_MODE governs this and how to relax it if the metadata has fallen behind the live API.

IMPLEMENTATION NOTES

If you have taken the advice above and created classes using the amzn-api script you can probably ignore this section. This section is intended to help those trying to create the lightest weight possible AWS API class.

Just a reminder for those wanting to go lite...

Read the documentation AWS provides for the API. You need to understand the request parameters and headers required.
Examine the Botocore data for the API. That might help you understand that structures required for the calling parameters.
Use the aws CLI script in debug mode to see the actual payloads and how they are formatted.

Headers

X-Amz-Target

Most of the newer AWS APIs are invoked as HTTP POST operations and accept a header X-Amz-Target in lieu of the CGI parameter Action to specify the specific API action. Some APIs also want the version in the target, some don't. There is sparse documentation about the nuances of using the REST interface directly to call AWS APIs, but you kinda sorta figure it out by parsing the Botocore data for a particular API.

When invoking an API, the class uses the api value to indicate that the action should be set in the X-Amz-Target header. We also check to see if the version needs to be attached to the action value as required by some APIs.

if ( $self->get_api ) {
  if ( $self->get_version) {
    $self->set_target(sprintf('%s_%s.%s', $self->get_api, $self->get_version, $self->get_action));
  }
  else {
    $self->set_target(sprintf('%s.%s', $self->get_api, $self->get_action));
  }

  $request->header('X-Amz-Target', $self->get_target());
}

DynamoDB and KMS seem to be able to use this in lieu of query variables Action and Version, although again, there seems to be a lot of inconsistency (and sometimes flexibility) in the APIs. DynamoDB uses DynamoDB_YYYYMMDD.Action while KMS does not require the version that way and prefers TrentService.Action (with no version). There is no explanation in any of the documentations I have been able to find as to what "TrentService" might actually mean. Again, your best approach is to read Amazon's documentation and look at their sample requests for guidance. You can also look to the Botocore project for information regarding the service. Checkout the service-2.json file within the sub-directory botocore/botocore/data/{api-version}/{service-name} which contains details for each service.

In general, the AWS API ecosystem is very organic. Each service seems to have its own rules and protocol regarding what the content of the headers should be.

As noted, this generic API interface tries to make it possible to use one class Amazon::API as a sort of gateway to the APIs. The most generic interface is simply sending query variables and not much else in the header. Services like EC2 conform to that protocol and can be invoked with relatively little fanfare.

use Amazon::API;
use Data::Dumper;

print Dumper(
  Amazon::API->new(
    service => 'ec2',
    version => '2016-11-15'
  )->invoke_api('DescribeInstances')
);

Note that invoking the API in this fashion, version is required.

For more hints regarding how to call a particular service, you can use the AWS CLI with the --debug option. Invoke the service using the CLI and examine the payloads sent by the Botocore library.

Rolling a New API

Once again, your best bet is to use the amzn-api script to roll a class from the Botocore metadata, but if you really want to create your own class the lite way read on.

The Amazon::API class will stub out methods for the API if you pass an array of API method names. The stub is equivalent to:

sub some_api {
  my $self = shift;

  $self->invoke_api('SomeApi', @_);
}

Some will also be happy to know that the class will create an equivalent CamelCase version of the method.

As an example, here is a possible implementation of Amazon::CloudWatchEvents that implements one of the API calls.

package Amazon::CloudWatchEvents;

use strict;
use warnings;

use parent qw(Amazon::API);

sub new {
  my ($class, $options) = @_;

  my $self = $class->SUPER::new(
    { %{$options},
      api         => 'AWSEvents',
      service     => 'events',
      api_methods => [qw( ListRules )],
    }
  );

  return $self;
}

Then...

use Data::Dumper;

print Dumper(Amazon::CloudWatchEvents->new->ListRules({}));

Of course, creating a class for the service is optional. It may be desirable however to create higher level and more convenient methods that aid the developer in utilizing a particular API.

Overriding Methods

Because the class does some symbol table munging, you cannot easily override the methods in the usual way.

sub ListRules {
  my $self = shift;
  ...
  $self->SUPER::ListRules(@_)
}

Instead, you should re-implement the method as implemented by this class.

sub ListRules {
  my $self = shift;
  ...
  $self->invoke_api('ListRules', @_);
}

Content-Type

Yet another piece of evidence that suggests the organic nature of the Amazon API ecosystem is their use of different Content-Type headers. Some of the variations include:

application/json
application/x-amz-json-1.0
application/x-amz-json-1.1
application/x-www-form-urlencoded

Accordingly, the invoke_api() method can be passed the Content-Type or will try to make its best guess based on the service protocol or the type of object being passed as parameters. There is a hash of service names and service types that this module uses to determine the content type required by the service. If services are added that hash needs to be updated.

You can also set the default content type used for the calling service by passing the content_type option to the constructor.

$class->SUPER::new(
  content_type => 'application/x-amz-json-1.1',
  api          => 'AWSEvents',
  service      => 'events'
);

ADDITIONAL HINTS

Bad Request

If you send the wrong headers or payload you're liable to get a 400 Bad Request. You may also get other errors that can be misleading when you send incorrect parameters. When in doubt compare your requests to requests from the AWS CLI using the --debug option.

1. Set log_level to "trace" to see low-level request and response diagnostics from Amazon::API.

Trace output may contain sensitive application data; see "Trace logging and sensitive data".

2. Execute the AWS CLI with the --debug option and compare the request and response with that of your calls.
Payloads

Pay attention to the payloads that are required by each service. Do not assume that sending nothing when you have no parameters to pass is correct. For example, the ListSecrets API of SecretsManager requires at least an empty JSON object.
```
$api->invoke_api('ListSecrets', {});
```
Failure to send at least an empty JSON object will result in a 400 response.

VERSION

This documentation refers to version 2.8.1 of Amazon::API.

DIAGNOSTICS

To enable diagnostic output, set log_level to debug or trace.

Logging

Amazon::API uses Log::Log4perl for diagnostic logging. The default log level is INFO, which suppresses the DEBUG and TRACE diagnostic messages emitted by Amazon::API.

Set log_level to "debug" for operational diagnostic logging or "trace" for low-level protocol and serialization diagnostics.

Note however that TRACE level logging may expose potentially sensitive application level data.

Trace logging and sensitive data

Trace-level logging is intended for low-level diagnostic use and may include API request parameters, response payloads, serialized values, pagination tokens, and other application data.

Although Amazon::API sanitizes AWS credential and authentication material before logging, application payloads are not guaranteed to be redacted. Depending on the AWS service and operation, trace output may therefore contain sensitive values such as passwords, secrets, tokens, keys, or personally identifiable information.

Logging at the debug level is intended to be operationally safe; trace is explicitly diagnostic and may contain sensitive application data.

Trace logging should be enabled only when necessary and trace logs should be handled as potentially sensitive data.

BUGS AND LIMITATIONS

This module has not been tested on Windows OS. Please report any issues found by opening an issue here:

https://github.com/rlauer6/perl-Amazon-API/issues

FAQs

Why should I use this module instead of Paws?

Maybe you shouldn't. Paws is a community supported project and may be a better choice for most people. The programmers who created Paws are luminaries in the pantheon of Perl programming (alliteration intended). If you don't want to install all of the AWS services but only need to use a single service, Amazon::API may be the right choice for you. Paws may also have some edge cases for some of the seldom used services and you might find this module easier to use and debug.

Does it perform better than Paws?

It depends on how you define better. Amazon::API has fewer dependencies and will load faster than Paws. It will consume less memory as well. API calls to AWS services will be about the same, with an edge to Amazon::API which returns plain Perl objects instead of Moose classes. The overhead introduced by this module and Paws may be insignificant compared to the API performance.

Does this work for all AWS APIs?

For the Botocore protocols supported by Amazon::API, there is no reason to believe it should not work -- and there is substantial evidence that it does.

Amazon::API supports the query, ec2, json, rest-json, and rest-xml protocol families. Request serialization, successful response deserialization, and AWS error-response handling have been tested against Botocore's own protocol test fixtures.

At the time this documentation was updated, the suite included 600+ request and successful-response cases plus 30+ error-response cases. All of them pass.

This is an important distinction from testing a few representative AWS services. The Botocore protocol tests exercise the underlying behavior shared by hundreds of service operations: URI parameters, query parameters, JSON and XML bodies, headers, payloads, blobs, timestamps, namespaces, wrappers, empty structures, errors, and other protocol details.

In other words, the serialization layer is no longer considered experimental. The supported Botocore protocols have broad automated coverage against the same protocol fixtures used to define Botocore's behavior.

That does not prove that every operation of every AWS service has been called against a live AWS endpoint. Individual services can still contain unusual behavior or exceptions that are not completely described by the service metadata or represented in the protocol test corpus. If you encounter one, please report it.

The current exception is smithy-rpc-v2-cbor, which is not supported.

Some services also have unusual behavior that makes a generic generated client a poor fit. S3 is the most obvious example. For S3, use a purpose-built client such as Amazon::S3::Lite, Amazon::S3, or Net::Amazon::S3 instead.

This code does not use "Modern Perl". Why?

This code has evolved over the years from being a simple way to make RESTful calls to a few Amazon APIs, to an API that now incorporates the use of the Botocore metadata to support nearly every API service.

The code did not start out as a well designed attempt to interpret the Botocore data like the Paws project...and that has turned out to be a happy accident. By avoiding frameworks like Moose the resulting interface is about as lightweight as it can possibly be. The code does however embrace Perl Best Practices to create a more maintainable and familiar distribution.

How do I pass AWS credentials to the API?

There is a bit of magic here as Amazon::API will use Amazon::Credentials transparently if you do not explicitly pass the credentials object. I've taken great pains to try to make the aforementioned module somewhat useful and secure.

See Amazon::Credentials.

Can I use more than one set of credentials to invoke different APIs?

Yes. See Amazon::Credentials.

How stable is the interface?

As of version 2.1.0 the interface is quite stable. I now consider this project "production ready".

Why are you using XML::Simple when it clearly says "DO NOT"?

It's simple. And it seems easier to build than other modules that almost do the same thing.

Note: This project uses a modified vendored version of XML::Simple. Upstream bugs affecting the behavior required by Amazon::API are unlikely to be addressed, so the parser is maintained locally. In particular, the current upstream version ignores the NormaliseSpace option and unconditionally strips whitespace inside elements.

How do I stop the client from rejecting a parameter value?

If a request dies with a validation error naming an enum, length, or pattern constraint, the value you passed does not match this client's Botocore metadata. Most often that is a typo. If instead AWS has changed the API since your metadata was generated, relax validation with $Amazon::API::VALIDATE_MODE -- set it to warn to downgrade the error to a warning, or off to skip the check. See "Request Validation".

I tried to use this with XYZ service and it didn't work. What should I do?

Serialization is tested against the Botocore protocol test corpus, but that corpus cannot exercise every service-specific behavior. If a request still fails, enable debugging and compare the generated request with the AWS CLI using --debug.

If response deserialization is the problem, setting decode_always to false returns the raw response body and can help determine whether the problem is HTTP decoding, protocol handling, or shape serialization.

If you find a service behavior that is not represented correctly, please report it. The Botocore service metadata and protocol fixtures are usually the best starting point for determining whether the exception belongs in the generic serializer or requires service-specific handling.

Why aren't all AWS services included in the distribution or available on CPAN?

Because the service classes are generated artifacts rather than independently maintained Perl distributions.

There are hundreds of AWS services represented by Botocore. Generating and publishing every possible Amazon::API::* distribution whenever the Botocore metadata changes would create hundreds of distributions whether anyone needed them or not, and many would become stale as soon as the next Botocore snapshot was adopted.

That works against one of the basic design goals of Amazon::API: install and maintain only the AWS APIs that an application actually uses.

Generated service distributions are therefore built on demand and stored in the public Amazon::API DarkPAN:

https://cpan.openbedrock.net/orepan2

The DarkPAN uses the standard CPAN repository layout and can be consumed by normal Perl installation tools. Once a service distribution has been requested and generated, it remains available there for reuse.

CPAN remains the distribution channel for Amazon::API and the supporting framework and tools. The DarkPAN is used for the much larger and continuously changing collection of generated service distributions.

I downloaded Amazon::API::SQS but there's no documentation. Where is it?

The generated Amazon::API::* service distributions intentionally do not include the full AWS service documentation.

See "BETTER TOGETHER" for information about Amazon::API::Help which provides documentation for services, API methods, request objects, response objects, and parameters.

Which Botocore version was this Amazon::API built against?

use Amazon::API::BuildInfo;

my ($version, $commit) =
    Amazon::API::BuildInfo->botocore_version;

BETTER TOGETHER

The motivation behind Amazon::API has been to provide a lightweight implementation of Amazon APIs for Perl. Several companion projects have been developed with the same philosophy.

Amazon::API::Help

A complete help system for exposing API methods, request objects and response objects. This is shipped separately so that the run-time Amazon::API is as lean as possible.

# get help for a service
amzn-api-help sts

# get help for a method
amzn-api-help sqs ListQueues

# get help for a request object
amzn-api-help sqs ListQueuesRequest

Note: method help will expose the request objects, response objects and parameters

Amazon::Credentials

A lightweight, robust credential provider used transparently by Amazon::API to locate and refresh AWS credentials from a variety of sources including environment variables, instance metadata, container credentials, and web identity tokens.

Amazon::S3::Lite

A lightweight S3 client providing the most commonly used S3 operations without the overhead of a full SDK.

Amazon::Lambda::Runtime

A complete implementation of the AWS Lambda Runtime API that allows Perl developers to write and deploy Lambda functions.

Amazon::Lambda::Runtime::Builder

A complete build system for creating Perl Lambdas.

Log::Log4perl::Appender::CloudWatch

A Log::Log4perl appender that sends log events directly to AWS CloudWatch Logs. Supports buffered writes, automatic stream creation, and both flat and dot-notation Log::Log4perl configuration. Safe to use with Amazon::API services in the same process without re-entrant logging conflicts.

LICENSE AND COPYRIGHT

This module is free software. It may be used, redistributed and/or modified under the same terms as Perl itself.

AUTHOR

Rob Lauer - <rlauer@tresurersbriefcase.com>

To install Amazon::API, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Amazon::API

CPAN shell

perl -MCPAN -e shell
install Amazon::API

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	Go to GitHub issues (only if GitHub is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

SYNOPSIS

DESCRIPTION

Try It Using Docker

BACKGROUND AND MOTIVATION

Take the Luddite approach

Build your own API classes with just what you need

Amazon::API DarkPAN

Build from the cloned https://github.com/rlauer6/Amazon-API.git project

Use the Botocore metadata to build classes locally

THE APPROACH

BOTOCORE SUPPORT

Response Serialization

Protocol Compatibility Testing

Request Validation

ERRORS

METHODS AND SUBROUTINES

new

invoke_api

decode_response

print_error

submit

generate_xml

EXPORTED METHODS

get_api_service

create_urlencoded_content

param_n

service

CAVEATS

IMPLEMENTATION NOTES

Headers

X-Amz-Target

Rolling a New API

Overriding Methods

Content-Type

ADDITIONAL HINTS

VERSION

DIAGNOSTICS

Logging

Trace logging and sensitive data

BUGS AND LIMITATIONS

FAQs

Why should I use this module instead of Paws?

Does it perform better than Paws?

Does this work for all AWS APIs?

This code does not use "Modern Perl". Why?

How do I pass AWS credentials to the API?

Can I use more than one set of credentials to invoke different APIs?

How stable is the interface?

Why are you using XML::Simple when it clearly says "DO NOT"?

How do I stop the client from rejecting a parameter value?

I tried to use this with XYZ service and it didn't work. What should I do?

Why aren't all AWS services included in the distribution or available on CPAN?

I downloaded Amazon::API::SQS but there's no documentation. Where is it?

Which Botocore version was this Amazon::API built against?

BETTER TOGETHER

LICENSE AND COPYRIGHT

SEE ALSO

AUTHOR

Module Install Instructions

Keyboard Shortcuts