/ 
Config-Abstraction-0.40
/ README.md

NAME

Config::Abstraction - Merge and manage configuration data from different sources

VERSION

Version 0.40

SYNOPSIS

Pattern 1: Environment overrides file overrides in-code defaults

The most common pattern for twelve-factor-style apps. The data argument supplies defaults, a YAML file supplies site configuration, and environment variables allow per-deployment overrides without touching any file.

# config/base.yaml
#   database:
#     host: db.example.com
#     port: 5432
#     user: app

use Config::Abstraction;

my $config = Config::Abstraction->new(
    data        => { database => { host => 'localhost', port => 5432 } },
    config_dirs => ['config'],
    env_prefix  => 'APP_',
);

# In production, set APP_DATABASE__HOST=db.prod.example.com in the environment.
# That silently overrides the file value, which in turn overrides the default.
my $host = $config->get('database.host');
my $port = $config->get('database.port');

Pattern 2: Command-line arguments override everything

Useful for CLI tools where operator flags must win over every other source.

# Run as: myscript.pl --APP_LOGLEVEL=debug --APP_DATABASE__HOST=localhost

use Config::Abstraction;

my $config = Config::Abstraction->new(
    config_dirs => ['config'],
    env_prefix  => 'APP_',
);

# @ARGV is consumed and stripped during new(); the resulting config already
# has cli-layer values at the highest precedence.
my $loglevel = $config->get('loglevel');       # 'debug'  (from --APP_LOGLEVEL)
my $db_host  = $config->get('database.host');  # 'localhost' (from --APP_DATABASE__HOST)

Pattern 3: Multi-file layering (base + local override)

Separates shared defaults from machine-specific tweaks. Every developer has a base.yaml; only the production server has local.yaml.

# config/base.yaml   -- checked into version control
#   database:
#     host: localhost
#     user: dev
#
# config/local.yaml  -- NOT checked in; production only
#   database:
#     host: db.prod.example.com
#     user: produser
#     password: s3cr3t

use Config::Abstraction;

my $config = Config::Abstraction->new(
    config_dirs => ['config'],   # loads base.yaml then local.yaml
    env_prefix  => 'APP_',
);

# On a dev machine (no local.yaml): host='localhost', user='dev'
# On the prod server:               host='db.prod.example.com', user='produser'
my $db = $config->get('database.host');
my $user = $config->get('database.user');

DESCRIPTION

Config::Abstraction is a flexible configuration management layer that sits above Config::* modules. It provides a simple way to layer multiple configuration sources with predictable merge order. It lets you define sources such as:

Sources are applied in the order they are provided. Later sources override earlier ones unless a key is explicitly set to undef in the later source.

In addition to using drivers to load configuration data from multiple file formats (YAML, JSON, XML, and INI), it also allows levels of configuration, each of which overrides the lower levels. So, it also integrates environment variable overrides and command line arguments for runtime configuration adjustments. This module is designed to help developers manage layered configurations that can be loaded from files and overridden at run-time for debugging, offering a modern, robust and dynamic approach to configuration management.

Merge Precedence

Sources are applied in the order shown below. Each row wins over every row above it. When the same key appears in multiple sources, the highest-priority source always determines the final value - including when that value is undef.

Priority   Source                         Set with
--------   ------                         --------
   1 (lo)  data constructor argument      data => { key => 'default' }
   2        base.*  config files          config/base.yaml, base.json, ...
   3        base.{env}.* config files     config/base.prod.yaml, ...
   4        local.* config files          config/local.yaml, local.json, ...
   5        local.{env}.* config files    config/local.prod.yaml, ...
   6        default / script-name files   config/default.yaml, myapp.yaml, ...
   7        config_file / config_files    config_file => '/etc/myapp.yaml'
   8        Environment variables         APP_DATABASE__HOST=db.prod.example.com
   9 (hi)  CLI arguments (@ARGV)          --APP_DATABASE__HOST=db.prod.example.com

The active environment is set by the environment constructor option, or auto-detected from {env_prefix}ENV (e.g. APP_ENV), PLACK_ENV, or NODE_ENV. When no environment is configured, rows 3 and 5 are skipped.

Within the file tier (rows 2-7), later files override earlier files using Hash::Merge LEFT_PRECEDENT: every key in the later file wins over the same key in an earlier file, even when the later value is undef (YAML ~). Nested hashes are merged recursively, so a local.yaml that only sets database.host will not erase database.port from base.yaml.

Example - what wins for the key C<database.host> when APP_ENV=prod:

data              =>  'localhost'              (overridden by base.yaml)
base.yaml         =>  'db.example.com'         (overridden by base.prod.yaml)
base.prod.yaml    =>  'db.prod.example.com'    (overridden by local.yaml)
local.yaml        =>  'db.local.example.com'   (overridden by local.prod.yaml)
local.prod.yaml   =>  'db.prod-local.example.com'  (overridden by $APP_DATABASE__HOST)
$APP_DATABASE__HOST  =>  'db.override.example.com'    <-- this wins

KEY FEATURES

SUPPORTED FILE FORMATS

ENVIRONMENT VARIABLE HANDLING

Configuration values can be overridden via environment variables. Environment variables use double underscores (__) to denote nested configuration keys and single underscores remain as part of the key name under the prefix namespace.

For example:

APP_DATABASE__USER becomes database.user (nested structure)

  $ export APP_DATABASE__USER="env_user"

will override any value set for `database.user` in the configuration files.

APP_LOGLEVEL becomes APP.loglevel (flat under prefix namespace)

APP_API__RATE_LIMIT becomes api.rate_limit (mixed usage)

This allows you to override both top-level and nested configuration values using environment variables.

Configuration values can be overridden via the command line (@ARGV). For instance, if you have a key in the configuration such as database.user, you can override it by adding "--APP_DATABASE__USER=other_user_name" to the command line arguments. This will override any value set for database.user in the configuration files.

EXAMPLE CONFIGURATION FLOW

METHODS

new

Constructor for creating a new configuration object.

Options:

If just one argument is given, it is assumed to be the name of a file.

get(key)

Retrieve a configuration value using dotted key notation (e.g., 'database.user'). Returns undef if the key doesn't exist or if key is undef.

EXAMPLE

my $cfg = Config::Abstraction->new(
    data        => { database => { host => 'localhost', port => 5432 } },
    config_dirs => [],
);

my $host = $cfg->get('database.host');   # 'localhost'
my $port = $cfg->get('database.port');   # 5432
my $miss = $cfg->get('database.user');   # undef  -- key absent

API SPECIFICATION

Input

key  -- SCALAR  -- dotted key path (e.g. 'database.host').
                   C<undef> is allowed and returns C<undef> silently.

Output

SCALAR or reference -- the value stored under C<key>, or C<undef> if absent.

MESSAGES

(none) -- missing keys return undef, no warning is raised.

PSEUDOCODE

if key is undef: return undef
call _ensure_loaded
if flatten mode: return config[key] (direct lookup)
parts = split sep_char from key
ref = config hashref
for each part:
  if ref is not a HASH: return undef
  if part not in ref:   return undef
  ref = ref[part]
return ref

exists(key)

Test whether a configuration key is present, using dotted key notation (e.g., 'database.user'). Returns 1 when the key exists (even if its value is undef), 0 otherwise. Returns 0 when key is undef.

EXAMPLE

my $cfg = Config::Abstraction->new(
    data        => { timeout => undef, retries => 3 },
    config_dirs => [],
);

$cfg->exists('timeout');   # 1 -- key present even though value is undef
$cfg->exists('retries');   # 1
$cfg->exists('missing');   # 0
$cfg->exists(undef);       # 0

API SPECIFICATION

Input

key  -- SCALAR  -- dotted key path.  C<undef> returns C<0>.

Output

boolean

MESSAGES

(none)

all()

Returns the entire merged configuration as a hashref, or undef when no configuration data was found. When flatten => 1 was given to the constructor the keys are dotted strings (e.g. 'database.host'); otherwise the hash is nested.

The special key config_path within the returned hashref is an arrayref listing every file that was loaded, in load order.

EXAMPLE

my $cfg = Config::Abstraction->new(
    data        => { host => 'localhost', port => 5432 },
    config_dirs => [],
);

my $all = $cfg->all();
# { host => 'localhost', port => 5432, config_path => [] }

API SPECIFICATION

Input

(none)

Output

HASHREF  -- the full merged config (includes C<config_path> key).
undef    -- when the merged config is empty and no data was supplied.

MESSAGES

(none) -- returns undef silently when empty.

explain_sources()

Returns a hashref describing where each configuration key came from and in what order the sources that set it were applied.

Each key of the returned hashref is a dotted key name (e.g. 'database.user'). The corresponding value is a hashref with two fields:

Keys set by exactly one source have a single-element sources list. Keys whose value was never overridden will show the same value in both the top-level field and the sole sources entry.

USAGE EXAMPLE

use Config::Abstraction;

local $ENV{APP_HOST} = 'prod.example.com';

my $cfg = Config::Abstraction->new(
    data        => { host => 'localhost', port => 5432 },
    config_dirs => ['/etc/myapp'],
);

use Data::Dumper;
print Dumper( $cfg->explain_sources() );
# {
#   'host' => {
#     value   => 'prod.example.com',
#     sources => [
#       { type => 'data', label => 'constructor data argument', value => 'localhost' },
#       { type => 'env',  label => 'APP_HOST',                  value => 'prod.example.com' },
#     ],
#   },
#   'port' => {
#     value   => 5432,
#     sources => [
#       { type => 'data', label => 'constructor data argument', value => 5432 },
#     ],
#   },
# }

API SPECIFICATION

Input

None (instance method; takes no arguments beyond $self).

Output

HASHREF where each key is a dotted config key and each value is:

{
    value   => SCALAR,
    sources => [
        {
            type  => 'data'|'file'|'env'|'argv',
            label => SCALAR,
            value => SCALAR,
        },
        ...
    ],
}

prefer_env(key)

Return the value that an environment variable provided for key, bypassing any later sources (e.g. CLI arguments) that may have overridden it. Falls back to the normal merged value from get(key) when no environment variable contributed to key.

EXAMPLE

local $ENV{APP_DATABASE__HOST} = 'env-host';
my $host = $cfg->prefer_env('database.host');
# Returns 'env-host' even if --APP_DATABASE__HOST=cli-host was also passed.

API SPECIFICATION

Input

key -- SCALAR -- dotted key path.

Output

SCALAR  -- the env-layer value, or C<get(key)> when no env var set it.

prefer_file(key)

Return the value that a configuration file provided for key, bypassing environment variables and CLI arguments that may have overridden it. Falls back to the normal merged value from get(key) when no file contributed to key.

EXAMPLE

my $host = $cfg->prefer_file('database.host');
# Returns the file-sourced value even if APP_DATABASE__HOST is set.

API SPECIFICATION

Input

key -- SCALAR -- dotted key path.

Output

SCALAR  -- the file-layer value, or C<get(key)> when no file set it.

prefer_data(key)

Return the value that the data constructor argument provided for key, bypassing files, environment variables, and CLI arguments that may have overridden it. Falls back to the normal merged value from get(key) when data did not contribute to key.

EXAMPLE

my $cfg = Config::Abstraction->new(
    data        => { timeout => 30 },
    config_dirs => ['/etc/myapp'],
);
my $t = $cfg->prefer_data('timeout');   # always 30, regardless of files/env

API SPECIFICATION

Input

key -- SCALAR -- dotted key path.

Output

SCALAR  -- the data-layer value, or C<get(key)> when C<data> did not set it.

prefer_argv(key)

Return the value that a CLI argument provided for key. Falls back to the normal merged value from get(key) when no CLI argument contributed to key.

Because CLI arguments are the highest-precedence source, this method is primarily useful for writing self-documenting code or for detecting whether a key was explicitly supplied on the command line.

EXAMPLE

my $level = $cfg->prefer_argv('log.level');
# Equivalent to $cfg->get('log.level') unless you specifically need to
# confirm the value came from @ARGV.

API SPECIFICATION

Input

key -- SCALAR -- dotted key path.

Output

SCALAR  -- the argv-layer value, or C<get(key)> when no CLI arg set it.

encrypt_value($plaintext)

Encrypt a plaintext string using the configured AES-256-GCM key and return an ENC[AES256GCM,...] token suitable for storing in a configuration file.

Each call generates a fresh random nonce, so the same plaintext produces a different token every time. The token is authenticated (GCM tag); any modification causes decryption to croak.

Requires CryptX (Crypt::AuthEnc::GCM, Crypt::PRNG).

EXAMPLE

# Generate a token to paste into base.yaml:
my $cfg = Config::Abstraction->new(
    encryption_key => $hex_key,
    config_dirs    => [],
    lazy           => 1,
);
my $token = $cfg->encrypt_value('s3cr3t_password');
# ENC[AES256GCM,QkJCQkJCQkJCQkJCO6Gfb0o5jwqB1R...]

# Or use config-dump from the command line:
#   config-dump --encrypt-value 's3cr3t_password' --encryption-key $KEY

API SPECIFICATION

Input

plaintext  -- SCALAR  -- the string to encrypt.  May be empty.

Output

SCALAR  -- the ENC[AES256GCM,...] token (always a printable ASCII string).

MESSAGES

"<class>: no encryption key configured ..." -- croak when no key is available.
"<class>: CryptX (Crypt::AuthEnc::GCM) is required ..." -- croak when CryptX absent.

PSEUDOCODE

call _ensure_loaded
key = _get_encryption_key() -- croak if undef
load Crypt::AuthEnc::GCM and Crypt::PRNG -- croak if absent
nonce = 12 random bytes
gcm = new GCM('AES', key)
gcm.iv_add(nonce)
ciphertext = gcm.encrypt_add(plaintext)
tag = gcm.encrypt_done()
return "ENC[AES256GCM," + base64url(nonce + ciphertext + tag) + "]"

merge_defaults

Merge the configuration hash into the given hash.

package MyPackage;
use Params::Get;
use Config::Abstraction;

sub new
{
  my $class = shift;

  my $params = Params::Get::get_params(undef, \@_) || {};

  if(my $config = Config::Abstraction->new(env_prefix => "${class}::")) {
    $params = $config->merge_defaults(defaults => $params, merge => 1, section => $class);
  }

  return bless $params, $class;
}

Options:

Remote configuration directories (Newcastle Connection)

Any entry in config_dirs whose path begins with /../ is treated as a remote specification rather than a local directory:

/../hostname/path/to/dir

The hostname and directory are extracted, and the same standard files searched locally (base.yaml, local.yaml, base.json, etc.) are fetched from the remote machine via File::Slurp::Remote over SSH.

When the hostname resolves to the local machine -- localhost, 127.0.0.1, ::1, or the value returned by Sys::Hostname::hostname() (checked as both fully-qualified and short form, case-insensitively) -- the /../host/ wrapper is silently unwrapped and the enclosed path is processed through the normal local file pipeline instead. No SSH connection is made. This means a configuration written for a shared remote host degrades gracefully when run on that host itself:

# On any other machine: fetches /etc/myapp over SSH
# On cfg-server itself: reads /etc/myapp from disk directly
config_dirs => ['/../cfg-server/etc/myapp']

Remote directories participate in the normal merge pipeline and can be freely mixed with local ones:

my $cfg = Config::Abstraction->new(
    config_dirs => [
        '/etc/myapp',                        # local
        '/../deploy@cfg-server/etc/myapp',   # remote via SSH (local on cfg-server)
    ],
);

SSH authentication is handled by the system SSH client. No extra constructor options are required; use your SSH agent or ~/.ssh/config for host-specific settings.

File::Slurp::Remote must be installed for remote directories to work. If it is absent the directory is silently skipped and a warning is emitted.

Why /../ (the Newcastle Connection convention)

Several syntaxes were considered for marking a config_dirs entry as remote. Each alternative was rejected for a concrete reason:

The Newcastle Connection prefix is therefore the only choice that is:

AUTOLOAD

This module supports dynamic access to configuration keys via AUTOLOAD. Nested keys are accessible using the separator, so $config->database_user() resolves to $config->{database}->{user}, when sep_char is set to '_'.

$config = Config::Abstraction->new(
    data => {
        database => {
            user => 'alice',
            pass => 'secret'
        },
        log_level => 'debug'
    },
    flatten => 1,
    sep_char => '_'
);

my $user = $config->database_user();        # returns 'alice'

# or
$user = $config->database()->{'user'};      # returns 'alice'

# Attempting to call a nonexistent key
my $foo = $config->nonexistent_key();       # dies with error

ENCRYPTED VALUES

Config::Abstraction supports transparent AES-256-GCM encryption of individual configuration values. This lets you store secrets (passwords, API keys, tokens) in config files without exposing them as plaintext, even when those files are committed to version control.

Quick start

Step 1 -- generate a key:

# 32 random bytes, encoded as 64 hex chars
perl -e 'use Crypt::PRNG qw(random_bytes); use MIME::Base64 qw(encode_base64url);
         print encode_base64url(random_bytes(32)), "\n"'

Store the result in an environment variable or a key file (outside version control):

export ENCRYPTION_KEY=<the 44-char base64url output>

Step 2 -- encrypt a secret value:

perl -MConfig::Abstraction -e '
  my $cfg = Config::Abstraction->new(data => {});
  print $cfg->encrypt_value("my_secret_password"), "\n";
'

This prints something like:

ENC[AES256GCM,QkJCQkJCQkJCQkJCO6Gfb0o5...]

Step 3 -- paste the token into your config file:

# config/base.yaml
database:
  host: db.example.com
  user: myapp
  password: 'ENC[AES256GCM,QkJCQkJCQkJCQkJCO6Gfb0o5...]'

Step 4 -- load and use normally:

my $cfg = Config::Abstraction->new(config_dirs => ['config']);
# $cfg->get('database.password') returns 'my_secret_password' -- already decrypted

Key configuration

The encryption key is resolved in this order (first match wins):

The key file should contain the key on its first line in any supported format. Never commit the key to version control.

Token format

ENC[AES256GCM,<base64url(nonce || ciphertext || tag)>]

Behaviour when no key is configured

If no key is found, ENC[...] tokens are left as literal strings. This means the feature is purely opt-in: existing deployments without a key configured are unaffected.

Requirements

CryptX (Crypt::AuthEnc::GCM, Crypt::PRNG) must be installed:

cpanm CryptX

COMMON PITFALLS

1. new() returns undef when no configuration is found

new() returns undef, not a blessed object, when no configuration data is found and no data argument was supplied. Every caller must check the return value.

my $cfg = Config::Abstraction->new(config_dirs => ['/etc/myapp']);
die "No configuration found" unless defined $cfg;
my $host = $cfg->get('database.host');   # safe

Forgetting the check leads to a cryptic "Can't call method on undef" error later, with a stack trace that points to get() rather than to the missing config file.

2. merge_defaults() requires a named argument, not a bare hashref

# WRONG -- Params::Get fast-path returns the whole config unchanged
my $merged = $cfg->merge_defaults(\%my_defaults);

# RIGHT
my $merged = $cfg->merge_defaults(defaults => \%my_defaults);

With the first form, Params::Get treats the single hashref as the entire parameter bag and returns the full config without merging anything.

3. Shallow merge in merge_defaults() silently drops nested keys from defaults

Without merge => 1, merge_defaults() uses a plain Perl hash merge ({ %defaults, %config }) at the top level. If the config contains a nested hash for a key, it entirely replaces the corresponding nested hash in your defaults - any keys that exist only in the defaults' nested hash are silently discarded.

my $cfg = Config::Abstraction->new(
    data        => { db => { host => 'localhost', port => 5432 } },
    config_dirs => [],
);

my $merged = $cfg->merge_defaults(defaults => { db => { user => 'guest' } });
# $merged->{db}{user} is UNDEF -- the whole 'db' hash was replaced by the config's version

# To combine nested keys from both sides, pass merge => 1:
my $merged = $cfg->merge_defaults(defaults => { db => { user => 'guest' } }, merge => 1);
# $merged->{db}{user} is 'guest', $merged->{db}{host} is 'localhost'

4. undef from a higher-priority source permanently wins

Hash::Merge LEFT_PRECEDENT means that an explicit undef (YAML ~) in a higher-priority source overrides a real value in a lower-priority source, including when the lower-priority value is defined.

# base.yaml:  timeout: 30
# local.yaml: timeout: ~

my $t = $cfg->get('timeout');   # undef -- local.yaml's null wins over base.yaml

This is intentional: it lets a local config deliberately unset a value. If you want to detect whether a key was explicitly nulled versus simply absent, use explain_sources() and inspect the sources list.

5. Double underscore vs. single underscore in environment variable names

Single underscores are part of the key name; double underscores create a nesting level.

APP_LOG_LEVEL=debug       => key 'log_level'   (single underscore, flat key)
APP_DATABASE__HOST=db     => key 'database.host' (double underscore, nested)
APP_API__RATE_LIMIT=100   => key 'api.rate_limit' (double underscore + single)

A common mistake is using single underscores expecting nested keys:

APP_DATABASE_HOST=db   # produces key 'database_host', NOT 'database.host'

6. Using AUTOLOAD requires sep_char set to '_'

AUTOLOAD translates method names to config keys using sep_char. The default sep_char is '.', but method names cannot contain dots. Set sep_char => '_' to use AUTOLOAD, and be aware that this makes single underscores into hierarchy separators.

my $cfg = Config::Abstraction->new(
    data    => { database => { host => 'localhost' } },
    sep_char => '_',
    config_dirs => [],
);
my $host = $cfg->database_host();   # works
my $bad  = $cfg->no_such_key();    # dies: No such config key 'no_such_key'

7. Absolute config_file paths require an empty or omitted config_dirs

On Unix, File::Spec->catfile('/etc', '/absolute/path.yaml') concatenates the two strings instead of letting the absolute path take over, producing a wrong path. When config_file is an absolute path, either omit config_dirs entirely (the constructor sets it to [''] automatically) or pass config_dirs => [''] explicitly.

# WRONG on Unix -- produces '/etc/etc/myapp/app.yaml'
Config::Abstraction->new(
    config_file => '/etc/myapp/app.yaml',
    config_dirs => ['/etc'],
);

# RIGHT
Config::Abstraction->new( config_file => '/etc/myapp/app.yaml' );

8. Tests must isolate from the developer's real config files

A call to new() without config_dirs will scan /etc, ~/.conf, ~/.config, and other default locations and load any base.yaml or local.yaml it finds there. In a test suite this injects real host configuration into your test object, causing non-deterministic failures.

Always pass config_dirs => [] in tests that use only in-memory data:

my $cfg = Config::Abstraction->new(
    data        => { key => 'value' },
    config_dirs => [],              # do not scan the filesystem
);

9. lazy => 1 defers errors until the first accessor call

With lazy => 1, new() always returns a blessed object - it cannot return undef for a missing config, and any schema validation errors surface at the first get() or all() call rather than at construction time. See the lazy option documentation in "new" for the full list of debugging implications.

VERSION HISTORY

Notable changes by release. Full details are in the Changes file.

LIMITATIONS

BUGS

It should be possible to escape the separator character either with backslashes or quotes.

Due to the case-insensitive nature of environment variables on Windows, it may be challenging to override values using environment variables on that platform.

REPOSITORY

https://github.com/nigelhorne/Config-Abstraction

SUPPORT

This module is provided as-is without any warranty.

Please report any bugs or feature requests to bug-config-abstraction at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Config-Abstraction. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

You can find documentation for this module with the perldoc command.

perldoc Config::Abstraction

SEE ALSO

AUTHOR

Nigel Horne, <njh at nigelhorne.com>

FORMAL SPECIFICATION

get

get : Config x Key → Value ∪ {⊥}
get(c, k) ≜ if k = ⊥ then ⊥
            else lookup(c.config, split(c.sep_char, k))
lookup(h, [])      ≜ h
lookup(h, p:rest)  ≜ if p ∉ dom(h) then ⊥
                      else lookup(h[p], rest)

encrypt_value

encrypt_value : Config x Plaintext → Token
encrypt_value(c, p) ≜
  let k  = resolve_key(c)  where k ≠ ⊥
  let n  ~ Uniform(Bytes^12)            -- fresh random nonce
  let ct = AES256GCM_enc(k, n, p)
  let t  = GCM_tag(k, n, p)
  in "ENC[AES256GCM," || base64url(n || ct || t) || "]"

exists

exists : Config x Key → {0, 1}
exists(c, k) ≜ if k = ⊥ then 0
               else 1 if lookup(c.config, split(c.sep_char, k)) ≠ ⊥
               else 0

all

all : Config → HashRef ∪ {⊥}
all(c) ≜ if |dom(c.config)| = 0 then ⊥ else c.config

LICENCE AND COPYRIGHT

Copyright 2025-2026 Nigel Horne.

Usage is subject to the GPL2 licence terms. If you use it, please let me know.