NAME

Database::BI::Model::Geocode - Resolve a location column to coordinates, once per distinct value

VERSION

0.013.0

SYNOPSIS

Geocode a table's place column and build map markers:

use Database::BI::Model::Geocode;

my $geocode = Database::BI::Model::Geocode->new();
my $result  = $geocode->resolve_column($rows, 'city');

for my $marker (@{ $result->{'markers'} }) {
    printf "  %s: lat=%.4f lon=%.4f (%d rows)\n",
        $marker->{'value'}, $marker->{'lat'}, $marker->{'lon'},
        $marker->{'rows'};
}

my $c = $result->{'counts'};
printf "Plotted %d of %d rows (%d skipped, %d unresolved)\n",
    $c->{'plotted'}, $c->{'rows'}, $c->{'skipped'}, $c->{'unresolved'};

Re-use remembered answers so the next request costs nothing:

# Load what was stored from a previous run.
my $spec   = 'table:sales';
my $models = decode_json(Mojo::File->new('.bi/models.json')->slurp);
my $cache  = Database::BI::Model::Geocode->cache_from_models($models, $spec);

# Resolve; cached values are used without asking the geocoder.
my $result = $geocode->resolve_column($rows, 'city', cache => $cache);

# Save the updated cache back to the models file.
my $updated = Database::BI::Model::Geocode->models_with_cache(
    $models, $spec, $result->{'cache'},
    backend => $result->{'backend'},
    built   => '2026-10-18T12:00:00',
);
Mojo::File->new('.bi/models.json')->spew(encode_json($updated));

Warm the cache for a whole column without needing rows:

# Extract unique city values from your own data.
my @cities = do { my %s; grep { !$s{$_}++ } map { $_->{'city'} } @{ $rows } };

my $result = $geocode->resolve_values(\@cities,
    cache  => $cache,
    region => 'GB',
    max    => 1000,
);

printf "Resolved %d / %d new places\n",
    $result->{'counts'}{'resolved'}, $result->{'counts'}{'attempted'};

Ask whether the geocoder is available before offering it in the UI:

my $geocode = Database::BI::Model::Geocode->new();
my ($ok, $why) = $geocode->geocoder_available();

unless ($ok) {
    carp "Map panel disabled: $why";
}

Inject a test double so geocoding works without a real geocoder:

package MyGeocoderStub;
sub new            { bless {}, shift }
sub valid_location { my ($self, $v) = @_; $v && $v !~ /\A\d+\z/ }
sub geocode_many   {
    my ($self, $places, %opts) = @_;
    return { 'London' => { lat => 51.5, long => -0.12, database => 'stub' } };
}

my $geocode = Database::BI::Model::Geocode->new(geocoder => MyGeocoderStub->new);
my $result  = $geocode->resolve_column($rows, 'city');

DESCRIPTION

Geo::Coder::Free resolves a place name to a Geo::Location::Point. A table column holds something less tidy: values repeated thousands of times, blank cells, numbers that drifted in from an adjacent column. This module does the three things the column then needs.

Ask once per distinct value.: The column is reduced to a set before any lookup happens. A repeated city name costs one lookup, so a 100,000-row file with a dozen cities costs twelve lookups, not 100,000.
Refuse a value that is not a location before any backend sees it.: $geocoder->valid_location($value) is the gate the geocoder's own guard applies before looking anything up. This module asks the same question first, so a pure-digit string (a row number that ended up in the wrong column), an empty cell, or a Perl reference is counted and reported as skipped rather than sent to a backend that would reject it anyway.
Look the rest up without letting one value end the batch.: The batch is requested with on_error => 'undef'. A value the geocoder refuses after the pre-filter is a miss; a backend that throws an exception is still a throw, because nothing here wraps the geocoder in an eval.

The counts the module returns add up, always.

skipped + cached + attempted + not_attempted  ==  distinct
resolved + unresolved                         ==  cached + attempted

lookups is the number the geocoder itself reports for the batch. With Geo::Coder::Free 0.48 and later it may be lower than attempted, because the geocoder counts database reads and a value it answered from a warm process cache costs zero reads. It never exceeds attempted.

The cache

A cache entry is a [lat, lon] arrayref for a place that resolved, or the scalar 0 for a place that was looked up and not found. That 0 sentinel prevents a column of unresolvable values from being retried on every page request.

The cache is passed per call and returned per call. The caller's hashref is never modified: a request that dies part way through cannot leave a half-written cache behind. The controller is responsible for persisting the returned cache, which keeps the model_dir-inside-data_dir guard in the one place that already owns it.

A previously stored cache is used even when the geocoder is not available. Places already resolved continue to be plotted; only new lookups are unavailable.

Without the geocoder

Geo::Coder::Free is a recommended dependency, not a required one. It is loaded the first time it is needed, not at compile time. When it is absent or older than the minimum version:

$geocode->geocoder_available() returns (0, $reason).
resolve_column and resolve_values still return a complete result: all values are placed in not_attempted, cached answers are still plotted, and the error_map_geocoder_missing message explains the situation.
Nothing croaks. The page renders with whatever the cache already holds.

CONSTRUCTOR

new

my $geocode = Database::BI::Model::Geocode->new(%options);
my $geocode = Database::BI::Model::Geocode->new(\%options);  # hashref also accepted

Create a new resolver. Accepts a flat key/value list or a hashref; both forms are handled by Params::Get.

API SPECIFICATION

INPUT

geocoder             => object,   # optional; duck-typed: must provide valid_location() and geocode_many()
openaddr             => string,   # optional; path to OpenAddresses data (default: $ENV{OPENADDR_HOME})
min_geocoder_version => string,   # optional; minimum version to accept (default: '0.48')
i18n                 => object,   # optional; must implement maketext($key, @args)

geocoder is the injection seam for tests. An injected geocoder is used exactly as given: it is not version-checked, so any object that responds to valid_location() and geocode_many() works. A geocoder that fails either can() check causes an immediate croak (error_geocoder_interface), not a silent degradation.

openaddr overrides $ENV{OPENADDR_HOME}. It is forwarded to Geo::Coder::Free->new(openaddr => ...). When it is absent and the environment variable is not set, the geocoder uses its built-in MaxMind data only, which resolves city and country names but not street addresses. warn_map_geocoder_unavailable_data is emitted in that case.

min_geocoder_version lets the caller pin the floor, or a test simulate a too-old installation on a machine that has the current version.

i18n is an optional object whose maketext($key, @args) method is called in place of the built-in %MESSAGES table. The key is one of the constants defined in %MESSAGES; the args are the same positional values sprintf would receive.

OUTPUT

Returns $self.

Croaks with error_geocoder_interface if the injected geocoder does not provide both valid_location() and geocode_many().

Never croaks because the geocoder module is absent or too old: that is a runtime condition reported by geocoder_available(), not a constructor error.

SUBROUTINES/METHODS

resolve_column

my $result = $geocode->resolve_column(\@rows, $column_name, %options);

Resolve one named column of a table. \@rows is an arrayref of hashrefs (the records from the data pipeline), and $column_name is the key to look up in each row.

This is the main entry point for the map controller. It runs the same resolution as "resolve_values", then adds per-marker row counts and the markers list that a map rendering loop needs.

API SPECIFICATION

INPUT

\@rows          => arrayref of hashrefs, required
$column_name    => non-empty string, required
cache    => hashref,            # optional; { value => [lat, lon] | 0 }
region   => two-letter string,  # optional; country hint for the geocoder (e.g. 'GB')
max      => positive integer,   # optional; max new lookups; default 500

\@rows is the full record set as returned by the pipeline. Rows that are not hashrefs produce an undef cell, which is classified as skipped.

$column_name must be a non-empty string. It is used as a hash key; it is not validated against a schema, so a column name with spaces or special characters is accepted if the data uses one.

cache holds the already-known answers for this column. Entries are either a [lat, lon] arrayref (resolved) or 0 (looked up and not found). The caller's hashref is never modified; the returned cache key holds the updated copy.

region is passed straight through to the geocoder as a country hint. It is not derived from the Accept-Language header, because a language is not a country.

max is counted in new lookups, not rows. A cached answer costs nothing and does not count toward it. Values held back by the cap go into not_attempted and are not stored in the cache, so the next request picks them up.

OUTPUT

Returns a hashref:

{
    available     => 1 | 0,
    points        => { $value => { lat => $n, lon => $n, rows => $n, first_row => $n, backend? => $s } },
    markers       => [ { value => $s, lat => $n, lon => $n, rows => $n, first_row => $n } ],
    counts        => {
        rows          => $n,   # total rows in \@rows
        plotted       => $n,   # rows whose value resolved to a point
        unusable      => $n,   # rows - plotted
        distinct      => $n,   # unique values seen
        skipped       => $n,   # refused before any lookup (empty, ref, digit-only)
        cached        => $n,   # found in the cache (resolved or not)
        attempted     => $n,   # sent to the geocoder this request
        lookups       => $n,   # lookups the geocoder reports (may be < attempted)
        resolved      => $n,   # total with a position (cache + new)
        unresolved    => $n,   # total without (cache + new)
        not_attempted => $n,   # held back by the cap, or skipped because no geocoder
    },
    skipped       => [ $value, ... ],   # values refused before lookup
    unresolved    => [ $value, ... ],   # values tried (now or before) and not found
    not_attempted => [ $value, ... ],   # held back by the cap
    messages      => [ { key => $k, text => $t }, ... ],
    cache         => { $value => [lat, lon] | 0 },   # updated copy; never the caller's hashref
    backends      => { $backend_name => $count },
    backend       => $backend_name | undef,          # undef when nothing resolved
    capped        => 0 | 1,
}

points holds one entry per value that produced a position, whether the position came from the cache or from this request. markers is the same data in first-seen row order, which reads better on a map than alphabetical order.

backend on a point entry is set only for values looked up in this request; cached values do not carry one because the cache stores a position, not which database gave it. The top-level backend is the name of the backend that answered the most lookups this request.

The count invariant always holds:

skipped + cached + attempted + not_attempted  ==  distinct

MESSAGES

error_rows_arrayref                -- \@rows is not an arrayref
error_column_required              -- $column_name is empty or undef
error_map_loc_column               -- column not found in any row
error_map_geocoder_missing         -- no geocoder, or version below the floor
warn_map_rows_skipped              -- some rows could not be plotted
warn_map_geocode_capped            -- the cap held some values back
warn_map_geocoder_unavailable_data -- OPENADDR_HOME not set; coarse resolution only

EXAMPLE

my $cache   = Database::BI::Model::Geocode->cache_from_models($models, $spec);
my $result  = $geocode->resolve_column($rows, 'city',
    cache  => $cache,
    region => 'GB',
    max    => 200,
);

if ($result->{'capped'}) {
    carp "Some cities were not looked up; load the page again to finish them.";
}

for my $m (@{ $result->{'markers'} }) {
    printf "[%.4f, %.4f]  %s (%d rows)\n",
        $m->{'lat'}, $m->{'lon'}, $m->{'value'}, $m->{'rows'};
}

resolve_values

my $result = $geocode->resolve_values(\@values, %options);
my $result = $geocode->resolve_values(\@values, \%options);

Resolve a flat list of cell values, without the context of any row structure. Use this when you want to warm the cache for a column's distinct values, or when you have already extracted the values yourself and do not need per-marker row counts.

The returned hashref is identical to "resolve_column"'s, except that counts.rows, counts.plotted, counts.unusable, and markers are absent, and warn_map_rows_skipped is never emitted.

API SPECIFICATION

INPUT

\@values => arrayref, required;  elements may be any scalar or undef
cache    => hashref,             # optional; { value => [lat, lon] | 0 }
region   => string,              # optional; two-letter country hint
max      => positive integer,    # optional; new lookups allowed; default 500

Duplicate values in \@values cost one lookup each. undef, references, empty strings, and pure-digit strings are refused before any lookup and counted as skipped.

OUTPUT

Returns a hashref (same shape as "resolve_column" but without markers, counts.rows, counts.plotted, and counts.unusable):

{
    available     => 1 | 0,
    points        => { $value => { lat => $n, lon => $n, backend? => $s } },
    counts        => {
        distinct      => $n,
        skipped       => $n,
        cached        => $n,
        attempted     => $n,
        lookups       => $n,
        resolved      => $n,
        unresolved    => $n,
        not_attempted => $n,
    },
    skipped       => [ $value, ... ],
    unresolved    => [ $value, ... ],
    not_attempted => [ $value, ... ],
    messages      => [ { key => $k, text => $t }, ... ],
    cache         => { $value => [lat, lon] | 0 },
    backends      => { $name => $count },
    backend       => $name | undef,
    capped        => 0 | 1,
}

Croaks with error_values_arrayref if \@values is not an arrayref.

EXAMPLE

# Build the distinct-value list yourself and warm the cache offline.
my %seen;
my @cities = grep { defined($_) && !$seen{$_}++ } map { $_->{'city'} } @{ $rows };

my $result = $geocode->resolve_values(\@cities,
    cache  => $cache,
    region => 'GB',
    max    => 2000,
);

printf "New lookups: %d  Resolved: %d  Unresolved: %d\n",
    $result->{'counts'}{'attempted'},
    $result->{'counts'}{'resolved'},
    $result->{'counts'}{'unresolved'};

resolve_column

See full documentation above, in "SUBROUTINES/METHODS".

geocoder_available

my ($is_available, $reason) = $geocode->geocoder_available();

Ask whether this object can make new geocoder lookups. Returns a list of two values: a boolean (1 or 0) and a human-readable reason string when the answer is no.

Loads Geo::Coder::Free lazily on the first call if it has not been loaded yet. Never croaks: a panel uses this to decide whether to show the location option, so the answer must always be a string, not an exception.

API SPECIFICATION

INPUT

None. (No arguments accepted.)

OUTPUT

($available, $reason)

$available => 1      # geocoder is ready
$available => 0      # geocoder is not available
$reason    => undef  # when $available is 1
$reason    => string # error_map_geocoder_missing text; when $available is 0

EXAMPLE

my ($ok, $reason) = $geocode->geocoder_available();

if (!$ok) {
    # Disable the "Map by location column" panel option.
    $stash->{'map_loc_disabled'} = 1;
    $stash->{'map_loc_disabled_reason'} = $reason;
}

cache_from_models

my $cache = Database::BI::Model::Geocode->cache_from_models(\%models, $spec_key);

Extract the geocode cache for one dataset spec from a model store. Returns a copy of the stored places hashref so the caller cannot accidentally modify the model store.

This is a class method: call it on the package name, or on an instance if you prefer; either way the invocant is ignored.

API SPECIFICATION

INPUT

\%models  => hashref, required;  the structure returned by _read_models()
$spec_key => string, required;   the dataset spec (e.g. 'table:sales')

OUTPUT

Returns a plain hashref:

{ $place_name => [lat, lon] | 0, ... }

Returns an empty hashref {} for an unknown spec, a spec that has never been geocoded, or a \%models argument that is not a hashref. Never croaks.

EXAMPLE

my $models = decode_json(Mojo::File->new($model_path)->slurp);
my $cache  = Database::BI::Model::Geocode->cache_from_models(
    $models, 'table:sales'
);

# $cache is now { 'Ramsgate' => [51.336, 1.416], 'Paris' => 0, ... }

models_with_cache

my $updated_models = Database::BI::Model::Geocode->models_with_cache(
    \%models, $spec_key, \%cache,
    backend => 'OpenAddresses',
    built   => '2026-10-18T12:00:00',
);

Produce a new model store in which the geocode cache for $spec_key is replaced with \%cache. Every other key in the model store, and every other spec's geocode data, is carried over unchanged.

This is a class method; the invocant is ignored.

This module never writes the file. The caller is responsible for passing $updated_models to _write_models() or its equivalent, which is where the writability and data_dir guards live. The separation keeps those guards in the one place that already owns them (section 7 of the phase spec).

API SPECIFICATION

INPUT

\%models  => hashref, required;  the existing model store (not modified)
$spec_key => string, required;   the dataset spec to update
\%cache   => hashref, required;  { place => [lat, lon] | 0 }
backend   => string, optional;   name of the geocoder backend that resolved
built     => string, optional;   ISO-8601 timestamp of when the cache was built

OUTPUT

Returns a new hashref. The caller's \%models is never modified.

{
    ...all existing model keys...,
    geo => {
        ...all existing geo specs...,
        $spec_key => {
            places  => \%cache,
            backend => $backend,  # if supplied
            built   => $built,    # if supplied
        },
    },
}

EXAMPLE

my $updated = Database::BI::Model::Geocode->models_with_cache(
    $models,
    'path:/data/sales.csv',
    $result->{'cache'},
    backend => $result->{'backend'},
    built   => '2026-10-18T09:15:42',
);

# Now write the updated store to disk.
Mojo::File->new($model_path)->spew(encode_json($updated));

shared_cache_from_models

my $cache = Database::BI::Model::Geocode->shared_cache_from_models(\%models);

Extract the place answers every dataset shares, as a copy the caller may modify. This is the cache a map request should pass to resolve_column: a place geocoded for one file is then plotted for every other file that names it, without a second lookup.

A model store written before the cache was shared has no shared entry; this returns {} and the per-dataset caches remain readable with cache_from_models. A caller that reads both gets the old answers once and writes them back to the shared entry (see models_with_shared_cache).

This is a class method; the invocant is ignored.

API SPECIFICATION

INPUT

\%models  => hashref, required;  the structure returned by _read_models()

OUTPUT

{ $place_name => [lat, lon] | 0, ... }

Returns an empty hashref for a store with no shared cache, for a malformed entry, or for a \%models argument that is not a hashref. Never croaks.

EXAMPLE

my $models = decode_json(Mojo::File->new($model_path)->slurp);
my $cache  = Database::BI::Model::Geocode->shared_cache_from_models($models);

# $cache is now { 'Ramsgate' => [51.336, 1.416], 'Paris' => 0, ... }

models_with_shared_cache

my $updated_models = Database::BI::Model::Geocode->models_with_shared_cache(
    \%models, \%cache,
    backend => 'OpenAddresses',
    built   => '2026-10-18T12:00:00',
);

Produce a new model store in which the shared place cache holds \%cache. The cache is merged, not replaced: a request writes back every place it saw, and another request may have answered a different place since this one read the store. Other model keys, and every per-dataset geo entry, are carried over unchanged.

This module never writes the file, and does not add a per-dataset entry. The caller passes the result to _write_models() or its equivalent, where the writability and data_dir guards live.

API SPECIFICATION

INPUT

\%models => hashref, required;  the existing model store (not modified)
\%cache  => hashref, required;  { place => [lat, lon] | 0 }
backend  => string, optional;   name of the geocoder backend that resolved
built    => string, optional;   ISO-8601 timestamp of the cache update

OUTPUT

Returns a new hashref. The caller's \%models is never modified.

{
    ...all existing model keys...,
    geo => {
        ...all existing per-dataset entries...,
        shared => {
            places  => \%merged_cache,
            backend => $backend,  # if supplied
            built   => $built,    # if supplied
        },
    },
}

EXAMPLE

my $updated = Database::BI::Model::Geocode->models_with_shared_cache(
    $models,
    $result->{'cache'},
    backend => $result->{'backend'},
    built   => '2026-10-18T09:15:42',
);

Mojo::File->new($model_path)->spew(encode_json($updated));

COMMON PITFALLS

The cache sentinel 0 is not "false -- never looked up".

A cache entry of 0 means the place was looked up and not found. An absent key means it has never been attempted. The module treats these two states differently: a 0 entry contributes to counts.unresolved without triggering a new lookup; an absent key enters the classification loop and may reach the geocoder.

# WRONG -- checking definedness misses the 0 sentinel.
my $resolved = defined($cache{$place});

# CORRECT -- use the internal _has_position logic.
my $resolved = (ref($cache{$place}) eq 'ARRAY');

Pure-digit strings are skipped, not attempted.

A value like '42' fails valid_location() and goes into @skipped. It is never stored in the cache. This is the correct behaviour for a row number that ended up in the wrong column, but it surprises callers who expect every non-empty value to be attempted.

The returned cache is a copy; the caller's hashref is never modified.

Do not compare $cache_you_passed_in with $result-{'cache'}> expecting them to be the same reference. The update is in $result-{'cache'}>.

resolve_column croaks when the column is absent; resolve_values does not.

resolve_column checks that the named column appears in at least one row. If it does not, it croaks with error_map_loc_column. resolve_values does not do this check, because it receives values, not rows.

max counts new lookups, not rows or distinct values.

A value already in cache does not count toward max, even if it was looked up in a previous request. A value skipped by valid_location() does not count either.

new() always succeeds; geocoder load failure is not a constructor error.

You can always call new() without checking whether Geo::Coder::Free is installed. The load failure is deferred to the first call that needs the geocoder, and is reported through geocoder_available() and the messages array in the result, not through an exception.

cache_from_models and models_with_cache are class methods.

You may call them on an instance for convenience, but the invocant is not used. The pattern Database::BI::Model::Geocode->cache_from_models(...) is clearer.

The geocoder batch receives on_error = 'undef'.>

A value the geocoder rejects after the valid_location() pre-filter is returned as undef from the batch; it does not croak. A backend that throws an exception (a real database error, a network failure) still propagates upward, because nothing here wraps the geocoder in an eval.

ENCODING

Place names (values in the location column): Passed to the geocoder as byte strings. UTF-8 place names such as "Koeln" or "Beijing" are supported if the geocoder backend handles them. The internal code does not re-encode or validate the bytes; it compares them as opaque strings for deduplication and caching.
Column names: Matched against row hash keys with a string equality test. The column name must be byte-for-byte identical to the key in the row hashrefs. Column names with non-ASCII bytes are accepted.
Spec keys (the $spec_key argument): Used as hash keys in the model store. Any byte sequence is safe; UTF-8 spec keys are stored and retrieved without modification.
Message strings: The built-in %MESSAGES table contains ASCII-only text. If your application needs non-ASCII messages (for example, a French translation), supply an i18n object to new() whose maketext method renders the key in the desired locale.
Geocoder result strings (backend names): Stored and reported as returned by the geocoder. The module does not impose any encoding on them.

DIAGNOSTICS

All user-visible messages are keyed in %Database::BI::Model::Geocode::MESSAGES.

error_map_geocoder_missing

Geocoding a location column needs Geo::Coder::Free %s or later, and %s

The geocoder module is not installed (the second %s is "it is not installed") or is too old (the second %s is the installed version number). The first %s is the floor this module checks for - 0.48 unless the caller pinned min_geocoder_version. Install or upgrade Geo::Coder::Free.

error_map_loc_column

Column "%s" is not in this dataset, so there is nothing to geocode

The $column_name argument to resolve_column() did not appear as a key in any row. Check the column name, including case and any spaces.

error_geocoder_interface

Geocode: the geocoder must provide %s; Geo::Coder::Free %s or later does

An injected geocoder object does not respond to valid_location() or geocode_many(). Either use the real Geo::Coder::Free or add the missing method to your stub.

error_max_invalid

Geocode: option "max" must be a positive integer, not "%s"

The max option was supplied but is not a positive integer string. Passing max => 0 and max => 'all' both trigger this error.

error_values_arrayref

Geocode: the values must be an arrayref, not "%s"

The first argument to resolve_values() is not an arrayref. Pass \@values (a reference), not @values (a flat list).

error_rows_arrayref

Geocode: the rows must be an arrayref, not "%s"

The first argument to resolve_column() is not an arrayref.

error_column_required

Geocode: the column name is required

The $column_name argument to resolve_column() is undef or an empty string.

warn_map_rows_skipped

Plotted %d of %d rows; %d hold no location that could be plotted ...

Some rows in the input had a cell that was empty, a bare number, or a place that the geocoder could not find. The count of plotted rows is less than the total. Not an error: the map renders with the rows that could be resolved.

warn_map_geocode_capped

Geocoded %d of the %d places that had not been looked up before; ...

The max cap prevented all pending values from being looked up in this request. The un-looked-up values are in not_attempted and are not stored in the cache, so the next request picks them up.

warn_map_geocoder_unavailable_data

The geocoder has no OpenAddresses data configured (OPENADDR_HOME), ...

The geocoder was loaded successfully but OPENADDR_HOME is not set and openaddr was not passed to new(). Resolution will succeed for broad place names (cities, countries) but will fail for street-level addresses. Set the environment variable or pass openaddr => $path to new().

FORMAL SPECIFICATION

The following sections use Z calculus notation to describe the exact state transitions and invariants of each public operation. Notation: ∈ (member of), ¬in; (not member of), ⊆ (subset), ∪ (union), ∩ (intersection), ∧ (and), ∨ (or), ¬ (not), ∀ (for all), ∃ (there exists), → (maps to), ≤ (less than or equal), ≠ (not equal).

FORMAL SPECIFICATION

State transitions and invariants for each operation are described below.

_resolve -- value classification and batch geocoding

Let V be the multiset of input values.

Define the following disjoint sets that partition distinct(V):

SKIPPED          = { v | v = undef, ref(v), or len(v) = 0,
                         or geocoder.valid_location(v) = false }
CACHED_RESOLVED  = { v | v in dom(CACHE) and _has_position(CACHE(v)) }
CACHED_UNRESOLVED = { v | v in dom(CACHE) and not _has_position(CACHE(v)) }
NOT_ATTEMPTED    = { v | geocoder unavailable } (excludes SKIPPED and CACHED)
PENDING          = distinct(V) \ (SKIPPED ∪ CACHED_RESOLVED ∪ CACHED_UNRESOLVED
                                  ∪ NOT_ATTEMPTED)

Partition invariant:

|SKIPPED| + |CACHED_RESOLVED| + |CACHED_UNRESOLVED|
    + |NOT_ATTEMPTED| + |PENDING| = |distinct(V)|

Cap application (MAX = the max option, default 500):

ATTEMPT        = PENDING[0 .. min(MAX, |PENDING|) - 1]
CAPPED         = |PENDING| > MAX
NOT_ATTEMPTED' = NOT_ATTEMPTED ∪ PENDING[MAX .. |PENDING| - 1]   (when CAPPED)

Geocoding step -- for each v in ATTEMPT:

BATCH = geocoder.geocode_many(ATTEMPT, on_error => 'undef')

for v in ATTEMPT:
    (lat, lon) = coordinates(BATCH(v))
    if defined(lat) and defined(lon):
        CACHE'(v)  = [lat, lon]
        POINTS'(v) = { lat, lon, backend? }
    else:
        CACHE'(v)  = 0
        UNRESOLVED_NEW += {v}

Count invariants (always hold):

skipped + cached + attempted + not_attempted = distinct(V)
resolved + unresolved = cached + attempted
resolved  = |CACHED_RESOLVED| + |ATTEMPT ∩ POINTS'|
unresolved = |CACHED_UNRESOLVED| + |ATTEMPT \ POINTS'|
lookups   <= attempted

Cache monotonicity:

dom(CACHE') ⊇ dom(CACHE)
For all v in dom(CACHE): CACHE'(v) = CACHE(v)  (existing entries not modified)

resolve_column -- row augmentation

Given the output R of _resolve applied to the column's cell sequence CELLS:

row_count  : Value -> N   where row_count(v) = |{ i | CELLS[i] = v }|
first_row  : Value -> N   where first_row(v)  = min{ i | CELLS[i] = v }

For each v in dom(R.points):
    R.points(v).rows      = row_count(v)
    R.points(v).first_row = first_row(v)

R.markers = [ { value: v, ...R.points(v) }
              | v <- sort_by(first_row, dom(R.points)) ]

R.counts.rows     = |CELLS|
R.counts.plotted  = sum{ row_count(v) | v in dom(R.points) }
R.counts.unusable = R.counts.rows - R.counts.plotted

cache_from_models -- safe extraction

cache_from_models(models, spec)
    = if ref(models) = HASH and ref(models.geo.spec) = HASH
                            and ref(models.geo.spec.places) = HASH
        then copy(models.geo.spec.places)
        else {}

The result is a shallow copy: the caller cannot modify the model store by modifying the returned hashref, but values inside the cache (the [lat, lon] arrayrefs) are shared references.

models_with_cache -- safe merge

models_with_cache(models, spec, cache, backend?, built?)
    = { ...models,
        geo: { ...models.geo,
               spec: { ...models.geo.spec,
                        places:  copy(cache),
                        backend: backend (if defined),
                        built:   built   (if defined) } } }

The caller's models hashref is never modified; a new top-level hashref is returned.

STATE DIAGRAM

The object is in one of three states, according to whether it has a geocoder yet. A geocoder given to new is one straight away; otherwise one is looked for on the first call that needs it.

  new()
    |
    +--[bad arguments, or a geocoder lacking
    |   valid_location or geocode_many]-----> croaks; no object
    |
    +--[geocoder => $g]---------------------> READY
    |
    v
+-------------------+
|    CONSTRUCTED    |  Geocoder not yet lazy-loaded.
|  _geocoder not    |  resolve_*() and geocoder_available()
|  yet called       |  may trigger a load on first call.
+-------------------+
     |          |
first call,     first call,
module OK       module absent, too old or will not
     |          load, or its new() answers nothing
     v          v
+---------+  +------------+
|  READY  |  | UNAVAILABLE|
|         |  |            |
| Geocoder|  | No lookups |
| loaded  |  | possible.  |
| and     |  | Cache still|
| working.|  | served.    |
+---------+  +------------+
     |          |
resolve_*()  resolve_*()
geocodes +   cache only;
cache        not_attempted
     |          |
     v          v
+---------+  +------------+
|  READY  |  | UNAVAILABLE|
| (no     |  | (no        |
| change) |  | change)    |
+---------+  +------------+

Once the object is READY or UNAVAILABLE that does not change: neither a module that appears nor one that goes away moves it, and the module is looked for once.

There are two ways for a call to croak and leave the state as it was. Arguments a resolve_*() call refuses are refused before any load is attempted. A module that loads and whose new() dies sends that exception to the caller and remembers nothing, so the object stays CONSTRUCTED and the next call tries again.

Resolution itself is stateless: all state is in the result hashref and the cache the caller manages.

Valid state transitions summary:

[START]          -> CONSTRUCTED      (new() with no geocoder)
[START]          -> READY            (new() given a geocoder)
[START]          -> ERROR            (new() refuses its arguments; croaks, no object)
CONSTRUCTED      -> READY            (first call, module OK)
CONSTRUCTED      -> UNAVAILABLE      (first call, no module that can be used)
CONSTRUCTED      -> CONSTRUCTED      (arguments refused, or the module's new() dies; croaks)
READY            -> READY            (resolve_*(), or arguments refused)
UNAVAILABLE      -> UNAVAILABLE      (resolve_*(), or arguments refused)

Value-level state transitions within a single resolve_* call. Each distinct value takes the first of these that applies, from the top. CACHE is the cache in the result, a copy: the cache the caller passed in is never modified.

RECEIVED
   |
   +--[undef / ref / empty]-----------> SKIPPED
   |
   +--[in cache, has position]---------> CACHED_RESOLVED --> POINTS
   |
   +--[in cache, no position (= 0)]----> CACHED_UNRESOLVED --> UNRESOLVED
   |
   +--[geocoder unavailable]-----------> NOT_ATTEMPTED
   |
   +--[valid_location() throws]--------> EXCEPTION
   |
   +--[valid_location() = false]-------> SKIPPED
   |
   +--[within cap]--------------------> ATTEMPT
   |                                        |
   |                                        +--[geocoder returns position]--> RESOLVED_NEW --> POINTS, CACHE==[lat,lon]
   |                                        |
   |                                        +--[geocoder returns no position]--> UNRESOLVED_NEW, CACHE==0
   |                                        |
   |                                        +--[geocoder throws]--> EXCEPTION
   |
   +--[beyond cap]-------------------> NOT_ATTEMPTED

EXCEPTION is the whole request failing, on purpose: it goes to the caller, who has no result and a cache exactly as it was, so nothing can be half-written.

AUTHOR

Nigel Horne <njh@nigelhorne.com>

LICENCE AND COPYRIGHT

Usage is subject to the GPL2 licence terms. If you use it, please let me know.

To install Database::BI, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Database::BI

CPAN shell

perl -MCPAN -e shell
install Database::BI

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	Go to GitHub issues (only if GitHub is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

VERSION

SYNOPSIS

DESCRIPTION

The cache

Without the geocoder

CONSTRUCTOR

new

API SPECIFICATION

INPUT

OUTPUT

SUBROUTINES/METHODS

resolve_column

API SPECIFICATION

INPUT

OUTPUT

MESSAGES

EXAMPLE

resolve_values

API SPECIFICATION

INPUT

OUTPUT

EXAMPLE

resolve_column

geocoder_available

API SPECIFICATION

INPUT

OUTPUT

EXAMPLE

cache_from_models

API SPECIFICATION

INPUT

OUTPUT

EXAMPLE

models_with_cache

API SPECIFICATION

INPUT

OUTPUT

EXAMPLE

shared_cache_from_models

API SPECIFICATION

INPUT

OUTPUT

EXAMPLE

models_with_shared_cache

API SPECIFICATION

INPUT

OUTPUT

EXAMPLE

COMMON PITFALLS

ENCODING

DIAGNOSTICS

FORMAL SPECIFICATION

FORMAL SPECIFICATION

_resolve -- value classification and batch geocoding

resolve_column -- row augmentation

cache_from_models -- safe extraction

models_with_cache -- safe merge

STATE DIAGRAM

AUTHOR

LICENCE AND COPYRIGHT

Module Install Instructions

Keyboard Shortcuts