/ 
Sim-OPT-0.925
/ Sim::OPT::CalcMarginals

NAME

Sim::OPT::CalcMarginals - calculate marginal and level-specific response distributions from Sim::OPT results

SYNOPSIS

Using a configuration file from Perl

The normal Perl API is run_config. The configuration is kept in a separate Perl file and the module performs the calculation described by that file:

use Sim::OPT::CalcMarginals qw(run_config);

my $result = run_config('marginals_config.pl');

$result is a hash reference containing summary information and the paths of the three files that were created.

Running from the command line

If Sim::OPT is installed in Perl's normal module search path, the same configuration file can be run directly from the shell with:

perl -MSim::OPT::CalcMarginals=run_config \
     -e 'run_config(shift)' marginals_config.pl

If you are running from an unpacked Sim::OPT source tree and the module has not yet been installed, add that distribution's lib directory to Perl's search path. From the top directory of the Sim::OPT distribution, use:

perl -Ilib -MSim::OPT::CalcMarginals=run_config \
     -e 'run_config(shift)' marginals_config.pl

The last argument is the configuration filename. Relative input and output paths in that configuration are interpreted relative to the directory that contains the configuration file, not relative to the shell's current working directory.

This module does not itself inspect @ARGV. A future command-line wrapper can therefore call run_config without changing the calculation code or the configuration-file format.

Calling the calculation directly from Perl

Instead of a configuration file, a Perl program may pass a configuration hash reference directly to run:

use Sim::OPT::CalcMarginals qw(run);

my $result = run({
    file       => 'amtry-0_totres.csv',
    levels     => {
         1 => 5,  2 => 5,  3 => 5,
         4 => 3,  5 => 3,  6 => 3,  7 => 3,
         8 => 3,  9 => 3, 10 => 3, 11 => 3,
        12 => 3, 13 => 3, 14 => 3, 15 => 3,
        16 => 3, 17 => 3, 18 => 3, 19 => 3,
        20 => 3, 21 => 3, 22 => 3,
    },
    column     => 2,
    divisions  => 100,
    best       => 60,
    worst      => 80,
}, '.');

The optional second argument is the base directory from which relative paths are resolved. If it is omitted, the current directory is used.

DESCRIPTION

Sim::OPT::CalcMarginals analyzes a Sim::OPT-style CSV results file in two related ways.

Each non-empty input row is expected to encode the state of the discrete variables in CSV field 0. A typical field looks like:

1-5_2-5_3-4_4-1_5-3_..._22-3

Each variable-level token means that the indicated variable took the indicated discrete level on that row. The configuration says which variables are to be considered and how many levels each variable is allowed to have.

The module then calculates:

1. Empirical marginal distributions of the discrete variables

For every requested variable, the module counts how often each of its levels occurs, independently of the levels taken by the other variables. It also expresses those counts as percentages of all observations counted for that variable.

For example, if variable 8 has levels 1, 2, and 3, the marginal result answers:

How often is variable 8 at level 1?
How often is variable 8 at level 2?
How often is variable 8 at level 3?

These are the ordinary empirical marginal distributions of the discrete variables encoded in field 0.

2. Response distributions conditional on every variable level

The configuration key column selects one numerical CSV field, using zero-based indexing. Thus column => 2 means the third CSV field.

For every requested variable and every one of its levels, the module collects the selected numerical response values observed while that variable was at that level. It divides the interval between best and worst into the requested number of equal bins and counts how many response values fall into each bin.

Thus a row labelled 8-2 in a distribution file means:

the distribution of the selected numerical response among observations
for which variable 8 was at level 2

It does not mean the marginal distribution of variable 8 itself; that is reported separately in marginals_*.csv.

CONFIGURATION FILE

A configuration file is an ordinary Perl file whose final value is a hash reference. For example:

{
    file => 'amtry-0_totres.csv',

    levels => {
         1 => 5,  2 => 5,  3 => 5,
         4 => 3,  5 => 3,  6 => 3,  7 => 3,  8 => 3,  9 => 3,
        10 => 3, 11 => 3, 12 => 3, 13 => 3, 14 => 3, 15 => 3,
        16 => 3, 17 => 3, 18 => 3, 19 => 3, 20 => 3, 21 => 3,
        22 => 3,
    },

    column     => 2,
    divisions  => 100,
    best       => 60,
    worst      => 80,
};

The terminating semicolon is recommended. Because the file is loaded as Perl code, it must evaluate to the hash reference shown above.

file

The input CSV filename. An absolute path is used as written. A relative path is resolved relative to the directory containing the configuration file when run_config is used.

levels

A hash reference mapping each requested variable number to its number of allowed levels. For example:

levels => { 1 => 5, 2 => 5, 3 => 3 }

means that variables 1 and 2 have levels 1 through 5, while variable 3 has levels 1 through 3.

Every requested variable must be present in field 0 of every processed row. A level outside the declared range is treated as a configuration/data mismatch and causes the calculation to stop with an explanatory error. This validation is intentional: it prevents a real level from being silently omitted.

column

Zero-based index of the CSV field containing the numerical response whose level-specific distributions are required.

For example:

column => 2

selects the third comma-separated field.

A non-numeric value in the selected response field is not used in the response histograms. The row is still used for the discrete-variable marginal counts, provided its field-0 variable encoding is valid.

divisions

The number of equal histogram bins spanning the interval between best and worst.

For example, with:

best      => 60,
worst     => 80,
divisions => 100,

the numerical interval has width 20 and each bin has width 0.2. The output uses bin centres, therefore the centres are 60.1, 60.3, 60.5, and so on up to 79.9 when the objective is minimization.

Each in-range numerical observation is assigned to exactly one integer bin. The upper numerical endpoint is explicitly assigned to the last internal bin. This avoids the overlapping-boundary floating-point problem that can occur when every bin boundary is repeatedly recomputed and compared independently.

best and worst

These are semantic endpoints of the response interval, not merely names for its numerical minimum and maximum.

The module infers the optimization direction automatically:

best < worst    minimization
best > worst    maximization

For example:

best  => 60,
worst => 80,

means that smaller response values are better and the objective is minimization. Conversely:

best  => 80,
worst => 60,

means that larger response values are better and the objective is maximization.

Internally, histogram binning is always performed from the numerical lower endpoint to the numerical upper endpoint. In the output distribution files, however, the bins are currently ordered from best toward worst. Thus a minimization problem with best 60 and worst 80 is written from low to high, whereas a maximization problem with best 80 and worst 60 is written from high to low.

Values numerically outside the interval bounded by best and worst are not included in the response histograms. They are counted separately in the terminal summary as lying on either the better-than-best side or the worse-than-worst side, according to the inferred objective direction.

output_dir

Optional directory for the generated CSV files. If omitted, results are written next to the configuration file. A relative output_dir is resolved relative to the configuration file's directory.

The directory must already exist.

HOW THE CALCULATIONS WORK

Discrete marginal distributions

For each non-empty row, the module parses field 0 into a mapping from variable number to observed level. For every variable named in levels, it increments one counter for the observed level.

If variable v has L declared levels, the count for level l is:

count(v,l) = number of processed rows in which variable v is at level l

The percentage written for that level is:

100 * count(v,l) / sum_over_all_levels(count(v,*))

Consequently, the level percentages for each variable sum to 100%, apart from normal floating-point formatting effects.

Level-specific response histograms

Let the numerical endpoints be lower and upper, irrespective of whether they correspond to best or worst. With divisions = D, the bin width is:

width = (upper - lower) / D

For an in-range response value x below the upper endpoint, its internal bin index is:

int((x - lower) / width)

A value exactly equal to upper is assigned to bin D - 1. The centre of internal bin b is:

lower + (b + 0.5) * width

For each requested variable-level combination, the module increments the appropriate bin whenever the selected response is numeric and lies within the configured interval.

Percentage response distributions

The raw count histogram for each variable level is normalized independently. If N(v,l) is the number of in-range numerical responses observed while variable v is at level l, each percentage bin is:

100 * bin_count(v,l,b) / N(v,l)

Therefore each non-empty row of distributions_percentages_*.csv sums to 100%. This makes curves for levels having different numbers of observations comparable by shape rather than by sample size.

OUTPUT FILES

For an input file named name.csv, three files are written.

marginals_name.csv

Contains the empirical marginal distribution of the discrete variables, with four columns:

variable,level,count,percentage

Each row gives one level of one variable. count is the raw number of observations at that level and percentage is its share of that variable's observations.

This file is suitable, for example, for bar plots showing how frequently the different levels of a variable occur.

distributions_counts_name.csv

Contains the level-specific response histograms as raw counts.

The first column is labelled variable-level. The remaining columns are the histogram bin centres. Each subsequent row, such as 8-2, contains the raw bin counts for the selected response when variable 8 is at level 2.

This file is useful when absolute sample counts matter.

distributions_percentages_name.csv

Has the same arrangement as distributions_counts_name.csv, but every variable-level histogram is normalized independently to percentages.

This is normally the most useful file for comparing distribution shapes. One can make one graph per variable and superimpose the rows for all levels of that variable. For example, a five-level variable can be shown as five lines on the same plot, all sharing the bin-centre scale from the first row.

For a line plot in a spreadsheet, the bin-centre row can be used as the horizontal category labels. For an XY plot, the same bin-centre row can be used as the numerical X values.

TERMINAL SUMMARY

After a run, the module prints a concise diagnostic summary including:

  • inferred objective direction;

  • configured best and worst values;

  • number of processed rows;

  • number of numeric responses in the selected CSV column;

  • numerical interval and bin width;

  • counts of responses outside the configured interval on the better-than-best and worse-than-worst sides; and

  • paths of the three generated files.

The same information is also available programmatically from the hash reference returned by run or run_config.

RETURN VALUE

run and run_config return a hash reference containing:

objective
best
worst
lower
upper
divisions
bin_width
processed_rows
numeric_responses
outside_better
outside_worse
marginal_file
counts_file
percentages_file

This allows other Sim::OPT code to use the calculation without parsing the human-readable terminal messages.

FUNCTIONS

run_config($path)

Loads the configuration file at $path, resolves relative paths from the configuration file's directory, performs the complete calculation, writes the three output files, prints the diagnostic summary, and returns the result hash reference described above.

This is the recommended entry point when the run specification is stored in a separate file.

run($config, $base_directory)

Runs the calculation directly from a configuration hash reference.

$base_directory determines how relative input and output paths are resolved. If it is omitted or empty, the current directory is used.

load_config($path)

Loads a configuration file and verifies that it returns a hash reference. In list context it returns the configuration hash reference and the directory containing the configuration file.

Most callers should use run_config instead of calling load_config directly.

INPUT FORMAT NOTES

The current reader deliberately follows the simple format used by the existing Sim::OPT result files. It separates CSV fields on literal commas and parses field 0 as underscore-separated variable-level tokens.

Consequently, this version is intended for simple comma-separated files whose fields do not themselves contain quoted commas. If full RFC-style CSV quoting is required in the future, the input layer should be replaced with a dedicated CSV parser without changing the marginal or histogram calculations.

ERRORS AND VALIDATION

The module stops with an explanatory error when, among other things:

  • the configuration is not a hash reference;

  • the input file is missing;

  • levels is missing or invalid;

  • column or divisions is invalid;

  • best and worst are non-numeric or equal;

  • a requested variable is absent from field 0 of an input row;

  • an observed variable level exceeds the declared level range; or

  • the requested output directory does not exist.

These checks are intended to make configuration mistakes visible rather than silently producing incomplete distributions.

AUTHOR

Gian Luca Brunetti, <gianluca.brunetti@polimi.it>

ACKNOWLEDGEMENTS

The initial design and implementation of CalcMarginals were developed by Gian Luca Brunetti with assistance from AI.

COPYRIGHT AND LICENSE

Copyright (C) 2008-2025 by Gian Luca Brunetti, gianluca.brunetti@gmail.com. This software is distributed under a dual licence, open-source (GPL v3) and proprietary. The present copy is GPL. By consequence, this is free software. You can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 3.