Revision history for Database-BI
0.020.0 Wed Nov 4 09:25:26 PM EST 2026
[ Enhancements ]
- The line graph panel has Lines by, which draws one line for each value of a
column (series=). It was reachable only by writing the URL.
- Canvas: Add current view offers Lines by for a line card.
- /histogram takes scheme=, which fills the bars with the first colour of a
categorical scheme. The histogram panel and Add current view offer it as
Bar colour, and a canvas histogram card keeps it.
- Canvas: Add current view offers Colour by, Colour scheme, Bubble size and
the radii for a scatter card, as the scatter panel does.
- /treemap takes color=, which shades each tile by a second column, with
color_agg= (default avg) and color_scheme= (sequential, default YlOrRd).
A legend shows the scale. The treemap panel, Add current view and a
canvas treemap card all offer it. It was a 400 before.
- /combo takes scheme=: the bars take its first colour and the line its
second. The combo panel, Add current view and the card offer it.
- HTML::D3 0.29 is now required. Besides the two above, it escapes the
text in the stacked, combo and treemap tooltips, where a label holding
markup ran when the chart was hovered.
- Geocoding goes through a Geo::Coder::List chain: Geo::Coder::Free's
local data first, then OpenStreetMap's Nominatim (Geo::Coder::OSM) for
the places it does not know. Requests are kept to one a second by
LWP::UserAgent::Throttled, from a client named Database-BI. A map page
asks Nominatim about at most map_geocode_online_max places (default 20)
and leaves the rest for the next visit or script/bi-geocode.
map_geocode_online => 0 keeps every lookup local. A Nominatim that
fails is not taken to mean the place does not exist: the place is
tried again later. Geo::Coder::List, Geo::Coder::OSM and
LWP::UserAgent::Throttled are new recommends.
- Geo::Coder::List keeps the answers it finds in a cache under
<model_dir>/geocode-cache, shared by /map and script/bi-geocode, so a
place Nominatim found is not asked again for a month. The local data
is asked through the same cache, with or without Nominatim, and a place
whose local lookup fails is left for later rather than ending the
page. Nominatim is asked over https, one request a lookup
(Geo::Coder::OSM asked over http, and was redirected).
- /map takes loc= more than once, for a place written across columns:
loc=Address&loc=Country looks up "Brezje 8, Cerknica, Slovenia". The
map panel has a Followed by column (guessed when one is called
Country, and remembered with the other), canvas map cards keep the
columns as a list, and script/bi-geocode takes --col more than once.
- script/bi-geocode takes --offline and --online-max.
- When a geocoded map cannot plot some rows, the note that says so opens
onto a list of their places: how many rows each stands for, why (no
place, not a place name, not found, or not looked up yet), and a link
to those rows in the table it came from.
- The note that some rows of a map could not be plotted names only the
reasons that apply, each with its rows ("8 whose place was not found"),
rather than every reason there could be. Beside it, one link opens
all those rows in the table at once.
- A new filter operator, in, keeps the rows whose cell is any value of a
JSON list: f=place:in:["Paris","Rome"]. It is what that link uses.
- With HTML::OSM 0.13, every map has a scale bar, and a map coloured by
series=, color= rules or size= has its key drawn on the map. The key
lists only the colours of the markers in view, so it changes as the
map is zoomed and moved; bubble sizes are always listed. A map with
no zoom= opens on every marker, as the map panel says; it used to open
at zoom 12 on the markers' midpoint, which could show none of them.
HTML::OSM 0.13 is now the recommended minimum: with an older one a map
page says which version it needs, as it did below 0.12.
- A map no longer waits for its places to be looked up before it is
shown. It draws what the place cache knows, and a progress bar names
each place as it is looked up ("Looking up 5 of 10: Brezje 8,
Cerknica, Slovenia"), with a Cancel button. Each answer is saved as
it comes (POST /geo/lookup), so a run that is cancelled or interrupted
keeps what it found, and the map is drawn again when the last place is
in. After Cancel the places left are listed as not looked up yet, with
a link to look them up. The progress bar asks one place at a time, so
map_geocode_online_max no longer leaves places for the next visit, where
each paid for a slow local lookup again. geocode=wait keeps the old
behaviour, which a client that is not a browser still gets;
geocode=cached looks nothing up.
- The application's secret is random for each start unless the config
sets secrets => [...]. Mojolicious used the application's name, which
anyone can read; it now signs the places a map page lists.
[ Bug fixes ]
- A CSV whose first data row had a value in a column that is mostly
empty showed only the rows with a value there: a bank export whose
first transaction had a cheque number showed 4 of its 362 rows. The
column rows are keyed on was the first one filled in the first row,
and Database::Abstraction drops a row whose key is empty. It is now
the column with the fewest empty cells in the whole file, read as
Database::Abstraction reads it: a row such as "Joe\"s Bar" no longer
ends the count, and a line that will not parse is skipped.
- DataSource read no rows when its caller had set $/ (slurp mode, or a
record length): the header was read as the whole file. The methods
that read a file now use "\n" whatever the caller's $/ is.
- On Windows a path: spec reached through a symlink was filed in the
model store under the link's name, not the file's, because realpath
there leaves links alone: definitions saved through the link were not
found from the file. canonical_spec now follows the links first.
- Deleting a canvas card rewrote a bookmark's cv= of more than nine
digits, which names no card, as cv=1e+20. It is left alone, and a
search or card index that is not one is no longer renumbered.
- A map marker's popup showed only the first of the rows at its place:
"3 rows" and one event. It now lists them all, a value every row
shares once and the columns that differ as a small table. Past
map_popup_rows (default 10) the rest are counted, with a link to them
in the table, and the table scrolls, so a busy place does not cover the
map. The same holds for rows at one coordinate.
- A map card on a canvas is coloured and sized as the map it was saved
from. It stores color and size in its encode, which only the map page's
options were read for, so a card drew plain pins.
- Calc::date_normalise and the date functions in calculated fields gave
the caller back a changed $!: on BSD, mktime and localtime can
leave errno set when they succeed (a CPAN Testers failure in t/unit.t).
Fixes https://www.cpantesters.org/cpan/report/ef94d1fc-d7aa-11f1-839c-f58cc80b69bd
- script/bi-geocode could not open a path: spec: it looked for a table
named after the whole file name, extension included.
- /scatter no longer saves scheme= in the chart it hands to a canvas when
series= is absent, where the scheme is ignored and unchecked.
- A scatter added to a canvas from /scatter kept only x and y: the card now
colours by series and sizes bubbles as the page does, and the page saves
min_r= and max_r= with it. A bad saved scheme draws the default colours.
- A line chart added to a canvas from /graph?series= drew every row as one
line through all the groups. The card now draws one line per value.
- script/bi-geocode died on every run ('option "max" must be a positive
integer, not "0"'), and its summary counted cache hits from a key that
does not exist and compared rows with places.
- A canvas map card whose saved color rules held something other than a
rule (PUT and import store a card as sent) was a 500; such entries are
now ignored. A card whose encode, options or filters are not of the
right shape is drawn as if they were absent, rather than failing.
- cv=abc on a canvas was read as card 0, which was then drawn unfiltered
as the source of the cross-filter (and perl warned). Only a whole
number names a card.
- Removing a text card left a bookmark's cv= naming the removed card,
which then named the card that moved into its place. It is dropped.
- A negative offset in a page of rows is read as 0, and a map's unplotted
rows ignore anything in the rows that is not a row.
- A column-listing error on a table page lost only the last "at FILE line
N" of a rethrown error; it now goes through the same tidying as every
other error shown to a reader.
[ Tests ]
- t/edge_cases.t attacks the id choice (escaped quotes, unparseable and
stuck reads, a read failing mid-file, 50,000 rows, 2,000 columns, a
megabyte of quotes, the caller's $/, descriptor leaks and a full
descriptor table), bookmark renumbering, cv= values, stored cards of the
wrong shape and map popups (hostile cells, 100,000 rows at a marker).
- t/function.t covers the geocoding progress run, the remaining
DataSource, Calc and Geocode helpers, and the Dashboard helpers and
canvas, lookup, extraction and /geo/lookup routes no part reached.
- t/function.t tries _follow_links on an NT-form link target only where
readlink gives the target back as written: Cygwin turns a target with
backslashes into a Windows path (a CPAN Testers failure on Cygwin).
Fixes https://www.cpantesters.org/cpan/report/4d672bda-d7ac-11f1-8016-ac2316d9a04a
- The geocoding tests pass without the recommended modules, as on CI:
t/transition.t stands in for Geo::Coder::List, t/unit.t raises the
"needs Geo::Coder::List" message with it hidden, and t/integration.t
counts the geocoder as installed only with Geo::Coder::List as well.
- t/scatter.t: series=, scheme=, size=, min_r= and max_r= on /scatter and
on a canvas scatter card, and the scatter fields of both panels.
- t/graph-series.t: series= on /graph, on a canvas line card and in both
panels.
- t/histogram-scheme.t: scheme= on /histogram, on a canvas card and in both
panels.
- t/geocode-online.t: the geocoding chain against a stand-in Nominatim
(t/lib/BI/Test/Nominatim.pm), so no test reaches the network.
- t/treemap-combo-colour.t: color=, color_agg= and color_scheme= on
/treemap, including the Other tile; scheme= on /combo; both canvas cards
and all four panels.
0.019.0 Tue Nov 3 11:46:08 AM EST 2026
[ Enhancements ]
- Stored lookups: POST /lookups/save adds the columns of another file under
data_dir to every matching row, on every page, chart, export and canvas.
nolk= skips one for a view; lk= applies one stored for another dataset.
- Field extraction: rx() in a calculated field takes a value out of a column
with a regular expression, and POST /extract/save stores a pattern whose
named groups become columns everywhere.
- Accelerated datasets: script/bi-extract builds a SQLite copy of a large
file under <model_dir>/extracts/, which the pages read while the file is
unchanged. Once it changes, the file is read and the page says so.
- A calculated field can read a cross-row column (cs=).
- MariaDB and MySQL: mariadb:server/table opens a table of a server named in
mariadb_servers, anywhere a source is taken. The spec holds only the name.
- Export PDF on every chart page.
- Canvas: text cards, for titles and explanations. Cards save as SVG, PNG
and PDF, showing the whole chart and a pie's legend.
- Saved charts carry "Database::BI" at the bottom right.
- A website, in website/html/.
- script/database-bi FILE starts the server showing that database.
- Dark mode, chosen on a new Preferences page. Saved charts, printed pages
and embedded canvases stay light.
- Every feature can be reached from the pages: Lookups and Extract fields
panels, a canvas Share panel (link, embed code, JSON download) with Import
canvas to read it back, and Forget remembered columns on the map panel.
- A refused request in a browser shows the reason on a page with a way back.
- POST /geo/save takes repeated col=/kind= pairs, and replace=1.
- Canvases in the main navigation opens a new /canvases page.
- Database::Abstraction 0.54, File::Temp 0.2308 and HTML::D3 0.27 are now
required.
[ Bug Fixes ]
- Security: error replies that repeat a parameter were sent as HTML, so a
crafted link such as /bar?range=<script> ran script on this site.
scheme= reached a chart's script unchecked. A cell holding
</script><script> ran on /bar, /pie, /stacked and /graph.
- On Windows, a symlink was not followed when checking that a file is under
data_dir, so a link could point a lookup outside it.
- Saved definitions for a path: dataset reached through a symlink were not
found by its /open page.
- week() was 0 on Windows.
- A saved definition that could no longer be applied made every page of its
dataset a 400, so the panel that mends it could not be reached. The table
page is now drawn without it, with a note saying what is wrong.
- A saved group's column was missing from the table page and canvas cards,
and groups saved from the panel never matched a row.
- The map panel's Remember columns added to the saved columns rather than
replacing them.
- A panel reopened after a save could list what was there before.
- A SQLite .db file holding a table named after the file was read as text.
- A SQLite file could not be opened when its first table's name had
characters other than letters, digits and underscores.
- /open, /join and /combine left out calculated fields and span= time
buckets. cs= on the table page could not read them.
- A filter on a cross-row column, a calculated column or a time bucket found
no rows.
- lt, le, gt and ge treated a blank or a word as zero; they now compare
numbers only, and read $1,234.56 and (1,234.56).
- A column of codes (SKU123) or dates was read as numbers; 1e3 was read as 13.
- Merge hung on BSDs with the File::Temp that Perl 5.28 ships.
- hypnotoad script/database-bi never served the application.
- Page links on a table opened with no other query were broken.
- A file with no header row split quoted fields and dropped empty ones.
Sorting did nothing on such files, on XLSX and Berkeley DB, and on columns
such as "First Name". XLSX and Berkeley DB are now read by
Database::Abstraction.
- /api/range ignored joins and combines.
- DataSource->updated() now gives the file's mtime.
- /embed/:id drew no charts. A canvas card's CSV download always failed.
- Writes to analyses.json reported success when nothing was saved; they now
answer 503. A store that could not be parsed was replaced by the next
save; it is now kept as <file>.corrupt.<time>.
- A canvas or models.json of the wrong shape was a 500. PUT, PATCH and
import now refuse or drop what POST does.
- /export and /folder hung on a bad span= or calculated field.
- 500s and exception pages now give a message instead: an unreadable data
or staging directory, SQLite export without a driver, a JSON URL that
holds no rows, a failing map renderer, a sorted export or a page on a
read-only backend, an unknown heatmap scheme.
- Errors no longer show readers Perl source paths, line numbers or @INC.
- An absolute data_dir defeated the check that keeps analyses.json out of it.
- Non-numeric page=, per_page=, topn= and bins= are ignored.
- Line chart exports cut off rotated labels, and saved charts lost their
fonts.
- Smaller fixes in Insights, calculated fields, Summary, Ask, the map and
links built from non-ASCII text.
- Tutorial: new chapters on navigation, Ask, bi-geocode and the settings
file, and the existing ones brought up to date.
0.018.0 Wed Oct 28 08:41:28 PM EDT 2026
[ Enhancements ]
- Cross-row columns: cs=<col>:<fn>:<alias>[:<arg>] with by= and order=
adds a running total, moving average, rank, z-score or share of the
total per row. A Cross-row panel builds them; charts and exports
carry the new column.
- Canvas cards can carry their own cross-row plan (cross_row), replacing
the page's, run on the card's own filtered rows. Add current view
collects several columns; the card header's Edit cross-row edits the
plan as chips and warns when a rename affects a plotted column.
PATCH /api/analyses/:id/visuals/:visual replaces a card's plan.
- Word cloud (GET /wordcloud, and canvas cards with kind=wordcloud),
drawn with HTML::TagCloud. Each word links to its rows, or on a
canvas cross-filters to them.
- Relative time ranges: ?range=<name>&rcol=<column> on every data route
and saved canvas (last_24_hours, today, yesterday, last_7_days,
last_30_days, last_90_days, this_month, last_month, this_quarter,
last_quarter, this_year, last_year). An unknown name or missing rcol=
is a 400. The dashboard filter bar and canvas toolbar have a Time
range menu.
- The filter UI adapts to the column's detected type: date pickers,
numeric operators, or string operators.
- New "between" filter operator (col:between:YYYY-MM-DD:YYYY-MM-DD);
lt/le/gt/ge compare ISO dates as dates.
- New measures for the pivot, column statistics and canvas cards:
median, stdev, first, last, range and p<N> percentiles.
- Time buckets: span= groups a chart or pivot by hour, day, week, month,
quarter or year; /graph?agg= plots one point per bucket.
- Stored time patterns accept time=, e.g. 27/10/2026 14:05.
- bi-alert: --suppress, --recovery (exit 3), --notify-cmd, and several
analyses and thresholds in one run.
- Response compression, ETag/304 and HTML minification via
Mojolicious::Plugin::Buffer (new prerequisite); downloads opt out.
- Optional page cache: page_cache_dir and page_cache_duration store
/catalog and /tutorial in a CHI File cache.
- Internal: every data route's filter stage is one function,
_reduce_records.
- Main navigation links to Canvases, Catalog and Open file on every page.
- Chart, pivot, stats and insights pages show a "Drawn from:" strip naming
the time range, filters, cross-row columns, combined files and
de-duplication, with a link back to change them.
- Every page has a full breadcrumb, from Home to the page itself, each
step named after what it shows: Home > sales > Bar chart: amount by
region > sales (region = East) > Pie chart: Count by region. The
trail is read from each page's back links, so it is no longer cut at
two levels; a file opened from the browser sits under its folder, and
a canvas is named. Back links to another site are not followed, and
the trail is at most eight steps.
[ Bug Fixes ]
- Two charts on one canvas no longer collide; the second card draws.
- A card's cross-row column no longer appears in the cards beside it.
- A card naming a column no plan produces is refused, instead of
printing a heading over empty cells.
- A day bound on a timestamp column (e.g. le 2026-10-27) now includes
the rows stamped on that day.
- A stale ?dd= bookmark no longer makes canvas marks unfocusable.
- Canvas help now describes what the page actually does.
- Charts, pivots, stats and insights of a combined (c=), de-duplicated
(d=1) or inner/outer-joined (jtype=) view now draw from the rows the
table shows; before, the panels dropped those parameters, and d=1 was
ignored by every chart route. Export keeps jtype= too.
- Opening any toolbar panel closes all the others: the histogram panel
left Cross-row and Word cloud open, and Merge, Combine and Export left
chart panels open and their own buttons disabled.
- The Word cloud, Cross-row and Ask toolbar buttons are styled like the
others; all of them now share one btn-tool class.
- Toolbar and filter-bar controls on the same row now line up top and
bottom: they share one height and are centred, rather than aligned on
their text baselines.
- The pivot panel's Limit to top, ranked by, Time bucket and "of column"
fields are the same size as its Row, Column and Measure fields.
- Canvas controls line up: the Time range bar, parameter bar and toolbar
share the table toolbar's height, the Time range menus and Apply
button are styled, and card-header buttons are one height.
- Every dropdown and text field is white with one border colour, from a
single rule in the layout; the table page's filter bar and pivot fields
showed the browser's grey, and pivot fields turned near-black when the
OS was in dark mode.
- An unknown canvas (/a/<id>) is an HTML page with the breadcrumb and
navigation, not a line of plain text.
- Folder view levels keep the link back to the table they came from,
and its crumb is an absolute link.
- A missing page is a 404 with the breadcrumb and navigation, not
Mojolicious's bare page. /view/<table> for a table that does not
exist is now a 404 naming it, rather than the home page with an
error that showed the data directory.
- A view that is both joined (j=) and combined (c=) shows both, from
/join or /combine: /combine ignored j=, and Merge after Combine
dropped c=. The table now shows the rows its charts draw from.
- Tests: the read-only model_dir and export-write-failure tests skip
where chmod cannot revoke access (Cygwin, Windows ACLs, root), using
Test::Permissions; they failed on Cygwin (CPAN Testers).
Fixes https://www.cpantesters.org/cpan/report/b2c11820-d297-11f1-a488-efeb21b3859a
- Tests: t/unit.t no longer segfaults at exit under Devel::Cover on
perl 5.42 (the CI coverage step); its LLM-response helper was a
lexical "my sub".
- Text::CSV is a prerequisite: script/bi-alert's fallback reader needed
it and it was not declared.
- Documentation: cross-row columns (cs=, by=, order=), compression and the
page cache (page_cache_dir, page_cache_duration), breadcrumbs and 404
pages, j= with c= on /join and /combine, and an API SPECIFICATION for
every public function. Util.pm's licence now matches the
distribution's (GPL2). Fixed POD that rendered "= >" and "&>".
- The templates and controller comments are ASCII again.
0.017.1 Sun Oct 25 09:23:34 PM EDT 2026
[ Bug Fixes ]
- script/database-bi: shebang is perl again; the script starts morbo
itself when given no arguments. 0.017.0's morbo shebang caused each
worker to spawn another morbo, growing PERL5LIB until exec failed with
E2BIG.
- GET /map: HTML::OSM older than 0.12 now produces a warning page instead
of a 500. 0.017.0 passed search => 0 and max_zoom unconditionally;
older HTML::OSM rejected the unknown keys.
- GET /map: ?agg= removed (it was accepted and ignored).
- /join: schema-type mismatch warnings now fire correctly. DataSource
delegates schema() to its inner backend; the controller compares types
across join steps and reports differing columns.
- data_dir may be absolute. A relative value is still resolved against
the application home; an absolute one was previously appended to it.
- Database::BI::Extract: removed unused _get_or_build_extract; corrected
class docs (the web app does not materialise extracts).
- script/bi-alert: corrected docs -- it writes to STDOUT, not email.
--dry-run is accepted for compatibility only.
- Parallel test runs (HARNESS_OPTIONS=j10) are now deterministic. Upload
staging moved to a per-deployment uploads_dir config key (.uploads by
default); test files each use their own temp directory.
- Browser fixture: selector quoting no longer depends on the JSON
backend's allow_nonref setting.
- t/integration.t: %INC restored after Test::Without::Module block,
eliminating "Subroutine ... redefined" noise.
- Every page now sets <html lang> from the resolved language. 36 render
calls were missing the stash key.
- Geo::Coder::Free minimum raised to 0.48 throughout (floor constant,
error messages, docs). The floor was 0.47, but 0.48 introduced the
per-lookup read counts the application uses.
- _resolve_language now falls back to the default language when the
resolved code has no template directory, as documented.
[ New Features ]
- GET /map: ?size=<col> draws bubble markers scaled 6-30 px.
- GET /map: ?series=<col>&scheme=<name> colours markers by category with
a legend. Mutually exclusive with ?color=.
[ Documentation ]
- Complete /map parameter reference; HTML::OSM 0.12 floor;
Nominatim search always suppressed.
- "Command-line tools" section covering all three scripts.
- Canvas, analyses API, sharing/embedding, catalog, insights, ask,
summary and geo endpoints (previously README-only).
[ Tests ]
- t/launcher.t: asserts perl shebang on every shipped script.
- t/map.t and t/integration.t: drive both renderer states via the
controller's _map_renderer_version probe.
- t/edge_cases.t: covers uploads_dir resolution (absolute, relative, unset).
- t/map.t: covers the canvas in both renderer states.
- Every test file now gives the app its own model_dir temp directory.
- t/browser-canvas-map-degraded.t: browser test for the degraded map card.
- Map-rendering assertions in t/map.t, t/integration.t and t/unit.t now
require HTML::OSM 0.12 (not merely installed).
- browser.yml: no-renderer, no-renderer-canvas, no-geocoder and renderer
jobs added; each pins or verifies one optional module.
- BI::Test::Browser: config keys written before app startup; fixture
builds a language home for non-shipped locales; app_setup coderef
supported.
- t/browser-fixture.t: checks fixture behaviour without a browser.
- t/browser-map-page-renderer.t, t/browser-map-geocoder.t,
t/browser-canvas-map-geocoder.t: browser coverage for optional-module
degraded states.
- t/browser-offline-assets.t: skips map assertions below HTML::OSM 0.12;
adds D3-drawn bar assertions.
- t/recommended-versions.t: asserts tests and application code gate
recommended modules at the declared version floor.
- t/browser-language.t: browser coverage for configured, default and
fallback languages.
- .github/workflows/actionlint.yml: lints all workflows on push/PR
(pinned actionlint 1.7.12 + shellcheck 0.10.0); fixed 24 shellcheck
findings in existing workflows.
- browser.yml: keyboard job handles missing HTML::OSM without failing;
browser-map-panel.t no longer dies when a card has no script.
- Every browser job publishes a coverage report to the job summary.
- t/workflow-browser.t: guards the renderer job's file list.
0.017.0 Sat Oct 24 06:12:53 PM EDT 2026
[ New Features ]
- GET /geo/list, POST /geo/save, POST /geo/delete: coordinate-column
preference CRUD. Saves the user's choice of lat=, lon=, or loc=
column for a given l= spec in .bi/models.json under geo.<spec>.columns.
The Map panel now fetches /geo/list on open and pre-populates the
column selects from the saved preferences, falling back to the existing
name-heuristic when none are saved. A "Remember columns" checkbox
fires POST /geo/save on form submit.
- GET /map: new optional URL params.
?zoom=N (0-19): passes an initial zoom level to HTML::OSM instead of
auto-fitting to the marker bounds.
?tiles=name: selects a named tile source from the tile_sources config
key (unknown name returns 400). Arbitrary URLs are never accepted.
?heat=1: adds a Leaflet heatmap layer over the regular markers using
HTML::OSM->add_heatmap (present in HTML::OSM >= 0.11).
?weight=<col>: numeric column that drives heatmap point intensity;
values are normalised to [0,1]; defaults to equal weight 1 when
absent.
?max_zoom=N (0-28): sets the Leaflet tile layer maxZoom option so the
user cannot zoom past what the tile server covers. Also readable
from the map_max_zoom config key; the URL param takes precedence.
Requires HTML::OSM >= 0.12.
- GET /map: Nominatim live-search box is now suppressed (search => 0
passed to HTML::OSM). The Reset Map button is retained; only the
search input and Nominatim egress are removed. Requires HTML::OSM >=
0.12.
- script/bi-geocode: new CLI warming script.
Opens a dataset spec, geocodes every distinct value of the named
column, and writes the resolved coordinates into the shared geocode
cache in .bi/models.json. Idempotent: a second run against a warm
cache resolves zero places and writes nothing. Suitable for cron.
Flags: --spec, --col, --data-dir, --model-dir, --openaddr, --region,
--dry-run, --verbose.
[ Dependency changes ]
- HTML::OSM minimum bumped from 0.11 to 0.12 (search => 0 and max_zoom
constructor params; tile_url was already present in 0.11).
0.016.0 Fri Oct 23 08:58:00 AM EDT 2026
[ New Features ]
- POST /api/ask: Natural language to analysis spec.
Translates a plain-English question into an analysis spec by
calling the Anthropic Messages API. The model receives only column
names and types -- no raw row data, no file paths. The returned spec
is validated (column names and kind checked) before being returned.
The feature is absent (503) when ai_api_key is not configured.
Database::BI::Ask -- new module.
Ask... button added to the dashboard toolbar when ai_api_key is set.
New test suite t/ask.t.
- GET /insights: Automatic statistical insights panel.
Detects outliers (Tukey IQR fence), period-over-period deltas,
top movers by dimension, and seasonality (lag-1 autocorrelation
on monthly totals, threshold r>=0.6). No LLM or external service
required. Every insight includes a view_url linking to the chart
or stats page that shows the underlying data.
Database::BI::Insights -- new module; pure arithmetic, no eval.
New test suite t/insights.t (218 subtests).
- Insights button added to the dashboard toolbar (next to Column stats).
- POST /api/summary: Narrative summary generation.
Two paths: deterministic template sentences (offline, always available)
and optional LLM polish via the Anthropic Messages API (opt-in,
activated by setting ai_api_key in the app config and passing ai=1).
Only aggregate insight facts are sent to the LLM -- no raw row data,
no file paths. Results are cached in-process for 1 hour.
Database::BI::Summary -- new module.
Summarize button added to /insights page; Use AI opt-in checkbox
shown when ai_api_key is configured in the app.
New test suite t/summary.t.
[ Bug Fixes ]
- Fixed https://matrix.perl-magpie.org/results/fbe5436e-cd3d-11f1-9632-bd4d6e8775ea
0.015.0 Tue Oct 20 08:10:43 PM EDT 2026
[ New Features ]
- GET /tutorial: in-application tutorial with a chapter for every page
feature (Database::BI::Tutorial). Each chapter has numbered steps, a
tip, and a Try it link. Chapters for joins, pivot, calc fields, canvas,
and five chart types include a worked walk-through. F1 opens the chapter
for the current page; a "Help:" link in the header does the same.
t/tutorial.t enforces that every GET page has a chapter and no sentence
exceeds 25 words.
- Dataset catalog (GET /catalog): lists all files in data_dir with type,
size, last-modified date, and which saved analyses reference each file.
- Sharing and embedding: POST /api/analyses/:id/share issues a signed
HMAC token. GET /share/:id?token= renders a read-only canvas;
GET /embed/:id?token= renders a chrome-less embed page.
GET /a/:id/visual/:n/csv downloads one visual's data as CSV.
- Extract cache (Database::BI::Extract): query results are materialised as
SQLite files under model_dir/extracts/ keyed by source fingerprint.
Deleting the extracts/ directory leaves the app fully functional.
- bi-alert CLI (script/bi-alert): evaluates a saved analysis against a
threshold (--threshold alias:op:value). Exit 0 = ok, 1 = crossed,
2 = error. Schedule via cron.
[ Enhancements ]
- Map markers cluster with Leaflet.markercluster: overlapping pins show a
count at low zoom and split into individual markers as you zoom in.
- Geocode answers are now shared across all datasets in the model store.
A place is resolved once and any other dataset that names it plots it
immediately without another lookup.
- D3, Leaflet, and tile-server URLs are all configurable, so a deployment
with no outbound network access can serve every asset itself.
- Map is a canvas visual kind. New Canvas and Add to Existing Canvas both
start from the map's lat/lon or geocoded column spec. Canvas map cards
support in-place column editing without a page reload.
- Add visual panel on canvas: append a table, chart, or map card to an
open canvas without leaving the page.
- New Canvas seeds itself from the current table or chart, including its
kind, columns, and options, so a pie chart stays a pie on the canvas.
- Multi-table SQLite files support a nested spec (table:store/customer).
The join and combine panels list each internal table separately.
- Map marker popups now show all non-coordinate columns by default.
Pass ?popup=col to restrict the popup to specific columns.
- data/virginia_counties.csv added: all 95 Virginia counties with 2020
census populations, lat/lon centroids, and region labels. Tutorial
walk-throughs for map, pie, bar, scatter, and stats use this file.
- Canvas home-page cards support Ctrl+click and rubber-band drag for
multi-select; a floating action bar offers bulk delete.
- Canvas freshness banner warns when a source file has changed since the
analysis was last saved.
[ Bug Fixes ]
- Canvas treemap cards no longer return 500.
- Chart options (orientation, max_items, show_values) are now preserved
when seeding a canvas from a chart page.
- D3 is now loaded before chart cards, so inline chart scripts run
correctly on first render.
- Cross-filtered canvas charts now aggregate the filtered rows, not the
full source.
- Stored calc fields now appear in /view/:table (not only in chart views).
0.014.0 Mon Oct 19 09:49:36 AM EDT 2026
[ New Features ]
- GET /map renders the filtered dataset on an OpenStreetMap map via
HTML::OSM (optional). Use lat=/lon= for coordinate columns already in
the data, or loc=<column> to geocode a place-name column. Rows sharing
a position are collapsed to one marker; out-of-range coordinate columns
return 400; markers and popup columns are configurable.
- Database::BI::Model::Geocode resolves place-name columns via
Geo::Coder::Free (optional, >= 0.47). Lookups are one-per-distinct-value,
cached in the model store (.bi/models.json), capped per request
(map_geocode_max), and skipped rather than failed for blanks and bare
numbers. Without the geocoder, previously resolved places are still
plotted.
[ Enhancements ]
- Full keyboard accessibility audit. Canvas and chart resize handles are
now sliders in the tab order (arrow keys, Shift for bigger steps, Home/End
for min/max); card size can also be set from a Size menu on each card.
Canvas cards are draggable to a chosen grid cell from the keyboard
(arrow keys in the header, Shift+arrow for edge). Pivot column headers
sort (Enter/Space) and move (Shift+Left/Right) from the keyboard.
Home-page card tooltips appear on focus. Stacked and multi-series line
chart legends are toggle switches. Treemap tile values are read in the
tile name. Every size change and card move is announced via a live region.
- Browser test suite (t/browser-*.t) driven by headless Firefox via
Marionette. Covers canvas keyboard controls, chart resize/export/a11y,
pivot keyboard, and home-page focus behaviour. Runs as a separate CI
job (.github/workflows/browser.yml); skips automatically when Firefox
is unavailable.
[ Bug Fixes ]
- GET /combo: ?bar_val= and ?line_val= are now the canonical parameter
names (?bar= and ?line= kept as aliases). ?cat= is now the canonical
x-axis parameter (?x= kept as alias), consistent with /stacked.
0.013.0 Fri Oct 16 04:08:48 PM EDT 2026
[ New Features ]
- Saved canvases: multi-visual workspace pages (/a/:id) that combine
charts and table views. Full CRUD API at /api/analyses. Canvas
pages support cross-filtering, drill-through hierarchies,
parameterisation (?p.<name>=), bookmarks with undo/redo, per-visual
PNG download, and print to PDF/paper.
[ Enhancements ]
- Chart resize: all chart pages (line, pie, bar, heatmap, scatter,
stacked, combo, treemap, histogram) now show a drag handle at the
bottom-right corner of the chart SVG. Dragging it resizes the chart
proportionally without a page reload or D3 re-render.
0.012.1 Thu Oct 15 12:00:00 PM EDT 2026
[ Bug Fixes ]
- Fix 'Argument "" isn't numeric in int' warnings logged to the console
when numeric query parameters (?topn=, ?max=, ?max_cols=, ?per_page=,
?page=, ?min_r=, ?max_r=) are submitted with an empty value.
- Fix ?bins=0 reverting to 10 bins instead of clamping to 1: the
|| 10 empty-string guard must not swallow the literal value 0.
histogram_view now uses the defined-and-non-empty pattern so that
?bins=0 correctly passes through the $bins < 1 clamp.
[ Tests ]
- Add t/binned.t: end-to-end contract tests for /histogram pre-binned
mode (label/count correctness, bins=N clamping, max-lands-in-top-bin,
no d3.bin() call, bin()/histogram label agreement, virtual column
binning, flat fixture, empty/error state).
0.012.0 Wed Oct 14 09:10:15 PM EDT 2026
[ New Features ]
- GET /pivot: crosstab/pivot table. Accepts ?rows=, ?cols=, and
?m=col:fn:alias (sum|avg|min|max|count|count_distinct); ?totals=1
adds subtotals and a grand-total row; ?h= applies post-aggregation
HAVING filters; ?topn=N+?topn_col= keeps the top-N row groups;
?max_cols=N collapses tail column combos into "Other";
?format=csv|sqlite|json|xlsx exports the result.
- GET /stats: per-column statistics (row count, null count, distinct
count, min, max, mean, median).
- GET /scatter: scatter/bubble plot (?x=, ?y=, ?series=, ?size=,
?scheme=, ?min_r=, ?max_r=). The scatter panel now exposes all
parameters: "Colour by" (any column), "Bubble size" (numeric
columns only), colour scheme (tableau10|category10|set2|set3|paired
-- enabled only when a series column is chosen), and Min/Max radius
1-40 (enabled only when a size column is chosen). ?scheme= is
validated server-side; an unrecognised name returns 400.
- GET /stacked: stacked bar chart (?cat=, ?series=, ?val=).
- GET /combo: combo bar+line chart (?cat=, ?bar_val=, ?line_val=).
- GET /treemap: treemap (?group=, ?size=, ?topn=N).
- GET /histogram: histogram with server-side binning (?col=, ?bins=N;
default 10 bins; max 200).
- GET /line: alias for GET /graph (all /graph params accepted).
- GET /graph: new ?series=<col> parameter for multi-series line charts.
- Calculated fields (Database::BI::Calc): recursive-descent evaluator
supporting arithmetic, comparison, IF, string, numeric, and date
functions; bin(); coalesce; isnull. All chart and pivot routes
accept ?c=name:expr (repeatable). Stored fields managed via
GET /calc/list, POST /calc/save, POST /calc/delete.
- Date normalisation: stored date-format patterns convert non-ISO
columns to ISO-8601 before aggregation. Managed via
GET /dates/list, POST /dates/save, POST /dates/delete.
- Named value groups: virtual "${col}_group" columns map raw values
to analyst-defined labels. Managed via GET /groups/list,
POST /groups/save, POST /groups/delete.
- Redis backend: DataSource accepts redis => 'redis://...' and the
redis:tablename|redis://... spec format.
[ Enhancements ]
- GET /join and GET /combine now support pagination (?page=N,
?per_page=M, ?sort=<col>, ?dir=asc|desc).
- GET /export and POST /export now accept ?page=N and ?per_page=M.
- f= filters and d=1 dedup applied before pagination so the N+1
row count reflects the reduced set (reduce-before-paginate).
- bar_view and pie_view: new ?topn=N parameter; remainder collapsed
into "Other".
- Added X-BI-Warnings response header to GET /export.
- All chart render() calls now use a unified chart_html stash key
and include a chart => { kind, title, encode, options } DTO.
- Added DataSource::dbi_source() for SQL-capability detection and
Database::Join zero-copy ATTACH path.
- Added _grouped_totals() as the single aggregation entry point for
all chart routes, with SQL push-down and Perl fold fallback.
[ Bug Fixes ]
- /join and /combine: no-params default now returns the first 200
rows with a pagination bar, consistent with /view and /open.
- f= filters on paginated views no longer produce short pages.
- d=1 dedup now deduplicates the full result before slicing.
- has_next recomputed after in-memory filter/dedup reduction.
- Chart category order no longer changes when a filter is applied:
Perl fold now returns groups in the same ascending key order as
the SQL path.
- Aggregation on SQLite sources pushes SQL-compatible f= filters to
the database; non-SQL-able filters still fall back to Perl fold.
- DataSource::dbi_source() no longer propagates exceptions from
non-SQL backends; returns undef instead.
- CSV and other non-SQL sources no longer emit "Bad limit clause"
warnings; LIMIT/OFFSET is only sent to SQL-backed sources.
[ Dependencies ]
- HTML::D3 >= 0.20 (radius_range support for scatter bubble charts;
0.19 added the five new snippet methods).
- Database::Abstraction >= 0.51 (redis key support).
- Database::Join >= 0.009.0 (aggregate, limit/offset, dbi_source).
0.011.0 Sun Oct 11 09:20:48 PM EDT 2026
[ Enhancements ]
- Added schema browser (GET /tables) for multi-table SQLite databases.
Opening a SQLite file with more than one internal table now shows a
card grid listing every table with its columns rather than silently
opening the first table. Clicking a card opens that table via the
existing /open view. The schema page is inserted into the breadcrumb
trail so users can navigate back to it.
- Added ?it=<table> parameter to GET /open, allowing a specific internal
table to be opened from a multi-table SQLite file.
- Added sqlite_tables() accessor to DataSource, returning the list of
internal table names discovered during construction.
- Added internal_table constructor option to DataSource, letting callers
request a specific internal SQLite table instead of the auto-detected
first match.
- Added data/store.db: sample SQLite database with three joinable tables
(customer, product, sales) for development and demonstration.
- Fixed: multi-table symlink for .db-extension SQLite files now uses
.sql as the symlink extension so Database::Abstraction opens it via
the DBI/SQLite backend rather than falling through to the CSV reader.
NOTE: /tables tests currently cover SQLite only. MySQL, PostgreSQL,
and other multi-table driver tests will be added in a future release
once the corresponding Database::Abstraction drivers are available.
- Server-side pagination for /view/:table and /open. Both routes now accept
?page=N (1-based, default 1) and ?per_page=M (default 200, max 2000).
The controller uses the N+1 trick: it requests per_page+1 rows via
LIMIT/OFFSET and treats the extra row as a "has next page" sentinel rather
than issuing a COUNT(*) query. This is critical for multi-GB SQLite files
(e.g. openaddresses.db) where COUNT(*) would block the server for minutes.
A pagination bar (Prev / current page / Next) is shown when at least two
pages exist. All URL parameters (filters, dedup, per_page, etc.) are
preserved across page navigations.
- BerkeleyDB files larger than 100 MB now return an informative error page
instead of causing disk thrashing. Files whose magic bytes identify them as
BerkeleyDB but whose DB_File tie() fails (e.g. Berkeley DB version 9 files
opened with libdb-5.3) would previously fall through to D::A's CSV path,
causing a full read of the binary file. DataSource now detects the magic
bytes in _detect_file_info, refuses files above 100 MB immediately, and
returns error_unsupported_db for smaller files where tie() fails.
- Three-level breadcrumb for chart pages. When a bar, line-graph, pie,
or heatmap chart is launched from a table view that was itself reached
from the schema browser (/tables), the chart page now shows the full
navigation chain (e.g. Home > Store schema > Table view > Chart) rather
than the previous two-level "Home > Back to table". The dashboard
panel forms forward the current page's back_url as back2=/back2_label=
parameters; the chart controllers promote them to the outer breadcrumb
level so all four levels are rendered correctly.
- SQLite hover tooltip on the home-page file picker. Hovering over a
database card in the "Choose a database" list now shows a dark popover
with the file type (e.g. "SQLite database - 3 tables"), last-modified
date, and file size. GET /api/stat now returns a table_count integer
for SQLite-extension files (.sql, .sqlite, .sqlite3, .db); the count
is omitted for Berkeley DB .db files or other non-SQLite payloads.
The home-page db-cards now carry data-path attributes so the JS fetch
can call /api/stat on first mouseenter (lazy, not on page load).
0.010.0 Tue Oct 6 11:29:33 AM EDT 2026
[ Enhancements ]
- Added Folder view (GET /folder): browse any data file grouped by the
letter prefix of a chosen code column (e.g. Order # values like A10,
CT4, B3). Records are organised into a two- or three-level folder
hierarchy -- top level shows distinct first letters (A-Z), clicking a
letter reveals sub-prefix folders (e.g. CA, CB, CC inside C) and any
items with that exact prefix, clicking a sub-prefix shows the matching
records in a table. A "Folder view..." toolbar button opens a column-
picker panel identical in style to the chart panels; the code column is
auto-detected when the panel is opened but the user can override it via
the dropdown. Breadcrumb navigation lets the user step back to any
folder level or return to the source table via "Back to table".
- upload_file now guards move_to() with eval { ...; 1 } and checks -f
$dest after the call. A failed save returns a JSON 500 with the key
error_upload_save instead of propagating an unhandled exception.
The orphaned upload subdirectory is removed before the error response
is sent. (Uses the eval { ...; 1 } idiom to avoid false positives
from a stale $@ left by Mojolicious internal dispatch code.)
- All chart pages (/graph, /pie, /heatmap, /bar) now use responsive SVG
layout (HTML::D3 0.18 `responsive => 1`): charts scale to the available
container width so they render correctly on mobile and narrow windows.
- /bar: added ?max=N parameter -- when N > 0, bars beyond the top N are
collapsed into a single "Other" bar (HTML::D3 `max_bars`). Prevents
unreadable charts from high-cardinality categorical columns.
- /bar: category-column name is now shown as an axis label below the bars
(HTML::D3 `x_label`).
- /pie: added ?scheme= parameter to select the D3 categorical colour
palette (tableau10, category10, set2, set3, paired; default tableau10).
Mirrors the existing ?scheme= parameter already on /heatmap.
- /join: added ?jtype=left|inner|outer parameter to control join semantics.
`left` (default) keeps all rows from the left table; `inner` returns
only rows that match in both; `outer` returns all rows from either table.
Exposed in the "Merge data" panel as a "Which rows to include" dropdown.
- /join: schema-type-mismatch warnings from Database::Join (e.g. "column
'score' has type TEXT in left table but INTEGER in right table") are now
captured and displayed as a yellow notice bar above the data table,
instead of landing silently in the server error log.
- DataSource now implements updated() -- proxies to Database::Abstraction's
updated() or stats the file directly. This allows Database::Join to
invalidate its SQLite spill cache when a source file changes on disk.
0.009.0 Wed Sep 23 09:14:24 PM EDT 2026
[ Enhancements ]
- Added a bar chart (GET /bar). Choose a categorical column and an
optional numeric value column (omit to count rows per category); the
chart uses HTML::D3 >= 0.18's new render_bar_chart_snippet. A
"Bar chart..." toolbar button opens a configuration panel identical
in style to the pie/graph/heatmap panels. Requires HTML::D3 >= 0.18.
- Bar chart drill-down: clicking any bar navigates to the source table
filtered to that category, with a "Back to bar chart" breadcrumb link
so the user can return to the chart.
- Pie and bar chart drill-down breadcrumbs now work correctly when the
source table was opened via GET /import (URL import). The middle
breadcrumb links back to the unfiltered imported table (clearing any
active filter), and the third crumb links back to the chart.
Previously the back2/back2_label parameters were silently discarded
by import_url, so the chart back-link never appeared.
- Data tables now scroll horizontally when the number of columns exceeds
the screen width. The table is wrapped in a div.table-scroll
container (overflow-x: auto) so the toolbar and breadcrumbs stay at
full width while only the table body scrolls.
[ Bug Fixes ]
- Fixed a test failure on CPAN testers running CGI::Lingua < 0.67 where
language negotiation for non-English Accept-Language headers returned
the wrong result. Minimum CGI::Lingua version is now 0.85.
- Fixed non-ASCII EM-dash and currency characters in source comments and
one runtime error message in Dashboard.pm, which could produce mojibake
on systems that interpret .pm files as Latin-1.
- IPC::System::Simple is now declared as a runtime dependency (PREREQ_PM)
in addition to a test dependency, because DataSource.pm uses
"use autodie qw(:all)" at compile time on every server startup.
0.008.1 Tue Sep 22 09:13:04 PM EDT 2026
[ Bug Fixes ]
- Fixed mojibake in pie/heatmap/line-graph chart labels when the data
contains non-ASCII characters (e.g. author names from /import).
Root cause: HTML::D3 < 0.17 returned UTF-8 byte strings from its
render_*_snippet methods; Mojolicious::Renderer then called
Mojo::Util::encode('UTF-8', ...) on the TT output, which treated
those bytes as Latin-1 and double-encoded them (e.g. U+00F6 "o
umlaut" appeared as "o" + two garbage bytes). Fixed by decoding
the snippet HTML to a Perl character string (utf8::decode) in
graph_view, pie_view, and heatmap_view before passing it to the
TT stash. Requires HTML::D3 >= 0.17, which fixes the same bug
at the source by using JSON::MaybeXS with utf8 => 0 throughout.
- Fixed pie chart PNG (and SVG) export clipping long legend labels on
the right. The canvas/SVG width was calculated using a hardcoded
240 px legend column regardless of label length. The legend width
is now measured dynamically with ctx.measureText() on an off-screen
canvas so the export image is always wide enough for the longest
label, with a 10 px right-side margin.
0.008.0 Tue Sep 22 02:33:40 PM EDT 2026
[ Enhancements ]
- Added support for TSV (tab separated) files.
- Added support for .sqlite and .sqlite3 file extensions as aliases
for SQLite databases. Files with either extension are treated
identically to .sql: sqlite_master is probed for table names,
mismatched stems are auto-corrected via a symlink, and the
extensions appear in the filesystem browser and upload
accepted-types list.
- Multi-table joins now pass backend => 'auto' to Database::Join
(>= 0.004.0). Datasets up to join_max_rows combined rows
(default 10,000; configurable in database_bi.conf) join in Perl
memory; larger datasets automatically spill to a temporary SQLite
file so peak RAM is bounded by the result set rather than the sum
of all source tables.
- Added a heatmap chart (GET /heatmap). Choose two categorical columns
for the X and Y axes; optionally pick a numeric column to sum per cell
(omit to count rows). Colour scheme is configurable (YlOrRd, Blues,
Greens, Purples, RdPu, YlGnBu). A "Heatmap..." toolbar button in the
dashboard opens a panel to configure and launch the chart. Uses the
new render_heatmap_snippet method from HTML::D3 >= 0.16.
- Bumped HTML::D3 minimum dependency from 0.14 to 0.16.
- Line graph reference lines: the Min line is now amber (#f59e0b)
instead of blue, so it no longer clashes with the steelblue main
line. Data-point dots whose Y value equals the current REF.min are
recoloured amber when the Min checkbox is checked; dots at REF.max
are recoloured red when the Max checkbox is checked. Unchecking a
reference-line checkbox immediately reverts the affected dots to
steelblue. Dot colours are also updated after zoom and reset.
- Charts generated from URL-imported data (/graph, /pie, /heatmap) now
show two attribution lines below the chart: "Source: <url>" and
"Accessed on <date>", where the date is the time of the current
request (when the data was fetched/served), not necessarily today.
Both lines are also embedded in SVG and PNG exports: the SVG gains
two <text> elements appended below the chart area (height and viewBox
extended by 38 px); the PNG canvas is extended by the same amount and
the text is drawn with canvas 2D fillText. URLs longer than 90
characters are truncated with "..." in exports to avoid overflow.
Charts from local files are unaffected.
- Added support for Database::Abstraction's remote file syntax:
paths of the form /../hostname/dir/file.ext can now be passed to
GET /open?path= and to path: specs in /join, /combine, /graph,
/pie, /heatmap, and /api/columns. The hostname portion is
validated against a safe-identifier regex (same as D::A's own
host validation). Only supported file extensions are accepted.
Net::SFTP::Foreign is loaded lazily and used to download the
remote file to a process-local temp directory; the temp dir is
kept alive for the DataSource object's lifetime and cleaned up
automatically on destruction. Net::SFTP::Foreign is now a
declared runtime dependency.
- Remote files with non-standard extensions (e.g. .log, .dat) are
now handled correctly. After download, DataSource sniffs the
file content (SQLite magic bytes, XML preamble, or first-line
separator) and renames the temp copy to the appropriate standard
extension (.db, .xml, .tsv, .psv, or .csv) so that
_detect_file_info and Database::Abstraction can discover it
automatically. Previously, a remote file whose extension was not
in the recognised list caused a "Can't find a file" error even
when the download succeeded.
- Added CSS-only tooltips to all path input fields (join panel,
combine panel, and the home-page "Browse your files" input).
Hovering or focusing any of these fields shows a dark popover
explaining the local and remote (/../hostname/) path syntax with
examples. The tooltip uses position:absolute + opacity transition
and is shown via :hover/:focus-within with no JavaScript.
- Added novalidate to the join, combine, open, and import forms so
that the browser's built-in "Please fill out this field" validation
popup no longer appears on hover; DataSource validates inputs
server-side instead.
- Added support for user@hostname syntax in the remote file path:
paths of the form /../user@hostname/dir/file.ext are now accepted
and the user portion is passed to Net::SFTP::Foreign as the login
name.
- Added a "Copy link" button to the filter toolbar. The button
appears only when at least one filter is active and copies the
current URL (including all ?f= parameters) to the clipboard, so
users can bookmark or share a custom-filtered view. Uses the
modern Clipboard API with an execCommand fallback for plain-HTTP
deployments. Clicking briefly shows "Copied!" feedback before
reverting to the original label.
[ Tests ]
- Transaction 33: full TSV lifecycle (view, open, filter, export,
upload).
- Transaction 34: full .sqlite3 lifecycle (open, filter, export,
browse, idempotency, table-name mismatch auto-correction).
- Transaction 35: full .sqlite lifecycle (same phases as T34).
- Transaction 37: heatmap chart lifecycle (val mode, count mode, 400
for missing/invalid params, idempotency, toolbar button and panel
presence).
- Transaction 36: copy-link button lifecycle (absent without filters,
present with one or more filters, JS clipboard paths present, CSS
classes defined).
- Transaction 39: remote file path (/../hostname/dir/file.ext)
lifecycle (path parsing, hostname validation, extension guard,
/open rendering, /api/columns spec resolution, idempotency,
non-standard extension via _sniff_data_ext content detection,
user@hostname credential passing).
Net::SFTP::Foreign is stubbed in tests so no real SSH connection
is required.
- basic.t: added "GET /heatmap" smoke subtest, "SQLite3 format
(.sqlite3 extension)" subtest, and "SQLite format (.sqlite
extension)" subtest.
0.007.0 Wed Sep 16 07:33:53 PM EDT 2026
[ Enhancements ]
- Added pie chart support via the new GET /pie route. A "Pie chart..."
toolbar button opens a panel where the user chooses "Slice by" (the
category column to group by) and "Sum of" (the numeric column to
aggregate), plus an optional "Donut" toggle. The controller sums each
category using the same accounting-notation handling as the line graph,
then delegates rendering to HTML::D3 0.14's new render_pie_chart_snippet
(animated, sorted by value, capped at 12 slices with an "Other" tail,
legend enabled). HTML::D3 minimum bumped from 0.13 to 0.14.
- Pie and line graph pages now show a two-level breadcrumb
("Home > Back to table") matching the dashboard style.
- The table view now has a quick-filter bar below the toolbar. A column
select, operator select (equals, not equals, contains, starts with, less
than, less or equal, greater than, greater or equal, is empty, is not
empty), and a value input let the user add a filter condition without
opening the "Merge data" panel. Pressing Enter in the value field
triggers Apply. Active filter chips now each have an individual x-remove
button to drop a single condition without clearing all filters.
[ Bug Fixes ]
- Opening the line graph panel after the pie chart panel (or vice versa)
now correctly hides the previously open panel. The graph panel's
otherPanels list was missing 'pie-panel'.
0.006.0 Tue Sep 15 08:03:20 PM EDT 2026
[ Enhancements ]
- Added a 'home' link on the table display page
- XLSX files are now supported directly via DBD::Excel (each worksheet
is a separate table). The upload endpoint, filesystem browser, and
drag-and-drop drop zone all accept .xlsx files. DBD::Excel added to
TEST_REQUIRES.
- "Combine data" now allows a file to be drag and dropped
- Multi-table left joins are now delegated to Database::Join (>= 0.003.0)
instead of the bespoke _left_join helper. Column collision handling
(prefixing right-table columns with "label.col" when a name clashes) is
provided by Database::Join's new collision_prefix parameter.
DataSource now exposes selectall_arrayref() and an improved columns()
fallback so that DataSource objects can be passed directly to
Database::Join as component databases.
- Database::Join added to PREREQ_PM (runtime dependency).
- DataSource::new now silently sanitizes table names derived from filenames:
characters that are illegal in SQL identifiers (hyphens, dots, spaces,
etc.) are replaced with underscores, and a digit-leading stem is prefixed
with '_'. The original filename stem is kept as the dbname so
Database::Abstraction still locates the correct file on disk. This means
files like "Transactions-2026-09-08.csv" and "1data.csv" can now be
opened via /open without error. Only path-separator characters ('/', '\',
NUL) and the empty string continue to croak error_table_name_invalid.
- A "Totals" checkbox in the toolbar appends a summary row to the bottom of
any data table. When checked, every column whose values are all numeric
(using the same parseAccounting logic as the sort and graph features, so
accounting-notation negatives are handled correctly) shows its sum; date
columns (YYYY-MM-DD, M/D/YYYY) are excluded from totalling even though
parseFloat can extract a number from them. Non-numeric columns are left
blank. The checkbox state is persisted in localStorage alongside column
order and sort direction, so the totals row reappears automatically on
page reload. The summary row follows column drag-to-reorder automatically
because moveCol() operates on all table rows including tfoot.
- CSV and PSV files with no header row are now handled automatically.
When every value in the first line fails the safe-identifier check,
_detect_file_info tests whether any value looks like a date (YYYY-MM-DD,
M/D/YYYY) or a signed/accounting-notation number. If so, the file is
treated as header-less: all rows (including the first) are read directly
and column names are synthesised from the inferred data types — date
values become "Date", numeric/currency values become "Amount", and free
text becomes "Description". Duplicate types are disambiguated with a
numeric suffix (Date2, Amount2, ...). This allows bank and accounting
exports such as "Transactions-current.csv" to open without error or any
manual renaming of headers.
- CHI in-process caching is now active for all data sources. URL-backed
tables (fetched via LWP from a remote web page) are cached for 15 minutes
using a TTL-based key, so repeated visits to the same /import URL avoid
a network round-trip on every page load. File-backed tables are cached
using a key that encodes the file's mtime, so a changed file naturally
produces a cache miss without any explicit invalidation — the stale entry
is orphaned and reclaimed when the process restarts or the CHI Memory
driver's GC runs. The cache TTL and driver can be overridden in
database_bi.conf via the "cache" key: { driver => 'File', ttl_url => '5 min' }.
CHI added to PREREQ_PM and cpanfile.
- DataSource::columns() now returns undef for URL-backed tables instead of
delegating to Database::Abstraction's columns(). D::A fetches URL data
lazily (in _open(), not in new()), so calling _db->columns() after a CHI
cache hit (which bypasses _open()) would trigger an unexpected live network
request — failing with a non-200 error even when valid cached data was
available. Returning undef lets _get_columns derive column names from the
cached data records, which is correct since URL/HTML tables have no
canonical column order anyway.
- Rows and columns can now be selected and deleted in the virtual table view.
A checkbox column is injected at the left edge of every data table;
clicking a row's checkbox selects it (turning the row pink) and Ctrl+click
on any column header selects the column (highlighting all its cells in red).
A "Delete selected" button appears in the toolbar as soon as any rows or
columns are selected; clicking it removes them from the in-browser view
without touching the underlying file. Row selection supports a
select-all/deselect-all checkbox in the header. The record count in the
toolbar is updated after deletion. Sorting, drag-to-reorder, and the
Totals row all continue to work correctly after deletions.
- Line graphs now animate on initial page load (requires HTML::D3 >= 0.13).
The line draws left-to-right via a stroke-dashoffset transition (1200 ms,
easeLinear); data-point circles fade in after the line finishes (300 ms
fade, 1200 ms delay). Zoom and reset redraws are not animated — a
per-page initialDrawDone flag ensures the effect fires only once.
When the viewer's OS has prefers-reduced-motion enabled the animation
is skipped entirely and the chart appears instantly.
HTML::D3 minimum version bumped to 0.13 in PREREQ_PM and cpanfile.
[ Bug Fixes ]
- When attempting to drag and drop a file larger than MAX_UPLOAD_BYTES,
a better error message is displayed which stays on the screen for
3 seconds
- CSV and PSV files whose first line is blank (e.g. a file containing only
a single newline character) are now treated as empty files rather than
croaking "no column with a safe identifier name". _detect_file_info now
returns the _file_is_empty sentinel when the parsed first line yields
zero column names, causing _init_backend to skip Database::Abstraction
entirely and fetch_all to return []. The /open endpoint renders the
standard empty-table view ("No records found") instead of an error page.
Tests added to t/edge_cases.t.
- Files whose names contain spaces (e.g. "transactions for Nigel.xlsx")
now open correctly. Database::Abstraction validates its dbname argument
as a SQL identifier and rejected names with spaces with an "unsafe dbname"
error at fetch time. DataSource::_init_backend now creates a temporary
directory with a symlink using the sanitized name whenever the raw
filename stem contains illegal SQL characters, so D::A sees only the
safe name. Regression test added as Transaction 27.
- The "Line graph..." toolbar button was silently broken: clicking it did
nothing. buildYSelect() referenced SEL from the table-management IIFE
where it was declared, but the graph-panel IIFE lives in a separate
<script> block and cannot access that scope. The ReferenceError thrown
during initGraphBtn() prevented the click listener from ever being
registered. Fixed by removing the cross-IIFE reference entirely:
buildYSelect() now uses r.cells[idx] instead of r.cells[idx + SEL],
because Array.from(tHead.rows[0].cells) already includes the injected
sel-th at position 0, so both the header and data arrays are shifted
identically and no additional offset is needed. A regression test in
Transaction 22 verifies that buildYSelect does not reintroduce
the idx+SEL double-offset.
- The Y-axis dropdown in the "Line graph..." panel was listing non-numeric
columns (e.g. Description) as valid Y-axis choices, and omitting the
correct numeric ones (e.g. Balance). Root cause: buildYSelect() used
r.cells[idx + SEL] where idx came from headers.forEach() over
tHead.rows[0].cells. Because injectSelCol shifts both thead (sel-th)
and tbody (checkbox) by one position, the header and data arrays are
already aligned; adding SEL read one cell to the right of the header
under test, causing every column's numeric check to be evaluated against
the data of the column to its right. Fix: changed to r.cells[idx].
- Spreadsheet::ParseXLSX added to PREREQ_PM. DBD::Excel 0.07 only
handles .xls (its source skips .xlsx files entirely); DataSource now
reads .xlsx directly via Spreadsheet::ParseXLSX in _detect_file_info,
bypassing D::A the same way headerless CSV files do.
Excel::Writer::XLSX added to TEST_REQUIRES for fixture creation.
Transaction 28 covers XLSX open lifecycle including spaced filenames.
- Synthesised column headers for header-less CSV/PSV files are now
capitalised (Date, Amount, Description) so they display correctly as
table headers in the browser.
- Berkeley DB files (.db extension) can now be opened via /view/:table,
/open, and the filesystem browser. Database::Abstraction detects the
BerkeleyDB magic bytes automatically and opens the file via DB_File
(a Perl core module). No code change was required in Database::BI;
the existing @SUPPORTED_EXT and _detect_file_info already handled .db
files correctly. Transaction 29 confirms the end-to-end lifecycle.
0.005.2 Fri Sep 11 06:08:10 AM EDT 2026
[ Bug Fixes ]
- Fixed SIGABRT crash (DBI XS assertion failure) in t/edge_cases.t on
DEBUGGING-enabled perls (observed: perl 5.44.0 + DBI 1.651 + DBD::CSV
0.64). When a 0-byte CSV file was opened, _detect_file_info returned
the generic "not CSV" sentinel ({}), causing _init_backend to create a
Database::Abstraction object without the max_slurp_size guard. D::A
fell back to the DBD::CSV path, which issued "SELECT * FROM <table>" on
an empty file; DBD::CSV's EOF error left DBI's internal Errstr SV in a
shape that violated an XS assertion, crashing the process during global
destruction after all test assertions had already passed (so the TAP
plan was never written and the harness reported FAIL).
Fix: _detect_file_info now returns { _file_is_empty => 1, file_size => 0 }
for 0-byte files. _init_backend detects this sentinel and short-circuits
before creating D::A; fetch_all returns [] immediately. DBI is never
contacted and no crash occurs.
References: https://www.cpantesters.org/cpan/report/117b2a76-a4e5-11f1-9192-ffecfaad6fbc
- Reference lines (Min / Avg / Max) on the line graph now recalculate
correctly after a brush-to-zoom or Reset zoom interaction. Two bugs
were responsible:
* The previous implementation tried to hook into the zoom by listening
for a D3 brush "end" DOM event, but D3 fires brush callbacks through
its own internal dispatch, not through DOM events. The listener
never fired. The fix patches the global `redraw` function (a
function declaration, not a `const`) and recalculates the reference
values (min / avg / max) from the data currently being drawn
(`newData`) rather than the full-dataset values stored at page load.
This ensures the lines always fall within the visible y-domain even
after zooming into a subset of the data.
* `drawRefLines()` was called synchronously during the 500 ms D3
transition. Although `y.domain()` is updated before the transition
begins, the y-axis visually animates to the new domain over 500 ms;
appending lines mid-transition produced a brief positional mismatch.
The call is now deferred with `setTimeout(drawRefLines, ms + 20)`.
- Requires HTML::D3 >= 0.11 for the companion fix: the zoomable line
chart y-domain now uses `Math.min(0, d3.min(...))` as its lower bound
instead of hard-coding 0. This ensures data points and reference
lines with negative Y values (e.g. accounting-notation amounts) are
plotted within the visible chart area rather than appearing off-screen
below the x-axis.
0.005.1 Sun Aug 30 08:29:03 PM EDT 2026
[ Bug Fixes ]
- Fixed compatibility with Database::Abstraction >= 0.41. D::A 0.41 changed
dbname to fall back to the class-name suffix rather than the table parameter,
so a dynamically-generated package like Database::BI::_DB::Orders was used
as the filename stem (Orders.csv) instead of the lowercase table name
(orders.csv), causing "Can't find a file" errors on case-sensitive Linux
filesystems. DataSource::_init_backend now passes dbname => $table
explicitly so the filename stem is always correct regardless of D::A version.
Minimum Database::Abstraction version bumped to 0.41.
- Columns containing accounting-style negative amounts - values written as
(1,234.56) rather than -1,234.56 - are now recognised as numeric and
handled correctly throughout the application:
* Column sort: clicking an Amount header now sorts correctly, with
parenthesised values treated as negative numbers rather than being
sorted as text or treated as positive values.
* Y-axis selector: columns whose values use accounting notation are now
included in the "numeric columns" list for line graphs, so Amount (and
similar columns) appear in the Y-axis dropdown.
* Line graph: Y values in accounting notation are plotted at the correct
negative position on the chart rather than at their absolute value.
The fix adds a parseAccounting() helper in the browser (shared by sort
and graph detection) and updates the server-side graph_view Y-strip logic
to detect the leading '(' before stripping non-numeric characters.
0.005.0 Sat Aug 29 02:22:00 PM EDT 2026
[ Enhancements ]
- The "Line graph..." toolbar button is now greyed out (disabled) when the
current view has no numeric column that can serve as the Y axis.
Hovering over the disabled button shows the tooltip "Line graphs need a
numeric column". The check runs once at page load using the same
isNumericVal / isDateVal logic as the Y-axis column selector, so date
columns and fully-text columns both correctly suppress the button.
- When a view has exactly one numeric column the Y-axis dropdown is
replaced by a read-only "auto-selected" label showing the column name,
so the user only needs to choose the X axis before clicking Plot.
- Upload subdirectories older than 24 hours are now evicted automatically
on every server startup. Recent uploads (from the current or previous
session within the last 24 hours) are preserved. The explicit
"Clear upload cache" button (POST /uploads/clear) continues to remove
all entries immediately regardless of age.
- Graph hover tooltips now show all columns for the data point, not just
the X and Y values. Mousing over a dot displays every field from that
row (excluding the axes columns) as additional key/value lines below the
label and value. Requires HTML::D3 >= 0.09.
- Line graph now supports brush-to-zoom: drag across a range on the X axis
to zoom in; a "Reset zoom" button returns to the full view. Uses
HTML::D3's render_zoomable_line_chart_snippet.
- Line graph toolbar now shows the number of plotted data points
(e.g. "Plotting 6 points") so users know immediately how much data
is in view, especially after filtering.
- GET /export now accepts format=json to download the current view as a
JSON array of objects. CSV and SQLite remain available as before.
- GET /api/columns now accepts a ?spec= parameter in the unified
"table:name" or "path:/abs/path" format used by /join and /graph,
so JavaScript can pass the l= pipeline spec directly without parsing it.
[ Bug Fixes ]
- Graph page correctly returns HTTP 200 with a "No plottable data" message
when the chosen Y column contains no numeric values (e.g. a text column),
rather than crashing or producing an empty chart.
- The first X-axis label on the line graph is no longer cropped. D3
rotates labels -45 degrees with text-anchor:end, so the leftmost
label extends to a negative SVG x-coordinate. The fix widens the
SVG via a viewBox that starts at x=-60: the formerly-negative region
becomes part of the SVG viewport, so no CSS overflow trick is needed.
- t/filter.t subtest "import_url: non-numeric table_index is sanitized to 0"
now passes on GitHub Actions. Database::Abstraction fetches URLs via
LWP::UserAgent::Cached (not plain LWP::UserAgent); the test was mocking
LWP::UserAgent::get but that mock was bypassed by Cached's override of
simple_request. The mock now targets LWP::UserAgent::simple_request
(the base-class method that Cached delegates to after its cache check),
which intercepts the call regardless of which LWP subclass D::A uses.
LWP::UserAgent::Cached is also added to PREREQ_PM and cpanfile so CI
installs it; previously it was absent and the feature silently failed
on fresh environments.
0.004.0 Thu Aug 27 09:36:22 PM EDT 2026
[ Enhancements ]
- Date columns now sort chronologically when clicking the column header.
The sort routine detects slash-separated dates (M/D/YYYY and D/M/YYYY)
and ISO 8601 dates (YYYY-MM-DD) and converts them to a YYYYMMDD integer
key before comparing, so 1/5/2017 correctly sorts after 12/27/2016.
For slash dates the format is auto-detected by scanning for a component
that exceeds 12: if the second component ever exceeds 12 the column is
month-first (M/D/YYYY); if the first component exceeds 12 it is
day-first (D/M/YYYY); ambiguous columns default to month-first.
- Add line graph feature: a "Line graph..." button on any data view opens
a column-picker panel where the user selects X and Y axes. Clicking
"Plot" navigates to GET /graph, which opens the same data pipeline as
the current view (joins, combines, filters, dedup) and renders an
interactive D3.js v7 line chart with hover tooltips via HTML::D3.
The graph page includes a toolbar injected into the HTML::D3 output
with a "Back to table" link (returning to the exact URL the user was
on) and "Export SVG" / "Export PNG" buttons. SVG is serialized with
XMLSerializer; PNG is rasterized via a Canvas element.
Non-numeric Y values (currency symbols, commas, etc.) are stripped and
the numeric portion is used; rows where Y is still non-numeric after
stripping are silently skipped.
[ Bug Fixes ]
- Drag-and-dropped files whose original filesystem path cannot be
determined are no longer added to the "recently opened" list.
Previously the temporary .uploads/<tempdir>/ path was recorded as a
fallback, producing a useless entry that pointed inside the app's
upload cache. Now, when no file:// URI is available from the
DataTransfer text/uri-list data (e.g. Firefox, email attachments),
the entry is omitted entirely. Files whose original path IS
recoverable continue to appear in the list under that path.
[ Test Improvements ]
- Added Transaction 20 subtest covering graph UI polish and date-sort JS:
verifies biDateKey helper and monthFirst detection are present, Plot
precedes Cancel in the graph panel DOM, chart centering CSS is injected,
and the back link carries the correct colour/size/weight.
0.003.2 Wed Aug 26 09:24:45 PM EDT 2026
[ Bug Fixes ]
- Large CSV files (> 16 KB) with column names containing spaces — such as
bank-export files with an "Account Number" column — now render correctly.
Files exceeding Database::Abstraction's default 16 KB slurp threshold
previously fell through to the DBD::CSV SQL path, which sanitizes column
names (lowercases them and replaces spaces with underscores, so "Account
Number" becomes "account_number"). The template then tried to access the
original name in a restricted hash that only permitted the sanitized key,
producing "Attempt to access disallowed key 'Account Number' in a
restricted hash". DataSource::_detect_file_info now returns the file
size, and _init_backend passes it as max_slurp_size to
Database::Abstraction, ensuring the Text::xSV::Slurp path is always
taken for CSV/PSV files so original column names are preserved regardless
of file size.
[ Test Improvements ]
- Added path-K and path-K2 subtests in t/path.t asserting that
_detect_file_info now returns file_size in its result hashref for
CSV/PSV files and omits it for the empty-hashref (non-CSV) case.
- Added end-to-end regression subtest BV-large-spaced-header in t/domain.t:
creates a > 16 KB CSV with "Account Number" as a column header, opens
it via /open, and asserts all rows are visible and no restricted-hash
crash occurs.
0.003.1 Wed Aug 26 04:54:25 PM EDT 2026
[ Enhancements ]
- Allow files to be removed from the recently opened and recently saved
lists
[ Bug Fixes ]
- Drag-and-dropped files now appear in "Recently opened" under their
original filesystem location, not the temporary .uploads/ copy. The
drop handler now extracts the file:// URI from the DataTransfer
text/uri-list item (provided by desktop file managers such as Nautilus
and Finder) and stashes it in sessionStorage before starting the
upload. The dashboard page then reads that stash and records the real
path. If no file:// URI is available (browser drag, email attachment,
etc.), the entry is omitted entirely so a useless temp path never
- Dragging and dropping the same file more than once no longer creates
duplicate entries in the "Recently opened" section on the home page.
Each upload creates a new random .uploads/<tempdir>/ subdirectory, so
the href changed on every drop even for the same file, bypassing the
existing href-based deduplication. The dedup key is now normalised:
upload entries are keyed by display name ("upload:<filename>"); all
other entries (browser-opened files, URL imports) continue to be keyed
by their full href so that two different files with the same name from
different directories can both appear in the list.
- Opening a file whose name has a mixed-case stem (e.g. AccountHistory.csv)
now works correctly on case-sensitive Linux filesystems. The controller
was lowercasing the stem before calling open_table, causing DataSource to
look for "accounthistory.csv" which does not exist. The stem is now
passed verbatim from the filesystem path; URL-sourced names (from
/view/:table and table: join specs) are still normalised to lowercase as
before because data/ files are lowercase by convention.
- CSV files whose first column header is not a valid SQL identifier (e.g.
"Account Number" from a bank export) now open without error.
DataSource::_detect_file_info previously passed the first column name
directly to Database::Abstraction as the id parameter; D::A validates id
against $SAFE_IDENTIFIER and croaked "unsafe id column name".
_detect_file_info now scans the header and picks the first column name
that IS a safe identifier.
- CSV files where the first safe-identifier column is always empty (e.g.
"Check" in a US bank statement) now show all rows. Database::Abstraction
uses empty_is_undef => 1, so an empty Check cell becomes undef and the
row-existence sentinel grep discards every row where Check is blank.
_detect_file_info now reads the first data row and skips safe-identifier
columns whose value there is empty, landing on a reliably populated column
(e.g. "Description") as the id.
- If no column header in a CSV satisfies the safe-identifier constraint,
DataSource now croaks error_no_safe_id with a clear message asking the
user to rename at least one column, rather than silently returning 0 rows.
- Bumped minimum CGI::Lingua to fix https://www.cpantesters.org/cpan/report/7dcee1d4-b74d-11f1-882f-ce9e7f79262c
[ Test Improvements ]
- Added targeted tests in t/function.t, t/path.t, and t/domain.t that
would have caught all three bugs above before they reached production:
unit-level _detect_file_info subtests for each id-selection scenario,
new CFG paths G-J in path.t, and end-to-end HTTP domain subtests that
assert all rows are visible for mixed-case stems, spaced column headers,
and empty sentinel columns.
0.003.0 Wed Aug 26 16:54:40 UTC 2026
[ Enhancements ]
- Platform and language are now detected per-request rather than read
statically from config. CGI::Info inspects the User-Agent header to
select 'mobile', 'tablet', or 'web'; CGI::Lingua selects the best
ISO 639-1 language from the Accept-Language header against the set of
languages that have a template directory under templates/<platform>/.
Config values (platform, language) remain as fallbacks for when a
module is unavailable, the header is absent, or no template directory
exists for the detected value.
- Add "Refresh" button (↻) to the toolbar next to Export. Pressing it
reloads the current view from disk, picking up any changes to the
source file(s) since the page was last loaded. All URL state (joins,
filters, dedup) is preserved because location.reload() reuses the
current URL.
- Add "Hide duplicates / Show duplicates" toggle button to the toolbar.
When active (filled blue), duplicate rows are removed from the displayed
table and from any CSV or SQLite export. The button text reflects the
current state: "Hide duplicates" when duplicates are visible, "Show
duplicates" when they are suppressed. State is carried in the ?d=1
query parameter so the URL remains shareable and bookmarkable.
- Added "Clear upload cache" button (POST /uploads/clear).
Mojo::File->list() requires {dir => 1} to
include subdirectories; without it the upload subdirs are silently
skipped and the action always reports 0 files freed.
[ Bug Fixes ]
- Files opened via the "recently saved" section are no longer also added
to the "recently opened" list. The saved-section card href now carries
a transient ?from=saved marker; the dashboard JS reads and removes it
via history.replaceState (keeping the address bar clean) and skips the
bi:recent write when the marker is present.
- Recently saved files on the home page now open with a single click;
double-click was required before, making the feature effectively
inaccessible on touch devices and to users who did not discover the
interaction. Missing files (greyed-out cards) still block navigation.
0.002.0 Wed Aug 26 08:21:12 EDT 2026
[ Enhancements ]
- Add "Recently saved" section to the home page: the last 5 files written
via POST /export appear as cards below "Recently opened". Each card
fetches /api/stat on page load -- missing files are immediately greyed
out and their cards block navigation. Mouseover shows saved-on date,
last-modified time, file size, and a "File not found on disk" warning
when the file has been moved or deleted.
- Add "Combine data" feature (GET /combine): stack rows from two or more
data files vertically into a unified view. All columns from all sources
appear as headers; cells are blank where a source file lacks that column.
Equivalent to a SQL UNION ALL across heterogeneous schemas.
- New "Combine data..." toolbar button (teal) sits next to "Merge data...";
both panels are mutually exclusive (opening one closes the other).
- Export pipeline (GET /export, POST /export) now handles c= params so
combined views can be downloaded or saved to disk.
- Internal helpers migrated from Sub::Private (:Private) to Sub::Protected
(:Protected) in Dashboard.pm and DataSource.pm for correct OO dispatch
in production (morbo/hypnotoad) without the CHECK-phase stash-deletion
side-effect that made :Private OO calls fail at runtime.
- "Merge data" button tooltip improved; join chip no longer shows key
annotation for combine summaries that have no join key.
0.001.0 Tue Aug 25 08:14:37 PM EDT 2026
- Initial CPAN release.
- Mojolicious web application presenting arbitrary flat data files
(CSV, PSV, XML, SQLite) as styled, sortable, reorderable HTML tables.
- Home page card grid; filesystem browser (/browse, /open).
- In-memory left-join engine (/join) with repeatable join steps.
- Server-side result filters (eq, ne, contains, starts, lt, le, gt, ge,
empty, notempty operators).
- CSV and SQLite export via GET /export (stream download) and
POST /export (filesystem write).
- Drag-and-drop file upload (POST /upload).
- JSON APIs: /api/columns, /api/dirs, /api/stat.
- URL import: fetch and display an HTML table from any public URL
(/import).
- localStorage persistence for column order and sort state.
Keyboard Shortcuts
Global
s
Focus search bar
?
Bring up this help dialog
GitHub
gp
Go to pull requests
gi
Go to GitHub issues (only if GitHub is preferred repository)