Changes for version 0.3424 - 2026-11-16

  • read_table()
    • keep_cols and drop_cols read faster when the columns left out are quoted, or the file is an .xlsx. A quoted field in such a column is now scanned for its closing quote without its text being copied, and an .xlsx cell in one is no longer built only to be freed. The table is unchanged. Times are the fastest of five processes, each reading once into an aoh on perl 5.44.0; peak memory did not change.
      • file kept before after 20,000 x 200 CSV, half the columns quoted 3 0.070 s 0.043 s 20,000 x 200 CSV, every column quoted 3 0.103 s 0.054 s 50,000 x 20 CSV, 10 quoted 200-byte texts 10 0.074 s 0.052 s 50,000 x 20 CSV, 10 quoted 3-line notes 10 0.090 s 0.071 s 10,000 x 100 .xlsx of numbers 3 0.139 s 0.084 s 10,000 x 100 .xlsx of shared strings 3 0.218 s 0.138 s 20,000 x 200 CSV, every column read 200 0.305 s 0.298 s 10,000 x 100 .xlsx, every column read 100 0.198 s 0.200 s
  • density()
    • density() now refuses an nb above 2147483647 with "invalid 'nb'", as bw_sj(), bw_ucv() and bw_bcv() already did; that is R's own bound, since as.integer() is NA above it. On a 32-bit perl with 64-bit integers an nb of 2**32 truncated to 0, and with bw => 'SJ', 'ucv' or 'bcv' the bandwidth step divided by it and wrote into an empty table.
    • density() needs about half the memory at large n. The interpolation grid and the output x values are now computed when needed rather than stored, each work buffer is freed as soon as it is dead rather than all at the end, and an output point's grid interval is found arithmetically instead of by bisection (bisection stays for a grid so narrow that adjacent points are equal). The estimate is unchanged bit for bit. Times are the fastest of five processes on perl 5.44.0, memory is the peak resident size:
      • x values n before after 1,000,000 512 0.094 s 134 MB 0.093 s 118 MB 100,000 4,194,304 4.13 s 663 MB 3.94 s 339 MB 1,000,000 16,777,216 21.20 s 2679 MB 20.44 s 1375 MB
  • interpolate()
    • interpolate() with method 'spline' or 'polynomial' refuses an order too large for a C int instead of truncating it: order => 4294967297 became 1 and was silently run as linear interpolation.
  • sum(), mean(), sd(), var()
    • When the running total overflows, these read their arguments a second time, which FETCHes a tied array again. That second read now croaks as the first does if an element has become undef, and croaks if the array's length has changed, instead of skipping the element or reading a different number of values and dividing by the first count.

Modules

Get basic statistical functions, like in R, but with Perl using XS for performance