#!perl# Note: This script is a CLI for Riap function /App/td/td# and generated automatically using Perinci::CmdLine::Gen version 0.502use5.010001;usestrict;usewarnings;our$AUTHORITY='cpan:PERLANCAR';# AUTHORITYour$DATE='2024-06-26';# DATEour$DIST='App-td';# DISTour$VERSION='0.112';# VERSIONmy$cmdline= Perinci::CmdLine::Any->new(url=>"/App/td/td",program_name=>"td",read_config=> 0,read_env=> 0,);$cmdline->run;# ABSTRACT: Manipulate table data# PODNAME: td__END__=pod=encoding UTF-8=head1 NAMEtd - Manipulate table data=head1 VERSIONThis document describes version 0.112 of td (from Perl distribution App-td), released on 2024-06-26.=head1 SYNOPSISB<td> B<L<--help|/"--help, -h, -?">> (or B<L<-h|/"--help, -h, -?">>, B<L<-?|/"--help, -h, -?">>)B<td> B<L<--version|/"--version, -v">> (or B<L<-v|/"--version, -v">>)B<td> [B<L<--case-insensitive|/"--case-insensitive, -i">>|B<L<-i|/"--case-insensitive, -i">>] [B<L<--detail|/"--detail, -l">>|B<L<-l|/"--detail, -l">>|B<L<--no-detail|/"--detail, -l">>|B<L<--nodetail|/"--detail, -l">>] [(B<L<--exclude-column|/"--exclude-columns-json=s">>=I<str>)+|B<L<--exclude-columns-json|/"--exclude-columns-json=s">>=I<json>|(B<L<-E|/"--exclude-columns-json=s">>=I<str>)+] [B<L<--format|/"--format=s">>=I<name>|B<L<--json|/"--json">>] [(B<L<--include-column|/"--include-columns-json=s">>=I<str>)+|B<L<--include-columns-json|/"--include-columns-json=s">>=I<json>|(B<L<-I|/"--include-columns-json=s">>=I<str>)+] [B<L<--lines|/"--lines=s, -n">>=I<str>|B<L<-n|/"--lines=s, -n">>=I<str>] [B<L<--(no)naked-res|/"--naked-res">>] [B<L<--no-header-column|/"--no-header-column">>] [B<L<--page-result|/"--page-result">>[=I<program>]|B<L<--view-result|/"--view-result">>[=I<program>]] [B<L<--repeated|/"--repeated, -r">>|B<L<-r|/"--repeated, -r">>|B<L<--no-repeated|/"--repeated, -r">>|B<L<--norepeated|/"--repeated, -r">>] [B<L<--weight-column|/"--weight-column=s">>=I<str>] -- E<lt>I<L<action|/"--action=s*">>E<gt> [I<L<argv|/"--argv=s@">>] ...=head1 DESCRIPTIONB<What is td?>I<td> receives table data from standard input and performs an action on it. Ithas functionality similar to some Unix commands like I<head>, I<tail>, I<wc>,I<cut>, I<sort> except that it operates on table rows/columns instead oflines/characters. This is convenient to use with CLI scripts that output tabledata.B<What is table data?>A I<table data> is JSON-encoded data in the form of either: C<hos> (hash ofscalars, which is viewed as a two-column table where the columns are C<key> andC<value>), C<aos> (array of scalars, which is viewed as a 1-column array where thecolumn is C<elem>), C<aoaos> (array of arrays of scalars), or C<aohos> (array ofhashes of scalars).The input can also be an I<enveloped> table data, where the envelope is an array:C<[status, message, content, meta]> and C<content> is the actual table data. Thiskind of data is produced by C<Perinci::CmdLine>-based scripts and can containmore detailed table specification in the C<meta> hash, which C<td> can parse.B<What scripts/modules output table data?>CLI scripts that are written using L<Perinci::CmdLine> framework outputenveloped table data. There are at least hundreds of such scripts on CPAN. Someexamples include: L<lcpan> (from L<App::lcpan>), L<pmlist> (fromL<App::PMUtils>), and L<bencher> (from L<Bencher>).C<TableData::*> modules contain table data. They can easily be output to CLIusing the L<tabledata> utility (from L<App::TableDataUtils>).CSV output from any module/script can be easily converted to table data usingthe L<csv2td> utility:% csv2td YOUR.csv | td ...% program-that-outputs-csv | csv2td - | td ...Table data can also be converted from several other formats e.g. JSON, YAML,XLS/XLSX/ODS.B<What scripts/modules accept table data?>This I<td> script, for one, accepts table data.If a module/script expects CSV, you can feed it table data and convert the tabledata to CSV using L<td2csv> utility.Several other formats can also be converted to table data, e.g. JSON,YAML, XLS/XLSX/ODS.B<Using td>First you might want to use the C<info> action to see if the input is a tabledata:% osnames -l --json | td infoIf input is not valid JSON, a JSON parse error will be displayed. If input isvalid JSON but not a table data, another error will be displayed. Otherwise,information about the table will be displayed (form, number of columns, columnnames, number of rows, and so on).Next, you can use these actions:# List available actions% td actions# Convert table data (which might be hash, aos, or aohos) to aoaos form% list-files -l --json | td as-aoaos# Convert table data (which might be hash, aos, or aoaos) to aohos form% list-files -l --json | td as-aohos# Display table data on the browser using datatables (to allow interactive sorting and filtering)% osnames -l | td cat --format html+datatables# Convert table data to CSV% list-files -l --json | td as-csv# Calculate arithmetic average of numeric columns% list-files -l --json | td avg# Append a row at the end containing arithmetic average of number columns% list-files -l --json | td avg-row# Count number of columns% osnames -l --json | td colcount# Append a single-column row at the end containing number of columns% osnames -l --json | td colcount-row# Return the column names only% lcpan related-mods Perinci::CmdLine | td colnames# append a row containing column names% lcpan related-mods Perinci::CmdLine | td colnames-row# Only show first 5 rows% osnames -l --json | td head -n5# Show all but the last 5 rows% osnames -l --json | td head -n -5# Check if input is table data and show information about the table% osnames -l --json | td info# Count number of rows% osnames -l --json | td rowcount% osnames -l --json | td wc ;# shorter alias# Append a single-column row containing row count% osnames -l --json | td rowcount-row% osnames -l --json | td wc-row ;# shorter alias# Add a row number column (1, 2, 3, ...)% list-files -l --json | td rownum-col# Select some columns% osnames -l --json | td select value description# Select all columns but some% osnames -l --json | td select '*' -E value -E description# Return the rows in a random order% osnames -l --json | td shuf# Pick 5 random rows from input% osnames -l --json | td shuf -n5% osnames -l --json | td pick -n5 ;# synonym for 'shuf'# Sort by column(s) (add "-" prefix to for descending order)% osnames -l --json | td sort value tags% osnames -l --json | td sort -- -value# Return sum of all numeric columns% list-files -l --json | td sum# Append a sum row% list-files -l --json | td sum-row# Only show last 5 rows% osnames -l --json | td tail -n5# Show rows from the row 5 onwards% osnames -l --json | td tail -n +5# Remove adjacent duplicate rows:% command ... | td uniq% command ... | td uniq -i ;# case-insensitive% command ... | td uniq --repeated ;# only shows the duplicate rows% command ... | td uniq -i C1 -i C2 ;# only use columns C1 & C2 to check uniqueness% command ... | td uniq -E C5 -E C6 ;# use all columns but C5 & C6 to check uniqueness# Remove non-adjacent duplicate rows:% command ... | td nauniq% command ... | td nauniq -i ;# case-insensitive% command ... | td nauniq --repeated ;# only shows the duplicate rows% command ... | td nauniq -i C1 -i C2 ;# only use columns C1 & C2 to check uniqueness% command ... | td nauniq -E C5 -E C6 ;# use all columns but C5 & C6 to check uniqueness# Transpose table (make first column of rows as column names in the# transposed table)% osnames -l --json | td transpose# Transpose table (make columns named 'row1', 'row2', 'row3', ... in the# transposed table)% osnames -l --json | td transpose --no-header-column# Use Perl code to filter rows. Perl code gets row in $row or $_# (scalar/aos/hos) or $rowhash (always a hos) or $rowarray (always aos).# There are also $rownum (integer, starts at 0) and $td (table data object).# Perl code is eval'ed in the 'main' package with strict/warnings turned# off. The example below selects videos that are larger than 480p.% media-info *.mp4 | td grep 'use List::Util qw(min); min($_->{video_height}, $_->{video_width}) > 480'# Use Perl code to filter columns. Perl code gets column name in $colname or# $_. There's also $colidx (column index, from 1) and $td (table data# object). If table data form is 'hash' or 'aos', it will be transformed# into 'aoaos'. The example below only select even columns that match# /col/i. Note that most of the time, 'td select' is better. But when you# have a lot of columns and want to select them programmatically, you have# grep-col.% somecd --json | td grep-col '$colidx % 2 == 0 && /col/i'# Use Perl code to transform row. Perl code gets row in $row or $_# (scalar/hash/array) and is supposed to return the new row. As in 'grep',# $rowhash, $rowarray, $rownum, $td are also available as helper. The# example below adds a field called 'is_landscape'.% media-info *.jpg | td map '$_->{is_landscape} = $_->{video_height} < $_->{video_width} ? 1:0; $_'# Use perl code to sort rows. Perl sorter code gets row in $a & $b or $_[0]# & $_[1] (hash/array). Sorter code, like in Perl's standard sort(), is# expected to return -1/0/1. The example belows sort videos by height,# descendingly then by width, descendingly.% media-info *.mp4 | td psort '$b->{video_height} <=> $a->{video_height} || $b->{video_width} <=> $b->{video_width}'=head1 OPTIONSC<*> marks required options.=head2 Main options=over=item B<--action>=I<s>*Action to perform on input table.Valid values:["actions","as-aoaos","as-aohos","as-csv","avg","avg-row","cat","colcount","colcount-row","colnames","colnames-row","grep","grep-col","grep-row","head","info","map","map-row","nauniq","pick","psort","rowcount","rowcount-row","rownum-col","select","shuf","sort","sum","sum-row","tail","transpose","uniq","wc","wc-row"]Can also be specified as the 1st command-line argument.=item B<--argv-json>=I<s>Arguments (JSON-encoded).See C<--argv>.Can also be specified as the 2nd command-line argument and onwards.=item B<--argv>=I<s@>Arguments.Default value:[]Can also be specified as the 2nd command-line argument and onwards.Can be specified multiple times.=back=head2 Actions action options=over=item B<--detail>, B<-l>(No description)=back=head2 Head action options=over=item B<--lines>=I<s>, B<-n>(No description)=back=head2 Nauniq action options=over=item B<--case-insensitive>, B<-i>(No description)=item B<--exclude-column>=I<s@>, B<-E>(No description)Can be specified multiple times.=item B<--exclude-columns-json>=I<s>See C<--exclude-column>.=item B<--include-column>=I<s@>, B<-I>(No description)Can be specified multiple times.=item B<--include-columns-json>=I<s>See C<--include-column>.=item B<--repeated>, B<-r>Allow/show duplicates.For shuf/pick actions, setting this option means sampling with replacement whichmakes a single row can be sampled/picked multiple times. The default is tosample without replacement.For uniq/nauniq actions, setting this option means instructing to returnduplicate rows instead of the unique rows.=back=head2 Output options=over=item B<--format>=I<s>Choose output format, e.g. json, text.Default value:undefOutput can be displayed in multiple formats, and a suitable default format ischosen depending on the application and/or whether output destination isinteractive terminal (i.e. whether output is piped). This option specificallychooses an output format.=item B<--json>Set output format to json.=item B<--naked-res>When outputing as JSON, strip result envelope.Default value:0By default, when outputing as JSON, the full enveloped result is returned, e.g.:[200,"OK",[1,2,3],{"func.extra"=>4}]The reason is so you can get the status (1st element), status message (2ndelement) as well as result metadata/extra result (4th element) instead of justthe result (3rd element). However, sometimes you want just the result, e.g. whenyou want to pipe the result for more post-processing. In this case you can useC<--naked-res> so you just get:[1,2,3]=item B<--page-result>Filter output through a pager.This option will pipe the output to a specified pager program. If pager programis not specified, a suitable default e.g. C<less> is chosen.=item B<--view-result>View output using a viewer.This option will first save the output to a temporary file, then open a viewerprogram to view the temporary file. If a viewer program is not chosen, asuitable default, e.g. the browser, is chosen.=back=head2 Pick action options=over=item B<--lines>=I<s>, B<-n>(No description)=item B<--repeated>, B<-r>Allow/show duplicates.For shuf/pick actions, setting this option means sampling with replacement whichmakes a single row can be sampled/picked multiple times. The default is tosample without replacement.For uniq/nauniq actions, setting this option means instructing to returnduplicate rows instead of the unique rows.=item B<--weight-column>=I<s>Select a column that contains weight.=back=head2 Select action options=over=item B<--exclude-column>=I<s@>, B<-E>(No description)Can be specified multiple times.=item B<--exclude-columns-json>=I<s>See C<--exclude-column>.=item B<--include-column>=I<s@>, B<-I>(No description)Can be specified multiple times.=item B<--include-columns-json>=I<s>See C<--include-column>.=back=head2 Shuf action options=over=item B<--repeated>, B<-r>Allow/show duplicates.For shuf/pick actions, setting this option means sampling with replacement whichmakes a single row can be sampled/picked multiple times. The default is tosample without replacement.For uniq/nauniq actions, setting this option means instructing to returnduplicate rows instead of the unique rows.=item B<--weight-column>=I<s>Select a column that contains weight.=back=head2 Tail action options=over=item B<--lines>=I<s>, B<-n>(No description)=back=head2 Transpose action options=over=item B<--no-header-column>Don't make the first column as column names of the transposed table; instead create column named 'row1', 'row2', ....=back=head2 Uniq action options=over=item B<--case-insensitive>, B<-i>(No description)=item B<--exclude-column>=I<s@>, B<-E>(No description)Can be specified multiple times.=item B<--exclude-columns-json>=I<s>See C<--exclude-column>.=item B<--include-column>=I<s@>, B<-I>(No description)Can be specified multiple times.=item B<--include-columns-json>=I<s>See C<--include-column>.=item B<--repeated>, B<-r>Allow/show duplicates.For shuf/pick actions, setting this option means sampling with replacement whichmakes a single row can be sampled/picked multiple times. The default is tosample without replacement.For uniq/nauniq actions, setting this option means instructing to returnduplicate rows instead of the unique rows.=back=head2 Other options=over=item B<--help>, B<-h>, B<-?>Display help message and exit.=item B<--version>, B<-v>Display program's version and exit.=back=head1 COMPLETIONThis script has shell tab completion capability with support for severalshells.=head2 bashTo activate bash completion for this script, put:complete -C td tdin your bash startup (e.g. F<~/.bashrc>). Your next shell session will thenrecognize tab completion for the command. Or, you can also directly execute theline above in your shell to activate immediately.It is recommended, however, that you install modules using L<cpanm-shcompgen>which can activate shell completion for scripts immediately.=head2 tcshTo activate tcsh completion for this script, put:complete td 'p/*/`td`/'in your tcsh startup (e.g. F<~/.tcshrc>). Your next shell session will thenrecognize tab completion for the command. Or, you can also directly execute theline above in your shell to activate immediately.It is also recommended to install L<shcompgen> (see above).=head2 other shellsFor fish and zsh, install L<shcompgen> as described above.=head1 HOMEPAGEPlease visit the project's homepage at L<https://metacpan.org/release/App-td>.=head1 SOURCESource repository is at L<https://github.com/perlancar/perl-App-td>.=head1 AUTHORperlancar <perlancar@cpan.org>=head1 CONTRIBUTINGTo contribute, you can send patches by email/via RT, or send pull requests onGitHub.Most of the time, you don't need to build the distribution yourself. You cansimply modify the code, then test via:% prove -lIf you want to build the distribution (e.g. to try to install it locally on yoursystem), you can install L<Dist::Zilla>,L<Dist::Zilla::PluginBundle::Author::PERLANCAR>,L<Pod::Weaver::PluginBundle::Author::PERLANCAR>, and sometimes one or two otherDist::Zilla- and/or Pod::Weaver plugins. Any additional steps required beyondthat are considered a bug and can be reported to me.=head1 COPYRIGHT AND LICENSEThis software is copyright (c) 2024, 2023, 2022, 2021, 2020, 2019, 2017, 2016, 2015 by perlancar <perlancar@cpan.org>.This is free software; you can redistribute it and/or modify it underthe same terms as the Perl 5 programming language system itself.=head1 BUGSPlease report any bugs or feature requests on the bugtracker website L<https://rt.cpan.org/Public/Dist/Display.html?Name=App-td>When submitting a bug or request, please include a test-file or apatch to an existing test-file that illustrates the bug or desiredfeature.=cut