/ 
App-CSVUtils-1.023
River stage one • 3 direct dependents • 4 total dependents
/ App::CSVUtils::Manual::Cookbook

NAME

App::CSVUtils::Manual::Cookbook - App::CSVUtils cookbook

VERSION

This document describes version 1.023 of App::CSVUtils::Manual::Cookbook (from Perl distribution App-CSVUtils), released on 2023-03-31.

DESCRIPTION

This document lists the various tasks you can do with the scripts included in App::CSVUtils distribution.

Example 1.csv content:

name,age,rank
andi,25,120
budi,28,115
cinta,19,142
derry,33,121

Example 2.csv content:

name,gender
andi,M
budi,M
cinta,F
erni,F

ADDING FIELDS

With csv-add-field, we add a new field named foo with the content calculated from Perl code:

% csv-add-fields 1.csv foo -e '$main::rownum * 2'
name,age,rank,foo
andi,25,120,4
budi,28,115,6
cinta,19,142,8
derry,33,121,10

You can specify at which position the new field will be using one of the options --at, --after, --before.

Adding a field containing line number

% csv-add-fields 1.csv linenum --at 1 -e '++$i'
linenum,name,age,rank
1,andi,25,120
2,budi,28,115
3,cinta,19,142
4,derry,33,121

Other ways:

% csv-add-fields 1.csv linenum --at 1 -e '$rownum-1'
% csv-add-fields 1.csv linenum --at 1 -e '$data_rownum'

See also: "Adding line number to CSV rows".

ADDING ROWS

TODO.

ADDING SUMMARY ROW (TOTAL, AVERAGE, ETC)

TODO.

CONVERTING TO OTHER FORMATS

To CSV

TODO.

To JSON

TODO.

To Perl data structure

TODO.

To TSV

TODO.

To YAML

TODO.

DEALING WITH NEWLINES IN CSV

TODO.

DEALING WITH NON-STANDARD CSV

The utilities allow you to set input's field separator (`--sep-char`), quote character (`--quote-char`), and escape character (`--escape-char`), so if you have semicolon as the field separator, e.g.:

name;age;rank
andi;20;staff
budi;30;manager
cinta;17;associate

and you want to sort by age:

% csv-sort-rows INPUT.CSV --sep-char ';' --by-field age

These parameters will be passed to Text::CSV's attributes with the corresponding names.

Likewise, you can customize output's field separator (`--output-sep-char`), quote character (`--output-quote-char`), and escape character (`--output-escape-char`).

FILTERING FIELDS

Keywords: selecting fields, grepping fields

With csv-select-fields:

% csv-select-fields 1.csv -f age -f name
age,name
25,andi
28,budi
19,cinta
33,derry

Note that the order of the fields is as specified.

You can select multiple fields using --include-field-pat option. You can exclude fields using --exclude-field and --exclude-field-pat.

FILTERING (SELECTING) ROWS

TODO.

GETTING INFORMATION ABOUT CSV

% csv-info 1.csv
{
   "data_row_count" : 6,
   "field_count" : 3,
   "fields" : [
      "name",
      "age",
      "rank"
   ],
   "file_size" : 78,
   "header_row_count" : 0,
   "row_count" : 6
}

Listing the field names

% csv-list-field-names 1.csv
+------+-------+
| name | index |
+------+-------+
| age  | 2     |
| name | 1     |
| rank | 3     |
+------+-------+

% csv-list-field-names 1.csv --json
[
   200,
   "OK",
   [
      {
         "index" : 2,
         "name" : "age"
      },
      {
         "index" : 1,
         "name" : "name"
      },
      {
         "index" : 3,
         "name" : "rank"
      }
   ],
   {
      "stream" : 0,
      "table.fields" : [
         "name",
         "index"
      ],
      "title" : "csv-list-field-names 1.csv --json"
   }
]

MERGING CSV FILES

Keywords: concatenating, combining.

See also: "SPLITTING CSV"

Merging CSV files

% csv-concat 1.csv 2.csv
name,age,rank,name,gender
andi,25,120,
budi,28,115,
cinta,19,142,
derry,33,121,
andi,,,M
budi,,,M
cinta,,,F
erni,,,F

Adding field containing filename

% csv-concat <(csv-add-fields 1.csv filename --at 1 -e '"1.csv"') <(csv-add-fields 2.csv filename --at 1 -e '"2.csv"')

MODIFYING FIELDS

Keywords: munging fields

MODIFYING ROWS

Keywords: munging rows

Adding line number to CSV rows

% csv-munge-rows 1.csv -e '$_->[0] = ++$i . "|$_->[0]"'
name,age,rank
1|andi,25,120
2|budi,28,115
3|cinta,19,142
4|derry,33,121

Other ways:

% csv-munge-rows 1.csv -e '$_->[0] = $data_rownum . "|$_->[0]"'

See also: "Adding a field containing line number".

Shuffling every row

% csv-munge-rows 1.csv -e 'use List::Util "shuffle"; $_=[shuffle @$_]'

MODIFYING VALUES

Quoting all values, including where quotes are not necessary

Sample file input.csv:

name,age
budi",21
"toni saputra, s.h",32

% csv-csv input.csv --output-always-quote
"name","age"
"budi","21"
"toni saputra, s.h","32"

Removing non-necessary quotes around values

Sample file input.csv:

"name","age"
"budi","21"
"toni saputra, s.h","32"

% csv-csv input.csv --no-output-always-quote
name,age
budi",21
"toni saputra, s.h",32

PERFORMING SET OPERATIONS WITH CSV

See csv-setop.

TODO.

SORTING FIELDS

See csv-sort-fields.

TODO.

SORTING ROWS

See csv-sort-rows.

TODO.

SPLITTING CSV

See csv-split.

TODO.

See also "MERGING CSV FILES".

TRANSPOSING

Transposing, like in a two-dimensional matrix, means switching row and colum indices to produce a new CSV where the fields become rows and vice versa. For example:

name,age
andi,20
budi,30
cinta,17

becomes:

name,andi,budi,cinta
age,20,30,17

To do this:

% csv-transpose INPUT.CSV

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/App-CSVUtils.

SOURCE

Source repository is at https://github.com/perlancar/perl-App-CSVUtils.

SEE ALSO

Acme::CPANModules::WorkingWithCSV

AUTHOR

perlancar <perlancar@cpan.org>

CONTRIBUTING

To contribute, you can send patches by email/via RT, or send pull requests on GitHub.

Most of the time, you don't need to build the distribution yourself. You can simply modify the code, then test via:

% prove -l

If you want to build the distribution (e.g. to try to install it locally on your system), you can install Dist::Zilla, Dist::Zilla::PluginBundle::Author::PERLANCAR, Pod::Weaver::PluginBundle::Author::PERLANCAR, and sometimes one or two other Dist::Zilla- and/or Pod::Weaver plugins. Any additional steps required beyond that are considered a bug and can be reported to me.

COPYRIGHT AND LICENSE

This software is copyright (c) 2023, 2022, 2021, 2020, 2019, 2018, 2017, 2016 by perlancar <perlancar@cpan.org>.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=App-CSVUtils

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.