From Code to Community: Sponsoring The Perl and Raku Conference 2025 Learn more

 / 
Locale-TextDomain-OO-Extract-2.017
River stage one • 1 direct dependent • 1 total dependent
/ Locale::TextDomain::OO::Extract

NAME Locale::TextDomain::OO::Extract - Extracts internationalization data

VERSION

2.017

DESCRIPTION

This module extracts internationalization data.

The extractor runs the following steps:

preprocess the file
find all possible positions in content
match exactly and create a stack
map the stack to a lexicon entry

If you want to write the lexicon structure to pot files see folder example of this distribution how it works.

SYNOPSIS

There are different extract module for different files. All this extracted data are stored into one or selective lexicons. At the end of extraction this lexicons can be stored into pot files or anywhere else.

Use an existing extractor


            

              
              
use strict;
use warnings;
use Locale::TextDomain::OO::Extract::...;
use Path::Tiny qw(path);
my $extractor = Locale::TextDomain::OO::Extract::...->new(
    # all parameters are optional
    lexicon_ref => \my %lexicon,  # default is {}
    domain      => 'MyDomain',    # default is q{}
    category    => 'LC_MESSAGES', # default is q{}
    debug_code  => sub {          # default is undef
        my ($group, $message) = @_;
        print $group, ', ', $message, "\n";
        return;
    },
);
my @files = (
    'relative_dir/filename1.suffix',
    'relative_dir/filename2.suffix',
);
for ( @files ) {
    $extractor->clear;
    $extractor->filename($_);
    $extractor->content_ref( \( path($_)->slurp_utf8 ) );
    $extractor->extract;
}
# do something with that
# maybe write a pot file using Locale::PO
... = extractor->lexicon_ref;

Write your own extractor


            

              
              
package MyExtractor;
use Moo;
extends qw(
    Locale::TextDomain::OO::Extract::Base::RegexBasedExtractor
);
with qw(
    Locale::TextDomain::OO::Extract::Role::File
);

Optional method to uncomment or interpolate the file content or anything else.


            

              
              
sub preprocess {
    my $self = shift;
    my $content_ref = $self->content_ref;
    # modify anyhow
    ${$content_ref}=~ s{\\n}{\n}xmsg;
    return;
}

Map the reference, all the matches or defaults. See Perl extractor how it works. Maybe ignore some stack entries.


            

              
              
sub stack_item_mapping {
    my $self = shift;
    my $match = $_->{match};
    $self->add_message({
        reference    => ...,
        domain       => ...,
        category     => ...,
        msgctxt      => ...,
        msgid        => ...,
        msgid_plural => ...,
    });
    return;
}

Match all positions, the searched string is starting with. You can match to the end of the searched string but there is no need for.


            

              
              
my $start_rule = qr{ ... }xms;

Match exactly the different things. All the values from () are stored in stack. Prepare the stack in a way you are able to merge the data. Maybe an empty match helps to have all on the right position.

'or' means: if not then try the following. 'or' is only allowed between 2 array references.


            

              
              
my $rules = [
    [
        'begin',
        qr{ ... ( ... ) ...}xms, # match this
        'and',
        qr{ ... ( ... ) ...}xms, # then that
        'end',
    ]
    'or',
    [
        'begin',
        [
            qr{ ... ( ... ) ... }xms,
        ],
        'or',
        [
            qr{ ... ( ... ) ... }xms,
            'or',
            qr{ ... ( ... ) ... }xms,
        ],
        'end',
    ],
];

Tell your extractor what steps he should run.


            

              
              
sub extract {
    my $self = shift;
    $self->start_rule($start_rule);
    $self->rules($rules);
    $self->preprocess;
    $self->SUPER::extract;
    for ( @{ $self->stack } ) {
        $self->stack_item_mapping;
    }
    return;
}

The whole process (extract, translate, clean, format)

See also module Locale::TextDomain::OO::Extract::Process


            

              
              
1.1. If the PO file not exists for a language create it with the right header
     or change the defaults after initial write (spew).
       |
       |     .------------------------------------.
       |     |                                    |
       v     v                                    |
   .-------------.                                |
   |    de.po    |-.                              |
   '-------------' |                              |
     '-------------'                              |
       |         |                                |
.------'   2.1. Read the existing PO files        |
|               of all languages                  |
|               into the lexicon sturcture.       |
|                |                                |
|                v                                |
|   .-------------------.                         |
|   | lexicon structure |<--------------------.   |
|   | de:: fr:: ...     |<----------------.   |   |
|   '-------------------'                 |   |   |
|       |       |   |                     ^   ^   ^
|       |       |   '-------------------->|->-|->-|->-.
|       |       |                         ^   ^   ^   |
|       |   3.1. Remove all gettext       |   |   |   |
|       |        references               |   |   |   |
|       |       |                         |   |   |   |
|       |       '-------------------------'   |   |   |
|       |                                     |   |   |
|       |   .------------------.              |   |   |
|       |   | MyProjectFile.pm |-.            |   |   |
|       |   '------------------' |            |   |   |
|       |     '------------------'            |   |   |
|       |              |                      |   |   |
|       |   4. Extract the files              |   |   |
|       |      of your project.               |   |   |
|       |   4.1. Add new message id's         |   |   |
|       |        if not already exists.       |   |   |
|       |   4.2. Add the new references.      |   |   |
|       |              |                      |   |   |
|       |              '----------------------'   |   |
|       |                                         |   |
|   5.1. Merge new and changed messages.          |   |
|   5.2. Write back the PO files.                 |   |
|       |                                         ^   |
|       '---------------------------------------->O   |
v                                                 ^   |
O->-----.                                         |   |
v       |                                         |   |
|   6.1. Translate the PO files.                  |   |
|       |                                         ^   |
|       '---------------------------------------->O   |
v                                                 ^   |
O->-----.                                         |   |
v       |                                         |   |
|   7.1. Clean all messages                       |   |
|        without any reference.                   |   |
|       |                                         |   |
|       '-----------------------------------------'   |
|                                                     |
'-------.                                             |
        |                                             |
    8.1. Write MO files.                              |
        |                                             |
        v                                             |
.-------------.                                       |
|    de.mo    |-.                                     |
'-------------' |                                     |
  '-------------'                                     |
                                                      |
Update the references in region files.                |
                                                      |
1.2. If the PO file not exists for a region           |
     create it with the right header.                 |
           |                                          |
           v                                          |
    .----------.                                      |
    | de-at.po |-.                                    v
    '----------' |<-----------------------------------O
      '----------'                                    ^
         |   |                                        |
.--------'   |                                        |
|            |                                        |
|   2.1. Read the existing PO files                   |
|        of all regions                               |
|        into the lexicon sturcture.                  |
|   3.2. Remove all references.                       |
|   4.3. Add the new references (from language)       |
|        for existing region messages only.           |
|   5.1. Merge changed messages.                      |
|   5.2. Write back to PO files.                      |
|   6.2. Translate the PO files.                      |
|   7.2. Clean all messages                           |
|        without any reference.                       |
|             |                                       |
|             '---------------------------------------'
'--------.
         |
8.2. Write MO files.
         |
         v
    .----------.
    | de-at.mo |-
    '----------' |
      '----------'

SUBROUTINES/METHODS

none

EXAMPLE

Inside of this distribution is a directory named example. Run this *.pl files.

DIAGNOSTICS

none

CONFIGURATION AND ENVIRONMENT

none

DEPENDENCIES

none

INCOMPATIBILITIES

not known

BUGS AND LIMITATIONS

none

SEE ALSO

Locale::TextDoamin::OO

AUTHOR

Steffen Winkler

LICENSE AND COPYRIGHT

Copyright (c) 2014 - 2025, Steffen Winkler <steffenw at cpan.org>. All rights reserved.

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.