NAME

I18N::LangTags - functions for dealing with RFC1766-style language tags

SYNOPSIS

use I18N::LangTags qw(is_language_tag same_language_tag
                      extract_language_tags super_languages
                      similarity_language_tag is_dialect_of
                      locale2language_tag alternate_language_tags
                      encode_language_tag
                     );

...or whatever of those functions you want to import. Those are all the exportable functions -- you're free to import only some, or none at all. By default, none are imported.

If you don't import any of these functions, assume a &I18N::LangTags:: in front of all the function names in the following examples.

DESCRIPTION

Language tags are a formalism, described in RFC 1766, for declaring what language form (language and possibly dialect) a given chunk of information is in.

This library provides functions for common tasks involving language tags as they are needed in a variety of protocols and applications.

Please see the "See Also" references for a thorough explanation of how to correctly use language tags.

the function is_language_tag($lang1)

Returns true iff $lang1 is a formally valid language tag.

is_language_tag("fr")            is TRUE
is_language_tag("x-jicarilla")   is FALSE
    (Subtags can be 8 chars long at most -- 'jicarilla' is 9)

is_language_tag("i-Klikitat")    is TRUE
    (True without regard to the fact noone has actually
     registered Klikitat -- it's a formally valid tag)

is_language_tag("fr-patois")     is TRUE
    (Formally valid -- altho descriptively weak!)

is_language_tag("Spanish")       is FALSE
is_language_tag("french-patois") is FALSE
    (No good -- first subtag has to match
     /^([xXiI]|[a-zA-Z]{2})$/ -- see RFC1766)

the function extract_language_tags($whatever)

Returns a list of whatever looks like formally valid language tags in $whatever. Not very smart, so don't get too creative with what you want to feed it.

extract_language_tags("fr, fr-ca, i-mingo")
  returns:   ('fr', 'fr-ca', 'i-mingo')

extract_language_tags("It's like this: I'm in fr -- French!")
  returns:   ('It', 'in', 'fr')
(So don't just feed it any old thing.)

the function same_language_tag($lang1, $lang2)

Returns true iff $lang1 and $lang2 are acceptable variant tags representing the same language-form.

same_language_tag('x-kadara', 'i-kadara')  is TRUE
   (The x/i- alternation doesn't matter)
same_language_tag('X-KADARA', 'i-kadara')  is TRUE
   (...and neither does case)
same_language_tag('en',       'en-US')     is FALSE
   (all-English is not the SAME as US English)
same_language_tag('x-kadara', 'x-kadar')   is FALSE
   (these are totally unrelated tags)

same_language_tag works by just seeing whether encode_language_tag($lang1) is the same as encode_language_tag($lang2).

(Yes, I know this function is named a bit oddly. Call it historic reasons.)

the function similarity_language_tag($lang1, $lang2)

Returns an integer representing the degree of similarity between tags $lang1 and $lang2 (the order of which does not matter), where similarity is the number of common elements on the left, without regard to case and to x/i- alternation.

similarity_language_tag('fr', 'fr-ca')           is 1
   (one element in common)
similarity_language_tag('fr-ca', 'fr-FR')        is 1
   (one element in common)

similarity_language_tag('fr-CA-joual',
                        'fr-CA-PEI')             is 2
similarity_language_tag('fr-CA-joual', 'fr-CA')  is 2
   (two elements in common)

similarity_language_tag('x-kadara', 'i-kadara')  is 1
   (x/i- doesn't matter)

similarity_language_tag('en',       'x-kadar')   is 0
similarity_language_tag('x-kadara', 'x-kadar')   is 0
   (unrelated tags -- no similarity)

similarity_language_tag('i-cree-syllabic',
                        'i-cherokee-syllabic')   is 0
   (no B<leftmost> elements in common!)

the function is_dialect_of($lang1, $lang2)

Returns true iff language tag $lang1 represents a subdialect of language tag $lang2.

Get the order right! It doesn't work the other way around!

is_dialect_of('en-US', 'en')            is TRUE
  (American English IS a dialect of all-English)

is_dialect_of('fr-CA-joual', 'fr-CA')   is TRUE
is_dialect_of('fr-CA-joual', 'fr')      is TRUE
  (Joual is a dialect of (a dialect of) French)

is_dialect_of('en', 'en-US')            is FALSE
  (all-English is a NOT dialect of American English)

is_dialect_of('fr', 'en-CA')            is FALSE

is_dialect_of('en',    'en'   )            is TRUE
is_dialect_of('en-US', 'en-US')            is TRUE
  (B<Note:> these are degenerate cases)

is_dialect_of('i-mingo-tom', 'x-Mingo') is TRUE
  (the x/i thing doesn't matter, nor does case)

the function super_languages($lang1)

Returns a list of language tags that are superordinate tags to $lang1 -- it gets this by removing subtags from the end of $lang1 until nothing (or just "i" or "x") is left.
```
super_languages("fr-CA-joual")  is  ("fr-CA", "fr")

super_languages("en-AU")  is  ("en")

super_languages("en")  is  empty-list, ()

super_languages("i-cherokee")  is  empty-list, ()
 ...not ("i"), which would be illegal as well as pointless.
```
If $lang1 is not a valid language tag, returns empty-list in a list context, undef in a scalar context.

A notable and rather unavoidable problem with this method: "x-mingo-tom" has an "x" because the whole tag isn't an IANA-registered tag -- but super_languages('x-mingo-tom') is ('x-mingo') -- which isn't really right, since 'i-mingo' is registered. But this module has no way of knowing that. (But note that same_language_tag('x-mingo', 'i-mingo') is TRUE.)

More importantly, you assume at your peril that superordinates of $lang1 are mutually intelligible with $lang1. Consider this carefully.
the function locale2language_tag($locale_identifier)

This takes a locale name (like "en", "en_US", or "en_US.ISO8859-1") and maps it to a language tag. If it's not mappable (as with, notably, "C" and "POSIX"), this returns empty-list in a list context, or undef in a scalar context.
```
locale2language_tag("en") is "en"

locale2language_tag("en_US") is "en-US"

locale2language_tag("en_US.ISO8859-1") is "en-US"

locale2language_tag("C") is undef or ()

locale2language_tag("POSIX") is undef or ()

locale2language_tag("POSIX") is undef or ()
```
I'm not totally sure that locale names map satisfactorily to language tags. Think REAL hard about how you use this. YOU HAVE BEEN WARNED.
the function encode_language_tag($lang1)

This function, if given a language tag, returns an encoding of it such that:

* tags representing different languages never get the same encoding.

* tags representing the same language always get the same encoding.

* an encoding of a formally valid language tag always is a string value that is defined, has length, and is true if considered as a boolean.

Note that the encoding itself is not a formally valid language tag. Note also that you cannot, currently, go from an encoding back to a language tag that it's an encoding of.

Note also that you must consider the encoded value as atomic; i.e., you should not consider it as anything but an opaque, unanalysable string value. (The internals of the encoding method may change in future versions, as the language tagging standard changes over time.)

encode_language_tag returns undef if given anything other than a formally valid language tag.

The reason encode_language_tag exists is because different language tags may represent the same language; this is normally treatable with same_language_tag, but consider this situation:

You have a data file that expresses greetings in different languages. Its format is "[language tag]=[how to say 'Hello']", like:
```
en-US=Hiho
fr=Bonjour
i-mingo=Hau'
```
And suppose you write a program that reads that file and then runs as a daemon, answering client requests that specify a language tag and then expect the string that says how to greet in that language. So an interaction looks like:
```
greeting-client asks:    fr
greeting-server answers: Bonjour
```
So far so good. But suppose the way you're implementing this is:
```
my %greetings;
die unless open(IN, "<in.dat");
while(<IN>) {
  chomp;
  next unless /^([^=]+)=(.+)/s;
  my($lang, $expr) = ($1, $2);
  $greetings{$lang} = $expr;
}
close(IN);
```
at which point %greetings has the contents:
```
"en-US"   => "Hiho"
"fr"      => "Bonjour"
"i-mingo" => "Hau'"
```
And suppose then that you answer client requests for language $wanted by just looking up $greetings{$wanted}.

If the client asks for "fr", that will look up successfully in %greetings, to the value "Bonjour". And if the client asks for "i-mingo", that will look up successfully in %greetings, to the value "Hau'".

But if the client asks for "i-Mingo" or "x-mingo", or "Fr", then the lookup in %greetings fails. That's the Wrong Thing.

You could instead do lookups on $wanted with:
```
use I18N::LangTags qw(same_language_tag);
my $repsonse = '';
foreach my $l2 (keys %greetings) {
  if(same_language_tag($wanted, $l2)) {
    $response = $greetings{$l2};
    last;
  }
}
```
But that's rather inefficient. A better way to do it is to start your program with:
```
use I18N::LangTags qw(encode_language_tag);
my %greetings;
die unless open(IN, "<in.dat");
while(<IN>) {
  chomp;
  next unless /^([^=]+)=(.+)/s;
  my($lang, $expr) = ($1, $2);
  $greetings{
              encode_language_tag($lang)
            } = $expr;
}
close(IN);
```
and then just answer client requests for language $wanted by just looking up
```
$greetings{encode_language_tag($wanted)}
```
And that does the Right Thing.
the function alternate_language_tags($lang1)

This function, if given a language tag, returns all language tags that are alternate forms of this language tag. (There is little alternation in the current language tagging formalism, but extensions to the formalism are under consideration which could add a great deal of alternation.)

Examples from the current formalism:
```
alternate_language_tags('en')           is   ()
alternate_language_tags('x-mingo-tom')  is   ('i-mingo-tom')
alternate_language_tags('x-klikitat')   is   ('i-klikitat')
alternate_language_tags('i-klikitat')   is   ('x-klikitat')
```
This function returns undef if given anything other than a formally valid language tag.

NOTE ABOUT FUTURE VERSIONS

I will need to change this library when RFC1766 is superceded. Currently (1999-03-06) there are plans to do just that, by adding a whole series of three-letter language codes like "nav" for Navajo -- which would currently be a formally invalid language code. Be sure to check CPAN for updates of this library.

ABOUT LOWERCASING

I've considered making all the above functions that output language tags return all those tags strictly in lowercase. Having all your language tags in lowercase does make some things easier. But you might as well just lowercase as you like, or call encode_language_tag($lang1) where appropriate.

ABOUT UNICODE PLAINTEXT LANGUAGE TAGS

In some future version of I18N::LangTags, I plan to include support for RFC2482-style language tags -- which are basically just normal language tags with their ASCII characters shifted into Plane 14.

COPYRIGHT

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

AUTHOR

Sean M. Burke sburke@netadventure.net

To install I18N::LangTags, copy and paste the appropriate command in to your terminal.

cpanm

cpanm I18N::LangTags

CPAN shell

perl -MCPAN -e shell
install I18N::LangTags

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)