/ 
Unicode-Stringprep-1.09_70091229
River stage two • 6 direct dependents • 87 total dependents
/ Unicode::Stringprep

NAME

Unicode::Stringprep - Preparation of Internationalized Strings (RFC 3454)

SYNOPSIS

use Unicode::Stringprep;
use Unicode::Stringprep::Mapping;
use Unicode::Stringprep::Prohibited;

my $prepper = Unicode::Stringprep->new(
  3.2,
  [ { 32 => '<SPACE>'},  ],
  'KC',
  [ @Unicode::Stringprep::Prohibited::C12, @Unicode::Stringprep::Prohibited::C22,
    @Unicode::Stringprep::Prohibited::C3, @Unicode::Stringprep::Prohibited::C4,
    @Unicode::Stringprep::Prohibited::C5, @Unicode::Stringprep::Prohibited::C6,
    @Unicode::Stringprep::Prohibited::C7, @Unicode::Stringprep::Prohibited::C8,
    @Unicode::Stringprep::Prohibited::C9 ],
  1 );
$output = $prepper->($input)

DESCRIPTION

This module implements the stringprep framework for preparing Unicode text strings in order to increase the likelihood that string input and string comparison work in ways that make sense for typical users throughout the world. The stringprep protocol is useful for protocol identifier values, company and personal names, internationalized domain names, and other text strings.

The stringprep framework does not specify how protocols should prepare text strings. Protocols must create profiles of stringprep in order to fully specify the processing options.

FUNCTIONS

This module provides a single function, new, that creates a perl function implementing a stringprep profile.

This module exports nothing.

new($unicode_version, $mapping_tables, $unicode_normalization, $prohibited_tables, $bidi_check)

Creates a blessed function reference that implements a stringprep profile.

$unicode_version is the Unicode version specified by the stringprep profile. Currently, this parameter must be 3.2.

$mapping_tables provides the mapping tables used for stringprep. It may be a reference to a hash or an array. A hash must map Unicode codepoints (as integers, e. g. 0x0020 for U+0020) to replacement strings (as perl strings). An array may contain pairs of Unicode codepoints and replacement strings as well as references to nested hashes and arrays. Unicode::Stringprep::Mapping provides the tables from RFC 3454, Appendix B. For further information on the mapping step, see RFC 3454, section 3.

$unicode_normalization is the Unicode normalization to be used. Currently, '' (no normalization) and 'KC' (compatibility composed) are specified for stringprep. For further information on the normalization step, see RFC 3454, section 4.

$prohibited_tables provides the list of prohibited output characters for stringprep. It may be a reference to an array. The array contains pairs of codepoints, which define the start and end of a Unicode character range (as integers). The end character may be undef, specifying a single-character range. The array may also contain references to nested arrays. Unicode::Stringprep::Prohibited provides the tables from RFC 3454, Appendix C. For further information on the prohibition checking step, see RFC 3454, section 5.

$bidi_check must be set to true if additional checks for bidirectional characters are required. For further information on the bidi checking step, see RFC 3454, section 6.

The function returned can be called with a single parameter, the string to be prepared, and returns the prepared string. It will die if the input string is invalid (so use eval if necessary).

For performance reasons, it is strongly recommended to call the new function as few times as possible, i. e. once per stringprep profile. It might also be better not to use this module directly but to use (or write) a module implementing a profile, such as Net::IDN::Nameprep.

IMPLEMENTING PROFILES

You can easily implement a stringprep profile without subclassing:

package ACME::ExamplePrep;

use Unicode::Stringprep;

use Unicode::Stringprep::Mapping;
use Unicode::Stringprep::Prohibited;

*exampleprep = Unicode::Stringprep->new(
  3.2,
  [ \@Unicode::Stringprep::Mapping::B1, ],
  '',
  [ \@Unicode::Stringprep::Prohibited::C12,
    \@Unicode::Stringprep::Prohibited::C22, ],
  1,
);

This binds ACME::ExamplePrep::exampleprep to the function created by Unicode::Stringprep->new.

Usually, it is not necessary to subclass this module. Sublassing this module is not recommended.

DATA TABLES

The following modules contain the data tables from RFC 3454. These modules are automatically loaded when loading Unicode::Stringprep.

  • Unicode::Stringprep::Unassigned

    @Unicode::Stringprep::Unassigned::A1	# Appendix A.1

  • Unicode::Stringprep::Mapping

    @Unicode::Stringprep::Mapping::B1	# Appendix B.1
@Unicode::Stringprep::Mapping::B2	# Appendix B.2
@Unicode::Stringprep::Mapping::B2	# Appendix B.3

  • Unicode::Stringprep::Prohibited

    @Unicode::Stringprep::Prohibited::C11	# Appendix C.1.1
@Unicode::Stringprep::Prohibited::C12	# Appendix C.1.2
@Unicode::Stringprep::Prohibited::C21	# Appendix C.2.1
@Unicode::Stringprep::Prohibited::C22	# Appendix C.2.2
@Unicode::Stringprep::Prohibited::C3	# Appendix C.3
@Unicode::Stringprep::Prohibited::C4	# Appendix C.4
@Unicode::Stringprep::Prohibited::C5	# Appendix C.5
@Unicode::Stringprep::Prohibited::C6	# Appendix C.6
@Unicode::Stringprep::Prohibited::C7	# Appendix C.7
@Unicode::Stringprep::Prohibited::C8	# Appendix C.8
@Unicode::Stringprep::Prohibited::C9	# Appendix C.9

  • Unicode::Stringprep::BiDi

    @Unicode::Stringprep::BiDi::D1	# Appendix D.1
@Unicode::Stringprep::BiDi::D2	# Appendix D.2

CAVEATS

perl Version

You should use perl 5.8.3 or higher.

While this module does work with earlier perl versions, there are some limitations:

Perl 5.6 does not promote strings to UTF-8 automatically. You have to make sure that you only pass valid UTF-8 strings to this module.

Perl 5.6 to 5.7 come with Unicode databases earlier than version 3.2. Strings that contain characters for which the normalisation has been changed are not prepared correctly.

Perl 5.6 to 5.8.2 can't handle surrogate characters (U+D800..U+DFFF) in strings. If a profile tries to map these characters, they won't be mapped (currently no stringprep profile does this). If a profile prohibits these characters, this module may fail to detect them (currently, all profiles do that, so you have to make sure that these characters are not present).

If any of these problems is detected, this module will issue a warning unless $Unicode::Stringprep::WARNINGS is set to false.

Unicode Corrigendum #5

Unicode 3.2, on which the Stringprep specification is based, contained a problem in the language of the specification for Normalization Forms NFC and NFKC.

This has been fixed in Unicode 4.1 (Corrigendum #5, http://unicode.org/versions/corrigendum5.html).

This module re-produces the old, broken normalization method because this is, strictly speaking, required by the Stringprep specification.

However, please note that other Stringprep implementations may simply ignore the issue or

AUTHOR

Claus Färber <CFAERBER@cpan.org>

LICENSE

Copyright 2007-2009 Claus Färber.

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

SEE ALSO

Unicode::Normalize, RFC 3454 (http://www.ietf.org/rfc/rfc3454.txt)