NAME

Text::BibTeX - interface to read and parse BibTeX files

SYNOPSIS

use Text::BibTeX;

my $bibfile = Text::BibTeX::File->new("foo.bib");
my $newfile = Text::BibTeX::File->new(">newfoo.bib");

while ($entry = Text::BibTeX::Entry->new($bibfile))
{
   next unless $entry->parse_ok;

      .             # hack on $entry contents, using various
      .             # Text::BibTeX::Entry methods
      .

   $entry->write ($newfile);
}

DESCRIPTION

The Text::BibTeX module serves mainly as a high-level introduction to the Text::BibTeX library, for both code and documentation purposes. The code loads the two fundamental modules for processing BibTeX files (Text::BibTeX::File and Text::BibTeX::Entry), and this documentation gives a broad overview of the whole library that isn't available in the documentation for the individual modules that comprise it.

In addition, the Text::BibTeX module provides a number of miscellaneous functions that are useful in processing BibTeX data (especially the kind that comes from bibliographies as defined by BibTeX 0.99, rather than generic database files). These functions don't generally fit in the object-oriented class hierarchy centred around the Text::BibTeX::Entry class, mainly because they are specific to bibliographic data and operate on generic strings (rather than being tied to a particular BibTeX entry). These are also documented here, in "MISCELLANEOUS FUNCTIONS".

Note that every module described here begins with the Text::BibTeX prefix. For brevity, I have dropped this prefix from most class and module names in the rest of this manual page (and in most of the other manual pages in the library).

STRINGS AND UNICODE

By default (binmode => 'utf-8'), Text::BibTeX reads its input as UTF-8 and every string it gives you -- entry types, keys, field names, values, name parts, formatted names -- is a Perl character string, normalised once, when it is read, to Unicode Normalization Form C (or whichever form you ask for with the normalization option). Bytes that are not valid UTF-8 are replaced by U+FFFD, with a warning; to read files in other encodings use the encoding option of "open" in Text::BibTeX::File, or pass in a filehandle with an :encoding(...) layer. Strings you pass in may be character strings or UTF-8 encoded byte strings.

Unicode is supported throughout: names (entry types, keys, field and macro names) may contain any characters; field names and entry types are lowercased, and macro names compared, using full Unicode case mappings; and splitting and formatting names, "purify_string" and "change_case" classify and case-map characters (whole grapheme clusters, for initials) according to Unicode. The Unicode tables are built from the Unicode database of the perl Text::BibTeX was built with; "unicode_version" tells you which version that is.

TeX markup is passed through unchanged. BibTeX's own rules for TeX "special characters" such as {\'e} (which count as one character for initials, and are treated specially by "purify_string" and "change_case") are followed, but TeX is never converted to Unicode or back; do that separately if you need to.

With binmode => 'bytes' nothing is decoded or normalised: strings are passed through as byte strings, exactly as they appear in the input.

ERRORS AND WARNINGS

Problems found while parsing (syntax errors, undefined macros, malformed names, and so on) are reported as Text::BibTeX::Error objects. Each entry keeps the errors found while it was parsed (see "errors" in Text::BibTeX::Entry), and each error is also passed to Perl's warn, with the same text as in earlier versions, for example:

foo.bib, line 12, warning: undefined macro "jan"

so it goes wherever STDERR goes and can be caught with $SIG{__WARN__}. These warnings follow the warnings pragma of the calling code, using the category Text::BibTeX: they are on by default, are silenced by no warnings 'Text::BibTeX', and become exceptions under use warnings FATAL => 'Text::BibTeX'. Misuse of the library is always reported with die. See Text::BibTeX::Error for details, including how to take over reporting completely.

UPGRADING FROM 0.x

Version 1.00 replaced the C library at the heart of Text::BibTeX. Programs written for earlier versions should mostly work unchanged, but note the following:

Strings are now returned as Perl character strings by default (as if you had given binmode => 'utf-8', which was the recommended option). If your code expects UTF-8 encoded byte strings, pass binmode => 'bytes' to "open" in Text::BibTeX::File, "new" in Text::BibTeX::Entry and "new" in Text::BibTeX::Name. In 'utf-8' mode, invalid UTF-8 in the input is now replaced (with a warning) rather than passed through.
Errors and warnings are reported through Perl's warn (see "ERRORS AND WARNINGS") instead of being written directly to the C library's standard error stream; usage errors throw exceptions instead of terminating the program.
Field names and entry types are lowercased according to Unicode, so ÉDITOR becomes éditor (it used to become Éditor), and "purify_string" and "change_case" now handle non-ASCII letters.
Error recovery is better: after a syntax error the parser resynchronises at the next entry, and a missing closing brace no longer causes the following entry to be lost. Some messages after the first error in a damaged file may therefore differ. "N characters of junk" warnings are now reported with the entry that follows the junk.
Any Perl filehandle can be read from, including in-memory handles and handles with I/O layers.
The C library (derived from btparse) is compiled into the module and is no longer installed as a separate shared library. The biblex, bibparse and dumpnames programs and the btparse manual pages that came with it are gone: the BibTeX language is described in Text::BibTeX::Language, and the rules for splitting and formatting names, purifying strings, changing case and processing field values are described in the documentation of the modules that use them.
Perl 5.26 or later is required.

MODULES AND CLASSES

The Text::BibTeX library includes a number of modules, many of which provide classes. Usually, the relationship is simple and obvious: a module provides a class of the same name---for instance, the Text::BibTeX::Entry module provides the Text::BibTeX::Entry class. There are a few exceptions, though: most obviously, the Text::BibTeX module doesn't provide any classes itself, it merely loads two modules (Text::BibTeX::Entry and Text::BibTeX::File) that do. The other exceptions are mentioned in the descriptions below, and discussed in detail in the documentation for the respective modules.

The modules are presented roughly in order of increasing specialization: the first three are essential for any program that processes BibTeX data files, regardless of what kind of data they hold. The later modules are specialized for use with bibliographic databases, and serve both to emulate BibTeX 0.99's standard styles and to provide an example of how to define a database structure through such specialized modules. Each module is fully documented in its respective manual page.

Text::BibTeX: Loads the two fundamental modules (Entry and File), and provides a number of miscellaneous functions that don't fit anywhere in the class hierarchy.
Text::BibTeX::File: Provides an object-oriented interface to BibTeX database files. In addition to the obvious attributes of filename and filehandle, the "file" abstraction manages properties such as the database structure and options for it.
Text::BibTeX::Entry: Provides an object-oriented interface to BibTeX entries, which can be parsed from File objects, arbitrary filehandles, or strings. Manages all the properties of a single entry: type, key, fields, and values. Also serves as the base class for the structured entry classes (described in detail in Text::BibTeX::Structure).
Text::BibTeX::Value: Provides an object-oriented interface to values and simple values, high-level constructs that can be used to represent the strings associated with each field in an entry. Normally, field values are returned simply as Perl strings, with macros expanded and multiple strings "pasted" together. If desired, you can instruct Text::BibTeX to return Text::BibTeX::Value objects, which give you access to the original form of the data.
Text::BibTeX::Structure: Provides the Structure and StructuredEntry classes, which serve primarily as base classes for the two kinds of classes that define database structures. Read this man page for a comprehensive description of the mechanism for implementing Perl classes analogous to BibTeX "style files".
Text::BibTeX::Bib: Provides the BibStructure and BibEntry classes, which serve two purposes: they fulfill the same role as the standard style files of BibTeX 0.99, and they give an example of how to write new database structures. These ultimately derive from, respectively, the Structure and StructuredEntry classes provided by the Structure module.
Text::BibTeX::BibSort: One of the BibEntry class's base classes: handles the generation of sort keys for sorting prior to output formatting.
Text::BibTeX::BibFormat: One of the BibEntry class's base classes: handles the formatting of bibliographic data for output in a markup language such as LaTeX.
Text::BibTeX::Name: A class used by the Bib structure and specific to bibliographic data as defined by BibTeX itself: parses individual author names into "first", "von", "last", and "jr" parts.
Text::BibTeX::NameFormat: Also specific to bibliographic data: puts split-up names (as parsed by the Name class) back together in a custom way.

The BibTeX data language itself, as Text::BibTeX parses it, is described in Text::BibTeX::Language.

For a first time through the library, you'll probably want to confine your reading to Text::BibTeX::File and Text::BibTeX::Entry. The other modules will come in handy eventually, especially if you need to emulate BibTeX in a fairly fine grained way (e.g. parsing names, generating sort keys). But for the simple database hacks that are the bread and butter of the Text::BibTeX library, the File and Entry classes are the bulk of what you'll need. You may also find some of the material in this manual page useful, namely "CONSTANT VALUES" and "UTILITY FUNCTIONS".

EXPORTS

The Text::BibTeX module has a number of optional exports, most of them constant values described in "CONSTANT VALUES" below. The default exports are a subset of these constant values that are used particularly often, the "entry metatypes" (also accessible via the export tag metatypes). Thus, the following two lines are equivalent:

use Text::BibTeX;
use Text::BibTeX qw(:metatypes);

Some of the various subroutines provided by the module are also exportable. bibloop, split_list, purify_string, and change_case are all useful in everyday processing of BibTeX data, but don't really fit anywhere in the class hierarchy. They may be imported from Text::BibTeX using the subs export tag. check_class and display_list are also exportable, but only by name; they are not included in any export tag. (These two mainly exist for use by other modules in the library.) For instance, to use Text::BibTeX and import the entry metatype constants and the common subroutines:

use Text::BibTeX qw(:metatypes :subs);

Another group of subroutines exists for direct manipulation of the macro table maintained by the underlying C library. These functions (see "Macro table functions", below) allow you to define, delete, and query the value of BibTeX macros (or "abbreviations"). They may be imported en masse using the macrosubs export tag:

use Text::BibTeX qw(:macrosubs);

CONSTANT VALUES

The Text::BibTeX module makes a number of constant values available. These correspond to the values of various enumerated types in the underlying C library.

Each group of constants is optionally exportable using an export tag given in the descriptions below.

Entry metatypes: BTE_UNKNOWN, BTE_REGULAR, BTE_COMMENT, BTE_PREAMBLE, BTE_MACRODEF. The metatype method in the Entry class always returns one of these values. The latter three describe, respectively, comment, preamble, and string entries; BTE_REGULAR describes all other entry types. BTE_UNKNOWN should never be seen (it's mainly useful for C code that might have to detect half-baked data structures). Export tag: metatypes.
AST node types: BTAST_STRING, BTAST_MACRO, BTAST_NUMBER. Used to distinguish the three kinds of simple values---strings, macros, and numbers. The SimpleValue class' type method always returns one of these three values. See also Text::BibTeX::Value. Export tag: nodetypes.
Name parts: BTN_FIRST, BTN_VON, BTN_LAST, BTN_JR, BTN_NONE. Used to specify the various parts of a name after it has been split up. These are mainly useful when using the NameFormat class. See also Text::BibTeX::Name and Text::BibTeX::NameFormat. Export tag: nameparts.
Join methods: BTJ_MAYTIE, BTJ_SPACE, BTJ_FORCETIE, BTJ_NOTHING. Used to tell the NameFormat class how to join adjacent tokens together; see "FORMATTING RULES" in Text::BibTeX::NameFormat. Export tag: joinmethods.

UTILITY FUNCTIONS

Text::BibTeX provides several functions that operate outside of the normal class hierarchy. Of these, only bibloop is likely to be of much use to you in writing everyday BibTeX-hacking programs; the other two (check_class and display_list) are mainly provided for the use of other modules in the library. They are documented here mainly for completeness, but also because they might conceivably be useful in other circumstances.

bibloop (ACTION, FILES [, DEST])

Loops over all entries in a set of BibTeX files, performing some caller-supplied action on each entry. FILES should be a reference to the list of filenames to process, and ACTION a reference to a subroutine that will be called on each entry. DEST, if given, should be a Text::BibTeX::File object (opened for output) to which entries might be printed.

The subroutine referenced by ACTION is called with exactly one argument: the Text::BibTeX::Entry object representing the entry currently being processed. Information about both the entry itself and the file where it originated is available through this object; see Text::BibTeX::Entry. The ACTION subroutine is only called if the entry was successfully parsed; any syntax errors will result in a warning message being printed, and that entry being skipped. Note that all successfully parsed entries are passed to the ACTION subroutine, even preamble, string, and comment entries. To skip these pseudo-entries and only process "regular" entries, then your action subroutine should look something like this:

sub action {
   my $entry = shift;
   return unless $entry->metatype == BTE_REGULAR;
   # process $entry ...
}

If your action subroutine needs any more arguments, you can just create a closure (anonymous subroutine) as a wrapper, and pass it to bibloop:

sub action {
   my ($entry, $extra_stuff) = @_;
   # ...
}

my $extra = ...;
Text::BibTeX::bibloop (sub { &action ($_[0], $extra) }, \@files);

If the ACTION subroutine returns a true value and DEST was given, then the processed entry will be written to DEST.

check_class (PACKAGE, DESCRIPTION, SUPERCLASS, METHODS)

Ensures that a PACKAGE implements a class meeting certain requirements. First, it inspects Perl's symbol tables to ensure that a package named PACKAGE actually exists. Then, it ensures that the class named by PACKAGE derives from SUPERCLASS (using the universal method isa). This derivation might be through multiple inheritance, or through several generations of a class hierarchy; the only requirement is that SUPERCLASS is somewhere in PACKAGE's tree of base classes. Finally, it checks that PACKAGE provides each method listed in METHODS (a reference to a list of method names). This is done with the universal method can, so the methods might actually come from one of PACKAGE's base classes.

DESCRIPTION should be a brief string describing the class that was expected to be provided by PACKAGE. It is used for generating warning messages if any of the class requirements are not met.

This is mainly used by the supervisory code in Text::BibTeX::Structure, to ensure that user-supplied structure modules meet the rules required of them.

display_list (LIST, QUOTE)

Converts a list of strings to the grammatical conventions of a human language (currently, only English rules are supported). LIST must be a reference to a list of strings. If this list is empty, the empty string is returned. If it has one element, then just that element is returned. If it has two elements, then they are joined with the string " and " and the resulting string is returned. Otherwise, the list has N elements for N >= 3; elements 1..N-1 are joined with commas, and the final element is tacked on with an intervening ", and ".

If QUOTE is true, then each string is encased in single quotes before anything else is done.

This is used elsewhere in the library for two very distinct purposes: for generating warning messages describing lists of fields that should be present or are conflicting in an entry, and for generating lists of author names in formatted bibliographies.

MISCELLANEOUS FUNCTIONS

In addition to loading the File and Entry modules, Text::BibTeX loads the XSUB code which bridges the Perl modules to the underlying C library. This XSUB code provides a number of miscellaneous utility functions, most of which are put into other packages in the Text::BibTeX family for use by the corresponding classes. (For instance, the XSUB code loaded by Text::BibTeX provides a function Text::BibTeX::Entry::parse, which is actually documented as the parse method of the Text::BibTeX::Entry class---see Text::BibTeX::Entry. However, for completeness this function---and all the other functions that become available when you use Text::BibTeX---are at least mentioned here. The only functions from this group that you're ever likely to use are described in "Generic string-processing functions".

Startup/shutdown functions

initialize (): Does nothing; kept for compatibility.
cleanup (): Deletes all macros from the global macro table.
unicode_version (): Returns the version of the Unicode database the C library's character tables were built from (that of the perl Text::BibTeX was built with).

Generic string-processing functions

split_list (STRING, DELIM [, FILENAME [, LINE [, DESCRIPTION [, OPTS]]]])

Splits a string on a fixed delimiter according to the BibTeX rules for splitting up lists of names. With BibTeX, the delimiter is hard-coded as "and"; here, you can supply any string. This is not the same as Perl's split function: an instance of DELIM in STRING is only a delimiter if

it is surrounded by whitespace (so "Anderson" is not split);
it is not at the beginning or end of STRING (so "and" on its own is one item);
it is at brace-depth zero (so "{Barnes and Noble}" is one item).

The comparison is case-insensitive, so

Candy and Apples AnD {Green Eggs and Ham}

splits into "Candy", "Apples", and "{Green Eggs and Ham}". Empty items (as in "Smith and and Jones") are returned as undef, with a warning. OPTS is a hash ref of the same binmode and normalization arguments as with, e.g. Text::BibTeX::File->open(); the default is 'utf-8' mode. FILENAME, LINE and DESCRIPTION (such as "name") are used in warnings.

Returns the list of strings resulting from splitting STRING on DELIM.

isplit_list (STRING, DELIM [, FILENAME [, LINE [, DESCRIPTION]]])

Splits a string on a fixed delimiter in the same way as "split_list". The strings returned are of the same kind as STRING: character strings if it was a character string, byte strings otherwise. Use split_list to specify binmode and normalization arguments.

Returns the list of strings resulting from splitting STRING on DELIM.

purify_string (STRING [, OPTIONS])

"Purifies" STRING in the BibTeX way (usually for generation of sort keys). A purified copy of the input string is returned: a character string if STRING was one, a byte string otherwise. OPTIONS is currently unused.

Purification keeps letters and digits (in any script, with any combining marks attached to them) and spaces, converts hyphens and ties ("~") to spaces, and removes (almost) everything else.

"Almost", because TeX "special characters" (used for accented and non-English letters) are handled specially. A special character is a brace group that starts at brace-depth zero and whose first character is a backslash. For instance, the string

{\foo bar}Herr M\"uller went from {P{\r r}erov} to {\AA}rhus

contains two special characters: "{\foo bar}" and "{\AA}". Neither the \"u nor the \r r are special characters, because they are not at the right brace depth.

In a special character, a control sequence that is one of LaTeX's "foreign letters" (\oe, \ae, \aa, \o, \l, \ss, plus uppercase versions) is converted to an English approximation by stripping the backslash and lowercasing the second character (if any), so {\AA} becomes Aa. All other control sequences in a special character are removed, as are all non-alphanumeric characters. Outside special characters, control sequences lose only their backslash. So the string above becomes

barHerr Muller went from Pr rerov to Aarhus

Something has gone wrong with P{\r r}erov: the accented `r' should be a special character, starting at brace-depth zero. Written as

{\foo bar}Herr M\"uller went from P{\r r}erov to {\AA}rhus

the result is more sensible:

barHerr Muller went from Prerov to Aarhus

The "nonsense" special character {\foo bar} is a common trick to put text in a string solely for generating sort keys: TeX ignores it, as long as \foo is defined as a macro that does nothing.

purify_string is adequate for sorting by English-language conventions. To follow the conventions of other languages, use a collation library (such as Unicode::Collate) on the purified string.

change_case (TRANSFORM, STRING [, OPTIONS])

Transforms the case of STRING according to TRANSFORM, which is one of 'u' (uppercase), 'l' (lowercase), or 't' ("title capitalization", as for non-book titles). The transformed copy is returned: a character string if STRING was one, a byte string otherwise. An invalid TRANSFORM throws an exception. OPTIONS is currently unused.

Case conversion uses the full Unicode case mappings, so the result may be longer than the input ("ß" uppercases to "SS"). Title capitalization means that everything is converted to lowercase, except the first letter of the first word and of words immediately following a colon or sentence-ending punctuation, which are converted to titlecase (this only differs from uppercase for a few digraphs, such as "ǆ"). For instance,

Flying Squirrels: Their Peculiar Habits. Part One

becomes

Flying squirrels: Their peculiar habits. Part one

Text in braces is treated differently. In a special character (see "purify_string"), control sequences that are LaTeX's non-English letters are converted (\AE becomes \ae when lowercasing); other control sequences, including accents, are preserved; and all other text, at any depth, is converted to lowercase or uppercase (to lowercase for title capitalization). Brace groups that are not special characters are left completely untouched.

For example, with title capitalization,

A Guide to \LaTeXe: Document Preparation ...

becomes

A guide to \latexe: Document preparation ...

which is probably not what you want. Writing {\LaTeXe} keeps the control sequence intact, but {\LaTeXe} is then a special character that is not a non-English letter, so purify_string removes it, and a sort key generated from the title would be "A Guide to Document Preparation". The solution (for any title with a TeX command that produces actual text) is to bury the control sequence at brace-depth two:

A Guide to {{\LaTeXe}}: Document Preparation ...

Entry-parsing functions

Although these functions are provided by the Text::BibTeX module, they are actually in the Text::BibTeX::Entry package. That's because they are implemented in C, and thus loaded with the XSUB code that Text::BibTeX loads; however, they are actually methods in the Text::BibTeX::Entry class. Thus, they are documented as methods in Text::BibTeX::Entry.

_parse (ENTRY_STRUCT, PARSER [, PRESERVE])
_parse_s (ENTRY_STRUCT, TEXT [, PRESERVE [, UTF8]]): PARSER is a Text::BibTeX::Parser object, which wraps a C parser reading from a Perl filehandle.

Macro table functions

These functions allow direct access to the global macro table. In the normal course of events, macro definitions always accumulate, and are only defined as a result of parsing a macro definition (@string) entry, so (as in BibTeX) macros defined in one file are available in the files read after it. If, for example, you wish to start fresh with new macros for every file, use delete_all_macros (or the reset_macros option to "open" in Text::BibTeX::File); to keep each file's macros to itself, use the private_macros option. Macro names are case-insensitive. When Text::BibTeX is loaded it defines the standard three-letter month macros (jan to dec). If you wish to pre-define certain macros, use add_macro_text. (But note that the Bib structure, as part of its mission to emulate BibTeX 0.99, defines the standard "month name" macros for you.)

add_macro_text (MACRO, TEXT [, FILENAME [, LINE]]): Defines a new macro, or redefines an old one. MACRO is the name of the macro, and TEXT is the text it should expand to. FILENAME and LINE are just used to generate any warnings about the macro definition. The only such warning occurs when you redefine an old macro: its value is overridden, and add_macro_text() issues a warning saying so.
delete_macro (MACRO): Deletes a macro from the macro table. If MACRO isn't defined, takes no action.
delete_all_macros (): Deletes all macros from the macro table, even the predefined month names.
macro_length (MACRO): Returns the length of a macro's expansion text. If the macro is undefined, returns 0; no warning is issued.
macro_text (MACRO [, FILENAME [, LINE]]): Returns the expansion text of a macro, as a (UTF-8 encoded) byte string. If the macro is not defined, issues a warning and returns undef. FILENAME and LINE, if supplied, are used for generating this warning; they should be supplied if you're looking up the macro as a result of finding it in a file.

Name-parsing functions

These are both private functions for the use of the Name class, and therefore are put in the Text::BibTeX::Name package. You should use the interface provided by that class for parsing names in the BibTeX style.

_split (NAME_STRUCT, NAME, FILENAME, LINE, NAME_NUM, KEEP_CSTRUCT)
free (NAME_STRUCT)

Name-formatting functions

These are private functions for the use of the NameFormat class, and therefore are put in the Text::BibTeX::NameFormat package. You should use the interface provided by that class for formatting names in the BibTeX style.

create ([PARTS [, ABBREV_FIRST]])
free (FORMAT_STRUCT)
_set_text (FORMAT_STRUCT, PART, PRE_PART, POST_PART, PRE_TOKEN, POST_TOKEN)
_set_options (FORMAT_STRUCT, PART, ABBREV, JOIN_TOKENS, JOIN_PART)
format_name (NAME_STRUCT, FORMAT_STRUCT [, UTF8])

BUGS AND LIMITATIONS

The parser reads ahead in its input as far as it needs to: don't read from a filehandle yourself while a Text::BibTeX::File (or Text::BibTeX::Entry::parse) is reading from it.

Name splitting follows BibTeX's rules, which are designed for Western names.

Any crash, memory error or leak in the C library should be considered a bug. It is tested with AddressSanitizer and UndefinedBehaviorSanitizer on randomly generated and mutated input (see fuzz/bt_fuzz.c in the distribution).

AUTHOR

Greg Ward <gward@python.net>

COPYRIGHT

Copyright (c) 1997-2000 by Gregory P. Ward. All rights reserved. This file is part of the Text::BibTeX library. This library is free software; you may redistribute it and/or modify it under the same terms as Perl itself.

To install Text::BibTeX, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Text::BibTeX

CPAN shell

perl -MCPAN -e shell
install Text::BibTeX

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	Go to GitHub issues (only if GitHub is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)