NAME

Rinci - Language-neutral metadata for your code

VERSION

version 1.1.0

SPECIFICATION VERSION

1.1

ABSTRACT

This document describes Rinci, a set of extensible, language-neutral metadata specifications for your code (functions/methods, variables, packages, classes, and so on). Rinci allows various helper tools, from code generator to web middleware to documentation generator to other protocols, to act on your code, making your life easier as a programmer. Rinci also allows better interoperability between programming languages. Rinci is geared towards dynamic scripting languages like Perl, Python, Ruby, PHP, JavaScript, but is not limited to those languages.

WHAT IS RINCI? (TERMINOLOGIES)

Rinci is a set of specifications of metadata for your code entities. Each different type of code entity, like function/method, variable, namespace, etc, has its own specification. The metadata is a set of name/value pairs stored in a regular mapping variable (a.k.a. hash in Perl/Ruby, or dictionary in Python, or array in PHP, or object in JavaScript/JSON; but from here on the term hash will be used). Each specification will describe what metadata property names (a.k.a. hash keys) are known and what value should each property contain. So the Rinci::function specification will describe metadata for functions/methods, Rinci::package will describe metadata for namespace/package, and so on.

Rinci defines properties pertaining to documentation (like summary, description, examples, links), function argument and return value validation (args and result), dependencies (deps), standardized feature description (features), also a generic undo protocol for supporting undoable operations. The specification is extensible: you can define more properties, or more deps, or more features.

WHAT ARE THE BENEFITS OF RINCI?

By adding Rinci metadata to your code, you can write/use tools to do various things to your program. Rinci is designed with code generation and function wrapping in mind. At the time of this writing, several tools (mostly modules in Sub::Spec::* namespace) exist in Perl to do the following:

• Sub::Spec::Wrapper

  Wrap functions with a single generated function that can do the following: validate input (using information from the args property), validate return value (the return property), add execution time-limiting (the timeout property), add automatic retries (the retry), interactive confirmation, logging, and more.

• Sub::Spec::Exporter

  A replacement for Exporter or Sub::Exporter if your functions are equipped with Rinci metadata. Automatically provide export tags (using information in the tags property). Automatically wrap functions using Sub::Spec::Wrapper when exporting.

• Sub::Spec::To::* modules

  Convert function metadata to user/API documentation in POD or other format. Basically all they do is just arranging the various available properties like summary, description, args, return, examples, see and so on and format them to the final target.

• Sub::Spec::CmdLine

  Run functions on the command line. Automatically convert command-line options/arguments to function arguments. Generate help/usage message (for --help). Check dependencies (e.g. you can specify that in order to run your functions, you need some executables/other functions to exist, an environment variable being set, and so on). Even do bash shell completion (using Sub::Spec::BashComplete).

• Serabi

  A PSGI application (a set of PSGI middlewares, really) to serve function requests over HTTP, using the Rinci::HTTP protocols. Suitable for creating an API service.

• Sub::Spec::Use

  Use remote packages and import their functions, transparently like you would use local Perl modules. The remote server can be any Rinci::HTTP-compliant service, even when implemented in other languages.

• Sub::Spec::Gen::*

  Since Rinci metadata are just normal data structure, they can be easily generated. The Sub::Spec::Gen::* Perl modules can generate functions as well as their metadata, for example to access table data (like from a regular array or from a SQL database).

More tools might be written in the future.

RINCI VS ...

Some features offered by Rinci (or Rinci tools) are undoubtedly already offered by your language or existing language libraries. For example, for documentation Perl already has POD and Python has docstrings. There are numerous libraries for argument validation in every language. Python has decorators that can be used to implement various features like argument validation and logging. Perl has subroutine attributes to stick additional metadata to your subroutines and variables. And so on.

The benefits that Rinci offer include richer metadata, language neutrality, extensibility, and manipulability.

<Richer metadata>. Rinci strives to provide enough metadata for tools to do various useful things. For example, description and summary supports translations. Argument specification is pretty rich, with a quite powerful and flexible schema language.

Language neutrality. You can share metadata between languages, including documentation and rules for argument validation. Perl 6 allows very powerful argument validation, for example, but it is language-specific. With Rinci you can easily share validation rules and generate validators in Perl and JavaScript (and other target languages).

Manipulability. Being a normal data structure, your Rinci metadata is easier to manipulate (clone, merge, modify, what have you) as well as access (from your language and others).

Other things to consider. If you stack multiple decorators in Python, for example, it usually results in wrapping your Python function multiple times, which can add overhead. A single wrapper like Sub::Spec::Wrapper, on the other hand, uses a single level of wrapping to minimize subroutine call overhead.

Working together. There is no reason why Rinci metadata has to compete against existing features from language/libraries. A code generator for Rinci metadata can generate code that utilize those features. For example, the timeout property can be implemented in Python using decorator, if you want. Rinci basically just provides a way for you to express desired properties/constraints/behaviours, separate from the implementation. A tool is free to implement those properties using whatever technique is appropriate.

SPECIFICATION

Note: Examples are usually written in Perl, but this does not mean they only apply to a particular language.

Terminologies

Code entities, or just entities for short, are elements in your code that can be given metadata. Currently supported entities are function/method, namespace/package, and variable. Other entities planned to be supported: class, object, library, application.

Entity URIs. Sometimes code entities need to be expressed as URIs, for example as links or when requesting over Rinci HTTP protocol. To refer to local entities, use the ent scheme (which can be optional in places). Namespace parts is separated by ., independent of what the implementation language uses (e.g. Perl and Ruby use :: but Python uses ., all should use . regardless).

ent:Package.SubPkg.func
Package.SubPkg.func        # ent: scheme is sometimes optional
ent:Package.SubPkg         # namespaces are entities themselves
ent:                       # the top level namespace
ent:Package.SubPkg.$var    # notice the syntax

To refer to remote entities served using Rinci HTTP protocol, you can just use the normal http or https scheme, where the URL scheme might differ according to each server's policy:

http://example.org/api/v1/Package/SubPkg/func
https://example.org/Package.SubPkg.$var

Specification common to all metadata

This section describes specification common to all kinds of Rinci metadata.

Metadata. Metadata is just a normal hash. Each hash key is called a property. Any hash key in the metadata that starts with _ (underscore) is ignored, this can be used to store extra data, or experimental properties. Otherwise, the property must be known and valid. A property is valid if the property name matches the regex pattern /\A[A-Za-z][A-Za-z0-9_]*([A-Za-z][A-Za-z0-9_]*)*\z/ (in other words, a word or a dotted word like "foo" or "foo.bar") and the name is known (listed in the specification), and the property values conform to the specification for that property.

Custom properties. The specification is extensible: you can add properties of your own. Currently the mechanism to formally add a custom property to the specification is unspecified, but at least in Perl it is recommended that the distribution which does this updates the schema in Rinci::METATYPE::Schema. Popular custom properties might make it into core specification in later revision or version.

Where to put the metadata. The specification does not specify where to put metadata in, it is up to you or the tools you want to use. It can be separated in another file or put alongside the code. In the case of function metadata, for example, many of the Sub::Spec::* Perl modules expect the function metadata to be put in the %SPEC package variable, with the name of the function as the hash key. Consult the tool that you are using to find out for sure.

Common properties. Below are properties common to all metadata:

Property: v => FLOAT (required)

Declare specification version. This property is required. It should have the value of 1.1. If v is not specified, it is assumed to be 1.0 and metadata is assumed to be the old, Sub::Spec 1.0.x metadata.

Example:

v => 1.1

Property: name => STR

The name of the entity. Useful when aliasing entity (and reusing the metadata) and wanting to find out the canonical/original entity.

Examples:

name => 'foo'
name => '$var'  # only in languages where variables have prefix

Property: summary => STR

A one-line summary. It should be plain text without any markup. It is like the summary clause in Data::Sah.

Example:

name => 'foo',
summary => 'Perform the foo ritual',

Property: tags => ARRAY OF STR

A list of tags, useful for categorization. Can also be used by tools, e.g. Sub::Spec::Exporter in Perl uses the tags property of the function metadata as export tags.

Example:

# tag a function as beta
tags => ['beta']

Property: description => STR

A longer description text. The text should be in marked up in format specified by text_markup and is suggested to be formatted to 78 columns. It is like the description Data::Sah clause.

Example:

{
    name => 'foo',
    summary => 'Perform the foo ritual',
    description => <<EOT,

Foo ritual can be performed by humans or machines. This program will perform a
machine-based ritual using [the best available
algorithm](http://example.org/foo-with-bar-algo.html).

Note that you still have to perform foo ritual manually from time to time, just
to be safe.

EOT
}

Property: links => ARRAY OF HASHES

List to related entities or resources. Can be used to generate a SEE ALSO and/or LINKS sections in documentation. Each link is a hash with the following keys:

• uri => STR (required)

• title => STR

  A short plaintext title for the link.

• description => STR

  A longer marked up text description for the link. Suggested to be formatted to 76 columns.

• tags => ARRAY OF STR

  Can be used to categorize or select links. For generating SEE ALSO sections, use the tag 'see'.

Example:

# links in the Bar::foo function metadata
links => [
    {
        url   => "http://example.org/foo-with-bar-algo.html",
        title => "Article describing foo using Bar algorithm",
    },
    {
        url   => "Baz::foo",
        title => "Another implementation of foo using the Bar algorithm",
        tags  => ['related']
    },
],

Property: text_markup => STR

Specify text markup format for the content of properties which contain marked-up text, like description. The default is markdown (http://en.wikipedia.org/wiki/Markdown) and other valid values include org (http://orgmode.org/) and none.

NOTE: Other popular text markup formats might be added in the future. Text format should be readable as-is (so-called "plain text markup format") as some tools, like Sub::Spec::CmdLine in Perl will just dump the content description as text help message instead of going through the trouble of passing the content through some formatters.

# use Org format for all marked up text
text_markup => 'org'

NOTE: Supporting different markup format for different properties, e.g. markdown for description but org for other properties (or perhaps different format between arg's description and link's description) might be considered in the future if necessary, but for now such practice is not recommended.

Entity-specific specifications

Each entity-specific specification is described on a separate subdocument. Currently these specifications are defined:

• Rinci::function - Metadata for functions/methods

• Rinci::package - Metadata for namespaces/packages

• Rinci::variable - Metadata for variables

These specifications are planned or considered, but not yet defined:

• Rinci::class - Metadata for classes

• Rinci::object - Metadata for objects

• Rinci::application - Metadata for applications

• Rinci::library - Metadata for libraries

• Rinci::distribution - Metadata for software distribution

• Rinci::language - Metadata for programming languages

• Rinci::author - Metadata for software authors

• Rinci::project - Metadata for software projects

• Rinci::repository - Metadata for code repository (like git, svn)

FAQ

What does Rinci mean?

Rinci is taken from Indonesian word perincian or rincian, meaning specification or detail.

Why use Sah for data schema?

Sah is a flexible and extensible schema language, while still not being language-specific, making it easy for code generator tools to generate validator code in various target languages (Perl, Ruby, etc).

HISTORY

Below is the history of major versions of the specifications (those which break backward compatibility). For more detailed changes between revisions, see the Changes file in the distribution.

1.1 (Jan 2012)

To clearly separate specification from implementation, rename specification from Sub::Spec to Rinci (the namespace Sub::Spec is still used for implementations, along with Var::Spec, Package::Spec). Split specification, each for functions, variables, packages. Bump specification version from 1.0 to 1.1 due to several incompatibilities like changed args and result properties, terminologies, defaults. Versioning property (v) now required.

1.0 (Aug 2011)

First release version of Sub::Spec.

0.x (Feb-Aug 2011)

Series of Sub::Spec drafts.

SEE ALSO

Related specifications

Sah schema language, Data::Sah

Rinci::HTTP

Related ideas/concepts

.NET attributes, http://msdn.microsoft.com/en-us/library/z0w1kczw.aspx

Python Decorators, http://www.python.org/dev/peps/pep-0318/ , http://wiki.python.org/moin/PythonDecorators

Other related links

Acmeism, http://www.acmeism.org/

AUTHOR

Steven Haryanto <stevenharyanto@gmail.com>

COPYRIGHT AND LICENSE

This software is copyright (c) 2012 by Steven Haryanto.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

cpanm Rinci

perl -MCPAN -e shell
install Rinci