NAME
Message::Passing - a perl subset of Logstash <http://logstash.net>
SYNOPSIS
logstash --input STDIN --output STDOUT
{"foo": "bar"}
{"foo":"bar"}DESCRIPTION
A lightweight but inter-operable subset of logstash http://logstash.net
This implementation is currently a prototype, and as such should be considered alpha and subject to change at any point.
BASIC PREMISE
You have data for discrete events, represented by a hash (and serialized as JSON).
This could be a text log line, an audit record of an API event, a metric emitted from your application that you wish to aggregate and process - anything that can be a simple hash really..
You want to be able to shove these events over the network easily, and aggregate them / filter and rewrite them / split them into worker queues.
This module is designed as a simple framework for writing components that let you do all of these things, in a simple and easily extensible manor.
For a practical example, You generate events from a source (e.g. ZeroMQ output of logs and performance metrics from your Catalyst FCGI or Starman workers) and run one script that will give you a central application log file, or push the logs into ElasticSearch.
There are a growing set of components you can plug together to make your logging solution.
Getting started is really easy - you can just use the logstash command installed by the distribution. If you have a common config that you want to repeat, or you want to write your own server which does something more flexible than the normal script allows, then see Message::Passing::DSL.
To dive straight in, see the documentation for the command line utility logstash, and see the examples in Message::Passing::Manual::Cookbook.
For more about how the system works, see Message::Passing::Manual::Concepts.
COMPONENTS
Below is a non-exhaustive list of components available.
INPUTS
Inputs receive data from a source (usually a network protocol).
They are responsible for decoding the data into a hash before passing it onto the next stage.
Inputs include:
You can easily write your own input, just use AnyEvent, and consume Message::Passing::Role::Input.
FILTER
Filters can transform a message in any way.
Examples include:
- Message::Passing::Filter::Null - Returns the input unchanged.
- Message::Passing::Filter::All - Stops any messages it receives from being passed to the output. I.e. literally filters all input out.
- Message::Passing::Filter::T - Splits the incoming message to multiple outputs.
You can easily write your own filter, just consume Message::Passing::Role::Filter.
Note that filters can be chained, and a filter can return undef to stop a message being passed to the output.
OUTPUTS
Outputs send data to somewhere, i.e. they consume messages.
- Message::Passing::Output::STDOUT
- Message::Passing::Output::AMQP
- Message::Passing::Output::ZeroMQ
- Message::Passing::Output::WebHooks
- Message::Passing::Output::ElasticSearch
- Message::Passing::Output::Test
SEE ALSO
- Log::Message::Structured - For creating your log messages.
- Log::Dispatch::Message::Passing - use Message::Passing outputs from Log::Dispatch.
THIS MODULE
This is a simple MooseX::Getopt script, with one input, one filter and one output.
METHODS
build_chain
Builds and returns the configured chain of input => filter => output
start
Class method to call the run_log_server function with the results of having constructed an instance of this class, parsed command line options and constructed a chain.
This is the entry point for the logstash script.
AUTHOR
Tomas (t0m) Doran <bobtfish@bobtfish.net>
SPONSORSHIP
This module exists due to the wonderful people at Suretec Systems Ltd. <http://www.suretecsystems.com/> who sponsored it's development for its VoIP division called SureVoIP <http://www.surevoip.co.uk/> for use with the SureVoIP API - <http://www.surevoip.co.uk/support/wiki/api_documentation>
COPYRIGHT
Copyright Suretec Systems Ltd. 2012.
Logstash (upon which many ideas for this project is based, but which we do not reuse any code from) is copyright 2010 Jorden Sissel.
LICENSE
GNU Affero General Public License, Version 3
If you feel this is too restrictive to be able to use this software, please talk to us as we'd be willing to consider re-licensing under less restrictive terms.