To develop a new codec for Logstash, you build a self-contained Ruby gem
whose source code lives in its own GitHub repository. The Ruby gem can then be
hosted and shared on RubyGems.org. You can use the example codec
implementation as a starting point. (If you’re unfamiliar with
Ruby, you can find an excellent quickstart guide at
https://www.ruby-lang.org/en/documentation/quickstart/.)

You can now create your own Logstash plugin in seconds! The generate subcommand of bin/logstash-plugin creates the foundation
for a new Logstash plugin with templatized files. It creates the correct directory structure, gemspec files, and dependencies so you
can start adding custom code to process data with Logstash.

Logstash provides infrastructure to automatically generate documentation for
plugins. We use the asciidoc format to write documentation so any comments in
the source code will be first converted into asciidoc and then into html.

The configuration, or config section allows you to define as many (or as few)
parameters as are needed to enable Logstash to process events.

There are several configuration attributes:

:validate - allows you to enforce passing a particular data type to Logstash
for this configuration option, such as :string, :password, :boolean,
:number, :array, :hash, :path (a file-system path), uri (starting in 5.0.0), :codec (since
1.2.0), :bytes (starting in 1.5.0). Note that this also works as a coercion
in that if I specify "true" for boolean (even though technically a string), it
will become a valid boolean in the config. This coercion works for the
:number type as well where "1.2" becomes a float and "22" is an integer.

:default - lets you specify a default value for a parameter

:required - whether or not this parameter is mandatory (a Boolean true or false)

:list - whether or not this value should be a list of values. Will typecheck the list members, and convert scalars to one element lists. Note that this mostly obviates the array type, though if you need lists of complex objects that will be more suitable.

:deprecated - informational (also a Boolean true or false)

:obsolete - used to declare that a given setting has been removed and is no longer functioning. The idea is to provide an informed upgrade path to users who are still using a now-removed setting.

The Logstash register method is like an initialize method. It was originally
created to enforce having super called, preventing headaches for newbies.
(Note: It may go away in favor of initialize, in conjunction with some
enforced testing to ensure super is called.)

public means the method can be called anywhere, not just within the class.
This is the default behavior for methods in Ruby, but it is specified explicitly
here anyway.

You can also assign instance variables here (variables prepended by @).
Configuration variables are now in scope as instance variables, like @message

The encode method takes an event and serializes it (encodes) into another
format. Good examples of encode methods include the simple plain
codec, the slightly more involved msgpack
codec, and even an avro
codec.

In most cases, your encode method should have an @on_event.call() statement.
This call will output data per event in the described way.

A require statement in Ruby is used to include necessary code. In some cases
your plugin may require additional files. For example, the collectd plugin
uses
the types.db file provided by collectd. In the main directory of your plugin,
a file called vendor.json is where these files are described.

The vendor.json file contains an array of JSON objects, each describing a file
dependency. This example comes from the
collectd
codec plugin:

sha1 is the sha1 signature used to verify the integrity of the file
referenced by url.

url is the address from where Logstash will download the file.

files is an optional array of files to extract from the downloaded file.
Note that while tar archives can use absolute or relative paths, treat them as
absolute in this array. If files is not present, all files will be
uncompressed and extracted into the vendor directory.

At the bottom of the gemspec file is a section with a comment:
Gem dependencies. This is where any other needed gems must be mentioned. If
a gem is necessary for your plugin to function, it is a runtime dependency. If
a gem are only used for testing, then it would be a development dependency.

You can also have versioning requirements for your dependencies—including other
Logstash plugins:

After creating an account,
obtain an API
key from RubyGems.org. By default, RubyGems uses the file ~/.gem/credentials
to store your API key. These credentials will be used to publish the gem.
Replace username and password with the credentials you created at
RubyGems.org:

Some of the many benefits of having your plugin in the logstash-plugins
repository are:

Discovery Your plugin will appear in the Logstash Reference,
where Logstash users look first for plugins and documentation.

Documentation Your plugin documentation will automatically be added to the
Logstash Reference.

Testing With our testing infrastructure, your plugin will be continuously
tested against current and future releases of Logstash. As a result, users will
have the assurance that if incompatibilities arise, they will be quickly
discovered and corrected.

Code Review Your plugin must be reviewed by members of the community for
coherence, quality, readability, stability and security.

Tests Your plugin must contain tests to be accepted. These tests are also
subject to code review for scope and completeness. It’s ok if you don’t know
how to write tests — we will guide you. We are working on publishing a guide to
creating tests for Logstash which will make it easier. In the meantime, you can
refer to http://betterspecs.org/ for examples.

To begin migrating your plugin to logstash-plugins, simply create a new
issue in
the Logstash repository. When the acceptance guidelines are completed, we will
facilitate the move to the logstash-plugins organization using the recommended
github process.