The first alpha of syslog-ng 3.4 was released more than a year ago, and the first stable release earlier this year, development on syslog-ng 3.5 has been going on steadlily since then, with a lot of interesting features and changes pouring in. While the 3.5 release will not bring as many overwhelming changes as the previous two releases did, the sheer number of improvements is still substantial. Our focus was on smaller developments all over the place this time, instead of big sweeping changes. In this post, we will go over most of these features, with use-cases, examples and a few words about why they are so useful and awesome.

These are only bigger features, there has been a lot of small tweaking going on, and of course, all of the fixes that went into prior versions will also be in 3.5 too, and there's still a chance that something else may find its way in before the feature freeze.

One of the major new features in the 3.5 release will be support for multi-line messages, a feature that has been available in syslog-ng PE for a good while, and which has been ported to and improved upon to the open source edition. Two variants of multi-line are supported, which will be detailed below. Both of them are available for the file() and pipe() sources only.

Indented multi-line

The easiest variant is indented multi-line, where each line can be followed by others, indented by whitespace, and the message continues until the first non-indented line. This is the format used by the Linux kernel too, from version 3.5, for /dev/log. This type of multi-line can be used as follows:

With the indentedmulti-line-mode() setting, this would turn into two log messages:

First line\n Continuation 1;\n Continuation 2;
Second line

Regexp-based multiline

If multi-line input is not based on indentation, one can use the regexpmulti-line-mode() instead, which makes two new settings available: multi-line-prefix() and multi-line-garbage(). These can be used to define the start and the end of a log message: any string between a the beginning matching prefix and a matching garbage will be considered a single message. That is, the prefix will be included, the garbage will not be: it will be discarded.

Of course, none of these settings need to be set, you can just use the defaults, and it will just work!

Riemann

Did you know that syslog-ng is far more than a log collector and processor? No? You do now. When you have access to a lot of logs, and a tremendous amount of power in parsing them, you can use these tools for monitoring easily! And when we're talking monitoring, riemann is a great asset in our toolbox, and with the new destination, we can easily forward metrics to it:

To turn some text into *upper-* or *lower-case*, these two template
functions come in handy. They simply turn all their arguments into
the respecitve case.

$(delimit DELIMITERSNEW-DELIMITERTEXT)

Sometimes one has a delimited string, where one wishes to replace
the delimiters. This function does just that: give it a list of
delimiters (you may need to quote it, if it contains whitespace), a
replacement, and a text to replace delimiters in, and it does the
rest.
For example, to replace tabs and spaces with a vertical bar, one
could write a template like this:

template("$(delimit \"\t \" \"|\" $MESSAGE)\n")

$(env VARIABLE...)

Have you ever felt the need to check an environment variable from
within syslog-ng? I did, and now I can.

While syslog-ng supported sending log messages into various data stores and message queues for a while (SQL at first, MongoDB, JSON and AMQP later), even when those supported different types of data than strings, we could not do anything else. Until now. It is now possible - at places where it makes sense - to annotate templates with type hints, which the destination driver can optionally use. Type hinting is implemented for the mongodb() destination and the $(format-json) template function for now. When no type hint is specified, syslog-ng defaults to string.

To add type hints, simply wrap the respective template with the hinted type, like this:

Currently the following type hints exist: boolean (anything that begins with a t or 1 is true, anything that begins with f or 0 is false, everything else is an error), string, literal (same as string, but not quoted if it would be quoted otherwise), int32 (int is an alias for this), int64, and datetime. Only UNIX timestamps can be type-hinted to datetime, anything else will likely result in a casting error.

It is also possible to control what happens when type casting fails: syslog-ng can drop the whole message, drop the property, or fall back to string. It can also do all of these silently:

options {
typecast(on-error(silently-drop-property));
};

Using this feature with $(format-json) is very similary too:

$(format-json date=datetime("$UNIXDATE") pid=int64("$PID"))

With this feature in place, you can now store your non-string values with their proper types!

Unit suffixes make it considerably easier to set limits and describe numbers within the syslog-ng configuration. We no longer need to spell out sizes to the byte precious, it is now enough to write: log-fifo-size(200MiB). Now, syslog-ng will understand suffixes for kilo-, mega-, and giga-bytes, (K, M, G, respectively) either in base-10 or base-2 (with an extra i after the suffix). One can also omit the trailing b from the end.

So, to set the log-fifo-size() to 2097152 bytes, one can simply use 2MiB. Or, to set it to 2000000, 2Mb. That's a whole lot easier, isn't it? No more counting zeros, no more silly typos in a ten-digit number, no more pain, but easily readable units!

Apart from the features above, there have been a lot of other changes and improvements in the code base:

We now have a (mostly) non-recursive, quiet build system, which is
not only much faster than what we had before, but more reliable too,
and easier to glance through its logs, too.

A few old settings that were never used were removed (the
username() and password() settings of the
mongodb() driver), some were renamed and deprecated:
the replace() key transformation function of
value-pairs() was renamed to
replace-prefix(), as that makes the intent clearer. In
this latter case, the old name is still valid, but obsoleted.

Our integration with systemd is
much tighter now: syslog-ng can notify systemd when it is ready.
This also means that when systemd support is enabled, certain
systemd libraries will have to be installed: syslog-ng no longer
carries a convenience copy.

A new filter was implemented: in-list, with which one
can implement efficient white- or blacklists.
To use it, you will need a file with one value a line, and do
something along these lines:

syslog-ng now supports Linux 3.5+-style /dev/kmsg, and
will use that instead of /proc/kmsg when a sufficiently
recent kernel is detected (assuming one is using the
system() source).
The new kernel log format supports structured messages, and
syslog-ng is smart enough to parse them, and make them accessible
like all other message properties (with a .linux. prefix).

Gergely Nagy

A tiny mouse, a hacker.

Senior software engineer, Debian developer,
GPL zealot and a few other boring things. If you are a recruiter,
I strongly recommend reading
my list of conditions and
preferences, which I prepared just to make your life
easier.