Logstash Lines: SNMP input

Welcome to Logstash Lines! With these weekly series, we're keeping you up to date with what's new in Logstash, including the latest commits and releases.

Did you know that Logstash 6.2 is already available? Try it and let us know what you think.

Important Notes for Upgrades for Persistent Queue users

If you use the Logstash Persistent Queue we have an important message with regard to upgrading your Logstash instance. We’ve found a few different issues that can affect the way data is serialized in some versions of Logstash. We’ve made some fixes, but an unfortunate consequence of those fixes is that the queue data files may not work when upgrading between Logstash versions. This is something that we did not intend, and are striving not to repeat in the future.

When upgrading Logstash it is important that you either fully drain the queue before upgrading, or fully remove the queue data files. The location of the queue data files is determined by the path.queue setting in your logstash.yml. To drain the queue fully you must:

Set queue.drain: true in your logstash.yml

Start Logstash

Issue a graceful shutdown to Logstash by sending it the SIGTERM signal. This will shut down all inputs and continue processing the pipeline until the queue is empty

Perform the upgrade

We’re aiming to improve this situation in 6.3.0, which shouldn’t require these extra steps in updates from that point onward. Our goal is to provide a seamless experience, and we’re taking a close look at the root causes of this situation.

Logstash SNMP Input v0.1.0.beta1 is out!

We're happy to announce you can now perform SNMP polling with Logstash with the first beta version of SNMP input plugin, which has just been released.

The project is quite young but already has a big list of desirable features on its roadmap. This release is intended to collect feedback from those of you that value this capability and can, better than anyone else, help us shape the plugin into an SNMP polling tool you'd want to use.

Our colleague Rob went through the effort of understanding and solving a severe limitation of the existing caching implementation in the DNS filter. This synchronized access to the hit cache caused a bottleneck in Logstash, dropping the event rate of a pipeline from 41k to 600 events per second with the introduction of the DNS filter. After applying his pull request #42 and properly sizing the hit cache for his particular use case, we were able to observe the throughput reach 40k eps, a mere 2% performance impact of performing DNS resolution once the cache warmed up.

Other changes

Repositories under: elastic/logstash-plugins

The filter now only calls filter_matched on events that actually matched.
This fixes issues where all events would have success-related actions happened
when no match had actually happened (
add_tag, add_field, remove_tag,
remove_field)
#99