Metricbeat: new HAProxy module

Filebeat: The return of the Symlinks

Filebeat used to somehow unintentionally follow symlinks when opening files. So we deprecated them in 1.3 and were planning to remove the support for symlinks in 5.0. However, we’ve got some pretty strong pushback from Kubernetes users (K8s uses symlinks to map log files with useful file names), so we’ve listened and added back support for them. However, the symlink support is off by default and marked as experimental at this point.

Filebeat: close_removed and clean_removed are now on by default

This will keep the registry file from growing to quickly. For background: in 1.x the registry file was a map using the filename as a key. In 5.x the registry file is a list. A side effect of this change is that the registry file grows even if the same file names are reused (think of the usual syslog file names). By making these two options on by default, we’re making sure the registry file stays small.

import_dashboards: automatically select the right Beat

Now that the import_dashboard scripts were rewritten in Go and download the beats-dashboards package when executed, we wanted to make sure that it does the “right thing” when called without any arguments. So we now adjust the binary at build time to have a different `-beat` argument depending on the package.

Packetbeat: Improve handling of HTTP messages larger than 10MB

Old implementation dropped messages larger than 10MB (hard coded), to protect against memory DoS. However, in some cases the transaction was still recorded, but the parsing of the headers was incomplete. This PR makes the support for large messages explicit, by adding a mode to the parser that "sees" the segments without storing them.