Navigation

UDPLog is a system for emitting application log events via UDP and shipping
them via RabbitMQ or Scribe for further processing. The idea is that the
applications sends its structured log events to a dedicated shipping daemon on
the same machine, which in turn passes it on to one or more remote services. As
this uses UDP, emitting events is non-blocking fire-and-forget. A system like
Logstash can then be used to process and store log events.

Warning

The U in UDPLog stands for unreliable, and that’s just what it is. Log
messages are delivered on a best-effort basis and are not guaranteed in
any way, even though the default mode is to send logs to a daemon running
on the same machine.

Do not use UDPLog for any data that must be reliable, such as any
information used for billing of any sort!

A log event a combination of a category identifier (ASCII, matching the regular
expression ^[0-9A-Za-z_]+$) and a set of name/value pairs. The UDP wire
protocol represents an event as a single datagram composed of the the category,
a colon character, an optional whitespace character and the name/value pairs
rendered as a JSON object:

In general, it is better to have to fewer distinct categories, and to have
multiple types of entries in the same category if they have common fields.
Using ElasticSearch and Logstash, you can then define a mapping for each
category. To distinguish the events in the same category, you can add another
field like event.

In general, an application shouldn’t have more than a few categories, and
categories can span multiple applications.

The UDPLog libraries already adds a timestamp to each event. You do not need to
add a timestamp to your logs, unless you want to record the exact time your
event happened as opposed to the time the log was created. In that case, set
the timestamp field.

All times should be expressed in seconds, not micro or milliseconds. Generally,
you’ll want a floating point number of seconds. Timestamps are expressed as
floating point seconds since the UNIX epoch.