Configure a time-based lookup

If your lookup table has a field that represents time, you can use it to create a time-based lookup. This is also referred to as a temporal lookup. You can configure all four lookup types as time-based lookups.

Simple time-based lookups attempt to match the event timestamp with the timestamp of a record in the lookup table, and then perform operations like adding one or more fields to the event from the matched record.

You can also define time-bounded lookups, which use the event time to define a range of time within which to match lookup records. For example, you could create a time-bounded lookup that matches the first lookup table record with a timestamp that falls within 10 seconds before the event timestamp.

Defining time-based lookups

To create a simple time-based lookup, add the following lines to your lookup stanza in transforms.conf:

time_field = <field_name>
time_format = <string>

Here are the definitions of these settings.

Setting

Description

Default

time_field

Identifies the field in the lookup table that represents the timestamp. The search processor applies the first matching entry in descending order.

Defining time-bounded lookups

To create a time bounded lookup, add these optional settings to your time-based lookup configuration:

max_offset_secs = <integer>
min_offset_secs = <integer>

Here are the definitions of these settings:

Setting

Description

Default

max_offset_secs

The maximum amount of time in seconds that an event timestamp can be later than the lookup record timestamp, for a match to occur.

2000000000 (effectively no default)

min_offset_secs

The minimum amount of time in seconds that an event timestamp can be later than the lookup record timestamp, for a match to occur.

0

The max_offset_secs and min_offset_secs settings define the earliest and latest times within which the search processor can search for matching records in the lookup table. The search processor calculates the earliest and latest time values from the event time like this:

Within this window of time, the search processor applies a match in descending order of time up to the point where we get max_matches number of matches for that event. If max_matches is not set, it defaults to 1. For more information about max_matches see Add field matching rules to your lookup configuration.

Time-based lookup example

Here's an example of a CSV lookup that uses DHCP logs to identify users on a network based on their IP address and the timestamp. The DHCP logs are in a file, dhcp.csv, which contains the timestamp, IP address, and the user's name and MAC address.

If you wanted to turn this into a time-bounded lookup, you could add the following settings to the [dhcpLookup] stanza in transforms.conf:

max_offset_secs = 10
min_offset_secs = 0

This would cause the lookup to match events to the first lookup table record with a timestamp that falls within a range of time bounded by the event timestamp and ten seconds before the event timestamp.

Comments

@Woodcock

As you can see, we've updated the topic to better explain the matching logic for time-bounded lookups. It's also been restructured to do a better job of differentiating between simple "time-based" lookups and slightly more complex "time-bounded" lookups. Hope this helps.

Mness, Splunker

January 10, 2018

This does not describe the algorithm for matching and really, that is the most important part. Does it use the min/max offsets only in 1 direction? Does it match "closest time that is not bigger" or "closest time that is not smaller" or what?

Configure a time-based lookup

Enter your email address, and someone from the documentation team will respond to you:

Send me a copy of this feedback

Please provide your comments here. Ask a question or make a suggestion.

Feedback submitted, thanks!

You must be logged into splunk.com in order to post comments.
Log in now.

Please try to keep this discussion focused on the content covered in this documentation topic.
If you have a more general question about Splunk functionality or are experiencing a difficulty with Splunk,
consider posting a question to Splunkbase Answers.

0
out of 1000 Characters

Your Comment Has Been Posted Above

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website.
Learn more (including how to update your settings) here »