Welcome to Splunk Answers, a Q&A forum for users to find answers to questions about deploying, managing, and using Splunk products. Contributors of all backgrounds and levels of expertise come here to find solutions to their issues, and to help other users in the Splunk community with their own questions.

This quick tutorial will help you get started with key features to help you find the answers you need. You will receive 10 karma points upon successful completion!

Refine your search:

Monitor advice needed

0

After reading the docs and looking in forums, I thought I had a understanding of monitor and what it does...I guess not.Is a monitor, set up to follow from tail, only supposed to index data that is written to a directory from the time of monitor creation? I manually made a monitor in my inputs.conf file after I saw that the monitor I set up in the manager was grabbing events that were pre-dated.

[monitor:///dir/path]
blacklist = dir/
followTail = 1

I didn't see events logging right away after restarting splunk, so I thought it was working properly...that is, only indexing events that are new. I came in today to find my license had exceeded limit over night and splunk has indexed events from last year.

Someone tell me what is wrong with this. Is there a way to set up a monitor that only indexes new events? Why is my monitor indexing the whole file?

People who like this

2 Answers

With the option followTail enabled Splunk is going to monitor only events being added into the monitored stanza after restarting Splunk. Maybe the old files are in compress format and their modification time has been changed. Or maybe you have an incorrect timestamp extraction problem, and the events are not really from last year.

You can find more information on how to troubleshoot this problem by reviewing the content of this twiki page:

well there is nothing wrong, this is the way splunk monitors directories: starting at the moment you add it in a monitor [stanza] splunk is 'eating' up any readable file in this directory. How should splunk know what you consider as old data?

if you really want to have only new data to be indexed, move the 'old logs' out of the way before you [monitor] the directory.

after that splunk will index only the new files coming in and will forget about the already indexed files.