​Troubleshooting license violation error in splunk:

Handling lice violation is very common issue faced by Splunk admins.Sometimes it may take longer time to identify the root cause of violation if environment is large.Below we have collected most useful queries which will be helpful to identify the source of violation.

What happens when I exceed my Enterprise license limit?"If you exceed your licensed daily volume on any one calendar day, you will get a violation warning. The message persists for 14 days. If you have 5 or more violations on an Enterprise license or 3 violations on a Free license in a rolling 30-day period, search will be disabled. Search capabilities return when you have fewer than 5 (Enterprise) or 3 (Free) violations in the previous 30 days or when you apply a new license with a larger volume limit.

Note: During a license violation period, Splunk does not stop indexing your data. Splunk only blocks access while you exceed your license.

Note: Searches to the _internal index are not disabled even during a licensing-enforcement period, so you can still access the Indexing Status dashboard, or run searches against _internal to diagnose the licensing problem. "

so: you can exceed your Enterprise license 4 times within 30 days--the 5th time, search will be disabled. You can exceed your Free licenses 2 times, and the 3rd time, search will be disabled.

2- If some forwarders are not necessary, turn splunk forwarder off on those boxes.Why did you deployed a forwarder on every single box in the first place !!!

3- If some useless files are being indexed, be more selective.Disable the inputs, or use whitelist/blacklists to limit the scopeexample to drop the core files, or to index only *.log files:[montitor:///var/log] blacklist=\.core$ [monitor:///mypath/*\.log]

4 - If some servers are sending to much data (syslog by example)disable the routing to splunk, or select the components to send.example on syslog.conf (send only critical and errors, and every event from my application)*.CRITICAL splunk.mydomain.com *.ERROR splunk.mydomain.com myapplication.* splunk.mydomain.com

6- Search for duplicates events in the logs, please check they exists in the original logs, or if the same log file is being indexed several times (some log rotation may cause that)here are searches to find duplicates in splunk :* | eval raw=_raw | convert ctime(_indextime) as indextime | stats count first(indextime) as first last(indextime) as last by raw | where count > 1 | table count first last rawThen drilldown to the source to figure.