Re: DNS Latency Dashboard

This is a great solution if you can afford the $$$ to up your reporter to the size needed to do query logging. Have you looked at creating some summary jobs so that it could be used for long term trending or anomly detection? I'm guessing that Splunk would have some issues with running the dashboard on a 20,000+ QPS grid over a week timeframe. But having access quickly to the domains driving the longer recursive queries would be of great value in troubleshooting issues. We still use the SNMP recursive latancy data that already exists by my round about syslog injection of the data into the reporting tool. It of course doesn't have this level of detal.

Re: DNS Latency Dashboard

I am very familiar with the reporting appliance. What index aare you talking about? Without the syslog data you do not get that data. We are a very large enterprise and when we tried tuning it down it did not work. I need to go onto my lab notes and give some specifics. It comes down to this. The respones and queries are in the info priority which has the most traffic of all the priorities. That makes since since it is only information. We tried using the percentages on the Reporting Appliance settings but I do not think they work as advertised. We tried a bunch of different settings and none worked.

That is complex to setup the first time, but we have now been running it for years. Just the "uncatorgorized" syslogs are 30 gig a day on our grid. I need just a few hundred meg of those logs to get some specifc data on the grid. That process has let us stay well under the 20 gig limit on our reporter.

We had this same discussion years ago I remember you. The problem is that the Reporting and Appliance application that infoblox set up does not let you use the wonderful features of splunk where you can do that kind of granualarity pre-processing that you would need to do.

Re: DNS Latency Dashboard

I am now curious to try this. What version NIOS and what version reporter? The problem is still a matter of volume. Limit query to 10 minutes or a 100 mb. I think I get a 100mbs in seconds. I really want to make the Reporting appliance show some value.

Re: DNS Latency Dashboard

So how are you getting the captured traffic into the indexer? I see the settings, not saving it to local disk. I see an SCP transfer. I am clued out how you get the query capture to populate the indexes.

I think what you are referring to is using the "Infoblox Data Connector" to receive SCP uploads of captured query/response data from your DNS grid members, let Data Connector do the pre-processing and then the Splunk forwarder on the data connector forward that data to the Indexer.

You'll find the deployment guide and user guide for Data Connector under the "Tech Docs" section on the Infoblox Support Site

Re: DNS Latency Dashboard

By the way, index for this data is already pre-defined on the Indexer on latest NIOS versions and the index is ib_dns_capture. There are around 4-6 reports/dashboards that would give you stats from the new data, such as the "DNS Domains Queries by Client" report/dashboard.

Since this additional data contains queries/responses in standard syslog format, you could perhaps additionally look into using rex/erex/ifx to extract additional fields, if required.

So how are you getting the captured traffic into the indexer? I see the settings, not saving it to local disk. I see an SCP transfer. I am clued out how you get the query capture to populate the indexes.

I"m not sure who you were replying to, but we do not push the query logs to the reporter. We created a "data collector" appliance long before that was an offering from Infoblox. On that Linux box I get all the dns log captures via SCP, parse and forward them on to various locations, one of the destinations is actually the Infoblox Data connector appliance which only forwards them on to the cloud as we could not replicate that functionality with our custom appliance.

We get close to the above functionality by using the SNMP values for recursive DNS queries, but it does not have the details that the full query logging provides with the above report. We do some alerting on the recursive DNS latency and the reporting tool, but to trouble shoot it currently, we just grep the flat files of the query logs.

Re: DNS Latency Dashboard

That is very interesting. The fact that my lab is virtual appliancs gives me the opportunity that I did not have when I first started on this journey. Since I am just using Temp licenses and it is just a "lab" I am going to try the Data collector out. I have also set up some Linux servers in my lab. One of those is collecting syslogs. I did not even think to run those back through the reporter! If I have questions I know where to ask. Friday, Monday I downloaded and read all the documentation on the data collector. Today I plan to start trying to set it up in the lab. I will post here my lessons learned.

When I went to try the queries fromn the dash board I only had the query being captured.

I was able to set up the data connector and the reporter has a source type of ib;dns;capture. That seems right. The problem is that I now have 10,000 plus sources. All the captures are there as csv files. yikes. That is not something is desired. What did I set up wrong? I suspect it is because I foillowed the documentation and checked the block on retain captured queries/replies on the local disk. I just unchecked that block.