Tech tips and tricks from the world of network traffic and security monitoring

Show me a list of top 100 flows that uploaded data to servers outside North America last week

or even the basic query

Show me the top 100 large volume flows today

This kind of query is surprisingly difficult for large datasets over long timeframes. You use Flow Trackers in Trisul for exactly this purpose. In this blog, lets see how you can create a flow tracker of your own.

How this works?

Essentially Trisul is a streaming analytics tool. It maintains a table, also known as a sketch, of top flows matching criteria your specify. This “sketch” is then flushed to disk every 5 minutes. Out of the box, Trisul has 6 flow trackers. One each for Total Volume, Upload, Download, Duration, TCP Payload Upload, and TCP Payload Download. Therefore every 5 minutes, 6 topper sketches are flushed to disk, each containing 100 top flows in that group. So when you run these long term range based queries, the sketches are used rather than the raw flows.

Example : Track top flows to servers outside America (US/CA/MX)

Let us create a flow tracker of our own. One that tracks all flows that have been marked as involving countries other than US, CA, MX.

Step 1 : First tag all flows with country codes

We first need to mark the flows with a country code. For that we use an Automatic Flow Tagger . If you’ve used them you would know that tags are nothing but text labels added to flow records based on observed metrics.

Go to Tools > Flow Taggers > Manage

Create a new Automatic Flow Tagger , select the Country counter group, and enter * to match all country codes.

Step 2 : Create a flow tracker based on the country tags

We now create a new flow tracker for flows that do not involve US, MX, CA country codes.

Go to Tools > Flow Tracker > Manage

Create a New Flow Tracker and fill out as shown below

Note that we used negated keys to track flows to countries other than US/UK/MX

Thats all it takes. You only have to restart Trisul to start tracking these flows.

Other types

You can use hostnames, IP ranges (even really large ranges such as 100.0.0.0~200.0.0.0), Port ranges (Port-1000~Port-2000) also for flow trackers. All the TCP/IP tuples can be used as well as the tags in the above example. Trisul detects the tuple automatically from what you’ve entered.

Viewing

You may now view flow tracker activity from Tools > Flow Tracker or from Retro Tools > Flow Tracker