The overall objective of this research
is to develop high performance data mining algorithms and tools that will
provide support required to analyze the massive data sets generated by
various processes that monitor computing and information systems. This
research is being conducted as a part of MINDS (Minnesota Intrusion
Detection System) project that is developing a suite of data mining
techniques to automatically detect attacks against computer networks and
systems. Figure 1 illustrates the MINDS system applied to real network
traffic data. There are several integral parts of the research within this
project:

Filtering / Preprocessing / Known attack detection
module

Scan detector

Anomaly detection algorithms

Summarization of attacks using association pattern
analysis

Figure 1. Architecture of MINDS System

The MINDS system is being used by the University of
Minnesota (UM) network security analysts, Paul Dokas, in a production
mode, as follows. The UM network traffic is independently monitored using
SNORT, an open source network intrusion detection system based on
signatures, as well as using MINDS. As the first step in MINDS, the net
flow tools are used to collect the network traffic data from CISCO
routers. This data is filtered to remove network connections not
interesting for analysis and preprocessed to collect basic features (such
as source and destination IP addresses, source and destination ports,
protocols) and to extract some derived features (such as number of flows
to unique destination IP addresses inside the network within the last T
seconds from the same source). Such created data is fed into the MINDS
system. The known attack detection module detects network connections that
correspond to attacks for which the models are known. The remaining
connections are fed to the anomaly detection modules, which assigns a
score that reflects how anomalous the connection is compared to the normal
network traffic. Connections that are highly anomalous are analyzed by the
UM network security analysts to determine if they are truly intrusions are
false alarms. Highly anomalous connections are further analyzed and
summarized using association pattern analysis. Such association rules
provide a concise characterization of the detected anomalies, and are
helpful in creating new signatures and models for emerging attacks.

The University of Minnesota network security analysts
has been using MINDS successfully to detect novel intrusions that could
not be identified using state-of-the-art signature-based tools such as
SNORT. Many of these attacks detected by MINDS, have already been on the
CERT/CC list of recent advisories and incident notes. In the absence of
manual screening of incoming network connections, it is not possible to
provide any estimate of MINDS's detection rate (i.e. the fraction of
attacks that are identified by MINDS as anomalous). However, nearly all
connections that are ranked highly by our anomaly detection algorithms are
found to be interesting and anomalous by the network security analyst on
our team. Summarization of anomalous connections using association pattern
analysis has been very helpful in understanding nature of cyber attacks as
well as in creating new signature rules for intrusion detection systems.