Wednesday, April 27, 2016

Encrypted connections are used to
transfer sensitive data between two hosts over the unsecured
internet. Surveys show that 25% - 35% of enterprise traffic is SSL
encrypted. The number can be as high as 70% in some specific
industries. But, can SSL encrypted traffic ensure security ?

Study shows that websites using SSL are
no way more protected than websites that are not encrypted. Attackers
use advanced techniques to conceal their communication in an SSL
connection. And, to detect and prevent those attacks we need to look
through an encrypted SSL connection to find out malicious contents.
SSL Inspection is a technique using which encrypted SSL traffic can
be decrypted and sent to other security appliances, which can analyze
it further to detect harmful contents and prevent possible damages.

How SSL can be used by attackers

If malicious communications are not
encrypted, it can be detected by standard security appliances like
IDS, IPS, firewalls easily. But, these security tools are not able to
decrypt an encrypted connection and look through it. So, many a times
attackers take advantage of that to make attacks. They use SSL
connection to conceal their malicious communication.

If the initial communication for
infection is through an approved port and a seemingly secured
browser, it can bypass the firewall/IPS easily. In fact, it is
easier to attack an organization using applications that use
encryption.

Many a times attackers infect
computers with malware and create a botnet. Then they exploit the
computational resources of the infected computers for malicious
purposes like making DDoS attacks, spreading malware or for more
attacks. For malware families like Zeus, the communication with the
Command & Control Server of the botnet is concealed within an
SSL connection. The malware first opens an SSL connection and then
use that for communication with the Command & Control Server
for sending stolen sensitive data of the victims.

Attackers can use SSL for Phishing
attacks also. They can send malicious link to employees of an
organization via emails and trick them to click on it. On clicking
on the link, it may take the victims to an malicious SSL server
controlled by the attackers. If the communication is through some
approved ports, the firewall/IPS may not detect it. And, the
attackers can infect the computers with malware to create a botnet.
After that, they can easily exfiltrate sensitive data like financial
account data of the organization using an encrypted SSL connection.

What is SSL Inspection ?

Currently, many security devices cannot
inspect encrypted traffic and the few that can decrypt SSL traffic
cause significant performance degradation and are very expensive.

SSL inspectors work with secure network
gateways to monitor inbound and outbound SSL traffic. They decrypt
inbound and outbound SSL traffic, including the web and email
communication and send the suspicious traffic to other security
devices like IDS, IPS, network forensic device, advanced network
gateways etc for further inspection and analysis. If the decrypted
traffic is sent to active security tools like IPS, the suspected
traffic is analyzed and proper actions are taken to prevent possible
damages.

But, there may be cases when SSL
inspectors may not want to decrypt certain SSL traffic, such as
patient data in a hospital. So, it must whitelist and filter SSL
traffic for inspection.

SSL inspectors should be able to
process large amount of data quickly. They usually contain high
performance compute engines that have hardware performance
accelerators to handle SSL traffic, which enables them to monitor SSL
traffic in real time.

An SSL inspection appliance detects
an SSL session and looks at its policy to determine whether the
traffic should be inspected. If the SSL traffic is suspicious, it
decrypts the data and send the decrypted data to other security tools
for further analysis.

IDS/IPS, firewalls and network
gateways – if malicious traffic is found on further analysis by
these devices, the packets are dropped and the SSL session is
killed.

Email filtering devices

Data Loss Prevention devices –
when SSL inspectors send decrypted traffic to these devices, they
can do pattern matching to look for sensitive data such as social
security numbers, credit card information, bank account and routing
data etc to prevent data exfiltration from an organization.

Forensics and investigative tools

SSL dectyptors can also be used for
cloud services monitoring. All secure services running in the cloud
look same at TCP/IP layer. The traffic can be differentiated only
when they are decrypted.

Monday, April 25, 2016

Every time we swipe out credit cards in
a point of sale device, our credit card number is read. And, no doubt
we need to encrypt it for maintaining security. Format Preserving
Encryption or FPE is an encryption technology in which the format of
the ciphertext output remains same as format of the plaintext input.
So, that would mean if we encrypt a 16 digit credit card number using
FPE, the encrypted output will be another 16 digit number.

But, why do we need that ? Let's
understand it in more details.

Challenges of encrypting credit card
numbers

We can use a block cipher to encrypt
credit card numbers. But, there are certain challenges with that
approach.

If we encrypt a 16 digit credit
card number using a block cipher, the output will be 34 bytes long.
This may break existing applications that expect the credit card
number to be a 16 digit number only.

The 34 byte ciphertext of a 16
digit credit card number obtained using block cipher will contain
hexadecimal values containing alphanumeric and special characters.
The ciphertext output may not be another credit card number. And,
that may break existing applications.

If the ciphertext is decrypted and
encrypted again, it should retain its value. It should not depend on
any random seed value to initialize the encryption as it is done in
a block cipher.

FPE is an encryption using which credit
card numbers can be encrypted in such a way that field length and
data type of the plaintext credit card number is preserved across
encryption, which would mean, the encrypted output of a 16 digit
credit card number will be another 16 digit number which can
integrate well with the existing applications.

So, we can say, FPE is like a random
permutation which, in this case, takes a 16 digit number as input and
gives another 16 digit number as output. But, for a large domain, it
is infeasible to precompute a truly random permutation and remember
it. FPE uses a secret key to generate pseudorandom permutation of a
number in such a way that the computation time for a single value is
less and computationally feasible.

How is Format Preserving Encryption
done

Format Preserving Encryption
uses a block cipher like AES as a primitive. So, if the block cipher
algorithm is secure, FPE will be unbreakable.

There are a number of algorithms for
Format Preserving Encryption. Some of them are mentioned below :

FPE using Prefix Cipher :

Let's say, there are N numbers from 0
to N-1. First, a block cipher is applied on each of these integers.
As a result, we would get another set of integers called weights of
those N integers. Next, we can sort the integers as per their
weights.

Let's say, S be the set of allowed
values for the inputs. In this algorithm, first a block cipher will
be applied to each of the inputs. If the output is not in S, block
cipher will be applied again on that input, until the output
ciphertext is in S.

As this pseudorandom permutation is
one-to-one and the domain is finite, the iteration is guaranteed to
stop.

FPE using Feistel Network

In this algorithm,
the input is first split into two halves L1 and R1
and the following operations are performed on each half :

L2 = R1

R2 = L1
XOR F(ki, R1)

Here, a single key k is used with a
different tweak in each round, using the round count as the tweak.

Acceptance of FPE

National
Institute of Standards and Technology has recommended Format
Preserving Encryption for encrypting sensitive data like credit card
numbers, Social Security Numbers etc.

This was an introductory article on
Format Preserving Encryption just to give some basic information.
Hope it solved its purpose.

Friday, April 22, 2016

A Conficker or Downup or
Downadup or Kido is a computer worm that infects a
Microsoft Windows machine using some vulnerability in the Microsoft
Windows Operating System software and creates a botnet of infected
computers to steal sensitive information of users including banking
credentials, credit card information etc using keyloggers. This
malware uses advanced malware techniques and is extremely difficult
to control. Since its discovery in 2008, the malware has infected
millions of computers.

How does Conficker malware infect a computer ?

Conficker is delivered to an infected
system as a Dynamic Link Library or DLL. It cannot run as a
standalone program.

The worm first infects a Windows system
using certain vulnerabilities in the system and then exploits
shellcode to inject the DLL into the running Windows server service.
And then, it creates a registry entry to ensure that it runs
everytime the machine reboots.

After infecting a computer, Conficker
uses a list of websites to find out the IP address of the infected
machine. It then uses the IP address to download a small HTTP server
and opens that in the infected machine.

Once the HTTP server is up, the worm
then scans for other vulnerable machines. Once it finds a vulnerable
target machine to infect, it sends the URL of the currently infected
machine as a payload to the target vulnerable machine. The remote
target machine then downloads the worm from the URL sent and starts
infecting other vulnerable machines.

To infect a remote computer in the
network, the worm first tries with credentials of the currently
logged on user. If it is unsuccessful, it gains a list of user
accounts in the target machine and tries to login using each of the
username and a list of commonly used weak passwords. The worm then
drops a copy of itself in the admin share of the target.

Conficker then creates a remotely
scheduled job to activate the copy.

Conficker can also infect a computer
using removable drives or USB drives. For that, it first copies
itself to the drives using a random file name. It then changes the
autorun.inf file to show an additional option to “Open folder to
view files” with “Publisher not specified”, when the drive
connects with a computer. If a user cannot understand the trick and
selects that option, a copy of the worm will start running in the
computer.

After infecting a computer, the worm
generates a list of domain names using a randomization function
seeded with current UTC system date. All the infected machines try to
connect to the same set of domain names for updates.

Variants of Conficker malware

There are a number of variants of
Conficker worm.

Conficker.A – This is the
first version of the Conficker worm. It relies on Windows Server
Service vunerability for its propagation.

Conficker.C – It uses
50,000+ randomly generated domain names so that the security
community cannot block all of the domain registration associated
with the A & B variants. It also uses P&P coordination
channel for updates.

Conficker.D – It changed
the domain name registration algorithm to generate a large pool of
domain names. This variant just updates existing Conficker.C
infected machines and does not spread by attacking new systems.

Conficker.E – It is
another update to the Conficker.C code base.

System changes after infection of Conficker malware

After infecting a Windows computer, the
worm makes a couple of system changes.

Conficker changes system settings
of the infected computer so that the victim cannot view hidden
files.

It stops Windows Security Center
Service which notifies user about security settings.

It stops Windows Update Auto
Update Service.

It also stops Microsoft Error
Reporting Service.

Conficker resets the infected
computer's system restore point and prevents recovery of the system
using system restore.

It disables TCP/IP Tuning

It also disables third-party
security software to avoid detection.

It deletes backup files.

It increases traffic on port 445.

Access to administrator shared
files get access denied errors.

It checks for internet
connectivity in the infected system by trying to connect to a list
of websites.

Depending on system date, it
builds a URL to download files. The generated URL typically has a
domain name that is based on the current system date.

It increases network traffic in
the infected computer, making the system slow.

How to remove Conficker malware from an
infected system ?

There are a number of security tools
provided by various anti-virus vendors. Some of the links are given
below :

Wednesday, April 20, 2016

A Fast Flux Network is
a network of compromised computers and some public DNS records that
change frequently. As a result, the IP address associated with the
corresponding domain name changes frequently. This technique is often
used by the attackers to hide their malicious websites from
detection.

Why Fast Flux
Networks

Attackers typically
compromise one or more victim computer systems with malware and
exploit those to establish a fraudulent website like a Phishing
website. The problem of the attackers with this approach is, these
websites can be easily tracked down by public DNS name and IP address
to shut them down immediately.

So, the attackers
started using server address obfuscation. They often use a group of
proxy servers to redirect network. But, this approach also does not
prove to be much convenient for them because of limited scalability.
Moreover, these websites can still be tracked down quickly by
international cooperation.

So, the attackers
started using Fast Flux Networks.

The basic idea behind a
Fast Flux Network is to associate multiple IP addresses to a
malicious domain name. These IP addresses are swapped in and out with
extremely high frequency, may be in every 3 minutes, with the help of
changing DNS records. As a result, a browser connecting to the same
malicious website in every three minutes will see different IP
address each time and connect to the actual malicious website via
different infected computers every time.

How Fast Flux
Network works

In Fast Flux Networks,
attackers compromise a number of computer systems with malware and
then exploit their bandwidth and computation power to build the Fast
Flux Network.

In Fast Flux Networks,
attackers often use a number of compromised computers as front end
systems. These front end systems get the requests from the victims to
connect to the malicious website and redirect those requests to the
back-end servers.

So, the large pool of
rotating IP addresses do not correspond to the actual back-end
servers. Instead, they fluctuate among many front end servers which
in turn funnel the requests and redirect them to the actual back-end
servers.

Fast Flux motherships
are the main controlling elements behind the front end servers. They
are similar to Command & Control or C & C servers, though
they have much more features compared to the C & C servers.

This mothership node is
hidden by the front end servers, which make them extremely difficult
to track down. They often host both DNS and HTTP services and use web
server virtual hosting configuration to manage content availability.

Types of Fast Flux
Networks

There are two types of
Fast Flux Networks that are widely used by the attackers :

Single Flux
Network

Double Flux
Network

Single Flux
Network

In a Single Flux
Network, when a victim makes a request to the attacker's website, the
request first reaches a front end redirector. This redirector
redirects the request to the target website. There are typically a
number of nodes that are used as front end redirectors. So, if one
node gets detected and shut down, many other compromised hosts can
take its position. Using this Single Flux Network, the DNS
record corresponding to resolving the IP address of the attacker's
website change frequently, may be in every 3 minutes.

Let's understand this
with an example.

Suppose, a victim makes
a request to access the website malicious.attacker.com using a URL
published by the attackers. So, the victim's computer will first
make a request to resolve the domain name for malicious.attacker.com.

At this point, the DNS
root nameserver will be queried first and it would return the
nameserver responsible for the top level domain .com. Next, .com
nameserver will be queried to get the nameserver responsible for
attacker.com, say ns.attacker.com. Now, ns.attacker.com will be
queried to get the IP address for malicious.attacker.com.

For normal DNS lookup,
this DNS record usually remains constant. But, in Single Flux
Network, this DNS record change frequently to contain multiple IP
addresses in round robin fashion.

Double Flux
Network

For a Double Flux
Network, the victim sends similar DNS query to resolve the IP address
for malicious.attacker.com and queries are made in similar fashion to
get nameserver of .com and then ns.attacker.com.

But, here the
nameserver ns.attacker.com is actually a part of the Double Flux
Network and its own IP address itself changes frequently. When a DNS
request reaches ns.attacker.com, the nameserver forwards the queries
to corresponding mothership node and then a connection is made to the
target system.

Advantages of Fast
Flux Networks for attackers

There are a couple of
reasons because of which the attackers use Fast Flux Networks for
their fraudulent activities.

Easy to operate

Fast Flux Networks are
easy to operate for the attackers. They just need one powerful
back-end server to serve the actual contents and DNS information. The
published URL first points to the front end servers which redirect
the requests to the actual back-end server. So, the attackers need to
maintain only a few number of core systems to host their malicious
website, instead of maintaining many servers to host their fraudulent
website.

Difficult to
investigate

Fast Flux Networks make
criminal investigations much difficult. Security experts typically
recover a handful of IP addresses corresponding to the disposable
front end servers. They are, in most of the cases, spread across
multiple jurisdiction, continents, regional languages and time zones.
And, that complicates the investigation to a large extent.

Hidden back end
servers

Back end servers are
hidden by the front end nodes. As a result, it takes much longer to
identify and shut down those core back end servers.

Monday, April 18, 2016

Zeus, ZeuS or Zbot is a Trojan malware
package which is used by the attackers to steal sensitive data of
users, especially banking credentials, causing heavy financial losses
to the users. This malware was first identified in July 2007 and
became more widespread since 2009. Attackers use this malwre to
create a botnet and then use that to steal banking credentials of the
victim.

How does Zeus malware infect a computer ?

Zeus is one of the most widely known
notorious malware which is used by the attackers to create a botnet
which silently harvests financial data of the victims and send it
back to the attackers. Zeus is basically the name of the malware
toolkit which is widely distributed and used by the underworld
miscreants to create the information stealing trojans.

A Zeus malware toolkit typically has
the following components :

Builder

Configuration File

Exe File

Server

Builder

Builder is used by the miscreants to
create the malware executable file as well as the configuration file.
The malware usually uses encryption mechanism to obfuscate itself.

Configuration File

Configuration file is downloaded at the
time of execution of the executable malware code. It contains the
following information along with some other information :

URL from which the Zeus executable
will be downloaded

URL to which the stolen data of
victims will be sent back

A set of IP/domain pairs that will
be written into the infected host file to hijack DNS requests of the
victim's computer

This configuration file is typically
stored in the server controlled by the attackers. The bot
periodically queries the server to retrieve the information contained
in the configuration file. A bot owner may upload a new configuration
file at the server at any time to change the configuration.

Exe File

Different underworld botnet customers
who use same version of Zeus toolkit typically have the same exe
file. But, the configuration file differs from one botnet to the
other.

Server

The server component of the toolkit is
basically a set of php scripts which are used to monitor, command and
collect information from the infected computers.

Steps of infection

Firstly the attackers use some
social engineering to trick the victims to download the malware.
They may send phishing email with malicious links or use some other
methods to convince the victims to click on a malicious link. The
malware can even get downloaded by exploiting security
vulnerabilities of commonly used software of the victim's computer.

The malware copies itself to a
location, execute the copy and then delete it to avoid detection.

After installation, the malware
changes browser settings of the victim's computer. Zeus typically
uses Man-In-The-Browser attack (What
is a Man-In-The-Browser Attack ? ) to steal sensitive
credentials of the victims.

The malware then infects code to
other processes in the victim's computer.

The injected code hooks apis in
each process.

Next, the configuration file gets
downloaded from the server controlled by the attackers.

The malware then uses api hooks to
steal sensitive data, especially banking credentials of the victims.

How does Zeus malware steal sensitive data of
victims ?

As mentioned earlier, Zeus uses
Man-In-The-Browser attack to steal banking credentials of the
victims. It typically follows the steps mentioned below to steal
sensitive data of the victims :

After infecting a computer, the
trojan installs a malicious extension to the victim's browser. When
the user restarts the browser, it gets loaded automatically.

The
extension registers a handler for every page load, which tracks all
the pages loaded by the browser and matches them with a list of
known websites.

Whenever
the user loads a page of a banking website, the extension registers
a button event handler.

The
user authenticates to the banking website giving his credentials.
When the user fills up a form for financial transaction, the
extension intercepts the communication. It notes down the data
entered by the user, but modifies the data and sends the modified
data to the banking web application.

The
web application performs the transaction as per the modified data
and sends the receipt.

The
extension again intercepts the communication. It modifies the data
in the receipt with the data entered by the user originally.

The
user gets the modified receipt filled up with data provided by him.

How to prevent Zeus malware ?

Zeus
malware is one of the most widely known notorious malware which is
difficult to detect. But, a user can always follow some simple steps
to prevent infection of the malware.

The
malware mostly uses some social engineering to infect a victim's
computer. Do not click on any link if you are not very sure of its
authenticity. Do not open email attachments if you are not sure of
the sender. And, always avoid downloading software from untrusted
sources.

The
malware exploits security vulnerabilities of commonly used software
to infect a computer. So, always keep your computer updated with
recent security patches of all the commonly used software.

Update
your Operating Systems with recent patches for the same reason.

Keep
your browser updated with recent patches. It would reduce the
security vulnerabilities present in the browser software.

Always
keep your system updated with recent patches of anti-malware
programs from a trusted source.

Closely
monitor any changes in browser settings is one option of preventing
this attack. Browser extensions and scripting should be limited.
And, do not use any browser extension if you are not very sure about
its authenticity.

Users
should educate themselves about Zeus malware and its attacks and use
their common sense while using sensitive banking web applications.

So,
beware of various malware programs and how to prevent them, so that
you can protect your data in a better way. And, stay safe, stay
secured.

Sunday, April 17, 2016

When sensitive data is transferred
between two hosts, an encrypted communication like TLS is used for
that purpose. And, digital certificates have a major role to play in
TLS communication.

A TLS server usually issues its digital
certificate from a trusted Certificate Authority or CA. When a TLS
client wants to establish a TLS connection with the server, the
server sends that digital certificate to the client. The client
verifies the certificate from the trusted CA and then proceeds with
establishing the connection.

But, there are a number of recent
incidences where a digital certificate is mistakenly issued by a CA
or a CA has been compromised. It is extremely difficult to detect such
fraudulent cases in current scenario. Our browsers cannot detect if a
malicious website is using a mistakenly issued certificate or a
certificate issued by a compromised or rogue CA. As a result, user
security gets compromised.

Certificate Transparency is an
experimental IETF (Internet Engineering Task Force) open standard
which is used to monitor and audit digital certificates and to detect
suspicious certificates and CAs within a reasonable time frame.

Functional Components

Certificate Transparency has three new
functional components to the current SSL certificate system :

Certificate Log

Certificate Monitor

Certificate Auditor

Certificate Log

A Certificate Log is a simple
network service that maintains a record of SSL certificates. Any
interested entity can submit certificates to the Certificate Log,
though it is expected that a CA or server operator submit a
certificate to the log. A log is a single, ever growing and append
only entity. When an interested party submits a valid digital
certificate to the log, the log returns a Signed Certificate
Timestamp or SCT. A valid digital certifcate can have a
single valid SCT. So, if the log has seen the digital certificate
before, it may return the same SCT which it has already returned
before.

Anyone who gets a SCT of a certificate
from a particular log, can verify it with the log. If there is any
inconsistency, that would mean the certificate is compromised.

A Certificate Log is cryptographically
assured. It uses a Merkle Hash Tree to maintain the log. A
Merkle Hash Tree is a binary tree whose nodes are hashes of two child
nodes. (More information on Merkle Tree can be found here : What
is a Merkle Tree ? ) The Merkle Hash Tree of a log contains
hashes of individual certificates in its leaves. And, every non-leaf
node is a hash of its pair of child nodes. The root hash of the
Merkle Tree is known as Merkle Tree Hash. The log server signs
its Merkle Tree Hash with its secret key and it is known as Signed
Tree Head or STH.

The Merkle Tree uses the hashing
algorithm SHA-256. And, to sign various data structures, it uses
elliptic curve signatures using NIST P-256 curve or RSA signatures
using a key of at least 2048 bits.

So, to summarize, the input of the
Merkle Hash Tree is a list of data entries or digital certificates.
These data entries are hashed to form the leaves of the Merkle Tree.
And, the output is a 32 byte Merkle Tree Hash.

Periodically, may be once in an hour,
the log server appends all its newly submitted certificates to the
log. Then, it creates a separate Merkle Tree Hash and combines it
with the old Merkle Tree Hash. The new Merkle Tree Hash is then
signed to create a new Signed Tree Head or STH. And, this process
continues for all certificates ever submitted to the log.

The log also publishes a list of
acceptable root certificates. A submitted certificate accompanies all
the additional certificates required to verify the certificate chain
up to required root certificate. When a TLS client receives a digital
certificate along with SCT, it should verify the SCT with the
signature input of SCT data and the certificate. They should also
verify the signature with the log's public keys. If the log fails to
provide a certificate with a valid SCT or presents multiple views of
the Merkle Tree, that would indicate a security violation.

Monitor

Monitors are publicly run servers that
periodically contact all the log servers to detect suspicious
certificates. They watch the certificate logs and detect illegitimate
or unauthorized certificates, unusual certificate extensions or
certificate with strange permissions. They also make sure that the
log certificates are visible to the log. Monitors periodically
fetches entries from the log servers and monitors them.

To monitor the log certificates, the
Monitors typically follow the steps below :

They fetch the Signed Tree Head or
STH from the log.

They verify the STH signature.

They fetch all the entries in the
tree corresponding to the STH.

They verify that the Merkle Tree
constructed from the fetched entries produce the same hash as that
of the STH.

Monitors are likely to be operated by
CAs. They can also operate as standalone entities and provide services
to the CAs and server operators. And, Monitors can be run by server
operators like Google, Microsoft, Yahoo etc.

Auditor

An Auditor is a software component that
fetches partial information from the log as input and verifies that
the information is consistent. They can also verify that a particular
certificate is present in the log.

An Auditor may be an integral component
of a TLS client like browser. It may periodically send a batch of
SCTs to its auditing component and verify the authenticity of the
certificates. It asynchronously contact the logs and perform
verification.

Basic operations of Certificate
Transparency

So, to summarize, here is how
Certificate Transparency operates :

Though anyone can submit a
certificate to the log, it is expected that a CA or server operator
submits a digital certificate. It is also possible for a CA to
submit a certificate before its issuance.

After submitting
the certificate to the log, the log returns an SCT or Signed
Certificate Timestamp and an MMD or Maximum Merge Delay. MMD
indicates the time by which the certificate will be added to the
log, so that the certificate can be issued within reasonable
timeframe.

During TLS handshake, the TLS
client receives the certificate along with the SCT and its signature
chain.

TLS clients can use the SCT and
the log to verify that the certificate is not misissued.

Please note that, the presence of
SCT alone does not guarantee that the certificate is not misissued.
The interested party must verify the SCT with the log.

Security

Misissued certificates that have not
been publicly logged, will not have a valid SCT. So, the TLS client
can detect its illegitimacy immediately.

TLS client can also verify the received
SCT with the log. If there is any inconsistency, it will flag a
security violation.

Interested party like domain owners can
monitor the log and detect a misissue of a certificate. In fact,
Certificate Transparency gives any interested party the opportunity
to monitor and verify the integrity of the certificates.

The main advantage of Certificate
Transparency is, it can detect a security violation within a few
hours, instead of a few days, a few weeks or even a few months. And,
this in turn speeds up the overall mitigation process in case of a
misissue of a certificate.

Friday, April 15, 2016

Deep Packet Inspection or DPI
is a technology
using which one can
examine the data part of the network packets and search for protocol
non-compliance, virus, spam, intrusions and other statistical
information
and decide whether the packet should be passed or dropped or should
be routed to a different destination for further processing.

DPI
is both a hardware and a software solution. A DPI device monitors the
payload of each packet passing through it and detects protocols,
application, inappropriate URL's, intrusion attempts and even malware
present in the data packet. It is often used to enhance network
security. And, Internet Service Providers often use it to decide on
data usage, data limits, bandwidth, compliance with regulations,
prioritization of traffic, load balancing or collection of
statistical data from its subscribers.

Let's
understand first how DPI works and how DPI technology has evolved.

OSI
Model and Flow of Data Packets

To
understand how DPI works and how this technology has evolved, we need
to understand how a data packet flows through OSI protocol stack.

As
per the OSI model, the communication system between the sender and
receiver of a network packet is partitioned into seven layers :

The
Application Layer
– responsible for interacting with the application software

The
Presentation Layer
– responsible for compression, encryption and formatting of data
being presented

The
Session Layer – responsible for creating, managing and ending a session's
communication

The
Transport Layer
– responsible for sequencing and delivery of data

The
Network Layer
– responsible for the addressing and routing of the network
packets

The
Data Link Layer
– responsible for formatting the packet as per the medium of
transmission of packets

The
Physical Layer –
responsible for defining the actual media and characteristics of the
transmitted data

When
we type a URL in the address bar of a browser, the data typically
flows through the OSI protocol stack in the following way :

We
type the URL in the address bar of the browser. The Application
Layer interacts with the corresponding software, here the web
browser. The browser makes an HTTP request to access the webpage
from the web server. The request is passed through the next layer of
the OSI model – the Presentation Layer.

The
Presentation Layer is concerned with the actual format of data being
presented. When the browser receives the data from the web server,
the Presentation Layer presents it in a proper format like JPEG,
MPEG, MOV, HTML etc. This layer can also encrypt and compress the
data.

The
next layer of the OSI model is the Session Layer. This layer is
responsible for creating, managing and ending session's
communication between the sender and receiver of the data. The
Session Layer, the Presentation Layer and the Application Layer are
mainly responsible for composing the payload of a packet.

The
Transport Layer deals with the sequencing and delivery of the data.
It segments the data into packets, sequence the packets, establishes
a connection between the source and destination of the packets and
then, sends those across through the next layer of the OSI model.
Please note that, the Transport Layer is not concerned with the
managing and ending of sessions. It only processes the connection
between the sender and the receiver of the data.

The
Network Layer is responsible for the addressing and routing of the
network packets. It deals with how the network packets will travel
from one part of the network to the other. However, it is not
concerned with whether the packets received are error free. The
Transport Layer takes care of that.

The
Data Link Layer formats the packets as per the medium used for
transmitting the packets – e.g. Wireless medium, ethernet
connection etc.

The
Physical Layer does not change the actual data of the packets. It
defines the actual media and characteristics of the transmitted
data. The Physical Layer, the Data Link Layer, the Network Layer and
the Transport Layer are mainly responsible for composing the headers
of network packets.

Lineage
of Packet Inspection

Initially,
Packet Inspection used to be used in traditional Firewalls. They
would use this technology to monitor and filter packets for network
security. Later, this technology gradually evolved to Deep Packet
Inspection. Now, DPI is widely used in modern Next Generation
Firewalls for enhancing network security, though the usage of DPI is
not at all limited to that. It is widely used for content
optimization, network and subscriber analysis and content regulation.

Shallow
Packet Inspection

Shallow
Packet Inspection
is widely used in traditional Firewalls. It works mainly in the first
three layers of the OSI model. This technology examines mainly the
headers of the network packets to decide on whether the packet should
be passed or should be dropped.

Shallow
Packet Inspection mainly observes the source and destination IP
addresses, the number of packets the message is broken into, the
total number of hops in routing the packet and synchronization data
for reassembling the packets etc to decide on whether the packet
should be processed further.

Medium
Packet Inspection

Medium
Packet Inspection
is widely used in application proxies. They examine the packet
headers and limited amount of payload of the packet. And, that
information is then matched against a pre-loaded parse list, which
can be easily updated by the system administrators. A parse list
allows specific packet types based on the data format types and
associated location in the internet, rather than their IP addresses
alone.

Medium
Packet Inspection technology can look into the Presentation Layer of
the packet's payload, which enables it to detect certain file
formats. Using Medium Packet Inspection devices, administrators can
thus prevent client computers from receiving flash files from
Youtube, image files from social networking sites etc. Medium Packet
Inspection can even prioritize some packets based on associated
application commands and file formats of the data. It can dig into
the packet to identify application protocol commands associated with
it and then permit or deny it as per that information. (What is Web Application Firewall and how does it work ? )

Medium
Packet Inspection was quite an advancement from Shallow Packet
Inspection. But, the problem with this technology is it is quite
poorly scalable, which limits its usefulness to a large extent.

Deep
Packet Inspection

Medium
Packet Inspection technology can look into the payload of the packets
only up to a certain extent. So, Medium Packet Inspection devices
have only limited application awareness. And, we needed something
more.

Deep
Packet Inspection technology evolved for that purpose. It looks into
the payload of the packets and can identify the origin and content of
each packet to take further decisions.

Deep
Packet Inspection devices use expressions to define patterns of
interest in network data streams. It can handle the packets based on
specific patterns present in the payload of the packets.

So,
a Deep Packet Inspection Device can look into payload of all the data
packets passed through it in real time. So that would mean, a Deep
Packet Inspection Device can look inside all the traffic from a
specific IP address, pick out all the HTTP traffic, capture all the
traffic that are meant for or coming from a specific mail server and
reassemble those emails when a user types out.

Applications
of Deep Packet Inspection technology

Deep
Packet Inspection technology has several applications. Some
major applications are listed below :

Network
Security

It
is widely used to Next Generation Firewalls ( What
is Next Generation Firewalls ? ) to monitor
and filter traffic per application basis instead of port basis, which
enables it to troubleshoot network problems in a better way.

Anti-malware

A
Deep Packet Inspection device can detect and filter a wide range of
malware including trojans, viruses, spyware, adware and other
malicious applications. It can do that by mainly taking a couple of
approach mentioned below :

URL
Detection
– Deep Packet Inspection devices can compare incoming and embedded
URLs against a database of that of known malicious websites.

Object
Detection
– Deep Packet Inspection devices can look into the traffic to
search for potentially harmful executables and objects and then,
analyze them to detect malware.

Signature
Detection
– Deep Packet Inspection devices can look into the payload of data
packets to search for the presence of signatures of known malware.
Signature matching is done using a database of known signatures of
malware and it usually takes help of security service providers to
update the signature database.

URL
Filtering

Deep
Packet Inspection devices can look into the traffic to search for
requested URLs and block URLs which are potentially harmful or
inappropriate.

Protocols
and Application Recognition

Deep
Packet Inspection technology can look into the traffic to distinguish
between email services including IMAP, POP3 and SMTP. It can identify
protocols like HTTP, FTP, TCP etc. It can also look into the payload
of data traffic to see the presence of certain file types like Flash,
Youtube, Windows Media etc. It can identify a wide variety of
tunneling, session, peer-to-peer, messaging and VoIP protocols so
that it can route the data for further processing.

Network
Management

Deep
Packet Inspection technology can be used to maintain QoS (Quality of
Service) for the end users. It can be used to differentiate between
different types of traffic and to prioritize or throttle down those
different types of traffic to maintain basic QoS.

Billing
and Metering of Traffic

Deep
Packet Inspection technology can be used by the Internet Service
Providers to offer subscribers different levels of access like usage,
data limits, bandwidth etc. It can also be used for the purpose of
compliance of certain regulations of traffic, prioritization of
traffic and load balancing.

Subscriber
Analysis

Sometimes
Deep Packet Inspection technology is used by the Internet Service
Providers to gather statistical information of their subscribers. For
example, the ISPs can gather information on web browsing habits of
their subscribers and later, to use that to enhance marketing
revenues.

Application
Distribution and Load Balancing

Deep
Packet Inspection technology can be used to look into the packet
content and then, to redirect them to different destinations for the
purpose of load balancing and fault tolerance.

Content
Regulations

Deep
Packet Inspection technology can be used to examine the traffic and
to block content that are potentially harmful or unlawful.

Copyright
Enforcement

Deep
Packet Inspection technology can be used to look into the packet
content and automatically detect and block unauthorized sharing of
copyrighted contents including music or video files.

So,
Deep Packet Inspection is quite an advancement of technology and it
is completely upto us to decide on how we are going to use it for our
own benefits. This article was intended to give basic infomation on
Deep Packet Inspection. Hope it solved the purpose.