Revision as of 15:44, 30 March 2011

Home

Overview

ModSecurity is an Apache web server module that provides a web application firewall engine. The ModSecurity Rules Language engine is extrememly flexible and robust and has been referred to as the "Swiss Army Knife of web application firewalls." While this is certainly true, it doesn't do much implicitly on its own and requires rules to tell it what to do. In order to enable users to take full advantage of ModSecurity out of the box, we have developed the Core Rule Set (CRS) which provides critical protections against attacks across most every web architecture.

Unlike intrusion detection and prevention systems, which rely on signatures specific to known vulnerabilities, the CRS is based on generic rules which focus on attack payload identification in order to provide protection from zero day and unknown vulnerabilities often found in web applications, which are in most cases custom coded.

Detection Categories

In order to provide generic web applications protection, the Core Rules use the following techniques:

Protocol compliance:

HTTP request validation - This first line of protection ensures that all abnormal HTTP requests are detected. This line of defense eliminates a large number of automated and non targeted attacks as well as protects the web server itself.

Global constraints - Limiting the size and length of different HTTP protocol attributes, such as the number and length of parameters and the overall length of the request. Ensuring that these attributed are constrained can prevent many attacks including buffer overflow and parameter manipulation.

Generic Attack Detection - Detect application level attacks such as described in the OWASP top 10. These rules employ context based patterns match over normalized fields. Detected attacks include:

SQL injection and Blind SQL injection.

Cross Site Scripting (XSS).

OS Command Injection and remote command access.

File name injection.

ColdFusion, PHP and ASP injection.

E-Mail Injection

HTTP Response Splitting.

Universal PDF XSS.

Trojans & Backdoors Detection - Detection of attempts to access Trojans & backdoors already installed on the system. This feature is very important in a hosting environment when some of these backdoors may be uploaded in a legitimate way and used maliciously.

Other:

Error Detection - Prevent application error messages and code snippets from being sent to the user. This makes attacking the server much harder and is also a last line of defense if an attack passes through.

XML Protection – The Core Rule Set can be set to examine XML payload for most signatures.

Let's talk here

ModSecurity Communities

Further development of ModSecurity and the Core Rule Set occurs through mailing list discussions and occasional workshops, and suggestions for improvement are welcome. For more information, please contact us.

Bug Tracker

Demo

Installation

Quick Start

Core Rule Set Quick Setup

=============

To activate the rules for your web server installation:

1) The modsecurity_crs_10_config.conf includes management rules and directives
that can control important CRS functions. Pay attention to
the SecRuleEngine setting (On by default) and that the SecDefaultAction
directive is set to "pass". The 49 inbound blocking and 59 outbound blocking
rules files use the "block" action which
inherits this setting. The effectively means that you can toggle the
SecDefaultAction setting to decide if you would like to deny on an
anomaly scoring/correlation match.

Update the PARANOID_MODE variable setting if you want to become more
aggressive in your detection. Caution - this will cause more false positives.

Should also update the appropriate anomaly scoring levels that will be propagated
to the inbound/outbound blocking files.

The ModSecurity Core Rule Set is provided to you under the terms and
conditions of GPL version 2

This directory contains the files for Core ModSecurity Rule Set
The rules are compatible with ModSecurity 2.5 (as of version 1.4.3)

Using ModSecurity requires rules. In order to enable users to take full
advantage of ModSecurity immediately, Trustwave is providing a free
Core rule set. Unlike intrusion detection and prevention systems which
rely on signature specific to known vulnerabilities, the Core Rule Set
provides generic protection from unknown vulnerabilities often found in web
application that are in most cases custom coded. This is what we call "Attack
Payload Detection."

CRS < 2.0 - Self-Contained Rules

- If a rule triggered, it would either deny or pass and log
- No intelligence was shared between rules

Not optimal from a rules management perspective (handling false positives/exceptions)

- Editing the regex could blow it up
- Typical method was to copy/paste rules into custom rules files and then edit rule logic
and disable core rule ID.
- Heavily customized rules were less likely to be updated by the user

Not optimal from a security perspective

- Not every site had the same risk tolerance
- Lower severity alerts were largely ignored
- Individual low severity alerts are not important but several low severity events
in the same transaction are.

Rules - Detection and Management

Rules logic has changed by decoupling the inspection/detection from the blocking functionality

- Rules log.pass and set transactional variables (tx) to track anomaly scores and to
store meta-data about the rule match
- This TX rule match data can be used by other 3rd party rules (converter Emerging Threats
Snort web attack rules) to more accurately correlate identified attacks with their
attack vector locations.
- TX data of previous strong rule matches can also be used to conditionally apply weaker signatures
that normally would have a high fasle positive rate.
- Rules also increase anomaly scores for both the attack category and global score which allows
users to set a threshold that is appropriate for them.
- This also allows several low severity events to trigger alerts while individual ones are suppressed.
- Exceptions may be handled by either increasing the overall anomaly score threshold, or
by adding rules to a local custom exceptions file where TX data of previous rule matches
may be inspected and anomaly scores re-adjusted based on the false positive criteria.

User can now globally update which variables to inspect and the anomaly score settings in the
modsecurity_crs_10_config.conf file.

- PARANOID_MODE setting which will apply rules to locations that have a higher false positive rate
- INBOUND_ANOMALY_SCORE setting will be populated in the inbound blocking file and if a transaction
score at the end of phase:2 is equal to or greater than this number, it will be denied.
- OUTBOUND_ANOMALY_SCORE setting will be populated in the outbound blocking file and it a transaction
score at the end of phase:4 is equal to or greater than this number, it will be denied.

The CRS rules themselves are configured with the pass action, which allows all the rules to be processed
and for the proposed anomaly scoring/collaborative detection concept to work. The inbound/outbound anomaly
score levels may be set in the modsecurity_crs_10_config.conf file. These scores will be evaluated in the
modsecurity_crs_49_inbound_blocking.conf and modsecurity_crs_59_outbound_blocking.conf files.

One of the top feedback items we have heard is that the CRS events in the Apache error_log file
were very chatty. This was due to each rule triggering its own error_log entry. What most people
wanted was for 1 correlated event to be generated that would give the user a higher level
determination as to what the event category was.

To that end- each CRS rule will generate an audit log event Message entry but they will not log
to the error_log on their own. These rules are now considered basic or reference events and
may be reviewed in the audit log if the user wants to see what individual events contributed
to the overall anomaly score and event designation.

After the transaction has completed (in the logging phase), the rules in the
base_rules/modsecurity_crs_60_correlation.conf file will conduct further post-processing by
analyzing any inbound events with any outbound events in order to provide a more
intelligent/priority correlated event.

- Was there an inbound attack?
- Was there an HTTP Status Code Error (4xx/5xx level)?
- Was there an application information leak?

If an inbound attack was detected
and either an outbound application status code error or infolead was detected, then the overall
event severity is raised -

- 0: Emergency - is generated from correlation where there is an inbound attack and
an outbound leakage.
- 1: Alert - is generated from correlation where there is an inbound attack and an
outbound application level error.

In order to provide generic web applications protection, the Core Rule Set
uses the following techniques:

HTTP Protocol Validation and Protection

Detecting violations of the HTTP protocol and a locally
defined usage policy. This first line of protection ensures that all abnormal HTTP
requests are detected. This line of defense eliminates a large number of
automated and non targeted attacks as well as protects the web server itself.

Automation Detection

Automated clients are both a security risk and a
commercial risk. Automated crawlers collect information from your site, consume
bandwidth and might also search for vulnerabilities on the web site. Automation
detection is especially useful for generic detection of comments spam.

Detecting bots, crawlers, scanners and other surface malicious activity.
Not aimed against targeted attacks, but against general malicious internet activity

optional_rules/modsecurity_crs_42_comment_spam.conf

This rules file is only relevant if you are concerned about comment SPAM attacks.
The rules file will run an RBL check against the source IP address at SPAMHAUS and will
cache the response for 1 day. If the client sends subsequent requests, it will be denied
without having to re-run an RBL check.

This file will also look for comment SPAM posting attacks which submit URL links.

Common Web Attacks Protection

Common Web Attacks Protection Rules on the second level address the common web
application security attack methods. These are the issues that can appear in
any web application. Some of the issues addressed are:

base_rules/modsecurity_crs_40_generic_attacks.conf

optional_rules/modsecurity_crs_40_experimental.conf

The rules in this file are considered BETA quality as they have not been rigorously tested.
They attempt to address advanced attacks such as HTTP Parameter Pollution or use new rule
features or techniques.

base_rules/modsecurity_crs_42_tight_security.conf

This rules file attempts to identify all directory traversal variations. It is prone to a high
level of false positives so set PARANOID_MODE if you want to run these rules.

base_rules/modsecurity_crs_41_sql_injection.conf

- SQL injection and blind SQL injection

base_rules/modsecurity_crs_41_xss.conf

- Cross site scripting (XSS)

base_rules/modsecurity_crs_41_phpids_converter.conf

base_rules/modsecurity_crs_41_phpids_filters.conf

SpiderLabs received authorization from PHPIDS (http://phpids.net/) to convert their
rules and include them in the CRS

The issue to overcome is that the PCRE RegExs used in the rules are pretty poor. What we want
to do is to combine the *what* of our generic attack payload detection (attack payloads) with
the *where* (attack vector - URL + Parameter Name) of the ET known vuln data. The approach we
took was to have most of the ET rules look for the attack vector data and then simply check all
saved TX data for a corresponding attack vector match.

Trojan Protection

ModSecurity Core Rule Set detects access to back doors
installed on a web server. This feature is very important in a hosting
environment when some of this backdoors may be uploaded in a legitimate way and
used maliciously. In addition the Core Rule Set includes a hook for adding
an Anti-Virus program such as ClamAV for checking file uploads.

base_rules/modsecurity_crs_45_trojans.conf

InfoLeakages

If all fails, the Core Rule Set will detect errors sent by
the web server. Detecting and blocking errors prevents attackers from
collecting reconnaissance information about the web application and also server
as a last line of defense in case an attack was not detected eariler.

base_rules/modsecurity_crs_50_outbound.conf

Request Header Tagging

This concept is similar to anti-SPAM SMTP apps that will add additional mime headers
to emails providing the SPAM detection analysis information. The CRS is attempting
to mimic this concept at the HTTP layer by adding additional request headers that
provide insight into any ModSecurity events that may have triggered during processing.
The advantage of this approach is that it allows a WAF to be in a detection-only mode
while still providing attack data to the destination application server. The recieving
app server may then inspect the WAF request headers and make a determination whether
or not to process the transaction. This concept is valuable in distributed web environments
and hosting architectures where a determination to block may only be appropriate at the
destination app server.

optional_rules/modsecurity_crs_49_header_tagging.conf

This rules file will take all of the TX attack variable data and populate Apache ENV
variables that Apache can then use to add X-WAF-Event request header data to the
request.

- Updated converted PHPIDS signatures (https://svn.php-ids.org/svn/trunk/lib/IDS/default_filter.xml)
- Updated PHPIDS rules logic to first search for payloads in ARGS and then if there is no match found
then search more generically in request_body|request_uri_raw
- Updated PHPIDS rules logic to only set TX variables and to not log. This allows for more clean
exceptions in the 48 file which can then expire/delete false positive TX matches and adjust the
anomaly scores. These rules will then inspect for any TX variables in phase:5 and create appropriate
alerts for any variable matches that exist.

Bug Fixes:

- Added Anomaly Score check to the 60 correlation file to recheck the anomaly score at the end of
phase:4 which would allow for blocking based on information leakage issues.

Project About

PROJECT INFOWhat does this OWASP project offer you?

RELEASE(S) INFOWhat releases are available for this project?

what

is this project?

Name: OWASP ModSecurity Core Rule Set Project (home page)

Purpose: ModSecurity is an Apache web server module that provides a web application firewall engine. The ModSecurity Rules Language engine is extrememly flexible and robust and has been referred to as the "Swiss Army Knife of web application firewalls." While this is certainly true, it doesn't do much implicitly on its own and requires rules to tell it what to do. In order to enable users to take full advantage of ModSecurity out of the box, we have developed the Core Rule Set (CRS) which provides critical protections against attacks across most every web architecture.

Unlike intrusion detection and prevention systems, which rely on signatures specific to known vulnerabilities, the CRS is based on generic rules which focus on attack payload identification in order to provide protection from zero day and unknown vulnerabilities often found in web applications, which are in most cases custom coded.

Release description: ModSecurity is a web application firewall that can work either embedded or as a reverse proxy. It provides protection from a range of attacks against web applications and allows for HTTP traffic monitoring, logging and real-time analysis.