This rule will be evaluated against all inbound logs that match the cloudwatch:events schema defined in conf/logs.json.
In this case, all CloudWatch events will generate an alert, which will be sent to the alerts Athena table.

Let’s modify the rule to page the security team if anyone ever uses AWS root credentials:

fromstream_alert.shared.ruleimportrule@rule(logs=['cloudwatch:events'],outputs=['pagerduty:csirt','slack:security'])defcloudtrail_root_account_usage(record):"""Page security team for any usage of AWS root account"""return(record['detail']['userIdentity']['type']=='Root'andrecord['detail']['userIdentity'].get('invokedBy')isNoneandrecord['detail']['eventType']!='AwsServiceEvent')

Now, any AWS root account usage is reported to PagerDuty, Slack, and the aforementioned Athena table.
In order for this to work, your datasources and outputs must be configured so that:

CloudTrail logs are being sent to StreamAlert via CloudWatch events

The pagerduty:csirt and slack:security outputs have the proper credentials

A single legitimate root login may generate hundreds of alerts in the span of a few minutes

There is no distinction between different AWS account IDs

We can generalize the rule to alleviate these issues:

fromhelpers.baseimportget_first_key# Find first key recursively in recordfromstream_alert.shared.ruleimportmatcher,rule# This could alternatively be defined in matchers/matchers.py to be more shareable_PROD_ACCOUNTS={'111111111111','222222222222'}@matcherdefprod_account(record):"""Match logs for one of the production AWS accounts"""return(rec.get('account')in_PROD_ACCOUNTSorget_first_key(rec,'userIdentity',{}).get('accountId')in_PROD_ACCOUNTS)@rule(logs=['cloudtrail:events','cloudwatch:events'],# Rule applies to these 2 schemasmatchers=['prod_account'],# Must be satisfied before rule is evaluatedmerge_by_keys=['useragent'],# Merge alerts with the same 'useragent' key-value pairmerge_window_mins=5,# Merge alerts every 5 minutesoutputs=['pagerduty:csirt','slack:security']# Send alerts to these 2 outputs)defcloudtrail_root_account_usage(record):"""Page security team for any usage of AWS root account"""return(get_first_key(record,'userIdentity',{}).get('type')=='Root'andnotget_first_key(record,'invokedBy')andget_first_key(record,'eventType')!='AwsServiceEvent')

To simplify rule logic, you can extract common routines into custom helper methods.
These helpers are defined in helpers/base.py and can be called from within a matcher or rule (as shown here).

Since rules are written in Python, you can make them as sophisticated as you want!

context can pass extra instructions to the alert processor for more precise routing:

# Context provided to the pagerduty-incident output with# instructions to assign the incident to a user.@rule(logs=['osquery:differential'],outputs=['pagerduty:csirt'],context={'pagerduty-incident':{'assigned_user':'valid_user'}})defmy_rule(record,context):context['pagerduty-incident']['assigned_user']=record['username']returnTrue

A single consolidated alert will be sent showing the common keys and the record differences.
All of the specified merge keys must have the same value in order for two records to be merged,
but those keys can be nested anywhere in the record structure.

req_subkeys defines sub-keys that must exist in the incoming record (with a non-zero value) in order for it to be evaluated.

This feature should be used if you have logs with a loose schema defined in order to avoid raising a KeyError in rules.

# The 'columns' key must contain sub-keys of 'address' and 'hostnames'@rule(logs=['osquery:differential'],outputs=['aws-lambda:my-function'],req_subkeys={'columns':['address','hostnames']})defosquery_host_check(rec):# If all logs did not have the 'address' sub-key, this rule would# throw a KeyError. Using req_subkeys avoids this.returnrec['columns']['address']=='127.0.0.1'