Role in IT decision-making process:Align Business & IT GoalsCreate IT StrategyDetermine IT NeedsManage Vendor RelationshipsEvaluate/Specify Brands or VendorsOther RoleAuthorize PurchasesNot Involved

Work Phone:

Company:

Company Size:

Industry:

Street Address

City:

Zip/postal code

State/Province:

Country:

Occasionally, we send subscribers special offers from select partners. Would you like to receive these special partner offers via e-mail?YesNo

Your registration with Eweek will include the following free email newsletter(s):News & Views

By submitting your wireless number, you agree that eWEEK, its related properties, and vendor partners providing content you view may contact you using contact center technology. Your consent is not required to view content or use site features.

By clicking on the "Register" button below, I agree that I have carefully read the Terms of Service and the Privacy Policy and I agree to be legally bound by all such terms.

WEBINAR:On-Demand

A new open-source intrusion detection system (IDS) effort is officially getting underway on Nov. 5 with the launch of the OPNids project.

The OPNids effort is being led by threat hunting firm CounterFlow AI and security appliance provider Deciso, which also leads the Opensense security platform project. OPNids is built on top of the open-source Suricata IDS, providing a new layer of machine learning-based intelligence to help improve incident response and threat hunting activities.

"We created a pipeline that will actually take the Suricata logs and analyze the packets to provide context around any alerts," Randy Caldejon, CEO and co-founder of CounterFlow AI, told eWEEK. "We like to call this alert triage. It's like taking it to the last mile of what the analysts would do anyhow because typically when there's an alert, they want some context."

Further reading

The Suricata project got started in 2009 by the Open Information Security Foundation as an alternative open-source option to the Snort IDS that was already in market.

"What we're doing with the machine learning is taking advantage of the data that is flowing at line rates from Suricata and rather than just storing the data to disk we figured we should analyze the data while it's in motion," he said. "What we have is not one single model; we built an agent so analysts can write their own scripts."

How It Works

At the core of OPNids is the DragonFly Machine Learning Engine (MLE), which uses a streaming data analytics model to ingest line rate network data from Suricata.

"Most machine learning techniques are what are traditionally known as batch techniques, where you get a big pool of data offline and you apply a sophisticated algorithm onto that," Andrew Fast, chief data scientist at CounterFlow AI, told eWEEK. "We are taking a different approach, with streaming analytics, which is a newer branch of machine learning that is not as widely used."

Fast explained that with streaming analytics rather than looking at a collected pool of data, the DragonFly MLE is collecting statistics and making decisions as the data flows through the IDS. The DragonFly MLE can also support offline training and then online scoring at wire speed, he said. With machine learning, training is used to help tune a model, while scoring is the output of the model.

Caldejon added that OPNids can also enable post-processing of data for additional analysis, with an approach known as Filebeat, which is commonly used in the Elastic Stack. As such, OPNids data can be forwarded to an Elastic search engine for additional analysis. Apache Kafka streaming is also supported, as is syslog for general log capabilities.

Moving forward, Caldejon said the plan is to also integrate OPNids with threat intelligence gateway capabilities, taking in third-party threat data as a factor that helps the MLE make decisions.

OPNids Pro

Alongside the open-source effort, there is a commercially supported version of OPNids in the works. The commercial version has a hardware edition where software is all preloaded and integrated. Additionally, Caldejon said the Pro edition of OPNids will include a packet caching capability. The packet caching capability will take network packets and write them to disk, saving the PCAP (packet capture) for additional offline analysis, he said.

Both the Pro and standard editions of OPNids are fully bundled application images including the MLE and Suricata. Caldejon said users can download and get the system running within 10 minutes.

"It's got Suricata, it's got the MLE, and it has a nice UI [user interface], so even unsophisticated users can manage the technology," he said. "Typically, a lot of open-source projects are all CLI [command line interface] and expect people to be experts at the command line, but we wanted to make this accessible to data scientists as well, and they usually don't have those skills, so yeah, OPNids is a one-stop shop, everything is bundled."

Sean Michael Kerner is a senior editor at eWEEK and InternetNews.com. Follow him on Twitter @TechJournalist.

Advertiser Disclosure:
Some of the products that appear on this site are from companies from which QuinStreet receives compensation. This compensation may impact how and where products appear on this site including, for example, the order in which they appear. QuinStreet does not include all companies or all types of products available in the marketplace.