Detecting Insider Threats with Elastic @ RedOwl

Elastic is a fundamental part of our platform at RedOwl, so when they brought the Elastic{ON} Tour to New York we were excited to attend and share with attendees how we make use of the Elastic Stack to help our clients better understand their Insider Risk threats.

For me personally it was a great opportunity to learn about the progress towards GA for Elastic Stack 5.0 and understand the thinking behind the Elastic Stack and X-Pack reorganization. Given our background in analytics, the introduction to Prelert was very interesting and started a number of conversations internally about how we might be able to leverage this new functionality in the future.

During the breaks we took the opportunity to meet with some of the Elastic engineers on site, as well as spend some time at the Ask Me Anything booth chatting with other attendees about our platform. In particular, it was great to discuss more of the details of Prelert with Gaurav Gupta, VP of Product at Elastic, as we're very intrigued by the approach it takes of putting analytics functionality into a new node type within the Elastic cluster. We also networked with a number of attendees on topics ranging from how the Elastic ecosystem allows our solution to scale within extremely large enterprises to the applicability of our behavioral analysis to the performance of athletes and sports teams!

When Elastic asked us to present at this year's event, we were very eager to take part. We're huge supporters of their products, and being able to help others see the possibilities beyond simple search was an exciting opportunity. The whole process was very straightforward – from slide templates through to presenting on the day, and we were happy to see both a full house and some great questions!

For our presentation we wanted to cover a few bases. First, I spoke about the problem space that RedOwl operates within, and how our application's ability to combine intelligence from multiple streams of unstructured data sets it apart both in functionality and in implementation challenges. I also explained how Elasticsearch fits into our overall stack alongside Apache Nifi, RabbitMQ and other supporting technologies. In the second part, Russell Snyder, Principal Engineer at RedOwl, gave a brief history of how and why we moved from Hadoop to Elasticsearch, and explained some of the complex queries and aggregations we use to provide insight to our users. I hope you find it interesting!

Adam Reeve serves as RedOwl's Principal Architect where he is responsible for the security, performance, scalability, and resilience of the platform. Adam has over 20 years of professional software engineering experience, most recently working on highly scalable service architectures serving billions of requests per day at Tumblr. Prior to that, he spent a number of years working in the finance sector.