Monday, February 13, 2017

Chris Sanders started a poll on Twitter asking "Would you rather get a real-time alert with partial context immediately, or a full context alert delayed by 30 mins?" I answered by saying I would prefer full context delayed by 30 minutes. I also replied with the text at left, from my first book The Tao of Network Security Monitoring (2004). It's titled "Real Time Isn't Always the Best Time."

Dustin Webber then asked "if you have [indicators of compromise] IOC that merit 'real-time' notification then you should be in the business of prevention. Right?"

Long ago I decided to not have extended conversations over Twitter, as well as to not try to compress complex thoughts into 140 characters -- hence this post!

To Dustin's point, I agree that if it is possible to generate a match (or "alert," etc.) with 100% accuracy (or possibly near 100%, depending on the severity of the problematic event), i.e., with no chance or almost no chance of a false positive, then it is certainly worth seeking a preventive action for that problematic event. To use a phrase from the last decade, "if you can detect it, why can't you prevent it?"

However, there are likely cases where zero- or low-false positive events do not have corresponding preventive actions. Two come to mind.

First, although you can reliably detect a problem, you may not be able to do anything about it. The security team may lack the authority, or technical capability, to implement a preventive action.

Second, although you can reliably detect a problem, you may not want to do anything about it. The security team may desire to instead watch an intruder until such time that containment or incident mitigation is required.

Sunday, February 12, 2017

Yesterday my friend Bamm Visscher published a series of Tweets on detection. I thought readers might like to digest it as a lightly edited blog post. Here, then, is the first ever (as far as I can remember) guest post on TaoSecurity Blog. Enjoy.

When you receive new [threat] intel and apply it in your detection environment, keep in mind all three analysis opportunities: RealTime, Batch, and Hunting.

If analysts can quickly and accurately process events generated by the [RealTime] signature, it's a good sign the indicator should be part of RealTime detection. If an analyst struggles to determine if a [RealTime alert] has detected malicious activity, it's likely NOT appropriate for RealTime detection.

Hunting is the soft science of detection, and best done with a team of diverse skills. Intelligence, content development, and detection should all work together. Don't fear getting skunked on your hunting trips. Keep investing time. The rewards are accumulative. Be sure to pass Hunting rewards into Batch Analysis and RealTime detection operations in the form of improved context.

The biggest mistake organizations make is not placing emphasis outside of RealTime detection, and "shoe-horning" [threat] intelligence into RealTime operations. So called "Atomic Indicators" tend to be the biggest violator of shoe-horning. Atomic indicators are easy to script into signature driven detection devices, but leave an analyst wondering what he is looking at and for.

Do not underestimate the NEGATIVE impact of GOOD [threat] intelligence inappropriately placed into RealTime operations! Mountains of indiscernible events will lead to analyst fatigue and increase the risk of good analyst missing a real incident.

Thursday, February 09, 2017

A reader asked me to explain the differences between two of my books. I decided to write a public response.

If you visit the TaoSecurity Books page, you will see two different types of books. The first type involves books which list me as author or co-author. The second involves books to which I have contributed a chapter, section, or foreword.

This post will only discuss books which list me as author or co-author.

In July 2004 I published The Tao of Network Security Monitoring: Beyond Intrusion Detection. This book was the result of everything I had learned since 1997-98 regarding detecting and responding to intruders, primarily using network-centric means. It is the most complete examination of NSM philosophy available. I am particularly happy with the NSM history appendix. It cites and summarizes influential computer security papers over the four decade history of NSM to that point.

The main problem with the Tao is that certain details of specific software versions are very outdated. Established software like Tcpdump, Argus, and Sguil function much the same way, and the core NSM data types remain timeless. You would not be able to use the Bro chapter with modern Bro versions, for example. Still, I recommend anyone serious about NSM read the Tao.

The introduction describes the Tao using these words:

Part I offers an introduction to Network Security Monitoring, an operational framework for the collection, analysis, and escalation of indications and warnings (I&W) to detect and respond to intrusions. Part I begins with an analysis of the terms and theory held by NSM practitioners. The first chapter discusses the security process and defines words like security, risk, and threat. It also makes assumptions about the intruder and his prey that set the stage for NSM operations. The second chapter addresses NSM directly, explaining why NSM is not implemented by modern NIDS' alone. The third chapter focuses on deployment considerations, such as how to access traffic using hubs, taps, SPAN ports, or inline devices. Part II begins an exploration of the NSM “product, process, people” triad. Chapter 4 is a case study called the “reference intrusion model.” This is an incident explained from the point of view of an omniscient observer. During this intrusion, the victim collected full content data in two locations. We will use those two trace files while explaining the tools discussed in Part II. Following the reference intrusion model, I devote chapters to each of the four types of data which must be collected to perform network security monitoring – full content, session, statistical, and alert data. Each chapter describes open source tools tested on the FreeBSD operating system and available on other UNIX derivatives. Part II also includes a look at tools to manipulate and modify traffic. Featured in Part II are little-discussed NIDS' like Bro and Prelude, and the first true open source NSM suite, Sguil.Part III continues the NSM triad by discussing processes. If analysts don’t know how to handle events, they’re likely to ignore them. I provide best practices in one chapter, and follow with a second chapter explicitly for technical managers. That material explains how to conduct emergency NSM in an incident response scenario, how to evaluate monitoring vendors, and how to deploy a NSM architecture.Part IV is intended for analysts and their supervisors. Entry level and intermediate analysts frequently wonder how to move to the next level of their profession. I offer some guidance for the five topics with which a security professional should be proficient: weapons and tactics, telecommunications, system administration, scripting and programming, and management and policy. The next three chapters offer case studies, showing analysts how to apply NSM principles to intrusions and related scenarios.Part V is the offensive counterpart to the defensive aspects of Parts II, III, and IV. I discuss how to attack products, processes, and people. The first chapter examines tools to generate arbitrary packets, manipulate traffic, conduct reconnaissance, and exploit flaws inn Cisco, Solaris, and Microsoft targets. In a second chapter I rely on my experience performing detection and response to show how intruders attack the mindset and procedures upon which analysts rely.An epilogue on the future of NSM follows Part V. The appendices feature several TCP/IP protocol header charts and explanations. I also wrote an intellectual history of network security, with abstracts of some of the most important papers written during the last twenty-five years. Please take the time to at least skim this appendix, You'll see that many of the “revolutionary ideas” heralded in the press were in some cases proposed decades ago.

The Tao lists as 832 pages. I planned to write 10 more chapters, but my publisher and I realized that we needed to get the Tao out the door. ("Real artists ship.") I wanted to address ways to watch traffic leaving the enterprise in order to identify intruders, rather than concentrating on inbound traffic, as was popular in the 1990s and 2000s. Therefore, I wrote Extrusion Detection: Security Monitoring for Internal Intrusions.

I've called the Tao "the Constitution" and Extrusion "the Bill of Rights." These two books were written in 2004-2005, so they are tightly coupled in terms of language and methodology. Because Extrusion is tied more closely with data types, and less with specific software, I think it has aged better in this respect.

The introduction describes Extrusion using these words:

Part I mixes theory with architectural considerations. Chapter 1 is a recap of the major theories, tools, and techniques from The Tao. It is important for readers to understand that NSM has a specific technical meaning and that NSM is not the same process as intrusion detection. Chapter 2 describes the architectural requirements for designing a network best suited to control, detect, and respond to intrusions. Because this chapter is not written with specific tools in mind, security architects can implement their desired solutions regardless of the remainder of the book. Chapter 3 explains the theory of extrusion detection and sets the stage for the remainder of the book. Chapter 4 describes how to gain visibility to internal traffic. Part I concludes with Chapter 5, original material by Ken Meyers explaining how internal network design can enhance the control and detection of internal threats.Part II is aimed at security analysts and operators; it is traffic-oriented and requires basic understanding of TCP/IP and packet analysis. Chapter 6 offers a method of dissecting session and full content data to unearth unauthorized activity. Chapter 7 offers guidance on responding to intrusions, from a network-centric perspective. Chapter 8 concludes part III by demonstrating principles of network forensics.Part III collects case studies of interest to all types of security professionals. Chapter 9 applies the lessons of Chapter 6 and explains how an internal bot net was discovered using Traffic Threat Assessment. Chapter 10 features analysis of IRC bot nets, contributed by LURHQ analyst Michael Heiser. An epilogue points to future developments. The first appendix, Appendix A, describes how to install Argus and NetFlow collection tools to capture session data. Appendix B explains how to install a minimal Snort deployment in an emergency. Appendix C, by Tenable Network Security founder Ron Gula, examines the variety of host and vulnerability enumeration techniques available in commercial and open source tools. The book concludes with Appendix D, where Red Cliff Consulting expert Rohyt Belani offers guidance on internal host enumeration using open source tools.

At the same time I was writing Tao and Extrusion, I was collaborating with my friends and colleagues Keith Jones and Curtis Rose on a third book, Real Digital Forensics: Computer Security and Incident Response. This was a ground-breaking effort, published in October 2005. What made this book so interesting is that Keith, Curtis and I created workstations running live software, compromised each one, and then provided forensic evidence for readers on a companion DVD.

This had never been done in book form, and after surviving the process we understood why! The legal issues alone were enough to almost make us abandon the effort. Microsoft did not want us to "distribute" a forensic image of a Windows system, so we had to zero out key Windows binaries to satisfy their lawyers.

The primary weakness of the book in 2017 is that operating systems have evolved, and many more forensics books have been written. It continues to be an interesting exercise to examine the forensic practices advocated by the book to see how they apply to more modern situations.

After those three titles, I was done with writing for a while. However, in 2012 I taught a class for Black Hat in Abu Dhabi. I realized many of the students lacked the fundamental understanding of how networks operated and how network security monitoring could help them detect and respond to intrusions. I decided to write a book that would explain NSM from the ground up. While I assumed the reader would have familiarity with computing and some security concepts, I did not try to write the book for existing security experts.

The main drawback of the book is the integration of Security Onion coverage. SO is a wonderful open source suite, partly because it is kept so current. That makes it difficult for a print book to track changes in the software installation and configuration options. While you can still use PNSM to install and use SO, you are better off relying on Doug Burks' excellent online documentation.

PNSM is an awesome resource for learning how to use SO and other tools to detect and respond to intrusions. I am particularly pleased with chapter 9, on NSM operations. It is a joke that I often tell people to "read chapter 9" when anyone asks me about CIRTs.

The introduction describes PNSM using these words:

Part I, “Getting Started,” introduces NSM and how to think about sensor placement.