Sonatype Blog

NSA & Open Source: Another Controversy Brewing?

I attended the NSA Open Source Industry Day in Maryland and thought I’d summarize what did and didn't surprise me. We'll see if these observations prove controversial or helpful! More importantly we'll see if organizations can effectively manage, govern, and secure their applications given the reality of open source, agile development practices and component-based development.

What wasn’t surprising?

Organizations are looking for an open source savior. They are looking for someone or something to hold accountable. They want a “single throat to choke”. They are conditioned to think this way based on the historical support they receive from ISVs that license proprietary software. And now, they turn to vendors like Red Hat for open source infrastructure software support. But that model doesn’t work for the hundreds or thousands of open source components that are sourced from different projects or forges. A new model is needed.

Open source = Linux, MySQL, JBoss. When people think open source, they think large infrastructure software – they don’t think of the open source components that are used to build applications. Many are not aware or planning for the fact that 80% of an application is comprised of open source components. Until they accept this reality, it will be difficult for them to implement the processes necessary to secure component-based applications.

Disagreement about open source contributions. While people agreed that classified or secret system work shouldn’t be shared, there was disagreement about non-sensitive work. And there were questions about the value of sharing this work. This led to a discussion about how custom change leads to forked internal versions that are difficult to maintain and the change control issues associated with leveraging new open source versions. Here is an interesting article about open source in government misconceptions.

Disagreement about whether access to code gave hackers an advantage. Lots of discussion about how “many eyes make for safer code”, and interesting conversation about whether access to code gave hackers an advantage. Some thought so, but others argued that it doesn’t matter because hackers can simply reverse engineer what they need from the binary.

Organizations can’t keep up with the latest versions of software. Many panelists and attendees lamented their ability to keep their software current – even with large infrastructure software. When thinking about this in the context of components, the problem is even more daunting.

What was surprising?

People still buy into the golden repository fallacy. Perhaps this shouldn’t be surprising, but the knee-jerk reaction to many open source challenges was “we just need to ensure that developers use the current version”. Or “we should make sure that only the most current version is available for download from the Internet”. Although it’s great to have a repository to provide some control (we do provide the Nexus Repository Manager by the way!), we know that there is no such thing as a golden repository. There are many problems with this approach: Developers will bypass it; it will be too restrictive for initial development efforts, etc. Sure a repository is a start, but you need guidance and governance throughout the software lifecycle, including support for the production environment.

People still think an approval process is the way to manage OSS. One organization spoke about how they have an open source review board. Told from the perspective of a CISO - here is how it goes… warning, it’s painful!!

The owner of the OSS component (the person that wants to use it) - they write up an in-depth review, including the business value… It goes to legal and we make sure we don't have a legacy issue, or a contract issue. Once legal is done, we go to review board – and depending on the timing of the project - they prioritize it with other items in the queue. It then goes through a security review - what functionality does it provide, what else will it touch? We have to understand the baseline - I have to know my enterprise and what my network looks like today. We factor in privacy requirements, transactional requirements. Do we do these securely? Are there protocols being used? Once that is done, we go deeper into the code using automated tools. At the end of the full review - legal, technical, security - we make a risk management decision. Do we want it in our enterprise? It's give and take with the developer - we like this, we need this changed…

I’m not arguing that these aren’t important considerations. I’m just saying that this approach doesn’t work based on the volume, variety, complexity and release cadence of components – even if the workflow is automated. Think about this approach in an agile development environment that leverages short sprints! This is why developers resist policy efforts. Developers are forced to bypass the process to make their deadlines. Or they have to rely on components that have already been approved, even if they aren’t the best component for the job.

People equate app security to DAST / SAST. While these technologies certainly play a role, some organizations force fit these technologies into managing open source usage. But how many organizations really want to get involved with open source component code? They turn to components for “out of the box” functionality. They turn to components so they don’t have build it themselves. They don’t want to be involved in the code – they often don’t even have the code. And why wait to scan source code for components until they are used in the development process when information is already known about those components? Information that can be used to select optimal components at the beginning of the development process. While DAST/SAST have their place, component-based development can’t be managed the same way that we manage the custom code that stitches components together.

Key Realizations

Do you agree with these observations? Do you agree that we need to act? Based on the conversations took place at the conference, these common themes indicate solid progress on the open source front:

Supply chain management. Given that the application development process is supported by components sourced externally, using supply chain principles to manage the development lifecycle was a common theme. There is a realization that the entire software lifecycle needs to be supported, and that organizations are responsible for the application regardless of what constitutes the application. People understand how components can be used to drive developer efficiency, as long as security, licensing and quality risk can be managed.

Automation is key. Even though I didn’t hear DevOps uttered once, there was plenty of discussion around automating the development and delivery process. While this automation is typically focused on the application delivery toolchain (IDE, build tools, CI, CD, etc.), it can lead to conversations around policy automation. Automated policies can provide guidance and enforcement directly in the tools that are used to develop and delivery software applications.

Give developers what they want. There is a growing realization that security needs to provide developers what they want. The CISO understands that developers will bypass their controls if they hinder the development process. They realize that they have to overcome the “false positive” and “false negative” burden that is the result of many application security tools. And they are starting to realize that providing information to developers in their context requires integration into the tools that they use. Look, for example, at how Sonatype's CLM platform integrates open source security data and automate the policies directly in the IDE where developers work every day.

Modify security to work with how today’s apps are developed. Security and quality efforts must be aligned with modern development approaches – and that’s not just accommodating the use of open source. There were plenty of conversations about agile development, and how security and quality efforts have to fit within that construct. Among other things, it means that policy and compliance efforts must fit within short agile development cycles.

Organizations must take responsibility for code & components that are externally sourced. There is a growing realization that developers using open source must manage it effectively. But this responsibility should not fall on the developer alone; the entire organization needs to support the effort. And organizations must do so in a way that supports the development process naturally, not by force fitting legacy approaches. The answer can't be we just won't use open source – there is too much value that would be lost. Here is a video from Goldman that represents the value that open source provides.

Silos (of all kind) need to be removed. While DevOps leads the way by breaking down Development and IT Ops silos, it’s not just organization friction that stifles agility. In the keynote, Dr. Patrick Dowd, NSA CTO and Chief Architect, talked about how they are moving away from an enclave-based architecture because the “silos” made it difficult to analyze data that crossed enclaves. Instead, they have moved to an architecture that leverages open source technologies to deliver cloud-based utility, data, and storage support. Organizations realize they are more likely to meet agility and speed goals if they dismantle organization, technology and architecture silos.

If you attended the event, or if you work in the government sector, do you agree with these observations?

If you work in another industry, it might help to think about how this perspective relates to your industry.

Interesting points which I recognise from experiencing them myself. A couple of points though:

1) I think it's a bit disingenuous to post this with the headline "NSA & Open Source: ..." This posting was (in general) about the inability of organisations to manage and/or recognise their OSS utilisation. Whilst the conference in question was hosted by the NSA, I think you're piggy-backing on the recent controversy regarding the NSA (PRISM, Glen Greenwald, etc.).

2) Giving developers what they want isn't necessarily in line with what an organisation can accommodate. Take OSS licensing as an example; organisations may be developing a number of different products, some being distributed to end-customers, some for internal use. Depending on the target audience for the delivered product, different OSS licenses may or may not be acceptable to a organisation. Therefore an appropriate review needs to take place to confirm whether an OSS component meets the licensing requirement for the target audience (c.f. GPL vs. BSD licenses).

You state that "Organizations must take responsibility for code & components that are externally sourced." Yup ... agree with that whole heartily ... now how do we address that?

How specifically do organisations maintain their development velocity whilst having the control of their use of OSS such that they don't introduce security or licensing issues into their products prior to release.

Nexus Procurement might be part of the solution - but how does an organisation join that up when the developers I speak aren't even aware that there could be issues with the inclusion of OSS components in their builds.

Dg.

mtroester

Hello David -

Thanks for commenting on the blog post. I plead guilty to selecting a title that drafts off of the recent NSA news. And it may not be highly controversial, but there was disagreement about both the usage of open source, the ramifications of open source, and how best to manage open source in the context of government. My post touched on these conversations - clearly not everyone agreed on the these topics.

The "giving the developers" what they want was a direct quote from one of the panelists - the implication was not to give the developers what they want at all cost, or to give the developers what they want regardless of how that might expose an organization. The point being that if the guidance or governance mechanism is not developer friendly, then it will likely be ignored or bypassed. This is due to the fact that the developers are measured based on their ability to meet deadlines. There was no implication that an organization should allow a developer to use open source software with a license that exposes the organization, and certainly no implication that developers should be allowed to cut corners that would introduce security vulnerabilities. The point was that the "review" process needs to be done in a way that fits the development process. From a Sonatype perspective, we believe this means:

Providing developers the information they need up front directly in their IDE to pick the right components from the start. Components that have acceptable licenses, components that are free or in-line with organizational policies for security, etc. The right components are "defined" based on policies created by the legal team (for licensing) and by the security team (for security).

That guidance is provided across the development lifecycle and that guidance is integrated in the tools that developers use today, vs. forcing the developers to learn a new tool. So, our CLM solution is integrated into the IDE, into the repository manager and into the build/CI/CD environments.

That support needs to extend into the production environment - organizations need to be able to monitor and address new vulnerabilities or issues that are identified for components that constitute production applications.

So, our answer to how organizations maintain development velocity while mitigating the risk of OSS is our Component Lifecycle Management solution. Certainly Nexus plays a role - although procurement can be used to control what is being consumed from the repository, our experience shows that this is not enough - that support needs to be included throughout the lifecycle. We have extended Nexus with the ability to manage the staging and deployment process based on security, licensing and architecture policies. We refer to this option as Nexus Pro CLM Edition. Organizations can expand their component support with the full CLM, which provides support for Eclipse, as well as support for production applications.