Are You Protecting Your DevOps Software 'Factory'?

A new study out today shows that DevSecOps could stand to use a healthier dose of OpSec, as many DevOps tools are left exposed on the public Internet with little to no security controls.

So much of the education about the intersection of DevOps and security focuses on application security testing and secure development practices. But DevSecOps is about more than just securing the software product itself. It's also crucial to protect the "factory" that produces those applications — namely, the development infrastructure and DevOps toolchain.

Unfortunately, according to a new study conducted by researchers with IntSights, a statistically significant number of DevOps organizations are falling down on that second part of the equation.

"Given DevOps tools sit in the cloud, they are more vulnerable to reconnaissance by hackers," says Alon Arvatz, co-founder of IntSights. "As opposed to traditional IT tools and servers that are still protected by the company network, a misconfigured DevOps tool will expose its data directly to the Internet, meaning hackers don't need to use any special hacking tools, just simple scanning tools that are available online."

Arvatz and his team examined nearly 26,000 URLs of different DevOps tools and servers from a range of organizations and did a simple test by trying to connect to them through a browser.

"No fancy attack tools, or port scanning, or any preliminary data, except for using open OSINT [open source intelligence] tools and websites to create the list," the report explains.

They found that more than 23% of those tested were accessible from the Web, with a range of access levels.

"Some were totally exposed without any user/password combination, exposing company data, user lists, internal server names, etc.," the report explained. "Most were protected with a simple login page, and a minority with a more robust cloud access security broker."

The trouble is that even those tools and servers that did have nominal security controls in place still left enough breadcrumbs and openings to make it easier for attackers to gain entry. For example, many organizations used DevOps tool names for a Web-facing server — such as Jenkins, Kibana, Trello, Jira, and so on. Additionally, most tools don't have built-in multifactor authentication, leaving the security of the system up to a simple username and password combo.

Ideally, many standard technologies and practices of DevOps can be used advantageously for security purposes. For example, the use of infrastructure as code and automation of systems can provide a very efficient means for helping organizations consistently lock down their development and production application infrastructure.

"Because we have this infrastructure as code, we're getting a lot of reuse," explains Paula Thrasher, director of digital services for General Dynamics, a large federal IT integrator, who says that prior to a DevOps-oriented pipeline, her teams could expect 60% reuse of design patterns and infrastructure. Now that number is pushing 90%. "Which basically means 90% of the stuff in our production is a reusable standard, and not a special-snowflake bespoke server. That's huge, because it just takes down the attack surface."

Of course, on the flip side, that means that one mistake is amplified across an entire organization. As an organization scales, a misconfiguration makes it into every single instance a team fires up rather than just the one in a single insecure bespoke server. So, the stakes are higher.

Arvatz explains that DevOps can do a lot to help raise security posture in an organization and most of the tools involved have the capability to be used securely. But at the same time, organizations need security-conscious administrators in charge.

"Some [DevOps] tools do offer inherently higher security in their basic configuration, and most cloud platforms offers robust security defenses, but some tools rely entirely on their operator knowledge and expertise," he says. "The general shortage of experienced employees in the DevOps and security fields and the fact that most if not all tools are cloud-based make them prone to human errors."

In addition to renaming DevOps tools on Web-facing servers and implementing multifactor authentication, Arvatz and his team suggest a number of other best practices for these tools. These include using proxy servers, stopping the use of default ports, and meticulously keeping up on the patching of infrastructure. They also suggest blocking access to the servers altogether from the Web when feasible, though they acknowledge that may not always be practical.

Ericka Chickowski specializes in coverage of information technology and business innovation. She has focused on information security for the better part of a decade and regularly writes about the security industry as a contributor to Dark Reading. View Full Bio

Its alarming to see that externally facing infrastructure is not even being password protected (a very weak form of protection). What is even more alarming is that these devops tools are utilizing company data instead of test data.

With Continuous Integration, DevOps says that you can install modified software as soon as it passes the test suite. Even if you could have a test suite that would test every required function for the application, how can a test suite insure that you haven't added vulnerabilities. Remember that many people think that Agile means no documentation (rather than don't write more documentation than you need) so you don't know what to test for.

There was a song entitled "Do You Believe in Magic". I get the feeling that a lot of IT management feels that Agile, Scrum, DevOps, SecOps are magic wands and everything will go perfectly so long as you recite the proper incantations at your stand-up meetings. They don't think that you need to think about anything else. (And the product looks like they didn't think about anything.) Before you expect them to understand the need for security, you need to get them to understand how to write software that works.

Online attackers are constantly developing new, innovative ways to break into the enterprise. This Dark Reading Tech Digest gives an in-depth look at five emerging attack trends and exploits your security team should look out for, along with helpful recommendations on how you can prevent your organization from falling victim.

Certain versions between 2.x to 5.x (refer to advisory) of the NetApp Service Processor firmware were shipped with a default account enabled that could allow unauthorized arbitrary command execution. Any platform listed in the advisory Impact section may be affected and should be upgraded to a fixed...

An XML External Entity Injection (XXE) vulnerability in the Management System (console) of BlackBerry AtHoc versions earlier than 7.6 HF-567 could allow an attacker to potentially read arbitrary local files from the application server or make requests on the network by entering maliciously crafted X...