DevOps: Breaking the Development-Operations barrier

What is DevOps?

DevOps gave us an edge

“DevOps has helped us do very frequent releases, giving us an edge on time to market. We are now able to make daily product releases as opposed to 6-month releases, and push fixes to our customers in a span of a few hours.”

— Hamesh Chawla, VP of Engineering at Zephyr

DevOps is a set of practices that automates the processes between software development and IT teams, in order that they can build, test, and release software faster and more reliably. The concept of DevOps is founded on building a culture of collaboration between teams that historically functioned in relative siloes. The promised benefits include increased trust, faster software releases, ability to solve critical issues quickly, and better manage unplanned work.

At Atlassian, DevOps is the next most famous portmanteau (combining of two words) next to Brangelina (Brad Pitt and Angelina Jolie), bringing together the best of software development and IT operations. And like our jokes, it requires some explaining.

At its essence, DevOps is a culture, a movement, a philosophy.

It's a firm handshake between development and operations that emphasizes a shift in mindset, better collaboration, and tighter integration. It unites agile, continuous delivery, automation, and much more, to help development and operations teams be more efficient, innovate faster, and deliver higher value to businesses and customers.

Who's doing DevOps?

Chef is the company behind the Chef Automate platform for DevOps workflows. Tens of thousands of developers use Chef to test, automate, and manage infrastructure. At the forefront of the DevOps evolution, the Seattle-based company has been releasing products like Chef, InSpec, Habitat, and Chef Automate to advance new ways of developing and shipping software and applications. To experiment with and refine its own DevOps practices, Chef relies on the Atlassian platform.

History of DevOps

The DevOps movement started to coalesce some time between 2007 and 2008, when IT operations and software development communities got vocal about what they felt was a fatal level of dysfunction in the industry.

They railed against the traditional software development model, which called for those who write the code to be organizationally and functionally apart from those who deploy and support that code.

Developers and IT/Ops professionals had separate (and often competing) objectives, separate department leadership, separate key performance indicators by which they were judged, and often worked on separate floors or even separate buildings. The result was siloed teams concerned only with their own fiefdoms, long hours, botched releases, and unhappy customers.

Surely there’s a better way, they said. So the two communities got together and started talking – with people like Patrick Dubois, Gene Kim, and John Willis driving the conversation.

What began in online forums and local meet-ups is now a major theme in the software zeitgeist, which is probably what brought you here! You and your team are feeling the pain caused by siloed teams and broken lines of communication within your company.

You’re using agile methodologies for planning and development, but still struggling to get that code out the door without a bunch of drama. You’ve heard a few things about DevOps and the seemingly magical effect it can have on teams and think “I want some of that magic.”

The bad news is that DevOps isn’t magic, and transformations don’t happen overnight. The good news is that you don’t have to wait for upper management to roll out a large-scale initiative. By understanding the value of DevOps and making small, incremental changes, your team can embark on the DevOps journey right away. Let’s look at each of these benefits in detail.

Infrastructure as code allowed us to perform 10x more builds without adding a single person to our team. — Michael Knight, Build Engineer at Atlassian

What's in it for you?

Collaboration and trust

Culture is the #1 success factor in DevOps. Building a culture of shared responsibility, transparency and faster feedback is the foundation of every high performing DevOps team.

Teams that work in siloes often don't adhere to the 'systems thinking' of DevOps. 'Systems thinking' is being aware of how your actions not only affect your team, but all the other teams involved in the release process. Lack of visibility and shared goals means lack of dependency planning, misaligned priorities, finger pointing, and 'not our problem' mentality, resulting in slower velocity and substandard quality. DevOps is that change in mindset of looking at the development process holistically and breaking down the barrier between Dev and Ops.

Release faster and work smarter

Speed is everything. Teams that practice DevOps release more frequently, with higher quality and stability.

Lack of automated test and review cycles block the release to production and poor incident response time kills velocity and team confidence. Disparate tools and processes increase OPEX, lead to context switching, and slow down momentum. Through automation and standardized tools and processes, teams can increase productivity and release more frequently with fewer hiccups.

Accelerate time to resolution

The team with the fastest feedback loop is the team that thrives. Full transparency and seamless communication enable DevOps teams to minimize downtime and resolve issues faster than ever before.

If critical issues aren't resolved quickly, customer satisfaction tanks. Key issues slip through the cracks in the absence of open communication, resulting in increased tension and frustration among teams. Open communication helps Dev and Ops teams swarm on issues, fix incidents, and unblock the release pipeline faster.

Better manage unplanned work

Unplanned work is a reality that every team faces–a reality that most often impacts team productivity. With established processes and clear prioritization, the Dev and Ops teams can better manage unplanned work while continuing to focus on planned work.

Transitioning and prioritizing unplanned work across different teams and systems is inefficient and distracts from work at hand. However, through raised visibility and proactive retrospection, teams can be better anticipate and share unplanned work.

The CALMS Framework for DevOps

Culture

If we could sum up DevOps culture in one word, it’d be “collaboration” – and if we were allowed two words, they’d be “cross-functional collaboration.” (Ok, that’s more like three words.)

All the tooling and automation in the world are useless if they aren’t accompanied by a genuine desire on the part of development and IT/Ops professionals to work together. Because DevOps doesn’t solve tooling problems. It solves human problems. Therefore, it’s unlikely you’ll poke your head out of the cubicle one day, look around, and discover that teams at your company embody DevOps culture. But there are simple things you can do to nurture it.

Think of DevOps much like agile, but with the operations included. Forming project- or product-oriented teams to replace function-based teams is a step in the right direction. Include development, QA, product management, design, operations, project management, and any other skill set the project requires. At Atlassian, we even embed marketing with our product teams.

Few things foster collaboration like sharing a common goal and having a plan to reach it together. At some companies, switching suddenly to project-based teams is too much, too soon. So take smaller steps. Development teams can – and should – invite appropriate members of the operations team to join sprint planning sessions, daily stand-ups, and sprint demos. Operations teams can invite key developers. It’s an agile and organic way to keep on the pulse of each other’s projects, ideas, and struggles. The time spent listening and cross-pollinating subject-area knowledge pays for itself by making release management and emergency troubleshooting far more efficient.

And speaking of emergencies, they’re an effective test of DevOps culture. Do developers, operations, and customer support swarm on the problem and resolve it as a team? Does everyone start with the assumption that their teammates made the best decisions possible with the information and resources they had at the time? Is the incident post-mortem about fixing processes instead of pointing fingers? If the answer is “yes,” that’s a good indication that your team functions with DevOps culture at its core.

Note that the most successful companies are on board with DevOps culture across every department, and at all levels of the org chart. They have open channels of communication, and talk regularly. They make sure everyone’s goals are aligned, and adjust as needed. They assume that keeping customers happy is just as much product management’s responsibility as it is the development team’s responsibility. They understand that DevOps isn’t one team’s job. It’s everyone’s job.

Automation

Build, test, deploy, and provisioning automation are typical starting points for teams who don’t have them in place already. And hey: what better reason for developers, testers, and operators to work together than building systems to benefit everyone?

Teams new to automation usually start with continuous delivery: the practice of running each code change through a gauntlet of automated tests, often facilitated by cloud-based infrastructure, then packaging up successful builds and promoting them up toward production using automated deploys. As you might guess, continuous delivery is not a quick and easy thing to set up, but the return on investment is well worth it.

Why? Computers execute tests more rigorously and faithfully than humans. These tests catch bugs and security flaws sooner, allowing developers to fix them more easily. And the automated deploys alert IT/Ops to server “drift” between environments, which reduces or eliminates surprises when it’s time to release.

Another of DevOps’ major contributions is the idea of “configuration as code.” Developers strive to create modular, composable applications because they are more reliable and maintainable. That same thinking can be extended to the infrastructure that hosts them, whether it lives in the cloud or on the company's own network.

True, systems are always changing. But we can create a facade of immutability by using code for provisioning so that re-provisioning a compromised server becomes faster than repairing it – not to mention more reliable. It reduces risk, too. Both development and operations can incorporate new languages or technologies via the provisioning code, and share the updates with each other. Compatibility issues become immediately apparent, instead of manifesting in the middle of a release.

“Configuration as code” and “continuous delivery” aren’t the only types of automation seen in the DevOps world, but they’re worth special mention because they help break down the wall between development and operations. And when DevOps uses automated deploys to send thoroughly tested code to identically provisioned environments, “Works on my machine!” becomes irrelevant.

Lean

When we hear “lean” in the context of software, we usually think about eliminating low-value activities and moving quickly – being scrappy, being agile. Even more relevant for DevOps are the concepts of continuous improvement and embracing failure.

A DevOps mindset sees opportunities for continuous improvement everywhere. Some are obvious, like holding regular retrospectives so your team’s processes can improve. Others are subtle, like A/B testing different on-boarding approaches for new users of your product.

We have agile development to thank for making continuous improvement a mainstream idea. Early adopters of the agile methodology proved that a simple product in the hands of customers today is more valuable than a perfect product in the hands of customers six months from now. If the product is improved continuously, customers will stick around.

And guess what: failure is inevitable. So you might as well set up your team to absorb it, recover, and learn from it (some call this “being anti-fragile”). At Atlassian, we believe that if you’re not failing once in a while, you’re not trying hard enough.

We challenge our teams with big, hairy, audacious goals and make sure they have the autonomy and the resources to meet them. We hire smart, ambitious people and expect them to fail sometimes.

In the context of DevOps, failure is not a punishable offense. Teams assume that things are bound to go pear-shaped at some point, so they build for fast detection and rapid recovery. (Read up on Nexflix’s Chaos Monkey for an excellent example.) Postmortems focus on where processes fell down and how to strengthen them – not on which team member f'ed up the code. Why? Because continuous improvement and failure go hand in hand.

DevOps has evolved so that development owns more operations – and that’s how Chef works. We can’t just throw it over the wall anymore. Our engineers are responsible for QA, writing, and running their own tests to get the software out to customers. — Julian Dunn, Product Manager at Chef

Measurement

It’s hard to prove your continuous improvement efforts are actually improving anything without data. Fortunately, there are loads of tools and technologies for measuring performance like how much time users spend in your product, whether that blog post generated any sales, or how often critical alerts pop up in your logs.

Although you can measure just about anything, that doesn’t mean you have to (or should) measure everything. Take a page from agile development and start with the basics:

How long did it take to go from development to deployment?

How often do recurring bugs or failures happen?

How long does it take to recover after a system failure?

How many people are using your product right now?

How many users did you gain / lose this week?

With a solid foundation in place, it’s easier to capture more sophisticated metrics around feature usage, customer journeys, and service level agreements (SLAs). The information you get comes in handy when it’s time for road mapping and spec’ing out your next big move.

All this juicy data will help your team make decisions, but it’s even more powerful when shared with other teams – especially teams in other departments. For example, your marketing team wants shiny new features they can sell. But meanwhile, you’re seeing high customer churn because the product is awash in technical debt. Providing user data that supports your roadmap – even if it’s light on features and heavy on fixes - makes it easier to build consensus and get buy in from stakeholders.

Sharing

The long-standing friction between development and operations teams is largely due to a lack of common ground. We believe that sharing responsibility and success will go a long way toward bridging that divide. Developers can win instant goodwill by helping to carry one of operations’ biggest burdens: the pager. DevOps is big on the idea that the same people who build an application should be involved in shipping and running it.

This doesn’t mean that you hire developers and simply expect them to be excellent operators as well. It means that developers and operators pair with each other in each phase of the application’s lifecycle.

Teams that embrace DevOps often have a rotating role whereby developers address issues caught by end users while, at the same, troubleshooting production problems. This person responds to urgent customer-reported issues, creating patches when necessary, and works through the backlog of customer-reported defects. The “developer on support” learns a lot about how the application is used in the wild. And by being highly available to the operations team, the development teams builds trust and mutual respect.

Slogging through the rough patches together makes celebrating successes all the more sweet. You’ll know DevOps culture has taken hold at your company when you see the development team bring bagels for the operations team on release day.

Positive feedback from peers motivates us as much as our paychecks and career ambitions. Publicly recognizing a teammate who detected a nasty bug before it went live means a lot. If your department has discretionary budget for employee kudos, don’t let it go unused!