Etiquetas

Processes, Risks, and Continuous Delivery

As we adopt DevOps practices within IBM, we quickly find that there are many business processes which can get in the way of continuous delivery. Most of these processes are put in place to mitigate risk of releasing products that may expose IBM to legal action (e.g. license infringements or regulatory non-compliance), or that could jeopardize IBM's reputation (e.g. through poor quality or security vulnerabilities). There is a lot of work currently underway to streamline these processes. Essentially, we need to engage the process owners and work together with them to enable product teams to release more frequently [1]. As we do this, it is important to recognize how continuous delivery changes some aspects of the risks that the processes are intended to mitigate.

One interesting aspect that changes is the relationship between "deployment" and "release". Among the evolving patterns that SaaS teams are using with continuous delivery are "Dark Launch" and "Ramp-ups" [2]. Dark Launch is the practice of deploying a new feature into production without enabling any new capabilities in the user interface. Typically, Dark Launch involves the exercising and monitoring of back-end services related to the new feature to help confirm that everything will work when the new capabilities are made available to users. Ramp-ups are the practice of making a new capability available to a small and then gradually increasing subset of users. Ramp-ups allow incremental feedback and additional confidence that a new feature is ready for full production.

If we consider "release" to be the process of making new features available to users, then one of the interesting aspects of Dark Launch and Ramp-ups is that they effectively decouple the process of deployment from the process of release. Features can be deployed into production as soon as they are ready for use and the release of those features to users can be controlled separately via switches [3]. By carefully controlling the number of users who are exposed to a new feature via Ramp-ups, the risks associated with releasing that new features are significantly less than they would be if the feature was exposed to all users of the system at once. Continuous delivery also brings the ability to very quickly roll back a change or deploy a fix if a problem is found. This also changes the nature of the risks that our processes are intended to mitigate. So even as Continuous Delivery can be inhibited by processes, it also introduces opportunities to refine and improve those processes based on how it changes the risks involved with deployment and release.

As we work to improve our processes, we need to be careful that this streamlining doesn't turn into a game of "whack-a-mole" in which steps are removed from one process only to appear in a different process. Our goal is to establish a culture of continuous improvement for our processes - one that continually evolves to leverage and enable new patterns that emerge from Continuous Delivery and DevOps.