Lean, Scrum and Flow Product Development

Pair programming is an agile software development technique in which two programmers work together at one workstation. One, the driver, writes code while the other, the observer, pointer or navigator, reviews each line of code as it is typed in.[i] Pair programming is considered a wasteful technique because people tend to believe that navigators aren’t writing code. In fact, this simplistic view lays on the foundation that the more people writing code, the faster you product will release. Yet, things usually go even worse because products suffer from maintainability and complexity problems in short and long run as a result of poor software development practices, inexperienced teams and occasional quality reviews. Some studies have shown that paired programmers are only 15% slower than two independent individual programmers, but produce 15% fewer bugs[ii]. I decided to give it a try when I came across queuing theory, which is a statistics and probabilistic approach to evaluate how systems (teams) perform when jobs are nonrepetitive, nonhomogeneous and they arrive in an unpredictable arrival rate. It sounds familiar!!!!. Isn’t it? Some additional concepts are needed before continuing: Server(s) are the resources whom perform the job. Arrival process describes the arrival pattern to the queue. Service process is the amount of time that it takes to accomplish the work. Queuing discipline describes how we manage the arrival work (FIFO First input first output, LIFO last input, first output,etc)

Next chart depicts the relation between capacity utilization and service process for a system with unpredictable arrival rate and unpredictable service time, also defined as M/M/1/∞

Notice that cycle time increases exponentially as you increase capacity utilization thus, the effect of overloading teams drives to blocking system’s workflow more and more. So, the effect of loading systems from 70% to 80% capacity utilization increases cycle time around 25 times.

What should we do to keep capacity utilization under certain limits? My favorite choice is limiting WIP. Another valuable countermeasure is the pair programming technique because it helps decrease the team’s capacity utilization to acceptable levels (behind the scenes) which also leads to reduce cycle time and to improve flow. This technique also strives to reduce silo mentality, promoting code ownership through the constant rotation of pairs. Again, queuing theory confirms that mathematically systems perform better when there’s only one queue (product backlog) and several servers (team members) whom can deal with work.

Thus, if your goal is to increase system’s performance, you should try to seed and promote T shape servers. Those whom are experts in a single area and should potentially develop abilities to collaborate with other areas. Finally, a fow more benefits from the pair programming practice: developers learn from each other new development skills and continuous rotation creates a strong sense of “team”.

An interesting opportunity showed up a few weeks ago just after returning from my paternity leave. My brain was still stuck and asleep when my co-workers were presenting me the last “Death or Death” project. They told me we had only 4 months to deliver a new software before the Nothing.

*Thanks to Michael Ende for describing the life of most software developers in this world.

After some workshops that we held to create a common understanding about the problem, we discussed about our critical situation and we decided to promote our agile principles and values even more. We reinforced next ideas:

Delivering the highest quality software iteratively

Building and designing it incrementally

Communicating constantly with customer

Focusing on customer needs

Thus, the following techniques or tools were planned to be used in our last *cough project:

The intent of this blog isn’t to create a prescriptive recipe for your software development process but to provide insights into the reasons to using any specific technique.

#impact mapping
Despite the fact that this tool is designed for strategic planning and very useful for managing projects in the long run. We decided to use this tool for helping the team visualize the product backlog as a report (information radiator) in the short term. This simplification of the diagram seeks to respond to the following questions at first sight:

What the customer wants in plain text

How to provide value. In User Story format. As a I want so that

Sometimes It is needed to break down the “how” item into smaller pieces following User Story format in order to give even more detail about the content of the How node or Epic.

#MobProgramming
This technique which I discovered thanks to the promotion of Woody Zuill consists of one team, one active keyboard and one projector.
As they promote: It’s just like doing full-team pair programming.
There are two roles involved: navigators who discuss, think, design and guide the driver who is in charge of writing the code that navigators are dictating. Every 15 minutes the driver role is rotated.
Team’s feedback after two weeks is terrific. They highlighted the following emergent behaviors:

Alignment: The whole team took part of the architecture. In fact, all team members coded and designed the emergent architecture.

The whole team defined a solid foundation for coding standards and code quality rules.

More meaningful DONE DEFINITION.

They learnt a lot from each other and specially programmers from senior developers.

#BDD
I told to someone some time ago that the book Specification by example from Godjko Adzic had radically changed the way I understood software development. Although, I didn’t have much experience (only 2 projects), it has become an irreplaceable tool for any project that I work in.

As an Agile coach I try to encourage the team to practice BDD and follow next rules:

Technology changes but domain remains. We avoid testing presentation layer and we make effort to test only our business services. Presentation layer is either delegated to exploratory testing or programmed with an automation tool whether it’s worth investing in it.

Using Ubiquitous Domain language. Formal language must be shared by all members of the software development team – both software developers and non-technical team members.

Testing the real system. Continuous Integration machine provides feedback on a daily basis about the health of the system. We are especially interested in performance issues or integration problems with external components. We are promoting to integrate external components as soon as possible or mocking them until a stable version is ready to integrate.

Embrace BDD refactoring. Re-read and re-write your tests many times in order to minimize misunderstanding or ambiguity and search for inadequate feature or scenario definition. The purpose of the scenario is to describe what the system has to do. Likewise, the feature describes the acceptance test. We use the standard agile framework of a User story for its definition.

As I am writing this blog is I wonder about the possibility to create a specification quality control policy which is a check list for early identification of common issues affected by specifications and help reduce rework.

#New Product Development
Although I got excellent results following the Scrum framework along these years I have been progressively more interested in Lean Software Development, Kanban and New product development and less interested in estimations, the usage of them from middle and upper layers, conflicts provoked by estimations and the relationship with team commitment. Thus, our current software development process is a mix of ideas from different sources and frameworks.

Once every two weeks we release a new version and facilitate a review for our stakeholders. This cadence reduces the team’s coordination cost and creates a sense of urgency to deliver value as soon as possible and receive feedback from stakeholders. Besides, we have also limited work in progress (WIP) in order to provide enough flexibility and adaptability to variability. I took a change to facilitate a Systems thinking analysis meeting to create an awareness of potential effects of modifying the value and how to increase and decrease it. The team has managed this WIP internally so far with responsibility adapting it to the continuous context change.

Although our Done definition includes an statement that states: 0 bugs, some of them have shown up  and we decided to create a queue for them. The queue size is very little (only 6 bugs) and team usually reacts very quickly to keep the queue under control. One of the interesting effects of limiting WIP is that developers are capable of dealing with variability (bugs) faster than other teams that I led in the past.

#Visual Management
First blog entry described how we are going to use our metrics cycle time and lead time but it only mentioned cumulative flow diagrams. Now it’s time I explained to you in more detail the usage of this metric.
This metric aims at helping track and monitor how user stories are moving through various stages of the process to being “done”.

From a cumulative flow diagram we can see:

Where the bottlenecks are in our flow. Based on Theory of Constraints introduced by Eli Goldratt we must exploit the bottleneck, optimizing the throughput of the system adding more capacity or changing how the system is performing. Thus, if we apply continuous improvement frameworks like PDCA or Build – > Measure -> Learn we can easily evaluate if our policies are improving the flow.

If demand is seasonal and take steps to adjust capacity in that case.

If we are delivering value at the end of the process and how to improve them (global view of the process).

Finally, we use release burndown chart to indicate the progress of the team against the product backlog. The diagram is updated every week.

Update on 13th of March. Even though I received some unexpected news from Skype Education and I have decided to go ahead with the plan but using Google Hangout.

Build phase: In order to help me improve my English Communication skills I have decided to create a classroom on Skype Hangout for people to talk about Agile, Lean, Kanban, Scrum or XP and exchange experiences.

This experiment will take place on Saturdays or/and Sundays once a month. Please, have a look at the web page to learn about the dates.

Measure phase: How to measure project’ success: Amount of demand and feedback from those who join the classroom.

Jidoka is the Japanese term used by LEAN practitioners to stop the production line when workers in the factory floor discover a defect. A retrospective is a delayed demonstration of the Jidoka term which aims at evaluating and bringing up continuous improvement actions. Such actions must be focused on the components that interact together to deliver a software product valuable for your costumers: product (software quality), people and processes.

Sometimes, poor variability reduction policies led by team and management layers (including Scrum Masters and Agile Coaches) source problems such as: poor technical skills, requirements ambiguity, poor requirement specifications or scope. Such problems make teams suffer from stressful situations and excess of pressure that must be treated even before the retrospective takes place.

Explicit rules

One of the Kanban foundations for leading the continuous improvement movement is to design explicit process rules which aim to improve flow and reduce risks. As an Agile Coach I have worked together with the team to design a process for the early detection of bottlenecks and “STOP THE LINE” when required.

User Stories

Despite the fact that it’s hard to split epics into small same size user stories, we try to break user stories into vertical slices of the product. In order to provide value as quickly as possible, we ask ourselves the following question:

“What is team going to undertake in only one day?

Bottlenecks, Cycle Time and Traffic Lights

We currently have a visual management metric to measure cycle time and to help us detect bottlenecks.

*Notice that team manually updates the diagram, computing the amount of time it took them to complete the user story. Meanwhile, the agile coach also updates the digital version which automatically collects valuable metrics like the average cycle time, average trend or the standard deviation.

**Horizontal axis depicts the time line and the vertical axis contains the time consumed for each user story.

At the bottom of the diagram, there’s a green area (up to 2 days of cycle time) that indicates that our process is healthy and it’s flowing. No additional action is required.

The yellow area (3rd day of cycle time) is an optional step and we set up a new topic just after the stand-up meeting. Team works together to identify early actions to remove the bottleneck.

At the top of the diagram (more than 4 days of cycle time) we are forced to evaluate the current situation of the bottleneck on a daily basis after stand up meeting. We must focus our attention on the whole process, resources availability and people capacity, work in progress and pending actions. This topic usually takes 10 to 15 minutes.

*Due to the fact that cycle time is a delayed indicator, we are researching to replace the cycle time report with the cumulative flow diagram. It’s a tool used in queuing theory that depicts the quantity of work in a given state, showing work in progress, queue in time, and departure.