Category Archives: Agile Principles and Practices

Understanding “what is systems thinking” is important to developing a deeper understanding of Agile. I’m an engineer at heart. I’ve been trained to analyze problems objectively and come up with well-thought-out solutions. That approach comes naturally to me but it is something that needs to be learned and reinforced. Here’s a dialog I had with my wife that illustrates this:

Wife: I will never buy another *Brand X* washing machine again!

Me: Why is that?

Wife: The clothes don’t smell fresh!

What’s wrong with that picture? People often rush to judgment like that without fully analyzing a situation and/or make a hasty assessment based on some kind of personal bias that’s not very objective. Think about it – in this situation, there are many things that might cause the clothes to not smell fresh so it’s probably premature to blame the washing machine and all models of washing machines built by *Brand X* so quickly but people do that all the time.

“Systems Thinking” is a framework for looking at something as a “system” and understanding how all the components of that system contribute to achieving whatever result it is supposed to accomplish. For example, the process of washing clothes in a washing machine depends on the type of detergent, type of fabric softener, the need to operate the washing machine properly, and the need to clean the washing machine drum periodically, etc. to achieve the desired result of having fresh-smelling clothes. For a more detailed definition of what “Systems Thinking” is, check out this blog post I wrote some time ago:

“Systems Thinking” is a powerful tool I learned a long time ago when I first read Peter Senge’s book: “The Fifth Discipline – The Art and Practice of the Learning Organization” in the 1990’s. That has been a very powerful tool for me that I’ve used over-and-over again in many situations.

The practice of systems thinking can be complex – you can use the phrase to refer to a set of tools – such as causal loop diagrams, stock and flow diagrams and simulation models – that help us map and explore dynamic complexity. “For example, systems thinkers often describe the world in terms of reinforcing and balancing processes, limits, delays, patterns of behavior over time, and so forth.” – Barry Richmond, isee systems, inc.

However, Without adding a lot of complexity, a lot can be gained from simply developing a unique perspective on reality – a perspective that sharpens our awareness of the whole and of how the parts within those wholes interrelate. The biggest obstacle to systems thinking; however, is our tendency to over-simplify something that is complex to force-fit it into binary, black-and-white terms rather than trying to understand the complexity of it at a deeper level. My wife’s emotional reaction to the washing machine is an example of that. Her instant reaction was that it must be that *@!# washing machine and I’ll never buy another washing machine like that again!

Here’s an example from a LinkedIn discussion I recently participated in that is more directly relevant to the subject of Agile Project Management:

“Ultimately Project Management is a type X/violent approach to delivery. Where Lean/Agile is a type Y/non-violent approach to delivery”

What’s wrong with that statement? It makes a very broad-based assessment of what “project management is based on some very biased opinions of what project management is and attempts to characterize the whole practice of project management that way. It’s equivalent to my wife’s statement that “I will never buy another *Brand X* washing machine again!”. Anyone who thinks that way will have a very difficult time adopting a true systems thinking approach just as my wife had to adjust her thinking to think about the problem with the washing machine in a broader and more objective perspective. There’s a saying that I think is very relevant to this that says:

“It’s easier to accept a simple myth and move on than it is to take the time to understand complex reality”

As long as people cling on to some of the simply myths and stereotypes that exist about what “project management” is, it will be difficult for them to see “Project Management” and, more specifically, “Agile Project Management” in a fresh new perspective. Another statement made by the same person in the LinkedIn discussion was:

“The term Agile PM is as disconcerting as a Scrum Project Manager”

People like to see things in binary, black-and-white terms and have difficulty seeing the possibility that all project managers might not fit into that stereotype.

I’ve just finished developing a new online training course on “Understanding Agile at a Deeper Level” that includes a module on “Systems Thinking” because I believe it is very important to Agile Project Management. The new course also has a lot of material on the principles and values behind both Agile and Scrum that I think will help people see things in a fresh new perspective based on a deeper understanding of Agile and Scrum beyond just the “mechanics” of basic Scrum practices. This new course will help people take a systems thinking approach to understand Agile and Scrum at a deeper level and see it in a broader perspective of how it fits within a business enterprise as a whole. This new course is in final review now and should be released within the next week. You can view a brief video summary of the course here:

I recently participated in a discussion on LinkedIn that was initiated by someone who suggested several possible roles for a Business Analyst in Agile/Scrum that didn’t seem consistent with Agile principles at all. I believe that modifying and extending Agile and Scrum may be necessary to fit the situation, but it has to be done intelligently and I think it takes some skill to figure out what makes sense and what doesn’t.

We all know that there are Agile “zealots” who insist on rigid adherence to doing Agile/Scrum “by the book” without any deviation. On the other hand, there are people who “wing it” and treat Agile/Scrum practices like a “cafeteria menu” where you can pick and choose the principles and practices you want to adopt and which ones to ignore. Neither one of those approaches makes sense in my opinion but there’s a lot of “gray area” between those extremes. So, how do you determine what makes sense and what doesn’t make sense? I don’t think there’s a clear answer to that question but here’s some guidelines that I think are useful.

There’s a big difference between:

Taking a proven framework like Scrum and modifying it and extending it in a way that is consistent with Agile principles and practices, and

Just starting from scratch ignoring Scrum and all other Agile methodologies, principles, and practices and attempting to put together some kind of ad-hoc approach

There’s an analogy to the martial arts that I think fits pretty well. There are a variety of different kinds of martial arts but they all have some similarity and they all require some level of knowledge, proficiency, and discipline in how they’re practiced to be good at it. You don’t just go out and start doing martial arts without any training and experience to know how to do it. Check out this article I wrote on “Stages of Mastery in Agile”.

The essence of the “Shu-ha-ri” martial arts philosophy is that you should be at a level of proficiency before you start improvising and “improvisation without knowledge and proficiency is just amateurism”. I think that is also very applicable to Agile. It takes a considerable amount of skill to figure out how to modify and extend Scrum and other Agile methodologies to fit a given situation.

The key message is that people shouldn’t underestimate the level of skill it takes to modify and extend an Agile/Scrum approach to fit a given situation. That’s a key advantage of some predefined frameworks like SAFe but, on the other hand, even with some of the predefined frameworks, it takes some skill to adapt an Agile approach to fit a business and there is no “silver bullet”.

There is a lot of religious fervor about Agile. I was reviewing one of Dean Leffingwell’s slide presentations and I particularly like a comment he made that “Agile is only a tool – it’s not a religion”. It inspired me to write this blog post.

There is a lot of religious fervor about Agile and there’s a lot of polarization between proponents of Agile and more traditional plan-driven project management approaches. There’s a lot of good reasons why Agile makes sense in many situations but some people seem to just do it mechanically – becoming Agile becomes a goal in itself without a real understanding of why it makes sense in a given situation.

Don Reinertsen has written a good book called “The Principles of Product Development Flow” which I think provides some valuable insight into the underlying principles of why Agile makes sense. Since the principles are general enough to apply to any product development effort (agile or plan-driven), it also provides an objective foundation for comparing Agile and alternative plan-driven approaches so that someone can make an intelligent and objective assessment regarding how these two seemingly competitive approaches.

The book goes a little too far in my view in developing a quantitative and mathematically sound basis for the principles in the book based on statistics. In the real world, I don’t think very many people would really apply that kind of rigor to analyzing these principles; but nonetheless, an understanding of the principles, at least at a high level, is very valuable and it does provide an objective foundation for understanding the benefits of Agile at a deeper level. I’ve summarized the eight principles here:

Economics – Take an Economic View

“Why do we want to change the product development process? The answer: To increase profits. As obvious as this might seem, this economic view of product development is surprisingly uncommon. Instead, product developers create a large number of proxy objectives: increase innovation, improve quality, conform to plan, shorten development cycles, eliminate waste, etc. Unfortunately, they rarely understand how these proxy objectives influence profits.”

I’ve seen a number of instances where companies blindly pursue some of those “proxy objectives” such as “increasing innovation” without really understanding the economic impact. “Increasing innovation” should not be an end-in-itself and it reaches a point of diminishing returns at some point and it might begin to impact other proxy variables such as quality. For that reason, it is good to put things in an economic context. The economic context provides a framework for making intelligent decisions about how much to focus on these “proxy variables”.

Here’s another example of how the economic context can provide a sound framework for decision-making: It is generally best to defer decisions on product features as long as possible; however, some decisions should be made early and should not be significantly deferred because if they have a significant economic impact.

Queues – Actively Manage Queues

“Queues matter because they are economically important, they are poorly managed and they have the potential to be much better managed. Queues profoundly affect the economics of product development. They cause valuable work products to sit idle, waiting to access busy resources.… queues hurt cycle time, quality, and efficiency.”

Developing requirements far-in-advance that sit in a queue waiting for development can be inefficient and wasteful because:

The requirements may change prior to going into development and much of the effort involved in developing the requirements might have been wasted, and/or

Speculation in the requirements that are done too far into the future can result in erroneous assumptions that make their way into development without being questioned

But, on the other hand, we shouldn’t blindly accept the notion that developing requirements far-in-advance never makes sense. There are a lot of reasons why at least developing high-level requirements early might make sense to guide the overall direction and architecture of the project and I’m sure that there are even some circumstances where developing more detailed requirements upfront also makes sense (you have to use the economic impact as a guide for making that decision).

Variability – Understand and Exploit Variability

“There is probably no aspect of product development that is more misunderstood than variability. Because variability sometimes leads to bad outcomes, it is inevitable that some observers will incorrectly generalize that variability always leads to bad outcomes.”

Reducing variability will many times improve efficiency but that isn’t always the case. For example, Breaking up large requirements into smaller ones that are of a more uniform size reduces variability and can improve flow, however, at some point further attempts to reduce variability do not have economic value.

For example, if we might use a rule-of-thumb that if a user story is greater than 13 story points, it needs to be broken down, but there’s probably little value in breaking down a story with less than 13 story points into smaller story points just to reduce variability.

Batch Size – Reduce Batch Size

“Ask a typical product developer what their batch size is, and why, and you will get a puzzled look…Product developers simply don’t think in terms of batch size. As a result, they underutilize one of the most important tools to improve flow… many of the most important improvements in product development such as concurrent engineering, rapid prototyping, and agile software methods, are recognizable as batch size reductions. However, because developers fail to recognize the underlying mechanism of action behind these improvements, they lose the chance to apply this principle more broadly throughout their development process.”

Examples of batch size inefficiencies include:

Project scope – taking on more in a single project than is truly necessary

Project Funding – The entire project is conceived and funded as a single large batch proposal

Requirements definition – the tendency to define 100% of the requirements before the project starts

WIP Constraints – Apply WIP Constraints

Use Work in Process (WIP) constraints to manage overall flow. For example,

Control the number of projects taken on at any one time to avoid over-saturating development resources

Use specialized resources wisely to maximize their impact on overall flow

Cadence, Synchronization, and Flow Control – Control Flow Under Uncertainty: Cadence and Synchronization

“Cadence is the use of a regular predictable rhythm within a process. This rhythm transforms unpredictable events into predictable events. It plays an important role in preventing variability from accumulating in a sequential process…”

Examples of cadence are using fixed-length sprints to establish a pace for the development effort. Examples of the use of synchronization include:

Concurrent development on multiple paths at the same time

Concurrent testing of multiple subsystems

Fast Feedback – Get Feedback as Fast as Possible

Fast feedback can lower the expected loss by truncating unproductive paths more quickly or raise the expected gain because we can exploit an emergent opportunity by rapidly redirecting resources.

“Sophisticated military organizations can provide very advanced models of centrally coordinated, decentralized control. There is an impression that military organizations seek robotic compliance from subordinates to the orders of superiors. In reality, the modern military focuses on responding to a fluid battlefield, rather than executing a predetermined plan. It views war as a domain of inherent uncertainty, where the side that can best exploit uncertainty will win.”

I recommend Don Reinertsen’s book for anyone who wants more in-depth understanding of these principles (this is just a very high-level overview). Many people get immersed in the mechanics of Scrum and forget about the values and principles of Agile. Don Reinertsen’s book goes even deeper into the principles of product development which provides a good basis for understanding both Agile (adaptive) approaches and Waterfall (plan-driven) approaches at a deeper level.

Is Agile Like the Steam Engine? I recently participated in a LinkedIn discussion in which the author of the discussion made a statement that “Agile is by definition a software development methodology”. I didn’t agree with that statement at all…Its unfortunate that the word “Agile” has acquired that connotation in actual usage by a number of people but that’s just sloppy use of terminology (similar to the sloppy use of the word “Waterfall” that I’ve talked about) and its also somewhat narrow thinking.

The author of that discussion defended his statement that “Agile is by definition a software development methodology” by saying that “The Agile Manifesto starts with these words ‘We are uncovering better ways of developing software'”. That statement doesn’t work either in my opinion – I’m sure that was a key motivator behind creating the Agile Manifesto but that was in 2001 which was 12-13 years ago and our concept of what Agile is has grown significantly since that time.

I was trying to think of a good analogy to illustrate this point and somehow I thought of the steam engine. The steam engine was originally invented in 1698 by Thomas Savery who was working on the problem of how to pump water out of coal mines. Thomas Newcomen improved on the design but it wasn’t until Scotsman James Watt improved on the steam engine in the second half of the 18th century that it became truly a viable piece of machinery that helped start the industrial revolution. Even though that started the industrial revolution across the whole world, very few people in the 1700’s would likely imagine that that was only the beginning and the same principles originally used to use to pump water out of coal mines would have a much broader usage in the future including powering nuclear power plants.

The roots of Agile go back a lot further than 2001 – you can very easily trace the roots of Agile to TQM and Lean Manufacturing long before 2001 and the evolution of iterative methodologies like RUP certainly also had an influence. The period prior to 2001 was probably equivalent to pumping water out of coal mines

Learning to apply The Agile principles to software development in the Agile Manifesto in 2001 was a real breakthrough that is probably similar to James Watt learning how to apply steam to operate industrial machinery that started the industrial revolution across the world. It had major impact, but that was only the beginning

Just as James Watt probably never envisioned the use of steam power in nuclear power plants, the signers of the Agile Manifesto probably never fully envisioned widespread implementation of those principles in many other areas beyond software development

In common usage, some people think that the word “Agile” only applies to software development; to other people, the definition is even narrower and you’re only truly Agile if you’re doing Scrum and doing it strictly by the book with no variation. in my opinion, this really is just sloppy use of terminology and narrow thinking in how the word “Agile” is being used. Our understanding of what “Agile” is is still at a very early stage probably very similar to when James Watt learned how to apply steam technology to operate industrial machinery and limiting our thinking about what “Agile” is may prevent us from imagining new applications and ways of using Agile principles and practices that no one has even dreamed of yet.