Tag Archives: values

How much documentation does it take to run a project with ten people working for six months? For some organizations it takes way too much:

This binder (about 3 or 4 inches thick) is all the documentation associated with such a project. In looking carefully at the project, creating the documentation took far more time than the time spent on designing, writing and testing the software. Yet, the documentation does not produce any value. Only the software produces value. The Agile Manifesto, asks us to focus on the outcome (working software) and to make tradeoffs to minimize the means (comprehensive documentation).

The Agile Manifesto asks us to challenge our assumptions about documentation. In many work environments, documentation is an attempt to address some interesting and important needs:

Knowledge sharing among stakeholders and the people working on a project.

Knowledge sharing across time as people come in and out of a project.

Verification and traceability for contracts or other compliance needs.

Decision-making and analysis for business and technical problems.

Management oversight and control.

Various aspects of individual accountability.

Documentation is usually heavier (more comprehensive) the more the following circumstances exist in an organization:

Geographical distribution of people.

Lack of trust between people, departments or organizations.

Regulated work environments.

Depth of management hierarchy.

Number of people directly and indirectly involved.

Knowledge and skill sets highly segregated between people.

Culture of respect for written texts.

Working Software

What if the software itself could address the needs that often documentation is used to address? Let’s look at them in turn:

Knowledge sharing among stakeholders and the people working on a project.
If the software is functional at all stages, as supported by Agile methods such as Scrum and Extreme Programming, then the software becomes an effective representation of the knowledge of all the people who have participated in building it.

Knowledge sharing across time as people come in and out of a project.
Software that is technically excellent is often easier to understand for people who are new to it. For example, excellence in user experience and design means new users can get up to speed on software faster. Use of good design patterns and automated testing allows new developers to understand existing software easily.

Verification and traceability for contracts or other compliance needs.
Test-driven development (code) and specification by example (scripting and code) are forms of traceable, executable documentation that easily stay in-sync with the underlying software system.

Decision-making and analysis for business and technical problems.
In particular, diagrams can help a great deal here. However, electronic tools for creating such diagrams can be slow and awkward. Consider the practice of Agile Modelling (basically using a whiteboard and taking photos) as a good alternative to precise technical diagramming if you are doing problem-solving.

Management oversight and control.
Reports and metrics drive much of the traditional documentation in an organization. Simplifying reports and metrics often leads to a clearer picture of what is going on, reduces the opportunities to “game” the system, and always results in lower levels of documentation. As well, some reports and metrics can be generated 100% through automated means. All that said, the fundamental premise in the Agile manifesto is that management should base decisions on what is actually built – the “Working software” by looking at it and using it.

Various aspects of individual accountability.
If you really need this, a good version control system can give you the information for this. Sign-offs and other types of accountability documentation are typically just waste that doesn’t actually help in process improvement. Most people who are in high-compliance environments already have licenses and/or security clearances that provide this accountability. If you software is working, however, then this isn’t even a concern as trust is built and bureaucracy can be reduced.

In my recent training programs as research for this article, I have surveyed over 100 people on one aspect of documentation – code documentation. Every individual surveyed is either currently coding or has a coding background, and every single person had the same answer to a simple scenario question:

Imagine that you have just joined a new organization and you are about to start working as a software developer. One of the existing team members comes up to you and introduces himself. He has with him a piece of paper with a complicated-looking diagram and a full binder that looks to be holding about 250 pages. He asks you, “you need to get up to speed quickly on our existing system – we’re starting you coding tomorrow – would you prefer to go over the architecture diagram with me or would you prefer to review the detailed specifications and design documents.” He indicates the one-page diagram and the binder respectively. Which would you prefer?

(I’ve put up a Survey Monkey one-question survey: Code Documentation Preference to extend the reach of this question. It should take you all of 60 seconds to do it. I’ll post results when I write the next Agile Manifesto essay in a month or two.)

The fact that everyone answers the same way is interesting. What is even more interesting to me is that if you think through this scenario, it is actually almost the worst-case scenario where you might want documentation for your developers. That means that in “better” cases where documentation for developers may not be as urgent or necessary, then the approach of just going to talk with someone is a lot better.

Documentation and Maps

The problem with documentation is the same problem we have with maps: “the map is not the territory” (quote from the wisdom of my father, Garry Berteig). We sometimes forget this simple idea. When we look at, say, Google Maps, we always have in the back of our consciousness that the map is just a guide and it is not a guarantee. We know that if we arrive at a place, we will see the richness of the real world, not the simplified lines and colours of a map. We don’t consider maps as legally binding contracts (usually). We use maps to orient ourselves… as we look around at our reality. We can share directions using maps, but we don’t share purpose or problems with maps. And finally, maps assume that physical reality is changing relatively slowly (even Google Maps).

Many times when we create documentation in organizations, however, we get confused about the map versus the territory.

Agility and Documentation

Of course, code is a funny thing: all code is documentation too. The code is not the behaviour. But in software, code (e.g. Java, ASM, Scheme, Prolog, Python, etc.) is as close as possible to the perfect map. Software is (mostly) deterministic. Software (mostly) doesn’t change itself. Software (mostly) runs in a state absent from in-place human changes to that software. Software (mostly) runs on a system (virtual or physical) that has stable characteristics. The code we write is a map. From this perspective, documentation becomes even less important if we have people that already understand the language(s)/platform(s) deeply.

Affiliated Promotions:

Try our automated online Scrum coach: Scrum Insight - free scores and basic advice, upgrade to get in-depth insight for your team. It takes between 8 and 11 minutes for each team member to fill in the survey, and your results are available immediately. Try it in your next retrospective.

In IT and high-tech, there is a “natural” prevailing culture that makes this first value incredibly difficult. This difficulty is rooted in traditional “scientific management“, but made even more so by a critical additional factor that is mostly invisible: techies solve problems with tools.

Management wants to define processes with clearly described activities, clear inputs and outputs, and clear sources and recipients of the activity (see the description of SIPOC for an explanation of this thinking). Techies build tools to automate these well-defined processes to improve their efficiency, quality and reliability.

Management creates organizational roles with detailed descriptions, detailed goals and detailed performance measurements (see the description of RACI for an explanation of this thinking). Techies build tools to carefully constrain people to these detailed roles to improve efficiency, quality and reliability.

Management has money. Techies want some of that money. So they build the tools to help management get what they really want: a completely automated organization of computers, machines and robots.

The culture of technology is to solve problems with individuals and interactions by introducing processes and tools. The culture of technology is (almost) inherently anti-Agile.

The culture of technology is to solve problems with individuals and interactions by introducing processes and tools. The culture of technology is (almost) inherently anti-Agile.

Individuals and Interactions

Let’s look at the first part of this value in a bit more depth. When we think about work, most of us work with other people. We bring our unique skills, personality and interests to work, and we work with other people who also bring unique skills, personality and interests. In a high-bureaucracy, high-technology work environment, it is easy to forget about all this uniqueness and instead objectify people. When people sense they are being objectified, mostly they feel bad about it. We want to be acknowledged as thinking, feeling, unique beings with agency. Objectification, no matter the source or the rationale, is depressing and de-humanizing. The Agile Manifesto implicitly recognizes this concept and asks us who follow the Manifesto to try to shift our value-focus.

There are many aspects to this concept of humanizing work. Some things that come to mind immediately include recognizing and encouraging people’s capacity for:

creativity and innovation

learning and problem-solving

caring about others

pride in work

complementarity with others

responsibility

Processes and Tools

This side of the value is also interesting. Processes and tools do not have agency. They do not improve on their own. Instead, processes and tools only either remain the same or degrade. Processes and tools are forces for stasis: they encourage maintenance of the status quo. Only humans introduce new processes and tools.

Technologists live in a philosophical double-standard: we build processes and tools for others to use and which we frequently would not like used on ourselves. (We will discuss the cases where me might both build and benefit from processes and tools in a bit.) This is one of the challenges of the type of work we do in technology, but it also applies to many other types of work. So how do we solve this conundrum? I would assert that the principles of the Agile Manifesto and the various Agile methods and techniques are all answers to this question. They show us possible ways to implement this value (and the others) without getting stuck in processes and tools.

Only humans introduce new processes and tools.

What are Processes Good For, What are Tools Good For?

Some processes are good. Some amount of process is good. How do we determine what is good? Well, it largely depends on context. Some examples:

If a close family member is living in a distant location then the advances in communication tools are extremely helpful: the telegraph, the telephone, the cell phone, email, Skype. These tools create connections where otherwise there would be little or none.

If a great deal of data is created while running a marketing campaign and needs to be stored and manipulated, then computers are amazing tools for this. Computers are much much better than human minds and manual record-keeping for this sort of work.

If you create a fantastic new soup, from scratch, for some special occasion and you want to remember how to make and even share how to make it with others, then you document the process in a recipe.

Context, Emphasis and Crisis

Context here is important. The value of Individuals and Interactions over Processes and Tools is basically a statement that given the right circumstances we can use processes and tools, but that our default approach to work and problem-solving should be to focus on individuals and their interactions. Depending on the state of your work environment this is easier or harder.

For example, a startup company founded by three long-time friends who have not yet employed anyone else is almost certainly going to solve most problems that come up through discussion amongst founders and through the development of their skills and capabilities. As a company gets larger, however, there is pressure to adopt more and more processes and tools. This pressure comes from a deep source: lack of trust. At about 12 people, you reach the limit of the number of people you can have and still have anyone do anything (this limit is referred to obliquely in “The Wisdom of Teams” by Katzenbach and Smith). After 12 people, it becomes harder to avoid role specialization and some basic forms of processes and tools. In other words, bureaucracy starts growing as the organization grows. Even at this size, however, it is still relatively easy to have a very strong emphasis on individuals and interactions. There is another important limit: somewhere around 150 to 200 people, any hope of 100% mutual trust among the members of the organization is lost. This is the point at which processes and tools “naturally” start to truly take over. (This transition can happen even in much smaller organizations if the culture does not emphasize trust-based interactions.)

In small trust-based organizations, crisis is usually addressed by the mechanisms of mutual respect, skill development, informal agreements, and strengthening the interactions between people. In a large organization with low trust, crisis is almost always addressed by the creation of new bureaucracy: sign-offs, audits, traceability, procedures, policies, processes and tools.

The true test of the an organization’s commitment to the first value of the Agile Manifesto is, therefore, how it responds to crisis. When someone makes a mistake, can we help them develop the skill and the support networks to avoid the mistake in the future? Or do we put in place even more restrictive constraints on what that person does and how they do it?

In a large organization with low trust, crisis is almost always addressed by the creation of new bureaucracy.

Beyond IT and High-Tech

For now, all that needs to be said is that this particular value of the Agile Manifesto does not in any way directly refer to software or software development. As such, it is pretty easy to see how it could be applied in many other types of work. However, there are some types of work where processes and tools really do take precedence over individuals and interactions. If we want to apply the concepts of Agile universally (or near-universally), we have to examine some of these exceptions. I will leave that for a future essay.

Affiliated Promotions:

Try our automated online Scrum coach: Scrum Insight - free scores and basic advice, upgrade to get in-depth insight for your team. It takes between 8 and 11 minutes for each team member to fill in the survey, and your results are available immediately. Try it in your next retrospective.

In 2001, 17 people got together for a world-changing discussion about software development. They tried to find the common values and principles by which people could do better at the work of software development, which was in a terrible crisis (and still, to some extent is). They were successful in that they created a list of 4 values and 12 principles to guide people trying to find better ways of developing software: the Agile Manifesto was born.

It is time to think about the concepts behind the values and principles. To acknowledge that the Agile Manifesto (for software development) can be re-stated at a much deeper level. To abstract Agile software development to Agile work in general. This is my goal over a series of essays about the Agile Manifesto.

Let’s start with an analysis of the values of the Agile Manifesto in relationship to the concept of “value”.

The Agile Manifesto Values

In the Agile Manifesto we can read the four values:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

While a few of these already are quite general, let’s dig a bit deeper by starting with the second value, “working software over comprehensive documentation”. What does this value really refer to? Why do we care about software over documentation. Why “working” vs. “comprehensive”? This is where Lean thinking can help us. The notion of “customer value-added” or just value added work is any work that changes the form, fit or function of that which you are delivering and simultaneously is work that a customer would pay for independently of all the other activities and results you may be spending time on. In this specific example of software and documentation, we can try to imagine what it would be like to say to a potential customer, “we can give you documentation for our software, but we won’t be able to give you the actual software itself… but we’d still like to be paid.” I’m sure it will be clear that it would be a very unusual customer who would agree to such a proposal. Thus, we see that the value of software development activities is in producing the software itself, and the documentation is by necessity of secondary importance.

But what if we are writing a book of fiction? Surely this is documentation! But, it is not. To make the analogy to the type of documentation mentioned in the Agile Manifesto, we would sell a book not just with the story itself, but also with in-depth instructions on how to use a book, how to read, how to interpret our feelings as we read, etc. And not just that, but we would also provide a set of notes to the publisher about exactly how we wrote the book: the time and place of each paragraph written, our original outlines, our research including much that was thrown away, all our conversations with people as we struggled to sort out various plot, character and setting elements, and possibly even all our edits that we had thrown away. This is the documentation to which, by analogy, the Agile Manifesto is referring.

Perhaps now we can look at a connection between the first value and the second value of the Manifesto: that documentation, tools and processes are all much of the same thing. They all belong to the same abstract category, namely, the means used to achieve a particular end. Of course, we all know that both the means and the ends are important, although we may not all agree on their relative importance. Nevertheless, we can probably agree on some extreme outliers that will help us come to the point I wish to make. For example, we can agree that killing someone merely to get to the front of the grocery store line and save a minute or two of time is an extreme case where the end clearly does not justify the means. Likewise, we can also agree that refusing honestly given help out of a desire for independence when it ends with the death of our children by starvation is putting means too much in the fore. Balance is required, therefore. The Agile Manifesto acknowledges this balance by its epilogue to the values, “That is, while there is value in the items on the right, we value the items on the left more.” The other two values of the Agile Manifesto which mention contract negotiation and following a plan are similarly pointing out activities of the means that are non-value added, in Lean terms.

The Agile Manifesto, is stating four values, is, quite directly, pointing to those things which in life are fundamentally of value as ends, not just as means. Individuals and interactions, working software, customer collaboration and responding to change, are all valuable. In order to abstract away from software, then, and create a more general statement of the nature of Agility, we need to explore the idea of value.

The Idea of Value

If we are in business, determining value, while possibly complicated, is not usually too obscure an effort. We look at exchanges of money to see where the “market” agrees there is value. The price of a product or service is only representative of value if someone will actually pay. Therefore, businesses look at return on investment, profit margins and the like to determine value. Similarly, in other types of markets such as the stock market, value is determined by an exchange of money. But underlying this exchange of money is a decision made by an individual human being (or several, or many) to refuse all the other potential uses of that money for the one specific use of making a particular purchase. This choice is based on all sorts of factors. In economics, we talk about these factors mostly in rational selfish terms: what sort of benefit does the purchaser get from the purchase. Factors such as risk, short-term vs. long-term, net present value, trade-offs, etc. certainly can play a role in such decisions. But in business (and in particular marketing and sales), we know that there are also lots of non-rational forces at work in deciding to spend money on a particular something. Value, therefore, has a great deal to do with how a particular person both feels and understands the current proposed “investment” opportunity.

Feeling and understanding arise from many specific factors, as discussed, but what can we say generally about feeling and understanding? They are internal states of a person’s mind (or heart, if you like). Those internal states have been the subject of much discussion in the realm of psychology, sociology, economics, and philosophy. But quite simply, those internal states are greatly determined by perception. How a person perceives a situation is the immediate general factor that determines those internal states. Of course, perception is a general term that includes sensory perception, but also the kinds of prejudices we have, the categories into which we place things conceptually, the internal language we use to describe things, and our existing emotional and mental constructs. So, fundamentally, value is perceived depending on all these perceptual factors.

The Agile Manifesto authors, therefore, had (and perhaps still have) a perception of value which places individuals and interactions, working software, customer collaboration and responding to change all in a position of more value than those other items. But this perception of value may not be in alignment with other people’s perception of value. Still, we can see already in 13 short years that there is broad, if not universal, agreement on these statements of value. Why is this? Why has Agile become so popular over a relatively sustained length of time with a trajectory that still seems to have it growing in popularity for years to come?

Agile values address a deep need in people in the software development discipline and indeed, by analogy, into much of the work world.

In my next essay on the Agile Manifesto, we will take a look in more depth at the first value of the Agile Manifesto: Individuals and Interactions over Processes and Tools.

Some links to commentary on the values of the Agile Manifesto while you wait for my next essay instalment!…

Affiliated Promotions:

Try our automated online Scrum coach: Scrum Insight - free scores and basic advice, upgrade to get in-depth insight for your team. It takes between 8 and 11 minutes for each team member to fill in the survey, and your results are available immediately. Try it in your next retrospective.

Agile Advice was started in 2005. In ten years, we have published over 850 articles (an average of just about 2 per week!). Here are some collections of the ten “best” articles. I hope you enjoy looking back at (or discovering for the first time!) some of the things that have made this such a great joy for me.

I also want to acknowledge that there are a number of other contributors to Agile Advice besides me (Mishkin). These contributors are all experts, all have great experiences, and all are fantastic people to know. I’m grateful for their contributions since they have all made Agile Advice a better place to browse!

Affiliated Promotions:

Try our automated online Scrum coach: Scrum Insight - free scores and basic advice, upgrade to get in-depth insight for your team. It takes between 8 and 11 minutes for each team member to fill in the survey, and your results are available immediately. Try it in your next retrospective.

Hi Charles, although I appreciate your concerns about the word commitment, there is still huge support for adding the five values back to the Scrum Guide, including using that “bad” word. I would like to present to you an argument for the use of the word commitment by telling a story.

Affiliated Promotions:

Try our automated online Scrum coach: Scrum Insight - free scores and basic advice, upgrade to get in-depth insight for your team. It takes between 8 and 11 minutes for each team member to fill in the survey, and your results are available immediately. Try it in your next retrospective.

Hi Everyone. A Scrum Alliance colleague of mine mentioned that in a conversation with Jeff Sutherland (one of the authors of Scrum), Jeff suggested that the community could request and vote on a change to the Scrum Guide (the official description of Scrum) in order the add the values back to the guide. The values are normally listed as focus, commitment, courage, openness and respect. Please go and vote (and / or discuss) this on the Scrum Guides User Voice page. Voting is super easy – you just need to sign in with Facebook.

Affiliated Promotions:

Try our automated online Scrum coach: Scrum Insight - free scores and basic advice, upgrade to get in-depth insight for your team. It takes between 8 and 11 minutes for each team member to fill in the survey, and your results are available immediately. Try it in your next retrospective.

Affiliated Promotions:

Try our automated online Scrum coach: Scrum Insight - free scores and basic advice, upgrade to get in-depth insight for your team. It takes between 8 and 11 minutes for each team member to fill in the survey, and your results are available immediately. Try it in your next retrospective.

When I am speaking with executives, ScrumMasters and other leaders of change in organizations, I often present a simple 3-layer model to understand the relationship between the various moving parts in the Agile Framework:

The Agile Values and Principles – These describe the culture and, in the Agile Manifesto, are the definition of the word “Agile” as applied to software development. I didn’t write the Agile Manifesto so I don’t get to re-define the word Agile. To give an example: in the manifesto it says “The best architectures, requirements and designs emerge out of self-organizing teams.” As a former enterprise architect at Charles Schwab, I struggled with what I saw as incredibly wasteful up-front architectural activities when I knew that developers would (sometimes) ignore my glorious ivory-tower plans! Therefore, if you are still doing up-front architecture and forcing your teams to comply to that architecture, you aren’t Agile. Therefore, as an individual, a team or an organization, you need to make a conscious decision to “BE” Agile or not… and if you decide not, then please don’t call yourselves Agile.

The Agile Toolkit – There are many hundreds of distinct tools in the Agile toolkit including Scrum, OpenAgile and other “large” Agile methods, as well as the Planning Game, Product Box, Test-Driven Development and other “small” Agile techniques. Any group of people trying to BE Agile, will need to use dozens or even hundreds of different Agile tools. I call them tools because the analogy with construction tools is a very good one. Scrum is like a hammer. But you can’t do much with just a hammer. Scrum is a great, simple tool. But you always need other tools as well to actually get stuff done. All the tools in the Agile Toolkit are compatible with the Agile Values and Principles. Even so, it is possible to use the Agile Tools without being Agile. A Scrum team that never gets together face-to-face is not an Agile team: “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.” (Video conferencing doesn’t count.)

The Agile Organization – When you start using a tool, there is a learning period. We start by being conscious of our incompetence and as we persist, we become competent… but it isn’t natural or habitual yet. Eventually, with continued use, we become unconscious of the tool. IDE’s and version control are like this in most organizations: we don’t even think about them! But getting through that initial stage requires us to change; to develop new skills. This process usually requires discomfort or pain (including psychological pain). An organization attempting to BE Agile and to use many of the tools in the Agile Toolkit will need to make many changes and often these will be difficult. For example, incorporating the Product Owner role from Scrum into your organization requires new role definitions, new performance evaluation practices and criteria, new compensation systems, new communication and reporting mechanisms, new authority and accountability processes, etc. etc. All of the changes required are about creating Enterprise Agility throughout the whole organization, beyond just software or IT. These extensive changes are often started in a very ad hoc manner, but at some point they need to become systematic. This is an important decision point for executive management: are we going to be Pragmatic about our Enterprise Agile adoption, or are we going to be Transformative about our Enterprise Agile adoption.

I sometimes also call this the “Agile Ecosystem” since it is a constantly evolving set of ideas (processes, tools, resources) that does not have a clearly defined boundary. For example, the technique of Value Stream Mapping comes from Lean manufacturing but has also be broadly adopted by Agile practitioners.

Affiliated Promotions:

Try our automated online Scrum coach: Scrum Insight - free scores and basic advice, upgrade to get in-depth insight for your team. It takes between 8 and 11 minutes for each team member to fill in the survey, and your results are available immediately. Try it in your next retrospective.

The Agile Manifesto is the founding document of the Agile movement. It can be found at http://www.agilemanifesto.org and if you haven’t read it, it is strongly recommended! Living values and principles is an act of striving for excellence. There are no mechanisms in Scrum to force people to live the values and principles of the Agile Manifesto. Scrum relies on individual team members to strive to develop an understanding and practice of Agile values and principles in and of their own volition. If Team Members do not live the values and principles of the Agile Manifesto in their team many other things will take priority such as creating a complex document. The team could think of Scrum as a tool for project delivery, without really working to change the culture of their organization. Since Scrum empowers individuals and makes obstacles visible, if the team doesn’t live the Agile Manifesto principles then they may be disconnected from the roots of Scrum and make it a lesser version of itself, sometimes known as Scrumerfall (where you blend some elements of Scrum with other elements of waterfall which provides little to no benefit).

Affiliated Promotions:

Try our automated online Scrum coach: Scrum Insight - free scores and basic advice, upgrade to get in-depth insight for your team. It takes between 8 and 11 minutes for each team member to fill in the survey, and your results are available immediately. Try it in your next retrospective.