Agile practices now have research support

Adam Smith was wrong. Well … he was not wrong in his conclusion but he was partially wrong in his basic assumption that human always pursue their self-interest.

Through the work of many scientists, we have begun to see evidence across several disciplines that people are in fact more cooperative and selfless—or behave far less selfishly—than we have assumed. In fact, recent research shows that in any society majority of us behave cooperatively rather than selfishly (though some people do behave selfishly).

The essence of agile is iterative development and self-organizing team (What makes Agile agile?). Latest research suggests that iterative approach with trial and error is the best way to navigate through our environment which has become exceedingly complex. Such research is inspired by biology and evolution.

Now you have research evidence that we are indeed tuned to work like that. Here are the references to 2 books and couple of article which highlight the latest development in this field.

However, what surprises me most is that there not a single reference to agile manifesto or agile development methodology in any of them. All of them invariably refer to Wikipedia and Open Source movement but I have not seen any similar publication referring to Agile.

The Unselfish Gene

Ted Cadsby, in an article in HBR, neatly summarizes the status of research in this field. He points out the following:

In experiment about cooperative behavior – in no society examined under controlled conditions have the majority of people consistently behaved selfishly. Most organizations would be better off helping us to engage and embrace our collaborative, generous sentiments than assuming that we are driven purely by self-interest.

Wikipedia and open source movement – people contribute because they want to cooperate. The interpretation that the voluntary contributions of participants are an attempt to improve their reputations and long-term employment prospects has been refuted by the decade of empirical research.

Cognitively and emotionally, we may be able to “feel” what others are feeling – Neuroscience also shows that a reward circuit is triggered in our brains when we cooperate with one another, and that provides a scientific basis for saying that at least some people want to cooperate, given a choice, because it feels good.

Agile proponents have always maintained that “Self-organization” through “Collaboration” and “Trust” is the key to successful software development. If you look at these 3 of the 12 principles behind agile manifesto, you will notice that it revolves around trust, face-to-face conversation, motivated individuals and self-organizing teams.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

The best architectures, requirements, and designs emerge from self-organizing teams.

The article goes on to point out 7 building blocks of cooperative systems. The first building block is Communication – in agile manifesto the first item is just that “Individuals and interactions”.

There is another article by the same author which emphasizes that many of our current problems are too complex to be reduced to a single-cause explanation. Complexity arises from the interconnections between things — how parts within a system interact via intricate feedback mechanisms. The information signals we need to make sense of complex things are buried in a lot of noise, and we, unfortunately, are not adept at digging for cues.

Adapt: Why Success Always Starts With Failure

This book is by Tim Hartford, the author of “The Undercover Economist”. If you have not read the book the here is a nice summarization of the key ideas. There are references to numerous recent researches in this field. The main ideas brought forward in the book are:

Astounding complexity emerges in response to a simple process: try out a few variants on what you already have, weed out the failures, copy the success – and repeat forever. Trial and error is a tremendously powerful process for solving problems in a complex world.

Given that life is so unpredictable, what seemed initially like an inferior option may turn out to be exactly the solution we need. It’s sensible in many areas in life to leave room for exploring parallel possibilities.

In addition to these the author makes a case for making systems loosely coupled so that failures can be tolerated. This will ensure that the cost of one failure is not too much and it does not bring down everything with it.

The key to Agile is “iteration”. The third principle states “Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.” The emphasis is on doing – learning – adapting.

The Origin of Wealth: Evolution, Complexity, and the Radical Remaking Of Economics

This book, written by Eric Beinhocker, is rich in survey of new ideas in economics, incorporating discoveries from science and psychology. This presentation highlights the key ideas in the book. The key premise in the book is that our economy is a complex, adaptive system which implies that:

Equilibrium, and with it much of the economic thinking that has dominated the past century, is out.

A new tool set with its own techniques and theories are available to explain economic phenomena.

Wealth is a product of evolutionary processes.

This book also emphasizes on the need to iteration and self-organization.

Here is a diagram from the presentation showing how cycles have evolved.