Agile Testing Days: Deception and Estimation: How We Fool Ourselves

The second keynote at the Agile Testing Days was presented from Linda Rising on Hardwired for deception means trouble with estimates. This is the write-up of her session.

Linda Rising stated that she attended some talks in the morning, and she felt that she heard those talks before – maybe twenty, thirty years ago. She commented on the Agile manifesto, that states software is created by people, but it rather should be software is created by people’s brain.

Rising introduced the definition of deception as consciously or unconsciously leading another or yourself to believe something that is not true. In short, her message was: we naturally deceive ourselves and others – constantly.

Rising made the point that insurance companies are betting that they are spending money based on an assumption how long someone will be living. Controversy she mentioned that we estimate that we will live about ten years longer than estimated. A similar problem we have is with statistics. Current figures state that the chance for a marriage to hold is currently 50-50, but who is telling himself in front of the wedding altar thinks about having a 50-50 chance?

Rising claimed that people are hardwired to be optimistic, to see what we want to see, to refuse to think about or process information we do no like. She explained that decisions are made based upon emotions though we might think that we’re deciding rationally. Our hard-wiring goes as far as to distort reality so that it fits our view of the world. The information we decide to take in, is already filtered by our biases.We deceive ourselves at the end of the day with a “rational” argument about it. And of course we believe that we’re not biased in any sense. This starts with information intake already.

She cited Max Planck with

A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.

She remembered a reviewer comment on one of her papers on mathematics: “I wouldn’t believe this, even if it were true.” Semmelweis made the point after researching doctors, and noticing a lower death rate once doctors washed their hands between surgeries. He was locked away, since his point was not accepted at that time.

Rising mentioned a study, where participants where given positive information first, then negative information in a series of experiments. In another set of the same series, people were given the negative information first, and the positive information later. She continued then by asking the audience for opinions what would happen. Those who received the positive information first, gave significantly higher predictions of success, while the tendency was exaggerated when subjects were asked to make their judgments against the clock. She bridged the gap to estimations under pressure.

Rising cited a survey about drivers, where most of them said they were above average. She quoted an survey on online dating participants tend to exaggerate about age and weight as well as in the recreational activities like extreme sports. This fact holds for 90% of the participants.

Further Rising explained that we believe that we improve in order to keep our selfish, lacking judgement. Parents are teaching this deceive to their children. She raised the example that children shall lie when they get a present, and have to tell the auntie that they like the gift. Another example was the question from a wife “Does this new dress make me look fat?, which most husbands have heard one time or the other.

On size estimates Rising explained that humans get tricked by verticals. Having a thin, tall glass looks more full than a smaller but broader one. Even our own bodies trick us. We eat more from larger container or when served larger portions. We eat more from all-you-can eat buffets. Names and presentation of food distorts test.

Linda Rising told the story how she received the 9/11 incidents. She was at an airport, and got no information why her flight was not going. She was sent home, and now has a very different look back on things, even in the picture of deceiving yourself. Stating the happenings after 9/11 she said that air travel decreased by 20% afterwards. Many people estimated that their chances of survival were better in a car than in a plane. Since people were more getting in their cars increasing driving cars on the roads, on year after 9/11 the number of killed pedestrians by cars has reached three times the number of casualties from the four planes on 9/11.

Another example Rising told was the one from smokers. Smokers live five years shorter, while frequent flying may yield an average of one day earlier death. Still most smokers will state that they will be dying more probably by flying rather than smoking.

We are hardwired to fear what we cannot control. That’s why we are hardwired to distort information. As a result we tend to overestimate our ability to do anything: code, test, solve problems, … take your pick. Rising raised the point that data also does not help in this regard. It still does not work to simply collect data. She cited a Harvard research on Brook’s Law called “The Experience Trap”. Managers still haven’t learned the lessons from this decade old books. In the debrief managers said that they made this mistake and it won’T happen again. Going into another iteration, they did the same mistake again.

Rising bridged the gap to Agile working. The only way to achieve estimates that are “good enough” is to take small steps. Experiment and learn from failure as well as success. You must involve others because you will deceive yourself about your own estimates. Agile is a multi-legged stool. You can’t fake it. In small steps, you must include all the elements.

2 thoughts on “Agile Testing Days: Deception and Estimation: How We Fool Ourselves”

Interesting. This keeps me in mind of James Lyndsay’s talk at the SIGiST conference in September in London where he talked on the Irrational Tester and how easy it is to let our biases and personal prejudices get the better of us when we are testing.

Rising certainly makes a valid point about estimation. According to my earlier agile estimates I would be one of the most productive developers out there.
Generally when estimating we are using a best case scenario, where we expect that not only are we coding without error, but everyone else is and all the components my code has to integrate with has no bugs. I guess this goes right back to our fear of loosing control mentioned in your blogpost.
The unfortunate reality is that there are generally only a few best case development scenarios but a much larger amount of development scenarios that will not allow us to finish development within the initial estimate. We simply do not have the capacity to anticipate everyone else’s errors and bugs and if we could we would simply end up estimating the worst case scenario.
As Rising did mention, estimation makes sense only for small pieces of work. I started breaking down requirements into smaller tasks that I feel comfortable to estimate. Now I’m not a the most productive developer, but at least I’m delivering (mostly) on time.