Search form

estimation

When dealing with stakeholders who do not have much experience in project management we inevitably encounter a situation where we have to provide “quick” estimates early in the project life cycle. The science of project management encourages us to use plus-minus estimates, ranges and coarse estimates (see more on the topic: Five Proper Ways to Present Project Estimates).

However, usually the stakeholders are very keen to hear very precise numbers like “the project duration is going to be 9 months and 3 days” or “the total budget is going to be $245,433”.

So, how do you explain to your customers that as soon as you started generating project estimates (especially the early ones), you have entered the domain of uncertainty?

I usually like to demonstrate the principle of uncertainty by picking a simple example completely unrelated to the project at hand and dissect it piece by piece right in front of their eyes.

Will the customer want Feature X?

In other words, will the customer decide to add Feature X to the dozens (hundreds, thousands) of other features already added to the project scope?

Example: Will the customer decide to add the “Landscaping” feature to the “Build a New Home” project?

Will the customer want the “Honda Civic” or “Ferrari” version of Feature X?

Will the customer prefer to go with a cheaper and simpler version of the chosen feature, or opt for a luxury kind?

If you implement the “Honda Civic” version of Feature X, will the customer later change his mind and demand the “Ferrari” version after all?

Example: I am sure it never happened on your projects, but is there a chance that the customer who initially decided to go with ten Eastern White Pines, will change her mind and opt for fifty Little Gem Magnolias?

How often have you been in a situation when your manager (customer, stakeholder, etc.) taps you on the shoulder and says something to the effect of:

Hey, Bob, I have this little project for you. Here is what I need (proceeds to describe scope at a very high level). How long do you think it would take your team to deliver this? Can you provide me with your estimate by the end of the day (week)?

Every project manager is aware of the fact that as soon as we said “project estimate”, we, whether we like it or not, said “uncertainty”. So here is a million-dollar question:

How to provide quick estimates when very little is known about the final product?

Option 1: Plus-Minus Qualifiers

Example: This project should take six plus/minus two months

Duration = 6 months ± 2 months

Option 2: Ranges

Example: This project should take anywhere between four and eight months

Duration = 4 months – 8 months

Option 3: Cases

Example:

Best case – 4 months

Most likely – 6 months

Worst case – 8 months

Option 4: Coarse Dates and Time Periods

Example: “3 Quarters” instead of “9 months” (or 180 days)

Duration = 3 Quarters

Option 5: Confidence Factors

Example: “We are 90% sure that the project will be done in between 90 and 120 days”

And how do you present your project estimates? Please leave your comments below.

I recently came across the “Olympic Proportions: Cost and Cost Overrun at the Olympics 1960-2012” article published by Bent Flyvbjerg in 2012. One of the figures that really got my attention was a table describing the actual costs and budget overruns of the Olympic Games between 1968 and 2012 (see Table 1).

There are currently three theories attempting to explain our helplessness when it comes to accurate project estimation. In a nutshell, here are explanations by three different schools of thought:

The Standard Economic Theory Explanation - If companies take rational risks in order to earn abnormal incomes, these poor outcomes are inevitable. One of the key laws of the financial theory states that in a perfect market, the higher the expected return of an asset, the higher is the inherent risk associated with it.

The "Mass Delusion" Explanation - Modern business decision making is seriously flawed because of the delusional optimism (optimism bias) that forces people to overestimate the benefits and underestimate the costs of future projects.

The Machiavelli Factor Explanation – executives and politicians are deliberately cooking the books in order to make the project proposals look more attractive. In addition, executives are rewarded with heavy incentives for rosy forecasts and face very minor penalties when their predictions proved to be wrong.

I personally tend to lean towards the blend of theories #2 and #3. In my experience the following happens:

Introduction

The project to deliver the most expensive warship in the world is $2.3 billion over the budget and 2 years (and counting) late. The US Navy’s newest $13 billion aircraft carrier is still not ready for combat because of mechanical delays that have already put it two years behind schedule, according to the Pentagon’s top weapons tester.

The USS Gerald R. Ford (see Figure 1 for more info) was supposed to be ready by September 2016, but Michael Gilmore, the Defense Department’s director of operational test and evaluation, said in a June 28 memo that the warship had ongoing launch and recovery problems.

Figure 1

The construction of the ship started in November 13, 2009 and is still ongoing. Click on the video below to watch the time lapse.

Introduction

The Kashagan Oil Field has been discovered in 2000 in the Northern part of the Caspian Sea (see Figure 1). It was quickly confirmed that it was the world's largest discovery in the last 30 years, combined with the Tengiz Field located nearby, with a projected output close to that of the Ghawar field in Saudi Arabia. The recoverable oil reserves of the Kashagan Filed were estimated to be at 19-20 billion barrels.

Figure 1

The consortium consisting of several global energy players along with the government of Kazakhstan was formed in order to develop the reserves. The number of players and the make-up of the conglomerate changed several times over project lifecycle (we will come back to this fact at a later time), but as of 2012 the group consisted of the following players:

Eni (Italy) - 16.81% stake

KazMunayGas (Kazakhstan) - 16.81% stake

Royal Dutch Shell (UK/Netherlands) - 16.81% stake

Total S.A. (France) - 16.81% stake

ExxonMobil (USA) - 16.81% stake

China National Petroleum Corporation (China) - 8.4%

Inpex (Japan) - 7.56% stake

The Project

The project started in 2001 with an expected completion date of 2005. The allocated budget for this venture was US$10 billion. The oilfield was expected to become operational around 2005 and produce anywhere between 90,000 and 370,000 barrels (at peak production) a day. Needless to say, this project was viewed in Kazakhstan as one of the most important initiatives for the young country and was expected to generate considerable income for the government.

However the project fell 8 years behind the schedule and ended up costing a bit more than the original forecast. Depending on who one chooses to believe, the cost of the venture as of 2015 was either US$50 billion (official number provided by the government of Kazakhstan), US$115 billion (CNN Money's estimate) , or close to US$150 billion (number being mentioned in some publications).

Why Your IT Project May Be Riskier Than You Think

I am a big fan of professor Flyvbjerg and his work and as a result needless to say I read this article with great interest. The article describes the phenomenon called the "black swan projects" or as he puts it "high-impact events that are rare and unpredictable but in retrospect seem not so improbable". In other words, these are the endeavours that are vastly over budget and sometimes so bad, that they kill the companies that conceived them. Some of the examples are:

"The average overrun was 27%—but that figure masks a far more alarming one. Graphing the projects’ budget overruns reveals a “fat tail”—a large number of gigantic overages. Fully one in six of the projects we studied was a black swan, with a cost overrun of 200%, on average, and a schedule overrun of almost 70%. This highlights the true pitfall of IT change initiatives: It’s not that they’re particularly prone to high cost overruns on average, as management consultants and academic studies have previously suggested. It’s that an unusually large proportion of them incur massive overages - that is, there are a disproportionate number of black swans."

The article proposes the following steps to be taken in order to address this challenge:

One of the prevailing superstitions in project management is the "fixed pie" assumption. In other words, both sides assume that the project results, speaking mathematically, are binary - either the team delivers the project on time or it doesn't; either the project is on budget or over, etc. Fortunately, negotiations are typically not like NBA Finals when team A meets team B in a seven-game series where the winner gets the Larry O'Brien Championship Trophy and the loser goes home empty-handed.

In my experience, situations on most projects are similar to the fable involving two kids quarrelling over the ownership of an orange. Finally, their father enters the room and, operating under the fixed pie assumption, cuts the orange in two equal halves distributing the fruit between the brother and the sister. Interestingly enough, the brother eats the orange and throws away the peel. The sister uses the peel from her half as an ingredient in pastry while disposing of the fruit.

The situations between customers and project teams are often similar to the fable described previously: the underlying interests, constraints and risk tolerances of both parties are rarely identical. The proverbial pie usually looks quite different to each party. Hence, a good project manager can increase the size of the pie by looking for things that are of low cost to him and his team and high value to the customers (and vice versa).

Consider the following interaction between an experienced construction project manager and a customer:

Customer: “I would like to add another clause to our contract. If the work on the new mall is not finished by the deadline in the contract, I want your company to pay a penalty of $5,000,000”

PM: “Hmm, we have already signed the contract without the late penalty clause; I am not sure how our management would react to that …”

Customer: “I am sorry, but I have been instructed by my boss not to proceed ahead without this modification to the contract”

PM: “And may I ask you why you guys feel the need to add this clause?”

Sheila: You know about the problems we had with the release 4.0 of this product. You guys were late by 3 months, went almost 50% over budget and delivered way less features that we expected. Besides even the stuff you did deliver had serious quality issues.

Aha! So the previous similar project has been a failure on all three fronts: late, over budget and delivered less scope than expected. I wonder if optimistic estimation had anything to do with that?

Dan: Yes, and because of all these problems we lost two of our customers and several others warned us not to contact them until we have e-Merchant 5.0 ready with all the necessary features …

And these failures are beginning to have a strategic impact on the company's well-being ...

John: Yes, I understand your concerns and that is why we decided to invest a bit more time in scope definition in order to be able to better estimate project size, risks and duration.

Sounds like a very smart move to me. After all it is impossible to come up with any meaningful estimates until one knows the scope of work.

Dan: How much time are you planning to spend on requirements gathering?

John: We estimated that we would need about four weeks to elicit and document all the requirements and another week to for the team to conduct document inspections and generate estimates.

As a part of my project and portfolio management consulting practice I frequently get involved in advising various companies on how to run their real-life projects. Several years ago I was requested by a CEO of one organization to sit in a meeting where several stakeholders were supposed to discuss various aspects of the new "do-or-die" project. The ensuing conversation was so hilarious, that I excused myself from the room as soon as I could and recorded the entire exchange before it escaped my mind.

So, my challenge to you: how many estimation/negotiation handling mistakes were you able to spot in the following conversation?

John the project manager at the ABC Software Inc. a producer of e-Commerce software was preparing for a meeting with Dan, a VP of Sales and one of the original founders of the company and Sheila, a Senior Product Director. They were supposed to discuss the next major release of the company’s e-Merchant product.

Sheila: You know about the problems we had with the release 4.0 of this product. You guys were late by 3 months, went almost 50% over budget and delivered way less features that we expected. Besides even the stuff you did deliver had serious quality issues.

Dan: Yes, and because of all these problems we lost two of our customers and several others warned us not to contact them until we have e-Merchant 5.0 ready with all the necessary features …

John: Yes, I understand your concerns and that is why we decided to invest a bit more time in scope definition in order to be able to better estimate project size, risks and duration.

Dan: How much time are you planning to spend on requirements gathering?