Problems and Risks with Estimating Software Size

The following sections explain why software project managers and developers have such an excessive problem with estimating their tasks, and why the estimates they do provide are so filled with risk.

The Problem with Estimating

Software Engineers are notoriously poor estimators. We know this because 15 percent of all software projects fail to meet their goals and overruns of 100 - 200 percent are common in software projects. As an industry, we are not poor performers, although it may seem that way, because the prediction of performance is poor. There are many reasons why we set unreasonable expectations, and few will deny that estimating is a difficult task.

When performance doesn't meet the estimate, there are two possible causes: poor performance or poor estimates. In the software world, we have sufficient evidence that our estimates stink, but almost no evidence that people in general don't work hard enough or intelligently enough.

- Tom DeMarco, Why Does Software Cost So Much?

DeMarco is right. Developers are often blamed for being aggressive, unresponsive and uncaring toward the customer, unable to do the job, or just plain lazy. Most of the time, none of these accusations is true. The truth is that developers have been asked to do the impossible. Correcting this impression relies on what DeMarco calls the management of expectations. And it all begins with the estimation of software size.

It's not an easy task, though. Among the various reasons estimating is so complex:

● The problem may not be well understood by developers and/or customers; either or both may be missing facts or might have them distorted by unsubstantiated opinions and biases.

● There is little or no historical data upon which to base future estimates.

● Estimators may become frustrated when trying to explain how big a system will be before it is built - maybe even before it is designed.

● The developing organization has no standard estimating process (or, where standards exist, they are not followed), resulting in a lack of consistency in the estimation process.

● Management and/or customer demand quick estimates and possibly misunderstand that code can be written right away, skipping analysis and design steps.

● Stakeholders believe that requirements can be fixed at the beginning of a project, or customers believe that they cannot afford time for requirements specification.

● Management uses estimates as performance or motivational goals.

● Developers are optimistic and desire to please their management, peers, and customers.

● Mistakes are hidden rather than being reported and evaluated, resulting in false impressions of past performance.

● Assessment abilities are impaired by egos involved in performance.

● Early estimates and schedules are viewed as performance goals; estimates are made prematurely, before the concept exploration is complete; and confusion arises between desired versus realistic schedules.

● Reluctance to re-estimate arises.

● Insufficient visibility into other parts of the system (legacy system interfaces, hardware, etc.) may require additional software and therefore may affect size; incomplete understanding of system-level constraints or special system limitations exists.

It is little wonder that the very complexity and difficulty of predicting software size culminates in project risks.

The Risks of Estimating

The general principles of software project management risk will be discussed in "Determining Project Risks". Many of the typical software risks originate with estimates of size, effort, cost, and schedule, which are based on the "problem with estimating". Here is a brief review of some risks resulting from the problem, along with a brief set of suggestions for mitigating those risks.

When incomplete or incorrect estimates are given, there is an obvious risk of disappointing the customer, possibly losing future business. Equally serious is the misestimate of a fixed-price contract - a too-optimistic estimate will result in the contractor losing money (perhaps a great deal) as well as losing face.

Inaccurate estimates will require adjustments to the schedule, to squeeze the optimal one into a shorter time frame, which almost always results in the introduction of defects.

Some Suggestions to Mitigate Size-Related Risks

There are ways to reduce the effects of size estimation risks, all of which are good project management practices in any case:

● Produce a WBS, decomposed to the lowest level possible, to use in a "divide and conquer" approach; small components are easier to estimate.

● Review assumptions with all stakeholders, including operations, maintenance, and support departments.

● Wherever possible, do the research into past organizational experiences instead of just guessing. This can frequently be done in the absence of historical data. Anecdotal evidence can be illuminating.

● Stay in close communication, using a common language, with developers working on other components of the system (perhaps in parallel).

● Update estimates at frequent intervals, as shown in Figure (a). Estimation accuracy does improve over the course of the life cycle.

● Use multiple size estimating methods to increase confidence.

● Educate software development staff in estimation methods.

Both Boehm and Capers Jones have published graphs that are astonishing in the possible degree of estimating inaccuracy in the early stages of the life cycle - specifically because these are the figures upon which we generally base initial project go/no-go decisions. When customers ask for ballpark estimates, they deserve to know that the "ballpark" is not terribly accurate - it is generally in the range of +/ 35% above or below the final actual size/effort/schedule/cost (estimates are too low more frequently than they are too high). Again, this phenomenon is demonstrated in Figure (a).

The contents available on this website are copyrighted by TechPlus unless otherwise indicated. All rights are reserved by TechPlus, and content may not be reproduced, published, or transferred in any form or by any means, except with the prior written permission of TechPlus.