Getting the right project management answer, a useful answer, means asking the right questions of our people, metrics, analytics, and data mining efforts. Too many questions have no useful meaning and too many answers have no corresponding question that needed to be answered. Here is a real world example and a few key principles that got us the answers we needed and vaulted our projects on to success.

First, Ask A Question That Needs Answering

I did something that everyone said was useless and made no sense. I answered the question of how long was it taking to fix defects in our product. I just added up how long it took from when the defects were reported to when the defects were put in a product build and then divided by the total number of defects reported. I got an average. One well respected engineering manager retorted “That makes no sense! You are mixing large and small defects and defects from different parts of the product. You are averaging out apples and oranges and getting useless fruit cocktail!”

There was only one hitch with this manager’s theory (see theorizing vs managing in projects) — this average was very good at predicting when our defects got fixed. It was much better than the estimates, plans and projections found in all the PowerPoint slides from equally well respected engineers and managers. The numbers further showed that we would not be done for months (see defect reports can be our best friend), when all the “happy” slides said we were sure to be done by next week or maybe in two weeks. “Oh, no!” they said. “It won’t take months. We are the experts and have been managing these efforts longer than you have.”

What happened?

We finished up about four months later, when we finally got all the defects identified and fixed.

Make Sure Our Answer Is Really For The Question Asked

As it happened, the Quality department decided that they liked the idea of calculating how long it was taking to fix defects. So they computed their own number. It was huge. Much larger than what I had computed. Suddenly, the development teams liked my numbers! I liked what Quality was trying to do, so I had a few meetings with them to understand what we were doing differently.

Simply put, Quality was asking and answering the wrong question. For the next ten years — yes, ten years — Quality would continually get it wrong and the numbers they produced would never get into any real use nor help projects deliver on time. Instead they were relegated to being part of the “standard charts” produced on a Quality dashboard. These charts were never predictive of when defects would be fixed nor when the the project was to be completed.

What happened?

They never understood the concept of asking the right question.

What was the right question in this case?

It was very simple.

It was a question we asked every day. It was asked constantly. It was the source of great and often heated debates.

The question was “How long will it take to fix those defects that were just reported today?”

Simple yes? No, not really.

Quality would never answer this question. Even after discussing it with them, for years, they insisted that the question they asked and answered was the right one.

What did they answer? “How long has it taken to fix those defects we just fixed today?”

That is it.

Isn’t it the same question?

No. Not even close for our purposes.

Just Because We Can Compute A Number Doesn’t Mean It Is Useful

The first reality check is that no one, no customer, no senior manager, no product manager, and no project manager ever asked that question Quality was answering.

When these same folks were asked, as an experiment, what question they wanted me to answer (“how long will it take to fix today’s reported defects vs how long did it take to fix the now fixed defects in the product”) — I often got a “are you nuts” look and a repeat of wanting to know when the current defects we just identified today would get fixed.

The differences in calculations were simple. In my case, I’d take the defects that were reported found in any one day, and compute the average on how long it took to fix them.

Quality did it a bit differently. They took the defects that were reported fixed on any given day, and computed the average of how long it had been since they were reported.

There primary argument for taking this approach was that on any given day that we fixed defects (installed them in the product), we knew all the information we needed and could compute a number.

The problem is, they said, with my approach was that to determine how long it took to fix the defects reported today, we had to wait until they were all fixed to know a number. I agreed. (OK, so how would you do it? It is a bit tricky. Share your ideas in the comments.)

Quality was computing something they found easy to compute, rather than answering the question being asked.

Truly Answering A Key Question Can Have A Huge Impact

Asking the right question is not as easy as it sounds. Just because a question can be asked or an answer can be computed, doesn’t mean that it is useful. Pay attention to the questions that are asked in our project, and then try to see what information we have that allows us to answer that question. When we simply did that, we suddenly knew how long things were really taking, and we started meeting our promises, building customer confidence, and hitting our deadlines.

Are you producing any metrics or numbers that don’t seem to answer a question people ask everyday?

Thank you for sharing. Sharing and comments tells me which articles are most helpful to my readers:

3 thoughts on “How To Ask The Right Project Management Question And Get The Right Answer”

Comments from around the web:

Ramesh Kumar Keshvan • Hi Bruce, I was asked by a interviewer about the methodology used in deploying the Projects, i was unable to meet the expectation, He was looking for the standards available like PMI Standards or Critical Chain, or Prince 2, everyones expectation remains for us to answer the way they handle the Projects which is too critical and as everyone knows they are totally driven by situations. Thanks Ramesh

Bruce Benson • Ramesh,

I believe this is pretty normal. The other kind of interview I often encounter is the one where they ask questions to see if you are just like the person who just left and they want someone exactly like that person.

I do remind people that, just like in a project, we need to deliver what the customer expects. While I know that having PMI knowledge does not guarantee success (nor a good PM) I do know it makes the customer comfortable and confident. Once I have the job, I then seek to exceed their expectations and they really have no idea what concepts made the difference (PMI, lean, agile, hard work, experience, etc.).

Good luck in all your future interviews.

Bruce

Paul,

We calculated the average for each day as well as a 30 day rolling average. What we saw is that the speed of fixing defects usually started out pretty slow at the beginning of a project (averaged weeks), hit its fastest fix rate during the peak reporting period for defects (fixes averaged about 7 days, some short periods of 5 days, one project got an average fix rate to 3 days – but had lousy quality), and then slowed down a bit around the time the project was complete (product released, software went live). These were big projects that ended up with about 15,000 reported defects over their development.

What this meant in part was that basing the fix rate on when a defect comes in tells us about what is happening today (on that day) and probably tomorrow – which it did well.

Quality based it upon the build, and with every build we’d have many defects that were from earlier in the project, up to 12 months ago. So that number didn’t tell us anything about how quickly we could fix defects coming in today or tomorrow.

Thanks for the ideas.

Bruce

Paul Boossays:

I think they could get closer to your answer if they did their calculation over a number of days (you’d need to determine the right sample size) and then averaged.

They certainly sound like they should be the same, but they aren’t the same question. Theirs is a statistically insignificant sample. Yours is a sample of random events where the events are known to be deviating around the average (though you don’t know the amount of deviation until they complete).