08 June 2013

The Flaw of Expected Values

No matter how well-managed they are, projects tend to finish late and over budget. We keep doing things to correct this problem, but project failure rates have remained constant for decades.

It turns out that one of the reasons, perhaps the principal reason, is that the math we use for estimating project cost and duration is fatally flawed; it gives us consistently optimistic estimates.

The fatal flaw is the Flaw of Averages, eloquently described in Sam Savage's book of the same name.

In project planning and estimating terms, that's the Flaw of Expected Values.

Whether it’s about a whole project or a single task, “How long will it take?” doesn’t have just one answer. It has a whole bunch of answers, each with its own probability of being right.

One of those many answers is the expected value - the theoretical average over theoretically many similar projects.

Here's the irony:

If you are really good at estimation and your expected values are reliable, about half your projects should finish late and over budget. Why would you launch a project knowing it has a 50% probability of failing?

That's what "average" or "expected" means; about half the time the number will be smaller and about half the time it will be bigger.

That's not all. The time and cost to do a task have fairly firm minimums, but no maximum short of abandoning the effort. This skews the possibilities toward the late and over-budget side.

And then there's the concurrency tax.

Suppose you have a project with ten concurrent tasks, each with an expected duration of six months. CPM will wrongly estimate that the project will be done in six months.

Since these are all expected finishes, each one has a 50% probability of being late. Heads, we're early, tails, we're late. All of them coming in on time or early is equivalent to tossing ten heads in a row - one chance in 1,024. So the expected project duration should be greater than six months. The question is, "How much greater?" That's a question you can't answer if all you have is expected values.

When you choose a target, or make a resourcing decision, you also choose the probability of success (or the chance of failure if the glass is half-empty). Whether you’re conscious of the choice or not, you’re making a risk management decision. Making the choice conscious and informed is where Probability Management and SIPmath comes in.

SIPmath is not a proprietary "methodology" or product. It's an easily-applied way of quantifying and calculating uncertainty that's simple enough to do in Excel. It's open standards, open-source code, a growing library of resources, and a non-profit organization to promote and develop the tools and techniques.

A planned cost or duration doesn't have just one value. It has many possible values, each with its own probability of becoming the actual. This uncertainty – many possible values, each with its own probability – applies to every value in the project model.

With SIPmath, we calculate not with single expected values, but many values. What CPM does once, we do hundreds or thousands of times. We simulate many different mixes of possible inputs, producing many possible outcomes and probabilities.

You still have to commit to plan numbers for cost and duration, if only to coordinate effort. Using SIPmath you can choose your plan numbers with a clear understanding of your odds of success. Learn more at sipmath.org .

No comments:

Post a Comment