28 August 2013

SIP math and Monte Carlo Simulation

Thinking about the contrast between SIP math and MCS.

SIPmath has its roots in Monte Carlo Simulation, but the implementation, application architecture, and data architecture are different. MCS munges generation, data, and use together. SIPmath extracts the data part and puts it in a SIP where, being pure data, it can be cataloged, and passed around as easily as you attach a picture to an email. How it's generated and how it's used are separate concerns.

Under ideal conditions, where there's lots of data, each value in a SIP is valid because it has actually happened, and the frequencies in the SIP are the same as the reality. That is, a well-formed SIP is correct by construction. Since the rest is simple arithmetic, avoiding implementation errors and independent validation are both fairly simple.

MCS stratified sampling and SIPmath are the same except for where in the workflow the samples are taken.

On the other hand, MCS generating random values from a curve that approximates the data, is approximate by construction.  We can only hope to get close to the fidelity that comes effortlessly in a SIP composed from history.

08 June 2013

The Flaw of Expected Values

No matter how well-managed they are, projects tend to finish late and over budget. We keep doing things to correct this problem, but project failure rates have remained constant for decades.

It turns out that one of the reasons, perhaps the principal reason, is that the math we use for estimating project cost and duration is fatally flawed; it gives us consistently optimistic estimates.

The fatal flaw is the Flaw of Averages, eloquently described in Sam Savage's book of the same name.

In project planning and estimating terms, that's the Flaw of Expected Values.

09 March 2013

The Expected Finish Isn't

Conventional planning tools produce one or more expected values -- expected finish, expected cost.

"Expected value" is also known as the average or mean. But, an average over what? An average assumes a bunch of things whose values can be added up. Average time or average cost implies a large number of activities whose cost and duration can be averaged.

It also implies that the activities that finish early and below budget will provide the savings to underwrite the activities that finish late and over budget.

More generally, if the calculation of expected value is a valid calculation, the sum of the actual costs of a large number of activities should be close to the sum of their expected costs. Is this what happens in the real world?

Silly question -- it doesn't. Relative to expected values, task and project finishes range from a little early to a lot late, slightly under-budget to major overrun. The sum of the actuals is inevitably greater than the sum of the averages.

A sure sign of insanity is doing the same thing over and over again expecting a different result.