This stage of the planning process comes under different names: Strategy, Project Proposal, Business Case, Project Charter, ... The names tend to reflect the degree of approval obtained or being sought, and the kind of detail expected. In any case, this is the first place in the planning process that the planned activities, projected costs, and timelines come together for stakeholder inspection and decision-making. It includes a plan, but probably not at the level of detail used for managing the project.

"How long will it take?" doesn't have just one answer. It has a bunch of them, each with its own probability of being right. Which of the many values we commit to is a *risk management choice*, not a discovery.

When we give a decision maker an estimate that something will cost $100,000, or take six months to do, or that annual sales will be 20,000 units, we're leaving out an important piece of information: the probability of meeting or improving on that estimate--the probability of success. We're also not telling them what a change in the estimate would do to that probability. In other words, we're not showing the full range of choices in our estimates; we're usurping the decision maker's authority to decide what they'll accept as the odds of success or failure.

Probabilities can be more useful than estimates, and they can have a dramatic impact on our understanding of the situation.

Which is more important for you to know: that the project is estimated to take 100 days, or that, with that deadline, the odds are 4:1 against success?

Giving decision makers a complete picture of estimate uncertainties is vital to effective planning. It'll take some extra tools:

- tools for presenting uncertain data,
- tools for modeling and calculating with uncertainty, and
- tools for capturing and quantifying input uncertainty.

All of these will need to work with probability distributions, what Professor Savage calls a Stochastic Information Package and another statistician might call a "randomized uniform partition" We'll abbreviate it to just "distribution" except when we mean something else. For any particular measure, a distribution quantifies uncertainty, makes the connection between possible values and their probabilities, and makes Monte Carlo Simulation dead-simple.

## Tool #1: Enhanced Presentation

A distribution is a large vector of randomly ordered numbers that no one wants to look at directly. The best way to present a distribution is graphically, and the toolkit will include a variety of graphics optimized for presentation to stakeholders. Some will be static. Others will be interactive and let us change assumptions to see how the results are affected.

Nothing else we do will be of any value unless the model and its results can be presented in a way that's intuitive and quickly grasped by decision makers. Sheets of numbers and formulas won't do.

## Tool #2: Monte Carlo Simulation

Planning with distributions is most easily, and simply, done with modeling and Monte Carlo Simulation, taking full advantage of the computer power available to us. The key feature of this approach is that *uncertainty in the model inputs is reflected throughout the model calculations* and no information is lost in the process. Moreover, the math is four-function calculator level, so there's no mystery about how the results are produced.

This can be done with not too much difficulty with a spreadsheet such as Excel, and there are a variety of add-ins to MS Excel and MS Project that make it easier. There are also dedicated applications and programming platforms with useful functions built in. The prices range from free to thousands of dollars per seat. In the posts that follow, I'll be using plain Excel, sometimes augmented with the XLSim add-in.

## Tool #3: Sampling History

The best source of information for our inputs is quantified experience. If we have the records, we can examine similar activities and use what we learn to build a suitable distribution. If there's a lot of history, we can use a resampling approach; if not, we'll need an informed estimate.

## Tool #4: Calibrated Estimators

Where there isn't usable history, we need subject matter experts (SMEs) to develop an estimated distribution using whatever hard information is available, coupled with their expertise as estimators.

Two problems with using SME estimates is that, without training, SMEs tend to produce biased estimates and to suffer from overconfidence in the accuracy of these estimates. Fortunately, training techniques are available to correct both of these problems, producing "calibrated estimators" whose estimates can be relied on.

## The Bottom Line

There is uncertainty in the data we use as inputs to our models that can't just be waved away. This input uncertainty translates into uncertain results. Fuzz in, fuzz out.

For the Strategy to give decision makers the information they need, we need to show them the possibilities and probabilities, so that they can make informed goal setting and risk management decisions.

Meeting this requirement will provide more than ample fodder for future posts.

This is great.

ReplyDeleteI often use the analogy of a weather forecaster saying, "Tomorrow it will rain." without specifiyin 20% chance thereof.

The prediction/forecast is wrong unless there is a confidence level associated with it.

Tony

In the case of a weather forecast, you're probably wrong anyway.

ReplyDeleteBut it is a good analogy.