Published here December 2003. 

PART II | Reason 3 | Shareholder Value | Stakeholder Value
Finding the Right Metrics | Metrics as "Observables" and the Clairvoyant Test
Don't Forget Financial Metrics | Be Careful Using Balanced Scorecards
Each Organization Needs Its Own Metrics | PART IV

Metrics as "Observables" and the Clairvoyant Test

To the extent possible, metrics should be observables; that is, characteristics of projects or project outcomes that can be observed and measured in the real world. Since estimating project value requires forecasting the future, metrics don't, obviously, all have to be things we can observe today. Metrics can, for example, include a projected future state of some observable, for example, an improvement in a reliability-of-service statistic important to customer satisfaction.

A useful device for checking whether a metric is observable is the so-called "clairvoyant test" devised by my college mentor, Professor Ron Howard. Before accepting what appears to be a good metric, consider whether a clairvoyant could give an unequivocal value for that metric given that a project decision is made in a specific way. Oftentimes, the clairvoyant test points out inexactness of what initially appears to be a well-defined metric. For example, "customer satisfaction" doesn't pass the clairvoyant test. However, "percent reduction in recorded customer complaints" and "company ranking in the next industry customer satisfaction survey" are metrics that do pass the test.

Metrics that don't pass the clairvoyant test are vague. They create inconsistency and imprecision when used for estimating. More importantly, if the metrics are not observables, they cannot be monitored so that actual values can be compared against estimates. Observable metrics allow project proponents to be held accountable for achieving estimates submitted as part of project proposals, which is essential for minimizing biases and gaming in project forecasts.

Finding the Right Metrics  Finding the Right Metrics

Home | Issacons | PM Glossary | Papers & Books | Max's Musings
Guest Articles | Contact Info | Search My Site | Site Map | Top of Page