Many project managers do not conduct a post implementation review that is essential for establishing the overall success, failures, challenges and lessons learned. For those who do, it is usually within a month or two of completion of the project and usually focuses on the traditional metrics, success/failures and how the project team did in performing the project implementation. However, for many projects, including IT projects, this may be insufficient time to get a clear picture of the project's business success or failure.
To illustrate how time plays a part in reflecting holistic project success/failure metrics, consider the track record of the Empire State building (ESB).
"The building was the brain-child of John J. Raskob, the vice-president of General Motors, who wanted this new building to exceed the height of the rival car manufacturer's Chrysler Building, still under construction when the plans were released on August 29, 1929. The program given to the architects called for a tight schedule of completion one and a half years after the start of the project."
"The Empire State Building in New York City was completed '[in] One year and 45 days... (ahead of schedule); Cost $40,948,900 (including land). The Building Alone [cost] $24,718,000 (the onset of the depression halved the anticipate cost of the building.)'"
So the ESB was completed ahead of schedule, under budget, and was to the specifications as designed. If the measurement were to be done on traditional PM metrics the project would be a complete success! However, if the metrics also looked at rented space, it would tell a completely different story. For the very reason that it came in at half the production cost (the great depression), rental rates at the building's opening was a meager 20%. In fact it was nicknamed the "Empty State Building." So if measured on rented space on completion of the project, it would be a failure. But let's expand the time-line. You have to go to 1948 or 17 years later for the building to have enough tenants to turn a profit. Yet, today it is again the tallest building in New York, has always been the icon for New York and, as of 2002, was 97% occupied.
That example brings up an important point. Success metrics may have to be monitored over a lengthy period of time to determine the true success/failure of the project, something that may not be immediately apparent shortly after completion.
This Time factor of measurement is very import, as many projects are either creating something new, or implementing a new process or/and system. Metrics that look at many of the goals of a system need time to realize their true impact, "...despite management's introduction of an extensive set of organizational change initiatives, managerial goals of improved flexibility and responsiveness are not immediately attained."
One last important metric is "usage". Whether in construction, like the Empire State building described above, or a computer system, it is the beneficial use of the product that ultimately determines the success of most projects. Therefore, this should be an important metric that is identified during project approval stage and one that should be monitored throughout product's life span.
In Michael Schrage's July 15th 2005 article in CIO Magazine "IT's Hardest Puzzle", Michael describes a CMR system implementation that was done on time and within budget.
"However, the first six months of the CRM deployment ended up costing the company more money per unit sale. Even worse, the company's threats to discipline the salespeople who had gamed the CRM ruined the sales force's desire to work with the system. The vice president of sales made such a fuss about how his most creative salespeople were being "punished" for their ingenuity that harsh words were exchanged. He was asked to leave As a result, customer satisfaction dropped."
"The lesson here is simple: Whether CIOs implement ERPs, SCMs or CRMs, only the most naive C-level executive focuses on whether the system actually works. Success will be - and should be - measured by how well that system is used."
30. Shang, Shari & Seddon, Peter B., October 2002, p277 (271 - 299), Assessing and managing the benefits of enterprise systems: the business manager's perspective, Information Systems Journal, 12 (4)
31. Bernthal, Paul, May 2005, p54 (53-56), Measurement Gets Strategic, T+D Magazine, 59 (5)
32. Cooke-Davies, Terry, March 2004, pp. 6-8, Consistently doing the right projects and doing them right: What metrics do you need?, Journal of the Australian Institute of Project Management, 24 (1)
33. Wideman, 2002
34. New York Skyscrapers - One Hundred Years of High-Rises, Available On-Line at: http://www.greatgridlock.net/NYC/nyc2a.html
35. Empire State Building Official Internet Site, Available On-Line at:
36. Melville, Nigel, June 2004, pp 283-322, Review: Information Technology and Organizational Performance: An Integrative Model of IT Business Value, MIS Quarterly, 28 (2)
37. Schrage, Michael, July 15, 2005, IT's Hardest Puzzle, CIO, Available On-Line at: http://www.cio.com/archive/071505/leadership.html