The fact that project failures rates in the IT industry are considerably higher than for other types of engineering projects has been well documented. Over the years many studies have shown high failure rates in the IT sector, and among the latest reports is a July 2008 study by US Government Accounting Office (GAO) that found that of 840 federally funded projects 49% of them were poorly planned, poorly performing or both. While some would like to believe that the situation is better in private organizations a 2008 study by the Information Systems Audit and Control Association found that 43% of 400 respondents admitted that their organization had had a recent project failure.
Clearly, failure rates in fields such as civil engineering are much lower. While construction projects do at times go over budget and encounter schedule delays, the majority of projects do get completed and the end product usually fulfils its intended purpose. If the failure rates experienced in the IT sector were replicated in civil engineering projects our cities would be littered with abandoned construction projects, the electrical supply to our homes would work intermittently and many of our bridges would have gaping holes that would routinely swallow vehicles brave enough to attempt a crossing.
Given that IT Project Managers use the same basic tools, techniques and principles as their counterparts in civil engineering, the difference in success rates raises some interesting questions.
Superficially, all three questions are probably affirmative. However, if we want to improve the success rates in the IT sector, we need to go beyond the superficial answers and understand the problems in a much more fundamental way. There is an old proverb that says "you can't truly solve a problem until you truly understand the nature of the problem you're solving". In short, when we fail to understand the essence of a problem, we run the risk of adopting solutions that simply don't work.
Many IT organizations have been down that road. In the aftermath of a failed project, one of the most common reactions is for the organization to adopt a program of process improvement. In theory by defining more robust processes for the planning, execution and control of projects, success rates can be improved. Having witnessed many organizations' attempts at process improvement, I can report that it's an idea that is far harder to implement than it might at first appear.
A good number of these organizations end up no better off than they were before the program began. The reasons behind the failure of process improvement initiatives are often complex. So in large part, it often comes down to the fact that the underlying problems the organization is facing are poorly understood and hence the process improvement initiatives are poorly attuned to solving the organizations' real needs.
1. Chaos Report - Standish Group
2. Dobbs Defining Success, S. Ambler, Oct 2007
3. GAO-08-1051T - United States Government Accountability Office testimony ("OMB and agencies need to improve planning, management and oversight of projects"), Jul 2008
4. IT Week Magazine - 19 May 2008
5. Or as Albert Einstein observed: "The significant problems we face cannot be solved at the same level we were at when we created them."