When it comes to eventual feed back for purposes of improving the portfolio, a key metric is obviously the answer to the question: "Was the product a success?" Measuring product success raises some major issues. On what basis can we agree that the product was indeed a success?
- Was it bought?
- Was it used?
- Did users like it, i.e. did they evince a measure of "satisfaction"?
- Did it make money?
- Was the product an enabler of the intended benefits?
- And did it produce those benefits in sufficient measure?
And, we might add, if it did, who cares if the product was late and over budget?
Of course, there is always room for improvement - otherwise we'd never make any progress. But in the IT world, the very idea of upgrades has become a part of the business plan. We can reckon that almost any software will either be reworked as an upgrade version, or supplanted by a newer and better application. Indeed, if that does not happen on a one to two year cycle, we may even be disappointed.
This attitude has become so all pervasive, that we even get disappointed when more solid artifacts fail to be significantly upgraded, and that applies across the spectrum from cell phones, to household equipment, to sports gear, to cars. Eventually, it comes down to whether the driver for change is a real need to improve over the previous model, or is it a device to be able to sell the latest "new and improved" must-have?
Theoretically, the "market" will, or should, decide but that is not always the case. Certainly, if the product fails in a big way on any of the questions we raised above, the product and hence the project will have been a failure. But the opposite does not necessarily hold true. For instance, the answers to the above questions may all be very positive simply because of a very slick and powerful selling ("marketing") campaign. A campaign designed to convince people that this product is just what they want to fill a need - even if they have not recognized that need before!
Clearly there is a lesson here for the project management fraternity. It is not sufficient these days to be "on time, on budget", nor even that the product works just the way it should and satisfies all the "requirements". At the time of the transfer of the project's product into "the care, custody and control" of the users, the product needs to be marketed. It needs to be sold into the market place, that is, into its working environment.
Then, and only then, upon completion of a successful marketing campaign, will the product be a success and, therefore, the project can also be called a success. Of course, that costs money, and management must be prepared for it because that is not strictly a part of the project's "construction" budget.