Monday, May 10, 2010

Why Software Projects Fail

According to a study done by the Standish Group, the U.S. government and businesses spent over $81 billion on canceled software projects, and another $59 billion for budget overruns annually.

Their survey claims that only about one-sixth of all projects were completed on time and within budget, nearly one third of all projects were canceled outright, and well over half were considered "challenged."

Of the challenged or canceled projects, the average project was 189 percent over budget, 222 percent behind schedule, and contained only 61 percent of the originally specified features.


What are the primary causes of this failure rate?

We define a failed software project as one that is over budget, behind schedule, or fails to deliver value for the users of the software. Most people talk about the first two, over budget and behind schedule. One wonders how many companies actual evaluate the "value" they get from their software projects.


Here are our reasons why software projects fail. In a future post, we will address how to mitigate these issues.



Unrealistic Schedules

(Poor Cost and Schedule Estimation, Failure to Plan, Never Reviewing Project Progress, Ignoring Late Failure Warning Signals)

Pushing for an aggressive schedule does not accelerate the work, it actually delays it. When faced with an unrealistic schedule, development teams often behave irrationally. They race through the requirements, produce a superficial design and rush into coding. The mad scramble to build something - anything - results in a poor-quality product that has the wrong functions, is seriously defective and ends up being late anyway.

Setting correct schedules can be difficult. If management has software development experience, that is a help. However, our general observation is that most projects are not allowed sufficient time, and with many projects, the time from the testing and quality assurance phases of the project is often severely trimmed.


Inappropriate Staffing

(Lack of Resources, Lack of Executive Support, Skills that Do Not Match the Job)

The only way to complete an software development project rapidly and efficiently is to assign an adequate number of people with the proper skills and then protect them from interruptions and distractions. This helps build the motivation and effective teamwork needed for quality results. When managers fail to provide timely, adequate and properly trained resources, their projects will generally fail.

Software development continues to be a rapidly changing field. Programmers need continual training to stay current. It also takes a continual desire to learn on the part of programmers.


Changing Requirements During Development

(Incomplete and Vague Requirements, Poor User Input and Lack of User Involvement, Stakeholder Conflicts, Communication Breakdowns)

To start designing and building products, software developers must know what product to build. Unfortunately, management, marketing and even the customers often don't know what they want. Or worse, they think they do know and then change their minds partway through the job. While the requirements normally change in the early phases of a job, there's a point beyond which changes will waste time and money and disrupt the work.

This area also leads to the lack of value of the software project for the company. If the project's requirements are not thought out to a reason degree of completeness, the resulting software often provides little value.



Poor-Quality Work

(Poor System Architecture, Inadequate Testing, Testing in the Production Environment, Lack of Quality Assurance, Not Conforming to Industry Standards)

A common scenario is one where a manager of a software project that has to meet an accelerated delivery date set by their boss. They may unmercifully push their software developers, who rush through the design and coding and skip all of the quality reviews and inspections. Testing finds many defects, but management argues for delivering the software and fixing defects later. Management meets the deadline, but the system is a disaster. It is so unreliable that the software has to be fixed every time a change is made in the product or product mix.

When executives push for unrealistic schedules, the project either will be late in delivering a working product or will produce a product that doesn't work. There's a saying about software quality: "If it doesn't have to work, we can build it really fast."


Believing in Magic

(The Big Red Do-it Button, Unrealistic Expectations)

Commercial off-the-shelf software, or COTS, is an attractive way to save development time and money. But COTS isn't a silver bullet. If not properly managed, it can be a disaster. A COTS product that works perfectly in demonstrations, for example, may crash when subjected to different hardware configurations, higher data rates or even data-entry errors. You must test the product thoroughly enough to expose previously untested conditions. If the program is troublesome when stress-tested, it will almost certainly be troublesome when used in production.

In addition, the use of COTS may force users to adopt inefficient processes in order to use the software. This inefficiency will reduce the value of the software. A full explanation of the "buy versus build" evaluation process is beyond the scope of this entry. However, there are costs associated with configuring COTS, adjusting processes, and process inefficiencies that must be compared with the cost of writing software from scratch.


Conclusions

It is important to allocate sufficient time and resources to complete your software projects successfully. It is equally important to take time in the planning and requirements phase to make sure the project is able to deliver real "value" to the users. Finally, it is important that software be sufficiently tested to minimize the cost to the business of software errors.