Tuesday, May 19, 2015

So many bugs!

This is a true story. The company name has been changed.

Background

The replacement of Acme Corporation’s equipment maintenance system would directly affect the work of about one-third of Acme’s work force, and provide data for the daily work of nearly everyone else in the company.   

After the new software was selected, the implementation project was expected to take nine months.

The situation

Early in the project, the project team found bugs in the software, but not a lot of them at first. They reported these bugs to the vendor and obtained incident numbers.

The vendor had a consultant on site setting up a prototype. Even before timelines started to slip, the project manager noticed that the consultant appeared to be spending more time on troubleshooting bugs than on creating the prototype.

In addition, the project manager recognized that the amount of attention paid to bugs indicated a risk: that the software may not work as expected and might not be implemented on schedule.

Action taken

The project manager contacted the vendor and asked for a report on all outstanding bugs, along with a status update on each one. However, the vendor was unable to come up with such a list. To the project manager, the inability to produce a status for bugs indicated that the vendor did not have a good tracking mechanism, and the risk became an issue that needed to be managed.

The project manager immediately assigned an analyst on the project team to track all bugs, and arranged a weekly status call with the vendor. In the status call, the bug list was reviewed, and priorities were also established, in order to ensure that the highest priority bugs were fixed first.The tracking of bugs and testing and installation of fixes became high priority work on the project.

Although bugs were being fixed, more were found and the overall number of bugs continued to climb. Many of the defects were also critical – Acme could not possibly implement the system with these types of problems.

The next one’s better…

The vendor explained that many bugs reported were already fixed in the next release of the product, and that it would be faster to install the next release than to continue to fix bugs in the current release. The timeline was in danger of slipping, so Acme agreed to have the vendor install the next release.

The vendor installed the next release of the software, but the bugs were even worse than before. The go-live date had to be delayed because the software was too defective to implement. Acme was committed to the software product selected. They decided to have the project team continue to manage the tracking of bugs and testing and installation of the fixes.

Outcome

The go-live date was delayed again.  The new system was finally implemented after thirteen months, four months after originally scheduled. The cost of implementation was higher than expected due to the number of delays and amount of re-testing required.

Conclusion

The software selected was well established in the marketplace, with over forty successful large implementations of its product. I wonder if Acme relied on reputation and could have done a more thorough job of checking references. Acme was committed to the product and probably would have selected it even if it was buggy. However, more information about the problems would have allowed them to plan extra time and money for the implementation.

Although the project was completed late and over budget, the project manager was recognized as having done an exceptional job of managing the implementation.

For the Acme Corporation, on-time and on-budget were not the only items to consider in evaluating the success of the implementation. The early recognition of the bugs as a problem, the immediate action on tracking and managing bugs, and the establishing of control over the priorities of the vendor were key to getting the product to run reliably, which was the most important outcome for Acme.

Copyright 2015 Debbie Gallagher