Why your IT project is being audited

You’re the manager of a system implementation or custom development project and have just been told that the project is being audited. There’s a possibility that you aren’t exactly thrilled about this. After all, the project is going well, or maybe it’s over and the new system is humming along just fine. A common response by IT folks who haven’t been exposed much to audits is “but I didn’t do anything wrong, so why am I being audited?”  The purpose of this article is to provide some information on the reasons why your IT project could be audited. You’ll notice that the reasons have nothing to do with you!

Types of auditors

First of all, you may have noticed by now that there are different types of auditors checking out your IT environment. Here are some of the most common types of auditors that audit systems and IT processes in your organization every year:

·         Financial statement;

·         Controls certification readiness;

·         Internal audit; and

·         Regulatory.

The financial statement auditors are responsible for signing an opinion regarding the financial statements of your organization. Let’s include here the folks who have to audit your controls for Sarbanes-Oxley, and any other internal controls audits. They will be from the same audit firm as the financial statement auditors and will generally perform one integrated audit to support both the financial statement opinion and the internal controls opinion. The systems focus of these audits is on IT processes and systems that could have an impact on the financial statements.

In preparation for the internal controls certification and audit, your organization may have hired auditors from another firm to help you get ready. They may be documenting your IT environment and providing feedback on control weaknesses, so that you can remediate prior to the controls certification audit. These auditors also will be financial-statement focused.

If your organization has an internal audit department, you will be used to seeing them almost every year. You may have also noticed that their focus is often different from that of the financial statement auditors. The internal auditors go beyond the financial statements to look at systems that affect operations, whether or not they have no financial statement impact. They frequently also have a mandate to verify that the organization’s policies and procedures are being followed, or that value is being achieved for funds spent. The internal audit group may or may not be involved in assisting the financial statement auditors, depending on their mandate in your organization.

Regulators frequently also perform audits. For example, if you work for a financial institution, you are subject to audits by the Office of the Superintendent of Financial Institutions. Other industries also have regulatory auditors, and if you are in one of those industries, you know that these auditors will visit you every year. Their purpose is to verify that relevant regulations are being adhered to by the IT systems and processes in your organization.

The annual audit

The financial statements for your organization rely heavily on the systems in your organization – gone are the days of binders with 14-column paper! In performing their work, the auditors also rely on reports and data from those systems. As a result, the auditors are obligated to verify the management of many aspects of your financial systems.  They will check out security, networking, communications, databases, and application implementation and maintenance. They may also evaluate business continuity planning and IT strategy.

Depending on a variety of factors, the financial statement auditors may look at different aspects of your systems environment each year, say networking this year and databases next year. There may also be elements, like security, that do not follow a rotation plan, so are examined each year.

The internal auditors generally have a specific set of objectives each year, based on your organization’s risk assessment and audit plan. As a result, they will look at different aspects each year that they visit you.

As for regulatory auditors, the work they do will depend on the type of regulatory body they are and other factors relating to the industry and regulatory environment.

The project auditor

In addition to the usual array of annual auditors, you may also receive a series of visits from a specialist in IT project risk. This person or team will be hired by your project sponsor or client, who is looking for regular feedback throughout the project, so that risks can be managed timely. The hiring of this additional audit team is likely to occur in industries where there are a number of stakeholders to satisfy, or when the project is very large and high risk. The project risk audit team is likely to examine your project more frequently and in more detail than the various types of annual auditors.

The project audit

As part of the annual audit planning process, the audit team evaluates your organization and the risks in your environment. In this risk assessment, they are considering inherent risk, which is the level of risk that exists if there are no controls in your environment. So, for example, what would be the impact if everyone in the organization could sign a cheque, or if no one tested that new application system?

The audit team will then focus their work on the areas identified as highest risk. Although the areas of greatest risk vary by type of industry, a new system nearly always has a high probability of being identified as a medium or high risk. As a result, your IT project has a high likelihood of being audited, by the financial statement and controls certification auditors, the certification readiness team, the internal audit team, regulatory auditors, and maybe also project risk auditors.

As noted earlier, the various auditors have different focus areas and objectives. As a result, you may find that your project is audited several times.

It’s not about you

So, as you can see, the project audit isn’t about you at all. It’s driven by the audit team’s assessment of risk.

Copyright 2015 Debbie Gallagher

The middleman

This is a true story. The company names have been changed.

The project manager gets in the middle

Acme Corporation, a large financial institution, decided to eliminate its personal credit card business to focus on commercial customers. As a result, Acme agreed to sell its database of customer information to a competitor.

Standard Consulting was engaged to extract the customer data and transform it into a format that the competitor could load into its own system. The competitor was creating a new marketing database system, and had hired a team of developers to build it for them.

Acme’s project manager estimated that it would take Standard about four months to complete the work.

Acme’s management was concerned about the possibility of disclosure to the competitor of any data in the database beyond the specific customer data that had been sold.  To alleviate management’s concern, the Acme project manager decided to act as the communication channel between Standard and the competitor. The consultants from Standard could not speak directly with the competitor or with the competitor’s development firm. Instead, they directed their questions by email to the project manager at Acme. Then the Acme project manager would obtain an answer from the competitor and direct it back to Standard.

The communication process was cumbersome and time-consuming. Because the questions and answers were all exchanged by email, there was no opportunity to ensure that the receiving party completely understood the question being asked and as a result, it would sometimes not be answered satisfactorily. Follow-up questions were needed, and the process was repeated.

Because the competitor’s marketing database was still being developed, some data requirements were still changing, resulting in changes to the extraction programs being built. Sometimes these changes were not communicated to Standard via Acme quickly enough and as a result, test migrations of data would fail.

The project manager had other responsibilities, so answers to questions, and follow-ups for changes or issues were slow. The project ran late, but still the process continued. Some issues required significant analysis and discussion between Standard and Acme’s competitor, but all of these took place via email through the Acme project manager. 

Well into overtime

The project had now run for twelve months instead of four, and still it was not finished. Acme’s management was concerned that it might be sued by the competitor for non-delivery of the customer data. The Acme project manager was replaced.

The new project manager met with the Acme and Standard teams to develop a comprehensive list of outstanding tasks and new dates by which the tasks needed to be done.

In addition, she had brief team meetings every day. At the daily meeting, the tasks due that day were determined to have been completed or to have slipped. If a task had slipped, remedial action was taken immediately. Representatives of the competitor attended part of the meeting by phone, so that questions, answers, and outstanding issues could be resolved immediately.

The project finished after sixteen months, a full twelve months later than the original estimate of four months. Acme’s competitor did successfully load the customer data into the new marketing database system, and Acme did not get sued.

Conclusion

The original time estimate for the work was much too low. The Acme project manager did not realize the impact on the timeline of having so many parties involved in the process. Project managers who have successfully delivered multi-party, multi-location projects are painfully aware of how time-consuming the coordination effort is. Anything that can be resolved in an hour with one company involved can easily take ten hours with three parties involved. There is significant effort in the scheduling, discussion, resolution, and follow up for every issue. This data migration project had four parties: Acme; Standard; the competitor; and the competitor’s development team. Dealing with multiple locations is also time-consuming, as the immediacy of face-to-face contact is lost. Sometimes, when the related team is not physically present, the local team forgets to keep the off-site team up to date. This project had two locations, one with Acme and Standard, and another with the competitor and its development team.

Although the estimate of four months was too low, the project should not have taken sixteen months. The communication process designed by the original project manager was rigid. However, it might have been made workable by assigning that liaison role to someone else. Because the project manager had other responsibilities, the management of the communication process often fell to a lower priority and caused project delays. 

The project manager should have placed a resource in that analyst/communication role full time.  That analyst should have been given the authority to speak with both parties by phone and in person, in addition to using email correspondence. This would have improved the response time to questions and issues considerably. It also would have reduced the number of times that emails went back and forth between parties who were trying to understand the issue well enough to get it resolved. Key hiring criteria for that analyst role would be verbal and written communication skills, organization, documentation (to capture decisions and agreements), and ability to prioritize issues. The cost of that one additional person on the job would have been significantly less expensive than running the project so late.

When it was clear that the project tasks and issues were not being completed on time, the new project manager did the right thing by ensuring very frequent meetings to review completion status and follow up.

The new project manager also brought the competitor to the status meetings, a step that helped immeasurably in getting issues resolved quickly.

Copyright 2015 Debbie Gallagher

Head in the sand

This is a true story. The company name has been changed.

The data conversion

Acme Corporation was a retailer based in Canada and in France. They were replacing the separate Canadian and French systems with one common system, including financial as well as sales and distribution modules.

Early in the project, a software developer in France was assigned to design and create data conversion programs to load data from the two old systems into the new system. The developer was familiar with the French legacy system but not familiar with the Canadian system or the new system. The developer’s experience with the French legacy system provided her with some knowledge of the business and its system requirements. However, she had always worked for a supervisor or project manager, and did not have any experience at managing work.

Several members of the project team expressed their concerns to the project manager. Surely the one developer would not have time to plan and develop all of the programs needed to convert data from both legacy systems. The project manager was not worried, as the developer had always got assigned tasks done in the past.

After several weeks, there was no visible action from the developer, and the project manager was still unconcerned, so a couple of team leads on the project decided to try and move things along. The Purchasing/Accounts Payable team lead called a meeting to discuss data conversion requirements for vendor master and purchase orders. The Sales/Accounts Receivable team lead called a meeting to discuss conversion requirements for sales orders, pricing, and customer master.

The developer took some notes at these meetings and said she was going to start coding extracts from the French legacy system. She did not provide any confirmation to the team members about what would be delivered. In addition, she spent no time working on extraction of data from the Canadian system.

The team leads continued to press the project manager for answers about how the data conversion was going to get done with only one resource. After a few more weeks, the project manager assigned a second developer to write the data conversion programs for the Canadian data. The two developers had no shared design to ensure that the separate work done for the two legacy systems would have similar results in the new system.

The team leads kept voicing their concerns to the project manager, that the data conversion was not well planned, that there might not be consistency in the data from the two countries and that the programs might not be done on time. The project manager saw no need for the developers to stop their work to do some planning.

At the project status meetings, it became clear that the team leads and the developers had different ideas about what was to be delivered. As a result, additional meetings were held and the programmers took notes on additional files to be converted.

When data was loaded into the new system for testing, the team members were shocked at how many fields were missing or incorrect. The team leads again approached the project manager with their concerns about the data conversion and whether it would be on time and of acceptable quality.

The project manager still thought it didn’t make sense to stop and coordinate efforts between the two countries or to create a schedule at this late date. He decided that the developers were too busy working to take time out for planning.

The project manager starts to worry

When the go-live date was less than a month away, the rest of the system implementation was going well, but the data conversion work was significantly behind, the project manager got worried. Conveniently for the project manager, he was directed by the steering committee to delay the project go-live date by three months. Acme was acquiring another company and the system implementation would be delayed for a few months to allow Acme to focus on store amalgamations and training the new employees on Acme’s way of selling products. The project manager was very relieved. He did not have to be the one to defer the go-live date due to a late and inadequate data conversion.

Conclusion

The project manager made an invalid assumption right from the start and kept to it even despite evidence that proved him wrong. He thought that since the French developer had always got the job done when given assigned tasks that she would also be able to deliver when she needed to define, plan, and manage the work as well. Project management was not a skill the developer had learned and so the assumption was unfair.

In addition, although the project manager monitored the progress of the rest of the project very carefully, he didn’t pay much attention to the scope, resourcing and delivery of the data conversion. He assumed that data conversion was a small and easy task compared to the rest of the project. If he had ensured that scope was defined and a schedule was developed at the start, he would have realized the extent of the work to be done. Even some preliminary discussions with other project managers would have given him better insight into the likely challenges of the data conversion work. Except for the company’s timely acquisition of a competitor, the project manager would have faced an embarrassing and costly project delay, due to his reluctance to define and plan the data conversion.

Unfortunately, both developers were uncomfortable about asking for help. They would have been wise to insist to the project manager that they needed help with the planning and coordination of the two countries’ requirements.

Copyright 2015 Debbie Gallagher

Put money aside for more lumber!

This is a true story. The name of the city has been changed.

Starting with lessons learned

The City of Yorktown was starting a project to replace its financial and human resources systems. The city had a history of technology projects that were completed late and over budget, and the sponsor for this project was determined that this project would be different. He and the project manager decided to start with lessons learned.

The project manager spent time researching what other cities had done and what had worked well. As a result of this research, there were some things she decided were important to Yorktown’s project: (a) all departments would be involved in and committed to the preparation of the business case; (b) the structure of the contracts with the various vendors involved would ensure one vendor was responsible for managing all of the vendors; (c) the vendors would be treated as business partners; and (d) the budget would have plenty of contingencies.

Implementing the lessons

Before choosing a software package and vendors, the business case and requirements were prepared. In every department at Yorktown, the department heads signed off on the cost savings and other benefits expected from the new system, including agreement that specific budget cuts would be made to the department once the system was implemented. The cost savings and budget cuts would be made over a five-year period as staff retired, since there were to be no layoffs.

The Request for Proposal (RFP) was issued with a requirement that the vendors bid together as a team, and that the system integrator take responsibility for managing the other vendors. The proposal was to include software and hardware recommendations.

After the applications and vendors were selected, the business benefits were re-worked, taking into account the specific package that had been chosen and its particular capabilities. Again, each Yorktown department head signed off on the business benefits, including the planned budget reductions. At the same time, the costs were refined, based on current information about the package, the infrastructure and the implementation fees.

The contracts with the vendors were negotiated and included some important elements: (a) the system integrator was to have responsibility for managing the software and hardware vendors; and (b) the vendors were asked to put up some extra contingency dollars. The additional dollars were not laid out in the contract itself but were negotiated at the same time and were to be managed outside of the contract.

The project manager knew that each vendor would already have included some unallocated budget in their own bids, in order to ensure they were not going to lose money on this fixed-fee price. She too had a miscellaneous category in her own project budget. However, she also wanted all parties to provide additional assurance by sharing costs for situations on the project where the responsibility for picking up the extra costs could not be clearly defined. She felt this would prevent cost overruns and also lead to the city and vendors acting as partners instead of each protecting their own budget.

The shared contingency dollars were to be spent on a case-by-case basis when agreed by all parties during the project. The total of the shared miscellaneous budget was approximately one percent of the project budget

Using the shared contingency dollars

The implementation was to take seven months. Almost immediately, Yorktown and the vendors used nearly a quarter of the shared reserve of funds, as additional hardware needs were identified during the set up of the infrastructure.

During the rest of the project, several smaller items were agreed to be paid out of the shared reserve fund. Most of the remainder of the reserve was used for additional processors, storage, and database licensing.

In each case, when an issue arose that would cost more money, Yorktown and the vendors would, as expected, try to negotiate so that another party would pay for the new item. However, when it became apparent that some items could not be agreed upon, the parties agreed to pay out of the shared reserve.

The outcome

The project completed on time and on budget, and is expected to deliver the planned business benefits. Yorktown and the vendors did maintain a collegial relationship during the entire implementation.

Conclusions

The time the project manager spent on planning and research was time well spent. The project manager learned a lot from her discussions with other cities. She focused on making sure the key success factors were in place and that potential points of failure were mitigated.

She ensured commitment from the business users by involving them in defining the system requirements and having them take responsibility for budget reductions related to implementation of the new systems.

The contracts were structured so that one vendor could manage the others, reducing the conflict that can arise when no one vendor is responsible for the outcome.

The shared reserve was an innovative concept that allowed the project manager to honour her commitment to the sponsor and city council to complete the project without going over budget. Just as important, because the reserve was shared by Yorktown and the vendors, there was more of an implied partnership between the parties. They had a means to resolve budget issues that might otherwise cause each to dig in their heels to protect their own budget.

According to the project manager, who built in an extra contingency, when each party already had a contingency, the old expression should be “Measure Twice, Cut Once – And Have Money Handy For More Lumber!”

Copyright 2015 Debbie Gallagher

The penny pincher

This is a true story. The company names have been changed.

Losing staff and saving money

Acme Corporation, a British manufacturer, was implementing a new billing system, including development of custom reports, modifications, and interfaces. Acme assigned twenty of its own staff either part-time or full-time to the project. In addition, Acme engaged Standard Consulting, who assigned twenty part-time and full-time consultants to the project.  The consulting fees were expected to be about two million Euros for the nine-month project.

The first phase of the project went well, with the smallest Acme division starting to use the new billing system after seven months, as scheduled. The remaining divisions, billing over ninety-five per cent of Acme’s customers, were to be live nine months after the start of implementation.

During the eighth month, one of Acme’s four technology staff quit with very little notice. Acme’s IT manager wanted to get a replacement hired quickly, but was also concerned about cost. When the Standard project manager suggested that a Standard consultant could be added to the team, the Acme IT manager declined due to the additional fees. Then the Standard project manager offered to assist in the recruiting process, but the Acme IT manager declined that option as well. Instead, the Acme IT manager brought on a resource from a foreign country, which would save five thousand Euros for the remaining time on the project.  

The foreign developer arrived within a week, and in less than two days it was obvious that the developer was not qualified and also spoke very poor English. By the time the developer was terminated, a person-month had been lost. Acme decided to add another Standard consultant to the project to replace the terminated developer. This arrangement started immediately.

Now a second Acme developer quit his job, also with short notice. While recruiting was under way for the empty positions, another Standard consultant was added to the project part-time.

Then, a third member of the Acme technology team left his job with short notice. This time, the Acme manager asked the Standard manager to assist with the process of hiring a contractor, to ensure the new person was qualified to work with the new software and development tools. The new contractor started right away to catch up on the lagging work.

Go-live

The Acme and Standard technology team members worked longer hours to try and catch up. However, enough time had already been lost that it could not be completely caught up, and the go-live date had to be delayed by a month.

During the project, the project team ran test billings successfully. So, in order to save money, Acme’s IT manager told the Standard project manager that no go-live support would be needed, and the Standard consultants could leave. Standard Consulting’s project manager thought it was wiser to keep a few of the consultants on site for a couple of weeks after go-live to support Acme. This would have cost Acme about twenty thousand Euros, which had already been included in the budget for consulting fees. However, Acme was insistent that they didn’t want to pay for it and that Standard should leave.

After go-live, Standard’s project manager was shocked to find out that Acme was extremely unhappy. Upon investigation, Standard’s manager found out that Acme had not properly run a step that was required prior to running the billings. Acme’s staff did not have the experience to solve the problem quickly and the billing had been delayed. Acme’s CIO had been concerned that she would lose her job if they could not produce invoices after spending millions on implementing the new software.

After all the work that had been done well, Standard’s project manager was very concerned that Acme could end up dissatisfied. In order to ease Acme’s concerns, he assigned a consultant to automate the pre-billing step and did not bill Acme for the work.

Conclusions

Acme’s decision to choose the unqualified foreign contractor to save five thousand Euros out of two million Euros in fees was short-sighted, as the go-live date ended up being postponed for an entire month.

The Standard Consulting project manager discovered that it is possible for a project to go well and then for lack of go-live support, the delivery team can lose credibility. He may have found it worthwhile to manage this risk by assigning one consultant to stay or be on-call for a week or two at go-live and not bill the client, or bill at a discounted rate. Providing a bit of free service is what he ended up doing, and perhaps they’d all have been happier if it had been done earlier.

It is tempting during a project, when budget revisions are needed, to consider reducing or eliminating the go-live support budget. In this case, where the client had continuous turnover, leaving only the consultants with knowledge, it was an unwise decision. It may have saved a few dollars but nearly prevented the company from running its business. 

Something I wonder:  When the billing couldn’t be made to work, and his boss, the CIO, was in danger of losing her job, why didn’t the manager phone the consultants to ask for help?

Copyright 2015 Debbie Gallagher

The meddler

This is a true story. The company name has been changed.

Changing direction

Acme Corporation, a manufacturer, was building a new data warehouse with multi-dimensional analysis and reporting capabilities. There were two main teams, one building the technical infrastructure and one developing the multi-dimensional cubes and reports.

The project was already underway when the project manager left and a new project manager was hired. At the first technical team meeting after the new project manager joined, the technical team lead discovered that his team was trying a new approach instead of what had been agreed as the path to developing the technical architecture. The team members explained that the project manager had decided that they should try a different approach. The technical lead spoke with the project manager to find out why he had been left out of the discussion. The project manager was surprised. She had certainly not intended to leave out the team lead. She had dropped by to discuss a different matter, when an informal discussion on the technical progress had come up and the change in direction had been agreed. The technical lead pointed out to the project manager that once the discussion turned to alternative approaches, he should have been involved in the discussion. At the very least, he should have been informed of the outcome, as it affected his team’s scheduled tasks and his resources. The project manager hadn’t meant to leave out the team lead, the new approach was a good one, and so she didn’t see the harm.

A few days later, the technical lead discovered a member of his team was working on a different assignment. The project manager had approved the reassignment. The team lead discussed this change with the project manager, who explained that she had just bumped into the other project manager in the elevator. This had not been a formal meeting. The other project manager happened to have been short-handed, so she had agreed that the team member could help out. Again, the technical lead protested that he had been left out of the decision and not been informed. The project manager didn’t think this was a problem. After all, the technical lead had not been left out deliberately, it was just informal, and the decision was a good one, so she didn’t understand his concern.

This pattern of events continued. It didn’t take long for team members to bypass the technical lead and start going straight to the project manager to resolve any requests or issues. The technical lead became extremely frustrated. Repeated meetings with the project manager did not correct the situation. So instead, the technical lead just tried to keep up to date on what was going on with his team and understand the progress they were making. The technical lead disliked working this way, but his repeated discussions with the project manager had not corrected the problem.

The technical lead regains control

The cube and report team started to have significant problems. The users had rejected the prototype cubes and reports. Significant re-design and re-work was required. End users and developers were blaming each other for the rejected work. In addition, the cube and report team had a lot of difficulty in doing the re-work. This team now required a lot of time from the project manager.

As a result of the upheaval in the cube and report team, the project manager was too busy to address informal questions about the technical team’s work. The technical lead regained control of his resources and scheduled tasks, and kept the project manager informed about progress through the weekly status reports and meetings. The work proceeded according to schedule.

As the cube and report problems were resolved and the project neared completion, the project manager congratulated the technical lead on a job well done. The technical lead pointed out that he had delivered the work with the appropriate quality on time, even though the project manager had been unavailable to interfere. The two laughed about the situation together.

Moving on

On their next project together, the technical lead met with the project manager at the beginning of the project. The project manager agreed to try very hard to avoid interfering. They met weekly to discuss progress, issues and explore alternative approaches.

Sometimes, the project manager slipped back into her old ways, but the team lead would remind her to stop meddling. They would laugh together about it, and the project manager would back off.

Conclusions

There were a couple of reasons for the project manager’s behaviour. First, she was new to the organization and felt a need to prove herself in the new environment. As a result, she was spending extra time getting involved in details to reassure herself that the project was going to succeed. In addition, the project manager was a very easygoing, approachable person, so team members and other project managers felt comfortable chatting about their project issues and getting her involved.

The project manager was usurping the authority of her own technical lead who was competent and did not require close oversight. By doing so, she was working very long hours doing more work than she really needed to. She could have been focusing the whole time on output and issues, and instead got involved unnecessarily in the details of the technical team.

It is fortunate that the project manager and the technical lead had a friendly relationship and were able to laugh about this problem and solve it. Overriding the technical lead’s authority was really no laughing matter.

Copyright 2015 Debbie Gallagher

It's a technically elegant solution

This is a true story. The company names have been changed.

The schedule

Acme Corporation, a European manufacturer, was implementing a new billing and accounts receivable system. Acme engaged their implementation partner, Standard Consulting, to develop the data conversion programs.

Acme’s legacy billing and accounts receivable system was more than twenty years old. It was running on hardware and an operating system that were no longer sold. In addition, it was custom software, developed by a person who was no longer available to support it or explain how it worked.

The client team made high-level decisions about what data they needed converted to their new system to support internal and external reporting, as well as long-term billing needs. They wrote these requirements in a strategy document, which they gave to the Standard Consulting project manager.

Standard’s project manager assigned a technical team lead to review the strategy document, and prepare a schedule and cost estimate. Standard’s project manager was satisfied with the schedule and cost, and so was Acme. The data conversion programs were to be ready for testing by the client team in five weeks.

Falling behind schedule

After three weeks, the data conversion budget was half used. This was about as expected. However, the technical team leader had not submitted her status reports, and when pressed, provided only vague verbal reassurances that everything was under control. She seemed unclear on how she would know that her development team was finished.

Although there were several components required to convert the billing and receivables data, there were no components ready to test by the five-week deadline. Again, the technical lead was vague and assured the project manager that the programs were progressing.

After about seven weeks, programs were ready for testing, and the client lead was asked to review the sample data in the new system. The client lead was shocked. This data wasn’t even close to what was required. There were significant differences between what was stated in the high-level requirements document and what had been developed. In addition, there were several components that had not been built at all.

The technical team lead was disappointed, as she had developed a perfectly elegant solution to the technical challenge of extracting data from the legacy system, and the client was not appropriately appreciative of the effort or outcome.

Getting the programs corrected for go-live

The Standard Consulting project manager added a business analyst to the technical team. He was to review the design of the conversion programs compared to the requirements, and determine what re-work and new development was required. However, there were no detailed design documents to review, only the original high-level strategy document. The business analyst assessed each component by developing the detailed design, working with the technical lead and billing lead to solve discrepancies between what was needed and what had been built. Standard Consulting was committed to delivering what had been committed to Acme. Significant parts of the data conversion programs were re-written, and the missing components were developed.

Testing was very time consuming. The programs had not been built for the volume of data and complexity that was really required. As a result, whenever one bug was fixed, others would appear. More than a dozen rounds of testing and re-work were required for some components. The client couldn’t keep up with the testing, so Standard had to provide additional testers. The data conversion costs went three hundred per cent over budget. The client and the technical team worked nights and weekends to complete testing and program revisions.

The go-live date was not delayed. However, only the basic functionality could be used right away, because not all of the conversion components were ready. The technical team continued to work for another month to complete the remaining components.

Conclusions

The project schedule prepared by the technical team leader should have been the first sign to the project manager that something was amiss. The technical lead had been focused on the technical complexities of extracting data from the legacy system, and had planned only technical deliverables. She had included a task for preparation and a milestone for sign off of technical design, but no preparation and sign off of detailed requirements.

The next sign of a problem was the lack of status reporting from the technical lead. When she tried to give just vague reassurances, the project manager was rightly concerned, but should have pushed harder and earlier for proper status reporting, with progress described for each component. This would have allowed the project manager to identify earlier that components outlined in the strategy document were missing.

In the end, the client go-live deadline was achieved, and all of the components were finished. The data conversion programs ran successfully and loaded correct data into the new system. However, the client was not happy. Acme was not interested in the technical elegance of the delivered programs. What they wanted, and didn’t get, was programs that met their requirements and were delivered according to the originally agreed schedule.

Copyright 2015 Debbie Gallagher

I won't trouble the client with it

This is a true story. The company names have been changed.

Choosing the product

Acme Corporation, a large Asian retail company, was expanding into the U.S. and engaged Standard Consulting to select, negotiate the software purchase, and implement a financial system to be used for their new U.S. subsidiary.

Acme wanted to move along quickly. Based on an existing professional relationship, Acme trusted Standard’s capabilities and expertise. Because Acme was setting up a new subsidiary, there were very few employees so far. Employees who had been hired had joined Acme recently, so had no experience yet in Acme processes and procedures. As a result of all of these factors, Acme decided to rely heavily on the expertise of Standard’s project manager in assessing the strengths and weaknesses of the various products, and recommending a product for Acme.

The project manager selected a system that was new to the North American market, but was widely used in Asia’s manufacturing industry. The vendor was planning to build additional functionality into the product to accommodate retail businesses.

The project manager was satisfied with the vendor’s plan to accommodate the U.S. market and the retail business requirements. She wanted to help the client feel comfortable with the selected product and vendor, and didn’t want to worry the client. So, she decided to skip the project risk assessment; it was only a list of potential problems that might cause unnecessary concerns for the client.

Problems surface

As the project got under way, the team started finding bugs right away. Many bugs were serious and affected basic financial processes. As the implementation continued, it also became apparent that the vendor’s new functionality for retailers was not going to be delivered, and was not even under development. As time went on, more and more bugs were found and, like the earlier bugs, several were serious, with the potential to prevent operation of the software by Acme.

The project team was heavily reliant on Standard Consulting, as there were very few Acme employees hired yet. Those who were on staff were all new hires and did not know yet what their business required. As a result, the project team had to guess at what some of the system configuration should be.

The project manager was pleased with the vendor’s intention to fix up all the bugs, and decided not to bother the client with any worries about the continuing and increasing incidence of serious bugs. The project team managed to develop a number of complicated workarounds to enable operation of the system, despite the lack of retail functionality, and to eliminate reliance on the worst of the buggy programs. The project manager continued to assure the client that everything would work out well.

More problems after go-live

The system did go live, but was very complicated to use and unstable. The project costs went over budget and because Acme hadn’t been made aware of the extent of the problems created by the vendor, Standard was left to cover most of the shortfall in fees.

Acme had no internal IT support in the U.S. yet, and hired Standard Consulting to provide support services. In the support services contract, Standard did not distance itself from the vendor’s problems, so Standard was often responsible for resolving serious errors caused by the vendor.

When Acme hired a CFO for its U.S. operation, and he discovered the complexity of using and managing the new system, with its instability and complicated workarounds, he complained to Standard Consulting. However, given the buggy software and existing system functionality, not much could be done in the short term to make the software easier to use or more stable. After go-live, the vendor did develop some of the retail functionality and did fix many of the more serious bugs. However, Acme and its CFO were stuck with several clumsy workarounds.

Conclusions

Skipping the risk assessment was an error in judgement by the project manager. She wasn’t lazy, but was overly concerned about keeping the client protected from the reality of the project risks.

The project manager significantly underestimated the risks of implementing software that was being installed for the first time in a new country and a new industry. She also did not warn the client of the risk of buying undeveloped features or modules (also known as vapourware). Because she was too optimistic, the project manager didn’t ensure that the vendor’s promises about the new retail functionality were written into the contract. There was also no provision in the contract for recourse when the serious nature of the numerous bugs was discovered.

Another sizeable risk in this project was the lack of availability of experienced client staff to participate in the product selection and the implementation. An experienced user has the best insight into what the company’s needs are and what configuration and processes are suitable. Acme didn’t have anyone available to fill this role, thus creating significant risk of selecting a product that was a poor fit, or of implementing the product poorly.

Later, when developing the support agreement, the project manager did not separate the responsibilities of Standard Consulting from the responsibilities of the vendor, thereby creating unnecessary risk for Standard. This error also reduced risk for the vendor, and may have decreased the vendor’s incentive to fix problems.

Due to the project manager’s concern about making sure the client always felt comfortable, the client was never fully aware of the level of risk being assumed in the implementation of this product.

Optimism is not necessarily a good quality in a project manager. Setting the expectations of the client is a major responsibility of the project manager and is best achieved with a healthy dose of realism.

Copyright 2015 Debbie Gallagher

The boss

This is a true story. The company name has been changed.

The team’s view

Acme Corporation was implementing a new ERP system with many modules such as general ledger, payables, procurement, sales orders, manufacturing, and receivables.

Not all teams could be accommodated in the project workroom, so the project manager had to choose a team to be situated in an adjacent building.

The manufacturing team lead was knowledgeable about the business requirements and already familiar with the product. In addition, the project manager felt this team needed less interaction with and support from the other teams, so the project manager selected the manufacturing team to work in the alternate location.

The manufacturing team lead did indeed turn out to be capable of managing the team in a separate location. She was able to assist the team with proper resolution of most issues that came up. In addition, she had a good understanding of which issues required input or approval from other teams and ensured they were involved in necessary decision-making.

The manufacturing team lead called the project manager every day to report progress for the day and to ensure he was up to date on the status of all of the important issues related to the manufacturing team. She also made sure issues were escalated to him where appropriate.

The project manager was so pleased with his choice of which group to separate from the others. The manufacturing team lead clearly was capable of running the team herself with little oversight.

What the project manager didn’t realize was that the manufacturing team members no longer saw him as the person in charge of the project. He was assumed to be too remote and lacking insight into their issues.

The manufacturing team started to question the time that their team lead spent every day calling the project manager. They were fine on their own, why should she waste time keeping the project manager up to date? They also questioned the need to escalate issues to the project manager. After all, what use would his input be, since he didn’t know anything about what they were doing?

Putting the project manager back in charge

The manufacturing team lead told the project manager what she was noticing, that the manufacturing team was increasingly seeing her as the authority for anything related to the project, and didn’t recognize the project manager’s right to manage issues that affected them.

The project manager was shocked. After all, he was always up to date and knowledgeable about the manufacturing team’s progress and problems, due to the team lead’s diligence in reporting to him daily.

The project manager started to visit the manufacturing team in the other building every day. He started with brief casual conversations with each team member. The project manager started to ask team members how the project was going and to discuss the issues and progress of the manufacturing team.

At first, the manufacturing team members were puzzled. What was the point of the project manager dropping by? Didn’t he have anything better to do than to bother them with chitchat? However, as the project manager began to discuss progress and issues, he started to regain the confidence of the manufacturing team members.

After he had been visiting the site daily for a few weeks, the negative comments about the project manager were nearly eliminated. Instead, the team members started to agree that the project manager ought to be kept up to date daily, and that certain issues ought to be escalated to him.

It wasn’t much longer before team members were comfortable with the project manager having input into the key issues facing the team.

Conclusions

Because of the efforts of the manufacturing team lead, the project manager was well informed about the progress and issues of the team. However, because the project manager was not visible to the team members, they did not see him as a having an understanding of their issues and successes. In addition, they did not see that it was necessary to ensure the project manager was informed on key issues, so they could be considered in light of the needs of the entire project.

This could have become a problem, if the resolution of an issue had required that the project manager make a decision that over-ruled the preferences of the manufacturing team. For example, perhaps a decision would have to be made to satisfy the needs of the project as a whole, but would be inconvenient for the manufacturing team to implement. In the existing situation, the project manager would not be seen by the manufacturing team as competent to understand the issue and intervene for the benefit of the project.

Fortunately, the manufacturing team lead recognized the problem and told the project manager what was happening. Since the project manager was dealing with an existing visibility problem, he was wise to introduce himself gradually into the team’s daily work. Since the team already saw the manufacturing team lead as the authority figure, a more forceful approach by the project manager could have made matters worse.

Gaining the respect of the team members required the project manager to make a daily investment of time in getting to know the manufacturing team members and allowing them to get to know him as well. The amount of time he spent each day was small, perhaps fifteen or twenty minutes, and was time well spent.

Copyright 2015 Debbie Gallagher

Crossing our fingers; that's our schedule

This is a true story. The company names have been changed.

Two projects

Acme was starting a project to implement a new logistics application, and engaged Standard Consulting as their implementation partner. When the logistics project started, a new manufacturing system was already in progress, and was also expected to complete before the logistics project.

There were a lot of custom reports required for both manufacturing and logistics. Acme had identified a few hundred necessary reports, including fifty reports for logistics that were critical to go-live. The report development for the manufacturing module had not gone well: many reports were incomplete and some reports that were done were buggy. Acme decided to take more control of the reports development process by assigning an Acme employee as the reporting team lead.

The schedule for reporting

The Standard Consulting project manager asked the new reporting team lead for his schedule, but the reporting team lead had no intention of developing one, as it seemed a waste of time. The project manager asked, “How do you know you’ll be done the critical reports in time for go-live?” The report team lead replied, “We’ll cross our fingers and hope for the best. After all, we always come through in the end.”

The project manager continued to insist on a schedule. Finally, the project manager prevailed. She had the two most experienced report developers stop work on reports. These two developers were assigned to define the development tool to use for each report, as well as the estimated effort for coding, testing, and promotion to production environment. The project manager used this information to build a schedule.

The reporting lead started to realize there was a lot more to report development than just the coding. In addition, the schedule made it clear that the fifty critical reports for go-live could not possibly be completed in time.

However, the reporting lead was worried about looking bad, and didn’t want anyone to know about the possibility that the reports might not get finished. He asked the project manager to avoid discussing the report schedule issue at the status meeting. Despite the schedule, he hoped the reports would get done on time.

The project manager insisted that since the problem had been identified, it had to be raised as a project issue. The reporting lead finally agreed.

Getting some help

The project manager and the reporting lead met with the logistics team and explained the problem. The logistics team was upset about the reports at first, but then agreed to help solve the problem.

The logistics team agreed to prioritize their reports and, with discussion, it became clear that only two reports actually had to be run on the day of go-live. Several others were not needed until the end of the first week, others at the two-week mark, and most at month end. There were even some reports that were not needed until year-end, which was six months after go-live.

When the reports issue was raised at the project status meeting, the manufacturing team offered to prioritize their remaining reports to free up resources to work on the distribution reports. In addition, the Acme project director offered to provide some extra report writing staff.

The manufacturing and distribution modules both were live on time. In addition, the critical reports for both modules were available by the newly prioritized dates.

Conclusions

The reporting lead’s approach of crossing his fingers and hoping for the best was not realistic. Once he understood and made the other team members aware of the reporting delivery problems, everyone pitched in to help solve the problem. The logistics team prioritized their reports realistically, the manufacturing team delayed some of their reports to free up resources, and the project director provided additional help. None of this could have happened if the team lead kept his problem secret.

Even though the client was responsible for its own report development, the consulting firm project manager was right to insist on a proper schedule for the reports. The reports were needed for the implementation, for which the project manager was responsible. Without the schedule, no one had any idea of the extent of work involved, and it was impossible to predict success or failure.

Copyright 2015 Debbie Gallagher

The expert sub-contractor

This is a true story. The company names have been changed.

Engaging the sub-contractor

When Acme Consulting was preparing to implement a work order management and call centre processing application at Standard Limited, Acme had several staff resources with expertise in the work order application. However, their only employee with call centre experience had never implemented this particular package. Acme had asked a sub-contractor to participate in the sales process and, now that they had won the contract, to work on the implementation.

Acme’s project manager had used the sub-contractor once before. On that previous assignment, the client staff had made some vague complaints about the sub-contractor, indicating her documentation and follow ups were inadequate. However, the client wasn’t very specific, and didn’t complain very much. The project manager knew that many other companies had used the sub-contractor and that she was generally recognized as having a lot of experience with that particular call centre system.

Standard’s project

As soon as the project started, the sub-contractor was dissatisfied.

·         As planned, Acme did hire the sub-contractor to do some of the call centre module work, but because this was a larger engagement, Acme also assigned their own resource to work on the team. The other resource had industry experience, but he did not have experience with this particular package, so was assigned a more junior role on the team. The sub-contractor complained to the project manager that she was not assigned enough hours on the project, as she expected to be assigned all of the call centre tasks.

·         The sub-contractor worried that her performance would be monitored by the more junior Acme resource.

·         The sub-contractor did not want to use Acme’s implementation methodology, but the project manager, the Acme resource and the client insisted on following it.

·         Differences in approach and work habits, such as attention to detail and quality of documentation, caused additional friction between the sub-contractor and the junior resource.

The project manager met with the Acme resource and the sub-contractor together to clarify roles and responsibilities. The sub-contractor agreed to follow the Acme Consulting methodology and both agreed to work together on ensuring the implementation was delivered according to the client’s requirements.

The problems continue

However, the client liked the more junior person’s work habits and approach, and over time, the junior resource informally led the team. In addition, although the sub-contractor appeared to be knowledgeable about the call centre software, the client and the junior resource did a lot of detailed analysis and didn’t always need the sub-contractor’s expertise.

There were occasions when the client felt the sub-contractor had given incorrect information about how the product worked, so the junior resource corrected the problems and did not charge the client for the time spent in re-work. Although the client was not overcharged, she felt that the sub-contractor should have been more knowledgeable so the work would not have to be re-done.

During the project, Standard Limited required some custom programming work to be done, and Acme Consulting had no resources of the right type available to do the work. So, when the sub-contractor proposed using her own staff to do the custom programming, Acme’s project manager did not object.

As the project continued, it became evident that the sub-contractor had continued to provide additional staff for more custom programming projects at Standard Limited. The sub-contractor team’s custom work became a source of confusion on the project. Who was responsible for delivery? Did Acme Consulting guarantee the work of the sub-contractor’s staff?

At the end of the project, Standard was generally pleased with the work done by Acme Consulting resources. However, they were very unhappy with the work done by the sub-contractor. In addition, when there were problems with the custom work, Standard called the Acme Consulting project manager, who had to refer them back to the sub-contractor.

Conclusions

Although the project manager had a queasy feeling about the complaints regarding the sub-contractor at the previous client, he ignored them because she was generally known as an expert. That queasy feeling was an intuitive recognition of risk and should not have been ignored. The project manager could have inquired further about the vague complaints at the previous client, and made a more informed decision about whether or not to use the sub-contractor at Standard Limited.

The sub-contractor assumed that Acme would use her for all of the implementation work because had no specific agreement had been made. Acme Consulting could have used the sub-contractor during the sales process and paid her for her time instead of agreeing to put her on the implementation project. Alternatively, a different resource could have been used, perhaps a different sub-contractor, or an Acme Consulting resource from another city or country.

Once Acme Consulting did decide to use the sub-contractor, she should have been managed more rigorously from the start. For example, a written agreement should have been drafted, covering the amount of work that was awarded to her, the methodology to be used, the quality of work expected, and how it would be measured. The agreement should have also specified how to terminate the sub-contractor in the event that the work delivered did not meet the standard required by the client.

The sub-contractor should have been specifically prohibited from presenting her own staff to work on other projects at the same client. The confusion caused by this practice led to a lack of satisfaction with Acme Consulting that was not directly attributable to Acme’s own work. 

In retrospect, the project manager says he made too many allowances for the sub-contractor, and should not have “fluffed off” his vague feelings of discontent with the sub-contractor’s work at the previous client.

Copyright 2015 Debbie Gallagher