In previous articles, I described why your IT project might be audited, and an overview of the major steps in the audit for an application development or implementation project. In this article, I’ll provide some insight into the evidence needed to demonstrate for the audit that the project team actually executed the controls.
Examination and testing of evidence is something the auditor is required to do in order to satisfy auditing standards (see my earlier article What to expect when your IT project is audited).
Let’s follow some examples for a few common project controls. I’ll go over the kind of evidence that might be used, as well as some common problems with evidence.
Evidence that testing occurred
A common control on IT projects is that the application is tested to verify that it works and meets business requirements.
Relevant evidence that testing took place and that the application worked often focuses on the signed test scripts that were executed. These are particularly useful evidence if they include expected results, as well as a notation and signature indicating that the test passed. Additional supporting documents might be the task completion status for testing tasks on the project schedule, or a summary document capturing test pass/fail results for all scripts.
The auditor may also examine test planning documentation in order to assess the approach. For example, testing strategy, testing plans, project schedules showing testing tasks and resources, and test scripts.
As mentioned in my previous article Why your IT project is being audited, the various auditors examining your IT project may have differing scope and purpose. These differences will affect the evidence they examine. For example, if an internal audit group needed to verify that the company methodology was followed, they may need to see evidence to support specific tests mandated by your company’s own standards.
Over the course of performing many project audits, I noticed that often project teams had documentation to support test planning, but sometimes had problems with sufficient evidence supporting the test execution. This is a significant distinction and is not just fussiness on the part of the auditor. It is necessary that the evidence support the specific control.
Evidence problems related to test execution can occur when testers do not sign their completed test scripts, neglect to indicate that the test result was a pass, or even throw away their completed test results. In addition, sometimes a tester will keep scripts showing failure, but neglect to keep the follow up test results showing that the problem was corrected.
Evidence of business approval
In addition to the testing itself, it is customary for a key business representative to approve the test results for the project.
Sometimes the testing manager or project manager will ask the business approver to respond to an email, indicating approval of the test results. This approach sounds straightforward enough but in practice sometimes leads to inadequate evidence.
A common problem with email signoffs is that requesters and approvers may treat them casually, leading to problems in using them as audit evidence later. One evidence problem occurs when the wording does not make it clear what the approver is approving. For example, if the testing manager sends an email to the approver, “Please approve the attached,” attaches the test results, then the approver replies “Approved”, the test results are no longer attached and there is no indication in the email itself of what is being approved.
A simple solution is to be more specific in the request email. For example, “Please approve the test results, which show successful completion of all general ledger scripts, details attached.” With this approach, if the attachment is removed in the email reply process, there is still an indication of what is being approved.
A significant evidence issue can arise when the approver adds irrelevant information to the approval email. For example, “Results ok, please let me know the resolution of the problem with ap-open.”
The approver has started an entirely new subject, as ap-open has nothing to do with the specific approval being requested. Unfortunately, there is no indication that the ap-open problem is different from the subject of the approval, so it leaves the auditor wondering how the approval could have been granted when there appears to be a problem outstanding.
The auditor has no choice, if presented with this email as an approval, except to examine the situation further and obtain further evidence that the ap-open problem had no relationship to the approval being granted. The auditor, remember, is mandated to examine the documents to obtain sufficient evidence, and the wording of the email suggests that the approval may not be valid. This kind of email subject change invalidates the approval given until shown otherwise.
One more problem can occur with email approvals, when the email deletion policy eliminates the evidence prior to the audit! Remember to copy the approval emails to a place where they will be retained.
Evaluating your own evidence
In trying to determine in advance whether your evidence might be acceptable to the auditors, review the documents and ask yourself this question “If I wasn’t on the project team, and this set of documents was everything I knew about the whether the control happened, would I have proof that the work was performed and found to be acceptable?”
What frequently happens, I think, is that project team members do perform the work and obtain the approvals. However, in many cases, these aspects of the work may not be well documented, leading to problems in supporting the audit later.
Now that you know some of the evidence pitfalls, perhaps you can avoid them on your next audit.
Copyright 2015 Debbie Gallagher