In
previous articles, I described why your IT project might be audited, and an
overview of the major steps in the audit for an application development or
implementation project. In this article, I’ll provide some insight into the
evidence needed to demonstrate for the audit that the project team actually
executed the controls.
Examination
and testing of evidence is something the auditor is required to do in order to
satisfy auditing standards (see my earlier article What to expect when your IT project is audited).
Let’s
follow some examples for a few common project controls. I’ll go over the kind
of evidence that might be used, as well as some common problems with evidence.
Evidence that testing occurred
A
common control on IT projects is that the application is tested to verify that it
works and meets business requirements.
Relevant
evidence that testing took place and that the application worked often focuses
on the signed test scripts that were executed. These are particularly useful
evidence if they include expected results, as well as a notation and signature indicating
that the test passed. Additional supporting documents might be the task
completion status for testing tasks on the project schedule, or a summary document
capturing test pass/fail results for all scripts.
The
auditor may also examine test planning documentation in order to assess the
approach. For example, testing strategy, testing plans, project schedules
showing testing tasks and resources, and test scripts.
As
mentioned in my previous article Why your
IT project is being audited, the various auditors examining your IT project
may have differing scope and purpose. These differences will affect the
evidence they examine. For example, if an internal audit group needed to verify
that the company methodology was followed, they may need to see evidence to
support specific tests mandated by your company’s own standards.
Over
the course of performing many project audits, I noticed that often project
teams had documentation to support test planning,
but sometimes had problems with sufficient evidence supporting the test execution. This is a significant
distinction and is not just fussiness on the part of the auditor. It is
necessary that the evidence support the specific control.
Evidence
problems related to test execution can occur when testers do not sign their
completed test scripts, neglect to indicate that the test result was a pass, or
even throw away their completed test results. In addition, sometimes a tester
will keep scripts showing failure, but neglect to keep the follow up test results
showing that the problem was corrected.
Evidence of business approval
In
addition to the testing itself, it is customary for a key business
representative to approve the test results for the project.
Sometimes
the testing manager or project manager will ask the business approver to
respond to an email, indicating approval of the test results. This approach
sounds straightforward enough but in practice sometimes leads to inadequate
evidence.
A
common problem with email signoffs is that requesters and approvers may treat
them casually, leading to problems in using them as audit evidence later. One
evidence problem occurs when the wording does not make it clear what the approver
is approving. For example, if the
testing manager sends an email to the approver, “Please approve the attached,”
attaches the test results, then the approver replies “Approved”, the test
results are no longer attached and there is no indication in the email itself of
what is being approved.
A
simple solution is to be more specific in the request email. For example, “Please
approve the test results, which show successful completion of all general
ledger scripts, details attached.” With this approach, if the attachment is
removed in the email reply process, there is still an indication of what is
being approved.
A
significant evidence issue can arise when the approver adds irrelevant
information to the approval email. For example, “Results ok, please let me know
the resolution of the problem with ap-open.”
The
approver has started an entirely new subject, as ap-open has nothing to do with
the specific approval being requested. Unfortunately, there is no indication
that the ap-open problem is different from the subject of the approval, so it
leaves the auditor wondering how the approval could have been granted when
there appears to be a problem outstanding.
The
auditor has no choice, if presented with this email as an approval, except to
examine the situation further and obtain further evidence that the ap-open
problem had no relationship to the approval being granted. The auditor, remember, is mandated to examine
the documents to obtain sufficient evidence, and the wording of the email
suggests that the approval may not be valid. This kind of email subject change
invalidates the approval given until shown otherwise.
One
more problem can occur with email approvals, when the email deletion policy
eliminates the evidence prior to the audit! Remember to copy the approval
emails to a place where they will be retained.
Evaluating your own evidence
In
trying to determine in advance whether your evidence might be acceptable to the
auditors, review the documents and ask yourself this question “If I wasn’t on
the project team, and this set of documents was everything I knew about the whether the control happened, would I have
proof that the work was performed and found to be acceptable?”
Conclusion
What
frequently happens, I think, is that project team members do perform the work
and obtain the approvals. However, in
many cases, these aspects of the work may not be well documented, leading to problems
in supporting the audit later.
Now
that you know some of the evidence pitfalls, perhaps you can avoid them on your
next audit.
Copyright
2015 Debbie Gallagher
