Article, published on IJOnline, highlighting the importance of choosing the right model auditor, the risks of de-scoping the model audit, and how Navigant Consulting\'s is different and less risky.

1. Model Audit – do you know what risk you are taking on?
Andy Hucknall – Navigant Consulting
18 May 2009
If you have been involved with a project / infrastructure finance or PPP/PFI deal that
has a financial model (i.e. all of them!), you may have heard that the model needs
auditing, but why is it needed, what is a model audit and how is it carried out? It may
not sound like the most exciting part of transaction, but that does not make it any less
important.
The model audit market in the UK has been dominated by three players; but we at Navigant
Consulting are the new entrants, led by a senior team with more than 15 years of combined
experience. We have a different approach, focussing on risk rather price, and the feedback
that we are getting from both sponsors, and particularly funders, is that this is long overdue.
Model auditors for the past few years have been primarily selected on the basis of the
cheapest quote, leading to a “de-scoping” of the model audit. Model auditors have been doing
the minimum amount of work possible in order to keep the costs down. Typically, it is the
detailed ‘bottom up’ review of the financial model’s formulae that is avoided, with reliance
placed upon a ‘shadow model’ that seeks to re-perform the calculations.
Given the current riskier environment, and competitive dialogue procurement requiring project
sponsors to commit to their numbers at an earlier stage, we believe that “de-scoping” of the
model audit is a risky move.
We have always advocated the approach of combining the bottom-up approach of reviewing
every unique formula with high level analytical review, re-performance of key calculations,
sensitivity analysis and documentation review to ensure that the model is consistent with the
project documentation. Our team considers this to be less risky than using the shadow
modelling approach.
Why do you need a model audit?
Banks require a model audit as a condition precedent to lending, but it is the project sponsors
who pay for the model audit. It has been in the project sponsors’ interest to keep the cost
down, so they have pushed to de-scope the model audit as much as possible, relying upon
the model auditor’s professional indemnity (PI) insurance if something goes wrong.
We are increasingly seeing the banks demanding a full scope model audit as they see the
model audit as a key part of the risk management process. There are numerous documented
cases of undetected critical errors in spreadsheets leading to ill-informed decisions that can
result in mis-pricing, lower returns on investment than expected and misinformation being
provided to stakeholders and regulatory bodies. All may result in financial loss or reputational
damage.
Do you really want to take on a significant financial risk based upon a potentially erroneous
financial model? Assuming that the answer is no, de-scoping the model audit is a false
economy.

2. What is a model audit?
A model audit is an independent review of whether a financial model meets an agreed set of
objectives. The model auditor, thus, reports whether those objectives have been met, and any
exceptions.
A typical set of objectives may be that:
• the calculations in the model are arithmetically correct
• the results are materially reliable, accurate, complete and consistent with the
assumptions contained in the model
• certain ratios are calculated correctly and in line with the definitions from the
financing agreement
• the accounting treatments and assumptions applied within the Model are
consistent with current local GAAP and with key provisions of the Project’s
financing documentation
• the tax assumptions applied within the Model are consistent with current local tax
legislation and with key provisions of the Project’s financing documentation as
provided
• any unexplained trends or variations in key financial and banking indicators in
Model outputs are identified through analytical review
• any unexplained inconsistent or unintuitive cash-flow trends (including revenues,
costs, taxes, depreciation) or variations in key financial indicators based on the
inputs and the Project’s commercial structure are identified through analytical
review of Model outputs
• the results produced from changes to underlying assumptions accurately and
completely reflect the potential impact of those changes
If a model auditor is de-scoping, which of the above objectives can be removed? It is doubtful
whether any of the above should be removed, thus other techniques are used to limit the
scope (and the hence the cost) such as limiting the number of iterations of the model and
documentation that are included in the fee (with excessive cost for additional iterations),
charging for “extras” that should be included, such as third party duty of care letters, or
providing less affirmative final reports (audit opinion).
The last point is particularly important. A clear, affirmative statement in the final report that the
objectives have been met provides comfort to those relying on the report. If the model audit
has been de-scoped, the final report wording may simply state the agreed procedures were
performed, thus potentially absolving the model auditor of liability for omissions.
How should a model audit be undertaken?
A model audit should, first and foremost, be tailored to achieve the agreed objectives, and
thus, each model audit should be planned individually.
It should combine a detailed ‘bottom-up’ review of each unique formula with ‘top-down’
analysis, such as:
• the reperformance of key calculations based upon the Project’s documentation
• analytical review of trends in key outputs
• sensitivity analysis
• commercial sense checking
This combination of detailed ‘bottom up’ review with ‘top down’ analysis provides the best
possible chance of identifying material errors.
Experience shows that the detailed ‘bottom up’ review and ‘top down’ analysis identify
different errors that using only way technique may not identify, thus de-scoping the model

3. audit by ignoring the detailed ‘bottom up’ review significantly increases the risk of material
errors not being identified.
Academic research by the expert in this field, Prof. Ray Panko (University of Hawai’i), in his
paper “Spreadsheet Errors: What We Know, What We Think We Can Do” to the European
Spreadsheet Risk Interest Group (EuSpRIG) Symposium 2000 demonstrated that the
independent cell by cell inspection is almost the only way of systematically identifying errors.
We believe that the model audit should be assessed and planned by senior, experienced
staff, and that the senior staff should undertake the parts of the model audit identified as
higher risk during the planning stage.
Our senior team are all qualified accountants with a statutory audit background, and bring this
risk management approach to model auditing, whilst our team are all experienced
professionals with a corporate finance background. We just do not believe that model audit is
something that can be undertaken by graduates in their first job without wider finance
experience.
In many cases, the model auditor will simply limit costs by both avoiding work such as the
detailed review of formulae, and by using more junior, inexperienced staff, relying upon their
PI cover.
Shadow modelling – the risk
Some model auditors dismiss the detailed review of formulae as a “tick and bash” exercise,
even calling it ‘traditional’ model audit as if imply to imply it is out of date.
The alternative proposed by a few model auditors is ‘shadow modelling’ whereby project
inputs are put into their own model, and results compared, i.e. re-performance. That is not to
say that re-performance is not useful, rather it should be one of a range of techniques that is
used, rather than the only one.
Shadow modelling, thus, is limited to the “top down” elements of a full scope review, saving
cost, but increasing risk.
Your choice?
Given that shadow modelling is being used by several firms, always ask what approach your
model auditor will be using! If you do choose to de-scope the model audit to save money, be
aware of the extra risk that you are taking on, and ask is it worth it?
The model audit is a key financial risk management tool, not a box ticking exercise. As such,
it needs to include a proper scope of services to mitigate the financial risk that is being
committed to.
Given that financial commitment is occurring earlier in the process under competitive
dialogue, we are increasingly seeing some independent review taking place earlier in the
process, often known as a ‘pre-preferred bidder’. This is common practice in the rest of
Europe, but we hope and expect that project sponsors will require some independent review
of their financial models before preferred bidder.
Above all, you need to know what risk you are taking on - particularly if you are prepared to
risk a de-scoped model audit using shadow modelling.
Andy Hucknall is an Associate Director at Navigant Consulting.