Presentation by Joop Vrolijk, Netherlands, on Developing performance audit in transition countries, at the workshop organised by SIGMA for the Lebanese Court of Accounts on Introducing performance auditing in Supreme Audit Institutions, held in Beirut 28-29 June 2016.

1. © OECD
AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
Developing performance auditing in
Supreme Audit Institutions
Developing Performance Audit in Transition Countries
Joop Vrolijk, Netherlands
Beirut, 28 June 2016

2. AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
List of Content
1. What is Performance Audit?
2. What were triggers to start Performance Auditing?
3. How did SAI’s in SIGMA countries start?
4. Examples of Performance Audits
5. Performance Audit: a System Based approach
6. Conditions for success
7. Conclusions
2

3. © OECD
AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
EUROPEAN
COMMISSION
1.1 What is performance audit?
Three crucial questions:
1 What are the goals (policy/organisation)
2 To what extent the goals are achieved (in an
economical, efficient and effective way)?
3 If goals are not met, what is / are the reason(s)?
3

4. © OECD
AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
EUROPEAN
COMMISSION
1.2 Relation 3 E’s
4

5. AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU 2. What were triggers to start Performance Auditing?
- Donors (EU)
- Extension mandate
- Complying with INTOSAI standards
- Development of institution (SDP’s)
Was mostly: donor driven
Now more: becoming ISSAI proof
5

6. AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
3. How did SAI’s in SIGMA countries start?
• Mostly with support donors (EU) in twinning’s or TA projects by means of pilot audits
(EU chapter 32 requirement!);
• Mostly separate PA unit established: PA staff later on used as trainers for other PA staff
(ToT).
• Start with simple subject: no potential political risks but impact/attention of public,
easy for beginners, within a period which can be overseen (motivation of staff!)
• Simple subject mostly related to efficiency in financial management. Not really
effectiveness of organisation, let alone government policies, which can have high
failure risk
 too complicated
 takes too much time
 no information available
6

7. AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
4. Examples of Performance Audits
• Economy:
 Financial management of EU project ““Support to the development of land market” (Bulgaria
2004)
• Efficiency of:
 Government organisational procedure “ Process of employment of civil servants in the public
institutions” (B+H, 2009)
 Government public procurement system “ Public procurement of IT equipment” (B+H , 2010)
 Financial management of EU project “ Child Welfare Reform” (Bulgaria, 2004)
 Government management system “ Vehicle management” (Serbia, 2013)
 Government management system “Property administration” (Montenegro, 2011)
• Effectiveness of:
 Government policy “Necessity of liberalisation of electricity market (Albania, 2014)
 Municipal management system “Municipal waste system management” (Kosovo, 2013)
7

8. © OECD
AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
EUROPEAN
COMMISSION
5.1 Performance audit: a system
approach
Introduction of PA takes time:
 choose a step-by-step-approach
 start looking at systems
8

9. AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU 5.2 Example of Step-by step approach:
development of PA in B+H
Basis SIGMA Peer reviews and assessments
• Start in 2005 in three SAIs:
• Small steps: pilot audits
• In 2008:
• No interest of Parliaments in PA reports
• Findings and recommendations were too vague
• In 2012
• PA has reached satisfactory professional level.
• All reports had some impact, although not all recommendations were implemented.
• PA is appreciated by auditees, parliamentary committees and media
• PA had positive profile
• However PA was resource-intensive. SIGMA recommended
• consider splitting up larger topics into a sequence of smaller scale audits with one final, summarising
report at the end
• Narrow scope of PA’s
• Go for potential quick wins: topics that due to relative simplicity not need much resources 9

10. © OECD
AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
EUROPEAN
COMMISSION
5.3 A system approach: the policy cycle
Look from the AUDITEE’S perspective
• Preparation
• Designing
• Execution
• Monitoring
• Evaluation
• Follow up
10

11. AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU 6.1 Pre-requisites for starting Performance Auditing
- Legal mandate
- Sufficient resources available for PA -> Statutory audits
- Be part of overall SAI audit strategy (vision on development of PA)
- Acceptance that PA is time consuming and can have failure’s in starting up
phase
- Interest from and support by Parliament
- Organisational structure
- Availability of PA methodology and devoted and skilled staff
11

12. AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
6.2 What kind of structure is successful?
That depends. SAIs need tailor made solutions:
• Start with new small PA team (unit?) trained by PA external experts. That’s learning by
doing, after first experiences drafting a methodology (manual) and train other SAI staff
(ToT)
• Advantage: critical mass of PA staff available; give boost to development of PA;
driver
• Disadvantage: special position (and treatment!) in SAI
• Add to existing structures PA functions and start with combined Fin.Audit /PA audits; a
small central co-ordination point/working group for support and exchanging
experiences is needed
• Advantage: integration of PA in regular SAI audit activities; spreading PA over
whole SAI community; benefit from Financial Audit findings
• Disadvantage: without strict co-ordination PA methodology PA staff in SAI will
feel lost; difficult to develop PA SAI wise 12

13. 7. Conditions for having external impact
• Subject relevant for society: programming
• Capacity: quantative (complex audits)
and qualitative (skills)
• Timing of the publication (to whom will
message have to be addressed?)
• Being up to date (no surprises)
• Public relations (presentation, articles)
AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
13

14. AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
8. Conclusions
Step by step approach is mostly used and appeared to be effective
• start with piloting with external support
• draft methodology while piloting
• train special group of dedicated staff
• start with simple subject on economy and efficiency of financial
management systems with focus on desirable impact
IMPORTANT
Introduction and development of PA will take time.
14

15. AjointinitiativeoftheOECDandtheEuropeanUnion,
principallyfinancedbytheEU
THANK YOU!
QUESTIONS?
15