1. Performance AuditAdding ValueICGFM Conference May 19, 2011 Lily Bi, CIA, CGEIT, CISA Director, Standards and Guidance Institute of Internal Auditors

3. Internal Audit

4. Concept and Benefits of Performance Audit

5. Increase your ability to work with management in a positive and constructive partnership

6. The International Standards for Professional Practice of Internal AuditingAnalyze risks and develop a risk-based performance audit Learn a value-for-money approach for performance audit Final Thoughts – Trend of Internal Audit Profession

7. Program Topics Unit 1 - Understand the Landscape Unit 2 - Management Functions and Performance Measures Unit 3 - International Standards For Performance Audit Unit 4 - Risk-Based Approach (Case Study) Unit 5 - Value-for-Money Approach (Case Study) Unit 6 – Final Thoughts

8. Working Agreement P = Participation O = Openness S = Sense of fun E = Enthusiasm

10. The definition of internal Auditing

11. The definition of performance audit

13. Road Map of Internal Audit Modern Internal Audit 1941 - Internal Audit, a separate and distinctive discipline.

14. About the IIA Established in 1941, global headquarters in Altamonte Springs, Florida, USA Nonprofit professional association 170,000 members worldwide 103 national institutes worldwide Key focus: Standards-setting body for internal auditors Professional certifications Global research center Principal educator Global voice for the profession

15. Definition of Internal Auditing

16. Images of Internal Auditors Which metaphor do you like? Magnifying glass Telescope Compass Hunting dogs Watch dogs Policemen Consultants Eyes and ears of the Audit Committee

17. Definition of Internal Auditing Internal auditing is an independent, objectiveassurance and consulting activity designed to add value and improve an organization’s operations. It helps an organization accomplish objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes. Source: International Professional Practices Framework (IPPF) The Institute of Internal Auditors

18. Internal Auditing Is Add Value Independent Assurance Activity designed to Improve Operations Objective Consulting Activity

19. Internal Auditing Helps To Help To The Effectiveness of Organization accomplish it’s Objectives Risk Management Process Evaluate Control Process Improve Governance Process

20. Performance Audit

21. Definitions of PA INTOSAI: Performance auditing is an independent examination of the efficiency and effectiveness of government undertakings, programs, or organizations, with due regard to economy, and the aim of leading to improvements. US Government Auditing Standards:Performance audits are defined as engagements that provide assurance or conclusions based on an evaluation of sufficient, appropriate evidence against stated criteria, such as specific requirements, measures, or defined business practices. Performance audits provide objective analysis so that management and those charged with governance and oversight can use the information to improve program performance and operations, reduce costs, facilitate decision making by parties with responsibility to oversee or initiate corrective action, and contribute to public accountability.

22. Working Definition of PA Performance Audit is an independent and objective examination of a program, function, operation or the management systems of a governmental entity to: assure the entity’s objectives are carried out in an economic, efficient and effective way, and identify opportunity for improvement

23. Financial vs. Compliance vs. Performance Auditing

24. What Makes this Performance Audit? An Example: “…to determine whether laws, contracts, policies and procedures have been properly observed and whether all business transactions were conducted in accordance with established policies and with success. In this connection, the auditors are to make suggestions for the improvement of existing facilities and procedures, criticisms of contracts with suggestions for improvement, etc.”

25. Benefit of Performance Audit

26. Benefit of PA – Adding Value Relevant Focus on the key initiatives Flexible Define the scope of the audit based on risk Improving organizational performance Strengthen the governance Fraud prevention and detection Gaining public trust

27. Internal Audit Value Assurance = Governance, Risk Management, Control Insight = Catalyst, Analyses, Assessments Objectivity = Integrity, Accountability, Independence

28. Exercise - Connect the Dots o o o o o o o o o Connect all nine dots using just 4 lines without taking the pencil off the paper

29. Think Outside the Box o o o o o o o o o

31. Seeing the organization through the eyes of management

34. Communication

35. Customer Satisfaction

37. Management’s Roles

38. Performance Auditor’s Roles Evaluate the management processes and identify the heart of the problem Alert to actual and potential changes Identify the opportunity for improvement All units, programs, systems and activities are subject to internal auditor’s evaluations

39. See though the Eyes of Management Almost every deviation or deficiency results from the violation of some principle of management or good administration. See the organization and its activities through the eyes of management

40. Three Simple Questions to Ask Management What can go wrong? How do you it won’t go wrong? So what?

41. Performance Measures

42. Types of Management Performance Measures INPUTS - Measures of service efforts, e.g., number of hours, amount of materials. OUTPUTS - Measures of service level, e.g., number of residences served, amount of service provided. OUTCOMES - Measures of service accomplishments, e.g., measures related to program goals, including effectiveness of quality. EFFICIENCY - Measures that relate service efforts to service accomplishments, e.g., output/unit of input, productivity indexes.

43. Principles Measure only what are important to the organization Use of output-oriented measures Identify the total costs of service delivery Focus on continuous process improvement Performance measures should interconnect throughout the organization

44. One Example – Five Performance Categories: Effectiveness – the degree to which process output conforms to requirements Efficiency – the degree to which the process produces the output at a minimum cost of resources Quality – the degree to which the product or service meets customer expectations Timeliness – the degree to which a unit of work was done correctly and on time Safety – the measure of health and the working environment of the organization

45. Unit 3 International Standards For Performance Audit International Professional Practices Framework - IPPF from the IIA

46. Why the Standards Matter TheStandards Lead Represent Advancement of theProfession

47. Road Map of Internal Audit - Changestothe IIA Standards

48. The IIA’s IPPF International Professional Practices Framework

49. Mandatory Strongly recommended AUTHORITATIVE Guidance Authoritative=

50. Code of Ethics Integrity The integrity of internal auditors establishes trust and thus provides the basis for reliance on their judgment. Objectivity Internal auditors exhibit the highest level of professional objectivity in gathering, evaluating, and communicating information about the activity or process being examined. Internal auditors make a balanced assessment of all the relevant circumstances and are not unduly influenced by their own interests or by others in forming judgments. Confidentiality Internal auditors respect the value and ownership of information they receive and do not disclose information without appropriate authority unless there is a legal or professional obligation to do so. Competency Internal auditors apply the knowledge, skills, and experience needed in the performance of internal auditing services.

51. International Standards for Professional Practice of Internal Auditing

53. They set the bar that every auditor should comply with.

54. They give you a reference guide for how to conduct yourself.

55. They lay the ground work, but are not the ultimate goal.

58. Independence and Objectivity………………………………..1100

59. Proficiency and Due Professional Care……………….….1200

61. Nature of Work.……………………………………………….…………2100

62. Engagement Planning…………………………………….……..…2200

63. Performing the Engagement…………………………..……… 2300

64. Communicating Results………………………………..….………2400

65. Monitoring Progress………………………………………….……. 2500

68. Translated into 21 languages

69. Recognized or referenced by International Standards Setting Bodies, such as:

70. INTOSAI (IIA Standards are recognized globally for public sector audit professions)

71. Basel Committee on Banking Supervision

72. OECD Internal Audit Function

73. Referenced on the mandated legislation or regulation in countries or territories, such as

76. The importance of clearly defined business objectives and associated performance measures (goals) to a performance audit

77. Risk assessment using a Risk/Control Matrix methodology

79. IIA Standards Related to Performance Audit Process

80. Plan Performance Audit The most important part of an audit is the planning phase. Standard 2010 – Planning: The chief audit executive must establish risk-based plans to determine the priorities of the internal audit activity, consistent with the organization’s goals.

81. Plan Performance Audit Standard 2201 – Planning Considerations: In planning the engagement, internal auditors must consider: The objectives of the activity being reviewed and the means by which the activity controls its performance; The significant risks to the activity, its objectives, resources, and operations and the means by which the potential impact of risk is kept to an acceptable level; The adequacy and effectiveness of the activity’s risk management and control processes compared to a relevant control framework or model; and The opportunities for making significant improvements to the activity’s risk management and control processes.

82. Risk-based Performance Audit Start with an organization’s objectives and associated performance measures. Focus on an evaluation of performance risks and controls related to those objectives. Help the organization achieve the desirable goals and protect it from bad or undesirable things happening. Help reduce the chance of missed opportunities. Provide suggestions for improvement in controls designed to mitigate the risks associated with meeting performance objectives.

83. Risk Assessment Formula

84. Identification of Objectives Objectives are the things an organization wants to accomplish. Objectives should be S.M.A.R.T.

85. Objectives Cascade Mission Vision Objective 3 Objective 2 Objective 1 Sub-Objective Sub-Objective Sub-Objective Sub-Objective Sub-Objective Sub-Objective Sub-Objective Sub-Objective Sub-Objective

86. What is Risk Risks are things that could prevent an organization from meeting its objectives. IIA definition - Risk is the possibility of an event occurring that will have an impact on the achievement of objectives. Risk is measured in terms of impact and likelihood.

87. Business Risk Examples Erroneous records and/or information Business interruption (Government shutdown) Public criticism or legal action High costs Loss or destruction of assets Customer dissatisfaction due to ineffective program/service design Fraud or conflict of interest Inappropriate mgmt. policy and/or decision making process

88. Focusing on the “Real Risks” Operational 20% Strategic & Business 60% Financial 15% Compliance 5%

89. H High Risk Impact Total Audit Universe Low L H Likelihood Risk Assessment

91. Avoidance

92. Transfer

94. Considers their effect on event likelihood and impact, in relation to risk appetite and cost versus benefit

96. Residual (Net) - AFTER RISK REPONSEResponses Residual Risk

97. Exercise: Rain and Umbrella When it rains, where are Inherent and Residual Risk (IR and RR)?

98. When it rains, where are IR and RR? IR IR IR IR IR IR IR RR CR RR RR RR RR IR = All the raindrops RR = The raindrops outside the umbrella CR = Control Risk, possibility the umbrella leaks Risk Appetite = How big the umbrella is

99. What is Control Controls are things that help meet an organization's objectives. IIA Definition Control - any action taken by management, the board, and other parties to manage risk and increase the likelihood that established objectives and goals will be achieved. Management plans, organizes, and directs the performance of sufficient actions to provide reasonable assurance that objectives and goals will be achieved.

100. Control to Mitigate These Risks Erroneous records and/or information Business interruption Public criticism or legal action High costs Loss or destruction of assets Customer dissatisfaction due to ineffective program/service design Fraud or conflict of interest Inappropriate mgmt. policy and/or decision making process

101. Risk Management and Control Two sides of the same coin: Risk is managed by having in place the right controls to safeguard against its occurrence; Internal control exists only in relation to what they do to mitigate risk. Risk management and internal control are integrated parts of an entity’s overall governance and management system.

105. Mandated H AUDIT RESOURCES High Risk Impact Total Audit Universe * Low L H Likelihood Validate the Audit Plan Special Request

106. Case Study State Department of Fruit and Vegetable

108. Three E’s Performance Measures

109. Difference between Risk-Based and Value-for-Money approaches

110. Twelve Attributes for Evaluating Effectiveness

112. Value-for-Money Definition: VFM is utility derived from every purchase or every sum of money spent. VFM is based not only on the minimum purchase price (economy) but also on the maximum efficiency and effectiveness of the purchase. Looks at how well an organization provides value for money. Focuses on economy, efficiency, and effectiveness Based on the Twelve Attributes for Evaluating Effectiveness

113. Audit Performance Measures – 3E’s The principle of ECONOMY is keeping costs low. It requires that the resources used by the audited entity for its activities shall be made available in due time, in appropriate quantity and quality and at the best price. The principle of EFFICIENCY is getting the most from available resources. It is concerned with the best relationship between resources employed, conditions given and results achieved. The principle of EFFECTIVENESS is meeting the objectives set. It is concerned with attaining the specific aims or objectives set and/or achieving the intended results.

114. 12 Attributes For Evaluating Effectiveness Costs and Productivity Responsiveness Financial Results Working Environment Protection of Assets Monitoring and Reporting Management Direction Relevance Appropriateness Achievement of Intended Results Acceptance Secondary Impacts

115. Conducting Performance Audit- Planning Gather background information on the audit area. Understand the organization’s business, objectives, mission, etc. Interview management and staff. Use the twelve attributes to scope the audit by looking at each attribute to choose which are most applicable. For the selected attributes, form questions to be answered during the next phase.

116. Conducting Performance Audit- Examining and Evaluating The questions are answered through: - Interviews with management, employees and others - Industry research - Performance measures (criteria) - Benchmarking (criteria) - Other management and audit reports. - Site visits.

117. Conducting Performance Audit- Reporting and Following Up Communicating Results Phase Issues should be communicated to client throughout the audit. The report is written and presented to the client. Following Up Management implements action items from the report. Audit assists as required.

118. Case Study State Department of Fruit and Vegetable

122. Guided by international standards and enhanced emphasis on quality

123. Adoption of risk-based methodologies

124. Consulting service + assurance service

125. More independence and enhanced stature

126. Add value to the organization and stronger alignment

127. More strategic approach to staffing: out-sourcing and co-sourcing

128. Integration of IT and non-IT audit resources

129. Enhanced use of technology tools/services

132. “Step up to the plate” in risk management

133. Enhance internal audit knowledge of the business

134. Streamline internal audit processes and operations

136. Improving opportunities to achieve goals