This document provides an overview of data analytics and computer-assisted audit techniques (CAATs) for internal auditors. It discusses conducting the audit process, including planning, determining objectives, obtaining and verifying information from IT systems and databases. It also covers using CAATs for data analysis, test techniques, and audit procedures. The document describes challenges for auditors in obtaining data access, defining metrics, and minimizing system impact. Finally, it discusses various CAAT types and their usage, including sampling, parallel simulation, and snapshot techniques.
1. 3/25/2019
1
Data Analytics - 2
Analytics in the Audit
based on Data Analytics for
Internal Auditors
by Richard Cascarino
About Jim Kaplan, CIA, CFE
President and Founder of AuditNet®,
the global resource for auditors (now
available on iOS, Android and
Windows devices)
Auditor, Web Site Guru,
Internet for Auditors Pioneer
Recipient of the IIA’s 2007 Bradford
Cadmus Memorial Award.
Author of “The Auditor’s Guide to
Internet Resources” 2nd Edition
Page 2
1
2
2. 3/25/2019
2
About AuditNet® LLC
• AuditNet®, the global resource for auditors, is available on the
Web, iPad, iPhone, Windows and Android devices and features:
• Over 3,000 Reusable Templates, Audit Programs,
Questionnaires, and Control Matrices
• Training without Travel Webinars focusing on fraud, data
analytics, IT audit, and internal audit
• Audit guides, manuals, and books on audit basics and using
audit technology
• LinkedIn Networking Groups
• Monthly Newsletters with Expert Guest Columnists
• Surveys on timely topics for internal auditors
• NASBA Approved CPE Sponsor
Introductions
Page 3
The views expressed by the presenters do not necessarily represent
the views, positions, or opinions of AuditNet® LLC. These materials,
and the oral presentation accompanying them, are for educational
purposes only and do not constitute accounting or legal advice or
create an accountant-client relationship.
While AuditNet® makes every effort to ensure information is
accurate and complete, AuditNet® makes no representations,
guarantees, or warranties as to the accuracy or completeness of the
information provided via this presentation. AuditNet® specifically
disclaims all liability for any claims or damages that may result from
the information contained in this presentation, including any
websites maintained by third parties and linked to the AuditNet®
website.
Any mention of commercial products is for information only; it does
not imply recommendation or endorsement by AuditNet® LLC
3
4
3. 3/25/2019
3
About Richard Cascarino, MBA,
CIA, CISM, CFE, CRMA
• Principal of Richard Cascarino &
Associates based in Colorado USA
• Over 28 years experience in IT audit
training and consultancy
• Past President of the Institute of
Internal Auditors in South Africa
• Member of ISACA
• Member of Association of Certified
Fraud Examiners
• Author of Data Analytics for Internal
Auditors
5
Today’s Agenda
Conducting the Audit
Audit Planning
Determining Audit objectives
Obtaining Information from IT Systems for Analysis
Databases / Big Data
The Download process
Access to data
Data verification
Use of Computer Assisted Audit Techniques
Test Techniques
CAATs for Data Analysis
Generalized Audit Software
Audit Procedures
CAAT Usage
Page 6
5
6
4. 3/25/2019
4
Risk Analysis and Internal
Auditing
Estimating the significance of the risk
Assessing the likelihood or frequency of the risk
Considering how the risk should be managed
What actions need to be taken
What controls need to be effected
Preventative procedures - reduce the
significance or likelihood of the risk occurring
Displacement procedures - offset the effect if
it does occur
Risks are normally evaluated before the
mitigating effects of controls are considered
7
Risk Analysis and Internal
Auditing
Estimating the significance of the risk
Assessing the likelihood or frequency of the risk
Considering how the risk should be managed
What actions need to be taken
What controls need to be effected
Preventative procedures - reduce the
significance or likelihood of the risk occurring
Displacement procedures - offset the effect if
it does occur
Risks are normally evaluated before the
mitigating effects of controls are considered
8
7
8
5. 3/25/2019
5
Several Ways of Defining
Risk
Risk is the possibility of loss
Risk is the probability of loss
Risk expresses a possible loss over a specified
period of time
Risk is the potential for realising unwanted,
negative consequences
Risk measures the probability and severity of
adverse effects
Risk is a function of the probability that an event
will occur and the consequence if it does
9
Other Ways of Seeing Risk
Velocity
Readiness/Preparedness
Capacity
Controllability
Monitorability
Interdependencies
Frequency of occurrence
Volatility
Maturity
Degree of confidence
9
10
6. 3/25/2019
6
The One we Worry About -
Velocity
Speed of onset
How quickly does the risk descend upon us?
Do we have much warning?
Speed of impact
Do we feel the effects right away, or does the
pain slowly increase?
Does it spread and impact us in other ways;
e.g. reputation?
Speed of reaction
Even if we see it coming, do we have the agility
to timely react?
A Risk-based Planning Approach
Typical audit scope issues:
Audit frequency
Fixed frequency
Random frequencies
Conditional approach based on analytical review or risk analysis
Audit intensity
Not always more time in the riskier areas
Audit timing
Involves a variety of objectives and constraints
12
11
12
7. 3/25/2019
7
Risk-based Audit Steps
Define the audit universe of auditable units
Identify the appropriate risk factors reflecting
management's' concerns
Select an appropriate format for evaluating risk
factors
Assess a concern index for each unit reflecting its
riskiness over several risk factors
Based on the risk rating, assign an audit frequency
on a methodical and standardised basis
Produce the audit coverage plan
13
Design of Audit Steps
All risk evaluation designed to restrict audit work to
high-impact areas
Better return on investment
More defensible recommendations
More saleable results
Does not remove the need for actual audit work
Generally audit testing falls into the categories:
Do the controls work?
Do they meet their control objectives?
Are they efficient?
14
13
14
8. 3/25/2019
8
Selecting Controls for Testing
15
–Establish "prime" Controls for an Area
–Identify Controls covering several Areas
–Identify Stand-alone Controls
–Controls which provide Evidence
–Do NOT try to prove a Negative
Primary Areas of Concern
16
–Complex Systems cannot be re-created manually
–Many computer records are intelligible only to computers
–Most systems allow multiple access
–"Computers can be trusted"
–Disasters really mean Disaster
15
16
9. 3/25/2019
9
Where to Start
Establish audit objectives and requirements
Gain executive-level support
Ascertain degree to which management is
performing monitoring role
Select appropriate technology solutions
Identify information sources and gain access
Understand business processes and identify
key controls and risks
Build audit skill set
Manage and report results
Big Data Definition
No single standard definition…
“Big Data” is data whose scale, diversity, and
complexity require new architecture,
techniques, algorithms, and analytics to
manage it and extract value and hidden
knowledge from it…
18
17
18
10. 3/25/2019
10
How much data?
Google processes 20 PB a day (2008)
Wayback Machine has 3 PB + 100
TB/month (3/2009)
Facebook has 2.5 PB of user data + 15
TB/day (4/2009)
eBay has 6.5 PB of user data + 50 TB/day
(5/2009)
CERN’s Large Hydron Collider (LHC)
generates 15 PB a year
Type of Data
Relational Data (Tables/Transaction/Legacy
Data)
Text Data (Web)
Semi-structured Data (XML)
Graph Data
Social Network, Semantic Web (RDF), …
Streaming Data
You can only scan the data once
19
20
11. 3/25/2019
11
Characteristics of Big Data:
1-Scale (Volume)
Data Volume
44x increase from 2009 2020
From 0.8 zettabytes to 35zb
Data volume is increasing
exponentially
21
Exponential increase in
collected/generated data
Characteristics of Big Data:
2-Complexity (Varity)
Various formats, types, and
structures
Text, numerical, images, audio,
video, sequences, time series,
social media data, multi-dim
arrays, etc…
Static data vs. streaming data
A single application can be
generating/collecting many
types of data
22
To extract knowledge all these types of
data need to linked together
21
22
12. 3/25/2019
12
Characteristics of Big Data:
3-Speed (Velocity)
Data is begin generated fast and need to be
processed fast
Online Data Analytics
Late decisions missing opportunities
Examples
E-Promotions: Based on your current location, your purchase
history, what you like send promotions right now for store next to
you
Healthcare monitoring: sensors monitoring your activities and body
any abnormal measurements require immediate reaction
23
3 Vs of Big Data
The “BIG” in big data isn’t just about volume
24
23
24
13. 3/25/2019
13
The 4V’s
25
Big Data Usage
Transactional
•Fraud detection
•Financial services / stock
markets
Sub-Transactional
•Weblogs
•Social/online media
•Telecoms events
Non-Transactional
•Web pages, blogs etc
•Documents
•Physical events
•Application events
•Machine events
25
26
14. 3/25/2019
14
Main Big Data Technologies
Hadoop NoSQL Databases Analytic Databases
Hadoop
•Low cost, reliable scale-
out architecture
•Distributed computing
Proven success in Fortune
500 companies
•Exploding interest
NoSQL Databases
•Huge horizontal scaling
and high availability
•Highly optimized for
retrieval and appending
•Types
• Document stores
• Key Value stores
• Graph databases
Analytic RDBMS
•Optimized for bulk-load and
fast aggregate query
workloads
•Types
• Column-oriented
• MPP (Massively
Parallel Processing)
• In-memory
Challenges for the Auditor
How to efficiently and cost effectively sustain
controls assessment and testing efforts?
How to know on a timely basis when control
deficiencies occur?
How to quantify the impact of control
deficiencies?
How to improve effectiveness of controls
How to gain assurance over ongoing
effectiveness of controls
27
28
15. 3/25/2019
15
Deficiencies of Traditional
Approach
Retrospective view
analysis frequently occurs long after
transaction has taken place, too late for action
Lack of timely visibility into control risks and
deficiencies
Alternatively
Independently test all transactions for
compliance with controls at, or soon after, point
at which they occur
Not feasible with Big Data
Importing the Data
30
Bring a copy to the audit machine
Copies can be reanalyzed later if need be
Live data moves on
You cannot corrupt live data working on a copy
Bringing it into the audit software
Depends on the software
Most modern systems can import from a variety of data types
What’s where in the data
Data layout is critical
May automatically extract the data layout from metadata (data
about the data)
ODBC databases
Excel layouts etc.
If the structure is flat you will need the file layout from IT (Make
sure it’s up-to-date)
29
30
16. 3/25/2019
16
Acquiring the Data
31
If all you can get is the hard copy
Can they print it to a file instead
Comma Delimited if possible
Fred Smith, Internal Audit,3/13/2011,
Individual data fields separated by commas
Easy for the software to identify individual fields
If it’s a printout scan it
1 field of 120 characters for example
The audit software will allow you to define fields within the 120
characters
You can even define different layouts for different rows
Verifying the Data
32
You’ve got the data – now what?
Make sure it’s what you asked for
Timeliness – does it reflect the right period?
Accuracy – is it the live data?
Completeness – is it all the data?
It’s embarrassing to come to an adverse conclusion only to find you were
given the “wrong” file / layout etc.
Its even worse if you came to a non-adverse conclusion
Check against known
Control totals
Dates
Transactions
Never believe what the first printout tells you
31
32
17. 3/25/2019
17
Challenges for the Auditor
Obtaining appropriate data access
Defining appropriate measurement metrics
Setting appropriate thresholds for exceptions
reporting
Developing appropriate metrics to prioritize
exceptions
Minimizing impact on systems’ operational
performance
Source code review
–Requires programming skill
–Slow
–Expensive
–Boring
–Proves little
–May be useful for specialized review
33
34
18. 3/25/2019
18
Confirmation of Results
35
e.g. Debtors certification
–Slow
–Uncertain
–Only shows up errors in your favor
–Very labor intensive
Test Data
36
–Selected to test both correct data and errors
–Require little technical background
but Lacks Objectivity
–Influenced by what is expected
–Assumes program tested is "LIVE" program
35
36
19. 3/25/2019
19
Integrated Test Facility (ITF)
37
–Establishes a "dummy" entity
–Process data together with live data
–Excluded from live results
–Under the auditor's control but
–May result in system catastrophe
Advantages of an ITF
38
–Little technical training required
–Low processing cost
–Tests system as it routinely operates
–Understood by all involved
–Tests manual function as well as computer
37
38
20. 3/25/2019
20
Disadvantages of an ITF
39
–ITF transactions must be removed before
they interfere with live totals
–High cost if live systems require modification
to implement
–Test data affects live files - danger of
destruction
–Difficult to identify all exception conditions
–Quantity of test data will be limited
Snapshot Technique
40
–A form of transaction trail
–Identifiable inputs "tagged"
–Trail produced for all processing logic
–Useful in high-volume systems
–Used extensively by I.S. staff in testing systems
39
40
21. 3/25/2019
21
Sampling
41
–"Liars, Damned Liars and Statistics"
–A tool for audit quality control
–May be the only tool possible in a high-volume system
–Not well understood by auditors
–At computer speeds 100% sampling may be practicable
May not be desirable
Parallel Simulation
42
Uses same input data
Uses same files
Uses different programs
From a different source
To produce the same results?
41
42
22. 3/25/2019
22
CAAT Types and Their Usage
43
–Application audit tools are not always CAATs
–"Any tangible aid that assists an auditor"
Tools to obtain information
Tools to evaluate controls
Tools to verify controls
Automated tools
Automated Tools (CAATs)
44
Test Data Generators
Flowcharting Packages
Specialized Audit Software
Generalized Audit Software
Utility Programs
43
44
23. 3/25/2019
23
Specialized Audit Software
45
Can accomplish any audit task but
–High development and maintenance cost
–Require specific I.S. skills
–Must be "verified" if not written by the auditor
–High degree of obsolescence
Generalized Audit Software
46
"Prefabricated" audit tests
Each use is a one-off
Auditor has direct control
Lower development cost
Fast to implement
IDEA
ACL
Arbutus Analyzer
45
46
24. 3/25/2019
24
Application of GAS
47
Detective examination of files
Verification of processing controls
File interrogations
Management inquiries
Types of Audit Software
48
Program generators
Macro languages
Audit-specific tools
Data downloaders
Micro-based software
47
48
26. 3/25/2019
26
Determining the Appropriate
CAAT
51
Depends on the Audit Objective and
selected technique
Application Audit Techniques
Purposes
–1 To verify processing operation
–2 To verify the results of processing
Common CAAT Problems
52
–Getting the wrong files
–Getting the wrong layout
–Documentation is out of date
–Prejudging results
Never believe what the first printout tells you
51
52
27. 3/25/2019
27
In any Application System
53
–Try to identify the controls the user relies on
–Documentation is often misleading
–Not everything needs to be audited
–Program logic mirrors business logic
–You can always ask for help
Industry-Related Software
54
–Audit procedures commonly available for:
Accounts receivable
Payroll
General ledger
Inventory
–May be customizable
–Industry-related audit software available for:
Insurance
Health care
Financial services
53
54
28. 3/25/2019
28
Industry- Related Drawbacks
55
–Requires
Conversion of input to standard package
layouts
Selection of appropriate parameters
A degree of IS skill for conversion
–Software itself normally
Cost-effective
Efficient
Customized Audit Software
56
–To run in unique circumstances
–To perform unique audit tests
–To produce output in unique formats
–Expensive to develop
–Normally require a high level of IS skills
–May not tell you what you think they do
–May be the only viable solution
55
56
29. 3/25/2019
29
Information Retrieval Software
–Report writers and Query Languages
–Not specifically written for auditors
–Can perform many common audit routines
–Includes
Report writers
Program generators
4th generation languages
Excel as a CAAT
58
57
58
31. 3/25/2019
31
Questions?
Any Questions?
Don’t be Shy!
AuditNet® and cRisk Academy
If you would like
forever access to this
webinar recording
If you are watching
the recording, and
would like to obtain
CPE credit for this
webinar
Previous AuditNet®
webinars are also
available on-demand
for CPE credit
http://criskacademy.com
http://ondemand.criskacade
my.com
Use coupon code: 50OFF
for a discount on this
webinar for one week
61
62
32. 3/25/2019
32
Data Analysis Webinar Series
March 26 - Analytics Techniques
April 2 - Analysis and Monitoring
April 16 - Data Analytics Software
April 23 - Using the Analysis
Thank You!
Jim Kaplan
AuditNet® LLC
1-800-385-1625
Email:info@auditnet.org
www.auditnet.org
Richard Cascarino & Associates
Cell: +1 970 819 7963
Tel +1 303 747 6087 (Skype Worldwide)
Tel: +1 970 367 5429
eMail: rcasc@rcascarino.com
Web: http://www.rcascarino.com
Skype: Richard.Cascarino
Page 64
63
64