The presentation provides an overview of data analytics concepts and tools that can be used for internal auditing. It discusses how audit analytics can help challenge traditional audit views and provide additional services while maintaining independence. Examples are given of how analytics can be used for monitoring controls, enhancing audits, and ad-hoc analysis of risks. Key lessons focus on ensuring diversity in analytic teams and being prepared to replace personnel. The presentation emphasizes using a toolbox approach to tools and affordably sourcing analytic talent from interns with the needed skills. Maintaining independence is discussed in the context of facilitating rather than directly implementing risk responses or managing risk.

1. Nate Anderson, Internal Audit, Sears
Cliff Nuxoll, Internal Audit, Sears

2. PRESENTATION OBJECTIVES
• Overview of data analytics concepts
– Summarize audit analytics concepts & tools
– Reinforce concepts through examples & lessons
– Analytics team best practices
– Present practical tools & approaches to analytics
• Challenge traditional view of Audit Analytics
– Consider services Audit can provide while remaining
independent and objective

3. OUTLINE
• Audit analytics
– Overview
• Key ingredients to audit analytics
– Methodology & Approach
– Building an analytics team
– Overview of commonly used tools
• Analytics in action
– Monitoring controls
– Audit aids
– Ad-hoc analysis
• Lessons learned
• Maintaining Independence & Objectivity

4. AUDIT ANALYTICS OVERVIEW
• Definition
• Industry Insights
• Key Trends
• Key Ingredients

5. AD-HOC ANALYSIS
Auditor obtains
useful data
Data is loaded
for analysis
Results
of analysis
Summary
insights
Goals: Test general hypothesis (e.g., determine
root cause for sample of negative margin sales)

6. AUDIT AUTOMATION
Auditor
aid engaged
Automated
routine
Results for
auditor
Analytics
Routine/Program
Goals: Improve efficiency, accuracy, or
effectiveness of audit processes

7. CONTINUOUS AUDITING / MONITORING
Analytics
Routine/Program
Data feed
to audit
Automated
routine
Output for
action/decision
Goal: Enable risk monitoring, support risk
decision, and/or facilitate control activity

8. STATISTICAL ANALYSIS / MODELING
Data feed
to audit
Stats/modeling
routine
Output for
action/decision
Goal: Descriptive statistics procedure or modeling to test
hypothesis, increase understanding, or make prediction

9. INDUSTRY INSIGHTS
• PwC 2014 State of the IA Profession Survey
• Protiviti 2015 IA Capabilities & Needs Survey

10. PWC 2014 STATE OF PROFESSION SURVEY
How is Internal Audit doing?
• 49% (senior mgmt) & 60% (board) believe IA is delivering
on expectations
• 45% (senior mgmt) & 70% (board) believe IA adds
significant value
• 29% (senior mgmt) & 51% (board) believe IA is leveraging
technology effectively in execution of audit services
Where are the opportunities for IA to improve?
• #1 area respondents want greater IA involvement in:
– Increased reliance on big data & analytics (80%)
• “[IA] functions should always be looking to add value by
expanding their capabilities in [data analytics].”

11. PROTIVITI 2015 IA SURVEY
• 5 of 7 areas (out of 36 total) where audit improvement is most
urgently needed relate to analytics.
• Data analytics skills were the top area of desired growth in 2013
(4 of top 5) and 2014 (6 of top 9)
“Need to
Improve” Rank
1 Auditing IT Security
1 (tie) Computer-assisted audit tools (CAATs)
3 Data analysis tools – data manipulation
4 Marketing internal audit internally
5 Fraud – monitoring
6 Data analysis tools – statistical analysis
7 Continuous auditing

12. PROTIVITI 2015 IA SURVEY
• “There continues to be significant dialogue
among internal audit functions about the need
to leverage technology-enabled auditing tools,
but they are not achieving progress.”
• “CAEs and internal audit leaders should
consider whether this is becoming a never-
ending journey”
• “Will [audit analytics] continue to be discussed
but not implemented?”

13. KEY TRENDS
• Democratization of data
• Visualization growth
• On-demand computing power

14. KEY TRENDS: DEMOCRATIZATION OF DATA
Major growth in data
Unstructured Structured
80% 20%
Majority is unstructured &
raises new opportunities &
concerns
New methods to
store, access &
analyze unstructured
data

15. KEY TRENDS: DATA VISUALIZATION GROWTH
Significant
advances in
visualization
tools

16. KEY TRENDS: ON-DEMAND COMPUTING POWER
Leverage cloud for
power & storage

17. KEY INGREDIENTS TO AUDIT ANALYTICS
Approach
Tools
Team
Methodology

18. AUDIT ANALYTICS METHODOLOGY
Problem to
analyze
Get/Process
data
Analyze
results
Measure
insights
Apply
learnings

19. ELEMENTS OF AGILE PHILOSOPHY
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.
Just do it.

20. AGILE MANIFESTO
“We are uncovering ways of developing software
by doing it and helping others do it. Through this
work we have come to value:
That is, while there is value in the items on the
right, we value the items on the left more.”
Individuals & interactions Over Processes & tools
Working software Over Comprehensive documentation
Customer collaboration Over Contract negotiation
Responding to change Over Following a plan

21. AGILE ELEMENTS WITHIN OUR
APPROACH
• Agile
– Obsess over problem to be solved
– No “analysis paralysis”
– Delivery early, often, and modestly (small releases)
– Improve incrementally
– Learn from reality quickly and with little money
• Traditional
– Dangerous set up: Design everything, code
everything, promise to deliver big later.
– Rigid scope and plan
– Over-reliant on consultants

22. ATTRIBUTES OF AGILE TEAMS
• Culture of transparency without penalties
• Reward early experimentation (and failure)
• Self-organizing and self-managing teams
• Cross-functional teams
“I had never failed. I’ve just found
10,000 ways which do not work.”
- Thomas Edison

23. CHANGING WITH TECHNOLOGY
Leverage data
warehouses
Leverage big data
Leverage open
source
1970 2015Time
Complexity

24. AUDIT ANALYTICS TEAM
Insights
Coder
Analyst
Business
Expert

25. SKILLSET: BUSINESS EXPERT
• Leverages personal insights and relationships
• Focus on solving real world problems
• Business unit experience
• Prioritize risks Problem to
analyze
Get/Process
data
Analyze
results
Measure
insights
Apply
learnings

26. SKILLSET: CODER
• Knows where and how to gather data
• Able to code in multiple languages
• Works well with key IT practitioners
• Developer experience
Problem to
analyze
Get/Process
data
Analyze
results
Measure
insights
Apply
learnings

27. SKILLSET: ANALYST
• Evaluate key risks based on data
• Drive solutions based on analysis
• Excellent problem solver
• Can visualize results
Problem to
analyze
Get/Process
data
Analyze
results
Measure
insights
Apply
learnings

28. ANALYTICS LEADERSHIP TEAM
CAE
AnalystsBusiness
Experts
Coders
• Sponsor key to success
• Must be open to any
approach that gets results
• Strong practitioner
• Great business knowledge
• Strong practitioner
• Understands how to
manage IT resources and
projects
Analysts
IT Audit
Lead
Corporate
Audit Lead

29. TYPICAL ANALYTICS PROCESS FLOW
Requirements
Business Expert Coder Analyst

30. LESSONS LEARNED: RESOURCING
1. Diversity is critical.
2. Be ready to replace key personnel.
Auditors Coders
Coders Business
Experts

31. AUDIT ANALYTICS TOOLS
Visualize
Analyze
Organiz
Acquire

32. MICROSOFT OFFICE SUITE
Acquire /
ETL
Organize Analyze Visualize Price Difficulty

33. TOP AUDIT ANALYTICS SOFTWARE
Acquire /
ETL
Organize Analyze Visualize Price Difficulty

34. GARTNER MAGIC QUADRANT – BI TOOLS
Top tier
Open source
Completeness of Vision
AbilitytoExecute

35. TOP VISUALIZATION SOFTWARE
Acquire /
ETL
Organize Analyze Visualize Price Difficulty

36. MICROSOFT BI TOOLSET
Acquire /
ETL
Organize Analyze Visualize Price Difficulty

37. TOP BI OPEN SOURCE (FREE)
Acquire /
ETL
Organize Analyze Visualize Price Difficulty

38. TECHNOLOGIST TOOLS
Acquire /
ETL
Organize Analyze Visualize Price Difficulty

39. ANALYTICS SOLUTION EXAMPLES
• Monitoring Controls
– Patriot Act Compliance
– Pharmacy Compliance
– Gift Card Compliance
• Audit Enhancement
– Access Benchmark
• Ad-Hoc Risk Analytics
– Gift card analytics
– Employee Store Risks
– Telecom spend

40. MONITORING CONTROLS
• Hosted web applications
– Patriot act compliance
– Pharmacy compliance
– Gift Card compliance
• Collaboration between business & audit
• Aid business in mitigating significant risks

41. PATRIOT ACT COMPLIANCE
• Replaced pre-existing weekly Excel reports with
continuous online tracking system – accuracy
improvement of 500%
• Findings are generated nightly and appended to the
current report
• Related transaction details are populated under each
finding

42. PHARMACY POLICY COMPLIANCE
• Requested by Legal to protect against costly fines
• LDAP-authenticated system requires Pharmacists and
Pharmacy Managers to agree/ disagree to policy on a
weekly basis
• Users sign in and enter pharmacy location number

43. PHARMACY POLICY COMPLIANCE
• Once signed into the system with a user id and location
number, users come to the policy page
• Upon agreement, user information and pharmacy location are
logged
• In the case of a
disagreement,
Managers & Directors
are notified via email
to take appropriate
action

44. GIFT CARD COMPLIANCE
Periodic review and action (sign-off) on potential risk events:
• Required sign-off
• Business unit management oversight of sign-off, participation,
risk events

45. AUDIT ENHANCEMENT
• Hosted web application
– Access benchmark
• Improves audit activities
• Typically enhances:
– Efficiency
– Effectiveness
– Uniformity of approach

46. ACCESS BENCHMARK
Concept:
- Access list repository for audit & IT compliance
- Regular snapshots of access for critical IT assets
- Enables self-service access reviews by control owners

47. ACCESS BENCHMARK – COVERAGE
Sarbanes-Oxley IT
Components
Count
Environments (LDAP, AD, etc.) 10+
Applications 50+
Databases 150+
Systems 200+
Datasets 50+
Production Directories 50+
Utilities 5+
• Implemented across LDAP, Active Directory, mainframe
hosts, Sun, AIX, Linux, HP-UX, Windows, AS/400, MySQL,
SQL Server, DB2, Oracle, Teradata, Informix, PeopleSoft,
etc.

48. ACCESS BENCHMARK – WALK-THROUGH
• Primary functions:
– Admin – Add IT assets, map reviewers, manage access
– Reviewer – Down/upload of mapped access reviews
– Auditor – Download of completed reviews

49. ACCESS BENCHMARK – REVIEWER VIEW
# of accounts
requiring review
All IT assets
related to user
Download
current list
Relevant
technology layer

50. ACCESS BENCHMARK – REVIEWER VIEW
Enabled drag and drop of
completed access reviews

51. Upload occurs; data
validation performed
ACCESS BENCHMARK – REVIEWER VIEW

52. ACCESS BENCHMARK – AUDITOR VIEW
Download List
Select technology
layer
Select review “as
of” date

53. ACCESS BENCHMARK – BENEFITS
• Effective access reviews and re-certifications
• Uniformity in approach & quality
• Enables 100% coverage (all IT assets & accounts)
• Solution is scalable (can leverage for SOX, PCI, etc.)
• Accurate “critical information asset” inventory
• Value of weekly access snapshots

54. AUDIT ENHANCEMENT “MUST HAVES”
• Ready access to:
– employee & contractor data
– Key transactional data access (e.g., point-of-sale)
• Statistical aides (assist with sample selection, etc.)
• Focus on repetitive activities in areas such as compliance

55. AD-HOC RISK ANALYTICS
• Conducted with desktop software
– Gift card analytics (tableau)
– Store employee risks (power bi)
– Telecom spend (tableau)
• Enhances risk assessments, audits
• Requires savvy & assertive auditors

56. GIFT CARD ACTIVITY OVER TIME
Continuous
control
implemented
Flawed program
launched; quickly
addressed

57. SUSPICIOUS ACTIVITY BY STATE
States with
significant activity
States where no
activity is allowed

58. SUSPICIOUS ACTIVITY BY DISTRICT
Districts with significant
suspicious activity

59. STORE EMPLOYEE RISKS
Shifts < 3 hours
Qty of edits
Qty of self-corrects
Qty of self-corrects

60. STORE EMPLOYEE RISKS
High qty of
self-corrections
to hours
High qty of
manual
hours edits
High qty of
both concerns

61. TELECOM SPEND
• Where is biggest cost recovery opportunity?
– Over allocation / overcharge
– Obscure service charges
– International call/text usage
– Unneeded feature removal
– Closed sites / lines not in use
– Call/text/data plan optimization
– General use overage

62. TELECOM SPEND: VENDOR 1
Quickly highlight key cost
recovery opportunities
~$350k
savings
proposed

63. TELECOM SPEND: VENDOR 2
Quick overview of amount of
recovery by reason
~$2.2m
savings
proposed
Top recovery reason:
Unused lines/circuits

64. TELECOM SPEND: CLOSED SITE/ UNUSED LINES
SHMC-38445 and SHMC-99999 may be
false positives; need more data
Abnormally large sites:
- Store
- Corporate

65. Significant number
relate to corporate
TELECOM SPEND: BY SITE

66. Identify greatest opportunities
for preventive controls
TELECOM SPEND: DRILL-DOWN ON CORPORATE
Visualization Summary:
• Quick, big-picture view
• Convey conclusions & approach to key stakeholders

67. LESSONS LEARNED
• Most valuable technical skill
• Toolbox approach
• Affordably sourcing team

68. MOST VALUABLE TECHNICAL SKILLS
1. SQL. And then really advanced SQL.
Learn it.
Love it.
Live it.
Essential for finding, browsing, evaluating, analyzing, and
filtering data
2. Excel – Lots can be done before limitations emerge
3. Tableau – Includes all essential ingredients
4. Depends on the need, familiarity, etc.

69. TOOLBOX APPROACH: BEST TOOL WINS
• What step are you on in your data analytics journey?
• How to move forward without:
– Looking too far ahead
– Spending unnecessary $$$
• Successful tools for Sears Holdings:
– Everyone: Excel, Access
– Front-end team: ACL, Tableau
– Back-end team
• Linux servers (free, powerful server)
• MySQL (free, powerful database)
• Cassandra (free, powerful NoSQL database)

70. AFFORDABLY SOURCING TEAM
1. Coders as interns
– Freedom and creativity of role should appeal to them
– Do not ask them to be auditors
2. Data analysts as interns
– Subject matter is attractive (fraud, security, etc.)
3. Auditors with coding background
– Increases likelihood of obtaining versatile data analytics
practitioners

71. ENTERPRISE RISK MANAGEMENT FAN
* Internal Audit acts as facilitator and host only

72. INDEPENDENCE & OBJECTIVITY
“Independence is the freedom from conditions that threaten
the ability of the internal audit activity to carry out internal
audit responsibilities in an unbiased manner.”
“Objectivity is an unbiased mental attitude that allows
internal auditors to perform engagements in such a manner
that they believe in their work product and that no quality
compromises are made. Objectivity requires that internal
auditors do not subordinate their judgment on audit
matters to others.”
– Section 1100 – Independence and Objectivity
International Standards for the Professional Practice of
Internal Auditing

73. INDEPENDENCE IMPAIRMENT THOUGHTS
• Are we “implementing risk responses on management’s
behalf”?
• Are we “taking accountability for risk management”?
• Are we remaining able to audit these controls without bias?
1. We are remaining independent of the performance of the
control, we are unbiased, while we are increasing our control
oversight.
2. We do not make risk response decisions; we do not manage
risk for management.
Most Importantly: If we never have to answer these questions, how
much value are we adding?

74. THANK YOU
Contact Information
Nate Anderson
nate.anderson@searshc.com
Cliff Nuxoll
cliff.nuxoll@searshc.com