1. Measuring Impact:
Ways to connect corporate
learning to strategic
business outcomes

2. Housekeeping
#LearningImpact
@Danij2
@CarolLeaman
@Axonify @Bersin
Event is being
recorded
Please
leverage the
Q&A window

3. Dani Johnson
Vice President, Learning and
Development Research
Bersin by Deloitte
Deloitte Consulting LLP
Carol Leaman
Chief Executive Officer
Axonify

4. Copyright © 2016 Deloitte Development LLC. All rights reserved.4 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Who we are
Global provider of leading practices, trends, and benchmarking research in talent management,
learning, and strategic HR.
6 research practices
• Human Resources
• Leadership Development
• Learning & Development
• Talent Acquisition
• Talent Management
• Tools & Technology
Offerings
• WhatWorks® Membership: Research, Tools, Education, Consulting
• IMPACT: The industry’s leading conference on the Business of Talent
• Advisory Services & Consulting
Human
Resources
Talent
Acquisition
Leadership
Development
Learning &
Development
Talent
Management
Tools &
Technology

5. Copyright © 2016 Deloitte Development LLC. All rights reserved.5 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
• Why is learning measurement important,
especially now?
• How do we improve learning measurement–
including connecting learning to business
outcomes?
• Case Study – measurement in practice.
Agenda

6. Learning measurement is becoming more important…
but L&D often isn’t good at it.

7. Copyright © 2016 Deloitte Development LLC. All rights reserved.7 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Measuring learning is becoming more important.
Source: High-Impact Talent Analytics, Bersin by Deloitte, 2013
State of Learning Measurement, Bersin by Deloitte, 2015;
Knowledge Advisors: The Business Drivers of Talent Analytics, Knowledge Advisors, 2014
Business leaders are looking
to L&D for data insights.
51%
Percent of surveyed leaders
asking for human capital
data

8. Copyright © 2016 Deloitte Development LLC. All rights reserved.8 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Measuring learning is becoming more important.
Source: High-Impact Talent Analytics, Bersin by Deloitte, 2013
State of Learning Measurement, Bersin by Deloitte, 2015;
Knowledge Advisors: The Business Drivers of Talent Analytics, Knowledge Advisors, 2014
81%
77%
58%
56%
15%
Finance
Operations
Sales
Marketing
HR
Other departments use
data to make decisions.
Business leaders are looking
to L&D for data insights.
51%
Percent of surveyed leaders
asking for human capital
data

9. Copyright © 2016 Deloitte Development LLC. All rights reserved.9 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Measuring learning is becoming more important.
Source: High-Impact Talent Analytics, Bersin by Deloitte, 2013
State of Learning Measurement, Bersin by Deloitte, 2015;
Knowledge Advisors: The Business Drivers of Talent Analytics, Knowledge Advisors, 2014
81%
77%
58%
56%
15%
Finance
Operations
Sales
Marketing
HR
Other departments use
data to make decisions.
Business leaders are looking
to L&D for data insights.
51%
Percent of surveyed leaders
asking for human capital
data
Research indicates that
better measurement ties to
better performance.
Alignment
Efficiency
Effectiveness

10. Copyright © 2016 Deloitte Development LLC. All rights reserved.10 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Only five percent of L&D organizations we
surveyed excel at using data to:
5% • Align with the business,
• Efficiently run the L&D function,
• Increase the effectiveness of
learning methods.
Source: State of Learning Measurement, Bersin by Deloitte, 2015

11. Copyright © 2016 Deloitte Development LLC. All rights reserved.11 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Help improve learning measurement by:
Transforming data
into decisions.
Create partnerships and
access to data
Aligning measurement with
business priorities.
Build and maintain relationships
with stakeholders and understand
how they define success
Measuring
business Metrics.
Focus practices, resources and
tools on measuring business – not
busy - matters

12. Aligning measurement with business priorities.

13. Copyright © 2016 Deloitte Development LLC. All rights reserved.13 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Focusing on business (not learning…) outcomes.
…which are typically reflected in learning business plans
69%
58%
48%
38%
35%
27%
19%
Budget
Program investment priorities / plans
Staffing
Volume (e.g. learners, hours)
Business impact goals
Customer satisfaction targets
We do not have a business plan for L&D.
Source: State of Learning Measurement, Bersin by Deloitte, 2015

14. Copyright © 2016 Deloitte Development LLC. All rights reserved.14 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Building ongoing relationships with stakeholders.
…L&D should create and maintain alignment with business needs
Source: State of Learning Measurement, Bersin by Deloitte, 2015
77%
64%
42%
40%
Meet with line managers and business leaders
to diagnose problems and develop solutions
Enlist executives or business leaders as
sponsors
Hold regular reviews with a steering committee
of business and HR leaders
Require line managers and business leaders to
sign off on new learning initiatives

15. Copyright © 2016 Deloitte Development LLC. All rights reserved.15 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Building ongoing relationships with stakeholders.
…Learners are stakeholders too!
2%
3%
2%
3%
10%
9%
14%
15%
20%
24%
11%
16%
20%
23%
25%
32%
34%
40%
36%
39%
31%
33%
38%
35%
39%
38%
34%
30%
28%
21%
56%
48%
40%
39%
26%
21%
18%
15%
16%
16%
Not important Somewhat important Very important Essential
Sharing knowledge with my team
Web search for resources
Professional networks / communities
External news feeds and blogs
Content curated from external sources
Self-directed study / courses
Internal company documents
Performance support tools
Company training / e-learning
General meetings and conversations
Source: LearningintheWorkplaceSurvey, Centrefor Learning&PerformanceTechnologies/ JaneHart,, 2014

16. Focusing on measuring business metrics.

17. Copyright © 2016 Deloitte Development LLC. All rights reserved.17 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Using metrics typically considered valuable by the business.
0%
25%
50%
75%
100%
0% 25% 50% 75% 100%
Valuable
Tracked / Measured
Impact on behavior / job
performance
Sponsor / client
satisfaction
Business results
Impact on talent /
organizational metrics
Line manager satisfaction / feedback
Content access methods
Learning scores /
transfer
Learner
demographics
Hours delivered /
time spent learning
Costs / expenses
Learner
satisfaction /
feedback
Enrollments, starts,
usage, and completions
Source: State of Learning Measurement, Bersin by Deloitte, 2015

18. Copyright © 2016 Deloitte Development LLC. All rights reserved.18 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Thinking holistically about measurement.
Outcome
(the business)
Efficiency
(L&D)
Alignment
(managers /
leaders)
Effectiveness
(learners)
• Time to develop
• Cost to develop / deliver
• Cost of development per minute / hour
• Cost per participant
• Resource hours to develop / deliver
• Total enrollments
• Learning offerings correspond to competency model
• Learning has strong executive sponsors
• Managers support participation among team
members
• Program objectives align with business needs
• New behaviors reflect what was introduced in a
learning offering
• New knowledge and skills
• Learner satisfaction
• Adoption
• Key performance indicators (KPIs)
• Revenue
• Profitability and growth
• Market share
• Customer / satisfaction
• Employee engagement / retention
• Productivity
Source: State of Learning Measurement, Bersin by Deloitte, 2015

19. Copyright © 2016 Deloitte Development LLC. All rights reserved.19 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Measuring against established business objectives.
Frequency with which L&D groups evaluate themselves against…
Source: State of Learning Measurement, Bersin by Deloitte, 2015
16%
22%
28%
59%
28%
7%
9%
7%
21%
18%
19%
10%
Satisfaction targets
Progress toward strategic initiatives
Volume targets (e.g., programs, hours delivered)
Sales performance or targets
Weekly/Monthly After learning events Rarely/Never

20. Transforming data into decisions.

21. Copyright © 2016 Deloitte Development LLC. All rights reserved.21 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Applying the basics.
Dedicate resources and
manpower 11%
Of companies surveyed spend
more than 5% of their budget on
measurement. 28% don’t even
know.
Build relationships with
stakeholders 1.2x
Companies that regularly engage
with stakeholders are more likely to
have high-impact measurement
capabilities.
Have a plan
19%
Of those we surveyed said their
companies don’t have a learning
business plan for L&D.
Source: State of Learning Measurement, Bersin by Deloitte, 2015;
Becoming a High-Impact Learning Organization, 8/2012

22. Copyright © 2016 Deloitte Development LLC. All rights reserved.22 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Kirkpatrick method is only the beginning
Source: Key Trends in the Measurement of Talent Development, 3/2012
Rethinking yesterday’s tools and practices.
Too simplistic…
Does liking a course or
passing a test really
translate directly into
better performance or
business results?
Too hard…
There are no clear,
consistent definitions and
it’s rarely simple or easy
to isolate learning
contribution (…or L&D’s).
And too backward
Finding out after training
that people were not
satisfied – or that results
didn’t improve – doesn’t
tell you what to change.

23. Copyright © 2016 Deloitte Development LLC. All rights reserved.23 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Utility / usefulness
Stakeholder satisfaction
Experience API (xAPI) Employee engagement
Learning records stores Readiness / promotions
Information technology Adaptive learning Employee retention
Finance and accounting Web analytics Time to proficiency
Marketing Social listening Activity / contributions
Business intelligence CRM systems Influence and feedback
New friends New tools New metrics
Building (some) new capabilities.

24. Copyright © 2016 Deloitte Development LLC. All rights reserved.24 Measuring Impact: How to Connect Corporate Learning to Strategic Business Outcomes
Keep business priorities at the forefront
Learning measurement may require a new mindset.
• Relationships
• Learning business
plan
• Frequent
measurement
• Holistic approach
• Identify who will
benefit
• Practices, resources,
capabilities, & tools
Actionable data
through:
• Partnerships
• Data transparency
FocusAlign Transform

25. Case Study: Measurement in practice.

26. Case Study: Large Retailer
• Over 75,000 employees
• Currently focused on logistics and distribution network
• Over 150 facilities across the U.S.
• Over 400,000 learning sessions (1.4M Questions) per Month
• 1 Million Safety Observations recorded per Month

32. Assess Ÿ Predict Ÿ Adjust Ÿ Optimize

33. Assess Hope

34. Assess Ÿ Predict Ÿ Adjust Ÿ Optimize

35. Assess Ÿ Predict Ÿ Adjust Ÿ Optimize
Large Retailer

36. Assess Ÿ Predict Ÿ Adjust Ÿ Optimize
Did the number of
safety incidents
meet expectations?
Large Retailer

37. Assess Ÿ Predict Ÿ Adjust Ÿ Optimize
Did they actively
handle the materials
correctly?
Did the number of
safety incidents
meet expectations?
Large Retailer

38. How do we measure the behaviors?

39. How do we measure the behaviors?

40. How do we measure the behaviors?

41. Assess Ÿ Predict Ÿ Adjust Ÿ Optimize
Did they gain
knowledge on
correct materials
handling?
Did they actively
handle the materials
correctly?
Did the number of
safety incidents
meet expectations?
Large Retailer

42. How do we measure knowledge?

43. How do we measure knowledge?

44. How do we measure knowledge?

45. How do we measure knowledge?

46. Assess Ÿ Predict Ÿ Adjust Ÿ Optimize
Material Handling
Learning Modules
and Reinforcement
(Pull Hook & Layer
Picking)
Did they gain
knowledge on
correct materials
handling?
Did they actively
handle the materials
correctly?
Did the number of
safety incidents
meet expectations?
Large Retailer

47. How do know if it’s the right content?
High level of
knowledge
Correct behavior
demonstrated
Good content
High level of
knowledge
Incorrect
behavior
demonstrated
Content may be
an issue

48. Assess Ÿ Predict Ÿ Adjust Ÿ Optimize
Material Handling
Learning Modules
and Reinforcement
(Pull Hook & Layer
Picking)
Did they gain
knowledge on
correct materials
handling?
Did they actively
handle the materials
correctly?
Did the number of
safety incidents
meet expectations?
Large Retailer

49. Dani Johnson
Vice President, Learning and
Development Research
Bersin by Deloitte
Deloitte Consulting LLP
Carol Leaman
Chief Executive Officer
Axonify
Q&A

50. Thank you!

51. About Deloitte
As used in this document, “Deloitte” means Deloitte Consulting LLP, a subsidiary of Deloitte LLP. Please see www.deloitte.com/us/about for a detailed description of the legal structure of Deloitte LLP and its
subsidiaries. Certain services may not be available to attest clients under the rules and regulations of public accounting.
This communication contains general information only, and none of Deloitte Touche Tohmatsu Limited, its member firms, or their related entities (collectively, the “Deloitte Network”) is, by means of this
communication, rendering professional advice or services. Before making any decision or taking any action that may affect your finances or your business, you should consult a qualified professional adviser. No
entity in the Deloitte Network shall be responsible for any loss whatsoever sustained by any person who relies on this communication.
Copyright © 2016 Deloitte Development LLC. All rights reserved
Member of Deloitte Touche Tohmatsu Limited

52. About Axonify

53. Axonify focuses on driving employee knowledge in
corporate enterprise, ensuring that every
employee has what they need - in their heads or
at their fingertips - to effectively do their jobs.
For the first time ever, organizations can measure
the impact of their learning efforts on knowledge
growth through to employee behaviors and
business results.

54. A Little Bit About Axonify
Axonify counts amongst its customers the world’s largest beverage company, the world’s largest
retailer, the world’s largest automotive company, the world’s largest professional services firm, the
world’s largest pharmaceutical company and the world’s largest multinational conglomerate.
10 M 79%
>80% 78%
Microlearning
bites per month
and growing
Voluntary
participation rates
“Axonify has
made me more
knowledgeable”
“Axonify has
made me more
effective at my
job (i.e. work
safer, sell more)”
Axonify Delivers Axonify Users Say
Source: 2014 Axonify End User Survey

55. Some of Our Customers
Visit www.axonify.com to learn more