The ‘Measuring the Business Impact of Learning’ benchmarking survey, conducted by LEO Learning and Watershed (on behalf of Learning Technologies Group) is entering its fourth year. With the survey launching on November 1st and closing on December 13th, LEO Learning and Watershed are holding a webinar to reflect on the results so far, plus discuss how organizations they’re working with have overcome the barriers in measurement planning and implementation. The insights are drawn from their group experience working with a range of clients in this field and should be valuable for anyone who wants to get going in learning analytics and sustainable business impact assessment. Join your hosts as they cover the state of the world of measurement, and you’ll receive: An understanding of how well-known organizations have overcome the barriers in measurement planning and implementation. Real-world examples of how to get management buy-in, designing for data, building data ecosystems, implementing a learning analytics strategy and more. The opportunity to take this years ‘Measuring the Business Impact of Learning’ survey, and see the results coming in live!

1. #CLOwebinar
The presentation will begin at the top of the hour.
A dial in number will not be provided.
Listen to today’s webinar using your computer’s
speakers or headphones.
Welcome to the webinar!

2. #CLOwebinar
Tools You Can Use
Audio Control
– A dial in number will not be provided.
– The audio will stream through your
headphones or computer speakers.
– Also check your computer’s volume for
external speakers or headsets.

3. #CLOwebinar
Questions and Handouts
You can submit questions by clicking on
this icon here.
Download a PDF of the slides here.

4. #CLOwebinar
1. May I receive a copy of the slides?
YES! Click on the handouts located on the right portion of
your screen.
2. May I review the webinar recording at a later date?
YES! You may log in again using today’s link to review the
presentation on-demand.
3. Is this webinar HRCI or SHRM certified?
YES! The HRCI and SHRM certification codes will be sent in
the follow up email.
Frequently Asked Questions

5. Learning Design Director
LEO Learning
Measuring the business impact of
learning: What we’ve learned
Ella Richardson
Director of Strategy
Watershed Analytics
Tim Dickinson
Chief Strategy Officer
Learning Technologies Group
Piers Lea

6. Agenda
The state of
measurement
Biggest barriers to
measurement
Practical steps to
overcome barriers
Takeaways
in a nutshell

7. Over 1000 respondents over 3 years

8. ?
Survey

13. ?
What are your results?

14. Barriers to measurement:
Getting Started
1
Getting buy in Getting Data
2 3

15. But First: Remember measurement is ongoing & iterative
Watershed Method for Learning
Analytics
Categories of analytics
+
Complexities of analytics

16. Start with what you can control.
Barrier 1
I don’t know how to get started.

17. Have they
learned it?
What do people
need to learn?
Are people
doing that?
What do people
need to do?
Did they
complete it?
What training
is required?
Was the goal
achieved?
Business
goal
Learning Design
Measurement
Getting Started: Link design to measurement

18. How In Practice: Get your team to understand
Develop a bespoke
measurement model
and processes
Upskill the trainers
to operate as
performance
consultants
Put measurement
first in the design
and delivery
process
Building internal measurement capability

19. Evaluate
Analysis
Consult
Measurement
Process
Action
Design
DevelopImplement
Evaluate
Analyse
Consult
• Clarify the business goals
• Define the business objectives
• Create a business impact action plan
• Identify the business
data sources
• Write the learning
objectives
• Specify the learning outcomes
• Identify the learning data
sources
• Refine the measurement
action plan
• Identify whether the required data can
be captured
• Baseline the data
• Design how the data will be displayed
• Pilot the learning
• Gather the business and learning data
• Check the quality of the data and
move to full roll-out
• Explore the business and
learning data for insights
• Report back on initial findings
and improve training
• Develop a deep
understanding of the
business performance gaps
• Create a draft
improvement plan
• Modify programme /
manage learners as
required
A bespoke
measurement
process

20. Getting Started: Map out your ecosystem
Identify your
current tools and
resources
Question your
current ecosystem
Choose an area to
test your new
measurement
It’s easier than you think
1 2 3

21. Getting Started: Test it out

24. 8% behavioral change within month of launch

25. xAPI cohort
● Learn from peers
● Team-based projects
Free LRS
● Essentials LRS
● SCORM cloud
● Learning Locker
Getting Started: DIY

26. xAPI
IPK
API
LRS
Behr built it themselves.

27. Outcome
There was a 2% sales lift during
the four-week period when video and
assessment were used as planned.

28. Show them what it could look like.
Barrier 2
Competing Priorities: How to get budget and time?

29. Getting Buy-In: Define a strategy
Key components of a measurement strategy
No.1
Co-create
measurement
project
No.2
Establish the
business drivers
No.3
Map out
stakeholders
No.4
Document
chain of
evidence
No.5
Identify data
types and
source
No.6
Define, develop
tech: Integration
& tools
No.7
Pilot the
solution
No.8
Document
strategy and
business case
No.9
Define the
change
programme
No.10
Measure impact
of measurement
strategy

30. The chain of evidence
Engagement Learning Behaviour Results

31. Chain of Evidence
Engagement
Knowledge, skills &
attitudes
Performance
Behaviors
Business Change
Has the learning
landed?
Has learning taken
place?
Have the
performance
behaviors
changed?
Has business
performance
improved?
Learner Change
& Performance2
Business
impact3Learning
Delivery
1
© LEO 2019

32. Getting Buy-In:
Share the ecosystem vision

33. Getting Buy-In: Share the ecosystem vision
Read more: bit.ly/2NCbMkk

34. Get aligned and don’t be afraid to ask for help.
Barrier 3
How do you get access to the data?

35. Getting Data: Vendor Alignment
Share the vision
and how your
vendor fits in
Build a partnership
based on
collaboration
Validate
interoperability
before taking out
the checkbook

36. Getting Data: Vendor Alignment

37. Measurement – sources of evidence
Engagement
• Portal/learning visits and
behaviour data (which
courses, by whom, for
how long)
• Star rating
• User Surveys
• Learner voice
• User acceptance
testing
• Interviews
• Annual staff survey
• Individual 360
• Survey of departmental
satisfaction
K, S, A
• Upfront diagnostics
(including 360)
• Post intervention Online
assessments (immediate
& after 3 months)
• Marked work (essays,
reports)
• Peer assessment
• Demonstration
• Confidence measure
taken after every
intervention
Performance
Behaviours
• User surveys
• Manager surveys
• Interviews (in
manageable numbers)
• Surveys of coaches and
mentors
• User observation
• Service user feedback
(net promoter score)
• Activity monitoring
software
• Use of learning journals
Business Change
• Business Metrics & KPIs
• Profitability, Growth,
Productivity etc.
• Improved staff retention
• Faster turnaround times
• Money saved by new
approach
• Money saved by
outcomes

38. Getting Data: Ask for help
Build alliances
within the
organisation, early
Get the best team
round the table

39. Key Takeaways
Get Started
○ Design for data
○ Start small, test and learn
○ DIY: Learn about it yourself
Get Buy in
○ Define a measurement strategy
○ Map out your ecosystem/what’s available to you already
Get Data
○ Align vendors and tools for long term success
○ Ask for help

40. ?
Q&A

41. Leolearning.com | watershedlrs.com
Access free tools and resources at:
bit.ly/2Q6ssSt

42. #CLOwebinar
Register for the next webinar!
Register for all upcoming Chief Learning Officer Webinars at
chieflearningofficer.com/webinars