SlideShare une entreprise Scribd logo
1  sur  41
Télécharger pour lire hors ligne
Measurement at 8Ls
Crafting an Evaluation Strategy
Learning DevCamp 2019 | S106
Megan Torrance | mtorrance@torrancelearning.com
Megan Torrance
mtorrance@torrancelearning.com
@MMTorrance | @xAPIGnome | @Tlearning
How do we know
if training works?
What do we
actually measure?
What we
measure
What we
know we
need to
measure
Along the continuum of things we can measure
Nothing Useless
Overkill
Who cares?
Make a plan for
implementation
Evaluate for
organizational insights
Establish a
strategy for
learning evaluation
Evaluate Katzell-
Kirkpatrick Levels
Evaluate
engagement &
experience
Take first
next steps
Great! We rolled it out on
the LMS & the intranet.
How’s the
course going?
YEAH! So are you
seeing success?
People really like it!
We’ll want some changes.
… how do we measure
success?
8 Ways to Evaluate Learning S106 Learning DevCamp 2019
8 Ls of
Evaluation
Stories &
why##
%
A word about Kirkpatrick Levels
www.worklearning.com
Katzell-Kirkpatrick
8 Ways to Evaluate Learning S106 Learning DevCamp 2019
LEVEL 1
SATISFACTION
Do participants like it?
Check out
Will Thalheimer’s
Performance-based
Smile Sheets
LEVEL 1
SATISFACTION
Do participants like it?
HOW TO
• Surveys from LMS
• Surveys
• Interviews
• Ask instructor or observer
• Online reviews
IT’S NOT
A measure
of effectiveness
MORE INFO
Will Thalheimer’s
Performance-based
Smile Sheets
LEVEL 2
KNOWLEDGE
Did they learn it?
Learning =
Doing
LEVEL 2
KNOWLEDGE
Did they learn it?
HOW TO
• Knowledge checks in the
course content
• Tests immediate after
• Tests delayed over time
• Adaptive testing
• Confidence based testing
IT’S NOT
A measure of ability
to do anything other
than pass the test
LEVEL 3
BEHAVIORS
Do they do it (correctly)
out in the real world?
Check out
Brinkerhoff’s
Success Case
Method
LEVEL 3
BEHAVIORS
Do they do it (correctly)
out in the real world?
HOW TO
• Surveys of learners
• Surveys of managers
HOW TO, part 2
• Business process
performance data
MORE INFO
Making Sense of xAPI
Brinkerhoff Success Case
Method
LEVEL 4
RESULTS
Does it matter?
Did it solve the problem?
Hint: you measure this with
business metrics,
not training ones!
LEVEL 4
RESULTS
Does it matter?
Did it solve the problem?
HOW TO
• Measure organizational
results
IT’S NOT
“clean” – get over it
MORE INFO
Ask your org leaders how
they measure their business
(hint: it’s probably not
training data)
$ of solving problem
$ of program
ROI =
Make a plan for
implementation
Evaluate for
organizational insights
Establish a
strategy for
learning evaluation
Evaluate Katzell-
Kirkpatrick Levels
Evaluate
engagement &
experience
Take first
next steps
LIKES,
LOOKS &
LAUNCHES
PARTICIPATION
Who? How many? Where?
How long? Dropoff?
Often useful until you
gather other data
LIKES,
LOOKS &
LAUNCHES
PARTICIPATION
Who? How many? Where?
How long? Dropoff?
HOW TO
• LMS, enrollment,
completion stats
• Social engagement stats
• Google Analytics
• Whatever your tools will
give you
IT’S NOT
useful without the
qualitative data to
understand the why
IT IS …
Useful
What you share while you
collect other data
LX
LEARNING
EXPERIENCE
What actually happened?
Check out xAPI for
digital learning
LX
LEARNING
EXPERIENCE
What actually happened?
HOW TO MORE INFO
• xAPI
• Whatever tool data you
can get
Making Sense of xAPI
IT’S NOT
useful without the
qualitative data to
understand the why
LEADERS
INSIGHTS
What do leaders see?
Well, from
here it looks
like …
LEADERS
INSIGHTS
What do leaders see?
HOW TO IT’S NOT
• 1:1 interviews quantitative
LESSONS
LEARNED
What did we learn?
* This is
about YOU!
LESSONS
LEARNED
What did we learn?
HOW TO IT’S NOT
• Retrospective
• Start / Stop / Continue
an evaluation of the
training so much as an
evaluation of our process for
getting here
Make a plan for
implementation
Evaluate for
organizational insights
Establish a
strategy for
learning evaluation
Evaluate Katzell-
Kirkpatrick Levels
Evaluate
engagement &
experience
Take first
next steps
STAKES
TO PUT IN THE
GROUND NOW
Omigosh we need a control!
Yes – I just suggested
you do this before
setting your strategy
STAKES
TO PUT IN THE
GROUND NOW
Omigosh we need a control!
You may realize you need to start measuring
something now to get a baseline.
Make a plan for
implementation
Evaluate for
organizational insights
Establish a
strategy for
learning evaluation
Evaluate Katzell-
Kirkpatrick Levels
Evaluate
engagement &
experience
Take first
next steps
35% of TD pros’ organizations
evaluated the business results of
learning programs to any extent.
Elaine Beich, ATD Foundations of Talent Development, 2016 research
What are you already
measuring? (yeah you!)
What will make the biggest
impact to start measuring?
What is the easiest to start
measuring? (quick wins!)
Who else will you need to
involve? (friendly partners!)
What barriers will you face?
For crying out loud fill out
your session evaluations.
Megan Torrance | mtorrance@torrancelearning.com
What will you start measuring in
30 days? 60 days? 90 days?
References
https://www.worklearning.com/2018/01/30/donald-
kirkpatrick-was-not-the-originator-of-the-four-level-model-
of-learning-evaluation/
https://keydifferences.com/difference-between-
qualitative-and-quantitative-data.html
https://www.amazon.com/Success-Case-Method-
Quickly-Working/dp/1576751856
https://www.td.org/books/atds-guide-to-talent-
development
https://roiinstitute.net/
https://www.td.org/td-at-work/making-sense-of-xapi

Contenu connexe

Tendances

Selling xAPI / Getting Buy-in for TorranceLearning Download May 2016
Selling xAPI  / Getting Buy-in for TorranceLearning Download May 2016Selling xAPI  / Getting Buy-in for TorranceLearning Download May 2016
Selling xAPI / Getting Buy-in for TorranceLearning Download May 2016TorranceLearning
 
The Impacts of the Tin Can API: How 8 Companies are Using the Tin Can API (xAPI)
The Impacts of the Tin Can API: How 8 Companies are Using the Tin Can API (xAPI)The Impacts of the Tin Can API: How 8 Companies are Using the Tin Can API (xAPI)
The Impacts of the Tin Can API: How 8 Companies are Using the Tin Can API (xAPI)Rustici Software
 
LSCon 2017 Making Future-focused Platform Decisions with the xAPI
LSCon 2017 Making Future-focused Platform Decisions with the xAPILSCon 2017 Making Future-focused Platform Decisions with the xAPI
LSCon 2017 Making Future-focused Platform Decisions with the xAPITorranceLearning
 
Webinar: Sharing Statements a Collaborative Project
Webinar: Sharing Statements a Collaborative ProjectWebinar: Sharing Statements a Collaborative Project
Webinar: Sharing Statements a Collaborative ProjectRustici Software
 
Abigail Howe Xerte presentation
Abigail Howe Xerte presentationAbigail Howe Xerte presentation
Abigail Howe Xerte presentationJISC RSC Eastern
 
Case Study: xAPI for vets and pets
Case Study: xAPI for vets and petsCase Study: xAPI for vets and pets
Case Study: xAPI for vets and petsMegan Bowe
 
Self-Directed Learning
Self-Directed LearningSelf-Directed Learning
Self-Directed LearningBrian Chorba
 
Visible Paths: Tracing the Development of an Online Tutoring Program
Visible Paths: Tracing the Development of an Online Tutoring ProgramVisible Paths: Tracing the Development of an Online Tutoring Program
Visible Paths: Tracing the Development of an Online Tutoring Programnjolemore
 
Creating a Data Driven L&D Team - an xAPI Case Study - DevLearn 2018
Creating a Data Driven L&D Team - an xAPI Case Study - DevLearn 2018Creating a Data Driven L&D Team - an xAPI Case Study - DevLearn 2018
Creating a Data Driven L&D Team - an xAPI Case Study - DevLearn 2018Margaret Roth
 

Tendances (12)

Selling xAPI / Getting Buy-in for TorranceLearning Download May 2016
Selling xAPI  / Getting Buy-in for TorranceLearning Download May 2016Selling xAPI  / Getting Buy-in for TorranceLearning Download May 2016
Selling xAPI / Getting Buy-in for TorranceLearning Download May 2016
 
The Impacts of the Tin Can API: How 8 Companies are Using the Tin Can API (xAPI)
The Impacts of the Tin Can API: How 8 Companies are Using the Tin Can API (xAPI)The Impacts of the Tin Can API: How 8 Companies are Using the Tin Can API (xAPI)
The Impacts of the Tin Can API: How 8 Companies are Using the Tin Can API (xAPI)
 
LSCon 2017 Making Future-focused Platform Decisions with the xAPI
LSCon 2017 Making Future-focused Platform Decisions with the xAPILSCon 2017 Making Future-focused Platform Decisions with the xAPI
LSCon 2017 Making Future-focused Platform Decisions with the xAPI
 
Webinar: Sharing Statements a Collaborative Project
Webinar: Sharing Statements a Collaborative ProjectWebinar: Sharing Statements a Collaborative Project
Webinar: Sharing Statements a Collaborative Project
 
Design Implications of the Experience API (Tin Can API)
Design Implications of the Experience API (Tin Can API)Design Implications of the Experience API (Tin Can API)
Design Implications of the Experience API (Tin Can API)
 
Abigail Howe Xerte presentation
Abigail Howe Xerte presentationAbigail Howe Xerte presentation
Abigail Howe Xerte presentation
 
cmi5-xapi-camp
cmi5-xapi-campcmi5-xapi-camp
cmi5-xapi-camp
 
Case Study: xAPI for vets and pets
Case Study: xAPI for vets and petsCase Study: xAPI for vets and pets
Case Study: xAPI for vets and pets
 
Feedback
FeedbackFeedback
Feedback
 
Self-Directed Learning
Self-Directed LearningSelf-Directed Learning
Self-Directed Learning
 
Visible Paths: Tracing the Development of an Online Tutoring Program
Visible Paths: Tracing the Development of an Online Tutoring ProgramVisible Paths: Tracing the Development of an Online Tutoring Program
Visible Paths: Tracing the Development of an Online Tutoring Program
 
Creating a Data Driven L&D Team - an xAPI Case Study - DevLearn 2018
Creating a Data Driven L&D Team - an xAPI Case Study - DevLearn 2018Creating a Data Driven L&D Team - an xAPI Case Study - DevLearn 2018
Creating a Data Driven L&D Team - an xAPI Case Study - DevLearn 2018
 

Similaire à 8 Ways to Evaluate Learning S106 Learning DevCamp 2019

Creating a competitive training plan
Creating a competitive training plan Creating a competitive training plan
Creating a competitive training plan Henry John Nueva
 
Improving Organizational Performance using the Experience API
Improving Organizational Performance using the Experience APIImproving Organizational Performance using the Experience API
Improving Organizational Performance using the Experience APIAli Shahrazad
 
Using Net Promoter Score (NPS) to Increase Course Engagement
Using Net Promoter Score (NPS) to Increase Course EngagementUsing Net Promoter Score (NPS) to Increase Course Engagement
Using Net Promoter Score (NPS) to Increase Course EngagementLambda Solutions
 
If agile is so great, why do we constrain it to software projects
If agile is so great, why do we constrain it to software projectsIf agile is so great, why do we constrain it to software projects
If agile is so great, why do we constrain it to software projectsJohn McIntyre
 
Measures that Matter: Capture and Communicate the Value of Learning Programs
Measures that Matter: Capture and Communicate the Value of Learning ProgramsMeasures that Matter: Capture and Communicate the Value of Learning Programs
Measures that Matter: Capture and Communicate the Value of Learning ProgramsNoelle Akins
 
Lambda Solutions Webinar | Capturing Data to Improve the eLearning Experience...
Lambda Solutions Webinar | Capturing Data to Improve the eLearning Experience...Lambda Solutions Webinar | Capturing Data to Improve the eLearning Experience...
Lambda Solutions Webinar | Capturing Data to Improve the eLearning Experience...Lambda Solutions
 
Learning Analytics Primer: Getting Started with Learning and Performance Anal...
Learning Analytics Primer: Getting Started with Learning and Performance Anal...Learning Analytics Primer: Getting Started with Learning and Performance Anal...
Learning Analytics Primer: Getting Started with Learning and Performance Anal...Watershed
 
What is xAPI and is it contagious?
What is xAPI and is it contagious?What is xAPI and is it contagious?
What is xAPI and is it contagious?TorranceLearning
 
Addressing the Risks & Opportunities of Implementing an Outcomes Based Strategy
Addressing the Risks & Opportunities of Implementing an Outcomes Based Strategy Addressing the Risks & Opportunities of Implementing an Outcomes Based Strategy
Addressing the Risks & Opportunities of Implementing an Outcomes Based Strategy Blackbaud Pacific
 
Does Your Content Work for People? Essentials for Evaluating Your Client's Co...
Does Your Content Work for People? Essentials for Evaluating Your Client's Co...Does Your Content Work for People? Essentials for Evaluating Your Client's Co...
Does Your Content Work for People? Essentials for Evaluating Your Client's Co...GatherContent
 
Measures That Matter: How to Capture and Communicate the Value of Your Learni...
Measures That Matter: How to Capture and Communicate the Value of Your Learni...Measures That Matter: How to Capture and Communicate the Value of Your Learni...
Measures That Matter: How to Capture and Communicate the Value of Your Learni...Casey Cramer
 
Measuring the Business Impact of Learning Workshop. Royal Institution, London...
Measuring the Business Impact of Learning Workshop. Royal Institution, London...Measuring the Business Impact of Learning Workshop. Royal Institution, London...
Measuring the Business Impact of Learning Workshop. Royal Institution, London...Gareth Jones
 
What Do Users Really Think? Surveying Users About Your Help Content
What Do Users Really Think? Surveying Users About Your Help ContentWhat Do Users Really Think? Surveying Users About Your Help Content
What Do Users Really Think? Surveying Users About Your Help Contentpatricia_gale
 
Becoming a Salesforce / Data Driven Non-Profit
Becoming a Salesforce / Data Driven Non-ProfitBecoming a Salesforce / Data Driven Non-Profit
Becoming a Salesforce / Data Driven Non-ProfitSarah Parise
 
Things we can learn from digital start-ups | Digital conference | 27 October ...
Things we can learn from digital start-ups | Digital conference | 27 October ...Things we can learn from digital start-ups | Digital conference | 27 October ...
Things we can learn from digital start-ups | Digital conference | 27 October ...CharityComms
 
Career Benchmarking Using Insights from the Salesforce Trailblazer Community ...
Career Benchmarking Using Insights from the Salesforce Trailblazer Community ...Career Benchmarking Using Insights from the Salesforce Trailblazer Community ...
Career Benchmarking Using Insights from the Salesforce Trailblazer Community ...Spanning Cloud Apps
 
No More Excuses. Create a Testing Plan with No Traffic, Time, or Budget
No More Excuses.  Create a Testing Plan with No Traffic, Time, or BudgetNo More Excuses.  Create a Testing Plan with No Traffic, Time, or Budget
No More Excuses. Create a Testing Plan with No Traffic, Time, or BudgetNTEN
 
Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evalu...
Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evalu...Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evalu...
Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evalu...Lambda Solutions
 

Similaire à 8 Ways to Evaluate Learning S106 Learning DevCamp 2019 (20)

Creating a competitive training plan
Creating a competitive training plan Creating a competitive training plan
Creating a competitive training plan
 
Improving Organizational Performance using the Experience API
Improving Organizational Performance using the Experience APIImproving Organizational Performance using the Experience API
Improving Organizational Performance using the Experience API
 
Setting the right strategic direction
Setting the right strategic directionSetting the right strategic direction
Setting the right strategic direction
 
Using Net Promoter Score (NPS) to Increase Course Engagement
Using Net Promoter Score (NPS) to Increase Course EngagementUsing Net Promoter Score (NPS) to Increase Course Engagement
Using Net Promoter Score (NPS) to Increase Course Engagement
 
If agile is so great, why do we constrain it to software projects
If agile is so great, why do we constrain it to software projectsIf agile is so great, why do we constrain it to software projects
If agile is so great, why do we constrain it to software projects
 
Measures that Matter: Capture and Communicate the Value of Learning Programs
Measures that Matter: Capture and Communicate the Value of Learning ProgramsMeasures that Matter: Capture and Communicate the Value of Learning Programs
Measures that Matter: Capture and Communicate the Value of Learning Programs
 
Lambda Solutions Webinar | Capturing Data to Improve the eLearning Experience...
Lambda Solutions Webinar | Capturing Data to Improve the eLearning Experience...Lambda Solutions Webinar | Capturing Data to Improve the eLearning Experience...
Lambda Solutions Webinar | Capturing Data to Improve the eLearning Experience...
 
Learning Analytics Primer: Getting Started with Learning and Performance Anal...
Learning Analytics Primer: Getting Started with Learning and Performance Anal...Learning Analytics Primer: Getting Started with Learning and Performance Anal...
Learning Analytics Primer: Getting Started with Learning and Performance Anal...
 
What is xAPI and is it contagious?
What is xAPI and is it contagious?What is xAPI and is it contagious?
What is xAPI and is it contagious?
 
Addressing the Risks & Opportunities of Implementing an Outcomes Based Strategy
Addressing the Risks & Opportunities of Implementing an Outcomes Based Strategy Addressing the Risks & Opportunities of Implementing an Outcomes Based Strategy
Addressing the Risks & Opportunities of Implementing an Outcomes Based Strategy
 
CALPACT New Media Webinar: Social Media Measurement and Evaluation
CALPACT New Media Webinar: Social Media Measurement and EvaluationCALPACT New Media Webinar: Social Media Measurement and Evaluation
CALPACT New Media Webinar: Social Media Measurement and Evaluation
 
Does Your Content Work for People? Essentials for Evaluating Your Client's Co...
Does Your Content Work for People? Essentials for Evaluating Your Client's Co...Does Your Content Work for People? Essentials for Evaluating Your Client's Co...
Does Your Content Work for People? Essentials for Evaluating Your Client's Co...
 
Measures That Matter: How to Capture and Communicate the Value of Your Learni...
Measures That Matter: How to Capture and Communicate the Value of Your Learni...Measures That Matter: How to Capture and Communicate the Value of Your Learni...
Measures That Matter: How to Capture and Communicate the Value of Your Learni...
 
Measuring the Business Impact of Learning Workshop. Royal Institution, London...
Measuring the Business Impact of Learning Workshop. Royal Institution, London...Measuring the Business Impact of Learning Workshop. Royal Institution, London...
Measuring the Business Impact of Learning Workshop. Royal Institution, London...
 
What Do Users Really Think? Surveying Users About Your Help Content
What Do Users Really Think? Surveying Users About Your Help ContentWhat Do Users Really Think? Surveying Users About Your Help Content
What Do Users Really Think? Surveying Users About Your Help Content
 
Becoming a Salesforce / Data Driven Non-Profit
Becoming a Salesforce / Data Driven Non-ProfitBecoming a Salesforce / Data Driven Non-Profit
Becoming a Salesforce / Data Driven Non-Profit
 
Things we can learn from digital start-ups | Digital conference | 27 October ...
Things we can learn from digital start-ups | Digital conference | 27 October ...Things we can learn from digital start-ups | Digital conference | 27 October ...
Things we can learn from digital start-ups | Digital conference | 27 October ...
 
Career Benchmarking Using Insights from the Salesforce Trailblazer Community ...
Career Benchmarking Using Insights from the Salesforce Trailblazer Community ...Career Benchmarking Using Insights from the Salesforce Trailblazer Community ...
Career Benchmarking Using Insights from the Salesforce Trailblazer Community ...
 
No More Excuses. Create a Testing Plan with No Traffic, Time, or Budget
No More Excuses.  Create a Testing Plan with No Traffic, Time, or BudgetNo More Excuses.  Create a Testing Plan with No Traffic, Time, or Budget
No More Excuses. Create a Testing Plan with No Traffic, Time, or Budget
 
Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evalu...
Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evalu...Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evalu...
Measuring the Impact of eLearning: Turning Kirkpatrick’s Four Levels of Evalu...
 

Plus de TorranceLearning

Harnessing Change: Agile Methods for Instructional Designers
Harnessing Change: Agile Methods for Instructional DesignersHarnessing Change: Agile Methods for Instructional Designers
Harnessing Change: Agile Methods for Instructional DesignersTorranceLearning
 
ATD Virtual Conference: Leveraging Agile Methods in L&D
ATD Virtual Conference: Leveraging Agile Methods in L&DATD Virtual Conference: Leveraging Agile Methods in L&D
ATD Virtual Conference: Leveraging Agile Methods in L&DTorranceLearning
 
xAPI Intro for Instructional Designers Learning While Working 2019
xAPI Intro for Instructional Designers Learning While Working 2019xAPI Intro for Instructional Designers Learning While Working 2019
xAPI Intro for Instructional Designers Learning While Working 2019TorranceLearning
 
Agile & LLAMA For Instructional Designers DevCamp 2019 S101
Agile & LLAMA For Instructional Designers DevCamp 2019 S101Agile & LLAMA For Instructional Designers DevCamp 2019 S101
Agile & LLAMA For Instructional Designers DevCamp 2019 S101TorranceLearning
 
LLAMA Intro to Agile May 2019 Download
LLAMA Intro to Agile May 2019 DownloadLLAMA Intro to Agile May 2019 Download
LLAMA Intro to Agile May 2019 DownloadTorranceLearning
 
xAPI Ecosystem xAPI Party May 2019
xAPI Ecosystem xAPI Party May 2019xAPI Ecosystem xAPI Party May 2019
xAPI Ecosystem xAPI Party May 2019TorranceLearning
 
xAPI Introduction - TorranceLearning xAPI Party Download May 2019
xAPI Introduction - TorranceLearning xAPI Party Download May 2019xAPI Introduction - TorranceLearning xAPI Party Download May 2019
xAPI Introduction - TorranceLearning xAPI Party Download May 2019TorranceLearning
 
Training2019 Session 419 LLAMA - Agile Project Management for Instructional D...
Training2019 Session 419 LLAMA - Agile Project Management for Instructional D...Training2019 Session 419 LLAMA - Agile Project Management for Instructional D...
Training2019 Session 419 LLAMA - Agile Project Management for Instructional D...TorranceLearning
 
Training 2019 Session 206 - xAPI Intro for Instructional Designers
Training 2019 Session 206 - xAPI Intro for Instructional DesignersTraining 2019 Session 206 - xAPI Intro for Instructional Designers
Training 2019 Session 206 - xAPI Intro for Instructional DesignersTorranceLearning
 
Intro to xAPI & Ecosystem - xAPI Party Dec 2018
Intro to xAPI & Ecosystem - xAPI Party Dec 2018Intro to xAPI & Ecosystem - xAPI Party Dec 2018
Intro to xAPI & Ecosystem - xAPI Party Dec 2018TorranceLearning
 
xAPI Intro for Instructional Designers - DevLearn18
xAPI Intro for Instructional Designers - DevLearn18xAPI Intro for Instructional Designers - DevLearn18
xAPI Intro for Instructional Designers - DevLearn18TorranceLearning
 
Online Learning Conf 2018 xAPI Geek Free - Torrance
Online Learning Conf 2018 xAPI Geek Free - TorranceOnline Learning Conf 2018 xAPI Geek Free - Torrance
Online Learning Conf 2018 xAPI Geek Free - TorranceTorranceLearning
 
xAPI Geek Free - Learning DevCamp 2018
xAPI Geek Free - Learning DevCamp 2018xAPI Geek Free - Learning DevCamp 2018
xAPI Geek Free - Learning DevCamp 2018TorranceLearning
 
Agile PM for Instructional Design - Learning DevCamp 2018
Agile PM for Instructional Design - Learning DevCamp 2018Agile PM for Instructional Design - Learning DevCamp 2018
Agile PM for Instructional Design - Learning DevCamp 2018TorranceLearning
 
Building the xAPI Learning Ecoystem
Building the xAPI Learning EcoystemBuilding the xAPI Learning Ecoystem
Building the xAPI Learning EcoystemTorranceLearning
 
Intro to Agile Project Mgmt for Instructional Design
Intro to Agile Project Mgmt for Instructional DesignIntro to Agile Project Mgmt for Instructional Design
Intro to Agile Project Mgmt for Instructional DesignTorranceLearning
 
ATD ICE 2018 Building the xAPI Ecosystem Houck & Torrance
ATD ICE 2018 Building the xAPI Ecosystem Houck & TorranceATD ICE 2018 Building the xAPI Ecosystem Houck & Torrance
ATD ICE 2018 Building the xAPI Ecosystem Houck & TorranceTorranceLearning
 
ATD ICE 2018 xAPI Geek Free
ATD ICE 2018 xAPI Geek FreeATD ICE 2018 xAPI Geek Free
ATD ICE 2018 xAPI Geek FreeTorranceLearning
 
LSCon 407 Building the xAPI Learning Ecosystem of Your Dreams
LSCon 407 Building the xAPI Learning Ecosystem of Your DreamsLSCon 407 Building the xAPI Learning Ecosystem of Your Dreams
LSCon 407 Building the xAPI Learning Ecosystem of Your DreamsTorranceLearning
 

Plus de TorranceLearning (20)

Harnessing Change: Agile Methods for Instructional Designers
Harnessing Change: Agile Methods for Instructional DesignersHarnessing Change: Agile Methods for Instructional Designers
Harnessing Change: Agile Methods for Instructional Designers
 
ATD Virtual Conference: Leveraging Agile Methods in L&D
ATD Virtual Conference: Leveraging Agile Methods in L&DATD Virtual Conference: Leveraging Agile Methods in L&D
ATD Virtual Conference: Leveraging Agile Methods in L&D
 
xAPI Intro for Instructional Designers Learning While Working 2019
xAPI Intro for Instructional Designers Learning While Working 2019xAPI Intro for Instructional Designers Learning While Working 2019
xAPI Intro for Instructional Designers Learning While Working 2019
 
Agile & LLAMA For Instructional Designers DevCamp 2019 S101
Agile & LLAMA For Instructional Designers DevCamp 2019 S101Agile & LLAMA For Instructional Designers DevCamp 2019 S101
Agile & LLAMA For Instructional Designers DevCamp 2019 S101
 
LLAMA Intro to Agile May 2019 Download
LLAMA Intro to Agile May 2019 DownloadLLAMA Intro to Agile May 2019 Download
LLAMA Intro to Agile May 2019 Download
 
xAPI Ecosystem xAPI Party May 2019
xAPI Ecosystem xAPI Party May 2019xAPI Ecosystem xAPI Party May 2019
xAPI Ecosystem xAPI Party May 2019
 
xAPI Introduction - TorranceLearning xAPI Party Download May 2019
xAPI Introduction - TorranceLearning xAPI Party Download May 2019xAPI Introduction - TorranceLearning xAPI Party Download May 2019
xAPI Introduction - TorranceLearning xAPI Party Download May 2019
 
Training2019 Session 419 LLAMA - Agile Project Management for Instructional D...
Training2019 Session 419 LLAMA - Agile Project Management for Instructional D...Training2019 Session 419 LLAMA - Agile Project Management for Instructional D...
Training2019 Session 419 LLAMA - Agile Project Management for Instructional D...
 
Training 2019 Session 206 - xAPI Intro for Instructional Designers
Training 2019 Session 206 - xAPI Intro for Instructional DesignersTraining 2019 Session 206 - xAPI Intro for Instructional Designers
Training 2019 Session 206 - xAPI Intro for Instructional Designers
 
Intro to xAPI & Ecosystem - xAPI Party Dec 2018
Intro to xAPI & Ecosystem - xAPI Party Dec 2018Intro to xAPI & Ecosystem - xAPI Party Dec 2018
Intro to xAPI & Ecosystem - xAPI Party Dec 2018
 
xAPI Intro for Instructional Designers - DevLearn18
xAPI Intro for Instructional Designers - DevLearn18xAPI Intro for Instructional Designers - DevLearn18
xAPI Intro for Instructional Designers - DevLearn18
 
Online Learning Conf 2018 xAPI Geek Free - Torrance
Online Learning Conf 2018 xAPI Geek Free - TorranceOnline Learning Conf 2018 xAPI Geek Free - Torrance
Online Learning Conf 2018 xAPI Geek Free - Torrance
 
xAPI Geek Free - Learning DevCamp 2018
xAPI Geek Free - Learning DevCamp 2018xAPI Geek Free - Learning DevCamp 2018
xAPI Geek Free - Learning DevCamp 2018
 
Agile PM for Instructional Design - Learning DevCamp 2018
Agile PM for Instructional Design - Learning DevCamp 2018Agile PM for Instructional Design - Learning DevCamp 2018
Agile PM for Instructional Design - Learning DevCamp 2018
 
Building the xAPI Learning Ecoystem
Building the xAPI Learning EcoystemBuilding the xAPI Learning Ecoystem
Building the xAPI Learning Ecoystem
 
What is xAPI?
What is xAPI? What is xAPI?
What is xAPI?
 
Intro to Agile Project Mgmt for Instructional Design
Intro to Agile Project Mgmt for Instructional DesignIntro to Agile Project Mgmt for Instructional Design
Intro to Agile Project Mgmt for Instructional Design
 
ATD ICE 2018 Building the xAPI Ecosystem Houck & Torrance
ATD ICE 2018 Building the xAPI Ecosystem Houck & TorranceATD ICE 2018 Building the xAPI Ecosystem Houck & Torrance
ATD ICE 2018 Building the xAPI Ecosystem Houck & Torrance
 
ATD ICE 2018 xAPI Geek Free
ATD ICE 2018 xAPI Geek FreeATD ICE 2018 xAPI Geek Free
ATD ICE 2018 xAPI Geek Free
 
LSCon 407 Building the xAPI Learning Ecosystem of Your Dreams
LSCon 407 Building the xAPI Learning Ecosystem of Your DreamsLSCon 407 Building the xAPI Learning Ecosystem of Your Dreams
LSCon 407 Building the xAPI Learning Ecosystem of Your Dreams
 

Dernier

Olympus 38DL Plus Ultrasonic Thickness Gauge
Olympus 38DL Plus Ultrasonic Thickness GaugeOlympus 38DL Plus Ultrasonic Thickness Gauge
Olympus 38DL Plus Ultrasonic Thickness GaugeStephenKim86
 
3BBE: THE FUTURE OF ECOMMERCE PRESENTATION - LOUIS MALAYBALAY
3BBE: THE FUTURE OF ECOMMERCE PRESENTATION - LOUIS MALAYBALAY3BBE: THE FUTURE OF ECOMMERCE PRESENTATION - LOUIS MALAYBALAY
3BBE: THE FUTURE OF ECOMMERCE PRESENTATION - LOUIS MALAYBALAYLouis Malaybalay
 
Optimize Your CRM Customization and Beyond
Optimize Your CRM Customization and BeyondOptimize Your CRM Customization and Beyond
Optimize Your CRM Customization and BeyondBoundify
 
Unleashing the Power of Fandom: A Short Guide to Fan Business
Unleashing the Power of Fandom: A Short Guide to Fan BusinessUnleashing the Power of Fandom: A Short Guide to Fan Business
Unleashing the Power of Fandom: A Short Guide to Fan Businesstompeter3736
 
10 Tips for Great Teams CSUN Conference 2024
10 Tips for Great Teams CSUN Conference 202410 Tips for Great Teams CSUN Conference 2024
10 Tips for Great Teams CSUN Conference 2024Nate Evans
 
NVIDIA's overall business overview Presentation.pptx
NVIDIA's overall business overview Presentation.pptxNVIDIA's overall business overview Presentation.pptx
NVIDIA's overall business overview Presentation.pptxKrutik Rakade
 
Shravan Kumaran and sanjay kumaran.pdf..
Shravan Kumaran and sanjay kumaran.pdf..Shravan Kumaran and sanjay kumaran.pdf..
Shravan Kumaran and sanjay kumaran.pdf..ranjithapriya2
 
Record of Module Forensic photography in
Record of Module Forensic photography inRecord of Module Forensic photography in
Record of Module Forensic photography inalexademileighpacal
 
Strategic Resources Corporate Presentation - March 2024 Update
Strategic Resources Corporate Presentation - March 2024 UpdateStrategic Resources Corporate Presentation - March 2024 Update
Strategic Resources Corporate Presentation - March 2024 UpdateAdnet Communications
 
Presented by Sabri international .......
Presented by Sabri international .......Presented by Sabri international .......
Presented by Sabri international .......SABRI INTERNATIONAL
 
NewBase 14 March 2024 Energy News issue - 1707 by Khaled Al Awadi_compress...
NewBase  14 March  2024  Energy News issue - 1707 by Khaled Al Awadi_compress...NewBase  14 March  2024  Energy News issue - 1707 by Khaled Al Awadi_compress...
NewBase 14 March 2024 Energy News issue - 1707 by Khaled Al Awadi_compress...Khaled Al Awadi
 
Importance of Commercial Vehicle Insurance.pptx
Importance of Commercial Vehicle Insurance.pptxImportance of Commercial Vehicle Insurance.pptx
Importance of Commercial Vehicle Insurance.pptxBonano Insurance
 
Meet Raj Shamani: A Trailblazing Entrepreneur
Meet Raj Shamani: A Trailblazing EntrepreneurMeet Raj Shamani: A Trailblazing Entrepreneur
Meet Raj Shamani: A Trailblazing Entrepreneurramya202104
 
Mist Cooling & Fogging System Company in Egypt
Mist Cooling & Fogging System Company in EgyptMist Cooling & Fogging System Company in Egypt
Mist Cooling & Fogging System Company in Egyptopstechsanjanasingh
 
ICv2 Hobby Games White Paper 2024 - State of the Industry
ICv2 Hobby Games White Paper 2024 - State of the IndustryICv2 Hobby Games White Paper 2024 - State of the Industry
ICv2 Hobby Games White Paper 2024 - State of the IndustryDennisViau
 
Dashboards y paneles - CP Home - Area de Operaciones
Dashboards y paneles - CP Home - Area de OperacionesDashboards y paneles - CP Home - Area de Operaciones
Dashboards y paneles - CP Home - Area de OperacionesLPI ONG
 
How The Hustle Milestone Referral Program Got 300K Subscribers
How The Hustle Milestone Referral Program Got 300K SubscribersHow The Hustle Milestone Referral Program Got 300K Subscribers
How The Hustle Milestone Referral Program Got 300K SubscribersFlyyx Tech
 
The 10 Most Influential Women Making Difference In 2024.pdf
The 10 Most Influential Women Making Difference In 2024.pdfThe 10 Most Influential Women Making Difference In 2024.pdf
The 10 Most Influential Women Making Difference In 2024.pdfInsightsSuccess4
 

Dernier (20)

Olympus 38DL Plus Ultrasonic Thickness Gauge
Olympus 38DL Plus Ultrasonic Thickness GaugeOlympus 38DL Plus Ultrasonic Thickness Gauge
Olympus 38DL Plus Ultrasonic Thickness Gauge
 
3BBE: THE FUTURE OF ECOMMERCE PRESENTATION - LOUIS MALAYBALAY
3BBE: THE FUTURE OF ECOMMERCE PRESENTATION - LOUIS MALAYBALAY3BBE: THE FUTURE OF ECOMMERCE PRESENTATION - LOUIS MALAYBALAY
3BBE: THE FUTURE OF ECOMMERCE PRESENTATION - LOUIS MALAYBALAY
 
Optimize Your CRM Customization and Beyond
Optimize Your CRM Customization and BeyondOptimize Your CRM Customization and Beyond
Optimize Your CRM Customization and Beyond
 
WAM Corporate Presentation Mar 12 2024_Video.pdf
WAM Corporate Presentation Mar 12 2024_Video.pdfWAM Corporate Presentation Mar 12 2024_Video.pdf
WAM Corporate Presentation Mar 12 2024_Video.pdf
 
Unleashing the Power of Fandom: A Short Guide to Fan Business
Unleashing the Power of Fandom: A Short Guide to Fan BusinessUnleashing the Power of Fandom: A Short Guide to Fan Business
Unleashing the Power of Fandom: A Short Guide to Fan Business
 
10 Tips for Great Teams CSUN Conference 2024
10 Tips for Great Teams CSUN Conference 202410 Tips for Great Teams CSUN Conference 2024
10 Tips for Great Teams CSUN Conference 2024
 
NVIDIA's overall business overview Presentation.pptx
NVIDIA's overall business overview Presentation.pptxNVIDIA's overall business overview Presentation.pptx
NVIDIA's overall business overview Presentation.pptx
 
Shravan Kumaran and sanjay kumaran.pdf..
Shravan Kumaran and sanjay kumaran.pdf..Shravan Kumaran and sanjay kumaran.pdf..
Shravan Kumaran and sanjay kumaran.pdf..
 
Record of Module Forensic photography in
Record of Module Forensic photography inRecord of Module Forensic photography in
Record of Module Forensic photography in
 
Strategic Resources Corporate Presentation - March 2024 Update
Strategic Resources Corporate Presentation - March 2024 UpdateStrategic Resources Corporate Presentation - March 2024 Update
Strategic Resources Corporate Presentation - March 2024 Update
 
WAM Corporate Presentation Mar 12 2024.pdf
WAM Corporate Presentation Mar 12 2024.pdfWAM Corporate Presentation Mar 12 2024.pdf
WAM Corporate Presentation Mar 12 2024.pdf
 
Presented by Sabri international .......
Presented by Sabri international .......Presented by Sabri international .......
Presented by Sabri international .......
 
NewBase 14 March 2024 Energy News issue - 1707 by Khaled Al Awadi_compress...
NewBase  14 March  2024  Energy News issue - 1707 by Khaled Al Awadi_compress...NewBase  14 March  2024  Energy News issue - 1707 by Khaled Al Awadi_compress...
NewBase 14 March 2024 Energy News issue - 1707 by Khaled Al Awadi_compress...
 
Importance of Commercial Vehicle Insurance.pptx
Importance of Commercial Vehicle Insurance.pptxImportance of Commercial Vehicle Insurance.pptx
Importance of Commercial Vehicle Insurance.pptx
 
Meet Raj Shamani: A Trailblazing Entrepreneur
Meet Raj Shamani: A Trailblazing EntrepreneurMeet Raj Shamani: A Trailblazing Entrepreneur
Meet Raj Shamani: A Trailblazing Entrepreneur
 
Mist Cooling & Fogging System Company in Egypt
Mist Cooling & Fogging System Company in EgyptMist Cooling & Fogging System Company in Egypt
Mist Cooling & Fogging System Company in Egypt
 
ICv2 Hobby Games White Paper 2024 - State of the Industry
ICv2 Hobby Games White Paper 2024 - State of the IndustryICv2 Hobby Games White Paper 2024 - State of the Industry
ICv2 Hobby Games White Paper 2024 - State of the Industry
 
Dashboards y paneles - CP Home - Area de Operaciones
Dashboards y paneles - CP Home - Area de OperacionesDashboards y paneles - CP Home - Area de Operaciones
Dashboards y paneles - CP Home - Area de Operaciones
 
How The Hustle Milestone Referral Program Got 300K Subscribers
How The Hustle Milestone Referral Program Got 300K SubscribersHow The Hustle Milestone Referral Program Got 300K Subscribers
How The Hustle Milestone Referral Program Got 300K Subscribers
 
The 10 Most Influential Women Making Difference In 2024.pdf
The 10 Most Influential Women Making Difference In 2024.pdfThe 10 Most Influential Women Making Difference In 2024.pdf
The 10 Most Influential Women Making Difference In 2024.pdf
 

8 Ways to Evaluate Learning S106 Learning DevCamp 2019

Notes de l'éditeur

  1. Opening discussion: We’re all professionals here so let’s jump right on. How do we know if training works? Goal: conversation about ways to know if things are working – not specific examples.
  2. We recognize there’s a gap. I clearly don’t need to tell you why this is important. So … what are we going to do about it? Here’s our plan for tonight.
  3. This is not a research-based model. This is based on what I’ve seen & done in practice & learned from people who do research.
  4. Give overview of 8Ls, history & adaptation.
  5. Qualitative data provides insights and understanding about a problem. Methods of collecting qualitative data: Focus groups Observation Interviews Archival materials like newspapers Quantitative data computes the values and can be expressed in numerical terms. Methods of collecting quantitative data: Surveys Experiments Observations and interviews https://keydifferences.com/difference-between-qualitative-and-quantitative-data.html
  6. Story: the SAP team where people loved it and people hated it … but the performance is the same.
  7. Emerged from the work of Dr. Donald Kirkpatrick and Dr. Jack Phillips Return on Investment or ROI is a metric applied across many business functions, and for good reason. It’s the fundamental measurement of performance in business. It’s the statement of profitability over time in relation to the money invested in the effort. Because it is difficult to get a true ROI measurement, ROI is best used when deciding between 2 executions: for example, a website vs. a tradeshow; an additional sales representative in the field vs. more direct mail pieces; and so on. It worthwhile noting that evaluation may serve a number of key quality control functions in addition to establishing training’s business value. Some of the other objectives of evaluation are to: Improve the quality of learning programs Determine if a program meet its objectives Identify potential strengths and weaknesses in the learning program Develop a cost/benefit analysis of training and an Human Resources Development (HRD) investments Support marketing of training or HRD programs Determine a program’s appropriateness for the target audience Assist in decision making about program investments and establish funding priorities