The most successful CLOs today run learning like a business. This means aligning learning to the business goals, carefully planning key programs and agreeing on measures of success, and then actively managing the programs to achieve the planned outcomes, effectiveness and efficiency. Talent Development Reporting Principles (TDRp) is a fast-growing, industry-led initiative designed help CLOs do just this — run learning like a business. TDRp introduces three common statements to organize outcome, effectiveness and efficiency measures, and three recommended reports for senior leaders to use in managing L&D, including one designed for the CEO. With the help of leading practitioners and industry thought leaders, TDRp answers the age-old questions in the context of running learning like a business: What data should we collect and how should the measures be defined? What do CLOs, CEOs and other senior leaders want to see and how should it be presented? How can we show the important role learning plays in achieving organizational goals? Join Dave Vance, former CLO for Caterpillar and author of The Business of Learning, to learn more about running learning like business and how TDRp can help. Vance, who is co-author of the TDRp papers, will provide an introduction to TDRp, including recommendations for categorizing measures and creating standard statements and reports. See how the principles of running learning like a business combined with the recommendations of TDRp can help you be a better and more strategic business partner in your organization, delivering the right outcomes with greater effectiveness and efficiency. For more information go to TDRprinciples.org.

1. Using Talent Development Reporting
Principles to Run Learning Like a
Business
You can listen to today’s webinar using your computer’s speakers
or you may dial into the teleconference.
If you would like to join the teleconference,
please dial 1.408.600.3600 and enter access code 921 466 798 #
You will be on hold until the seminar begins.
#CLOwebinar

2. Using Talent Development Reporting
Principles to Run Learning Like a
Business
Speaker: Dave Vance
Former President
Caterpillar University
Moderator: Deanna Hartley
Senior Editor
Chief Learning Officer magazine
#CLOwebinar

3. Tools You Can Use
• Q&A
– Click on the Q&A icon on
your floating toolbar on the
top of your screen.
– Type in your question in the
space provided
– Click on “Send.”
#CLOwebinar

4. Tools You Can Use
• Polling
– The poll will appear on the
right side of your screen
– Select the best option for
each question
– Click on “Submit”
#CLOwebinar

5. Frequently Asked Questions
Will I receive a copy of the webinar recording?
YES
Will I receive a copy of the slides?
YES
Please allow up to 2 business days to receive these materials
#CLOwebinar

6. Using Talent Development Reporting
Principles to Run Learning Like a
Business
Deanna Hartley
Senior Editor
Chief Learning Officer magazine
#CLOwebinar

7. Using Talent Development Reporting
Principles to Run Learning Like a
Business
Dave Vance
Former President
Caterpillar University
#CLOwebinar

8. Chief Learning Officer Webinar
Sponsored by Skillsoft
September 13, 2012
David Vance

9. The basics of running learning like a
business
Talent Development Reporting Principles
(TDRP)
◦ What is it?
◦ How can it help?
Q&A
Chief Learning Officer Webinar September 13, 2012 8

10. Help our organizations achieve their goals
◦ Achieve their goals more quickly
◦ Achieve their goals at lower cost
Best accomplished by running learning like a
business focusing on
◦ Outcomes (Are we doing the right things?)
◦ Effectiveness (How well?)
◦ Efficiency (How much and at what cost?)
Chief Learning Officer Webinar September 13, 2012 9

11. 1. Align learning to business goals
2. Plan it carefully
3. Execute and report with discipline
4. Measure and evaluate
Chief Learning Officer Webinar September 13, 2012 10

12. Proactive, strategic exercise
◦ Meet with CEO or business unit leader
◦ Learn about next year’s goals and priorities
◦ Learn who the stakeholders are
If not done proactively, at least be sure the
requested learning does support the
highest-priority business goals
◦ May mean saying “No” to some requests
Chief Learning Officer Webinar September 13, 2012 11

13. A: Proactively to align learning to business goals
B: Align to business goals as requests for learning come in
C: No process to ensure alignment but most of our learning is
aligned to business goals
D: No process and much of our learning is probably not
aligned to business goals
E: We really don’t know what our business goals are
Chief Learning Officer Webinar September 13, 2012 12

14. Meet with stakeholders to understand
◦ Business need
◦ Learning’s role (if any)
Agree with stakeholder on
◦ Specific programs, type of learning
◦ Target audience, location, completion dates
◦ Number of participants
◦ Roles and responsibilities
◦ Measures of success
Expected impact of learning on business goal
◦ Costs
Chief Learning Officer Webinar September 13, 2012 13

15. If done proactively
◦ Develop a business case for key learning
All costs
Benefits
◦ Create a business plan for learning including
Strategic alignment
Expected results from learning
Business case
Measurement and evaluation strategy
◦ Have plan approved by CEO and senior leaders
Chief Learning Officer Webinar September 13, 2012 14

16. A: Written business plan for learning approved by the CEO,
governing body or other senior leaders
B: Written business plan for learning but not approved by
senior leaders
C: PowerPoint business plan for learning approved or shared
with senior leaders
D: We have L&D goals but no formal plan. We respond to
requests as they come in.
Chief Learning Officer Webinar September 13, 2012 15

17. Manage your execution to deliver planned
outcomes, effectiveness, and efficiency
Review progress monthly within L&D
◦ Year-to-date results versus plan
◦ Will you make plan?
◦ If not, what is the forecast?
◦ What action should you take?
Review progress at least quarterly with CEO,
senior leaders
◦ Focusing on most important business goals, key measures
Chief Learning Officer Webinar September 13, 2012 16

18. A: Report results to our CEO or top-level leader at least
quarterly
B: Report results to our SVP of HR or other direct report to the
top-level leader at least quarterly
C: Report semi-annually or annually
D: Don’t report results to others
Chief Learning Officer Webinar September 13, 2012 17

19. Execute your measurement and evaluation
strategy. For example,
◦ Level 1 for all courses
◦ Level 2 where appropriate
◦ Level 3 for 10 key programs
◦ Levels 4 and 5 for three key programs
Did you achieve your goals?
If not, what can you learn? What can you
improve?
If you exceeded goals, what can you learn?
Chief Learning Officer Webinar September 13, 2012 18

20. Conceptually not difficult
Challenging in practice
Requires new skills for some, confidence to
apply them for others
Some common objections
◦ No standards in place to provide guidance
◦ No internal processes to support it
◦ Not comfortable with business-like approach
TDRp can help
Chief Learning Officer Webinar September 13, 2012 19

21. Brings principles, standards, definitions and
recommended statements and reports to
talent development
Answers key questions:
◦ What does the CLO need to run learning like a business?
What data should we collect?
How are the measures defined?
◦ What do the CEO and senior leaders want to see and how
should it be presented?
Chief Learning Officer Webinar September 13, 2012 20

22. Began in fall 2010 by Kent Barnett (CEO,
Knowledge Advisors) and Tamar Elkeles (VP of
Learning and Organization Development,
Qualcomm)
Engaged industry thought leaders like Fitz-enz,
Bassi, Phillips, Brinkerhoff, Bersin and CLOs of
major organizations
TDRp for L&D completed in March 2012. Now
being extended to rest of HR.
Chief Learning Officer Webinar September 13, 2012 21

23. Standard Measures:
• Outcome measures: What Reports
is the impact on the
business?
• Efficiency measures: How much? How many? At what
cost?
• Effectiveness measures: How well?
Statements
Program/ Initiative System Wide
Analytics Analytics
Standard Measures
Specific methodology (e.g. Systems and processes
Phillips, Brinkerhoff) for organizing data,
calculating measures
Data
Data Sources and Systems
Chief Learning Officer Webinar September 13, 2012 22

24. Three Statements:
• Outcome Statement
• Effectiveness
Reports
Statement
• Efficiency Statement
Program/ Initiative System Wide
Analytics
Statements Analytics
Standard Measures
Specific methodology (e.g. Systems and processes
Phillips, Brinkerhoff) for organizing data,
calculating measures
Data
Data Sources and Systems
Chief Learning Officer Webinar September 13, 2012 23

25. Three Reports:
• Summary Report
• Program Report Reports
• Operations Report
Statements
Program/ Initiative System Wide
Analytics Analytics
Standard Measures
Specific methodology (e.g. Systems and processes
Phillips, Brinkerhoff) for organizing data,
calculating measures
Data
Data Sources and Systems
Chief Learning Officer Webinar September 13, 2012 24

26. • Three standard statements • Summary statements show
◦ Outcome ◦ Last year’s actual
◦ Effectiveness ◦ Plan (or goal) for this year
◦ Efficiency ◦ Year-to-date results
• Standard measures are • Detail statements show
used but choice of ◦ Monthly, quarterly, trend data
measures depends on ◦ Granularity
organization ◦ Without plan
Chief Learning Officer Webinar September 13, 2012 25

27. Shows alignment of learning to goals
Expected impact of learning on those goals
◦ Isolated impact of learning (level 4)
Participant’s forecast
Sponsor’s forecast, estimate
Statistical estimate (after the training is completed)
◦ Application (level 3)
◦ Proxies for impact
In some cases may be number of participants
◦ Agreed upon with stakeholder upfront
Chief Learning Officer Webinar September 13, 2012 26

28. Results through June
2011 For 2012
Priority Business Outcomes and Learning Impact Actual Plan Jun YTD % of Plan
1 Revenue: Increase Sales by 20%
Corporate Goal or Actual % 10% 20% 17% 85%
Application of Training: % Applying 3 key concepts % NA 80% 85% 106%
2 Leadership: Increase EOS (1) Leadership Score by 5 Points
Corporate Goal or Actual Points 0 pts 5 pts 3 pts 60%
Impact of Training: Time spent coaching direct reports Minutes/wk NA 60 50 83%
3 Safety: Reduce Injuries by 20%
Corporate Goal or Actual % 10% 20% 15% 75%
Impact of Training: 70% reduction in injuries due to L&D % 5% 14% 11% 79%
4 Call Center Satisfaction: Improve Satisfaction Score by 4 Points
Corporate Goal or Actual Points 1.6 4.0 2.9 73%
Impact of Training: High, Medium, Low High/Medium/Low NA High High
5 Comply with New Regulations
Corporate Goal or Actual % in compliance 100% 100% 100%
Impact of Training: Essential for compliance NA Essential Essential
6 Innovation: Generate 10 New Patentable Ideas
Corporate Goal or Actual # 4 10 7 70%
Impact of Training: None planned NA NA NA
Chief Learning Officer Webinar September 13, 2012

29. Kirkpatrick/Phillips five levels
◦ Level 1: Satisfaction, Forecasts of levels 3-5
Participant
Sponsor
◦ Level 2: Amount learned
◦ Level 3: Application rate
◦ Level 4: Impact
◦ Level 5: Value (net benefit or ROI)
Some will have only L1, L2
Chief Learning Officer Webinar September 13, 2012 28

30. Chief Learning Officer Webinar September 13, 2012

31. • Volume • Utilization
◦ Participants ◦ Courses, classes, instructors
◦ Courses, classes, hours ◦ By ILT, vILT
- Offered, taken • Effort to develop and maintain
◦ By ILT, vILT, WBT, etc. • Reach
• Costs • Cycle times
◦ Development, delivery, ◦ Performance consulting,
management development, delivery
◦ Tuition
◦ Opportunity
◦ Cost reduction due to e-
learning
*See a complete list in the Definition of
Terms and Measurements document
Chief Learning Officer Webinar September 13, 2012 30

32. Chief Learning Officer Webinar September 13, 2012

33. • Three levels of reports • Executive-level reports
◦ Summary Report (for CEO) show
◦ Program Report (for CLO) ◦ Last year’s actual
◦ Operations Report (for CLO) ◦ Plan (or goal) for this year
◦ Year-to-date results
◦ Highly customized, ◦ Forecast for this year
pulling the most important • Detailed reports for
measures from the managers may show
statements ◦ Monthly, quarterly, trend data
◦ Granularity
◦ Without plan or forecast
Chief Learning Officer Webinar September 13, 2012 32

34. Results Through June
2011 For 2012
Impact of Learning Actual Plan Jun YTD % Plan Forecast
On Top Ten Business Goals (1)
Sponsor Estimate of Impact On 5 point scale 2.3 pts 2.5 pts 2.4 pts 2.4 pts
Application Rate % who applied it 65% 75% 71% 73%
Total Participants Number 25,000 30,000 17,268 58% 32,000
On Top Three Business Goals
Increase in Sales by 20%
Corporate Goal or Actual % 2% 20% 17% 20%
Application of Training (application rate) % NA 80% 85% 85%
Total participants Number 386 1,000 826 83% 1,115
Increase Leadership Score by 5 points
Corporate Goal or Actual points 0 pts 5 pts 3 pts 4 pts
Impact of Training (time spent coaching) minutes/week NA 60 50 50
Total participants Number 0 400 168 42% 400
Reduce Injuries by 20%
Corporate Goal or Actual % 10% 20% 9% 20%
Impact of Training: 70% reduction in injuries due to L&D % 5% 14% 6% 43% 14%
Total participants Number 4,216 11,000 5,826 53% 11,300
Total Participants in All Programs Number 40,121 50,000 22,013 44% 48,000
Unique Participants in All Programs Number 20,263 23,000 18,407 80% 24,000
Chief Learning Officer Webinar September 13, 2012

35. Results Through June
2011 For 2012
Actual Plan Jun YTD % Plan Forecast
Effectiveness
Participant Feedback % favorable 80% 85% 87% 102% 85%
Sponsor Feedback % favorable 75% 80% 77% 96% 78%
Learning Score 78% 85% 83% 84%
Application rate % who applied it 61% 75% 78% 79%
Efficiency
% of courses developed on time % 82% 95% 88% 94%
% of employees reached by L&D % 85% 88% 72% 88%
% of ee's with development plan % 82% 98% 95% 96%
E-learning course utilization rate % taken by 20+ 83% 97% 91% 97%
L&D Investment
L&D Expenditure Million $ $15.8 $20.2 $9.9 49% $20.2
Opportunity Cost Million $ $3.4 $2.9 $1.3 45% $2.9
Cost Reduction (internal to L&D) Thousand $ $63 $295 $168 57% $325
Note: (1) Learning is expected to contribute to achievement of all ten top goals except Goal 8: Open Office in Beijing
Key programs in support of the top ten goals include Marketing, Leadership, Safety, Fraud Detection, Business Acumen,
Engineering, Performance Mgt, New Employee Orientation, and Innovation.
Chief Learning Officer Webinar September 13, 2012

36. Chief Learning Officer Webinar September 13, 2012

37. Chief Learning Officer Webinar September 13, 2012

38. Chief Learning Officer Webinar September 13, 2012

39. Simple, consistent approach to reporting
◦ Use standard definitions to extract and convert data into
three types of measures
Outcome, effectiveness, and efficiency
◦ Create three standard statements with these measures
Outcome, Effectiveness, and Efficiency
◦ Create three types of customized reports from these
statements
Summary, Program (Initiative), and Operations
Chief Learning Officer Webinar September 13, 2012 38

40. Executive Reporting Process
Senior Executives Learning Executives 39
Executive L&D Summary L&D Program L&D Operational
Reports Report Report Report
Quarterly Monthly Monthly
Learning Learning
Outcome
Statements Statement
Effectivenes Efficiency
Guiding Principles
s Statement
Statement
Data Sets
Outcomes Effectiveness Efficiency
Extract, convert and calculate Standard Measures
Data
Sources
Financial Data Learning Management Evaluation Other Sources:
System System HRIS, CRM, ERP
Chief Learning Officer Webinar September 13, 2012

41. Step 1: Align learning to goals
◦ => The Outcome Statement
Step 2: Plan carefully
◦ => The three statements
Step 3: Execute with discipline
◦ => The three reports
Step 4: Measure and evaluate
◦ => All statements and reports
Chief Learning Officer Webinar September 13, 2012

42. Running learning like a business
◦ The Business of Learning by David Vance
◦ The Chief Learning Officer by Tamar Elkeles and Jack Phillips
◦ Strategic Learning Alignment by Rita Smith
Resources Available at TDRprinciples.org
◦ 56-page white paper, 18-page overview, 4-page summary
◦ Definition of terms and measures for L&D
◦ PowerPoints
◦ Sample write-up of an L&D Summary Report for a CEO
Contact Dave Vance for more information:
dave@poudrerivergroup.com
Chief Learning Officer Webinar September 13, 2012 41

43. Questions & Answers
Dave Vance
Former President
Caterpillar University
#CLOwebinar

44. Join Our Next CLO Webinar
Virtual Training: Building a
Global Training Strategy
Wednesday, September 19, 2012
CLO Webinars start at 2 p.m. Eastern / 11 a.m. Pacific
Register at www.clomedia.com/events
Join the CLO Network: http://network.clomedia.com/
#CLOwebinar