Training needs analysis, skills auditing, training evaluation, calculating training ROI and strategic learning and development best practice principles and processes
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
Training needs analysis, skills auditing and training
1. TRAINING NEEDS ANALYSIS, SKILLS
AUDITING AND TRAINING
EVALUATION/ROI
8-10 JUNE 2016
CHARLES COTTER
www.slideshare.net/CharlesCotter
2. TRAINING PROGRAMME OVERVIEW
• Understanding and Applying the ADDIE training process/cycle
• Applying the Training Needs Analysis process
• Applying the Skills Auditing process
• Compiling Workplace Skills Plans (WSP) and Annual Training
Reports (ATR)
• Training Evaluation and Calculating Training Return on Investment
(ROI)
• Strategic Learning and Development
3. INTRODUCTORY LEARNING ACTIVITY
• Individual activity:
• Complete the statement by inserting one (1) word only. In order
to conduct an effective training needs analysis, I need to/to be
.…………………………………..
• Now find other learners with the same word as you.
• Jot these words down on the flip-chart.
• Each learner will have the opportunity to elaborate on their
chosen word.
5. GENERIC TRAINING AND
DEVELOPMENT PROCESS/CYCLE
• Step 1: Identify the overall business needs
• Step 2: Identify the success criteria
• Step 3: Establish individual training needs
• Step 4: Design and develop a learning solution
• Step 5: Review the learning solution
• Step 6: Production of learning and instructional materials
• Step 7: Implement the learning solution (delivery of training)
• Step 8: Evaluate and Review the impact of training
7. PERFORMING A TRAINING NEEDS
ANALYSIS (TNA)
• The objectives and benefits of a training
needs analysis
• Levels of training needs
• SDF responsibility in TNA
• The TNA process
8. OBJECTIVES AND BENEFITS OF TNA
• Provide necessary information on participants
• Identify employee difficulties and performance problems
• Identify important topic areas
• Provide documentation and materials for training
• Provide information on attitudes to training
• Increase employee involvement
• Help establish contact with subject specialists
• Help estimate the cost of training
• Save time, money and resources
• Help tailor the service more accurately
• Provide a means of measuring training effectiveness
9. LEVELS OF TNA
• Organizational Needs
Identify key problem areas that affect performance on an organization-
wide basis.
• Job or Occupational Needs
The knowledge, skills, information, equipment, materials etc. needed
to perform to standard in the job and the steps in performing the job
• Individual Needs
This level ensures that individuals who need training are the ones who
actually get it and that the training introduced bridges the gap
between actual, current and desired, future performance.
10. SDF RESPONSIBILITY IN TNA
• The role of the SDF is to offer expert advice
to management about what training can and
cannot achieve
• To facilitate the needs assessment process so
that it is a participative, transparent and in-
house consulting role.
11. 6-STEP TNA PROCESS
• Step One: Situational and problem analysis
• Step Two: Envisioning the desired end state
• Step Three: Identifying the TNA methods
• Step Four: Data collection
• Step Five: Reaching, sharing and presenting TNA
findings
• Step Six: Compile an Implementation Plan of Action
12. STEP ONE: SITUATIONAL AND PROBLEM
ANALYSIS
• Focuses on getting a clear and realistic understanding of the situation or problem
that we think training may address and on defining the starting point for the TNA.
• Fundamental questions to ask?
• A situational analysis is conducted to assess the unique requirements of a
situation when there is a perception that everything is not as it should be.
• Indicators (of a problem)?
• Three types of problems:
Managerial
System
Job performance
• Preferred environmental scanning tools: PESTEL and SWOT Analyses
14. STEP TWO: ENVISIONING THE
DESIRED END STATE
• Encourages you to ’draw a picture" of what the future will look like
once the training needs have been addressed. A vision is a mental
picture of a possible future state. Focus on:
What would success look like?
Where do we want to be?
Do we have the whole picture?
• Step 2 represents the information that allows you to do a kind of
gap analysis - between the present and the future. It becomes
possible to work out how to get from now to then. Additional
useful questions to ask are:
What are the challenges to getting the desired results?
Who else do we need to involve or convince to get the desired results?
15. LEARNING ACTIVITY 1
• Group Discussion:
• By referring to the TNA process, apply steps
1-2 to a defined organizational context.
16. STEP THREE: IDENTIFYING THE TNA
METHODS
• Who to ask?
Who knows the most?
Who wants to be involved?
Who would have a different perspective, who might derail the process if they are not involved?
Who can provide objective information?
• What to ask?
How do you see the situation?
What do you think needs to be done?
What training is needed?
Is there anyone else we should talk to?
• What data collection method/s to use?
Job analysis
Performance appraisal
Skills Auditing
18. STEP FOUR: DATA COLLECTION
• This step focuses on collecting, interpreting or
processing the information so that you can
work out what it is really telling you.
• You need to:
Identify the issues
Prioritise the issues
Prepare the information to share with others
19. LEARNING ACTIVITY 2
• Group Discussion:
• By referring to the TNA process, apply steps
3-4 to a defined organizational context.
20. STEP 5: REACHING, SHARING AND
PRESENTING TNA FINDINGS
• Your results must be shared in a way that will
help you to move forward.
• Decide:
What you want to share
How you want to share it
21. STEP 6: COMPILE AN
IMPLEMENTATION PLAN OF ACTION
• Now you must draw up a plan of action.
• The objective is to ensure that everything you
have done up to now does not go to waste.
• Incorporation of findings into Workplace Skills
Plan (WSP).
22. LEARNING ACTIVITY 3
• Group Discussion:
• By referring to the TNA process, apply steps
5-6 to a defined organizational context.
23. SKILLS AUDITING PRESENTATION
OVERVIEW
• Definition, purpose and outcome of a Skills
Audit
• Diagnosis: Current Skills Audit practices and
processes
• Defining and Measuring of Competence
• Building a Business Case for Skills Audits
(Benefits and Costs)
24. FUNDAMENTALS OF SKILLS AUDITING
• Costs and Benefits of Skills Audits
• Understanding the concept, “competence”
“Applied Competence is the union of practical, foundational and
reflexive competence”
• Types of evidence
• Techniques and Approaches for Conducting a Skills Audit
Panel approach
Consultant approach
One-on-one approach
Alternative approaches
25. INTRODUCTORY QUESTIONS
• Q1: For any business manager, what is their
most crucial financial and business risk
mitigation tool?
• Q2: For any Learning & Development
Manager/Skills Development Facilitator, what
is their most under-utilized people/skills risk
mitigation tool?
26. ORIGIN OF THE WORD, “AUDIT”
• The word audit originates from the Latin word
‘audire’ which means to “listen”.
• An audit is a systematic, objective risk
management tool for how well the workplace is
complying with regulatory and policy
requirements.
28. DEFINITION, PURPOSE AND
OUTCOME OF SKILLS AUDITING
• A skills audit is a snapshot that allows an organization to determine the
level of skills and knowledge of the workforce.
• It is compared against the competencies that are required in order to
determine the gaps and to focus training and development accordingly.
• Skills audits are conducted to determine training needs within an
organization in order for that organization to improve its skills and
knowledge.
• A skills audit establishes an individual’s current competence against the
skills matrix for a particular position.
• A skills audit gathers more information than current qualifications levels.
• The outcome of the skills audit process is a skills gap analysis.
31. BEST PRACTICE CRITERIA: SKILLS
AUDITING
• #1: A job analysis must be used as a basis for the skills audit
• #2: Definitive performance standards must be developed, written,
and provided to all stakeholders, regardless of the type of rating
• #3: Raters are trained to use the rating instrument properly
• #4: Formal appeal mechanisms must be in place and assessment
results need to be reviewed to ensure fairness and reliability
• #5: Multiple techniques/approaches are utilized and ratings are
supported with documented examples of behaviour
32. BEST PRACTICE CRITERIA: SKILLS
AUDITING
• #6: Employees are given a chance to improve their skills through targeted
development opportunities
• #7: The 6 E’s - the Skills Auditing process is efficient, effective, economical,
educational, ethical and evidentiary
• #8: Compliance with the following principles of Skills Audits:
Fairness
Validity
Reliability
Transparency/ Openness
Constructive feedback
Objective
• #9: The outcome of the skills audit generates predictive analytics and business
intelligence, providing the organization with a strategic competitive advantage
• #10: Skills Auditing must be a holistic, systematic, integrated and aligned
approach
33. LEARNING ACTIVITY 4
• Individual activity:
• Review and evaluate your organization’s
current skills audit process against the ten
(10) best practice criteria.
• Identify gaps and recommend improvement
strategies to address these process gaps.
35. STRATEGIC IMPERATIVE OF SKILLS
AUDITING
• The key piece of information an organization needs to improve and to
deliver to its Mission Statement and strategy is to know what skills and
knowledge the organization requires and what skills and knowledge the
organization currently has. This information is essential for a number of
reasons:
Without this information you don't know where to improve.
With this information your training and development will be better planned
and more focused.
Recruiting needs are better defined and more likely to result in the most
appropriate candidate.
Placement decisions are easier with knowledge of current competence levels.
Career pathing and succession planning is assisted with accurate information
on individuals.
• Meyer, Mabaso & Lancaster (2001) recommend proactive needs
identification and a more futuristic approach to the assessment of
training needs.
36. KEY BENEFITS OF A SKILLS AUDIT
• Valid and valuable Workplace Skills Plans (WSP)
• Improved skills and knowledge
• Lower training and development costs because development efforts are more focused
• Business intelligence - acquisition and use of information that can be used for purposes such as
internal employee selection and placement
• Increased productivity as people are better matched to their positions
• The results of a skills audit can be reported for each division to show individual and divisional
competency gaps against competency needs.
• This assists with the collation of a WSP that complies with the provisions of the Skills Development
Act and Seta regulations.
• Lancaster, Mabaso & Meyer (2001, p86) claim that “the skills plan can only be produced after the
organization has conducted a skills audit and a comprehensive needs analysis”
• Certain SETA’s have included skills auditing as one of the requirements for the discretionary grant.
Organizations that conduct skills audits in a structured manner, may submit levy claims against
Grant D of the skills development regulations.
37. 5 C’S – THE KEY BENEFITS OF A SKILLS
AUDIT
• Compliance
+
• Competitive
+
• Cash
+
• Credibility
+
• Competence
= Clean Skills Audit
38. COSTS OF SKILLS AUDITS
• Training
• Time
• Administrative expenses (e.g. stationery)
• Information system/software
• Communication
• Use of consultants (where necessary)
39. POTENTIAL REPERCUSSIONS OF NOT
CONDUCTING A VALID SKILLS AUDIT
• Invalid and unreliable training plans
• Training plans that are not specific to individual, departmental and
organizational needs
• Little or no commitment to training & development by
management and staff, as plans are not seen as value-adding
• Little or no alignment of training and development to
organizational strategy and objectives
• Non-implementation of the Workplace Skills Plan and therefore the
organization will not be able to claim reporting grants
40. LEARNING ACTIVITY 5
• Group activity:
• Develop a business case for Skills Auditing i.e.do
the benefits outweigh the costs?
• Do skills audits comply with:
Viability
Feasibility
Sustainability
42. COMPETENCE
• “Applied Competence is the union of practical, foundational and reflexive
competence”
• Practical Competence - the demonstrated ability to perform a set of tasks in an
authentic context. A range of actions or possibilities is considered and decisions
are made about which actions to follow and to perform the chosen action.
• Foundational Competence - the demonstrated understanding of what the learner
is doing and why. This underpins the practical competence and therefore the
actions taken.
• Reflexive Competence - the learner demonstrates the ability to integrate or
connect performance with understanding so as to show that s/he is able to adapt
to changed circumstances appropriately and responsibly, and to explain the reason
behind an action.
• Thus competence is understood as including the individual’s learning,
understanding and ability to transfer and apply learned skills and knowledge
across a wide range of work contexts.
45. TECHNIQUES/APPROACHES TO A
SKILLS AUDIT
• Panel approach
• Consultant approach
• One-on-one approach
• Alternative approaches:
Competence-based self-assessment with validation by direct manager or supervisor
360 degree reviews
Focus groups
Assessment centres
Assessment by subject matter experts
46. EVIDENCE
• Types of evidence:
Direct
Indirect
Historical
• Evaluation of evidence (VACCS):
Validity
Authenticity
Consistency
Currency
Sufficiency
48. SKILLS AUDITING PROCESS
• Step 1: Determine Skills Requirements
• Step 2: Audit actual skills
• Step 3: Determine development needs and
plan for training/restructuring
49. STEP 1: DETERMINE SKILLS
REQUIREMENTS
• In order to determine skills requirements, an organization should identify
current and future skills requirements per job.
• The end result is a skills matrix with related competency definitions.
Definitions can be allocated against various proficiency levels per job,
such as basic, intermediate and complex.
• Objective: Determine the critical or required skills (elicited from job
profiles, your strategy, or competency matrix).
• Skills matrix process:
Step 1: Workshop with a project team (include Subject Matter Experts)
Step 2: Use outcomes analysis to derive skills/knowledge factors and unit
standard titles
Step 3: Use results of outcomes analysis and value chain process to develop a
skills matrix and titles matrix
Step 4: Verify matrices with SMEs and finalise
52. STEP 2: AUDIT ACTUAL SKILLS
• Step 2 involves an individual self-audit and skills
audit
• Results are collated into reporting documents
that may include statistical graphs, qualitative
reports and recommendations
• A skills audit includes auditing qualifications,
experience and training (knowledge)
• Conducting a Skills Gap Analysis
54. SKILLS AUDIT RATING SCALE
Rating Description Definition
0 No evidence of competence An individual does not currently display any form or level of competence
in the skill listed. He or she may require formal training and exposure to
the skill in the workplace.
0.25 Some evidence of competence The individual may demonstrate part competence, but definitely needs
formal training and exposure to the skill in the workplace.
0.5 Evidence of competence, needs
further training
An individual is competent, but needs to improve. Training is the most
effective solution. The individual may be at a lower level than the
position requires, i.e. at linear, instead of complex level.
0.75 Evidence of competence, needs more
exposure to the skill
The individual is competent and has undergone training. Further
exposure in the workplace would ensure improvement and full
competence. The individual may be at a lower level than the position
requires, i.e. at linear, instead of intermediate level.
1 Full evidence of competence The individual is competent in the skills at the level allocated to his/ her
position.
55. STEP 3: DETERMINE DEVELOPMENT NEEDS AND PLAN
FOR TRAINING/RESTRUCTURING
• Once skills audit information has been collected, an analysis of the
results may be used for planning purposes relating to training and
development and other Human Resource interventions.
• Recommendations are then discussed and agreed actions are
implemented.
• This skill shortfall forms the basis of a Training Needs Analysis
(TNA) so that the company can reach the desired skill base amongst
its employees.
• A gap analysis is the outcome of the skills audit process.
• Information that is provided through the skills audit can be used for
the multiple HRM and business purposes.
56. REPORTING SKILLS AUDIT RESULTS
• The reporting framework is generated according to the purpose you want to use
the skills data for.
• These reports are vital as they may be used to inform organizational training and
development strategy, Workplace Skills Plans, individual development plans and
performance management interventions etc.
• These reports must be stored in a manner that respects the confidentiality of
individual employees.
• It is therefore important to agree on and communicate who has access to skills
audit results, and how these people may use the results upfront.
• Skills audit reports may take on a number of forms:
Individual competency profiles
Divisional radar report
Organizational pie chart
57. INFORMATION EXTRACTION FROM
REPORTS
Individual name & employee number
0%
20%
40%
60%
80%
100%
120%
C
om
m
unication
Interpersonal
C
ontrolling
Adm
inistration
Planning
C
om
puter
D
rafting
legal docs
Property
law
Interview
ing
Department name - Divisional Radar Report
0
20
40
60
80
100
Communication Skills (B)
Interpersonal Skills (B/E)
Administration skills (D)
Planning Skills (D)
Management Skills (G)
Marketing (B/C)
Loans Processing (A/B/C/D)
Computer skills (D)
Company name
ORGANISATIONAL
STRATEGIC COMPETENCY PROFILE
66%
73%
75%
69%
75%
71%69%
A. Strategic competency listing B. C. D. E. F. G.
58. LEARNING ACTIVITY 6
• Group Discussion:
• Apply steps 1-3 of the skills audit process, to
a defined organizational context.
59. WORKPLACE SKILLS PLAN (WSP)
• The purpose of a WSP
• Objectives and scope of a WSP
• The value and importance of a WSP
• Alignment with the Business and HRM strategy
• Compiling a WSP
60. COMPILING A WSP
• Step 1: Develop an occupation classification matrix
• Step 2: Populate the occupation classification matrix
• Step 3: Establish the company’s skills development priorities
• Step 4: Define the education and training required for achieving
the strategic skills development priorities
• Step 5: Define the number and characteristics of training
beneficiaries that will be trained in the Levy-Grant Year
• Step 6: Define the quality assurance measures for each of the
planned education and training activities
61. STEP 3: ESTABLISH THE COMPANY’S SKILLS
DEVELOPMENT PRIORITIES
• Know-how
• Up-skilling
• Re-skilling
• Multi-skilling
• Refresher training
• Initial training
62. ANNUAL TRAINING REPORT (ATR)
• Defining an Annual Training Report (ATR)
Basically this report consists of all attendance registers, proof of expenditure, training provider
used in this report the SETA can establish whether training was done or is in the process of
being done.
• Submission requirements
This report reflects the education, training and development activities of the organisation that
were implemented.
The annual training report does exactly what it says; it basically tells the SETA what training
has taken place in the organisation in the previous 12 months, and which employees received
training.
It also contains information to the type, cost and delivery method of the training it goes
further to assess whether the training that the employees received is in line with the planned
training for the organisation, industry and critical skills identified by the SETA.
The ATR allows employers to monitor the achievement of the skills priorities and skills
development objectives that were outlined in the WSP.
63. LEARNING ACTIVITY 7
• Group Discussion:
• By referring to the process of compiling a WSP,
describe the efficiency and effectiveness of this
process in a defined organizational context.
• By referring to the process of compiling an ATR,
describe the efficiency and effectiveness of this
process in a defined organizational context.
• Identify gaps and recommend improvement
strategies.
64. DEFINING TRAINING ROI FORMULA
ROI is a key financial metric of the value of training investments and costs. It
is a ratio of net benefits to costs, expressed as a percentage.
The formula can be expressed as:
[(monetary benefits – cost of the training) / cost of the training] x 100
65. LEARNING ACTIVITY 8
• Individual activity:
• Complete the Training ROI Scorecard, 16-
point checklist. Identify gaps and recommend
improvement strategies.
67. 4-PHASE, TRAINING ROI PROCESS
• The calculation of training ROI should be
approached in an iterative, 4-phase approach:
Phase 1: Preparatory
Phase 2: Initiation
Phase 3: Analysis
Phase 4: Consolidatory
70. DIAGNOSTIC ACTIVITY
• Group Discussion:
• By referring to the 6 levels, determine your
organization’s level of training proficiency.
71. DIAGNOSIS OF CURRENT TRAINING AND
DEVELOPMENT PRACTICES
• How efficient is the training process; is the
attendance of scheduled training programmes
good and are learners satisfied post-training? –
LEVEL 1: EFFICIENT
• What is the submission rate of PoE’s and is there
a good success rate? – LEVEL 2: EDUCATIONAL
• What is the degree of transfer and application of
learning to the workplace and improved
behavioural change and performance? – LEVEL
3: EFFECTIVE
72. DIAGNOSIS OF CURRENT TRAINING AND
DEVELOPMENT PRACTICES
• What is the impact of training programmes on
organizational business results and metrics e.g.
productivity; competence; customer service etc. – LEVEL
4: VALUABLE
• What is the Return-on-Investment (ROI) of the training
programmes? Do the benefits exceed the costs? – LEVEL
5: ECONOMICAL
• To what extent do training programmes directly
contribute to the achievement of strategic objectives;
drive innovation; generate business solutions and create
sustainable competitive advantages for the organization?
– LEVEL 6: STRATEGIC
75. L&D METRICS
• Measures of training activity (concerning how
much training and development occurred with
the focus on formalised, structured learning)
• Measures of training results (concerning how
well training and development achieved its goals)
• Measures of training efficiency (concerning the
extent to which training and development
maximises resources in pursuit of its mission)
76. TRAINING ROI REPORT
• Executive Summary
• Introduction and Contextualization
• Analysis and Results
• Recommendations and Training Improvement Plan
• Conclusion
• Reference to Attachments
77. LEARNING ACTIVITY 9
• Group Discussion:
• By referring to the 4-step ROI process,
describe how the measurement of the impact
of training can be measured at your
organization.
82. STRATEGIC IMPERATIVE OF LEARNING
AND DEVELOPMENT
• Deloitte (2014): The Nine Critical Talent Imperatives inter alia:
Accelerating Time to Competency
Driving Performance & Development
Improving Management & Leadership
• Building a leading L&D function will likely not only drive
performance, but also improve employee engagement.
• “For far too long, training has been a passive, organizational back-
seat driver. It should come to prominence by enabling and
ultimately, driving strategy and it’s achievement.” (Cotter, 2015)
• If skills shortages are seen as a top threat to business expansion,
leadership will turn to learning managers for a response
(justification).
84. 10 CRITICAL SUCCESS FACTORS FOR THE TRANSFORMATION OF
TRAINING TO A STRATEGIC LEARNING SOLUTION
• #1: Top management support and ownership
• #2: Vibrant and effective Performance Management
System (PMS)
• #3: Direct and active engagement, consultation and
participation of line management in all learning
processes
• #4: Training Managers need to adopt and apply a
strategic mind-set (conceptual thinking)
• #5: Establishment of a learning organizational culture
85. CULTURE IS CRITICAL
• L&D should be a poster child of “people
investment” in your company—creating not
only great training, but also reinforcing the
culture of learning.
• Research on learning culture shows that,
among all of the different investments in
learning that can be made, creating a culture
of learning is the most important of all.
88. 10 CRITICAL SUCCESS FACTORS FOR THE TRANSFORMATION OF
TRAINING TO A STRATEGIC LEARNING SOLUTION
• #6: Holding individuals accountable for application of
learning by means of e.g. learner contracts/agreements
• #7: When utilizing outsourced training providers ensure
performance-directed, Service Level Agreements are in
place
• #8: Learning and Development must be embedded in the
business strategy
• #9: Learning strategy must precede structure
• #10: Commitment to training as an investment and not
cost item
90. LEARNING ACTIVITY 10
• Group Discussion:
• Evaluate the current degree of compliance to
the 10 best practice criteria. Identify gaps and
recommend improvement strategies.
91. ANALYSIS
• Required Thinking – Laboratory Scientist
• Accurate sourcing of training needs by means of properly
performed and scientifically reliable and valid skills audits
• Accurate sourcing of performance gaps by means of a vibrant
performance management system/process
• “Training is not the Alpha and Omega and the cure for all
organizational ills.”
• (Vertical) Alignment with Strategic Business Plan and Strategic
Workforce Plan and horizontal integration (bundling) with other
key HRM functions/processes
92. DESIGN
• Required Thinking – Architect
• O-R-C-A – Outcomes; Resources; Capabilities and Activities
• Contract learning curriculum design specialists
• Ensure quality assurance of all learning materials and assessment
tools
• Transform to a technology-driven or web-based methodology e.g.
e- or m-learning, MOOC’s or gamification
• “Organizations should redesign their learning architecture”
(Deloitte, 2015)
97. DEVELOPMENT
• Required Thinking – Construction Manager
• Applied Competency-based methodology (SAQA definition:
foundational; practical and reflexive)
• Contract a diverse, task team of subject matter and development
experts
• Review, pilot and consult with line management to determine
relevance, compatibility and value of learning offering
• “Companies should focus on building a complete learning
experience.” (Deloitte, 2015)
98. IMPLEMENTATION
• Required Thinking – Postman, because they always
deliver
• Due diligence to verify competence of trainers
• SAPTA – Certified Professional Trainers (CPT)
www.saptaonline.org
99. EVALUATION
• Required Thinking – Engineer
• Develop policy, processes, systems and
learning analytics to measure the impact of
learning beyond levels 1-3
• Revision of formative and summative
assessment practices
• Training ROI
100. LEARNING ACTIVITY 11
• Group Discussion:
• Describe how strategic principles can be
applied to the ADDIE training cycle to
transform to a strategic impact and level.