Title of the presentation is Bite The Bullet and which goes very well with the subject in focus.This slide set was created out of the ATD book on measuring impact of sales trainings for the purpose of internal training and have no commercial value.
very interesting insights into concepts like ROI and aligning training with business strategy and showing its impact on business.
Credits: ATD, Patricia Pulliam Phillips, PhD
Jack J. Phillips, PhD
and Rachel Robinson
11. Operating Standards and Philosophy
These standards provide clear guidance about the specific ways to implement the ROI Methodology
to ensure consistent, reliable practice in evaluating sales training.
15. ACHIEVING BUSINESS ALIGNMENT
Activity Based Approach
Business need is not linked to the sales training in terms
of monetary impact.
Assessment of performance issues that will be addressed
in sales training are not captured in a quantifiable,
measurable manner.
Specific measurable, quantifiable objectives are not
clarified.
Participants are not fully prepared to participate in the
program.
The environment is not prepared to support the transfer
of learning to behavior change and business impact.
Partnerships with key managers to support the
participant have not been identified and developed.
Results or benefit-cost analysis in real, tangible, and
objective measures—including monetary impact—are not
captured.
Planning and reporting is input focused.
Result Based Approach
Program is linked to specific business impact measures
such as revenue, productivity improvement, new
customers, etc.
There is an assessment of performance effectiveness that
needs to improve.
Specific, measurable objectives for behavior change and
the related business impact are identified.
Results expectations are communicated with and in
partnership with participants.
Work environment is prepared to support transfer of
learning to behavior change and business impact.
Partnerships are established with key managers and
clients prior to training to ensure participation and
support.
Results and benefit-cost analysis are
measured.
Planning and reporting is outcome focused.
16. Clarifying Stakeholders’ needs
To ensure sales training programs lead to results, they must first be positioned for success.
The ultimate need for sales training is in the potential payoff of the investment.
I. Is this program worth implementing?
II. Is this a problem worth solving?
III. Is this an opportunity worth pursuing?
IV. Is there likelihood for a positive ROI?
21. Improving Response Rate
• Provide advance communication.
• Communicate the purpose.
• Identify who will see the results.
• Let the target audience know that they are part of a
sample.
• Add emotional appeal.
• Design for simplicity.
• Make it look professional and attractive.
• Use the local manager support.
• Pilot test the questionnaire.
• Recognize the expertise of participants.
• Consider the use of incentives.
• Have an executive sign the introductory letter.
• Report the use of results.
• Provide an update to create pressure to respond.
• Present previous responses.
• Introduce the questionnaire during the program.
• Use follow-up reminders.
• Consider a captive audience.
• Consider the appropriate medium for easy response.
• Estimate the necessary time to complete the
questionnaire.
• Show the timing of the planned steps.
• Personalize the process.
• Collect data anonymously or confidentially.
• Send a copy of the results to the participants.
23. Timing of Data
Collection
Level 1 Level 2
Level 3 Level 4
At the end of the sales training
program
During or at the end of the
Program
Three to six months after the sales
training program is complete.
Either at the time of Level 3 or a
little later
24. ROI Analysis
Isolating The Effects Of Sales Training
Other Factors Are Always Present
Without Isolation, There Is Only Evidence, Not Proof
Other Factors Have Protective Owners
To Do It Right Is Not Easy
26. Comparison Group Analysis
1. Highly Accurate
2. Highly Credible
1. Practically Challenging
2. Research Oriented
3. Chances of contamination
4. Environmental factors
27. Trend Line Analysis
1. Simple
2. Inexpensive
1. Not always accurate
2. Lack of data
3. No consideration for external
influences
28. Forecasting Methods
1. Can Predict performance in business
measures with accuracy
2. Credible & reliable results
1. When multiple variable enter the process
2. Complex
3. Lack of mathematical methods to convert
input into Output
29. Expert Estimation
1. Focus Group Approach
i. Credible
ii. Low-cost
i. Identifying the right factors
ii. Establishing link between performance &
Training
30. Expert Estimation
2. Questionnaire Approach
i. Credible
ii. Low-cost
iii. Less
analysis
i. It is just an estimate
ii. Establishing link between
performance & Training
iii. Collecting quality data
31. Selecting Isolation Techniques
a) Feasibility of the technique
b) Accuracy provided with the technique
c) Credibility of the technique with the target audience
d) Cost of implementation
e) The amount of disruption in normal work activities as the technique
is implemented
a)Participant, staff, and management time needed with the particular
technique.
32. Types of Data
Hard Data v/s Soft Data
• Output
• Quality
• Cost
• Time
• Customer satisfaction
• Customer loyalty
• Brand awareness
• Reputation.
Tangible vs. Intangible Data
• Tangible benefits of a program
are those benefits that have
been converted to money.
• Intangible benefits of a program
are those benefits that have not
been converted to money.
Data Conversion
34. Five Steps To Data Conversion
1) Focus on the unit of measure.
2) Determine the value of each unit.
3) Calculate the change in the performance of the measure
4) Determine the annual improvement in the measure.
5) Calculate the total monetary value of the improvement
35. ROI Calculation
1. Benefit-cost Ratio 2. ROI Percentage
BCR = 150,000
100,000
Benefit
Cost
= 1.5 or 1.5:1E.g.
ROI = 150,000-100,000
100,000
Benefit-cost
Cost
X 100 = 50%E.g.X 100
Payback Period =
150,000
100,000
= .50X12months = 6 Months
36. Intangible Benefits
• Job satisfaction
• Organizational commitment
• Teamwork
• Customer satisfaction.
When you report ROI, always balance it with the intangible benefits.
This balance places the ultimate benefits of the sales training initiative into perspective.
37. Reporting & Using ROI
Evaluations of sales training programs are essentially useless if the results are never
communicated
1. Identify The Need
2. Identify the audience
4. Select the media
3. Develop the report.
5. Evaluate the results.
38. Develop The Report
1. Detailed Reports
3. General Audience Reports
2. Executive Summary
4. Single Page Report
• Need for the program
• Need for the evaluation
• Evaluation methodology
• Results
• Conclusions and next steps
• Appendices.
5. Macro-Level Scorecards
39. Evaluate The Results
How to evaluate the success of our communication?
• Observe reaction to the information and the communication process
• Ask participants if they know what the data mean
• Follow up on actions taken as a result of the communication
Remember, there are no perfect ROI studies—someone will find an improvement opportunity in
everything you do.
The term sales has a far-reaching scope that goes well beyond car salesmen, department store clerks, or bank tellers. Sales professions have become prominent in a vast number of industries, spanning both the private and public sectors. Almost everyone has a product or service to sell.
More and more, companies are recognizing that sales is a profession, and thus an increased emphasis on sales training is imperative.
Within the context
of sales training, it answers the question: For every dollar invested in sales training
or enablement, how many dollars were returned, above and beyond the investment?
BCR= Program Benefits/program cost
ROI%= PB-PC x 100
PC
The system begins with the five-level ROI framework, developed in the 1970s and becoming prominent in the 1980s (Phillips, 1983). Today, this framework is used to
categorize results for all types of programs and projects.
This third part of the evaluation puzzle ensures consistent decision making around the application of the ROI Methodology. When the 12 guiding principles shown in Table 1-2 are followed, consistent results can be achieved. Additionally, these guiding principles help maintain a conservative and credible approach to data collection and analysis.
The 12 guiding principles serve as a decision-making tool and influence decisions on the best approach by which to collect data, the best source and timing for data collection, the most appropriate approach for isolation and data conversion, the costs to be included, and the stakeholders to whom results are reported. Adhering to the 12 guiding principles provides credibility when reporting results to executives.
The fourth part of the ROI Methodology evaluation puzzle includes case applications and practice, which allow for a deeper understanding of the ROI Methodology’s comprehensive evaluation process. Case application is a way to provide evidence of a sales training program’s success. The case studies in this book provide excellent examples of application of the ROI Methodology. While practitioners who are beginning their pursuit of the ROI Methodology can learn from these case studies, as well as those found in other publications, the best learning comes from actual application. Conducting your own sales training ROI study will allow you to see how the framework, process model, and operating standards come together. Your first ROI study serves as a starting line for your track record of sales training program success.
Finally, the last part of the ROI Methodology evaluation puzzle is implementation. While it is important to conduct a sales training ROI study, one study alone adds very little value to your efforts to continuously improve and account for your investments. The key to a successful sales training function is to sustain the use of ROI. Building the philosophy of the ROI Methodology into everyday decisions about your practice is imperative for attaining credibility and consistency in training effectiveness. Implementing the ROI Methodology requires assessing the organization’s culture for accountability and its readiness for evaluating sales training programs at the ROI level. It also requires defining the purpose for pursuing this level of evaluation, building expertise and capability, and creating tools, templates, and standard processes.
As the sales profession has matured, accountability for sales training effectiveness has increased. In the past, we could provide nearly any talent development solution,
including sales training, to clients and measure the success of that solution based on self-reports. But in a world of tighter budgets, less time, and fewer resources, this is
no longer enough. Instead, the very real demands for sales training accountability require that we shift from an activity-based approach to a results-based approach to
sales training,
Positioning occurs through the establishment of objectives, Objectives are core to business alignment, They are also the first step in the ROI process, Objectives drive the design of the sales training. They evolve from the needs assessment process and drive the evaluation process. Objectives serve as the catalyst between what stakeholders want and need and what they receive and accomplish from a program or initiative. The first step toward developing objectives is clarifying stakeholder needs. The problem or opportunity can be obvious, such as:
Sales have decreased 30 percent from last year.
Very low market share in a market with only a few competitors.
Sales are flat—no growth.
Product returns have increased 20 percent in six months.
Customer satisfaction has declined two years in a row.
Or they may be less obvious, such as:
OO We want our customers to be more engaged.
OO The sales force should be more consultative.
OO Every sales professional must have negotiation skills.
OO We need more just-in-time training.
OO Let’s organize a business development conference.
OO Create a great place to work for the sales team.
When the payoff need is addressed, the specific business measures that need to improve to address the payoff need are identified. These “business
needs” represent hard data, categorized in broad categories of output, quality, cost, and time; they may also represent soft data such as measures of satisfaction, image,
and reputation. Examples of business measures in need of improvement may be sales, new accounts, market share, sales expense, customer satisfaction, customer loyalty,
product returns, and customer complaints.
When planning a sales training program evaluation, two documents are completed in as much detail as possible.
These seven key areas are addressed in detail in the ROI analysis plan. Figure 2-3shows a completed ROI analysis plan. Planning in detail what you are going to ask, how you are going to ask, who you are going to ask, when you are going to ask, and who will do the asking, along with the key steps in the ROI analysis will help ensure successful execution. Additionally, having clients sign off on the plans will ensure support for the evaluation approach when the evaluation results are presented.
The most-often used methods of data collection for ROI evaluation are questionnaires,interviews, focus groups, action plans, and performance records.
While the ROI calculation is an annual benefit, it is unlikely that you will wait a full year to capture Level 4 data. Senior executives usually want to see results sooner than later. If the sales training program was introduced to solve a problem (such as, unsatisfactory sales revenue), the concern will either go away, executives and senior managers will forget, or a decision will be made without the data about the sales training program.
When using the ROI Methodology, data analysis should include the isolation of program effects on the data, calculation of fully loaded program costs, conversion of data to monetary values when appropriate, and the ROI calculation. Isolating the effects of a sales training or enablement program on business impact data is one of the most challenging, yet necessary, steps in the ROI Methodology.
Before reviewing the specific methods, it is helpful to highlight two important issues. First, although it is possible to isolate for the effect of the sales training program at Level 3, application, isolation is usually applied to Level 4, impact. The business impact connected to the program is the key issue because when calculating ROI, the
improvement in business measures is reported in monetary terms and more than any other level, it must be credible. After the business impact data have been collected,
the next step is to isolate the effects of the program. This step demonstrates the proof that the sales training program made a difference,
The most accurate and credible approach to isolate the effects of a sales training program is a comparison group analysis, known as the control group arrangement. This
approach involves the use of an experimental group that participates in the sales training program and a control group that does not. The composition of both groups should be as similar as possible, and, if feasible, the selection of participants for each group should be on a random basis.
The key element in this approach is to track the trend using historical data; project where the trend would be without help from a program; then after the sales training
occurs, track the actual data over the same period of time as the pre-program data. Then the comparison can be made between what the forecast data show and what the
actual data show.
A more analytical approach to trend-line analysis is the use of forecasting methods that predict a change in performance variables based on the correlation of other variables. This approach represents a mathematical interpretation of the trend-line analysis when other variables enter the situation at the time of sales training.
An easily implemented method to isolate the effect of sales training is to obtain information directly from experts who understand the business performance measures.
The experts could be any number of individuals. For many sales training programs, the participants are the experts. After all, their performance is in question and the
measure is reflecting their individual performance. They may know more about the relationships between the different factors, including the impact of training, than any
other individual.
1. Participants should know in advance that they are expected to provide this type of data along with an explanation of why the information is needed and how it
will be used.
2. Ideally, participants should see a copy of this questionnaire and discuss it while they are involved in the program.
3. Participants should be reminded of the requirement prior to the time to collect data.
4. Participants should be provided with examples of how the questionnaire can be completed, using likely scenarios and types of data.
5. The immediate manager should guide participants through the process and review and approve the data, if necessary.
ROI is developed through the comparison of the monetary benefits of a program and the cost (or investment) in that program. It is an economic indicator, meaning that the
metric indicates the financial return on the investment. To develop this measure, impact data (Level 4 results) are converted to monetary value then compared to the
cost of the program. Before we describe the development of the ROI calculation, it is important to review the ways in which impacts are often described.
When it has been decided to convert a measure to monetary value and you’ve chosen the technique that you are going to use to calculate the monetary value, follow the five steps to complete the data conversion process.
So when do you use which? Many times both metrics are reported to give both perspectives. Because the BCR comes from the public sector, it is more often used in
public sector reporting. However, the ROI is gaining traction in those settings. For private sector organizations, the ROI is the primary metric.
As described earlier, intangible benefits are those benefits that are not converted to monetary value; but they are important and sometimes just as important as the actual
ROI calculation.
As described earlier, intangible benefits are those benefits that are not converted to monetary value; but they are important and sometimes just as important as the actual
ROI calculation.
One of the obstacles perhaps most difficult to overcome is receiving inadequate, insufficient, or disappointing news. Addressing a bad-news situation is an issue for most project leaders and other stakeholders involved in a project.
As described earlier, intangible benefits are those benefits that are not converted to monetary value; but they are important and sometimes just as important as the actual
ROI calculation.