Trying to improve business results via training? One-hit wonders will sabotage your efforts. Use research-proven strategies to yield long-term retention of your training.

1. when
remembering
REALLY
matters
Presented by
Sharon Boller,
President & Chief Product Officer

2. WHO AM I?
Game-lover, learner, instructional designer,
product owner, game designer, dog-lover and
owner, Mom, wife, cyclist.
Oh…and president and chief product officer,
Bottom-Line Performance.

3. OUR GAME PLAN
1 The requirements to learn something
The challenge of remembering… and the cost of
2 forgetting
3 The research on remembering…and forgetting
4 Strategies for remembering…and for learning
5 Two case studies that use the strategies

4. What’s Required to LEARN?
Motivation Relevant Practice Specific, timely feedback
Ability to retrieve later
Right?????

5. MEMORY TEST #1
1 x 12 = ____
2 x 12 = ____
3 x 12 = ____
4 x 12 = ____
5 x 12 = ____
6 x 12 = ____
7 x 12 = ____
8 x 12 = ____
9 x 12 = ____
10 x 12 =____
11 x 12 =____
12 x 12 =____
Times Tables
Raise your hand if you feel VERY
CONFIDENT that you can answer this set
of multiplication problems without
pausing.

6. MEMORY TEST #2
Spelling Words
Write down 5 words as I say them…don’t
think hard. Just write the words.
Receive
Believe
Experience
Neighbor
Friend

7. MEMORY TEST #3
1. What year did the Civil
War begin?
2. Where did Robert E.
Lee surrender, ending
the Civil War?
3. When did Abraham
Lincoln deliver the
Gettysburg address?
History
Three American history questions; write
your answers to each one. Do NOT use
your cell phone, iPad, or any other device
to answer. Use your brain only.

8. So…why did
you
remember
some of this
stuff, but
not other
stuff?

9. Some stuff you
need to know
“cold”
Other stuff you
can “find/
locate”

10. Does
Memory
HERE
?
Matter
You are a customer support
engineer for a medical device
company. You train lab technicians
how to operate and troubleshoot
the $250,000 piece of machinery
their hospital purchased from your
company.
Will the hospital be okay if its lab techs have to find/
locate info on using the equipment – or does the
hospital expect lab techs to know common procedures
“cold?”

11. Does
Memory
HERE
?
Matter
You are a rep in a sales meeting. Your
customer just shared her needs, and she
wants a response. You need to mentally
think through your company’s eight
product lines, select the RIGHT ONE, and
then share the appropriate product
benefits and features. She’s waiting….
Can you Google that? How long will she wait while you
find/locate info? Or…does she expect you to formulate
a fairly immediate response based on what you
ALREADY KNOW AND CAN RECALL?

12. Does
Memory
HERE
?
Matter
You are a “skip tracer.” Your job is to
recover cars if their owners have
defaulted on a loan. There’s a lot of
defaults, and the skip tracing
process contains numerous steps!
How efficient will you be at doing your job if you are
constantly finding/locating? What’s okay to look up…
and what do you simply need to know and know how
to do from memory?
What assets go unrecovered if you are slow at the job?

13. Does
Memory
HERE
?
Matter
You are a lab technician. You just
dropped a vial of blood being tested
for HIV. It just went SPLAT!
You KNOW the blood poses a risk.
You take annual training for this…but
that was months ago. You recall
some steps, but you are not sure of
all of them.
With blood splattered can you look up the procedure or
do you need to be able to recall it and apply it to the
situation at hand?

14. PEOPLE NEED TO
REMEMBER A
LOT OF STUFF IN
THE WORKPLACE Product Knowledge
Industry Facts
Policies & Procedures
Research Data
Sales Messages
What else do people need to
“know cold” (i.e. from memory?)

15. Per ATD, we’re spending about $164 BILLION
on training per year. Are we getting results?
Time saved…
or not wasted
in the first
place?
Money
saved…or
money
gained?
People producing
more, faster,
more accurately
or feeling less
stressed?

16. WHAT’S REMEMBERED?
During training? A week later?

17. How much forgetting?
30% in 3 to 6 days’ time
50% in 3 to 6 days’ time
70% in 3 to 6 days’ time
90% (or more) in 3 to 6 days’ time
A
B
C
D

18. IT DEPENDS

19. Let’s be REALLY
generous and
assume 30%
forgetting.

20. Which 30% is the
business okay
with?

21. REMEMBERING IS
HARD
FORGETTING IS
EASY

22. FOUR STRATEGIES TO USE
Spaced intervals – 1 not a single “glop”
2 Repetition – several instances of it
3 right after making mistake
4 Stories
Feedback – with requirement to do it

23. Spaced Intervals &
Repetition
Micro Spacing Macro Spacing
Single Session Single Session Single Session
or
Day 1
Day 8
Day 16

24. Too Much
Nothing

25. SPACING RESEARCH
Spaced Repetition in the New York Times:
http://www.theknowledgeguru.com/spaced-repetition-featured-new-york-times/
Learning
Research
by
Annie
Murphy
Paul:
Distributed
Prac9ce
and
Spaced
Repe99on
h<p://www.theknowledgeguru.com/learning-­‐research-­‐annie-­‐murphy-­‐paul-­‐distributed-­‐prac9ce-­‐
repe99on/
Spaced Repetition: What Research tells us by Dr. Karl Kapp:
http://www.theknowledgeguru.com/spaced-retrieval-retrieval-practice-knowledge-
guru-research-tells-us/
Learning
vs.
Retrieval
UCLA
Bjork
Learning
and
ForgeIng
Lab:
h<p://bjorklab.psych.ucla.edu/research.html

26. FEEDBACK LOOPS

27. STORIES
Light up our
whole brain
Elicit
POWERFUL
Empathic
response
Personalize.
Contextualize.

28. when remembering
REALLY
matters Things I have
learned in this
session::
1.
What

29. Case Study #1 – ExactTarget Product
Launch (2012 case)
1. SaaS:
Mul9ple
Product
Lines
and
Mul9ple
Product
Launches
• 9
dis9nct
product
lines
within
organiza9on
• Product
line
releases
each
month
2. Employees,
clients,
and
partners
had
training
overload;
company
needed
a
way
to
“mix
it
up.”
3. MobileConnect
was
one
of
the
largest
product
launches
in
company
history.
Success
was
cri/cal.

30. LEARN + REMEMBER
Motivation Relevant Practice
Specific, timely feedback
Ability to retrieve later
Spaced Intervals, repetition Story

31. MOTIVATION
Intrinsic is about person
realizing learning will
help them in their jobs.
Extrinsic can couple with
– but not replace –
intrinsic. In this
instances, motivation
was prizes AND
recognition.

32. MOTIVATION

33. MOTIVATION
Meet
Paul
from
Australia
Earned
9tle
as
THE
“Mobile
Connect
Guru”
via
game
play.
“I
have
never
felt
more
valued
in
company
that
I
have
since
becoming
the
Mobile
Connect
Guru”

34. STORY as challenge

35. STORY - context

36. STORY - context

37. Spaced Iterations, Relevant Practice,
Feedback + REPETITION

38. Repetition – Interation 1

39. Repetition – Interation 2

40. Repetition – Interation 3

41. Feedback + Re-do

42. MORE Spacing, Repetition

43. Business Results
Quicker pipeline built: “Of all the launches done in the
two years previous to the MobileConnect launch, the sales
team built one of the quickest pipelines for this product.”
Dramatic increase in first-call support resolution.
Greatly increased contract values.

44. Case
Study #2
Dialysis
Treat-ment
Business Problems:
1. Patient post-training drop rate very high.
2. Time req’d to train a single patient longer than desired.
3. Only 1 patient per month could be trained.
As a result, less marketshare and growth in marketshare
than desired.
Training Goal:
“Patients can safely and confidently
perform ongoing treatments at home.”

45. They had GLOP

46. We created spacing

47. Multiple repetitions

48. Multiple Repetitions

49. Practice, Feedback

50. Stories for empathy
Characters in the eLearning told stories about
their own situations; these mirrored the
realities of the patients going through training.
“One of biggest hurdles I had to face was learning
to insert my own needles. I don’t care how brave
you are, having to stick two needles in my arm
every time I did a treatment was scary.”
“I was worried about hurting David when I had to
stick him with needles…I was not confident I could
do it.”

51. RESULTS?
• Yes! 6% reduction in “patient drops” in patients who
received NxSteps training over those who were trained via
another method. (Training for a single patient can typically
cost $30K or more to do so 6% is a big deal.)
• A major NxStage customer who used NxSteps program
viewed it as so successful in improving bottom-line $
results for company that it is working to replicate program
on peritoneal side of business.

52. Steps
You
Can
Take
In
Design
Ask:
“What business problem is this
learning solution solving?”
To solve the problem, consider
what is ESSENTIAL for people
to know, do, believe.
Distinguish between need to
be able to “find/locate” from
need to “know it cold”
1)

53. Steps
You
Can
Take
In
Design
If SMEs & stakeholders
insist on too much
content, ask:
“What 30% - 90% are you okay
with people forgetting…or
never learning in the first
place?”
2)

54. Steps
You
Can
Take
In
Design
Think beyond “one-hit
wonder.”
If remembering REALLY matters,
then spacing and repetition HAVE
to be part of your design.
3)
Spaced Intervals, repetition

55. 4)
DESIGN FOR: LEARN + REMEMBER
Motivation Relevant Practice
Specific, timely feedback
Ability to retrieve later
Spaced Intervals, repetition Story

56. Too Much
Nothing
5) Remember…

57. Want to
find
me?
Email:
sharon@
bottomlineperformance.com
Twitter: @Sharon_Boller