Training in Implementation: Actionable Research Approaches (TIARA): Module 2, Theory & Approaches

1. Implementation Science and
Practice
Theory and Approaches
June 2019
Anne Sales
salesann@umich.edu

2. Outline for today
• Brief recap of last session
• Example: Implementing goals of care conversations in long term care settings
• Introduction to frameworks in implementation research and practice
• Types and uses of frameworks
• Digging deeper
• Process: KTA, our stepped framework
• Determinants: TICD
• Evaluation: RE-AIM (more in Session 3)
• Using a determinants framework– TICD
• Assessing positive and negative determinants (facilitators and barriers)– preparatory phase
• Analyzing determinants– preparatory/planning
• Designing implementation interventions
• What do you think might work and why?
• Example of a designed implementation intervention
• Practical considerations

3. Quick reminder about how we define
implementation and its science

4. Activities at each step
Gap or variation
analysis
Literature review
Determinants
assessment using
framework
Map strategies and
behavior change
techniques to
determinants
Assemble elements into a coherent
implementation intervention
Use appropriate levels of
analysis and adjust for
clustering as appropriate;
typically requires mixed
methods

5. Implementing goals of care
conversations in long term care
settings
An example of implementation research in action

6. Following our process framework
• What is the gap?
• Patients/residents* are admitted
to long term care settings without
clear, standardized approaches to
holding conversations about what
they want to get out of their care
• We don’t know how many VA LTC
residents have had such
conversations
• Decisions about life sustaining
treatments are based on a variety
of inputs
• Resident wishes are often not known
*In LTC settings, residents = patients

7. The LSTDI recommendation
• All Veterans receiving care in Veterans Health Administration
facilities who are seriously ill or have a high probability of a life
threatening illness in the near future should have a goals of care
conversation documented using the LSTDI progress note template
• VHA Handbook 1004.03 covers many different situations (“triggering
events”) that should result in a Goals of Care Conversation
• Available at https://www.ethics.va.gov/LST.asp
• Note that a new standardized template for documenting the
conversations is an important component of the initiative
• Innovations
• Standardized mandated conversations at specific time points or under
certain circumstances
• Documentation using a new template
• Persistent orders

8. The solution
• National program
• Life Sustaining Treatment Decision
initiative (LSTDI)
• https://www.ethics.va.gov/LST.asp
• Extensive training and support
• Electronic health record template
• Standardize documentation of
conversations and decisions
• Monitoring progress
• Online dashboard available for
continuous monitoring

9. Steps in supporting implementation
• Map process of admitting new
residents to LTC setting
• Why focus on admission process?
• Assess possible areas for behavior
change
• Assess who needs to change what
behavior when
• The TACT principle
• Target
• Action
• Context
• Time
• Doing what (A)
• To whom (T)
• In what context (C)
• When (T)
• https://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-7-207

10. Decision to admit
Veteran to CLC unit
Initial
conversation
about goals of
care
Current LST
template
completed
MDS
assessment
MDS complete
MDS data
transmitted to
databases
Care planning
Change in
status
Reassessment:
goals of care + MDSAttending
MD
MD/
NP/P
A
MD/
NP/PA
Interdisciplinary team–
MDS RN lead
MD/NP/PA
Timeframe? 2 weeks after admission
Timeframe?
MDS = Minimum Data Set, a required periodic
assessment for all residents in LTC settings

11. Decision to admit
Veteran to CLC
unit
Initial
conversation
about goals
of care
CWAD
template
completed
MDS
assessment
MDS
complete
MDS data
transmitted
to Austin
Care
planning
Change in
status
Reassessment:
goals of care +
MDSAttending
MD
MD/
NP
MD/
NP
Interdisciplinary team–
MDS RN lead
MD/NP
Timeframe? 2 weeks after admission
Timeframe?
ReassessingDocumentingDecidingDiscussingAssessing

12. Behaviors
• Assessing
• Status
• Preferences
• Goals
• Discussing
• With resident
• With staff
• With peers
• Decision making
• What to write/box to check
• When to reassess
• Documenting
• Writing down in appropriate place
• Reassessing

13. Next steps?

14. Learning Health Sciences
Frameworks and models provide important
guidance for doing implementation
• Process frameworks
• Describing
• Guiding the process
• Determinant frameworks
• Understanding
• Explaining what
influences
implementation
• Evaluation frameworks
• How well the process
worked
Nilsen, Implementation Science 2015
http://www.implementationscience.com/content/10/1/53
These are an important
codification of knowledge in
implementation practice and
research

15. Learning Health Sciences
Process frameworks
• Describe or guide
process of
implementation
• Often steps in a cycle

16. Learning Health Sciences
Evaluation frameworks
• Specify aspects of
implementation for
evaluation
• How do you measure
the success of
implementation?

17. Learning Health Sciences
Determinants frameworks
• Specify
types/classes/domains
of factors that
influence the success
of implementation
• Damschroder LJ et al. 2009
https://implementationscience.biomedc
entral.com/articles/10.1186/1748-5908-
4-50 The Consolidated Framework for
Implementation Research (CFIR)
• Francis et al. 2012
https://implementationscience.biomedc
entral.com/articles/10.1186/1748-5908-
7-35 The Theoretical Domains
Framework (TDF)
• Flottorp et al. 2013
https://implementationscience.biomedc
entral.com/articles/10.1186/1748-5908-
8-35 : The Tailored Implementation in
Chronic Disease (TICD) checklist

18. How to use these three types
of frameworks
• Process
• Planning your work
• Evaluation
• Did your efforts work?
• Determinants
• Figure out what you need to do
• Design one or more implementation interventions
Page #

19. Focusing on determinants
• Step 3
• Assess barriers and facilitators to implementing
your innovation/evidence based
practice/policy/program
• Why?

20. Reasons to assess barriers and
facilitators (determinants of
implementation success)
• To overcome them
• Planning
• Awareness
• To study them
• We have plenty of catalogs
• Listing them is not necessarily all that helpful
• Determinants frameworks are catalogs of barriers
and facilitators
• Organized into some kind of taxonomy or system for
classifying them

21. So how do you use these?
• Assessing barriers and facilitators
• Conduct interviews guided by your selected
framework
• Analyze interviews using the same (possibly
additional) framework(s)
• Evaluate the types of barriers and/or facilitators that
need to be addressed
• Prioritize– which are most important?
• Assess feasibility– which can be overcome, and how?
• If feasible, link to implementation strategies and/or
behavior change techniques

22. Conducting interviews
• Conducting interviews using the
TICD (or any determinants
framework)
• Construct an interview guide
based on the framework
• Flottorp et al. 2013 paper has some
guidance
• We have developed a fairly flexible
interview guide that can be used for
different implementation projects

23. What do you get from interviews?
• Quote:
• “I think we all agree that it’s a
good thing to discuss goals of
care with people who are
admitted to our nursing home.
I’m not sure whether we have to
do it on admission, or if we have
to wait until we have a good
relationship with the Veteran.
Some Veterans are reluctant to
talk about end of life initially–
it’s not part of their culture.”
• Quote:
• “I personally believe that this is a
really important thing to do, and
I’m really comfortable talking
with Veterans about end of life
and their own situations.
However, my colleagues aren’t
all so comfortable with it. Some
of them seem to think that it will
make the Veteran deeply
unhappy and possibly
considering suicide.”

24. Template analysis

25. General qualitative analysis
• Goal is to produce a codebook and themes
• Open coding
• Constant comparison
• Is it like something else?
• Is it different?
• What is the underlying theme?
• Themes and possibly a framework emerges from the
analysis

26. Rapid template analysis
• Framework already exists
• CFIR
• TDF
• TICD
• Other/combination
• Interview ideally has been developed based on
the framework(s)
• Analysis starts with coding into existing constructs
or categories within the framework(s)

27. Coding into TICD
• Quote from interview:
• “I think we all agree that it’s a
good thing to discuss goals of care
with people who are admitted to
our nursing home. I’m not sure
whether we have to do it on
admission, or if we have to wait
until we have a good relationship
with the Veteran. Some Veterans
are reluctant to talk about end of
life initially– it’s not part of their
culture.”

28. Questions
• What do you do when you find a statement that
doesn’t fit?
• What about when a statement fits into more than
one category?
• What do you do after you finish coding?

29. Coding into a determinants
framework
• After you finish coding all interviews or other
qualitative information
• Review which constructs have information in them
• Review the domains for these constructs
• Which of these are most important?
• Criteria for importance
• What barriers have to be overcome?
• Which will matter the most for implementing the practice you
intend to implement?
• What is feasible?
• What do key stakeholders think is most important and/or
feasible?

30. After coding and review
• Begin to assess what kinds of strategies and/or
behavior change techniques can be used to
overcome barriers
• Build one or more implementation interventions
using these strategies and behavior change
techniques

31. Learning Health Sciences
Digging into the first two domains
(Domain/Construct/Concept)
• Guideline/Innovation Factors
• Recommendation
• Quality of evidence
• Strength of recommendation
• Clarity
• Cultural appropriateness
• Accessibility
• Source
• Consistency with other guidelines
• Recommended clinical intervention
• Feasibility
• Accessibility
• Recommended behavior
• Compatibility
• Effort
• Trialability
• Observability
• Individual Health Professional Factors
• Knowledge and skills
• Domain knowledge
• Awareness and familiarity with recommendation
• Knowledge about own practice
• Skills needed to adhere
• Cognitions
• Agreement with recommendations
• Attitudes towards guidelines in general
• Expected outcome
• Intention and motivation
• Self-efficacy
• Learning style
• Emotions
• Professional behavior
• Nature of the behavior
• Capacity to plan change
• Self-monitoring or feedback

32. The LSTDI recommendation
• All Veterans receiving care in Veterans Health Administration facilities
who are seriously ill or have a high probability of a life threatening illness
in the near future should have a goals of care conversation documented
using the LSTDI progress note template
• VHA Handbook 1004.03 covers many different situations (“triggering
events”) that should result in a Goals of Care Conversation
• Available at https://www.ethics.va.gov/LST.asp
• Note that a new standardized template for documenting the
conversations is an important component of the initiative
• Innovations
• Standardized mandated conversations at specific time points or under certain
circumstances
• Documentation using a new template
• Persistent orders

33. Analyzing the recommendation
• All Veterans receiving care
in Veterans Health
Administration facilities
who are seriously ill or
have a high probability of a
life threatening illness in
the near future should
have a goals of care
conversation documented
using the LSTDI progress
note template
Recommendation High Low
Quality of evidence
Strength of
recommendation
Clarity
Cultural
appropriateness
Accessibility of the
recommendation
Source of the
recommendation
Consistency with other
guidelines

34. • All Veterans receiving care
in Veterans Health
Administration facilities
who are seriously ill or
have a high probability of a
life threatening illness in
the near future should
have a goals of care
conversation documented
using the LSTDI progress
note template
Recommended
clinical
intervention High Low
Feasibility
Accessibility of the
intervention
Recommended
behavior High Low
Compatibility
Effort
Trialability
Observability

35. • All Veterans receiving care
in Veterans Health
Administration facilities
who are seriously ill or
have a high probability of a
life threatening illness in
the near future should
have a goals of care
conversation documented
using the LSTDI progress
note template
Knowledge and
skills High Low
Domain knowledge
Awareness and
familiarity with the
recommendation
Knowledge about
own practice
Skills needed to
adhere

36. Cognitions (including
attitudes) High Low
Agreement with
recommendations
Attitudes towards
guidelines in general
Expected outcome
Intention and
motivation
Self-efficacy
Learning style
Emotions
Professional
behavior High Low
Nature of the
behavior
Capacity to
plan change
Self-monitoring
or feedback

37. This gets us through the first two
domains
Five more to go: Patient Factors; Professional Interactions; Incentives and
Resources; Capacity for Organizational Change; Social, Political and Legal Factors

38. Back to the definition
• “under organizational
constraints”
• Measuring these
• Can use interviews
• Might want to consider surveys

39. Using TICD
• Going beyond the
evidence and the
perceptions of
individuals and
teamwork
• Factors under the
control of the
organization
• Questions in the
interviews

40. More questions
• Focus on
professional
interactions,
incentives and
resources, capacity
for change
• Leadership issues
come into play in
these areas

41. Other ways of getting
data
• Surveys
• Organizational Readiness to
Change Assessment (ORCA)
• Only partially what its name
implies
• Weiner’s theory of organizational
readiness to change
• Some organizational readiness to
change
• Other assessments
• Evidence
• Context
• Facilitation

42. Now what?
• We know a lot about
determinants
• Things that will help us get the
innovation implemented
• Things that are likely to get in the
way of implementation
• We’ve interviewed, analyzed,
surveyed and coded
• How do we get from all of this to
doing implementation?

43. Designing implementation interventions
• Implementation strategies
• Powell et al. 2015
• Proctor et al. 2013
• Other taxonomies
• EPOC 2016
• Lists of different strategies used
to do implementation
• Mapping is not trivial
• Behavior change techniques
• 2013 paper by Susan Michie and
colleagues
• Catalogs and creates taxonomy of
micro-strategies
• Behavior change techniques (BCTs)
• Things that have evidence that
they can support changing
behavior

44. Using LSTDI example
• Quote:
• “I think we all support the idea
of having goals of care
conversations with Veterans in
our nursing home. But as the
manager of this unit, I don’t
know how many Veterans have
had goals of care conversations
and documentation. It makes it
hard to know how much I need
to push people.”
• Problem/coded factor from
TICD:
• Individual health professional
factors
• Lack of knowledge about own
practice
• Also
• Capacity for organizational change
• Monitoring and feedback

45. From Powell et al.
2015
• Implementation strategy
• Audit and provide feedback
• Feedback intervention
• Possibly
• Add components of behavior
change techniques
• Role modeling
• Brief description of how to conduct a
goals of care conversation
• Video link

46. Pragmatic advice
• Why is feedback important?
• Important for learning
• Without feedback, it’s very difficult for
us to learn effectively
• Understand our performance on key
metrics
• But we get a lot of poorly designed
(or completely un-designed)
feedback
• Organizational dashboards
• Verbal feedback without much
concrete information
• Feedback without action planning
is not likely to be effective

47. But not all strategies are feasible
• Feedback feasibility
• Need existing data streams
• Need ability to rapidly analyze
data and create feedback reports
• Issues to consider:
• Cost
• Acceptability
• Other pragmatic issues
• “Change payment incentives” is
a strategy that may not be
feasible except in the very long
run

48. Summarizing for today
• There are multiple sub-steps between Step 2
and Step 3
• A. Process mapping
• B. Analyzing the innovation
• C. An initial assessment of possible areas where
there may be strong barriers and/or facilitators
• Valence of the determinants
• How strong are they?
• How likely are they?
• People with content expertise can help with this analysis
• D. Collect data to check out your hypotheses about
what will and won’t work
• Use your data to design implementation interventions