1. Making the CYP-IAPT Feedback and
Outcomes Monitoring Clinically Useful:
A Workshop
Duncan Law
5th Nov 2013
CYP-IAPT National Conference
Hertfordshire Partnership
University NHS Foundation Trust
© Duncan Law

2. Feelings about routine feedback & outcomes monitoring?
Hate them
0
Mixed feelings
5
© Duncan Law
Love them
10

3. Values behind outcomes and feedback
1. Demonstrate/celebrate/prove
effectiveness of interventions and service
1. Enhance collaborative practice:
– Better service user voice in therapy
– Facilitates good clinical practice
© Duncan Law

4. Feedback & Outcomes
Monitoring
Demonstrating success
© Duncan Law

5. Demonstrating effectiveness
• 90% target
– at least two time points with repeated normed
measure
– Plus, at lest one report of Education
Employment and Training (EET)
– http://www.iapt.nhs.uk/silo/files/90.pdf
© Duncan Law

6. It’s not (just) about measurement,
it’s about a culture of collaboration
©Tom Frost
© Duncan Law

7. Feedback & Outcomes
Monitoring
Young peoples views
© Duncan Law

8. Video
• http://www.youtube.co First, a word from the
m/watch?v=SOayZgia people who matter
M7U
© Duncan Law

9. Young peoples views:
“Knowing when and how to use
the outcome measures is
important. They must become an
integrated part of the session - not
something that is kept as an
abstract form-filling exercise.”
“Outcomes tools can be really
useful, not only for the therapists
but for the young people. They
allow you to evaluate where
things are going well and where
they could be improved.
Feedback is important for us to
learn and grow. However, I do
think you must use outcomes
tools with the correct intention and
in the right way.”
© Duncan Law

10. Young peoples views:
“Whenever I was doing a
questionnaire, it felt like a tick box
exercise. I didn’t know why I was
doing it. In fact no one seemed to
understand the point of the
questionnaire, it was just a
standard process. Eventually I
stopped thinking what I wrote on it
as it just seemed pointless.”
“Young people never get any
feedback about what the
information they have given has
been used for or how it has been
implemented in services. This
makes it seem like the
questionnaires are just so that
they can tick a box to say they
have used outcome tools without
actually using the information
given.”
© Duncan Law

11. Feedback & Outcomes
Monitoring
The evidence base
© Duncan Law

12. Mike Lambert
•
•
•
•
Feedback to clinicians on outcomes trajectories
Reduced drop-out
Better outcomes
No advice given to clinicians on how to use the feedback
•
Lambert, M. J. (2007). PRESIDENTIAL ADDRESS: A program of research aimed at improving
psychotherapy outcome in routine care: What we have learned from a decade of research.
Psychotherapy Research, 17, 1-14.
© Duncan Law

13. Len Bickman
•
•
•
•
Big RCT (28 sites in 10 states) ‘real world CAMHS’
Feedback weekly or every three months
Faster improvement with weekly feedback
Even better if clinicians looked at the feedback! (paraphrased)
•
Bickman, L., Kelley, S., Breda, C., De Andrade, A, & Riemer, M. (2011): Effects of routine
feedback to clinicians on youth mental health outcomes: A randomized cluster design, Psychiatric
Services, 62(12), p.1423-1429
© Duncan Law

14. Feedback & Outcomes
Monitoring
Making them clinically useful
© Duncan Law

15. 6 useful questions that forms can help with
Assessment/Choice
Partnership/ongoing work
Review & Close
• “What’s the problem?”
• “What do you want to change?”
• “How are we getting on together?”
• “How are things going?”
• “Have we done as much as we need to?”
• “How has this been generally?”

16. Feedback & Outcomes
Monitoring
Clinicians views
© Duncan Law

17. clinicians talking about their experience
http://www.youtube.com/watch?v=5srUEXW
wMKU
© Duncan Law

18. How it can feel to try out new ideas and approaches
Apprehension
Integrated in
practice
Feeling Clunky
Learning taken from
the Closing the Gap
Project
Neus Abrines
Jasmine Hoffman

19. Let’s give it a go!
© Duncan Law

20. In pairs, one clinician, one young person
Clinicians: try explain to the young person:
– the importance of completing a measure
– to complete it as openly and honestly as possible
Young people: during the role play, consider what the
clinician said or did that made you:
– Appreciate the importance of completing the measure
– want to fill the measure in openly and honestly
– DO NOT share this detail with your partner yet
© Duncan Law

21. How did that feel?
Young people:
• What did your partner do that made you feel that
completing the measure was:
– Important
– Worthwhile
– Useful
– Helpful
• What did your partner do to convince you to complete
the measure as honestly as possible?
© Duncan Law

22. How did that feel?
Clinicians:
• Did you feel confident in what you were saying to the
young person?
• What techniques did you try to use to convince them?
• What would have helped you to feel more confident?
© Duncan Law

23. Feedback & Outcomes
Monitoring
Making them clinically useful
© Duncan Law

24. Forms, forms and more forms!
Let’s brainstorm: what are all these forms for?
© Duncan Law

25. Barriers to using measures
(with thanks to UPROMISE)

26. Balancing the seesaw of service and therapeutic
need
(with thanks to UPROMISE)
SERVICE EVALUATION
CLINICAL MEANINGFULNESS
SDQ
RCADS
CGAS
Goals
Symptom rating scales
Session rating scales
CHI-ESQ

27. 6 useful questions that forms can help with
Assessment/Choice
Partnership/ongoing work
Review & Close
• “What’s the problem?”
• “What do you want to change?”
• “How are we getting on together?”
• “How are things going?”
• “Have we done as much as we need to?”
• “How has this been generally?”

28. At Assessment/ Choice
“What’s the problem?”
• SDQ
and/or
• RCADS
“What do you want to
change?”
• Goals (GBO)
and
• ORS/CORS
© Duncan Law

29. Session-by-session
“How are things going?”
• Symptom Tracker (in collaboration with child/family)
or
• Impact Tracker “how are you doing” or ORS/CORS
and
• Goals Based Outcomes (GBOs), or ORS/CORS
© Duncan Law

30. An Example of
a Symptom
Tracker
Worries
(GAD)
© Duncan Law 2012
Hertfordshire Partnership
NHS Foundation Trust

31. Goals Based
Outcome Tool:
What does it look
like?
Hertfordshire Partnership
NHS Foundation Trust

32. Session-by-Session
“How are we getting on together?”
• SRS
or
• Session Feedback Questionnaire
© Duncan Law

33. Introducing the RCADS: 2 ways
• In the following video, a clinician will introduce
the RCADS to a young person and ask her to
complete it in two different ways:
http://vimeo.com/57925678
While watching, consider
• What are the pros and cons of asking a young
person to complete measures in these ways?
• What might you do differently?
• What questions might the young person have?

34. Interpreting RCADS Scores
• What do you
make of this?
• What might you
want to discuss
in supervision?
•
What do you want to
know?
•
What questions would
you ask?

35. Feeding back scores to clients
• The following video clip shows a therapist
discussing scores on a measure with a young
person: http://vimeo.com/57925681
• Do you think this was done in a collaborative
manner?
• How could it have been more collaborative?

36. What’s going on here?
© Duncan Law

37. 16
Subscale Score
14
12
10
8
6
4
2
0
Session 1
Session 2
Session 3
Session 4
Session
© Duncan Law
Session 5
Session 6

38. The stats bit!
Clinical Cutoffs
A score below (or
sometimes above) which a
person is unlikely to have a
problem measured by a tool
Reliable Change
A change in scores is reliable
if it is larger than that
expected by measurement
error
With thanks to Andy Fugard
© Duncan Law

39. What’s going on here?
© Duncan Law

40. Depression symptom tracker - 17 year old boy
Reliable Change = 6
Clinical cutoff = 12
© Duncan Law

41. Service User Feedback &
Outcomes Monitoring
Resources and Further Information
© Duncan Law, Stephen Butler, & Alex Goforth

42. CYP-IAPT Service user feedback and outcomes framework
© Duncan Law

43. CYP-IAPT Resources
http://www.iapt.nhs.uk/silo/files/apractical-guide-to-using-serviceuser-feedback--outcome-tools.pdf
© Duncan Law
http://www.iapt.nhs.uk/cy
p-iapt/routine-outcomemonitoring-as-part-ofiapt/

44. Other Resources
• www.corc.uk.net
• www.myapt.org.uk
• www.youtube.com/channel/UCVr_XGnjA229P5b
UgfZ-y3w
• www.vimeo.com/tag:cyp+iapt
• Information on using feedback and outcomes
including training videos, information
© Duncan Law

45. Feelings about routine feedback & outcomes monitoring?
Hate them
0
Mixed feelings
5
© Duncan Law
Love them
10

46. © Duncan Law

47. Thanks for coming along
We hope you found this presentation useful
Feel free to use all are parts of it to help
train other staff in your organisation & to
further implement CYP-IAPT
If you do use it please reference: ‘Duncan Law’
These slides must not be used in training for commercial gain
© Duncan Law