Evaluating postgraduate CBT training in New Zealand. The study evaluated the transfer of training for graduates of a postgraduate CBT program from 2000-2009. Of the 88 graduates contacted, 11 participated by providing recorded therapy sessions that were rated using the Cognitive Therapy Scale. Seven of the 11 participants achieved scores above the competence threshold. However, recruitment of graduates was difficult due to fears of negative evaluation, difficulties obtaining recordings, and the expense of competency ratings. Improving supervision access, technological support, and disseminating feedback may help increase participation in future evaluations.

1. Evaluating postgraduate CBT training in New zealand dr Beverly Haarhoff Massey University, Auckland. Transfer of training: “Fade away or show and tell!”

2. Agenda Why is the evaluation of maintenance and transfer of training in psychotherapy important? Obstacles and difficulties in assessing transfer Brief look at current state of play in terms of research in the CBT field Our study (descriptive data) Problems A way forward

3. Defining transfer of training “Transfer of training is defined as the generalisation and maintenance of new information, knowledge, attitudes and skills into the everyday practice of trainees” Baldwin & Ford, 1988.

4. Transfer of training why measure? Management and allocation of resources Increasing access to evidenced based therapies (IAPT program) What works? For whom? To improve patient/client outcome

5. Transfer of training in Psychotherapy? *Difficult area to research Comparatively few studies (increasing in CBT) Lack of funding (the field falls between psychotherapy & education) Competing psychotherapy training programs and paradigms (profession specific, short-term, comprehensive, scholar-practitioner, scientist-practitioner, ‘apprenticeship model’, emphasis on personal therapy etc.)

6. Evaluating CBT transfer of training More complications! What represents competence? (which competecies? General or specific?) Trainee Self-report ? Supervisor ‘indirect’ report? Objective scoring of competence? Evaluation of patient outcome? The willingness of graduates to participate? Small participant samples (limited access programs)

7. Transfer of training in CBT Empirical studies have increased since 1999 Still heavily reliant on the self report of trainees The few studies using ‘objective’ observation methods such as the Cognitive Therapy Scale (Young & Beck, 1980) measure overall competence have have generally been positive. Very few linked to outcome or praticing CBT therapists

8. Recent studies McManus, Westbrook, Vazquez-Montes, Fennell & Kennerley, 2010. Oxford Diploma CBT Course 278 trainees between 1998-2009 Increased competence Clinical Psychologists achieved consistently better results Age negatively correlated with improved competency

9. Brosan, Reynolds & Moore, 2006 Naturalistic study Practitioners recruited from a range of professions mid t/m recorded CBT session rated using the CTS 5380 practitioners approached 47 responded & only 24 submitted tapes Psychologists found to be the most competent However a number of accredited therapist scored below the expected level of competence

10. The Massey University Postgraduate CBT diploma Minimum of two years Part-time Phase one: theoretical: four papers taught in block mode namely Theory and Practice of CBT, CBT for Depression, CBT for the Anxiety Disorders and CBT for chronic and complex problems. Phase two: Supervised clinical practicum over two semesters. x Two clients, two case studies and two verbal case presentations. (all supervision in-house) Final oral exam with the focus on case conceptualisation

11. The study Questions: ★Is CBT competence maintained and transferred beyond training? Is therapist self-reported competence in using CBT related to observed use of CBT? Is professional development as therapist related to self-reported competence and observed competence? Is work involvement positively related to self-reported and observed competence as a CBT therapist?

12. Measures *Cognitive Therapy Scale (Young & Beck, 1980, 1988) (the most widely used & validated instrument for assessing CBT competence McManus et al., 2010) Adapted Survey of PGDipCBT (Kennedy-Merrick et al., 2006) Therapist Professional Development questionnaire (Orlinsky & Rønnestad, 2005) Psychotherapist work involvement questionnaire (Orlinsky & Rønnestad, 2005)

13. The Cognitive Therapy Scale (Young & Beck, 1980, 1988) General therapeutic procedures & interpersonal effectiveness Agenda Feedback Understanding Interpersonal effectiveness Collaboration Pacing & efficient use of time ‘Red-line’ = 39-44 (RCT) Specific CBT skills Guided discovery Focusing on key cognitions and behaviours Strategy for change Application of CBT Techniques Homework Each item rated on a (0-6) Likert scale yielding a total score of 66

14. Participants The participants were drawn from two groups of CBT trainees Group one: trainees enrolled in the PGDipCBT during 2009 *Group two: Graduates from the PGDipCBT (2000-2009) Mental health practitioners across multiple professions: psychologists, social workers, nurses, psychiatric registrars and consultants, GP’s, Occupational therapists, psychotherapists & counsellors

15. Participants : Group Two Of the 88 graduates between 2000-2009, 42% (n=37) were not contactable Of the remaining 58% (n=51), 20 were currently not practicing CBT and 9 declined to participate. A total of 29 agreed to participate in the study but of this group only 12 provided recorded examples of their clinical work. Of this sample one tape was unusable and only 11 clinicians eventually took part (12.5% of the total sample)

16. Results: Cognitive Therapy Scale (Young & Beck,1980,1988)

17. Competent? Out of 11 participants n=7 (64%) achieved competence scoring above the 39 ‘red-line’ competence for RCT criteria 3 of 7 were clinical psychologists, 43%

18. Participants below the ‘red-line’ Poor performance in specific CBT competencies and skills

19. Difficulties: Participant recruitment

20. Difficulties Fear of negative evaluation Obtaining recorded clinical material (patient consent, graduate resistance, technological difficulties) Expensive (scoring of competence measures)

21. A way forward Providing a strong rationale for participation in competency evaluation Building in positive reinforcement for participation Building a culture of accountability Improving access to supervision Technological support & know how Deseminating feedback from research

22. Acknowledgements Robyn Gedye (PhD candidate) Dr Mei Williams Senior Lecturer Massey University Lynley Stenhouse (Clinical Psychologist) Sarah Kennedy-Merrick PhD Study participants Graduates of the Massey University Post Graduate Diploma in CBT 2000-2009.

23. questions