This presentation takes a detailed look at unmet needs in mental health services for children.

1. Building a Science on Implementation of Evidence-based Practices in Children’s Mental Health Kimberly Eaton Hoagwood, Ph.D. Columbia University December 11, 2005

4. Ringel &Sturm, 2000; NIMH, 2001 Unmet Need for Mental Health Services 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% White African- American Latino Other

5. National Averages of Use and Need Don’t Match National Average MH Need for Children at 6-17: 7.09% National Average MH Use for Children at 6-17: 7.45% Data Source: NSAF wave 1 and 2, Sturm, 2001

7. Beyond the Linear Model Basic Research Clinical Trial (Efficacy) Treatment Development Effectiveness Trial Treatment Deployment

8. Schoenwald & Hoagwood, 2001

9. The Rise in Popularity of the term “Evidence-Based” (Hoagwood & Johnson, 2003) EBT EBP EBM 1900-1990 0 0 0 1990-1995 3 7 76 1995-2002 63 459 5,425

10. Psychotherapies provided in routine clinical care have little to no effect (Weisz et al., 1995) Mean Effect Sizes Weisz et al., 1995 Children & Adolescents Adults University Clinic settings

29. New York State Implementation Model System & Policy Context Financial policies, methods of reimbursement, state policies Organizational Context Culture Climate Structure Clinical Care Improvement Training on EBP’s, supervision, consultation and support Engagement Empowerment Attitudes, Beliefs & Expectancies of Families & Youth Improved Child & Family Outcomes Attitudes, Beliefs & Expectancies of Clinicians and Supervisors Improved Implementation Efficiency & Effectiveness

32. Children and Youth participating in the CATS research project ID’s Generated 1068 Assigned 650 Declined 50 Ineligible 165 CATS 450 Conferenced-In 51 Comparison 149 Children 287 Adolescents 163 Children 108 Adolescents 41 Unassigned 204

33. Acknowledgements CATS Consortium The CATS Consortium is a cooperative multi site treatment study performed by nine independent teams in collaboration with the New York State Office of Mental Health. The New York State collaborators are Kimberly Eaton Hoagwood, Ph.D., Chip Felton, M.S.W., Sheila Donahue, Ph.D., Anita Appel, M.S.W., James Rodriguez, Ph.D., (NYSPI), Laura Murray, Ph.D., (NYSPI), David Fernandez, M.A., (NYSPI), Joanna Legerski, B.S. (NYSPI), Michelle Chung, B.A., Jacob Gisis, B.S., Jennifer Sawaya, B.A., Sudha Mehta, M.P.H. (OMH), Jessica Mass Levitt, Ph.D. (NYSPI). The Principal investigators and co-investigators from the nine sites are Robert Abramovitz, M.D., (JBFCS),), Reese Abright, M.D., (St. Vincents Hospital), Peter D’Amico, (North Shore/Long Island Jewish), Giussepe Constantino, Ph.D., (Lutheran Hospital), Carrie Epstein, C.S.W.-R., (Safe Horizon), Jennifer Havens, M.D., (Columbia University), Sandra Kaplan, M.D., (North Shore/LIJ), Jeffrey Newcorn, M.D., (Mt. Sinai), Moises Perez, Ph.D., (Alianza Dominicana), Raul Silva, M.D., (NYU/Bellevue), Heike Thiel de Bocanegra, Ph.D., (Safe Horizon),), Juliet Vogel, Ph.D. (North Shore/LIJ). The Scientific Advisors to the project are: Leonard Bickman, Ph.D., (Vanderbilt University), Peter S. Jensen, M.D., (Columbia University), Mary McKay, Ph.D., (Mount Sinai Medical School), Susan Essock, Ph.D., (Mount Sinai Medical School), Sue Marcus, Ph.D., (Mount Sinai Medical School), Wendy Silverman, Ph.D. (Florida International University), Robert Pynoos, M.D. (University of California, Los Angeles); Allan Steinberg, Ph.D. (University of California, Los Angeles); Lawrence Palinkas, Ph.D. (University of California at San Diego); and Joseph Cappelleri, Ph.D., (Pfizer Corporation). The Treatment Developers and Scientific Consultants to the project are: Judy Cohen, M.D., (University of Pittsburgh), Anthony Mannarino, Ph.D., (University of Pittsburgh), Christopher Layne, Ph.D., (Brigham Young University), William Saltzman, Ph.D ., (UCLA).

39. Treatment as Usual Show Rates McKay et al., 2005

41. First Interview Results

43. CATS Assessment vs. Treatment Show Rates

44. Assessment Rate by Site

45. Treatment Show Rate by Site

46. Treatment Settings

47. Average Number of Treatment Sessions for Treatment Completers

48. Treatment Sessions N=446

50. Autism Conduct Depression Oppositional Substance None None CBT Parent/Teacher Training CBT None Multisystemic Therapy CBT + parents; IPT; Relaxation Anger Coping; Assertiveness; PSST Behavior Tx; Family Tx ABA; FCT None None None None Play Therapy; GIST Juvenile Justice; Individual Tx Family Tx; Individual Tx Relaxation; Individual Tx Individual Therapy ADHD Behavior Therapy None None Biofeedback; Play Tx; GIST None None Group Therapy None Group Therapy Group Therapy Anxiety CBT; Exposure; Modeling CBT+ parents; Ed support None EMDR; Play Tx; GIST None Problem Level 1 best support Level 2 good support Level 3 some support Level 4 no support Level 5 known risks Example: Chorpita (2002) EBT Analysis

52. Level 2 CBT + parents Interpersonal Relaxation 88% 85% 100% MA; PhD MA; PhD; MD MA; PhD clinic clinic school CBT 94% MA; PhD Clinic; school 1.74 1.40 1.51 1.48 Level 1 Intervention Finish 14 to 18 12 to 18 11 to 18 9 to 18 Age Staff Setting Effect NS 49% PR; 41% HA; 10% C NS 84% NS; 18%PR; 3%AA Ethn 7 to 8 weeks 12 weeks 5 to 8 weeks 5 to 16 weeks Length Evidence: Interventions for Depression

53. Level 2 CBT + parents Interpersonal Relaxation 88% 85% 100% MA; PhD MA; PhD; MD MA; PhD clinic clinic school CBT 94% MA; PhD Clinic; school 1.74 1.40 1.51 1.48 Level 1 Intervention Finish 14 to 18 12 to 18 11 to 18 9 to 18 Age Staff Setting Effect NS 49% PR; 41% HA; 10% C NS 84% NS; 18%PR; 3%AA Ethn 7 to 8 weeks 12 weeks 5 to 8 weeks 5 to 16 weeks Length Evidence: Interventions for Depression

54. Level 2 CBT + parents Interpersonal Relaxation 88% 85% 100% MA; PhD MA; PhD; MD MA; PhD clinic clinic school CBT 94% MA; PhD Clinic; school 1.74 1.40 1.51 1.48 Level 1 Intervention Finish 14 to 18 12 to 18 11 to 18 9 to 18 Age Staff Setting Effect NS 49% PR; 41% HA; 10% C NS 84% NS; 18%PR; 3%AA Ethn 7 to 8 weeks 12 weeks 5 to 8 weeks 5 to 16 weeks Length Evidence: Interventions for Depression

56. Level 2 CBT + parents Edu support 93% 85% MA; PhD N/A clinic clinic CBT 95% UG; MA; PhD Clinic; school 1.05 1.78 N/A Level 1 Intervention Finish 14 to 18 6 to 17 2 to 17 Age Staff Setting Effect NS 92% C 54% NS; 33% C; 7% Arm; 6%AA Ethn 12 weeks 12 weeks 3 to 16 weeks Length Evidence: Interventions for Anxiety

57. Level 2 CBT + parents Edu support 93% 85% MA; PhD N/A clinic clinic CBT 95% UG; MA; PhD Clinic; school 1.05 1.78 N/A Level 1 Intervention Finish 7 to 12 6 to 17 2 to 17 Age Staff Setting Effect NS 92% C 54% NS; 33% C; 7% Arm; 6%AA Ethn 12 weeks 12 weeks 3 to 16 weeks Length Evidence: Interventions for Anxiety

59. Example (Chorpita et al., 2005) Internalizing Externalizing All Ext Int

60. Anxiety and Phobias (Chorpita et al) Depression All Dep A/P Ext Int

62. New York State Implementation Model System & Policy Context Financial policies, methods of reimbursement, state policies Organizational Context Culture Climate Structure Clinical Care Improvement Training on EBP’s, supervision, consultation and support Engagement Empowerment Attitudes, Beliefs & Expectancies of Families & Youth Improved Child & Family Outcomes Attitudes, Beliefs & Expectancies of Clinicians and Supervisors Improved Implementation Efficiency & Effectiveness

68. Parent Empowerment Research Study Basic Design 40 Parent Advocates/Family Support Specialists (PA/FSS) 20 PA/FSS PEP Training 20 PA/FSS Training as Usual 120 Parent/Caregivers Receiving PA/FS Services 6 per PA/FSS 120 Parent/Caregivers Receiving PA/FS Services 6 per PA/FSS

74. Glisson & Himmelgarn (1998) Parameter Estimates for Hypothesized Six-Variable Model Service Quality County Demographics Service Outcomes (problem levels) -.13* .12* -.05 -.24* -.03 .02 -.36* .01 .06 -.20* * p < .05 Organizational Climate Interorganizational Services Coordination Interorganizational Relationships

75. Organizational Context: Implications for the Transport of Evidence-Based Treatments To Mental Health Provider Organizations Sonja K. Schoenwald, Ph.D. Family Services Research Center Psychiatry & Behavioral Sciences Medical University of South Carolina

82. Therapist Effects Pre-Post Differences In CBCL Total Scores by Adherence

85. Adherence & Organization: Direct Effects on Outcomes

94. Culture Structure Psychological Climate Organizational Climate Attitudes Social Norms Self-Efficacy Beliefs & Expectations Behavioral Intention Models of Diffusion, Organizational Implementation & Social Processes Systems Context Organizational Properties Individual & Shared Perceptions Behavior Structural Determinants of Organizational Innovation Social Determinants of Organizational Innovation Adapted from Glisson 2002

99. Service System Organization Service Delivery Client Practitioner Intervention Schoenwald & Hoagwood, 2001 Organiza-tional mandates Ethnicity/ cultural iden Salary level/ Criteria for increases Clarity of intervention Organiza-tional mission Gender Endorsement of intervention Complexity of intervention Interagency working relationship Organiza-tional climate Source of payment Age and dev- elopmental status Training of practitioner Similarity of int to std practice Legal mandate for referrals Organiza- tional culture Physical location of sessions Source of referral Supervisor/ Researcher Intervention specification Manual? Financing methods Personnel policies Length of sessions Family context Adherence monitoring Focus of intervention Policies of referral source, pay Structure, hierarchy Frequency of sessions Nature of referral problems Specialized training Nature of intervention theory

100. Intervention Characteristics Theoretical foundation, strength of research support, clinical foundation, precision, availability of manual, specificity of manual, clarify of model Practitioner Characteristics Clinical adherence to model, frequency of clinical supervision, structure of clinical supervision, type of clinician, treatment orientation of clinician Client Characteristics Referral problem(s), family context, client’s ethnicity/cultural identification Service Delivery Characteristics Referral source, frequency of treatment sessions, length of treatment sessions, setting/location of treatment sessions, setting/location of the clinic or school Service System Characteristics Salary incentives to adopt EBPs, policies and practices of referral sources, source of payments for the specific EBP, financing/payment mechanisms, legal mandates of referral sources, strength of interagency relationships DOMAIN 1 DOMAIN 2 DOMAIN 3 DOMAIN 4 DOMAIN 5 DOMAIN 6 Key Readiness Factors Service Agency Characteristics Endorsement by site leadership, structure of organization, size of organization, culture and climate of organization, policies and practices within the organization

101. Dimensions of Organizational Readiness (DOOR) Question: how important are the following factors: Frequency of clinical supervision required to deliver the EBT (e.g., consultations between clinicians & supervisors) Length of each treatment session required to deliver the EBT Strength of the research supporting the EBT … outside agencies … state mental health authorities … consumer advocacy … families or youth in the service setting … clinical staff (e.g., therapists, social workers, psychologists, psychiatrists) Support for the EBT by …. Clinic Directors Clinicians Consumers Researchers To Me

102. Mean Ratings of Readiness Factors Scale

103. Total Mean Ratings of Readiness Factors

104. Comparison of Respondent Group Ratings Scale