Presentation given by Geoffrey Nelson and Tim Aubry, CAN at the Ninth European Research Conference on Homelessness, "Homelessness in Times of Crisis", Warsaw, September 2014 http://feantsaresearch.org/spip.php?article222&lang=en

1. Implementation Evaluation of Canada’s At Home/Chez Soi Housing First Program
Geoffrey Nelson, National Research Team, At Home/Chez Soi Project, Dept. of Psychology, Wilfrid Laurier University, Waterloo, Ontario, Canada Tim Aubry, National Research Team, At Home/Chez Soi Project, School of Psychology, University of Ottawa, Ottawa, Ontario, Canada 9th European Research Conference – Homelessness in Times of Crisis, FEANTSA, Warsaw, Friday, September 19, 2014

2. / 2
Overview of Presentation
• Background and Description of the At Home/Chez Soi Study
• Fidelity Findings
• Qualitative Implementation Findings

3. / 3
At Home/Chez Soi Project Description
In 2008 the Federal government allocated $110 million
At Home/Chez Soi is:
The largest study of its kind in the world (>2100 participants)
Pathways Housing First model
Implemented in 5 Canadian cities: Vancouver, Winnipeg, Toronto, Montreal, Moncton
Funding – 85% services / 15% research
A randomized controlled trial

4. / 4
Housing First
•Housing is immediate and permanent
•Recovery & choice emphasized
•Housing is primarily scattered site, private market
•Supports include guaranteed rent subsidies – clients pay 30%
•Additional services (>1 visit) are voluntary
•No readiness/maintenance conditions
•Tenancy rights
•Draws upon Pathways to Housing program model
•Two service delivery approaches (ACT and ICM) for two different populations (High and Moderate Need)

5. / 5

6. / 6
Vancouver
•Outcomes of individuals with serious substance use issues
• 3rd arm – a congregate setting

7. / 7
•Focus on Aboriginal homeless population
• 3rd arm included traditional healing approaches (e.g., medicine wheel)
Winnipeg

8. / 8
Toronto
• 3rd arm – ethno-culturally appropriate based on anti- racist/anti-oppression model
• Also emphasis on primary care/chronic physical illnesses

9. / 9
• Intervention included test of a vocational approach – individualized placement and support (IPS)
Montreal

10. / 10
Moncton
• Blended intervention - “flexible ACT” (FACT) model
• Focus on smaller cities and rural areas

11. / 11
Implementation of Housing First
•National and local training events
•Technical assistance visits
•Communities of practice
•Quality Assurance Team conducts fidelity assessments

12. / 12
What is the Purpose of the Mixed Methods Fidelity and Implementation Evaluation?
Quantitative fidelity assessment
•Looks at degree of implementation of critical ingredients
•Rates programs numerically using scale
•Provides feedback for quality improvement
Qualitative process and implementation evaluations uncovers:
•Strengths
•Challenges/Trouble spots

13. / 13
Can Housing First Be Adapted to Local Context and Culture?
Yes, but must be careful that the adaptation does not deviate from Housing First elements/principles
•Toronto’s Anti-racism, Anti-oppression ICM program
•Winnipeg’s Aboriginal-focused medicine wheel program But “no,” when the adaptation goes against the basic principles of Housing First, because it enters a “zone of drastic mutation” (Hall & Loucks, 1978)

14. Housing
First
Fidelity
Adaptation

15. / 15
Why is Housing First Fidelity Important?
Fidelity matters because it is related to outcomes. The greater the fidelity, the better the outcomes of Housing First
•Gilmer et al. (2014)
•Davidson et al. (2014)

16. / 16
Relationship of Fidelity with Program Processes and Participant Outcomes
•Higher fidelity in the in the 12 programs (5 ACT, 7 ICM) was related to greater direct and indirect service time and more contacts with participants (corr. = .55 to .60)
•Higher fidelity was associated with greater housing stability (% of time in housing) (OR = 1.11) and larger improvement in quality of life (d = .10) and community functioning (d = .11) over the course of the study

17. / 17
Housing First Fidelity Domains
Housing and recovery-oriented approach – 37 item scale with items falling into 5 domains; each item completed for each program
•Housing choice and structure
•Separation of housing and services
•Treatment philosophy (recovery orientation)
•Service array
•Program structure

18. / 18
Sample Housing First Fidelity Items
Housing choice and structure domain
•Housing choice
•Housing availability
•Permanent housing
•Affordable housing
•Integrated housing
•Privacy

19. / 19
Housing Availability Item – Anchors
All items are rated on a 4-point scale, with 4 indicating high fidelity, and 1 indicating low fidelity
•Extent to which program helps participants move quickly into units of their choosing.
1.Less than 55% of program participants move into a unit of their choosing within 6 weeks of having a housing subsidy or receiving a voucher
2.55-69% of program participants move into a unit of their choosing within 6 weeks of having a housing subsidy or receiving a voucher
3.70-84% of program participants move into a unit of their choosing within 6 weeks of having a housing subsidy or receiving a voucher
4.85% of program participants move into a unit of their choosing within 6 weeks of having a housing subsidy or receiving a voucher

20. / 20
Fidelity Assessment
External quality assurance team
Data Collected
Time 1 – 9-13 Months of Program Operation
Time 2 – 26-31 Months of Program Operation
Staff Interviews
84
89
Consumer Focus Groups
10
11
Chart Reviews
100
102
Number of Programs
10
10
Number of Sites
5
5

21. / 21
Summary of Fidelity Rating Findings
•10 programs rated on 37 fidelity items in 5 domains
•Overall, strong fidelity to the Housing first model
•Also, improvement noted from early to later implementation (71% program ingredients rated above 3 on a 4-point scale at early implementation and 78% of program ingredients rated above 3 at later implementation)
•Average fidelity rating across all items is 3.47 at early implementation and 3.62 at later implementation
•Variation at individual program level and by service delivery type (e.g., more challenges for ICM regarding service array domain)
•Both fidelity and adaptation are important

22. / 22
Qualitative Implementation Evaluation
•Aim: to better understand the reasons for implementation fidelity strengths and challenges/trouble spots in the At Home/ Chez Soi project
•Qualitative implementation evaluations were conducted in years one and three of the project
•Primary questions:
1.What are implementation strengths?
2.What are implementation challenges/trouble spots?

23. / 23
Qualitative Implementation Evaluation
Site qualitative research teams and national qualitative research team
Data Collected
Time 1 – 9-13 Months of Program Operation
Time 2 – 26-31 Months of Program Operation
Key Informant Interviews
72
36
Focus Groups
35(211)
17(99)
Number of Programs
10
10
Number of Sites
5
5

24. / 24
Methods
•Sampling and sample
•Purposeful sampling
•283 participants total at T1 and 135 total at T2
•Data collection
•Interviews in English or French
•At participants’ workplaces or site offices
•Interviews audio recorded and transcribed verbatim
•Data analysis
•Thematic analysis for site reports and cross-site report

25. / 25
Results – Q1: Implementation Strengths
•Partnerships and collaboration between services teams, and with participants, community organizations, and professionals were important strengths
•One respondent at the Vancouver site stated, “I’ve never worked in such a supportive and collaborative place. The team is so committed to what it does, and I think it’s reflected in our communication and the changes we’ve seen in our clients.”

26. / 26
Results – Q1: Implementation Strengths
•Despite low vacancy rates and lack of affordable housing in every city, all sites reported success with specific aspects of housing and rehousing participants
•Across sites, this was achieved primarily by 1) creatively matching participants with housing units; 2) learning from past experiences; 3) achieving stability; 4) planning preventative strategies to avoid future rehousing incidents; and 5) forming collaborative relationships with landlords and property management companies
•“Compared to other landlords, those in our project get a great deal of support… We underestimated how important that support is. It’s like an extra set of hands helping them do their job.” (Vancouver team member)

27. / 27
Results – Q1: Implementation Strengths
•Presence of dedicated specialists and professionals on housing and service teams
•For example, the Moncton site reported that the addition of a home economist, psychiatric consultant, and vocational specialist to their ACT team enabled program delivery and support
•Toronto noted that their service provision success could be attributed to the high level of clinical expertise from team leaders and psychiatrists

28. / 28
Results – Q2: Implementation Challenges/Trouble Spots
•Staffing
•Substantial staff turn-over across sites due to heavy workloads, pressure to meet project timelines, and concerns about job security
•This is a common problem in these types of programs and not something unique or unusual about the At Home/ Chez Soi project
•Scattered-site housing model
•Vancouver, Winnipeg, Montreal, and Moncton faced challenges related to scattered-site being the sole model of housing for the project, particularly for people with more complex physical and mental health challenges
•Winnipeg suggested congregate-type housing as an alternative; Vancouver introduced the idea of “step-up, step-down” options; and Moncton opened a transitional apartment building

29. / 29
Results – Q2: Implementation Challenges/Trouble Spots
•Participant feelings of isolation and loneliness
•Isolation is a broader social issue which is more acute for this population
•Sites attributed these difficult experiences to participants’ changing social networks, leaving friends behind, and vast lifestyle and living arrangement changes
•“…the person that is addicted lives a life of loneliness… You can’t be around your friends, so the drug is your best friend… An agency has to make sure that you can be there to complement that… until they have been able and have had time to build new friends.” (Moncton key informant)

30. / 30
Results – Q2: Implementation Challenges/Trouble Spots
•Some sites had problems integrating peer support into the program
•Some difficulty identifying and training potential peer specialists
•Moncton and Winnipeg sites had concerns about appropriately defining and clarifying the roles that peer support workers should play on service teams
•Documentation
•Sites noted the need to more clearly document housing and service teams’ work in their case notes
•Certain implementation challenges and low implementation fidelity for certain criteria could be attributed to inconsistent record and chart- keeping by team members

31. / 31
Results – Q2: Implementation Challenges/Trouble Spots
•Most sites identified some trouble spots regarding housing between the second and third years of implementation
•Challenges around finding good quality, affordable, vacant housing units in areas in which participants desired to live
•Housing restrictions related to smoke-free and pet-free policies (Vancouver)
•In Toronto, issues of tenancy management, relationships between housing teams and participants, dealing with unit damage, and rehousing participants emerged as trouble-spots in housing
•“So sometimes, the only interaction we had with clients was when there was chaos, when their was a real problem that was going on… I mean, the landlords are calling us and complaining… but we have no idea who the person is so that was a disadvantage to me that’s going forward.” (Toronto team member)

32. / 32
Results – Q2: Implementation Challenges/Trouble Spots
•Vocational and educational goal supports
•The Vancouver site reported that initially engaging and/ or re-engaging participants in work, volunteer, and educational activities had become difficult
•Vocational specialists often had to tend to emergency crises rather than develop more proactive, long-term goals with participants
•Toronto team members found limited variety and availability of educational and employment opportunities within the community

33. / 33
Conclusions
•Findings provide information about identifying and maintaining strengths, as well as identifying and potentially preventing challenges or trouble spots when implementing complex community-based housing interventions