Exploring the value of mixed methods within the At Home/Chez Soi housing first project: a strategy to evaluate the implementation of a complex population health intervention for people with mental illness who have been homeless

A WISH to be Housed: Exploring the Winter Interim Solution to Homelessness (WISH) Temporary Accommodation Model in London, Canada

Temporary housing programs (THPs) aim to serve the homeless population. This article explores the impacts of a THP, the Winter Interim Solution to Homelessness (WISH) in London, Canada, which applied a barrier-free, harm reduction model. Adopting an intersectional lens and interpretive description methodology, we analyzed data collected from WISH residents, utilizing a thematic analytic approach. Identified themes included: (1) "(It's) kinda like a real home" (WISH is unlike other shelters); (2) "It's like a buddy system" (A sense of community was fostered); (3) "It's the same size as a jail cell" (Problems with infrastructure); (4) "To keep us on focus" (Un/supportive staff/volunteers); (5) "I'm not sure what I'm going to do after" (The dearth of permanent housing creates trauma); and (6) "Too much bloody money in too little hands" (Distrust of housing providers). Although WISH was helpful to some residents, the temporary nature of the program limited its long-term impact.

Patterns and effects of social integration on housing stability, mental health and substance use outcomes among participants in a randomized controlled Housing First trial

INTRODUCTION

Recent research has shown that social integration, involving community integration and social support, can be protective resources for the health of homeless individuals. However, it is not clear how social integration affects health in the transition from homelessness into housing, and subsequent housing retention. This paper examines, through mixed quantitative and qualitative methods, how social integration changes over time, and how these changes relate to housing stability, substance use and mental health outcomes among a sample of homeless individuals experiencing mental illness participating in the At Home/Chez Soi Housing First randomized controlled trial in Canada.

METHODS

Longitudinal quantitative data (baseline, 6, 12, 24 month) and qualitative data ( 18 month) from the five trial sites (Moncton, Montreal, Toronto, Winnipeg and Vancouver) were examined using mixed models, mediation analyses, and thematic analysis.

RESULTS

Social integration (i.e., social network size, social interest, psychological integration) increased over time among study participants, with social network size increasing significantly more among Housing First participants than Treatment as Usual participants. Social network size mediated the effect of the intervention on the percentage of days in stable housing, indicating that the Housing First intervention may have increased participants' social network size, which in turn increased the percentage of days stably housed. No significant mediation of social integration on the effects of the intervention on mental health and substance use outcomes was found. Findings from the qualitative interviews support and expand upon these quantitative findings.

Recovery education for people experiencing housing instability: An evaluation protocol

BACKGROUND

Recovery education centers (RECs) offer recovery supports through education rather than traditional health services. The Supporting Transitions and Recovery Learning Centre (STAR) in Toronto, Canada, is among the few that are internationally focused on individuals with histories of homelessness. Although research suggests that RECs positively impact participants, there is a paucity of rigorous studies and none address the engagement and impacts on homeless individuals.

AIMS

This protocol describes a realist-informed evaluation of STAR, specifically examining (1) if STAR participation is more effective in promoting 12-month recovery outcomes than participation in usual services for individuals experiencing housing instability and mental health challenges and (2) how STAR participation promotes recovery and other positive outcomes.

METHODS

This study uses a quasi-experimental mixed methods design. Personal empowerment (primary outcome) and recovery, housing stability, social functioning, health service use and quality of life (secondary outcomes) data were collected at baseline, and 6 and 12 months. Intervention group participants were recruited at the time of STAR registration while control group participants were recruited from community agencies serving this population after screening for age and histories of housing instability. Interviews and focus groups with service users and providers will identify the key intervention ingredients that support the process of recovery.

RESULTS

From January 2017 to July 2018, 92 individuals were recruited to each of the intervention and control groups. The groups were mostly similar at baseline; the intervention group's total empowerment score was slightly higher than the control group's (M (SD): 2.94 (0.23) vs 2.84 (0.28), p = .02), and so was the level of education. A subset of STAR participants (n = 20) and nine service providers participated in the qualitative interviews and focus groups.

CONCLUSION

This study will offer important new insights into the effectiveness of RECs, and expose how key REC ingredients support the process of recovery for people experiencing housing instability.

Effect of Housing First on Suicidal Behaviour: A Randomised Controlled Trial of Homeless Adults with Mental Disorders

OBJECTIVE

This study attempted to determine if Housing First (HF) decreased suicidal ideation and attempts compared to treatment as usual (TAU) amongst homeless persons with mental disorders, a population with a demonstrably high risk of suicidal behaviour.

METHOD

The At Home/Chez Soi project is an unblinded, randomised control trial conducted across 5 Canadian cities (Vancouver, Winnipeg, Toronto, Montreal, Moncton) from 2009 to 2013. Homeless adults with a diagnosed major mental health disorder were recruited through community agencies and randomised to HF ( n = 1265) and TAU ( n = 990). HF participants were provided with private housing units and received case management support services. TAU participants retained access to existing community supports. Past-month suicidal ideation was measured at baseline and 6, 12, 18, and 21/24 months. A history of suicide attempts was measured at baseline and the 21/24-month follow-up.

RESULTS

Compared to baseline, there was an overall trend of decreased past-month suicidal ideation (estimate = -.57, SE = .05, P < 0.001), with no effect of treatment group (i.e., HF vs. TAU; estimate = -.04, SE = .06, P = 0.51). Furthermore, there was no effect of treatment status (estimate = -.10, SE = .16, P = 0.52) on prevalence of suicide attempts (HF = 11.9%, TAU = 10.5%) during the 2-year follow-up period.

CONCLUSION

This study failed to find evidence that HF is superior to TAU in reducing suicidal ideation and attempts. We suggest that HF interventions consider supplemental psychological treatments that have proven efficacy in reducing suicidal behaviour. It remains to be determined what kind of suicide prevention interventions (if any) are specifically effective in further reducing suicidal risk in a housing-first intervention.

What Happens After the Demonstration Phase? The Sustainability of Canada's At Home/Chez Soi Housing First Programs for Homeless Persons with Mental Illness

This research examined the sustainability of Canada's At Home/Chez Soi Housing First (HF) programs for homeless persons with mental illness 2 years after the end of the demonstration phase of a large (more than 2000 participants enrolled), five-site, randomized controlled trial. Qualitative interviews were conducted with 142 participants (key informants, HF staff, and persons with lived experience) to understand sustainability outcomes and factors that influenced those outcomes. Also, a self-report HF fidelity measure was completed for nine HF programs that continued after the demonstration project. A cross-site analysis was performed, using the five sites as case studies. The findings revealed that nine of the 12 HF programs (75%) were sustained, and that seven of the nine programs reported a high level of fidelity (achieving an overall score of 3.5 or higher on a 4-point scale). The sites varied in terms of the level of systems integration and expansion of HF that were achieved. Factors that promoted or impeded sustainability were observed at multiple ecological levels: broad contextual (i.e., dissemination of research evidence, the policy context), community (i.e., partnerships, the presence of HF champions), organizational (i.e., leadership, ongoing training, and technical assistance), and individual (i.e., staff turnover, changes, and capacity). The findings are discussed in terms of the implementation science literature and their implications for how evidence-based programs like HF can be sustained.

Implementing Housing First Across Sites and Over Time: Later Fidelity and Implementation Evaluation of a Pan-Canadian Multi-site Housing First Program for Homeless People with Mental Illness

This article examines later fidelity and implementation of a five-site pan-Canadian Housing First research demonstration project. The average fidelity score across five Housing First domains and 10 programs was high in the first year of operation (3.47/4) and higher in the third year of operation (3.62/4). Qualitative interviews (36 key informant interviews and 17 focus groups) revealed that staff expertise, partnerships with other services, and leadership facilitated implementation, while staff turnover, rehousing participants, participant isolation, and limited vocational/educational supports impeded implementation. The findings shed light on important implementation "drivers" at the staff, program, and community levels.

Evaluation of the implementation of the Montreal At Home/Chez Soi project

Background

Homelessness and mental disorders constitute a major problem in Canada. The purpose of the At Home/Chez Soi pilot project was to house and provide supports to marginalised groups. Policymakers are in a better position to nurture new, complex interventions if they know which key factors hinder or enable their implementation. This paper evaluates the implementation process for the Montreal site of this project.

Methods

We collected data from 62 individuals, through individual interviews, focus groups, questionnaires, observations and documentation. The implementation process was analysed using a conceptual framework with five constructs: Intervention Characteristics (IC), Context of Implementation (CI), Implementation Process (IP), Organizational Characteristics (OC) and Strategies of Implementation (SI).

Results

The most serious obstacle to the project came from the CI construct, i.e., lack of support from provincial authorities and key local resources in the homelessness field. The second was within the OC construct. The chief hindrances were numerous structures, divergent values among stakeholders, frequent turnover of personnel and team leaders; lacking staff supervision and miscommunication. The third is related to IC: the complex, unyielding nature of the project undermined its chances of success. The greatest challenges from IP were the pressure to perform, along with stress caused by planning, deadlines and tension between teams. Conversely, SI construct conditions (e.g., effective governing structures, comprehensive training initiatives and toolkits) were generally very positive even with problems in power sharing and local leadership. For the four other constructs, the following proved useful: evidence of the project’s scope and quality, great needs of services consolidation, generous financing and status as a research pilot project, enthusiasm and commitment toward the project, substantially improved services, and overall user satisfaction.

Conclusion

This study demonstrated the difficulty of implementing a complex project in the healthcare system. While the project faced many barriers, minimal conditions were also achieved. At the end of the study period, major tensions between organizations and teams were significantly reduced, supporting its full implementation. However, in late 2013, the project was unsustainable, calling into question the relevance of achieving a significant number of positive conditions in each area of the framework.

Early implementation evaluation of a multi-site housing first intervention for homeless people with mental illness: a mixed methods approach

This research sought to determine whether the implementation of Housing First in a large-scale, multi-site Canadian project for homeless participants with mental illness shows high fidelity to the Pathways Housing First model, and what factors help or hinder implementation. Fidelity ratings for 10 Housing First programs in five cities were made by an external quality assurance team along five key dimensions of Housing First based on 84 key informant interviews, 10 consumer focus groups, and 100 chart reviews. An additional 72 key informant interviews and 35 focus groups yielded qualitative data on factors that helped or hindered implementation. Overall, the findings show a high degree of fidelity to the model with more than 71% of the fidelity items being scored higher than 3 on a 4-point scale. The qualitative research found that both delivery system factors, including community and organizational capacity, and support system factors, training and technical assistance, facilitated implementation. Fidelity challenges include the availability of housing, consumer representation in program operations, and limitations to the array of services offered. Factors that accounted for these challenges include low vacancy rates, challenges of involving recently homeless people in program operations, and a lack of services in some of the communities. The study demonstrates how the combined use of fidelity assessment and qualitative methods can be used in implementation evaluation to develop and improve a program.

Are we producing the right kind of actionable evidence for the social determinants of health?

Globally, health and social inequities are growing and are created, actively maintained, and aggravated by existing policies and practices. The call for evidence-based policy making to address this injustice seems a promising strategy to facilitate a reversal of existing strategies and the design of new effective programming. Acting on evidence to address inequities requires congruence between identifying the major drivers of disparities and the study of their causes and solutions. Yet, current research on inequities tends to focus on documenting disparities among individuals or subpopulations with little focus on identifying the macro-social causes of adverse population health. Moreover, the research base falls far short of a focus on the solutions to the complex multilevel drivers of disparities. This paper focuses upon recommendations to refocus and improve the public health research evidence generated to inform and create strong evidence-based recommendations for improving population health.