Use of learning collaboratives by the center for practice innovations to bring IPS to scale in New York State

Staff Time in the Community: An Enduring Critical Component of Individual Placement and Support in the Digital Age

OBJECTIVE

This study examined the relationship between individual placement and support (IPS) employment specialists' time spent in the community and employment outcomes in the current digital age, featuring increased technology use and online hiring practices.

METHODS

The authors examined the relationship between employment outcomes and IPS employment specialists' time spent in the community at 78 sites in 2018 and 84 sites in 2019.

RESULTS

The amount of time staff spent in the community was significantly and positively associated with better employment outcomes.

CONCLUSIONS

These data support the continued importance of employment specialists' spending time in the community with employers and IPS recipients to achieve optimal outcomes for recipients.

Learning Collaboratives: a Strategy for Quality Improvement and Implementation in Behavioral Health

Learning collaboratives are increasingly used in behavioral health. They generally involve bringing together teams from different organizations and using experts to educate and coach the teams in quality improvement, implementing evidence-based practices, and measuring the effects. Although learning collaboratives have demonstrated some effectiveness in general health care, the evidence is less clear in behavioral health and more rigorous studies are needed. Learning collaboratives may contain a range of elements, and which elements are included in any one learning collaborative varies widely; the unique contribution of each element has not been established. This commentary seeks to clarify the concept of a learning collaborative, highlight its common elements, review evidence of its effectiveness, identify its application in behavioral health, and highlight recommendations to guide technical assistance purveyors and behavioral health providers as they employ learning collaboratives to improve behavioral health access and quality.

Behavioral Health Workforce Development in the era of COVID-19: Examples From a State-Funded Intermediary Organization

Intermediary and purveyor organizations (IPOs) play a key role in disseminating and implementing behavioral health evidence-based practices. The COVID-19 pandemic created a time of crisis and disruption to behavioral health care delivery. Using the conceptual framework of basic, targeted, and intensive technical assistance (TA) from the Training and Technology Transfer Centers, case studies are used to describe how programs at The Center for Practice Innovations a state funded-intermediary organization, adapted its training and technical assistance to be delivered entirely remotely, to include content related to COVID-19 and to provide guidance on telehealth-based behavioral health care.

Impact of the COVID-19 Pandemic on a Statewide Individual Placement and Support Employment Initiative

The COVID-19 pandemic has had an enormous impact on the provision of behavioral health care services across the United States. This column examines this impact within the context of New York State's supported employment initiative, which involved 89 implementation sites before the start of the pandemic. The pandemic caused changes to the training and implementation supports provided, the number of sites providing these services, and the ways in which sites provided supported employment services. Although mean self-assessed implementation fidelity decreased modestly, employment outcomes that dipped early in the pandemic rebounded quickly to prepandemic levels.

Self- Evaluation Tool to Support Implementation of Treatment for Tobacco Use Disorder in Behavioral Health Programs

Tobacco use in people with behavioral health conditions remain two to three times higher than the general population causing premature death and impacting recovery negatively across several domains. Intermediary organizations can provide practical tools, training, and technical assistance to help programs improve capacity to treat tobacco use. This report describes the construction and application of the Tobacco Integration Self-Evaluation Tool (TiSET) for behavioral health programs, a 20-item scale inspired by the DDCMHT and additional content from the Facility Tobacco Policy and Treatment Practices Self-Evaluation tool that one of the study authors (JW) used previously with addiction treatment programs. Completing the TiSET is an important step for behavioral health programs to evaluate their ability to effectively treat people that use tobacco. An important next step is to use those results to facilitate a quality improvement process. We include large agency example illustrating how the TiSET can be applied in real-world practice.

Applying Implementation Drivers to Scale-up Evidence-Based Practices in New York State

Mental health authorities in several states, often working with academic partners, have played important roles in disseminating evidence-based practices (EBPs) for adults diagnosed with serious mental illness. This work has been facilitated by intermediary organizations that work directly with providers to implement EBPs. This report uses two case studies to describe how the Center for Practice Innovations (CPI), an intermediary organization, has used the Active Implementation Research Network’s nine implementation drivers to successfully implement EBPs across the large state of New York. One case study focuses on supported employment and the second on integrated treatment for co-occurring mental health and substance use conditions. We provide these case studies to illustrate how intermediary organizations can use implementation science to organize and select effective support strategies to disseminate and implement a range of EBPs within a state system.

A Feasibility Cluster Randomized Controlled Trial of Individual Placement and Support (IPS) for Patients With Offending Histories

Objective: To examine the feasibility of conducting a fully powered randomized controlled trial (RCT) of Individual Placement and Support (IPS). IPS is a form of supported employment which aims to put people into open employment quickly and in accordance with their preferences. It is delivered by employment specialists collocated within clinical teams, and provides time unlimited support for the individual and their employer, along with welfare benefits counselling. Method: A feasibility cluster RCT of treatment as usual (TAU) plus IPS versus TAU alone was conducted over 12 months among patients with offending histories in a community forensic setting in the UK. The feasibility criteria were to achieve 50% recruitment rate; 50% completion rate for IPS; 50% completion rate of all outcome measures; and 80% acceptability rating for IPS. The primary efficacy outcome was the proportion of people in open employment at 12 months. The secondary outcomes were other vocational and educational activities; Brief Psychiatric Rating Scale; Rosenberg's Self-esteem Scale; Client Service Receipt Inventory; quality of life using the SF12-v2 and EQ5-D3; Social Functioning Questionnaire; Work Limitation Questionnaire; and reoffending. Results: Participants' mean age was 39.2 years. The majority were male (88.9), White British (72.2), and single (72.2%). Over 72% had no higher qualification beyond secondary education; mean years in education was 10.4. Over one third had schizophrenia, one fifth had depression, and the rest had personality disorder as their primary diagnosis. Participants had a lifetime average of 7.5 convictions for 15.5 offences. The recruitment rate of all referrals was 38.3% (IPS n = 11; TAU n = 7). Completion rate for IPS was 54.5, with 45.5% acceptability rating. Completion rates for outcome measures for the groups at baseline and 12 months ranged from 22.2 to 100%. The proportion of people in open employment at 12 months were 9.1 and 0% for IPS and TAU respectively. Conclusion: It is not feasible to conduct a full RCT of IPS in community forensic settings in the UK owing to recruitment and retention difficulties. Conducting a trial of this kind requires a large pool of patients from multiple sites and longer IPS implementation and recruitment periods than those of this study. Clinical Trial Registration: www.ClinicalTrials.gov, identifier NCT02442193.

Barriers and facilitators to the implementation of individual placement and support (IPS) for patients with offending histories in the community: The United Kingdom experience

Introduction

We aimed to identify the barriers and facilitators to the implementation of a high fidelity individual placement and support service in a community forensic mental health setting.

Method

In-depth interviews were conducted with clinical staff ( n = 11), patients ( n = 3), and employers ( n = 5) to examine barriers and facilitators to implementation of a high fidelity individual placement and support service. Data was analysed using thematic analysis, and themes were mapped onto individual placement and support fidelity criteria.

Results

Barriers cited included competing interests between employment support and psychological therapies, perceptions of patients’ readiness for work, and concerns about the impact of returning to work on welfare benefits. Facilitators of implementation included clear communication of the benefits of individual placement and support, inter-disciplinary collaboration, and positive attitudes towards the support offered by the individual placement and support programme among stakeholders. Offences, rather than mental health history, were seen as a key issue from employers’ perspectives. Employers regarded disclosure of offending or mental health history as important to developing trust and to gauging their own capacity to offer support.

Conclusions

Implementation of individual placement and support in a community mental health forensic setting is complex and requires robust planning. Future studies should address the barriers identified, and adaptations to the individual placement and support model are needed to address difficulties encountered in forensic settings.

Using Instructional Design, Analyze, Design, Develop, Implement, and Evaluate, to Develop e-Learning Modules to Disseminate Supported Employment for Community Behavioral Health Treatment Programs in New York State

Background

Implementation science lacks a systematic approach to the development of learning strategies for online training in evidence-based practices (EBPs) that takes the context of real-world practice into account. The field of instructional design offers ecologically valid and systematic processes to develop learning strategies for workforce development and performance support.

Objective

This report describes the application of an instructional design framework-Analyze, Design, Develop, Implement, and Evaluate (ADDIE) model-in the development and evaluation of e-learning modules as one strategy among a multifaceted approach to the implementation of individual placement and support (IPS), a model of supported employment for community behavioral health treatment programs, in New York State.

Methods

We applied quantitative and qualitative methods to develop and evaluate three IPS e-learning modules. Throughout the ADDIE process, we conducted formative and summative evaluations and identified determinants of implementation using the Consolidated Framework for Implementation Research (CFIR). Formative evaluations consisted of qualitative feedback received from recipients and providers during early pilot work. The summative evaluation consisted of levels 1 and 2 (reaction to the training, self-reported knowledge, and practice change) quantitative and qualitative data and was guided by the Kirkpatrick model for training evaluation.

Results

Formative evaluation with key stakeholders identified a range of learning needs that informed the development of a pilot training program in IPS. Feedback on this pilot training program informed the design document of three e-learning modules on IPS: Introduction to IPS, IPS Job development, and Using the IPS Employment Resource Book. Each module was developed iteratively and provided an assessment of learning needs that informed successive modules. All modules were disseminated and evaluated through a learning management system. Summative evaluation revealed that learners rated the modules positively, and self-report of knowledge acquisition was high (mean range: 4.4-4.6 out of 5). About half of learners indicated that they would change their practice after watching the modules (range: 48-51%). All learners who completed the level 1 evaluation demonstrated 80% or better mastery of knowledge on the level 2 evaluation embedded in each module. The CFIR was used to identify implementation barriers and facilitators among the evaluation data which facilitated planning for subsequent implementation support activities in the IPS initiative.

Conclusion

Instructional design approaches such as ADDIE may offer implementation scientists and practitioners a flexible and systematic approach for the development of e-learning modules as a single component or one strategy in a multifaceted approach for training in EBPs.

Relationship Between Self-Assessed Fidelity and Self-Reported Employment in the Individual Placement and Support Model of Supported Employment

OBJECTIVE

A growing body of literature demonstrates that high-fidelity implementation of the individual placement and support (IPS) model of supported employment increases the chances of achieving desired outcomes. This study examined the relationship between IPS fidelity, as self-reported by program sites, and employment outcomes and determined whether this relationship was maintained over time.

METHODS

A total of 78 outpatient programs in New York State provided data on self-reported IPS fidelity and employment outcomes. Pearson correlations were used to determine the relationship between fidelity scores and competitive employment rates. A mixed-effects model examined the relationship between repeated fidelity and employment measures over time.

RESULTS

A significant positive relationship was found between better self-reported IPS fidelity and greater employment. The relationship between IPS fidelity and employment was sustained over time (up to one year).

CONCLUSIONS

Higher-fidelity implementation of the IPS model, as self-assessed by program sites, was associated with higher employment rates, which were sustained over time.

Is There a Role for Fidelity Self-Assessment in the Individual Placement and Support Model of Supported Employment?

OBJECTIVE

Fidelity assessments help ensure that evidence-based practices are implemented properly. Although assessments are typically conducted by independent raters, some programs have implemented self-assessments because of resource constraints. Self-assessments were compared with independent assessments of programs implementing individual placement and support supported employment.

METHODS

Eleven community-based outpatient programs in New York State completed both self- and independent assessments. Intraclass correlation coefficients and paired t tests were used to compare scores from self- and independent assessments.

RESULTS

For both assessment methods, mean scores for all programs were within the range of fair fidelity. Self- and independent assessment total scores were not significantly different; however, significant differences were found on some scale items in this small sample.

CONCLUSIONS

Self-assessment may be valid for examining a program's overall functioning and useful when resource constraints prevent independent assessment. Independent assessors may be able to identify nuances, particularly on individual assessment items, that can point to areas for program improvement.

Prevalence and Quality of Individual Placement and Support (IPS) Supported Employment in the United States

The individual placement and support (IPS) model of supported employment for people with serious mental illness is an evidence-based practice. Factors including a national learning community promoting IPS and enforcement of the Supreme Court's Olmstead decision have spurred the growth of IPS nationwide. In this study we first evaluated the national prevalence and quality of IPS programs. We then evaluated the impact of learning community membership and Olmstead settlements on IPS program penetration and quality across the United States. We interviewed representatives from 48 state behavioral health agencies and 51 state vocational rehabilitation agencies. Survey questions examined the number of IPS programs in each state, the presence of an Olmstead settlement mandating employment services for people with serious mental illness, and the presence of three indicators of quality in IPS programs: collaboration between state behavioral health and vocational rehabilitation agencies, regular, independent fidelity monitoring, and technical assistance and training for IPS programs. Respondents from 38 (75%) states, including 19 states in the IPS Learning Community and 19 outside the learning community, reported a total of 523 IPS programs nationwide (M = 14, SD = 16). The state IPS program penetration rate (number of IPS programs per 1,000,000 people) ranged from 0.05 to 16.62 (M = 3.61, SD = 3.62) among states with IPS. The penetration rate was similar for learning community and non-learning community states with IPS, but learning community states were much more likely than non-learning community states with IPS to report the presence of each of three quality indicators. Eleven states reported Olmstead or other settlements that positively impacted employment services for people with serious mental illness, but among the 38 states with IPS programs, Olmstead states did not differ from non-Olmstead states in IPS program penetration or on the quality indicators. Nationally, most states provide IPS programs, but the within-state penetration rate and quality of implementation vary widely. While learning community and non-learning community states with IPS do not differ in the prevalence of IPS programs, learning community states are much more likely to report key quality indicators, which may enhance these states' potential for sustaining and expanding IPS. Olmstead settlements have not yet shown a direct impact on the penetration and quality of IPS, but as the Department of Justice continues to enforce the Supreme Court's Olmstead decision, their significance may increase.

Sustaining and Expanding Evidence-Based Supported Employment: The Role of State Leaders Participating in a Learning Community

State leaders often promote implementation of evidence-based practices but have difficulty sustaining and expanding them over time. This paper examines the activities of leaders in 13 states that have successfully implemented, sustained, and expanded evidence-based supported employment, known as Individual Placement and Support (IPS), for 4 to 12 years. We interviewed state leaders from 13 states participating in a learning community regarding the composition of their leadership team, participation in the learning community, interagency collaboration, state policy alignment, financing, training, and monitoring of fidelity and outcome. To assess state-level performance in implementing, sustaining, and expanding IPS services, we obtained measures of sustainment, expansion, program fidelity, and employment in the subsequent year and compared them to a priori benchmarks. The majority of states (between 69 % and 77 %) met benchmarks for sustainment, expansion, fidelity, and employment. States varied widely in specific actions to advance IPS, but all had established leadership teams, participated in the national learning community, and built an infrastructure supporting IPS. Leaders in 13 states participating in a learning community have adopted and maintained multiple strategies to sustain and expand evidence-based supported employment at a high level of fidelity with good employment outcomes.

Recent advances in supported employment for people with serious mental illness

PURPOSE OF REVIEW

Supported employment is a vocational rehabilitation approach that eschews prevocational training in favor of rapid job search for competitive work and follow-along supports to sustain employment, with the individual placement and support (IPS) model being the most standardized and researched model. This review covers recent research on the IPS model of supported employment.

RECENT FINDINGS

The evidence base for IPS-supported employment continues to grow, with seven new randomized controlled trials published, most conducted abroad, resulting in a total of 23 controlled studies showing that IPS is effective at improving work outcomes. Several reviews have concluded that competitive work improves quality of life in persons with serious mental illness, and some research has suggested that employment may confer clinical benefits. Encouraging research has been published on modifications of the IPS model (such as inclusion of supported education), augmentations (such as cognitive remediation) and adaptations for underserved populations (such as persons with criminal justice involvement). Recent studies have also described the effective strategies for implementing and sustaining IPS; others have reported its cost-effectiveness.

SUMMARY

Research continues to accumulate on the effectiveness of IPS-supported employment, adaptations of the model and overcoming programme and policy barriers to its widespread implementation.