Adolescent, parent, and provider attitudes toward a machine learning based clinical decision support system for selecting treatment for youth depression

BACKGROUND

Machine learning based clinical decision support systems (CDSSs) have been proposed as a means of advancing personalized treatment planning for disorders, such as depression, that have a multifaceted etiology, course, and symptom profile. However, machine learning based models for treatment selection are rare in the field of psychiatry. They have also not yet been translated for use in clinical practice. Understanding key stakeholder attitudes toward machine learning based CDSSs is critical for developing plans for their implementation that promote uptake by both providers and families.

METHODS

In Study 1, a prototype machine learning based Clinical Decision Support System for Youth Depression (CDSS-YD) was demonstrated to focus groups of adolescents with a diagnosis of depression (n = 9), parents (n = 11), and behavioral health providers (n = 8). Qualitative analysis was used to assess their attitudes towards the CDSS-YD. In Study 2, behavioral health providers were trained in the use of the CDSS-YD and they utilized the CDSS-YD in a clinical encounter with 6 adolescents and their parents as part of their treatment planning discussion. Following the appointment, providers, parents, and adolescents completed a survey about their attitudes regarding the use of the CDSS-YD.

RESULTS

All stakeholder groups viewed the CDSS-YD as an easy to understand and useful tool for making personalized treatment decisions, and families and providers were able to successfully use the CDSS-YD in clinical encounters. Parents and adolescents viewed their providers as having a critical role in the use the CDSS-YD, and this had implications for the perceived trustworthiness of the CDSS-YD. Providers reported that clinic productivity metrics would be the primary barrier to CDSS-YD implementation, with the creation of protected time for training, preparation, and use as a key facilitator.

CONCLUSIONS

Machine learning based CDSSs, if proven effective, have the potential to be widely accepted tools for personalized treatment planning. Successful implementation will require addressing the system-level barrier of having sufficient time and energy to integrate it into practice.

Adolescent, parent, and provider attitudes toward a machine-learning based clinical decision support system for selecting treatment for youth depression

Background

Machine-learning based clinical decision support systems (CDSSs) have been proposed as a means of advancing personalized treatment planning for disorders, such as depression, that have a multifaceted etiology, course, and symptom profile. However, machine-learning based models for treatment selection are rare in the field of psychiatry. They have also not yet been translated for use in clinical practice. Understanding key stakeholder attitudes toward machine learning-based CDSSs is critical for developing plans for their implementation that promote uptake by both providers and families.

Methods

In Study 1, a machine-learning based Clinical Decision Support System for Youth Depression (CDSS-YD) was demonstrated to focus groups of adolescents with a diagnosis of depression (n = 9), parents (n = 11), and behavioral health providers (n = 8). Qualitative analysis was used to assess their attitudes towards the CDSS-YD. In Study 2, behavioral health providers were trained in the use of the CDSS-YD and they utilized the CDSS-YD in a clinical encounter with 6 adolescents and their parents as part of their treatment planning discussion. Following the appointment, providers, parents, and adolescents completed a survey about their attitudes regarding the use of the CDSS-YD.

Results

All stakeholder groups viewed the CDSS-YD as an easy to understand and useful tool for making personalized treatment decisions, and families and providers were able to successfully use the CDSS-YD in clinical encounters. Parents and adolescents viewed their providers as having a critical role in the use the CDSS-YD, and this had implications for the perceived trustworthiness of the CDSS-YD. Providers reported that clinic productivity metrics would be the primary barrier to CDSS-YD implementation, with the creation of protected time for training, preparation, and use as a key facilitator.

Conclusions

The CDSS-YD has the potential to be a widely accepted and useful tool for personalized treatment planning. Successful implementation will require addressing the system-level barrier of having sufficient time and energy to integrate it into practice.

Toward a more comprehensive understanding of organizational influences on implementation: the organization theory for implementation science framework

Introduction

Implementation is influenced by factors beyond individual clinical settings. Nevertheless, implementation research often focuses on factors related to individual providers and practices, potentially due to limitations of available frameworks. Extant frameworks do not adequately capture the myriad organizational influences on implementation. Organization theories capture diverse organizational influences but remain underused in implementation science. To advance their use among implementation scientists, we distilled 70 constructs from nine organization theories identified in our previous work into theoretical domains in the Organization Theory for Implementation Science (OTIS) framework.

Methods

The process of distilling organization theory constructs into domains involved concept mapping and iterative consensus-building. First, we recruited organization and implementation scientists to participate in an online concept mapping exercise in which they sorted organization theory constructs into domains representing similar theoretical concepts. Multidimensional scaling and hierarchical cluster analyses were used to produce visual representations (clusters) of the relationships among constructs in concept maps. Second, to interpret concept maps, we engaged members of the Cancer Prevention and Control Research Network (CPCRN) OTIS workgroup in consensus-building discussions.

Results

Twenty-four experts participated in concept mapping. Based on resulting construct groupings' coherence, OTIS workgroup members selected the 10-cluster solution (from options of 7-13 clusters) and then reorganized clusters in consensus-building discussions to increase coherence. This process yielded six final OTIS domains: organizational characteristics (e.g., size; age); governance and operations (e.g., organizational and social subsystems); tasks and processes (e.g., technology cycles; excess capacity); knowledge and learning (e.g., tacit knowledge; sense making); characteristics of a population of organizations (e.g., isomorphism; selection pressure); and interorganizational relationships (e.g., dominance; interdependence).

Discussion

Organizational influences on implementation are poorly understood, in part due to the limitations of extant frameworks. To improve understanding of organizational influences on implementation, we distilled 70 constructs from nine organization theories into six domains. Applications of the OTIS framework will enhance understanding of organizational influences on implementation, promote theory-driven strategies for organizational change, improve understanding of mechanisms underlying relationships between OTIS constructs and implementation, and allow for framework refinement. Next steps include testing the OTIS framework in implementation research and adapting it for use among policymakers and practitioners.

Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy

BACKGROUND

Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers' implementation of evidence-based psychosocial interventions-(1) audit and feedback and (2) active learning.

METHODS

Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples.

RESULTS

In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ2=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ2=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples.

CONCLUSIONS

Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04096274 . Registered on 19 September 2019.

Increasing Access to Organization Theories for Implementation Science

Background

Organization theories offer numerous existing, highly relevant, yet largely untapped explanations of the organizational dynamics underlying evidence-based intervention (EBI) implementation. Rooted in ideas regarding power, autonomy, and control, organization theories can explain how and why organizations adopt, implement, and sustain EBI use. Although they have gained visibility, organization theories remain underused in implementation research, perhaps due to their inaccessibility to implementation scientists. To improve access to organization theory among implementation scientists, we summarized organization theories with relevance to implementation science.

Methods

Led by the Cancer Prevention and Control Research Network (CPCRN) Organization Theory for Implementation Science workgroup, we employed a modified Delphi process to reach a consensus among 18 experts at the intersection of organization and implementation science regarding organization theories with relevance to implementation science. From texts that described the organization theories, using standardized abstraction forms, two investigators independently abstracted information regarding constructs, propositions regarding how or why constructs might influence implementation, the potential relevance of organization theories' propositions for implementation, and overviews of each theory. The investigators then reconciled discrepancies until reaching consensus. A third investigator reviewed reconciled abstraction forms for accuracy, coherence, and completeness.

Findings

We identified nine organization theories with relevance to implementation science: contingency, complexity, institutional, network, organizational learning, resource dependence, sociotechnical, and transaction cost economics. From the theories, we abstracted 70 constructs and 65 propositions. An example proposition from institutional theory is: "Coercive, mimetic, and normative pressures contribute to organizations…within an organizational field [becoming increasingly similar]." These propositions can be operationalized as levers to facilitate EBI implementation.

Conclusions

To increase use in the field, organization theories must be made more accessible to implementation scientists. The abstraction forms developed in this study are now publicly available on the CPCRN website with the goal of increasing access to organization theories among an interdisciplinary audience of implementation scientists through the CPCRN Scholars program and other venues. Next steps include consolidating organization theory constructs into domains and translating the resulting framework for use among researchers, policymakers and practitioners, aiding them in accounting for a comprehensive set of organization theory constructs thought to influence EBI implementation.

Implementing Person-Centered Care Planning: A Randomized Controlled Trial

OBJECTIVE

Person-centered care is a key quality indicator and central to promoting integrated and recovery-oriented services. Person-centered care planning (PCCP) is a manualized intervention promoting the collaborative cocreation of a recovery-oriented care service plan on the basis of an individual's most valued life goals. This cluster randomized controlled trial tested the effect of PCCP training on person-centered care delivery in community mental health clinics.

METHODS

Fourteen clinic sites were randomly assigned to receive either PCCP training (N=7; experimental condition) or service planning as usual (N=7; control condition). Data were collected from online surveys, and service plans were completed by 60 provider teams. The Person-Centered Care Planning Assessment Measure was administered via chart review at baseline, 12 months, and 18 months, and surveys were used to measure supervision, implementation leadership, and program type. The main effect was examined with linear mixed-effects regression models, with observations over time.

RESULTS

Analyses controlling for service user and program characteristics revealed that at 12 months, the group assigned to PCCP training showed significant improvements in delivering person-centered care compared with the control group (b=1.10, SE=0.50, p=0.03). At 18 months, this effect was even more pronounced (b=1.47, SE=0.50, p=0.01), representing a medium-to-large effect size of d=0.71 (95% confidence interval=0.23-1.20).

CONCLUSIONS

These findings indicate that training providers in PCCP increases provider competency in delivering person-centered care. Using an objective measure of person-centered care, the authors show that a comprehensive training strategy can target both the philosophical shift and the technical skills needed to promote client recovery.

A Mixed Methods Study of Organizational Readiness for Change and Leadership During a Training Initiative Within Community Mental Health Clinics

This longitudinal mixed-methods study explored variation in organizational readiness for change and leadership behavior across seven organizations during a 12-month training initiative in person-centered care planning. Quantitative data was used to examine trajectories of organizational readiness for change and leadership behavior over time and qualitative data explored provider perspectives on the trajectory of these organizational factors during the 12-month training initiative. Findings indicated that levels of organizational readiness for change and leadership behavior varied across clinics, but most experienced a significant positive change at the mid-point of the training. Organizational readiness for change was positively correlated with leaderships behaviors across time. Provider focus group findings gave insight into their initial resistance to adopting the new practice and their increasing receptivity in the second 6 months due to increased understanding of the practice and leadership endorsement. Increasing provider openness to a new practice prior to training and having a consistently engaged leadership have the potential to improve the efficiency of a training initiative.

Do Social Workers Lead Differently? Examining Associations with Leadership Style and Organizational Factors

What training leaders need to successfully shepherd high quality human service delivery remains less understood. This study aims to evaluate associations between social work training, leadership style, and organizational factors. Participants included leaders (N=49) and employees (N=224) working in 10 behavioral health organizations who completed a survey. Hierarchical linear models were used to analyze multilevel associations. Analyses indicated that leaders with social work training were rated higher in transformational leadership and organizational factors (e.g., lesser stress) on average within both samples. Findings begin an empirical argument for the value of social work leaders for the success of human service organizations.

Strategies for Sustaining Fidelity: A Multi-state Qualitative Analysis in Housing First Programs

Little is known about long-term fidelity of evidence-based interventions (EBIs) under changing conditions. This study examines how staff at 'mature' (eight or more years in operation) Housing First (HF) programs strategize to sustain EBI fit in different geographic areas in the Mid-Atlantic/Northeastern United States. Six focus groups (FGs) at three purposively selected HF programs were conducted with separate FGs for case managers and supervisors at each site. FG discussions elicited participants' service approaches and strategies in addressing fidelity amidst ongoing changes affecting each program. Thematic content analysis of FG transcripts was conducted using the five HF fidelity domains (housing choice/structure, separation of housing and services, service philosophy, service array, and program structure) as a priori themes with inductive content analyses conducted on data in each theme. Strategies for rigor were employed. Case managers (N = 17) and supervisors (N = 16) were predominantly white (76%) and female (60%). Across the themes, challenges included lack of affordable housing and choice, funders' restrictions and practice 'drift.' Strategies included community engagement and hiring, strong leadership and 'bending the rules.' There were no differences across sites. Later-stage implementation challenges show the need for continued vigilance in fidelity to EBIs. Among the strategies used to address fidelity in this study, the pursuit of pro-active community engagement to attract knowledgeable staff as well as increase local buy-in was considered pivotal at all three sites. These findings underscore the need to attend to the external setting as well as to internal program operations.

Getting to the table: Agency characteristics and evidence-based intervention adoption in children's mental health care

Scaling evidence-based interventions (EBI) for children and families across healthcare systems can expand public health impact. Research has identified EBI adoption determinants. However, less understood are characteristics of agencies that opt in across the stages of adoption. This study examined the relationship between agency (N=69) characteristics (e.g., revenue) and four adoption stages during a large-scale trial of an EBI for children with significant behavioral difficulties and their families. 48 (70%) of agencies demonstrated interest, 28 (41%) scheduled an informational meeting, 20 (29%) received training, and 16 (22%) demonstrated EBI uptake. Analyses indicated no differences in characteristics and initial interest. However, agencies with small-sized revenue had significantly reduced odds at other adoption stages. Implications for strategies to bring EBI access to scale are discussed.

Advancing the use of organization theory in implementation science

Healthcare settings and systems have been slow to adopt and implement many effective cancer prevention and control interventions. Understanding the factors that determine successful implementation is essential to accelerating the translation of effective interventions into practice. Many scholars have studied the determinants of implementation, and much of this research has been guided by the Consolidated Framework for Implementation Research (CFIR). The CFIR categorizes implementation determinants at five levels (characteristics of the intervention, inner setting, individual, processes, and outer setting). Of these five levels, determinants at the level of the outer setting are the least developed. Extensive research in fields other than healthcare suggest that determinants at the level of the outer setting (e.g., funding streams, contracting practices, and public policy) play a central role in shaping when and how an organization implements new structures and practices. Thus, a more comprehensive understanding of outer-setting determinants is critical to efforts to accelerate the implementation of effective cancer control interventions. The Cancer Prevention and Control Research Network (CPCRN) created a cross-center workgroup to review organizational theories and begin to contribute to the creation of a future framework of constructs related to outer setting determinants. In this paper, we report findings from the review of three organizational theories: Institutional Theory, Transaction Cost Economics, and Contingency Theory. To demonstrate the applicability of this work to implementation science and practice, we have applied findings to three case studies of CPCRN researchers' efforts to implement colorectal cancer screening interventions in Federally Qualified Health Centers.

Elucidating the influence of supervisors' roles on implementation climate

Background

Supervisors play an essential role in implementation by diffusing and synthesizing information, selling implementation, and translating top management’s project plans to frontline workers. Theory and emerging evidence suggest that through these roles, supervisors shape implementation climate—i.e., the degree to which innovations are expected, supported, and rewarded. However, it is unclear exactly how supervisors carry out each of these roles in ways that contribute to implementation climate—this represents a gap in the understanding of the causal mechanisms that link supervisors’ behavior with implementation climate. This study examined how supervisors’ performance of each of these roles influences three core implementation climate domains (expectations, supports, and rewards).

Materials and methods

A sequenced behavioral health screening, assessment, and referral intervention was implemented within a county-based child welfare agency. We conducted 6 focus groups with supervisors and frontline workers from implementing work units 6 months post-implementation (n = 51) and 1 year later (n = 40) (12 groups total). Participants were asked about implementation determinants, including supervision and implementation context. We audio-recorded, transcribed, and analyzed focus groups using an open coding process during which the importance of the supervisors’ roles emerged as a major theme. We further analyzed this code using concepts and definitions related to middle managers’ roles and implementation climate.

Results

In this work setting, supervisors (1) diffused information about the intervention proactively, and in response to workers’ questions, (2) synthesized information by tailoring it to workers’ individual needs, (3) translated top managements’ project plans into day-to-day tasks through close monitoring and reminders, and (4) justified implementation. All four of these roles appeared to shape the implementation climate by conveying strong expectations for implementation. Three roles (diffusing, synthesizing, and mediating) influenced climate by supporting workers during implementation. Only one role (diffusing) influenced climate by conveying rewards.

Conclusions

Supervisors shaped implementation climate by carrying out four roles (diffusing, synthesizing, mediating, and selling). Findings suggest that the interaction of these roles convey expectations and support for implementation (two implementation climate domains). Our study advances the causal theory explaining how supervisors’ behavior shapes the implementation climate, which can inform implementation practice.

The Availability of Supervision in Routine Mental Health Care

Clinical supervision is an embedded resource for practice quality in community mental health organizations. Supervision has been found to increase provider competence and decrease stress. In addition, supervision has been associated with service user outcomes including decreased depressive symptoms. However, little is known about the availability and nature of supervision in real world settings. The primary aims of this study were to identify available supervision and the extent to which contextual factors are related to that availability. The data source for this study was a multi-state and multi-site (N=14) NIMH-funded trial survey of providers (N=273). Supervision was measured by hours per week (quantity) and by utilization of best practice activities (content). Univariate, chi-square, independent samples t-tests, and ANOVA analyses were used to assess supervision content and quantity and to examine subgroup differences. Participants reported an average of 2.17 hours of supervision per week and 28.6% of participants endorsed best practice content. Supervision quantity varied significantly across sites (p<.05) and program type (p<.05) while content did not. Individual role within the organization had a significant relationship with reported supervision content (p<.001). In these settings, organizations are exercising discretion in how to utilize supervision within the available time. Supervision time also varied by program type, increasing with the intensity of services. Findings demonstrate that reports of availability vary according to position within the organization and the intensity of services within a given program type. Implications for workforce development, access to quality services, and implementation of evidence-based practices are discussed.

Unpacking Clinical Supervision in Transitional and Permanent Supportive Housing: Scrutiny or Support?

Behavioral health organizations use clinical supervision to ensure professional development and practice quality. This qualitative study examined 35 service coordinators' perspectives on supervision in two distinct supportive housing program types (permanent and transitional). Thematic analysis of in-depth interviews yielded three contrast themes: support versus scrutiny, planned versus impromptu time, and housing first versus treatment first. Supervisory content and format resulted in differential perceptions of supervision, thereby influencing opportunities for learning. These findings suggest that unpacking discrete elements of supervision enactment in usual care settings can inform implementation of recovery-oriented practice.

Sorting it out: Eliciting consumer priorities for recovery in supportive housing

OBJECTIVE

This study aims to understand participant priorities in their personal recovery journey and their perspectives of recovery domains.

METHODS

A card sort data gathering technique was employed to elicit priorities in recovery from consumers in supportive housing programs serving formerly homeless adults with severe mental illnesses in New York City. Participants (N=38) were asked to sort 12 cards printed with recovery domains in order of importance and describe the meaning attached to each domain.

RESULTS

Mental health (95%), physical health (89%), and housing (92%) were the domains most frequently included and prioritized in the top three rankings. Family (76%) and partner (74%) were also frequently included and endorsed as most important second only to mental health. Housing was prioritized yet rated most important less often (58%). Work, school, hobbies, program, friends and neighborhood were less frequently endorsed. 'Card sort talk' revealed critical understanding of participants' priorities and their reasons for endorsing other domains less frequently.

CONCLUSIONS

Most important to participants was regaining functional independence through improved mental and physical health and access to housing. With underlying principles of efficiency and empowerment, card sort is a promising engagement technique for providers to elicit consumer priorities in their own recovery.

A comparison of how behavioral health organizations utilize training to prepare for health care reform

BACKGROUND

Under the Affordable Care Act, States have obtained Medicaid waivers to overhaul their behavioral health service systems to improve quality and reduce costs. Critical to implementation of broad service delivery reforms has been the preparation of organizations responsible for service delivery. This study focused on one large-scale initiative to overhaul its service system with the goal of improving service quality and reducing costs. The study examined the participation of behavioral health organizations in technical assistance efforts and the extent to which organizational factors related to their participation.

METHODS

This study matched two datasets to examine the organizational characteristics and training participation for 196 behavioral health organizations. Organizational characteristics were drawn from the Substance Abuse and Mental Health Services Administration National Mental Health Services Survey (N-MHSS). Training variables were drawn from the Clinical Technical Assistance Center's master training database. Chi-square analyses and multivariate logistic regression models were used to examine the proportion of organizations that participated in training, the organizational characteristics (size, population served, service quality, infrastructure) that predicted participation in training, and for those who participated, the type (clinical or business) and intensity of training (webinar, learning collaborative, in-person) they received.

RESULTS

Overall 142 (72. 4%) of the sample participated in training. Organizations who pursued training were more likely to be large in size (p = .02), serve children in addition to adults (p < .01), provide child evidence-based practices (p = .01), and use computerized scheduling (p = .01). Of those trained, 95% participated in webinars, 64% participated in learning collaboratives and 35% participated in in-person trainings. More organizations participated in business trainings than clinical (63.8 vs. 59.2%). Organizations serving children had higher odds of participating in both clinical training (OR = 5.91, p < .01) and business training (OR = 4.24, p < .01) than those that did not serve children.

CONCLUSIONS

The majority of organizations participated in trainings indicating desire for technical assistance to prepare for health care reform. Larger organizations and organizations serving children were more likely to participate potentially indicating increased interest in preparation. Over half participated in business trainings highlighting interest in learning to improve efficiency. Further understanding is needed to support organizational readiness for health care reform initiatives among behavioral health organizations.

I have my own lease-So why the service plan again? Perspectives on service planning in supportive housing

OBJECTIVE

This study aimed to understand multiple stakeholder perspectives implementing a recovery-oriented approach to service planning in supportive housing programs serving people with lived experience of mental illnesses.

METHOD

Multiple stakeholders (N = 57) were recruited to participate in focus groups (N = 8), including 4 with tenants, 2 with service coordinators, 1 with supervisors, and 1 with leadership. Supportive housing programs were purposively sampled from a recovery-oriented organization serving 1,500 people annually. Stakeholders' experiences with service planning and implementing a recovery-oriented approach to service planning were explored. The authors conducted inductive thematic analyses combined with a conceptual matrix, which yielded themes across and within multiple stakeholder focus groups.

RESULTS

Three themes emerged: (a) an institutional reminder-service planning experiences elicited negative emotions and served to remind people of experiences in institutional settings, (b) one-size-fits-all service planning-stakeholders perceived the use of quality assurance tools within the planning process as rigid to others' interests beyond their own, and (c) rules and regulations-reconciling funder requirements (e.g., completion dates) while also tailoring services to tenants' particular situations challenged providers.

CONCLUSIONS AND IMPLICATIONS FOR PRACTICE

Even in a recovery-oriented organization, findings suggest that service planning in supportive housing has limitations in responding to each tenant's iterative recovery process. Further, in this context where people can make their home, stakeholders questioned whether the very presence of ongoing service planning activities is problematic. However, tenant-service coordinator relationships predicated on mutual respect and esteem overcame some service planning limitations. (PsycINFO Database Record

Case Manager Perspectives on the Role of Treatment in Supportive Housing for People with Severe Mental Illness

OBJECTIVE

Growing recognition exists of housing as a social determinant of health, and thus, health care reform initiatives are expanding the reach of health care beyond traditional settings. One result of this expansion is increased Medicaid funds for supportive-housing programs for people with severe mental illnesses. This qualitative study explores the ways in which case managers working in a supportive housing program approach treatment and how their approach is influenced by both program requirements and their beliefs about mental illness.

METHOD

The study is part of a longitudinal qualitative study on recovery for people with severe mental illnesses living in supportive housing. Multiple interviews (n = 55) with 24 case managers from a residential-continuum supportive-housing program were conducted over 18 months. To provide an in-depth view of case manager perspectives, the study uses thematic analysis with multiple coders.

RESULTS

Overall, case managers understand supportive housing as being a treatment program but predominantly characterize treatment as medication management. The following themes emerged: believing medication to be the key to success in the program, persuading residents to take medication, and questioning the utility of the program for residents who were not medication adherent.

CONCLUSIONS

Case managers understand supportive housing to be a treatment program; however, given the external constraints and their own beliefs about mental illness, case managers often equate treatment with taking medication. Study findings demonstrate the need to train case managers about mental health recovery and integrated health care. The findings also have implications for policies that tie housing to services.

Proceedings of the 3rd Biennial Conference of the Society for Implementation Research Collaboration (SIRC) 2015: advancing efficient methodologies through community partnerships and team science : Seattle, WA, USA. 24-26 September 2015

Introduction to the 3rd Biennial Conference of the Society for Implementation Research Collaboration: advancing efficient methodologies through team science and community partnerships

Cara Lewis, Doyanne Darnell, Suzanne Kerns, Maria Monroe-DeVita, Sara J. Landes, Aaron R. Lyon, Cameo Stanick, Shannon Dorsey, Jill Locke, Brigid Marriott, Ajeng Puspitasari, Caitlin Dorsey, Karin Hendricks, Andria Pierson, Phil Fizur, Katherine A. Comtois

A1: A behavioral economic perspective on adoption, implementation, and sustainment of evidence-based interventions

Lawrence A. Palinkas

A2: Towards making scale up of evidence-based practices in child welfare systems more efficient and affordable

Patricia Chamberlain

A3: Mixed method examination of strategic leadership for evidence-based practice implementation

Gregory A. Aarons, Amy E. Green, Mark. G. Ehrhart, Elise M. Trott, Cathleen E. Willging

A4: Implementing practice change in Federally Qualified Health Centers: Learning from leaders’ experiences

Maria E. Fernandez, Nicholas H. Woolf, Shuting (Lily) Liang, Natalia I. Heredia, Michelle Kegler, Betsy Risendal, Andrea Dwyer, Vicki Young, Dayna Campbell, Michelle Carvalho, Yvonne Kellar-Guenther

A3: Mixed method examination of strategic leadership for evidence-based practice implementation

Gregory A. Aarons, Amy E. Green, Mark. G. Ehrhart, Elise M. Trott, Cathleen E. Willging

A4: Implementing practice change in Federally Qualified Health Centers: Learning from leaders’ experiences

Maria E. Fernandez, Nicholas H. Woolf, Shuting (Lily) Liang, Natalia I. Heredia, Michelle Kegler, Betsy Risendal, Andrea Dwyer, Vicki Young, Dayna Campbell, Michelle Carvalho, Yvonne Kellar-Guenther

A5: Efficient synthesis: Using qualitative comparative analysis and the Consolidated Framework for Implementation Research across diverse studies

Laura J. Damschroder, Julie C. Lowery

A6: Establishing a veterans engagement group to empower patients and inform Veterans Affairs (VA) health services research

Sarah S. Ono, Kathleen F. Carlson, Erika K. Cottrell, Maya E. O’Neil, Travis L. Lovejoy

A7: Building patient-practitioner partnerships in community oncology settings to implement behavioral interventions for anxious and depressed cancer survivors

Joanna J. Arch, Jill L. Mitchell

A8: Tailoring a Cognitive Behavioral Therapy implementation protocol using mixed methods, conjoint analysis, and implementation teams

Cara C. Lewis, Brigid R. Marriott, Kelli Scott

A9: Wraparound Structured Assessment and Review (WrapSTAR): An efficient, yet comprehensive approach to Wraparound implementation evaluation

Jennifer Schurer Coldiron, Eric J. Bruns, Alyssa N. Hook

A10: Improving the efficiency of standardized patient assessment of clinician fidelity: A comparison of automated actor-based and manual clinician-based ratings

Benjamin C. Graham, Katelin Jordan

A11: Measuring fidelity on the cheap

Rochelle F. Hanson, Angela Moreland, Benjamin E. Saunders, Heidi S. Resnick

A12: Leveraging routine clinical materials to assess fidelity to an evidence-based psychotherapy

Shannon Wiltsey Stirman, Cassidy A. Gutner, Jennifer Gamarra, Dawne Vogt, Michael Suvak, Jennifer Schuster Wachen, Katherine Dondanville, Jeffrey S. Yarvis, Jim Mintz, Alan L. Peterson, Elisa V. Borah, Brett T. Litz, Alma Molino, Stacey Young McCaughanPatricia A. Resick

A13: The video vignette survey: An efficient process for gathering diverse community opinions to inform an intervention

Nancy Pandhi, Nora Jacobson, Neftali Serrano, Armando Hernandez, Elizabeth Zeidler- Schreiter, Natalie Wietfeldt, Zaher Karp

A14: Using integrated administrative data to evaluate implementation of a behavioral health and trauma screening for children and youth in foster care

Michael D. Pullmann, Barbara Lucenko, Bridget Pavelle, Jacqueline A. Uomoto, Andrea Negrete, Molly Cevasco, Suzanne E. U. Kerns

A15: Intermediary organizations as a vehicle to promote efficiency and speed of implementation

Robert P. Franks, Christopher Bory

A16: Applying the Consolidated Framework for Implementation Research constructs directly to qualitative data: The power of implementation science in action

Edward J. Miech, Teresa M. Damush

A17: Efficient and effective scaling-up, screening, brief interventions, and referrals to treatment (SBIRT) training: a snowball implementation model

Jason Satterfield, Derek Satre, Maria Wamsley, Patrick Yuan, Patricia O’Sullivan

A18: Matching models of implementation to system needs and capacities: addressing the human factor

Helen Best, Susan Velasquez

A19: Agency characteristics that facilitate efficient and successful implementation efforts

Miya Barnett, Lauren Brookman-Frazee, Jennifer Regan, Nicole Stadnick, Alison Hamilton, Anna Lau

A20: Rapid assessment process: Application to the Prevention and Early Intervention transformation in Los Angeles County

Jennifer Regan, Alison Hamilton, Nicole Stadnick, Miya Barnett, Anna Lau, Lauren Brookman-Frazee

A21: The development of the Evidence-Based Practice-Concordant Care Assessment: An assessment tool to examine treatment strategies across practices

Nicole Stadnick, Anna Lau, Miya Barnett, Jennifer Regan, Scott Roesch, Lauren Brookman-Frazee

A22: Refining a compilation of discrete implementation strategies and determining their importance and feasibility

Byron J. Powell, Thomas J. Waltz, Matthew J. Chinman, Laura Damschroder, Jeffrey L. Smith, Monica M. Matthieu, Enola K. Proctor, JoAnn E. Kirchner

A23: Structuring complex recommendations: Methods and general findings

Thomas J. Waltz, Byron J. Powell, Matthew J. Chinman, Laura J. Damschroder, Jeffrey L. Smith, Monica J. Matthieu, Enola K. Proctor, JoAnn E. Kirchner

A24: Implementing prolonged exposure for post-traumatic stress disorder in the Department of Veterans Affairs: Expert recommendations from the Expert Recommendations for Implementing Change (ERIC) project

Monica M. Matthieu, Craig S. Rosen, Thomas J. Waltz, Byron J. Powell, Matthew J. Chinman, Laura J. Damschroder, Jeffrey L. Smith, Enola K. Proctor, JoAnn E. Kirchner

A25: When readiness is a luxury: Co-designing a risk assessment and quality assurance process with violence prevention frontline workers in Seattle, WA

Sarah C. Walker, Asia S. Bishop, Mariko Lockhart

A26: Implementation potential of structured recidivism risk assessments with justice- involved veterans: Qualitative perspectives from providers

Allison L. Rodriguez, Luisa Manfredi, Andrea Nevedal, Joel Rosenthal, Daniel M. Blonigen

A27: Developing empirically informed readiness measures for providers and agencies for the Family Check-Up using a mixed methods approach

Anne M. Mauricio, Thomas D. Dishion, Jenna Rudo-Stern, Justin D. Smith

A28: Pebbles, rocks, and boulders: The implementation of a school-based social engagement intervention for children with autism

Jill Locke, Courtney Benjamin Wolk, Colleen Harker, Anne Olsen, Travis Shingledecker, Frances Barg, David Mandell, Rinad S. Beidas

A29: Problem Solving Teletherapy (PST.Net): A stakeholder analysis examining the feasibility and acceptability of teletherapy in community based aging services

Marissa C. Hansen, Maria P. Aranda, Isabel Torres-Vigil

A30: A case of collaborative intervention design eventuating in behavior therapy sustainment and diffusion

Bryan Hartzler

A31: Implementation of suicide risk prevention in an integrated delivery system: Mental health specialty services

Bradley Steinfeld, Tory Gildred, Zandrea Harlin, Fredric Shephard

A32: Implementation team, checklist, evaluation, and feedback (ICED): A step-by-step approach to Dialectical Behavior Therapy program implementation

Matthew S. Ditty, Andrea Doyle, John A. Bickel III, Katharine Cristaudo

A33: The challenges in implementing muliple evidence-based practices in a community mental health setting

Dan Fox, Sonia Combs

A34: Using electronic health record technology to promote and support evidence-based practice assessment and treatment intervention

David H. Lischner

A35: Are existing frameworks adequate for measuring implementation outcomes? Results from a new simulation methodology

Richard A. Van Dorn, Stephen J. Tueller, Jesse M. Hinde, Georgia T. Karuntzos

A36: Taking global local: Evaluating training of Washington State clinicians in a modularized cogntive behavioral therapy approach designed for low-resource settings

Maria Monroe-DeVita, Roselyn Peterson, Doyanne Darnell, Lucy Berliner, Shannon Dorsey, Laura K. Murray

A37: Attitudes toward evidence-based practices across therapeutic orientations

Yevgeny Botanov, Beverly Kikuta, Tianying Chen, Marivi Navarro-Haro, Anthony DuBose, Kathryn E. Korslund, Marsha M. Linehan

A38: Predicting the use of an evidence-based intervention for autism in birth-to-three programs

Colleen M. Harker, Elizabeth A. Karp, Sarah R. Edmunds, Lisa V. Ibañez, Wendy L. Stone

A39: Supervision practices and improved fidelity across evidence-based practices: A literature review

Mimi Choy-Brown

A40: Beyond symptom tracking: clinician perceptions of a hybrid measurement feedback system for monitoring treatment fidelity and client progress

Jack H. Andrews, Benjamin D. Johnides, Estee M. Hausman, Kristin M. Hawley

A41: A guideline decision support tool: From creation to implementation

Beth Prusaczyk, Alex Ramsey, Ana Baumann, Graham Colditz, Enola K. Proctor

A42: Dabblers, bedazzlers, or total makeovers: Clinician modification of a common elements cognitive behavioral therapy approach

Rosemary D. Meza, Shannon Dorsey, Shannon Wiltsey-Stirman, Georganna Sedlar, Leah Lucid

A43: Characterization of context and its role in implementation: The impact of structure, infrastructure, and metastructure

Caitlin Dorsey, Brigid Marriott, Nelson Zounlome, Cara Lewis

A44: Effects of consultation method on implementation of cognitive processing therapy for post-traumatic stress disorder

Cassidy A. Gutner, Candice M. Monson, Norman Shields, Marta Mastlej, Meredith SH Landy, Jeanine Lane, Shannon Wiltsey Stirman

A45: Cross-validation of the Implementation Leadership Scale factor structure in child welfare service organizations

Natalie K. Finn, Elisa M. Torres, Mark. G. Ehrhart, Gregory A. Aarons

A46: Sustainability of integrated smoking cessation care in Veterans Affairs posttraumatic stress disorder clinics: A qualitative analysis of focus group data from learning collaborative participants

Carol A. Malte, Aline Lott, Andrew J. Saxon

A47: Key characteristics of effective mental health trainers: The creation of the Measure of Effective Attributes of Trainers (MEAT)

Meredith Boyd, Kelli Scott, Cara C. Lewis

A48: Coaching to improve teacher implementation of evidence-based practices (EBPs)

Jennifer D. Pierce

A49: Factors influencing the implementation of peer-led health promotion programs targeting seniors: A literature review

Agathe Lorthios-Guilledroit, Lucie Richard, Johanne Filiatrault

A50: Developing treatment fidelity rating systems for psychotherapy research: Recommendations and lessons learned

Kevin Hallgren, Shirley Crotwell, Rosa Muñoz, Becky Gius, Benjamin Ladd, Barbara McCrady, Elizabeth Epstein

A51: Rapid translation of alcohol prevention science

John D. Clapp, Danielle E. Ruderman

A52: Factors implicated in successful implementation: evidence to inform improved implementation from high and low-income countries

Melanie Barwick, Raluca Barac, Stanley Zlotkin, Laila Salim, Marnie

Davidson

A53: Tracking implementation strategies prospectively: A practical approach

Alicia C. Bunger, Byron J. Powell, Hillary A. Robertson

A54: Trained but not implementing: the need for effective implementation planning tools

Christopher Botsko

A55: Evidence, context, and facilitation variables related to implementation of Dialectical Behavior Therapy: Qualitative results from a mixed methods inquiry in the Department of Veterans Affairs

Sara J. Landes, Brandy N. Smith, Allison L. Rodriguez, Lindsay R. Trent, Monica M. Matthieu

A56: Learning from implementation as usual in children’s mental health

Byron J. Powell, Enola K. Proctor

A57: Rates and predictors of implementation after Dialectical Behavior Therapy Intensive Training

Melanie S. Harned, Marivi Navarro-Haro, Kathryn E. Korslund, Tianying Chen, Anthony DuBose, André Ivanoff, Marsha M. Linehan

A58: Socio-contextual determinants of research evidence use in public-youth systems of care

Antonio R. Garcia, Minseop Kim, Lawrence A. Palinkas, Lonnie Snowden, John Landsverk

A59: Community resource mapping to integrate evidence-based depression treatment in primary care in Brazil: A pilot project

Annika C. Sweetland, Maria Jose Fernandes, Edilson Santos, Cristiane Duarte, Afrânio Kritski, Noa Krawczyk, Caitlin Nelligan, Milton L. Wainberg

A60: The use of concept mapping to efficiently identify determinants of implementation in the National Institute of Health--President’s Emergent Plan for AIDS Relief Prevention of Mother to Child HIV Transmission Implementation Science Alliance

Gregory A. Aarons, David H. Sommerfeld, Benjamin Chi, Echezona Ezeanolue, Rachel Sturke, Lydia Kline, Laura Guay, George Siberry

A61: Longitudinal remote consultation for implementing collaborative care for depression

Ian M. Bennett, Rinad Beidas, Rachel Gold, Johnny Mao, Diane Powers, Mindy Vredevoogd, Jurgen Unutzer

A62: Integrating a peer coach model to support program implementation and ensure long- term sustainability of the Incredible Years in community-based settings

Jennifer Schroeder, Lane Volpe, Julie Steffen

A63: Efficient sustainability: Existing community based supervisors as evidence-based treatment supports

Shannon Dorsey, Michael D Pullmann, Suzanne E. U. Kerns, Nathaniel Jungbluth, Lucy Berliner, Kelly Thompson, Eliza Segell

A64: Establishment of a national practice-based implementation network to accelerate adoption of evidence-based and best practices

Pearl McGee-Vincent, Nancy Liu, Robyn Walser, Jennifer Runnals, R. Keith Shaw, Sara J. Landes, Craig Rosen, Janet Schmidt, Patrick Calhoun

A65: Facilitation as a mechanism of implementation in a practice-based implementation network: Improving care in a Department of Veterans Affairs post-traumatic stress disorder outpatient clinic

Ruth L. Varkovitzky, Sara J. Landes

A66: The ACT SMART Toolkit: An implementation strategy for community-based organizations providing services to children with autism spectrum disorder

Amy Drahota, Jonathan I. Martinez, Brigitte Brikho, Rosemary Meza, Aubyn C. Stahmer, Gregory A. Aarons

A67: Supporting Policy In Health with Research: An intervention trial (SPIRIT) - protocol and early findings

Anna Williamson

A68: From evidence based practice initiatives to infrastructure: Lessons learned from a public behavioral health system’s efforts to promote evidence based practices

Ronnie M. Rubin, Byron J. Powell, Matthew O. Hurford, Shawna L. Weaver, Rinad S. Beidas, David S. Mandell, Arthur C. Evans

A69: Applying the policy ecology model to Philadelphia’s behavioral health transformation efforts

Byron J. Powell, Rinad S. Beidas, Ronnie M. Rubin, Rebecca E. Stewart, Courtney Benjamin Wolk, Samantha L. Matlin, Shawna Weaver, Matthew O. Hurford, Arthur C. Evans, Trevor R. Hadley, David S. Mandell

A70: A model for providing methodological expertise to advance dissemination and implementation of health discoveries in Clinical and Translational Science Award institutions

Donald R. Gerke, Beth Prusaczyk, Ana Baumann, Ericka M. Lewis, Enola K. Proctor

A71: Establishing a research agenda for the Triple P Implementation Framework

Jenna McWilliam, Jacquie Brown, Michelle Tucker

A72: Cheap and fast, but what is “best?”: Examining implementation outcomes across sites in a state-wide scaled-up evidence-based walking program, Walk With Ease

Kathleen P Conte

A73: Measurement feedback systems in mental health: Initial review of capabilities and characteristics

Aaron R. Lyon, Meredith Boyd, Abigail Melvin, Cara C. Lewis, Freda Liu, Nathaniel Jungbluth

A74: A qualitative investigation of case managers’ attitudes toward implementation of a measurement feedback system in a public mental health system for youth

Amelia Kotte, Kaitlin A. Hill, Albert C. Mah, Priya A. Korathu-Larson, Janelle R. Au, Sonia Izmirian, Scott Keir, Brad J. Nakamura, Charmaine K. Higa-McMillan

A75: Multiple pathways to sustainability: Using Qualitative Comparative Analysis to uncover the necessary and sufficient conditions for successful community-based implementation

Brittany Rhoades Cooper, Angie Funaiole, Eleanor Dizon

A76: Prescribers’ perspectives on opioids and benzodiazepines and medication alerts to reduce co-prescribing of these medications

Eric J. Hawkins, Carol A. Malte, Hildi J. Hagedorn, Douglas Berger, Anissa Frank, Aline Lott, Carol E. Achtmeyer, Anthony J. Mariano, Andrew J. Saxon

A77: Adaptation of Coordinated Anxiety Learning and Management for comorbid anxiety and substance use disorders: Delivery of evidence-based treatment for anxiety in addictions treatment centers

Kate Wolitzky-Taylor, Richard Rawson, Richard Ries, Peter Roy-Byrne, Michelle Craske

A78: Opportunities and challenges of measuring program implementation with online surveys

Dena Simmons, Catalina Torrente, Lori Nathanson, Grace Carroll

A79: Observational assessment of fidelity to a family-centered prevention program: Effectiveness and efficiency

Justin D. Smith, Kimbree Brown, Karina Ramos, Nicole Thornton, Thomas J. Dishion, Elizabeth A. Stormshak, Daniel S. Shaw, Melvin N. Wilson

A80: Strategies and challenges in housing first fidelity: A multistate qualitative analysis

Mimi Choy-Brown, Emmy Tiderington, Bikki Tran Smith, Deborah K. Padgett

A81: Procurement and contracting as an implementation strategy: Getting To Outcomes® contracting

Ronnie M. Rubin, Marilyn L. Ray, Abraham Wandersman, Andrea Lamont, Gordon Hannah, Kassandra A. Alia, Matthew O. Hurford, Arthur C. Evans

A82: Web-based feedback to aid successful implementation: The interactive Stages of Implementation Completion (SIC)^TM tool

Lisa Saldana, Holle Schaper, Mark Campbell, Patricia Chamberlain

A83: Efficient methodologies for monitoring fidelity in routine implementation: Lessons from the Allentown Social Emotional Learning Initiative

Valerie B. Shapiro, B.K. Elizabeth Kim, Jennifer L. Fleming, Paul A. LeBuffe

A84: The Society for Implementation Research Collaboration (SIRC) implementation development workshop: Results from a new methodology for enhancing implementation science proposals

Sara J. Landes, Cara C. Lewis, Allison L. Rodriguez, Brigid R. Marriott, Katherine Anne Comtois

A85: An update on the Society for Implementation Research Collaboration (SIRC) Instrument Review Project

Trajectories of Recovery Among Formerly Homeless Adults With Serious Mental Illness

OBJECTIVE

Recovery from mental illness is possible, but individuals with co-occurring disorders and homelessness face challenges. Although a nonlinear recovery course is assumed, few studies have analyzed recovery over time. This mixed-methods study examined recovery trajectories over 18 months after enrollment in supportive housing programs of 38 participants with DSM axis I diagnoses.

METHODS

Qualitative interview data were quantified through consensual ratings to generate a recovery score for four waves of data collection based on eight recovery domains culled from the literature. Case study analyses were conducted of participants whose scores varied by one standard deviation or more between baseline and 18 months to identify which domains were important.

RESULTS

Most of the 38 participants (N=23) had no significant change in recovery; seven had a negative trajectory, and eight had a positive trajectory. Case studies of these 15 participants indicated domains that contributed to change: significant-other relationships (N=9), engagement in meaningful activities (N=9), mental health (N=7), family relationships (N=6), general medical health (N=5), housing satisfaction (N=5), employment (N=2), and substance use (N=1). Except for mental health and substance use (which contributed only to negative trajectories), the influence of domains was both positive and negative. Domains were intertwined; for example, variation in relationships was linked to changes in meaningful activities.

CONCLUSIONS

This study showed little change in recovery over time for most participants and a decline in mental health for a small minority. Findings underscore the importance of social relationships and meaningful activities among individuals with serious mental illness, who experience complex challenges.

Person-centered care planning and service engagement: a study protocol for a randomized controlled trial

Background

Service disengagement is a pervasive challenge the mental health care system faces. Mental health services are of little value should persons with mental illnesses continue to opt out of receiving them. Consumers attribute disengagement from care to an absence of choice in their treatment. In response, the mental health system is adopting a person-centered model, based upon recovery principles, to engage consumers more actively in their care. Person-centered care planning is a promising practice involving collaboration to develop and implement an actionable plan to assist the person in achieving personal recovery goals.

Methods/design

This study design combines a parallel-group randomized controlled trial of community mental health organizations with qualitative methods to assess the effectiveness of person-centered care planning. Participants at 14 sites in Delaware and Connecticut will be randomized to treatment as usual or the person-centered care planning intervention. Participants will be in leadership (n = 70) or supervisory or direct care (n = 210) roles. The person-centered care planning intervention involves intensive staff training and 12 months of ongoing technical assistance. Quantitative survey data will be collected at baseline, 6 months and 12 months measuringperson-centered care planning competency and organizational factors. Consumer outcomes (engagement, medication adherence, functioning and consumer satisfaction) will be assessed by Medicaid and state-level data. Qualitative data focused on process factors will include staff and consumer interviews and focus groups. In this intent-to-treat analysis, we will use mixed-effects multivariate regression models to evaluate the differential impact of the person-centered care planning intervention on each consumer and implementation outcome as well as the extent to which clinician assessments of organizational factors are associated with the implementation outcome. Mixed methods will triangulate and strengthen the interpretation of outcomes.

Discussion

The aim of this study is to generate valuable guidance for state systems engaged in scale-up and transformation efforts. Targeted staff selection for training to support sustainability will serve to provide further insight into important intervention implementation strategies. Person-centered care planning has the potential to enhance the impact of all evidence-based and recovery-oriented practices and bring practice into line with the emerging national guidelines in health care reform.

Trial registration

This trial was registered with ClinicalTrials.gov (Identifier: NCT02299492) on 21 November 2014 as New York University Protocol Record PCCP-13-9762, Person-Centered Care Planning and Service Engagement.