A Rasch model analysis of evidence-based treatment practices used in the criminal justice system

Implementation Strategy Fidelity and Organizational Outcomes in a Randomized Trial: Implications for Reentry from Corrections to Community Treatment

Background: This study examined fidelity of implementation strategies used in an organizational process improvement intervention (OPII) designed to improve evidence-based practices related to assessments for drug-involved clients exiting incarceration. Leadership was studied as a moderating factor between fidelity and outcomes. Methods: A mixed-method cluster randomized design was used to randomize 21 sites to early- or delayed-start within 9 research centers. Parent study protocol was reviewed to develop fidelity constructs (i.e., responsiveness, dose, quality, adherence). Outcomes were site-level success in achieving goals and objectives completed during the OPII (e.g., percent goals achieved). Correlations, analyses of covariance, regressions and moderation analyses were performed. Qualitative interviews assessed facilitators/barriers to implementation. Results: Fidelity constructs related to outcomes. No differences were found in fidelity by early or delayed condition. At low levels of leadership, high staff responsiveness (i.e., engagement in the OPII) related to poorer outcome. Conclusions: It is important to consider contextual factors (e.g., leadership) that may influence implementation strategy fidelity when deploying evidence-based practices. Findings are relevant to researchers, clinicians, administrators and policy makers, and suggest that goal completion during implementation of evidence-based practices requires monitoring of leadership competence, fidelity to implementation strategies (i.e., staff responsiveness to strategies) and attendance to goal importance.

Evidence-Based Practices and Use among Employees and Students at an Austrian Medical University

Developed in the pre-internet era in the early 1980s, empirical medical practice, i.e., evidence-based practice (EBP) has become crucial in critical thinking and statistical reasoning at the point-of-care. As little evidence is available so far on how EBP is perceived in the Austrian academic context, we conducted a cross-sectional online survey among a nonrandom purposive sample of employees and students at the Medical University Vienna, Austria (total n = 1247, 59.8% females). The German questionnaire assessed both EBP capability beliefs and EBP use, with the respective indices both yielding good internal consistency. We conducted subgroup comparisons between employees (n = 638) and students (n = 609). In line with Bandura's self-efficacy theory, we found a correlation between EBP capability beliefs and EBP use, with higher scores reported in the employee group. The results indicated that the participants did not strictly follow the sequential EBP steps as grounded in the item-response theory. Since its emergence, EBP has struggled to overcome the dominating traditional way of conducting medicine, which is also known as eminence-based medicine, where ad hoc decisions are based upon expert opinions, and nowadays frequently supplemented by quick online searches. Medical staff and supervisors of medical students should be aware of the existing overlaps and synergies of these potentially equivalent factors in clinical care. There is a need for intensifying the public and scientific debate on how to deal with the divergence between EBP theory and EBP practice.

Protocol for a quasi-experimental, 950 county study examining implementation outcomes and mechanisms of Stepping Up, a national policy effort to improve mental health and substance use services for justice-involved individuals

BACKGROUND

The criminal justice system is the largest provider of mental health services in the USA. Many jurisdictions are interested in reducing the use of the justice system for mental health problems. The national Stepping Up Initiative helps agencies within counties work together more effectively to reduce the number of individuals with mental illness in jails and to improve access to mental health services in the community. This study will compare Stepping Up counties to matched comparison counties over time to (1) examine the effectiveness of Stepping Up and (2) test hypothesized implementation mechanisms to inform multi-agency implementation efforts more broadly.

METHODS

The study will survey 950 counties at baseline, 18 months, and 36 months in a quasi-experimental design comparing implementation mechanisms and outcomes between 475 Stepping Up counties and 475 matched comparison counties. Surveys will be sent to up to four respondents per county including administrators of jail, probation, community mental health services, and community substance use treatment services (3800 total respondents). We will examine whether Stepping Up counties show faster improvements in implementation outcomes (number of justice-involved clients receiving behavioral health services, number of behavioral health evidence-based practices and policies [EBPPs] available to justice-involved individuals, and resources for behavioral health EBPP for justice-involved individuals) than do matched comparison counties. We will also evaluate whether engagement of hypothesized mechanisms explains differences in implementation outcomes. Implementation target mechanisms include (1) use of and capacity for performance monitoring, (2) use and functioning of interagency teams, (3) common goals and mission across agencies, and (4) system integration (i.e., building an integrated system of care rather than adding one program or training). Finally, we will characterize implementation processes and critical incidents using survey responses and qualitative interviews.

DISCUSSION

There are few rigorous, prospective studies examining implementation mechanisms and their relationship with behavioral health implementation outcomes in justice and associated community behavioral health settings. There is also limited understanding of implementation mechanisms that occur across systems with multiple goals. This study will describe implementation outcomes of Stepping Up and will elucidate target mechanisms that are effective in multi-goal, multi-agency systems.

Organizational Context and Individual Adaptability in Promoting Perceived Importance and Use of Best Practices for Substance Use

This study examines associations among organizational context, staff attributes, perceived importance, and use of best practices among staff in community-based, juvenile justice (JJ) agencies. As part of the National Institute on Drug Abuse's Juvenile Justice-Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS) study, 492 staff from 36 JJ agencies were surveyed about the perceived importance and use of best practices within their organization in five substance use practice domains: screening, assessment, standard referral, active referral, and treatment support. Structural equation models indicated that supervisory encouragement and organizational innovation/flexibility were associated with greater individual adaptability. Adaptability (willingness to try new ideas, use new procedures, adjust quickly to change), was positively correlated with importance ratings. Importance ratings were positively associated with reported use of best practices. Organizational climates that support innovation likely affect use of practices through staff attributes and perceptions of the importance of such services.

CORRELATES OF INTERORGANIZATIONAL SERVICE COORDINATION IN COMMUNITY CORRECTIONS

Because weak interagency coordination between community correctional agencies (e.g., probation and parole) and community-based treatment providers has been identified as a major barrier to the use of evidence-based practices (EBPs) for treating druginvolved offenders, this study sought to examine how key organizational (e.g., leadership, support, staffing) and individual (e.g., burnout, satisfaction) factors influence interagency relationships between these agencies. At each of 20 sites, probation/parole officials (n = 366) and community treatment providers (n = 204) were surveyed about characteristics of their agencies, themselves, and interorganizational relationships with each other. Key organizational and individual correlates of interagency relationships were examined using hierarchical linear models (HLM) analyses, supplemented by interview data. The strongest correlates included Adaptability, Efficacy, and Burnout. Implications for policy and practice are discussed.

Effects of an Organizational Linkage Intervention on Inter-Organizational Service Coordination Between Probation/Parole Agencies and Community Treatment Providers

Weak coordination between community correctional agencies and community-based treatment providers is a major barrier to diffusion of medication-assisted treatment (MAT)--the inclusion of medications (e.g., methadone and buprenorphine) in combination with traditional counseling and behavioral therapies to treat substance use disorders. In a multisite cluster randomized trial, experimental sites (j = 10) received a 3-h MAT training plus a 12-month linkage intervention; control sites (j = 10) received the 3-h training alone. Hierarchical linear models showed that the intervention resulted in significant improvements in perceptions of interagency coordination among treatment providers, but not probation/parole agents. Implications for policy and practice are discussed.

Substance abuse and batterer programmes in California, USA: factors associated with treatment outcomes

The association between substance abuse and intimate partner violence is quite robust. A promising area to improve treatment for the dual problems of substance abuse and violence perpetration is the identification of client characteristics and organisational and programme factors as predictors of health outcomes. Therefore, we examined associations of client, organisational and programme factors with outcomes in community health settings. Directors of 241 substance use disorder programmes (SUDPs) and 235 batterer intervention programmes (BIPs) reported outcomes of programme completion and substance use and violence perpetration rates at discharge; data collection and processing were completed in 2012. SUDPs having more female, non-white, younger, uneducated, unemployed and lower income clients reported lower completion rates. In SUDPs, private, for-profit programmes reported higher completion rates than public or private, non-profit programmes. SUDPs with lower proportions of their budgets from government sources, and higher proportions from client fees, reported better outcomes. Larger SUDPs had poorer programme completion and higher substance use rates. Completion rates in SUDPs were higher when clients could obtain substance- and violence-related help at one location, and programmes integrated violence-prevention contracting into care. In BIPs, few client, organisational and programme factors were associated with outcomes, but the significant factors associated with programme completion were consistent with those for SUDPs. Publicly owned and larger programmes, and SUDPs lacking staff to integrate violence-related treatment, may be at risk of poorer client outcomes, but could learn from programmes that perform well to yield better outcomes.

Effects of a strategy to improve offender assessment practices: Staff perceptions of implementation outcomes

BACKGROUND

This implementation study examined the impact of an organizational process improvement intervention (OPII) on a continuum of evidence based practices related to assessment and community reentry of drug-involved offenders: Measurement/Instrumentation, Case Plan Integration, Conveyance/Utility, and Service Activation/Delivery.

METHODS

To assess implementation outcomes (staff perceptions of evidence-based assessment practices), a survey was administered to correctional and treatment staff (n=1509) at 21 sites randomly assigned to an Early- or Delayed-Start condition. Hierarchical linear models with repeated measures were used to examine changes in evidence-based assessment practices over time, and organizational characteristics were examined as covariates to control for differences across the 21 research sites.

RESULTS

Results demonstrated significant intervention and sustainability effects for three of the four assessment domains examined, although stronger effects were obtained for intra- than inter-agency outcomes. No significant effects were found for Conveyance/Utility.

CONCLUSIONS

Implementation interventions such as the OPII represent an important tool to enhance the use of evidence-based assessment practices in large and diverse correctional systems. Intra-agency assessment activities that were more directly under the control of correctional agencies were implemented most effectively. Activities in domains that required cross-systems collaboration were not as successfully implemented, although longer follow-up periods might afford detection of stronger effects.

Assessing Adolescent Substance Abuse Programs with Updated Quality Indicators: The Development of a Consumer Guide for Adolescent Treatment

When adolescent substance abuse requires treatment, few parents know which treatment features are important and which treatment programs are effective. There are few resources to help them select appropriate care. We describe early work on an evaluation method and comparative treatment guide for parents based upon the premise that the quality of a program and its potential effectiveness is a function of the number and frequency of evidence-based treatment practices (EBPs) delivered. Thus, we describe the development of and measurement approach for a set of EBPs toward the goal of developing a Consumer Guide to Adolescent Substance Abuse Treatment.

Dissemination and implementation of evidence-based practices for child and adolescent mental health: a systematic review

OBJECTIVE

Although there has been a dramatic increase in the number of evidence-based practices (EBPs) to improve child and adolescent mental health, the poor uptake of these EBPs has led to investigations of factors related to their successful dissemination and implementation. The purpose of this systematic review was to identify key findings from empirical studies examining the dissemination and implementation of EBPs for child and adolescent mental health.

METHOD

Of 14,247 citations initially identified, 73 articles drawn from 44 studies met inclusion criteria. The articles were classified by implementation phase (exploration, preparation, implementation, and sustainment) and specific implementation factors examined. These factors were divided into outer (i.e., system level) and inner (i.e., organizational level) contexts.

RESULTS

Few studies used true experimental designs; most were observational. Of the many inner context factors that were examined in these studies (e.g., provider characteristics, organizational resources, leadership), fidelity monitoring and supervision had the strongest empirical evidence. Albeit the focus of fewer studies, implementation interventions focused on improving organizational climate and culture were associated with better intervention sustainment as well as child and adolescent outcomes. Outer contextual factors such as training and use of specific technologies to support intervention use were also important in facilitating the implementation process.

CONCLUSIONS

The further development and testing of dissemination and implementation strategies is needed to more efficiently move EBPs into usual care.

Addressing substance abuse and violence in substance use disorder treatment and batterer intervention programs

BACKGROUND

Substance use disorders and perpetration of intimate partner violence (IPV) are interrelated, major public health problems.

METHODS

We surveyed directors of a sample of substance use disorder treatment programs (SUDPs; N=241) and batterer intervention programs (BIPs; N=235) in California (70% response rate) to examine the extent to which SUDPs address IPV, and BIPs address substance abuse.

RESULTS

Generally, SUDPs were not addressing co-occurring IPV perpetration in a formal and comprehensive way. Few had a policy requiring assessment of potential clients, or monitoring of admitted clients, for violence perpetration; almost one-quarter did not admit potential clients who had perpetrated IPV, and only 20% had a component or track to address violence. About one-third suspended or terminated clients engaging in violence. The most common barriers to SUDPs providing IPV services were that violence prevention was not part of the program's mission, staff lacked training in violence, and the lack of reimbursement mechanisms for such services. In contrast, BIPs tended to address substance abuse in a more formal and comprehensive way; e.g., one-half had a policy requiring potential clients to be assessed, two-thirds required monitoring of substance abuse among admitted clients, and almost one-half had a component or track to address substance abuse. SUDPs had clients with fewer resources (marriage, employment, income, housing), and more severe problems (both alcohol and drug use disorders, dual substance use and other mental health disorders, HIV + status). We found little evidence that services are centralized for individuals with both substance abuse and violence problems, even though most SUDP and BIP directors agreed that help for both problems should be obtained simultaneously in separate programs.

CONCLUSIONS

SUDPs may have difficulty addressing violence because they have a clientele with relatively few resources and more complex psychological and medical needs. However, policy change can modify barriers to treatment integration and service linkage, such as reimbursement restrictions and lack of staff training.

Organizational characteristics that foster early adoption of cultural and linguistic competence in outpatient substance abuse treatment in the United States

Recent years have seen an increased interest in developing culturally and linguistically responsive systems of care in substance abuse treatment in the United States. This study examines the extent to which external and internal organizational pressures contributed to the degree of adoption of culturally and linguistically responsive practices in the nation's outpatient substance abuse treatment system early in the period of development of this system of care. Findings show that a higher degree of adoption of culturally competent practices was most likely in treatment programs with high dependence on external funding and regulation. Internally, programs with a larger number of professionals were associated with the lowest degree of adoption, while managers' cultural sensitivity contributed significantly to a high degree of adoption of these responsive practices. Considering the passage of recent legislation enforcing the use of cultural and linguistic competence in health care, implications of these baseline findings on early adoption patterns are discussed for future research and health care policy evaluation.

Cultural competence in outpatient substance abuse treatment: measurement and relationship to wait time and retention

BACKGROUND

Culturally competent practice is broadly acknowledged to be an important strategy to increase the quality of services for racial/ethnic minorities in substance abuse treatment. However, few empirically derived measures of organizational cultural competence exist, and relatively little is known about how these measures affect treatment outcomes.

METHOD

Using a nationally representative sample of outpatient substance abuse treatment (OSAT) programs, this study used item response theory to create two measures of cultural competence-organizational practices and managers' culturally sensitive beliefs-and examined their relationship to client wait time and retention using Poisson regression modeling.

RESULTS

The most common and precisely measured organizational practices reported by OSAT managers included matching providers and clients based on language/dialect; offering cross-cultural training; and fostering connections with community and faith-based organizations connected to racial and ethnic minority groups. The most culturally sensitive belief among OSAT managers was support for language/dialect matching for racial and ethnic minority clients. Results of regression modeling indicate that organizational practices were not related to either outcome. However, managers' culturally sensitive beliefs were negatively associated with average wait time (p<0.05), and positively associated with average retention (p<0.01).

CONCLUSIONS

Managers' culturally sensitive beliefs-considered to be influential for effective implementation of culturally competent practices-may be particularly relevant in influencing wait time and retention in OSAT organizations that treat Latinos and African American clients.

Associations among state and local organizational contexts: Use of evidence-based practices in the criminal justice system

This study used hierarchical linear modeling (HLM) to examine the extent to which the organizational characteristics of state corrections agencies and local criminal justice facilities interacted in their associations with the extent to which local facilities are using evidence-based substance abuse treatment practices (EBPs). The study used data collected from two nationally representative surveys - one of state executives and the other of local prison wardens, justice administrators, and treatment directors - which were conducted as part of the National Criminal Justice Treatment Practices survey [NCJTP; Taxman, F.S., Young, D., Wiersema, B., Mitchell, S., Rhodes, A.G., 2007. The National Criminal Justice Treatment Practices Survey: Multi-level survey methods and procedures. J. Subst. Abuse Treat. 32, 225-238], and includes both adult criminal and juvenile justice samples. Results indicated that several state organizational characteristics were either associated with more EBP use or interacted with local organizational characteristics in associations with EBP use, including: (1) systems integration at the state level was associated with greater EBP use; (2) state staffing adequacy and stability accentuated the association between local training and resources for new programs and EBP use (i.e., in states with better staffing, the relationship between training/resources and EBP use in local facilities was stronger); and (3) state executives' attitudes regarding the missions and goals of corrections tended to diminish the extent to which corresponding local administrator attitudes were associated with EBP use. The study has implications for future research focused on EBP diffusion and implementation in correctional environments, particularly attempts to influence EBP use by working through state agencies.

Availability and capacity of substance abuse programs in correctional settings: A classification and regression tree analysis

OBJECTIVE TO BE ADDRESSED: The purpose of this study was to investigate the structural and organizational factors that contribute to the availability and increased capacity for substance abuse treatment programs in correctional settings. We used classification and regression tree statistical procedures to identify how multi-level data can explain the variability in availability and capacity of substance abuse treatment programs in jails and probation/parole offices.

METHODS

The data for this study combined the National Criminal Justice Treatment Practices (NCJTP) Survey and the 2000 Census. The NCJTP survey was a nationally representative sample of correctional administrators for jails and probation/parole agencies. The sample size included 295 substance abuse treatment programs that were classified according to the intensity of their services: high, medium, and low. The independent variables included jurisdictional-level structural variables, attributes of the correctional administrators, and program and service delivery characteristics of the correctional agency.

RESULTS

The two most important variables in predicting the availability of all three types of services were stronger working relationships with other organizations and the adoption of a standardized substance abuse screening tool by correctional agencies. For high and medium intensive programs, the capacity increased when an organizational learning strategy was used by administrators and the organization used a substance abuse screening tool. Implications on advancing treatment practices in correctional settings are discussed, including further work to test theories on how to better understand access to intensive treatment services. This study presents the first phase of understanding capacity-related issues regarding treatment programs offered in correctional settings.

Filling service gaps: Providing intensive treatment services for offenders

Consistent with the few studies that have previously examined treatment prevalence and access in the adult and juvenile justice systems, the recent National Criminal Justice Treatment Practices (NCJTP) survey indicated that there is a particular need to expand intensive treatment modalities for offenders in both institutional and community corrections settings. Applying multilevel modeling techniques to NCJTP survey data, this study explores conditions and factors that may underlie the wide variation among states in the provision of intensive treatment for offenders. Results indicate that states' overall rates of substance abuse and dependence, funding resources, and the state governor's political party affiliation were significantly associated with intensive treatment provision. Numerous factors that have been implicated in recent studies of evidence-based practice adoption, including state agency executives' views regarding rehabilitation, agency culture and climate, and other state-level measures (e.g., household income, crime rates, expenditures on treatment for the general population) were not associated with treatment provision. Future research should examine further variations in offenders' service needs, the role of legislators' political affiliations, and how other factors may interact with administrator characteristics in the adoption and expansion of intensive treatment services for offenders.

Organizational context, systems change, and adopting treatment delivery systems in the criminal justice system

The correctional system does not include service provision as a primary goal, even though individuals in prison, jail, and on probation/parole have large unmet substance abuse treatment needs. In response to mandates in the U.S. Constitution for basic health care, services are provided for incarcerated offenders, but generally do not include substance abuse treatment. The system does little to extend any type of health care service to individuals in community settings. This leaves the majority of offenders (6 million under community supervision in the U.S.) basically unattended, even with substance abuse disorders that are four times greater than the general public. The challenge of adapting the correctional system to be part of an integrated service provision system - working in conjunction with the public and private community-based service delivery sector - has intrigued researchers and policy makers over the last two decades. A series of articles using data from the National Criminal Justice Treatment Practices survey have examined factors that influence the adoption of a myriad of substance abuse treatment services for offender populations in various settings. These articles explore the factors that affect adoption and implementation, and provide guidance on issues relevant to organizational change and a dual mission of correctional agencies to advance public safety and public health. This special issue of Drug and Alcohol Dependence is devoted to understanding organizational constructs and factors to improve health outcomes for offenders.

Competing values among criminal justice administrators: The importance of substance abuse treatment

This study applied latent class analysis (LCA) to examine heterogeneity in criminal justice administrators' attitudes toward the importance of substance abuse treatment relative to other programs and services commonly offered in criminal justice settings. The study used data collected from wardens, probation and/or parole administrators, and other justice administrators as part of the National Criminal Justice Treatment Practices survey (NCJTP), and includes both adult criminal and juvenile justice samples. Results of the LCA suggested that administrators fell into four different latent classes: (1) those who place a high importance on substance abuse treatment relative to other programs and services, (2) those who place equal importance on substance abuse treatment and other programs and services, (3) those who value other programs and services moderately more than substance abuse treatment, and (4) those who value other programs and services much more than substance abuse treatment. Latent class membership was in turn associated with the extent to which evidence-based substance abuse treatment practices were being used in the facilities, the region of the country in which the administrator worked, and attitudes toward rehabilitating drug-using offenders. The findings have implications for future research focused on the impact that administrators' attitudes have on service provision as well as the effectiveness of knowledge dissemination and diffusion models.

Research on the diffusion of evidence-based treatments within substance abuse treatment: a systematic review

This article provides a comprehensive review of research studies that have examined the diffusion of evidence-based treatments (EBTs) within the field of substance abuse treatment. Sixty-five research studies were identified and were grouped into one of three major classifications: attitudes toward EBTs, adoption of EBTs, and implementation of EBTs. This review suggests significant progress has been made with regard to the advancement of the fields' knowledge about attitudes toward and the extent to which specific EBTs have been adopted in practice, as well as with regard to the identification of organizational factors related to EBT adoption. In an effort to advance the substance abuse treatment field toward evidence-based diffusion practices, recommendations are made for greater use of methodologically rigorous experimental or quasi-experimental designs, psychometrically sound instruments, and integration of quantitative and qualitative data collection.

COD services in community correctional settings: an examination of organizational factors that affect service delivery

Little is known about service delivery for those in the correctional system with co-occurring mental health and substance abuse disorders (COD). Using data from the National Criminal Justice Treatment Practices survey (NCJTP) on community corrections and jail agencies, this study found that correctional organizations that offer COD services differ from agencies that do not. Organizations that respond to the needs of COD offenders tend to be more "service friendly," rank higher on measures of various dimensions of organizational climate that embrace learning and goal achievement, and tend to adopt more innovations, particularly those classified as evidence-based practices. These survey findings indicate that correctional agencies who serve COD offenders are in a better position to be responsive to the changing service needs of the populations, and are more likely to be able to balance public safety and treatment goals. Future steps to develop support environments for COD assessment and treatment services are discussed in light of these findings.