Risk assessment matters, but only when implemented well: A multisite study in juvenile probation

Are risk assessment tools more accurate than unstructured judgments in predicting violent, any, and sexual offending? A meta-analysis of direct comparison studies

We conducted a pre-registered meta-analysis of studies that directly compared the predictive validity of risk assessment tools to unstructured judgments of risk for violent, any, or sexual offending. A total of 31 studies, containing 169 effect sizes from 45,673 risk judgments, met inclusion criteria. Based on the results of three-level mixed-effects meta-regression models, the predictive validity of total scores on risk assessment tools was significantly higher than that of unstructured judgments for predictions of violent, any, and sexual offending. Tools continued to outperform unstructured judgments after accounting for risk of bias. This finding was also robust to variations in population, assessment context, and outcome measurement. Although this meta-analysis provides support for the use of risk assessment tools, it also highlights limitations and gaps that future research should address.

Healthcare expenditures for people with substance use disorders in drug courts compared to their peers in traditional courts

Individuals within the criminal justice system are at greater risk of substance use-related morbidity and mortality and have substantial healthcare needs. In this quasi-experimental study, we assessed utilization patterns of Massachusetts Medicaid Program (MassHealth) services and associated expenditures among drug court probationers compared to a propensity score-matched sample of traditional court probationers. Risk of reoffending, employment status, age, and living arrangement data were used to calculate propensity scores and match probationers between the two court types, producing a final sample of 271 in each court (N=542). Utilization of services and associated expenditures were analyzed using a two-part model to address the skewed distribution of the data and to control for residual differences after matching from the perspective of the payer (i.e., MassHealth). The largest categories of MassHealth spending were prescription pharmaceuticals, hospital inpatient visits, and physician visits. In the unadjusted analysis, drug court probationers exhibited greater MassHealth services utilization and expenditures than traditional court probationers. However, drug courts enrolled more females, more people at higher risk of reoffending, and more people with opioid use disorders. After controlling for differences between the two court types, the difference in MassHealth services utilization and associated expenditures did not reach statistical significance. Drug court probationers were more likely to engage with healthcare services but did not incur significantly greater expenditures than traditional court probationers after controlling for differences between the samples.

Examining the measurement precision of behavior problems among a sample of primarily rural youth on juvenile probation and their parents

Accurate and timely intervention in the justice system is particularly critical in rural communities, given documented barriers to accessible, evidence-based services for youth. As youth in the juvenile justice system have a high prevalence of behavioral health needs, accurate assessment of those needs is a critical first step in linking youth to appropriate care. The goal of the current study is to examine the reliability of a brief assessment (the Brief Problem Checklist [BPC]) among a sample of 222 justice-involved youth and their caregivers who primarily reside in rural communities in the United States. Using a series of reliability analyses and tests of agreement, we examined whether youth and caregiver BPC produces reliable scales, the strength of the convergence among each of the BPC scales, and youth and caregiver agreement on the BPC scales. Findings support the reliability of the BPC, but not inter-rater reliability. Poor agreement between youth and caregiver reports exists for both youth internalizing and externalizing problems. Additionally, the BPC was significantly related to several theoretically relevant constructs, including treatment, substance use disorder severity, and family history of substance use. These findings lend merit to discussions about the need for more research on the reliability and validity of assessment instruments before their widespread use in guiding youth- and agency case planning decisions, along with informing conclusions about program effectiveness.

Using the VERA-2R, professional and organisational aspects

Introduction

Violent extremism risk assessments of individuals suspected or convicted of terrorism are relevant for legal decisions, in prison and probation settings, and in inter-professional risk collaboration. These risk assessment reports by professionals should be applicable to and usable for the different judicial contexts. Informal and formal clinical practice evaluations, in the form of practitioners feedback and standardised evaluation of professional violent extremism risk reports are needed to gain insight in the use and quality of violent extremism risk assessments.

Methods

In this study we examined how forensic professionals from three different countries (Sweden, Belgium and the Netherlands) use the VERA-2R in different judicial contexts. We also investigated which organizational aspects are important for the use of the VERA-2R. We focused on the perspective of the forensic professionals and their judicial organisations. We did a standardised survey among 86 VERA-2R trained professionals and a standardised interview with 20 executives and managers of organizations working with the VERA-2R.

Results

This study showed that professionals find the VERA-2R useful for structuring information and speaking a common risk language. However, using the VERA-2R comes with a variety of challenges, both on the professional and organisational level. VERA-2R trained professionals had few opportunities to use the instrument and when they did, they were not always offered regular supervision, intervision and booster training. Also, organisational issues in collaboration between judicial partner organisations and the lack of risk transfer information to professionals came to light.

Discussion

More research on the topic of risk transfer is needed. Policy implications are advised, for example the development of booster trainings, more organizational support, regulations on re-assessments, providing expertise and knowledge to indirect stakeholders and clear writing guidelines.

Risk-need-responsivity: Evaluating need-to-service matching with reach, effectiveness, adoption, implementation, maintenance

With a sample of 125 adults under community supervision (71.20% male, 76.00% White, mean age = 33.17 years), this study evaluated need-to-service matching using an evaluation framework from implementation science. Need-to-service matching is a case management strategy intended to align service referrals in case plans with justice-involved persons' criminogenic needs. The results indicated that need-to-service matching reached a high percentage of its target population at 81.70%. Within criminogenic need areas, good match frequencies ranged from 80.00% in family/marital problems to 98.29% in alcohol/drug problems. Clinical staff also met the adherence benchmark applied by the current study, which required a 75.00% match between individuals' criminogenic needs and the services they received. Justice-involved persons had, on average, 90.46% of their criminogenic needs matched with at least one service referral. Over-prescription of services (i.e., recommendation of services that were not needed) was high, with frequencies in need areas ranging from 60.98% in education/employment to 82.21% in antisocial patterns. Methods from implementation science are useful for structuring evaluations of need-to-service matching, understanding implementation success and failure, and generating recommendations for improving implementation practice. The field would benefit greatly from benchmarks for need-to-service matching evaluation elements.

Facilitators of Practitioner Adherence to a Risk/Need Assessment Tool: Hypothesis Testing on a Survey of Juvenile Probation Officers

We used a statewide survey to test hypotheses about the predictors of juvenile probation officers' adherence to the Youth Level of Service/Case Management Inventory (YLS/CMI) risk/need assessment (RNA) tool, focusing on (1) the consistency and quality with which officers completed the tool, and (2) the extent to which they used it in decisions. While some hypotheses had been tested in prior quantitative studies, others were based on insights from case studies. Results showed that leadership and climate variables were consistently important in predicting adherence, though these tended to operate indirectly through their effects on other facilitators. Probation officer attitudes, either toward the YLS/CMI or to evidence-based practices, were also important across adherence measures. Although inconsistent in their effects across dependent variables, quality assurance of officer decision-making, external office relationships, and county YLS/CMI policies also predicted adherence.

The impact of a brief RNR-based training on Turkish juvenile probation officers' punitive and rehabilitative attitudes and recidivism risk perceptions

The present quasi-experimental study examined the impact of a brief training program based on the risk-need-responsivity (RNR) model on Turkish juvenile probation officers' (JPOs) punitive and rehabilitative attitudes toward justice-involved youth and recidivism risk perceptions. Fifty-nine JPOs were recruited through three probation offices in Istanbul, Turkey. Thirty-six JPOs, who received a 1-day training in the RNR model of offending behavior, were compared to JPOs in a wait-list control condition (n = 23). Participants in both conditions completed surveys at baseline and 1-week posttraining. Mixed-factorial analysis of variances revealed a significantly higher decrease in JPOs' punitive attitudes from pre- to posttest, in the training condition compared to the control group, with a medium effect size. Rehabilitative attitudes decreased in both conditions, while recidivism risk perceptions did not change from pre- to posttest in either condition. Future research could expand on these promising results using a more intensive training program and a randomized-controlled design in a larger sample of JPOs.

Supervision trajectories of male juvenile offenders: growth mixture modeling on SAVRY risk assessments

BACKGROUND

Structured risk/need assessment tools are increasingly used to orientate risk reduction strategies with juvenile offenders. The assumption is that the risk/need items on these tools are sufficiently sensitive to measure changes in the individual, family and/or contextual characteristics of juvenile offenders. However, there is very little research demonstrating the capacity of these tools to measure changes in juvenile offenders. Congruent with the developmental and life-course criminology theories (DLC) the objective of this study is to explore the existence of heterogeneous trajectories of juvenile offenders across the juvenile justice system as measured through five empirical risk/need areas based on the Structured Assessment of Violence Risk in Youth (SAVRY), one of the most widely applied risk assessment tools for juveniles.

METHODS

This longitudinal study included 5205 male juvenile offenders who transitioned through the Catalan juvenile justice system between 2006 and 2014. During intervention they received at least two, and a maximum of seven, consecutive SAVRY risk/need assessments over an 18-month period. The heterogeneity of latent class trajectories was explored through growth mixture modeling (GMM). The trajectory class membership was linked to covariates through multinomial logistic regression analyses.

RESULTS

Through GMM three to four heterogeneous trajectories, with high quality of separation, were identified in each of the risk/need areas. The trajectories with low risk/needs (45-77% of the sample) remained low and presented a very limited increase in risk/needs during the 18-month period. The high risk/need trajectories (20-37% of the sample) showed a limited decrease or no change. Between 5 and 13% of the sample had large reductions in their risk/needs levels, and approximately 5% showed a large increase in risk/needs.

CONCLUSIONS

In line with the DLC theories this study shows that trajectories on criminogenic risk/needs can be heterogeneous and indicate distinct rates of change over time. The results of this study also may suggest a limited sensibility to measure change over time of SAVRY's risk and protective items. Suggestions to improve the sensitivity of measuring change over time, such as shorter time frames or future-oriented time frames for the scoring of the items, are offered.

Commentary: Risk Assessment in the Age of Evidence-Based Practice and Policy

Risk assessment has come to be recognized as a key component of evidence-based practice and policy in psychiatric and correctional agencies. At the same time, however, there is significant debate in scientific, policy, and public arenas regarding the role of risk assessment instruments in mental health and criminal justice decision-making, and questions regarding the level of evidence supporting their usefulness. It is in light of these conflicting realities that the current commentary considers Williams, Wormith, Bonta and Sitarenios' (2017) re-examination of the Singh, Grann, and Fazel (2011) meta-analysis and recommendations made in "The Use of Meta-Analysis to Compare and Select Offender Risk Instruments." Additional limitations in the extant risk assessment research are identified and their implications for evidence-based practice and policy are discussed.