Development of a computerized adaptive substance use disorder scale for screening and measurement: the CAT-SUD

Validation of the structured interview section of the penn computerized adaptive test for neurocognitive and clinical psychopathology assessment (CAT GOASSESS)

Large-scale studies and burdened clinical settings require precise, efficient measures that assess multiple domains of psychopathology. Computerized adaptive tests (CATs) can reduce administration time without compromising data quality. We examined feasibility and validity of an adaptive psychopathology measure, GOASSESS, in a clinical community-based sample (N = 315; ages 18-35) comprising three groups: healthy controls, psychosis, mood/anxiety disorders. Assessment duration was compared between the Full and CAT GOASSESS. External validity was tested by comparing how the CAT and Full versions related to demographic variables, study group, and socioeconomic status. The relationships between scale scores and criteria were statistically compared within a mixed-model framework to account for dependency between relationships. Convergent validity was assessed by comparing scores of the CAT and the Full GOASSESS using Pearson correlations. The CAT GOASSESS reduced interview duration by more than 90 % across study groups and preserved relationships to external criteria and demographic variables as the Full GOASSESS. All CAT GOASSESS scales could replace those of the Full instrument. Overall, the CAT GOASSESS showed acceptable psychometric properties and demonstrated feasibility by markedly reducing assessment time compared to the Full GOASSESS. The adaptive version could be used in large-scale studies or clinical settings for intake screening.

Initial Validation of a Computerized Adaptive Test for Substance Use Disorder Identification in Adolescents

PURPOSE

Computerized adaptive tests (CATs) are highly efficient assessment tools that couple low patient and clinician time burden with high diagnostic accuracy. A CAT for substance use disorders (CAT-SUD-E) has been validated in adult populations but has yet to be tested in adolescents. The purpose of this study was to perform initial evaluation of the K-CAT-SUD-E (i.e., Kiddy-CAT-SUD-E) in an adolescent sample compared to a gold-standard diagnostic interview.

METHODS

Adolescents (N = 156; aged 11-17) with diverse substance use histories completed the K-CAT-SUD-E electronically and the substance related disorders portion of a clinician-conducted diagnostic interview (K-SADS) via tele-videoconferencing platform. The K-CAT-SUD-E assessed both current and lifetime overall SUD and substance-specific diagnoses for nine substance classes.

RESULTS

Using the K-CAT-SUD-E continuous severity score and diagnoses to predict the presence of any K-SADS SUD diagnosis, the classification accuracy ranged from excellent for current SUD (AUC = 0.89, 95% CI = 0.81, 0.95) to outstanding (AUC = 0.93, 95% CI = 0.82, 0.97) for lifetime SUD. Regarding current substance-specific diagnoses, the classification accuracy was excellent for alcohol (AUC = 0.82), cannabis (AUC = 0.83) and nicotine/tobacco (AUC = 0.90). For lifetime substance-specific diagnoses, the classification accuracy ranged from excellent (e.g., opioids, AUC = 0.84) to outstanding (e.g., stimulants, AUC = 0.96). K-CAT-SUD-E median completion time was 4 min 22 s compared to 45 min for the K-SADS.

CONCLUSIONS

This study provides initial support for the K-CAT-SUD-E as a feasible accurate diagnostic tool for assessing SUDs in adolescents. Future studies should further validate the K-CAT-SUD-E in a larger sample of adolescents and examine its acceptability, feasibility, and scalability in youth-serving settings.

The Spanish language version of the TAPS tool: protocol for a validation and implementation study in primary care

BACKGROUND

The TAPS Tool ("Tobacco, Alcohol, Prescription drug, and illicit Substance use") is a screening and brief assessment for detecting unhealthy substance use in healthcare settings that was developed by the National Institute on Drug Abuse Clinical Trials Network and validated in a multisite study. Our team developed a Spanish language version of the TAPS Tool that supports provider- and self-administration screening using a mobile/web-based platform, the TAPS Electronic Spanish Platform (TAPS-ESP).

METHODS

This article describes the protocol and rationale for a study to validate the TAPS-ESP in a sample of Spanish-speaking primary care patients recruited from a network of community-based clinics in Texas (target N = 1,000). The TAPS-ESP will be validated against established substance use disorder diagnostic measures, alternative screening tools, and substance use biomarkers. The study will subsequently examine barriers and facilitators to screening with the TAPS-ESP from a provider workflow perspective using qualitative interviews with providers.

DISCUSSION

Validating a Spanish language version of the TAPS Tool could expand access to evidence-based, linguistically accurate, and culturally relevant substance use screening and brief assessment for an underserved health disparity population.

TRIAL REGISTRATION

The study was registered with www.

CLINICALTRIALS

gov : NCT05476588, 07/22/2022.

Participants' Engagement With and Results From a Web-Based Integrative Population Mental Wellness Program (CHAMindWell) During the COVID-19 Pandemic: Program Evaluation Study

BACKGROUND

The COVID-19 pandemic involved a prolonged period of collective trauma and stress during which substantial increases in mental health concerns, like depression and anxiety, were observed across the population. In this context, CHAMindWell was developed as a web-based intervention to improve resilience and reduce symptom severity among a public health care system's patient population.

OBJECTIVE

This program evaluation was conducted to explore participants' engagement with and outcomes from CHAMindWell by retrospectively examining demographic information and mental health symptom severity scores throughout program participation.

METHODS

We examined participants' symptom severity scores from repeated, web-based symptom screenings through Computerized Adaptive Testing for Mental Health (CAT-MH) surveys, and categorized participants into symptom severity-based tiers (tier 1=asymptomatic to mild; tier 2=moderate; and tier 3=severe). Participants were provided tier-based mindfulness resources, treatment recommendations, and referrals. Logistic regressions were conducted to evaluate associations between demographic variables and survey completion. The McNemar exact test and paired sample t tests were performed to evaluate changes in the numbers of participants in tier 1 versus tier 2 or 3 and changes in depression, anxiety, and posttraumatic stress disorder severity scores between baseline and follow-up.

RESULTS

The program enrolled 903 participants (664/903, 73.5% female; 556/903, 61.6% White; 113/903, 12.5% Black; 84/903, 9.3% Asian; 7/903, 0.8% Native; 36/903, 4% other; and 227/903, 25.1% Hispanic) between December 16, 2020, and March 17, 2022. Of those, 623 (69%) completed a baseline CAT-MH survey, and 196 completed at least one follow-up survey 3 to 6 months after baseline. White racial identity was associated with completing baseline CAT-MH (odds ratio [OR] 1.80, 95% CI 1.14-2.84; P=.01). Participants' odds of having symptom severity below the clinical threshold (ie, tier 1) were significantly greater at follow-up (OR 2.60, 95% CI 1.40-5.08; P=.001), and significant reductions were observed across symptom domains over time.

CONCLUSIONS

CHAMindWell is associated with reduced severity of mental health symptoms. Future work should aim to address program engagement inequities and attrition and compare the impacts of CHAMindWell to a control condition to better characterize its effects.

Clinical characterization and differentiation of B-SNIP psychosis Biotypes: Algorithmic Diagnostics for Efficient Prescription of Treatments (ADEPT)-1

Clinically defined psychosis diagnoses are neurobiologically heterogeneous. The B-SNIP consortium identified and validated more neurobiologically homogeneous psychosis Biotypes using an extensive battery of neurocognitive and psychophysiological laboratory measures. However, typically the first step in any diagnostic evaluation is the clinical interview. In this project, we evaluated if psychosis Biotypes have clinical characteristics that can support their differentiation in addition to obtaining laboratory testing. Clinical interview data from 1907 individuals with a psychosis Biotype were used to create a diagnostic algorithm. The features were 58 ratings from standard clinical scales. Extremely randomized tree algorithms were used to evaluate sensitivity, specificity, and overall classification success. Biotype classification accuracy peaked at 91 % with the use of 57 items on average. A reduced feature set of 28 items, though, also showed 81 % classification accuracy. Using this reduced item set, we found that only 10-11 items achieved a one-vs-all (Biotype-1 or not, Biotype-2 or not, Biotype-3 or not) area under the sensitivity-specificity curve of .78 to .81. The top clinical characteristics for differentiating psychosis Biotypes, in order of importance, were (i) difficulty in abstract thinking, (ii) multiple indicators of social functioning, (iii) conceptual disorganization, (iv) severity of hallucinations, (v) stereotyped thinking, (vi) suspiciousness, (vii) unusual thought content, (viii) lack of spontaneous speech, and (ix) severity of delusions. These features were remarkably different from those that differentiated DSM psychosis diagnoses. This low-burden adaptive algorithm achieved reasonable classification accuracy and will support Biotype-specific etiological and treatment investigations even in under-resourced clinical and research environments.

A randomized controlled trial on using predictive algorithm to adapt level of psychological care for community college students: STAND triaging and adapting to level of care study protocol

BACKGROUND

There is growing interest in using personalized mental health care to treat disorders like depression and anxiety to improve treatment engagement and efficacy. This randomized controlled trial will compare a traditional symptom severity decision-making algorithm to a novel multivariate decision-making algorithm for triage to and adaptation of mental health care. The stratified levels of care include a self-guided online wellness program, coach-guided online cognitive behavioral therapy, and clinician-delivered psychotherapy with or without pharmacotherapy. The novel multivariate algorithm will be comprised of baseline (for triage and adaptation) and time-varying variables (for adaptation) in four areas: social determinants of mental health, early adversity and life stressors, predisposing, enabling, and need influences on health service use, and comprehensive mental health status. The overarching goal is to evaluate whether the multivariate algorithm improves adherence to treatment, symptoms, and functioning above and beyond the symptom-based algorithm.

METHODS/DESIGN

This trial will recruit a total of 1000 participants over the course of 5 years in the greater Los Angeles Metropolitan Area. Participants will be recruited from a highly diverse sample of community college students. For the symptom severity approach, initial triaging to level of care will be based on symptom severity, whereas for the multivariate approach, the triaging will be based on a comprehensive set of baseline measures. After the initial triaging, level of care will be adapted throughout the duration of the treatment, utilizing either symptom severity or multivariate statistical approaches. Participants will complete computerized assessments and self-report questionnaires at baseline and up to 40 weeks. The multivariate decision-making algorithm will be updated annually to improve predictive outcomes.

DISCUSSION

Results will provide a comparison on the traditional symptom severity decision-making and the novel multivariate decision-making with respect to treatment adherence, symptom improvement, and functional recovery. Moreover, the developed multivariate decision-making algorithms may be used as a template in other community college settings. Ultimately, findings will inform the practice of level of care triage and adaptation in psychological treatments, as well as the use of personalized mental health care broadly.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05591937, submitted August 2022, published October 2022.

Computer adaptive testing to assess impairing behavioral health problems in emergency department patients with somatic complaints

Objectives

To assess: (1) the prevalence of mental health and substance use in patients presenting to the emergency department (ED) through use of a computer adaptive test (CAT‐MH), (2) the correlation among CAT‐MH scores and self‐ and clinician‐reported assessments, and (3) the association between CAT‐MH scores and ED utilization in the year prior and 30 days after enrollment.

Methods

This was a single‐center observational study of adult patients presenting to the ED for somatic complaints (97%) from May 2019 to March 2020. The main outcomes were computer‐adaptive‐assessed domains of suicidality, depression, anxiety, post‐traumatic stress disorder (PTSD), and substance use. We conducted Pearson correlations and logistic regression for objectives 2 and 3, respectively.

Results

From a sample of 794 patients, the proportion of those at moderate/severe risk was: 24.1% (suicidality), 8.3% (depression), 16.5% (anxiety), 12.3% (PTSD), and 20.4% (substance use). CAT‐MH domains were highly correlated with self‐report assessments (r = 0.49–0.79). Individuals who had 2 or more ED visits in the prior year had 62% increased odds of being in the intermediate‐high suicide risk category (odds ratio [OR], 1.62; 95% confidence interval [CI], 1.07–2.44) compared to those with zero prior ED visits. Individuals who scored in the intermediate‐high‐suicide risk group had 63% greater odds of an ED visit within 30 days after enrollment compared to those who scored as low risk (OR, 1.63; 95% CI, 1.09, 2.44).

Conclusion

The CAT‐MH documented that a considerable proportion of ED patients presenting for somatic problems had mental health conditions, even if mild. Mental health problems were also associated with ED utilization.

Development of a computerized adaptive substance use disorder scale for screening, measurement and diagnosis - The CAT-SUD-E

Introduction

The Computerized Adaptive Test for Substance Use Disorder (CAT-SUD), an adaptive test based on multidimensional item response theory, has been expanded to include 7 specific Diagnostic and Statistical Manual, 5th edition (DSM-5) defined SUDs. Initial testing of the new measure, the CAT-SUD expanded (CAT-SUD-E) is reported here.

Methods

275 Community-dwelling adults (ages 18-68) responded to public and social-media advertisements. Participants virtually completed both the CAT-SUD-E and the Structured Clinical Interview for DSM-5, Research Version (SCID) to assess the validity of the CAT-SUD-E in determining whether participants met criteria for specific DSM-5 SUDs. Diagnostic classifications were based on 7 SUDs, each with 5 items, for current and lifetime SUDs.

Results

For SCID-based presence of any lifetime SUD, predictions based on the overall CAT-SUD-E diagnosis and severity score were AUC=0.92, 95% CI = 0.88, 0.95 for current and AUC=0.94, 95% CI = 0.91, 0.97 for lifetime. For individual diagnoses, classification accuracy for current SUDs ranged from an AUC=0.76 for alcohol to AUC=0.92 for nicotine/tobacco. Classification accuracy for lifetime SUDs ranged from an AUC=0.81 for hallucinogens to AUC=0.96 for stimulants. Median CAT-SUD-E completion time was under 4 min.

Conclusions

The CAT-SUD-E quickly produces similar results as lengthy structured clinical interviews for overall SUD and substance-specific SUDs, with high precision and accuracy, through a combination of fixed-item responses for diagnostic classification and adaptive SUD severity measurement. The CAT-SUD-E harmonizes information from mental health, trauma, social support and traditional SUD items to provide a more complete characterization of SUD and provides both diagnostic classification and severity measurement.

Using computerised adaptive tests to screen for perinatal depression in underserved women of colour

BACKGROUND

Compared with traditional screening questionnaires, computerised adaptive tests for severity of depression (CAT-DI) and computerised adaptive diagnostic modules for depression (CAD-MDD) show improved precision in screening for major depressive disorder. CAT measures have been tailored to perinatal women but have not been studied in low-income women of colour despite high rates of perinatal depression (PND).

OBJECTIVE

This study aimed to examine the concordance between CAT and traditional measures of depression in a sample of primarily low-income black and Latina women.

METHODS

In total, 373 women (49% black; 29% Latina) completed the Patient Health Questionnaire-9 (PHQ-9), CAD-MDD and CAT-DI at 845 visits across pregnancy and postpartum. We examined the concordance between continuous CAT-DI and PHQ-9 scores and between binary measures of PND diagnosis on CAD-MDD and the PHQ-9 (cut-off score >10). We examined cases with a positive PND diagnosis on the CAD-MDD but not on the PHQ-9 ('missed' cases) to determine whether clinic notes were consistent with CAD-MDD results.

FINDINGS

CAT-DI and PHQ-9 scores were significantly associated (concordance correlation coefficient=0.67; 95% CI 0.58 to 0.74). CAD-MDD detected 5% more case of PND compared with PHQ-9 (p<0.001). The average per-visit rate of PND was 14.4% (14.5% in blacks, 14.9% in Latinas) on the CAD-MDD, and 9.5% (9.8% in blacks, 8.8% in Latinas) on the PHQ-9. Clinical notes were available on 60% of 'missed' cases and validated CAD-MDD PND diagnosis in 89% of cases.

CONCLUSIONS

CAD-MDD detected 5% more cases of PND in women of colour compared with traditional tests, and the majority of these cases were verified by clinician notes.

CLINICAL IMPLICATIONS

Use of CAT in routine clinic care may address health disparities in PND screening.

Development and Validation of Computerized Adaptive Assessment Tools for the Measurement of Posttraumatic Stress Disorder Among US Military Veterans

IMPORTANCE

Veterans from recent and past conflicts have high rates of posttraumatic stress disorder (PTSD). Adaptive testing strategies can increase accuracy of diagnostic screening and symptom severity measurement while decreasing patient and clinician burden.

OBJECTIVE

To develop and validate a computerized adaptive diagnostic (CAD) screener and computerized adaptive test (CAT) for PTSD symptom severity.

DESIGN, SETTING, AND PARTICIPANTS

A diagnostic study of measure development and validation was conducted at a Veterans Health Administration facility. A total of 713 US military veterans were included. The study was conducted from April 25, 2017, to November 10, 2019.

MAIN OUTCOMES AND MEASURES

The participants completed a PTSD-symptom questionnaire from the item bank and provided responses on the PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) (PCL-5). A subsample of 304 participants were interviewed using the Clinician-Administered Scale for PTSD for DSM-5.

RESULTS

Of the 713 participants, 585 were men; mean (SD) age was 52.8 (15.0) years. The CAD-PTSD reproduced the Clinician-Administered Scale for PTSD for DSM-5 PTSD diagnosis with high sensitivity and specificity as evidenced by an area under the curve of 0.91 (95% CI, 0.87-0.95). The CAT-PTSD demonstrated convergent validity with the PCL-5 (r = 0.88) and also tracked PTSD diagnosis (area under the curve = 0.85; 95% CI, 0.79-0.89). The CAT-PTSD reproduced the final 203-item bank score with a correlation of r = 0.95 with a mean of only 10 adaptively administered items, a 95% reduction in patient burden.

CONCLUSIONS AND RELEVANCE

Using a maximum of only 6 items, the CAD-PTSD developed in this study was shown to have excellent diagnostic screening accuracy. Similarly, using a mean of 10 items, the CAT-PTSD provided valid severity ratings with excellent convergent validity with an extant scale containing twice the number of items. The 10-item CAT-PTSD also outperformed the 20-item PCL-5 in terms of diagnostic accuracy. The results suggest that scalable, valid, and rapid PTSD diagnostic screening and severity measurement are possible.

Development and Validation of a Computerized Adaptive Assessment Tool for Discrimination and Measurement of Psychotic Symptoms

OBJECTIVE

Time constraints limit the use of measurement-based approaches in research and routine clinical management of psychosis. Computerized adaptive testing (CAT) can reduce administration time, thus increasing measurement efficiency. This study aimed to develop and test the capacity of the CAT-Psychosis battery, both self-administered and rater-administered, to measure the severity of psychotic symptoms and discriminate psychosis from healthy controls.

METHODS

An item bank was developed and calibrated. Two raters administered CAT-Psychosis for inter-rater reliability (IRR). Subjects rated themselves and were retested within 7 days for test-retest reliability. The Brief Psychiatric Rating Scale (BPRS) was administered for convergent validity and chart diagnosis, and the Structured Clinical Interview (SCID) was used to test psychosis discriminant validity.

RESULTS

Development and calibration study included 649 psychotic patients. Simulations revealed a correlation of r = .92 with the total 73-item bank score, using an average of 12 items. Validation study included 160 additional patients and 40 healthy controls. CAT-Psychosis showed convergent validity (clinician: r = 0.690; 95% confidence interval [95% CI]: 0.610-0.757; self-report: r = .690; 95% CI: 0.609-0.756), IRR (intraclass correlation coefficient [ICC] = 0.733; 95% CI: 0.611-0.828), and test-retest reliability (clinician ICC = 0.862; 95% CI: 0.767-0.922; self-report ICC = 0.815; 95%CI: 0.741-0.871). CAT-Psychosis could discriminate psychosis from healthy controls (clinician: area under the receiver operating characteristic curve [AUC] = 0.965, 95% CI: 0.945-0.984; self-report AUC = 0.850, 95% CI: 0.807-0.894). The median length of the clinician-administered assessment was 5 minutes (interquartile range [IQR]: 3:23-8:29 min) and 1 minute, 20 seconds (IQR: 0:57-2:09 min) for the self-report.

CONCLUSION

CAT-Psychosis can quickly and reliably assess the severity of psychosis and discriminate psychotic patients from healthy controls, creating an opportunity for frequent remote assessment and patient/population-level follow-up.

Reducing Opioid Mortality in Illinois (ROMI): A case management/peer recovery coaching critical time intervention clinical trial protocol

Individuals with a history of opioid use are disproportionately represented in Illinois jails and prisons and face high risks of overdose and relapse at community reentry. Case management and peer recovery coaching are established interventions that may be leveraged to improve linkage to substance use treatment and supportive services during these critical periods of transition. We present the protocol for the Reducing Opioid Mortality in Illinois (ROMI), a type I hybrid effectiveness-implementation randomized trial of a case management, peer recovery coaching and overdose education and naloxone distribution (CM/PRC + OEND) critical time intervention (CTI) compared to OEND alone. The CM/PRC + OEND is a novel, 12-month intervention that involves linkage to substance use treatment and support for continuity of care, skills building, and navigation and engagement of social services that will be implemented using a hub-and-spoke model of training and supervision across the study sites. At least 1000 individuals released from jails and prisons spanning urban and rural settings will be enrolled. The primary outcome is engagement in medication for opioid use disorder. Secondary outcomes include health insurance enrollment, mental health service engagement, and re-arrest/recidivism, parole violation, and/or reincarceration. Mixed methods will be used to evaluate process and implementation outcomes including fidelity to, barriers to, facilitators of, and cost of the intervention. Videoconferencing and other remote processes will be leveraged to modify the protocol for safety during the COVID-19 pandemic. Results of the study may improve outcomes for vulnerable persons at the margin of behavioral health and the criminal legal system.

Examining the Utility of a General Substance Use Spectrum Using Latent Trait Modeling

BACKGROUND

Polysubstance use (PSU; lifetime use of multiple substances) is common among individuals with problematic alcohol/substance use and is associated with poor prognosis and poor physical/mental health. Furthermore, simultaneous co-use of substances, such that drug effects overlap, is also common and related to unique risks (e.g. overdose). Despite the importance of PSU, current diagnostic systems continue to conceptualize problems with alcohol/substances as class-specific constructs (e.g. Stimulant Use Disorder), which essentially ignore many unique PSU processes.

METHODS

The current study modeled problems with alcohol, cannabis, stimulants, sedatives, opiates, and simultaneous co-use of these substances as a manifestation of a general substance use continuum versus as correlated class-specific constructs in a sample of young-adults(n = 2482) using confirmatory factor analysis. Utility of the models was evaluated by examining associations between the general substance use spectrum and class-specific latent factors with measures of anxiety, ADHD, adult antisocial problems, borderline symptoms, neuroticism, and intelligence in a subset of the sample(n=847).

RESULTS

Findings supported the conceptualization of problems with all substances, including co-use of substances, as being manifestations of a general substance use spectrum, as class-specific constructs were not differentially associated with other measures of psychological dysfunction. Examination of this general substance use spectrum indicated that all substances, separately and co-use, were robustly informative of this spectrum, but tended to discriminate between different severity levels.

DISCUSSION

The general substance use spectrum allows for integration of information from the use and co-use of all substances to provide better assessment of overall problems with substances compared to class-specific constructs.

Digital health for assessment and intervention targeting tobacco and cannabis co-use

Purpose of review

This article aims to summarize current research on digital health for assessment and intervention targeting tobacco and cannabis co-use and to answer the following questions: Which digital tools have been used? Which populations have been targeted? And what are implications for future research?

Recent findings

Ecological Momentary Assessment (EMA) via text messages or interactive voice response calls has been used to capture co-use patterns within a time window or co-administration of both substances via blunts among young adults. Feasibility of multicomponent interventions targeting dual cessation of both substances among adult co-users with cannabis use disorder, delivered via smartphone apps, online, and computer modules has been demonstrated.

Summary

Digital tools, particularly those using EMAs and mobile sensors, should be expanded to assess co-use of emerging tobacco and cannabis products. Digital cessation interventions should be tailored to different groups of co-users and address specific mechanisms underlying different co-use patterns.