Comparing the effects of sampling designs on the diagnostic accuracy of eight promising screening algorithms for pediatric bipolar disorder

Discriminant diagnostic validity of paediatric bipolar disorder screening tests: A systematic review and meta-analysis

INTRODUCTION

Bipolar disorders (BD) are among the most significantly impairing of childhood and adolescent psychiatric disorders. Although BD symptoms may begin in adolescence, they are frequently not diagnosed until adulthood, and accordingly BD scales could aid diagnostic assessment in paediatric populations. This review aims to synthesis the evidence for the accuracy of BD symptom index tests for discriminating BD from non-BD (other diagnoses or healthy controls) in paediatric population. Additionally, several theoretically relevant moderators of diagnostic accuracy were evaluated.

METHODS

A systematic search across three databases were conducted from 1980 to 2022, augmented by grey literature database searches, citation chaining and contacting authors. Data from eligible studies were synthesized using meta-analysis. A multilevel model was fitted to account for nested effect sizes, with 31 potential moderators examined in univariate and multivariate models.

RESULTS

Twenty-Eight studies were eligible, yielding 115 effect sizes for analysis. Meta-analytic modelling indicated BD symptom index tests have a high diagnostic accuracy (g = 1.300; 95% CI: 0.982 - 1.619; p < .001) in paediatric population. Accuracy was relative to the type of comparison group, index test content, index test informant and index test's scale or subscale.

CONCLUSIONS

Screening tests based on mania content, caregiver report and non-healthy comparison groups have clinical utility in identifying paediatric BD. Other informant-and-content combination may not accurately identify paediatric BD. Unlike healthy controls, tests derived from studies using non-healthy comparison groups, represent BD symptom non-specificity and BD symptom overlap with other disorders, providing external validity and clinical utility.

Where There’s Smoke: Validating a Nonproprietary Single-Item Burnout-Impacting-Safety Scale

This study aimed to assess the utility of a single survey item to predict the impact of burnout on safety and quality of healthcare providers as perceived by their colleagues. The primary objective is to determine if the item predicts the frequency of patient safety event reports within certain clinical departments. The secondary objective will be to determine if there is an acceptable cutoff score for the item which predicts low versus high numbers of safety events reported by healthcare providers in each clinical department. Participants were 424 healthcare providers in an academic medical center in the mid-Atlantic region of the United States. The item was designed to assess for the perception of the impact of burnout on work in terms of quality or safety using a 5-point Likert scale. Data from a patient safety event reporting system was accessed for the year of survey completion (2017). A negative binomial regression was used to assess the ability of the item to predict reported patient safety event reports. The item was found to significantly predict objective safety event data. Sensitivity and specificity, as well as receiver operating characteristic (ROC) curve analyses, were conducted to determine appropriateness of cutoff scores to identify low- and high-risk clinical departments. The item was found to demonstrate adequate sensitivity (82%) using a cutoff score of 4 on the survey item. However, the area under the curves (AUCs) which assess diagnostic accuracy fell in the poor range. These results suggest that healthcare administrators could deploy this single item as a brief pulse or screener of teams of individuals who are within a work unit and use a cutoff score of 4 as a means to assess for hot spots where healthcare provider burnout may be putting patients at high risk in terms of safety.

Validation of the Distress Thermometer in patients with advanced cancer receiving specialist palliative care in a hospice setting

BACKGROUND

Ongoing assessment of psychological reaction to illness in palliative and end of life care settings is recommended, yet validated tools are not routinely used in clinical practice. The Distress Thermometer is a short screening tool developed for use in oncology, to detect individuals who would benefit from further psychological assessment. However the optimal cut-off to detect indicative psychological morbidity in patients with advanced cancer receiving specialist palliative care is unclear.

AIM

To provide the first validation of the Distress Thermometer in an advanced cancer population receiving specialist palliative care in a UK hospice setting.

DESIGN

Receiver Operating Characteristics analysis was used to compare the sensitivity and specificity of cut-offs indicative of psychological morbidity on the Distress Thermometer in comparison to the Hospital Anxiety and Depression Scale.

SETTING/PARTICIPANTS

Data were derived from 202 patients with advanced cancer who were approached on admission to inpatient or day hospice care, with 139 patients providing complete data on both measures.

RESULTS

The area under the curve was optimal using a Distress Thermometer cut-off score of ⩾6 for total distress and for anxiety, and a cut-off score of ⩾4 optimal when screening for depression.

CONCLUSIONS

The Distress Thermometer is a valid, accurate screening tool to be used in advanced cancer but with caution in relation to the lack of specificity. With little variation between the area under the curve scores, arguably a Distress Thermometer cut-off score of ⩾5 is most appropriate in screening for all types of psychological morbidity if sensitivity is to be prioritised.

Evidence Base Update for Brief, Free, and Accessible Youth Mental Health Measures

Evidence-based assessment (EBA) is foundational to high-quality mental health care for youth and is a critical component of evidence-based practice delivery, yet is underused in the community. Administration time and measure cost are barriers to use; thus, identifying and disseminating brief, free, and accessible measures are critical. This Evidence Base Update evaluates the empirical literature for brief, free, and accessible measures with psychometric support to inform research and practice with youth. A systematic review using PubMed and PsycINFO identified measures in the following domains: overall mental health, anxiety, depression, disruptive behavior, traumatic stress, disordered eating, suicidality, bipolar/mania, psychosis, and substance use. To be eligible for inclusion, measures needed to be brief (50 items or less), free, accessible, and have psychometric support for their use with youth. Eligible measures were evaluated using adapted criteria established by De Los Reyes and Langer (2018) and were classified as having excellent, good, or adequate psychometric properties. A total of 672 measures were identified; 95 (14%) met inclusion criteria. Of those, 21 (22%) were "excellent," 34 (36%) were "good," and 40 (42%) were "adequate." Few measures had support for their use to routinely monitor progress in therapy. Few measures with excellent psychometric support were identified for disordered eating, suicidality, psychosis, and substance use. Future research should evaluate existing measures for use with routine progress monitoring and ease of implementation in community settings. Measure development is needed for disordered eating, suicidality, psychosis, and substance use to increase availability of brief, free, accessible, and validated measures.

The Critical Need for Help-Seeking Controls in Clinical High-Risk Research

Despite rapidly growing knowledge of the clinical high-risk (CHR) state for psychosis, the vast majority of case-control studies have relied on healthy volunteers as a reference point for drawing inferences about the CHR construct. Researchers have long recognized that results generated from this design are limited by significant interpretive concerns, yet little attention has been given to how these concerns affect the growing field of CHR research. We argue that overreliance on healthy controls in CHR research threatens the validity of inferences concerning group differences, hinders advances in understanding the development of psychosis, and limits clinical progress. We suggest that the combined use of healthy and help-seeking (i.e., psychiatric) controls is a necessary step for the next generation of CHR research. We then evaluate methods for help-seeking control studies, identify the available CHR studies that have used such designs, discuss select findings in this literature, and offer recommendations for research.

Is the Finding Too Good to Be True? Moving from "More Is Better" to Thinking in Terms of Simple Predictions and Credibility

In 2018, De Los Reyes and Langer expanded the scope of the Evidence Base Updates series to include reviews of psychological assessment techniques. In keeping with the goal of offering clear "take-home messages" about the evidence underlying the technique, experts have proposed a rubric for evaluating the reliability and validity support. Changes in the research environment and pressures in the peer review process, as well as a lack of familiarity with some statistical methods, have created a situation in which many findings that appear "excellent" in the rubric are likely to be "too good to be true," in the sense that they are unlikely to generalize to clinical settings or are unlikely to be reproduced in independent samples. We describe several common scenarios in which published results are often too good to be true, including internal consistency, interrater reliability, correlation, standardized mean differences, diagnostic accuracy, and global model fit statistics. Simple practices could go a long way toward improving design, reporting, and interpretation of findings. When effect sizes are in the "excellent" range for issues that have been challenging, scrutinize before celebrating. When benchmarks are available based on theory or meta-analyses, results that are moderately better than expected in the favorable direction (i.e., Cohen's q ≥ +.30) also invite critical appraisal and replication before application. If readers and reviewers pull for transparency and do not unduly penalize authors who provide it, then change in research quality will be faster and both generalizability and reproducibility are likely to benefit.

Evidence for Differential Predictive Performance of the Prime Screen Between Black and White Help-Seeking Youths

OBJECTIVE

Self-report screening instruments for emerging psychosis have the potential to improve early detection efforts by increasing the number of true positives among persons deemed to be at "clinical high risk" of the disorder, but their practical utility depends on their validity across race. This study sought to examine whether a commonly used self-report screening tool for psychosis risk performed equally among black and white youths in its ability to predict clinical high-risk status.

METHODS

Black (N=58) and white (N=50) help-seeking individuals ages 12-25 (61% female) were assessed with the Prime Screen and the Structured Interview for Psychosis-Risk Syndromes (SIPS). A logistic regression model estimated race differences in the strength of the relation between Prime Screen scores and SIPS-defined risk status.

RESULTS

Higher Prime Screen scores significantly predicted clinical high-risk status among white (p<.01) but not black participants. Among black youths without clinical high risk, self-reported Prime Screen scores more closely resembled scores for youths (black or white) with clinical high risk than scores of white peers who were also without clinical high risk.

CONCLUSIONS

Results suggest that consideration of race or ethnicity and associated cultural factors is important when screening for clinical high-risk status. Findings support the need to develop culturally valid early psychosis screening tools to promote appropriately tailored early intervention efforts.

The predictive validity of the Strengths and Difficulties Questionnaire for child attention-deficit/hyperactivity disorder

We need accurate screening measures for attention-deficit/hyperactivity disorder (ADHD) to ensure that children with the disorder are referred for assessment without raising concern for children with normal behaviour. The Strengths and Difficulties Questionnaire (SDQ) provides hyperactivity-inattention (HI), conduct, emotional and peer problem subscales and impact scores that may be used for screening. The aim of the study was to investigate the predictive validity of the Danish version of the parent SDQ HI subscale at the child age of 7 years for subsequent clinically diagnosed ADHD (age 8-15 years). Participants were part of the Danish National Birth Cohort (N = 51,096), and children with ADHD were identified through the Danish National Health registries (n = 943). Receiver operating characteristic analysis showed that the screening accuracy for the HI scores was good (area under the curve = .84). With Cox multivariate regression analysis, we found that SDQ HI subscale scores ≥ 7 with impact gave a nearly 14-fold [hazard ratio (HR) = 13.59] increased risk for ADHD, while conduct and emotional problems indicated low risk (HRs of 1.62 and 1.67, respectively). For the HI subscale to be a sensitive measure for ADHD, a low cutoff (4) was needed, but gave many false screening positives (PPV = .02). Although the diagnostic accuracy of the parent version of the SDQ HI subscale for predicting ADHD was good, our results question the feasibility of screening the general child population for ADHD with only the parent SDQ HI subscale.

Toward Automatic Risk Assessment to Support Suicide Prevention

Background: Suicide has been considered an important public health issue for years and is one of the main causes of death worldwide. Despite prevention strategies being applied, the rate of suicide has not changed substantially over the past decades. Suicide risk has proven extremely difficult to assess for medical specialists, and traditional methodologies deployed have been ineffective. Advances in machine learning make it possible to attempt to predict suicide with the analysis of relevant data aiming to inform clinical practice. Aims: We aimed to (a) test our artificial intelligence based, referral-centric methodology in the context of the National Health Service (NHS), (b) determine whether statistically relevant results can be derived from data related to previous suicides, and (c) develop ideas for various exploitation strategies. Method: The analysis used data of patients who died by suicide in the period 2013-2016 including both structured data and free-text medical notes, necessitating the deployment of state-of-the-art machine learning and text mining methods. Limitations: Sample size is a limiting factor for this study, along with the absence of non-suicide cases. Specific analytical solutions were adopted for addressing both issues. Results and Conclusion: The results of this pilot study indicate that machine learning shows promise for predicting within a specified period which people are most at risk of taking their own life at the time of referral to a mental health service.

Development and Validation of Objective and Quantitative Eye Tracking-Based Measures of Autism Risk and Symptom Levels

OBJECTIVE

The primary aim of this study was to develop and validate eye tracking-based measures for estimating autism spectrum disorder (ASD) risk and quantifying autism symptom levels.

METHOD

Eye tracking data were collected from youth during an initial evaluation visit, with administrators blinded to all clinical information. Consensus diagnoses were given by the multidisciplinary team. Participants viewed a 5-minute video that included 44 dynamic stimuli from 7 distinct paradigms while gaze was recorded. Gaze metrics were computed for temporally defined regions of interest. Autism risk and symptom indices aggregated gaze measures showing significant bivariate relationships with ASD diagnosis and Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) symptom severity levels in a training sample (75%, n = 150). Receiver operating characteristic curve analysis and nonparametric correlations were used to cross-validate findings in a test sample (25%; n = 51).

RESULTS

Most children (n = 201, 92%) completed a valid eye tracking assessment (ages 1.6─17.6; 80% male; ASD n = 91, non-ASD n = 110). In the test subsample, the autism risk index had high accuracy for ASD diagnosis (area under the curve [AUC] = 0.86, 95% CI =0.75-0.95), whereas the autism symptom index was strongly associated with ADOS-2 total severity scores (r = 0.41, p < .001). Validity was not substantively attenuated after adjustment for language, nonverbal cognitive ability, or other psychopathology symptoms (r = 0.40-0.67, p > .001).

CONCLUSION

Eye tracking measures appear to be useful quantitative, objective measures of ASD risk and autism symptom levels. If independently replicated and scaled for clinical use, eye tracking-based measures could be used to inform clinical judgment regarding ASD identification and to track autism symptom levels.

Discriminative Validity of the Dimensional Obsessive-Compulsive Scale for Separating Obsessive-Compulsive Disorder From Anxiety Disorders

Objective: We investigated the diagnostic efficiency and clinical utility of the Dimensional Obsessive-Compulsive Scale (DOCS) and subscales for distinguishing obsessive-compulsive disorder (OCD) from anxiety disorders (ADs). Method: A total of 369 participants (167 male, M_age = 29.61 years) diagnosed with DSM-IV OCD or AD, recruited from specialty clinics across the United States, completed clinical interviews and self-report questionnaires, including the DOCS. Receiver operating characteristic analyses and diagnostic likelihood ratios (DiLRs) determined discriminative validity and provided clinical utility. Logistic regressions tested for incremental validity in the DOCS-total scale and subscales in predicting OCD status. Results: The DOCS-total scale and Contamination subscale performed best in differentiating between OCD and AD diagnosis (DOCS-total: Area under curve [AUC] = .75, p < .001; Contamination: AUC = .70, p < .001) as compared with the other subscales. At high scores (DOCS-total: 28+, Contamination: 6+), Contamination was more effective than the DOCS-total in differentiating OCD from ADs, with high scores in Contamination quadrupling OCD odds and DOCS-total by about threefold (Contamination DiLR+ = 4.04, DOCS-total DiLR+ = 2.82). At low scores (DOCS-total: 0-9, Contamination: 0-2), the converse was true, with low scores in Contamination cutting OCD odds by half and DOCS-total by one fifths (Contamination DiLR- = 0.52, DOCS-total DiLR- = 0.23). Conclusion: At high scores, the Contamination subscale is the most helpful subscale to differentiate OCD and ADs. For low scores, the DOCS-total scale performs the best among the scales.

Dysregulation in children: Origins and implications from age 5 to age 28

Research shows that childhood dysregulation is associated with later psychiatric disorders. It does not yet resolve discrepancies in the operationalization of dysregulation. It is also far from settled on the origins and implications of individual differences in dysregulation. This study tested several operational definitions of dysregulation using Achenbach attention, anxious/depressed, and aggression subscales. Individual growth curves of dysregulation were computed, and predictors of growth differences were considered. The study also compared the predictive utility of the dysregulation indexes to standard externalizing and internalizing indexes. Dysregulation was indexed annually for 24 years in a community sample (n = 585). Hierarchical linear models considered changes in dysregulation in relation to possible influences from parenting, family stress, child temperament, language, and peer relations. In a test of the meaning of dysregulation, it was related to functional and psychiatric outcomes in adulthood. Dysregulation predictions were further compared to those of the more standard internalizing and externalizing indexes. Growth curve analyses showed strong stability of dysregulation. Initial levels of dysregulation were predicted by temperamental resistance to control, and change in dysregulation was predicted by poor language ability and peer relations. Dysregulation and externalizing problems were associated with negative adult outcomes to a similar extent.

Evidence-Based Assessment from Simple Clinical Judgments to Statistical Learning: Evaluating a Range of Options Using Pediatric Bipolar Disorder as a Diagnostic Challenge

Reliability of clinical diagnoses is often low. There are many algorithms that could improve diagnostic accuracy, and statistical learning is becoming popular. Using pediatric bipolar disorder as a clinically challenging example, we evaluated a series of increasingly complex models ranging from simple screening to a supervised LASSO regression in a large (N=550) academic clinic sample. We then externally validated models in a community clinic (N=511) with the same candidate predictors and semi-structured interview diagnoses, providing high methodological consistency; the clinics also had substantially different demography and referral patterns. Models performed well according to internal validation metrics. Complex models degraded rapidly when externally validated. Naïve Bayesian and logistic models concentrating on predictors identified in prior meta-analyses tied or bettered LASSO models when externally validated. Implementing these methods would improve clinical diagnostic performance. Statistical learning research should continue to invest in high quality indicators and diagnoses to supervise model training.

The predictive value of childhood subthreshold manic symptoms for adolescent and adult psychiatric outcomes

BACKGROUND

Childhood subthreshold manic symptoms may represent a state of developmental vulnerability to Bipolar Disorder (BD) and may also be associated with other adverse psychiatric outcomes. To test this hypothesis we examined the structure and predictive value of childhood subthreshold manic symptoms for common psychiatric disorders presenting by early adulthood.

METHODS

Subthreshold manic symptoms at age 11 years and lifetime clinical outcomes by age 19 years were ascertained in the TRacking Adolescents' Individual Lives Survey (TRAILS), a prospective Dutch community cohort. We used latent class analysis to identify subthreshold manic symptom profiles at baseline. The association between class membership and subsequent clinical diagnoses of BD (comprising BD-I, BD-II, mania and hypomania), depressive, anxiety and substance abuse disorders was determined using Cox proportional-hazard ratio (HR) models.

RESULTS

At age 11 years, we identified a normative (n=916; 47%), a mildly symptomatic (n=843; 43%) and a highly symptomatic class (n=198; 10%). Referenced to the normative class, the sex- and age-adjusted risk of new-onset BD by the age of 19 years was significantly increased in the mildly (HR=2.01, 95%CI 1.13-3.59) and highly symptomatic classes (HR=5.02, 95%CI 2.48-10.16). These estimates remained significant after further adjustments for cognitive and family function, parental socioeconomic status, parental psychiatric morbidity, and comorbid disorders at baseline (p-value for linear trend across classes<0.01). Class membership did not show significant associations with incident depressive, anxiety and substance abuse disorders in the fully adjusted regression models.

LIMITATIONS

The period of risk for adult-onset BD extends beyond the observational period of the study.

CONCLUSIONS

Elevated childhood subthreshold manic symptoms are associated with increased risk of BD by early adulthood and are therefore a potentially useful phenotype for the early identification of at-risk individuals.

Comparing the CASI-4R and the PGBI-10 M for Differentiating Bipolar Spectrum Disorders from Other Outpatient Diagnoses in Youth

We compared 2 rating scales with different manic symptom items on diagnostic accuracy for detecting pediatric bipolar spectrum disorder (BPSDs) in outpatient mental health clinics. Participants were 681 parents/guardians of eligible children (465 male, mean age = 9.34) who completed the Parent General Behavior Inventory-10-item Mania (PGBI-10 M) and mania subscale of the Child and Adolescent Symptom Inventory-Revised (CASI-4R). Diagnoses were based on KSADS interviews with parent and youth. Receiver operating characteristic (ROC) analyses and diagnostic likelihood ratios (DLRs) determined discriminative validity and provided clinical utility, respectively. Logistic regressions tested for incremental validity in the CASI-4R mania subscale and PGBI-10 M in predicting youth BPSD status above and beyond demographic and common diagnostic comorbidities. Both CASI-4R and PGBI-10 M scales significantly distinguished BPSD (N = 160) from other disorders (CASI-4R: Area under curve (AUC) = .80, p < 0.0005; PGBI-10 M: AUC = 0.79, p < 0.0005) even though scale items differed. Both scales performed equally well in differentiating BPSDs (Venkatraman test p > 0.05). Diagnostic likelihood ratios indicated low scores on either scale (CASI: 0-5; PGBI-10 M: 0-6) cut BPSD odds to 1/5 of those with high scores (CASI DLR- = 0.17; PGBI-10 M DLR- = 0.18). High scores on either scale (CASI: 14+; PGBI-10 M: 20+) increased BPSD odds about fourfold (CASI DLR+ = 4.53; PGBI-10 M DLR+ = 3.97). Logistic regressions indicated the CASI-4R mania subscale and PGBI-10 M each provided incremental validity in predicting youth BPSD status. The CASI-4R is at least as valid as the PGBI-10 M to help identify BPSDs, and can be considered as part of an assessment battery to screen for pediatric BPSDs.

Diagnostic Efficiency of the Child and Adolescent Symptom Inventory (CASI-4R) Depression Subscale for Identifying Youth Mood Disorders

This study examined the diagnostic and clinical utility of the Child and Adolescent Symptom Inventory-4 R (CASI-4 R) Depressive and Dysthymia subscale for detecting mood disorders in youth (ages 6-12; M = 9.37) visiting outpatient mental health clinics. Secondary analyses (N = 700) utilized baseline data from the Longitudinal Assessment of Manic Symptoms study. Semistructured interviews with youth participants and their parents/caregivers determined psychiatric diagnoses. Caregivers and teachers completed the CASI-4 R. CASI-4 R depressive symptom severity and symptom count scores each predicted mood disorder diagnoses. Both caregiver scores (symptom severity and symptom count) of the CASI-4 R subscale significantly identified youth mood disorders (areas under the curve [AUCs] = .78-.79, ps < .001). The symptom severity version showed a small but significant advantage. Teacher symptom severity report did not significantly predict mood disorder diagnosis (AUC = .56, p > .05), whereas the teacher symptom count report corresponded to a small effect size (AUC = .61, p < .05). The CASI-4 R Depression scale showed strong incrememental validity even controlling for the other CASI-4 R scales. Caregiver subscale cutoff scores were calculated to assist in ruling in (diagnostic likelihood ratio [DLR] = 3.73) or ruling out (DLR = 0.18) presence of a mood disorder. The CASI-4 R Depressive subscale caregiver report can help identify youth mood disorders, and using DLRs may help improve diagnostic accuracy.

Diagnostic efficiency of the SDQ for parents to identify ADHD in the UK: a ROC analysis

Early, accurate identification of ADHD would improve outcomes while avoiding unnecessary medication exposure for non-ADHD youths, but is challenging, especially in primary care. The aim of this paper is to test the Strengths and Difficulties Questionnaire (SDQ) using a nationally representative sample to develop scoring weights for clinical use. The British Child and Adolescent Mental Health Survey (N = 18,232 youths 5-15 years old) included semi-structured interview DSM-IV diagnoses and parent-rated SDQ scores. Areas under the curve for SDQ subscales were good (0.81) to excellent (0.96) across sex and age groups. Hyperactivity/inattention scale scores of 10+ increased odds of ADHD by 21.3×. For discriminating ADHD from other diagnoses, accuracy was fair (<0.70) to good (0.88); Hyperactivity/inattention scale scores of 10+ increased odds of ADHD by 4.47×. The SDQ is free, easy to score, and provides clinically meaningful changes in odds of ADHD that can guide clinical decision-making in an evidence-based medicine framework.

Diagnostic Efficiency of Caregiver Report on the SCARED for Identifying Youth Anxiety Disorders in Outpatient Settings

This study investigated the diagnostic and clinical utility of the parent-rated Screen for Child Anxiety Related Emotional Disorders (SCARED-P) for detecting youth anxiety disorders. Youth ages 6 to 12 years, 11 months were recruited from 9 outpatient mental health clinics (N = 707). Consensus diagnoses were based on semistructured interviews (Schedule for Affective Disorders and Schizophrenia for School-Age Children) with youth and caregivers; 31% were diagnosed with at least one anxiety disorder. Caregivers completed the SCARED-P to describe youth anxiety levels. SCARED-P scores were not considered during the consensus diagnoses. Areas under the curve (AUCs) from receiver operating characteristic analyses and diagnostic likelihood ratios (DLRs) quantified performance of the SCARED-P total score and subscale scores (generalized anxiety disorder and separation anxiety disorder). SCARED-P total scores had variable efficiency (AUCs = .69-.88), and Generalized Anxiety Disorder and Separation Anxiety subscale scores were excellent (AUCs = .86-.89) for identifying specific anxiety disorders. Optimal subscale cutoff scores were computed to help rule in (DLRs = 2.7-5.4) or rule out (DLRs < 1.0) anxiety disorders among youth. Results suggest that the Generalized Anxiety Disorder and Separation Anxiety SCARED-P subscales accurately identify their respective matched diagnoses. DLRs may aid clinicians in screening for youth anxiety disorders and improve accuracy of diagnosis.

Comparing the Diagnostic Accuracy of Five Instruments for Detecting Posttraumatic Stress Disorder in Youth

The purpose of the study was to compare diagnostic accuracy of five posttraumatic stress disorder (PTSD) measures in a large outpatient sample of youths 11-18 years of age. Index tests included a parent report (a rationally derived scale from the Child Behavioral Checklist), a teacher report (the Teacher Report Form), and three youth reports-a PTSD scale from the Youth Self Report (YSR), Child PTSD Symptom Scale, and Child and Adolescent Trauma Survey. Interviews with the youth and caregiver using Schedule for Affective Disorders and Schizophrenia for School-Age Children generated criterion diagnoses of PTSD. Diagnoses were blind to scores on the index tests. Based on consensus diagnoses (N = 458), 10% of youth had PTSD. Area under the curve (AUC) from receiver operating characteristic analyses and multilevel likelihood ratios evaluated test performance. All youth reports (AUCs .67-.73) outperformed the teacher report (AUCs .42-.48) at identifying PTSD. The YSR outperformed the caregiver reports (AUCs .57-.58). Combining tests did not improve prediction of PTSD. The YSR predicted PTSD even after controlling for a self-reported traumatic event, but checklist ratings of traumatic events had no incremental value after controlling for YSR scores. When a youth endorsed few symptoms, the likelihood of the youth having PTSD was low. Very high scores on the YSR were associated with a moderate increase in the likelihood of PTSD diagnosis. The YSR appeared to be a useful diagnostic aid for youth PTSD and could facilitate differential diagnosis of youth PTSD in outpatient settings.

Further evidence that severe scores in the aggression/anxiety-depression/attention subscales of child behavior checklist (severe dysregulation profile) can screen for bipolar disorder symptomatology: a conditional probability analysis

BACKGROUND

Previous work shows that children with high scores (2SD, combined score≥210) on the Attention Problems, Aggressive Behavior, and Anxious-Depressed (A-A-A) subscales of the Child Behavior Checklist (CBCL) are more likely than other children to meet criteria for bipolar (BP)-I disorder. However, the utility of this profile as a screening tool has remained unclear.

METHODS

We compared 140 patients with pediatric BP-I disorder, 83 with attention deficit hyperactivity disorder (ADHD), and 114 control subjects. We defined the CBCL-Severe Dysregulation profile as an aggregate cutoff score of ≥210 on the A-A-A scales. Patients were assessed with structured diagnostic interviews and functional measures.

RESULTS

Patients with BP-I disorder were significantly more likely than both control subjects (Odds Ratio [OR]: 173.2; 95% Confidence Interval [CI], 21.2 to 1413.8; P<0.001) and those with ADHD (OR: 14.6; 95% CI, 6.2 to 34.3; P<0.001) to have a positive CBCL-Severe Dysregulation profile. Receiver Operating Characteristics analyses showed that the area under the curve for this profile comparing children with BP-I disorder against control subjects and those with ADHD was 99% and 85%, respectively. The corresponding positive predictive values for this profile were 99% and 92% with false positive rates of <0.2% and 8% for the comparisons with control subjects and patients with ADHD, respectively.

LIMITATIONS

Non-clinician raters administered structured diagnostic interviews, and the sample was referred and largely Caucasian.

CONCLUSIONS

The CBCL-Severe Dysregulation profile can be useful as a screen for BP-I disorder in children in clinical practice.

Unfiltered Administration of the YMRS and CDRS-R in a Clinical Sample of Children

The objective of this study is to evaluate discriminative validity of the Young Mania Rating Scale (YMRS) and Children's Depression Rating Scale-Revised (CDRS-R) in a clinical sample of children when administered in an unfiltered manner (i.e., regardless of whether symptoms occur in a mood episode). The Kiddie Schedule for Affective Disorders and Schizophrenia is the gold standard for assessing psychiatric disorders in children and was used to make diagnoses in this study. Using a sample of 707 treatment-seeking youth (ages 6-12 years, Mage = 9.7 years, 67.6% male), receiver operating curve analyses were performed and diagnostic likelihood ratios (DLRs) were calculated to evaluate the ability to change the odds and differentiate bipolar disorder from other disorders (using the YMRS) and depression from other disorders (using the CDRS-R). Using unfiltered administration, the YMRS achieved good discriminative validity when classifying bipolar disorder compared to other disorders (Area Under the Curve [AUC] = .86) and increased odds of a bipolar diagnosis given a score in the highest quintile (DLR = 6.12). Using unfiltered administration, the CDRS-R achieved moderate to good discriminative validity in classifying depressive disorders (DD) compared to other disorders (AUCBD in comparison = .78; AUCBD not in comparison = .84) and slightly increased odds of DD given a score in the highest quintile (DLRBD in comparison = 3.12; DLRBD not in comparison = 5.08). The YMRS and CDRS-R have moderate to good discriminative validity when administered in an unfiltered way in a sample of treatment seeking youth.

Clinical decision making about child and adolescent anxiety disorders using the Achenbach system of empirically based assessment

Anxiety disorders are common among children but can be difficult to diagnose. An actuarial approach to the diagnosis of anxiety may improve the efficiency and accuracy of the process. The objectives of this study were to determine the clinical utility of the Achenbach Child Behavior Checklist (CBCL) and Youth Self Report (YSR), two widely used assessment tools, for diagnosing anxiety disorders in youth and to aid clinicians in incorporating scale scores into an actuarial approach to diagnosis through a clinical vignette. Demographically diverse youth, 5 to 18 years of age, were drawn from two samples; one (N = 1,084) was recruited from a research center, and the second (N = 651) was recruited from an urban community mental health center. Consensus diagnoses integrated information from semistructured interview, family history, treatment history, and clinical judgment. The CBCL and YSR internalizing problems T scores discriminated cases with any anxiety disorder or with generalized anxiety disorder from all other diagnoses in both samples (ps < .0005); the two scales had equivalent discriminative validity (ps > .05 for tests of difference). No other scales, nor any combination of scales, significantly improved on the performance of the Internalizing scale. In the highest risk group, Internalizing scores greater than 69 (CBCL) or greater than 63 (YSR) resulted in a Diagnostic Likelihood Ratio of 1.5; low scores reduced the likelihood of anxiety disorders by a factor of 4. Combined with other risk factor information in an actuarial approach to assessment and diagnosis, the CBCL and YSR Internalizing scales provide valuable information about whether a youth is likely suffering from an anxiety disorder.

A primer on receiver operating characteristic analysis and diagnostic efficiency statistics for pediatric psychology: we are ready to ROC

OBJECTIVE

To offer a practical demonstration of receiver operating characteristic (ROC) analyses, diagnostic efficiency statistics, and their application to clinical decision making using a popular parent checklist to assess for potential mood disorder.

METHOD

Secondary analyses of data from 589 families seeking outpatient mental health services, completing the Child Behavior Checklist and semi-structured diagnostic interviews.

RESULTS

Internalizing Problems raw scores discriminated mood disorders significantly better than did age- and gender-normed T scores, or an Affective Problems score. Internalizing scores <8 had a diagnostic likelihood ratio <0.3, and scores >30 had a diagnostic likelihood ratio of 7.4.

CONCLUSIONS

This study illustrates a series of steps in defining a clinical problem, operationalizing it, selecting a valid study design, and using ROC analyses to generate statistics that support clinical decisions. The ROC framework offers important advantages for clinical interpretation. Appendices include sample scripts using SPSS and R to check assumptions and conduct ROC analyses.

Differentiating bipolar disorder from unipolar depression and ADHD: the utility of the general behavior inventory

Adolescence and early adulthood are the peak ages for the onset of unipolar and bipolar mood disorders. Moreover, for most individuals with attention-deficit/hyperactivity disorder (ADHD), symptoms and impairment begin in childhood but persist well into adolescence and adulthood (e.g., Barkley, 2010). Thus, adolescence and early adulthood represent a developmental window wherein individuals can be affected by mood disorders, ADHD, or both. Because treatment protocols for unipolar depression (UPD), bipolar disorder (BD), and ADHD are quite different, it is crucial that assessment instruments used among adolescents and young adults differentiate between these disorders. The primary objectives of this study were to evaluate the predictive and diagnostic validity of General Behavior Inventory (GBI; Depue et al., 1981) scores in discriminating BD from UPD and ADHD. Participants were drawn from adolescent (n = 361) and young adult (n = 614) samples. Based on findings from logistic regression and receiver-operating characteristics analyses, the diagnostic efficiency of the GBI scales range from fair (discriminating UPD from BD) to good (discriminating BD participants from nonclinical controls). Multilevel diagnostic likelihood ratios are also provided to facilitate individual decision making.

The 7 up 7 down inventory: a 14-item measure of manic and depressive tendencies carved from the General Behavior Inventory

The aim of this study was to develop and validate manic and depressive scales carved from the full-length General Behavior Inventory (GBI). The brief version was designed to be applicable for youths and adults and to improve separation between mania and depression dimensions. Data came from 9 studies (2 youth clinical samples, aggregate N = 738, and 7 nonclinical adult samples, aggregate N = 1,756). Items with high factor loadings on the 2 extracted dimensions of mania and depression were identified from both data sets, and final item selection was based on internal reliability criteria. Confirmatory factor analyses described the 2-factor model's fit. Criterion validity was compared between mania and depression scales, and with the full-length GBI scales. For both mania and depression factors, 7 items produced a psychometrically adequate measure applicable across both aggregate samples. Internal reliability of the Mania scale was .81 (youth) and .83 (adult) and for Depression was .93 (youth) and .95 (adult). By design, the brief scales were less strongly correlated with each other than were the original GBI scales. Construct validity of the new instrument was supported in observed discriminant and convergent relationships with external correlates and discrimination of diagnostic groups. The new brief GBI, the 7 Up 7 Down Inventory, demonstrates sound psychometric properties across a wide age range, showing expected relationships with external correlates. The new instrument provides a clearer separation of manic and depressive tendencies than the original. (PsycINFO Database Record (c) 2013 APA, all rights reserved).

An inexpensive family index of risk for mood issues improves identification of pediatric bipolar disorder

Family history of mental illness provides important information when evaluating pediatric bipolar disorder (PBD). However, such information is often challenging to gather within clinical settings. This study investigates the feasibility and utility of gathering family history information using an inexpensive method practical for outpatient settings. Families (N=273) completed family history, rating scales, and the Mini-International Neuropsychiatric Interview (Sheehan et al., 1998) and the Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children (Kaufman et al., 1997) about youths 5-18 (median=11) years of age presenting to an outpatient clinic. Primary caregivers completed a half-page Family Index of Risk for Mood issues (FIRM). All families completed the FIRM quickly and easily. Most (78%) reported 1+ relatives having a history of mood or substance issues (M=3.7, SD=3.3). A simple sum of familial mood issues discriminated cases with PBD from all other cases (area under receiver operating characteristic [AUROC]=.63, p=.006). FIRM scores were specific to youth mood disorder and not attention-deficit/hyperactivity disorder or disruptive behavior disorder. FIRM scores significantly improved the detection of PBD even controlling for rating scales. No subset of family risk items performed better than the total. Family history information showed clinically meaningful discrimination of PBD. Two different approaches to clinical interpretation showed validity in these clinically realistic data. Inexpensive and clinically practical methods of gathering family history can help to improve the detection of PBD.

Portability of a screener for pediatric bipolar disorder to a diverse setting

Robust screening measures that perform well in different populations could help improve the accuracy of diagnosis of pediatric bipolar disorder. Changes in sampling could influence the performance of items and potentially influence total scores enough to alter the predictive utility of scores. Additionally, creating a brief version of a measure by extracting items from a longer scale might cause differential functioning due to context effects. The authors of current study examined both sampling and context effects of a brief measure of pediatric mania. Caregivers of 813 youths completed the parent-reported version of the General Behavior Inventory (PGBI) at an academic medical center sample enriched for mood disorders. Caregivers of 481 youths completed the PGBI at a community mental health center. Caregivers of 799 youths completed 10 items extracted from the PGBI at a community setting. Caregivers of 159 youths completed both versions of the PGBI and a semistructured diagnostic interview. Differential item functioning indicated that across samples some items functioned differently; however, total observed scores were similar across all levels of mania. Receiver operating characteristic analysis indicated that the 10 extracted items discriminated bipolar disorder from nonbipolar behavior as well as when the items were embedded within the full measure. Findings suggest that the extracted items perform similarly to the embedded items in the community setting. Measurement properties appear sufficiently robust across settings to support clinical applications.

Internal consistency and associated characteristics of informant discrepancies in clinic referred youths age 11 to 17 years

In this study, we examined the internal consistency of informant discrepancies in reports of youth behavior and emotional problems and their unique relations with youth, caregiver, and family characteristics. In a heterogeneous multisite clinic sample of 420 youths (ages 11-17 years), high internal consistency estimates were observed across measures of informant discrepancies. Further, latent profile analyses identified systematic patterns of discrepancies, characterized by their magnitude and direction (i.e., which informant reported greater youth problems). In addition, informant discrepancies systematically and uniquely related to informants' own perspectives of youth mood problems, and these relations remained significant after taking into account multiple informants' reports of informant characteristics widely known to relate to informant discrepancies. These findings call into question the prevailing view of informant discrepancies as indicative of unreliability and/or bias on the part of informants' reports of youths' behavior.

Child Behavior Checklist Juvenile Bipolar Disorder (CBCL-JBD) and CBCL Posttraumatic Stress Problems (CBCL-PTSP) scales are measures of a single dysregulatory syndrome

BACKGROUND

The Child Behavior Checklist Juvenile Bipolar Disorder (CBCL-JBD) profile and Posttraumatic Stress Problems (CBCL-PTSP) scale have been used to assess juvenile bipolar disorder (JBD) and posttraumatic stress disorder (PTSD), respectively. However, their validity is questionable according to previous research. Both measures are associated with severe psychopathology often encompassing multiple DSM-IV diagnoses. Further, children who score highly on one of these scales often have elevated scores on the other, independent of PTSD or JBD diagnoses. We hypothesized that the two scales may be indicators of a single syndrome related to dysregulated mood, attention, and behavior. We aimed to describe and identify the overlap between the CBCL-JBD profile and CBCL-PTSP scales.

METHOD

Two thousand and twenty-nine (2029) children from a nationally representative sample (1073 boys, 956 girls; mean age = 11.98; age range = 6-18) were rated on emotional and behavior problems by their parents using the CBCL. Comparative model testing via structural equation modeling was conducted to determine whether the CBCL-JBD profile and CBCL-PTSP scale are best described as measuring separate versus unitary constructs. Associations with suicidality and competency scores were also examined.

RESULTS

The CBCL-JBD and CBCL-PTSP demonstrated a high degree of overlap (r = .89) at the latent variable level. The best fitting, most parsimonious model was one in which the CBCL-JBD and CBCL-PTSP items identified a single latent construct, which was associated with higher parental endorsement of child suicidal behavior, and lower functioning.

CONCLUSIONS

The CBCL-JBD profile and CBCL-PTSP scale overlap to a remarkable degree, and may be best described as measures of a single syndrome. This syndrome appears to be related to severe psychopathology, but may not conform to traditional DSM-IV classification. These results contribute to the ongoing debate about the utility of the CBCL-JBD and CBCL-PTSP profiles, and offer promising methods of empirically based measurement of disordered self-regulation in youth.

The assessment of children and adolescents with bipolar disorder

The overarching goal of this article is to examine the current best evidence for assessing bipolar disorder (BPD) in children and adolescents and provide a comprehensive, evidence-based approach to diagnosis. Evidence- based assessment strategies are organized around the "3 Ps" of clinical assessment: Predict important criteria or developmental trajectories, Prescribe a change in treatment choice, and inform Process of treating the youth and his/her family. The review characterizes BPD in youths-specifically addressing bipolar diagnoses and clinical subtypes; it then provides an actuarial approach to assessment using prevalence of disorder, risk factors, and questionnaires; discusses treatment thresholds; and identifies practical measures of process and outcomes. The clinical tools and risk factors selected for inclusion in this review represent the best empirical evidence in the literature. By the end of the article, clinicians will have a framework and set of clinically useful tools with which to effectively make evidence-based decisions regarding the diagnosis of BPD in children and adolescents.

A quantitative and qualitative review of neurocognitive performance in pediatric bipolar disorder

Bipolar disorder (BD) is an increasingly prevalent diagnosis in youth. As a result, there has been a corresponding increase in interest about neuropsychological and cognitive profiles in children and adolescents diagnosed with BD. Meta-analysis of the existing literature comparing individuals with BD to healthy controls indicated that the largest differences are observed for measures of verbal memory (d = 0.77). Moderate differences were found in the areas of attention (d = 0.62), executive functioning (d = 0.62), working memory (d = 0.60), visual memory (d = 0.51), visual perceptual skills (d = 0.48), and verbal fluency (d = 0.45). Small differences were found for measures of reading (d = 0.40), motor speed (d = 0.33), and full-scale intelligence quotient (IQ) (d = 0.32). Often, few studies have provided relevant information for a particular neurocognitive domain. Despite this, several domains displayed heterogeneity of effect sizes across studies. Methodological factors explained the variance in effect sizes to different extents depending upon the cognitive domain. The changing influence of method artifacts is likely due to variable coverage of cognitive domains across studies and the use of different measures across studies. Findings are consistent with previous meta-analyses of the adult BD neurocognitive literature, suggesting that many of the deficits observed in adults are present earlier in the course of the illness. Study reporting guidelines are offered that may help clarify the impact of illness definitions, mood state, medication status, and other methodological variables on neurocognition in pediatric BD.

Comparing the psychometric properties of multiple teacher report instruments as predictors of bipolar disorder in children and adolescents

The psychometric properties of four teacher report measures and their utility for accurate diagnosis of pediatric bipolar spectrum disorders (BPSDs) were examined. Participants were 191 youth (65% male; 62% African-American; 23% diagnosed with a BPSD), age 5-18 (M=10.16, SD=3.27) years, 70% recruited from a community mental health center and 30% recruited from a mood disorders clinic. Teachers "who knew the child best" were asked to complete the Achenbach Teacher Report Form (TRF) as well as teacher versions of the General Behavior Inventory (T-GBI), the Child Mania Rating Scale (CMRS-T), and the Young Mania Rating Scale (T-YMRS). Teacher response rates and missing data varied significantly depending on the age of the child. Exploratory factor analysis identified stable and interpretable factors; however, receiver operating characteristic (ROC) and logistic regression analyses showed that teacher report measures were not able to discriminate BPSD cases from non-BPSD cases, or from attention deficit hyperactivity disorder (ADHD) cases. Teacher report appears to be insufficiently specific or sensitive to BPSD for clinical diagnostic use, although teacher scales might have research utility.