Is the Finding Too Good to Be True? Moving from "More Is Better" to Thinking in Terms of Simple Predictions and Credibility

Assessment of suicidal thoughts and behaviors in adults: A systematic review of measure psychometric properties and implications for clinical and research utility

High-quality clinical care and research on suicidal thoughts and behaviors (STBs) depends on availability and implementation of reliable and valid measures of STBs. In contrast to studies examining STB risk factors, screening instruments, or treatment, little research has rigorously examined the content, characteristics, and psychometric properties of STB measures themselves. This systematic review (1) identified STB measures that conform to empirically supported definitions of STBs, and (2) identified peer-reviewed papers reporting on the psychometric properties of these measures in adults. Data on psychometric properties and other measure characteristics were extracted. A total of 21 eligible measures were identified in the first stage. In the second stage, 70 articles (with 79 independent samples) were included with psychometric data in adult samples for 19 measures. Although there was support for strong internal consistency and content validity across many measures, face validity and clinical utility concerns were prevalent. Few measures comprehensively assessed suicidal behaviors, and interview-based assessments tended to show the strongest psychometric properties and clinical utility. Findings are discussed in the context of recommendations for improving existing measures, including future research to increase utility and translatability across clinical settings, delivery methods, and diverse populations.

Measures of Suicidal Thoughts and Behaviors in Children and Adolescents: A Systematic Review and Recommendations for Use in Clinical and Research Settings

Empirically supported measures of suicidal thoughts and behaviors (STBs) are needed to serve as reference outcomes for suicide risk screening tools and to monitor severity and treatment progress in children and adolescents with STBs. The present paper systematically reviewed existing measures of STBs in youth and studies evaluating their psychometric properties and clinical utility. Measures were then evaluated on reliability, validity, and clinical utility. Sixteen articles (20 independent samples) were found with psychometric data with youth samples for eight measures. Interview-based measures were found to have the strongest psychometric support and clinical utility. Significant limitations exist for all self-report measures due to inherent characteristics of these measures that cannot be remedied through additional psychometric study. There is an urgent need for the development and validation of new self-report measures of STBs, particularly for preadolescent children, sexual and gender minority youth, and racial/ethnic minority youth.

The Brief Version of the Mentalization Scale (MentS-12): Evidence-Based Assessment of Mentalizing Capacity

Recognizing the need for a concise self-report measure of mentalizing capacity, we developed a 12-item iteration of the well-established Mentalization Scale (MentS). Using college student and community samples of Serbian adults (N = 566), we performed a precise selection of items and then examined the psychometric attributes of the shortened scale (MentS-12). The new scale maintains the original three-dimensional structure: self-related mentalization, other-related mentalization, and motivation to mentalize. MentS-12 proves to be both reliable and structurally consistent. To improve its utility in therapeutic contexts, we determined clinical change thresholds for both the complete and abbreviated forms. We hope that its feasibility stimulates the integration of the MentS-12 in longitudinal research projects and real-world clinical settings.

Development and validation of an 8-item version of the Real Relationship Inventory-Client form

OBJECTIVE

To develop and validate a very brief version of the 24-item Real Relationship Inventory-Client (RRI-C) form.

METHOD

Two independent samples of individual psychotherapy patients (Nsample1 = 700, Nsample2 = 434) completed the RRI-C along with other measures. Psychometric scale shortening involved exploratory factor analysis, item response theory analysis, confirmatory factor analysis (CFA), and multigroup CFA. Reliability and convergent and discriminant validity of the scale and subscales were also assessed.

RESULTS

The 8-item RRI-C (RRI-C-SF) preserves the two-factor structure: Genuineness (k = 4, α = .86) and Realism (k = 4, α = .87), which were correlated at r = .74. CFA provided the following fit indices for the bifactor model: X2/df = 2.16, CFI = .99, TLI = .96, RMSEA = .07, and SRMR = .03. Multigroup CFA showed that the RRI-C-SF was invariant across in-person and remote session formats. The RRI-C-SF demonstrated high reliability (α = .91); high correlation with the full-length scale (r = .96); and excellent convergent and discriminant validity with measures of other elements of the therapeutic relationship, personality characteristics, current mental health state, and demographic-clinical variables. Clinical change benchmarks were calculated to serve as valuable tools for both research and clinical practice.

CONCLUSION

The RRI-C-SF is a reliable measure that can be used for both research and clinical purposes. It enables a nuanced assessment of the genuineness and the realism dimensions of the real relationship.

Development of webcam-collected and artificial-intelligence-derived social and cognitive performance measures for neurodevelopmental genetic syndromes

This study focused on the development and initial psychometric evaluation of a set of online, webcam-collected, and artificial intelligence-derived patient performance measures for neurodevelopmental genetic syndromes (NDGS). Initial testing and qualitative input was used to develop four stimulus paradigms capturing social and cognitive processes, including social attention, receptive vocabulary, processing speed, and single-word reading. The paradigms were administered to a sample of 375 participants, including 163 with NDGS, 56 with idiopathic neurodevelopmental disability (NDD), and 156 neurotypical controls. Twelve measures were created from the four stimulus paradigms. Valid completion rates varied from 87 to 100% across measures, with lower but adequate completion rates in participants with intellectual disability. Adequate to excellent internal consistency reliability (α = 0.67 to 0.95) was observed across measures. Test-retest reproducibility at 1-month follow-up and stability at 4-month follow-up was fair to good (r = 0.40-0.73) for 8 of the 12 measures. All gaze-based measures showed evidence of convergent and discriminant validity with parent-report measures of other cognitive and behavioral constructs. Comparisons across NDGS groups revealed distinct patterns of social and cognitive functioning, including people with PTEN mutations showing a less impaired overall pattern and people with SYNGAP1 mutations showing more attentional, processing speed, and social processing difficulties relative to people with NFIX mutations. Webcam-collected performance measures appear to be a reliable and potentially useful method for objective characterization and monitoring of social and cognitive processes in NDGS and idiopathic NDD. Additional validation work, including more detailed convergent and discriminant validity analyses and examination of sensitivity to change, is needed to replicate and extend these observations.

The Autism Symptom Dimensions Questionnaire: Development and psychometric evaluation of a new, open-source measure of autism symptomatology

AIM

To describe the development and initial psychometric evaluation of a new, freely available measure, the Autism Symptom Dimensions Questionnaire (ASDQ).

METHOD

After development and revision of an initial 33-item version, informants completed a revised 39-item version of the ASDQ on 1467 children and adolescents (aged 2-17 years), including 104 with autism spectrum disorder (ASD).

RESULTS

The initial 33-item version of the ASDQ had good reliability and construct validity. However, only four specific symptom factors were identified, potentially due to an insufficient number of items. Factor analyses of the expanded instrument identified a general ASD factor and nine specific symptom factors with good measurement invariance across demographic groups. Scales showed good-to-excellent overall and conditional reliability. Exploratory analyses of predictive validity for ASD versus neurotypical and other developmental disability diagnoses indicated good accuracy for population and at-risk contexts.

INTERPRETATION

The ASDQ is a free and psychometrically sound informant report instrument with good reliability of measurement across a continuous range of scores and preliminary evidence of predictive validity. The measure may be a useful alternative to existing autism symptom measures but further studies with comparison of clinical diagnoses using criterion-standard instruments are needed.

WHAT THIS PAPER ADDS

The Autism Symptom Dimensions Questionnaire (ASDQ) is a new, freely available measure of autism symptoms. The ASDQ showed reliable and accurate measurement of autism symptoms. The measure had good screening efficiency for autism spectrum disorder relative to other developmental conditions.

Profiles of Suicidal Ideation Among Black Male Adolescents: Examination of Individual and Socioecological Predictors

OBJECTIVE

The current study utilized latent profile analysis to identify distinct profiles of suicidal ideation among Black male adolescents and compared profiles on socioecological determinants of suicide and psychological symptoms.

METHOD

A sample of 457 Black male adolescents (mean age = 15.31, SD = 1.26) completed self-report measures of suicidal ideation, racial discrimination, community violence exposure, anxiety symptoms, depressive symptoms, and posttraumatic stress symptoms.

RESULTS

Results of the latent profile analysis revealed a three-profile model: a low ideation profile, with low levels of all forms of suicidal ideation; a general death ideation profile with elevated general thoughts of death and dying; and a high, concealed ideation profile with high levels on all suicidal ideation items, except communicating the ideation to others. ANOVAs revealed that levels of psychological symptoms were significantly different for each profile, with the high, concealed ideation profile showing the highest levels. The low ideation profile had significantly lower scores than the two other profiles on community violence exposure, but the other two profiles did not differ significantly from one another. Further, the general death ideation profile had significantly higher scores on racial discrimination than the other two profiles, but the other two profiles did not differ significantly from one another.

CONCLUSIONS

The current study supports recent socio-cultural theories of suicidal ideation and behavior in Black youth and highlights the need for increased access to care and services for Black boys who are exposed to socioecological factors that heighten suicidal ideation.

The Proposed Specifiers for Conduct Disorder - Parent (PSCD-P): Convergent Validity, Incremental Validity, and Reactions to Unfamiliar Peer Confederates

Youth who experience psychopathy display multiple impairments across interpersonal (grandiose-manipulative [GM]), affective (callous-unemotional [CU]), lifestyle (daring-impulsive [DI]), and potentially antisocial and behavioral features. Recently, it has been acknowledged that the inclusion of psychopathic features can offer valuable information in relation to the etiology of Conduct Disorder (CD). Yet, prior work largely focuses on the affective component of psychopathy, namely CU. This focus creates uncertainty in the literature on the incremental value of a multicomponent approach to understanding CD-linked domains. Consequently, researchers developed the Proposed Specifiers for Conduct Disorder (PSCD; Salekin & Hare, 2016) as a multicomponent approach to assess GM, CU, and DI features in combination with CD symptoms. The notion of considering the wider set of psychopathic features for CD specification requires testing whether multiple personality dimensions predict domain-relevant criterion outcomes above-and-beyond a CU-based approach. Thus, we tested the psychometric properties of parents' reports on the PSCD (PSCD-P) in a mixed clinical/community sample of 134 adolescents (M_age = 14.49, 66.4% female). Confirmatory factor analyses resulted in a 19-item PSCD-P displaying acceptable reliability estimates and a bifactor solution consisting of GM, CU, DI, and CD factors. Findings supported the incremental validity of scores taken from the PSCD-P across multiple criterion variables, including (a) an established survey measure of parent-adolescent conflict; and (b) trained independent observers' ratings of adolescents' behavioral reactions to laboratory controlled tasks designed to simulate social interactions with unfamiliar peers. These findings have important implications for future research on the PSCD and links to adolescents' interpersonal functioning.

When Adolescents Experience Co-Occurring Social Anxiety and ADHD Symptoms: Links With Social Skills When Interacting With Unfamiliar Peer Confederates

Adolescents with elevated social anxiety commonly experience peer-related impairments - particularly with same-age, unfamiliar peers - stemming from their avoidant behaviors. Yet, peer-related impairments are not unique to social anxiety. For example, adolescents who experience social anxiety may also experience symptoms of attention deficit/hyperactivity disorder (ADHD), which also increase risk for peer-related impairments. Relative to social anxiety, peer-related impairments linked to ADHD symptoms more likely stem from hyperactivity (i.e., approach behaviors). These distinct pathways point to adolescents with elevated social anxiety and ADHD symptoms (i.e., social anxiety + ADHD) experiencing particularly high peer-related impairments, which commonly manifest as behavioral displays of low social skills when interacting with unfamiliar peers. We tested this notion in a mixed-clinical/community sample of 134 14- to 15-year-old adolescents and their parents. Adolescents participated in a series of social interaction tasks designed to simulate how adolescents interact with same-age, unfamiliar peers. Trained observers independently rated adolescents on observed social skills within these interactions. Both parents and adolescents completed parallel surveys of social anxiety and ADHD symptoms, which we used to identify social anxiety + ADHD adolescents as well as other combinations of social anxiety and ADHD symptoms (i.e., neither, elevated on one but not the other). Adolescents with social anxiety + ADHD displayed significantly lower social skills, relative to all other groups. Among adolescents, social anxiety + ADHD may have a compounding effect on social skills. As such, therapists working with social anxiety + ADHD adolescents should probe for peer-related impairments and factors implicated in the development and maintenance of these impairments.

Integrating multi-informant reports of youth mental health: A construct validation test of Kraemer and colleagues' (2003) Satellite Model

Accurately assessing youth mental health involves obtaining reports from multiple informants who typically display low levels of correspondence. This low correspondence may reflect situational specificity. That is, youth vary as to where they display mental health concerns and informants vary as to where and from what perspective they observe youth. Despite the frequent need to understand and interpret these informant discrepancies, no consensus guidelines exist for integrating informants' reports. The path to building these guidelines starts with identifying factors that reliably predict the level and form of these informant discrepancies, and do so for theoretically and empirically relevant reasons. Yet, despite the knowledge of situational specificity, few approaches to integrating multi-informant data are well-equipped to account for these factors in measurement, and those that claim to be well-positioned to do so have undergone little empirical scrutiny. One promising approach was developed roughly 20 years ago by Kraemer and colleagues (2003). Their Satellite Model leverages principal components analysis (PCA) and strategic selection of informants to instantiate situational specificity in measurement, namely components reflecting variance attributable to the context in which informants observe behavior (e.g., home/non-home), the perspective from which they observe behavior (e.g., self/other), and behavior that manifests across contexts and perspectives (i.e., trait). The current study represents the first construct validation test of the Satellite Model. A mixed-clinical/community sample of 134 adolescents and their parents completed six parallel surveys of adolescent mental health. Adolescents also participated in a series of simulated social interactions with research personnel trained to act as same-age, unfamiliar peers. A third informant (unfamiliar untrained observer) viewed these interactions and completed the same surveys as parents and adolescents. We applied the Satellite Model to each set of surveys and observed high internal consistency estimates for each of the six-item trait (α = 0.90), context (α = 0.84), and perspective (α = 0.83) components. Scores reflecting the trait, context, and perspective components displayed distinct patterns of relations to a battery of criterion variables that varied in the context, perspective, and source of measurement. The Satellite Model instantiates situational specificity in measurement and facilitates unifying conceptual and measurement models of youth mental health.

Evidence Base Update for Questionnaires of Emotion Regulation and Reactivity for Children and Adolescents

Objective: Emotion regulation (ER) is a multi-faceted and dynamic process relevant to both normative emotional development and transdiagnostic emotional dysfunction for a range of psychological disorders. There has been tremendous growth in ER research over the past decade, including the development of numerous new measures to assess ER. This Evidence Base Update included a systematic review to identify self- and informant-report questionnaire measures of ER for children and adolescents, including measures of ER strategies and effectiveness (or emotion dysregulation).Methods: PubMed, PsycInfo, and Health and Psychosocial Instruments databases were searched using the terms emotion OR affect AND regulation OR control OR reactivity OR response, as well as terms related to questionnaires and psychometrics, restricted to articles on youth (< 18 years old). Each measure's psychometrics was evaluated based on modified criteria by De Los Reyes and Langer (2018).Results: Nine-hundred ninety-seven papers were identified yielding 87 measures that met inclusion for review. Although the majority (60%) of identified ER measures could not be recommended based on these criteria, 8% were Excellent, 14% were Good, and 17% were Adequate. The recommended measures included: 11 general ER measures (5 focused on strategies, 5 focused on dysregulation/ effectiveness), 13 measures of ER as it relates to specific emotions or contexts such as irritability or peer stress (4 focused on strategies, 9 focused on dysregulation/effectiveness), and 11 measures of other constructs that include an ER subscale (all focused on dysregulation). Conclusions: The characteristics, strengths, and weaknesses of the recommended ER measures are described in order to guide measure selection for clinical or research uses. A synthesis of themes identified during this review includes commonly observed areas of weakness and gaps in the literature to provide a foundation for future research and measure development.

Using Evaluative Criteria to Review Youth Anxiety Measures, Part II: Parent-Report

This Evidence Base Update of parent-report measures of youth anxiety symptoms is a companion piece to our update on youth self-report anxiety symptom measures (Etkin et al., 2021). We rate the psychometric properties of the parent-report measures as Adequate, Good, or Excellent using criteria developed by Hunsley and Mash (2008) and Youngstrom et al. (2017). Our review reveals that the evidence base for parent-report measures is considerably less developed compared with the evidence base for youth self-report measures. Nevertheless, several measures, the parent-report Screen for Child Anxiety-Related Emotional Disorders, Multidimensional Anxiety Scale for Children, and Spence Children's Anxiety Scale, were found to have Good to Excellent psychometric properties. We conclude our review with suggestions about which parent-report youth anxiety measures are best suited to perform different assessment functions and directions for additional research to expand and strengthen the evidence base.

Using Evaluative Criteria to Review Youth Anxiety Measures, Part I: Self-Report

Evidence-based assessment serves several critical functions in clinical child psychological science, including being a foundation for evidence-based treatment delivery. In this Evidence Base Update, we provide an evaluative review of the most widely used youth self-report measures assessing anxiety and its disorders. Guided by a set of evaluative criteria (De Los Reyes & Langer, 2018), we rate the measures as Excellent, Good, or Adequate across their psychometric properties (e.g., construct validity). For the eight measures evaluated, most ratings assigned were Good followed by Excellent, and the minority of ratings were Adequate. We view these results overall as positive and encouraging, as they show that these youth anxiety self-report measures can be used with relatively high confidence to accomplish key assessment functions. Recommendations and future directions for further advancements to the evidence base are discussed.

Evaluating and Validating GBI Mania and Depression Short Forms for Self-Report of Mood Symptoms

Objective: To evaluate short forms of free self-report mania and depression scales, evaluating their reliability, content coverage, criterion validity, and diagnostic accuracy.Method: Youths age 11 to 18 years seeking outpatient mental health services at either an Academic medical clinic (N = 427) or urban Community mental health center (N = 313), completed the General Behavior Inventory (GBI) and other rating scales. Youths and caregivers completed semi-structured interviews to establish diagnoses and mood symptom severity, with GBI scores masked during diagnosis. Ten- and seven-item short forms, psychometric projections, and observed performance were tested first in the Academic sample and then externally cross-validated in the Community sample.Results: All short forms maintained high reliability (all alphas >.80 across both samples), high correlations with the full-length scales (r.85 to.96), excellent convergent and discriminant validity with mood, behavior, and demographic criteria, and diagnostic accuracy undiminished compared to using the full-length scales. Ten-item scales showed advantages in terms of coverage; the 7 Up showed slightly weaker performance.Conclusions: Present analyses evaluated and externally cross-validated short forms that maintain high reliability and content coverage, and show strong criterion validity and diagnostic accuracy - even when used in an independent sample with very different demographics and referral patterns. The short forms appear useful in clinical applications including initial evaluation, as well as in research settings where they offer an inexpensive quantitative score. Short forms are available in more than two dozen languages. Future work should further evaluate sensitivity to treatment effects and cultural invariance.

Sluggish Cognitive Tempo in Autism Spectrum Disorder, ADHD, and Their Comorbidity: Implications for Impairment

Objective: Sluggish Cognitive Tempo (SCT), characterized by lethargy and daydreaming, has most commonly been studied in community samples and in youth with Attention-Deficit/Hyperactivity Disorder (ADHD). Despite shared neurodevelopmental symptoms with ADHD, few studies have investigated SCT in Autism Spectrum Disorders (ASD). The current study investigated SCT symptoms in youth with ASD, ADHD, and comorbid ASD+ADHD to explore the relations between SCT and global and social impairment.Method: Caregivers of children and adolescents (n = 98; ages 6-17) diagnosed with ADHD (n = 46), ASD (n = 28), or ASD+ADHD (n = 24) completed measures of social impairment, SCT, and demographic variables.Results: All three clinical groups demonstrated comparable levels of SCT. Diagnosis and SCT independently contributed to parent-rated social impairment, while SCT and IQ, but not diagnosis, contributed to clinician-rated global functioning. Specifically, having comorbid ASD+ADHD, but not an ASD or ADHD diagnosis alone, significantly predicted greater social impairment.Conclusion: These results extend previous literature investigating SCT in ASD and provide evidence to suggest that SCT is associated with social and global impairment above and beyond the impairment associated with ADHD and/or ASD. These results may have implications for clinical assessment and treatment of ASD and ADHD.

Evidence Base Update for Brief, Free, and Accessible Youth Mental Health Measures

Evidence-based assessment (EBA) is foundational to high-quality mental health care for youth and is a critical component of evidence-based practice delivery, yet is underused in the community. Administration time and measure cost are barriers to use; thus, identifying and disseminating brief, free, and accessible measures are critical. This Evidence Base Update evaluates the empirical literature for brief, free, and accessible measures with psychometric support to inform research and practice with youth. A systematic review using PubMed and PsycINFO identified measures in the following domains: overall mental health, anxiety, depression, disruptive behavior, traumatic stress, disordered eating, suicidality, bipolar/mania, psychosis, and substance use. To be eligible for inclusion, measures needed to be brief (50 items or less), free, accessible, and have psychometric support for their use with youth. Eligible measures were evaluated using adapted criteria established by De Los Reyes and Langer (2018) and were classified as having excellent, good, or adequate psychometric properties. A total of 672 measures were identified; 95 (14%) met inclusion criteria. Of those, 21 (22%) were "excellent," 34 (36%) were "good," and 40 (42%) were "adequate." Few measures had support for their use to routinely monitor progress in therapy. Few measures with excellent psychometric support were identified for disordered eating, suicidality, psychosis, and substance use. Future research should evaluate existing measures for use with routine progress monitoring and ease of implementation in community settings. Measure development is needed for disordered eating, suicidality, psychosis, and substance use to increase availability of brief, free, accessible, and validated measures.