The Case for Cognitive Interviewing in Survey Item Validation: A Useful Approach for Improving the Measurement and Assessment of Substance Use Disorders

Highly Involved Parenting of Adolescents With ADHD: Examination of the Psychometric Properties of a Measure of "Helicopter Parenting"

OBJECTIVE

The concept of the "helicopter parent" was popularized in the 2000s and 2010s by Western culture, and it has recently begun to be examined by researchers to describe parental over-involvement and intrusive behavior that impedes transition into adulthood. Research has yet to investigate the viability of this construct for adolescents when parenting is needed to facilitate the development of autonomy. The present study examined the psychometric structure of a modified "helicopter parenting" measure adapted for use in a sample with increased likelihood of highly involved parenting: adolescents with ADHD.

METHODS

Adolescents (n = 333; age 13-18 years; 25% female) and their parents (n = 341, 91% female) completed a survey for a study on provider training in stimulant diversion prevention in 2016 and 2017. We modified a previously validated measure of "helicopter parenting" for young adults. Other previously established parenting measures were included. We conducted principal component analysis for both informants' reports of the modified measure. We examined associations between the components and informants' demographic characteristics and parenting measures to begin to examine convergent and discriminant validity.

RESULTS

Two components were identified for adolescent and parent reports and labeled parental Intervention and Day-to-day Monitoring and Planning. These components were differentially associated with demographic characteristics and other measures of parenting. For example, across reporters, parents exhibited less Day-to-Day Monitoring and Planning for older adolescents. Racially/ethnically minoritized parents and male adolescents reported more Intervention parenting. Modest-sized statistically significant associations were found between these indicators of highly involved parenting and the other measures of parenting.

CONCLUSION

Findings provide initial evidence of construct validity. Future work with more heterogeneous samples should examine if this measure captures adaptive parenting, or behaviors that interfere with developing independence, for adolescents with ADHD and neurotypically developing adolescents.

Examining the validity of the addictions neuroclinical assessment domains in a crowdsourced sample of adults with current alcohol use

Several dimensional frameworks for characterizing heterogeneity in alcohol use disorder (AUD) have been proposed, including the Addictions Neuroclinical Assessment (ANA). The ANA is a framework for assessing individual variability within AUD across three domains corresponding to the proposed stages of the addiction cycle: reward (binge-intoxication stage), negative emotionality (withdrawal-negative affect stage), and cognitive control (preoccupation-anticipation stage). Recent work has evaluated the ANA's three-factor structure and construct validity, primarily in treatment-seekers with AUD. We extended this research by examining the factor structure, bias across alcohol use severity, longitudinal invariance, and concurrent and predictive validity of a novel assessment of the ANA domains in adults with past 12-month regular (10 + alcohol units/week) alcohol use. Participants recruited from Prolific (N = 732), a crowdsourced data collection platform, completed various self-report measures. A test-retest subsample (n = 234) completed these measures 30 days later. Split-half exploratory factor analysis and confirmatory factor analysis supported the three-factor structure of the ANA. The overall factor structure was invariant across 30 days. Concurrently and prospectively, ANA domains demonstrated convergent validity concerning theoretically aligned alcohol-related, psychological, and personality measures. However, there was evidence of poor discriminant validity, and several cognitive control and reward items demonstrated bias across alcohol use severity. Future research is needed to improve the measurement of ANA domains using multimodal indicators, examine longitudinal changes in domains and their relationship with alcohol use severity, characterize phenotypic subgroups based on relative levels of domains, and compare the utility of the ANA with other proposed frameworks for measuring AUD heterogeneity. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Survey Uses May Influence Survey Responses

Traditional validation processes for psychological surveys tend to focus on analyzing item responses instead of the cognitive processes that participants use to generate these responses. When screening for invalid responses, researchers typically focus on participants who manipulate their answers for personal gain or respond carelessly. In this paper, we introduce a new invalid response process, discordant responding, that arises when participants disagree with the use of the survey and discuss similarities and differences between this response style and protective responding. Results show that nearly all participants reflect on the intended uses of an assessment when responding to items and may decline to respond or modify their responses if they are not comfortable with the way the results will be used. Incidentally, we also find that participants may misread survey instructions if they are not interactive. We introduce a short screener to detect invalid responses, the discordant response identifiers (DRI), which provides researchers with a simple validity tool to use when validating surveys. Finally, we provide recommendations about how researchers may use these findings to design surveys that reduce this response manipulation in the first place.

Validation of a brief screener for broad-spectrum mental and substance-use disorders in South Africa

In low-resource settings, valid mental health screening tools for non-specialists can be used to identify patients with psychiatric disorders in need of critical mental health care. The Mental Wellness Tool-13 (mwTool-13) is a 13-item screener for identifying adults at risk for common mental disorders (CMDs) alcohol-use disorders (AUDs), substance-use disorders (SUD), severe mental disorders (SMDs), and suicide risk (SR). The mwTool-13 is administered in two steps, specifically, only those who endorse any of the initial three questions receive the remaining ten questions. We evaluated the performance of mwTool-13 in South Africa against a diagnostic gold standard. We recruited a targeted, gender-balanced sample of adults, aged ≥18 years at primary and tertiary healthcare facilities in Eastern Cape Province. Of the 1885 participants, the prevalence of CMD, AUD, SMD, SR, and SUD was 24.4%, 9.5%, 8.1%, 6.0%, and 1.6%, respectively. The mwTool-13 yielded high sensitivities for CMD, SMD, and SR, but sub-optimal sensitivities for AUD and SUD (56.7% and 64.5%, respectively). Including a single AUD question in the initial question set improved the tool's performance in identifying AUD and SUD (sensitivity > 70%), while maintaining brevity, face-validity, and simplicity in the South African setting.

Longitudinal measurement invariance of constructs derived from the addiction cycle

OBJECTIVE

The Alcohol Addiction Research Domain Criteria (AARDoC) is an organizational framework for assessing heterogeneity in addictive disorders organized across the addiction cycle domains of incentive salience, negative emotionality, and executive functioning and may have benefits for precision medicine. Recent work found pretreatment self-report items mapped onto the addiction cycle domains and predicted 1- and 3-year alcohol use disorder treatment outcomes. Given the potential utility of the addiction cycle domains for predicting relevant treatment outcomes, this study sought to evaluate the longitudinal measurement invariance of the domains.

METHOD

We conducted a secondary analysis of individuals with alcohol use disorder (n = 1,383, 30.9% female, 76.8% non-Hispanic White, 11.2% Hispanic) who participated in the COMBINE study. Eleven items assessed at pre- and posttreatment were included in exploratory structural equation modeling (ESEM) and longitudinal invariance analyses.

RESULTS

The pre- and posttreatment ESEM models had factor loadings consistent with the three addiction cycle domains and fit the data well. The ESEM factor structure was invariant from pre- to posttreatment (representing configural invariance) and metric invariance (factor loadings) was largely supported, but analyses failed to support scalar invariance (item-level thresholds) of the addiction cycle domains.

CONCLUSIONS

A three-factor structure representing addiction cycle domains can be modeled using brief self-report measures pre- and posttreatment. Individuals demonstrated a downward shift in the level of item endorsement, indicating improvement with treatment. Although this 11-item measure might be useful at baseline for informing treatment decisions, results indicate the need to exercise caution in comparing the addiction cycle domains pre- to posttreatment within persons. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Student perceptions of inclusive pedagogy in undergraduate STEM classrooms

In university STEM classrooms, the incorporation of inclusive practices improves student performance, decreases disparities in the academic success of underrepresented students, and increases student retention and persistence in Science, Technology, Engineering, and Mathematics (STEM) programs. Inclusive pedagogical practices include effective instructional choices like active learning, providing rubrics, and other strategies that have been shown to support students from disadvantaged backgrounds. Additionally, explicitly inclusive practices such as addressing microaggressions and sharing pronouns can promote a sense of belonging for students. While a plethora of literature has shown these impacts and faculty have access to resources and training about inclusive pedagogy, we were interested in whether students are noticing these practices and how student identities impact their observations of instructional practices. We surveyed undergraduates (n = 74) from diverse STEM disciplines at a large land-grant university regarding their observation of 11 different inclusive pedagogical practices. Overall, students observed inclusive instructional practices more often than they observed explicitly diversity, equity, and inclusion (DEI)-related practices. For explicitly DEI-related practices, white students observed more practices than Students of Color. This suggests that more work needs to be done to train faculty in explicit DEI-related practices, especially with the goal of supporting Students of Color who have been historically excluded from STEM.

Constructs derived from the addiction cycle predict alcohol use disorder treatment outcomes and recovery 3 years following treatment

OBJECTIVE

The addiction cycle has been proposed as a framework for understanding the progression of alcohol use disorder (AUD) in terms of psychological and biological domains, including reward drinking/incentive salience, relief drinking/negative emotionality, and loss of control/executive functioning impairment. To have utility in clinical practice, self-report measures of these domains that are applicable across sociodemographic groups and associated with clinical outcomes are needed. This study sought to validate domains from self-report measures and to test whether domains are measurement invariant across sociodemographic groups and associated with treatment outcomes.

METHOD

Secondary analysis of individuals with AUD (n = 3,092) who participated in two alcohol clinical trials, Project Matching Alcohol Treatment to Client Heterogeneity (MATCH) and COMBINE. Factor analytic methods were used to derive addiction cycle domains at baseline. These domains were then examined as predictors of outcomes.

RESULTS

Fifteen self-report items were used as indicators of the addiction cycle domains, with sociodemographic differences in measurement by sex, age, race, education, and AUD symptoms. Relief/negative emotionality and reward/incentive salience were significantly associated with outcomes at 1 and 3 years following treatment, and executive functioning also predicted nonabstinent recovery at 3 years.

CONCLUSIONS

The results support the utility of domains relevant to the addiction cycle in predicting AUD treatment outcomes and recovery among individuals who sought treatment for AUD. The addiction cycle domains were more strongly associated with outcomes than other measures clinicians might use to predict outcomes (e.g., AUD symptoms). Future research should continue to develop and refine the items and test whether the addiction cycle domains can inform treatment planning. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Self-report assessment of alcohol sensitivity: An examination of the effects of different probes

OBJECTIVE

Level of response (LOR) to alcohol is associated with several alcohol-related risk factors and outcomes. However, existing self-report measures of LOR have important limitations. For example, the Self-Rating of the Effects of Alcohol Scale assesses a limited range of alcohol-related effects. Although the Alcohol Sensitivity Questionnaire (ASQ) samples a broader range of effects, it uses different probes across effects, confounding type of effect with method variation associated with the use of different probes. Focusing on the ASQ, we systematically evaluate variation in estimated LOR as a function of how number of drinks to achieve an effect is probed. Our approach addresses a major limitation of existing LOR measures which fail to account for sensitivity variability across drinking occasions.

METHOD

This study randomized 732 adult drinkers into one of four versions of the ASQ that assessed sensitivity to 15 alcohol-related effects, systematically varying the follow-up probes.

RESULTS

Accounting for (a) the minimum number of drinks consumed before feeling an effect and (b) the maximum number of drinks consumed without feeling an effect for all effects is superior to the original ASQ approach in predicting relevant outcomes.

CONCLUSIONS

Assessments of sensitivity should probe for minimum and maximum number of drinks across each of the effects. If impractical to probe for both, consistently probing for maximum number of drinks is desirable. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

What do Veterans with homeless experience want us to know that we are not asking? A qualitative content analysis of comments from a national survey of healthcare experience

Surveys of people who experience homelessness can portray their life and healthcare experiences with a level of statistical precision; however, few have explored how the very same surveys can deliver qualitative insights as well. In responding to surveys, people experiencing homelessness can use the margins to highlight health and social concerns that investigators failed to anticipate that standard question batteries miss. This study describes the unprompted comments of a large national survey of Veterans with homeless experiences. The Primary Care Quality-Homeless Services Tailoring (PCQ-HOST) survey presented 85 close-ended items to solicit social and psychological experiences, health conditions, and patient ratings of primary care. Amongst 5377 Veterans responding to the paper survey, 657 (12%) offered 1933 unprompted comments across nearly all domains queried. Using a team-based content analysis approach, we coded and organised survey comments by survey domain, and identified emergent themes. Respondents used comments for many purposes. They noted when questions called for more nuanced responses than those allowed, especially 'sometimes' or 'not applicable' on sensitive questions, such as substance use, where recovery status was not queried. On such matters, the options of 'no' and 'yes' failed to capture important contextual and historical information that mattered to respondents, such as being in recovery. Respondents also elaborated on negative and positive care experiences, often naming specific clinics or clinicians. This study highlights the degree to which members of vulnerable populations, who participate in survey research, want researchers to know the reasons behind their responses and topics (like chronic pain and substance use disorders) that could benefit from open-ended response options. Understanding patient perspectives can help improve care. Quantitative data from surveys can provide statistical precision but may miss key patient perspectives. The content that patients write into survey margins can highlight shortfalls of a survey and point towards future areas of inquiry. Veterans with homeless experience want to provide additional detail about their lives and care experiences in ways that transcend the boundaries of close-ended survey questions. Questions on substance use proved especially likely to draw comments that went beyond the permitted response options, often to declare that the respondent was in recovery. Respondents frequently clarified aspects of their care experiences related to pain, pain care, transportation and experiences of homelessness.

A mixed-methods approach to improve the measurement of alcohol-induced blackouts: ABOM-2

BACKGROUND

Alcohol-induced blackouts describe memory loss resulting from alcohol consumption. Approximately half of college students report experiencing a blackout in their lifetime. Blackouts are associated with an increased risk for negative consequences, including serious injury. Research has documented two types of blackouts, en bloc (EB) and fragmentary (FB). However, research is limited by the lack of a validated measure that differentiates between these two forms of blackout. This study used a mixed-methods approach to improve the assessment of FB and EB among young adults. Specifically, we sought to improve the existing Alcohol-Induced Blackout Measure (ABOM), which was derived from a relatively small pool of items that did not distinguish FB from EB.

METHODS

Study 1 used three rounds of cognitive interviewing with U.S. college students (N = 31) to refine existing assessment items. Nineteen refined blackout items were retained for Study 2. Study 2 used face validity, factor analysis, item response theory, and external validation analyses to test the two-factor blackout model among U.S. heavy-drinking college students (N = 474) and to develop and validate a new blackout measure (ABOM-2).

RESULTS

Iterative factor analyses demonstrated that the items were well represented by correlated EB and FB factors, consistent with our hypothesis. External validation analyses demonstrated convergent and discriminant validity. These analyses also provided preliminary evidence for the two factors having differential predictive validity (e.g., FB correlated with enhancement drinking motives, while EB correlated with coping and conformity motives).

CONCLUSIONS

The Alcohol-Induced Blackout Measure-2 (ABOM-2) improves the measurement of blackout experiences among college students. Its use could facilitate the examination of EB and FB as differential predictors of alcohol-related outcomes in future studies.

Psychographic segmentation to identify higher-risk teen peer crowds for health communications: Validation of Virginia's Mindset Lens Survey

Audience segmentation is necessary in health communications to ensure equitable resource distribution. Peer crowds, which are macro-level teen subcultures, are effective psychographic segments for health communications because each crowd has unique mindsets, values, norms, and health behavior profiles. These mindsets affect behaviors, and can be used to develop targeted health communication campaigns to reach those in greatest need. Though peer crowd research is plentiful, no existing peer crowd measurement tool has been formally validated. As such, we developed and validated Virginia's Mindset Lens Survey (V-MLS), a mindset-based teen peer crowd segmentation survey to support health communication efforts. Using an online convenience sample of teens (N = 1,113), we assessed convergent and discriminant validity by comparing the V-MLS against an existing, widely-used peer crowd survey (I-Base Survey^®) utilizing a multi-trait multi-method matrix. We also examined the V-MLS's predictive ability through a series of regressions using peer crowd scores to predict behaviors, experiences, and traits relevant to health communication campaign planning. The V-MLS demonstrated reliability and convergent and discriminant validity. Additionally, the V-MLS effectively distinguished teen peer crowds with unique health behaviors, experiences, and personal traits. When combined with appropriate information processing and campaign development frameworks, this new tool can complement existing instruments to inform message framing, tone, and style for campaigns that target at-risk teens to increase campaign equity and reach.