Impact of analytic decisions on test-retest reliability of individual and group estimates in functional magnetic resonance imaging: a multiverse analysis using the monetary incentive delay task

Empirical studies reporting low test-retest reliability of individual blood oxygen-level dependent (BOLD) signal estimates in functional magnetic resonance imaging (fMRI) data have resurrected interest among cognitive neuroscientists in methods that may improve reliability in fMRI. Over the last decade, several individual studies have reported that modeling decisions, such as smoothing, motion correction and contrast selection, may improve estimates of test-retest reliability of BOLD signal estimates. However, it remains an empirical question whether certain analytic decisions consistently improve individual and group level reliability estimates in an fMRI task across multiple large, independent samples. This study used three independent samples ( N s: 60, 81, 119) that collected the same task (Monetary Incentive Delay task) across two runs and two sessions to evaluate the effects of analytic decisions on the individual (intraclass correlation coefficient [ICC(3,1)]) and group (Jaccard/Spearman rho ) reliability estimates of BOLD activity of task fMRI data. The analytic decisions in this study vary across four categories: smoothing kernel (five options), motion correction (four options), task parameterizing (three options) and task contrasts (four options), totaling 240 different pipeline permutations. Across all 240 pipelines, the median ICC estimates are consistently low, with a maximum median ICC estimate of .43 - .55 across the three samples. The analytic decisions with the greatest impact on the median ICC and group similarity estimates are the Implicit Baseline contrast, Cue Model parameterization and a larger smoothing kernel. Using an Implicit Baseline in a contrast condition meaningfully increased group similarity and ICC estimates as compared to using the Neutral cue. This effect was largest for the Cue Model parameterization, however, improvements in reliability came at the cost of interpretability. This study illustrates that estimates of reliability in the MID task are consistently low and variable at small samples, and a higher test-retest reliability may not always improve interpretability of the estimated BOLD signal.

Individual and Community level Developmental Adversities: Associations with Marijuana and Alcohol Use in Late-Adolescents and Young Adults

Exposure to community and individual level stressors during adolescence has been reported to be associated with increased substance use. However, it remains unclear what the relative contribution of different community- and individual-level factors play when alcohol and marijuana use become more prevalent during late adolescence. The present study uses a large longitudinal sample of adolescents (Wave 1: N = 2017; 55% Female; 54.5% White, 22.3% Black, 8% Hispanic, 15% other) to evaluate the association and potential interactions between community- and individual-level factors and substance use from adolescence to young adulthood (Wave 1 to Wave 3 Age Mean [SD]: 16.7 [1.1], 18.3 [1.2], 19.3 [1.2]). Across three waves of data, multilevel modeling (MLM) is used to evaluate the association between community affluence and disadvantage, individual household socioeconomic status (SES, measured as parental level of education and self-reported public assistance) and self-reported childhood maltreatment with self-reported 12-month alcohol and 12-month marijuana use occasions. Sample-selection weights and attrition-adjusted weights are accounted for in the models to evaluate the robustness of the estimated effects. Across the MLMs, there is a significant positive association between community affluence and parental education with self-reported alcohol use but not self-reported marijuana use. In post hoc analyses, higher neighborhood affluence in older adolescents is associated with higher alcohol use and lower use in younger adolescents; the opposite association is found for neighborhood disadvantage. Consistent with past literature, there is a significant positive association between self-reported childhood maltreatment and self-reported 12-month alcohol and 12-month marijuana use. Results are largely consistent across weighted and unweighted analyses, however, in weighted analyses there is a significant negative association between community disadvantage and self-reported 12-month alcohol use. This study demonstrates a nuanced relationship between community- and individual-level factors and substance use during the transitional window of adolescence which should be considered when contextualizing and interpreting normative substance use during adolescence.

A multi-sample evaluation of the measurement structure and function of the modified monetary incentive delay task in adolescents

Interpreting the neural response elicited during task functional magnetic resonance imaging (fMRI) remains a challenge in neurodevelopmental research. The monetary incentive delay (MID) task is an fMRI reward processing task that is extensively used in the literature. However, modern psychometric tools have not been used to evaluate measurement properties of the MID task fMRI data. The current study uses data for a similar task design across three adolescent samples (N = 346 [Agemean 12.0; 44 % Female]; N = 97 [19.3; 58 %]; N = 112 [20.2; 38 %]) to evaluate multiple measurement properties of fMRI responses on the MID task. Confirmatory factor analysis (CFA) is used to evaluate an a priori theoretical model for the task and its measurement invariance across three samples. Exploratory factor analysis (EFA) is used to identify the data-driven measurement structure across the samples. CFA results suggest that the a priori model is a poor representation of these MID task fMRI data. Across the samples, the data-driven EFA models consistently identify a six-to-seven factor structure with run and bilateral brain region factors. This factor structure is moderately-to-highly congruent across the samples. Altogether, these findings demonstrate a need to evaluate theoretical frameworks for popular fMRI task designs to improve our understanding and interpretation of brain-behavior associations.

Prenatal androgen influences on the brain: A review, critique, and illustration of research on congenital adrenal hyperplasia

Sex hormones, especially androgens, contribute to sex and gender differences in the brain and behavior. Organizational effects are particularly important because they are thought to be permanent, reflecting hormone exposure during sensitive periods of development. In human beings, they are often studied with natural experiments in which sex hormones are dissociated from other biopsychosocial aspects of development, such as genes and experiences. Indeed, the greatest evidence for organizational effects on sex differences in human behavior comes from studies of females with congenital adrenal hyperplasia (CAH), who have heightened prenatal androgen exposure, female-typical rearing, and masculinized toy play, activity and career interests, spatial skills, and some personal characteristics. Interestingly, however, neuroimaging studies of females with CAH have revealed few neural mechanisms underlying these hormone-behavior links, with the exception of emotion processing; studies have instead shown reduced gray matter volumes and reduced white matter integrity most consistent with other disease-related processes. The goals of this narrative review are to: (a) describe methods for studying prenatal androgen influences, while offering a brief overview of behavioral outcomes; (b) provide a critical methodological review of neuroimaging research on females with CAH; (c) present an illustrative analysis that overcomes methodological limitations of previous work, focusing on person-specific neural reward networks (and their associations with sensation seeking) in women with CAH and their unaffected sisters in order to inform future research questions and approaches that are most likely to reveal organizational hormone effects on brain structure and function.

Personalized Neural Networks Underlie Individual Differences in Ethnic Identity Exploration and Resolution

This study examined how ethnic identity relates to large-scale brain networks implicated in social interactions, social cognition, self-definition, and cognitive control. Group Iterative Multiple Model Estimation (GIMME) was used to create sparse, person-specific networks among the default mode and frontoparietal resting-state networks in a diverse sample of 104 youths aged 17-21. Links between neural density (i.e., number of connections within and between these networks) and ethnic identity exploration and resolution were evaluated in the full sample. Ethnic identity resolution was positively related to frontoparietal network density, suggesting that having clarity about one's ethnic group membership is associated with brain network organization reflecting cognitive control. These findings help fill a critical knowledge gap about the neural underpinnings of ethnic identity.

Neural heterogeneity underlying late adolescent motivational processing is linked to individual differences in behavioral sensation seeking

Adolescent risk-taking, including sensation seeking (SS), is often attributed to developmental changes in connectivity among brain regions implicated in cognitive control and reward processing. Despite considerable scientific and popular interest in this neurodevelopmental framework, there are few empirical investigations of adolescent functional connectivity, let alone examinations of its links to SS behavior. The studies that have been done focus on mean-based approaches and leave unanswered questions about individual differences in neurodevelopment and behavior. The goal of this paper is to take a person-specific approach to the study of adolescent functional connectivity during a continuous motivational state, and to examine links between connectivity and self-reported SS behavior in 104 adolescents (M_Age  = 19.3; SD_Age  = 1.3). Using Group Iterative Multiple Model Estimation (GIMME), person-specific connectivity during two neuroimaging runs of a monetary incentive delay task was estimated among 12 a priori brain regions of interest representing reward, cognitive, and salience networks. Two data-driven subgroups were detected, a finding that was consistent between both neuroimaging runs, but associations with SS were only found in the first run, potentially reflecting neural habituation in the second run. Specifically, the subgroup that had unique connections between reward-related regions had greater SS and showed a distinctive relation between connectivity strength in the reward regions and SS. These findings provide novel evidence for heterogeneity in adolescent brain-behavior relations by showing that subsets of adolescents have unique associations between neural motivational processing and SS. Findings have broader implications for future work on reward processing, as they demonstrate that brain-behavior relations may attenuate across runs.

Intrauterine Device Use: A New Frontier for Behavioral Neuroendocrinology

Intrauterine devices (IUDs) are the most-used reversible contraceptive method for women in the world, but little is known about their potential modulation of brain function, cognition, and behavior. This is disconcerting because research on other hormonal contraceptives, especially oral contraceptives (OCs), increasingly shows that exogenous sex hormones have behavioral neuroendocrine consequences, especially for gendered cognition, including spatial skills. Effects are small and nuanced, however, partially reflecting heterogeneity. The goal of this paper is to introduce IUD use as a new frontier for basic and applied research, and to offer key considerations for studying it, emphasizing the importance of multimodal investigations and person-specific analyses. The feasibility and utility of studying IUD users is illustrated by: scanning women who completed a functional magnetic resonance imaging mental rotations task; taking an individualized approach to mapping functional connectivity during the task using network analyses containing connections common across participants and unique to individual women, focusing on brain regions in putative mental rotations and default mode networks; and linking metrics of brain connectivity from the individualized networks to both mental rotations task performance and circulating hormone levels. IUD users provide a promising natural experiment for the interplay between exogenous and endogenous sex hormones, and they are likely qualitatively different from OC users with whom they are often grouped in hormonal contraceptive research. This paper underscores how future research on IUD users can advance basic neuroendocrinological knowledge and women's health.

Ecological stress, amygdala reactivity, and internalizing symptoms in preadolescence: Is parenting a buffer?

Ecological stress during adolescent development may increase the sensitivity to negative emotional processes that can contribute to the onset and progression of internalizing behaviors during preadolescence. Although a small number of studies have considered the link among the relations between ecological stress, amygdala reactivity, and internalizing symptoms in childhood and adolescence, these studies have largely been small, cross-sectional, and often do not consider unique roles of parenting or sex. In the current study, we evaluated the interrelations between ecological stress, amygdala reactivity, subsequent internalizing symptoms, and the moderating roles of parenting and sex among 9- and 10-year-old preadolescents from the Adolescent Brain Cognitive Development (ABCD) Study ®. A subset of participants who met a priori quality control criteria for bilateral amygdala activation during the EN-back faces versus places contrast (N = 7,385; Mean Age = 120 months, SD = 7.52; 49.5% Female) were included in the study. A confirmatory factor analysis was performed to create a latent variable of ecological stress, and multiple structural equation models were tested to evaluate the association among baseline ecological stress and internalizing symptoms one year later, the mediating role of amygdala reactivity, and moderating effects of parental acceptance and sex. The results revealed a significant association between ecological stress and subsequent internalizing symptoms, which was greater in males than females. There was no association between amygdala reactivity during the Faces versus Places contrast and ecological stress or subsequent internalizing symptoms, and no mediating role of amygdala or moderating effect of parental acceptance on the association between ecological stress and internalizing symptoms. An alternative mediation model was tested which revealed that there was a small mediating effect of parental acceptance on the association between ecological stress and internalizing symptoms, demonstrating lower internalizing symptoms among preadolescents one year later. Given the lack of association in brain function, ecological stress and internalizing symptoms in preadolescents in this registered report, effects from comparable small studies should be reconsidered in larger samples.

Interactions between methodological and interindividual variability: How Monetary Incentive Delay (MID) task contrast maps vary and impact associations with behavior

INTRODUCTION

Phenomena related to reward responsiveness have been extensively studied in their associations with substance use and socioemotional functioning. One important task in this literature is the Monetary Incentive Delay (MID) task. By cueing and delivering performance-contingent reward, the MID task has been demonstrated to elicit robust activation of neural circuits involved in different phases of reward responsiveness. However, systematic evaluations of common MID task contrasts have been limited to between-study comparisons of group-level activation maps, limiting their ability to directly evaluate how researchers' choice of contrasts impacts conclusions about individual differences in reward responsiveness or brain-behavior associations.

METHODS

In a sample of 104 participants (Age Mean = 19.3, SD = 1.3), we evaluate similarities and differences between contrasts in: group- and individual-level activation maps using Jaccard's similarity index, region of interest (ROI) mean signal intensities using Pearson's r, and associations between ROI mean signal intensity and psychological measures using Bayesian correlation.

RESULTS

Our findings demonstrate more similarities than differences between win and loss cues during the anticipation contrast, dissimilarity between some win anticipation contrasts, an apparent deactivation effect in the outcome phase, likely stemming from the blood oxygen level-dependent undershoot, and behavioral associations that are less robust than previously reported.

CONCLUSION

Consistent with recent empirical findings, this work has practical implications for helping researchers interpret prior MID studies and make more informed a priori decisions about how their contrast choices may modify results.

Cortical and subcortical response to the anticipation of reward in high and average/low risk-taking adolescents

Since the first neurodevelopmental models that sought to explain the influx of risky behaviors during adolescence were proposed, there have been a number of revisions, variations and criticisms. Despite providing a strong multi-disciplinary heuristic to explain the development of risk behavior, extant models have not yet reliably isolated neural systems that underlie risk behaviors in adolescence. To address this gap, we screened 2017 adolescents from an ongoing longitudinal study that assessed 15-health risk behaviors, targeting 104 adolescents (Age Range: 17-to-21.4), characterized as high-or-average/low risk-taking. Participants completed the Monetary Incentive Delay (MID) fMRI task, examining reward anticipation to "big win" versus "neutral". We examined neural response variation associated with both baseline and longitudinal (multi-wave) risk classifications. Analyses included examination of a priori regions of interest (ROIs); and exploratory non-parametric, whole-brain analyses. Hypothesis-driven ROI analysis revealed no significant differences between high- and average/low-risk profiles using either baseline or multi-wave classification. Results of whole-brain analyses differed according to whether risk assessment was based on baseline or multi-wave data. Despite significant mean-level task activation, these results do not generalize prior neural substrates implicated in reward anticipation and adolescent risk-taking. Further, these data indicate that whole-brain differences may depend on how risk-behavior profiles are defined.

Adolescent Health Risk Behaviors: Convergent, Discriminant and Predictive Validity of Self-Report and Cognitive Measures

Self-report and cognitive tasks of reward sensitivity and self-regulation have influenced several developmental models that may explain the heightened engagement in risk behaviors during adolescence. Despite some inconsistencies across studies, few studies have explored the convergent, discriminant, and predictive validity of self-report and cognitive measures of these psychological characteristics in adolescence. The present study evaluated the convergent and discriminant validity of self-report and cognitive measures of reward sensitivity and self-regulation among 2017 adolescents (age M = 16.8, SD = 1.1; 56% female; 55% White, 22% Black, 8% Hispanic, 15% other race/ethnic; 49% 10th grade and 51% 12th grade). This study compared the predictive validity of an omnibus measure and specific measures of risk engagement. Convergent and discriminant validity from self-report to cognitive tasks were as predicted, although with weak convergent relationships. As hypothesized, compared to cognitive tasks, self-report measures consistently predicted risky behaviors and explained more variance in the models. These results demonstrate that while cognitive tasks can significantly predict certain risk behaviors, they require increased power to find the very small effects, raising questions about their use as implicit proxies for real world risk behavior.

Provider reasons for discontinuing long-term opioid therapy following aberrant urine drug tests differ based on the type of substance identified

OBJECTIVE

Urine drug testing (UDT) is increasingly performed as a means of identifying aberrant behavior that may be grounds for discontinuation of long-term opioid therapy (LTOT). Little is known, however, about the ways in which positive UDT results may differentially inform decisions to discontinue LTOT based on the type of substance for which the UDT screened positive. The aim of this study was to examine the likelihood of clinician-initiated discontinuation of LTOT attributed to positive UDT results across three discrete categories of substances: (1) cannabis, (2) alcohol or illicit substances (excluding cannabis), and (3) controlled prescription medications that were not prescribed.

DESIGN

This retrospective study utilized the US Department of Veterans Affairs (VA) Health Care System. Corporate Data Warehouse to assemble a sample of 600 patients with substance use disorders and matched controls who were discontinued from LTOT in 2012. Comprehensive manual medical record review identified UDT results in the year prior to discontinuation and reason(s) for discontinuation.

PATIENTS, PARTICIPANTS

Patients with one or more UDTs positive for a single substance (N = 185) comprised the study sample.

MAIN OUTCOME MEASURE(S)

Likelihood of clinician-initiated discontinuation attributed to a positive UDT across the three categories.

RESULTS

Patients with one or more UDTs positive for cannabis were more likely to be discontinued from opioid therapy as a result of the positive UDT compared to those with one or more UDTs positive for nonprescribed prescription medication (adjusted odds ratio [OR] = 18.05, 95% CI = 7.29-44.66). Similarly, patients with UDTs positive for alcohol or illicit substances were more likely to be discontinued for the positive UDTs relative to patients who tested positive for nonprescribed prescription medications (adjusted OR = 13.10, 95% CI = 4.81-35.68). No difference in UDT-related discontinuation decisions was evident between patients with UDTs positive for alcohol/illicit substances versus cannabis (adjusted OR = 1.47, 95% CI = 0.57-3.77).

CONCLUSIONS

High odds of UDT-related discontinuation were found in patients who tested positive for cannabis, alcohol, or illicit substances, relative to nonprescribed prescription medications.

Substance Use Disorder Treatment Following Clinician-Initiated Discontinuation of Long-Term Opioid Therapy Resulting from an Aberrant Urine Drug Test

BACKGROUND

It is unclear whether substance use disorder (SUD) treatment is offered to, or utilized by, patients who are discontinued from long-term opioid therapy (LTOT) following aberrant urine drug tests (UDTs).

OBJECTIVE

To describe the proportion of patients who were referred to, and engaged in, SUD treatment following LTOT discontinuation and to examine differences in SUD treatment referral and engagement based on the substances that led to discontinuation.

DESIGN

From a sample of 600 patients selected from a national cohort of Veterans Health Administration patients who were discontinued from LTOT, we used manual chart review to identify 169 patients who were discontinued because of a UDT that was positive for alcohol, cannabis, or other illicit or non-prescribed controlled substances.

MAIN MEASURES

We extracted sociodemographic, clinical, and health care utilization data from patients' electronic medical records.

KEY RESULTS

Forty-three percent of patients (n = 73) received an SUD treatment referral following LTOT discontinuation and 20% (n = 34) engaged in a new episode of SUD treatment in the year following discontinuation. Logistic regression models controlling for sociodemographic and clinical variables demonstrated that patients who tested positive for cannabis were less likely than patients who tested positive for non-cannabis substances to receive referrals for SUD treatment (aOR = 0.44, 95% CI = 0.23-0.84, p = 0.01) or engage in SUD treatment (aOR = 0.42, 95% CI = 0.19-0.94, p = 0.04). Conversely, those who tested positive for cocaine were more likely to receive an SUD treatment referral (aOR = 3.32, 95% CI = 1.57-7.06, p = 0.002) and engage in SUD treatment (aOR = 2.44, 95% CI = 1.00-5.96, p = 0.05) compared to those who did not have a cocaine-positive UDT.

CONCLUSIONS

There may be substance-specific differences in clinician referrals to, and patient engagement in, SUD treatment. This suggests a need for more standardized implementation of clinical guidelines that recommend SUD care, when appropriate, following LTOT discontinuation.

Suicidal ideation and suicidal self-directed violence following clinician-initiated prescription opioid discontinuation among long-term opioid users

OBJECTIVE

Little is known about patient outcomes following discontinuation of opioid therapy, which may include suicidal ideation (SI) and suicidal self-directed violence (SSV). The purpose of this study was to examine correlates of SI and non-fatal SSV in a sample of patients discontinued from long-term opioid therapy (LTOT).

METHOD

Five hundred-nine Veterans Health Administration (VHA) patients whose clinicians discontinued them from LTOT were selected from a national cohort of VHA patients who discontinued opioids in 2012. The sample comprised patients with a substance use disorder and matched controls. Patient electronic health records were manually reviewed to identify discontinuation reasons and the presence of SI or SSV in the 12months following discontinuation.

RESULTS

Forty-seven patients (9.2%) had SI only, while 12 patients (2.4%) had SSV. In covariate-adjusted logistic regression models, mental health diagnoses associated with having SI/SSV included post-traumatic stress disorder (aOR=2.56, 95% CI=1.23-5.32) and psychotic disorders (aOR=3.19, 95% CI=1.14-8.89). Other medical comorbidities, substance use disorder and pain diagnoses, opioid dose, and benzodiazepine prescriptions were unrelated to SI/SSV.

CONCLUSIONS

Among patients with a substance use disorder and matched controls, there are high rates of SI/SSV following opioid discontinuation, suggesting that these "high risk" patients may require close monitoring and risk prevention.