Putting the individual into reliability: Bayesian testing of homogeneous within-person variance in hierarchical models

Temporal stability of Bayesian belief updating in perceptual decision-making

Bayesian inference suggests that perception is inferred from a weighted integration of prior contextual beliefs with current sensory evidence (likelihood) about the world around us. The perceived precision or uncertainty associated with prior and likelihood information is used to guide perceptual decision-making, such that more weight is placed on the source of information with greater precision. This provides a framework for understanding a spectrum of clinical transdiagnostic symptoms associated with aberrant perception, as well as individual differences in the general population. While behavioral paradigms are commonly used to characterize individual differences in perception as a stable characteristic, measurement reliability in these behavioral tasks is rarely assessed. To remedy this gap, we empirically evaluate the reliability of a perceptual decision-making task that quantifies individual differences in Bayesian belief updating in terms of the relative precision weighting afforded to prior and likelihood information (i.e., sensory weight). We analyzed data from participants (n = 37) who performed this task twice. We found that the precision afforded to prior and likelihood information showed high internal consistency and good test-retest reliability (ICC = 0.73, 95% CI [0.53, 0.85]) when averaged across participants, as well as at the individual level using hierarchical modeling. Our results provide support for the assumption that Bayesian belief updating operates as a stable characteristic in perceptual decision-making. We discuss the utility and applicability of reliable perceptual decision-making paradigms as a measure of individual differences in the general population, as well as a diagnostic tool in psychiatric research.

Complementary benefits of multivariate and hierarchical models for identifying individual differences in cognitive control

Understanding individual differences in cognitive control is a central goal in psychology and neuroscience. Reliably measuring these differences, however, has proven extremely challenging, at least when using standard measures in cognitive neuroscience such as response times or task-based fMRI activity. While prior work has pinpointed the source of the issue - the vast amount of cross-trial variability within these measures - no study has rigorously evaluated potential solutions. Here, we do so with one potential way forward: an analytic framework that combines hierarchical Bayesian modeling with multivariate decoding of trial-level fMRI data. Using this framework and longitudinal data from the Dual Mechanisms of Cognitive Control project, we estimated individuals' neural responses associated with cognitive control within a color-word Stroop task, then assessed the reliability of these individuals' responses across a time interval of several months. We show that in many prefrontal and parietal brain regions, test-retest reliability was near maximal, and that only hierarchical models were able to reveal this state of affairs. Further, when compared to traditional univariate contrasts, multivariate decoding enabled individual-level correlations to be estimated with significantly greater precision. We specifically link these improvements in precision to the optimized suppression of cross-trial variability in decoding. Together, these findings not only indicate that cognitive control-related neural responses individuate people in a highly stable manner across time, but also suggest that integrating hierarchical and multivariate models provides a powerful approach for investigating individual differences in cognitive control, one that can effectively address the issue of high-variability measures.

On liking how you look and feeling sexy: Quantifying the overlap between sexual desirability and body satisfaction in a sample of Australian Grindr users

The body image field aims to cultivate positive body image. To do so, it must appreciate factors contributing to positive body image. Sexual desirability is one such factor. Using experience sampling data from Australian Grindr users, we aimed to quantify the overlap between sexual desirability and body satisfaction. We found sexual desirability and body satisfaction correlated strongly between- (r = .90) and within-persons (rMedian =.60). Using dynamic structural equation modeling, we analyzed 238 participants' data (T = 9058), finding that sexual desirability and body satisfaction were bidirectionally related - previous sexual desirability predicted current body satisfaction (β = 0.22) and vice versa (β = 0.17). Participants' average body satisfaction tended to be higher when sexual desirability contributed more to body satisfaction (r = .31) and was more stable across time (r = .19). We found sexual desirability and body satisfaction overlap considerably and that sexual desirability may contribute more to body satisfaction than vice versa. Our results suggest that (1) sexual desirability and its dynamics across time influence body satisfaction, and (2) research on sexual desirability and its relationship with body image should be a focus for the field. We discuss avenues for future research on sexual desirability.

On doing multi-act arithmetic: A multitrait-multimethod approach of performance dimensions in integrated multitasking

Here we present a systematic plan to the experimental study of test-retest reliability in the multitasking domain, adopting the multitrait-multimethod (MTMM) approach to evaluate the psychometric properties of performance in Düker-type speeded multiple-act mental arithmetic. These form of tasks capacitate the experimental analysis of integrated multi-step processing by combining multiple mental operations in flexible ways in the service of the overarching goal of completing the task. A particular focus was on scoring methodology, particularly measures of response speed variability. To this end, we present data of two experiments with regard to (a) test-retest reliability, (b) between-measures correlational structure, (c) and stability (test-retest practice effects). Finally, we compared participants with high versus low performance variability to assess ability-related differences in measurement precision (typically used as proxy to "simulate" patient populations), which is especially relevant in the applied fields of clinical neuropsychology. The participants performed two classic integrated multi-act arithmetic tasks, combining addition and verification (Exp. 1) and addition and comparison (Exp. 2). The results revealed excellent test-retest reliability for the standard and the variability measures. The analysis of between-measures correlational structure revealed the typical pattern of convergent and discriminant relationships, and also, that absolute response speed variability was highly correlated with average speed (r > 0.85), indicating that these measures mainly deliver redundant information. In contrast, speed-adjusted (relativized) variability revealed discriminant validity being correlated to a much lesser degree with average speed, indicating that this measure delivers additional information not already provided by the speed measure. Furthermore, speed-adjusted variability was virtually unaffected by test-retest practice, which makes this measure interesting in situations with repeated testing.

Understanding the Error in Psychopathology: Notable Intraindividual Differences in Neural Variability of Performance Monitoring

BACKGROUND

Abnormal performance monitoring is a possible transdiagnostic marker of psychopathology. Research on neural indices of performance monitoring, including the error-related negativity (ERN), typically examines group and interindividual (between-person) differences in mean/average scores. Intraindividual (within-person) variability in activity captures the capacity to dynamically adjust from moment to moment, and excessive variability appears maladaptive. Intraindividual variability in ERN represents a unique and largely unexamined dimension that might impact functioning. We tested whether psychopathology group differences (major depressive disorder [MDD], generalized anxiety disorder [GAD], obsessive-compulsive disorder [OCD]) or corresponding psychiatric symptoms account for intraindividual variability in single-trial ERN scores.

METHODS

High-density electroencephalogram (Electrical Geodesics, Inc.) was recorded during a semantic flanker task in 51 participants with MDD, 44 participants with GAD, 31 participants with OCD, and 56 psychiatrically-healthy participants. Mean ERN amplitude was scored 0-125ms following participant response across four fronto-central sites. Multilevel location-scale models were used to simultaneously examine interindividual and intraindividual differences in ERN.

RESULTS

Analyses indicated considerable intraindividual variability in ERN that was common across groups. However, we did not find strong evidence to support relationships between ERN and psychopathology groups or transdiagnostic symptoms.

CONCLUSIONS

These findings point to important methodological implications for studies of performance monitoring in healthy and clinical populations-the common assumption of fixed intraindividual variability (i.e., residual variance) may be inappropriate for ERN studies. Implementation of multilevel location-scale models in future research can leverage between-person differences in intraindividual variability in performance monitoring to gain a rich understanding of trial-to-trial performance monitoring dynamics.

Beneath the surface: Unearthing within-person variability and mean relations with Bayesian mixed models

Mixed-effects models are becoming common in psychological science. Although they have many desirable features, there is still untapped potential. It is customary to view homogeneous variance as an assumption to satisfy. We argue to move beyond that perspective, and to view modeling within-person variance as an opportunity to gain a richer understanding of psychological processes. The technique to do so is based on the mixed-effects location scale model that can simultaneously estimate mixed-effects submodels to both the mean (location) and within-person variance (scale). We develop a framework that goes beyond assessing the submodels in isolation of one another and introduce a novel Bayesian hypothesis test for mean-variance correlations in the distribution of random effects. We first present a motivating example, which makes clear how the model can characterize mean-variance relations. We then apply the method to reaction times (RTs) gathered from 2 cognitive inhibition tasks. We find there are more individual differences in the within-person variance than the mean structure, as well as a complex web of structural mean-variance relations. This stands in contrast to the dominant view of within-person variance (i.e., "noise"). The results also point toward paradoxical within-person, as opposed to between-person, effects: several people had slower and less variable incongruent responses. This contradicts the typical pattern, wherein larger means tend to be associated with more variability. We conclude with future directions, spanning from methodological to theoretical inquires, that can be answered with the presented methodology. (PsycInfo Database Record (c) 2021 APA, all rights reserved).