Retest Reliability of Integrated Speed-Accuracy Measures

One Score to Rule Them All? Comparing the Predictive and Concurrent Validity of 30 Hearts and Flowers Scoring Approaches

The Hearts and Flowers (H&F) task is a computerized executive functioning (EF) assessment that has been used to measure EF from early childhood to adulthood. It provides data on accuracy and reaction time (RT) across three different task blocks (hearts, flowers, and mixed). However, there is a lack of consensus in the field on how to score the task that makes it difficult to interpret findings across studies. The current study, which includes a demographically diverse population of kindergarteners from Boston Public Schools (N = 946), compares the predictive and concurrent validity of 30 ways of scoring H&F, each with a different combination of accuracy, RT, and task block(s). Our exploratory results provide evidence supporting the use of a two-vector average score based on Zelazo et al.'s approach of adding accuracy and RT scores together only after individuals pass a certain accuracy threshold. Findings have implications for scoring future tablet-based developmental assessments.

Dynamics of cognitive predictors during reading acquisition in a sample of children overrepresented for dyslexia risk

Literacy acquisition is a complex process with genetic and environmental factors influencing cognitive and neural processes associated with reading. Previous research identified factors that predict word reading fluency (WRF), including phonological awareness (PA), rapid automatized naming (RAN), and speech-in-noise perception (SPIN). Recent theoretical accounts suggest dynamic interactions between these factors and reading, but direct investigations of such dynamics are lacking. Here, we investigated the dynamic effect of phonological processing and speech perception on WRF. More specifically, we evaluated the dynamic influence of PA, RAN, and SPIN measured in kindergarten (the year prior to formal reading instruction), first grade (the first year of formal reading instruction) and second grade on WRF in second and third grade. We also assessed the effect of an indirect proxy of family risk for reading difficulties using a parental questionnaire (Adult Reading History Questionnaire, ARHQ). We applied path modeling in a longitudinal sample of 162 Dutch-speaking children of whom the majority was selected to have an increased family and/or cognitive risk for dyslexia. We showed that parental ARHQ had a significant effect on WRF, RAN and SPIN, but unexpectedly not on PA. We also found effects of RAN and PA directly on WRF that were limited to first and second grade respectively, in contrast to previous research reporting pre-reading PA effects and prolonged RAN effects throughout reading acquisition. Our study provides important new insights into early prediction of later word reading abilities and into the optimal time window to target a specific reading-related subskill during intervention.

Listening efficiency in adult cochlear-implant users compared with normally-hearing controls at ecologically relevant signal-to-noise ratios

Introduction

Due to having to work with an impoverished auditory signal, cochlear-implant (CI) users may experience reduced speech intelligibility and/or increased listening effort in real-world listening situations, compared to their normally-hearing (NH) peers. These two challenges to perception may be usefully integrated in a measure of listening efficiency: conceptually, the amount of accuracy achieved for a certain amount of effort expended.

Methods

We describe a novel approach to quantifying listening efficiency based on the rate of evidence accumulation toward a correct response in a linear ballistic accumulator (LBA) model of choice decision-making. Estimation of this objective measure within a hierarchical Bayesian framework confers further benefits, including full quantification of uncertainty in parameter estimates. We applied this approach to examine the speech-in-noise performance of a group of 24 CI users (M age: 60.3, range: 20-84 years) and a group of 25 approximately age-matched NH controls (M age: 55.8, range: 20-79 years). In a laboratory experiment, participants listened to reverberant target sentences in cafeteria noise at ecologically relevant signal-to-noise ratios (SNRs) of +20, +10, and +4 dB SNR. Individual differences in cognition and self-reported listening experiences were also characterised by means of cognitive tests and hearing questionnaires.

Results

At the group level, the CI group showed much lower listening efficiency than the NH group, even in favourable acoustic conditions. At the individual level, within the CI group (but not the NH group), higher listening efficiency was associated with better cognition (i.e., working-memory and linguistic-closure) and with more positive self-reported listening experiences, both in the laboratory and in daily life.

Discussion

We argue that listening efficiency, measured using the approach described here, is: (i) conceptually well-motivated, in that it is theoretically impervious to differences in how individuals approach the speed-accuracy trade-off that is inherent to all perceptual decision making; and (ii) of practical utility, in that it is sensitive to differences in task demand, and to differences between groups, even when speech intelligibility remains at or near ceiling level. Further research is needed to explore the sensitivity and practical utility of this metric across diverse listening situations.

Same same but different: Subtle but consequential differences between two measures to linearly integrate speed and accuracy (LISAS vs. BIS)

Condition-specific speed-accuracy trade-offs (SATs) are a pervasive issue in experimental psychology, because they sometimes render impossible an unambiguous interpretation of experimental effects on either mean response times (mean RT) or percentage of correct responses (PC). For between-participants designs, we have recently validated a measure (Balanced Integration Score, BIS) that integrates standardized mean RT and standardized PC and thereby controls for cross-group variation in SAT. Another related measure (Linear Integrated Speed-Accuracy Score, LISAS) did not fulfill this specific purpose in our previous simulation study. Given the widespread and seemingly interchangeable use of the two measures, we here illustrate the crucial differences between LISAS and BIS related to their respective choice of standardization variance. We also disconfirm the recently articulated hypothesis that the differences in the behavior of the two combined performance measures observed in our previous simulation study were due to our choice of a between-participants design and we demonstrate why a previous attempt to validate BIS (and LISAS) for within-participants designs has failed, pointing out several consequential issues in the respective simulations and analyses. In sum, the present study clarifies the differences between LISAS and BIS, demonstrates that the choice of the variance used for standardization is crucial, provides further guidance on the calculation and use of BIS, and refutes the claim that BIS is not useful for attenuating condition-specific SATs in within-participants designs.

Chronic stress and executive functioning: A specification-curve analysis

To investigate the association between chronic stress and executive functioning (EF), we assessed 514 young to middle-aged adults in three EF tasks (i.e., Number-Letter, 2-Back, Go/Nogo) that assessed shifting, updating, and inhibition. Chronic stress was assessed by various self-report measures and hair cortisol concentrations as indicators of subjective and objective chronic stress, respectively. In order to test the association between chronic stress and EF, we fit a structural equation model with a latent common EF factor predicted by subjective and objective chronic stress on Kaplan-Meier estimates of response times. Controlling for participants' sex, age household income and the delay between cognitive testing and hair sample collection, neither subjective nor objective chronic stress showed a meaningful association with common EF. Exploratory analyses suggested a moderation effect of income on the association between subjective chronic stress and common EF, with a smaller association for high-income participants. Additionally, we conducted a specification-curve analysis on the association between chronic stress and EF to assess the influence of different analysis choices on results in our dataset. This analysis confirmed the absence of a coherent association between chronic stress and EF by showing that the majority of analytical choices produced null effects and only a small number of analytical choices produced meaningful associations (negative or positive). Taken together, our findings suggest that common EF likely remains preserved under the influence of chronic stress. Our specification-curve analysis, however, also shows that chronic stress may also have either a positive or a negative effect on EF, depending on the choice of covariates and measures of chronic stress and EF. Consequently, more research on the role of these factors for the association between chronic stress and EF is needed to avoid the interpretation of non-replicable stress-EF associations caused by analytical choices or selection bias.