Validating a visual version of the metronome response task

Measuring mind wandering with experience sampling during task performance: An item response theory investigation

The tendency for individuals to mind wander is often measured using experience sampling methods in which probe questions embedded within computerized cognitive tasks attempt to catch episodes of off-task thought at random intervals during task performance. However, mind-wandering probe questions and response options are often chosen ad hoc and vary between studies with extant little guidance as to the psychometric consequences of these methodological decisions. In the present study, we examined the psychometric properties of several common approaches for assessing mind wandering using methods from item response theory (IRT). IRT latent modeling demonstrated that measurement information was generally distributed across the range of trait estimates according to when probes were presented in time. Probes presented earlier in time provided more information about individuals with greater tendency to mind wandering than probes presented later. Furthermore, mind-wandering ratings made on a continuous scale or using multiple categorical rating options provided more information about individuals' latent mind-wandering tendency - across a broader range of the trait continuum - than ratings dichotomized into on-task and off-task categories. In addition, IRT provided evidence that reports of "task-related thoughts" contribute to the task-focused dimension of the construct continuum, providing justification for studies conceptualizing these responses as a kind of task-related focus. Together, we hope these findings will help guide researchers hoping to maximize the measurement precision of their mind wandering assessment procedures.

Slow and steady: Validating the rhythmic visual response task as a marker for attentional states

A principal goal of attention research is to develop tasks with clear behavioral signatures of attentional fluctuations. Measures that index attentional states often fall under two broad umbrellas: decision tasks, in which participants make responses based on the changing requirements of each trial, and rhythm tasks, in which participants respond rhythmically to a uniform stimulus (e.g., a metronome tone). In the former, response speeding typically precedes errors (indicative of attention failures). In the latter, increased response variability precedes subjective reports of off-task states. We developed and validated the rhythmic visual response task (RVRT); a rhythm task incorporating trial-unique scene stimuli. The RVRT incorporates two important advances from both task categories: (1) it is free from the influence that differential decision-making has on fluctuations in attentional states, and (2) trial-unique stimuli enable later cognitive judgments to be mapped to specific moments in the task. These features allow a relatively unobtrusive measure of mind wandering that facilitates the downstream assessment of its consequences. Participants completed 900 trials of the RVRT, interrupted periodically by thought probes that assessed their attentional state. We found that both response time variance and speed predicted depth of mind wandering. Encouraged by these findings, we used the same analysis approach on archival data to demonstrate that the combination of variance and speed best predicted attentional states in several rhythm and decision task datasets. We discuss the implications of these findings and suggest future research that uses the RVRT to investigate the impact of spontaneous mind wandering on memory, decision-making, and perception.

Sustaining attention in visuomotor timing is associated with location-based binding

Maintaining focus of attention over prolonged periods can be challenging, especially when the target stimulus is absent from the temporal sequence. Prior research has shown that a temporal attentional cue filling in the temporal blank can improve sustained attention: in a sustained visual attention task requiring synchronizing finger tapping with a temporally regular sequence composed of brief flash disks interleaved with blank periods, task performance was improved when a continuous fixation point that served as a temporal attentional cue was presented superimposed on the disk stimulus. To test the hypothesis that binding the temporal attentional cue with the target temporal sequence by spatial overlapping is crucial for enhancing sustained attention, the present study conducted a series of three experiments that deconstructed the bound connection between the cue and the sequence stimulus. In Experiment 1, the cue was placed above or below a flash disk. In Experiment 2, the cue was between two vertically arranged flash disks. In Experiment 3, the cue was in a flash ring. No significant effect of sustained attention improvement was found in any of the three experiments. Experiment 4 further replicated these null results and the previously observed effect of sustained attention improvement when the temporal cue was superimposed on the sequence stimulus. Our finding demonstrates that binding by spatial overlapping during the temporal blank when the sequence stimulus is absent is critical for enhancing sustained attention, which should be beneficial for improving performance across a broader range of tasks that require prolonged maintenance of attention.

Examining the effect of expected test format and test difficulty on the frequency and mnemonic costs of mind wandering

Mind wandering, generally defined as task-unrelated thought, has been shown to constitute between 30% and 50% of individuals' thoughts during almost every activity in which they are engaged. Critically, however, previous research has shown that the demands of a given task can lead to either the up- or down-regulation of mind wandering and that engagement in mind wandering may be differentially detrimental to future memory performance depending on learning conditions. The goal of the current research was to gain a better understanding of how the circumstances surrounding a learning episode affect the frequency with which individuals engage in off-task thought, and the extent to which these differences differentially affect memory performance across different test formats. Specifically, while prior work has manipulated the conditions of encoding, we focused on the anticipated characteristics of the retrieval task, thereby examining whether the anticipation of later demands imposed by the expected test format/difficulty would influence the frequency or performance costs of mind wandering during encoding. Across three experiments, we demonstrate that the anticipation of future test demands, as modelled by expected test format/difficulty, does not affect rates of mind wandering. However, the costs associated with mind wandering do appear to scale with the difficulty of the test. These findings provide important new insights into the impact of off-task thought on future memory performance and constrain our understanding of the strategic regulation of inattention in the context of learning and memory.

A Nomothetic Span Approach to the Construct Validation of Sustained Attention Consistency: Re-Analyzing Two Latent-Variable Studies of Performance Variability and Mind-Wandering Self-Reports

The ability to sustain attention consistency is frequently assessed using either objective behavioral measures, such as reaction time (RT) variability, or subjective self-report measures, such as rates of task-unrelated thought (TUT). The current studies examined whether the individual-difference covariation in these measures provides a more construct valid assessment of attention consistency than does either alone. We argue that performance and self-report measures mutually validate each other; each measurement approach has its own sources of error, so their shared variance should best reflect the attention consistency construct. We reanalyzed two latent-variable studies where RT variability and TUTs were measured in multiple tasks (Kane et al. in J Exp Psychol Gen 145:1017-1048, 2016; Unsworth et al. in J Exp Psychol Gen 150:1303-1331, 2021), along with several nomological network constructs to test the convergent and discriminant validity of a general attention consistency factor. Confirmatory factor analyses assessing bifactor (preregistered) and hierarchical (non-preregistered) models suggested that attention consistency can be modeled as the shared variance among objective and subjective measures. This attention consistency factor was related to working memory capacity, attention (interference) control, processing speed, state motivation and alertness, and self-reported cognitive failures and positive schizotypy. Although bifactor models of general attention consistency provide the most compelling construct validity evidence for a specific ability to sustain attention, multiverse analyses of outlier decisions suggested they are less robust than hierarchical models. The results provide evidence for the general ability to sustain attention consistency and suggestions for improving its measurement.

A combined experimental-correlational approach to the construct validity of performance-based and self-report-based measures of sustained attention

The ability to sustain attention is often measured with either objective performance indicators, like within-person RT variability, or subjective self-reports, like mind wandering propensity. A more construct valid approach, however, may be to assess the covariation in these performance and self-report measures, given that each of these is influenced by different sources of measurement error. If the correlation between performance-variability and self-report measures reflects the sustained attention construct, then task manipulations aimed at reducing the sustained attention demands of tasks should reduce the correlation between them (in addition to reducing mean levels of variability and mind wandering). The current study investigated this claim with a combined experimental-correlation approach. In two experiments (Ns ~ 1,500 each), participants completed tasks that either maximized or minimized the demand for sustained attention. Our demand manipulations successfully reduced the mean levels of sustained attention failures in both the objective and subjective measures, in both experiments. In neither experiment, however, did the covariation between these measures change as a function of the sustained attention demands of the tasks. We can therefore claim only minimal support for the construct validity of our measurement approach to sustained attention.

Beyond mind wandering: Performance variability and neural activity during off-task thought and other attention lapses

To study the characteristics of attention lapses, a metronome response task and experience sampling were employed while recording fMRI data. Thought prompts queried several attention states (on-task, task-related interference, off-task, inattention). Off-task thoughts were probed on whether they arose in a spontaneous or constrained (i.e., directed) manner. Increased fMRI activation was observed in the default mode network during off-task thought and in subregions of the anterior cingulate cortex and inferior frontal gyrus during inattention. Activation also increased in the left hippocampus during constrained thoughts. Functional connectivity increased between the left superior temporal sulcus and right temporoparietal junction for constrained compared to spontaneous thoughts. Overall, behavioral results indicated a monotonic increase in performance variability from on-task to inattention. However, subtle but consistent differences were observed between self-reported attention state and performance. Results are discussed from perspectives of mind wandering frameworks, the function of brain networks, and the role of engagement in off-task thought.

Coexistence of thought types as an attentional state during a sustained attention task

Some studies have asked participants about attentional state on a scale from on-task to off-task, which set the middle option as attention focused on both, including the possibility of the coexistence of thoughts. In addition, studies using multidimensional probes explicitly assumed coexistence within spontaneous thoughts and task-focus dimensions. Although several studies have assumed the coexistence of some thought dimensions, none has explored whether these are different types of thoughts (task-focus, mind-wandering, task-related, external stimuli-related). To examine whether this coexistence of thought types occurred, we used thought probes to determine the degree of immersion in each. The participants responded to probes presented at random during a sustained attention task. The results revealed a mixture of thought types in many self-reports. In addition, the state of attentional allocation behind self-reports was estimated using the hidden Markov model. We observed the following attentional states: task-focused, task-unrelated, task-related, external stimuli-focused, and task-focused-but also focused on other thoughts. These results suggest that individuals can simultaneously allocate attention to thought types and discriminate between reporting. In some cases, probe options should also be considered for this coexistence. We also examined the relationship between self-reports and behavioral indexes, and discussed the necessity of separately measuring the degree of immersion for each thought type.

Temporal Structure in Sensorimotor Variability: A Stable Trait, But What For?

Human performance shows substantial endogenous variability over time, and this variability is a robust marker of individual differences. Of growing interest to psychologists is the realisation that variability is not fully random, but often exhibits temporal dependencies. However, their measurement and interpretation come with several controversies. Furthermore, their potential benefit for studying individual differences in healthy and clinical populations remains unclear. Here, we gather new and archival datasets featuring 11 sensorimotor and cognitive tasks across 526 participants, to examine individual differences in temporal structures. We first investigate intra-individual repeatability of the most common measures of temporal structures - to test their potential for capturing stable individual differences. Secondly, we examine inter-individual differences in these measures using: (1) task performance assessed from the same data, (2) meta-cognitive ratings of on-taskness from thought probes occasionally presented throughout the task, and (3) self-assessed attention-deficit related traits. Across all datasets, autocorrelation at lag 1 and Power Spectra Density slope showed high intra-individual repeatability across sessions and correlated with task performance. The Detrended Fluctuation Analysis slope showed the same pattern, but less reliably. The long-term component (d) of the ARFIMA(1,d,1) model showed poor repeatability and no correlation to performance. Overall, these measures failed to show external validity when correlated with either mean subjective attentional state or self-assessed traits between participants. Thus, some measures of serial dependencies may be stable individual traits, but their usefulness in capturing individual differences in other constructs typically associated with variability in performance seems limited. We conclude with comprehensive recommendations for researchers.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42113-022-00162-1.

Episodic-semantic interactions in spontaneous thought

The distinction between a semantic memory system, encompassing conceptual knowledge, and an episodic memory system, characterized by specific episodes, is one of the most important theoretical proposals in cognitive science. However, the distinction between systems has rarely been discussed in relation to spontaneous thought that comes to mind with reduced cognitive effort and intentionality. In this review, we propose that the growing research on spontaneous thought can contribute to current discussions on the interaction between the episodic and semantic systems. Firstly, we review research that shows that, as in deliberate retrieval, spontaneous thoughts are influenced by both episodic and semantic memory, as reflected by the mix of semantic and episodic elements in descriptions of spontaneous thoughts, as well as semantic priming effects in spontaneous thoughts. We integrate the current evidence based on the interplay between cues and semantic activation. Namely, we suggest that cues are key to access episodic memory and modulate the frequency of spontaneous thought, while semantic activation modulates the content of spontaneous thought. Secondly, we propose that spontaneous retrieval is a privileged area to explore the question of functional independence between systems, because it provides direct access to the episodic system. We review the evidence for spontaneous thought in semantic dementia, which suggests that episodic and semantic systems are functionally independent. We acknowledge the scarcity of evidence and suggest that future studies examine the contents of spontaneous thought descriptions and their neural correlates to test the functional relationship and inform the interaction between episodic and semantic systems.

Testing the construct validity of competing measurement approaches to probed mind-wandering reports

Psychology faces a measurement crisis, and mind-wandering research is not immune. The present study explored the construct validity of probed mind-wandering reports (i.e., reports of task-unrelated thought [TUT]) with a combined experimental and individual-differences approach. We examined laboratory data from over 1000 undergraduates at two U.S. institutions, who responded to one of four different thought-probe types across two cognitive tasks. We asked a fundamental measurement question: Do different probe types yield different results, either in terms of average reports (average TUT rates, TUT-report confidence ratings), or in terms of TUT-report associations, such as TUT rate or confidence stability across tasks, or between TUT reports and other consciousness-related constructs (retrospective mind-wandering ratings, executive-control performance, and broad questionnaire trait assessments of distractibility–restlessness and positive-constructive daydreaming)? Our primary analyses compared probes that asked subjects to report on different dimensions of experience: TUT-content probes asked about what they’d been mind-wandering about, TUT-intentionality probes asked about why they were mind-wandering, and TUT-depth probes asked about the extent (on a rating scale) of their mind-wandering. Our secondary analyses compared thought-content probes that did versus didn’t offer an option to report performance-evaluative thoughts. Our findings provide some “good news”—that some mind-wandering findings are robust across probing methods—and some “bad news”—that some findings are not robust across methods and that some commonly used probing methods may not tell us what we think they do. Our results lead us to provisionally recommend content-report probes rather than intentionality- or depth-report probes for most mind-wandering research.

The metronome response task for measuring mind wandering: Replication attempt and extension of three studies by Seli et al

Science requires replicable tools to measure its intended constructs. Attention research has developed tools that have been used in mind-wandering research, but mind-wandering measures often rely on response-inhibition, which introduces speed-accuracy trade-offs that may conflate errors for mind-wandering. We sought to replicate three studies that used an improved mind-wandering measure: the Metronome Response Task (MRT). In a large (N=300) multisite sample, the primary MRT finding was replicated, showing that continuous rhythmic response time variability reliably predicted self-reported mind-wandering. Our findings also show previously undetected differences between intentional and unintentional mind-wandering. While previously reported mediators (motivation) and moderators (confidence) did not replicate, additional covariates add predictive value and additional constructs (e.g., boredom, effort) demonstrate convergent validity. The MRT is useful for inducing and measuring mind-wandering and provides an especially replicable tool. The MRT's measurement of attention could support future models of the complete cycle of sustained attention.

Trait-Level Variability in Attention Modulates Mind Wandering and Academic Achievement

Although mind wandering remains ubiquitous in daily life, the processes that underlie and sustain this behavior remain poorly understood. Across two experiments, we studied the role of intrinsic temperament traits, which shape stable behavioral processes, in moderating the association between mind wandering and the real-life functional outcome of academic success. In Experiment 1, participants completed the Mind Wandering Questionnaire, the Adult Temperament Questionnaire, and reported their grade for the highest degree completed or in progress. Individuals with traits of low Effortful control, high Negative affect, and low Extraversion indicated more mind wandering. Effortful control moderated the relationship between mind wandering and academic success, with higher tendency for mind wandering associated with higher academic achievement for individuals with high Effortful control, and lower academic achievement for those with low Effortful control. Experiment 2 confirmed these links using the visual metronome response task, an objective measure of mind wandering. Together, these results suggest that the intrinsic temperament trait of Effortful control represents one of the key mechanisms behind the functional influence of mind wandering on real-life outcomes. This work places an innate ability to control attention at the very core of real life success, and highlights the need for studying mind wandering through an interdisciplinary lens that brings together cognitive, biological, social, and clinical theories in order to understand the fundamental mechanisms that drive this behavior.

Detecting Mind Wandering: An Objective Method via Simultaneous Control of Respiration and Fingertip Pressure

Mind wandering happens when one train of thought, related to a current undertaking, is interrupted by unrelated thoughts. The detection and evaluation of mind wandering can greatly help in understanding the attention control mechanism during certain focal tasks. Subjective assessments such as random thought-probe and spontaneous self-report are the ways previous research has assessed mind wandering. Here we propose a task in which participants are asked to simultaneously control respiration and fingertip pressure. They are instructed to click a force sensor at the exact moment of inhalation and exhalation of their respiration. The temporal synchronization between the respiratory signals and the fingertip force pulses offers an objective index to detect mind wandering. Twelve participants engaged in the proposed task in which self-reports of mind wandering are compared with the proposed objective index. The results show that the participants reported significantly more mind-wandering episodes during the trials with a larger temporal synchronization than they did during those trials with a smaller temporal synchronization. The findings suggest that the temporal synchronization might be used as an objective marker of mind wandering in attention training and exploration of the attention control mechanism.