Performance monitoring for sensorimotor confidence: A visuomotor tracking study

Exploring quantitative measures in metacognition of emotion

Metacognition of emotion (meta-emotion) refers to the ability to evaluate and identify one's emotional feelings. No previous study has defined and measured this construct through objective and quantitative procedures. We established a reliable method to measure meta-emotion. With a two-interval forced-choice procedure, participants selected which of two pictures elicited stronger positive emotion; via the Law of Comparative Judgment, their responses were used to compute individual psychological distances for the emotional responses triggered by the pictures. Then, participants were asked to judge whether a pre-exposed picture induced a stronger positive emotion than the median of that elicited by the whole picture set, followed by a confidence rating. By utilizing each individual's psychological distance, the correctness of a participant's emotional experience was quantified by d', and meta-emotion was quantified using meta-d', M-ratio, and M-diff as indices of metacognitive sensitivity and efficiency based on Signal-Detection Theory. Test-retest reliabilities, validated by Spearman correlation, were observed in meta-d', M-ratio, and marginally with M-diff, suggesting the stability of meta-emotion in the current design. This study unveils a validated procedure to quantify meta-emotion, extendable for assessing metacognition of other subjective feelings. Nevertheless, caution is warranted in interpretation, as the measured processes may be influenced by non-metacognitive factors.

Dopamine precursor depletion affects performance and confidence judgements when events are timed from an explicit, but not an implicit onset

Dopamine affects processing of temporal information, but most previous work has tested its role in prospective tasks, where participants know in advance when the event to be timed starts. However, we are often exposed to events whose onset we do not know in advance. We can evaluate their duration after they have elapsed, but mechanisms underlying this ability are still elusive. Here we contrasted effects of acute phenylalanine and tyrosine depletion (APTD) on both forms of timing in healthy volunteers, in a within-subject, placebo-controlled design. Participants were presented with a disc moving around a circular path and asked to reproduce the duration of one full revolution and to judge their confidence in performance. The onset of the revolution was either known in advance (explicit onset) or revealed only at the end of the trial (implicit onset). We found that APTD shortened reproduced durations in the explicit onset task but had no effect on temporal performance in the implicit onset task. This dissociation is corroborated by effects of APTD on confidence judgements in the explicit task only. Our findings suggest that dopamine has a specific role in prospective encoding of temporal intervals, rather than the processing of temporal information in general.

Preserved Metacognition for Undetected Visuomotor Deviations

Humans can successfully correct deviations of movements without conscious detection of such deviations, suggesting limited awareness of movement details. We ask whether such limited awareness impairs confidence (metacognition). We recorded functional magnetic resonance imaging data while 31 human female and male participants detected cursor deviations during a visuomotor reaching task and rated their confidence retrospectively. We show that participants monitor a summary statistic of the unfolding visual feedback (the peak cursor error) to detect visuomotor deviations and adjust their confidence ratings, even when they report being unaware of a deviation. Crucially, confidence ratings were as metacognitively efficient for aware and unaware deviations. At the neural level, activity in the ventral striatum tracks high confidence, whereas a broad network encodes cursor error but not confidence. These findings challenge the notion of limited conscious action monitoring and uncover how humans monitor their movements as they unfold, even when unaware of ongoing deviations.SIGNIFICANCE STATEMENT We are unaware of the small corrections we apply to our movements as long as our goals are achieved. Here, although we replicate the finding that participants deny perceiving small deviations they correct, we show that their confidence reliably reflects the presence or absence of a deviation. This observation shows they can metacognitively monitor the presence of a deviation, even when they deny perceiving it. We also describe the hemodynamic correlates of confidence ratings. Our study questions the extent to which humans are unaware of the details of their movements; describes a plausible mechanism for metacognition in a visuomotor task, along with its neural correlates; and has important implications for the construction of the sense of self.

Feeling lucky? prospective and retrospective cues for sensorimotor confidence

On a daily basis, humans interact with the outside world using judgments of sensorimotor confidence, constantly evaluating our actions for success. We ask, what sensory and motor-execution cues are used in making these judgements and when are they available? Two sources of temporally distinct information are prospective cues, available prior to the action (e.g., knowledge of motor noise and past performance), and retrospective cues specific to the action itself (e.g., proprioceptive measurements). We investigated the use of these two cues in two tasks, a secondary motor-awareness task and a main task in which participants reached toward a visual target with an unseen hand and then made a continuous judgment of confidence about the success of the reach. Confidence was reported by setting the size of a circle centered on the reach-target location, where a larger circle reflects lower confidence. Points were awarded if the confidence circle enclosed the true endpoint, with fewer points returned for larger circles. This incentivized accurate reaches and attentive reporting to maximize the score. We compared three Bayesian-inference models of sensorimotor confidence based on either prospective cues, retrospective cues, or both sources of information to maximize expected gain (i.e., an ideal-performance model). Our findings showed two distinct strategies: participants either performed as ideal observers, using both prospective and retrospective cues to make the confidence judgment, or relied solely on prospective information, ignoring retrospective cues. Thus, participants can make use of retrospective cues, evidenced by the behavior observed in our motor-awareness task, but these cues are not always included in the computation of sensorimotor confidence.

Microgravity induces overconfidence in perceptual decision-making

Does gravity affect decision-making? This question comes into sharp focus as plans for interplanetary human space missions solidify. In the framework of Bayesian brain theories, gravity encapsulates a strong prior, anchoring agents to a reference frame via the vestibular system, informing their decisions and possibly their integration of uncertainty. What happens when such a strong prior is altered? We address this question using a self-motion estimation task in a space analog environment under conditions of altered gravity. Two participants were cast as remote drone operators orbiting Mars in a virtual reality environment on board a parabolic flight, where both hyper- and microgravity conditions were induced. From a first-person perspective, participants viewed a drone exiting a cave and had to first predict a collision and then provide a confidence estimate of their response. We evoked uncertainty in the task by manipulating the motion's trajectory angle. Post-decision subjective confidence reports were negatively predicted by stimulus uncertainty, as expected. Uncertainty alone did not impact overt behavioral responses (performance, choice) differentially across gravity conditions. However microgravity predicted higher subjective confidence, especially in interaction with stimulus uncertainty. These results suggest that variables relating to uncertainty affect decision-making distinctly in microgravity, highlighting the possible need for automatized, compensatory mechanisms when considering human factors in space research.

Fine-Grained Motion Recognition in At-Home Fitness Monitoring with Smartwatch: A Comparative Analysis of Explainable Deep Neural Networks

The squat is a multi-joint exercise widely used for everyday at-home fitness. Focusing on the fine-grained classification of squat motions, we propose a smartwatch-based wearable system that can recognize subtle motion differences. For data collection, 52 participants were asked to perform one correct squat and five incorrect squats with three different arm postures (straight arm, crossed arm, and hands on waist). We utilized deep neural network-based models and adopted a conventional machine learning method (random forest) as a baseline. Experimental results revealed that the bidirectional GRU/LSTMs with an attention mechanism and the arm posture of hands on waist achieved the best test accuracy (F1-score) of 0.854 (0.856). High-dimensional embeddings in the latent space learned by attention-based models exhibit more clustered distributions than those by other DNN models, indicating that attention-based models learned features from the complex multivariate time-series motion signals more efficiently. To understand the underlying decision-making process of the machine-learning system, we analyzed the result of attention-based RNN models. The bidirectional GRU/LSTMs show a consistent pattern of attention for defined squat classes, but these models weigh the attention to the different kinematic events of the squat motion (e.g., descending and ascending). However, there was no significant difference found in classification performance.

Motor outcomes congruent with intentions may sharpen metacognitive representations

We can monitor our intentional movements and form explicit representations about our movements, allowing us to describe how we move our bodies. But it is unclear which information this metacognitive monitoring relies on. For example, when throwing a ball to hit a target, we might use the visual information about how the ball flew to metacognitively assess our performance. Alternatively, we might disregard the ball trajectory - given that it is not directly relevant to our goal - and metacognitively assess our performance based solely on whether we reached the goal of hitting the target. In two experiments we aimed to distinguish between these two alternatives and asked whether the distal outcome of a goal-directed action (hitting or missing a target) informs the metacognitive representations of our own movements. Participants performed a semi-virtual task where they moved their arm to throw a virtual ball at a target. After each throw, participants discriminated which of two ball trajectories displayed on the screen corresponded to the flight path of their throw and then rated their confidence in this decision. The task included two conditions that differed on whether the distal outcome of the two trajectories shown matched (congruent) or differed (incongruent). Participants were significantly more accurate in discriminating between the two trajectories, and responded faster in the incongruent condition and, accordingly, were significantly more confident on these trials. Crucially, we found significant differences in metacognitive performance (measured as meta-d'/d') between the two conditions only on successful trials, where the virtual ball had hit the target. These results indicate that participants successfully incorporated information about the outcome of the movement into both their discrimination and confidence responses. However, information about the outcome selectively sharpened the precision of confidence ratings only when the outcome of their throw matched their intention. We argue that these findings underline the separation between the different levels of information that may contribute to body monitoring, and we provide evidence that intentions might play a central role in metacognitive motor representations.

Quick balance skill improvement after short-term training with error amplification feedback for older adults

This study investigated behavioral and cortical mechanisms for short-term postural training with error amplification (EA) feedback in the elderly. Thirty-six elderly subjects (65.7 ± 2.2 years) were grouped (control and EA, n = 18) for training in stabilometer balance under visual guidance. During the training session (8 training rounds of 60 s in Day 2), the EA group received visual feedback that magnified errors to twice the real size, whereas the control group received visual feedback that displayed real errors. Scalp EEG and kinematic data of the stabilometer plate and ankle joint were recorded in the pre-test (Day 1) and post-test (Day 3). The EA group (-46.5 ± 4.7%) exhibited greater post-training error reduction than that of the control group (-27.1 ± 4.0%)(p = 0.020), together with a greater decline in kinematic coupling between the stabilometer plate and ankle joint (EA: -26.6 ± 4.8%, control: 2.3 ± 8.6%, p = 0.023). In contrast to the control group, the EA group manifested greater reductions in mean phase-lag index (PLI) connectivity in the theta (4-7 Hz)(p = 0.011) and alpha (8-12 Hz) (p = 0.027) bands. Only the EA group showed post-training declines in the mean PLI in the theta and alpha bands. Minimal spanning tree analysis revealed that EA-based training led to increases in the diameter (p = 0.002) and average eccentricity (p = 0.004) of the theta band for enhanced performance monitoring and reduction in the leaf fraction (p = 0.030) of the alpha band for postural response with enhanced automaticity. In conclusion, short-term EA training optimizes balance skill, favoring multi-segment coordination for the elderly, which is linked to more sophisticated error monitoring with less attentive control over the stabilometer stance.

Consensus Goals in the Field of Visual Metacognition

Despite the tangible progress in psychological and cognitive sciences over the last several years, these disciplines still trail other more mature sciences in identifying the most important questions that need to be solved. Reaching such consensus could lead to greater synergy across different laboratories, faster progress, and increased focus on solving important problems rather than pursuing isolated, niche efforts. Here, 26 researchers from the field of visual metacognition reached consensus on four long-term and two medium-term common goals. We describe the process that we followed, the goals themselves, and our plans for accomplishing these goals. If this effort proves successful within the next few years, such consensus building around common goals could be adopted more widely in psychological science.

Suprathreshold perceptual decisions constrain models of confidence

Perceptual confidence is an important internal signal about the certainty of our decisions and there is a substantial debate on how it is computed. We highlight three confidence metric types from the literature: observers either use 1) the full probability distribution to compute probability correct (Probability metrics), 2) point estimates from the perceptual decision process to estimate uncertainty (Evidence-Strength metrics), or 3) heuristic confidence from stimulus-based cues to uncertainty (Heuristic metrics). These metrics are rarely tested against one another, so we examined models of all three types on a suprathreshold spatial discrimination task. Observers were shown a cloud of dots sampled from a dot generating distribution and judged if the mean of the distribution was left or right of centre. In addition to varying the horizontal position of the mean, there were two sensory uncertainty manipulations: the number of dots sampled and the spread of the generating distribution. After every two perceptual decisions, observers made a confidence forced-choice judgement whether they were more confident in the first or second decision. Model results showed that the majority of observers were best-fit by either: 1) the Heuristic model, which used dot cloud position, spread, and number of dots as cues; or 2) an Evidence-Strength model, which computed the distance between the sensory measurement and discrimination criterion, scaled according to sensory uncertainty. An accidental repetition of some sessions also allowed for the measurement of confidence agreement for identical pairs of stimuli. This N-pass analysis revealed that human observers were more consistent than their best-fitting model would predict, indicating there are still aspects of confidence that are not captured by our modelling. As such, we propose confidence agreement as a useful technique for computational studies of confidence. Taken together, these findings highlight the idiosyncratic nature of confidence computations for complex decision contexts and the need to consider different potential metrics and transformations in the confidence computation.

The feeling of confidence in what we perceive can influence our future behaviour and learning. Understanding how the brain computes confidence is an important goal of researchers. As such, researchers have identified a host of potential models. Yet, rarely are a wide range of models tested against each other to find those that best predict choice behaviour. Our study had human participants compare their confidence for pairs of easy perceptual decisions, reporting if they had higher confidence in the first or second decision. We tested twelve models, covering all three types of models proposed in previous studies, finding strong support for two models. The winning Heuristic model combines all three factors affecting choice uncertainty with an idiosyncratic weighting to compute confidence. The other winning model uses a transformation where the strength of the sensory signal is scaled according to sensory uncertainty. We also assessed the agreement of confidence reports in identical decision scenarios. Humans had higher agreement than almost all model predictions. We propose using confidence agreement intentionally as a second performance benchmark of model fit.

Dynamic influences on static measures of metacognition

Humans differ in their capability to judge choice accuracy via confidence judgments. Popular signal detection theoretic measures of metacognition, such as M-ratio, do not consider the dynamics of decision making. This can be problematic if response caution is shifted to alter the tradeoff between speed and accuracy. Such shifts could induce unaccounted-for sources of variation in the assessment of metacognition. Instead, evidence accumulation frameworks consider decision making, including the computation of confidence, as a dynamic process unfolding over time. Using simulations, we show a relation between response caution and M-ratio. We then show the same pattern in human participants explicitly instructed to focus on speed or accuracy. Finally, this association between M-ratio and response caution is also present across four datasets without any reference towards speed. In contrast, when data are analyzed with a dynamic measure of metacognition, v-ratio, there is no effect of speed-accuracy tradeoff.

Eye Movements as a Window into Decision-Making

For over 100 years, eye movements have been studied and used as indicators of human sensory and cognitive functions. This review evaluates how eye movements contribute to our understanding of the processes that underlie decision-making. Eye movement metrics signify the visual and task contexts in which information is accumulated and weighed. They indicate the efficiency with which we evaluate the instructions for decision tasks, the timing and duration of decision formation, the expected reward associated with a decision, the accuracy of the decision outcome, and our ability to predict and feel confident about a decision. Because of their continuous nature, eye movements provide an exciting opportunity to probe decision processes noninvasively in real time. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Multitasking costs on metacognition in a triple-task paradigm

Multitasking situations, such as using one's phone while driving, are increasingly common in everyday life. Experimental psychology has long documented the costs of multitasking on task performance; however, little is known of the effects it has on the metacognitive processes that monitor such performance. The present study is a step toward filling this void by combining psychophysical procedures with complex multitasking. We devised a multimodal paradigm in which participants performed a sensorimotor tracking task, a visual discrimination task, and an auditory 2-back working memory task, either separately or concurrently, while also evaluating their task performance every ~15 s. Our main finding is that multitasking decreased participants' awareness of their performance (metacognitive sensitivity) for all three tasks. Importantly, this result was independent of the multitasking cost on task performance, and could not be attributed to confidence leak, psychological refractory period, or recency effects on self-evaluations. We discuss the implications of this finding for both metacognition and multitasking research.