Strategic Pauses Relieve Listeners from the Effort of Listening to Fast Speech: Data Limited and Resource Limited Processes in Narrative Recall by Adult Users of Cochlear Implants

Speech that has been artificially accelerated through time compression produces a notable deficit in recall of the speech content. This is especially so for adults with cochlear implants (CI). At the perceptual level, this deficit may be due to the sharply degraded CI signal, combined with the reduced richness of compressed speech. At the cognitive level, the rapidity of time-compressed speech can deprive the listener of the ordinarily available processing time present when speech is delivered at a normal speech rate. Two experiments are reported. Experiment 1 was conducted with 27 normal-hearing young adults as a proof-of-concept demonstration that restoring lost processing time by inserting silent pauses at linguistically salient points within a time-compressed narrative ("time-restoration") returns recall accuracy to a level approximating that for a normal speech rate. Noise vocoder conditions with 10 and 6 channels reduced the effectiveness of time-restoration. Pupil dilation indicated that additional effort was expended by participants while attempting to process the time-compressed narratives, with the effortful demand on resources reduced with time restoration. In Experiment 2, 15 adult CI users tested with the same (unvocoded) materials showed a similar pattern of behavioral and pupillary responses, but with the notable exception that meaningful recovery of recall accuracy with time-restoration was limited to a subgroup of CI users identified by better working memory spans, and better word and sentence recognition scores. Results are discussed in terms of sensory-cognitive interactions in data-limited and resource-limited processes among adult users of cochlear implants.

Translation directionality and the Inhibitory Control Model: a machine learning approach to an eye-tracking study

Introduction

Based on such physiological data as pupillometry collected in an eye-tracking experiment, the study has further confirmed the effect of directionality on cognitive loads during L1 and L2 textual translations by novice translators, a phenomenon called "translation asymmetry" suggested by the Inhibitory Control Model, while revealing that machine learning-based approaches can be usefully applied to the field of Cognitive Translation and Interpreting Studies.

Methods

Directionality was the only factor guiding the eye-tracking experiment where 14 novice translators with the language combination of Chinese and English were recruited to conduct L1 and L2 translations while their pupillometry were recorded. They also filled out a Language and Translation Questionnaire with which categorical data on their demographics were obtained.

Results

A nonparametric related-samples Wilcoxon signed rank test on pupillometry verified the effect of directionality, suggested by the model, during bilateral translations, verifying "translation asymmetry" at a textual level. Further, using the pupillometric data, together with the categorical information, the XGBoost machine-learning algorithm yielded a model that could reliably and effectively predict translation directions.

Conclusion

The study has shown that translation asymmetry suggested by the model was valid at a textual level, and that machine learning-based approaches can be gainfully applied to Cognitive Translation and Interpreting Studies.

Case report: Cortico-ocular interaction networks in NBA2K

The sport industry has never seen growth such as eSports'. Using synchronized monitoring of two biological processes on a 25-year-old gamer, we investigated how his brain (via EEG) and eyes (via pupil dilation) interacted dynamically over time as an integrated network during NBA2K playing time. After the spectral decomposition of the different Brain and Eye signals into seven frequency bands, we calculated the bivariate equal-time Pearson's cross-correlation between each pair of EEG/Eye spectral power time series. On average, our results show a reorganization of the cortico-muscular network across three sessions (e.g., new interactions, hemispheric asymmetry). These preliminary findings highlight the potential need for individualized, specific, adaptive, and periodized interventions and encourage the continuation of this line of research for the creation of general theories of networks in eSports gaming. Future studies should recruit larger samples, investigate different games, and explore cross-frequency coordination among other key organ systems.

Time-specific Components of Pupil Responses Reveal Alternations in Effort Allocation Caused by Memory Task Demands During Speech Identification in Noise

Daily communication may be effortful due to poor acoustic quality. In addition, memory demands can induce effort, especially for long or complex sentences. In the current study, we tested the impact of memory task demands and speech-to-noise ratio on the time-specific components of effort allocation during speech identification in noise. Thirty normally hearing adults (15 females, mean age 42.2 years) participated. In an established auditory memory test, listeners had to listen to a list of seven sentences in noise, and repeat the sentence-final word after presentation, and, if instructed, recall the repeated words. We tested the effects of speech-to-noise ratio (SNR; -4 dB, +1 dB) and recall (Recall; Yes, No), on the time-specific components of pupil responses, trial baseline pupil size, and their dynamics (change) along the list. We found three components in the pupil responses (early, middle, and late). While the additional memory task (recall versus no recall) lowered all components' values, SNR (-4 dB versus +1 dB SNR) increased the middle and late component values. Increasing memory demands (Recall) progressively increased trial baseline and steepened decrease of the late component's values. Trial baseline increased most steeply in the condition of +1 dB SNR with recall. The findings suggest that adding a recall to the auditory task alters effort allocation for listening. Listeners are dynamically re-allocating effort from listening to memorizing under changing memory and acoustic demands. The pupil baseline and the time-specific components of pupil responses provide a comprehensive picture of the interplay of SNR and recall on effort.

Multimodal monitoring: practical recommendations (dos and don'ts) in challenging situations and uncertainty

With the advancements in modern medicine, new methods are being developed to monitor patients in the intensive care unit. Different modalities evaluate different aspects of the patient's physiology and clinical status. The complexity of these modalities often restricts their use to the realm of clinical research, thereby limiting their use in the real world. Understanding their salient features and their limitations can aid physicians in interpreting the concomitant information provided by multiple modalities to make informed decisions that may affect clinical care and outcomes. Here, we present a review of the commonly used methods in the neurological intensive care unit with practical recommendations for their use.

Task-Evoked Pupillary Response as a Potential Biomarker of Dementia and Mild Cognitive Impairment: A Scoping Review

Pupil dilation functions as a proxy for cognitive effort and can be measured through automated pupillometry. The aim of this scoping review is to examine how individuals with cognitive impairment differ in task-evoked pupillary responses relative to cognitively healthy individuals. A systematic literature search across six databases was conducted to identify studies examining changes in pupillary responses evoked by cognitive tasks comparing patients with dementia to healthy controls. Eight articles met inclusion criteria and were included for review. Differences in task-evoked pupillary response between cognitively impaired and cognitively healthy participants were observed across studies. Pupil dilation is decreased in patients with Alzheimer's Disease compared to controls, with no difference observed in patients with mild cognitive impairment. A mild, non-significant trend towards reduced pupil dilation in patients with either Parkinson's Disease or Dementia with Lewy Bodies suggests a similar but less pronounced effect than in AD patients. Further research is required to examine the utility of task-evoked pupillary responses as a potential biomarker indexing cognitive decline in individuals transitioning to mild cognitive impairment and/or dementia.

The pupillometry of the possible: an investigation of infants' representation of alternative possibilities

Contrasting possibilities has a fundamental adaptive value for prediction and learning. Developmental research, however, has yielded controversial findings. Some data suggest that preschoolers might have trouble in planning actions that take into account mutually exclusive possibilities, while other studies revealed an early understanding of alternative future outcomes based on infants' looking behaviour. To better understand the origin of such abilities, here we use pupil dilation as a potential indicator of infants' representation of possibilities. Ten- and 14-month-olds were engaged in an object-identification task by watching video animations where three different objects with identical top parts moved behind two screens. Importantly, a target object emerged from one of the screens but remained in partial occlusion, revealing only its top part, which was compatible with a varying number of possible identities. Just as adults' pupil diameter grows monotonically with the amount of information held in memory, we expected that infants' pupil size would increase with the number of alternatives sustained in memory as candidate identities for the partially occluded object. We found that pupil diameter increased with the object's potential identities in 14- but not in 10-month-olds. We discuss the implications of these results for the foundation of humans' capacities to represent alternatives. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.

Interaural speech asymmetry predicts bilateral speech intelligibility but not listening effort in adults with bilateral cochlear implants

Introduction

Bilateral cochlear implants (BiCIs) can facilitate improved speech intelligibility in noise and sound localization abilities compared to a unilateral implant in individuals with bilateral severe to profound hearing loss. Still, many individuals with BiCIs do not benefit from binaural hearing to the same extent that normal hearing (NH) listeners do. For example, binaural redundancy, a speech intelligibility benefit derived from having access to duplicate copies of a signal, is highly variable among BiCI users. Additionally, patients with hearing loss commonly report elevated listening effort compared to NH listeners. There is some evidence to suggest that BiCIs may reduce listening effort compared to a unilateral CI, but the limited existing literature has not shown this consistently. Critically, no studies to date have investigated this question using pupillometry to quantify listening effort, where large pupil sizes indicate high effort and small pupil sizes indicate low effort. Thus, the present study aimed to build on existing literature by investigating the potential benefits of BiCIs for both speech intelligibility and listening effort.

Methods

Twelve BiCI adults were tested in three listening conditions: Better Ear, Poorer Ear, and Bilateral. Stimuli were IEEE sentences presented from a loudspeaker at 0° azimuth in quiet. Participants were asked to repeat back the sentences, and responses were scored by an experimenter while changes in pupil dilation were measured.

Results

On average, participants demonstrated similar speech intelligibility in the Better Ear and Bilateral conditions, and significantly worse speech intelligibility in the Poorer Ear condition. Despite similar speech intelligibility in the Better Ear and Bilateral conditions, pupil dilation was significantly larger in the Bilateral condition.

Discussion

These results suggest that the BiCI users tested in this study did not demonstrate binaural redundancy in quiet. The large interaural speech asymmetries demonstrated by participants may have precluded them from obtaining binaural redundancy, as shown by the inverse relationship between the two variables. Further, participants did not obtain a release from effort when listening with two ears versus their better ear only. Instead, results indicate that bilateral listening elicited increased effort compared to better ear listening, which may be due to poor integration of asymmetric inputs.

Quantitative pupillometry and radiographic markers of intracranial midline shift: A pilot study

Background

Asymmetric pupil reactivity or size can be early clinical indicators of midbrain compression due to supratentorial ischemic stroke or primary intraparenchymal hemorrhage (IPH). Radiographic midline shift is associated with worse functional outcomes and life-saving interventions. Better understanding of quantitative pupil characteristics would be a non-invasive, safe, and cost-effective way to improve identification of life-threatening mass effect and resource utilization of emergent radiographic imaging. We aimed to better characterize the association between midline shift at various anatomic levels and quantitative pupil characteristics.

Methods

We conducted a multicenter retrospective study of brain CT images within 75 min of a quantitative pupil observation from patients admitted to Neuro-ICUs between 2016 and 2020 with large (>1/3 of the middle cerebral artery territory) acute supratentorial ischemic stroke or primary IPH > 30 mm³. For each image, we measured midline shift at the septum pellucidum (MLS-SP), pineal gland shift (PGS), the ratio of the ipsilateral to contralateral midbrain width (IMW/CMW), and other exploratory markers of radiographic shift/compression. Pupil reactivity was measured using an automated infrared pupillometer (NeurOptics^®, Inc.), specifically the proprietary algorithm for Neurological Pupil Index^® (NPi). We used rank-normalization and linear mixed-effects models, stratified by diagnosis and hemorrhagic conversion, to test associations of radiographic markers of shift and asymmetric pupil reactivity (Diff NPi), adjusting for age, lesion volume, Glasgow Coma Scale, and osmotic medications.

Results

Of 53 patients with 74 CT images, 26 (49.1%) were female, and median age was 67 years. MLS-SP and PGS were greater in patients with IPH, compared to patients with ischemic stroke (6.2 v. 4.0 mm, 5.6 v. 3.4 mm, respectively). We found no significant associations between pupil reactivity and the radiographic markers of shift when adjusting for confounders. However, we found potentially relevant relationships between MLS-SP and Diff NPi in our IPH cohort (β = 0.11, SE 0.04, P = 0.01), and PGS and Diff NPi in the ischemic stroke cohort (β = 0.16, SE 0.09, P = 0.07).

Conclusion

We found the relationship between midline shift and asymmetric pupil reactivity may differ between IPH and ischemic stroke. Our study may serve as necessary preliminary data to guide further prospective investigation into how clinical manifestations of radiographic midline shift differ by diagnosis and proximity to the midbrain.

Pupil Dilation Reflects Emotional Arousal Via Poetic Language

We investigated pupillary responses to the world's shortest fixed verses, Japanese haiku as aesthetic poetry (AP) and senryu as comic poetry (CP), in comparison with non-poetry control stimuli (NP) comprised of slogans that had the same rhythm patterns. Native Japanese speakers without literary training listened to these stimuli while we recorded their pupil diameters. We found that participants' pupils were significantly dilated for CP compared to NP in an early time window. While AP also evoked larger dilations than NP, the latency for AP-related pupil dilation was relatively long. Thus, lay people experience quick and intense arousal in response to funny and humorous words, while aesthetic properties of words may also elicit intense but slower changes in listeners' arousal levels, presumably because they evoke more implicit and subtle emotional effects. This study is the first to provide evidence that poetic language elicits human pupillary dilation. A better understanding of the cognitive and neural substrates for the sensitive awareness of pleasures expressed via poetic language will provide insights for improving mental and physical health. Hence, pupillometry can act as a useful convenient measurement to delineate the sympathetic activation of emotional contexts via language.

Does mathematics anxiety moderate the effect of problem difficulty on cognitive effort?

A negative relationship between mathematics anxiety (MA) and mathematics performance is well documented. One suggested explanation for this relationship is that MA interferes with the cognitive processes needed when solving mathematics problems. A demand for using more cognitive effort (e.g., when performing harder mathematics problems), can be traced as an increase in pupil dilation during the performance. However, we lack knowledge of how MA affects this relationship between the problem difficulty and cognitive effort. This study investigated, for the first time, if MA moderates the effect of arithmetic (i.e., multiplication) problem difficulty on cognitive effort. Thirty-four university students from Norway completed multiplication tasks, including three difficulty levels of problems, while their cognitive effort was also measured by means of pupil dilation using an eye tracker. Further, the participants reported their MA using a questionnaire, and arithmetic competence, general intelligence, and working memory were measured with paper-pencil tasks. A linear mixed model analysis showed that the difficulty level of the multiplication problems affected the cognitive effort so that the pupil dilated more with harder multiplication problems. However, we did not find a moderating effect of MA on cognitive effort, when controlling for arithmetic competence, general intelligence, and working memory. This suggests that MA does not contribute to cognitive effort when solving multiplication problems.

Pupillometric findings in ATTRv patients and carriers: results from a single-centre experience

INTRODUCTION

Hereditary transthyretin amyloidosis (ATTRv) is a treatable multisystemic disease with great phenotypic heterogeneity. Among extra-neurological features, pupillary abnormalities have been reported, either related to amyloid deposition in the eye or to a progressive autonomic neuropathy.

OBJECTIVE

To evaluate the role of automated pupillometry, a non-invasive and rapid test able to provide objective and reproducible data on pupil size and reactivity, as a marker of disease severity in late-onset ATTRv patients.

PATIENTS AND METHODS

We performed automated pupillometry on a cohort of ATTRv patients and pre-symptomatic TTR mutation carriers and compared results to healthy controls. An exhaustive clinical and instrumental evaluation was performed on all enrolled subjects.

RESULTS

A statistically significant difference in most pupillometry parameters was found in ATTRv patients as compared to both carriers and healthy controls. Moreover, in ATTRv patients, we found a significant correlation between many pupillometry findings and disease duration, as well as widely accepted clinical scales and investigations (NIS, Sudoscan from feet, and Norfolk QoL-DN questionnaire).

CONCLUSIONS

We suggest pupillometry may play a role as a reliable and non-invasive biomarker to evaluate ATTRv disease severity and monitor its progression.

Baseline pupil size encodes task-related information and modulates the task-evoked response in a speech-in-noise task

Pupillometry data are commonly reported relative to a baseline value recorded in a controlled pre-task condition. In this study, the influence of the experimental design and the preparatory processing related to task difficulty on the baseline pupil size was investigated during a speech intelligibility in noise paradigm. Furthermore, the relationship between the baseline pupil size and the temporal dynamics of the pupil response was assessed. The analysis revealed strong effects of block presentation order, within-block sentence order and task difficulty on the baseline values. An interaction between signal-to-noise ratio and block order was found, indicating that baseline values reflect listener expectations arising from the order in which the different blocks were presented. Furthermore, the baseline pupil size was found to affect the slope, delay and curvature of the pupillary response as well as the peak pupil dilation. This suggests that baseline correction might be sufficient when reporting pupillometry results in terms of mean pupil dilation only, but not when a more complex characterization of the temporal dynamics of the response is considered. By clarifying which factors affect baseline pupil size and how baseline values interact with the task-evoked response, the results from the present study can contribute to a better interpretation of the pupillary response as a marker of cognitive processing.

Pupillometry as a Window into Young Children's Sustained Attention

Sustained attention is critical to cognition, social competence, and academic success. Importantly, sustained attention undergoes significant development over the early childhood period. Yet, how sustained attention fluctuates over time on task has not been clearly outlined, particularly in young children. In this study, we provide a first test of whether the pupillary response can be used as an indicator of moment-to-moment sustained attention over time on task in young children. Children aged 5 to 7 years (N = 41) completed a psychomotor vigilance task, where they were asked to press a button as fast as possible at the onset of a target stimulus. We measured reaction times over the course of the task, pupil size prior to target onset (baseline pupil size), and pupil size in response to target onset (task-evoked pupil size). The results showed a stereotypical vigilance decrement in children's response times: as time on task increased, reaction times increased. Critically, children's task-evoked pupil size decreased over time on task, while no such change was present in baseline pupil size. These results suggest that young children's waning sustained attention may be linked to a decrease in alertness while overall arousal is maintained. We discuss the importance of leveraging pupillometry to understand the mechanisms of sustained attention over individuals and development.

Evaluating early-stage disk halo changes after small incision lenticule extraction

PURPOSE

To investigate early changes in the disk halo under different light conditions after myopic small incision lenticule extraction (SMILE).

METHODS

This prospective study included 70 eyes of 70 myopic patients aged 18-33 years, with a spherical equivalent (SE) of -5.87 ± 1.86 D, who underwent SMILE. The subjective refraction, higher-order aberrations (HOAs), pupillometry, and halo were measured preoperatively and postoperatively to analyse disk halo variations and correlated factors.

RESULTS

At 5 cd/m2 and 1 cd/m2 luminance, the halo radius in the high myopia (HM) group reached a postoperative peak after 1 week (p = 0.000 and 0.019, respectively), and recovered to baseline after 3 months. In the low-to-moderate (LM) myopia group, the halo radius did not differ 1 week postoperatively compared to the preoperative level (p = 0.015), but significantly improved after 3 months (p = 0.000). The halo radius correlated with SE, uncorrected distance visual acuity (UDVA), ocular HOAs, coma, and the pupillary light reflex in the LM group at all time points, but there were no correlations in the HM group.

CONCLUSIONS

Halo symptoms occurred early after SMILE, but recovered within 3 months. The recovery process was slower in the HM group than in the LM group, and the halo radius correlated with SE, UDVA, ocular HOAs, coma, and the pupillary light reflex.

Infant temperamental fear, pupil dilation, and gaze aversion from smiling strangers

In childhood, higher levels of temperamental fear-an early-emerging proclivity to distress in the face of novelty-are associated with lower social responsivity and greater social anxiety. While the early emergence of temperamental fear in infancy is poorly understood, it is theorized to be driven by individual differences in reactivity and self-regulation to novel stimuli. The current study used eye tracking to capture infants' (N = 124) reactions to a video of a smiling stranger-a common social encounter-including infant gaze aversions from the stranger's face (indexing arousal regulation) and pupil dilation (indexing physiological reactivity), longitudinally at 2, 4, 6, and 8 months of age. Multilevel mixed-effects models indicated that more fearful infants took more time to look away from a smiling stranger's face than less fearful infants, suggesting that high-fear infants may have slower arousal regulation. At 2 and 4 months, more fearful infants also exhibited greater and faster pupil dilation before gaze aversions, consistent with greater physiological reactivity. Together, these findings suggest that individual differences in infants' gaze aversions and pupil dilation can index the development of fearful temperament in early infancy, facilitating the identification of, and interventions for, risk factors to social disruptions.

Development of reference intervals for pupillometry in healthy dogs

BACKGROUND

Pupillometry, the measurement of pupil size and reactivity to a stimulus, has various uses in both human and veterinary medicine. These reflect autonomic tone, with the potential to assess nociception and emotion. Infrared pupillometry reduces inaccuracies that may occur when the pupillary light reflex is determined subjectively by the examiner. To our knowledge, there are no published studies outlining normal reference intervals for automated pupillometry in dogs.

OBJECTIVE

The objective of this study was to develop de novo automated pupillometry reference intervals from 126 healthy canine eyes.

METHODS

The pupillary light reflex (PLR) was measured with a handheld pupillometer (NeurOptics™ PLR-200™ Pupillometer). Parameters recorded included maximum pupil diameter (MAX), minimum pupil diameter (MIN), percent constriction (CON), latency (LAT), average constriction velocity (ACV), maximum constriction velocity (MCV), average dilation velocity (ADV) and time to 75% pupil diameter recovery (T75). One measurement was obtained for each eye.

RESULTS

The following reference intervals were developed: MAX (6.05-11.30 mm), MIN (3.76-9.44 mm), CON (-37.89 to -9.64 %), LAT (0.11-0.30 s), ACV (-6.39 to -2.63 mm/ s), MCV (-8.45 to -3.75 mm/s), ADV (-0.21-1.77 mm/s), and T75 (0.49-3.20 s).

CLINICAL SIGNIFICANCE

The reference intervals developed in this study are an essential first step to facilitate future research exploring pupillometry as a pain assessment method in dogs.

Impact of acute intoxication on quantitative pupillometry assessment in the emergency department

Study Hypothesis/Objective

This prospective cohort study aimed to assess whether and to what extent different quantitative pupillometry (QP) metrics are associated with different intoxicant drug classes as well as investigate the potential benefit of QP as a tool in the rapid assessment of clinically intoxicated patients in the emergency department (ED).

Methods

Between February 25, 2019 and April 24, 2021, 325 patients were enrolled in the EDs of the Hospital of the University of Pennsylvania (HUP) and Penn Presbyterian Medical Center (PPMC). Patients deemed clinically intoxicated or in withdrawal by an attending emergency physician were considered for eligibility. Patients <18 years old, with a chief complaint indicative of head trauma or stroke or without a urine drug screen (UDS) positive for drugs of abuse were excluded. QP data were also collected from a cohort of 82 healthy control subjects.

Results

Neurological Pupil index (NPi) values did not vary significantly between control and study groups nor between study group patients with a UDS positive for opioids. With exception of latency of constriction, all other QP metrics for the study group were depressed relative to controls (P < 0.005).

Conclusions

This work demonstrated the feasibility of QP measurement in the ED, finding that NPi remains unaffected by clinical intoxication and therefore can potentially be used for ED patient evaluation without risk of confounding by key intoxicants of abuse. Future work will evaluate the value of QP as a means of rapid and reproducible neurological assessment to identify various pathologies.

Pupillary reflex dilation and pain index evaluation during general anesthesia using sufentanil: a double-blind randomized controlled trial

Aim: In a single-center, double-blind, randomized controlled trial, we evaluated whether pupillometry-controlled use of sufentanil is better than free-choice administration of sufentanil by anesthesiologists. Patients & methods: 61 patients undergoing daycare gynecological or abdominal surgery were enrolled. A pupillometry pain index score chart was introduced for administration guidance of sufentanil. Results: The first objective, patient well-being, did not show a significant difference with painkiller usage and health state index at day 1 postoperatively. Secondly, we experienced difficulty in interpretation of the pupillometry score. Thirdly, opioid usage was higher in the intervention group (20.1 vs 14.8 mcg; p = 0.017). Conclusion: The use of pupillometry and pain index chart for bolus sufentanil with our protocol showed an unwanted higher sufentanil usage without a significant difference in patient wellbeing. (Ethics Committee EC17/28/319 of the University Hospital of Antwerp. Registration at clinicaltrials.gov NCT03248908.).

Pupillometry tracks errors in interval timing

Recent primate studies suggest a potential link between pupil size and subjectively elapsed duration. Here, we sought to investigate the relationship between pupil size and perceived duration in human participants performing two temporal bisection tasks in the subsecond and suprasecond interval ranges. In the subsecond task, pupil diameter was greater during stimulus processing when shorter intervals were overestimated but also during and after stimulus offset when longer intervals were underestimated. By contrast, in the suprasecond task, larger pupil diameter was observed only in the late stimulus offset phase prior to response prompts when longer intervals were underestimated. This pattern of results suggests that pupil diameter relates to an error monitoring mechanism in interval timing. These results are at odds with a direct relationship between pupil size and the perception of duration but suggest that pupillometric variation might play a key role in signifying errors related to temporal judgments.

Human-in-the-Loop Optimization of Transcranial Electrical Stimulation at the Point of Care: A Computational Perspective

Individual differences in the responsiveness of the brain to transcranial electrical stimulation (tES) are increasingly demonstrated by the large variability in the effects of tES. Anatomically detailed computational brain models have been developed to address this variability; however, static brain models are not “realistic” in accounting for the dynamic state of the brain. Therefore, human-in-the-loop optimization at the point of care is proposed in this perspective article based on systems analysis of the neurovascular effects of tES. First, modal analysis was conducted using a physiologically detailed neurovascular model that found stable modes in the 0 Hz to 0.05 Hz range for the pathway for vessel response through the smooth muscle cells, measured with functional near-infrared spectroscopy (fNIRS). During tES, the transient sensations can have arousal effects on the hemodynamics, so we present a healthy case series for black-box modeling of fNIRS−pupillometry of short-duration tDCS effects. The block exogeneity test rejected the claim that tDCS is not a one-step Granger cause of the fNIRS total hemoglobin changes (HbT) and pupil dilation changes (p < 0.05). Moreover, grey-box modeling using fNIRS of the tDCS effects in chronic stroke showed the HbT response to be significantly different (paired-samples t-test, p < 0.05) between the ipsilesional and contralesional hemispheres for primary motor cortex tDCS and cerebellar tDCS, which was subserved by the smooth muscle cells. Here, our opinion is that various physiological pathways subserving the effects of tES can lead to state−trait variability, which can be challenging for clinical translation. Therefore, we conducted a case study on human-in-the-loop optimization using our reduced-dimensions model and a stochastic, derivative-free covariance matrix adaptation evolution strategy. We conclude from our computational analysis that human-in-the-loop optimization of the effects of tES at the point of care merits investigation in future studies for reducing inter-subject and intra-subject variability in neuromodulation.

Investigating Methods for Cognitive Workload Estimation for Assistive Robots

Robots interacting with humans in assistive contexts have to be sensitive to human cognitive states to be able to provide help when it is needed and not overburden the human when the human is busy. Yet, it is currently still unclear which sensing modality might allow robots to derive the best evidence of human workload. In this work, we analyzed and modeled data from a multi-modal simulated driving study specifically designed to evaluate different levels of cognitive workload induced by various secondary tasks such as dialogue interactions and braking events in addition to the primary driving task. Specifically, we performed statistical analyses of various physiological signals including eye gaze, electroencephalography, and arterial blood pressure from the healthy volunteers and utilized several machine learning methodologies including k-nearest neighbor, naive Bayes, random forest, support-vector machines, and neural network-based models to infer human cognitive workload levels. Our analyses provide evidence for eye gaze being the best physiological indicator of human cognitive workload, even when multiple signals are combined. Specifically, the highest accuracy (in %) of binary workload classification based on eye gaze signals is 80.45 ∓ 3.15 achieved by using support-vector machines, while the highest accuracy combining eye gaze and electroencephalography is only 77.08 ∓ 3.22 achieved by a neural network-based model. Our findings are important for future efforts of real-time workload estimation in the multimodal human-robot interactive systems given that eye gaze is easy to collect and process and less susceptible to noise artifacts compared to other physiological signal modalities.

Applying functional near-infrared spectroscopy and eye-tracking in a naturalistic educational environment to investigate physiological aspects that underlie the cognitive effort of children during mental rotation tests

Spatial cognition is related to academic achievement in science, technology, engineering, and mathematics (STEM) domains. Neuroimaging studies suggest that brain regions' activation might be related to the general cognitive effort while solving mental rotation tasks (MRT). In this study, we evaluate the mental effort of children performing MRT tasks by measuring brain activation and pupil dilation. We use functional near-infrared spectroscopy (fNIRS) concurrently to collect brain hemodynamic responses from children's prefrontal cortex (PFC) and an Eye-tracking system to measure pupil dilation during MRT. Thirty-two healthy students aged 9-11 participated in this experiment. Behavioral measurements such as task performance on geometry problem-solving tests and MRT scores were also collected. The results were significant positive correlations between the children's MRT and geometry problem-solving test scores. There are also significant positive correlations between dorsolateral PFC (dlPFC) hemodynamic signals and visuospatial task performances (MRT and geometry problem-solving scores). Moreover, we found significant activation in the amplitude of deoxy-Hb variation on the dlPFC and that pupil diameter increased during the MRT, suggesting that both physiological responses are related to mental effort processes during the visuospatial task. Our findings indicate that children with more mental effort under the task performed better. The multimodal approach to monitoring students' mental effort can be of great interest in providing objective feedback on cognitive resource conditions and advancing our comprehension of the neural mechanisms that underlie cognitive effort. Hence, the ability to detect two distinct mental states of rest or activation of children during the MRT could eventually lead to an application for investigating the visuospatial skills of young students using naturalistic educational paradigms.

Sensory suppression and increased neuromodulation during actions disrupt memory encoding of unpredictable self-initiated stimuli

Actions modulate sensory processing by attenuating responses to self- compared to externally generated inputs, which is traditionally attributed to stimulus-specific motor predictions. Yet, suppression has been also found for stimuli merely coinciding with actions, pointing to unspecific processes that may be driven by neuromodulatory systems. Meanwhile, the differential processing for self-generated stimuli raises the possibility of producing effects also on memory for these stimuli; however, evidence remains mixed as to the direction of the effects. Here, we assessed the effects of actions on sensory processing and memory encoding of concomitant, but unpredictable sounds, using a combination of self-generation and memory recognition task concurrently with EEG and pupil recordings. At encoding, subjects performed button presses that half of the time generated a sound (motor-auditory; MA) and listened to passively presented sounds (auditory-only; A). At retrieval, two sounds were presented and participants had to respond which one was present before. We measured memory bias and memory performance by having sequences where either both or only one of the test sounds were presented at encoding, respectively. Results showed worse memory performance - but no differences in memory bias -, attenuated responses, and larger pupil diameter for MA compared to A sounds. Critically, the larger the sensory attenuation and pupil diameter, the worse the memory performance for MA sounds. Nevertheless, sensory attenuation did not correlate with pupil dilation. Collectively, our findings suggest that sensory attenuation and neuromodulatory processes coexist during actions, and both relate to disrupted memory for concurrent, albeit unpredictable sounds.

Arousal-modulated memory encoding and retrieval in adults with autism spectrum disorder

Recently, we have shown that pupil dilation during a recognition memory task can serve as an index of memory retrieval difficulties in autism. At the time of publication, we were unaware of specific data-analysis methods that can be used to shed further light on the origins of such memory related pupil dilation. Specifically, by distinguishing "tonic" from "phasic" changes in pupil dilation and considering their temporal progression, it is possible to draw inferences about the functional integrity of a locus coeruleus-norepinephrine system (LC-NE) that is known to play a key role in regulating memory encoding and retrieval processes. We therefore apply these analyses to our previously published eye-tracking data of adults with ASD (N = 24) and neurotypical development (TD, N = 30) during the recognition memory task. In this re-analysis, we related pupil dilation during encoding and retrieval to recognition accuracy in a per-trial analysis of linear mixed models. In ASD, we replicated attenuated recognition accuracy, which was accompanied by attenuated pupil dilation during encoding and retrieval. Group differences in pupil dilation during retrieval occurred late during the trial (after 1.75 s) and indicated an altered top-down processing like attenuated attribution of semantic salience in response to previously encoded stimuli. In addition, only in the ASD group were higher pupil dilation during encoding and lower pupil dilation during retrieval associated with decreased recognition accuracy. This supports altered modulation of memory encoding and retrieval in ASD, with LC-NE phasic activity as promising underlying mechanism. LAY SUMMARY: We investigated the changes of pupil size during memory testing in autism spectrum disorder. Adults with ASD remembered fewer items correctly than neurotypical individuals (TD). This reduced memory was related to increased pupillary responses at study and decreased pupil dilation at test only for adults with ASD indicating a different modulation of memory by the locus coeruleus.