High Working Memory Load Impairs Language Processing during a Simulated Piloting Task: An ERP and Pupillometry Study

Effects of player-video game interaction on the mental effort of older adults with the use of electroencephalography and NASA-TLX

While player-video game interaction appears to affect older adults in gaming, there is limited knowledge regarding the cognitive demands associated with the anticipation of performing a button press, specifically focusing on the input and game elements relation (I/E relation) in game environments. The study aims to investigate the effects of lateral and rotational displacement amplitudes of game elements, triggered by a single button-press, on the cognitive effort of older adults. Both subjective and objective measurement methods were employed to assess these effects. A total of 48 older adults participated in three casual video game tasks encompassing lateral and rotational displacements at varying I/E relations (low, medium, and high). Results obtained from the NASA Task Load Index and electroencephalography (EEG) measurements revealed significant differences between the I/E relations. Specifically, the subjective rating of cognitive demand among older players was significantly impacted by a small rotation angle associated with a button press, leading to increased mental, physical, and temporal demands, along with decreased performance. Surprisingly, the analysis of EEG data, particularly the theta-alpha ratio, revealed significant interaction effects of I/E relations, button press type, and game type on the cognitive demand required during gameplay. These findings offer practical implications and point towards future avenues for developing player-video game interactions that are more cognitively friendly for older players in gaming environments.

Development and validation of an assessment index for quantifying cognitive task load in pilots under simulated flight conditions using heart rate variability and principal component analysis

To investigate whether high cognitive task load (CTL) for aircraft pilots can be identified by analysing heart-rate variability, electrocardiograms were recorded while cadet pilots (n = 68) performed the plane tracking, anti-gravity pedalling, and reaction tasks during simulated flight missions. Data for standard electrocardiogram parameters were extracted from the R-R-interval series. In the research phase, low frequency power (LF), high frequency power (HF), normalised HF, and LF/HF differed significantly between high and low CTL conditions (p < .05 for all). A principal component analysis identified three components contributing 90.62% of cumulative heart-rate variance. These principal components were incorporated into a composite index. Validation in a separate group of cadet pilots (n = 139) under similar conditions showed that the index value significantly increased with increasing CTL (p < .05). The heart-rate variability index can be used to objectively identify high CTL flight conditions.Practitioner summary: We used principal component analysis of electrocardiogram data to construct a composite index for identifying high cognitive task load in pilots during simulated flight. We validated the index in a separate group of pilots under similar conditions. The index can be used to improve cadet training and flight safety.Abbreviations: ANOVA: a one-way analysis of variance; AP: anti-gravity pedaling task; CTL: cognitive task load; ECG: electrocardiograms; HR: heart rate; HRV: heart-rate variability; HRVI: heart-rate variability index; PT: plane-tracking task; RMSSD: root-mean square of differences between consecutive R-R intervals; RT: reaction task; SDNN: standard deviation of R-R intervals; HF: high frequency power; HFnu: normalized HF; LF: low frequency power; LFnu: normalized LF; PCA: principal component analysis.

Linking tonic and phasic pupil responses to P300 amplitude in an emotional face-word Stroop task

The locus coeruleus-norepinephrine (LC-NE) system, which regulates arousal levels, is important for cognitive control, including emotional conflict resolution. Additionally, the LC-NE system is implicated in P300 generation. If the P300 is mediated by the LC-NE system, and considering the established correlations between LC activity and pupil dilation, P300 amplitude should correlate with task-evoked (phasic) pupil dilation on a trial-by-trial basis. However, prior studies, predominantly utilizing oddball-type paradigms, have not demonstrated correlations between concurrently recorded task-evoked pupil dilation and P300 responses. Using a recently developed emotional face-word Stroop task that links pupil dilation to the LC-NE system, here, we examined both intra- and inter-individual correlations between task-evoked pupil dilation and P300 amplitude. We found that lower accuracy, slower reaction times, and larger task-evoked pupil dilation were obtained in the incongruent compared to the congruent condition. Furthermore, we observed intra-individual correlations between task-evoked pupil dilation and P300 amplitude, with larger pupil dilation correlating with a greater P300 amplitude. In contrast, pupil dilation did not exhibit consistent correlations with N450 and N170 amplitudes. Baseline (tonic) pupil size also showed correlations with P300 and N170 amplitudes, with smaller pupil size corresponding to larger amplitude. Moreover, inter-individual differences in task-evoked pupil dilation between the congruent and incongruent conditions correlated with differences in reaction time and P300 amplitude, though these effects only approached significance. To summarize, our study provides evidence for a connection between task-evoked pupil dilation and P300 amplitude at the single-trial level, suggesting the involvement of the LC-NE system in P300 generation.

Fast & scrupulous: Gesture-based alarms improve accuracy and reaction times under various mental workload levels. An ERSP study

In high-risk environments, fast and accurate responses to warning systems are essential to efficiently handle emergency situations. The aim of the present study was twofold: 1) investigating whether hand action videos (i.e., gesture alarms) trigger faster and more accurate responses than text alarm messages (i.e., written alarms), especially when mental workload (MWL) is high; and 2) investigating the brain activity in response to both types of alarms as a function of MWL. Regardless of MWL, participants (N = 28) were found to be both faster and more accurate when responding to gesture alarms than to written alarms. Brain electrophysiological results suggest that this greater efficiency might be due to a facilitation of the action execution, reflected by the decrease in mu and beta power observed around the response time window observed at C3 and C4 electrodes. These results suggest that gesture alarms may improve operators' performances in emergency situations.

The effect of working memory load on inattentional deafness during aeronautical decision-making

Operating an aircraft requires pilots to handle a significant amount of multi-modal information, which creates a high working memory load. Detecting auditory alarms in this high-load scenario is crucial for aviation safety. According to cognitive control load theory, an increase in working memory load may enhance distractor interference, resulting in improved detection sensitivity for task-irrelevant stimuli. Therefore, understanding the effect of working memory load on auditory alarm detection is of particular interest in aviation safety research. The studies were designed to investigate the effect of storage load and executive function load of working memory on auditory alarm detection during aeronautical decision-making through three experiments. In Experiment 1 and 2, participants performed an aeronautical decision-making task while also detecting an auditory alarm during the retention interval of a working memory task (visual-spatial, visual-verbal and auditory-verbal). In Experiment 3, participants were required to detect an auditory alarm while performing the 2-back and 3-back aeronautical decision-making tasks. Experiment 1 found that the auditory alarm sensitivity was higher in conditions of low visual-spatial working memory storage load compare to high load conditions. Experiment 2 found that a high storage load of visual-verbal working memory reduced auditory alarm sensitivity but auditory-verbal working memory load did not. Experiment 3 found that, unlike storage load, auditory alarm sensitivity was stronger under high executive function load relative to low executive function load. These findings show that working memory storage load and executive function load have different effects on auditory alarm sensitivity. The relationship between executive function and auditory alarm sensitivity supports cognitive control load theory, while the impact of the storage function on auditory alarm sensitivity does not adhere to this theory.

Mathematical-heuristic modelling for human performance envelope

BACKGROUND: Using the theory of complex systems, some human functions (thinking, memory, language) and human relationships have been analyzed and explained. In order to study the limits of human performance (in Air Traffic Controllers and pilots) a new concept was created, called the Human Performance Envelope (HPE). OBJECTIVE: The aim of this paper is to apply the principles of the complex system to the analysis of the human factors of the HPE concept. Moreover, this paper’s objective is to create a mathematical model that will give the opportunity to study all the physiological ergonomic factors, not only the ones that are most commonly studied. The most studied factors are mental workload, stress and situation awareness (SA). By applying the mathematical model, it is possible to analyze all the physiological factors (stress, mental workload, fatigue, attention, vigilance and SA). METHODS: In the present paper the theory of complex systems (hybrid modelling) was applied to the Human Performance Envelope concept. A mathematical model was created, then it was validated and solved based on previous researches. RESULTS: Firstly, a literature analysis was performed on the complex systems application by the present researchers concerning pilots’ HPE. The proportional and inverse proportional relationships between the nine human factors were visually illustrated. Finally, a mathematical model was proposed, consisting of a set of equations, which were partially solved and validated by the experiments on pilots done by other researchers. CONCLUSIONS: Further research is required to validate the whole mathematical model, including physiological measurements (experiments) for the six ergonomic factors and the applied heuristic psychosocial methods for the others.

The effects of visual working memory load on detection and neural processing of task-unrelated auditory stimuli

While perceptual load has been proposed to reduce the processing of task-unrelated stimuli, theoretical arguments and empirical findings for other forms of task load are inconclusive. Here, we systematically investigated the detection and neural processing of auditory stimuli varying in stimulus intensity during a stimuli-unrelated visual working memory task alternating between low and high load. We found, depending on stimulus strength, decreased stimulus detection and reduced P3, but unaffected N1 amplitudes of the event-related potential to auditory stimuli under high as compared to low load. In contrast, load independent awareness effects were observed during both early (N1) and late (P3) time windows. Findings suggest a late neural effect of visual working memory load on auditory stimuli leading to lower probability of reported awareness of these stimuli.

The P300 Auditory Event-Related Potential May Predict Segregation of Competing Speech by Bimodal Cochlear Implant Listeners

Compared to normal-hearing (NH) listeners, cochlear implant (CI) listeners have greater difficulty segregating competing speech. Neurophysiological studies have largely investigated the neural foundations for CI listeners' speech recognition in quiet, mainly using the P300 component of event-related potentials (ERPs). P300 is closely related to cognitive processes involving auditory discrimination, selective attention, and working memory. Different from speech perception in quiet, little is known about the neurophysiological foundations for segregation of competing speech by CI listeners. In this study, ERPs were measured for a 1 vs. 2 kHz contrast in 11 Mandarin-speaking bimodal CI listeners and 11 NH listeners. Speech reception thresholds (SRTs) for a male target talker were measured in steady noise or with a male or female masker. Results showed that P300 amplitudes were significantly larger and latencies were significantly shorter for the NH than for the CI group. Similarly, SRTs were significantly better for the NH than for the CI group. Across all participants, P300 amplitude was significantly correlated with SRTs in steady noise (r = -0.65, p = 0.001) and with the competing male (r = -0.62, p = 0.002) and female maskers (r = -0.60, p = 0.003). Within the CI group, there was a significant correlation between P300 amplitude and SRTs with the male masker (r = -0.78, p = 0.005), which produced the most informational masking. The results suggest that P300 amplitude may be a clinically useful neural correlate of central auditory processing capabilities (e.g., susceptibility to informational masking) in bimodal CI patients.

High expectancy influences the role of cognitive load in inattentional deafness during landing decision-making

Neglecting a critical auditory alarm is a major obstacle to maintaining a safe environment, especially in aviation. Earlier studies have indicated that tasks with a higher perceptual or cognitive load in the visual modality influence the processing of auditory stimuli. It is unclear, however, whether other factors, such as memory failure, active neglect, or expectancy influence the effect of cognitive load on auditory alarm detection sensitivity during aeronautical decision-making. In this study, we investigated this issue in three laboratory experiments using the technique of signal detection analysis, in which participants were asked to make a landing decision based on indicators of the instrument landing system while also trying to detect an audible alarm. We found that the sensitivity of auditory alarm detection was reduced under conditions of high cognitive load and that this effect persisted even when the auditory detection response occurred first (before the landing decision response) and when the probability of an auditory alarm was 40%. However, the sensitivity of auditory detection was not influenced by cognitive load under high expectancy conditions (60% probability of alarm presentation). Furthermore, the value of the response bias was reduced under high cognitive load conditions when the probability of an auditory alarm was low (20%). With an increase in the level of expectancy (40% and 60% probability of alarm presentation), it was found that cognitive load did not influence the response bias. These findings indicate that visual cognitive load affects the sensitivity to an auditory alarm only at a low expectancy level (20% and 40% probability of alarm presentation). The effect of cognitive load on the sensitivity to an auditory alarm was not due to memory failure or active neglect and the response bias was more sensitive to the expectancy factor.

Attention Detection in Virtual Environments Using EEG Signals: A Scoping Review

The competitive demand for attention is present in our daily lives, and the identification of neural processes in the EEG signals associated with the demand for specific attention can be useful to the individual’s interactions in virtual environments. Since EEG-based devices can be portable, non-invasive, and present high temporal resolution technology for recording neural signal, the interpretations of virtual systems user’s attention, fatigue and cognitive load based on parameters extracted from the EEG signal are relevant for several purposes, such as games, rehabilitation, and therapies. However, despite the large amount of studies on this subject, different methodological forms are highlighted and suggested in this work, relating virtual environments, demand of attention, workload and fatigue applications. In our summarization, we discuss controversies, current research gaps and future directions together with the background and final sections.

Facing successfully high mental workload and stressors: An fMRI study

The present fMRI study aimed at highlighting patterns of brain activations and autonomic activity when confronted with high mental workload and the threat of auditory stressors. Twenty participants performed a complex cognitive task in either safe or aversive conditions. Our results showed that increased mental workload induced recruitment of the lateral frontoparietal executive control network (ECN), along with disengagement of medial prefrontal and posterior cingulate regions of the default mode network (DMN). Mental workload also elicited an increase in heart rate and pupil diameter. Task performance did not decrease under the threat of stressors, most likely due to efficient inhibition of auditory regions, as reflected by a large decrement of activity in the superior temporal gyri. The threat of stressors was also accompanied with deactivations of limbic regions of the salience network (SN), possibly reflecting emotional regulation mechanisms through control from dorsal medial prefrontal and parietal regions, as indicated by functional connectivity analyses. Meanwhile, the threat of stressors induced enhanced ECN activity, likely for improved attentional and cognitive processes toward the task, as suggested by increased lateral prefrontal and parietal activations. These fMRI results suggest that measuring the balance between ECN, SN, and DMN recruitment could be used for objective mental state assessment. In this sense, an extra recruitment of task‐related regions and a high ratio of lateral versus medial prefrontal activity may represent a relevant marker of increased but efficient mental effort, while the opposite may indicate a disengagement from the task due to mental overload and/or stressors.

Contrast Is in the Eye of the Beholder: Infelicitous Beat Gesture Increases Cognitive Load During Online Spoken Discourse Comprehension

We investigated how two cues to contrast-beat gesture and contrastive pitch accenting-affect comprehenders' cognitive load during processing of spoken referring expressions. In two visual-world experiments, we orthogonally manipulated the presence of these cues and their felicity, or fit, with the local (sentence-level) referential context in critical referring expressions while comprehenders' task-evoked pupillary responses (TEPRs) were examined. In Experiment 1, beat gesture and contrastive accenting always matched the referential context of filler referring expressions and were therefore relatively felicitous on the global (experiment) level, whereas in Experiment 2, beat gesture and contrastive accenting never fit the referential context of filler referring expressions and were therefore infelicitous on the global level. The results revealed that both beat gesture and contrastive accenting increased comprehenders' cognitive load. For beat gesture, this increase in cognitive load was driven by both local and global infelicity. For contrastive accenting, this increase in cognitive load was unaffected when cues were globally felicitous but exacerbated when cues were globally infelicitous. Together, these results suggest that comprehenders' cognitive resources are taxed by processing infelicitous use of beat gesture and contrastive accenting to convey contrast on both the local and global levels.

The role of hippocampal theta oscillations in working memory impairment in multiple sclerosis

Working memory (WM) problems are frequently present in people with multiple sclerosis (MS). Even though hippocampal damage has been repeatedly shown to play an important role, the underlying neurophysiological mechanisms remain unclear. This study aimed to investigate the neurophysiological underpinnings of WM impairment in MS using magnetoencephalography (MEG) data from a visual-verbal 2-back task. We analysed MEG recordings of 79 MS patients and 38 healthy subjects through event-related fields and theta (4-8 Hz) and alpha (8-13 Hz) oscillatory processes. Data was source reconstructed and parcellated based on previous findings in the healthy subject sample. MS patients showed a smaller maximum theta power increase in the right hippocampus between 0 and 400 ms than healthy subjects (p = .014). This theta power increase value correlated negatively with reaction time on the task in MS (r = -.32, p = .029). Evidence was provided that this relationship could not be explained by a 'common cause' confounding relationship with MS-related neuronal damage. This study provides the first neurophysiological evidence of the influence of hippocampal dysfunction on WM performance in MS.

Phasic activity of the locus-coeruleus is not a mediator of the relationship between fitness and inhibition in college-aged adults

Aerobic fitness is consistently and robustly associated with superior performance on assessments of cognitive control. One potential mechanism underlying this phenomenon is activation of the locus-coeruleus. Specifically, individuals with greater aerobic fitness may be better able to sustain engagement in a cognitively demanding task via a superior ability to meet the metabolic demands of this neural system. Accordingly, the present investigation examined 1) the relationship between aerobic fitness and phasic activation of the locus-coeruleus (indexed using pupillometry) and 2) the potential mediating influence of locus-coeruleus activity on the relationship between aerobic fitness and cognitive task performance. Participants performed an inhibition task while their pupillary responses were measured using an infrared eye tracker. A VO_2max test was then performed to determine individuals' aerobic fitness levels. Consistent with previous research, higher levels of aerobic fitness were related to shorter reaction time. However, phasic activity of the locus-coeruleus did not mediate this relationship - nor did it relate to aerobic fitness level. These results suggest that aerobic fitness does not relate to differences in locus-coeruleus activity in the context of cognitive control in college-aged adults.

Linear and non linear measures of pupil size as a function of hypnotizability

Higher arousal and cortical excitability have been observed in high hypnotizable individuals (highs) with respect to low hypnotizables (lows), which may be due to differences in the activation of ascending activating systems. The present study investigated the possible hypnotizability-related difference in the cortical noradrenergic tone sustained by the activity of the Locus Coeruleus which is strongly related to pupil size. This was measured during relaxation in three groups of participants—highs (N = 15), lows (N = 15) and medium hypnotizable individuals (mediums, N = 11)—in the time and frequency domains and through the Recurrence Quantification Analysis. ECG and Skin Conductace (SC) were monitored to extract autonomic indices of relaxation (heart interbeats intervals, parasympathetic component of heart rate variability (RMSSD) and tonic SC (MeanTonicSC). Most variables indicated that participants relaxed throughout the session. Pupil features did not show significant differences between highs, mediums and lows, except for the spectral Band Median Frequency which was higher in mediums than in lows and highs at the beginning, but not at the end of the session.Thus, the present findings of pupil size cannot account for the differences in arousal and motor cortex excitability observed between highs and lows in resting conditions.

Human Mirror Neuron System Based Alarms in the Cockpit: A Neuroergonomic Evaluation

Controlled Flight Into Terrain (CFIT) events still remain among the deadliest accidents in aviation. When facing the possible occurrence of such an event, pilots have to immediately react to the ground proximity alarm ("Pull Up" alarm) in order to avoid the impending collision. However, the pilots' reaction to this alarm is not always optimal. This may be at least partly due to the low visual saliency of the current alarm and the deleterious effects of stress that alleviate the pilot's reactions. In the present study, two experiments (in a laboratory and in a flight simulator) were conducted to (1) investigate whether hand gesture videos (a hand pulling back the sidestick) can trigger brainwave frequencies related to the mirror neuron system; (2) determine whether enhancing the visual characteristics of the "Pull Up" alarm could improve pilots' response times. Electrophysiological results suggest that hand gesture videos attracted more participants' attention (greater alpha desynchronization in the parieto-occipital area) and possibly triggered greater activity of the mirror neuron system (greater mu and beta desynchronizations at central electrodes). Results obtained in the flight simulator revealed that enhancing the visual characteristics of the original "Pull Up" alarm improved the pilots' reaction times. However, no significant difference in reaction times between an enlarged "Pull Up" inscription and the hand gesture video was found. Further work is needed to determine whether mirror neuron system based alarms could bring benefits for flight safety, in particular, these alarms should be assessed during a high stress context.

The "SuperAgers" construct in clinical practice: neuropsychological assessment of illiterate and educated elderly

INTRODUCTION

The demographic transition is a global event intensified during the last decades that represents population aging. Thus, the studies directed to the elderly 80 years of age or more with preserved cognitive functions (named SuperAgers) emerges as a possible path to full comprehension of the health of those aging with acceptable levels of functionality and independency.

OBJECTIVE

To evaluate the cognitive performance of the elderly over 80 years old, associating the results to their educational level.

METHOD

We evaluated 144 healthy elders with 80 years or more through the following cognitive tests Mini-Mental State Examination (MMSE), Cambridge Cognitive Examination (CAMCOG), Clock Drawing Test (CDT), and Verbal Fluency Test (VF) and compared the tests' scores with their educational level segmented in years of formal education, being the groups ILLITR (<1 year of schooling), 1TO4 (from 1 to 4 years of schooling), and 5MORE (>5 years of schooling).

RESULTS

There was positive influence of educational level on the cognitive tests' score, which indicates higher cognitive reserve of the elderly with higher educational levels.

CONCLUSION

The functionality and independence of the so-called SuperAgers is determined by the cognitive reserve acquired throughout life, mainly developed by the years of formal education.

Decoding P300 Variability Using Convolutional Neural Networks

Deep convolutional neural networks (CNN) have previously been shown to be useful tools for signal decoding and analysis in a variety of complex domains, such as image processing and speech recognition. By learning from large amounts of data, the representations encoded by these deep networks are often invariant to moderate changes in the underlying feature spaces. Recently, we proposed a CNN architecture that could be applied to electroencephalogram (EEG) decoding and analysis. In this article, we train our CNN model using data from prior experiments in order to later decode the P300 evoked response from an unseen, hold-out experiment. We analyze the CNN output as a function of the underlying variability in the P300 response and demonstrate that the CNN output is sensitive to the experiment-induced changes in the neural response. We then assess the utility of our approach as a means of improving the overall signal-to-noise ratio in the EEG record. Finally, we show an example of how CNN-based decoding can be applied to the analysis of complex data.

Potential Mechanisms for the Ketamine-Induced Reduction of P3b Amplitudes

In specific dosages, the N-methyl-D-aspartate receptor (NMDA) antagonist ketamine can be used to model transient psychotic symptoms in healthy individuals that resemble those of schizophrenia. Ketamine administration also temporarily impairs cognitive functions, which can be studied by event-related potentials (ERPs). ERPs also allow dissecting what stages of information processing are affected by ketamine and what stages remain functional. For tasks requiring the differentiation of targets and non-targets, it has repeatedly been shown that ketamine administration in healthy individuals leads to decreased amplitudes of the ERP component P3b in response to target stimuli. However, it could be argued that this ketamine-induced P3b reduction is the consequence of an increased difficulty to differentiate targets from non-targets, primarily mediated by ketamine's psychotomimetic rather than pharmacological effects. The current review of ERP studies seeks to clarify the issue whether P3b effects of ketamine may indeed be explained as the consequence of an experienced increase in task difficulty or whether alternative mechanisms are perhaps more plausible. The review first summarizes the effects of task difficulty on ERP components related to intentional stimulus categorization (P3b), involuntary attention switches to distractors (P3a), as well as sensory processing (P1, N1). Secondly, the ERP effects of task difficulty are contrasted with those observed in ketamine studies in healthy individuals. Findings show that P3b amplitudes are consistently diminished by an increased task difficulty, as well as after ketamine administration. In contrast and as most important difference, increased task difficulty leads to increased P3a amplitudes to distractors presented in same modality as targets, whereas ketamine leads to reduced P3a amplitudes for such distractors. This dissociation indicates that the decreased P3b amplitudes after ketamine cannot be explained by a drug-induced increase in task difficulty. The conjoint reductions of P3a and P3b amplitudes instead suggest that working memory operations, in particular working memory updating are impaired after ketamine, which is in line with previous behavioral findings.

Pupillary Measures of the Cognitive Effort in Auditory Novel Word Processing and Short-Term Retention

The use of the task-evoked pupillary responses (TEPRs) methodology is emerging in the psycholinguistics literature, as a sensitive, reliable and dynamic psychophysiological measure of the cognitive effort produced by various aspects of language processing. This preliminary study aimed to assess the functionality and effectiveness of a TEPRs design for measuring the cognitive effort required for the processing and spontaneous (non-explicitly prompted) short-term retention of novel phonological forms presented auditorily. Twenty-four young adult participants (aged 19-28 years, M = 20.3, SD = 2.13) were auditorily presented with a series of pseudowords differing in their number of syllables and their syllabic complexity. Then, they were asked to produce a response to a delayed pseudoword-color matching task aimed to induce the short-term retention of the novel forms. Results on the size and timing of the TEPRs reveal a significant pupillary activation, starting immediately after the presentation of the auditory stimuli, peaking at 1080 ms and not subsiding significantly during the protracted retention period. Moreover, the differential complexity of the novel words phonology significantly affected pupillary activation. Overall, these preliminary results point to the effectiveness of pupillometry as a technique for capturing the cognitive effort entailed in the short-term maintenance of novel word forms in the phonological loop, a process deemed crucial in the everyday novel word learning process. Results are discussed in view of future research that could establish and extend their implications.

Assessment of Ocular and Physiological Metrics to Discriminate Flight Phases in Real Light Aircraft

OBJECTIVE

The purpose of the present study was to find psychophysiological proxies that are straightforward to use and could be implemented in actual flight conditions to accurately discriminate pilots' workload levels.

BACKGROUND

Piloting an aircraft is a complex activity where cognitive limitations may jeopardize flight safety. There is a need to implement solutions to monitor pilots' workload level to improve flight safety. There has been recent interest in combining psychophysiological measurements. Most of these studies were conducted in flight simulators at the group level, limiting the interpretation of the results.

METHODS

We conducted an experiment with 11 pilots performing two standard traffic patterns in a light aircraft. Five metrics were derived from their ocular and cardiac activities and were evaluated through three flight phases: takeoff, downwind, and landing.

RESULTS

Statistical analyses showed that the saccadic rate was the most efficient metric to distinguish between the three flight phases. In addition, a classifier trained on the ocular data collected from the first run predicted the flight phase within a second run with an accuracy of 75%. No gain in the classifier accuracy has been found by combining cardiac and ocular metrics.

CONCLUSIONS

Ocular-based metrics may be more suitable than cardiac ones to provide relevant information on pilots' flying activity in operational settings.

APPLICATIONS

Electrocardiographic and eye-tracking devices could be implemented in future cockpits as additional flight data for accident analysis, an objective pilot's state evaluation for training, and proxies for human-machine interactions to improve flight safety.

Pupil Sizes Scale with Attentional Load and Task Experience in a Multiple Object Tracking Task

Previous studies have related changes in attentional load to pupil size modulations. However, studies relating changes in attentional load and task experience on a finer scale to pupil size modulations are scarce. Here, we investigated how these changes affect pupil sizes. To manipulate attentional load, participants covertly tracked between zero and five objects among several randomly moving objects on a computer screen. To investigate effects of task experience, the experiment was conducted on three consecutive days. We found that pupil sizes increased with each increment in attentional load. Across days, we found systematic pupil size reductions. We compared the model fit for predicting pupil size modulations using attentional load, task experience, and task performance as predictors. We found that a model which included attentional load and task experience as predictors had the best model fit while adding performance as a predictor to this model reduced the overall model fit. Overall, results suggest that pupillometry provides a viable metric for precisely assessing attentional load and task experience in visuospatial tasks.

Neural and psychophysiological correlates of human performance under stress and high mental workload

In our anxiogenic and stressful world, the maintenance of an optimal cognitive performance is a constant challenge. It is particularly true in complex working environments (e.g. flight deck, air traffic control tower), where individuals have sometimes to cope with a high mental workload and stressful situations. Several models (i.e. processing efficiency theory, cognitive-energetical framework) have attempted to provide a conceptual basis on how human performance is modulated by high workload and stress/anxiety. These models predict that stress can reduce human cognitive efficiency, even in the absence of a visible impact on the task performance. Performance may be protected under stress thanks to compensatory effort, but only at the expense of a cognitive cost. Yet, the psychophysiological cost of this regulation remains unclear. We designed two experiments involving pupil diameter, cardiovascular and prefrontal oxygenation measurements. Participants performed the Toulouse N-back Task that intensively engaged both working memory and mental calculation processes under the threat (or not) of unpredictable aversive sounds. The results revealed that higher task difficulty (higher n level) degraded the performance and induced an increased tonic pupil diameter, heart rate and activity in the lateral prefrontal cortex, and a decreased phasic pupil response and heart rate variability. Importantly, the condition of stress did not impact the performance, but at the expense of a psychophysiological cost as demonstrated by lower phasic pupil response, and greater heart rate and prefrontal activity. Prefrontal cortex seems to be a central region for mitigating the influence of stress because it subserves crucial functions (e.g. inhibition, working memory) that can promote the engagement of coping strategies. Overall, findings confirmed the psychophysiological cost of both mental effort and stress. Stress likely triggered increased motivation and the recruitment of additional cognitive resources that minimize its aversive effects on task performance (effectiveness), but these compensatory efforts consumed resources that caused a loss of cognitive efficiency (ratio between performance effectiveness and mental effort).