Pupillary responses on the visual backward masking task reflect general cognitive ability

Pupil dilation deconvolution reveals the dynamics of attention at high temporal resolution

The size of the human pupil increases as a function of mental effort. However, this response is slow, and therefore its use is thought to be limited to measurements of slow tasks or tasks in which meaningful events are temporally well separated. Here we show that high-temporal-resolution tracking of attention and cognitive processes can be obtained from the slow pupillary response. Using automated dilation deconvolution, we isolated and tracked the dynamics of attention in a fast-paced temporal attention task, allowing us to uncover the amount of mental activity that is critical for conscious perception of relevant stimuli. We thus found evidence for specific temporal expectancy effects in attention that have eluded detection using neuroimaging methods such as EEG. Combining this approach with other neuroimaging techniques can open many research opportunities to study the temporal dynamics of the mind's inner eye in great detail.

Pupillometry

The measurement of pupil diameter in psychology (in short, “pupillometry”) has just celebrated 50 years. The method established itself after the appearance of three seminal studies ( Hess & Polt, 1960 , 1964 ; Kahneman & Beatty, 1966 ). Since then, the method has continued to play a significant role within the field, and pupillary responses have been successfully used to provide an estimate of the “intensity” of mental activity and of changes in mental states, particularly changes in the allocation of attention and the consolidation of perception. Remarkably, pupillary responses provide a continuous measure regardless of whether the participant is aware of such changes. More recently, research in neuroscience has revealed a tight correlation between the activity of the locus coeruleus (i.e., the “hub” of the noradrenergic system) and pupillary dilation. As we discuss in this short review, these neurophysiological findings provide new important insights to the meaning of pupillary responses for mental activity. Finally, given that pupillary responses can be easily measured in a noninvasive manner, occur from birth, and can occur in the absence of voluntary, conscious processes, they constitute a very promising tool for the study of preverbal (e.g., infants) or nonverbal participants (e.g., animals, neurological patients).

Recognition Memory Strength is Predicted by Pupillary Responses at Encoding While Fixation Patterns Distinguish Recollection from Familiarity

Thirty-five healthy participants incidentally encoded a set of man-made and natural object pictures, while their pupil response and eye movements were recorded. At retrieval, studied and new stimuli were rated as novel, familiar (strong, moderate, or weak), or recollected. We found that both pupil response and fixation patterns at encoding predict later recognition memory strength. The extent of pupillary response accompanying incidental encoding was found to be predictive of subsequent memory. In addition, the number of fixations was also predictive of later recognition memory strength, suggesting that the accumulation of greater visual detail, even for single objects, is critical for the creation of a strong memory. Moreover, fixation patterns at encoding distinguished between recollection and familiarity at retrieval, with more dispersed fixations predicting familiarity and more clustered fixations predicting recollection. These data reveal close links between the autonomic control of pupil responses and eye movement patterns on the one hand and memory encoding on the other. Moreover, the data illustrate quantitative as well as qualitative differences in the incidental visual processing of stimuli, which are differentially predictive of the strength and the kind of memory experienced at recognition.

In the eye of the listener: pupil dilation elucidates discourse processing

The current study investigated cognitive resource allocation in discourse processing by means of pupil dilation and behavioral measures. Short question-answer dialogs were presented to listeners. Either the context question queried a new information focus in the successive answer, or else the context query was corrected in the answer sentence (correction information). The information foci contained in the answer sentences were either adequately highlighted by prosodic means or not. Participants had to judge the adequacy of the focus prosody with respect to the preceding context question. Prosodic judgment accuracy was higher in the conditions bearing adequate focus prosody than in the conditions with inadequate focus prosody. Latency to peak pupil dilation was longer when new information foci were perceived compared to correction foci. Moreover, for the peak dilation, an interaction of focus type and prosody was found. Post hoc statistical tests revealed that prosodically adequate correction focus positions were processed with smaller peak dilation in comparison to all other dialog conditions. Thus, pupil dilation and results of a principal component analysis suggest an interaction of focus type and focus prosody in discourse processing.

Pupillary Stroop effects

We recorded the pupil diameters of participants performing the words' color-naming Stroop task (i.e., naming the color of a word that names a color). Non-color words were used as baseline to firmly establish the effects of semantic relatedness induced by color word distractors. We replicated the classic Stroop effects of color congruency and color incongruency with pupillary diameter recordings: relative to non-color words, pupil diameters increased for color distractors that differed from color responses, while they reduced for color distractors that were identical to color responses. Analyses of the time courses of pupil responses revealed further differences between color-congruent and color-incongruent distractors, with the latter inducing a steep increase of pupil size and the former a relatively lower increase. Consistent with previous findings that have demonstrated that pupil size increases as task demands rise, the present results indicate that pupillometry is a robust measure of Stroop interference, and it represents a valuable addition to the cognitive scientist's toolbox.

Neural correlates of fast pupil dilation in nonhuman primates: relation to behavioral performance and cognitive workload

Pupil dilation in humans has been previously shown to correlate with cognitive workload, whereby increased frequency of dilation is associated with increased degree of difficulty of a task. It has been suggested that frontal oculomotor brain areas control cognitively related pupil dilations, but this has not been confirmed due to lack of animal models of cognitive workload and task-related pupil dilation. This is the first report of a wavelet analysis applied to continuous measures of pupil size used to detect the onset of abrupt pupil dilations and the frequency of those dilations in nonhuman primates (NHPs) performing a trial-unique delayed-match-to-sample (DMS) task. A unique finding shows that electrophysiological recordings in the same animals revealed firing of neurons in frontal cortex correlated to different components of pupil dilation during task performance. It is further demonstrated that the frequency of fast pupil dilations (but not rate of eye movements) correlated with cognitive workload during task performance. Such correlations suggest that frontal neuron encoding of pupil dilation provides critical feedback to other brain areas involved in the processing of complex visual information.

Pupillometry as a measure of cognitive effort in younger and older adults

Two experiments examined the effectiveness of the pupillary response as a measure of cognitive load in younger and older adults. Experiment 1 measured the change in pupil size of younger and older adults while they listened to spoken digit lists that varied in length and retained them briefly for recall. In Experiment 2 changes in relative pupil size were measured while younger and older adults listened to sentences for later recall that varied in syntactic complexity and sentence length. Both age groups' pupil sizes were sensitive to the size of the memory set in Experiment 1 and sentence length in Experiment 2, with the older adults showing a larger effect of the memory load on a normalized measure of pupil size relative to the younger adults. By contrast, only the younger adults showed a difference in the pupillary response to a change in syntactic complexity, even with an adjustment for the reduced reactivity of the older pupil.

Pupillometric measures of attentional allocation to target and mask processing on the backward masking task in schizophrenia

Allocation of attentional resources during early visual processing was investigated in schizophrenia. Pupillary responses were recorded during a backward masking task as an index of resource allocation in schizophrenia patients (n=51) and nonpsychiatric controls (n=51). Two time-linked components of pupillary response waveforms appeared to differentially index resource allocation to targets versus masks. Two patient subgroups were identified: One with normal overall pupillary responses (resource allocation), but greater allocation on mask relative to target components, and another with abnormally small overall pupillary responses and similar allocation between target and mask components. Thus, misallocation of resources to masks contributed to masking deficits in one subgroup, whereas reduced resource allocation contributed to deficits in the other. The nature of resource-related deficits can vary across schizophrenia subgroups.

The coupling of emotion and cognition in the eye: Introducing the pupil old/new effect

The study presented here investigated the effects of emotional valence on the memory for words by assessing both memory performance and pupillary responses during a recognition memory task. Participants had to make speeded judgments on whether a word presented in the test phase of the experiment had already been presented ("old") or not ("new"). An emotion-induced recognition bias was observed: Words with emotional content not only produced a higher amount of hits, but also elicited more false alarms than neutral words. Further, we found a distinct pupil old/new effect characterized as an elevated pupillary response to hits as opposed to correct rejections. Interestingly, this pupil old/new effect was clearly diminished for emotional words. We therefore argue that the pupil old/new effect is not only able to mirror memory retrieval processes, but also reflects modulation by an emotion-induced recognition bias.

Effects of stimulus probability on pupillary dilation and reaction time in categorization

Stimuli presented with a low frequency elicit larger pupillary responses than those presented with a high frequency. Similar effects are known for motor responses observed in reaction time experiments. Utilizing this stimulus probability effect, we conducted a Go/NoGo reaction time experiment and measured pupillary dilation to evaluate categorization. We found no probability effect when participants used distinctive features to classify stimuli with different frequencies into the same category. If no distinctive features could be applied, however, the effect was observed in both pupillary and reaction time data. The stimulus probability effect can serve as a tool for the evaluation of categorization and is discussed in the context of orienting.

Tracking stimulus processing in Pavlovian pupillary conditioning

The aim of the present study was to evaluate whether the anticipatory pupillary dilation response is a useful indicator for the examination of complex differential conditioning problems like patterning. A human fear conditioning procedure with six groups (n=20 each) was used to examine conditioned stimulus (CS) processing when a compound stimulus was reinforced, but not its elements (positive patterning) or when the elements were reinforced, but not the compound (negative patterning), as well as modifications in which the compound was replaced by either a new compound or by a new element. We found evidence for conditioning within 2 s after CS onset. Group differences in differential conditioning indicated systematic differences in CS processing due to different discrimination task difficulties. This study showed that the pupil response is a suitable indicator for human Pavlovian conditioning with high time resolution within and between trials.

Time's arrow and pupillary response

The psychological arrow of time refers to our experience of the forward temporal progression of all natural processes. To investigate whether and how time's arrow is mentally coded in individual everyday events, a relatedness judgment task was used. The items each consisted of a verb (probe) and an adjective or participle (target). The temporal orientation between probe and target was varied either corresponding to the chronological orientation (e.g., shrinking-small) or corresponding to the reverse orientation (e.g., shrinking-large). Reaction times, error rates, and pupillary responses were recorded. Chronological items were processed faster than reverse items. These findings suggest that time's arrow is mentally coded in single everyday events. Pupil dilation and results of principal component analyses suggest top-down influences in the processing of temporally related items.

Pupillary responses and attentional allocation problems on the backward masking task in schizophrenia

Early visual information processing impairment has consistently been found on the backward masking task in patients with schizophrenia, but the nature of this impairment remains unclear. Pupillometry was used to measure attentional allocation during visual backward masking task performance in patients with schizophrenia (n=16) and nonpsychiatric controls (n=16). The extent of pupil dilation recorded during a cognitive task reflects the processing load placed on the nervous system by the task. Schizophrenia patients detected significantly fewer targets than controls only when the stimulus onset asynchrony (SOA) between targets and masks reached 317 ms. For both groups, peak pupil dilation responses were also significantly larger in the 317 ms SOA condition relative to a no-mask condition, suggesting that the processing load of the 317 SOA masking condition was greater than the no-mask condition. In addition, a principal components analysis of pupillary response waveforms identified time-related factors that appeared to differentially index attentional allocation to targets vs. masks. Patients with schizophrenia showed less dilation than controls on a middle factor that appeared to index attentional allocation to targets, but patients showed greater dilation than controls on a late factor that appeared to index attentional allocation to masks. That is, controls attended more to targets than to masks, but patients attended more to masks than to targets. These findings suggest that masking impairments at SOA intervals greater than 100-200 ms may be due abnormalities in attentional allocation mechanisms.