Dynamics and cortical distribution of neural responses to 2D and 3D motion in human

Visual-spatial dimension integration in digital pathology education enhances anatomical pathology learning

Literature review demonstrated a surprising lack of publications on digital e-learning pathology resources for senior medical undergraduates and interns. An interactive Digital Pathology Repository (iDPR) integrating two- and three-dimensional (2D, 3D) high-resolution anatomical pathology images with correlated digital histopathology was developed. The novel iDPR was rigorously evaluated using mixed methods to assess pathology knowledge gains (pre- and post-tests), quality impact analysis (questionnaire), user feedback (focus group discussions) and user visual behaviour (eye gaze tracking analysis of 2D/ 3D images).Exposure to iDPR appeared to improve user pathology knowledge, as observed by significantly increased test scores on topic-related quizzes (n = 69, p < 0.001). In addition, most users were highly satisfied with the key design elements of the iDPR tool. Focus group discussion revealed the iDPR was regarded as a relevant online learning resource, although some minor technical issues were also noted. Interestingly, visual behaviour trends indicated that specific diagnostic pathological lesions could be correctly identified faster in 3D images, when compared to 2D images.The iDPR offers promise and potential in pathology education for senior clinical students and interns, gauging from both qualitative and quantitative positive user feedback. With incorporation of image annotations and interactive functionality, and with further technology development, this would prove a useful tool for diagnostic pathology and telepathology. As images with added visual-spatial dimension can provide enhanced detail and aid more rapid diagnosis, future applications of the iDPR could include virtual reality or holographic images of anatomical pathology specimens.

Decoding Neural Responses to Motion-in-Depth Using EEG

Two stereoscopic cues that underlie the perception of motion-in-depth (MID) are changes in retinal disparity over time (CD) and interocular velocity differences (IOVD). These cues have independent spatiotemporal sensitivity profiles, depend upon different low-level stimulus properties, and are potentially processed along separate cortical pathways. Here, we ask whether these MID cues code for different motion directions: do they give rise to discriminable patterns of neural signals, and is there evidence for their convergence onto a single "motion-in-depth" pathway? To answer this, we use a decoding algorithm to test whether, and when, patterns of electroencephalogram (EEG) signals measured from across the full scalp, generated in response to CD- and IOVD-isolating stimuli moving toward or away in depth can be distinguished. We find that both MID cue type and 3D-motion direction can be decoded at different points in the EEG timecourse and that direction decoding cannot be accounted for by static disparity information. Remarkably, we find evidence for late processing convergence: IOVD motion direction can be decoded relatively late in the timecourse based on a decoder trained on CD stimuli, and vice versa. We conclude that early CD and IOVD direction decoding performance is dependent upon fundamentally different low-level stimulus features, but that later stages of decoding performance may be driven by a central, shared pathway that is agnostic to these features. Overall, these data are the first to show that neural responses to CD and IOVD cues that move toward and away in depth can be decoded from EEG signals, and that different aspects of MID-cues contribute to decoding performance at different points along the EEG timecourse.

Antero-Posterior vs. Lateral Vestibular Input Processing in Human Visual Cortex

Visuo-vestibular integration is crucial for locomotion, yet the cortical mechanisms involved remain poorly understood. We combined binaural monopolar galvanic vestibular stimulation (GVS) and functional magnetic resonance imaging (fMRI) to characterize the cortical networks activated during antero-posterior and lateral stimulations in humans. We focused on functional areas that selectively respond to egomotion-consistent optic flow patterns: the human middle temporal complex (hMT+), V6, the ventral intraparietal (VIP) area, the cingulate sulcus visual (CSv) area and the posterior insular cortex (PIC). Areas hMT+, CSv, and PIC were equivalently responsive during lateral and antero-posterior GVS while areas VIP and V6 were highly activated during antero-posterior GVS, but remained silent during lateral GVS. Using psychophysiological interaction (PPI) analyses, we confirmed that a cortical network including areas V6 and VIP is engaged during antero-posterior GVS. Our results suggest that V6 and VIP play a specific role in processing multisensory signals specific to locomotion during navigation.

Asymmetries between achromatic and chromatic extraction of 3D motion signals

Motion in depth (MID) can be cued by high-resolution changes in binocular disparity over time (CD), and low-resolution interocular velocity differences (IOVD). Computational differences between these two mechanisms suggest that they may be implemented in visual pathways with different spatial and temporal resolutions. Here, we used fMRI to examine how achromatic and S-cone signals contribute to human MID perception. Both CD and IOVD stimuli evoked responses in a widespread network that included early visual areas, parts of the dorsal and ventral streams, and motion-selective area hMT+. Crucially, however, we measured an interaction between MID type and chromaticity. fMRI CD responses were largely driven by achromatic stimuli, but IOVD responses were better driven by isoluminant S-cone inputs. In our psychophysical experiments, when S-cone and achromatic stimuli were matched for perceived contrast, participants were equally sensitive to the MID in achromatic and S-cone IOVD stimuli. In comparison, they were relatively insensitive to S-cone CD. These findings provide evidence that MID mechanisms asymmetrically draw on information in precortical pathways. An early opponent motion signal optimally conveyed by the S-cone pathway may provide a substantial contribution to the IOVD mechanism.

Visuomotor Correlates of Conflict Expectation in the Context of Motor Decisions

Many behaviors require choosing between conflicting options competing against each other in visuomotor areas. Such choices can benefit from top-down control processes engaging frontal areas in advance of conflict when it is anticipated. Yet, very little is known about how this proactive control system shapes the visuomotor competition. Here, we used electroencephalography in human subjects (male and female) to identify the visual and motor correlates of conflict expectation in a version of the Eriksen Flanker task that required left or right responses according to the direction of a central target arrow surrounded by congruent or incongruent (conflicting) flankers. Visual conflict was either highly expected (it occurred in 80% of trials; mostly incongruent blocks) or very unlikely (20% of trials; mostly congruent blocks). We evaluated selective attention in the visual cortex by recording target- and flanker-related steady-state visual-evoked potentials (SSVEPs) and probed action selection by measuring response-locked potentials (RLPs) in the motor cortex. Conflict expectation enhanced accuracy in incongruent trials, but this improvement occurred at the cost of speed in congruent trials. Intriguingly, this behavioral adjustment occurred while visuomotor activity was less finely tuned: target-related SSVEPs were smaller while flanker-related SSVEPs were higher in mostly incongruent blocks than in mostly congruent blocks, and incongruent trials were associated with larger RLPs in the ipsilateral (nonselected) motor cortex. Hence, our data suggest that conflict expectation recruits control processes that augment the tolerance for inappropriate visuomotor activations (rather than processes that downregulate their amplitude), allowing for overflow activity to occur without having it turn into the selection of an incorrect response.SIGNIFICANCE STATEMENT Motor choices made in front of discordant visual information are more accurate when conflict can be anticipated, probably due to the engagement of top-down control from frontal areas. How this control system modulates activity within visual and motor areas is unknown. Here, we show that, when control processes are recruited in anticipation of conflict, as evidenced by higher midfrontal theta activity, visuomotor activity is less finely tuned: visual processing of the goal-relevant location was reduced and the motor cortex displayed more inappropriate activations, compared with when conflict was unlikely. We argue that conflict expectation is associated with an expansion of the distance-to-selection threshold, improving accuracy while the need for online control of visuomotor activity is reduced.

The neural correlates of vertical disparity gradient and cue conflict in Panum's limiting case

Although Panum's limiting case has been extensively researched, only recently has it been discovered that in addition to horizontal disparity, the final perception of depth is influenced by (i) the vertical disparity gradient and (ii) the degree of cue conflict between 2D and 3D shapes. The present study examines the neural correlates of the two factors, using EEG while observers viewed several versions of stereoscopic stimuli, which depicted Panum's limiting case. In these patterns the vertical disparity gradient was varied from 0.1 to 0.6, while the degree of cue conflict was manipulated from low to high. The ERP data showed that the amplitude of the N170 component (exogenous) was modulated by the vertical disparity gradient and cue conflict. In contrast, the N270 component (endogenous) was modulated by cue conflict only. Such findings demonstrate that both factors affect the perception of depth in Panum's limiting case, but at different stages: the vertical disparity gradient at an early stage of processing (N170) and cue conflict at two stages (N170 and N270). Hence, vertical disparity gradient is related to low-level visual stimulus parameters and can modulate exogenous component, while cue conflict is related to both exogenous and endogenous components.

Children's Brain Responses to Optic Flow Vary by Pattern Type and Motion Speed

Structured patterns of global visual motion called optic flow provide crucial information about an observer's speed and direction of self-motion and about the geometry of the environment. Brain and behavioral responses to optic flow undergo considerable postnatal maturation, but relatively little brain imaging evidence describes the time course of development in motion processing systems in early to middle childhood, a time when psychophysical data suggest that there are changes in sensitivity. To fill this gap, electroencephalographic (EEG) responses were recorded in 4- to 8-year-old children who viewed three time-varying optic flow patterns (translation, rotation, and radial expansion/contraction) at three different speeds (2, 4, and 8 deg/s). Modulations of global motion coherence evoked coherent EEG responses at the first harmonic that differed by flow pattern and responses at the third harmonic and dot update rate that varied by speed. Pattern-related responses clustered over right lateral channels while speed-related responses clustered over midline channels. Both children and adults show widespread responses to modulations of motion coherence at the second harmonic that are not selective for pattern or speed. The results suggest that the developing brain segregates the processing of optic flow pattern from speed and that an adult-like pattern of neural responses to optic flow has begun to emerge by early to middle childhood.

Dorsal and ventral stream contributions to form-from-motion perception in a patient with form-from motion deficit: a case report

The main model of visual processing in primates proposes an anatomo-functional distinction between the dorsal stream, specialized in spatio-temporal information, and the ventral stream, processing essentially form information. However, these two pathways also communicate to share much visual information. These dorso-ventral interactions have been studied using form-from-motion (FfM) stimuli, revealing that FfM perception first activates dorsal regions (e.g., MT+/V5), followed by successive activations of ventral regions (e.g., LOC). However, relatively little is known about the implications of focal brain damage of visual areas on these dorso-ventral interactions. In the present case report, we investigated the dynamics of dorsal and ventral activations related to FfM perception (using topographical ERP analysis and electrical source imaging) in a patient suffering from a deficit in FfM perception due to right extrastriate brain damage in the ventral stream. Despite the patient's FfM impairment, both successful (observed for the highest level of FfM signal) and absent/failed FfM perception evoked the same temporal sequence of three processing states observed previously in healthy subjects. During the first period, brain source localization revealed cortical activations along the dorsal stream, currently associated with preserved elementary motion processing. During the latter two periods, the patterns of activity differed from normal subjects: activations were observed in the ventral stream (as reported for normal subjects), but also in the dorsal pathway, with the strongest and most sustained activity localized in the parieto-occipital regions. On the other hand, absent/failed FfM perception was characterized by weaker brain activity, restricted to the more lateral regions. This study shows that in the present case report, successful FfM perception, while following the same temporal sequence of processing steps as in normal subjects, evoked different patterns of brain activity. By revealing a brain circuit involving the most rostral part of the dorsal pathway, this study provides further support for neuro-imaging studies and brain lesion investigations that have suggested the existence of different brain circuits associated with different profiles of interaction between the dorsal and the ventral streams.

Learning the 3-D structure of objects from 2-D views depends on shape, not format

Humans can learn to recognize new objects just from observing example views. However, it is unknown what structural information enables this learning. To address this question, we manipulated the amount of structural information given to subjects during unsupervised learning by varying the format of the trained views. We then tested how format affected participants' ability to discriminate similar objects across views that were rotated 90° apart. We found that, after training, participants' performance increased and generalized to new views in the same format. Surprisingly, the improvement was similar across line drawings, shape from shading, and shape from shading + stereo even though the latter two formats provide richer depth information compared to line drawings. In contrast, participants' improvement was significantly lower when training used silhouettes, suggesting that silhouettes do not have enough information to generate a robust 3-D structure. To test whether the learned object representations were format-specific or format-invariant, we examined if learning novel objects from example views transfers across formats. We found that learning objects from example line drawings transferred to shape from shading and vice versa. These results have important implications for theories of object recognition because they suggest that (a) learning the 3-D structure of objects does not require rich structural cues during training as long as shape information of internal and external features is provided and (b) learning generates shape-based object representations independent of the training format.

Using Functional Near Infrared Spectroscopy (fNIRS) to Study Dynamic Stereoscopic Depth Perception

The parietal cortex has been widely implicated in the processing of depth perception by many neuroimaging studies, yet functional near infrared spectroscopy (fNIRS) has been an under-utilised tool to examine the relationship of oxy- ([HbO]) and de-oxyhaemoglobin ([HbR]) in perception. Here we examine the haemodynamic response (HDR) to the processing of induced depth stimulation using dynamic random-dot-stereograms (RDS). We used fNIRS to measure the HDR associated with depth perception in healthy young adults (n = 13, mean age 24). Using a blocked design, absolute values of [HbO] and [HbR] were recorded across parieto-occipital and occipital cortices, in response to dynamic RDS. Control and test images were identical except for the horizontal shift in pixels in the RDS that resulted in binocular disparity and induced the percept of a 3D sine wave that ‘popped out’ of the test stimulus. The control stimulus had zero disparity and induced a ‘flat’ percept. All participants had stereoacuity within normal clinical limits and successfully perceived the depth in the dynamic RDS. Results showed a significant effect of this complex visual stimulation in the right parieto-occipital cortex (p < 0.01, η² = 0.54). The test stimulus elicited a significant increase in [HbO] during depth perception compared to the control image (p < 0.001, 99.99 % CI [0.008–0.294]). The similarity between the two stimuli may have resulted in the HDR of the occipital cortex showing no significant increase or decrease of cerebral oxygenation levels during depth stimulation. Cerebral oxygenation measures of [HbO] confirmed the strong association of the right parieto-occipital cortex with processing depth perception. Our study demonstrates the validity of fNIRS to investigate [HbO] and [HbR] during high-level visual processing of complex stimuli.

Aesthetic preference recognition of 3D shapes using EEG

Recognition and identification of aesthetic preference is indispensable in industrial design. Humans tend to pursue products with aesthetic values and make buying decisions based on their aesthetic preferences. The existence of neuromarketing is to understand consumer responses toward marketing stimuli by using imaging techniques and recognition of physiological parameters. Numerous studies have been done to understand the relationship between human, art and aesthetics. In this paper, we present a novel preference-based measurement of user aesthetics using electroencephalogram (EEG) signals for virtual 3D shapes with motion. The 3D shapes are designed to appear like bracelets, which is generated by using the Gielis superformula. EEG signals were collected by using a medical grade device, the B-Alert X10 from advance brain monitoring, with a sampling frequency of 256 Hz and resolution of 16 bits. The signals obtained when viewing 3D bracelet shapes were decomposed into alpha, beta, theta, gamma and delta rhythm by using time-frequency analysis, then classified into two classes, namely like and dislike by using support vector machines and K-nearest neighbors (KNN) classifiers respectively. Classification accuracy of up to 80 % was obtained by using KNN with the alpha, theta and delta rhythms as the features extracted from frontal channels, Fz, F3 and F4 to classify two classes, like and dislike.

The steady-state visual evoked potential in vision research: A review

Periodic visual stimulation and analysis of the resulting steady-state visual evoked potentials were first introduced over 80 years ago as a means to study visual sensation and perception. From the first single-channel recording of responses to modulated light to the present use of sophisticated digital displays composed of complex visual stimuli and high-density recording arrays, steady-state methods have been applied in a broad range of scientific and applied settings.The purpose of this article is to describe the fundamental stimulation paradigms for steady-state visual evoked potentials and to illustrate these principles through research findings across a range of applications in vision science.

Vergence eye movements in patients with schizophrenia

Previous studies have shown that smooth pursuit eye movements are impaired in patients with schizophrenia. However, under normal viewing conditions, targets move not only in the frontoparallel plane but also in depth, and tracking them requires both smooth pursuit and vergence eye movements. Although previous studies in humans and non-human primates suggest that these two eye movement subsystems are relatively independent of one another, to our knowledge, there have been no prior studies of vergence tracking behavior in patients with schizophrenia. Therefore, we have investigated these eye movements in patients with schizophrenia and in healthy controls. We found that patients with schizophrenia exhibited substantially lower gains compared to healthy controls during vergence tracking at all tested speeds (e.g. 0.25 Hz vergence tracking mean gain of 0.59 vs. 0.86). Further, consistent with previous reports, patients with schizophrenia exhibited significantly lower gains than healthy controls during smooth pursuit at higher target speeds (e.g. 0.5 Hz smooth pursuit mean gain of 0.64 vs. 0.73). In addition, there was a modest (r≈0.5), but significant, correlation between smooth pursuit and vergence tracking performance in patients with schizophrenia. Our observations clearly demonstrate substantial vergence tracking deficits in patients with schizophrenia. In these patients, deficits for smooth pursuit and vergence tracking are partially correlated suggesting overlap in the central control of smooth pursuit and vergence eye movements.

How to use fMRI functional localizers to improve EEG/MEG source estimation

EEG and MEG have excellent temporal resolution, but the estimation of the neural sources that generate the signals recorded by the sensors is a difficult, ill-posed problem. The high spatial resolution of functional MRI makes it an ideal tool to improve the localization of the EEG/MEG sources using data fusion. However, the combination of the two techniques remains challenging, as the neural generators of the EEG/MEG and BOLD signals might in some cases be very different. Here we describe a data fusion approach that was developed by our team over the last decade in which fMRI is used to provide source constraints that are based on functional areas defined individually for each subject. This mini-review describes the different steps that are necessary to perform source estimation using this approach. It also provides a list of pitfalls that should be avoided when doing fMRI-informed EEG/MEG source imaging. Finally, it describes the advantages of using a ROI-based approach for group-level analysis and for the study of sensory systems.