The human egomotion network

All volitional movement in a three-dimensional space requires multisensory integration, in particular of visual and vestibular signals. Where and how the human brain processes and integrates self-motion signals remains enigmatic. Here, we applied visual and vestibular self-motion stimulation using fast and precise whole-brain neuroimaging to delineate and characterize the entire cortical and subcortical egomotion network in a substantial cohort (n=131). Our results identify a core egomotion network consisting of areas in the cingulate sulcus (CSv, PcM/pCi), the cerebellum (uvula), and the temporo-parietal cortex including area VPS and an unnamed region in the supramarginal gyrus. Based on its cerebral connectivity pattern and anatomical localization, we propose that this region represents the human homologue of macaque area 7a. Whole-brain connectivity and gradient analyses imply an essential role of the connections between the cingulate sulcus and the cerebellar uvula in egomotion perception. This could be via feedback loops involved updating visuo-spatial and vestibular information. The unique functional connectivity patterns of PcM/pCi hint at central role in multisensory integration essential for the perception of self-referential spatial awareness. All cortical egomotion hubs showed modular functional connectivity with other visual, vestibular, somatosensory and higher order motor areas, underlining their mutual function in general sensorimotor integration.

The cingulate oculomotor cortex

Knowledge about the relevance and extent of human eye movement control in the cingulate cortex to date is very limited. Experiments in non-human primates brought about evidence for a potentially central role of the dorsal bank of the cingulate sulcus in saccadic eye movements. In humans, a putative cingulate eye field (CEF) in the same region has been proposed; however, its function and location still remain controversial. Another area in the posterior cingulate cortex, the cingulate sulcus visual area (CSv), has been shown to respond to visual motion cues and also ocular motor tasks. In this study we used multi-band neuroimaging (n = 46) to comprehensively characterize oculomotor responses along the entire cingulate cortex during the most common types of eye movements. We were able to robustly localize the CEF to the anterior portion of the midcingulate gyrus. The region gave responses during all oculomotor tasks and is embedded within the ventral attention network. Area CSv, which is located in the anterior portion of the posterior cingulate gyrus, on the other hand responded to smooth pursuit and optokinetic nystagmus only. It likewise represents a node within the ventral attention network but at the same time seems to be a distinctive part of the somatomotor network. Our findings support an executive role of the CEF, suggesting a cognitive control function in maintaining and adapting different kinds of eye movements. CSv on the other hand might be an interface for relaying oculomotor, visual motion and broad sensory signals related to self-motion.

DeepVOG: Open-source pupil segmentation and gaze estimation in neuroscience using deep learning

BACKGROUND

A prerequisite for many eye tracking and video-oculography (VOG) methods is an accurate localization of the pupil. Several existing techniques face challenges in images with artifacts and under naturalistic low-light conditions, e.g. with highly dilated pupils.

NEW METHOD

For the first time, we propose to use a fully convolutional neural network (FCNN) for segmentation of the whole pupil area, trained on 3946 VOG images hand-annotated at our institute. We integrate the FCNN into DeepVOG, along with an established method for gaze estimation from elliptical pupil contours, which we improve upon by considering our FCNN's segmentation confidence measure.

RESULTS

The FCNN output simultaneously enables us to perform pupil center localization, elliptical contour estimation and blink detection, all with a single network and with an assigned confidence value, at framerates above 130 Hz on commercial workstations with GPU acceleration. Pupil centre coordinates can be estimated with a median accuracy of around 1.0 pixel, and gaze estimation is accurate to within 0.5 degrees. The FCNN is able to robustly segment the pupil in a wide array of datasets that were not used for training.

COMPARISON WITH EXISTING METHODS

We validate our method against gold standard eye images that were artificially rendered, as well as hand-annotated VOG data from a gold-standard clinical system (EyeSeeCam) at our institute.

CONCLUSIONS

Our proposed FCNN-based pupil segmentation framework is accurate, robust and generalizes well to new VOG datasets. We provide our code and pre-trained FCNN model open-source and for free under www.github.com/pydsgz/DeepVOG.