The dynamics of contour integration: A simultaneous EEG-fMRI study

Utilizing Motion Capture Systems for Instrumenting the OCRA Index: A Study on Risk Classification for Upper Limb Work-Related Activities

In the search to enhance ergonomic risk assessments for upper limb work-related activities, this study introduced and validated the efficiency of an inertial motion capture system, paired with a specialized platform that digitalized the OCRA index. Conducted in a semi-controlled environment, the proposed methodology was compared to traditional risk classification techniques using both inertial and optical motion capture systems. The inertial method encompassed 18 units in a Bluetooth Low Energy tree topology network for activity recording, subsequently analyzed for risk using the platform. Principal outcomes emphasized the optical system's preeminence, aligning closely with the conventional technique. The optical system's superiority was further evident in its alignment with the traditional method. Meanwhile, the inertial system followed closely, with an error margin of just ±0.098 compared to the optical system. Risk classification was consistent across all systems. The inertial system demonstrated strong performance metrics, achieving F1-scores of 0.97 and 1 for "risk" and "no risk" classifications, respectively. Its distinct advantage of portability was reinforced by participants' feedback on its user-friendliness. The results highlight the inertial system's potential, mirroring the precision of both traditional and optical methods and achieving a 65% reduction in risk assessment time. This advancement mitigates the need for intricate video setups, emphasizing its potential in ergonomic assessments.

Contour integration deficits at high spatial frequencies in children treated for anisometropic amblyopia

Purpose

This study was conducted to reexamine the question of whether children treated for anisometropic amblyopia have contour integration deficits. To do so, we used psychophysical methods that require global contour processing while minimizing the influence of low-level deficits: visibility, shape perception, and positional uncertainty.

Methods

Thirteen children with anisometropic amblyopia (age: 10.1 ± 1.8 years) and thirteen visually normal children (age: 10.8 ± 2.0 years) participated in this study. The stimuli were closed figures made up of Gabor patches either in noise or on a blank field. The contrast thresholds to detect a circular contour on a blank field, as well as the thresholds of aspect ratio and contour element number to discriminate a circular or elliptical contour in noise, were measured at Gabor spatial frequencies of 1.5, 3, and 6 cpd for amblyopic eyes (AEs), fellow eyes (FEs), and normal control eyes. Visual acuities and contrast sensitivity functions for AEs and FEs and the Randot stereoacuity were measured before testing.

Results

The AEs showed contrast deficits and degraded shape perception compared to the FEs at higher spatial frequencies (6 cpd). When the influence of abnormal contrast sensitivity and shape perception were minimized, the AEs showed contour integration deficits at spatial frequencies 3 and 6 cpd. These deficits were not related to basic losses in contrast sensitivity and acuity, stereoacuity, and visual crowding. Besides, no significant difference was found between the fellow eyes of the amblyopic children and the normal control eyes in the performance of contour integration.

Conclusion

After eliminating or compensating for the low-level deficits, children treated for anisometropic amblyopia still show contour integration deficits, primarily at higher spatial frequencies, which might reflect the deficits in global processing caused by amblyopia. Contour integration deficits are likely independent of spatial vision deficits. Refractive correction and/or occlusion therapies may not be sufficient to fully restore contour integration deficits, which indicates the need for the development of clinical treatments to recover these deficits.

Masking, crowding, and grouping: Connecting low and mid-level vision

An important task for vision science is to build a unitary framework of low- and mid-level vision. As a step on this way, our study examined differences and commonalities between masking, crowding and grouping-three processes that occur through spatial interactions between neighbouring elements. We measured contrast thresholds as functions of inter-element spacing and eccentricity for Gabor detection, discrimination and contour integration, using a common stimulus grid consisting of nine Gabor elements. From these thresholds, we derived a) the baseline contrast necessary to perform each task and b) the spatial extent over which task performance was stable. This spatial window can be taken as an indicator of field size, where elements that fall within a putative field are readily combined. We found that contrast thresholds were universally modulated by inter-element distance, with a shallower and inverted effect for grouping compared with masking and crowding. Baseline contrasts for detecting stimuli and discriminating their properties were positively linked across the tested retinal locations (parafovea and near periphery), whereas those for integrating elements and discriminating their properties were negatively linked. Meanwhile, masking and crowding spatial windows remained uncorrelated across eccentricity, although they were correlated across participants. This suggests that the computation performed by each type of visual field operates over different distances that co-varies across observers, but not across retinal locations. Contrast-processing units may thus lie at the core of the shared idiosyncrasies across tasks reported in many previous studies, despite the fundamental differences in the extent of their spatial windows.

Disjoint subspaces for common and distinct component analysis: Application to the fusion of multi-task FMRI data

BACKGROUND

Data-driven methods such as independent component analysis (ICA) makes very few assumptions on the data and the relationships of multiple datasets, and hence, are attractive for the fusion of medical imaging data. Two important extensions of ICA for multiset fusion are the joint ICA (jICA) and the multiset canonical correlation analysis and joint ICA (MCCA-jICA) techniques. Both approaches assume identical mixing matrices, emphasizing components that are common across the multiple datasets. However, in general, one would expect to have components that are both common across the datasets and distinct to each dataset.

NEW METHOD

We propose a general framework, disjoint subspace analysis using ICA (DS-ICA), which identifies and extracts not only the common but also the distinct components across multiple datasets. A key component of the method is the identification of these subspaces and their separation before subsequent analyses, which helps establish better model match and provides flexibility in algorithm and order choice.

COMPARISON

We compare DS-ICA with jICA and MCCA-jICA through both simulations and application to multiset functional magnetic resonance imaging (fMRI) task data collected from healthy controls as well as patients with schizophrenia.

RESULTS

The results show DS-ICA estimates more components discriminative between healthy controls and patients than jICA and MCCA-jICA, and with higher discriminatory power showing activation differences in meaningful regions. When applied to a classification framework, components estimated by DS-ICA results in higher classification performance for different dataset combinations than the other two methods.

CONCLUSION

These results demonstrate that DS-ICA is an effective method for fusion of multiple datasets.

Neural dynamics of cue reliability in perceptual decisions

To extract meaningful information from scenes, the visual system must combine local cues that can vary greatly in their degree of reliability. Here, we asked whether cue reliability mostly affects visual or decision-related processes, using visual evoked potentials (VEPs) and a model-based approach to identify when and where stimulus-evoked brain activity reflects cue reliability. Participants performed a shape discrimination task on Gaborized ellipses, while we parametrically and independently, varied the reliability of contour or surface cues. We modeled the expected behavioral performance as a linear function of cue reliability and established at what latencies and electrodes VEP activity reflected behavioral sensitivity to cue reliability. We found that VEPs were linearly related to the individual behavioral predictors at around 400 ms post-stimulus, at electrodes over parietal and lateral temporal cortex. The observed cue reliability effects were similar for variations in contour and surface cues. Notably, effects of cue reliability were absent at earlier latencies where visual shape information is typically reported, and also in data time-locked to the behavioral response, suggesting the effects are not decision-related. These results indicate that reliability of visual cues is reflected in late distributed perceptual processes.

Ventral stream hierarchy underlying perceptual organization in adolescents with autism

Object recognition relies on a hierarchically organized ventral visual stream, with both bottom-up and top-down processes. Here, we aimed at investigating the neural underpinnings of perceptual organization along the ventral visual stream in Autism Spectrum Disorders (ASD), and at determining whether this would be associated with decreased top-down processing in ASD. Nineteen typically developing (TD) adolescents and sixteen adolescents with ASD participated in an fMRI study where they had to detect visual objects. Five conditions displayed Gabor patterns (defined by texture and/or contour) with increasing levels of perceptual organization. In each condition, both groups showed similar abilities. In line with the expected cortical hierarchy, brain activity patterns revealed a progressive involvement of regions, from low-level occipital regions to higher-level frontal regions, when stimuli became more and more organized. The brain patterns were generally similar in both groups, but the ASD group showed greater activation than TD participants in the middle occipital gyrus and lateral occipital complex when perceiving fully organized everyday objects. Effective connectivity analyses suggested that top-down functional connections between the lower levels of the cortical hierarchy were less influenced by the meaning carried by the stimuli in the ASD group than in the TD group. We hypothesize that adolescents with ASD may have been less influenced by top-down processing when perceiving recognizable objects.

Learning Nonclassical Receptive Field Modulation for Contour Detection

This work develops a biologically inspired neural network for contour detection in natural images by combining the nonclassical receptive field modulation mechanism with a deep learning framework. The input image is first convolved with the local feature detectors to produce the classical receptive field responses, and then a corresponding modulatory kernel is constructed for each feature map to model the nonclassical receptive field modulation behaviors. The modulatory effects can activate a larger cortical area and thus allow cortical neurons to integrate a broader range of visual information to recognize complex cases. Additionally, to characterize spatial structures at various scales, a multiresolution technique is used to represent visual field information from fine to coarse. Different scale responses are combined to estimate the contour probability. Our method achieves state-of-the-art results among all biologically inspired contour detection models. This study provides a method for improving visual modeling of contour detection and inspires new ideas for integrating more brain cognitive mechanisms into deep neural networks.

Recurrent Processing of Contour Integration in the Human Visual Cortex as Revealed By fMRI-Guided TMS

Contour integration is a critical step in visual perception because it groups discretely local elements into perceptually global contours. Previous investigations have suggested that striate and extrastriate visual areas are involved in this mid-level processing of visual perception. However, the temporal dynamics of these areas in the human brain during contour integration is less understood. The present study used functional magnetic resonance imaging-guided transcranial magnetic stimulation (TMS) to briefly disrupt 1 of 2 visual areas (V1/V2 and V3B) and examined the causal contributions of these areas to contour detection. The results demonstrated that the earliest critical time window at which behavioral detection performance was impaired by TMS pluses differed between V1/V2 and V3B. The first critical window of V3B (90-110 ms after stimulus onset) was earlier than that of V1/V2 (120-140 ms after stimulus onset), thus indicating that feedback connection from higher to lower area was necessary for complete contour integration. These results suggested that the fine processing of contour-related information in V1/V2 follows the generation of a coarse template in the higher visual areas, such as V3B. Our findings provide direct causal evidence that a recurrent mechanism is necessary for the integration of contours from cluttered background in the human brain.

Moving Beyond ERP Components: A Selective Review of Approaches to Integrate EEG and Behavior

Relationships between neuroimaging measures and behavior provide important clues about brain function and cognition in healthy and clinical populations. While electroencephalography (EEG) provides a portable, low cost measure of brain dynamics, it has been somewhat underrepresented in the emerging field of model-based inference. We seek to address this gap in this article by highlighting the utility of linking EEG and behavior, with an emphasis on approaches for EEG analysis that move beyond focusing on peaks or “components” derived from averaging EEG responses across trials and subjects (generating the event-related potential, ERP). First, we review methods for deriving features from EEG in order to enhance the signal within single-trials. These methods include filtering based on user-defined features (i.e., frequency decomposition, time-frequency decomposition), filtering based on data-driven properties (i.e., blind source separation, BSS), and generating more abstract representations of data (e.g., using deep learning). We then review cognitive models which extract latent variables from experimental tasks, including the drift diffusion model (DDM) and reinforcement learning (RL) approaches. Next, we discuss ways to access associations among these measures, including statistical models, data-driven joint models and cognitive joint modeling using hierarchical Bayesian models (HBMs). We think that these methodological tools are likely to contribute to theoretical advancements, and will help inform our understandings of brain dynamics that contribute to moment-to-moment cognitive function.

Perceptual integration rapidly activates dorsal visual pathway to guide local processing in early visual areas

Rapidly grouping local elements into an organized object (i.e., perceptual integration) is a fundamental yet challenging task, especially in noisy contexts. Previous studies demonstrate that ventral visual pathway, which is widely known to mediate object recognition, engages in the process by conveying object-level information processed in high-level areas to modulate low-level sensory areas. Meanwhile, recent evidence suggests that the dorsal visual pathway, which is not typically attributable to object recognition, is also involved in the process. However, the underlying whole-brain fine spatiotemporal neuronal dynamics remains unknown. Here we used magnetoencephalography (MEG) recordings in combination with a temporal response function (TRF) approach to dissociate the time-resolved neuronal response that specifically tracks the perceptual grouping course. We demonstrate that perceptual integration initiates robust and rapid responses along the dorsal visual pathway in a reversed hierarchical manner, faster than the ventral pathway. Specifically, the anterior intraparietal sulcus (IPS) responds first (i.e., within 100 ms), followed by activities backpropagating along the dorsal pathway to early visual areas (EVAs). The IPS activity causally modulates the EVA response, even when the global form information is task-irrelevant. The IPS-to-EVA response profile fails to appear when the global form could not be perceived. Our results support the crucial function of the dorsal visual pathway in perceptual integration, by quickly extracting a coarse global template (i.e., an initial object representation) within first 100 ms to guide subsequent local sensory processing so that the ambiguities in the visual inputs can be efficiently resolved.

How the brain integrates local elements into a global object (i.e., perceptual integration) in noisy contexts constitutes a fundamental yet challenging question in cognitive neuroscience. Here, we recorded brain activity by using magnetoencephalography from human subjects watching glass-pattern stimuli to examine the fine spatiotemporal neuronal responses during perceptual integration. We demonstrate that high-level brain regions initially extract a coarse global form of the inputs, which is then relayed along the dorsal visual pathway in a reversed hierarchical manner to low-level areas to modulate local analysis. This global-to-local modulation mechanism is especially beneficial in noisy environments by rapidly making an “initial guess” to guide detail analysis so that the ambiguities in inputs can be efficiently resolved.

Contour Integration in Dynamic Scenes: Impaired Detection Performance in Extended Presentations

Since scenes in nature are highly dynamic, perception requires an on-going and robust integration of local information into global representations. In vision, contour integration (CI) is one of these tasks, and it is performed by our brain in a seemingly effortless manner. Following the rule of good continuation, oriented line segments are linked into contour percepts, thus supporting important visual computations such as the detection of object boundaries. This process has been studied almost exclusively using static stimuli, raising the question of whether the observed robustness and "pop-out" quality of CI carries over to dynamic scenes. We investigate contour detection in dynamic stimuli where targets appear at random times by Gabor elements aligning themselves to form contours. In briefly presented displays (230 ms), a situation comparable to classical paradigms in CI, performance is about 87%. Surprisingly, we find that detection performance decreases to 67% in extended presentations (about 1.9-3.8 s) for the same target stimuli. In order to observe the same reduction with briefly presented stimuli, presentation time has to be drastically decreased to intervals as short as 50 ms. Cueing a specific contour position or shape helps in partially compensating this deterioration, and only in extended presentations combining a location and a shape cue was more efficient than providing a single cue. Our findings challenge the notion of CI as a mainly stimulus-driven process leading to pop-out percepts, indicating that top-down processes play a much larger role in supporting fundamental integration processes in dynamic scenes than previously thought.

The Effect of Local Orientation Change on the Detection of Contours Defined by Constant Curvature: Psychophysics and Image Statistics

In the present study, we investigated the detection of contours defined by constant curvature and the statistics of curved contours in natural scenes. In Experiment 1, we examined the degree to which human sensitivity to contours is affected by changing the curvature angle and disrupting contour curvature continuity by varying the orientation of end elements. We find that (1) changing the angle of contour curvature decreased detection performance, while (2) end elements oriented in the direction (i.e., clockwise) of curvature facilitated contour detection regardless of the curvature angle of the contour. In Experiment 2 we further established that the relative effect of end—element orientation on contour detection was not only dependent on their orientation (collinear or cocircular), but also their spatial separation from the contour, and whether the contour shape was curved or not (i.e., C-shaped or S-shaped). Increasing the spatial separation of end-elements reduced contour detection performance regardless of their orientation or the contour shape. However, at small separations, cocircular end-elements facilitated the detection of C-shaped contours, but not S-shaped contours. The opposite result was observed for collinear end-elements, which improved the detection of S- shaped, but not C-shaped contours. These dissociative results confirmed that the visual system specifically codes contour curvature, but the association of contour elements occurs locally. Finally, we undertook an analysis of natural images that mapped contours with a constant angular change and determined the frequency of occurrence of end elements with different orientations. Analogous to our behavioral data, this image analysis revealed that the mapped end elements of constantly curved contours are likely to be oriented clockwise to the angle of curvature. Our findings indicate that the visual system is selectively sensitive to contours defined by constant curvature and that this might reflect the properties of curved contours in natural images.

Early suppression effect in human primary visual cortex during Kanizsa illusion processing: A magnetoencephalographic evidence

Detection of illusory contours (ICs) such as Kanizsa figures is known to depend primarily upon the lateral occipital complex. Yet there is no universal agreement on the role of the primary visual cortex in this process; some existing evidence hints that an early stage of the visual response in V1 may involve relative suppression to Kanizsa figures compared with controls. Iso-oriented luminance borders, which are responsible for Kanizsa illusion, may evoke surround suppression in V1 and adjacent areas leading to the reduction in the initial response to Kanizsa figures. We attempted to test the existence, as well as to find localization and timing of the early suppression effect produced by Kanizsa figures in adult nonclinical human participants. We used two sizes of visual stimuli (4.5 and 9.0°) in order to probe the effect at two different levels of eccentricity; the stimuli were presented centrally in passive viewing conditions. We recorded magnetoencephalogram, which is more sensitive than electroencephalogram to activity originating from V1 and V2 areas. We restricted our analysis to the medial occipital area and the occipital pole, and to a 40-120 ms time window after the stimulus onset. By applying threshold-free cluster enhancement technique in combination with permutation statistics, we were able to detect the inverted IC effect-a relative suppression of the response to the Kanizsa figures compared with the control stimuli. The current finding is highly compatible with the explanation involving surround suppression evoked by iso-oriented collinear borders. The effect may be related to the principle of sparse coding, according to which V1 suppresses representations of inner parts of collinear assemblies as being informationally redundant. Such a mechanism is likely to be an important preliminary step preceding object contour detection.

Multi-modal data fusion using source separation: Two effective models based on ICA and IVA and their properties

Fusion of information from multiple sets of data in order to extract a set of features that are most useful and relevant for the given task is inherent to many problems we deal with today. Since, usually, very little is known about the actual interaction among the datasets, it is highly desirable to minimize the underlying assumptions. This has been the main reason for the growing importance of data-driven methods, and in particular of independent component analysis (ICA) as it provides useful decompositions with a simple generative model and using only the assumption of statistical independence. A recent extension of ICA, independent vector analysis (IVA) generalizes ICA to multiple datasets by exploiting the statistical dependence across the datasets, and hence, as we discuss in this paper, provides an attractive solution to fusion of data from multiple datasets along with ICA. In this paper, we focus on two multivariate solutions for multi-modal data fusion that let multiple modalities fully interact for the estimation of underlying features that jointly report on all modalities. One solution is the Joint ICA model that has found wide application in medical imaging, and the second one is the the Transposed IVA model introduced here as a generalization of an approach based on multi-set canonical correlation analysis. In the discussion, we emphasize the role of diversity in the decompositions achieved by these two models, present their properties and implementation details to enable the user make informed decisions on the selection of a model along with its associated parameters. Discussions are supported by simulation results to help highlight the main issues in the implementation of these methods.

Effects of Spatial Frequency Similarity and Dissimilarity on Contour Integration

We examined the effects of spatial frequency similarity and dissimilarity on human contour integration under various conditions of uncertainty. Participants performed a temporal 2AFC contour detection task. Spatial frequency jitter up to 3.0 octaves was applied either to background elements, or to contour and background elements, or to none of both. Results converge on four major findings. (1) Contours defined by spatial frequency similarity alone are only scarcely visible, suggesting the absence of specialized cortical routines for shape detection based on spatial frequency similarity. (2) When orientation collinearity and spatial frequency similarity are combined along a contour, performance amplifies far beyond probability summation when compared to the fully heterogenous condition but only to a margin compatible with probability summation when compared to the fully homogenous case. (3) Psychometric functions are steeper but not shifted for homogenous contours in heterogenous backgrounds indicating an advantageous signal-to-noise ratio. The additional similarity cue therefore not so much improves contour detection performance but primarily reduces observer uncertainty about whether a potential candidate is a contour or just a false positive. (4) Contour integration is a broadband mechanism which is only moderately impaired by spatial frequency dissimilarity.

Mishaps, errors, and cognitive experiences: on the conceptualization of perceptual illusions

Although a visual illusion is often viewed as an amusing trick, for the vision scientist it is a question that demands an answer, which leads to even more questioning. All researchers hold their own chain of questions, the links of which depend on the very theory they adhere to. Perceptual theories are devoted to answering questions concerning sensation and perception, but in doing so they shape concepts such as reality and representation, which necessarily affect the concept of illusion. Here we consider the macroscopic aspects of such concepts in vision sciences from three classic viewpoints—Ecological, Cognitive, Gestalt approaches—as we see this a starting point to understand in which terms illusions can become a tool in the hand of the neuroscientist. In fact, illusions can be effective tools in studying the brain in reference to perception and also to cognition in a much broader sense. A theoretical debate is, however, mandatory, in particular with regards to concepts such as veridicality and representation. Whether a perceptual outcome is considered as veridical or illusory (and, consequently, whether a class of phenomena should be classified as perceptual illusions or not) depends on the meaning of such concepts.

Ensemble Empirical Mode Decomposition Analysis of EEG Data Collected during a Contour Integration Task

We discuss a data-driven analysis of EEG data recorded during a combined EEG/fMRI study of visual processing during a contour integration task. The analysis is based on an ensemble empirical mode decomposition (EEMD) and discusses characteristic features of event related modes (ERMs) resulting from the decomposition. We identify clear differences in certain ERMs in response to contour vs noncontour Gabor stimuli mainly for response amplitudes peaking around 100 [ms] (called P100) and 200 [ms] (called N200) after stimulus onset, respectively. We observe early P100 and N200 responses at electrodes located in the occipital area of the brain, while late P100 and N200 responses appear at electrodes located in frontal brain areas. Signals at electrodes in central brain areas show bimodal early/late response signatures in certain ERMs. Head topographies clearly localize statistically significant response differences to both stimulus conditions. Our findings provide an independent proof of recent models which suggest that contour integration depends on distributed network activity within the brain.

Collinear facilitation and contour integration in autism: evidence for atypical visual integration

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by impaired social interaction, atypical communication and a restricted repertoire of interests and activities. Altered sensory and perceptual experiences are also common, and a notable perceptual difference between individuals with ASD and controls is their superior performance in visual tasks where it may be beneficial to ignore global context. This superiority may be the result of atypical integrative processing. To explore this claim we investigated visual integration in adults with ASD (diagnosed with Asperger's Syndrome) using two psychophysical tasks thought to rely on integrative processing-collinear facilitation and contour integration. We measured collinear facilitation at different flanker orientation offsets and contour integration for both open and closed contours. Our results indicate that compared to matched controls, ASD participants show (i) reduced collinear facilitation, despite equivalent performance without flankers; and (ii) less benefit from closed contours in contour integration. These results indicate weaker visuospatial integration in adults with ASD and suggest that further studies using these types of paradigms would provide knowledge on how contextual processing is altered in ASD.