Multimodal deep neural decoding reveals highly resolved spatiotemporal profile of visual object representation in humans

Contribution of low-level image statistics to EEG decoding of semantic content in multivariate and univariate models with feature optimization

Spatio-temporal patterns of evoked brain activity contain information that can be used to decode and categorize the semantic content of visual stimuli. However, this procedure can be biased by low-level image features independently of the semantic content present in the stimuli, prompting the need to understand the robustness of different models regarding these confounding factors. In this study, we trained machine learning models to distinguish between concepts included in the publicly available THINGS-EEG dataset using electroencephalography (EEG) data acquired during a rapid serial visual presentation paradigm. We investigated the contribution of low-level image features to decoding accuracy in a multivariate model, utilizing broadband data from all EEG channels. Additionally, we explored a univariate model obtained through data-driven feature selection applied to the spatial and frequency domains. While the univariate models exhibited better decoding accuracy, their predictions were less robust to the confounding effect of low-level image statistics. Notably, some of the models maintained their accuracy even after random replacement of the training dataset with semantically unrelated samples that presented similar low-level content. In conclusion, our findings suggest that model optimization impacts sensitivity to confounding factors, regardless of the resulting classification performance. Therefore, the choice of EEG features for semantic decoding should ideally be informed by criteria beyond classifier performance, such as the neurobiological mechanisms under study.

Stratification of tumour cell radiation response and metabolic signatures visualization with Raman spectroscopy and explainable convolutional neural network

Reprogramming of cellular metabolism is a driving factor of tumour progression and radiation therapy resistance. Identifying biochemical signatures associated with tumour radioresistance may assist with the development of targeted treatment strategies to improve clinical outcomes. Raman spectroscopy (RS) can monitor post-irradiation biomolecular changes and signatures of radiation response in tumour cells in a label-free manner. Convolutional Neural Networks (CNN) perform feature extraction directly from data in an end-to-end learning manner, with high classification performance. Furthermore, recently developed CNN explainability techniques help visualize the critical discriminative features captured by the model. In this work, a CNN is developed to characterize tumour response to radiotherapy based on its degree of radioresistance. The model was trained to classify Raman spectra of three human tumour cell lines as radiosensitive (LNCaP) or radioresistant (MCF7, H460) over a range of treatment doses and data collection time points. Additionally, a method based on Gradient-Weighted Class Activation Mapping (Grad-CAM) was used to determine response-specific salient Raman peaks influencing the CNN predictions. The CNN effectively classified the cell spectra, with accuracy, sensitivity, specificity, and F1 score exceeding 99.8%. Grad-CAM heatmaps of H460 and MCF7 cell spectra (radioresistant) exhibited high contributions from Raman bands tentatively assigned to glycogen, amino acids, and nucleic acids. Conversely, heatmaps of LNCaP cells (radiosensitive) revealed activations at lipid and phospholipid bands. Finally, Grad-CAM variable importance scores were derived for glycogen, asparagine, and phosphatidylcholine, and we show that their trends over cell line, dose, and acquisition time agreed with previously established models. Thus, the CNN can accurately detect biomolecular differences in the Raman spectra of tumour cells of varying radiosensitivity without requiring manual feature extraction. Finally, Grad-CAM may help identify metabolic signatures associated with the observed categories, offering the potential for automated clinical tumour radiation response characterization.

Neural mechanisms of attentional bias to emotional faces in patients with chronic insomnia disorder

OBJECTIVE

This study used event-related potential (ERP) and resting-state functional connectivity (rs-FC) approaches to investigate the neural mechanisms underlying the emotional attention bias in patients with chronic insomnia disorder (CID).

METHODS

Twenty-five patients with CID and thirty-three demographically matched healthy controls (HCs) completed clinical questionnaires and underwent electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) scans. EEG analysis examined the group differences in terms of reaction times, P3 amplitudes, event-related spectral perturbations, and inter-trial phase synchrony. Subsequently, seed-based rs-FC analysis of the amygdala nuclei (including the central-medial amygdala [CMA] and basolateral amygdala [BLA]) was performed. The relationship between P3 amplitude, rs-FC and clinical symptom severity in patients with CID was further investigated by correlation analysis.

RESULTS

CID patients exhibited shorter reaction times than HCs in both standard and deviant stimuli, with the abnormalities becoming more pronounced as attention allocation increased. Compared to HCs, ERP analysis revealed increased P3 amplitude, theta wave power, and inter-trial synchrony in CID patients. The rs-FC analysis showed increased connectivity of the BLA-occipital pole, CMA-precuneus, and CMA-angular gyrus and decreased connectivity of the CMA-thalamus in CID patients. Notably, correlation analysis of the EEG and fMRI measurements showed a significant positive correlation between the P3 amplitude and the rs-FC of the CMA-PCU.

CONCLUSION

This study confirms an emotional attention bias in CID, specifically in the neural mechanisms of attention processing that vary depending on the allocation of attentional resources. Abnormal connectivity in the emotion-cognition networks may constitute the neural basis of the abnormal scalp activation pattern.