Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks

Brain-machine convergent evolution: Why finding parallels between brain and artificial systems is informative

Central nervous system neurons manifest a rich diversity of selectivity profiles-whose precise role is still poorly understood. Following the striking success of artificial networks, a major debate has emerged concerning their usefulness in explaining neuronal properties. Here we propose that finding parallels between artificial and neuronal networks is informative precisely because these systems are so different from each other. Our argument is based on an extension of the concept of convergent evolution-well established in biology-to the domain of artificial systems. Applying this concept to different areas and levels of the cortical hierarchy can be a powerful tool for elucidating the functional role of well-known cortical selectivities. Importantly, we further demonstrate that such parallels can uncover novel functionalities by showing that grid cells in the entorhinal cortex can be modeled to function as a set of basis functions in a lossy representation such as the well-known JPEG compression. Thus, contrary to common intuition, here we illustrate that finding parallels with artificial systems provides novel and informative insights, particularly in those cases that are far removed from realistic brain biology.

A shared model-based linguistic space for transmitting our thoughts from brain to brain in natural conversations

Effective communication hinges on a mutual understanding of word meaning in different contexts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We developed a model-based coupling framework that aligns brain activity in both speaker and listener to a shared embedding space from a large language model (LLM). The context-sensitive LLM embeddings allow us to track the exchange of linguistic information, word by word, from one brain to another in natural conversations. Linguistic content emerges in the speaker's brain before word articulation and rapidly re-emerges in the listener's brain after word articulation. The contextual embeddings better capture word-by-word neural alignment between speaker and listener than syntactic and articulatory models. Our findings indicate that the contextual embeddings learned by LLMs can serve as an explicit numerical model of the shared, context-rich meaning space humans use to communicate their thoughts to one another.

Using recurrent neural network to estimate irreducible stochasticity in human choice behavior

Theoretical computational models are widely used to describe latent cognitive processes. However, these models do not equally explain data across participants, with some individuals showing a bigger predictive gap than others. In the current study, we examined the use of theory-independent models, specifically recurrent neural networks (RNNs), to classify the source of a predictive gap in the observed data of a single individual. This approach aims to identify whether the low predictability of behavioral data is mainly due to noisy decision-making or misspecification of the theoretical model. First, we used computer simulation in the context of reinforcement learning to demonstrate that RNNs can be used to identify model misspecification in simulated agents with varying degrees of behavioral noise. Specifically, both prediction performance and the number of RNN training epochs (i.e., the point of early stopping) can be used to estimate the amount of stochasticity in the data. Second, we applied our approach to an empirical dataset where the actions of low IQ participants, compared with high IQ participants, showed lower predictability by a well-known theoretical model (i.e., Daw's hybrid model for the two-step task). Both the predictive gap and the point of early stopping of the RNN suggested that model misspecification is similar across individuals. This led us to a provisional conclusion that low IQ subjects are mostly noisier compared to their high IQ peers, rather than being more misspecified by the theoretical model. We discuss the implications and limitations of this approach, considering the growing literature in both theoretical and data-driven computational modeling in decision-making science.

Digital Twin Studies for Reverse Engineering the Origins of Visual Intelligence

What are the core learning algorithms in brains? Nativists propose that intelligence emerges from innate domain-specific knowledge systems, whereas empiricists propose that intelligence emerges from domain-general systems that learn domain-specific knowledge from experience. We address this debate by reviewing digital twin studies designed to reverse engineer the learning algorithms in newborn brains. In digital twin studies, newborn animals and artificial agents are raised in the same environments and tested with the same tasks, permitting direct comparison of their learning abilities. Supporting empiricism, digital twin studies show that domain-general algorithms learn animal-like object perception when trained on the first-person visual experiences of newborn animals. Supporting nativism, digital twin studies show that domain-general algorithms produce innate domain-specific knowledge when trained on prenatal experiences (retinal waves). We argue that learning across humans, animals, and machines can be explained by a universal principle, which we call space-time fitting. Space-time fitting explains both empiricist and nativist phenomena, providing a unified framework for understanding the origins of intelligence.

The motifs of radical embodied neuroscience

In this paper, I analyse how the emerging scientific framework of radical embodied neuroscience is different from contemporary mainstream cognitive neuroscience. To do so, I propose the notion of motif to enrich the philosophical toolkit of cognitive neuroscience. This notion can be used to characterize the guiding ideas of any given scientific framework in psychology and neuroscience. Motifs are highly unconstrained, open-ended concepts that support equally open-ended families of explanations. Different scientific frameworks-e.g., psychophysics or cognitive neuroscience-provide these motifs to answer the overarching themes of these disciplines, such as the relationship between stimuli and sensations or the proper methods of the sciences of the mind. Some motifs of mainstream cognitive neuroscience are the motif of encoding, the motif of input-output systems, and the motif of algorithms. The two first ones answer the question about the relationship between stimuli, sensations and experience (e.g., stimuli are input and are encoded by brain structures). The latter one answers the question regarding the mechanism of cognition and experience. The three of them are equally unconstrained and open-ended, and they serve as an umbrella for different kinds of explanation-i.e., different positions regarding what counts as a code or as an input. Along with the articulation of the notion of motif, the main aim of this article is to present three motifs for radical embodied neuroscience: the motif of complex stimulation, the motif of organic behaviour and the motif of resonance.

Optimization of the extraction process of total steroids from Phillinus gilvus (Schwein.) Pat. by artificial neural network (ANN)-response surface methodology and identification of extract constituents

Phillinus gilvus (Schwein.) Pat has pharmacological effects such as tonifying the spleen, dispelling dampness, and strengthening the stomach, in which sterol is one of the main compounds of P. gilvus, but there has not been thought you to its extraction and detailed identification of its composition, in the present study, we used artificial neural network (ANN) and response surface methodology (RSM) to optimize the conditions of ultrasonic-assisted extraction, and the parameters of the independent and interaction effects were evaluated. Ultra performance liquid chromatography-quadrupole-time of flight mass spectrometry (UPLC-Q-TOF-MS/MS) was used to identify the major components in the purified extract. The results showed that the optimal extraction process conditions were: ultrasonic time 96 min, ultrasonic power 140 W, liquid to material ratio 1:25 g/ml, and ultrasonic temperature 30.7 °C. The compliance rates of the predicted and experimental values for the artificial neural network model and the response surface model were 98.3% and 96.12%, respectively, indicating that both models have the potential to be used for optimizing the extraction process of P. gilvus in industry. A total of 120 compounds and 30 major steroids were identified by comparison with the reference compounds. Among the major steroidal components are these findings will contribute to the isolation and utilization of active ingredients in P. gilvus.

Scale matters: Large language models with billions (rather than millions) of parameters better match neural representations of natural language

Recent research has used large language models (LLMs) to study the neural basis of naturalistic language processing in the human brain. LLMs have rapidly grown in complexity, leading to improved language processing capabilities. However, neuroscience researchers haven't kept up with the quick progress in LLM development. Here, we utilized several families of transformer-based LLMs to investigate the relationship between model size and their ability to capture linguistic information in the human brain. Crucially, a subset of LLMs were trained on a fixed training set, enabling us to dissociate model size from architecture and training set size. We used electrocorticography (ECoG) to measure neural activity in epilepsy patients while they listened to a 30-minute naturalistic audio story. We fit electrode-wise encoding models using contextual embeddings extracted from each hidden layer of the LLMs to predict word-level neural signals. In line with prior work, we found that larger LLMs better capture the structure of natural language and better predict neural activity. We also found a log-linear relationship where the encoding performance peaks in relatively earlier layers as model size increases. We also observed variations in the best-performing layer across different brain regions, corresponding to an organized language processing hierarchy.

Model Virtues in Computational Cognitive Neuroscience

There is an abundance of computational models in cognitive neuroscience. A framework for what is desirable in a model, what justifies the introduction of a new one, or what makes one better than another is lacking, however. In this article, we examine key qualities ("virtues") that are desirable in computational models, and how these are interrelated. To keep the scope of the article manageable, we focus on the field of cognitive control, where we identified six "model virtues": empirical accuracy, empirical scope, functional analysis, causal detail, biological plausibility, and psychological plausibility. We first illustrate their use in published work on Stroop modeling and then discuss what expert modelers in the field of cognitive control said about them in a series of qualitative interviews. We found that virtues are interrelated and that their value depends on the modeler's goals, in ways that are not typically acknowledged in the literature. We recommend that researchers make the reasons for their modeling choices more explicit in published work. Our work is meant as a first step. Although our focus here is on cognitive control, we hope that our findings will spark discussion of virtues in other fields as well.

Shared functional specialization in transformer-based language models and the human brain

When processing language, the brain is thought to deploy specialized computations to construct meaning from complex linguistic structures. Recently, artificial neural networks based on the Transformer architecture have revolutionized the field of natural language processing. Transformers integrate contextual information across words via structured circuit computations. Prior work has focused on the internal representations ("embeddings") generated by these circuits. In this paper, we instead analyze the circuit computations directly: we deconstruct these computations into the functionally-specialized "transformations" that integrate contextual information across words. Using functional MRI data acquired while participants listened to naturalistic stories, we first verify that the transformations account for considerable variance in brain activity across the cortical language network. We then demonstrate that the emergent computations performed by individual, functionally-specialized "attention heads" differentially predict brain activity in specific cortical regions. These heads fall along gradients corresponding to different layers and context lengths in a low-dimensional cortical space.

Predicting the next sentence (not word) in large language models: What model-brain alignment tells us about discourse comprehension

Current large language models (LLMs) rely on word prediction as their backbone pretraining task. Although word prediction is an important mechanism underlying language processing, human language comprehension occurs at multiple levels, involving the integration of words and sentences to achieve a full understanding of discourse. This study models language comprehension by using the next sentence prediction (NSP) task to investigate mechanisms of discourse-level comprehension. We show that NSP pretraining enhanced a model's alignment with brain data especially in the right hemisphere and in the multiple demand network, highlighting the contributions of nonclassical language regions to high-level language understanding. Our results also suggest that NSP can enable the model to better capture human comprehension performance and to better encode contextual information. Our study demonstrates that the inclusion of diverse learning objectives in a model leads to more human-like representations, and investigating the neurocognitive plausibility of pretraining tasks in LLMs can shed light on outstanding questions in language neuroscience.

Taking consciousness for real: Increasing the ecological validity of the study of conscious vs. unconscious processes

The study of consciousness has developed well-controlled, rigorous methods for manipulating and measuring consciousness. Yet, in the process, experimental paradigms grew farther away from everyday conscious and unconscious processes, which raises the concern of ecological validity. In this review, we suggest that the field can benefit from adopting a more ecological approach, akin to other fields of cognitive science. There, this approach challenged some existing hypotheses, yielded stronger effects, and enabled new research questions. We argue that such a move is critical for studying consciousness, where experimental paradigms tend to be artificial and small effect sizes are relatively prevalent. We identify three paths for doing so-changing the stimuli and experimental settings, changing the measures, and changing the research questions themselves-and review works that have already started implementing such approaches. While acknowledging the inherent challenges, we call for increasing ecological validity in consciousness studies.

Multiple images captured from a single encounter do not promote face learning

Viewing multiple images of a newly encountered face improves recognition of that identity in new instances. Studies examining face learning have presented high-variability (HV) images that incorporate changes that occur from moment-to-moment (e.g., head orientation and expression) and over time (e.g., lighting, hairstyle, and health). We examined whether low-variability (LV) images (i.e., images that incorporate only moment-to-moment changes) also promote generalisation of learning such that novel instances are recognised. Participants viewed a single image, six LV images, or six HV images of a target identity before being asked to recognise novel images of that identity in a face matching task (training stimuli remained visible) or a memory task (training stimuli were removed). In Experiment 1 (n = 71), participants indicated which image(s) in 8-image arrays belonged to the target identity. In Experiment 2 (n = 73), participants indicated whether sequentially presented images belonged to the target identity. Relative to the single-image condition, sensitivity to identity improved and response biases were less conservative in the HV condition; we found no evidence of generalisation of learning in the LV condition regardless of testing protocol. Our findings suggest that day-to-day variability in appearance plays an essential role in acquiring expertise with a novel face.

Modular-integrative modeling: a new framework for building brain models that blend biological realism and functional performance

This Perspective presents the Modular-Integrative Modeling approach, a novel framework in neuroscience for developing brain models that blend biological realism with functional performance to provide a holistic view on brain function in interaction with the body and environment.

Neurobiological Causal Models of Language Processing

The language faculty is physically realized in the neurobiological infrastructure of the human brain. Despite significant efforts, an integrated understanding of this system remains a formidable challenge. What is missing from most theoretical accounts is a specification of the neural mechanisms that implement language function. Computational models that have been put forward generally lack an explicit neurobiological foundation. We propose a neurobiologically informed causal modeling approach which offers a framework for how to bridge this gap. A neurobiological causal model is a mechanistic description of language processing that is grounded in, and constrained by, the characteristics of the neurobiological substrate. It intends to model the generators of language behavior at the level of implementational causality. We describe key features and neurobiological component parts from which causal models can be built and provide guidelines on how to implement them in model simulations. Then we outline how this approach can shed new light on the core computational machinery for language, the long-term storage of words in the mental lexicon and combinatorial processing in sentence comprehension. In contrast to cognitive theories of behavior, causal models are formulated in the "machine language" of neurobiology which is universal to human cognition. We argue that neurobiological causal modeling should be pursued in addition to existing approaches. Eventually, this approach will allow us to develop an explicit computational neurobiology of language.

Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns

Contextual embeddings, derived from deep language models (DLMs), provide a continuous vectorial representation of language. This embedding space differs fundamentally from the symbolic representations posited by traditional psycholinguistics. We hypothesize that language areas in the human brain, similar to DLMs, rely on a continuous embedding space to represent language. To test this hypothesis, we densely record the neural activity patterns in the inferior frontal gyrus (IFG) of three participants using dense intracranial arrays while they listened to a 30-minute podcast. From these fine-grained spatiotemporal neural recordings, we derive a continuous vectorial representation for each word (i.e., a brain embedding) in each patient. Using stringent zero-shot mapping we demonstrate that brain embeddings in the IFG and the DLM contextual embedding space have common geometric patterns. The common geometric patterns allow us to predict the brain embedding in IFG of a given left-out word based solely on its geometrical relationship to other non-overlapping words in the podcast. Furthermore, we show that contextual embeddings capture the geometry of IFG embeddings better than static word embeddings. The continuous brain embedding space exposes a vector-based neural code for natural language processing in the human brain.

Effectiveness of a Biofeedback Intervention Targeting Mental and Physical Health Among College Students Through Speech and Physiology as Biomarkers Using Machine Learning: A Randomized Controlled Trial

Biofeedback therapy is mainly based on the analysis of physiological features to improve an individual's affective state. There are insufficient objective indicators to assess symptom improvement after biofeedback. In addition to psychological and physiological features, speech features can precisely convey information about emotions. The use of speech features can improve the objectivity of psychiatric assessments. Therefore, biofeedback based on subjective symptom scales, objective speech, and physiological features to evaluate efficacy provides a new approach for early screening and treatment of emotional problems in college students. A 4-week, randomized, controlled, parallel biofeedback therapy study was conducted with college students with symptoms of anxiety or depression. Speech samples, physiological samples, and clinical symptoms were collected at baseline and at the end of treatment, and the extracted speech features and physiological features were used for between-group comparisons and correlation analyses between the biofeedback and wait-list groups. Based on the speech features with differences between the biofeedback intervention and wait-list groups, an artificial neural network was used to predict the therapeutic effect and response after biofeedback therapy. Through biofeedback therapy, improvements in depression (p = 0.001), anxiety (p = 0.001), insomnia (p = 0.013), and stress (p = 0.004) severity were observed in college-going students (n = 52). The speech and physiological features in the biofeedback group also changed significantly compared to the waitlist group (n = 52) and were related to the change in symptoms. The energy parameters and Mel-Frequency Cepstral Coefficients (MFCC) of speech features can predict whether biofeedback intervention effectively improves anxiety and insomnia symptoms and treatment response. The accuracy of the classification model built using the artificial neural network (ANN) for treatment response and non-response was approximately 60%. The results of this study provide valuable information about biofeedback in improving the mental health of college-going students. The study identified speech features, such as the energy parameters, and MFCC as more accurate and objective indicators for tracking biofeedback therapy response and predicting efficacy. Trial Registration ClinicalTrials.gov ChiCTR2100045542.

Deep social neuroscience: the promise and peril of using artificial neural networks to study the social brain

This review offers an accessible primer to social neuroscientists interested in neural networks. It begins by providing an overview of key concepts in deep learning. It then discusses three ways neural networks can be useful to social neuroscientists: (i) building statistical models to predict behavior from brain activity; (ii) quantifying naturalistic stimuli and social interactions; and (iii) generating cognitive models of social brain function. These applications have the potential to enhance the clinical value of neuroimaging and improve the generalizability of social neuroscience research. We also discuss the significant practical challenges, theoretical limitations and ethical issues faced by deep learning. If the field can successfully navigate these hazards, we believe that artificial neural networks may prove indispensable for the next stage of the field's development: deep social neuroscience.

Using deep neural networks to disentangle visual and semantic information in human perception and memory

Mental representations of familiar categories are composed of visual and semantic information. Disentangling the contributions of visual and semantic information in humans is challenging because they are intermixed in mental representations. Deep neural networks that are trained either on images or on text or by pairing images and text enable us now to disentangle human mental representations into their visual, visual-semantic and semantic components. Here we used these deep neural networks to uncover the content of human mental representations of familiar faces and objects when they are viewed or recalled from memory. The results show a larger visual than semantic contribution when images are viewed and a reversed pattern when they are recalled. We further reveal a previously unknown unique contribution of an integrated visual-semantic representation in both perception and memory. We propose a new framework in which visual and semantic information contribute independently and interactively to mental representations in perception and memory.

Interpretable deep learning model for major depressive disorder assessment based on functional near-infrared spectroscopy

BACKGROUND

Major depressive disorder (MDD) affects a substantial number of individuals worldwide. New approaches are required to improve the diagnosis of MDD, which relies heavily on subjective reports of depression-related symptoms.

AIM

Establish an objective measurement and evaluation of MDD.

METHODS

Functional near-infrared spectroscopy (fNIRS) was used to investigate the brain activity of MDD patients and healthy controls (HCs). Leveraging a sizeable fNIRS dataset of 263 HCs and 251 patients with MDD, including mild to moderate MDD (mMDD; n = 139) and severe MDD (sMDD; n = 77), we developed an interpretable deep learning model for screening MDD and staging its severity.

RESULTS

The proposed deep learning model achieved an accuracy of 80.9% in diagnostic classification and 78.6% in severity staging for MDD. We discerned five channels with the most significant contribution to MDD identification through Shapley additive explanations (SHAP), located in the right medial prefrontal cortex, right dorsolateral prefrontal cortex, right superior temporal gyrus, and left posterior superior frontal cortex. The findings corresponded closely to the features of haemoglobin responses between HCs and individuals with MDD, as we obtained a good discriminative ability for MDD using cortical channels that are related to the disorder, namely the frontal and temporal cortical channels with areas under the curve of 0.78 and 0.81, respectively.

CONCLUSION

Our study demonstrated the potential of integrating the fNIRS system with artificial intelligence algorithms to classify and stage MDD in clinical settings using a large dataset. This approach can potentially enhance MDD assessment and provide insights for clinical diagnosis and intervention.

Deep learning and machine learning predictive models for neurological function after interventional embolization of intracranial aneurysms

Objective

The objective of this study is to develop a model to predicts the postoperative Hunt-Hess grade in patients with intracranial aneurysms by integrating radiomics and deep learning technologies, using preoperative CTA imaging data. Thereby assisting clinical decision-making and improving the assessment and prognosis of postoperative neurological function.

Methods

This retrospective study encompassed 101 patients who underwent aneurysm embolization surgery. 851 radiomic features were extracted from CTA images. 512 deep learning features are extracted from last layer of ResNet50 deep convolutional neural network model. The feature screening process pipeline encompassed intraclass correlation coefficient analysis, principal component analysis, U test, spearman correlation analysis, minimum redundancy maximum relevance algorithm and Lasso regression, to identify features most correlated with postoperative Hunt-Hess grading. In the model construction phase, three distinct models were constructed: radiomics feature-based model (RSM), deep learning feature-based model (DLM), and deep learning-radiomics feature fusion model (DLRSCM). The study also calculated the radiomics score and combined it with clinical data to construct a Nomogram for predictive modeling. DLM, RSM and DLRSCM model was constructed by 9 base algorithms and 1 ensemble learning algorithm - Stacking ensemble model. Model performance was evaluated based on the area under the Receiver Operating Characteristic (ROC) curve (AUC), Matthews Correlation Coefficient (MCC), calibration curves, and decision curves analysis.

Results

5 significant radiomic feature and 4 significant deep learning features were obtained through the feature selection process. These features were utilized for model construction. Bootstrap resampling method was used for internal validation of the models. In terms of model evaluation, the DLM model, the stacking ensemble algorithm results achieved an AUC of 0.959 and MCC of 0.815. In the RSM model, the stacking ensemble model AUC was 0.935 and MCC was 0.793. The stacking ensemble model in DLRSCM outperformed others, with an AUC of 0.968 and MCC of 0.820. Results indicated that the ANN performed optimally among all base models, while the stacked ensemble learning model exhibited the highest predictive performance.

Conclusion

This study demonstrates that the combination of radiomics and deep learning is an effective approach to predict the postoperative Hunt-Hess grade in patients with intracranial aneurysms. This holds significant value in the early identification of postoperative neurological complications and in enhancing clinical decision-making.

Deep evolutionary fusion neural network: a new prediction standard for infectious disease incidence rates

BACKGROUND

Previously, many methods have been used to predict the incidence trends of infectious diseases. There are numerous methods for predicting the incidence trends of infectious diseases, and they have exhibited varying degrees of success. However, there are a lack of prediction benchmarks that integrate linear and nonlinear methods and effectively use internet data. The aim of this paper is to develop a prediction model of the incidence rate of infectious diseases that integrates multiple methods and multisource data, realizing ground-breaking research.

RESULTS

The infectious disease dataset is from an official release and includes four national and three regional datasets. The Baidu index platform provides internet data. We choose a single model (seasonal autoregressive integrated moving average (SARIMA), nonlinear autoregressive neural network (NAR), and long short-term memory (LSTM)) and a deep evolutionary fusion neural network (DEFNN). The DEFNN is built using the idea of neural evolution and fusion, and the DEFNN + is built using multisource data. We compare the model accuracy on reference group data and validate the model generalizability on external data. (1) The loss of SA-LSTM in the reference group dataset is 0.4919, which is significantly better than that of other single models. (2) The loss values of SA-LSTM on the national and regional external datasets are 0.9666, 1.2437, 0.2472, 0.7239, 1.4026, and 0.6868. (3) When multisource indices are added to the national dataset, the loss of the DEFNN + increases to 0.4212, 0.8218, 1.0331, and 0.8575.

CONCLUSIONS

We propose an SA-LSTM optimization model with good accuracy and generalizability based on the concept of multiple methods and multiple data fusion. DEFNN enriches and supplements infectious disease prediction methodologies, can serve as a new benchmark for future infectious disease predictions and provides a reference for the prediction of the incidence rates of various infectious diseases.

Functional neuronal circuits emerge in the absence of developmental activity

The complex neuronal circuitry of the brain develops from limited information contained in the genome. After the genetic code instructs the birth of neurons, the emergence of brain regions, and the formation of axon tracts, it is believed that temporally structured spiking activity shapes circuits for behavior. Here, we challenge the learning-dominated assumption that spiking activity is required for circuit formation by quantifying its contribution to the development of visually-guided swimming in the larval zebrafish. We found that visual experience had no effect on the emergence of the optomotor response (OMR) in dark-reared zebrafish. We then raised animals while pharmacologically silencing action potentials with the sodium channel blocker tricaine. After washout of the anesthetic, fish could swim and performed with 75-90% accuracy in the OMR paradigm. Brain-wide imaging confirmed that neuronal circuits came 'online' fully tuned, without requiring activity-dependent plasticity. Thus, complex sensory-guided behaviors can emerge through activity-independent developmental mechanisms.

Are we really Bayesian? Probabilistic inference shows sub-optimal knowledge transfer

Numerous studies have found that the Bayesian framework, which formulates the optimal integration of the knowledge of the world (i.e. prior) and current sensory evidence (i.e. likelihood), captures human behaviours sufficiently well. However, there are debates regarding whether humans use precise but cognitively demanding Bayesian computations for behaviours. Across two studies, we trained participants to estimate hidden locations of a target drawn from priors with different levels of uncertainty. In each trial, scattered dots provided noisy likelihood information about the target location. Participants showed that they learned the priors and combined prior and likelihood information to infer target locations in a Bayes fashion. We then introduced a transfer condition presenting a trained prior and a likelihood that has never been put together during training. How well participants integrate this novel likelihood with their learned prior is an indicator of whether participants perform Bayesian computations. In one study, participants experienced the newly introduced likelihood, which was paired with a different prior, during training. Participants changed likelihood weighting following expected directions although the degrees of change were significantly lower than Bayes-optimal predictions. In another group, the novel likelihoods were never used during training. We found people integrated a new likelihood within (interpolation) better than the one outside (extrapolation) the range of their previous learning experience and they were quantitatively Bayes-suboptimal in both. We replicated the findings of both studies in a validation dataset. Our results showed that Bayesian behaviours may not always be achieved by a full Bayesian computation. Future studies can apply our approach to different tasks to enhance the understanding of decision-making mechanisms.

Harnessing the flexibility of neural networks to predict dynamic theoretical parameters underlying human choice behavior

Reinforcement learning (RL) models are used extensively to study human behavior. These rely on normative models of behavior and stress interpretability over predictive capabilities. More recently, neural network models have emerged as a descriptive modeling paradigm that is capable of high predictive power yet with limited interpretability. Here, we seek to augment the expressiveness of theoretical RL models with the high flexibility and predictive power of neural networks. We introduce a novel framework, which we term theoretical-RNN (t-RNN), whereby a recurrent neural network is trained to predict trial-by-trial behavior and to infer theoretical RL parameters using artificial data of RL agents performing a two-armed bandit task. In three studies, we then examined the use of our approach to dynamically predict unseen behavior along with time-varying theoretical RL parameters. We first validate our approach using synthetic data with known RL parameters. Next, as a proof-of-concept, we applied our framework to two independent datasets of humans performing the same task. In the first dataset, we describe differences in theoretical RL parameters dynamic among clinical psychiatric vs. healthy controls. In the second dataset, we show that the exploration strategies of humans varied dynamically in response to task phase and difficulty. For all analyses, we found better performance in the prediction of actions for t-RNN compared to the stationary maximum-likelihood RL method. We discuss the use of neural networks to facilitate the estimation of latent RL parameters underlying choice behavior.

Familiarity Processing through Faces and Names: Insights from Multivoxel Pattern Analysis

The way our brain processes personal familiarity is still debatable. We used searchlight multivoxel pattern analysis (MVPA) to identify areas where local fMRI patterns could contribute to familiarity detection for both faces and name categories. Significantly, we identified cortical areas in frontal, temporal, cingulate, and insular areas, where it is possible to accurately cross-classify familiar stimuli from one category using a classifier trained with the stimulus from the other (i.e., abstract familiarity) based on local fMRI patterns. We also discovered several areas in the fusiform gyrus, frontal, and temporal regions-primarily lateralized to the right hemisphere-supporting the classification of familiar faces but failing to do so for names. Also, responses to familiar names (compared to unfamiliar names) consistently showed less activation strength than responses to familiar faces (compared to unfamiliar faces). The results evinced a set of abstract familiarity areas (independent of the stimulus type) and regions specifically related only to face familiarity, contributing to recognizing familiar individuals.

Human attention during goal-directed reading comprehension relies on task optimization

The computational principles underlying attention allocation in complex goal-directed tasks remain elusive. Goal-directed reading, that is, reading a passage to answer a question in mind, is a common real-world task that strongly engages attention. Here, we investigate what computational models can explain attention distribution in this complex task. We show that the reading time on each word is predicted by the attention weights in transformer-based deep neural networks (DNNs) optimized to perform the same reading task. Eye tracking further reveals that readers separately attend to basic text features and question-relevant information during first-pass reading and rereading, respectively. Similarly, text features and question relevance separately modulate attention weights in shallow and deep DNN layers. Furthermore, when readers scan a passage without a question in mind, their reading time is predicted by DNNs optimized for a word prediction task. Therefore, we offer a computational account of how task optimization modulates attention distribution during real-world reading.

Functional neuroimaging as a catalyst for integrated neuroscience

Functional magnetic resonance imaging (fMRI) enables non-invasive access to the awake, behaving human brain. By tracking whole-brain signals across a diverse range of cognitive and behavioural states or mapping differences associated with specific traits or clinical conditions, fMRI has advanced our understanding of brain function and its links to both normal and atypical behaviour. Despite this headway, progress in human cognitive neuroscience that uses fMRI has been relatively isolated from rapid advances in other subdomains of neuroscience, which themselves are also somewhat siloed from one another. In this Perspective, we argue that fMRI is well-placed to integrate the diverse subfields of systems, cognitive, computational and clinical neuroscience. We first summarize the strengths and weaknesses of fMRI as an imaging tool, then highlight examples of studies that have successfully used fMRI in each subdomain of neuroscience. We then provide a roadmap for the future advances that will be needed to realize this integrative vision. In this way, we hope to demonstrate how fMRI can help usher in a new era of interdisciplinary coherence in neuroscience.

Modeling naturalistic face processing in humans with deep convolutional neural networks

Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces but less accurately capture individuation and dynamic features.

Simple and complex cells revisited: toward a selectivity-invariance model of object recognition

This paper presents a theoretical perspective on modeling ventral stream processing by revisiting the computational abstraction of simple and complex cells. In parallel to David Marr's vision theory, we organize the new perspective into three levels. At the computational level, we abstract simple and complex cells into space partitioning and composition in a topological space based on the redundancy exploitation hypothesis of Horace Barlow. At the algorithmic level, we present a hierarchical extension of sparse coding by exploiting the manifold constraint in high-dimensional space (i.e., the blessing of dimensionality). The resulting over-parameterized models for object recognition differ from existing hierarchical models by disentangling the objectives of selectivity and invariance computation. It is possible to interpret our hierarchical construction as a computational implementation of cortically local subspace untangling for object recognition and face representation, which are closely related to exemplar-based and axis-based coding in the medial temporal lobe. At the implementation level, we briefly discuss two possible implementations based on asymmetric sparse autoencoders and divergent spiking neural networks.

Studying Psychosis Using Natural Language Generation: A Review of Emerging Opportunities

Disrupted language in psychotic disorders, such as schizophrenia, can manifest as false contents and formal deviations, often described as thought disorder. These features play a critical role in the social dysfunction associated with psychosis, but we continue to lack insights regarding how and why these symptoms develop. Natural language generation (NLG) is a field of computer science that focuses on generating human-like language for various applications. The theory that psychosis is related to the evolution of language in humans suggests that NLG systems that are sufficiently evolved to generate human-like language may also exhibit psychosis-like features. In this conceptual review, we propose using NLG systems that are at various stages of development as in silico tools to study linguistic features of psychosis. We argue that a program of in silico experimental research on the network architecture, function, learning rules, and training of NLG systems can help us understand better why thought disorder occurs in patients. This will allow us to gain a better understanding of the relationship between language and psychosis and potentially pave the way for new therapeutic approaches to address this vexing challenge.

Artificial Neuronal Devices Based on Emerging Materials: Neuronal Dynamics and Applications

Artificial neuronal devices are critical building blocks of neuromorphic computing systems and currently the subject of intense research motivated by application needs from new computing technology and more realistic brain emulation. Researchers have proposed a range of device concepts that can mimic neuronal dynamics and functions. Although the switching physics and device structures of these artificial neurons are largely different, their behaviors can be described by several neuron models in a more unified manner. In this paper, the reports of artificial neuronal devices based on emerging volatile switching materials are reviewed from the perspective of the demonstrated neuron models, with a focus on the neuronal functions implemented in these devices and the exploitation of these functions for computational and sensing applications. Furthermore, the neuroscience inspirations and engineering methods to enrich the neuronal dynamics that remain to be implemented in artificial neuronal devices and networks toward realizing the full functionalities of biological neurons are discussed.

The nature and neurobiology of fear and anxiety: State of the science and opportunities for accelerating discovery

Fear and anxiety play a central role in mammalian life, and there is considerable interest in clarifying their nature, identifying their biological underpinnings, and determining their consequences for health and disease. Here we provide a roundtable discussion on the nature and biological bases of fear- and anxiety-related states, traits, and disorders. The discussants include scientists familiar with a wide variety of populations and a broad spectrum of techniques. The goal of the roundtable was to take stock of the state of the science and provide a roadmap to the next generation of fear and anxiety research. Much of the discussion centered on the key challenges facing the field, the most fruitful avenues for future research, and emerging opportunities for accelerating discovery, with implications for scientists, funders, and other stakeholders. Understanding fear and anxiety is a matter of practical importance. Anxiety disorders are a leading burden on public health and existing treatments are far from curative, underscoring the urgency of developing a deeper understanding of the factors governing threat-related emotions.

Convolutional neural networks for vision neuroscience: significance, developments, and outstanding issues

Convolutional Neural Networks (CNN) are a class of machine learning models predominately used in computer vision tasks and can achieve human-like performance through learning from experience. Their striking similarities to the structural and functional principles of the primate visual system allow for comparisons between these artificial networks and their biological counterparts, enabling exploration of how visual functions and neural representations may emerge in the real brain from a limited set of computational principles. After considering the basic features of CNNs, we discuss the opportunities and challenges of endorsing CNNs as in silico models of the primate visual system. Specifically, we highlight several emerging notions about the anatomical and physiological properties of the visual system that still need to be systematically integrated into current CNN models. These tenets include the implementation of parallel processing pathways from the early stages of retinal input and the reconsideration of several assumptions concerning the serial progression of information flow. We suggest design choices and architectural constraints that could facilitate a closer alignment with biology provide causal evidence of the predictive link between the artificial and biological visual systems. Adopting this principled perspective could potentially lead to new research questions and applications of CNNs beyond modeling object recognition.

A shared linguistic space for transmitting our thoughts from brain to brain in natural conversations

Effective communication hinges on a mutual understanding of word meaning in different contexts. The embedding space learned by large language models can serve as an explicit model of the shared, context-rich meaning space humans use to communicate their thoughts. We recorded brain activity using electrocorticography during spontaneous, face-to-face conversations in five pairs of epilepsy patients. We demonstrate that the linguistic embedding space can capture the linguistic content of word-by-word neural alignment between speaker and listener. Linguistic content emerged in the speaker's brain before word articulation, and the same linguistic content rapidly reemerged in the listener's brain after word articulation. These findings establish a computational framework to study how human brains transmit their thoughts to one another in real-world contexts.

Distinctive properties of biological neural networks and recent advances in bottom-up approaches toward a better biologically plausible neural network

Although it may appear infeasible and impractical, building artificial intelligence (AI) using a bottom-up approach based on the understanding of neuroscience is straightforward. The lack of a generalized governing principle for biological neural networks (BNNs) forces us to address this problem by converting piecemeal information on the diverse features of neurons, synapses, and neural circuits into AI. In this review, we described recent attempts to build a biologically plausible neural network by following neuroscientifically similar strategies of neural network optimization or by implanting the outcome of the optimization, such as the properties of single computational units and the characteristics of the network architecture. In addition, we proposed a formalism of the relationship between the set of objectives that neural networks attempt to achieve, and neural network classes categorized by how closely their architectural features resemble those of BNN. This formalism is expected to define the potential roles of top-down and bottom-up approaches for building a biologically plausible neural network and offer a map helping the navigation of the gap between neuroscience and AI engineering.

As without, so within: how the brain's temporo-spatial alignment to the environment shapes consciousness

Consciousness is constituted by a structure that includes contents as foreground and the environment as background. This structural relation between the experiential foreground and background presupposes a relationship between the brain and the environment, often neglected in theories of consciousness. The temporo-spatial theory of consciousness addresses the brain-environment relation by a concept labelled 'temporo-spatial alignment'. Briefly, temporo-spatial alignment refers to the brain's neuronal activity's interaction with and adaption to interoceptive bodily and exteroceptive environmental stimuli, including their symmetry as key for consciousness. Combining theory and empirical data, this article attempts to demonstrate the yet unclear neuro-phenomenal mechanisms of temporo-spatial alignment. First, we suggest three neuronal layers of the brain's temporo-spatial alignment to the environment. These neuronal layers span across a continuum from longer to shorter timescales. (i) The background layer comprises longer and more powerful timescales mediating topographic-dynamic similarities between different subjects' brains. (ii) The intermediate layer includes a mixture of medium-scaled timescales allowing for stochastic matching between environmental inputs and neuronal activity through the brain's intrinsic neuronal timescales and temporal receptive windows. (iii) The foreground layer comprises shorter and less powerful timescales for neuronal entrainment of stimuli temporal onset through neuronal phase shifting and resetting. Second, we elaborate on how the three neuronal layers of temporo-spatial alignment correspond to their respective phenomenal layers of consciousness. (i) The inter-subjectively shared contextual background of consciousness. (ii) An intermediate layer that mediates the relationship between different contents of consciousness. (iii) A foreground layer that includes specific fast-changing contents of consciousness. Overall, temporo-spatial alignment may provide a mechanism whose different neuronal layers modulate corresponding phenomenal layers of consciousness. Temporo-spatial alignment can provide a bridging principle for linking physical-energetic (free energy), dynamic (symmetry), neuronal (three layers of distinct time-space scales) and phenomenal (form featured by background-intermediate-foreground) mechanisms of consciousness.

A Novel Exploratory Graph-Based Analytical Tool for Functional Near-Infrared Spectroscopy in Naturalistic Experiments: An Illustrative Application in Typically Developing Children

Naturalistic paradigms are being increasingly applied to investigate human brain function. Compared with resting-state and task-based paradigms in neuroimaging, naturalistic stimuli and situations can be potentially more readily translated to daily-life applications. Among neuroimaging modalities, functional near-infrared spectroscopy (fNIRS) is particularly suitable for naturalistic investigations and applications. However, specific and tailored statistical analysis to interrogate brain function using naturalistic fNIRS is warranted. Here, we describe an exploratory graph-centrality-based approach to investigating participants' spatiotemporal similarities from the fNIRS signal. We illustrate the usefulness of our approach in a sample of typically developing children (10 males and 9 females; mean age of 5.2 years old; sd = 0.78) while they watch the Inscapes movie designed for neuroimaging acquisition. A node in the left dorsal prefrontal cortex presented similar responses across children, and those fNIRS responses were in line with scene transitions in the movie stimulus. Our results suggest the feasibility of applying centrality graph-based measures to investigate brain function in naturalistic fNIRS during development.

Toward a computational theory of manifold untangling: from global embedding to local flattening

It has been hypothesized that the ventral stream processing for object recognition is based on a mechanism called cortically local subspace untangling. A mathematical abstraction of object recognition by the visual cortex is how to untangle the manifolds associated with different object categories. Such a manifold untangling problem is closely related to the celebrated kernel trick in metric space. In this paper, we conjecture that there is a more general solution to manifold untangling in the topological space without artificially defining any distance metric. Geometrically, we can either embed a manifold in a higher-dimensional space to promote selectivity or flatten a manifold to promote tolerance. General strategies of both global manifold embedding and local manifold flattening are presented and connected with existing work on the untangling of image, audio, and language data. We also discuss the implications of untangling the manifold into motor control and internal representations.

Memristor-based neural networks: a bridge from device to artificial intelligence

Since the beginning of the 21st century, there is no doubt that the importance of artificial intelligence has been highlighted in many fields, among which the memristor-based artificial neural network technology is expected to break through the limitation of von Neumann so as to realize the replication of the human brain by enabling strong parallel computing ability and efficient data processing and become an important way towards the next generation of artificial intelligence. A new type of nanodevice, namely memristor, which is based on the variability of its resistance value, not only has very important applications in nonvolatile information storage, but also presents obsessive progressiveness in highly integrated circuits, making it one of the most promising circuit components in the post-Moore era. In particular, memristors can effectively simulate neural synapses and build neural networks; thus, they can be applied for the preparation of various artificial intelligence systems. This study reviews the research progress of memristors in artificial neural networks in detail and highlights the structural advantages and frontier applications of neural networks based on memristors. Finally, some urgent problems and challenges in current research are summarized and corresponding solutions and future development trends are put forward.

Artificial neural network model- and response surface methodology-based optimization of Atractylodis Macrocephalae Rhizoma polysaccharide extraction, kinetic modelling and structural characterization

Atractylodis Macrocephalae Rhizoma (AMR) is the dried rhizome of Atractylodes macrocephala Koidz, which is widely used in the development of health products. AMR contains a large number of polysaccharides, but at present there are fewer applications for these polysaccharides. In this study, the effects of different extraction methods on the Atractylodis Macrocephalae Rhizoma polysaccharide (AMRP) yield were investigated, and the conditions for ultrasound-assisted extraction were optimized by response surface methodology (RSM) and three neural network models (BP neural network, GA-BP neural network and ACO-GA-BP neural network). The best conditions were a liquid-to-solid ratio of 17 mL/g, ultrasonic power of 400 W, extraction temperature of 72 °C, and extraction time of 40 min, which yielded 31.31% AMRP. The kinetic equation of AMRP was determined and compared with the results predicted by three neural network models. It was finally determined that the extraction conditions, kinetic processes and kinetic equation predicted by the GA-ACO-BP neural network were optimal. In addition, AMRP was characterized using SEM, FTIR, HPLC, UV, XRD, and NMR, and the structural study revealed that AMRP has a rough exterior and a porous interior; moreover, it contains high levels of glucose (5.07%), arabinose (0.80%), and galactose (0.74%). AMRP has three crystal structures, consisting of two β-type monosaccharides and one α-type monosaccharide. Additionally, the effectiveness of AMRP as an antioxidant was demonstrated in an in vitro experiment.

Development as explanation: Understanding phenotypic stability and variability after the failure of genetic determinism

In the predominately gene-centered view of 20th century biology, the relationship between genotype and phenotype was essentially a relationship between cause and effect, between a plan and a product. Abandoning the idea of genes as inherited instructions or blueprints for phenotypes raises the question of how to best account for observed phenotypic stability and variability within and across generations of a population. We argue that the processes responsible for phenotypic stability and the processes responsible for phenotypic variability are one and the same, namely, the dynamics of development. This argument proposes that stability of phenotypic form is found not because of the transmission of genotypes, genetic programs, or the transfer of internal blueprints, but because similar internal and external conditions-collectively conceptualized as resources of development-can be reliably reconstituted in each generation. Variability of phenotypic form, which is an indispensable feature of any evolving system, relies on these same resources, but because the internal and external conditions of development are not reconstituted identically in succeeding generations, these conditions-and the phenotypes to which they give rise-will always be characterized by at least some variability.

Modeling surface color discrimination under different lighting environments using image chromatic statistics and convolutional neural networks

We modeled discrimination thresholds for object colors under different lighting environments [J. Opt. Soc. Am. 35, B244 (2018)]. First, we built models based on chromatic statistics, testing 60 models in total. Second, we trained convolutional neural networks (CNNs), using 160,280 images labeled by either the ground-truth or human responses. No single chromatic statistics model was sufficient to describe human discrimination thresholds across conditions, while human-response-trained CNNs nearly perfectly predicted human thresholds. Guided by region-of-interest analysis of the network, we modified the chromatic statistics models to use only the lower regions of the objects, which substantially improved performance.

Modeling surface color discrimination under different lighting environments using image chromatic statistics and convolutional neural networks

We modeled discrimination thresholds for object colors under different lighting environments [J. Opt. Soc. Am. 35, B244 (2018)]. First, we built models based on chromatic statistics, testing 60 models in total. Second, we trained convolutional neural networks (CNNs), using 160,280 images labeled by either the ground-truth or human responses. No single chromatic statistics model was sufficient to describe human discrimination thresholds across conditions, while human-response-trained CNNs nearly perfectly predicted human thresholds. Guided by region-of-interest analysis of the network, we modified the chromatic statistics models to use only the lower regions of the objects, which substantially improved performance.

Deep learning for studying drawing behavior: A review

In recent years, computer science has made major advances in understanding drawing behavior. Artificial intelligence, and more precisely deep learning, has displayed unprecedented performance in the automatic recognition and classification of large databases of sketches and drawings collected through touchpad devices. Although deep learning can perform these tasks with high accuracy, the way they are performed by the algorithms remains largely unexplored. Improving the interpretability of deep neural networks is a very active research area, with promising recent advances in understanding human cognition. Deep learning thus offers a powerful framework to study drawing behavior and the underlying cognitive processes, particularly in children and non-human animals, on whom knowledge is incomplete. In this literature review, we first explore the history of deep learning as applied to the study of drawing along with the main discoveries in this area, while proposing open challenges. Second, multiple ideas are discussed to understand the inherent structure of deep learning models. A non-exhaustive list of drawing datasets relevant to deep learning approaches is further provided. Finally, the potential benefits of coupling deep learning with comparative cultural analyses are discussed.

Neural mechanisms underlying the hierarchical construction of perceived aesthetic value

Little is known about how the brain computes the perceived aesthetic value of complex stimuli such as visual art. Here, we used computational methods in combination with functional neuroimaging to provide evidence that the aesthetic value of a visual stimulus is computed in a hierarchical manner via a weighted integration over both low and high level stimulus features contained in early and late visual cortex, extending into parietal and lateral prefrontal cortices. Feature representations in parietal and lateral prefrontal cortex may in turn be utilized to produce an overall aesthetic value in the medial prefrontal cortex. Such brain-wide computations are not only consistent with a feature-based mechanism for value construction, but also resemble computations performed by a deep convolutional neural network. Our findings thus shed light on the existence of a general neurocomputational mechanism for rapidly and flexibly producing value judgements across an array of complex novel stimuli and situations.

Emergent behaviour and neural dynamics in artificial agents tracking odour plumes

Tracking an odour plume to locate its source under variable wind and plume statistics is a complex task. Flying insects routinely accomplish such tracking, often over long distances, in pursuit of food or mates. Several aspects of this remarkable behaviour and its underlying neural circuitry have been studied experimentally. Here we take a complementary in silico approach to develop an integrated understanding of their behaviour and neural computations. Specifically, we train artificial recurrent neural network agents using deep reinforcement learning to locate the source of simulated odour plumes that mimic features of plumes in a turbulent flow. Interestingly, the agents' emergent behaviours resemble those of flying insects, and the recurrent neural networks learn to compute task-relevant variables with distinct dynamic structures in population activity. Our analyses put forward a testable behavioural hypothesis for tracking plumes in changing wind direction, and we provide key intuitions for memory requirements and neural dynamics in odour plume tracking.

Computational meaningfulness as the source of beneficial cognitive biases

The human brain has evolved to solve the problems it encounters in multiple environments. In solving these challenges, it forms mental simulations about multidimensional information about the world. These processes produce context-dependent behaviors. The brain as overparameterized modeling organ is an evolutionary solution for producing behavior in a complex world. One of the most essential characteristics of living creatures is that they compute the values of information they receive from external and internal contexts. As a result of this computation, the creature can behave in optimal ways in each environment. Whereas most other living creatures compute almost exclusively biological values (e.g., how to get food), the human as a cultural creature computes meaningfulness from the perspective of one's activity. The computational meaningfulness means the process of the human brain, with the help of which an individual tries to make the respective situation comprehensible to herself to know how to behave optimally. This paper challenges the bias-centric approach of behavioral economics by exploring different possibilities opened up by computational meaningfulness with insight into wider perspectives. We concentrate on confirmation bias and framing effect as behavioral economics examples of cognitive biases. We conclude that from the computational meaningfulness perspective of the brain, the use of these biases are indispensable property of an optimally designed computational system of what the human brain is like. From this perspective, cognitive biases can be rational under some conditions. Whereas the bias-centric approach relies on small-scale interpretable models which include only a few explanatory variables, the computational meaningfulness perspective emphasizes the behavioral models, which allow multiple variables in these models. People are used to working in multidimensional and varying environments. The human brain is at its best in such an environment and scientific study should increasingly take place in such situations simulating the real environment. By using naturalistic stimuli (e.g., videos and VR) we can create more realistic, life-like contexts for research purposes and analyze resulting data using machine learning algorithms. In this manner, we can better explain, understand and predict human behavior and choice in different contexts.

Population codes enable learning from few examples by shaping inductive bias

Learning from a limited number of experiences requires suitable inductive biases. To identify how inductive biases are implemented in and shaped by neural codes, we analyze sample-efficient learning of arbitrary stimulus-response maps from arbitrary neural codes with biologically-plausible readouts. We develop an analytical theory that predicts the generalization error of the readout as a function of the number of observed examples. Our theory illustrates in a mathematically precise way how the structure of population codes shapes inductive bias, and how a match between the code and the task is crucial for sample-efficient learning. It elucidates a bias to explain observed data with simple stimulus-response maps. Using recordings from the mouse primary visual cortex, we demonstrate the existence of an efficiency bias towards low-frequency orientation discrimination tasks for grating stimuli and low spatial frequency reconstruction tasks for natural images. We reproduce the discrimination bias in a simple model of primary visual cortex, and further show how invariances in the code to certain stimulus variations alter learning performance. We extend our methods to time-dependent neural codes and predict the sample efficiency of readouts from recurrent networks. We observe that many different codes can support the same inductive bias. By analyzing recordings from the mouse primary visual cortex, we demonstrate that biological codes have lower total activity than other codes with identical bias. Finally, we discuss implications of our theory in the context of recent developments in neuroscience and artificial intelligence. Overall, our study provides a concrete method for elucidating inductive biases of the brain and promotes sample-efficient learning as a general normative coding principle.

Recovering Individual Emotional States from Sparse Ratings Using Collaborative Filtering

A fundamental challenge in emotion research is measuring feeling states with high granularity and temporal precision without disrupting the emotion generation process. Here we introduce and validate a new approach in which responses are sparsely sampled and the missing data are recovered using a computational technique known as collaborative filtering (CF). This approach leverages structured covariation across individual experiences and is available in Neighbors, an open-source Python toolbox. We validate our approach across three different experimental contexts by recovering dense individual ratings using only a small subset of the original data. In dataset 1, participants (n=316) separately rated 112 emotional images on 6 different discrete emotions. In dataset 2, participants (n=203) watched 8 short emotionally engaging autobiographical stories while simultaneously providing moment-by-moment ratings of the intensity of their affective experience. In dataset 3, participants (n=60) with distinct social preferences made 76 decisions about how much money to return in a hidden multiplier trust game. Across all experimental contexts, CF was able to accurately recover missing data and importantly outperformed mean and multivariate imputation, particularly in contexts with greater individual variability. This approach will enable new avenues for affective science research by allowing researchers to acquire high dimensional ratings from emotional experiences with minimal disruption to the emotion-generation process.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-022-00161-2.