Synthetic PET: Analyzing large-scale properties of neural networks

Origins of Dissociations in the English Past Tense: A Synthetic Brain Imaging Model

Brain imaging studies of English past tense inflection have found dissociations between regular and irregular verbs, but no coherent picture has emerged to explain how these dissociations arise. Here we use synthetic brain imaging on a neural network model to provide a mechanistic account of the origins of such dissociations. The model suggests that dissociations between regional activation patterns in verb inflection emerge in an adult processing system that has been shaped through experience-dependent structural brain development. Although these dissociations appear to be between regular and irregular verbs, they arise in the model from a combination of statistical properties including frequency, relationships to other verbs, and phonological complexity, without a causal role for regularity or semantics. These results are consistent with the notion that all inflections are produced in a single associative mechanism. The model generates predictions about the patterning of active brain regions for different verbs that can be tested in future imaging studies.

Towards a Computational Comparative Neuroprimatology: Framing the language-ready brain

We make the case for developing a Computational Comparative Neuroprimatology to inform the analysis of the function and evolution of the human brain. First, we update the mirror system hypothesis on the evolution of the language-ready brain by (i) modeling action and action recognition and opportunistic scheduling of macaque brains to hypothesize the nature of the last common ancestor of macaque and human (LCA-m); and then we (ii) introduce dynamic brain modeling to show how apes could acquire gesture through ontogenetic ritualization, hypothesizing the nature of evolution from LCA-m to the last common ancestor of chimpanzee and human (LCA-c). We then (iii) hypothesize the role of imitation, pantomime, protosign and protospeech in biological and cultural evolution from LCA-c to Homo sapiens with a language-ready brain. Second, we suggest how cultural evolution in Homo sapiens led from protolanguages to full languages with grammar and compositional semantics. Third, we assess the similarities and differences between the dorsal and ventral streams in audition and vision as the basis for presenting and comparing two models of language processing in the human brain: A model of (i) the auditory dorsal and ventral streams in sentence comprehension; and (ii) the visual dorsal and ventral streams in defining "what language is about" in both production and perception of utterances related to visual scenes provide the basis for (iii) a first step towards a synthesis and a look at challenges for further research.

Modeling fMRI signals can provide insights into neural processing in the cerebral cortex

Every stimulus or task activates multiple areas in the mammalian cortex. These distributed activations can be measured with functional magnetic resonance imaging (fMRI), which has the best spatial resolution among the noninvasive brain imaging methods. Unfortunately, the relationship between the fMRI activations and distributed cortical processing has remained unclear, both because the coupling between neural and fMRI activations has remained poorly understood and because fMRI voxels are too large to directly sense the local neural events. To get an idea of the local processing given the macroscopic data, we need models to simulate the neural activity and to provide output that can be compared with fMRI data. Such models can describe neural mechanisms as mathematical functions between input and output in a specific system, with little correspondence to physiological mechanisms. Alternatively, models can be biomimetic, including biological details with straightforward correspondence to experimental data. After careful balancing between complexity, computational efficiency, and realism, a biomimetic simulation should be able to provide insight into how biological structures or functions contribute to actual data processing as well as to promote theory-driven neuroscience experiments. This review analyzes the requirements for validating system-level computational models with fMRI. In particular, we study mesoscopic biomimetic models, which include a limited set of details from real-life networks and enable system-level simulations of neural mass action. In addition, we discuss how recent developments in neurophysiology and biophysics may significantly advance the modelling of fMRI signals.

Modeling the BOLD correlates of competitive neural dynamics

Winner-take-all models are commonly used to model decision-making tasks where one outcome must be selected from several competing options. Related random walk and diffusion models have been used to explain such processes and apply them to psychometric and neurophysiological data. Recent model-based fMRI studies have sought to find the neural correlates of decision-making processes. However, due to the fact that hemodynamic responses likely reflect synaptic rather than spiking activity, the expected BOLD signature of winner-take-all circuits is not clear. A powerful way to integrate data from neurophysiology and brain imaging is by developing biologically plausible neural network models constrained and testable by neural and behavioral data, and then using Synthetic Brain Imaging - transforming the output of simulations with the model to make predictions testable against neuroimaging data. We developed a biologically realistic spiking winner-take-all model comprised of coupled excitatory and inhibitory neural populations. We varied the difficulty of a decision-making task by adjusting the contrast, or relative strength of inputs representing two response options. Synthetic brain imaging was used to estimate the BOLD response of the model and analyze its peak as a function of input contrast. We performed a parameter space analysis to determine values for which the model performs the task accurately, and given accurate performance, the distribution of the input contrast-BOLD response relationship. This underscores the need for models grounded in neurophysiological data for brain imaging analyses which attempt to localize the neural correlates of cognitive processes based on predicted BOLD responses.

A computational model of fMRI activity in the intraparietal sulcus that supports visual working memory

A computational model was developed to explain a pattern of results of fMRI activation in the intraparietal sulcus (IPS) supporting visual working memory for multiobject scenes. The model is based on the hypothesis that dendrites of excitatory neurons are major computational elements in the cortical circuit. Dendrites enable formation of a competitive queue that exhibits a gradient of activity values for nodes encoding different objects, and this pattern is stored in working memory. In the model, brain imaging data are interpreted as a consequence of blood flow arising from dendritic processing. Computer simulations showed that the model successfully simulates data showing the involvement of inferior IPS in object individuation and spatial grouping through representation of objects' locations in space, along with the involvement of superior IPS in object identification through representation of a set of objects' features. The model exhibits a capacity limit due to the limited dynamic range for nodes and the operation of lateral inhibition among them. The capacity limit is fixed in the inferior IPS regardless of the objects' complexity, due to the normalization of lateral inhibition, and variable in the superior IPS, due to the different encoding demands for simple and complex shapes. Systematic variation in the strength of self-excitation enables an understanding of the individual differences in working memory capacity. The model offers several testable predictions regarding the neural basis of visual working memory.

Are imaging and lesioning convergent methods for assessing functional specialisation?: investigations using an artificial neural network

This article presents an investigation of the relationship between lesioning and neuroimaging methods of assessing functional specialisation, using synthetic brain imaging (SBI) and lesioning of a connectionist network of past-tense formation. The model comprised two processing 'routes': one was a direct route between layers of input and output units, while the other, indirect, route featured an intermediate layer of processing units. Emergent specialisation within the network was assessed (1) by lesioning either the direct or indirect route and measuring past-tense performance for regular and irregular verbs, and (2) by measuring functional activation in each route when processing each verb type (SBI). SBI and lesioning approaches failed to converge when network activation was summed over each route in our SBI approach. Examination of individual network solutions suggested that the verb types might be using the indirect route differently in terms of the pattern of activation across the route, rather than in terms of gross activation. A subsequent SBI analysis compared patterns of activation in the indirect route and confirmed that these patterns were more similar between regular-type verbs than between regular and irregular verbs. As the spatial and temporal resolution of neuroimaging techniques improves, the results of this investigation suggest that the key to finding functional specialisation will be to distinguish local coding differences across behaviours that are the results of developmental processes. Other analyses suggest that lesioning data may be limited because, with increasing damage, they reveal the resting activations of a computational system rather than a computational specialisation per se.

Identification and validation of effective connectivity networks in functional magnetic resonance imaging using switching linear dynamic systems

Dynamic connectivity networks identify directed interregional interactions between modeled brain regions in neuroimaging. However, problems arise when the regions involved in a task and their interconnections are not fully known a priori. Objective measures of model adequacy are necessary to validate such models. We present a connectivity formalism, the Switching Linear Dynamic System (SLDS), that is capable of identifying both Granger-Geweke and instantaneous connectivity that vary according to experimental conditions. SLDS explicitly models the task condition as a Markov random variable. The series of task conditions can be estimated from new data given an identified model providing a means to validate connectivity patterns. We use SLDS to model functional magnetic resonance imaging data from five regions during a finger alternation task. Using interregional connectivity alone, the identified model predicted the task condition vector from a different subject with a different task ordering with high accuracy. In addition, important regions excluded from a model can be identified by augmenting the model state space. A motor task model excluding primary motor cortices was augmented with a new neural state constrained by its connectivity with the included regions. The augmented variable time series, convolved with a hemodynamic kernel, was compared to all brain voxels. The right primary motor cortex was identified as the best region to add to the model. Our results suggest that the SLDS model framework is an effective means to address several problems with modeling connectivity including measuring overall model adequacy and identifying important regions missing from models.

A link between neuroscience and informatics: large-scale modeling of memory processes

Utilizing advances in functional neuroimaging and computational neural modeling, neuroscientists have increasingly sought to investigate how distributed networks, composed of functionally defined subregions, combine to produce cognition. Large-scale, biologically realistic neural models, which integrate data from cellular, regional, whole brain, and behavioral sources, delineate specific hypotheses about how these interacting neural populations might carry out high-level cognitive tasks. In this review, we discuss neuroimaging, neural modeling, and the utility of large-scale biologically realistic models using modeling of short-term memory as an example. We present a sketch of the data regarding the neural basis of short-term memory from non-human electrophysiological, computational and neuroimaging perspectives, highlighting the multiple interacting brain regions believed to be involved. Through a review of several efforts, including our own, to combine neural modeling and neuroimaging data, we argue that large scale neural models provide specific advantages in understanding the distributed networks underlying cognition and behavior.

The organization of thinking: what functional brain imaging reveals about the neuroarchitecture of complex cognition

Recent findings in brain imaging, particularly in fMRI, are beginning to reveal some of the fundamental properties of the organization of the cortical systems that underpin complex cognition. We propose an emerging set of operating principles that govern this organization, characterizing the system as a set of collaborating cortical centers that operate as a large-scale cortical network. Two of the network's critical features are that it is resource constrained and dynamically configured, with resource constraints and demands dynamically shaping the network topology. The operating principles are embodied in a cognitive neuroarchitecture, 4CAPS, consisting of a number of interacting computational centers that correspond to activating cortical areas. Each 4CAPS center is a hybrid production system, possessing both symbolic and connectionist attributes. We describe 4CAPS models of sentence comprehension, spatial problem solving, and complex multitasking and compare the accounts of these models with brain activation and behavioral results. Finally, we compare 4CAPS with other proposed neuroarchitectures.

Exploring large-scale brain networks in functional MRI

Increasing emphasis has been recently put on large-scale network processing of brain functions. To explore these networks, many approaches have been proposed in functional magnetic resonance imaging (fMRI). Their objective is to answer the following two questions: (1) what brain regions are involved in the functional process under investigation? and (2) how do these regions interact? We review some of the key concepts and corresponding methods to cope with both issues.

Nonlinear local electrovascular coupling. I: A theoretical model

Here we present a detailed biophysical model of how brain electrical and vascular dynamics are generated within a basic cortical unit. The model was obtained from coupling a canonical neuronal mass and an expandable vasculature. In this proposal, we address several aspects related to electroencephalographic and functional magnetic resonance imaging data fusion: (1) the impact of the cerebral architecture (at different physical levels) on the observations; (2) the physiology involved in electrovascular coupling; and (3) energetic considerations to gain a better understanding of how the glucose budget is used during neuronal activity. The model has three components. The first is the canonical neural mass model of three subpopulations of neurons that respond to incoming excitatory synaptic inputs. The generation of the membrane potentials in the somas of these neurons and the electric currents flowing in the neuropil are modeled by this component. The second and third components model the electrovascular coupling and the dynamics of vascular states in an extended balloon approach, respectively. In the first part we describe, in some detail, the biophysical model and establish its face validity using simulations of visually evoked responses under different flickering frequencies and luminous contrasts. In a second part, a recursive optimization algorithm is developed and used to make statistical inferences about this forward/generative model from actual data.

Functional connectivity in fMRI: A modeling approach for estimation and for relating to local circuits

Although progress has been made in relating neuronal events to changes in brain metabolism and blood flow, the interpretation of functional neuroimaging data in terms of the underlying brain circuits is still poorly understood. Computational modeling of connection patterns both among and within regions can be helpful in this interpretation. We present a neural network model of the ventral visual pathway and its relevant functional connections. This includes a new learning method that adjusts the magnitude of interregional connections in order to match experimental results of an arbitrary functional magnetic resonance imaging (fMRI) data set. We demonstrate that this method finds the appropriate connection strengths when trained on a model system with known, randomly chosen connection weights. We then use the method for examining fMRI results from a one-back matching task in human subjects, both healthy and those with schizophrenia. The results discovered by the learning method support previous findings of a disconnection between left temporal and frontal cortices in the group with schizophrenia and a concomitant increase of right-sided temporo-frontal connection strengths. We then demonstrate that the disconnection may be explained by reduced local recurrent circuitry in frontal cortex. This method extends currently available methods for estimating functional connectivity from human imaging data by including both local circuits and features of interregional connections, such as topography and sparseness, in addition to total connection strengths. Furthermore, our results suggest how fronto-temporal functional disconnection in schizophrenia can result from reduced local synaptic connections within frontal cortex rather than compromised interregional connections.

Network analysis of single-subject fMRI during a finger opposition task

The analysis of functional magnetic resonance imaging (fMRI) data has typically relied on univariate methods to identify areas of brain activity related to cognitive and behavioral task performance. We investigated the ability of multivariate network analysis using a modified form of principal component analysis, the Scaled Subprofile Model (SSM), applied to single-subject fMRI data to identify patterns of interactions among brain regions over time during an anatomically well-characterized simple motor task. We hypothesized that each subject would exhibit correlated patterns of brain activation in several regions known to participate in the regulation of movement including the contralateral motor cortex and the ipsilateral cerebellum. EPI BOLD images were acquired in six healthy participants as they performed a visually and auditorally paced finger opposition task. SSM analysis was applied to the fMR time series on a single-subject basis. Linear combinations of the major principal components that predicted the expected hemodynamic response to the order of experimental conditions were identified for each participant. These combinations of SSM patterns were highly associated with the expected hemodynamic response, an indicator of local neuronal activity, in each participant (0.84 </= R(2) </= 0.97, all P's < 0.0001). As predicted, the combined pattern in each subject was characterized most prominently by relatively increased activations in contralateral sensorimotor cortex and ipsilateral cerebellum. Additionally, all subjects showed areas of relatively decreased activation in the ipsilateral sensorimotor cortex and contralateral cerebellum. The application of network analysis methods, such as SSM, to single-subject fMRI data can identify patterns of task-specific, functionally interacting brain areas in individual subjects. This approach may help identify individual differences in the task-related functional connectivity, track changes in task-related patterns of activity within or between fMRI sessions, and provide a method to identify individual differences in response to treatment.

fMRI MODELS OF DENDRITIC AND ASTROCYTIC NETWORKS

In order to elucidate the relationships between hierarchical structures within the neocortical neuropil and the information carried by an ensemble of neurons encompassing a single voxel, it is essential to predict through volume conductor modeling LFPs representing average extracellular potentials, which are expressed in terms of interstitial potentials of individual cells in networks of gap-junctionally connected astrocytes and synaptically connected neurons. These relationships have been provided and can then be used to investigate how the underlying neuronal population activity can be inferred from the measurement of the BOLD signal through electrovascular coupling mechanisms across the blood-brain barrier. The importance of both synaptic and extrasynaptic transmission as the basis of electrophysiological indices triggering vascular responses between dendritic and astrocytic networks, and sequential configurations of firing patterns in composite neural networks is emphasized. The purpose of this review is to show how fMRI data may be used to draw conclusions about the information transmitted by individual neurons in populations generating the BOLD signal.

Integrating neuroscientific data across spatiotemporal scales

A major challenge confronting neuroscientists is associated with the multiple spatial and temporal scales of investigation of neural structure and function. I shall discuss the use of computational neural modeling as one method to bridge some of the different spatial and temporal levels. This approach will be illustrated using large-scale, neurobiologically realistic network models of auditory and visual pattern recognition that relate neuronal dynamics to fMRI data. It will be demonstrated that the models are capable of exhibiting the salient features of both electrophysiological neuronal activities and fMRI values that are in agreement with empirically observed data.

Relating neuronal dynamics for auditory object processing to neuroimaging activity: a computational modeling and an fMRI study

We investigated the neural basis of auditory object processing in the cerebral cortex by combining neural modeling and functional neuroimaging. We developed a large-scale, neurobiologically realistic network model of auditory pattern recognition that relates the neuronal dynamics of cortical auditory processing of frequency modulated (FM) sweeps to functional neuroimaging data of the type obtained using PET and fMRI. Areas included in the model extend from primary auditory to prefrontal cortex. The electrical activities of the neuronal units of the model were constrained to agree with data from the neurophysiological literature regarding the perception of FM sweeps. We also conducted an fMRI experiment using stimuli and tasks similar to those used in our simulations. The integrated synaptic activity of the neuronal units in each region of the model, convolved with a hemodynamic response function, was used as a correlate of the simulated fMRI activity, and generally agreed with the experimentally observed fMRI data in the brain areas corresponding to the regions of the model. Our results demonstrate that the model is capable of exhibiting the salient features of both electrophysiological neuronal activities and fMRI values that are in agreement with empirically observed data. These findings provide support for our hypotheses concerning how auditory objects are processed by primate neocortex.

Brain network interactions in auditory, visual and linguistic processing

In the paper, we discuss the importance of network interactions between brain regions in mediating performance of sensorimotor and cognitive tasks, including those associated with language processing. Functional neuroimaging, especially PET and fMRI, provide data that are obtained essentially simultaneously from much of the brain, and thus are ideal for enabling one to assess interregional functional interactions. Two ways to use these types of data to assess network interactions are presented. First, using PET, we demonstrate that anterior and posterior perisylvian language areas have stronger functional connectivity during spontaneous narrative production than during other less linguistically demanding production tasks. Second, we show how one can use large-scale neural network modeling to relate neural activity to the hemodynamically-based data generated by fMRI and PET. We review two versions of a model of object processing - one for visual and one for auditory objects. The regions comprising the models include primary and secondary sensory cortex, association cortex in the temporal lobe, and prefrontal cortex. Each model incorporates specific assumptions about how neurons in each of these areas function, and how neurons in the different areas are interconnected with each other. Each model is able to perform a delayed match-to-sample task for simple objects (simple shapes for the visual model; tonal contours for the auditory model). We find that the simulated electrical activities in each region are similar to those observed in nonhuman primates performing analogous tasks, and the absolute values of the simulated integrated synaptic activity in each brain region match human fMRI/PET data. Thus, this type of modeling provides a way to understand the neural bases for the sensorimotor and cognitive tasks of interest.

Cerebral modeling and dynamic Bayesian networks

The understanding and the prediction of the clinical outcomes of focal or degenerative cerebral lesions, as well as the assessment of rehabilitation procedures, necessitate knowing the cerebral substratum of cognitive or sensorimotor functions. This is achieved by activation studies, where subjects are asked to perform a specific task while data of their brain functioning are obtained through functional neuroimaging techniques. Such studies, as well as animal experiments, have shown that sensorimotor or cognitive functions are the offspring of the activity of large-scale networks of anatomically connected cerebral regions. However, no one-to-one correspondence between activated networks adn functions can be found. Our research aims at understanding how the activation of large-scale networks derives from cerebral information processing mechanisms, which can only explain apparently conflicting activation data. Our work falls at the crossroads of neuroimaging interpretation techniques and computational neuroscience. Since knowledge in cognitive neuroscience is permanently evolving, our research aims more precisely at defining a new modeling formalism and at building a flexible simulator, allowing a quick implementation of the models, for a better interpretation of cerebral functional images. It also aims at providing plausible models, at eht level of large-scale networks, of cerebral information processing mechanisms in humans. In this paper, we propose a formalism, based on dynamic Bayesian networks (DBNs), that respects the following constraints: an oriented, networks architecture, whose nodes (the cerebral structures) can all be different, the implementation of causality--the activation of the structure is caused by upstream nodes' activation--the explicit representation of different time scales (from 1 ms for the cerebral activity to many seconds for a PET scan image acquisition), the representation of cerebral information at the integrated level of neuronal populations, the imprecision of functional neuroimaging data, the nonlinearity and the uncertainty in cerebral mechanisms, and brain's plasticity (learning, reorganization, modulation). One of the main problems, nonlinearity, has been tackled thanks to new extensions of the Kalman filter. The capabilities of the formalism's current version are illustrated by the modeling of a phoneme categorization process, explaining the different cerebral activations in normal and dyslexic subjects.

Language evolution: neural homologies and neuroinformatics☆

This paper contributes to neurolinguistics by grounding an evolutionary account of the readiness of the human brain for language in the search for homologies between different cortical areas in macaque and human. We consider two hypotheses for this grounding, that of Aboitiz and Garci;a [Brain Res. Rev. 25 (1997) 381] and the Mirror System Hypothesis of Rizzolatti and Arbib [Trends Neurosci. 21 (1998) 188] and note the promise of computational modeling of neural circuitry of the macaque and its linkage to analysis of human brain imaging data. In addition to the functional differences between the two hypotheses, problems arise because they are grounded in different cortical maps of the macaque brain. In order to address these divergences, we have developed several neuroinformatics tools included in an on-line knowledge management system, the NeuroHomology Database, which is equipped with inference engines both to relate and translate information across equivalent cortical maps and to evaluate degrees of homology for brain regions of interest in different species.

Simulating transcranial magnetic stimulation during PET with a large-scale neural network model of the prefrontal cortex and the visual system

Transcranial magnetic stimulation (TMS) exerts both excitatory and inhibitory effects on the stimulated neural tissue, although little is known about the neurobiological mechanisms by which it influences neuronal function. TMS has been used in conjunction with PET to examine interregional connectivity of human cerebral cortex. To help understand how TMS affects neuronal function, and how these effects are manifested during functional brain imaging, we simulated the effects of TMS on a large-scale neurobiologically realistic computational model consisting of multiple, interconnected regions that performs a visual delayed-match-to-sample task. The simulated electrical activities in each region of the model are similar to those found in single-cell monkey data, and the simulated integrated summed synaptic activities match regional cerebral blood flow (rCBF) data obtained in human PET studies. In the present simulations, the excitatory and inhibitory effects of TMS on both locally stimulated and distal sites were studied using simulated behavioral measures and simulated PET rCBF results. The application of TMS to either excitatory or inhibitory units of the model, or both, resulted in an increased number of errors in the task performed by the model. In experimental studies, both increases and decreases in rCBF following TMS have been observed. In the model, increasing TMS intensity caused an increase in rCBF when TMS exerted a predominantly excitatory effect, whereas decreased rCBF following TMS occurred if TMS exerted a predominantly inhibitory effect. We also found that regions both directly and indirectly connected to the stimulating site were affected by TMS.

Interpreting PET and fMRI measures of functional neural activity: the effects of synaptic inhibition on cortical activation in human imaging studies

Human brain imaging methods such as postiron emission tomography and functional magnetic resonance imaging have recently achieved widespread use in the study of both normal cognitive processes and neurological disorders. While many of these studies have begun to yield important insights into human brain function, the relationship between these measurements and the underlying neuronal activity is still not well understood. One open question is how neuronal inhibition is reflected in these imaging results. In this paper, we describe how large-scale modeling can be used to address this question. Specifically, we identify three factors that may play a role in how inhibition affects imaging results: (1) local connectivity; (2) context; and (3) type of inhibitory connection. Simulation results are presented that show how the interaction among these three factors can explain seemingly contradictory experimental results. The modeling suggests that neuronal inhibition can raise brain imaging measures if there is either low local excitatory recurrence or if the region is not otherwise being driven by excitation. Conversely, with high recurrence or actively driven excitation, inhibition can lower observed values.

A model of working memory: bridging the gap between electrophysiology and human brain imaging

Human neuroimaging methods such as positron emission tomography and functional magnetic resonance imaging have made possible the study of large-scale distributed networks in the behaving human brain. Although many imaging studies support and extend knowledge gained from other experimental modalities such as animal single-cell recordings, there have also been a substantial number of experiments that appear to contradict the animal studies. Part of the reason for this is that neuroimaging is an indirect measure of neuronal firing activity, and thus interpretation is difficult. Computational modeling can help to bridge the gap by providing a substrate for making explicit the assumptions and constraints provided from other sources such as anatomy, physiology and behavior. We describe a large-scale model of working memory that we have used to examine a number of issues relating to the interpretation of imaging data. The gating mechanism that regulates engagement and retention of short-term memory is revised to better reflect hypothesized underlying neuromodulatory mechanisms. It is shown that in addition to imparting better performance for the memory circuit, this mechanism also provides a better match to imaging data from working memory studies.

Neural modeling and functional brain imaging: an overview

This article gives an overview of the different functional brain imaging methods, the kinds of questions these methods try to address and some of the questions associated with functional neuroimaging data for which neural modeling must be employed to provide reasonable answers.

Synthetic brain imaging: grasping, mirror neurons and imitation

The article contributes to the quest to relate global data on brain and behavior (e.g. from PET, Positron Emission Tomography, and fMRI. functional Magnetic Resonance Imaging) to the underpinning neural networks. Models tied to human brain imaging data often focus on a few "boxes" based on brain regions associated with exceptionally high blood flow, rather than analyzing the cooperative computation of multiple brain regions. For analysis directly at the level of such data, a schema-based model may be most appropriate. To further address neurophysiological data, the Synthetic PET imaging method uses computational models of biological neural circuitry based on animal data to predict and analyze the results of human PET studies. This technique makes use of the hypothesis that rCBF (regional cerebral blood flow) is correlated with the integrated synaptic activity in a localized brain region. We also describe the possible extension of the Synthetic PET method to fMRI. The second half of the paper then exemplifies this general research program with two case studies, one on visuo-motor processing for control of grasping (Section 3 in which the focus is on Synthetic PET) and the imitation of motor skills (Sections 4 and 5, with a focus on Synthetic fMRI). Our discussion of imitation pays particular attention to data on the mirror system in monkey (neural circuitry which allows the brain to recognize actions as well as execute them). Finally, Section 6 outlines the immense challenges in integrating models of different portions of the nervous system which address detailed neurophysiological data from studies of primates and other species; summarizes key issues for developing the methodology of Synthetic Brain Imaging; and shows how comparative neuroscience and evolutionary arguments will allow us to extend Synthetic Brain Imaging even to language and other cognitive functions for which few or no animal data are available.

Predicting human functional maps with neural net modeling

Formidable difficulties exist in interpreting positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) hemodynamic signals in terms of the underlying neural activity. These include issues of spatial and temporal resolution and problems relating neuronal activity (i.e., action potentials) measured in nonhuman studies by single unit electrodes to hemodynamic measurements reflecting synaptic activity. Also, regional hemodynamic measurements correspond to a mixture of local and afferent synaptic activity. To surmount these difficulties, we propose using large-scale neurobiologically realistic models in which data at various spatial and temporal levels can be simulated and cross-validated by multiple disciplines, including functional neuroimaging. A delayed match-to-sample visual task is used to illustrate this approach.

Neural modeling, functional brain imaging, and cognition

The richness and complexity of data sets acquired from PET or fMRI studies of human cognition have not been exploited until recently by computational neural-modeling methods. In this article, two neural-modeling approaches for use with functional brain imaging data are described. One, which uses structural equation modeling, estimates the functional strengths of the anatomical connections between various brain regions during specific cognitive tasks. The second employs large-scale neural modeling to relate functional neuroimaging signals in multiple, interconnected brain regions to the underlying neurobiological time-varying activities in each region. Delayed match-to-sample (visual working memory for form) tasks are used to illustrate these models.