The Automatic Neuroscientist: A framework for optimizing experimental design with closed-loop real-time fMRI

Dynamic layer-specific processing in the prefrontal cortex during working memory

The dorsolateral prefrontal cortex (dlPFC) is reliably engaged in working memory (WM) and comprises different cytoarchitectonic layers, yet their functional role in human WM is unclear. Here, participants completed a delayed-match-to-sample task while undergoing functional magnetic resonance imaging (fMRI) at ultra-high resolution. We examine layer-specific activity to manipulations in WM load and motor response. Superficial layers exhibit a preferential response to WM load during the delay and retrieval periods of a WM task, indicating a lamina-specific activation of the frontoparietal network. Multivariate patterns encoding WM load in the superficial layer dynamically change across the three periods of the task. Last, superficial and deep layers are non-differentially involved in the motor response, challenging earlier findings of a preferential deep layer activation. Taken together, our results provide new insights into the functional laminar circuitry of the dlPFC during WM and support a dynamic account of dlPFC coding.

Neuroadaptive Bayesian optimisation to study individual differences in infants' engagement with social cues

Infants' motivation to engage with the social world depends on the interplay between individual brain's characteristics and previous exposure to social cues such as the parent's smile or eye contact. Different hypotheses about why specific combinations of emotional expressions and gaze direction engage children have been tested with group-level approaches rather than focusing on individual differences in the social brain development. Here, a novel Artificial Intelligence-enhanced brain-imaging approach, Neuroadaptive Bayesian Optimisation (NBO), was applied to infant electro-encephalography (EEG) to understand how selected neural signals encode social cues in individual infants. EEG data from 42 6- to 9-month-old infants looking at images of their parent's face were analysed in real-time and used by a Bayesian Optimisation algorithm to identify which combination of the parent's gaze/head direction and emotional expression produces the strongest brain activation in the child. This individualised approach supported the theory that the infant's brain is maximally engaged by communicative cues with a negative valence (angry faces with direct gaze). Infants attending preferentially to faces with direct gaze had increased positive affectivity and decreased negative affectivity. This work confirmed that infants' attentional preferences for social cues are heterogeneous and shows the NBO's potential to study diversity in neurodevelopmental trajectories.

Reinforcement learning for closed-loop regulation of cardiovascular system with vagus nerve stimulation: a computational study

Objective. Vagus nerve stimulation (VNS) is being investigated as a potential therapy for cardiovascular diseases including heart failure, cardiac arrhythmia, and hypertension. The lack of a systematic approach for controlling and tuning the VNS parameters poses a significant challenge. Closed-loop VNS strategies combined with artificial intelligence (AI) approaches offer a framework for systematically learning and adapting the optimal stimulation parameters. In this study, we presented an interactive AI framework using reinforcement learning (RL) for automated data-driven design of closed-loop VNS control systems in a computational study.Approach.Multiple simulation environments with a standard application programming interface were developed to facilitate the design and evaluation of the automated data-driven closed-loop VNS control systems. These environments simulate the hemodynamic response to multi-location VNS using biophysics-based computational models of healthy and hypertensive rat cardiovascular systems in resting and exercise states. We designed and implemented the RL-based closed-loop VNS control frameworks in the context of controlling the heart rate and the mean arterial pressure for a set point tracking task. Our experimental design included two approaches; a general policy using deep RL algorithms and a sample-efficient adaptive policy using probabilistic inference for learning and control.Main results.Our simulation results demonstrated the capabilities of the closed-loop RL-based approaches to learn optimal VNS control policies and to adapt to variations in the target set points and the underlying dynamics of the cardiovascular system. Our findings highlighted the trade-off between sample-efficiency and generalizability, providing insights for proper algorithm selection. Finally, we demonstrated that transfer learning improves the sample efficiency of deep RL algorithms allowing the development of more efficient and personalized closed-loop VNS systems.Significance.We demonstrated the capability of RL-based closed-loop VNS systems. Our approach provided a systematic adaptable framework for learning control strategies without requiring prior knowledge about the underlying dynamics.

Designing optimal behavioral experiments using machine learning

Computational models are powerful tools for understanding human cognition and behavior. They let us express our theories clearly and precisely and offer predictions that can be subtle and often counter-intuitive. However, this same richness and ability to surprise means our scientific intuitions and traditional tools are ill-suited to designing experiments to test and compare these models. To avoid these pitfalls and realize the full potential of computational modeling, we require tools to design experiments that provide clear answers about what models explain human behavior and the auxiliary assumptions those models must make. Bayesian optimal experimental design (BOED) formalizes the search for optimal experimental designs by identifying experiments that are expected to yield informative data. In this work, we provide a tutorial on leveraging recent advances in BOED and machine learning to find optimal experiments for any kind of model that we can simulate data from, and show how by-products of this procedure allow for quick and straightforward evaluation of models and their parameters against real experimental data. As a case study, we consider theories of how people balance exploration and exploitation in multi-armed bandit decision-making tasks. We validate the presented approach using simulations and a real-world experiment. As compared to experimental designs commonly used in the literature, we show that our optimal designs more efficiently determine which of a set of models best account for individual human behavior, and more efficiently characterize behavior given a preferred model. At the same time, formalizing a scientific question such that it can be adequately addressed with BOED can be challenging and we discuss several potential caveats and pitfalls that practitioners should be aware of. We provide code to replicate all analyses as well as tutorial notebooks and pointers to adapt the methodology to different experimental settings.

Real-time monitoring of infant theta power during naturalistic social experiences

Infant-directed speech and direct gaze are important social cues that shape infant's attention to their parents. Traditional methods for probing their effect on infant attention involve a small number of pre-selected screen-based stimuli, which do not capture the complexity of real-world interactions. Here, we used neuroadaptive Bayesian Optimization (NBO) to search a large 'space' of different naturalistic social experiences that systematically varied in their visual (gaze direct to averted) and auditory properties (infant directed speech to nonvocal sounds). We measured oscillatory brain responses (relative theta power) during episodes of naturalistic social experiences in 57 typically developing 6- to 12-month-old infants. Relative theta power was used as input to the NBO algorithm to identify the naturalistic social context that maximally elicited attention in each individual infant. Results showed that individual infants were heterogeneous in the stimulus that elicited maximal theta with no overall stronger attention for direct gaze or infant-directed speech; however, individual differences in attention towards averted gaze were related to interpersonal skills and greater likelihood of preferring speech and direct gaze was observed in infants whose parents showed more positive affect. Our work indicates NBO may be a fruitful method for probing the role of distinct social cues in eliciting attention in naturalistic social contexts at the individual level.

Closing the loop between brain and electrical stimulation: towards precision neuromodulation treatments

One of the most critical challenges in using noninvasive brain stimulation (NIBS) techniques for the treatment of psychiatric and neurologic disorders is inter- and intra-individual variability in response to NIBS. Response variations in previous findings suggest that the one-size-fits-all approach does not seem the most appropriate option for enhancing stimulation outcomes. While there is a growing body of evidence for the feasibility and effectiveness of individualized NIBS approaches, the optimal way to achieve this is yet to be determined. Transcranial electrical stimulation (tES) is one of the NIBS techniques showing promising results in modulating treatment outcomes in several psychiatric and neurologic disorders, but it faces the same challenge for individual optimization. With new computational and methodological advances, tES can be integrated with real-time functional magnetic resonance imaging (rtfMRI) to establish closed-loop tES-fMRI for individually optimized neuromodulation. Closed-loop tES-fMRI systems aim to optimize stimulation parameters based on minimizing differences between the model of the current brain state and the desired value to maximize the expected clinical outcome. The methodological space to optimize closed-loop tES fMRI for clinical applications includes (1) stimulation vs. data acquisition timing, (2) fMRI context (task-based or resting-state), (3) inherent brain oscillations, (4) dose-response function, (5) brain target trait and state and (6) optimization algorithm. Closed-loop tES-fMRI technology has several advantages over non-individualized or open-loop systems to reshape the future of neuromodulation with objective optimization in a clinically relevant context such as drug cue reactivity for substance use disorder considering both inter and intra-individual variations. Using multi-level brain and behavior measures as input and desired outcomes to individualize stimulation parameters provides a framework for designing personalized tES protocols in precision psychiatry.

Reproducibility in Neuroimaging Analysis: Challenges and Solutions

Recent years have marked a renaissance in efforts to increase research reproducibility in psychology, neuroscience, and related fields. Reproducibility is the cornerstone of a solid foundation of fundamental research-one that will support new theories built on valid findings and technological innovation that works. The increased focus on reproducibility has made the barriers to it increasingly apparent, along with the development of new tools and practices to overcome these barriers. Here, we review challenges, solutions, and emerging best practices with a particular emphasis on neuroimaging studies. We distinguish 3 main types of reproducibility, discussing each in turn. Analytical reproducibility is the ability to reproduce findings using the same data and methods. Replicability is the ability to find an effect in new datasets, using the same or similar methods. Finally, robustness to analytical variability refers to the ability to identify a finding consistently across variation in methods. The incorporation of these tools and practices will result in more reproducible, replicable, and robust psychological and brain research and a stronger scientific foundation across fields of inquiry.

How Machine Learning is Powering Neuroimaging to Improve Brain Health

This report presents an overview of how machine learning is rapidly advancing clinical translational imaging in ways that will aid in the early detection, prediction, and treatment of diseases that threaten brain health. Towards this goal, we aresharing the information presented at a symposium, "Neuroimaging Indicators of Brain Structure and Function - Closing the Gap Between Research and Clinical Application", co-hosted by the McCance Center for Brain Health at Mass General Hospital and the MIT HST Neuroimaging Training Program on February 12, 2021. The symposium focused on the potential for machine learning approaches, applied to increasingly large-scale neuroimaging datasets, to transform healthcare delivery and change the trajectory of brain health by addressing brain care earlier in the lifespan. While not exhaustive, this overview uniquely addresses many of the technical challenges from image formation, to analysis and visualization, to synthesis and incorporation into the clinical workflow. Some of the ethical challenges inherent to this work are also explored, as are some of the regulatory requirements for implementation. We seek to educate, motivate, and inspire graduate students, postdoctoral fellows, and early career investigators to contribute to a future where neuroimaging meaningfully contributes to the maintenance of brain health.

Automated deep brain stimulation programming with safety constraints for tremor suppression in patients with Parkinson's disease and essential tremor

Objective.Deep brain stimulation (DBS) programming for movement disorders requires systematic fine tuning of stimulation parameters to ameliorate tremor and other symptoms while avoiding side effects. DBS programming can be a time-consuming process and requires clinical expertise to assess response to DBS to optimize therapy for each patient. In this study, we describe and evaluate an automated, closed-loop, and patient-specific framework for DBS programming that measures tremor using a smartwatch and automatically changes DBS parameters based on the recommendations from a closed-loop optimization algorithm thus eliminating the need for an expert clinician.Approach.Bayesian optimization which is a sample-efficient global optimization method was used as the core of this DBS programming framework to adaptively learn each patient's response to DBS and suggest the next best settings to be evaluated. Input from a clinician was used initially to define a maximum safe amplitude, but we also implemented 'safe Bayesian optimization' to automatically discover tolerable exploration boundaries.Main results.We tested the system in 15 patients (nine with Parkinson's disease and six with essential tremor). Tremor suppression at best automated settings was statistically comparable to previously established clinical settings. The optimization algorithm converged after testing15.1±0.7settings when maximum safe exploration boundaries were predefined, and17.7±4.9when the algorithm itself determined safe exploration boundaries.Significance.We demonstrate that fully automated DBS programming framework for treatment of tremor is efficient and safe while providing outcomes comparable to that achieved by expert clinicians.

Tissue volume estimation and age prediction using rapid structural brain scans

The multicontrast EPImix sequence generates six contrasts, including a T1-weighted scan, in ~1 min. EPImix shows comparable diagnostic performance to conventional scans under qualitative clinical evaluation, and similarities in simple quantitative measures including contrast intensity. However, EPImix scans have not yet been compared to standard MRI scans using established quantitative measures. In this study, we compared conventional and EPImix-derived T1-weighted scans of 64 healthy participants using tissue volume estimates and predicted brain-age. All scans were pre-processed using the SPM12 DARTEL pipeline, generating measures of grey matter, white matter and cerebrospinal fluid volume. Brain-age was predicted using brainageR, a Gaussian Processes Regression model previously trained on a large sample of standard T1-weighted scans. Estimates of both global and voxel-wise tissue volume showed significantly similar results between standard and EPImix-derived T1-weighted scans. Brain-age estimates from both sequences were significantly correlated, although EPImix T1-weighted scans showed a systematic offset in predictions of chronological age. Supplementary analyses suggest that this is likely caused by the reduced field of view of EPImix scans, and the use of a brain-age model trained using conventional T1-weighted scans. However, this systematic error can be corrected using additional regression of T1-predicted brain-age onto EPImix-predicted brain-age. Finally, retest EPImix scans acquired for 10 participants demonstrated high test-retest reliability in all evaluated quantitative measurements. Quantitative analysis of EPImix scans has potential to reduce scanning time, increasing participant comfort and reducing cost, as well as to support automation of scanning, utilising active learning for faster and individually-tailored (neuro)imaging.

A guided multiverse study of neuroimaging analyses

For most neuroimaging questions the range of possible analytic choices makes it unclear how to evaluate conclusions from any single analytic method. One possible way to address this issue is to evaluate all possible analyses using a multiverse approach, however, this can be computationally challenging and sequential analyses on the same data can compromise predictive power. Here, we establish how active learning on a low-dimensional space capturing the inter-relationships between pipelines can efficiently approximate the full spectrum of analyses. This approach balances the benefits of a multiverse analysis without incurring the cost on computational and predictive power. We illustrate this approach with two functional MRI datasets (predicting brain age and autism diagnosis) demonstrating how a multiverse of analyses can be efficiently navigated and mapped out using active learning. Furthermore, our presented approach not only identifies the subset of analysis techniques that are best able to predict age or classify individuals with autism spectrum disorder and healthy controls, but it also allows the relationships between analyses to be quantified.

Behavioral Studies Using Large-Scale Brain Networks – Methods and Validations

Mapping human behaviors to brain activity has become a key focus in modern cognitive neuroscience. As methods such as functional MRI (fMRI) advance cognitive scientists show an increasing interest in investigating neural activity in terms of functional connectivity and brain networks, rather than activation in a single brain region. Due to the noisy nature of neural activity, determining how behaviors are associated with specific neural signals is not well-established. Previous research has suggested graph theory techniques as a solution. Graph theory provides an opportunity to interpret human behaviors in terms of the topological organization of brain network architecture. Graph theory-based approaches, however, only scratch the surface of what neural connections relate to human behavior. Recently, the development of data-driven methods, e.g., machine learning and deep learning approaches, provide a new perspective to study the relationship between brain networks and human behaviors across the whole brain, expanding upon past literatures. In this review, we sought to revisit these data-driven approaches to facilitate our understanding of neural mechanisms and build models of human behaviors. We start with the popular graph theory approach and then discuss other data-driven approaches such as connectome-based predictive modeling, multivariate pattern analysis, network dynamic modeling, and deep learning techniques that quantify meaningful networks and connectivity related to cognition and behaviors. Importantly, for each topic, we discuss the pros and cons of the methods in addition to providing examples using our own data for each technique to describe how these methods can be applied to real-world neuroimaging data.

Handling EEG artifacts and searching individually optimal experimental parameter in real time: a system development and demonstration

Objective.Neuroadaptive paradigms that systematically assess event-related potential (ERP) features across many different experimental parameters have the potential to improve the generalizability of ERP findings and may help to accelerate ERP-based biomarker discovery by identifying the exact experimental conditions for which ERPs differ most for a certain clinical population. Obtaining robust and reliable ERPs online is a prerequisite for ERP-based neuroadaptive research. One of the key steps involved is to correctly isolate electroencephalography artifacts in real time because they contribute a large amount of variance that, if not removed, will greatly distort the ERP obtained. Another key factor of concern is the computational cost of the online artifact handling method. This work aims to develop and validate a cost-efficient system to support ERP-based neuroadaptive research.Approach.We developed a simple online artifact handling method, single trial PCA-based artifact removal (SPA), based on variance distribution dichotomies to distinguish between artifacts and neural activity. We then applied this method in an ERP-based neuroadaptive paradigm in which Bayesian optimization was used to search individually optimal inter-stimulus-interval (ISI) that generates ERP with the highest signal-to-noise ratio.Main results.SPA was compared to other offline and online algorithms. The results showed that SPA exhibited good performance in both computational efficiency and preservation of ERP pattern. Based on SPA, the Bayesian optimization procedure was able to quickly find individually optimal ISI.Significance.The current work presents a simple yet highly cost-efficient method that has been validated in its ability to extract ERP, preserve ERP effects, and better support ERP-based neuroadaptive paradigm.

Is it time to put rest to rest?

The so-called resting state, in which participants lie quietly with no particular inputs or outputs, represented a paradigm shift from conventional task-based studies in human neuroimaging. Our foray into rest was fruitful from both a scientific and methodological perspective, but at this point, how much more can we learn from rest on its own? While rest still dominates in many subfields, data from tasks have empirically demonstrated benefits, as well as the potential to provide insights about the mind in addition to the brain. I argue that we can accelerate progress in human neuroscience by de-emphasizing rest in favor of more grounded experiments, including promising integrated designs that respect the prominence of self-generated activity while offering enhanced control and interpretability.

Early Stopping in Experimentation With Real-Time Functional Magnetic Resonance Imaging Using a Modified Sequential Probability Ratio Test

Introduction: Functional magnetic resonance imaging (fMRI) often involves long scanning durations to ensure the associated brain activity can be detected. However, excessive experimentation can lead to many undesirable effects, such as from learning and/or fatigue effects, discomfort for the subject, excessive motion artifacts and loss of sustained attention on task. Overly long experimentation can thus have a detrimental effect on signal quality and accurate voxel activation detection. Here, we propose dynamic experimentation with real-time fMRI using a novel statistically driven approach that invokes early stopping when sufficient statistical evidence for assessing the task-related activation is observed. Methods: Voxel-level sequential probability ratio test (SPRT) statistics based on general linear models (GLMs) were implemented on fMRI scans of a mathematical 1-back task from 12 healthy teenage subjects and 11 teenage subjects born extremely preterm (EPT). This approach is based on likelihood ratios and allows for systematic early stopping based on target statistical error thresholds. We adopt a two-stage estimation approach that allows for accurate estimates of GLM parameters before stopping is considered. Early stopping performance is reported for different first stage lengths, and activation results are compared with full durations. Finally, group comparisons are conducted with both early stopped and full duration scan data. Numerical parallelization was employed to facilitate completion of computations involving a new scan within every repetition time (TR). Results: Use of SPRT demonstrates the feasibility and efficiency gains of automated early stopping, with comparable activation detection as with full protocols. Dynamic stopping of stimulus administration was achieved in around half of subjects, with typical time savings of up to 33% (4 min on a 12 min scan). A group analysis produced similar patterns of activity for control subjects between early stopping and full duration scans. The EPT group, individually, demonstrated more variability in location and extent of the activations compared to the normal term control group. This was apparent in the EPT group results, reflected by fewer and smaller clusters. Conclusion: A systematic statistical approach for early stopping with real-time fMRI experimentation has been implemented. This dynamic approach has promise for reducing subject burden and fatigue effects.

A Bayesian optimization approach for rapidly mapping residual network function in stroke

Post-stroke cognitive and linguistic impairments are debilitating conditions, with limited therapeutic options. Domain-general brain networks play an important role in stroke recovery and characterizing their residual function with functional MRI has the potential to yield biomarkers capable of guiding patient-specific rehabilitation. However, this is challenging as such detailed characterization requires testing patients on multitudes of cognitive tasks in the scanner, rendering experimental sessions unfeasibly lengthy. Thus, the current status quo in clinical neuroimaging research involves testing patients on a very limited number of tasks, in the hope that it will reveal a useful neuroimaging biomarker for the whole cohort. Given the great heterogeneity among stroke patients and the volume of possible tasks this approach is unsustainable. Advancing task-based functional MRI biomarker discovery requires a paradigm shift in order to be able to swiftly characterize residual network activity in individual patients using a diverse range of cognitive tasks. Here, we overcome this problem by leveraging neuroadaptive Bayesian optimization, an approach combining real-time functional MRI with machine-learning, by intelligently searching across many tasks, this approach rapidly maps out patient-specific profiles of residual domain-general network function. We used this technique in a cross-sectional study with 11 left-hemispheric stroke patients with chronic aphasia (four female, age ± standard deviation: 59 ± 10.9 years) and 14 healthy, age-matched control subjects (eight female, age ± standard deviation: 55.6 ± 6.8 years). To assess intra-subject reliability of the functional profiles obtained, we conducted two independent runs per subject, for which the algorithm was entirely reinitialized. Our results demonstrate that this technique is both feasible and robust, yielding reliable patient-specific functional profiles. Moreover, we show that group-level results are not representative of patient-specific results. Whereas controls have highly similar profiles, patients show idiosyncratic profiles of network abnormalities that are associated with behavioural performance. In summary, our study highlights the importance of moving beyond traditional 'one-size-fits-all' approaches where patients are treated as one group and single tasks are used. Our approach can be extended to diverse brain networks and combined with brain stimulation or other therapeutics, thereby opening new avenues for precision medicine targeting a diverse range of neurological and psychiatric conditions.

Material characterization of thin planar structures using full-field harmonic vibration response measured with stroboscopic holography

We propose a novel material characterization method to estimate the Young's modulus of thin 2-D structures using non-modal noisy single frequency harmonic vibration data measured with holography. The method uses finite-difference discretization to apply the plate equation to all measured pixels inside the boundary of the vibrating structure and then treats the problem as a Bayesian optimization process to find the value of the Young's modulus by minimizing the Euclidian distance between the measured displacement field and repeatedly calculated displacement field using the plate equation. In order to assess the accuracy of the method, ground truth harmonic displacement magnitude fields of different plates were obtained using analytical solutions and the finite-element method and were used to estimate the Young's moduli. We applied Gaussian and non-Gaussian noise with different intensities to assess the robustness and accuracy of the proposed material characterization method in the presence of noise. We demonstrated that for multiple benchmarks for signal to noise ratio of down to 0 dB, our proposed method had errors of less than 5%. We also quantified the effects of uncertainties in the geometrical and material parameters as well as boundary conditions on the estimated Young's modulus. Furthermore, we studied the effects of the mesh size on the runtime and applied the method to experimental holography vibration measurement data of a copper plate.

Human-in-the-loop optimization of hip assistance with a soft exosuit during walking

Wearable robotic devices have been shown to substantially reduce the energy expenditure of human walking. However, response variance between participants for fixed control strategies can be high, leading to the hypothesis that individualized controllers could further improve walking economy. Recent studies on human-in-the-loop (HIL) control optimization have elucidated several practical challenges, such as long experimental protocols and low signal-to-noise ratios. Here, we used Bayesian optimization-an algorithm well suited to optimizing noisy performance signals with very limited data-to identify the peak and offset timing of hip extension assistance that minimizes the energy expenditure of walking with a textile-based wearable device. Optimal peak and offset timing were found over an average of 21.4 ± 1.0 min and reduced metabolic cost by 17.4 ± 3.2% compared with walking without the device (mean ± SEM), which represents an improvement of more than 60% on metabolic reduction compared with state-of-the-art devices that only assist hip extension. In addition, our results provide evidence for participant-specific metabolic distributions with respect to peak and offset timing and metabolic landscapes, lending support to the hypothesis that individualized control strategies can offer substantial benefits over fixed control strategies. These results also suggest that this method could have practical impact on improving the performance of wearable robotic devices.

Stimulating aged brains with transcranial direct current stimulation: Opportunities and challenges

Ageing involves significant neurophysiological changes that are both systematic while at the same time exhibiting divergent trajectories across individuals. These changes underlie cognitive impairments in elderly while also affecting the response of aged brains to interventions like transcranial direct current stimulation (tDCS). While the cognitive benefits of tDCS are more variable in elderly, older adults also respond differently to stimulation protocols compared to young adults. The age-related neurophysiological changes influencing the responsiveness to tDCS remain to be addressed in-depth. We review and discuss the premise that, in comparison to the better calibrated brain networks present in young adults, aged systems perform further away from a homoeostatic set-point. We argue that this age-related neurophysiological deviation from the homoeostatic optimum extends the leeway for tDCS to modulate the aged brain. This promotes the potency of immediate tDCS effects to induce directional plastic changes towards the homoeostatic equilibrium despite the impaired plasticity induction in elderly. We also consider how age-related neurophysiological changes pose specific challenges for tDCS that necessitate proper adaptations of stimulation protocols. Appreciating the distinctive properties of aged brains and the accompanying adjustment of stimulation parameters can increase the potency and reliability of tDCS as a treatment avenue in older adults.

ADOpy: a python package for adaptive design optimization

Experimental design is fundamental to research, but formal methods to identify good designs are lacking. Advances in Bayesian statistics and machine learning offer algorithm-based ways to identify good experimental designs. Adaptive design optimization (ADO; Cavagnaro, Myung, Pitt, & Kujala, 2010; Myung, Cavagnaro, & Pitt, 2013) is one such method. It works by maximizing the informativeness and efficiency of data collection, thereby improving inference. ADO is a general-purpose method for conducting adaptive experiments on the fly and can lead to rapid accumulation of information about the phenomenon of interest with the fewest number of trials. The nontrivial technical skills required to use ADO have been a barrier to its wider adoption. To increase its accessibility to experimentalists at large, we introduce an open-source Python package, ADOpy, that implements ADO for optimizing experimental design. The package, available on GitHub, is written using high-level modular-based commands such that users do not have to understand the computational details of the ADO algorithm. In this paper, we first provide a tutorial introduction to ADOpy and ADO itself, and then illustrate its use in three walk-through examples: psychometric function estimation, delay discounting, and risky choice. Simulation data are also provided to demonstrate how ADO designs compare with other designs (random, staircase).

Artificial Intelligence, Brains, and Beyond: Imperial College London Neurotechnology Symposium, 2020

In this report, we give an overview of the proceedings from the online Imperial College London Neurotechnology Symposium 2020. The first part deals with the fundamentals of how artificial intelligence (AI) can be used to inform research frameworks used in the field of neurotechnology. The second part goes a level higher and shows how AI can be used in cutting-edge cellular and molecular methodologies and their applications. The final part focuses on the efforts to “decode” neural systems in brain-computer interfaces to advance neuroprosthetics.

Optimizing neuromodulation based on surrogate neural states for seizure suppression in a rat temporal lobe epilepsy model

OBJECTIVE

Developing a new neuromodulation method for epilepsy treatment requires a large amount of time and resources to find effective stimulation parameters and often fails due to inter-subject variability in stimulation effect. As an alternative, we present a novel data-driven surrogate approach which can optimize the neuromodulation efficiently by investigating the stimulation effect on surrogate neural states.

APPROACH

Medial septum (MS) optogenetic stimulation was applied for modulating electrophysiological activities of the hippocampus in a rat temporal lobe epilepsy model. For the new approach, we implemented machine learning techniques to describe the pathological neural states and to optimize the stimulation parameters. Specifically, first, we found neural state surrogates to estimate a seizure susceptibility based on hippocampal local field potentials. Second, we modulated the neural state surrogates in a desired way with the subject-specific optimal stimulation parameters found by in vivo Bayesian optimization. Finally, we tested whether modulating the neural state surrogates affected seizure frequency.

MAIN RESULTS

We found two neural state surrogates: The first was hippocampal theta power by considering its well-known relationship with epilepsy, and the second was the output of pre-ictal state model (PriSM) which was built by characterizing the hippocampal activity during the pre-ictal period. The optimal stimulation parameters found by Bayesian optimization outperformed the other parameters in terms of modulating the surrogates toward anti-seizure neural state. When treatment efficacy was tested, the subject-specific optimal parameters for increasing theta power were more effective to suppress seizures than fixed stimulation parameter (7 Hz). However, modulation of the other neural state surrogate, PriSM, did not suppress seizures.

SIGNIFICANCE

The surrogate approach can save enormous time and resources to find subject-specific optimal stimulation parameters which can effectively modulate neural states and further improve therapeutic effectiveness. This approach can also be used for improving neuromodulation treatment of other neurological or psychiatric diseases.

Learning State-Dependent Neural Modulation Policies with Bayesian Optimization

Neural modulation is becoming a fundamental tool for understanding and treating neurological diseases and their implicated neural circuits. Given that neural modulation interventions have high dimensional parameter spaces, one of the challenges is selecting the stimulation parameters that induce the desired effect. Moreover, the effect of a given set of stimulation parameters may change depending on the underlying neural state. In this study, we investigate and address the state-dependent effect of medial septum optogenetic stimulation on the hippocampus. We found that pre-stimulation hippocampal gamma (33-50Hz) power influences the effect of medial septum optogenetic stimulation on during-stimulation hippocampal gamma power. We then construct a simulation platform that models this phenomenon for testing optimization approaches. We then compare the performance of a standard implementation of Bayesian optimization, along with an extension to the algorithm that incorporates pre-stimulation state to learn a state-dependent policy. The state-dependent algorithm outperformed the standard approach, suggesting that incorporating pre-stimulation can improve neural modulation interventions.

Discovery of Self-Assembling π-Conjugated Peptides by Active Learning-Directed Coarse-Grained Molecular Simulation

Electronically active organic molecules have demonstrated great promise as novel soft materials for energy harvesting and transport. Self-assembled nanoaggregates formed from π-conjugated oligopeptides composed of an aromatic core flanked by oligopeptide wings offer emergent optoelectronic properties within a water-soluble and biocompatible substrate. Nanoaggregate properties can be controlled by tuning core chemistry and peptide composition, but the sequence-structure-function relations remain poorly characterized. In this work, we employ coarse-grained molecular dynamics simulations within an active learning protocol employing deep representational learning and Bayesian optimization to efficiently identify molecules capable of assembling pseudo-1D nanoaggregates with good stacking of the electronically active π-cores. We consider the DXXX-OPV3-XXXD oligopeptide family, where D is an Asp residue and OPV3 is an oligophenylenevinylene oligomer (1,4-distyrylbenzene), to identify the top performing XXX tripeptides within all 20³ = 8000 possible sequences. By direct simulation of only 2.3% of this space, we identify molecules predicted to exhibit superior assembly relative to those reported in prior work. Spectral clustering of the top candidates reveals new design rules governing assembly. This work establishes new understanding of DXXX-OPV3-XXXD assembly, identifies promising new candidates for experimental testing, and presents a computational design platform that can be generically extended to other peptide-based and peptide-like systems.

Computational optimization of associative learning experiments

With computational biology striving to provide more accurate theoretical accounts of biological systems, use of increasingly complex computational models seems inevitable. However, this trend engenders a challenge of optimal experimental design: due to the flexibility of complex models, it is difficult to intuitively design experiments that will efficiently expose differences between candidate models or allow accurate estimation of their parameters. This challenge is well exemplified in associative learning research. Associative learning theory has a rich tradition of computational modeling, resulting in a growing space of increasingly complex models, which in turn renders manual design of informative experiments difficult. Here we propose a novel method for computational optimization of associative learning experiments. We first formalize associative learning experiments using a low number of tunable design variables, to make optimization tractable. Next, we combine simulation-based Bayesian experimental design with Bayesian optimization to arrive at a flexible method of tuning design variables. Finally, we validate the proposed method through extensive simulations covering both the objectives of accurate parameter estimation and model selection. The validation results show that computationally optimized experimental designs have the potential to substantially improve upon manual designs drawn from the literature, even when prior information guiding the optimization is scarce. Computational optimization of experiments may help address recent concerns over reproducibility by increasing the expected utility of studies, and it may even incentivize practices such as study pre-registration, since optimization requires a pre-specified analysis plan. Moreover, design optimization has the potential not only to improve basic research in domains such as associative learning, but also to play an important role in translational research. For example, design of behavioral and physiological diagnostic tests in the nascent field of computational psychiatry could benefit from an optimization-based approach, similar to the one presented here.

Active Acquisition for multimodal neuroimaging

In many clinical and scientific situations the optimal neuroimaging sequence may not be known prior to scanning and may differ for each individual being scanned, depending on the exact nature and location of abnormalities. Despite this, the standard approach to data acquisition, in such situations, is to specify the sequence of neuroimaging scans prior to data acquisition and to apply the same scans to all individuals. In this paper, we propose and illustrate an alternative approach, in which data would be analysed as it is acquired and used to choose the future scanning sequence: Active Acquisition. We propose three Active Acquisition scenarios based around multiple MRI modalities. In Scenario 1, we propose a simple use of near-real time analysis to decide whether to acquire more or higher resolution data, or acquire data with a different field -of -view. In Scenario 2, we simulate how multimodal MR data could be actively acquired and combined with a decision tree to classify a known outcome variable (in the simple example here, age). In Scenario 3, we simulate using Bayesian optimisation to actively search across multiple MRI modalities to find those which are most abnormal. These simulations suggest that by actively acquiring data, the scanning sequence can be adapted to each individual. We also consider the many outstanding practical and technical challenges involving normative data acquisition, MR physics, statistical modelling and clinical relevance. Despite these, we argue that Active Acquisition allows for potentially far more powerful, sensitive or rapid data acquisition, and may open up different perspectives on individual differences, clinical conditions, and biomarker discovery.

Efficiently searching through large tACS parameter spaces using closed-loop Bayesian optimization

Background

Selecting optimal stimulation parameters from numerous possibilities is a major obstacle for assessing the efficacy of non-invasive brain stimulation.

Objective

We demonstrate that Bayesian optimization can rapidly search through large parameter spaces and identify subject-level stimulation parameters in real-time.

Methods

To validate the method, Bayesian optimization was employed using participants’ binary judgements about the intensity of phosphenes elicited through tACS.

Results

We demonstrate the efficiency of Bayesian optimization in identifying parameters that maximize phosphene intensity in a short timeframe (5 min for >190 possibilities). Our results replicate frequency-dependent effects across three montages and show phase-dependent effects of phosphene perception. Computational modelling explains that these phase effects result from constructive/destructive interference of the current reaching the retinas. Simulation analyses demonstrate the method's versatility for complex response functions, even when accounting for noisy observations.

Conclusion

Alongside subjective ratings, this method can be used to optimize tACS parameters based on behavioral and neural measures and has the potential to be used for tailoring stimulation protocols to individuals.

Parameter Inference for Computational Cognitive Models with Approximate Bayesian Computation

This paper addresses a common challenge with computational cognitive models: identifying parameter values that are both theoretically plausible and generate predictions that match well with empirical data. While computational models can offer deep explanations of cognition, they are computationally complex and often out of reach of traditional parameter fitting methods. Weak methodology may lead to premature rejection of valid models or to acceptance of models that might otherwise be falsified. Mathematically robust fitting methods are, therefore, essential to the progress of computational modeling in cognitive science. In this article, we investigate the capability and role of modern fitting methods—including Bayesian optimization and approximate Bayesian computation—and contrast them to some more commonly used methods: grid search and Nelder–Mead optimization. Our investigation consists of a reanalysis of the fitting of two previous computational models: an Adaptive Control of Thought—Rational model of skill acquisition and a computational rationality model of visual search. The results contrast the efficiency and informativeness of the methods. A key advantage of the Bayesian methods is the ability to estimate the uncertainty of fitted parameter values. We conclude that approximate Bayesian computation is (a) efficient, (b) informative, and (c) offers a path to reproducible results.

Neurofeedback helps to reveal a relationship between context reinstatement and memory retrieval

Theories of mental context and memory posit that successful mental context reinstatement enables better retrieval of memories from the same context, at the expense of memories from other contexts. To test this hypothesis, we had participants study lists of words, interleaved with task-irrelevant images from one category (e.g., scenes). Following encoding, participants were cued to mentally reinstate the context associated with a particular list, by thinking about the images that had appeared between the words. We measured context reinstatement by applying multivariate pattern classifiers to fMRI, and related this to performance on a free recall test that followed immediately afterwards. To increase sensitivity, we used a closed-loop neurofeedback procedure, whereby higher classifier evidence for the cued category elicited increased visibility of the images from the studied context onscreen. Our goal was to create a positive feedback loop that amplified small fluctuations in mental context reinstatement, making it easier to experimentally detect a relationship between context reinstatement and recall. As predicted, we found that greater amounts of classifier evidence were associated with better recall of words from the reinstated context, and worse recall of words from a different context. In a second experiment, we assessed the role of neurofeedback in identifying this brain-behavior relationship by presenting context images again and manipulating whether their visibility depended on classifier evidence. When neurofeedback was removed, the relationship between classifier evidence and memory retrieval disappeared. Together, these findings demonstrate a clear effect of context reinstatement on memory recall and suggest that neurofeedback can be a useful tool for characterizing brain-behavior relationships.

Active Acquisition for multimodal neuroimaging

In many clinical and scientific situations the optimal neuroimaging sequence may not be known prior to scanning and may differ for each individual being scanned, depending on the exact nature and location of abnormalities. Despite this, the standard approach to data acquisition, in such situations, is to specify the sequence of neuroimaging scans prior to data acquisition and to apply the same scans to all individuals. In this paper, we propose and illustrate an alternative approach, in which data would be analysed as it is acquired and used to choose the future scanning sequence: Active Acquisition. We propose three Active Acquisition scenarios based around multiple MRI modalities. In Scenario 1, we propose a simple use of near-real time analysis to decide whether to acquire more or higher resolution data, or acquire data with a different field -of -view. In Scenario 2, we simulate how multimodal MR data could be actively acquired and combined with a decision tree to classify a known outcome variable (in the simple example here, age). In Scenario 3, we simulate using Bayesian optimisation to actively search across multiple MRI modalities to find those which are most abnormal. These simulations suggest that by actively acquiring data, the scanning sequence can be adapted to each individual. We also consider the many outstanding practical and technical challenges involving normative data acquisition, MR physics, statistical modelling and clinical relevance. Despite these, we argue that  Active Acquisition allows for potentially far more powerful, sensitive or rapid data acquisition, and may open up different perspectives on individual differences, clinical conditions, and biomarker discovery.

Dissociating frontoparietal brain networks with neuroadaptive Bayesian optimization

Understanding the unique contributions of frontoparietal networks (FPN) in cognition is challenging because they overlap spatially and are co-activated by diverse tasks. Characterizing these networks therefore involves studying their activation across many different cognitive tasks, which previously was only possible with meta-analyses. Here, we use neuroadaptive Bayesian optimization, an approach combining real-time analysis of functional neuroimaging data with machine-learning, to discover cognitive tasks that segregate ventral and dorsal FPN activity. We identify and subsequently refine two cognitive tasks, Deductive Reasoning and Tower of London, which maximally dissociate the dorsal from ventral FPN. We subsequently investigate these two FPNs in the context of a wider range of FPNs and demonstrate the importance of studying the whole activity profile across tasks to uniquely differentiate any FPN. Our findings deviate from previous meta-analyses and hypothesized functional labels for these FPNs. Taken together the results form the starting point for a neurobiologically-derived cognitive taxonomy.

Bayesian Optimization for Neuroimaging Pre-processing in Brain Age Classification and Prediction

Neuroimaging-based age prediction using machine learning is proposed as a biomarker of brain aging, relating to cognitive performance, health outcomes and progression of neurodegenerative disease. However, even leading age-prediction algorithms contain measurement error, motivating efforts to improve experimental pipelines. T1-weighted MRI is commonly used for age prediction, and the pre-processing of these scans involves normalization to a common template and resampling to a common voxel size, followed by spatial smoothing. Resampling parameters are often selected arbitrarily. Here, we sought to improve brain-age prediction accuracy by optimizing resampling parameters using Bayesian optimization. Using data on N = 2003 healthy individuals (aged 16-90 years) we trained support vector machines to (i) distinguish between young (<22 years) and old (>50 years) brains (classification) and (ii) predict chronological age (regression). We also evaluated generalisability of the age-regression model to an independent dataset (CamCAN, N = 648, aged 18-88 years). Bayesian optimization was used to identify optimal voxel size and smoothing kernel size for each task. This procedure adaptively samples the parameter space to evaluate accuracy across a range of possible parameters, using independent sub-samples to iteratively assess different parameter combinations to arrive at optimal values. When distinguishing between young and old brains a classification accuracy of 88.1% was achieved, (optimal voxel size = 11.5 mm³, smoothing kernel = 2.3 mm). For predicting chronological age, a mean absolute error (MAE) of 5.08 years was achieved, (optimal voxel size = 3.73 mm³, smoothing kernel = 3.68 mm). This was compared to performance using default values of 1.5 mm³ and 4mm respectively, resulting in MAE = 5.48 years, though this 7.3% improvement was not statistically significant. When assessing generalisability, best performance was achieved when applying the entire Bayesian optimization framework to the new dataset, out-performing the parameters optimized for the initial training dataset. Our study outlines the proof-of-principle that neuroimaging models for brain-age prediction can use Bayesian optimization to derive case-specific pre-processing parameters. Our results suggest that different pre-processing parameters are selected when optimization is conducted in specific contexts. This potentially motivates use of optimization techniques at many different points during the experimental process, which may improve statistical sensitivity and reduce opportunities for experimenter-led bias.

Human-in-the-loop Bayesian optimization of wearable device parameters

The increasing capabilities of exoskeletons and powered prosthetics for walking assistance have paved the way for more sophisticated and individualized control strategies. In response to this opportunity, recent work on human-in-the-loop optimization has considered the problem of automatically tuning control parameters based on realtime physiological measurements. However, the common use of metabolic cost as a performance metric creates significant experimental challenges due to its long measurement times and low signal-to-noise ratio. We evaluate the use of Bayesian optimization-a family of sample-efficient, noise-tolerant, and global optimization methods-for quickly identifying near-optimal control parameters. To manage experimental complexity and provide comparisons against related work, we consider the task of minimizing metabolic cost by optimizing walking step frequencies in unaided human subjects. Compared to an existing approach based on gradient descent, Bayesian optimization identified a near-optimal step frequency with a faster time to convergence (12 minutes, p < 0.01), smaller inter-subject variability in convergence time (± 2 minutes, p < 0.01), and lower overall energy expenditure (p < 0.01).

Decoding Time-Varying Functional Connectivity Networks via Linear Graph Embedding Methods

An exciting avenue of neuroscientific research involves quantifying the time-varying properties of functional connectivity networks. As a result, many methods have been proposed to estimate the dynamic properties of such networks. However, one of the challenges associated with such methods involves the interpretation and visualization of high-dimensional, dynamic networks. In this work, we employ graph embedding algorithms to provide low-dimensional vector representations of networks, thus facilitating traditional objectives such as visualization, interpretation and classification. We focus on linear graph embedding methods based on principal component analysis and regularized linear discriminant analysis. The proposed graph embedding methods are validated through a series of simulations and applied to fMRI data from the Human Connectome Project.

Neuroadaptive Bayesian Optimization and Hypothesis Testing

Cognitive neuroscientists are often interested in broad research questions, yet use overly narrow experimental designs by considering only a small subset of possible experimental conditions. This limits the generalizability and reproducibility of many research findings. Here, we propose an alternative approach that resolves these problems by taking advantage of recent developments in real-time data analysis and machine learning. Neuroadaptive Bayesian optimization is a powerful strategy to efficiently explore more experimental conditions than is currently possible with standard methodology. We argue that such an approach could broaden the hypotheses considered in cognitive science, improving the generalizability of findings. In addition, Bayesian optimization can be combined with preregistration to cover exploration, mitigating researcher bias more broadly and improving reproducibility.

Automated selection of brain regions for real-time fMRI brain-computer interfaces

OBJECTIVE

Brain-computer interfaces (BCIs) implemented with real-time functional magnetic resonance imaging (rt-fMRI) use fMRI time-courses from predefined regions of interest (ROIs). To reach best performances, localizer experiments and on-site expert supervision are required for ROI definition. To automate this step, we developed two unsupervised computational techniques based on the general linear model (GLM) and independent component analysis (ICA) of rt-fMRI data, and compared their performances on a communication BCI. Approach. 3 T fMRI data of six volunteers were re-analyzed in simulated real-time. During a localizer run, participants performed three mental tasks following visual cues. During two communication runs, a letter-spelling display guided the subjects to freely encode letters by performing one of the mental tasks with a specific timing. GLM- and ICA-based procedures were used to decode each letter, respectively using compact ROIs and whole-brain distributed spatio-temporal patterns of fMRI activity, automatically defined from subject-specific or group-level maps.

MAIN RESULTS

Letter-decoding performances were comparable to supervised methods. In combination with a similarity-based criterion, GLM- and ICA-based approaches successfully decoded more than 80% (average) of the letters. Subject-specific maps yielded optimal performances. Significance. Automated solutions for ROI selection may help accelerating the translation of rt-fMRI BCIs from research to clinical applications.

Real-time estimation of dynamic functional connectivity networks

Two novel and exciting avenues of neuroscientific research involve the study of task-driven dynamic reconfigurations of functional connectivity networks and the study of functional connectivity in real-time. While the former is a well-established field within neuroscience and has received considerable attention in recent years, the latter remains in its infancy. To date, the vast majority of real-time fMRI studies have focused on a single brain region at a time. This is due in part to the many challenges faced when estimating dynamic functional connectivity networks in real-time. In this work, we propose a novel methodology with which to accurately track changes in time-varying functional connectivity networks in real-time. The proposed method is shown to perform competitively when compared to state-of-the-art offline algorithms using both synthetic as well as real-time fMRI data. The proposed method is applied to motor task data from the Human Connectome Project as well as to data obtained from a visuospatial attention task. We demonstrate that the algorithm is able to accurately estimate task-related changes in network structure in real-time. Hum Brain Mapp 38:202-220, 2017. © 2016 Wiley Periodicals, Inc.

To work or not to work: Neural representation of cost and benefit of instrumental action

By definition, instrumental actions are performed in order to obtain certain goals. Nevertheless, the attainment of goals typically implies obstacles, and response vigor is known to reflect an integration of subjective benefit and cost. Whereas several brain regions have been associated with cost/benefit ratio decision-making, trial-by-trial fluctuations in motivation are not well understood. We review recent evidence supporting the motivational implications of signal fluctuations in the mesocorticolimbic system. As an extension of "set-point" theories of instrumental action, we propose that response vigor is determined by a rapid integration of brain signals that reflect value and cost on a trial-by-trial basis giving rise to an online estimate of utility. Critically, we posit that fluctuations in key nodes of the network can predict deviations in response vigor and that variability in instrumental behavior can be accounted for by models devised from optimal control theory, which incorporate the effortful control of noise. Notwithstanding, the post hoc analysis of signaling dynamics has caveats that can effectively be addressed in future research with the help of two novel fMRI imaging techniques. First, adaptive fMRI paradigms can be used to establish a time-order relationship, which is a prerequisite for causality, by using observed signal fluctuations as triggers for stimulus presentation. Second, real-time fMRI neurofeedback can be employed to induce predefined brain states that may facilitate benefit or cost aspects of instrumental actions. Ultimately, understanding temporal dynamics in brain networks subserving response vigor holds the promise for targeted interventions that could help to readjust the motivational balance of behavior.

Active SAmpling Protocol (ASAP) to Optimize Individual Neurocognitive Hypothesis Testing: A BCI-Inspired Dynamic Experimental Design

The relatively young field of Brain-Computer Interfaces has promoted the use of electrophysiology and neuroimaging in real-time. In the meantime, cognitive neuroscience studies, which make extensive use of functional exploration techniques, have evolved toward model-based experiments and fine hypothesis testing protocols. Although these two developments are mostly unrelated, we argue that, brought together, they may trigger an important shift in the way experimental paradigms are being designed, which should prove fruitful to both endeavors. This change simply consists in using real-time neuroimaging in order to optimize advanced neurocognitive hypothesis testing. We refer to this new approach as the instantiation of an Active SAmpling Protocol (ASAP). As opposed to classical (static) experimental protocols, ASAP implements online model comparison, enabling the optimization of design parameters (e.g., stimuli) during the course of data acquisition. This follows the well-known principle of sequential hypothesis testing. What is radically new, however, is our ability to perform online processing of the huge amount of complex data that brain imaging techniques provide. This is all the more relevant at a time when physiological and psychological processes are beginning to be approached using more realistic, generative models which may be difficult to tease apart empirically. Based upon Bayesian inference, ASAP proposes a generic and principled way to optimize experimental design adaptively. In this perspective paper, we summarize the main steps in ASAP. Using synthetic data we illustrate its superiority in selecting the right perceptual model compared to a classical design. Finally, we briefly discuss its future potential for basic and clinical neuroscience as well as some remaining challenges.