Rapid detection of neurons in widefield calcium imaging datasets after training with synthetic data

Neurophotonics beyond the surface: unmasking the brain's complexity exploiting optical scattering

The intricate nature of the brain necessitates the application of advanced probing techniques to comprehensively study and understand its working mechanisms. Neurophotonics offers minimally invasive methods to probe the brain using optics at cellular and even molecular levels. However, multiple challenges persist, especially concerning imaging depth, field of view, speed, and biocompatibility. A major hindrance to solving these challenges in optics is the scattering nature of the brain. This perspective highlights the potential of complex media optics, a specialized area of study focused on light propagation in materials with intricate heterogeneous optical properties, in advancing and improving neuronal readouts for structural imaging and optical recordings of neuronal activity. Key strategies include wavefront shaping techniques and computational imaging and sensing techniques that exploit scattering properties for enhanced performance. We discuss the potential merger of the two fields as well as potential challenges and perspectives toward longer term in vivo applications.

Accurate neuron segmentation method for one-photon calcium imaging videos combining convolutional neural networks and clustering

One-photon fluorescent calcium imaging helps understand brain functions by recording large-scale neural activities in freely moving animals. Automatic, fast, and accurate active neuron segmentation algorithms are essential to extract and interpret information from these videos. One-photon imaging videos' low resolution, high noise, and high background fluctuation pose significant challenges. Here, we develop a software pipeline to address the challenges of processing one-photon calcium imaging videos. We extend our previous two-photon active neuron segmentation algorithm, Shallow U-Net Neuron Segmentation (SUNS), to better suppress background fluctuations in one-photon videos. We also develop additional neuron extraction (ANE) to locate small or dim neurons missed by SUNS. To train our segmentation method, we create ground truth neurons by developing a manual labeling pipeline assisted with semi-automatic refinement. Our method is more accurate and faster than state-of-the-art techniques when processing simulated videos and multiple experimental datasets acquired over various brain regions with different imaging conditions.

CADENCE - Neuroinformatics Tool for Supervised Calcium Events Detection

CADENCE is an open Python 3-written neuroinformatics tool with Qt6 graphic user interface for supervised calcium events detection. In neuronal ensembles recording during calcium imaging experiments, the output of instruments such as Celena X, Zeiss LSM 5 Live confocal microscope and Miniscope is a movie showing flashing cells somata. There are few pipelines to convert video to relative fluorescence ΔF/F, from simplest ImageJ plugins to sophisticated tools like MiniAn (Dong et al. in Elife 11, https://doi.org/10.7554/eLife.70661 , 2022). Minian, an open-source miniscope analysis pipeline. Elife, 11.). While in some areas of study relative fluorescence ΔF/F may be the desired result in itself, researchers of neuronal ensembles are typically interested in a more detailed analysis of calcium events as indirect proxy of neuronal electrical activity. For such analyses, researchers need a tool to infer calcium events from the continuous ΔF/F curve in order to create a raster representation of calcium events for later use in analysis software, such as Elephant (Denker, M., Yegenoglu, A., & Grün, S. (2018). Collaborative HPC-enabled workflows on the HBP Collaboratory using the Elephant framework. Neuroinformatics, 19.). Here we present such an open tool with supervised calcium events detection.

Multifocal fluorescence video-rate imaging of centimetre-wide arbitrarily shaped brain surfaces at micrometric resolution

Fluorescence microscopy allows for the high-throughput imaging of cellular activity across brain areas in mammals. However, capturing rapid cellular dynamics across the curved cortical surface is challenging, owing to trade-offs in image resolution, speed, field of view and depth of field. Here we report a technique for wide-field fluorescence imaging that leverages selective illumination and the integration of focal areas at different depths via a spinning disc with varying thickness to enable video-rate imaging of previously reconstructed centimetre-scale arbitrarily shaped surfaces at micrometre-scale resolution and at a depth of field of millimetres. By implementing the technique in a microscope capable of acquiring images at 1.68 billion pixels per second and resolving 16.8 billion voxels per second, we recorded neural activities and the trajectories of neutrophils in real time on curved cortical surfaces in live mice. The technique can be integrated into many microscopes and macroscopes, in both reflective and fluorescence modes, for the study of multiscale cellular interactions on arbitrarily shaped surfaces.

A miniaturized mesoscope for the large-scale single-neuron-resolved imaging of neuronal activity in freely behaving mice

Exploring the relationship between neuronal dynamics and ethologically relevant behaviour involves recording neuronal-population activity using technologies that are compatible with unrestricted animal behaviour. However, head-mounted microscopes that accommodate weight limits to allow for free animal behaviour typically compromise field of view, resolution or depth range, and are susceptible to movement-induced artefacts. Here we report a miniaturized head-mounted fluorescent mesoscope that we systematically optimized for calcium imaging at single-neuron resolution, for increased fields of view and depth of field, and for robustness against motion-generated artefacts. Weighing less than 2.5 g, the mesoscope enabled recordings of neuronal-population activity at up to 16 Hz, with 4 μm resolution over 300 μm depth-of-field across a field of view of 3.6 × 3.6 mm2 in the cortex of freely moving mice. We used the mesoscope to record large-scale neuronal-population activity in socially interacting mice during free exploration and during fear-conditioning experiments, and to investigate neurovascular coupling across multiple cortical regions.

Injectable Fluorescent Neural Interfaces for Cell-Specific Stimulating and Imaging

Building on current explorations in chronic optical neural interfaces, it is essential to address the risk of photothermal damage in traditional optogenetics. By focusing on calcium fluorescence for imaging rather than stimulation, injectable fluorescent neural interfaces significantly minimize photothermal damage and improve the accuracy of neuronal imaging. Key advancements including the use of injectable microelectronics for targeted electrical stimulation and their integration with cell-specific genetically encoded calcium indicators have been discussed. These injectable electronics that allow for post-treatment retrieval offer a minimally invasive solution, enhancing both usability and reliability. Furthermore, the integration of genetically encoded fluorescent calcium indicators with injectable bioelectronics enables precise neuronal recording and imaging of individual neurons. This shift not only minimizes risks such as photothermal conversion but also boosts safety, specificity, and effectiveness of neural imaging. Embracing these advancements represents a significant leap forward in biomedical engineering and neuroscience, paving the way for advanced brain-machine interfaces.

Neurophotonics beyond the Surface: Unmasking the Brain's Complexity Exploiting Optical Scattering

The intricate nature of the brain necessitates the application of advanced probing techniques to comprehensively study and understand its working mechanisms. Neurophotonics offers minimally invasive methods to probe the brain using optics at cellular and even molecular levels. However, multiple challenges persist, especially concerning imaging depth, field of view, speed, and biocompatibility. A major hindrance to solving these challenges in optics is the scattering nature of the brain. This perspective highlights the potential of complex media optics, a specialized area of study focused on light propagation in materials with intricate heterogeneous optical properties, in advancing and improving neuronal readouts for structural imaging and optical recordings of neuronal activity. Key strategies include wavefront shaping techniques and computational imaging and sensing techniques that exploit scattering properties for enhanced performance. We discuss the potential merger of the two fields as well as potential challenges and perspectives toward longer term in vivo applications.

Sequentially activated discrete modules appear as traveling waves in neuronal measurements with limited spatiotemporal sampling

Numerous studies have identified traveling waves in the cortex and suggested they play important roles in brain processing. These waves are most often measured using macroscopic methods that are unable to assess the local spiking activity underlying wave dynamics. Here, we investigated the possibility that waves may not be traveling at the single neuron scale. We first show that sequentially activating two discrete brain areas can appear as traveling waves in EEG simulations. We next reproduce these results using an analytical model of two sequentially activated regions. Using this model, we were able to generate wave-like activity with variable directions, velocities, and spatial patterns, and to map the discriminability limits between traveling waves and modular sequential activations. Finally, we investigated the link between field potentials and single neuron excitability using large-scale measurements from turtle cortex ex vivo. We found that while field potentials exhibit wave-like dynamics, the underlying spiking activity was better described by consecutively activated spatially adjacent groups of neurons. Taken together, this study suggests caution when interpreting phase delay measurements as continuously propagating wavefronts in two different spatial scales. A careful distinction between modular and wave excitability profiles across scales will be critical for understanding the nature of cortical computations.

Robust single-shot 3D fluorescence imaging in scattering media with a simulator-trained neural network

Imaging through scattering is a pervasive and difficult problem in many biological applications. The high background and the exponentially attenuated target signals due to scattering fundamentally limits the imaging depth of fluorescence microscopy. Light-field systems are favorable for high-speed volumetric imaging, but the 2D-to-3D reconstruction is fundamentally ill-posed, and scattering exacerbates the condition of the inverse problem. Here, we develop a scattering simulator that models low-contrast target signals buried in heterogeneous strong background. We then train a deep neural network solely on synthetic data to descatter and reconstruct a 3D volume from a single-shot light-field measurement with low signal-to-background ratio (SBR). We apply this network to our previously developed computational miniature mesoscope and demonstrate the robustness of our deep learning algorithm on scattering phantoms with different scattering conditions. The network can robustly reconstruct emitters in 3D with a 2D measurement of SBR as low as 1.05 and as deep as a scattering length. We analyze fundamental tradeoffs based on network design factors and out-of-distribution data that affect the deep learning model's generalizability to real experimental data. Broadly, we believe that our simulator-based deep learning approach can be applied to a wide range of imaging through scattering techniques where experimental paired training data is lacking.

A Semi-supervised Pipeline for Accurate Neuron Segmentation with Fewer Ground Truth Labels

Recent advancements in two-photon calcium imaging have enabled scientists to record the activity of thousands of neurons with cellular resolution. This scope of data collection is crucial to understanding the next generation of neuroscience questions, but analyzing these large recordings requires automated methods for neuron segmentation. Supervised methods for neuron segmentation achieve state of-the-art accuracy and speed but currently require large amounts of manually generated ground truth training labels. We reduced the required number of training labels by designing a semi-supervised pipeline. Our pipeline used neural network ensembling to generate pseudolabels to train a single shallow U-Net. We tested our method on three publicly available datasets and compared our performance to three widely used segmentation methods. Our method outperformed other methods when trained on a small number of ground truth labels and could achieve state-of-the-art accuracy after training on approximately a quarter of the number of ground truth labels as supervised methods. When trained on many ground truth labels, our pipeline attained higher accuracy than that of state-of-the-art methods. Overall, our work will help researchers accurately process large neural recordings while minimizing the time and effort needed to generate manual labels.

Robust single-shot 3D fluorescence imaging in scattering media with a simulator-trained neural network

Imaging through scattering is a pervasive and difficult problem in many biological applications. The high background and the exponentially attenuated target signals due to scattering fundamentally limits the imaging depth of fluorescence microscopy. Light-field systems are favorable for high-speed volumetric imaging, but the 2D-to-3D reconstruction is fundamentally ill-posed, and scattering exacerbates the condition of the inverse problem. Here, we develop a scattering simulator that models low-contrast target signals buried in heterogeneous strong background. We then train a deep neural network solely on synthetic data to descatter and reconstruct a 3D volume from a single-shot light-field measurement with low signal-to-background ratio (SBR). We apply this network to our previously developed Computational Miniature Mesoscope and demonstrate the robustness of our deep learning algorithm on scattering phantoms with different scattering conditions. The network can robustly reconstruct emitters in 3D with a 2D measurement of SBR as low as 1.05 and as deep as a scattering length. We analyze fundamental tradeoffs based on network design factors and out-of-distribution data that affect the deep learning model's generalizability to real experimental data. Broadly, we believe that our simulator-based deep learning approach can be applied to a wide range of imaging through scattering techniques where experimental paired training data is lacking.

Functional neuroimaging as a catalyst for integrated neuroscience

Functional magnetic resonance imaging (fMRI) enables non-invasive access to the awake, behaving human brain. By tracking whole-brain signals across a diverse range of cognitive and behavioural states or mapping differences associated with specific traits or clinical conditions, fMRI has advanced our understanding of brain function and its links to both normal and atypical behaviour. Despite this headway, progress in human cognitive neuroscience that uses fMRI has been relatively isolated from rapid advances in other subdomains of neuroscience, which themselves are also somewhat siloed from one another. In this Perspective, we argue that fMRI is well-placed to integrate the diverse subfields of systems, cognitive, computational and clinical neuroscience. We first summarize the strengths and weaknesses of fMRI as an imaging tool, then highlight examples of studies that have successfully used fMRI in each subdomain of neuroscience. We then provide a roadmap for the future advances that will be needed to realize this integrative vision. In this way, we hope to demonstrate how fMRI can help usher in a new era of interdisciplinary coherence in neuroscience.

Dependence of ghost on the incident light angle into dichroic mirror

The dichroic mirror (DM) is a key component in microscope. We found a ghost in the reflection channel of a dual-channel fluorescence microscope and studied the relationship between the ghost and the incidence angle θ into the DM. The DM emission surface reflection generated ghost if the θ is not45 ° . We analyzed the distance and intensity relationship between the ghost and the primary image, which is θ -dependent and was demonstrated by imaging live cells and a stage micrometer. The ghost can be eliminated by placing the DM between objective and tube lens, but not between tube lens and detector, ensuring that the incident light into the DM is approximately parallel. Furthermore, the transmitted light of the DM is shifted towards a longer wavelength with increasing θ . Collectively, microscopists must carefully optimize the θ when designing a microscope to avoid the ghost.

Addressing annotation and data scarcity when designing machine learning strategies for neurophotonics

Machine learning has revolutionized the way data are processed, allowing information to be extracted in a fraction of the time it would take an expert. In the field of neurophotonics, machine learning approaches are used to automatically detect and classify features of interest in complex images. One of the key challenges in applying machine learning methods to the field of neurophotonics is the scarcity of available data and the complexity associated with labeling them, which can limit the performance of data-driven algorithms. We present an overview of various strategies, such as weakly supervised learning, active learning, and domain adaptation that can be used to address the problem of labeled data scarcity in neurophotonics. We provide a comprehensive overview of the strengths and limitations of each approach and discuss their potential applications to bioimaging datasets. In addition, we highlight how different strategies can be combined to increase model performance on those datasets. The approaches we describe can help to improve the accessibility of machine learning-based analysis with limited number of annotated images for training and can enable researchers to extract more meaningful insights from small datasets.

Improved Workflow for Analysis of Vascular Myocyte Time-Series and Line-Scan Ca2+ Imaging Datasets

Intracellular Ca2+ signals are key for the regulation of cellular processes ranging from myocyte contraction, hormonal secretion, neural transmission, cellular metabolism, transcriptional regulation, and cell proliferation. Measurement of cellular Ca2+ is routinely performed using fluorescence microscopy with biological indicators. Analysis of deterministic signals is reasonably straightforward as relevant data can be discriminated based on the timing of cellular responses. However, analysis of stochastic, slower oscillatory events, as well as rapid subcellular Ca2+ responses, takes considerable time and effort which often includes visual analysis by trained investigators, especially when studying signals arising from cells embedded in complex tissues. The purpose of the current study was to determine if full-frame time-series and line-scan image analysis workflow of Fluo-4 generated Ca2+ fluorescence data from vascular myocytes could be automated without introducing errors. This evaluation was addressed by re-analyzing a published "gold standard" full-frame time-series dataset through visual analysis of Ca2+ signals from recordings made in pulmonary arterial myocytes of en face arterial preparations. We applied a combination of data driven and statistical approaches with comparisons to our published data to assess the fidelity of the various approaches. Regions of interest with Ca2+ oscillations were detected automatically post hoc using the LCPro plug-in for ImageJ. Oscillatory signals were separated based on event durations between 4 and 40 s. These data were filtered based on cutoffs obtained from multiple methods and compared to the published manually curated "gold standard" dataset. Subcellular focal and rapid Ca2+ "spark" events from line-scan recordings were examined using SparkLab 5.8, which is a custom automated detection and analysis program. After filtering, the number of true positives, false positives, and false negatives were calculated through comparisons to visually derived "gold standard" datasets. Positive predictive value, sensitivity, and false discovery rates were calculated. There were very few significant differences between the automated and manually curated results with respect to quality of the oscillatory and Ca2+ spark events, and there were no systematic biases in the data curation or filtering techniques. The lack of statistical difference in event quality between manual data curation and statistically derived critical cutoff techniques leads us to believe that automated analysis techniques can be reliably used to analyze spatial and temporal aspects to Ca2+ imaging data, which will improve experiment workflow.