Simultaneous two-photon calcium imaging at different depths with spatiotemporal multiplexing

"There's plenty of room at the bottom": deep brain imaging with holographic endo-microscopy

Significance

Over more than 300 years, microscopic imaging keeps providing fundamental insights into the mechanisms of living organisms. Seeing microscopic structures beyond the reach of free-space light-based microscopy, however, requires dissection of the tissue-an intervention seriously disturbing its physiological functions. The hunt for low-invasiveness tools has led a growing community of physicists and engineers into the realm of complex media photonics. One of its activities represents exploiting multimode optical fibers (MMFs) as ultra-thin endoscopic probes. Employing wavefront shaping, these tools only recently facilitated the first peeks at cells and their sub-cellular compartments at the bottom of the mouse brain with the impact of micro-scale tissue damage.

Aim

Here, we aim to highlight advances in MMF-based holographic endo-microscopy facilitating microscopic imaging throughout the whole depth of the mouse brain.

Approach

We summarize the important technical and methodological prerequisites for stabile high-resolution imaging in vivo.

Results

We showcase images of the microscopic building blocks of brain tissue, including neurons, neuronal processes, vessels, intracellular calcium signaling, and red blood cell velocity in individual vessels.

Conclusions

This perspective article helps to understand the complexity behind the technology of holographic endo-microscopy, summarizes its recent advances and challenges, and stimulates the mind of the reader for further exploitation of this tool in the neuroscience research.

Optical constraints on two-photon voltage imaging

Significance

Genetically encoded voltage indicators (GEVIs) are a valuable tool for studying neural circuits in vivo, but the relative merits and limitations of one-photon (1P) versus two-photon (2P) voltage imaging are not well characterized.

Aim

We consider the optical and biophysical constraints particular to 1P and 2P voltage imaging and compare the imaging properties of commonly used GEVIs under 1P and 2P excitation.

Approach

We measure the brightness and voltage sensitivity of voltage indicators from commonly used classes under 1P and 2P illumination. We also measure the decrease in fluorescence as a function of depth in the mouse brain. We develop a simple model of the number of measurable cells as a function of reporter properties, imaging parameters, and desired signal-to-noise ratio (SNR). We then discuss how the performance of voltage imaging would be affected by sensor improvements and by recently introduced advanced imaging modalities.

Results

Compared with 1P excitation, 2P excitation requires ∼ 10 4 -fold more illumination power per cell to produce similar photon count rates. For voltage imaging with JEDI-2P in the mouse cortex with a target SNR of 10 (spike height to baseline shot noise), a measurement bandwidth of 1 kHz, a thermally limited laser power of 200 mW, and an imaging depth of > 300    μ m , 2P voltage imaging using an 80-MHz source can record from no more than ∼ 12 neurons simultaneously.

Conclusions

Due to the stringent photon-count requirements of voltage imaging and the modest voltage sensitivity of existing reporters, 2P voltage imaging in vivo faces a stringent tradeoff between shot noise and tissue photodamage. 2P imaging of hundreds of neurons with high SNR at a depth of > 300    μ m will require either major improvements in 2P GEVIs or qualitatively new approaches to imaging.

Simultaneous removal of noise and correction of motion warping in neuron calcium imaging using a pipeline structure of self-supervised deep learning models

Calcium imaging is susceptible to motion distortions and background noises, particularly for monitoring active animals under low-dose laser irradiation, and hence unavoidably hinder the critical analysis of neural functions. Current research efforts tend to focus on either denoising or dewarping and do not provide effective methods for videos distorted by both noises and motion artifacts simultaneously. We found that when a self-supervised denoising model of DeepCAD [Nat. Methods18, 1359 (2021)10.1038/s41592-021-01225-0] is used on the calcium imaging contaminated by noise and motion warping, it can remove the motion artifacts effectively but with regenerated noises. To address this issue, we develop a two-level deep-learning (DL) pipeline to dewarp and denoise the calcium imaging video sequentially. The pipeline consists of two 3D self-supervised DL models that do not require warp-free and high signal-to-noise ratio (SNR) observations for network optimization. Specifically, a high-frequency enhancement block is presented in the denoising network to restore more structure information in the denoising process; a hierarchical perception module and a multi-scale attention module are designed in the dewarping network to tackle distortions of various sizes. Experiments conducted on seven videos from two-photon and confocal imaging systems demonstrate that our two-level DL pipeline can restore high-clarity neuron images distorted by both motion warping and background noises. Compared to typical DeepCAD, our denoising model achieves a significant improvement of approximately 30% in image resolution and up to 28% in signal-to-noise ratio; compared to traditional dewarping and denoising methods, our proposed pipeline network recovers more neurons, enhancing signal fidelity and improving data correlation among frames by 35% and 60% respectively. This work may provide an attractive method for long-term neural activity monitoring in awake animals and also facilitate functional analysis of neural circuits.

Removing crosstalk signals in neuron activity by time multiplexed excitations in a two-photon all-optical physiology system

The two-photon all-optical physiology system has attracted great interest in deciphering neuronal circuits in vivo, benefiting from its advantages in recording and modulating neuronal activities at single neuron resolutions. However, the interference, or crosstalk, between the imaging and photostimulation beams introduces a significant challenge and may impede the future application of voltage indicators in two-photon all-optical physiology system. Here, we propose the time multiplexed excitation method to distinguish signals from neuronal activities and crosstalks from photostimulation. In our system, the laser pulses of the imaging beam and photostimulation beam are synchronized, and a time delay is introduced into these pulses to separate the fluorescence signal generated by these two beams. We demonstrate the efficacy of our system in eliminating crosstalk signals from photostimulation and evaluate its influence on both genetically encoded calcium indicators (GECIs) and genetically encoded voltage indicators (GEVIs) through in vivo experiments.

Fast topographic optical imaging using encoded search focal scan

A central quest in optics is to rapidly extract quantitative information from a sample. Existing topographical imaging tools allow non-contact and three-dimensional measurements at the micro and nanoscales and are essential in applications including precision engineering and optical quality control. However, these techniques involve acquiring a focal stack of images, a time-consuming process that prevents measurement of moving samples. Here, we propose a method for increasing the speed of topographic imaging by orders of magnitude. Our approach involves collecting a reduced set of images, each integrated during the full focal scan, whilst the illumination is synchronously modulated during exposure. By properly designing the modulation sequence for each image, unambiguous reconstruction of the object height map is achieved using far fewer images than conventional methods. We describe the theoretical foundations of our technique, characterise its performance, and demonstrate sub-micrometric topographic imaging over 100 µm range of static and dynamic systems at rates as high as 67 topographies per second, limited by the camera frame rate. The high speed of the technique and its ease of implementation could enable a paradigm shift in optical metrology, allowing the real-time characterisation of large or rapidly moving samples.

Graded arc beam in light needle microscopy for axially resolved, rapid volumetric imaging without nonlinear processes

High-speed three-dimensional (3D) imaging is essential for revealing the structure and functions of biological specimens. Confocal laser scanning microscopy has been widely employed for this purpose. However, it requires a time-consuming image-stacking procedure. As a solution, we previously developed light needle microscopy using a Bessel beam with a wavefront-engineered approach [Biomed. Opt. Express13, 1702 (2022)10.1364/BOE.449329]. However, this method applies only to multiphoton excitation microscopy because of the requirement to reduce the sidelobes of the Bessel beam. Here, we introduce a beam that produces a needle spot while eluding the intractable artifacts due to the sidelobes. This beam can be adopted even in one-photon excitation fluorescence 3D imaging. The proposed method can achieve real-time, rapid 3D observation of 200-nm particles in water at a rate of over 50 volumes per second. In addition, fine structures, such as the spines of neurons in fixed mouse brain tissue, can be visualized in 3D from a single raster scan of the needle spot. The proposed method can be applied to various modalities in biological imaging, enabling rapid 3D image acquisition.

Optimal real-time resonant scanner linearization using filtered Hermite interpolation

High-speed laser scanning microscopy frequently relies on resonant scanners due to their order of magnitude increase in imaging rate compared to conventional galvanometer scanners. However, the use of a nonlinear scan trajectory introduces distortion that must be corrected. This manuscript derives a new algorithm based on filtered Hermite polynomial interpolation that provides the optimal shot-noise-limited SNR for a fixed number of photons and provides higher spatial accuracy than previous methods. An open-source library is presented using the Intel advanced vector instruction set (AVX) to process up to 32 samples in parallel. Using this approach, I simultaneously demonstrate lower shot noise variance, moderately higher spatial accuracy and greater than 1 gigapixel per second interpolation rate on a desktop CPU.

High-speed low-light in vivo two-photon voltage imaging of large neuronal populations

Monitoring spiking activity across large neuronal populations at behaviorally relevant timescales is critical for understanding neural circuit function. Unlike calcium imaging, voltage imaging requires kilohertz sampling rates that reduce fluorescence detection to near shot-noise levels. High-photon flux excitation can overcome photon-limited shot noise, but photobleaching and photodamage restrict the number and duration of simultaneously imaged neurons. We investigated an alternative approach aimed at low two-photon flux, which is voltage imaging below the shot-noise limit. This framework involved developing positive-going voltage indicators with improved spike detection (SpikeyGi and SpikeyGi2); a two-photon microscope ('SMURF') for kilohertz frame rate imaging across a 0.4 mm × 0.4 mm field of view; and a self-supervised denoising algorithm (DeepVID) for inferring fluorescence from shot-noise-limited signals. Through these combined advances, we achieved simultaneous high-speed deep-tissue imaging of more than 100 densely labeled neurons over 1 hour in awake behaving mice. This demonstrates a scalable approach for voltage imaging across increasing neuronal populations.

Tutorial: multiphoton microscopy to advance neuroscience research

Multiphoton microscopy (MPM) employs ultrafast infrared lasers for high-resolution deep three-dimensional imaging of live biological samples. The goal of this tutorial is to provide a practical guide to MPM imaging for novice microscopy developers and life-science users. Principles of MPM, microscope setup, and labeling strategies are discussed. Use of MPM to achieve unprecedented imaging depth of whole mounted explants and intravital imaging via implantable glass windows of the mammalian nervous system is demonstrated.

Volumetric Imaging of Neural Activity by Light Field Microscopy

Recording the highly diverse and dynamic activities in large populations of neurons in behaving animals is crucial for a better understanding of how the brain works. To meet this challenge, extensive efforts have been devoted to developing functional fluorescent indicators and optical imaging techniques to optically monitor neural activity. Indeed, optical imaging potentially has extremely high throughput due to its non-invasive access to large brain regions and capability to sample neurons at high density, but the readout speed, such as the scanning speed in two-photon scanning microscopy, is often limited by various practical considerations. Among different imaging methods, light field microscopy features a highly parallelized 3D fluorescence imaging scheme and therefore promises a novel and faster strategy for functional imaging of neural activity. Here, we briefly review the working principles of various types of light field microscopes and their recent developments and applications in neuroscience studies. We also discuss strategies and considerations of optimizing light field microscopy for different experimental purposes, with illustrative examples in imaging zebrafish and mouse brains.

Optical gearbox enabled versatile multiscale high-throughput multiphoton functional imaging

To understand the function and mechanism of biological systems, it is crucial to observe the cellular dynamics at high spatiotemporal resolutions within live animals. The recent advances in genetically encoded function indicators have significantly improved the response rate to a near millisecond time scale. However, the widely employed in vivo imaging systems often lack the temporal solution to capture the fast biological dynamics. To broadly enable the capability of high-speed in vivo deep-tissue imaging, we developed an optical gearbox. As an add-on module, the optical gearbox can convert the common multiphoton imaging systems for versatile multiscale high-throughput imaging applications. In this work, we demonstrate in vivo 2D and 3D function imaging in mammalian brains at frame rates ranging from 50 to 1000 Hz. The optical gearbox's versatility and compatibility with the widely employed imaging components will be highly valuable to a variety of deep tissue imaging applications.

Two-photon calcium imaging of neuronal activity

In vivo two-photon calcium imaging (2PCI) is a technique used for recording neuronal activity in the intact brain. It is based on the principle that, when neurons fire action potentials, intracellular calcium levels rise, which can be detected using fluorescent molecules that bind to calcium. This Primer is designed for scientists who are considering embarking on experiments with 2PCI. We provide the reader with a background on the basic concepts behind calcium imaging and on the reasons why 2PCI is an increasingly powerful and versatile technique in neuroscience. The Primer explains the different steps involved in experiments with 2PCI, provides examples of what ideal preparations should look like and explains how data are analysed. We also discuss some of the current limitations of the technique, and the types of solutions to circumvent them. Finally, we conclude by anticipating what the future of 2PCI might look like, emphasizing some of the analysis pipelines that are being developed and international efforts for data sharing.

Depth random-access two-photon Bessel light-sheet imaging in brain tissue

Two-photon light-sheet fluorescence microscopy enables high-resolution imaging of neural activity in brain tissue at a high frame rate. Traditionally, light-sheet microscopy builds up a 3D stack by multiple depth scans with uniform spatial intervals, which substantially limits the volumetric imaging speed. Here, we introduce the depth random-access light-sheet microscopy, allowing rapid switching scanning depth for light-sheet imaging. With a low-cost electrically tunable lens and minimum modification of an existing two-photon light-sheet imaging instrument, we demonstrated fast random depth hopping light-sheet imaging at 100 frames per second in the live brain slice. Through depth random-access, calcium activities for an astrocyte were recorded on four user-selected detection planes at a refreshing rate of 25 Hz.

Generation of A Space-Variant Vector Beam with Catenary-Shaped Polarization States

We demonstrate the generation of a space-variant vector beam with catenary-shaped polarization states based on the polarization interferometry. With a spatial light modulator and a common path interferometric configuration, two orthogonally circularly polarized beams with different phase modulation overlap each other, yielding the vector beams. In addition, the polarization states of this vector beam are scalable to the arbitrary spatial distribution because of its great flexibility and universal applicability. It is expected that this vector beam may have many potential and intriguing applications in the micro/nano material processing, liquid crystal elements fabrication and optical micro-manipulation, and so on.

Axial motion estimation and correction for simultaneous multi-plane two-photon calcium imaging

Two-photon imaging in behaving animals is typically accompanied by brain motion. For functional imaging experiments, for example with genetically encoded calcium indicators, such brain motion induces changes in fluorescence intensity. These motion-related intensity changes or motion artifacts can typically not be separated from neural activity-induced signals. While lateral motion, within the focal plane, can be corrected by computationally aligning images, axial motion, out of the focal plane, cannot easily be corrected. Here, we developed an algorithm for axial motion correction for non-ratiometric calcium indicators taking advantage of simultaneous multi-plane imaging. Using temporally multiplexed beams, recording simultaneously from at least two focal planes at different z positions, and recording a z-stack for each beam as a calibration step, the algorithm separates motion-related and neural activity-induced changes in fluorescence intensity. The algorithm is based on a maximum likelihood optimisation approach; it assumes (as a first order approximation) that no distortions of the sample occurs during axial motion and that neural activity increases uniformly along the optical axis in each region of interest. The developed motion correction approach allows axial motion estimation and correction at high frame rates for isolated structures in the imaging volume in vivo, such as sparse expression patterns in the fruit fly brain.

Non-telecentric two-photon microscopy for 3D random access mesoscale imaging

Diffraction-limited two-photon microscopy permits minimally invasive optical monitoring of neuronal activity. However, most conventional two-photon microscopes impose significant constraints on the size of the imaging field-of-view and the specific shape of the effective excitation volume, thus limiting the scope of biological questions that can be addressed and the information obtainable. Here, employing a non-telecentric optical design, we present a low-cost, easily implemented and flexible solution to address these limitations, offering a several-fold expanded three-dimensional field of view. Moreover, rapid laser-focus control via an electrically tunable lens allows near-simultaneous imaging of remote regions separated in three dimensions and permits the bending of imaging planes to follow natural curvatures in biological structures. Crucially, our core design is readily implemented (and reversed) within a matter of hours, making it highly suitable as a base platform for further development. We demonstrate the application of our system for imaging neuronal activity in a variety of examples in zebrafish, mice and fruit flies.

Fluorescence imaging of large-scale neural ensemble dynamics

Recent progress in fluorescence imaging allows neuroscientists to observe the dynamics of thousands of individual neurons, identified genetically or by their connectivity, across multiple brain areas and for extended durations in awake behaving mammals. We discuss advances in fluorescent indicators of neural activity, viral and genetic methods to express these indicators, chronic animal preparations for long-term imaging studies, and microscopes to monitor and manipulate the activity of large neural ensembles. Ca2+ imaging studies of neural activity can track brain area interactions and distributed information processing at cellular resolution. Across smaller spatial scales, high-speed voltage imaging reveals the distinctive spiking patterns and coding properties of targeted neuron types. Collectively, these innovations will propel studies of brain function and dovetail with ongoing neuroscience initiatives to identify new neuron types and develop widely applicable, non-human primate models. The optical toolkit's growing sophistication also suggests that "brain observatory" facilities would be useful open resources for future brain-imaging studies.

Fast optical recording of neuronal activity by three-dimensional custom-access serial holography

Optical recording of neuronal activity in three-dimensional (3D) brain circuits at cellular and millisecond resolution in vivo is essential for probing information flow in the brain. While random-access multiphoton microscopy permits fast optical access to neuronal targets in three dimensions, the method is challenged by motion artifacts when recording from behaving animals. Therefore, we developed three-dimensional custom-access serial holography (3D-CASH). Built on a fast acousto-optic light modulator, 3D-CASH performs serial sampling at 40 kHz from neurons at freely selectable 3D locations. Motion artifacts are eliminated by targeting each neuron with a size-optimized pattern of excitation light covering the cell body and its anticipated displacement field. Spike rates inferred from GCaMP6f recordings in visual cortex of awake mice tracked the phase of a moving bar stimulus with higher spike correlation between intra compared to interlaminar neuron pairs. 3D-CASH offers access to the millisecond correlation structure of in vivo neuronal activity in 3D microcircuits.

Neurophotonic tools for microscopic measurements and manipulation: status report

Neurophotonics was launched in 2014 coinciding with the launch of the BRAIN Initiative focused on development of technologies for advancement of neuroscience. For the last seven years, Neurophotonics' agenda has been well aligned with this focus on neurotechnologies featuring new optical methods and tools applicable to brain studies. While the BRAIN Initiative 2.0 is pivoting towards applications of these novel tools in the quest to understand the brain, this status report reviews an extensive and diverse toolkit of novel methods to explore brain function that have emerged from the BRAIN Initiative and related large-scale efforts for measurement and manipulation of brain structure and function. Here, we focus on neurophotonic tools mostly applicable to animal studies. A companion report, scheduled to appear later this year, will cover diffuse optical imaging methods applicable to noninvasive human studies. For each domain, we outline the current state-of-the-art of the respective technologies, identify the areas where innovation is needed, and provide an outlook for the future directions.

Diesel2p mesoscope with dual independent scan engines for flexible capture of dynamics in distributed neural circuitry

Imaging the activity of neurons that are widely distributed across brain regions deep in scattering tissue at high speed remains challenging. Here, we introduce an open-source system with Dual Independent Enhanced Scan Engines for Large field-of-view Two-Photon imaging (Diesel2p). Combining optical design, adaptive optics, and temporal multiplexing, the system offers subcellular resolution over a large field-of-view of ~25 mm2, encompassing distances up to 7 mm, with independent scan engines. We demonstrate the flexibility and various use cases of this system for calcium imaging of neurons in the living brain.

Flexible simultaneous mesoscale two-photon imaging of neural activity at high speeds

Understanding brain function requires monitoring local and global brain dynamics. Two-photon imaging of the brain across mesoscopic scales has presented trade-offs between imaging area and acquisition speed. We describe a flexible cellular resolution two-photon microscope capable of simultaneous video rate acquisition of four independently targetable brain regions spanning an approximate five-millimeter field of view. With this system, we demonstrate the ability to measure calcium activity across mouse sensorimotor cortex at behaviorally relevant timescales.

High-speed, cortex-wide volumetric recording of neuroactivity at cellular resolution using light beads microscopy

Two-photon microscopy has enabled high-resolution imaging of neuroactivity at depth within scattering brain tissue. However, its various realizations have not overcome the tradeoffs between speed and spatiotemporal sampling that would be necessary to enable mesoscale volumetric recording of neuroactivity at cellular resolution and speed compatible with resolving calcium transients. Here, we introduce light beads microscopy (LBM), a scalable and spatiotemporally optimal acquisition approach limited only by fluorescence lifetime, where a set of axially separated and temporally distinct foci record the entire axial imaging range near-simultaneously, enabling volumetric recording at 1.41 × 10⁸ voxels per second. Using LBM, we demonstrate mesoscopic and volumetric imaging at multiple scales in the mouse cortex, including cellular-resolution recordings within ~3 × 5 × 0.5 mm volumes containing >200,000 neurons at ~5 Hz and recordings of populations of ~1 million neurons within ~5.4 × 6 × 0.5 mm volumes at ~2 Hz, as well as higher speed (9.6 Hz) subcellular-resolution volumetric recordings. LBM provides an opportunity for discovering the neurocomputations underlying cortex-wide encoding and processing of information in the mammalian brain.

Anisotropy of 2PA, 3PA, and Kerr effect in nonpolar ZnO

Anisotropic optical nonlinearity plays an important role in polarization-dependent optoelectronic devices. Taking the advantage of in-plane crystallographic axis, the polarization dependence of third- and fifth-order nonlinearity in nonpolar ZnO has been investigated by z scan. Here, we established the theory model of anisotropic relation of fifth-order nonlinearity in hexagonal wurtzite crystal. Both two-photon absorption (2PA) and three-photon absorption (3PA) coefficients exhibit anisotropic oscillations with a period of 180°, and the polarization modulation factor and anisotropy coefficient (r_p,σ) are measured as (2.10, 1.84) for β₂, and (1.66, 1.33) for β₃, respectively. This is the first time, to the best of our knowledge, that the anisotropy of the fifth-order optical nonlinear effect has been characterized and confirmed. It is verified that the self-defocusing effect at 500 nm is isotropy, while the self-focusing effect at 800 nm exhibits significant anisotropy. The maximum value of the nonlinear figure of merit is 5.84 along the [0010] crystalline direction, nearly 1.6 times the minimum value, revealing that ZnO exhibits potential for nonlinear applications containing polarization-related all-optical switching components.

Calcium-responsive contrast agents for functional magnetic resonance imaging

Calcium ions represent one of the key second messengers accompanying neural activity and synaptic signaling. Accordingly, dynamic imaging of calcium fluctuations in living organisms represents a cornerstone technology for discovering neural mechanisms that underlie memory, determine behavior, and modulate emotional states as well as how these mechanisms are perturbed by neurological disease and brain injury. While optical technologies are well established for high resolution imaging of calcium dynamics, physical limits on light penetration hinder their application for whole-brain imaging in intact vertebrates. Unlike optics, magnetic resonance imaging (MRI) enables noninvasive large-scale imaging across vertebrates of all sizes. This has motivated the development of several sensors that leverage innovative physicochemical mechanisms to sensitize MRI contrast to intracellular and extracellular changes in calcium. Here, we review the current state-of-the-art in MRI-based calcium sensors, focusing on fundamental aspects of sensor performance, in vivo applications, and challenges related to sensitivity. We also highlight how innovations at the intersection of reporter gene technology and gene delivery open potential opportunities for mapping calcium activity in genetically targeted cells, complementing the benefits of small molecule probes and nanoparticle sensors.

Precompensation of 3D field distortions in remote focus two-photon microscopy

Remote focusing is widely used in 3D two-photon microscopy and 3D photostimulation because it enables fast axial scanning without moving the objective lens or specimen. However, due to the design constraints of microscope optics, remote focus units are often located in non-telecentric positions in the optical path, leading to significant depth-dependent 3D field distortions in the imaging volume. To address this limitation, we characterized 3D field distortions arising from non-telecentric remote focusing and present a method for distortion precompensation. We demonstrate its applicability for a 3D two-photon microscope that uses an acousto-optic lens (AOL) for remote focusing and scanning. We show that the distortion precompensation method improves the pointing precision of the AOL microscope to < 0.5 µm throughout the 400 × 400 × 400 µm imaging volume.

High-precision coding in visual cortex

Individual neurons in visual cortex provide the brain with unreliable estimates of visual features. It is not known whether the single-neuron variability is correlated across large neural populations, thus impairing the global encoding of stimuli. We recorded simultaneously from up to 50,000 neurons in mouse primary visual cortex (V1) and in higher order visual areas and measured stimulus discrimination thresholds of 0.35° and 0.37°, respectively, in an orientation decoding task. These neural thresholds were almost 100 times smaller than the behavioral discrimination thresholds reported in mice. This discrepancy could not be explained by stimulus properties or arousal states. Furthermore, behavioral variability during a sensory discrimination task could not be explained by neural variability in V1. Instead, behavior-related neural activity arose dynamically across a network of non-sensory brain areas. These results imply that perceptual discrimination in mice is limited by downstream decoders, not by neural noise in sensory representations.

Ultrahigh-speed point scanning two-photon microscopy using high dynamic range silicon photomultipliers

Conventional two-photon microscopes use photomultiplier tubes, which enable high sensitivity but can detect relatively few photons per second, forcing longer pixel integration times and limiting maximum imaging rates. We introduce novel detection electronics using silicon photomultipliers that greatly extend dynamic range, enabling more than an order of magnitude increased photon detection rate as compared to state-of-the-art photomultiplier tubes. We demonstrate that this capability can dramatically improve both imaging rates and signal-to-noise ratio (SNR) in two-photon microscopy using human surgical specimens. Finally, to enable wider use of more advanced detection technology, we have formed the OpenSiPM project, which aims to provide open source detector designs for high-speed two-photon and confocal microscopy.

Fast, multiplane line-scan confocal microscopy using axially distributed slits

The inherent constraints on resolution, speed and field of view have hindered the development of high-speed, three-dimensional microscopy techniques over large scales. Here, we present a multiplane line-scan imaging strategy, which uses a series of axially distributed reflecting slits to probe different depths within a sample volume. Our technique enables the simultaneous imaging of an optically sectioned image stack with a single camera at frame rates of hundreds of hertz, without the need for axial scanning. We demonstrate the applicability of our system to monitor fast dynamics in biological samples by performing calcium imaging of neuronal activity in mouse brains and voltage imaging of cardiomyocytes in cardiac samples.

Temporally multiplexed dual-plane imaging of neural activity with four-dimensional precision

Spatiotemporal patterns of neural activity generate brain functions, such as perception, memory, and behavior. Four-dimensional (4-D: x, y, z, t) analyses of such neural activity will facilitate understanding of brain functions. However, conventional two-photon microscope systems observe single-plane brain tissue alone at a time with cellular resolution. It faces a trade-off between the spatial resolution in the x-, y-, and z-axes and the temporal resolution by a limited point-by-point scan speed. To overcome this trade-off in 4-D imaging, we developed a holographic two-photon microscope for dual-plane imaging. A spatial light modulator (SLM) provided an additional focal plane at a different depth. Temporal multiplexing of split lasers with an optical chopper allowed fast imaging of two different focal planes. We simultaneously recorded the activities of neurons on layers 2/3 and 5 of the cerebral cortex in awake mice in vivo. The present study demonstrated the proof-of-concept of dual-plane two-photon imaging of neural circuits by using the temporally multiplexed SLM-based microscope. The temporally multiplexed holographic microscope, combined with in vivo labeling with genetically encoded probes, enabled 4-D imaging and analysis of neural activities at cellular resolution and physiological timescales. Large-scale 4-D imaging and analysis will facilitate studies of not only the nervous system but also of various biological systems.

Identification and quantification of neuronal ensembles in optical imaging experiments

Recent technical advances in molecular biology and optical imaging have made it possible to record from up to thousands of densely packed neurons in superficial and deep brain regions in vivo, with cellular subtype specificity and high spatiotemporal fidelity. Such optical neurotechnologies are enabling increasingly fine-scaled studies of neuronal circuits and reliably co-active groups of neurons, so-called ensembles. Neuronal ensembles are thought to constitute the basic functional building blocks of brain systems, potentially exhibiting collective computational properties. While the technical framework of in vivo optical imaging and quantification of neuronal activity follows certain widely held standards, analytical methods for study of neuronal co-activity and ensembles lack consensus and are highly varied across the field. Here we provide a comprehensive step-by-step overview of theoretical, experimental, and analytical considerations for the identification and quantification of neuronal ensemble dynamics in high-resolution in vivo optical imaging studies.

Axially resolved volumetric two-photon microscopy with an extended field of view using depth localization under mirrored Airy beams

It is a great challenge in two-photon microscopy (2PM) to have a high volumetric imaging speed without sacrificing the spatial and temporal resolution in three dimensions (3D). The structure in 2PM images could be reconstructed with better spatial and temporal resolution by the proper choice of the data processing algorithm. Here, we propose a method to reconstruct 3D volume from 2D projections imaged by mirrored Airy beams. We verified that our approach can achieve high accuracy in 3D localization over a large axial range and is applicable to continuous and dense sample. The effective field of view after reconstruction is expanded. It is a promising technique for rapid volumetric 2PM with axial localization at high resolution.

High-NA two-photon single cell imaging with remote focusing using a diffractive tunable lens

Fast, volumetric structural and functional imaging of cellular and sub-cellular dynamics inside the living brain is one of the most desired capabilities in the neurosciences, but still faces serious challenges. Specifically, while few solutions for rapid 3D scanning exist, it is generally much easier to facilitate fast in-plane scanning than it is to scan axially at high speeds. Remote focusing in which the imaging plane is shifted along the optical axis by a tunable lens while maintaining the position of the sample and objective is a promising approach to increase the axial scan speed, but existing techniques often introduce severe optical aberrations in high-NA imaging systems, eliminating the possibility of diffraction-limited single-cell imaging. Here, we demonstrate near diffraction-limited, volumetric two-photon fluorescence microscopy in which we resolve the deep sub-micron structures of single microglia cells with axial scanning performed using a novel high-NA remote focusing method. Image contrast is maintained to within 7% compared to mechanical sample stepping and the focal volume remains nearly diffraction-limited over an axial range greater than 86 µm.

Dynamic distortion in resonant galvanometric optical scanners

High-speed optical systems are revolutionizing biomedical imaging in microscopy, DNA sequencing, and flow cytometry, as well as numerous other applications, including data storage, display technologies, printing, and autonomous vehicles. These systems often achieve the necessary imaging or sensing speed through the use of resonant galvanometric optical scanners. Here, we show that the optical performance of these devices suffers due to the dynamic mirror distortion that arises from the variation in torque with angular displacement. In one of two scanners tested, these distortions result in a variation of signal-to-noise (Strehl) ratio by an order of magnitude across the field of view, degrading transverse resolution by more than a factor of 2. This mirror distortion could be mitigated through the use of stiffer materials, such as beryllium or silicon carbide, at the expense of surface roughness, as these cannot be polished to the same degree of smoothness as common optical glasses. The repeatability of the dynamic distortion indicates that computational and optical corrective methods are also possible.

Integrated Neurophotonics: Toward Dense Volumetric Interrogation of Brain Circuit Activity-at Depth and in Real Time

We propose a new paradigm for dense functional imaging of brain activity to surmount the limitations of present methodologies. We term this approach "integrated neurophotonics"; it combines recent advances in microchip-based integrated photonic and electronic circuitry with those from optogenetics. This approach has the potential to enable lens-less functional imaging from within the brain itself to achieve dense, large-scale stimulation and recording of brain activity with cellular resolution at arbitrary depths. We perform a computational study of several prototype 3D architectures for implantable probe-array modules that are designed to provide fast and dense single-cell resolution (e.g., within a 1-mm3 volume of mouse cortex comprising ∼100,000 neurons). We describe progress toward realizing integrated neurophotonic imaging modules, which can be produced en masse with current semiconductor foundry protocols for chip manufacturing. Implantation of multiple modules can cover extended brain regions.

Design of a multi-modality DMD-based two-photon microscope system

We present the modular design and characterization of a multi-modality video-rate two-photon excitation (TPE) microscope based on integrating a digital micromirror device (DMD), which functions as an ultrafast beam shaper and random-access scanner, with a pair of galvanometric scanners. The TPE microscope system realizes a suite of new imaging functionalities, including (1) multi-layer imaging with 3D programmable imaging planes, (2) DMD-based wavefront correction, and (3) multi-focus optical stimulation (up to 22.7 kHz) with simultaneous TPE imaging, all in real-time. We also report the detailed optomechanical design and software development that achieves high level system automation. To verify the performance of different microscope functions, we have devised and performed imaging experiments on Drosophila brain, mouse kidney and human stem cells. The results not only show improved imaging resolution and depths via the DMD-based adaptive optics, but also demonstrate fast multi-focus stimulation for the first time. With the new imaging capabilities, e.g., tools for optogenetics, the multi-modality TPE microscope may play a critical role in the applications pertinent to neuroscience and biophotonics.

Inferring Neuronal Couplings From Spiking Data Using a Systematic Procedure With a Statistical Criterion

Recent remarkable advances in experimental techniques have provided a background for inferring neuronal couplings from point process data that include a great number of neurons. Here, we propose a systematic procedure for pre- and postprocessing generic point process data in an objective manner to handle data in the framework of a binary simple statistical model, the Ising or generalized McCulloch-Pitts model. The procedure has two steps: (1) determining time bin size for transforming the point process data into discrete-time binary data and (2) screening relevant couplings from the estimated couplings. For the first step, we decide the optimal time bin size by introducing the null hypothesis that all neurons would fire independently, then choosing a time bin size so that the null hypothesis is rejected with the strict criteria. The likelihood associated with the null hypothesis is analytically evaluated and used for the rejection process. For the second postprocessing step, after a certain estimator of coupling is obtained based on the preprocessed data set (any estimator can be used with the proposed procedure), the estimate is compared with many other estimates derived from data sets obtained by randomizing the original data set in the time direction. We accept the original estimate as relevant only if its absolute value is sufficiently larger than those of randomized data sets. These manipulations suppress false positive couplings induced by statistical noise. We apply this inference procedure to spiking data from synthetic and in vitro neuronal networks. The results show that the proposed procedure identifies the presence or absence of synaptic couplings fairly well, including their signs, for the synthetic and experimental data. In particular, the results support that we can infer the physical connections of underlying systems in favorable situations, even when using a simple statistical model.

Dynamical generation of multiple focal spot pairs with controllable position and polarization

We report a flexible method to dynamically generate multiple sub-wavelength focal spot pairs with adjustable polarization, position and number at the focal plane of a high numerical aperture (NA) objective lens. The desired incident field on the pupil plane can be analytically derived by employing the time-reversed method combined with the dipole antenna radiation theory. The numerical simulations of the corresponding tightly focused field are conducted using the Richards-Wolf vectorial diffraction theory. The validity of the presented method is demonstrated through experimental generation of several designed pupil fields with a versatile vectorial optical field generator and characterization of the produced focused fields with Stokes parameter measurements.

Two-Photon Voltage Imaging of Spontaneous Activity from Multiple Neurons Reveals Network Activity in Brain Tissue

Recording the electrical activity of multiple neurons simultaneously would greatly facilitate studies on the function of neuronal circuits. The combination of the fast scanning by random-access multiphoton microscopy (RAMP) and the latest two-photon-compatible high-performance fluorescent genetically encoded voltage indicators (GEVIs) has enabled action potential detection in deep layers in in vivo brain. However, neuron connectivity analysis on optically recorded action potentials from multiple neurons in brain tissue has yet to be achieved. With high expression of a two-photon-compatible GEVI, ASAP3, via in utero electroporation and RAMP, we achieved voltage recording of spontaneous activities from multiple neurons in brain slice. We provide evidence for the developmental changes in intralaminar horizontal connections in somatosensory cortex layer 2/3 with a greater sensitivity than calcium imaging. This method thus enables investigation of neuronal network connectivity at the cellular resolution in brain tissue.

Imaging volumetric dynamics at high speed in mouse and zebrafish brain with confocal light field microscopy

A detailed understanding of the function of neural networks and how they are supported by a dynamic vascular system requires fast three-dimensional imaging in thick tissues. Here we present confocal light field microscopy, a method that enables fast volumetric imaging in the brain at depths of hundreds of micrometers. It uses a generalized confocal detection scheme that selectively collects fluorescent signals from the in-focus volume and provides optical sectioning capability to improve imaging resolution and sensitivity in thick tissues. We demonstrate recording of whole-brain calcium transients in freely swimming zebrafish larvae and observe behaviorally correlated activities in single neurons during prey capture. Furthermore, in the mouse brain, we detect neural activities at depths of up to 370 μm and track blood cells at 70 Hz over a volume of diameter 800 μm × thickness 150 μm and depth of up to 600 μm.

3D Localization for Light-Field Microscopy via Convolutional Sparse Coding on Epipolar Images

Light-field microscopy (LFM) is a type of all-optical imaging system that is able to capture 4D geometric information of light rays and can reconstruct a 3D model from a single snapshot. In this paper, we propose a new 3D localization approach to effectively detect 3D positions of neuronal cells from a single light-field image with high accuracy and outstanding robustness to light scattering. This is achieved by constructing a depth-aware dictionary and by combining it with convolutional sparse coding. Specifically, our approach includes 3 key parts: light-field calibration, depth-aware dictionary construction, and localization based on convolutional sparse coding (CSC). In the first part, an observed raw light-field image is calibrated and then decoded into a two-plane parameterized 4D format which leads to the epi-polar plane image (EPI). The second part involves simulating a set of light-fields using a wave-optics forward model for a ball-shaped volume that is located at different depths. Then, a depth-aware dictionary is constructed where each element is a synthetic EPI associated to a specific depth. Finally, by taking full advantage of the sparsity prior and shift-invariance property of EPI, 3D localization is achieved via convolutional sparse coding on an observed EPI with respect to the depth-aware EPI dictionary. We evaluate our approach on both non-scattering specimen (fluorescent beads suspended in agarose gel) and scattering media (brain tissues of genetically encoded mice). Extensive experiments demonstrate that our approach can reliably detect the 3D positions of granular targets with small Root Mean Square Error (RMSE), high robustness to optical aberration and light scattering in mammalian brain tissues.

Spectro-temporal encoded multiphoton microscopy and fluorescence lifetime imaging at kilohertz frame-rates

Two-Photon Microscopy has become an invaluable tool for biological and medical research, providing high sensitivity, molecular specificity, inherent three-dimensional sub-cellular resolution and deep tissue penetration. In terms of imaging speeds, however, mechanical scanners still limit the acquisition rates to typically 10-100 frames per second. Here we present a high-speed non-linear microscope achieving kilohertz frame rates by employing pulse-modulated, rapidly wavelength-swept lasers and inertia-free beam steering through angular dispersion. In combination with a high bandwidth, single-photon sensitive detector, this enables recording of fluorescent lifetimes at speeds of 88 million pixels per second. We show high resolution, multi-modal - two-photon fluorescence and fluorescence lifetime (FLIM) - microscopy and imaging flow cytometry with a digitally reconfigurable laser, imaging system and data acquisition system. These high speeds should enable high-speed and high-throughput image-assisted cell sorting.

Scanless two-photon excitation with temporal focusing

Temporal focusing, with its ability to focus light in time, enables scanless illumination of large surface areas at the sample with micrometer axial confinement and robust propagation through scattering tissue. In conventional two-photon microscopy, widely used for the investigation of intact tissue in live animals, images are formed by point scanning of a spatially focused pulsed laser beam, resulting in limited temporal resolution of the excitation. Replacing point scanning with temporally focused widefield illumination removes this limitation and represents an important milestone in two-photon microscopy. Temporal focusing uses a diffusive or dispersive optical element placed in a plane conjugate to the objective focal plane to generate position-dependent temporal pulse broadening that enables axially confined multiphoton absorption, without the need for tight spatial focusing. Many techniques have benefitted from temporal focusing, including scanless imaging, super-resolution imaging, photolithography, uncaging of caged neurotransmitters and control of neuronal activity via optogenetics.

Simultaneous multiplane imaging with reverberation two-photon microscopy

Multiphoton microscopy has gained enormous popularity because of its unique capacity to provide high-resolution images from deep within scattering tissue. Here, we demonstrate video-rate multiplane imaging with two-photon microscopy by performing near-instantaneous axial scanning while maintaining three-dimensional micrometer-scale resolution. Our technique, termed reverberation microscopy, enables the monitoring of neuronal populations over large depth ranges and can be implemented as a simple add-on to a conventional design.

Parallelized volumetric fluorescence microscopy with a reconfigurable coded incoherent light-sheet array

Parallelized fluorescence imaging has been a long-standing pursuit that can address the unmet need for a comprehensive three-dimensional (3D) visualization of dynamical biological processes with minimal photodamage. However, the available approaches are limited to incomplete parallelization in only two dimensions or sparse sampling in three dimensions. We hereby develop a novel fluorescence imaging approach, called coded light-sheet array microscopy (CLAM), which allows complete parallelized 3D imaging without mechanical scanning. Harnessing the concept of an "infinity mirror", CLAM generates a light-sheet array with controllable sheet density and degree of coherence. Thus, CLAM circumvents the common complications of multiple coherent light-sheet generation in terms of dedicated wavefront engineering and mechanical dithering/scanning. Moreover, the encoding of multiplexed optical sections in CLAM allows the synchronous capture of all sectioned images within the imaged volume. We demonstrate the utility of CLAM in different imaging scenarios, including a light-scattering medium, an optically cleared tissue, and microparticles in fluidic flow. CLAM can maximize the signal-to-noise ratio and the spatial duty cycle, and also provides a further reduction in photobleaching compared to the major scanning-based 3D imaging systems. The flexible implementation of CLAM regarding both hardware and software ensures compatibility with any light-sheet imaging modality and could thus be instrumental in a multitude of areas in biological research.

Converting lateral scanning into axial focusing to speed up three-dimensional microscopy

In optical microscopy, the slow axial scanning rate of the objective or the sample has traditionally limited the speed of volumetric imaging. Recently, by conjugating either a movable mirror to the image plane in a remote-focusing geometry or an electrically tuneable lens (ETL) to the back focal plane, rapid axial scanning has been achieved. However, mechanical actuation of a mirror limits the axial scanning rate (usually only 10-100 Hz for piezoelectric or voice coil-based actuators), while ETLs introduce spherical and higher-order aberrations that prevent high-resolution imaging. In an effort to overcome these limitations, we introduce a novel optical design that transforms a lateral-scan motion into a spherical aberration-free axial scan that can be used for high-resolution imaging. Using a galvanometric mirror, we scan a laser beam laterally in a remote-focusing arm, which is then back-reflected from different heights of a mirror in the image space. We characterize the optical performance of this remote-focusing technique and use it to accelerate axially swept light-sheet microscopy by an order of magnitude, allowing the quantification of rapid vesicular dynamics in three dimensions. We also demonstrate resonant remote focusing at 12 kHz with a two-photon raster-scanning microscope, which allows rapid imaging of brain tissues and zebrafish cardiac dynamics with diffraction-limited resolution.

Fluorescence imaging with tailored light

Fluorescence microscopy has long been a valuable tool for biological and medical imaging. Control of optical parameters such as the amplitude, phase, polarization and propagation angle of light gives fluorescence imaging great capabilities ranging from super-resolution imaging to long-term real-time observation of living organisms. In this review, we discuss current fluorescence imaging techniques in terms of the use of tailored or structured light for the sample illumination and fluorescence detection, providing a clear overview of their working principles and capabilities.

MATRIEX imaging: multiarea two-photon real-time in vivo explorer

Two-photon laser scanning microscopy has been extensively applied to study in vivo neuronal activity at cellular and subcellular resolutions in mammalian brains. However, the extent of such studies is typically confined to a single functional region of the brain. Here, we demonstrate a novel technique, termed the multiarea two-photon real-time in vivo explorer (MATRIEX), that allows the user to target multiple functional brain regions distributed within a zone of up to 12 mm in diameter, each with a field of view (FOV) of ~200 μm in diameter, thus performing two-photon Ca²⁺ imaging with single-cell resolution in all of the regions simultaneously. For example, we demonstrate real-time functional imaging of single-neuron activities in the primary visual cortex, primary motor cortex and hippocampal CA1 region of mice in both anesthetized and awake states. A unique advantage of the MATRIEX technique is the configuration of multiple microscopic FOVs that are distributed in three-dimensional space over macroscopic distances (>1 mm) both laterally and axially but that are imaged by a single conventional laser scanning device. In particular, the MATRIEX technique can be effectively implemented as an add-on optical module for an existing conventional single-beam-scanning two-photon microscope without requiring any modification to the microscope itself. Thus, the MATRIEX technique can be readily applied to substantially facilitate the exploration of multiarea neuronal activity in vivo for studies of brain-wide neural circuit function with single-cell resolution.