Electrically tunable lens speeds up 3D orbital tracking

4D Single-particle tracking with asynchronous read-out single-photon avalanche diode array detector

Single-particle tracking techniques enable investigation of the complex functions and interactions of individual particles in biological environments. Many such techniques exist, each demonstrating trade-offs between spatiotemporal resolution, spatial and temporal range, technical complexity, and information content. To mitigate these trade-offs, we enhanced a confocal laser scanning microscope with an asynchronous read-out single-photon avalanche diode array detector. This detector provides an image of the particle's emission, precisely reflecting its position within the excitation volume. This localization is utilized in a real-time feedback system to drive the microscope scanning mechanism and ensure the particle remains centered inside the excitation volume. As each pixel is an independent single-photon detector, single-particle tracking is combined with fluorescence lifetime measurement. Our system achieves 40 nm lateral and 60 nm axial localization precision with 100 photons and sub-millisecond temporal sampling for real-time tracking. Offline tracking can refine this precision to the microsecond scale. We validated the system's spatiotemporal resolution by tracking fluorescent beads with diffusion coefficients up to 10 μm2/s. Additionally, we investigated the movement of lysosomes in living SK-N-BE cells and measured the fluorescence lifetime of the marker expressed on a membrane protein. We expect that this implementation will open other correlative imaging and tracking studies.

Simple, non-mechanical and automatic calibration approach for axial-scanning microscopy with an electrically tunable lens

We describe a simple and robust calibration approach for axial-scanning microscopy that realizes axial focus shifts with an electrically tunable lens (ETL). We demonstrate the calibration approach based on a microscope with an ETL placed close to the rear stop of the objective lens. By introducing a target-consisted of repeating lines at one known frequency and placed at a ~45° angle to the imaging path, the calibration method captures multiple images at different ETL currents and calibrates the dependence of the axial focus shift on the ETL current by evaluating the sharpness of the captured images. It calibrates the dependence of the magnification of the microscope on the ETL current by measuring the period of the repeating lines in the captured images. The experimental results show that different from the conventional calibration procedure, the proposed scheme does not involve any mechanical scanning and can simultaneously calibrate the dependence of the axial focus shift and the magnification on the ETL current. This might facilitate imaging studies that require the measurement of fine structures in a 3D volume. We also show the calibration procedure can be used to estimate the radius of a conner-arc sample, fabricated using laser micromachining. We believe that this easy-to-use calibration approach may facilitate use of ETLs for a variety of imaging platforms. It may also provide new insights for the development of novel 3D surface measurement methods. RESEARCH HIGHLIGHTS: The proposed calibration scheme does not involve any mechanical scanning and can simultaneously calibrate the dependence of the axial focus shift and the magnification on the electrically tunable lens (ETL) current. It might facilitate imaging studies that require the measurement of fine structures in a 3D volume, and the use of ETLs for a variety of imaging platforms. It may also provide new insights for the development of novel 3D surface measurement methods.

Recent Advances in Single-Molecule Tracking and Imaging Techniques

Since the early 1990s, single-molecule detection in solution at room temperature has enabled direct observation of single biomolecules at work in real time and under physiological conditions, providing insights into complex biological systems that the traditional ensemble methods cannot offer. In particular, recent advances in single-molecule tracking techniques allow researchers to follow individual biomolecules in their native environments for a timescale of seconds to minutes, revealing not only the distinct pathways these biomolecules take for downstream signaling but also their roles in supporting life. In this review, we discuss various single-molecule tracking and imaging techniques developed to date, with an emphasis on advanced three-dimensional (3D) tracking systems that not only achieve ultrahigh spatiotemporal resolution but also provide sufficient working depths suitable for tracking single molecules in 3D tissue models. We then summarize the observables that can be extracted from the trajectory data. Methods to perform single-molecule clustering analysis and future directions are also discussed.

Multicolor 3D Orbital Tracking

Feedback-based single-particle tracking (SPT) is a powerful technique for investigating particle behavior with very high spatiotemporal resolution. The ability to follow different species and their interactions independently adds a new dimension to the information available from SPT. However, only a few approaches have been expanded to multiple colors and no method is currently available that can follow two differently labeled biomolecules in 4 dimensions independently. In this proof-of-concept paper, the new modalities available when performing 3D orbital tracking with a second detection channel are demonstrated. First, dual-color tracking experiments are described studying independently diffusing particles of different types. For interacting particles where their motion is correlated, a second modality is implemented where a particle is tracked in one channel and the position of the second fluorescence species is monitored in the other channel. As a third modality, 3D orbital tracking is performed in one channel while monitoring its spectral signature in a second channel. This last modality is used to successfully readout accurate Förster Resonance Energy Transfer (FRET) values over time while tracking a mobile particle.

High-Speed and Low-Latency 3D Fluorescence Imaging for Robotic Microscope

In this study, we propose a high-speed and low-latency 3D fluorescence imaging method for robotic microscopes. The prototype system consists of a focus-tunable lens called a TAG lens, which operates at several hundred kHz, an image intensifier (I.I.) that enhances faint light such as fluorescence, and a high-speed vision system that can transfer acquired images to the host PC in 500 Hz. The proposed method can acquire images at arbitrary focal lengths at frame rates on the order of 1 kHz by synchronizing the focal-length fluctuation of the TAG lens and the exposure timing of the I.I., whose duration is a few hundred nanoseconds. The low-latency we aim for in this paper is on the order of a few milliseconds. A prototype system was developed to validate the proposed method. High-speed 3D tracking of the Brownian motion of a fluorescent bead of 0.5 μm diameter was demonstrated to verify the feedback performance of the proposed low-latency 3D fluorescence imaging method.

Closed-loop experimental optimization of tunable lenses

Tunable lenses (TLs) are optical devices that can change their optical power in response to an electrical signal. In many applications, they are often pushed to or beyond their temporal limits. Fast periodic and/or abrupt variations of the optical power induce undesired distortions in their transient response and produce a decrease in their performance. A low-cost focimetry system, along with a custom closed-loop iterative optimization algorithm, was developed to (1) characterize a TL's response at high speed and (2) optimize their performance in realistic TL working conditions. A significant lens performance improvement was found in about 23 iterations with a decrease in the area under the error curve and an improved effective time. Applying the closed-loop optimization algorithm in a depth scanning experiment enhanced the image quality. Quantitatively, the image quality was evaluated using the structural similarity index metric that improves in individual frames, on average, from 0.345 to 0.895.

Real-Time Feedback-Driven Single-Particle Tracking: A Survey and Perspective

Real-time feedback-driven single-particle tracking (RT-FD-SPT) is a class of techniques in the field of single-particle tracking that uses feedback control to keep a particle of interest in a detection volume. These methods provide high spatiotemporal resolution on particle dynamics and allow for concurrent spectroscopic measurements. This review article begins with a survey of existing techniques and of applications where RT-FD-SPT has played an important role. Each of the core components of RT-FD-SPT are systematically discussed in order to develop an understanding of the trade-offs that must be made in algorithm design and to create a clear picture of the important differences, advantages, and drawbacks of existing approaches. These components are feedback tracking and control, ranging from simple proportional-integral-derivative control to advanced nonlinear techniques, estimation to determine particle location from the measured data, including both online and offline algorithms, and techniques for calibrating and characterizing different RT-FD-SPT methods. Then a collection of metrics for RT-FD-SPT is introduced to help guide experimentalists in selecting a method for their particular application and to help reveal where there are gaps in the techniques that represent opportunities for further development. Finally, this review is concluded with a discussion on future perspectives in the field.

Three-Dimensional Tracking of Tethered Particles for Probing Nanometer-Scale Single-Molecule Dynamics Using a Plasmonic Microscope

Three-dimensional (3D) tracking of surface-tethered single particles reveals the dynamics of the molecular tether. However, most 3D tracking techniques lack precision, especially in the axial direction, for measuring the dynamics of biomolecules with a spatial scale of several nanometers. Here, we present a plasmonic imaging technique that can track the motion of ∼100 tethered particles in 3D simultaneously with sub-nanometer axial precision and single-digit nanometer lateral precision at millisecond time resolution. By tracking the 3D coordinates of a tethered particle with high spatial resolution, we are able to determine the dynamics of single short DNA and study its interaction with enzymes. We further show that the particle motion pattern can be used to identify specific and nonspecific interactions in immunoassays. We anticipate that our 3D tracking technique can contribute to the understanding of molecular dynamics and interactions at the single-molecule level.

Tunable lens using dielectric elastomer sandwiched by transparent conductive liquid

We propose and demonstrate a compact tunable lens with high transmittance using a dielectric elastomer sandwiched by transparent conductive liquid. The transparent conductive liquid not only serves as the refractive material of the tunable lens but also works as the compliant electrode of the dielectric elastomer. The overall dimensions of the proposed tunable lens are 16 mm in diameter and 10 mm in height, and the optical transmittance is as high as 92.2% at 380-760 nm. The focal power variation of the tunable lens is -23.71D at an actuation voltage of 3.0 kV. The rise and fall times are 60 ms and 185 ms, respectively. The fabrication process of the tunable lens is free of the deposition of opaque compliant electrodes. Such a tunable lens promises a potential solution in various compact imaging systems.

Image-free real-time 3-D tracking of a fast-moving object using dual-pixel detection

Real-time 3-D tracking of a fast-moving object has found important applications in industry, traffic control, sports, biomedicine, defense, etc. However, it is difficult to adopt typical image-based object tracking systems in a fast-moving object tracking in real time and for a long duration, because reliable and robust image processing and analysis algorithms are often computationally exhausted, and limited storage and bandwidth can hardly fulfill the great demand of high-speed photography. Here we report an image-free 3-D tracking approach. The approach uses only two single-pixel detectors and a high-speed spatial light modulator for data acquisition. By illuminating the target moving object with six single-period Fourier basis patterns, the approach is able to analytically calculate the position of the object with the corresponding single-pixel measurements. The approach is low-cost, and data- and computation-efficient. We experimentally demonstrate that the proposed approach can detect and track a fast-moving object at a frame rate of 1666 frames per second by using a 10,000 Hz digital micromirror device. Benefiting from the wide working spectrum of single-pixel detectors, the reported approach might be applicable for hidden fast-moving object tracking.

Quasi-simultaneous multi-focus imaging using a lock-in pixel image sensor and TAG lens

In this paper, a quasi-simultaneous multi-focus imaging technique named simulfocus imaging is reported. This technique was developed for measuring an entire object distributed in the depth direction beyond the depth of field (DOF) with high resolution in a single shot. Simulfocus imaging can acquire multiple focal planes in one shot by synchronizing a tunable acoustic gradient index (TAG) lens and a lock-in pixel image sensor. The TAG lens is a tunable-focus lens whose focal position can be changed at a high speed of several tens to several hundreds of kilohertz. The lock-in pixel image sensor is a special image sensor that can execute multiple exposures at an arbitrary timing during a single shooting. The sensor includes a number of photoelectron storage units in each pixel, and the units where the photoelectrons generated by each exposure are stored can be freely selected. Since an image can be acquired for a single storage unit, and the lock-in pixel image sensor has a number of storage units, the lock-in pixel image sensor can acquire multiple images in one shot. By assigning a specific exposure timing to each unit and synchronizing the exposure timing with the focus fluctuation of the TAG lens, it is possible to simultaneously acquire images in different focal planes. To evaluate the system, we conducted experiments to show the effectiveness of simulfocus imaging in microscope and telescope configurations. From the experimental results, it was confirmed that simulfocus was effective in both configurations.

The temporal structure of the inner retina at a single glance

The retina decomposes visual stimuli into parallel channels that encode different features of the visual environment. Central to this computation is the synaptic processing in a dense layer of neuropil, the so-called inner plexiform layer (IPL). Here, different types of bipolar cells stratifying at distinct depths relay the excitatory feedforward drive from photoreceptors to amacrine and ganglion cells. Current experimental techniques for studying processing in the IPL do not allow imaging the entire IPL simultaneously in the intact tissue. Here, we extend a two-photon microscope with an electrically tunable lens allowing us to obtain optical vertical slices of the IPL, which provide a complete picture of the response diversity of bipolar cells at a "single glance". The nature of these axial recordings additionally allowed us to isolate and investigate batch effects, i.e. inter-experimental variations resulting in systematic differences in response speed. As a proof of principle, we developed a simple model that disentangles biological from experimental causes of variability and allowed us to recover the characteristic gradient of response speeds across the IPL with higher precision than before. Our new framework will make it possible to study the computations performed in the central synaptic layer of the retina more efficiently.

Continuous focal translation enhances rate of point-scan volumetric microscopy

Two-Photon Laser-Scanning Microscopy is a powerful tool for exploring biological structure and function due to its ability to optically section through a sample with a tight focus. While it is possible to obtain 3D image stacks by moving a stage, this per-frame imaging process is time consuming. Here, we present a method for an easy-to-implement and inexpensive modification of an existing two-photon microscope to rapidly image in 3D using an electrically tunable lens to create a tessellating scan pattern which repeats with the volume rate. Using appropriate interpolating algorithms, the volumetric imaging rate can be increased by a factor up to four-fold. This capability provides the expansion of the two-photon microscope into the third dimension for faster volumetric imaging capable of visualizing dynamics on timescales not achievable by traditional stage-stack methods.

Speeded-Up Focus Control of Electrically Tunable Lens by Sparse Optimization

Electrically tunable lenses (ETL), also known as liquid lenses, can be focused at various distances by changing the electric signal applied on the lens. ETLs require no mechanical structures, and therefore, provide a more compact and inexpensive focus control than conventional computerized translation stages. They have been exploited in a wide range of imaging and display systems and enabled novel applications for the last several years. However, the optical fluid in the ETL is rippled after the actuation, which physically limits the response time and significantly hampers the applicability range. To alleviate this problem, we apply a sparse optimization framework that optimizes the temporal pattern of the electrical signal input to the ETL. In verification experiments, the proposed method accelerated the convergence of the focal length to the target patterns. In particular, it converged the optical power to the target at twice the speed of the simply determined input signal, and increased the quality of the captured image during multi-focal imaging.

Real-Time 3D Single Particle Tracking: Towards Active Feedback Single Molecule Spectroscopy in Live Cells

Single molecule fluorescence spectroscopy has been largely implemented using methods which require tethering of molecules to a substrate in order to make high temporal resolution measurements. However, the act of tethering a molecule requires that the molecule be removed from its environment. This is especially perturbative when measuring biomolecules such as enzymes, which may rely on the non-equilibrium and crowded cellular environment for normal function. A method which may be able to un-tether single molecule fluorescence spectroscopy is real-time 3D single particle tracking (RT-3D-SPT). RT-3D-SPT uses active feedback to effectively lock-on to freely diffusing particles so they can be measured continuously with up to photon-limited temporal resolution over large axial ranges. This review gives an overview of the various active feedback 3D single particle tracking methods, highlighting specialized detection and excitation schemes which enable high-speed real-time tracking. Furthermore, the combination of these active feedback methods with simultaneous live-cell imaging is discussed. Finally, the successes in real-time 3D single molecule tracking (RT-3D-SMT) thus far and the roadmap going forward for this promising family of techniques are discussed.

VIEW-MOD: a versatile illumination engine with a modular optical design for fluorescence microscopy

We developed VIEW-MOD (Versatile Illumination Engine with a Modular Optical Design): a compact, multi-modality microscope, which accommodates multiple illumination schemes including variable angle total internal reflection, point scanning and vertical/horizontal light sheet. This system allows combining and flexibly switching between different illuminations and imaging modes by employing three electrically tunable lenses and two fast-steering mirrors. This versatile optics design provides control of 6 degrees of freedom of the illumination source (3 translation, 2 tilt, and beam shape) plus the axial position of the imaging plane. We also developed standalone software with an easy-to-use GUI to calibrate and control the microscope. We demonstrate the applications of this system and software in biosensor imaging, optogenetics and fast 3D volume imaging. This system is ready to fit into complex imaging circumstances requiring precise control of illumination and detection paths, and has a broad scope of usability for a myriad of biological applications.

Tunable lenses: dynamic characterization and fine-tuned control for high-speed applications

Tunable lenses are becoming ubiquitous, in applications including microscopy, optical coherence tomography, computer vision, quality control, and presbyopic corrections. Many applications require an accurate control of the optical power of the lens in response to a time-dependent input waveform. We present a fast focimeter (3.8 KHz) to characterize the dynamic response of tunable lenses, which was demonstrated on different lens models. We found that the temporal response is repetitive and linear, which allowed the development of a robust compensation strategy based on the optimization of the input wave, using a linear time-invariant model. To our knowledge, this work presents the first procedure for a direct characterization of the transient response of tunable lenses and for compensation of their temporal distortions, and broadens the potential of tunable lenses also in high-speed applications.

MINFLUX monitors rapid molecular jumps with superior spatiotemporal resolution

Compared with localization schemes solely based on evaluating patterns of molecular emission, the recently introduced single-molecule localization concept called MINFLUX and the fluorescence nanoscopies derived from it require up to orders of magnitude fewer emissions to attain single-digit nanometer resolution. Here, we demonstrate that the lower number of required fluorescence photons enables MINFLUX to detect molecular movements of a few nanometers at a temporal sampling of well below 1 millisecond. Using fluorophores attached to thermally fluctuating DNA strands as model systems, we demonstrate that measurement times as short as 400 microseconds suffice to localize fluorescent molecules with ∼2-nm precision. Such performance is out of reach for popular camera-based localization by centroid calculation of emission diffraction patterns. Since theoretical limits have not been reached, our results show that emerging MINFLUX nanoscopy bears great potential for dissecting the motions of individual (macro)molecules at hitherto-unattained combinations of spatial and temporal resolution.

All-optical microscope autofocus based on an electrically tunable lens and a totally internally reflected IR laser

Microscopic imaging at high spatial-temporal resolution over long time scales (minutes to hours) requires rapid and precise stabilization of the microscope focus. Conventional and commercial autofocus systems are largely based on piezoelectric stages or mechanical objective actuators. Objective to sample distance is either measured by image analysis approaches or by hardware modules measuring the intensity of reflected infrared light. We propose here a truly all-optical microscope autofocus taking advantage of an electrically tunable lens and a totally internally reflected infrared probe beam. We implement a feedback-loop based on the lateral position of a totally internally reflected infrared laser on a quadrant photodetector, as an indicator of the relative defocus. We show here how to treat the combined contributions due to mechanical defocus and deformation of the tunable lens. As a result, the sample can be kept in focus without any mechanical movement, at rates up to hundreds of Hertz. The device requires only reflective optics and can be implemented at a fraction of the cost required for a comparable piezo-based actuator.

Development of Three-Dimensional Dental Scanning Apparatus Using Structured Illumination

We demonstrated a three-dimensional (3D) dental scanning apparatus based on structured illumination. A liquid lens was used for tuning focus and a piezomotor stage was used for the shift of structured light. A simple algorithm, which detects intensity modulation, was used to perform optical sectioning with structured illumination. We reconstructed a 3D point cloud, which represents the 3D coordinates of the digitized surface of a dental gypsum cast by piling up sectioned images. We performed 3D registration of an individual 3D point cloud, which includes alignment and merging the 3D point clouds to exhibit a 3D model of the dental cast.

Single Particle Tracking: From Theory to Biophysical Applications

After three decades of developments, single particle tracking (SPT) has become a powerful tool to interrogate dynamics in a range of materials including live cells and novel catalytic supports because of its ability to reveal dynamics in the structure-function relationships underlying the heterogeneous nature of such systems. In this review, we summarize the algorithms behind, and practical applications of, SPT. We first cover the theoretical background including particle identification, localization, and trajectory reconstruction. General instrumentation and recent developments to achieve two- and three-dimensional subdiffraction localization and SPT are discussed. We then highlight some applications of SPT to study various biological and synthetic materials systems. Finally, we provide our perspective regarding several directions for future advancements in the theory and application of SPT.

Single-Molecule Tracking and Its Application in Biomolecular Binding Detection

In the past two decades significant advances have been made in single-molecule detection, which enables the direct observation of single biomolecules at work in real time and under physiological conditions. In particular, the development of single-molecule tracking (SMT) microscopy allows us to monitor the motion paths of individual biomolecules in living systems, unveiling the localization dynamics and transport modalities of the biomolecules that support the development of life. Beyond the capabilities of traditional camera-based tracking techniques, state-of-the-art SMT microscopies developed in recent years can record fluorescence lifetime while tracking a single molecule in the 3D space. This multiparameter detection capability can open the door to a wide range of investigations at the cellular or tissue level, including identification of molecular interaction hotspots and characterization of association/dissociation kinetics between molecules. In this review, we discuss various SMT techniques developed to date, with an emphasis on our recent development of the next generation 3D tracking system that not only achieves ultrahigh spatiotemporal resolution but also provides sufficient working depth suitable for live animal imaging. We also discuss the challenges that current SMT techniques are facing and the potential strategies to tackle those challenges.

Selective plane illumination microscopy with a light sheet of uniform thickness formed by an electrically tunable lens

Light sheet microscopy is a powerful technique for rapid, three-dimensional fluorescence imaging of large specimen such as drosophila and zebrafish embryos. Yet, beam divergence results in a loss of axial resolution at the periphery of the light sheet. Here, we demonstrate how an electrically tunable lens can be utilized to maintain the minimal, diffraction-limited thickness of the light sheet over a wide field of view (>600 µm) at high frame rates (40 fps). This mode of operation is necessary for the application of fluorescence fluctuation spectroscopy in images. Microsc. Res. Tech. 81:924-928, 2018. © 2016 Wiley Periodicals, Inc.

Optical measurement of focal offset in tunable lenses

Electrically tunable lenses are becoming a widely used optical tool, and have brought significant innovation to microscopy methods. One current limitation of such systems is the difficulty of directly monitor the focal change in real time. Affordable and reliable feedback for such lenses, compatible with any microscopy setup, represents a much-needed improvement that is still not widely available. We discuss here the implementation and technical performance of an optical device to measure with a high frequency response the displacement of the focal offset of a commercial tunable lens with a precision in the range of the axial Point Spread Function (PSF) of the microscope. The technology presented is cost effective and can be employed on any microscopy setup.