Universal adaptive optics for microscopy through embedded neural network control

The resolution and contrast of microscope imaging is often affected by aberrations introduced by imperfect optical systems and inhomogeneous refractive structures in specimens. Adaptive optics (AO) compensates these aberrations and restores diffraction limited performance. A wide range of AO solutions have been introduced, often tailored to a specific microscope type or application. Until now, a universal AO solution - one that can be readily transferred between microscope modalities - has not been deployed. We propose versatile and fast aberration correction using a physics-based machine learning assisted wavefront-sensorless AO control (MLAO) method. Unlike previous ML methods, we used a specially constructed neural network (NN) architecture, designed using physical understanding of the general microscope image formation, that was embedded in the control loop of different microscope systems. The approach means that not only is the resulting NN orders of magnitude simpler than previous NN methods, but the concept is translatable across microscope modalities. We demonstrated the method on a two-photon, a three-photon and a widefield three-dimensional (3D) structured illumination microscope. Results showed that the method outperformed commonly-used modal-based sensorless AO methods. We also showed that our ML-based method was robust in a range of challenging imaging conditions, such as 3D sample structures, specimen motion, low signal to noise ratio and activity-induced fluorescence fluctuations. Moreover, as the bespoke architecture encapsulated physical understanding of the imaging process, the internal NN configuration was no-longer a "black box", but provided physical insights on internal workings, which could influence future designs.

Generalised adaptive optics method for high-NA aberration-free refocusing in refractive-index-mismatched media

Phase aberrations are introduced when focusing by a high-numerical aperture (NA) objective lens into refractive-index-mismatched (RIM) media. The axial focus position in these media can be adjusted through either optical remote-focusing or mechanical stage translation. Despite the wide interest in remote-focusing, no generalised control algorithm using Zernike polynomials has been presented that performs independent remote-focusing and RIM correction in combination with mechanical stage translation. In this work, we thoroughly review derivations that model high-NA defocus and RIM aberration. We show through both numerical simulation and experimental results that optical remote-focusing using an adaptive device and mechanical stage translation are not optically equivalent processes, such that one cannot fully compensate for the other without additional aberration compensation. We further establish new orthogonal modes formulated using conventional Zernike modes and discuss its device programming to control high-NA remote-focusing and RIM correction as independent primary modes in combination with mechanical stage translation for aberration-free refocusing. Numerical simulations are performed, and control algorithms are validated experimentally by fabricating graphitic features in diamond using direct laser writing.

Three-dimensional adaptive optical nanoscopy for thick specimen imaging at sub-50-nm resolution

Understanding cellular organization demands the best possible spatial resolution in all three dimensions. In fluorescence microscopy, this is achieved by 4Pi nanoscopy methods that combine the concepts of using two opposing objectives for optimal diffraction-limited 3D resolution with switching fluorescent molecules between bright and dark states to break the diffraction limit. However, optical aberrations have limited these nanoscopes to thin samples and prevented their application in thick specimens. Here we have developed an improved iso-stimulated emission depletion nanoscope, which uses an advanced adaptive optics strategy to achieve sub-50-nm isotropic resolution of structures such as neuronal synapses and ring canals previously inaccessible in tissue. The adaptive optics scheme presented in this work is generally applicable to any microscope with a similar beam path geometry involving two opposing objectives to optimize resolution when imaging deep in aberrating specimens.

3D super-resolution deep-tissue imaging in living mice

Stimulated emission depletion (STED) microscopy enables the three-dimensional (3D) visualization of dynamic nanoscale structures in living cells, offering unique insights into their organization. However, 3D-STED imaging deep inside biological tissue is obstructed by optical aberrations and light scattering. We present a STED system that overcomes these challenges. Through the combination of two-photon excitation, adaptive optics, red-emitting organic dyes, and a long-working-distance water-immersion objective lens, our system achieves aberration-corrected 3D super-resolution imaging, which we demonstrate 164 µm deep in fixed mouse brain tissue and 76 µm deep in the brain of a living mouse.

Multi-scale sensorless adaptive optics: application to stimulated emission depletion microscopy

Sensorless adaptive optics is commonly used to compensate specimen-induced aberrations in high-resolution fluorescence microscopy, but requires a bespoke approach to detect aberrations in different microscopy techniques, which hinders its widespread adoption. To overcome this limitation, we propose using wavelet analysis to quantify the loss of resolution due to the aberrations in microscope images. By examining the variations of the wavelet coefficients at different scales, we are able to establish a multi-valued image quality metric that can be successfully deployed in different microscopy techniques. To corroborate our arguments, we provide experimental verification of our method by performing aberration correction experiments in both confocal and STED microscopy using three different specimens.

Complex vectorial optics through gradient index lens cascades

Graded index (GRIN) lenses are commonly used for compact imaging systems. It is not widely appreciated that the ion-exchange process that creates the rotationally symmetric GRIN lens index profile also causes a symmetric birefringence variation. This property is usually considered a nuisance, such that manufacturing processes are optimized to keep it to a minimum. Here, rather than avoiding this birefringence, we understand and harness it by using GRIN lenses in cascade with other optical components to enable extra functionality in commonplace GRIN lens systems. We show how birefringence in the GRIN cascades can generate vector vortex beams and foci, and how it can be used advantageously to improve axial resolution. Through using the birefringence for analysis, we show that the GRIN cascades form the basis of a new single-shot Müller matrix polarimeter with potential for endoscopic label-free cancer diagnostics. The versatility of these cascades opens up new technological directions.

The manufacturing process for GRIN lenses causes a symmetric birefringence variation which is considered a deficiency. Here, the authors show how this birefringence can generate vector vortex beams and form the basis of a Müller matrix polarimeter with potential for endoscopic label-free cancer diagnostics.

Calibration of a phase-only spatial light modulator for both phase and retardance modulation

Liquid crystal spatial light modulators (SLMs) are usually configured and calibrated for phase modulation. However, as they are variable retarders, they also have application as polarization modulators. We show that conventional phase-only calibrations are insufficient for this purpose, and a separate retardance calibration is needed. To overcome this shortcoming we report a simple Twyman-Green interferometer-based setup to realize SLM phase and retardance calibration. For phase calibration, we identify the non-linear, spatially variant response to the drive voltage of the SLM using fringe analysis and both horizontally and vertically polarized incident light. For retardance calibration, we use incident light polarized at 45° and assess the intensity variation. The methods presented are compatible with in situ calibration of SLMs.

Aberrations in stimulated emission depletion (STED) microscopy

Like all methods of super-resolution microscopy, stimulated emission depletion (STED) microscopy can suffer from the effects of aberrations. The most important aspect of a STED microscope is that the depletion focus maintains a minimum, ideally zero, intensity point that is surrounded by a region of higher intensity. It follows that aberrations that cause a non-zero value of this minimum intensity are the most detrimental, as they inhibit fluorescence emission even at the centre of the depletion focus. We present analysis that elucidates the nature of these effects in terms of the different polarisation components at the focus for two-dimensional and three-dimensional STED resolution enhancement. It is found that only certain low-order aberration modes can affect the minimum intensity at the Gaussian focus. This has important consequences for the design of adaptive optics aberration correction systems.

Aberrations in 4Pi Microscopy

The combination of two opposing objective lenses in 4Pi fluorescence microscopy significantly improves the axial resolution and increases the collection efficiency. Combining 4Pi microscopy with other super-resolution techniques has resulted in the highest three-dimensional (3D) resolution in fluorescence microscopy to date. It has previously been shown that the performance of 4Pi microscopy is significantly affected by aberrations. However, a comprehensive description of 4Pi microscope aberrations has been missing. In this paper, we introduce an approach to describe aberrations in a 4Pi cavity through a new functional representation. We discuss the focusing properties of 4Pi systems affected by aberrations and discuss the implications for adaptive optics schemes for 4Pi microscopes based on this new insight.

Assignment of Opsonic Values to Pneumococcal Reference Serum 007sp for Use in Opsonophagocytic Assays for 13 Serotypes

Opsonophagocytic assays (OPAs) are routinely used for assessing the immunogenicity of pneumococcal vaccines, with OPA data often being utilized for licensure of new vaccine formulations. However, no reference serum for pneumococcal OPAs is available, making evaluation of data among different laboratories difficult. This international collaboration was initiated to (i) assign consensus opsonic indexes (OIs) to FDA pneumococcal reference serum lot 007sp (here referred to as 007sp) and a panel of serum samples used for calibration of the OPA and (ii) determine if the normalization of the OPA results obtained with test samples to those obtained with 007sp decreases the variability in OPA results among laboratories. To meet these goals, six participating laboratories tested a panel of serum samples in five runs for 13 serotypes. For each serum sample, consensus OIs were obtained using a mixed-effects analysis of variance model. For the calibration serum samples, normalized consensus values were also determined on the basis of the results obtained with 007sp. For each serotype, the overall reduction in interlaboratory variability was calculated by comparing the coefficients of variation of the unadjusted and the normalized values. Normalization of the results substantially reduced the interlaboratory variability, ranging from a 15% reduction in variability for serotype 9V to a 64% reduction for serotype 7F. Normalization also increased the proportion of data within 2-fold of the consensus value from approximately 70% (average for all serotypes) to >90%. On the basis of the data obtained in this study, pneumococcal reference standard lot 007sp will likely be a useful reagent for the normalization of pneumococcal OPA results from different laboratories. The data also support the use of the 16 FDA serum samples used for calibration of the OPA as part of the initial evaluation of new assays or periodic assessment of established assays.

Coma aberrations in combined two- and three-dimensional STED nanoscopy

Stimulated emission depletion (STED) microscopes, like all super-resolution methods, are sensitive to aberrations. Of particular importance are aberrations that affect the quality of the depletion focus, which requires a point of near-zero intensity surrounded by strong illumination. We present analysis, modeling, and experimental measurements that show the effects of coma aberrations on depletion patterns of two-dimensional (2D) and three-dimensional (3D) STED configurations. Specifically, we find that identical coma aberrations create focal shifts in opposite directions in 2D and 3D STED. This phenomenon could affect the precision of microscopic measurements and has ramifications for the efficacy of combined 2D/3D STED systems.

Modal-based phase retrieval for adaptive optics

We consider using phase retrieval (PR) to correct phase aberrations in an optical system. Three measurements of the point-spread function (PSF) are collected to estimate an aberration. For each measurement, a different defocus aberration is applied with a deformable mirror (DM). Once the aberration is estimated using a PR algorithm, we apply the aberration correction with the DM, and measure the residual aberration using a Shack-Hartmann wavefront sensor. The extended Nijboer-Zernike theory is used for modelling the PSF. The PR problem is solved using both an algorithm called PhaseLift, which is based on matrix rank minimization, and another algorithm based on alternating projections. For comparison, we include the results achieved using a classical PR algorithm, which is based on alternating projections and uses the fast Fourier transform.

Optimization-based wavefront sensorless adaptive optics for multiphoton microscopy

Optical aberrations have detrimental effects in multiphoton microscopy. These effects can be curtailed by implementing model-based wavefront sensorless adaptive optics, which only requires the addition of a wavefront shaping device, such as a deformable mirror (DM) to an existing microscope. The aberration correction is achieved by maximizing a suitable image quality metric. We implement a model-based aberration correction algorithm in a second-harmonic microscope. The tip, tilt, and defocus aberrations are removed from the basis functions used for the control of the DM, as these aberrations induce distortions in the acquired images. We compute the parameters of a quadratic polynomial that is used to model the image quality metric directly from experimental input-output measurements. Finally, we apply the aberration correction by maximizing the image quality metric using the least-squares estimate of the unknown aberration.

Snapshot coherence-gated direct wavefront sensing for multi-photon microscopy

Deep imaging in turbid media such as biological tissue is challenging due to scattering and optical aberrations. Adaptive optics has the potential to compensate the tissue aberrations. We present a wavefront sensing scheme for multi-photon scanning microscopes using the pulsed, near-infrared light reflected back from the sample utilising coherence gating and a confocal pinhole to isolate the light from a layer of interest. By interfering the back-reflected light with a tilted reference beam, we create a fringe pattern with a known spatial carrier frequency in an image of the back-aperture plane of the microscope objective. The wavefront aberrations distort this fringe pattern and thereby imprint themselves at the carrier frequency, which allows us to separate the aberrations in the Fourier domain from low spatial frequency noise. A Fourier analysis of the modulated fringes combined with a virtual Shack-Hartmann sensor for smoothing yields a modal representation of the wavefront suitable for correction. We show results with this method correcting both DM-induced and sample-induced aberrations in rat tail collagen fibres as well as a Hoechst-stained MCF-7 spheroid of cancer cells.

Semidefinite programming for model-based sensorless adaptive optics

Wavefront sensorless adaptive optics methodologies are widely considered in scanning fluorescence microscopy where direct wavefront sensing is challenging. In these methodologies, aberration correction is performed by sequentially changing the settings of the adaptive element until a predetermined image quality metric is optimized. An efficient aberration correction can be achieved by modeling the image quality metric with a quadratic polynomial. We propose a new method to compute the parameters of the polynomial from experimental data. This method guarantees that the quadratic form in the polynomial is semidefinite, resulting in a more robust computation of the parameters with respect to existing methods. In addition, we propose an algorithm to perform aberration correction requiring a minimum of N+1 measurements, where N is the number of considered aberration modes. This algorithm is based on a closed-form expression for the exact optimization of the quadratic polynomial. Our arguments are corroborated by experimental validation in a laboratory environment.

Renal transplantation in children younger than 6 years old

Herein we report our experience in renal transplantation in 38 children (40 transplants), ages 1 to 5 years, between 1989 and 2005. Demographics as well as patient and graft survivals are reported. Mean age at transplantation was 3.3 +/- 1.3 years, and mean weight was 14 kg (range, 5.7-25 kg); 92.5% were Caucasian, 7.5% African-Brazilian. The main etiology for end-stage renal disease (ESRD) was uropathic/vesicoureteral reflux (45%) followed by glomerulopathy (25%), congenital/hereditary diseases (10%), and hemolytic uremic syndrome (12.5%). Prior to transplantation, 5% were on hemodialysis, 85% on peritoneal dialysis, and 10% preemptive. All children were followed for at least 6 months posttransplantation, except 2 who died in the first month. In 75% of cases, kidneys were obtained from living-related donors, and in 25% from deceased donors. Thirty-nine kidneys were extraperitoneally placed. Primary immunosuppressant therapy consisted of cyclosporine (61%), tacrolimus (39%), mycophenolate (49%), and azathioprine (51%). A steroid-free protocol was used in 17% of patients. In the last 21 cases, basiliximab or daclizumab was added. There were 13 (32.5%) graft losses (4 artery/vein thromboses, 3 chronic rejections, 3 deaths, 3 other causes). The 5-year patient and graft survival rates were 89.6% and 72.2%. We have concluded that renal transplantation can be performed with good long-term results in children younger than 6 years old.

Dose--response effect of famotidine on patterns of gastro-oesophageal reflux

The present study attempts to assess the alteration in patterns of gastro-oesophageal reflux as assessed by 24-h oesophageal pH monitoring by varying degrees of H2-receptor blockade with famotidine. Subjects were 12 patients with complaints of daily heartburn who demonstrated at least 6% of acid mucosal contact time by 24-h ambulatory oesophageal pH monitoring. All subjects had a positive Bernstein test, and nine of the 12 subjects had erosive oesophagitis. The study was conducted as a double-blind crossover design utilizing 40 mg nocte, 20 mg b.d., and 40 mg b.d. and placebo treatments. Results indicated that all treatments significantly reduced the 24-h percentage acid contact time (P less than 0.05) compared to placebo. The two b.d. treatment regimens also significantly (P less than 0.05) reduced the number of episodes lasting longer than 5 min. Only the b.d. regimens successfully lowered the percentage of upright acid exposure. All treatments significantly (P less than 0.01) reduced the percentage of supine acid contact time, as well as the number of episodes lasting more than 5 min. It is concluded that gastro-oesophageal reflux disease may well require a b.d. dosing regimen with famotidine in order to achieve optimal mucosal healing and day time symptom control.

Effects of age and chronic renal failure on the urinary excretion kinetics of famotidine in man

The plasma and urine concentrations of famotidine, a new, potent H2-receptor antagonist, have been measured in 16 healthy young adults, 8 healthy elderly people and 18 patients with varying degrees of renal dysfunction after intravenous administration. Both the plasma elimination and renal excretion of famotidine were decreased in the elderly volunteers and renal patients. The renal clearance of famotidine averaged 4.43 ml/min/kg (310 ml/min) in normal young volunteers, which exceeded the mean creatinine clearance 1.55 ml/min/kg (109 ml/min), suggesting net secretion is a significant mechanism for elimination of famotidine. The ratio of famotidine renal clearance to creatinine clearance decreased as creatinine clearance decreased; these results suggest that the deterioration in the secretion process was much faster than that in glomerular filtration and are incompatible with the "intact nephron hypothesis". Nevertheless, both total body clearance and renal clearance were significantly correlated with creatinine clearance. The apparent half-life was also significantly correlated with creatinine clearance. Since famotidine is essentially free of dose-related adverse effects, dose adjustment in patients with mild renal insufficiency and in elderly people is not required; however, either a prolonged dosing interval or a decrease in daily dose during long-term therapy may be adapted for the patients with severe renal insufficiency to avoid accumulation and the potential undesirable effects.

Single-dose pharmacokinetics and bioavailability of famotidine in man. Results of multicenter collaborative studies

Pharmacokinetics and bioavailability of famotidine, a new H2-receptor antagonist, were investigated in healthy subjects in five clinical studies. Linear pharmacokinetics were observed following either intravenous or oral administration. Plasma clearance averaged 463 ml min-1. Renal clearance averaged 310 ml min-1, which exceeded the glomerular filtration rate. Renal excretion was the major route of elimination. Urinary recovery of unchanged drug following intravenous administration was about 67 per cent. Famotidine plasma half-life was approximately 2.6 h. Oral absorption was incomplete. The bioavailability averaged 43 per cent of the dose.