Comparison of methods for end-to-end co-optimization of optical systems and image processing with commercial lens design software

Curriculum learning for ab initio deep learned refractive optics

Deep optical optimization has recently emerged as a new paradigm for designing computational imaging systems using only the output image as the objective. However, it has been limited to either simple optical systems consisting of a single element such as a diffractive optical element or metalens, or the fine-tuning of compound lenses from good initial designs. Here we present a DeepLens design method based on curriculum learning, which is able to learn optical designs of compound lenses ab initio from randomly initialized surfaces without human intervention, therefore overcoming the need for a good initial design. We demonstrate the effectiveness of our approach by fully automatically designing both classical imaging lenses and a large field-of-view extended depth-of-field computational lens in a cellphone-style form factor, with highly aspheric surfaces and a short back focal length.

Generalized aberration regularization in the single-lens computational imaging system design

The restoration algorithm can reduce the aberration blur in the single-lens computational imaging system. The single lens and the restoration algorithm should be co-designed to improve the imaging performance of the whole computational imaging system. By investigating the influence of aberrations on computational imaging quality, we find there are aberrations that cannot be removed by restoration algorithm, which cause the loss of high-resolution information. However, such aberrations are not constrained in existing co-design methods. To solve this problem, we propose the generalized aberration regularization (GAR), which mainly constrains the aberrations that reduce the modulation transfer function (MTF) at the high frequency. Experimental results demonstrate that our approach can significantly improve the quality of high-resolution information in the recovery image compared with the existing co-design methods.

End-to-end aperture layout optimization and image restoration for optical sparse aperture systems

The optical sparse aperture (OSA) imaging technique is capable of improving the spatial resolution of a telescope while maintaining lower size, weight, and cost. The majority of OSA system researches separately focus on the design optimization of aperture layout and the method for image restoration, which have great design redundancy. In this Letter, an end-to-end design framework that simultaneously optimizes the aperture layout parameters of the OSA system and neural network parameters of image restoration is proposed, which achieves excellent imaging quality. The results show that adequate image mid-frequency information captured by the OSA system benefits network processing more than incomplete high-frequency information in a few directions. Based on this framework, we design a simplified OSA system on geostationary orbit. The simulation results show that our simplified OSA system with six sub-apertures measuring 1.2m each has a comparable imaging performance to a single-aperture system measuring 12 m.

Improved performance of a hybrid optical/digital imaging system with fast piecewise Wiener deconvolution

We quantitatively investigate how spatially varying deblurring algorithms can improve the imaging performance of hybrid optical/digital systems affected by field aberrations. To this end, we validate a theoretical model of the maximal gain that linear and spatially varying deblurring can bring to any given lens, and derive a practical algorithm to implement this type of deblurring with low computational complexity. The results demonstrate the usefulness to properly coordinate and balance the roles of the imaging optical system and raw image post-processing: optimal final imaging quality can be obtained by a lens that has been optically designed to reduce field aberrations at the price of lower average raw optical quality, associated with a fast and "slightly" spatially varying piecewise Wiener deconvolution algorithm.