Review of state-of-the-art artificial compound eye imaging systems

Continuous Optical Zoom Compound Eye Imaging Using Alvarez Lenses Actuated by Dielectric Elastomers

The compound eye is a natural multi-aperture optical imaging system. In this paper, a continuous optical zoom compound eye imaging system based on Alvarez lenses is proposed. The main optical imaging part of the proposed system consists of a curved Alvarez lens array (CALA) and two Alvarez lenses. The movement of the CALA and two Alvarez lenses perpendicular to the optical axis is realized by the actuation of the dielectric elastomers (DEs). By adjusting the focal length of the CALA and the two Alvarez lenses, the proposed system can realize continuous zoom imaging without any mechanical movement vertically to the optical axis. The experimental results show that the paraxial magnification of the target can range from ∼0.30× to ∼0.9×. The overall dimensions of the optical imaging part are 54 mm × 36 mm ×60 mm (L × W × H). The response time is 180 ms. The imaging resolution can reach up to 50 lp/mm during the optical zoom process. The proposed continuous optical zoom compound eye imaging system has potential applications in various fields, including large field of view imaging, medical diagnostics, machine vision, and distance detection.

Miniature bioinspired artificial compound eyes: microfabrication technologies, photodetection and applications

As an outstanding visual system for insects and crustaceans to cope with the challenges of survival, compound eye has many unique advantages, such as wide field of view, rapid response, infinite depth of field, low aberration and fast motion capture. However, the complex composition of their optical systems also presents significant challenges for manufacturing. With the continuous development of advanced materials, complex 3D manufacturing technologies and flexible electronic detectors, various ingenious and sophisticated compound eye imaging systems have been developed. This paper provides a comprehensive review on the microfabrication technologies, photoelectric detection and functional applications of miniature artificial compound eyes. Firstly, a brief introduction to the types and structural composition of compound eyes in the natural world is provided. Secondly, the 3D forming manufacturing techniques for miniature compound eyes are discussed. Subsequently, some photodetection technologies for miniature curved compound eye imaging are introduced. Lastly, with reference to the existing prototypes of functional applications for miniature compound eyes, the future development of compound eyes is prospected.

Development of a bio-inspired optical system that mimics accommodation and lighting regulation like the human eye

Bio-inspired optical systems have recently been developed using polarizers and liquid or rigid lenses. In this work, we propose a bio-inspired opto-mechatronic system that imitates the accommodation and regulation of light intensity as the human eye does. The system uses a polymeric lens as a cornea, an adjustable diaphragm as an iris, a tunable solid elastic lens as a crystalline lens, and a commercial sensor as a retina. We also present the development of the electronic control system to accommodate and regulate the amount of light that enters the system, for which two stepper motors, an Arduino control system, and light and movement sensors are used. The characterization of the system is presented together with the results obtained, where it can be seen that the system works in an acceptable range as the human eye does.

Optimal vector matching fusion method for bionic compound eye polarization compass and inertial sensor integration

The compound eyes of insects can extract the heading by sensing multi-directional polarization information. Inspired by this, the integration with bionic eye polarization compass and low-precision MEMS inertial sensors yields a new choice for autonomous navigation in a GPS-denied environment. However, the occlusion environments, such as jungles and buildings, seriously affects the attitude estimation accuracy by destroying the polarization information. Considering the above problem, this paper proposes a reliable attitude estimation method based on vector matching measurement models for integration with bionic compound eye polarization compass and low-cost MEMS inertial sensors, including the polarization vector, solar vector, and gravity vector. To realize the matching and fusion among these vector models, the vector optimization selection algorithm is designed according to the number of visible polarization sensors in the occlusion environment. Particularly, the vector optimization selection factor is designed according to the error between the measured and theoretical polarization vectors. Furthermore, when using the solar vector estimation model, the degree of the polarization is applied to improve the calculation accuracy of the solar vector. The gravity vector measurement model is employed to enhance the autonomy of the integration. Finally, the effectiveness of the proposed method is verified by simulated and outdoor experiments under tree obscuration.

Biomimetic Compound Eyes with Gradient Ommatidium Arrays

Compound eyes are high-performing natural optical perception systems with compact configurations, generating extensive research interest. Existing compound eye systems are often combinations of simple uniform microlens arrays; there are still challenges in making more ommatidia on the compound eye surface to focus to the same plane. Here, a biomimetic gradient compound eye is presented by artificially mimicking dragonflies. The multiple replication process efficiently endows compound eyes with the gradient characteristics of dragonfly compound eyes. Experimental results show that the manufactured compound eye allows multifocus imaging by virtue of the gradient ommatidium array arranged closely in a honeycomb pattern while ensuring excellent optical properties and compact configurations. Thousands of ommatidia showing a gradient trend at the millimeter scale while remaining relatively uniform at the micron scale have gradient focal lengths ranging from 260 to 450 μm. This gradient compound eye allows more ommatidia to focus on the same plane than traditional uniform compound eyes, which have experimentally been shown to capture more than 1100 in-plane clear images simultaneously, promising potential applications in micro-optical devices, optical imaging, and biochemical sensing.

Self-calibration algorithm for installation angle deviation of bionic polarization compound eyes

A self-calibration algorithm based on unsupervised optimization for polarizer installation angle deviation is proposed and used in a multi-aperture bionic polarization compound eye system. To simplify calibration operation, under the condition that the calibration-polarized light information is unknown, this algorithm fully exploits redundancy and random polarization information in the scene, and uses a non-convex multi-objective discrete parameter sorting optimization method to achieve angle self-calibration. Compared with ordinary calibration procedures, the algorithm requires less stringent conditions, achieves online calibration and is more accurate. It also can be applied to camera polarization arrays, division-of-focal-plane polarization cameras, and other polarization devices.

Measurement Technologies of Light Field Camera: An Overview

Visual measurement methods are extensively used in various fields, such as aerospace, biomedicine, agricultural production, and social life, owing to their advantages of high speed, high accuracy, and non-contact. However, traditional camera-based measurement systems, relying on the pinhole imaging model, face challenges in achieving three-dimensional measurements using a single camera by one shot. Moreover, traditional visual systems struggle to meet the requirements of high precision, efficiency, and compact size simultaneously. With the development of light field theory, the light field camera has garnered significant attention as a novel measurement method. Due to its special structure, the light field camera enables high-precision three-dimensional measurements with a single camera through only one shot. This paper presents a comprehensive overview of light field camera measurement technologies, including the imaging principles, calibration methods, reconstruction algorithms, and measurement applications. Additionally, we explored future research directions and the potential application prospects of the light field camera.

Toward next-generation endoscopes integrating biomimetic video systems, nonlinear optical microscopy, and deep learning

According to the World Health Organization, the proportion of the world's population over 60 years will approximately double by 2050. This progressive increase in the elderly population will lead to a dramatic growth of age-related diseases, resulting in tremendous pressure on the sustainability of healthcare systems globally. In this context, finding more efficient ways to address cancers, a set of diseases whose incidence is correlated with age, is of utmost importance. Prevention of cancers to decrease morbidity relies on the identification of precursor lesions before the onset of the disease, or at least diagnosis at an early stage. In this article, after briefly discussing some of the most prominent endoscopic approaches for gastric cancer diagnostics, we review relevant progress in three emerging technologies that have significant potential to play pivotal roles in next-generation endoscopy systems: biomimetic vision (with special focus on compound eye cameras), non-linear optical microscopies, and Deep Learning. Such systems are urgently needed to enhance the three major steps required for the successful diagnostics of gastrointestinal cancers: detection, characterization, and confirmation of suspicious lesions. In the final part, we discuss challenges that lie en route to translating these technologies to next-generation endoscopes that could enhance gastrointestinal imaging, and depict a possible configuration of a system capable of (i) biomimetic endoscopic vision enabling easier detection of lesions, (ii) label-free in vivo tissue characterization, and (iii) intelligently automated gastrointestinal cancer diagnostic.

Superhydrophobic and easy-to-clean full-packing nanopatterned microlens array with high-quality imaging

The high-quality imaging and easy cleaning property of microlens array (MLA) are two very important factors for its outdoor work. Herein, a superhydrophobic and easy-to-clean full-packing nanopatterned MLA with high-quality imaging is prepared by thermal reflow together with sputter deposition. Scanning electronic microscopy (SEM) images demonstrate that the sputter deposition method can improve 84% packing density of MLA prepared by thermal reflow to 100% and add nanopattern on the surface of microlens. The prepared full-packing nanopatterned MLA (npMLA) possess clear imaging with a significant increase of signal-to-noise ratio and higher transparency compared with the MLA prepared by thermal reflow. Besides for excellent optical properties, the full-packing surface displays a superhydrophobic property with a contact angle of 151.3°. Further, the full-packing contaminated by chalk dust become easier to be cleaned by nitrogen blowing and deionized water. As a result, the prepared full-packing is considered to be potential for various applications in the outdoor.

A monocular wide-field speed sensor inspired by the crabs' visual system for traffic analysis

The development of visual sensors for traffic analysis can benefit from mimicking two fundamental aspects of the visual system of crabs: their panoramic vision and their visual processing strategy adapted to a flat world. First, the use of omnidirectional cameras in urban environments allows for analyzing the simultaneous movement of many objects of interest over broad areas. This would reduce the costs and complications associated with infrastructure: installation, synchronization, maintenance, and operation of traditional vision systems that use multiple cameras with a limited field of view. Second, in urban traffic analysis, the objects of interest (e.g. vehicles and pedestrians) move on the ground surface. This constraint allows the calculation of the 3D trajectory of the vehicles using a single camera without the need to use binocular vision techniques.The main contribution of this work is to show that the strategy used by crabs to visually analyze their habitat (monocular omnidirectional vision with the assumption of a flat world ) is useful for developing a simple and effective method to estimate the speed of vehicles on long trajectories in urban environments. It is shown that the proposed method estimates the speed with a root mean squared error of 2.7 km h^-1.

A Meniscus Multifocusing Compound Eye Camera Based on Negative Pressure Forming Technology

To meet the challenge of preparing a high-resolution compound eye, this paper proposes a multi-focal-length meniscus compound eye based on MEMS negative pressure molding technology. The aperture is increased, a large field of view angle of 101.14° is obtained, and the ommatidia radius of each stage is gradually increased from 250 μm to 440 μm. A meniscus structure is used to improve the imaging quality of the marginal compound eye so that its resolution can reach 36.00 lp/mm. The prepared microlenses have a uniform shape and a smooth surface, and both panoramic image stitching and moving object tracking are achieved. This technology has great potential for application in many fields, including automatic driving, machine vision, and medical endoscopy.

Apposition compound-eye image scanner by glass plate optics

We propose a design approach for a thin image scanner using the concept of an apposition compound eye comprising many imaging units that take only one pixel image. Although light shielding between adjacent imaging units is always one of the main issues for an artificial compound eye, a simple plane structure using three aperture array layers on two glued glass plates prevents such stray light. Our prototyped scanner, with only 6.8-mm thickness as a packaged module, has 632 microlenses with 200-dpi resolution, resulting in a field of view of 80 mm. The evaluated images show no ghost images.

A bio-inspired polarization navigation sensor based on artificial compound eyes

Insect compound eyes are optical systems with small volume and a compact structure. The ommatidia in the dorsal rim area of some insects have polarized vision, which can perceive the polarization pattern of the sky and provide them with navigation information. In this paper, inspired by the polarization-sensitive compound eyes of insects, a bio-inspired polarization navigation sensor based on artificial compound eyes is designed. The sensor consists of an artificial compound eye, an integrated polarization detector and an integrated circuit. The optical path of the sensor uses the lens defocus method, which can ensure that the sensor obtains redundant polarization information. The integrated polarization detector is used to obtain the polarization information of the incident light, and the integrated circuit is responsible for the calculation. To extract effective information from images, we propose a multi-threshold segmentation method to filter and classify effective pixels. We use the least squares method to fit the inherent error of the sensor and then compensate it. The indoor calibration accuracy of the sensor is ±0.3°, and the outdoor calibration accuracy is ±0.5°. The sensor can provide accurate direction information for general smart mobile devices. The size of the sensor is 4 × 4 × 2 cm, and the weight is only 15 g. The key components of the sensor can be mass-produced, and it is a miniaturized and low-cost polarization navigation sensor.

Bio-inspired spherical compound eye camera for simultaneous wide-band and large field of view imaging

Natural compound eyes have excellent optical characteristics, namely large field of view, small size, no aberration, and sensitive to motion. Some arthropods have more powerful vision. For example, the Morpho butterfly's compound eyes can perceive the near-infrared and ultraviolet light that the human eye cannot see. This wide-band imaging with a large field of view has great potential in wide-area surveillance, all-weather panoramic imaging, and medical imaging. Hence, a wide-band spherical compound eye camera inspired by the Morpho butterfly's eye was proposed. The wide-band spherical compound eye camera which can achieve a large field of view (360° × 171°) imaging over a wide range of wavelengths from 400nm to 1000nm, mainly consists of three parts: a wide-band spherical compound eye with 234 sub-eyes for light collection, a wide-band optical relay system for light transmission, and a wide-band CMOS image sensor for photoelectric conversion. Our experimental results show that the wide-band spherical compound eye camera not only captures a large field of view without anomalous blurring or aberrations but also perceives near-infrared light that is not recognized by the human eye. These features make it possible for distortion-free panoramic vision and panoramic medical diagnosis.

A biomimetic compound eye lens for photocurrent enhancement at low temperatures

In this study, an artificial compound eye lens (ACEL) was fabricated using a laser cutting machine and polyvinyl alcohol (PVA) solution. A laser cutter was used to punch micro-sized holes (500 μm diameter-the smallest possible diameter) into an acrylic plate; this punched plate was then placed on the aqueous PVA solution, and the water was evaporated. The plate was used as the mold to obtain a polydimethylsiloxane (PDMS) micro lens array film, which was fixed to a dome-shaped three-dimensional-printed mold for further PDMS curing, and a hemispherical compound eye lens was obtained. Using a gallium nitride (GaN) photodetector, a light detection experiment was performed with the ACEL, bare lens, and no lens by irradiating light at various angles under low temperatures. The photodetector with the ACEL generated a high photocurrent under several conditions. In particular, when the light was irradiated at 0° and below -20 °C, the photocurrent of the GaN sensor with the ACEL increased by 61% and 81% compared with the photocurrent of the GaN sensor with the bare lens and without a lens, respectively. In this study, a sensor for detecting light with ACEL was demonstrated in low-temperature environments, such as indoor refrigerated storages and external conditions in Antarctica and Arctic.

Underwater blue-green LED communication using a double-layered, curved compound-eye optical system

Optical receiving systems with single-lens structures have problems such as low receiving efficiency and small field of view when applied to underwater optical wireless communication systems. In this study, a design scheme for a double-layered fly-eye-lens optical system with wide-angle focusing is proposed. Based on the analysis of the LED light source transmission model and seawater channel, the optical-power receiving equation of the fly-eye lens system is deduced. The fly-eye-lens receiving system was designed and simulated using Zemax according to the geometrical optics principle of the lens array. An experimental device for measuring the insertion loss and receiving efficiency of an underwater blue-green LED communication system was built, and the optical power of the receiving optical system was experimentally measured. For the link distances of 1, 3, and 5 m, the received optical power of the double-layered-compound eye system was higher than that of the single-layered system, with a power increase of 72%, 65%, and 60%, respectively. The results show that the double-layered fly-eye-lens receiving antenna can significantly improve the optical power received by the receiving end; therefore, this antenna structure has strong practicability and good development prospects in the field of underwater optical wireless communication.

Design and demonstration of a six-channel multiresolution imaging system

Multichannel imaging systems consist of multiple channels that have different imaging characteristics (fields-of-view and angular resolutions). We design and demonstrate a six-channel multiresolution imaging system that can achieve a relative magnification ratio of up to 10 times between the channels and at the same time result in different depths-of-field. The imaging system consists of two double-sided lens arrays made of PMMA material, a baffle to eliminate possible cross talk between neighboring channels, and a Sony full-frame image sensor. The imaging system was tested by capturing images of stationary and moving objects. The acquired images exhibit different resolutions, fields-of-view, and levels of blur proving our concept.

Bioinspired Compound Eyes for Diffused Light-Harvesting Application

Natural compound eyes endow arthropods with wide-field high-performance light-harvesting capability that enables them to capture prey and avoid natural enemies in dim light. Inspired by natural compound eyes, a curved artificial-compound-eye (cACE) photodetector for diffused light harvesting is proposed and fabricated, and its light-harvesting capability is systematically investigated. The cACE photodetector is fabricated by introducing a cACE as a light-harvesting layer on the surface of a silicon-based photodetector, with the cACE being prepared via planar artificial-compound-eye (pACE) template deformation. The distinctive geometric morphology of the as-prepared cACE effectively reduces its surface reflection and the dependence of the projected area on the incident light direction, thereby significantly improving the light-harvesting ability and output photocurrent of the silicon-based photodetector. Furthermore, the performances of cACE, pACE, and bare polydimethylsiloxane (PDMS)-attached photodetectors as diffused light detectors are investigated under different luminances. The cACE-photodetector output photocurrent is 1.395 and 1.29 times those of the bare PDMS-attached and pACE photodetectors, respectively. Moreover, this photodetector has a desirable geometric shape. Thus, the proposed cACE photodetector will facilitate development of high-performance photodetectors for luminance sensing.

Digital optofluidic compound eyes with natural structures and zooming capability for large-area fluorescence sensing

Compound eyes are ubiquitous natural biosensors that possess high temporal resolution and large fields of view (FOVs). While for solid materials based artificial imaging systems, flexible zooming ability while keeping the constant FOV is still challenging, as well as the low-cost fabrication. Herein, liquid compound eyes with natural structures are presented that synthesize optofluidics and bionics in a non-trivial manner, which enables the deformation-free zooming and flexible cell fluorescence sensing. Experimental results indicate that the innovatively manufactured bionic template possesses low roughness and uniform lens configuration with more than two thousands units, which endows the eyes with high-quality and low aberration imaging ability. Besides, digital controlled miscible liquids switching enables the focus of ommatidia simultaneously be adjusted from 150 μm to 5 mm with 100° view angle, and without bending the microlens curvature, to avoid FOV changing and image aberration. Due to large FOV and tunable ability, large-area cell fluorescence signal arrays and dynamic cell motion are imaged using this liquid compound eyes. This work presents novel strategy for compound lens manufacture at low-cost, and proposes deformation-free and continuous focus-tuning strategy, offering potentials for numerous applications, including biomedical sensing and adaptive imaging with large FOV.

Soft Array Surface-Changing Compound Eye

The field-of-view (FOV) of compound eyes is an important index for performance evaluation. Most artificial compound eyes are optical, fabricated by imitating insect compound eyes with a fixed FOV that is difficult to adjust over a wide range. The compound eye is of great significance in the field of tracking high-speed moving objects. However, the tracking ability of a compound eye is often limited by its own FOV size and the reaction speed of the rudder unit matched with the compound eye, so that the compound eye cannot better adapt to tracking high-speed moving objects. Inspired by the eyes of many organisms, we propose a soft-array, surface-changing compound eye (SASCE). Taking soft aerodynamic models (SAM) as the carrier and an infrared sensor as the load, the basic model of the variable structure infrared compound eye (VSICE) is established using an array of infrared sensors on the carrier. The VSICE model is driven by air pressure to change the array surface of the infrared sensor. Then, the spatial position of each sensor and its viewing area are changed and, finally, the FOV of the compound eye is changed. Simultaneously, to validate the theory, we measured the air pressure, spatial sensor position, and the FOV of the compound eye. When compared with the current compound eye, the proposed one has a wider adjustable FOV.

Continuous zoom compound eye imaging system based on liquid lenses

In this paper, a continuous zoom compound eye imaging system based on liquid lenses is proposed. The main imaging part of the system consists of a liquid compound eye, two liquid lenses and a planar image sensor. By adjusting the liquid injection volumes of the liquid compound eye and liquid lenses, the system can realize continuous zoom imaging without any mechanical movement of imaging components. According to the results of experiments, the paraxial magnification of the target can range from ∼0.019× to ∼0.037× at a fixed working distance. Moreover, the system can realize continuous focusing at a fixed paraxial magnification when the working distance ranges from ∼200mm to ∼300mm. Compared with the traditional artificial compound eye imaging systems, the proposed system increases the adjustability and matches the variable image surfaces of the liquid compound eye to a planar image sensor. The aspherical effects of the liquid compound eye and liquid lenses are also considered in the design of the system. The system is expected to be used for imaging in various scenes, such as continuous zoom panoramic imaging, 3D scanning measurement and so on.

Retina-like Imaging and Its Applications: A Brief Review

The properties of the human eye retina, including space-variant resolution and gaze characters, provide many advantages for numerous applications that simultaneously require a large field of view, high resolution, and real-time performance. Therefore, retina-like mechanisms and sensors have received considerable attention in recent years. This paper provides a review of state-of-the-art retina-like imaging techniques and applications. First, we introduce the principle and implementing methods, including software and hardware, and describe the comparisons between them. Then, we present typical applications combined with retina-like imaging, including three-dimensional acquisition and reconstruction, target tracking, deep learning, and ghost imaging. Finally, the challenges and outlook are discussed to further study for practical use. The results are beneficial for better understanding retina-like imaging.

Artificial Compound Eye Systems and Their Application: A Review

The natural compound eye system has many outstanding properties, such as a more compact size, wider-angle view, better capacity to detect moving objects, and higher sensitivity to light intensity, compared to that of a single-aperture vision system. Thanks to the development of micro- and nano-fabrication techniques, many artificial compound eye imaging systems have been studied and fabricated to inherit fascinating optical features of the natural compound eye. This paper provides a review of artificial compound eye imaging systems. This review begins by introducing the principle of the natural compound eye, and then, the analysis of two types of artificial compound eye systems. We equally present the applications of the artificial compound eye imaging systems. Finally, we suggest our outlooks about the artificial compound eye imaging system.

The Interface Thermal Resistance Evolution between Carbide-Bonded Graphene Coating and Polymer in Rapid Molding for Microlens Array

Surface rapid heating process is an efficient and green method for large-volume production of polymer optics by adopting 3D graphene network coated silicon molds with high thermal conductivity. Nevertheless, the heat transfer mechanism including the interface thermal resistance evolution between 3D graphene network coating and polymer has not been thoroughly revealed. In this study, the interface thermal resistance model was established by simplifying the contact situation between the coating and polymethylmethacrylate (PMMA), and then embedding into the finite element method (FEM) model to study the temperature variations of PMMA in surface rapid heating process. Heating experiments for graphene network were then carried out under different currents to provide the initial heat for heat transfer model. In addition, residual stress of the PMMA lens undergoing the non-uniform thermal history during molding was presented by the simulation model together. Finally, the optimal molding parameters including heating time and pressure will be determined according to calculation results of the interface thermal resistance model and microlens array molding experiment was conducted to illustrate that the interface thermal resistance model can predict the temperature of the polymer to achieve a better filling of microlens array with smooth surface and satisfactory optical performance.

Materials, Actuators, and Sensors for Soft Bioinspired Robots

Biological systems can perform complex tasks with high compliance levels. This makes them a great source of inspiration for soft robotics. Indeed, the union of these fields has brought about bioinspired soft robotics, with hundreds of publications on novel research each year. This review aims to survey fundamental advances in bioinspired soft actuators and sensors with a focus on the progress between 2017 and 2020, providing a primer for the materials used in their design.

Design method for the high optical efficiency and uniformity illumination system of the projector

How to balance the optical efficiency, illumination uniformity and the size of the illumination system is a challenging task in projector design. In this paper, we present a mathematical model describing the relationship between optical energy of the illumination system and the optical parameters and an optimization design method considering the light intensity distribution of the light source. By using the proposed method, two illumination systems are designed with different types of the digital micromirror device chips. In addition, we also propose a non-coaxial system to solve the deformation problem caused by the large flip angle of the DMD chip and further improve the illumination uniformity based on Scheimpflug principle. The optical efficiency and illumination uniformity of the illumination systems were verified and analyzed. The results indicate that the systems designed by the proposed method can provide a good design scheme and obtain a satisfactory utilization rate of the optical energy and higher uniformity.

Design and Integration of the Single-Lens Curved Multi-Focusing Compound Eye Camera

Compared with a traditional optical system, the single-lens curved compound eye imaging system has superior optical performance, such as a large field of view (FOV), small size, and high portability. However, defocus and low resolution hinder the further development of single-lens curved compound eye imaging systems. In this study, the design of a nonuniform curved compound eye with multiple focal lengths was used to solve the defocus problem. A two-step gas-assisted process, which was combined with photolithography, soft photolithography, and ultraviolet curing, was proposed for fabricating the ommatidia with a large numerical aperture precisely. Ommatidia with high resolution were fabricated and arranged in five rings. Based on the imaging experimental results, it was demonstrated that the high-resolution and small-volume single-lens curved compound eye imaging system has significant advantages in large-field imaging and rapid recognition.

Fully Light-Controlled Memory and Neuromorphic Computation in Layered Black Phosphorus

Imprinting vision as memory is a core attribute of human cognitive learning. Fundamental to artificial intelligence systems are bioinspired neuromorphic vision components for the visible and invisible segments of the electromagnetic spectrum. Realization of a single imaging unit with a combination of in-built memory and signal processing capability is imperative to deploy efficient brain-like vision systems. However, the lack of a platform that can be fully controlled by light without the need to apply alternating polarity electric signals has hampered this technological advance. Here, a neuromorphic imaging element based on a fully light-modulated 2D semiconductor in a simple reconfigurable phototransistor structure is presented. This standalone device exhibits inherent characteristics that enable neuromorphic image pre-processing and recognition. Fundamentally, the unique photoresponse induced by oxidation-related defects in 2D black phosphorus (BP) is exploited to achieve visual memory, wavelength-selective multibit programming, and erasing functions, which allow in-pixel image pre-processing. Furthermore, all-optically driven neuromorphic computation is demonstrated by machine learning to classify numbers and recognize images with an accuracy of over 90%. The devices provide a promising approach toward neurorobotics, human-machine interaction technologies, and scalable bionic systems with visual data storage/buffering and processing.

Curved retina-like camera array imaging system with adjustable super-resolution fovea

To achieve non-uniform imaging with a large field of view and high efficiency as well as to obtain an adjustable fovea with super-resolution, we proposed a curved retina-like camera array imaging system (CRCS), which is built by an eight-camera array distributed non-uniformly on a curved surface and a camera coaxial with Risley prisms located in the center. By the non-uniform imaging, the field of view of the developed prototype is 150^∘×40^∘ with a reduction of data redundancy by 87.62%. Besides, the experimental results show that CRCS can obtain clear and sharp images of farther targets of interest around the fovea field of view with a constant focal length.

Fabrication of self-aligning convergent waveguides of microlens arrays to collect and guide light

The optical properties of microlens arrays may be significantly affected by the optical crosstalk effect between adjacent lenses. Recently, this issue has triggered increasing attention in the scientific community. In this study, an integrated microlens array (MLA) consisting of self-aligning convergent waveguides of microlenses was fabricated. The optical crosstalk effect does not influence the performance of such system. Based on the self-focusing effect principle, self-writing of the waveguide array was achieved in a photosensitive polymer. The light collection and guiding performance of the MLA with and without thermal cross-linking treatment was analyzed in depth. The relation between the stray light and the filling rate of the MLA shows that a high filling rate decreases the optical crosstalk. Finally, an integrated MLA with a large area, high uniformity, and excellent optical performance was fabricated.

Large-scale microlens arrays on flexible substrate with improved numerical aperture for curved integral imaging 3D display

Curved integral imaging 3D display could provide enhanced 3D sense of immersion and wider viewing angle, and is gaining increasing interest among discerning users. In this work, large scale microlens arrays (MLAs) on flexible PMMA substrate were achieved based on screen printing method. Meanwhile, an inverted reflowing configuration as well as optimization of UV resin's viscosity and substrate's surface wettability were implemented to improved the numerical aperture (NA) of microlenses. The results showed that the NA values of MLAs could be increased effectively by adopting inverted reflowing manner with appropriate reflowing time. With decreasing the substrate's wettability, the NA values could be increased from 0.036 to 0.096, when the UV resin contact angles increased from 60.1° to 88.7°. For demonstration, the fabricated MLAs was combined to a curved 2D monitor to realize a 31-inch curved integral imaging 3D display system, exhibiting wider viewing angle than flat integral imaging 3D display system.

Foveal scanning based on an optical-phases array

We utilize the space-variant structure of the human fovea as a basis for a novel, to the best of our knowledge, foveal approach based on optical-phases array (OPA). This approach can be used to solve issues in 3D imaging and achieve a large field of view and high resolution with real-time application. A foveal scanning model based on the OPA is established. Simulations and experiments are performed to verify the models and illustrate the advantages of foveal scanning compared with traditional raster scanning. Simulations agree well with the theory, and the foveal approach has higher efficiency than traditional raster scanning. These results can serve as a reference for developing biomimetic sensors that mimic the human eye.

Target orientation detection based on a neural network with a bionic bee-like compound eye

The compound eye of insects has many excellent characteristics. Directional navigation is one of the important features of compound eye, which is able to quickly and accurately determine the orientation of an objects. Therefore, bionic curved compound eye have great potential in detecting the orientation of the target. However, there is a serious non-linear relationship between the orientation of the target and the image obtained by the curved compound eye in wide field of view (FOV), and an effective model has not been established to detect the orientation of target. In this paper, a method for detecting the orientation of the target is proposed, which combines a virtual cylinder target with a neural network. To verify the feasibility of the method, a fiber-optic compound eye that is inspired by the structure of the bee's compound eye and that fully utilizes the transmission characteristics and flexibility of optical fibers is developed. A verification experiment shows that the proposed method is able to realize quantitative detection of orientations using a prototype of the fiber-optic compound eye. The average errors between the ground truth and the predicted values of the horizontal and elevation angles of a target are 0.5951 ^° and 0.6748^°, respectively. This approach has great potential for target tracking, obstacle avoidance by unmanned aerial vehicles, and directional navigation control.

Multispectral curved compound eye camera

In this work, we propose a new type of multispectral imaging system, named multispectral curved compound eye camera (MCCEC). The so called MCCEC consists of three subsystems, a curved micro-lens array integrated with selected narrow-band optical filters, an optical transformation subsystem, and the data processing unit with an image sensor. The novel MCCEC system can achieve multi-spectral imaging at an ultra-large field of view (FOV), and obtain information of multiple spectrum segments at real time. Moreover, the system has the advantages of small size, light weight, and high sensitivity in comparison with conventional multispectral cameras. In current work, we mainly focus on the optical design of the MCCEC based on the overlap of FOV between the neighboring clusters of ommatidia to achieve the multispectral imaging at an ultra-large FOV. The optical layout of the curved micro-lens array, narrow-band filter array and the optical relay system for image plane transformation are carefully designed and optimized. The whole size of the optical system is 93 mm × 42 mm × 42 mm. The simulation results show that a maximum FOV of about 120° can be achieved for seven-waveband multispectral imaging with center wavelengths of 480 nm, 550 nm, 591 nm, 676 nm, 704 nm, 740 nm, and 767 nm. The new designed MCCEC has a great potential as an airborne or satellite-born payload for real time remote sensing and thus paves a new way for the design of compact and light-weight spectral-imaging cameras with an ultra large FOV.

Toward a locally adaptive optical protection filtering for human eyes and technical vision sensors

In the presence of direct sunlight or superbright light from artificial optical sources, the distribution of light intensity (brightness) over perceived scene objects typically has a dynamic range several orders of magnitude greater than the dynamic range of most optical sensors. In this paper, the locally adaptive optical protection (LAOP) filtering systems for technical vision sensors and human eyes (human visual system) are suggested. The LAOP filtering provides the reliable perception of the perceived scene objects with normal brightness simultaneously with preventing saturation ("blinding") of the optical sensors by light from the brightest objects. The characteristics of the key components of the LAOP filtering systems are discussed and tested experimentally.

Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review

Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging, and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modeling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research on insects' visual systems in the literature. These motion perception models or neural networks consist of the looming-sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation-sensitive neural systems of direction-selective neurons (DSNs) in fruit flies, bees, and locusts, and the small-target motion detectors (STMDs) in dragonflies and hoverflies. We also review the applications of these models to robots and vehicles. Through these modeling studies, we summarize the methodologies that generate different direction and size selectivity in motion perception. Finally, we discuss multiple systems integration and hardware realization of these bio-inspired motion perception models.