Using a Slit to Suppress Optical Aberrations in Laser Triangulation Sensors

In this paper, we present a laser triangulation sensor to measure the distance between the sensor and an object without contact using a diffraction slit rather than a traditional lens. We show that by replacing the lens with a slit, we can exploit the resulting diffraction pattern to have finer and yet simpler image analysis, yielding better estimation of the distance to the object. To test our hypothesis, we build a precision position table and a laser triangulation sensor, generate large data sets to test different estimation algorithms on various materials, and compare data acquisition using a traditional lens versus using a slit. We show that position estimation when using a slit is both more precise and more accurate than comparable methods using a lens.

Ultrasensitive Near-Infrared Polarization Photodetectors with Violet Phosphorus/InSe van der Waals Heterostructures

Near-infrared (NIR) polarization photodetectors with two-dimensional (2D) semiconductors and their van der Waals (vdW) heterostructures have presented great impact for the development of a wide range of technologies, such as in the optoelectronics and communication fields. Nevertheless, the lack of a photogenerated charge carrier at the device's interface leads to a poor charge carrier collection efficiency and a low linear dichroism ratio, hindering the achievement of high-performance optoelectronic devices with multifunctionalities. Herein, we present a type-II violet phosphorus (VP)/InSe vdW heterostructure that is predicted via density functional theory calculation and confirmed by Kelvin probe force microscopy. Benefiting from the type-II band alignment, the VP/InSe vdW heterostructure-based photodetector achieves excellent photodetection performance such as a responsivity (R) of 182.8 A/W, a detectivity (D*) of 7.86 × 1012 Jones, and an external quantum efficiency (EQE) of 11,939% under a 1064 nm photon excitation. Furthermore, the photodetection performance can be enhanced by manipulating the device geometry by inserting a few layers of graphene between the VP and InSe (VP/Gr/InSe). Remarkably, the VP/Gr/InSe vdW heterostructure shows a competitive polarization sensitivity of 2.59 at 1064 nm and can be integrated as an image sensor. This work demonstrates that VP/InSe and VP/Gr/InSe vdW heterostructures will be effective for promising integrated NIR optoelectronics.

Enhancement of Optical and Chemical Resistance Properties with a Novel Yellow Quinophthalone Derivative for Image Sensor Colorants

A novel quinophthalone derivative, 4,5,6,7-tetrachloro-2-(2-(3-hydroxy-1-oxo-1H-cyclopenta[b]naphthalen-2-yl)quinolin-4-yl)isoindoline-1,3-dione (TCHCQ), was designed and synthesized as a yellow colorant additive for green color filters in image sensors. The characteristics of the new material were evaluated in terms of optical, thermal, and chemical properties under solution and color filter film conditions. TCHCQ exhibited a significantly enhanced molar extinction coefficient in solution, being 1.21 times higher than that of the commercially used yellow colorant Y138. It also demonstrated excellent thermal stability, with a decomposition temperature (Td) exceeding 450 °C. Utilizing the nano-pigmentation process, TCHCQ was used to prepare nano-sized particles with an excellent average size of 35 nm. This enabled the fabrication of a color filter film with outstanding properties. The optical properties of the produced film revealed outstanding yellow colorant transmittance of 0.97% at 435 nm and 91.2% at 530 nm. The color filter film exhibited similar optical and thermal stability to Y138, with an improved chemical stability, as evidenced by a ΔEab value of 0.52. The newly synthesized TCHCQ is considered a promising candidate for use as a yellow colorant additive in image sensor color filters, demonstrating superior optical, thermal, and chemical stability.

Exposure Time Control Method for Higher Intermediate Frequency in Optical Heterodyne Imaging and Its Application to Electric-Field Imaging Based on Electro-Optic Effect

We propose and demonstrate a method for equivalent time sampling using image sensors to selectively detect only the target frequency. Shortening the exposure time of the image sensor and using equivalent time sampling allows for the detection of frequency components that are higher than the frame rate. However, the imaging system in our previous work was also sensitive to the frequency component at 1/4 of the frame rate. In this study, we control the phase relationship between the exposure time and observed signal by inserting an additional interval once every four frames to detect the target frequency selectively. With this technique, we conducted electric field imaging based on the electro-optic effect under high noise conditions in the low-frequency band to which the conventional method is sensitive. The results demonstrated that the proposed method improved the signal-to-noise ratio.

Efficacy of an Electric Toothbrush With Monitor in Dental Plaque Removal: A Crossover Randomized Controlled Trial

BACKGROUND AND PURPOSE

Plaque control is very important in the treatment of periodontitis. However, plaque is difficult to remove because one cannot see one's own oral cavity. The purpose of this study was to verify the plaque removal effect of a prototype device that has a built-in image sensor in the head of an electric toothbrush, enabling the user to brush while checking the condition of the tooth surface on a monitor in real time and to assess their sense of use.

MATERIALS AND METHODS

The subjects were 10 fifth-year students from the Graduate School of Dental Science, Fukuoka Dental College, Fukuoka, Japan. The subjects were divided into those who used electric toothbrushes while having the condition of the tooth surface checked with a monitor (monitor group) and those without a monitor (non-monitor group). O'Leary plaque control records before and after brushing and the brushing time were measured, and questionnaires were given to the subjects after brushing. Scaling and professional tooth cleaning were performed after completing the questionnaire. One week later, subjects were switched to the opposite group and had the same measurements and questionnaires. The Wilcoxon signed-rank test was used to compare both groups before and after the examination at a 5% significance level.

RESULTS

The monitor group had significantly better plaque removal than the non-monitor group. In addition, the monitor group spent significantly more time brushing than the control group.

CONCLUSION

Brushing while monitoring oral conditions in real time using an electric toothbrush with a built-in image sensor showed that significantly better plaque removal can be achieved with a longer brushing time.

Highly Crystalized MAPbX3 Perovskite Triangular Nanowire Arrays for Optoelectronic Applications

Here, a facile fabrication approach for the high-quality 1D perovskite triangular nanowire (TNW) array synthesis through space-confined effect is reported. A soft stamp containing 1D triangular linear array pattern is used to confine the MAPbX3 solution and to guide the growth of the nanowires along the prescribed direction with good crystallinity. The further constructed photodetectors based on the obtained MAPbI3 TNWs exhibit superior photoresponse properties with a responsivity of (125.2 ± 2.5) A W-1 and detectivity of (2.8 ± 0.8) × 1013 Jones at the wavelength of 650 nm. This excellent performance is attributed to the highly crystalline TNW with optical anisotropy and a small asymptotic height, which reduces the probability of the photon reflection and promotes the carrier transport. More interestingly, the increased surface area of the triangular device can present superior flexibility after a couple of thousands of bending cycles. Furthermore, by fabricating 7 × 7 photodetector arrays, the potential image sensor application is demonstrated. The perovskite nanowire fabrication approach is scalable and compatible with current semiconductor manufacturing, which indicates their great potential in broad applications.

Surface Passivation by Quantum Exclusion: On the Quantum Efficiency and Stability of Delta-Doped CCDs and CMOS Image Sensors in Space

Radiation-induced damage and instabilities in back-illuminated silicon detectors have proved to be challenging in multiple NASA and commercial applications. In this paper, we develop a model of detector quantum efficiency (QE) as a function of Si-SiO2 interface and oxide trap densities to analyze the performance of silicon detectors and explore the requirements for stable, radiation-hardened surface passivation. By analyzing QE data acquired before, during, and after, exposure to damaging UV radiation, we explore the physical and chemical mechanisms underlying UV-induced surface damage, variable surface charge, QE, and stability in ion-implanted and delta-doped detectors. Delta-doped CCD and CMOS image sensors are shown to be uniquely hardened against surface damage caused by ionizing radiation, enabling the stability and photometric accuracy required by NASA for exoplanet science and time domain astronomy.

Towards A Wireless Image Sensor for Real-Time Fluorescence Microscopy in Cancer Therapy

We present a mm-sized, ultrasonically powered lensless CMOS image sensor as a progress towards wireless fluorescence microscopy. Access to biological information within the tissue has the potential to provide insights guiding diagnosis and treatment across numerous medical conditions including cancer therapy. This information, in conjunction with current clinical imaging techniques that have limitations in obtaining images continuously and lack wireless compatibility, can improve continual detection of multicell clusters deep within tissue. The proposed platform incorporates a 2.4×4.7 mm2 integrated circuit (IC) fabricated in TSMC 0.18 μm, a micro laser diode (μLD), a single piezoceramic and off-chip storage capacitors. The IC consists of a 36×40 array of capacitive trans-impedance amplifier-based pixels, wireless power management and communication via ultrasound and a laser driver all controlled by a Finite State Machine. The piezoceramic harvests energy from the acoustic waves at a depth of 2 cm to power up the IC and transfer 11.5 kbits/frame via backscattering. During Charge-Up, the off-chip capacitor stores charge to later supply a high-power 78 mW μLD during Imaging. Proof of concept of the imaging front end is shown by imaging distributions of CD8 T-cells, an indicator of the immune response to cancer, ex vivo, in the lymph nodes of a functional immune system (BL6 mice) against colorectal cancer consistent with the results of a fluorescence microscope. The overall system performance is verified by detecting 140 μm features on a USAF resolution target with 32 ms exposure time and 389 ms ultrasound backscattering.

Monolithic Integration of Semi-Transparent and Flexible Integrated Image Sensor Array with a-IGZO Thin-Film Transistors (TFTs) and p-i-n Hydrogenated Amorphous Silicon Photodiodes

A novel approach to fabricating a transparent and flexible one-transistor-one-diode (1T-1D) image sensor array on a flexible colorless polyimide (CPI) film substrate is successfully demonstrated with laser lift-off (LLO) techniques. Leveraging transparent indium tin oxide (ITO) electrodes and amorphous indium gallium zinc oxide (a-IGZO) channel-based thin-film transistor (TFT) backplanes, vertically stacked p-i-n hydrogenated amorphous silicon (a-Si:H) photodiodes (PDs) utilizing a low-temperature (<90 °C) deposition process are integrated with a densely packed 14 × 14 pixel array. The low-temperature-processed a-Si:H photodiodes show reasonable performance with responsivity and detectivity for 31.43 mA/W and 3.0 × 1010 Jones (biased at -1 V) at a wavelength of 470 nm, respectively. The good mechanical durability and robustness of the flexible image sensor arrays enable them to be attached to a curved surface with bending radii of 20, 15, 10, and 5 mm and 1000 bending cycles, respectively. These studies show the significant promise of utilizing highly flexible and rollable active-matrix technology for the purpose of dynamically sensing optical signals in spatial applications.

IRIS: Integrated Retinal Functionality in Image Sensors

Neuromorphic image sensors draw inspiration from the biological retina to implement visual computations in electronic hardware. Gain control in phototransduction and temporal differentiation at the first retinal synapse inspired the first generation of neuromorphic sensors, but processing in downstream retinal circuits, much of which has been discovered in the past decade, has not been implemented in image sensor technology. We present a technology-circuit co-design solution that implements two motion computations-object motion sensitivity and looming detection-at the retina's output that could have wide applications for vision-based decision-making in dynamic environments. Our simulations on Globalfoundries 22 nm technology node show that the proposed retina-inspired circuits can be fabricated on image sensing platforms in existing semiconductor foundries by taking advantage of the recent advances in semiconductor chip stacking technology. Integrated Retinal Functionality in Image Sensors (IRIS) technology could drive advances in machine vision applications that demand energy-efficient and low-bandwidth real-time decision-making.

A Comprehensive Methodology for Optimizing Read-Out Timing and Reference DAC Offset in High Frame Rate Image Sensing Systems

This paper presents a comprehensive timing optimization methodology for power-efficient high-resolution image sensors with column-parallel single-slope analog-to-digital converters (ADCs). The aim of the method is to optimize the read-out timing for each period in the image sensor's operation, while considering various factors such as ADC decision time, slew rate, and settling time. By adjusting the ramp reference offset and optimizing the amplifier bandwidth of the comparator, the proposed methodology minimizes the power consumption of the amplifier array, which is one of the most power-hungry circuits in the system, while maintaining a small color linearity error and ensuring optimal performance. To demonstrate the effectiveness of the proposed method, a power-efficient 108 MP 3-D stacked CMOS image sensor with a 10-bit column-parallel single-slope ADC array was implemented and verified. The image sensor achieved a random noise of 1.4 e-rms, a column fixed-pattern noise of 66 ppm at an analog gain of 16, and a remarkable figure-of-merit (FoM) of 0.71 e-·nJ. The sensor utilized a one-row read-out time of 6.9 µs, an amplifier bandwidth of 1.1 MHz, and a reference digital-to-analog converter (DAC) offset of 512 LSB. This timing optimization methodology enhances energy efficiency in high-resolution image sensors, enabling higher frame rates and improved system performance. It could be adapted for various imaging applications requiring optimized performance and reduced power consumption, making it a valuable tool for designers aiming to achieve optimal performance in power-sensitive applications.

Integrated High-Temporal-Resolution and High-Density Subretinal Prosthesis Using a Correlated Double-Sampling Technique

This paper presents a 1600-pixel integrated neural stimulator with a correlated double-sampling readout (DSR) circuit for a subretinal prosthesis. The retinal stimulation chip inserted beneath the photoreceptor layer comprises an array of an active pixel sensor (APS) and biphasic pulse shaper. The DSR circuit achieves a high signal-to-noise ratio (SNR) of the APS with a short integration time to simultaneously improve the temporal and spatial resolutions of restored vision. This DSR circuit is adopted along with a 5 × 5-pixel tile, which reduces pixel size and improves the SNR by increasing the area occupied by storage capacitors. Moreover, a low-mismatch reference generator enables a low standard deviation between individual pulse shapers. The 1600-pixel retinal chip, fabricated using the 0.18 μm 1P6M CMOS process, occupies a total area of 4.3 mm × 3.3 mm and dissipates an average power of 3.4 mW; this was demonstrated by determining the stimulus current patterns corresponding to the illuminations of an LCD projector. Experimental results show that the proposed high-density stimulation array chip can achieve a high temporal resolution owing to its short integration time.

Design and Characterization of a Burst Mode 20 Mfps Low Noise CMOS Image Sensor

This paper presents a novel ultra-high speed, high conversion-gain, low noise CMOS image sensor (CIS) based on charge-sweep transfer gates implemented in a standard 180 nm CIS process. Through the optimization of the photodiode geometry and the utilization of charge-sweep transfer gates, the proposed pixels achieve a charge transfer time of less than 10 ns without requiring any process modifications. Moreover, the gate structure significantly reduces the floating diffusion capacitance, resulting in an increased conversion gain of 183 µV/e-. This advancement enables the image sensor to achieve the lowest reported noise of 5.1 e- rms. To demonstrate the effectiveness of both optimizations, a proof-of-concept CMOS image sensor is designed, taped-out and characterized.

Interface-Engineered Organic Near-Infrared Photodetector for Imaging Applications

We report a high-speed low dark current near-infrared (NIR) organic photodetector (OPD) on a silicon substrate with amorphous indium gallium zinc oxide (a-IGZO) as the electron transport layer (ETL). In-depth understanding of the origin of dark current is obtained using an elaborate set of characterization techniques, including temperature-dependent current-voltage measurements, current-based deep-level transient spectroscopy (Q-DLTS), and transient photovoltage decay measurements. These characterization results are complemented by energy band structures deduced from ultraviolet photoelectron spectroscopy. The presence of trap states and a strong dependency of activation energy on the applied reverse bias voltage point to a dark current mechanism based on trap-assisted field-enhanced thermal emission (Poole-Frenkel emission). We significantly reduce this emission by introducing a thin interfacial layer between the donor: acceptor blend and the a-IGZO ETL and obtain a dark current as low as 125 pA/cm² at an applied reverse bias of -1 V. Thanks to the use of high-mobility metal-oxide transport layers, a fast photo response time of 639 ns (rise) and 1497 ns (fall) is achieved, which, to the best of our knowledge, is among the fastest reported for NIR OPDs. Finally, we present an imager integrating the NIR OPD on a complementary metal oxide semiconductor read-out circuit, demonstrating the significance of the improved dark current characteristics in capturing high-quality sample images with this technology.

A Time-of-Flight Image Sensor Using 8-Tap P-N Junction Demodulator Pixels

This paper presents a time-of-flight image sensor based on 8-Tap P-N junction demodulator (PND) pixels, which is designed for hybrid-type short-pulse (SP)-based ToF measurements under strong ambient light. The 8-tap demodulator implemented with multiple p-n junctions used for modulating the electric potential to transfer photoelectrons to eight charge-sensing nodes and charge drains has an advantage of high-speed demodulation in large photosensitive areas. The ToF image sensor implemented using 0.11 µm CIS technology, consisting of an 120 (H) × 60 (V) image array of the 8-tap PND pixels, successfully works with eight consecutive time-gating windows with the gating width of 10 ns and demonstrates for the first time that long-range (>10 m) ToF measurements under high ambient light are realized using single-frame signals only, which is essential for motion-artifact-free ToF measurements. This paper also presents an improved depth-adaptive time-gating-number assignment (DATA) technique for extending the depth range while having ambient-light canceling capability and a nonlinearity error correction technique. By applying these techniques to the implemented image sensor chip, hybrid-type single-frame ToF measurements with depth precision of maximally 16.4 cm (1.4% of the maximum range) and the maximum non-linearity error of 0.6% for the full-scale depth range of 1.0-11.5 m and operations under direct-sunlight-level ambient light (80 klux) have been realized. The depth linearity achieved in this work is 2.5 times better than that of the state-of-the-art 4-tap hybrid-type ToF image sensor.

Thin and Scalable Hybrid Emission Filter via Plasma Etching for Low-Invasive Fluorescence Detection

Hybrid emission filters, comprising an interference filter and an absorption filter, exhibit high excitation light rejection performance and can act as lensless fluorescent devices. However, it has been challenging to produce them in large batches over a large area. In this study, we propose and demonstrate a method for transferring a Si substrate, on which the hybrid filter is deposited, onto an image sensor by attaching it to the sensor and removing the substrate via plasma etching. Through this method, we can transfer uniform filters onto fine micrometer-sized needle devices and millimeter-sized multisensor chips. Optical evaluation reveals that the hybrid filter emits light in the 500 to 560 nm range, close to the emission region of green fluorescent protein (GFP). Furthermore, by observing the fluorescence emission from the microbeads, a spatial resolution of 12.11 μm is calculated. In vitro experiments confirm that the fabricated device is able to discriminate GFP emission patterns from brain slices.

VLSI Design Based on Block Truncation Coding for Real-Time Color Image Compression for IoT

It has always been a major issue for a hospital to acquire real-time information about a patient in emergency situations. Because of this, this research presents a novel high-compression-ratio and real-time-process image compression very-large-scale integration (VLSI) design for image sensors in the Internet of Things (IoT). The design consists of a YEF transform, color sampling, block truncation coding (BTC), threshold optimization, sub-sampling, prediction, quantization, and Golomb-Rice coding. By using machine learning, different BTC parameters are trained to achieve the optimal solution given the parameters. Two optimal reconstruction values and bitmaps for each 4 × 4 block are achieved. An image is divided into 4 × 4 blocks by BTC for numerical conversion and removing inter-pixel redundancy. The sub-sampling, prediction, and quantization steps are performed to reduce redundant information. Finally, the value with a high probability will be coded using Golomb-Rice coding. The proposed algorithm has a higher compression ratio than traditional BTC-based image compression algorithms. Moreover, this research also proposes a real-time image compression chip design based on low-complexity and pipelined architecture by using TSMC 0.18 μm CMOS technology. The operating frequency of the chip can achieve 100 MHz. The core area and the number of logic gates are 598,880 μm² and 56.3 K, respectively. In addition, this design achieves 50 frames per second, which is suitable for real-time CMOS image sensor compression.

A CMOS Image Readout Circuit with On-Chip Defective Pixel Detection and Correction

Images produced by CMOS sensors may contain defective pixels due to noise, manufacturing errors, or device malfunction, which must be detected and corrected at early processing stages in order to produce images that are useful to human users and image-processing or machine-vision algorithms. This paper proposes a defective pixel detection and correction algorithm and its implementation using CMOS analog circuits, which are integrated with the image sensor at the pixel and column levels. During photocurrent integration, the circuit detects defective values in parallel at each pixel using simple arithmetic operations within a neighborhood. At the image-column level, the circuit replaces the defective pixels with the median value of their neighborhood. To validate our approach, we designed a 128×128-pixel imager in a 0.35μm CMOS process, which integrates our defective-pixel detection/correction circuits and processes images at 694 frames per second, according to post-layout simulations. Operating at that frame rate, our proposed algorithm and its CMOS implementation produce better results than current state-of-the-art algorithms: it achieves a Peak Signal to Noise Ratio (PSNR) and Image Enhancement Factor (IEF) of 45 dB and 198.4, respectively, in images with 0.5% random defective pixels, and a PSNR of 44.4 dB and IEF of 194.2, respectively, in images with 1.0% random defective pixels.

Uniformity Correction of CMOS Image Sensor Modules for Machine Vision Cameras

Flat-field correction (FFC) is commonly used in image signal processing (ISP) to improve the uniformity of image sensor pixels. Image sensor nonuniformity and lens system characteristics have been known to be temperature-dependent. Some machine vision applications, such as visual odometry and single-pixel airborne object tracking, are extremely sensitive to pixel-to-pixel sensitivity variations. Numerous cameras, especially in the fields of infrared imaging and staring cameras, use multiple calibration images to correct for nonuniformities. This paper characterizes the temperature and analog gain dependence of the dark signal nonuniformity (DSNU) and photoresponse nonuniformity (PRNU) of two contemporary global shutter CMOS image sensors for machine vision applications. An optimized hardware architecture is proposed to compensate for nonuniformities, with optional parametric lens shading correction (LSC). Three different performance configurations are outlined for different application areas, costs, and power requirements. For most commercial applications, the correction of LSC suffices. For both DSNU and PRNU, compensation with one or multiple calibration images, captured at different gain and temperature settings are considered. For more demanding applications, the effectiveness, external memory bandwidth, power consumption, implementation, and calibration complexity, as well as the camera manufacturability of different nonuniformity correction approaches were compared.

Smart Helmet-Based Proximity Warning System to Improve Occupational Safety on the Road Using Image Sensor and Artificial Intelligence

Recently, collisions between equipment and workers occur frequently on the road in construction and surface mining sites. To prevent such accidents, we developed a smart helmet-based proximity warning system (PWS) that facilitates visual and tactile proximity warnings. In this system, a smart helmet comprising an Arduino Uno board and a camera module with built-in Wi-Fi was used to transmit images captured by the camera to a smartphone via Wi-Fi. When the image was analyzed through object detection and a heavy-duty truck or other vehicle was detected in an image, the smartphone transmitted a signal to the Arduino via Bluetooth and, when a signal was received, an LED strip with a three-color LED visually alerted the worker and the equipment operator. The performance of the system tested the recognition distance of the helmet according to the pixel size of the detected image in an outdoor environment. The proposed personal PWS can directly produce visual proximity warnings to both workers and operators enabling them to quickly identify and evacuate from dangerous situations.

Novel Yellow Azo Pyridone Derivatives with Different Halide Atoms for Image-Sensor Color Filters

Novel yellow azo pyridone dye derivatives were synthesized for use in image-sensor color filters. The synthesized compounds have a basic chemical structure composed of azo, hydroxy, amide, and nitrile groups as well as different halide groups. New materials were evaluated on the basis of their optical, thermal, and surface properties under conditions mimicking those of a commercial device fabrication process. A comparison of their related performance revealed that, among the four prepared compounds, 5-((4,6-dichlorocyclohexa-2,4-dien-1-yl)diazenyl)-6-hydroxy-1,4-dimethyl-2-oxo-1,2-dihydropyridine-3-carbonitrile (Cl-PAMOPC) exhibited the best performance as an image-sensor color filter material, including a solubility greater than 0.1 wt% in propylene glycol monomethyl ether acetate solvent, a high decomposition temperature of 263 °C, and stable color difference values of 4.93 and 3.88 after a thermal treatment and a solvent-resistance test, respectively. The results suggest that Cl-PAMOPC can be used as a green dye additive in an image-sensor colorant.

Mixed-Dimensional Anti-ambipolar Phototransistors Based on 1D GaAsSb/2D MoS2 Heterojunctions

The incapability of modulating the photoresponse of assembled heterostructure devices has remained a challenge for the development of optoelectronics with multifunctionality. Here, a gate-tunable and anti-ambipolar phototransistor is reported based on 1D GaAsSb nanowire/2D MoS₂ nanoflake mixed-dimensional van der Waals heterojunctions. The resulting heterojunction shows apparently asymmetric control over the anti-ambipolar transfer characteristics, possessing potential to implement electronic functions in logic circuits. Meanwhile, such an anti-ambipolar device allows the synchronous adjustment of band slope and depletion regions by gating in both components, thereby giving rise to the gate-tunability of the photoresponse. Coupled with the synergistic effect of the materials in different dimensionality, the hybrid heterojunction can be readily modulated by the external gate to achieve a high-performance photodetector exhibiting a large on/off current ratio of 4 × 10⁴, fast response of 50 μs, and high detectivity of 1.64 × 10¹¹ Jones. Due to the formation of type-II band alignment and strong interfacial coupling, a prominent photovoltaic response is explored in the heterojunction as well. Finally, a visible image sensor based on this hybrid device is demonstrated with good imaging capability, suggesting the promising application prospect in future optoelectronic systems.

Two High-Precision Proximity Capacitance CMOS Image Sensors with Large Format and High Resolution

This paper presents newly developed two high-precision CMOS proximity capacitance image sensors: Chip A with 12 μm pitch pixels with a large detection area of 1.68 cm²; Chip B with 2.8 μm pitch 1.8 M pixels for a higher resolution. Both fabricated chips achieved a capacitance detection precision of less than 100 zF (10⁻¹⁹ F) at an input voltage of 20 V and less than 10 zF (10⁻²⁰ F) at 300 V due to the noise cancelling technique. Furthermore, by using multiple input pulse amplitudes, a capacitance detection dynamic range of up to 123 dB was achieved. The spatial resolution improvement was confirmed by the experimentally obtained modulation transfer function for Chip B with various line and space pattens. The examples of capacitance imaging using the fabricated chips were also demonstrated.

Optics-Free Chip-Scale Intraoperative Imaging Using NIR-Excited Upconverting Nanoparticles

We present an optics-free CMOS image sensor that incorporates a novel time-gated dual-photodiode pixel design to allow filter- and lens-less image acquisition of near-infrared-excited (NIR-excited) upconverting nanoparticles. Recent biomedical advances have highlighted the benefits of NIR excitation, but NIR interaction with silicon has remained a challenge, even with high-performance optical blocking filters. Using a secondary diode and a dual-photodiode design, this sensor is able to remove the 100s of mV of NIR background on pixels and bring it down to single-digit mV level, nearing its noise floor of 2.2 mV rms, not achievable with any optical filter. Non-linear effects of background cancellation using the diode pair has been mitigated using an initial one-time pixel-level curve fitting and calibration in a post-processing setting. This imager comprises a highly linear 11 fF metal-oxide-metal (MOM) capacitor and includes integrated angle-selective gratings to reject oblique light and enhance sharpness. Each pixel also includes two distinct correlated double sampling schemes, to remove low frequency flicker noise and systematic offset in the datapath. We demonstrate the performance of this imager using pulsed NIR-excited upconverting nanoparticles on standard United-States-Air-Force (USAF) resolution targets and achieve an SNR of 15 dB, while keeping NIR background below 6 mV. This 36-by-80-pixel array measures only 2.3 mm by 4.8 mm and can be thinned down to 25 µm, allowing it to become surgically compatible with intraoperative instruments and equipment, while remaining optics-free.

Simulating 50 keV X-ray Photon Detection in Silicon with a Down-Conversion Layer

Simulation results are presented that explore an innovative, new design for X-ray detection in the 20-50 keV range that is an alternative to traditional direct and indirect detection methods. Typical indirect detection using a scintillator must trade-off between absorption efficiency and spatial resolution. With a high-Z layer that down-converts incident photons on top of a silicon detector, this design has increased absorption efficiency without sacrificing spatial resolution. Simulation results elucidate the relationship between the thickness of each layer and the number of photoelectrons generated. Further, the physics behind the production of electron-hole pairs in the silicon layer is studied via a second model to shed more light on the detector's functionality. Together, the two models provide a greater understanding of this detector and reveal the potential of this novel form of X-ray detection.

Optical Efficiency Enhancement of Nanojet-Based Dielectric Double-Material Color Splitters for Image Sensor Applications

We propose a new type of color splitter, which guides a selected bandwidth of incident light towards the proper photosensitive area of the image sensor by exploiting the nanojet (NJ) beam phenomenon. Such splitting can be performed as an alternative to filtering out part of the received light on each color subpixel. We propose to split the incoming light thanks to a new type of NJ-based near-field focusing double-material element with an insert. To suppress crosstalk, we use a Deep-Trench Isolation (DTI) structure. We demonstrate that the use of a dielectric insert block allows for reduction in the size of the color splitting element. By changing the position of the DTI, the functionality of separating blue, green and red light can be improved.

Image Acquisition Device for Smart-City Access Control Applications Based on Iris Recognition

In this work, we present an eye-image acquisition device that can be used as an image acquisition front-end application in compact, low-cost, and easy-to-integrate products for smart-city access control applications, based on iris recognition. We present the advantages and disadvantages of iris recognition compared to fingerprint- or face recognition. We also present the main drawbacks of the existing commercial solutions and propose a concept device design for door-mounted access control systems based on iris recognition technology. Our eye-image acquisition device was built around a low-cost camera module. An integrated infrared distance measurement was used for active image focusing. FPGA image processing was used for raw-RGB to grayscale demosaicing and passive image focusing. The integrated visible light illumination meets the IEC62471 photobiological safety standard. According to our results, we present the operation of the distance-measurement subsystem, the operation of the image-focusing subsystem, examples of acquired images of an artificial toy eye under different illumination conditions, and the calculation of illumination exposure hazards. We managed to acquire a sharp image of an artificial toy eye sized 22 mm in diameter from an approximate distance of 10 cm, with 400 pixels over the iris diameter, an average acquisition time of 1 s, and illumination below hazardous exposure levels.

Electrically Switchable Color-Selective Organic Photodetectors for Full-Color Imaging

The present full-color imaging techniques rely on the use of broadband inorganic photodetectors with dedicated color filters, which is one of the practical challenges for large-area, flexible, and high-solution imaging applications. The development of high-performance color-selective photodetectors is one of the key solutions to overcome this challenge. In this work, an electrically switchable color-selective organic photodetector (OPD) comprising a double organic bulk heterojunction structure has been developed for full-color imaging. The color-selective sensing capability over the visible spectrum ranges can be realized by controlling the bias across the OPD, achieving a high responsivity of ∼200 mA/W, a large linear dynamic range of 122 dB, a viewing angle of 120°, and a -3 dB cutoff frequency of >50 kHz. A full-color imaging function has been demonstrated using electrically switchable red-, green-, and blue-color selective OPD sensors with an excellent operational stability. The results of this work provide a practical solution for applications in high-resolution full-color imaging and artificial vision.

Ambient Light Rejection Integrated Circuit for Autonomous Adaptation on a Sub-Retinal Prosthetic System

This paper introduces an ambient light rejection (ALR) circuit for the autonomous adaptation of a subretinal implant system. The sub-retinal implants, located beneath a bipolar cell layer, are known to have a significant advantage in spatial resolution by integrating more than a thousand pixels, compared to epi-retinal implants. However, challenges remain regarding current dispersion in high-density retinal implants, and ambient light induces pixel saturation. Thus, the technical issues of ambient light associated with a conventional image processing technique, which lead to high power consumption and area occupation, are still unresolved. Thus, it is necessary to develop a novel image-processing unit to handle ambient light, considering constraints related to power and area. In this paper, we present an ALR circuit as an image-processing unit for sub-retinal implants. We first introduced an ALR algorithm to reduce the ambient light in conventional retinal implants; next, we implemented the ALR algorithm as an application-specific integrated chip (ASIC). The ALR circuit was fabricated using a standard 0.35-μm CMOS process along with an image-sensor-based stimulator, a sensor pixel, and digital blocks. As experimental results, the ALR circuit occupies an area of 190 µm², consumes a power of 3.2 mW and shows a maximum response time of 1.6 s at a light intensity of 20,000 lux. The proposed ALR circuit also has a pixel loss rate of 0.3%. The experimental results show that the ALR circuit leads to a sensor pixel (SP) being autonomously adjusted, depending on the light intensity.

Design of an SVM Classifier Assisted Intelligent Receiver for Reliable Optical Camera Communication

Embedding optical camera communication (OCC) commercially as a favorable complement of radio-frequency technology has led to the desire for an intelligent receiver system that is eligible to communicate with an accurate light-emitting diode (LED) transmitter. To shed light on this issue, a novel scheme for detecting and recognizing data transmitting LEDs has been elucidated in this paper. Since the optically modulated signal is captured wirelessly by a camera that plays the role of the receiver for the OCC technology, the process to detect LED region and retrieval of exact information from the image sensor is required to be intelligent enough to achieve a low bit error rate (BER) and high data rate to ensure reliable optical communication within limited computational abilities of the most used commercial cameras such as those in smartphones, vehicles, and mobile robots. In the proposed scheme, we have designed an intelligent camera receiver system that is capable of separating accurate data transmitting LED regions removing other unwanted LED regions employing a support vector machine (SVM) classifier along with a convolutional neural network (CNN) in the camera receiver. CNN is used to detect every LED region from the image frame and then essential features are extracted to feed into an SVM classifier for further accurate classification. The receiver operating characteristic curve and other key performance parameters of the classifier have been analyzed broadly to evaluate the performance, justify the assistance of the SVM classifier in recognizing the accurate LED region, and decode data with low BER. To investigate communication performances, BER analysis, data rate, and inter-symbol interference have been elaborately demonstrated for the proposed intelligent receiver. In addition, BER against distance and BER against data rate have also been exhibited to validate the effectiveness of our proposed scheme comparing with only CNN and only SVM classifier based receivers individually. Experimental results have ensured the robustness and applicability of the proposed scheme both in the static and mobile scenarios.

A Compact High-Quality Image Demosaicking Neural Network for Edge-Computing Devices

Image demosaicking has been an essential and challenging problem among the most crucial steps of image processing behind image sensors. Due to the rapid development of intelligent processors based on deep learning, several demosaicking methods based on a convolutional neural network (CNN) have been proposed. However, it is difficult for their networks to run in real-time on edge computing devices with a large number of model parameters. This paper presents a compact demosaicking neural network based on the UNet++ structure. The network inserts densely connected layer blocks and adopts Gaussian smoothing layers instead of down-sampling operations before the backbone network. The densely connected blocks can extract mosaic image features efficiently by utilizing the correlation between feature maps. Furthermore, the block adopts depthwise separable convolutions to reduce the model parameters; the Gaussian smoothing layer can expand the receptive fields without down-sampling image size and discarding image information. The size constraints on the input and output images can also be relaxed, and the quality of demosaicked images is improved. Experiment results show that the proposed network can improve the running speed by 42% compared with the fastest CNN-based method and achieve comparable reconstruction quality as it on four mainstream datasets. Besides, when we carry out the inference processing on the demosaicked images on typical deep CNN networks, Mobilenet v1 and SSD, the accuracy can also achieve 85.83% (top 5) and 75.44% (mAP), which performs comparably to the existing methods. The proposed network has the highest computing efficiency and lowest parameter number through all methods, demonstrating that it is well suitable for applications on modern edge computing devices.

Scalable Single-Crystalline Organic 1D Arrays for Image Sensor

Optoelectronic applications of organic semiconductors demand single-crystalline structures with long-range order and suppressed defects for sustaining efficient carrier transport and long photocarrier lifetime, which are pivotal in photodetection, photovoltaic, and light emission. For integrated devices, an additional requirement of precise patterning is imposed, whereas the patterning of single-crystalline organic microstructures is still challenging because the molecular stacking is easily perturbed by disordered fluids in microdroplets. Herein, a capillary-bridge lithography is developed for driving the directional transport of capillary flows to control the confined crystallization of organic 1D single-crystalline arrays with aligned positioning and pure orientation. Through tuning the concentration and pressure, the size of organic 1D arrays in three dimensions can be controlled with 2.9-5.8 µm in width and 1.2 µm to 110 nm in height. Organic 1D array photodetectors exhibit a stable performance with on/off ratio of 180 and responsivity of 4.99 mA W^-1 . Based on the scalable fabrication of 1D array photodetectors, 20 × 20 multiplexed image sensors with high accuracy are demonstrated for capturing the light signals of capital letter "A," "B," and "C." This research will open opportunities for the large-scale fabrication of organic single-crystalline semiconductors toward the integrated optoelectronic modules.

Toward the Super Temporal Resolution Image Sensor with a Germanium Photodiode for Visible Light

The theoretical temporal resolution limit tT of a silicon photodiode (Si PD) is 11.1 ps. We call “super temporal resolution” the temporal resolution that is shorter than that limit. To achieve this resolution, Germanium is selected as a candidate material for the photodiode (Ge PD) for visible light since the absorption coefficient of Ge for the wavelength is several tens of times higher than that of Si, allowing a very thin PD. On the other hand, the saturation drift velocity of electrons in Ge is about 2/3 of that in Si. The ratio suggests an ultra-short propagation time of electrons in the Ge PD. However, the diffusion coefficient of electrons in Ge is four times higher than that of Si. Therefore, Monte Carlo simulations were applied to analyze the temporal resolution of the Ge PD. The estimated theoretical temporal resolution limit is 0.26 ps, while the practical limit is 1.41 ps. To achieve a super temporal resolution better than 11.1 ps, the driver circuit must operate at least 100 GHz. It is thus proposed to develop, at first, a short-wavelength infrared (SWIR) ultra-high-speed image sensor with a thicker and wider Ge PD, and then gradually decrease the size along with the progress of the driver circuits.

Fixed Pattern Noise Reduction and Linearity Improvement in Time-Mode CMOS Image Sensors

In the paper, a digital clock stopping technique for gain and offset correction in time-mode analog-to-digital converters (ADCs) has been proposed. The technique is dedicated to imagers with massively parallel image acquisition working in the time mode where compensation of dark signal non-uniformity (DSNU) as well as photo-response non-uniformity (PRNU) is critical. Fixed pattern noise (FPN) reduction has been experimentally validated using 128-pixel CMOS imager. The reduction of the PRNU to about 0.5 LSB has been achieved. Linearity improvement technique has also been proposed, which allows for integral nonlinearity (INL) reduction to about 0.5 LSB. Measurements confirm the proposed approach.

Automotive 3.0 µm Pixel High Dynamic Range Sensor with LED Flicker Mitigation

We present and discuss parameters of a high dynamic range (HDR) image sensor with LED flicker mitigation (LFM) operating in automotive temperature range. The total SNR (SNR including dark fixed pattern noise), of the sensor is degraded by floating diffusion (FD) dark current (DC) and dark signal non-uniformity (DSNU). We present results of FD DC and DSNU reduction, to provide required SNR versus signal level at temperatures up to 120 °C. Additionally we discuss temperature dependencies of quantum efficiency (QE), sensitivity, color effects, and other pixel parameters for backside illuminated image sensors. Comparing +120 °C junction vs. room temperature, in visual range we measured a few relative percent increase, while in 940 nm band range we measured 1.46x increase in sensitivity. Measured change of sensitivity for visual bands—such as blue, green, and red colors—reflected some impact to captured image color accuracy that created slight image color tint at high temperature. The tint is, however, hard to detect visually and may be removed by auto white balancing and temperature adjusted color correction matrixes.

Chip-Scale Angle-Selective Imager for In Vivo Microscopic Cancer Detection

We present an image sensor incorporating angle-selective gratings for resolution enhancement in contact imaging applications. Optical structures designed in the CMOS metal layers above each photodiode form the angle-selective gratings that limit the sensor angle of view to ±18 ^°, rejecting background light and deblurring the image. The imager is based on a high-gain capacitive transimpedance amplifier pixel using a custom 11fF MOM capacitor, achieving [Formula: see text] sensitivity. The pixel includes a leakage current minimization circuit to remove signal-dependent reset switch leakage and the corresponding dark current is [Formula: see text]. The resulting 4.7 mm by 2.25 mm sensor (80 by 36 pixels) is designed specifically for intraoperative cancer imaging in order to solve the pervasive challenge of identifying microscopic residual cancer foci in vivo, where they can be removed. We demonstrate imaging and detection of foci containing less than 200 cancer cells labeled with fluorescent biomarkers in 50 ms with signal-to-noise ratios greater than 15 dB and the detection of microscopic residual tumor in mice models. The absence of large optical elements enables extreme miniaturization, allowing manipulation within a small, morphologically complex, tumor cavity.

Ultrathin Polymer Nanofibrils for Solar-Blind Deep Ultraviolet Light Photodetectors Application

Solar-blind deep ultraviolet photodetectors (DUVPDs) based on conventional inorganic ultrawide bandgap semiconductors (UWBS) have shown promising application in various civil and military fields and yet they can hardly be used in wearable optoelectronic devices and systems for lack of mechanical flexibility. In this study, we report a non-UWBS solar-blind DUVPD by designing ultrathin polymer nanofibrils with a virtual ultrawide bandgap, which was obtained by grafting P3HT with PHA via a polymerization process. Optoelectronic analysis reveals that the P3HT-b-PHA nanofibrils are sensitive to DUV light with a wavelength of 254 nm but are virtually blind to both 365 nm and other visible light illuminations. The responsivity is 120 A/W with an external quantum efficiency of up to 49700%, implying a large photoconductive gain in the photoresponse process. The observed solar-blind DUV photoresponse is associated with the resonant mode due to the leakage mode of the ultrathin polymer nanofibrils. Moreover, a flexible image sensor composed of 10 × 10 pixels can also be fabricated to illustrate their capability for image sensing application. These results signify that the present ultrathin P3HT-b-PHA nanofibrils are promising building blocks for assembly of low-cost, flexible, and high-performance solar-blind DUVPDs.

Development of Rheumatoid Arthritis Classification from Electronic Image Sensor Using Ensemble Method

Rheumatoid arthritis (RA) is an autoimmune illness that impacts the musculoskeletal system by causing chronic, inflammatory, and systemic effects. The disease often becomes progressive and reduces physical function, causes suffering, fatigue, and articular damage. Over a long period of time, RA causes harm to the bone and cartilage of the joints, weakens the joints' muscles and tendons, eventually causing joint destruction. Sensors such as accelerometer, wearable sensors, and thermal infrared camera sensor are widely used to gather data for RA. In this paper, the classification of medical disorders based on RA and orthopaedics datasets using Ensemble methods are discussed. The RA dataset was gathered from the analysis of white blood cell classification using features extracted from the image of lymphocytes acquired from a digital microscope with an electronic image sensor. The orthopaedic dataset is a benchmark dataset for this study, as it posed a similar classification problem with several numerical features. Three ensemble algorithms such as bagging, Adaboost, and random subspace were used in the study. These ensemble classifiers use k-NN (K-nearest neighbours) and Random forest (RF) as the base learners of the ensemble classifiers. The data classification is accessed using holdout and 10-fold cross-validation evaluation methods. The assessment was based on set of performance measures such as precision, recall, F-measure, and receiver operating characteristic (ROC) curve. The performance was also measured based on the comparison of the overall classification accuracy rate between different ensembles classifiers and the base learners. Overall, it was found that for Dataset 1, random subspace classifier with k-NN shows the best results in terms of overall accuracy rate of 97.50% and for Dataset 2, bagging-RF shows the highest overall accuracy rate of 94.84% over different ensemble classifiers. The findings indicate that the efficiency of the base classifiers with ensemble classifier have substantially improved.

Nonorthogonal Aerial Optoelectronic Platform Based on Triaxial and Control Method Designed for Image Sensors

A traditional aerial optoelectronic platform consists of inside and outside multilayer gimbals, while an internal gimbal and drive components occupy the internal space where optical sensors are located. In order to improve the replaceability of optical sensors and to increase their available space, this paper introduces a nonorthogonal aerial optoelectronic platform based on three axes; we carried out research on its drive control method. A three-dimensional structure of an aerial optoelectronic platform was designed. A noncontact drive of a linear voice coil motor was introduced, and a drive control scheme of a proportional integral and a disturbance observer was adopted. Finally, simulations and experiments were carried out. Results showed that the aerial optoelectronic platform could effectively release three times the image sensor space, and the servo bandwidth was 60.2 Hz, which was much better than that of traditional two-axis and four-gimbal platforms. The stability accuracy of the system reached 4.9958 micron rad, which was obviously better than that of traditional gimbals. This paper provides a reference for the design of new optoelectronic platforms.

Tactile Image Sensors Employing Camera: A Review

A tactile image sensor employing a camera is capable of obtaining rich tactile information through image sequences with high spatial resolution. There have been many studies on the tactile image sensors from more than 30 years ago, and, recently, they have been applied in the field of robotics. Tactile image sensors can be classified into three typical categories according to the method of conversion from physical contact to light signals: Light conductive plate-based, marker displacement- based, and reflective membrane-based sensors. Other important elements of the sensor, such as the optical system, image sensor, and post-image analysis algorithm, have been developed. In this work, the literature is surveyed, and an overview of tactile image sensors employing a camera is provided with a focus on the sensing principle, typical design, and variation in the sensor configuration.

Deep Learning-Based Real-Time Multiple-Object Detection and Tracking from Aerial Imagery via a Flying Robot with GPU-Based Embedded Devices

In recent years, demand has been increasing for target detection and tracking from aerial imagery via drones using onboard powered sensors and devices. We propose a very effective method for this application based on a deep learning framework. A state-of-the-art embedded hardware system empowers small flying robots to carry out the real-time onboard computation necessary for object tracking. Two types of embedded modules were developed: one was designed using a Jetson TX or AGX Xavier, and the other was based on an Intel Neural Compute Stick. These are suitable for real-time onboard computing power on small flying drones with limited space. A comparative analysis of current state-of-the-art deep learning-based multi-object detection algorithms was carried out utilizing the designated GPU-based embedded computing modules to obtain detailed metric data about frame rates, as well as the computation power. We also introduce an effective target tracking approach for moving objects. The algorithm for tracking moving objects is based on the extension of simple online and real-time tracking. It was developed by integrating a deep learning-based association metric approach with simple online and real-time tracking (Deep SORT), which uses a hypothesis tracking methodology with Kalman filtering and a deep learning-based association metric. In addition, a guidance system that tracks the target position using a GPU-based algorithm is introduced. Finally, we demonstrate the effectiveness of the proposed algorithms by real-time experiments with a small multi-rotor drone.

High-Performance Low-Voltage Flexible Photodetector Arrays Based on All-Solid-State Organic Electrochemical Transistors for Photosensing and Imaging

The identifying characteristic of an organic electrochemical transistor (OECT) is the coupling between ionic and electronic charges within the entire volume of the channel. In this work, by taking advantage of the volumetric nature of the OECTs' response, a novel flexible photodetector is reported for the first time based on all-solid-state OECT with an excellent responsivity of up to 6.7 × 10⁶ A/W, detectivity as high as 3.6 × 10¹³ Jones, and a fast response of ∼0.13 s in the visible range, which are superior to those of the majority of the reported organic phototransistors (OPTs) based on field-effect transistors (FETs) and even better than those of FET-based phototransistors with two-dimensional (MoS₂ and graphene) and perovskite materials. The high performance of the devices was ascribed to the combination of the higher carrier mobility of poly(3,4-ethylenedioxythiophene) doped with poly(styrene sulfonate) (PEDOT:PSS) as a channel and the volumetric nature of the OECTs' response, and the charge density of the volumetric channel was efficiently modulated by incident light compared to FETs. Moreover, OECT-based OPTs with quantum dots (CdSe/ZnS) as a light sensitizer were characterized under ultraviolet light, and they exhibited excellent photosensitivity, which further verified the superiority of OECT for phototransistors. Furthermore, a flexible image sensor was fabricated for the first time by integrating flexible OECTs-OPTs into a 10 × 10 array, which can clearly identify the target image under a bending state, indicating the great potential of OECTs-OPTs in the application of low-power, ultrasensitive flexible photodetectors and imaging technology.

[Development Review of Novel Laparoscope Technology]

The proportion of laparoscopic surgeries is continuously increasing in general surgeries. Along with the development and application, new image sensor and digital image processing technology accelerated the emergence of novel laparoscope in recent years. Stereoscopic laparoscope (3D) appearing make the space orientation more accurate. new imaging methods and new structure design satisfy more clinical requirements; combination with optical technology (NBI technique, PDD technique, ICG technique) make intraoperative diagnosis possible.

On the Image Sensor Processing for Lane Detection and Control in Vehicle Lane Keeping Systems

Lane keeping systems for a keeping a vehicle in the desired lane is key to advanced driving assistance system in autonomous vehicles. This paper presents a cost-effective image sensor with efficient processing algorithm for lane detection and lane control applications to autonomous delivery systems. The algorithm includes (1) lane detection by inverse perspective mapping and random sample consensus parabola fitting and (2) lane control by pure pursuit steering controller and classical proportional integral speed controller based on a nonholonomic kinematic model. The image sensor experiments conducted on a 1/10 scale model car maneuvering in a straight⁻curve⁻straight lane validate the better processing performance before, during, and after the turning section over previous work. The image sensor with the processing algorithm achieves the average lane detection error within 5% and maximum cross-track error within 9% in real-time. The development shall pave the way to cost-effective autonomous delivery systems.

Bifunctional Etalon-Electrode To Realize High-Performance Color Filter Free Image Sensor

Organic photodiodes (OPDs), based on organic semiconductors with high absorption coefficients for visible light, are emerging as potential candidates for replacing silicon photodiodes in image sensors, particularly due to the possibility of realizing a thin thickness and exclusion of color filters, both of which can contribute to a dramatically enhanced degree of integration for image sensors. Despite years of research, techniques have not yet been developed that allow the OPD itself to have color selectivity while maintaining a thin (<1 μm) OPD thickness, in combination with a sufficiently high detectivity (>10¹² cm·Hz^0.5/W). To solve this issue, we introduce a concept of "etalon-electrode", which can perform the function of electrode and simultaneously the function of selective wavelength transparency. A strategically designed OPD architecture consisting of an etalon-electrode, a panchromatic organic active layer, and a counter electrode displays well-defined narrowband R-/G-/B-selective detectivity spectra depending on precision-adjusted thickness composition of the etalon-electrode. While a thin thickness of OPD is preserved at less than 800 nm including electrodes, active layer, and other buffer layers for all R-/G-/B-selective OPDs, high average detectivity values over 10¹² cm·Hz^0.5/W are demonstrated. Furthermore, the characteristic of imparting color selectivity by the etalon-electrode enables a more facile full color patterning, such that a prototype of a 10 × 10 image sensor with a pixel pitch of 500 μm is realized, resulting in accurate picturing of a well-defined full color image.

Improved Convolutional Pose Machines for Human Pose Estimation Using Image Sensor Data

In recent years, increasing human data comes from image sensors. In this paper, a novel approach combining convolutional pose machines (CPMs) with GoogLeNet is proposed for human pose estimation using image sensor data. The first stage of the CPMs directly generates a response map of each human skeleton’s key points from images, in which we introduce some layers from the GoogLeNet. On the one hand, the improved model uses deeper network layers and more complex network structures to enhance the ability of low level feature extraction. On the other hand, the improved model applies a fine-tuning strategy, which benefits the estimation accuracy. Moreover, we introduce the inception structure to greatly reduce parameters of the model, which reduces the convergence time significantly. Extensive experiments on several datasets show that the improved model outperforms most mainstream models in accuracy and training time. The prediction efficiency of the improved model is improved by 1.023 times compared with the CPMs. At the same time, the training time of the improved model is reduced 3.414 times. This paper presents a new idea for future research.

Noise Estimation for Image Sensor Based on Local Entropy and Median Absolute Deviation

Noise estimation for image sensor is a key technique in many image pre-processing applications such as blind de-noising. The existing noise estimation methods for additive white Gaussian noise (AWGN) and Poisson-Gaussian noise (PGN) may underestimate or overestimate the noise level in the situation of a heavy textured scene image. To cope with this problem, a novel homogenous block-based noise estimation method is proposed to calculate these noises in this paper. Initially, the noisy image is transformed into the map of local gray statistic entropy (LGSE), and the weakly textured image blocks can be selected with several biggest LGSE values in a descending order. Then, the Haar wavelet-based local median absolute deviation (HLMAD) is presented to compute the local variance of these selected homogenous blocks. After that, the noise parameters can be estimated accurately by applying the maximum likelihood estimation (MLE) to analyze the local mean and variance of selected blocks. Extensive experiments on synthesized noised images are induced and the experimental results show that the proposed method could not only more accurately estimate the noise of various scene images with different noise levels than the compared state-of-the-art methods, but also promote the performance of the blind de-noising algorithm.

A 512×512 SPAD Image Sensor with Integrated Gating for Widefield FLIM

We report on SwissSPAD2, an image sensor with 512×512 photon-counting pixels, each comprising a single-photon avalanche diode (SPAD), a 1-bit memory, and a gating mechanism capable of turning the SPAD on and off, with a skew of 250ps and 344ps, respectively, for a minimum duration of 5.75ns. The sensor is designed to achieve a frame rate of up to 97,700 binary frames per second and sub-40ps gate shifts. By synchronizing it with a pulsed laser and using multiple successive overlapping gates, one can reconstruct a molecule's fluorescent response with picosecond temporal resolution. Thanks to the sensor's number of pixels (the largest to date) and the fully integrated gated operation, SwissSPAD2 enables widefield FLIM with an all-solid-state solution and at relatively high frame rates. This was demonstrated with preliminary results on organic dyes and semiconductor quantum dots using both decay fitting and phasor analysis. Furthermore, pixels with an exceptionally low dark count rate and high photon detection probability enable uniform and high quality imaging of biologically relevant fluorescent samples stained with multiple dyes. While future versions will feature the addition of microlenses and optimize firmware speed, our results open the way to low-cost alternatives to commercially available scientific time-resolved imagers.

A CMOS SPAD Imager with Collision Detection and 128 Dynamically Reallocating TDCs for Single-Photon Counting and 3D Time-of-Flight Imaging

Per-pixel time-to-digital converter (TDC) architectures have been exploited by single-photon avalanche diode (SPAD) sensors to achieve high photon throughput, but at the expense of fill factor, pixel pitch and readout efficiency. In contrast, TDC sharing architecture usually features high fill factor at small pixel pitch and energy efficient event-driven readout. While the photon throughput is not necessarily lower than that of per-pixel TDC architectures, since the throughput is not only decided by the TDC number but also the readout bandwidth. In this paper, a SPAD sensor with 32 × 32 pixels fabricated with a 180 nm CMOS image sensor technology is presented, where dynamically reallocating TDCs were implemented to achieve the same photon throughput as that of per-pixel TDCs. Each 4 TDCs are shared by 32 pixels via a collision detection bus, which enables a fill factor of 28% with a pixel pitch of 28.5 μm. The TDCs were characterized, obtaining the peak-to-peak differential and integral non-linearity of -0.07/+0.08 LSB and -0.38/+0.75 LSB, respectively. The sensor was demonstrated in a scanning light-detection-and-ranging (LiDAR) system equipped with an ultra-low power laser, achieving depth imaging up to 10 m at 6 frames/s with a resolution of 64 × 64 with 50 lux background light.

An Image Signal Accumulation Multi-Collection-Gate Image Sensor Operating at 25 Mfps with 32 × 32 Pixels and 1220 In-Pixel Frame Memory

The paper presents an ultra-high-speed image sensor for motion pictures of reproducible events emitting very weak light. The sensor is backside-illuminated. Each pixel is equipped with the multiple collection gates (MCG) at the center of the front side. Each collection gate is connected to an in-pixel large memory unit, which can accumulate image signals captured by repetitive imaging. The combination of the backside illumination, image signal accumulation, and slow readout from the in-pixel signal storage after an image capturing operation offers a very high sensitivity. Pipeline signal transfer from the MCG to the in-pixel memory units enables the sensor to achieve a large frame count and a very high frame rate at the same time. A test sensor was fabricated with a pixel count of 32 × 32 pixels. Each pixel is equipped with four collection gates, each connected to a memory unit with 305 elements; thus, with a total frame count of 1220 (305 × 4) frames. The test camera achieved 25 Mfps, while the sensor was designed to operate at 50 Mfps.