Comparison of 21 artificial intelligence algorithms in automated diabetic retinopathy screening using handheld fundus camera

BACKGROUND

Diabetic retinopathy (DR) is a common complication of diabetes and may lead to irreversible visual loss. Efficient screening and improved treatment of both diabetes and DR have amended visual prognosis for DR. The number of patients with diabetes is increasing and telemedicine, mobile handheld devices and automated solutions may alleviate the burden for healthcare. We compared the performance of 21 artificial intelligence (AI) algorithms for referable DR screening in datasets taken by handheld Optomed Aurora fundus camera in a real-world setting.

PATIENTS AND METHODS

Prospective study of 156 patients (312 eyes) attending DR screening and follow-up. Both papilla- and macula-centred 50° fundus images were taken from each eye. DR was graded by experienced ophthalmologists and 21 AI algorithms.

RESULTS

Most eyes, 183 out of 312 (58.7%), had no DR and mild NPDR was noted in 21 (6.7%) of the eyes. Moderate NPDR was detected in 66 (21.2%) of the eyes, severe NPDR in 1 (0.3%), and PDR in 41 (13.1%) composing a group of 34.6% of eyes with referable DR. The AI algorithms achieved a mean agreement of 79.4% for referable DR, but the results varied from 49.4% to 92.3%. The mean sensitivity for referable DR was 77.5% (95% CI 69.1-85.8) and specificity 80.6% (95% CI 72.1-89.2). The rate for images ungradable by AI varied from 0% to 28.2% (mean 1.9%). Nineteen out of 21 (90.5%) AI algorithms resulted in grading for DR at least in 98% of the images.

CONCLUSIONS

Fundus images captured with Optomed Aurora were suitable for DR screening. The performance of the AI algorithms varied considerably emphasizing the need for external validation of screening algorithms in real-world settings before their clinical application.

A systematic review of camera monitor system display layout designs: Integration of existing knowledge

Despite the growing interest in mirrorless vehicles equipped with a camera monitor system (CMS), the human factors research findings on CMS display layout design have not been synthesized yet, hindering the application of the knowledge and the identification of future research directions. In an effort to address the 'lack of integration of the existing knowledge', this literature review addresses the following research questions: 1) what CMS display layout designs have been considered/developed by academic researchers and by automakers, respectively?; 2) among possible CMS display layout design alternatives, which ones have not yet been examined through human factors evaluation studies?; and 3) how do the existing human factors studies on the evaluation of different CMS display layout designs vary in the specifics of research? This review provides significant implications for the ergonomic design of CMS display layouts, including some potential design opportunities and future research directions.

Implications of the scale of detection for inferring co-occurrence patterns from paired camera traps and acoustic recorders

Multifunctional landscapes that support economic activities and conservation of biological diversity (e.g., cattle ranches with native forest) are becoming increasingly important because small remnants of native forest may comprise the only habitat left for some wildlife species. Understanding the co-occurrence between wildlife and disturbance factors, such as poaching activity and domesticated ungulates, is key to successful management of multifunctional landscapes. Tools to measure co-occurrence between wildlife and disturbance factors include camera traps and autonomous acoustic recording units. We paired 52 camera-trap stations with acoustic recorders to investigate the association between 2 measures of disturbance (poaching and cattle) and wild ungulates present in multifunctional landscapes of the Colombian Orinoquía. We used joint species distribution models to investigate species-habitat associations and species-disturbance correlations. One model was fitted using camera-trap data to detect wild ungulates and disturbance factors, and a second model was fitted after replacing camera-trap detections of disturbance factors with their corresponding acoustic detections. The direction, significance, and precision of the effect of covariates depended on the sampling method used for disturbance factors. Acoustic monitoring typically resulted in more precise estimates of the effects of covariates and of species-disturbance correlations. Association patterns between wildlife and disturbance factors were found only when disturbance was detected by acoustic recorders. Camera traps allowed us to detect nonvocalizing species, whereas audio recording devices increased detection of disturbance factors leading to more precise estimates of co-occurrence patterns. The collared peccary (Pecari tajacu), lowland tapir (Tapirus terrestris), and white-tailed deer (Odocoileus virginianus) co-occurred with disturbance factors and are conservation priorities due to the greater risk of poaching or disease transmission from cattle.

Wide-field imaging with smartphone based fundus camera: grading of severity of diabetic retinopathy and locating peripheral lesions in diabetic retinopathy

AIM

To assess the performance of smartphone based wide-field retinal imaging (WFI) versus ultra-wide-field imaging (UWFI) for assessment of sight-threatening diabetic retinopathy (STDR) as well as locating predominantly peripheral lesions (PPL) of DR.

METHODS

Individuals with type 2 diabetes with varying grades of DR underwent nonmydriatic UWFI with Daytona Plus camera followed by mydriatic WFI with smartphone-based Vistaro camera at a tertiary care diabetes centre in South India in 2021-22. Grading of DR as well as identification of PPL (DR lesions beyond the posterior pole) in the retinal images of both cameras was performed by senior retina specialists. STDR was defined by the presence of severe non-proliferative DR, proliferative DR or diabetic macular oedema (DME). The sensitivity and specificity of smartphone based WFI for detection of PPL and STDR was assessed. Agreement between the graders for both cameras was compared.

RESULTS

Retinal imaging was carried out in 318 eyes of 160 individuals (mean age 54.7 ± 9 years; mean duration of diabetes 16.6 ± 7.9 years). The sensitivity and specificity for detection of STDR by Vistaro camera was 92.7% (95% CI 80.1-98.5) and 96.6% (95% CI 91.5-99.1) respectively and 95.1% (95% CI 83.5-99.4) and 95.7% (95% CI 90.3-98.6) by Daytona Plus respectively. PPL were detected in 89 (27.9%) eyes by WFI by Vistaro camera and in 160 (50.3%) eyes by UWFI. However, this did not translate to any significant difference in the grading of STDR between the two imaging systems. In both devices, PPL were most common in supero-temporal quadrant (34%). The prevalence of PPL increased with increasing severity of DR with both cameras (p < 0.001). The kappa comparison between the 2 graders for varying grades of severity of DR was 0.802 (p < 0.001) for Vistaro and 0.753 (p < 0.001) for Daytona Plus camera.

CONCLUSION

Mydriatic smartphone-based widefield imaging has high sensitivity and specificity for detecting STDR and can be used to screen for peripheral retinal lesions beyond the posterior pole in individuals with diabetes.

One-field, two-field and five-field handheld retinal imaging compared with standard seven-field Early Treatment Diabetic Retinopathy Study photography for diabetic retinopathy screening

BACKGROUND/AIMS

To determine agreement of one-field (1F, macula-centred), two-field (2F, disc-macula) and five-field (5F, macula, disc, superior, inferior and nasal) mydriatic handheld retinal imaging protocols for the assessment of diabetic retinopathy (DR) as compared with standard seven-field Early Treatment Diabetic Retinopathy Study (ETDRS) photography.

METHODS

Prospective, comparative instrument validation study. Mydriatic retinal images were taken using three handheld retinal cameras: Aurora (AU; 50° field of view (FOV), 5F), Smartscope (SS; 40° FOV, 5F), and RetinaVue (RV; 60° FOV, 2F) followed by ETDRS photography. Images were evaluated at a centralised reading centre using the international DR classification. Each field protocol (1F, 2F and 5F) was graded independently by masked graders. Weighted kappa (Kw) statistics assessed agreement for DR. Sensitivity (SN) and specificity (SP) for referable diabetic retinopathy (refDR; moderate non-proliferative diabetic retinopathy (NPDR) or worse, or ungradable images) were calculated.

RESULTS

Images from 225 eyes of 116 patients with diabetes were evaluated. Severity by ETDRS photography: no DR, 33.3%; mild NPDR, 20.4%; moderate, 14.2%; severe, 11.6%; proliferative, 20.4%. Ungradable rate for DR: ETDRS, 0%; AU: 1F 2.23%, 2F 1.79%, 5F 0%; SS: 1F 7.6%, 2F 4.0%, 5F 3.6%; RV: 1F 6.7%, 2F 5.8%. Agreement rates of DR grading between handheld retinal imaging and ETDRS photography were (Kw, SN/SP refDR) AU: 1F 0.54, 0.72/0.92; 2F 0.59, 0.74/0.92; 5F 0.75, 0.86/0.97; SS: 1F 0.51, 0.72/0.92; 2F 0.60, 0.75/0.92; 5F 0.73, 0.88/0.92; RV: 1F 0.77, 0.91/0.95; 2F 0.75, 0.87/0.95.

CONCLUSION

When using handheld devices, the addition of peripheral fields decreased the ungradable rate and increased SN and SP for refDR. These data suggest the benefit of additional peripheral fields in DR screening programmes that use handheld retinal imaging.

Using a smartphone-based digital fundus camera for screening of retinal and optic nerve diseases in veterinary medicine: A preliminary investigation

BACKGROUND

Ophthalmoscopy is a valuable tool in clinical practice. We report the use of a novel smartphone-based handheld device for visualisation and photo-documentation of the ocular fundus in veterinary medicine.

METHODS

Selected veterinary patients of a referral ophthalmology service were included if one or both eyes had clear ocular media, allowing for examination of the fundus. Following pharmacological mydriasis, fundic images were obtained with a handheld fundus camera (Volk VistaView). For comparison, the fundus of a subset of animals was also imaged with a veterinary-specific fundus camera (Optomed Smartscope VET2).

RESULTS

The large field of view achieved by the Volk VistaView allowed for rapid and thorough observation of the ocular fundus in animals, providing a tool to visualise and record common pathologies of the posterior segment. Captured fundic images were sometimes overexposed, with the tapetal fundus artificially appearing hyperreflective when using the Volk VistaView camera, a finding that was less frequent when activating a 'veterinary mode' that reduced the sensitivity of the camera's sensor. The Volk VistaView compared well with the Optomed Smartscope VET2.

LIMITATION

The main study limitation was the small sample size.

CONCLUSIONS

The Volk VistaView camera was easy to use and provided good-quality fundic images in veterinary patients with healthy or diseased eyes, offering a wide field of view that was ideal for screening purposes.

Low-latency automotive vision with event cameras

The computer vision algorithms used currently in advanced driver assistance systems rely on image-based RGB cameras, leading to a critical bandwidth-latency trade-off for delivering safe driving experiences. To address this, event cameras have emerged as alternative vision sensors. Event cameras measure the changes in intensity asynchronously, offering high temporal resolution and sparsity, markedly reducing bandwidth and latency requirements1. Despite these advantages, event-camera-based algorithms are either highly efficient but lag behind image-based ones in terms of accuracy or sacrifice the sparsity and efficiency of events to achieve comparable results. To overcome this, here we propose a hybrid event- and frame-based object detector that preserves the advantages of each modality and thus does not suffer from this trade-off. Our method exploits the high temporal resolution and sparsity of events and the rich but low temporal resolution information in standard images to generate efficient, high-rate object detections, reducing perceptual and computational latency. We show that the use of a 20 frames per second (fps) RGB camera plus an event camera can achieve the same latency as a 5,000-fps camera with the bandwidth of a 45-fps camera without compromising accuracy. Our approach paves the way for efficient and robust perception in edge-case scenarios by uncovering the potential of event cameras2.

Optical Camera Communications in Healthcare: A Wearable LED Transmitter Evaluation during Indoor Physical Exercise

This paper presents an experimental evaluation of a wearable light-emitting diode (LED) transmitter in an optical camera communications (OCC) system. The evaluation is conducted under conditions of controlled user movement during indoor physical exercise, encompassing both mild and intense exercise scenarios. We introduce an image processing algorithm designed to identify a template signal transmitted by the LED and detected within the image. To enhance this process, we utilize the dynamics of controlled exercise-induced motion to limit the tracking process to a smaller region within the image. We demonstrate the feasibility of detecting the transmitting source within the frames, and thus limit the tracking process to a smaller region within the image, achieving an reduction of 87.3% for mild exercise and 79.0% for intense exercise.

Smartphone fundus photography by a physician in medical emergencies: An analytical cross-sectional study of 182 patients

Background Fundus examination is an integral part of the clinical evaluation of patients with medical emergencies. It is done at the bedside using a portable direct ophthalmoscope. Smartphone fundus photography (SFP) is a novel technique of retinal imaging. We evaluated the use of SFP by a physician in medical emergencies and compared it with direct ophthalmoscopy (DO) findings of the ophthalmologist. Methods We did a prospective study on patients admitted with medical emergencies with an indication for fundus examination. The SFP was done by the physician, and its findings were noted. These were compared with the DO findings of the ophthalmologist. Results Of the 182 patients studied, 111 (61%) had fundus findings by SFP and 95 (52.5%) by DO. Papilloedema (21.4%), haemorrhages (20%) and Roth spots (12.5%) were most common. DO missed early papilloedema and findings in the peripheral retina. Conclusions SFP is as effective as DO, in detecting retinal findings in patients with medical emergencies and can be performed at the bedside by the physician.

Reflection-Aware Generation and Identification of Square Marker Dictionaries

Square markers are a widespread tool to find correspondences for camera localization because of their robustness, accuracy, and detection speed. Their identification is usually based on a binary encoding that accounts for the different rotations of the marker; however, most systems do not consider the possibility of observing reflected markers. This case is possible in environments containing mirrors or reflective surfaces, and its lack of consideration is a source of detection errors, which is contrary to the robustness expected from square markers. This is the first work in the literature that focuses on reflection-aware square marker dictionaries. We present the derivation of the inter-marker distance of a reflection-aware dictionary and propose new algorithms for generating and identifying such dictionaries. Additionally, part of the proposed method can be used to optimize preexisting dictionaries to take reflection into account. The experimentation carried out demonstrates how our proposal greatly outperforms the most popular predefined dictionaries in terms of inter-marker distance and how the optimization process significantly improves them.

Dual stereo-digital image correlation system for simultaneous measurement of overlapped wings with a polarization RGB camera and fluorescent speckle patterns

Simultaneous monitoring of overlapped multi-wing structure by stereo-digital image correlation (stereo-DIC) may be used to quantify insect motion and deformation. We propose a dual stereo-DIC system based on multispectral imaging with a polarization RGB camera. Different fluorescent speckle patterns were fabricated on wings, which emit red and blue spectra under ultraviolet light that were imaged and separated using a polarization RGB camera and auxiliary optical splitting components. The resulting dual stereo-DIC system was validated through translation experiments with transparent sheets and reconstructed overlapped insect wings (cicadas). Dynamic measurements of the Ruban artificial flier indicate the efficacy of this approach to determining real insect flight behavior.

A Novel Virtual Wet Lab-Using a Smartphone Camera Adapter and a Video Conference Platform to Provide Real-Time Surgical Instruction

PURPOSE

Proctored surgical instruction has traditionally been taught through in-person interactions in either the operating room or an improvised wet lab. Because of the COVID-19 pandemic, live in-person instruction was not feasible owing to social distancing protocols, so a virtual wet lab (VWL) was proposed and implemented. The purpose of this article is to describe our experience with a VWL as a Descemet membrane endothelial keratoplasty (DMEK) skills-transfer course. This is the first time that a VWL environment has been described for the instruction of ophthalmic surgery.

METHODS

Thirteen participant surgeons took part in VWLs designed for DMEK skills transfer in September and October 2020. A smartphone camera adapter and a video conference software platform were the unique media for the VWL. After a didactic session, participants were divided into breakout rooms where their surgical scope view was broadcast live, allowing instructors to virtually proctor their participants in real time. Participants were surveyed to assess their satisfaction with the course.

RESULTS

All (100%) participants successfully injected and unfolded their DMEK grafts. Ten of the 13 participants completed the survey. Respondents rated the experience highly favorably.

CONCLUSIONS

With the use of readily available technology, VWLs can be successfully implemented in lieu of in-person skills-transfer courses. Further development catering to the needs of the participant might allow VWLs to serve as a viable option of surgical education, currently limited by geographical and social distancing boundaries.

Evaluation of photography using head-mounted display technology (ICAPS) for district Trachoma surveys

Background

As the prevalence of trachoma declines worldwide, it is becoming increasingly expensive and challenging to standardize graders in the field for surveys to document elimination. Photography of the tarsal conjunctiva and remote interpretation may help alleviate these challenges. The purpose of this study was to develop, and field test an Image Capture and Processing System (ICAPS) to acquire hands-free images of the tarsal conjunctiva for upload to a virtual reading center for remote grading.

Methodology/Principal findings

This observational study was conducted during a district-level prevalence survey for trachomatous inflammation—follicular (TF) in Chamwino, Tanzania. The ICAPS was developed using a Samsung Galaxy S8 smartphone, a Samsung Gear VR headset, a foot pedal trigger and customized software allowing for hands-free photography. After a one-day training course, three trachoma graders used the ICAPS to collect images from 1305 children ages 1–9 years, which were expert-graded remotely for comparison with field grades. In our experience, the ICAPS was successful at scanning and assigning barcodes to images, focusing on the everted eyelid with adequate examiner hand visualization, and capturing images with sufficient detail to grade TF. The percentage of children with TF by photos and by field grade was 5%. Agreement between grading of the images compared to the field grades at the child level was kappa = 0.53 (95%CI = 0.40–0.66). There were ungradable images for at least one eye in 199 children (9.1%), with more occurring in children ages 1–3 (18.5%) than older children ages 4–9 (4.2%) (χ² = 145.3, p<0.001).

Conclusions/Significance

The prototype ICAPS device was robust, able to image 1305 children in a district level survey and transmit images from rural Tanzania to an online grading platform. More work is needed to improve the percentage of ungradable images and to better understand the causes of disagreement between field and photo grading.

Trachoma is the leading infectious cause of blindness worldwide, caused by the bacterium Chlamydia trachomatis. Programs targeting trachoma elimination in endemic regions largely rely on periodic prevalence surveys to monitor progress, but training field graders requires active cases, which is becoming challenging as prevalence declines. Photography of the tarsal conjunctiva with remote interpretation via telemedicine may serve as a more auditable, effective, and cost-efficient method for surveys. We developed and evaluated the Image Capture and Processing System (ICAPS), a smartphone-based, hands-free, head-mounted camera system (Samsung Galaxy S8 with custom app, Samsung Gear VR headset, and a Bluetooth-linked foot pedal trigger). The ICAPS was easy to use in challenging field conditions, was able to upload images from Tanzania and link images to field data. The percentage of TF was 5% by both field grade and photo grade, with agreement kappa = 0.53. Additional field training and enhanced certification of photographers may help reduce the proportion of ungradable images; further research on reasons for mismatch of grades between field and photo is needed.

Partial-field illumination ophthalmoscope: improving the contrast of a camera-based retinal imager

Effective and accurate in vivo diagnosis of retinal pathologies requires high performance imaging devices, combining a large field of view and the ability to discriminate the ballistic signal from the diffuse background in order to provide a highly contrasted image of the retinal structures. Here, we have implemented the partial-field illumination ophthalmoscope, a patterned illumination modality, integrated to a high pixel rate adaptive optics full-field microscope. This non-invasive technique enables us to mitigate the low signal-to-noise ratio, intrinsic of full-field ophthalmoscopes, by partially illuminating the retina with complementary patterns to reconstruct a wide-field image. This new, to the best of our knowledge, modality provides an image contrast spanning from the full-field to the confocal contrast, depending on the pattern size. As a result, it offers various trade-offs in terms of contrast and acquisition speed, guiding the users towards the most efficient system for a particular clinical application.

Deep Learning for the Diagnosis of Stage in Retinopathy of Prematurity: Accuracy and Generalizability across Populations and Cameras

PURPOSE

Stage is an important feature to identify in retinal images of infants at risk of retinopathy of prematurity (ROP). The purpose of this study was to implement a convolutional neural network (CNN) for binary detection of stages 1, 2, and 3 in ROP and to evaluate its generalizability across different populations and camera systems.

DESIGN

Diagnostic validation study of CNN for stage detection.

PARTICIPANTS

Retinal fundus images obtained from preterm infants during routine ROP screenings.

METHODS

Two datasets were used: 5943 fundus images obtained by RetCam camera (Natus Medical, Pleasanton, CA) from 9 North American institutions and 5049 images obtained by 3nethra camera (Forus Health Incorporated, Bengaluru, India) from 4 hospitals in Nepal. Images were labeled based on the presence of stage by 1 to 3 expert graders. Three CNN models were trained using 5-fold cross-validation on datasets from North America alone, Nepal alone, and a combined dataset and were evaluated on 2 held-out test sets consisting of 708 and 247 images from the Nepali and North American datasets, respectively.

MAIN OUTCOME MEASURES

Convolutional neural network performance was evaluated using area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), sensitivity, and specificity.

RESULTS

Both the North American- and Nepali-trained models demonstrated high performance on a test set from the same population: AUROC, 0.99; AUPRC, 0.98; sensitivity, 94%; and AUROC, 0.97; AUPRC, 0.91; and sensitivity, 73%; respectively. However, the performance of each model decreased to AUROC of 0.96 and AUPRC of 0.88 (sensitivity, 52%) and AUROC of 0.62 and AUPRC of 0.36 (sensitivity, 44%) when evaluated on a test set from the other population. Compared with the models trained on individual datasets, the model trained on a combined dataset achieved improved performance on each respective test set: sensitivity improved from 94% to 98% on the North American test set and from 73% to 82% on the Nepali test set.

CONCLUSIONS

A CNN can identify accurately the presence of ROP stage in retinal images, but performance depends on the similarity between training and testing populations. We demonstrated that internal and external performance can be improved by increasing the heterogeneity of the training dataset features of the training dataset, in this case by combining images from different populations and cameras.

ESIDE: A computationally intelligent method to identify earthworm species (E. fetida) from digital images: Application in taxonomy

Earthworms (Crassiclitellata) being ecosystem engineers significantly affect the physical, chemical, and biological properties of the soil by recycling organic material, increasing nutrient availability, and improving soil structure. The efficiency of earthworms in ecology varies along with species. Therefore, the role of taxonomy in earthworm study is significant. The taxonomy of earthworms cannot reliably be established through morphological characteristics because the small and simple body plan of the earthworm does not have anatomical complex and highly specialized structures. Recently, molecular techniques have been adopted to accurately classify the earthworm species but these techniques are time-consuming and costly. To combat this issue, in this study, we propose a machine learning-based earthworm species identification model that uses digital images of earthworms. We performed a stringent performance evaluation not only through 10-fold cross-validation and on an external validation dataset but also in real settings by involving an experienced taxonomist. In all the evaluation settings, our proposed model has given state-of-the-art performance and justified its use to aid earthworm taxonomy studies. We made this model openly accessible through a cloud-based webserver and python code available at https://sites.google.com/view/wajidarshad/software and https://github.com/wajidarshad/ESIDE.

Calibration and location analysis of a heterogeneous binocular stereo vision system

In the dairy farming industry, we can obtain the temperature, color, and location information of dairy cows by patrol inspection robot so as to monitor the health status and abnormal behaviors of dairy cows. We build and calibrate a heterogeneous binocular stereo vision (HBSV) system comprising a high-definition color camera and infrared thermal camera and mount it on a patrol inspection robot. First, based on the traditional chessboard, an easy-to-make calibration board for the HBSV system is designed. Second, an accurate locating and sorting algorithm for the calibration points of the calibration board is designed. Then, the cameras are calibrated and the HBSV system is stereo-calibrated. Finally, target locating is achieved based on the above calibration results and Yolo target detection technology. In this paper, several experiments are carried out from many aspects. The target locating average error of HBSV system is 3.11%, which satisfies the needs of the dairy farming environment. The video's FPS captured by using HBSV is 7.3, which is 78% higher than that by using binocular stereo vision system and infrared thermal camera. The results show that the HBSV system has application value to a certain degree.

Detection of Burmese pythons in the near-infrared versus visible band

Human task performance studies are commonly used for detecting and identifying potential military threats. In this work, these principles are applied to detection of an environmental threat: the invasive Burmese python. A qualitative detection of Burmese pythons with a visible light camera and an 850 nm near-infrared (NIR) camera was performed in natural Florida backgrounds. The results showed that the difference in reflectivity between the pythons and native foliage was much greater in NIR, effectively circumventing the python's natural camouflage in the visible band. In this work, a comparison of detection performance in the selected near-infrared band versus the visible band was conducted. Images of foliage backgrounds with and without a python were taken in each band in daylight and at night with illumination. Intensities of these images were then calibrated and prepared for a human perception test. Participants were tasked with detecting pythons, and the human perception data was used to compare performance between the bands. The results show that the enhanced contrast in the NIR enabled participants to detect pythons at 20% longer ranges than the use of visible imagery.

Compressive recovery of smartphone RGB spectral sensitivity functions

Spectral response (or sensitivity) functions of a three-color image sensor (or trichromatic camera) allow a mapping from spectral stimuli to RGB color values. Like biological photosensors, digital RGB spectral responses are device dependent and significantly vary from model to model. Thus, the information on the RGB spectral response functions of a specific device is vital in a variety of computer vision as well as mobile health (mHealth) applications. Theoretically, spectral response functions can directly be measured with sophisticated calibration equipment in a specialized laboratory setting, which is not easily accessible for most application developers. As a result, several mathematical methods have been proposed relying on standard color references. Typical optimization frameworks with constraints are often complicated, requiring a large number of colors. We report a compressive sensing framework in the frequency domain for accurately predicting RGB spectral response functions only with several primary colors. Using a scientific camera, we first validate the estimation method with direct spectral sensitivity measurements and ensure that the root mean square errors between the ground truth and recovered RGB spectral response functions are negligible. We further recover the RGB spectral response functions of smartphones and validate with an expanded color checker reference. We expect that this simple yet reliable estimation method of RGB spectral sensitivity can easily be applied for color calibration and standardization in machine vision, hyperspectral filters, and mHealth applications that capitalize on the built-in cameras of smartphones.

Monitoring social distancing under various low light conditions with deep learning and a single motionless time of flight camera

The purpose of this work is to provide an effective social distance monitoring solution in low light environments in a pandemic situation. The raging coronavirus disease 2019 (COVID-19) caused by the SARS-CoV-2 virus has brought a global crisis with its deadly spread all over the world. In the absence of an effective treatment and vaccine the efforts to control this pandemic strictly rely on personal preventive actions, e.g., handwashing, face mask usage, environmental cleaning, and most importantly on social distancing which is the only expedient approach to cope with this situation. Low light environments can become a problem in the spread of disease because of people's night gatherings. Especially, in summers when the global temperature is at its peak, the situation can become more critical. Mostly, in cities where people have congested homes and no proper air cross-system is available. So, they find ways to get out of their homes with their families during the night to take fresh air. In such a situation, it is necessary to take effective measures to monitor the safety distance criteria to avoid more positive cases and to control the death toll. In this paper, a deep learning-based solution is proposed for the above-stated problem. The proposed framework utilizes the you only look once v4 (YOLO v4) model for real-time object detection and the social distance measuring approach is introduced with a single motionless time of flight (ToF) camera. The risk factor is indicated based on the calculated distance and safety distance violations are highlighted. Experimental results show that the proposed model exhibits good performance with 97.84% mean average precision (mAP) score and the observed mean absolute error (MAE) between actual and measured social distance values is 1.01 cm.

Validation of a Three-Dimensional Head and Neck Spheroid Model to Evaluate Cameras for NIR Fluorescence-Guided Cancer Surgery

Near-infrared (NIR) fluorescence-guided surgery is an innovative technique for the real-time visualization of resection margins. The aim of this study was to develop a head and neck multicellular tumor spheroid model and to explore the possibilities offered by it for the evaluation of cameras for NIR fluorescence-guided surgery protocols. FaDu spheroids were incubated with indocyanine green (ICG) and then included in a tissue-like phantom. To assess the capability of Fluobeam® NIR camera to detect ICG in tissues, FaDu spheroids exposed to ICG were embedded in 2, 5 or 8 mm of tissue-like phantom. The fluorescence signal was significantly higher between 2, 5 and 8 mm of depth for spheroids treated with more than 5 µg/mL ICG (p < 0.05). The fluorescence intensity positively correlated with the size of spheroids (p < 0.01), while the correlation with depth in the tissue-like phantom was strongly negative (p < 0.001). This multicellular spheroid model embedded in a tissue-like phantom seems to be a simple and reproducible in vitro tumor model, allowing a comparison of NIR cameras. The ideal configuration seems to be 450 μm FaDu spheroids incubated for 24 h with 0.05 mg/mL of ICG, ensuring the best stability, toxicity, incorporation and signal intensity.

Optical design and fabrication of a smartphone fundus camera

Fundus examination plays an important part in a medical setting. The fundus camera is one of the detection instruments used in obtaining fundus images, which can reflect information about disease and other conditions. However, traditional fundus cameras have many disadvantages in regard to data sharing, image recognition, and processing, as well as doctor-patient communication. In recent years, mobile medical systems have gradually become more prevalent in medical and health system environments. In this paper, we propose a design method for a smartphone fundus camera consisting of an illumination system and an imaging system. The end of the system can be combined with a smartphone to take the fundus images directly. We manufactured a prototype, designed an artificial eye model, and carried out a series of experiments. The results show that we can get fundus images clearly, and the imaging system will be able to correct refractive errors ranging from -8D∼+8D. The spatial resolution of the system is up to 15 µm. This is a portable device with an overall size of 160mm×160mm×80mm and a weight of 540 g. It has the advantages of lower price, simple operation, high resolution, and compact size, making it suitable as a portable ocular monitoring device.