Smartphone-based multispectral imaging: system development and potential for mobile skin diagnosis

Artificial intelligence-assisted grading for tear trough deformity

BACKGROUND

Various classification systems for tear trough deformity (TTD) have been published; however, their complexity can pose challenges in clinical use, especially for less experienced surgeons. It is believed that artificial intelligence (AI) technology can address some of these challenges by reducing inadvertent errors and improving the accuracy of medical practice. In this study, we aimed to establish a reliable and precise digital image grading model for TTD using smartphone-based photography enhanced using AI deep learning technology. This model is designed to aid and guide surgeons, particularly those who are less experienced or from younger generations, during clinical examinations and in making decisions regarding further surgical interventions.

MATERIALS AND METHODS

A total of 504 patients and 983 photos were included in the study. We adopted the Barton's grading system for TTD. All photos were taken using the same smartphone and processed and analyzed using the medical AI assistant (MAIA™) software. The photos were then randomly divided into two groups to establish training and testing models.

RESULTS

The confusion matrix for the training model demonstrated a sensitivity of 56%, specificity of 87.3%, F1 score of 0.57, and an area under the curve (AUROC) of 0.85. For the testing group, the sensitivity was 49.3%, specificity was 85%, F1 score was 0.49, and AUROC was 0.83. Representative heatmaps were also generated.

CONCLUSION

Our study is the first to demonstrate that tear trough deformities can be easily categorized using a built-in smartphone camera in conjunction with an AI deep learning program. This approach can reduce errors during clinical patient evaluations, particularly for less experienced practitioners.

Rapid in-situ calibration of computational micro-spectrometer with few-shot meta-learning

Computational micro-spectrometers comprised of detector arrays and encoding structure arrays, such as on-chip Fabry-Perot (FP) cavity filters, have great potential in many in-situ applications owing to their compact size and snapshot imaging ability. Given manufacturing deviation and environmental influence are inevitable, easy and effective calibration for spectrometer is necessary, especially for in-situ applications. Currently calibration strategies based on iterative algorithms or neural networks require accurate measurements of pixel-level (spectral) encoding functions through monochromator or large amounts of standard samples. These procedures are time-consuming and expensive, thereby impeding in-situ applications. Meta-learning algorithms with few-shot learning ability can address this challenge by incorporating the prior knowledge in the simulated dataset. In this work, we propose a meta-learning algorithm free of measuring encoding function or large amounts of standard samples to calibrate a micro-spectrometer with manufacturing deviation effectively. Our micro-spectrometer comprises 16 types of FP filters covering a wavelength range of 550-720 nm. The center wavelength of each filter type deviates from the design up to 6 nm. After calibration with 15 different color data, the average reconstruction error on the test dataset decreased from 7.2 × 10-3 to 1.2 × 10-3, and further decreased to 9.4 × 10-4 when the calibration data increased to 24. The performance is comparable to algorithms trained with measured encoding function both in reconstruction error and generalization ability. We estimated that the cost of in-situ calibration through reflectance measurements of color chart decreased to one percent of the cost through monochromator measurements. By exploiting prior deviation information in simulation data with meta-learning, the efficiency and cost of calibration are significantly improved, thereby facilitating the large-scale production and in-situ application of micro-spectrometers.

Learning to Recover Spectral Reflectance From RGB Images

This paper tackles spectral reflectance recovery (SRR) from RGB images. Since capturing ground-truth spectral reflectance and camera spectral sensitivity are challenging and costly, most existing approaches are trained on synthetic images and utilize the same parameters for all unseen testing images, which are suboptimal especially when the trained models are tested on real images because they never exploit the internal information of the testing images. To address this issue, we adopt a self-supervised meta-auxiliary learning (MAXL) strategy that fine-tunes the well-trained network parameters with each testing image to combine external with internal information. To the best of our knowledge, this is the first work that successfully adapts the MAXL strategy to this problem. Instead of relying on naive end-to-end training, we also propose a novel architecture that integrates the physical relationship between the spectral reflectance and the corresponding RGB images into the network based on our mathematical analysis. Besides, since the spectral reflectance of a scene is independent to its illumination while the corresponding RGB images are not, we recover the spectral reflectance of a scene from its RGB images captured under multiple illuminations to further reduce the unknown. Qualitative and quantitative evaluations demonstrate the effectiveness of our proposed network and of the MAXL. Our code and data are available at https://github.com/Dong-Huo/SRR-MAXL.

One-year results for myopia control of orthokeratology with different back optic zone diameters: a randomized trial using a novel multispectral-based topographer

AIM

To present the 1-year results of a prospective cohort study investigating the efficacy, potential mechanism, and safety of orthokeratology (ortho-k) with different back optic zone diameters (BOZD) for myopia control in children.

METHODS

This randomized clinical study was performed between Dec. 2020 and Dec. 2021. Participants were randomly assigned to three groups wearing ortho-k: 5 mm BOZD (5-MM group), 5.5 mm BOZD (5.5-MM group), and 6 mm BOZD (6-MM group). The 1-year data were recorded, including axial length, relative peripheral refraction (RPR, measured by multispectral refractive topography, MRT), and visual quality. The contrast sensitivity (CS) was evaluated by CSV-1000 instrument with spatial frequencies of 3, 6, 12, and 18 cycles/degree (c/d); the corneal higher-order aberrations (HOAs) were measured by iTrace aberration analyzer. The one-way ANOVA was performed to assess the differences between the three groups. The correlation between the change in AL and RPR was calculated by Pearson's correlation coefficient.

RESULTS

The 1-year results of 20, 21, and 21 subjects in the 5-MM, 5.5-MM, and 6-MM groups, respectively, were presented. There were no statistical differences in baseline age, sex, or ocular parameters between the three groups (all P>0.05). At the 1-year visit, the 5-MM group had lower axial elongation than the 6-MM group (0.07±0.09 vs 0.18±0.11 mm, P=0.001). The 5-MM group had more myopic total RPR (TRPR, P=0.014), with RPR in the 15°-30° (RPR 15-30, P=0.015), 30°-45° (RPR 30-45, P=0.011), temporal (RPR-T, P=0.008), and nasal area (RPR-N, P<0.001) than the 6-MM group. RPR 15-30 in the 5.5-MM group was more myopic than that in the 6-MM group (P=0.002), and RPR-N in the 5-MM group was more myopic than that in the 5.5-MM group (P<0.001). There were positive correlations between the axial elongation and the change in TRPR (r=0.756, P<0.001), RPR 15-30 (r=0.364, P=0.004), RPR 30-45 (r=0.306, P=0.016), and RPR-N (r=0.253, P=0.047). The CS decreased at 3 c/d (P<0.001), and the corneal HOAs increased in the 5-MM group (P=0.030).

CONCLUSION

Ortho-k with 5 mm BOZD can control myopia progression more effectively. The mechanism may be associated with greater myopic shifts in RPR.

Multispectral Imaging for Skin Diseases Assessment-State of the Art and Perspectives

Skin optical inspection is an imperative procedure for a suspicious dermal lesion since very early skin cancer detection can guarantee total recovery. Dermoscopy, confocal laser scanning microscopy, optical coherence tomography, multispectral imaging, multiphoton laser imaging, and 3D topography are the most outstanding optical techniques implemented for skin examination. The accuracy of dermatological diagnoses attained by each of those methods is still debatable, and only dermoscopy is frequently used by all dermatologists. Therefore, a comprehensive method for skin analysis has not yet been established. Multispectral imaging (MSI) is based on light-tissue interaction properties due to radiation wavelength variation. An MSI device collects the reflected radiation after illumination of the lesion with light of different wavelengths and provides a set of spectral images. The concentration maps of the main light-absorbing molecules in the skin, the chromophores, can be retrieved using the intensity values from those images, sometimes even for deeper-located tissues, due to interaction with near-infrared light. Recent studies have shown that portable and cost-efficient MSI systems can be used for extracting skin lesion characteristics useful for early melanoma diagnoses. This review aims to describe the efforts that have been made to develop MSI systems for skin lesions evaluation in the last decade. We examined the hardware characteristics of the produced devices and identified the typical structure of an MSI device for dermatology. The analyzed prototypes showed the possibility of improving the specificity of classification between the melanoma and benign nevi. Currently, however, they are rather adjuvants tools for skin lesion assessment, and efforts are needed towards a fully fledged diagnostic MSI device.

Compact and ultracompact spectral imagers: technology and applications in biomedical imaging

Significance

Spectral imaging, which includes hyperspectral and multispectral imaging, can provide images in numerous wavelength bands within and beyond the visible light spectrum. Emerging technologies that enable compact, portable spectral imaging cameras can facilitate new applications in biomedical imaging.

Aim

With this review paper, researchers will (1) understand the technological trends of upcoming spectral cameras, (2) understand new specific applications that portable spectral imaging unlocked, and (3) evaluate proper spectral imaging systems for their specific applications.

Approach

We performed a comprehensive literature review in three databases (Scopus, PubMed, and Web of Science). We included only fully realized systems with definable dimensions. To best accommodate many different definitions of "compact," we included a table of dimensions and weights for systems that met our definition.

Results

There is a wide variety of contributions from industry, academic, and hobbyist spaces. A variety of new engineering approaches, such as Fabry-Perot interferometers, spectrally resolved detector array (mosaic array), microelectro-mechanical systems, 3D printing, light-emitting diodes, and smartphones, were used in the construction of compact spectral imaging cameras. In bioimaging applications, these compact devices were used for in vivo and ex vivo diagnosis and surgical settings.

Conclusions

Compact and ultracompact spectral imagers are the future of spectral imaging systems. Researchers in the bioimaging fields are building systems that are low-cost, fast in acquisition time, and mobile enough to be handheld.

Smartphone-based single snapshot spatial frequency domain imaging

We report a handheld, smartphone-based spatial frequency domain imaging device. We first examined the linear dynamic range of the smartphone camera sensor. We then calculated optical properties for a series of liquid phantoms with varying concentrations of nigrosin ink and Intralipid, demonstrating separation of absorption and scattering. The device was then tested on a human wrist, where optical properties and hemoglobin-based chromophores were calculated. Finally, we performed an arterial occlusion on a human hand and captured hemodynamics using our device. We hope to lay the foundation for an accessible SFDI device with mass-market appeal designed for dermatological and cosmetic applications.

Diagnostics Using Non-Invasive Technologies in Dermatological Oncology

The growing incidence of skin cancer, with its associated mortality and morbidity, has in recent years led to the developing of new non-invasive technologies, which allow an earlier and more accurate diagnosis. Some of these, such as digital photography, 2D and 3D total-body photography and dermoscopy are now widely used and others, such as reflectance confocal microscopy and optical coherence tomography, are limited to a few academic and referral skin cancer centers because of their cost or the long training period required. Health care professionals involved in the treatment of patients with skin cancer need to know the implications and benefits of new non-invasive technologies for dermatological oncology. In this article we review the characteristics and usability of the main diagnostic imaging methods available today.

A biophotonic platform for quantitative analysis in the spatial, spectral, polarimetric, and goniometric domains

Advanced instrumentation and versatile setups are needed for understanding light interaction with biological targets. Such instruments include (1) microscopes and 3D scanners for detailed spatial analysis, (2) spectral instruments for deducing molecular composition, (3) polarimeters for assessing structural properties, and (4) goniometers probing the scattering phase function of, e.g., tissue slabs. While a large selection of commercial biophotonic instruments and laboratory equipment are available, they are often bulky and expensive. Therefore, they remain inaccessible for secondary education, hobbyists, and research groups in low-income countries. This lack of equipment impedes hands-on proficiency with basic biophotonic principles and the ability to solve local problems with applied physics. We have designed, prototyped, and evaluated the low-cost Biophotonics, Imaging, Optical, Spectral, Polarimetric, Angular, and Compact Equipment (BIOSPACE) for high-quality quantitative analysis. BIOSPACE uses multiplexed light-emitting diodes with emission wavelengths from ultraviolet to near-infrared, captured by a synchronized camera. The angles of the light source, the target, and the polarization filters are automated by low-cost mechanics and a microcomputer. This enables multi-dimensional scatter analysis of centimeter-sized biological targets. We present the construction, calibration, and evaluation of BIOSPACE. The diverse functions of BIOSPACE include small animal spectral imaging, measuring the nanometer thickness of a bark-beetle wing, acquiring the scattering phase function of a blood smear and estimating the anisotropic scattering and the extinction coefficients, and contrasting muscle fibers using polarization. We provide blueprints, component list, and software for replication by enthusiasts and educators to simplify the hands-on investigation of fundamental optical properties in biological samples.

Application of Image Sensors to Detect and Locate Electrical Discharges: A Review

Today, there are many attempts to introduce the Internet of Things (IoT) in high-voltage systems, where partial discharges are a focus of concern since they degrade the insulation. The idea is to detect such discharges at a very early stage so that corrective actions can be taken before major damage is produced. Electronic image sensors are traditionally based on charge-coupled devices (CCDs) and, next, on complementary metal oxide semiconductor (CMOS) devices. This paper performs a review and analysis of state-of-the-art image sensors for detecting, locating, and quantifying partial discharges in insulation systems and, in particular, corona discharges since it is an area with an important potential for expansion due to the important consequences of discharges and the complexity of their detection. The paper also discusses the recent progress, as well as the research needs and the challenges to be faced, in applying image sensors in this area. Although many of the cited research works focused on high-voltage applications, partial discharges can also occur in medium- and low-voltage applications. Thus, the potential applications that could potentially benefit from the introduction of image sensors to detect electrical discharges include power substations, buried power cables, overhead power lines, and automotive applications, among others.

Classification of dog skin diseases using deep learning with images captured from multispectral imaging device

Background

Dog-associated infections are related to more than 70 human diseases. Given that the health diagnosis of a dog requires expertise of the veterinarian, an artificial intelligence model for detecting dog diseases could significantly reduce time and cost required for a diagnosis and efficiently maintain animal health.

Objective

We collected normal and multispectral images to develop classification model of each three dog skin diseases (bacterial dermatosis, fungal infection, and hypersensitivity allergic dermatosis). The single models (normal image- and multispectral image-based) and consensus models were developed used to four CNN model architecture (InceptionNet, ResNet, DenseNet, MobileNet) and select well-performed model.

Results

For single models, such as normal image- or multispectral image-based model, the best accuracies and Matthew’s correlation coefficients (MCCs) for validation data set were 0.80 and 0.64 for bacterial dermatosis, 0.70 and 0.36 for fungal infection, and 0.82 and 0.47 for hypersensitivity allergic dermatosis. For the consensus models, the best accuracies and MCCs for the validation set were 0.89 and 0.76 for the bacterial dermatosis data set, 0.87 and 0.63 for the fungal infection data set, and 0.87 and 0.63 for the hypersensitivity allergic dermatosis data set, respectively, which supported that the consensus models of each disease were more balanced and well-performed.

Conclusions

We developed consensus models for each skin disease for dogs by combining each best model developed with the normal and multispectral images, respectively. Since the normal images could be used to determine areas suspected of lesion of skin disease and additionally the multispectral images could help confirming skin redness of the area, the models achieved higher prediction accuracy with balanced performance between sensitivity and specificity.

Comparative Study of Relative Peripheral Refraction in Children With Different Degrees of Myopia

Purpose

To investigate the difference in the retinal refraction difference value (RDV) using multispectral refractive topography (MRT).

Methods

Ninety myopic participants, who met the enrolment requirements, were examined with an automatic optometer after mydriasis. According to the value of the spherical equivalent (SE), the participants were divided into Emmetropia group (E, +0.5D < SE < −0.5D), Low Myopia (LM, −0.5D < SE ≤ −3D), and Moderate and high Myopia (MM, −3D < SE ≤ −10D). The ocular biological parameters were detected by optical biometrics (Lenstar 900, Switzerland), including axial length (AL), lens thickness (LT), and keratometry (K1, K2). Furthermore, the MRT was used to measure the retinal RDV at three concentric areas, with 15-degree intervals from fovea into 45 degrees (RDV-15, RDV 15–30, and RDV 30–45), and four sectors, including RDV-S (RDV-Superior), RDV-I (RDV-Inferior), RDV-T (RDV-Temporal), and RDV-N (RDV-Nasal).

Results

In the range of RDV-15, there was a significant difference in the value of RDV-15 between Group E (−0.007 ± 0.148) vs. Group LM (−0.212 ± 0.399), and Group E vs. Group MM (0.019 ± 0.106) (P < 0.05); In the range of RDV 15–30, there was a significant difference in the value of RDV 15–30 between Group E (0.114 ± 0.219) vs. Group LM (−0.106 ± 0.332), and Group LM vs. Group MM (0.177 ± 0.209; P < 0.05); In the range of RDV 30–45, there was a significant difference in the value of RDV 30–45 between Group E (0.366 ± 0.339) vs. Group LM (0.461 ± 0.304), and Group E vs. Group MM (0.845 ± 0.415; P < 0.05); In the RDV-S position, there was a significant difference in the value of RDV-S between Group LM (−0.038 ± 0.636) and Group MM (0.526 ± 0.540) (P < 0.05); In the RDV-I position, there was a significant difference in the value of RDV-I between Group E (0.276 ± 0.530) vs. Group LM (0.594 ± 0.513), and Group E vs. Group MM (0.679 ± 0.589; P < 0.05). In the RDV-T position, there was no significant difference in the value of RDV-T among the three groups. In the RDV-N position, there was a significant difference in the value of RDV-N between Group E (0.352 ± 0.623) vs. Group LM (0.464 ± 0.724), and Group E vs. Group MM (1.078 ± 0.627; P < 0.05). The RDV analysis in all directions among the three groups showed a significant difference between RDV-S and RDV-I in Group LM (P < 0.05). Moreover, the correlation analysis showed that SE negatively correlated with AL, RDV 30–45, RDV-S, RDV-I, and RDV-N.

Conclusions

In this study, there was a significant difference in the value of RDV among Group E, Group LM, and Group MM, and the value of RDV in Group MM was the highest on the whole. In the range of RDV 30–45, there was a growing trend with the increase in the degree of myopia among the three groups. Furthermore, the SE negatively correlated with AL, RDV 30–45, RDV-S, RDV-I, and RDV-N.

Mass production-enabled computational spectrometers based on multilayer thin films

Multilayer thin film (MTF) filter arrays for computational spectroscopy are fabricated using stencil lithography. The MTF filter array is a 6 × 6 square grid, and 169 identical arrays are fabricated on a single wafer. A computational spectrometer is formed by attaching the MTF filter array on a complementary metal–oxide–semiconductor (CMOS) image sensor. With a single exposure, 36 unique intensities of incident light are collected. The spectrum of the incident light is recovered using collected intensities and numerical optimization techniques. Varied light sources in the wavelength range of 500 to 849 nm are recovered with a spacing of 1 nm. The reconstructed spectra are a good match with the reference spectra, measured by a grating-based spectrometer. We also demonstrate computational pinhole spectral imaging using the MTF filter array. Adapting a spectral scanning method, we collect 36 monochromatic filtered images and reconstructed 350 monochromatic images in the wavelength range of 500 to 849 nm, with a spacing of 1 nm. These computational spectrometers could be useful for various applications that require compact size, high resolution, and wide working range.

Skin chromophore mapping by smartphone RGB camera under spectral band and spectral line illumination

SIGNIFICANCE

Multispectral imaging enables mapping of chromophore content changes in skin neoplasms, which helps to diagnose a pathology. Different types of light sources can be used for the imaging. Design of laser-based illuminators is more complicated and, consequently, they are more expensive than LED-based illuminators. On the other hand, spectral line illumination has the advantage of less complicated calculations, since only the discrete maximum wavelengths need to be considered. Spectral band and spectral line approaches for multispectral skin diagnostics have not been compared so far. This can help to evaluate the accuracy and effectiveness of both approaches.

AIM

To compare two specific illumination modalities-spectral band and spectral line illumination-from the point of performance for mapping of in vivo skin chromophores.

APPROACH

Three spectral images of the same skin malformations were captured by a smartphone RGB camera with two different add-on illuminators comprising LED emitters and laser emitters, respectively. Five types of benign skin neoplasms were included in our study. Concentrations of skin melanin, oxy- and deoxy-hemoglobin at image pixel groups were calculated using the Beer-Lambert law.

RESULTS

Skin chromophore maps and statistical analysis of mean concentrations' changes in the neoplasms compared to the surrounding skin are presented and discussed. The data of the laser emitters led to significantly higher (∼10 times) increase of the oxy-hemoglobin values in vascular neoplasms and much lower deoxy-hemoglobin values, if compared to the data obtained by the LED emitters.

CONCLUSIONS

Analysis of the obtained chromophore distribution maps and concentration variations in neoplasms led to conclusion that the spectral line illumination approach is more appropriate for this application. Considering only the peak wavelengths of illumination spectral bands leads to essentially different results if compared to those obtained by spectral line illumination and may cause misinterpretations in the clinical assessment of skin neoplasms.

Intelligent smartphone-based multimode imaging otoscope for the mobile diagnosis of otitis media

Otitis media (OM) is one of the most common ear diseases in children and a common reason for outpatient visits to medical doctors in primary care practices. Adhesive OM (AdOM) is recognized as a sequela of OM with effusion (OME) and often requires surgical intervention. OME and AdOM exhibit similar symptoms, and it is difficult to distinguish between them using a conventional otoscope in a primary care unit. The accuracy of the diagnosis is highly dependent on the experience of the examiner. The development of an advanced otoscope with less variation in diagnostic accuracy by the examiner is crucial for a more accurate diagnosis. Thus, we developed an intelligent smartphone-based multimode imaging otoscope for better diagnosis of OM, even in mobile environments. The system offers spectral and autofluorescence imaging of the tympanic membrane using a smartphone attached to the developed multimode imaging module. Moreover, it is capable of intelligent analysis for distinguishing between normal, OME, and AdOM ears using a machine learning algorithm. Using the developed system, we examined the ears of 69 patients to assess their performance for distinguishing between normal, OME, and AdOM ears. In the classification of ear diseases, the multimode system based on machine learning analysis performed better in terms of accuracy and F1 scores than single RGB image analysis, RGB/fluorescence image analysis, and the analysis of spectral image cubes only, respectively. These results demonstrate that the intelligent multimode diagnostic capability of an otoscope would be beneficial for better diagnosis and management of OM.

Biomedical Applications of Translational Optical Imaging: From Molecules to Humans

Light is a powerful investigational tool in biomedicine, at all levels of structural organization. Its multitude of features (intensity, wavelength, polarization, interference, coherence, timing, non-linear absorption, and even interactions with itself) able to create contrast, and thus images that detail the makeup and functioning of the living state can and should be combined for maximum effect, especially if one seeks simultaneously high spatiotemporal resolution and discrimination ability within a living organism. The resulting high relevance should be directed towards a better understanding, detection of abnormalities, and ultimately cogent, precise, and effective intervention. The new optical methods and their combinations needed to address modern surgery in the operating room of the future, and major diseases such as cancer and neurodegeneration are reviewed here, with emphasis on our own work and highlighting selected applications focusing on quantitation, early detection, treatment assessment, and clinical relevance, and more generally matching the quality of the optical detection approach to the complexity of the disease. This should provide guidance for future advanced theranostics, emphasizing a tighter coupling-spatially and temporally-between detection, diagnosis, and treatment, in the hope that technologic sophistication such as that of a Mars rover can be translationally deployed in the clinic, for saving and improving lives.

Novel technology at hand to measure skin hydration by Biodisplay smartphone touch screen panel

Skin hydration is generally evaluated using devices that measure capacitance or conductance. A new technology (Biodisplay) was developed to provide accurate measurements of skin hydration at the contact site. This study aimed to test the reliability of the Biodisplay by comparing its performance results with those of similar devices currently used to objectively assess skin hydration. For each of the 30 participants, skin hydration was measured at each of the defined points on the forearm three times using the Biodisplay and a Hydration probe (HP), an objective measurement device of skin hydration. We also evaluated skin hydration of the arm using both tools after applying moisturizers to evaluate interferences from skin care products. The reliability and reproducibility of each device were analyzed by intraclass correlation coefficients (ICC), and the correlation of the two devices was evaluated by Pearson's correlation coefficients (PCC). The Biodisplay demonstrated moderate-to-excellent reliability (ICC: (0.741-0.980)), but lower reliability than the HP (ICC: (0.953-0.980)). The skin hydration measurements made by the two devices were demonstrated to be significantly correlated, showing moderate correlations (PCC: 0.601). The Biodisplay can provide reasonably reliable and accurate measurements for skin hydration with the strong points of portability and accessibility.

Smartphone-based imaging systems for medical applications: a critical review

SIGNIFICANCE

Smartphones come with an enormous array of functionality and are being more widely utilized with specialized attachments in a range of healthcare applications. A review of key developments and uses, with an assessment of strengths/limitations in various clinical workflows, was completed.

AIM

Our review studies how smartphone-based imaging (SBI) systems are designed and tested for specialized applications in medicine and healthcare. An evaluation of current research studies is used to provide guidelines for improving the impact of these research advances.

APPROACH

First, the established and emerging smartphone capabilities that can be leveraged for biomedical imaging are detailed. Then, methods and materials for fabrication of optical, mechanical, and electrical interface components are summarized. Recent systems were categorized into four groups based on their intended application and clinical workflow: ex vivo diagnostic, in vivo diagnostic, monitoring, and treatment guidance. Lastly, strengths and limitations of current SBI systems within these various applications are discussed.

RESULTS

The native smartphone capabilities for biomedical imaging applications include cameras, touchscreens, networking, computation, 3D sensing, audio, and motion, in addition to commercial wearable peripheral devices. Through user-centered design of custom hardware and software interfaces, these capabilities have the potential to enable portable, easy-to-use, point-of-care biomedical imaging systems. However, due to barriers in programming of custom software and on-board image analysis pipelines, many research prototypes fail to achieve a prospective clinical evaluation as intended. Effective clinical use cases appear to be those in which handheld, noninvasive image guidance is needed and accommodated by the clinical workflow. Handheld systems for in vivo, multispectral, and quantitative fluorescence imaging are a promising development for diagnostic and treatment guidance applications.

CONCLUSIONS

A holistic assessment of SBI systems must include interpretation of their value for intended clinical settings and how their implementations enable better workflow. A set of six guidelines are proposed to evaluate appropriateness of smartphone utilization in terms of clinical context, completeness, compactness, connectivity, cost, and claims. Ongoing work should prioritize realistic clinical assessments with quantitative and qualitative comparison to non-smartphone systems to clearly demonstrate the value of smartphone-based systems. Improved hardware design to accommodate the rapidly changing smartphone ecosystem, creation of open-source image acquisition and analysis pipelines, and adoption of robust calibration techniques to address phone-to-phone variability are three high priority areas to move SBI research forward.

Utilization of Smartphone Depth Mapping Cameras for App-Based Grading of Facial Movement Disorders: Development and Feasibility Study

BACKGROUND

For the classification of facial paresis, various systems of description and evaluation in the form of clinician-graded or software-based scoring systems are available. They serve the purpose of scientific and clinical assessment of the spontaneous course of the disease or monitoring therapeutic interventions. Nevertheless, none have been able to achieve universal acceptance in everyday clinical practice. Hence, a quick and precise tool for assessing the functional status of the facial nerve would be desirable. In this context, the possibilities that the TrueDepth camera of recent iPhone models offer have sparked our interest.

OBJECTIVE

This paper describes the utilization of the iPhone's TrueDepth camera via a specially developed app prototype for quick, objective, and reproducible quantification of facial asymmetries.

METHODS

After conceptual and user interface design, a native app prototype for iOS was programmed that accesses and processes the data of the TrueDepth camera. Using a special algorithm, a new index for the grading of unilateral facial paresis ranging from 0% to 100% was developed. The algorithm was adapted to the well-established Stennert index by weighting the individual facial regions based on functional and cosmetic aspects. Test measurements with healthy subjects using the app were performed in order to prove the reliability of the system.

RESULTS

After the development process, the app prototype had no runtime or buildtime errors and also worked under suboptimal conditions such as different measurement angles, so it met our criteria for a safe and reliable app. The newly defined index expresses the result of the measurements as a generally understandable percentage value for each half of the face. The measurements that correctly rated the facial expressions of healthy individuals as symmetrical in all cases were reproducible and showed no statistically significant intertest variability.

CONCLUSIONS

Based on the experience with the app prototype assessing healthy subjects, the use of the TrueDepth camera should have considerable potential for app-based grading of facial movement disorders. The app and its algorithm, which is based on theoretical considerations, should be evaluated in a prospective clinical study and correlated with common facial scores.

Optical Technologies for the Improvement of Skin Cancer Diagnosis: A Review

The worldwide incidence of skin cancer has risen rapidly in the last decades, becoming one in three cancers nowadays. Currently, a person has a 4% chance of developing melanoma, the most aggressive form of skin cancer, which causes the greatest number of deaths. In the context of increasing incidence and mortality, skin cancer bears a heavy health and economic burden. Nevertheless, the 5-year survival rate for people with skin cancer significantly improves if the disease is detected and treated early. Accordingly, large research efforts have been devoted to achieve early detection and better understanding of the disease, with the aim of reversing the progressive trend of rising incidence and mortality, especially regarding melanoma. This paper reviews a variety of the optical modalities that have been used in the last years in order to improve non-invasive diagnosis of skin cancer, including confocal microscopy, multispectral imaging, three-dimensional topography, optical coherence tomography, polarimetry, self-mixing interferometry, and machine learning algorithms. The basics of each of these technologies together with the most relevant achievements obtained are described, as well as some of the obstacles still to be resolved and milestones to be met.

Solver-informed neural networks for spectrum reconstruction of colloidal quantum dot spectrometers

Recently, the miniature spectrometer based on the optical filter array has received much attention due to its versatility. Among many open challenges, designing efficient and stable algorithms to recover the input spectrum from the raw measurements is the key to success. Of many existing spectrum reconstruction algorithms, regularization-based algorithms have emerged as practical approaches to the spectrum reconstruction problem, but the reconstruction is still challenging due to ill-posedness of the problem. To alleviate this issue, we propose a novel reconstruction method based on a solver-informed neural network (NN). This approach consists of two components: (1) an existing spectrum reconstruction solver to extract the spectral feature from the raw measurements (2) a multilayer perceptron to build a map from the input feature to the spectrum. We investigate the reconstruction performance of the proposed method on a synthetic dataset and a real dataset collected by the colloidal quantum dot (CQD) spectrometer. The results demonstrate the reconstruction accuracy and robustness of the solver-informed NN. In conclusion, the proposed reconstruction method shows excellent potential for spectral recovery of filter-based miniature spectrometers.

Whole Slide Imaging: Technology and Applications

Pathology has benefited from advanced innovation with novel technology to implement a digital solution. Whole slide imaging is a disruptive technology where glass slides are scanned to produce digital images. There have been significant advances in whole slide scanning hardware and software that have allowed for ready access of whole slide images. The digital images, or whole slide images, can be viewed comparable to glass slides in a microscope, as digital files. Whole slide imaging has increased in adoption among pathologists, pathology departments, and scientists for clinical, educational, and research initiatives. Worldwide usage of whole slide imaging has grown significantly. Pathology regulatory organizations (ie, College of American Pathologists) have put forth guidelines for clinical validation, and the US Food and Drug Administration have also approved whole slide imaging for primary diagnosis. This article will review the digital pathology ecosystem and discuss clinical and nonclinical applications of its use.

Smartphone-based spectral imaging otoscope: System development and preliminary study for evaluation of its potential as a mobile diagnostic tool

We develop a novel smartphone-based spectral imaging otoscope for telemedicine and examine its capability for the mobile diagnosis of middle ear diseases. The device was applied to perform spectral imaging and analysis of an ear-mimicking phantom and a normal and abnormal tympanic membrane for evaluation of its potential for the mobile diagnosis. Spectral classified images were obtained via online spectral analysis in a remote server. The phantom experimental results showed that it allowed us to distinguish four different fluids located behind a semitransparent membrane. Also, in the spectral classified images of normal ears (n = 3) and an ear with chronic otitis media (n = 1), the normal and abnormal regions in each ear could be quantitatively distinguished with high contrast. These preliminary results thus suggested that it might have the potentials for providing quantitative information for the mobile diagnosis of various middle ear diseases.

Clinical evaluation of smartphone-based fluorescence imaging for guidance and monitoring of ALA-PDT treatment of early oral cancer

SIGNIFICANCE

India has one of the highest rates of oral cancer incidence in the world, accounting for 30% of reported cancers. In rural areas, a lack of adequate medical infrastructure contributes to unchecked disease progression and dismal mortality rates. Photodynamic therapy (PDT) has emerged as an effective modality with potential for treating early stage disease in resource-limited settings, while photosensitizer fluorescence can be leveraged for treatment guidance.

AIM

Our aim was to assess the capability of a simple smartphone-based device for imaging 5-aminolevulinic acid (ALA)-induced protoporphyrin IX (PpIX) fluorescence for treatment guidance and monitoring as part of an ongoing clinical study evaluating low-cost technology for ALA-based PDT treatment of early oral cancer.

APPROACH

A total of 29 subjects with <2  cm diameter moderately/well-differentiated microinvasive (  <  5  mm depth) oral squamous cell carcinoma lesions (33 lesions total, mean area ∼1.23  cm2) were administered 60  mg  /  kg ALA in oral solution and imaged before and after delivery of 100  J  /  cm2 total light dose to the lesion surface. Smartphone-based fluorescence and white light (WL) images were analyzed and compared with ultrasound (US) imaging of the same lesions.

RESULTS

We present a comparative analysis of pre- and post-treatment fluorescence, WL, and US images of oral lesions. There was no significant difference in the distribution of lesion widths measured by fluorescence and US (mean widths of 14.5 and 15.3 mm, respectively) and linear regression shows good agreement (R2  =  0.91). In general, PpIX fluorescence images obtained prior to therapeutic light delivery are able to resolve lesion margins while dramatic photobleaching (∼42  %  ) is visible post-treatment. Segmentation of the photobleached area confirms the boundaries of the irradiated zone.

CONCLUSIONS

A simple smartphone-based approach for imaging oral lesions is shown to agree in most cases with US, suggesting that this approach may be a useful tool to aid in PDT treatment guidance and monitoring photobleaching as part of a low-cost platform for intraoral PDT.

O. Microfluidic Point-of-Care Devices: New Trends and Future Prospects for eHealth Diagnostics

Point-of-care (PoC) diagnostics is promising for early detection of a number of diseases, including cancer, diabetes, and cardiovascular diseases, in addition to serving for monitoring health conditions. To be efficient and cost-effective, portable PoC devices are made with microfluidic technologies, with which laboratory analysis can be made with small-volume samples. Recent years have witnessed considerable progress in this area with "epidermal electronics", including miniaturized wearable diagnosis devices. These wearable devices allow for continuous real-time transmission of biological data to the Internet for further processing and transformation into clinical knowledge. Other approaches include bluetooth and WiFi technology for data transmission from portable (non-wearable) diagnosis devices to cellphones or computers, and then to the Internet for communication with centralized healthcare structures. There are, however, considerable challenges to be faced before PoC devices become routine in the clinical practice. For instance, the implementation of this technology requires integration of detection components with other fluid regulatory elements at the microscale, where fluid-flow properties become increasingly controlled by viscous forces rather than inertial forces. Another challenge is to develop new materials for environmentally friendly, cheap, and portable microfluidic devices. In this review paper, we first revisit the progress made in the last few years and discuss trends and strategies for the fabrication of microfluidic devices. Then, we discuss the challenges in lab-on-a-chip biosensing devices, including colorimetric sensors coupled to smartphones, plasmonic sensors, and electronic tongues. The latter ones use statistical and big data analysis for proper classification. The increasing use of big data and artificial intelligence methods is then commented upon in the context of wearable and handled biosensing platforms for the Internet of things and futuristic healthcare systems.

Applications of hyperspectral imaging in the detection and diagnosis of solid tumors

Hyperspectral imaging (HSI) is an emerging new technology in solid tumor diagnosis and detection. It incorporates traditional imaging and spectroscopy together to obtain both spatial and spectral information from tissues simultaneously in a non-invasive manner. This imaging modality is based on the principle that different tissues inherit different spectral reflectance responses that present as unique spectral fingerprints. HSI captures those composition-specific fingerprints to identify cancerous and normal tissues. It becomes a promising tool for performing tumor diagnosis and detection from the label-free histopathological examination to real-time intraoperative assistance. This review introduces the basic principles of HSI and summarizes its methodology and recent advances in solid tumor detection. In particular, the advantages of HSI applied to solid tumors are highlighted to show its potential for clinical use.

Hyperspectral imaging enabled by an unmodified smartphone for analyzing skin morphological features and monitoring hemodynamics

We propose a novel method and system that utilizes a popular smartphone to realize hyperspectral imaging for analyzing skin morphological features and monitoring hemodynamics. The imaging system works based on a built-in RGB camera and flashlight on the smartphone. We apply Wiener estimation to transform the acquired RGB-mode images into "pseudo"-hyperspectral images with 16 wavebands, covering a visible range from 470nm to 620nm. The processing method uses weighted subtractions between wavebands to extract absorption information caused by specific chromophores within skin tissue, mainly including hemoglobin and melanin. Based on the extracted absorption information of hemoglobin, we conduct real-time monitoring experiments in the skin to measure heart rate and to observe skin activities during a vascular occlusion event. Compared with expensive hyperspectral imaging systems, the smartphone-based system delivers similar results but with very-high imaging resolution. Besides, it is easy to operate, very cost-effective and has a wider customer base. The use of an unmodified smartphone to realize hyperspectral imaging promises a possibility to bring a hyperspectral analysis of skin out from laboratory and clinical wards to daily life, which may also impact on healthcare in low resource settings and rural areas.

Compressive Sensing Spectroscopy Using a Residual Convolutional Neural Network

Compressive sensing (CS) spectroscopy is well known for developing a compact spectrometer which consists of two parts: compressively measuring an input spectrum and recovering the spectrum using reconstruction techniques. Our goal here is to propose a novel residual convolutional neural network (ResCNN) for reconstructing the spectrum from the compressed measurements. The proposed ResCNN comprises learnable layers and a residual connection between the input and the output of these learnable layers. The ResCNN is trained using both synthetic and measured spectral datasets. The results demonstrate that ResCNN shows better spectral recovery performance in terms of average root mean squared errors (RMSEs) and peak signal to noise ratios (PSNRs) than existing approaches such as the sparse recovery methods and the spectral recovery using CNN. Unlike sparse recovery methods, ResCNN does not require a priori knowledge of a sparsifying basis nor prior information on the spectral features of the dataset. Moreover, ResCNN produces stable reconstructions under noisy conditions. Finally, ResCNN is converged faster than CNN.

Multispectral time-of-flight imaging using light-emitting diodes

Multispectral and 3-D imaging are useful for a wide variety of applications, adding valuable spectral and depth information for image analysis. Single-photon avalanche diode (SPAD) based imaging systems provide photon time-of-arrival information, and can be used for imaging with time-correlated single photon counting techniques. Here we demonstrate an LED based synchronised illumination system, where temporally structured light can be used to relate time-of-arrival to specific wavelengths, thus recovering reflectance information. Cross-correlation of the received multi-peak histogram with a reference measurement yields a time delay, allowing depth information to be determined with cm-scale resolution despite the long sequence of relatively wide (∼10 ns) pulses. Using commercial LEDs and a SPAD imaging array, multispectral 3-D imaging is demonstrated across 9 visible wavelength bands.

Analysis of skin morphological features and real-time monitoring using snapshot hyperspectral imaging

We propose a snapshot hyperspectral imaging system and methods for skin morphological feature analysis and real-time monitoring of skin activities. The analysis method includes a strategy using weighted subtractions between sub-channel images to extract absorption information due to specific chromophores within skin tissue, for example hemoglobin and melanin. Based on morphological analysis results, we carry out real-time monitoring of the skin features to verify the ability of this method to provide temporal responses of the skin tissue activities, which is experimentally shown to be useful in the measurement of heartrate, monitoring of the tissue recovery after a body exercise, and studying of the tissue response due to a vascular occlusion. Compared to conventional multispectral imaging system, the proposed system improves the device simplicity and is immune to motion artifacts. Coupled with the extraction algorithms, the hyperspectral imaging promises a robust skin assessment tool with abilities for qualitative visualization and potentially quantitative analysis of skin features, useful in the applications of cosmetics and clinical dermatology.

Fast visible and extended near-infrared multispectral fundus camera

We present a multispectral fundus camera that performs fast imaging of the ocular posterior pole in the visible and near-infrared (400 to 1300 nm) wavelengths through 15 spectral bands, using a flashlight source made of light-emitting diodes, and CMOS and InGaAs cameras. We investigate the potential of this system for visualizing occult and overlapping structures of the retina in the unexplored wavelength range beyond 900 nm, in which radiation can penetrate deeper into the tissue. Reflectance values at each pixel are also retrieved from the acquired images in the analyzed spectral range. The available spectroscopic information and the visualization of retinal structures, specifically the choroidal vasculature and drusen-induced retinal pigment epithelium degeneration, which are hardly visible in conventional color fundus images, underline the clinical potential of this system as a new tool for ophthalmic diagnosis.

Multimodal Device for Real-Time Monitoring of Skin Oxygen Saturation and Microcirculation Function

The present study introduces a recently developed compact hybrid device for real-time monitoring of skin oxygen saturation and temperature distribution. The prototype involves a snapshot hyperspectral camera, multi-wavelength illuminator, thermal camera, and built-in computer with custom-developed software. To validate this device in-vivo we performed upper arm vascular occlusion on eight healthy volunteers. Palm skin oxygen saturation maps were analyzed in real-time using k-means segmentation algorithm and two-layer optical diffuse model. The prototype system demonstrated a satisfying performance of skin hyperspectral measurements in the spectral range of 507–625 nm. The results confirmed the reliability of the proposed system for in-vivo assessment of skin hemoglobin saturation with oxygen and microcirculation.

Hyperspectral imaging of human skin aided by artificial neural networks

We developed a compact, hand-held hyperspectral imaging system for 2D neural network-based visualization of skin chromophores and blood oxygenation. State-of-the-art micro-optic multichannel matrix sensor combined with the tunable Fabry-Perot micro interferometer enables a portable diagnostic device sensitive to the changes of the oxygen saturation as well as the variations of blood volume fraction of human skin. Generalized object-oriented Monte Carlo model is used extensively for the training of an artificial neural network utilized for the hyperspectral image processing. In addition, the results are verified and validated via actual experiments with tissue phantoms and human skin in vivo. The proposed approach enables a tool combining both the speed of an artificial neural network processing and the accuracy and flexibility of advanced Monte Carlo modeling. Finally, the results of the feasibility studies and the experimental tests on biotissue phantoms and healthy volunteers are presented.

Smartphone Sensors for Health Monitoring and Diagnosis

Over the past few decades, we have witnessed a dramatic rise in life expectancy owing to significant advances in medical science and technology, medicine as well as increased awareness about nutrition, education, and environmental and personal hygiene. Consequently, the elderly population in many countries are expected to rise rapidly in the coming years. A rapidly rising elderly demographics is expected to adversely affect the socioeconomic systems of many nations in terms of costs associated with their healthcare and wellbeing. In addition, diseases related to the cardiovascular system, eye, respiratory system, skin and mental health are widespread globally. However, most of these diseases can be avoided and/or properly managed through continuous monitoring. In order to enable continuous health monitoring as well as to serve growing healthcare needs; affordable, non-invasive and easy-to-use healthcare solutions are critical. The ever-increasing penetration of smartphones, coupled with embedded sensors and modern communication technologies, make it an attractive technology for enabling continuous and remote monitoring of an individual's health and wellbeing with negligible additional costs. In this paper, we present a comprehensive review of the state-of-the-art research and developments in smartphone-sensor based healthcare technologies. A discussion on regulatory policies for medical devices and their implications in smartphone-based healthcare systems is presented. Finally, some future research perspectives and concerns regarding smartphone-based healthcare systems are described.

Multimodal endoscopic system based on multispectral and photometric stereo imaging and analysis

We propose a multimodal endoscopic system based on white light (WL), multispectral (MS), and photometric stereo (PS) imaging for the examination of colorectal cancer (CRC). Recently, the enhancement of the diagnostic accuracy of CRC colonoscopy has been reported; however, tumor diagnosis for a variety of lesion types remains challenging using current endoscopy. In this study, we demonstrate that our developed system can simultaneously discriminate tumor distributions and provide three-dimensional (3D) morphological information about the colon surface using the WL, MS, and PS imaging modalities. The results demonstrate that the proposed system has considerable potential for CRC diagnosis.

Smartphone-based multispectral imaging and machine-learning based analysis for discrimination between seborrheic dermatitis and psoriasis on the scalp

For appropriate treatment, accurate discrimination between seborrheic dermatitis and psoriasis in a timely manner is crucial to avoid complications. However, when they occur on the scalp, differential diagnosis can be challenging using conventional dermascopes. Thus, we employed smartphone-based multispectral imaging and analysis to discriminate between them with high accuracy. A smartphone-based multispectral imaging system, suited for scalp disease diagnosis, was redesigned. We compared the outcomes obtained using machine learning-based and conventional spectral classification methods to achieve better discrimination. The results demonstrated that smartphone-based multispectral imaging and analysis has great potential for discriminating between these diseases.

Experimental Demonstration of Remote and Compact Imaging Spectrometer Based on Mobile Devices

Imaging spectrometers show great potential for environmental and biomedical sensing applications. Selfie sticks, which are tools used to take photographs or videos, have gained global popularity in recent years. Few people have connected these two objects, and few people have researched the application of imaging spectrometers to perform scientific monitoring in point-of-use scenarios. In this paper, we develop a compact imaging spectrometer (35 g in weight, 18 mm in diameter, and 72 mm in length) that can be equipped on a motorized selfie stick to perform remote sensing. We applied this system to perform environmental and facial remote sensing via motorized scanning. The absorption of chlorophyll and hemoglobin can be found in the reflectance spectra, indicating that our system can be used in urban greening monitoring and point-of-care testing. In addition, this compact imaging spectrometer was also easily attached to an underwater dome port and a quad-rotor unmanned aerial vehicle to perform underwater and airborne spectral detection. Our system offers a route toward mobile imaging spectrometers used in daily life.

Visible and Extended Near-Infrared Multispectral Imaging for Skin Cancer Diagnosis

With the goal of diagnosing skin cancer in an early and noninvasive way, an extended near infrared multispectral imaging system based on an InGaAs sensor with sensitivity from 995 nm to 1613 nm was built to evaluate deeper skin layers thanks to the higher penetration of photons at these wavelengths. The outcomes of this device were combined with those of a previously developed multispectral system that works in the visible and near infrared range (414 nm⁻995 nm). Both provide spectral and spatial information from skin lesions. A classification method to discriminate between melanomas and nevi was developed based on the analysis of first-order statistics descriptors, principal component analysis, and support vector machine tools. The system provided a sensitivity of 78.6% and a specificity of 84.6%, the latter one being improved with respect to that offered by silicon sensors.

Pencil-like imaging spectrometer for bio-samples sensing

Spectrally-resolved imaging techniques are becoming central to the investigation of bio-samples. In this paper, we demonstrate the use of a WIFI-camera as a detection module to assemble a pencil-like imaging spectrometer, which weighs only 140 g and has a size of 3.1 cm in diameter and 15.5 cm in length. The spectrometer is standalone, and works wirelessly. A smartphone or network computer can serve as the data receiver and processor. The wavelength resolution of the spectrometer is about 17 nm, providing repeatable measurements of spatial two-dimensional images at various wavelengths for various bio-samples, including bananas, meat, and human hands. The detected spectral range is 400 nm - 675 nm and a white LED array lamp is selected as the light source. Based on the detected spectra, we can monitor the impacts of chlorophyll, myoglobin, and hemoglobin on bananas, pork, and human hands, respectively. For human hand scanning, a 3D spectral image data cube, which exhibits excellent signal to background noise ratio, can be obtained within 16 sec. We envisage that the adaptation of imaging spectrometer devices to the widely-accepted smartphone technology will help to carry out on-site studies in various applications. Besides, our pencil-like imaging spectrometer is cost-effective (<$300) and easy to assemble. This portable imaging spectrometer can facilitate the collection of large amounts of spectral image data. With the help of machine learning, we can realize object recognition based on spectral classification in the future.

Smartphone snapshot mapping of skin chromophores under triple-wavelength laser illumination

Chromophore distribution maps are useful tools for skin malformation severity assessment and for monitoring of skin recovery after burns, surgeries, and other interactions. The chromophore maps can be obtained by processing several spectral images of skin, e.g., captured by hyperspectral or multispectral cameras during seconds or even minutes. To avoid motion artifacts and simplify the procedure, a single-snapshot technique for mapping melanin, oxyhemoglobin, and deoxyhemoglobin of in-vivo skin by a smartphone under simultaneous three-wavelength (448–532–659 nm) laser illumination is proposed and examined. Three monochromatic spectral images related to the illumination wavelengths were extracted from the smartphone camera RGB image data set with respect to crosstalk between the RGB detection bands. Spectral images were further processed accordingly to Beer’s law in a three chromophore approximation. Photon absorption path lengths in skin at the exploited wavelengths were estimated by means of Monte Carlo simulations. The technique was validated clinically on three kinds of skin lesions: nevi, hemangiomas, and seborrheic keratosis. Design of the developed add-on laser illumination system, image-processing details, and the results of clinical measurements are presented and discussed.

Photoaging Mobile Apps as a Novel Opportunity for Melanoma Prevention: Pilot Study

Background

Around 90% of melanomas are caused by ultraviolet (UV) exposure and are therefore eminently preventable. Unhealthy tanning behavior is mostly initiated in early adolescence, often with the belief that it increases attractiveness; the problems related to skin atrophy and malignant melanoma are too far in the future to fathom. Photoaging desktop programs, in which an image is altered to predict future appearance, have been successful in positively influencing behavior in adiposity or tobacco prevention settings.

Objective

To develop and test a photoaging app designed for melanoma prevention.

Methods

We harnessed the widespread availability of mobile phones and adolescents’ interest in appearance to develop a free mobile app called Sunface. This app has the user take a self-portrait (ie, a selfie), and then photoages the image based on Fitzpatrick skin type and individual UV protection behavior. Afterward, the app explains the visual results and aims at increasing self-competence on skin cancer prevention by providing guideline recommendations on sun protection and the ABCDE rule for melanoma self-detection. The underlying aging algorithms are based on publications showing UV-induced skin damage by outdoor as well as indoor tanning. To get a first impression on how well the app would be received in a young target group, we included a total sample of 25 students in our cross-sectional pilot study with a median age of 22 (range 19-25) years of both sexes (11/25, 44% female; 14/25, 56% male) attending the University of Essen in Germany.

Results

The majority of enrolled students stated that they would download the app (22/25, 88%), that the intervention had the potential to motivate them to use sun protection (23/25, 92%) and that they thought such an app could change their perceptions that tanning makes you attractive (19/25, 76%). Only a minority of students disagreed or fully disagreed that they would download such an app (2/25, 8%) or that such an app could change their perceptions on tanning and attractiveness (4/25, 16%).

Conclusions

Based on previous studies and the initial study results presented here, it is reasonable to speculate that the app may induce behavioral change in the target population. Further work is required to implement and examine the effectiveness of app-based photoaging interventions within risk groups from various cultural backgrounds.