Smartphone-based multispectral imaging and machine-learning based analysis for discrimination between seborrheic dermatitis and psoriasis on the scalp

Deep learning-assisted multispectral imaging for early screening of skin diseases

INTRODUCTION

Melanocytic nevi (MN), warts, seborrheic keratoses (SK), and psoriasis are four common types of skin surface lesions that typically require dermatoscopic examination for definitive diagnosis in clinical dermatology settings. This process is labor-intensive and resource-consuming. Traditional methods for diagnosing skin lesions rely heavily on the subjective judgment of dermatologists, leading to issues in diagnostic accuracy and prolonged detection times.

OBJECTIVES

This study aims to introduce a multispectral imaging (MSI)-based method for the early screening and detection of skin surface lesions. By capturing image data at multiple wavelengths, MSI can detect subtle spectral variations in tissues, significantly enhancing the differentiation of various skin conditions.

METHODS

The proposed method utilizes a pixel-level mosaic imaging spectrometer to capture multispectral images of lesions, followed by reflectance calibration and standardization. Regions of interest were manually extracted, and the spectral data were subsequently exported for analysis. An improved one-dimensional convolutional neural network is then employed to train and classify the data.

RESULTS

The new method achieves an accuracy of 96.82 % on the test set, demonstrating its efficacy.

CONCLUSION

This multispectral imaging approach provides a non-contact and non-invasive method for early screening, effectively addressing the subjective identification of lesions by dermatologists and the prolonged detection times associated with conventional methods. It offers enhanced diagnostic accuracy for a variety of skin lesions, suggesting new avenues for dermatological diagnostics.

AutoMitoNetwork: Software for analyzing mitochondrial networks in autofluorescence images to enable label-free cell classification

High-resolution mitochondria imaging in combination with image analysis tools have significantly advanced our understanding of cellular function in health and disease. However, most image analysis tools for mitochondrial studies have been designed to work with fluorescently labeled images only. Additionally, efforts to integrate features describing mitochondrial networks with machine learning techniques for the differentiation of cell types have been limited. Herein, we present AutoMitoNetwork software for image-based assessment of mitochondrial networks in label-free autofluorescence images using a range of interpretable morphological, intensity, and textural features. To demonstrate its utility, we characterized unstained mitochondrial networks in healthy retinal cells and in retinal cells exposed to two types of treatments: rotenone, which directly inhibited mitochondrial respiration and ATP production, and iodoacetic acid, which had a milder impact on mitochondrial networks via the inhibition of anaerobic glycolysis. For both cases, our multi-dimensional feature analysis combined with a support vector machine classifier distinguished between healthy cells and those treated with rotenone or iodoacetic acid. Subtle changes in morphological features were measured including increased fragmentation in the treated retinal cells, pointing to an association with metabolic mechanisms. AutoMitoNetwork opens new options for image-based machine learning in label-free imaging, diagnostics, and mitochondrial disease drug development.

Single-cell analysis and machine learning identify psoriasis-associated CD8+ T cells serve as biomarker for psoriasis

Psoriasis is a chronic inflammatory skin disease, the etiology of which has not been fully elucidated, in which CD8+ T cells play an important role in the pathogenesis of psoriasis. However, there is a lack of in-depth studies on the molecular characterization of different CD8+ T cell subtypes and their role in the pathogenesis of psoriasis. This study aims to further expound the pathogenesy of psoriasis at the single-cell level and to explore new ideas for clinical diagnosis and new therapeutic targets. Our study identified a unique subpopulation of CD8+ T cells highly infiltrated in psoriasis lesions. Subsequently, we analyzed the hub genes of the psoriasis-specific CD8+ T cell subpopulation using hdWGCNA and constructed a machine-learning prediction model, which demonstrated good efficacy. The model interpretation showed the influence of each independent variable in the model decision. Finally, we deployed the machine learning model to an online website to facilitate its clinical transformation.

Artificial Intelligence: A Snapshot of Its Application in Chronic Inflammatory and Autoimmune Skin Diseases

Immuno-correlated dermatological pathologies refer to skin disorders that are closely associated with immune system dysfunction or abnormal immune responses. Advancements in the field of artificial intelligence (AI) have shown promise in enhancing the diagnosis, management, and assessment of immuno-correlated dermatological pathologies. This intersection of dermatology and immunology plays a pivotal role in comprehending and addressing complex skin disorders with immune system involvement. The paper explores the knowledge known so far and the evolution and achievements of AI in diagnosis; discusses segmentation and the classification of medical images; and reviews existing challenges, in immunological-related skin diseases. From our review, the role of AI has emerged, especially in the analysis of images for both diagnostic and severity assessment purposes. Furthermore, the possibility of predicting patients' response to therapies is emerging, in order to create tailored therapies.

Parameter-based transfer learning for severity classification of atopic dermatitis using hyperspectral imaging

BACKGROUND/PURPOSE

Because atopic dermatitis (AD) is a chronic inflammatory skin condition that causes structural changes, there is a growing need for noninvasive research methods to evaluate this condition. Hyperspectral imaging (HSI) captures skin structure features by exploiting light wavelength variations in penetration depth. In this study, parameter-based transfer learning was deployed to classify the severity of AD using HSI. Therefore, we aimed to obtain an optimal combination of classification results from the four models after constructing different source- and target-domain datasets.

METHODS

We designated psoriasis, skin cancer, eczema, and AD datasets as the source datasets, and the set of images acquired via hyperspectral camera as the target dataset for wavelength-specific AD classification. We compared the severity classification performances of 96 combinations of sources, models, and targets.

RESULTS

The highest classification performance of 83% was achieved when ResNet50 was trained on the augmented psoriasis dataset as the source, with the resulting parameters used to train the model on the target Near-infrared radiation (NIR) dataset. The second highest classification accuracy of 81% was achieved when ResNet50 was trained on the unaugmented psoriasis dataset as the source, with the resulting parameters used to train the model on the target R dataset. ResNet50 demonstrated potential as a generalized model for both the source and target data, also confirming that the psoriasis dataset is an effective training resource.

CONCLUSION

The present study not only demonstrates the feasibility of the severity classification of AD based on hyperspectral images, but also showcases combinations and research scalability for domain exploration.

Artificial intelligence in psoriasis: Where we are and where we are going

Artificial intelligence (AI) is a field of computer science that involves the development of programs designed to replicate human cognitive processes and the analysis of complex data. In dermatology, which is predominantly a visual-based diagnostic field, AI has become increasingly important in improving professional processes, particularly in the diagnosis of psoriasis. In this review, we summarized current AI applications in psoriasis: (i) diagnosis, including identification, classification, lesion segmentation, lesion severity and area scoring; (ii) treatment, including prediction treatment efficiency and prediction candidate drugs; (iii) management, including e-health and preventive medicine. Key challenges and future aspects of AI in psoriasis were also discussed, in hope of providing potential directions for future studies.

Spectral Filter Selection Based on Human Color Vision for Spectral Reflectance Recovery

Spectral filters are an important part of a multispectral acquisition system, and the selection of suitable filters can improve the spectral recovery accuracy. In this paper, we propose an efficient human color vision-based method to recover spectral reflectance by the optimal filter selection. The original sensitivity curves of the filters are weighted using the LMS cone response function. The area enclosed by the weighted filter spectral sensitivity curves and the coordinate axis is calculated. The area is subtracted before weighting, and the three filters with the smallest reduction in the weighted area are used as the initial filters. The initial filters selected in this way are closest to the sensitivity function of the human visual system. After the three initial filters are combined with the remaining filters one by one, the filter sets are substituted into the spectral recovery model. The best filter sets under L-weighting, M-weighting, and S-weighting are selected according to the custom error score ranking. Finally, the optimal filter set is selected from the three optimal filter sets according to the custom error score ranking. The experimental results demonstrate that the proposed method outperforms existing methods in spectral and colorimetric accuracy, which also has good stability and robustness. This work will be useful for optimizing the spectral sensitivity of a multispectral acquisition system.

Compact and ultracompact spectral imagers: technology and applications in biomedical imaging

Significance

Spectral imaging, which includes hyperspectral and multispectral imaging, can provide images in numerous wavelength bands within and beyond the visible light spectrum. Emerging technologies that enable compact, portable spectral imaging cameras can facilitate new applications in biomedical imaging.

Aim

With this review paper, researchers will (1) understand the technological trends of upcoming spectral cameras, (2) understand new specific applications that portable spectral imaging unlocked, and (3) evaluate proper spectral imaging systems for their specific applications.

Approach

We performed a comprehensive literature review in three databases (Scopus, PubMed, and Web of Science). We included only fully realized systems with definable dimensions. To best accommodate many different definitions of "compact," we included a table of dimensions and weights for systems that met our definition.

Results

There is a wide variety of contributions from industry, academic, and hobbyist spaces. A variety of new engineering approaches, such as Fabry-Perot interferometers, spectrally resolved detector array (mosaic array), microelectro-mechanical systems, 3D printing, light-emitting diodes, and smartphones, were used in the construction of compact spectral imaging cameras. In bioimaging applications, these compact devices were used for in vivo and ex vivo diagnosis and surgical settings.

Conclusions

Compact and ultracompact spectral imagers are the future of spectral imaging systems. Researchers in the bioimaging fields are building systems that are low-cost, fast in acquisition time, and mobile enough to be handheld.

Therapeutic application of machine learning in psoriasis: A Prisma systematic review

Dermatology, being a predominantly visual-based diagnostic field, has found itself to be at the epitome of artificial intelligence (AI)-based advances. Machine learning (ML), a subset of AI, goes a step further by recognizing patterns from data and teaches machines to automatically learn tasks. Although artificial intelligence in dermatology is mostly developed in melanoma and skin cancer diagnosis, advances in AI and ML have gone far ahead and found its application in ulcer assessment, psoriasis, atopic dermatitis, onychomycosis, etc. This article is focused on the application of ML in the therapeutic aspect of psoriasis.

An improved spectral estimation method based on color perception features of mobile phone camera

We use the mobile phone camera as a new spectral imaging device to obtain raw responses of samples for spectral estimation and propose an improved sequential adaptive weighted spectral estimation method. First, we verify the linearity of the raw response of the cell phone camera and investigate its feasibility for spectral estimation experiments. Then, we propose a sequential adaptive spectral estimation method based on the CIE1976 L*a*b* (CIELAB) uniform color space color perception feature. The first stage of the method is to weight the training samples and perform the first spectral reflectance estimation by considering the Lab color space color perception features differences between samples, and the second stage is to adaptively select the locally optimal training samples and weight them by the first estimated root mean square error (RMSE), and perform the second spectral reconstruction. The novelty of the method is to weight the samples by using the sample in CIELAB uniform color space perception features to more accurately characterize the color difference. By comparing with several existing methods, the results show that the method has the best performance in both spectral error and chromaticity error. Finally, we apply this weighting strategy based on the CIELAB color space color perception feature to the existing method, and the spectral estimation performance is greatly improved compared with that before the application, which proves the effectiveness of this weighting method.

Classification of dog skin diseases using deep learning with images captured from multispectral imaging device

Background

Dog-associated infections are related to more than 70 human diseases. Given that the health diagnosis of a dog requires expertise of the veterinarian, an artificial intelligence model for detecting dog diseases could significantly reduce time and cost required for a diagnosis and efficiently maintain animal health.

Objective

We collected normal and multispectral images to develop classification model of each three dog skin diseases (bacterial dermatosis, fungal infection, and hypersensitivity allergic dermatosis). The single models (normal image- and multispectral image-based) and consensus models were developed used to four CNN model architecture (InceptionNet, ResNet, DenseNet, MobileNet) and select well-performed model.

Results

For single models, such as normal image- or multispectral image-based model, the best accuracies and Matthew’s correlation coefficients (MCCs) for validation data set were 0.80 and 0.64 for bacterial dermatosis, 0.70 and 0.36 for fungal infection, and 0.82 and 0.47 for hypersensitivity allergic dermatosis. For the consensus models, the best accuracies and MCCs for the validation set were 0.89 and 0.76 for the bacterial dermatosis data set, 0.87 and 0.63 for the fungal infection data set, and 0.87 and 0.63 for the hypersensitivity allergic dermatosis data set, respectively, which supported that the consensus models of each disease were more balanced and well-performed.

Conclusions

We developed consensus models for each skin disease for dogs by combining each best model developed with the normal and multispectral images, respectively. Since the normal images could be used to determine areas suspected of lesion of skin disease and additionally the multispectral images could help confirming skin redness of the area, the models achieved higher prediction accuracy with balanced performance between sensitivity and specificity.

Improved method for spectral reflectance estimation and application to mobile phone cameras

We propose an improved method for estimating surface-spectral reflectance from the image data acquired by an RGB digital camera. We suppose a multispectral image acquisition system in the visible range, where a camera captures multiple images for the scene of an object under multiple light sources. First, the observed image data are described using the camera spectral sensitivities, the surface-spectral reflectance, the illuminant spectral power distributions, an additive noise term, and a gain parameter. Then, the optimal reflectance estimate is determined to minimize the mean-square error between the estimate and the original surface-spectral reflectance. We attempt to further improve the estimation accuracy and develop a novel linear estimator in a more general form than the Wiener estimator. Furthermore, we calibrate the imaging system using a reference standard sample. Finally, experiments are performed to validate the proposed method for estimating the surface-spectral reflectance using different mobile phone cameras.

Skin chromophore mapping by smartphone RGB camera under spectral band and spectral line illumination

SIGNIFICANCE

Multispectral imaging enables mapping of chromophore content changes in skin neoplasms, which helps to diagnose a pathology. Different types of light sources can be used for the imaging. Design of laser-based illuminators is more complicated and, consequently, they are more expensive than LED-based illuminators. On the other hand, spectral line illumination has the advantage of less complicated calculations, since only the discrete maximum wavelengths need to be considered. Spectral band and spectral line approaches for multispectral skin diagnostics have not been compared so far. This can help to evaluate the accuracy and effectiveness of both approaches.

AIM

To compare two specific illumination modalities-spectral band and spectral line illumination-from the point of performance for mapping of in vivo skin chromophores.

APPROACH

Three spectral images of the same skin malformations were captured by a smartphone RGB camera with two different add-on illuminators comprising LED emitters and laser emitters, respectively. Five types of benign skin neoplasms were included in our study. Concentrations of skin melanin, oxy- and deoxy-hemoglobin at image pixel groups were calculated using the Beer-Lambert law.

RESULTS

Skin chromophore maps and statistical analysis of mean concentrations' changes in the neoplasms compared to the surrounding skin are presented and discussed. The data of the laser emitters led to significantly higher (∼10 times) increase of the oxy-hemoglobin values in vascular neoplasms and much lower deoxy-hemoglobin values, if compared to the data obtained by the LED emitters.

CONCLUSIONS

Analysis of the obtained chromophore distribution maps and concentration variations in neoplasms led to conclusion that the spectral line illumination approach is more appropriate for this application. Considering only the peak wavelengths of illumination spectral bands leads to essentially different results if compared to those obtained by spectral line illumination and may cause misinterpretations in the clinical assessment of skin neoplasms.

Intelligent smartphone-based multimode imaging otoscope for the mobile diagnosis of otitis media

Otitis media (OM) is one of the most common ear diseases in children and a common reason for outpatient visits to medical doctors in primary care practices. Adhesive OM (AdOM) is recognized as a sequela of OM with effusion (OME) and often requires surgical intervention. OME and AdOM exhibit similar symptoms, and it is difficult to distinguish between them using a conventional otoscope in a primary care unit. The accuracy of the diagnosis is highly dependent on the experience of the examiner. The development of an advanced otoscope with less variation in diagnostic accuracy by the examiner is crucial for a more accurate diagnosis. Thus, we developed an intelligent smartphone-based multimode imaging otoscope for better diagnosis of OM, even in mobile environments. The system offers spectral and autofluorescence imaging of the tympanic membrane using a smartphone attached to the developed multimode imaging module. Moreover, it is capable of intelligent analysis for distinguishing between normal, OME, and AdOM ears using a machine learning algorithm. Using the developed system, we examined the ears of 69 patients to assess their performance for distinguishing between normal, OME, and AdOM ears. In the classification of ear diseases, the multimode system based on machine learning analysis performed better in terms of accuracy and F1 scores than single RGB image analysis, RGB/fluorescence image analysis, and the analysis of spectral image cubes only, respectively. These results demonstrate that the intelligent multimode diagnostic capability of an otoscope would be beneficial for better diagnosis and management of OM.

Low-Cost Hyperspectral Imaging with A Smartphone

Recent advances in smartphone technologies have opened the door to the development of accessible, highly portable sensing tools capable of accurate and reliable data collection in a range of environmental settings. In this article, we introduce a low-cost smartphone-based hyperspectral imaging system that can convert a standard smartphone camera into a visible wavelength hyperspectral sensor for ca. £100. To the best of our knowledge, this represents the first smartphone capable of hyperspectral data collection without the need for extensive post processing. The Hyperspectral Smartphone’s abilities are tested in a variety of environmental applications and its capabilities directly compared to the laboratory-based analogue from our previous research, as well as the wider existing literature. The Hyperspectral Smartphone is capable of accurate, laboratory- and field-based hyperspectral data collection, demonstrating the significant promise of both this device and smartphone-based hyperspectral imaging as a whole.

Smartphone-based imaging systems for medical applications: a critical review

SIGNIFICANCE

Smartphones come with an enormous array of functionality and are being more widely utilized with specialized attachments in a range of healthcare applications. A review of key developments and uses, with an assessment of strengths/limitations in various clinical workflows, was completed.

AIM

Our review studies how smartphone-based imaging (SBI) systems are designed and tested for specialized applications in medicine and healthcare. An evaluation of current research studies is used to provide guidelines for improving the impact of these research advances.

APPROACH

First, the established and emerging smartphone capabilities that can be leveraged for biomedical imaging are detailed. Then, methods and materials for fabrication of optical, mechanical, and electrical interface components are summarized. Recent systems were categorized into four groups based on their intended application and clinical workflow: ex vivo diagnostic, in vivo diagnostic, monitoring, and treatment guidance. Lastly, strengths and limitations of current SBI systems within these various applications are discussed.

RESULTS

The native smartphone capabilities for biomedical imaging applications include cameras, touchscreens, networking, computation, 3D sensing, audio, and motion, in addition to commercial wearable peripheral devices. Through user-centered design of custom hardware and software interfaces, these capabilities have the potential to enable portable, easy-to-use, point-of-care biomedical imaging systems. However, due to barriers in programming of custom software and on-board image analysis pipelines, many research prototypes fail to achieve a prospective clinical evaluation as intended. Effective clinical use cases appear to be those in which handheld, noninvasive image guidance is needed and accommodated by the clinical workflow. Handheld systems for in vivo, multispectral, and quantitative fluorescence imaging are a promising development for diagnostic and treatment guidance applications.

CONCLUSIONS

A holistic assessment of SBI systems must include interpretation of their value for intended clinical settings and how their implementations enable better workflow. A set of six guidelines are proposed to evaluate appropriateness of smartphone utilization in terms of clinical context, completeness, compactness, connectivity, cost, and claims. Ongoing work should prioritize realistic clinical assessments with quantitative and qualitative comparison to non-smartphone systems to clearly demonstrate the value of smartphone-based systems. Improved hardware design to accommodate the rapidly changing smartphone ecosystem, creation of open-source image acquisition and analysis pipelines, and adoption of robust calibration techniques to address phone-to-phone variability are three high priority areas to move SBI research forward.

Machine Learning Applications in the Evaluation and Management of Psoriasis: A Systematic Review

Background

Machine learning (ML), a subset of artificial intelligence (AI) that aims to teach machines to automatically learn tasks by inferring patterns from data, holds significant promise to aid psoriasis care. Applications include evaluation of skin images for screening and diagnosis as well as clinical management including treatment and complication prediction.

Objective

To summarize literature on ML applications to psoriasis evaluation and management and to discuss challenges and opportunities for future advances.

Methods

We searched MEDLINE, Google Scholar, ACM Digital Library, and IEEE Xplore for peer-reviewed publications published in English through December 1, 2019. Our search queries identified publications with any of the 10 computing-related keywords and "psoriasis" in the title and/or abstract.

Results

Thirty-three studies were identified. Articles were organized by topic and synthesized as evaluation- or management-focused articles covering 5 content categories: (A) Evaluation using skin images: (1) identification and differential diagnosis of psoriasis lesions, (2) lesion segmentation, and (3) lesion severity and area scoring; (B) clinical management: (1) prediction of complications and (2) treatment.

Conclusion

Machine learning has significant potential to aid psoriasis evaluation and management. Current topics popular in ML research on psoriasis are the evaluation of medical images, prediction of complications, and treatment discovery. For patients to derive the greatest benefit from ML advancements, it is helpful for dermatologists to have an understanding of ML and how it can effectively aid their assessments and decision-making.

Machine learning-enabled multiplexed microfluidic sensors

High-throughput, cost-effective, and portable devices can enhance the performance of point-of-care tests. Such devices are able to acquire images from samples at a high rate in combination with microfluidic chips in point-of-care applications. However, interpreting and analyzing the large amount of acquired data is not only a labor-intensive and time-consuming process, but also prone to the bias of the user and low accuracy. Integrating machine learning (ML) with the image acquisition capability of smartphones as well as increasing computing power could address the need for high-throughput, accurate, and automatized detection, data processing, and quantification of results. Here, ML-supported diagnostic technologies are presented. These technologies include quantification of colorimetric tests, classification of biological samples (cells and sperms), soft sensors, assay type detection, and recognition of the fluid properties. Challenges regarding the implementation of ML methods, including the required number of data points, image acquisition prerequisites, and execution of data-limited experiments are also discussed.