Spectral filter optimization for the recovery of parameters which describe human skin

Deep learning-based optical approach for skin analysis of melanin and hemoglobin distribution

SIGNIFICANCE

Melanin and hemoglobin have been measured as important diagnostic indicators of facial skin conditions for aesthetic and diagnostic purposes. Commercial clinical equipment provides reliable analysis results, but it has several drawbacks: exclusive to the acquisition system, expensive, and computationally intensive.

AIM

We propose an approach to alleviate those drawbacks using a deep learning model trained to solve the forward problem of light-tissue interactions. The model is structurally extensible for various light sources and cameras and maintains the input image resolution for medical applications.

APPROACH

A facial image is divided into multiple patches and decomposed into melanin, hemoglobin, shading, and specular maps. The outputs are reconstructed into a facial image by solving the forward problem over skin areas. As learning progresses, the difference between the reconstructed image and input image is reduced, resulting in the melanin and hemoglobin maps becoming closer to their distribution of the input image.

RESULTS

The proposed approach was evaluated on 30 subjects using the professional clinical system, VISIA VAESTRO. The correlation coefficients for melanin and hemoglobin were found to be 0.932 and 0.857, respectively. Additionally, this approach was applied to simulated images with varying amounts of melanin and hemoglobin.

CONCLUSION

The proposed approach showed high correlation with the clinical system for analyzing melanin and hemoglobin distribution, indicating its potential for accurate diagnosis. Further calibration studies using clinical equipment can enhance its diagnostic ability. The structurally extensible model makes it a promising tool for various image acquisition conditions.

Genetic optimization of mid-infrared filters for a machine learning chemical classifier

Miniaturized mid-infrared spectrometers present opportunities for applications that range from health monitoring to agriculture. One approach combines arrays of spectral filters with infrared photodetectors, called filter-array detector-array (FADA) microspectrometers. A paper recently reported a FADA microspectrometer in tandem with machine learning for chemical identification. In that work, a FADA microspectrometer with 20 filters was assembled and tested. The filters were band-pass, or band-stop designs that evenly spanned the microspectrometer's operating wavelength range. However, given that a machine learning classifier can be trained on an arbitrary filter basis, it is not apparent that evenly spaced filters are optimal. Here, through simulations with noise, we use a genetic algorithm to optimize six bandpass filters to best identify liquid and gaseous chemicals. We report that the classifiers trained with the optimized filter sets outperform those trained with evenly spaced filter sets and those handpicked to target the absorption bands of the chemicals investigated.

Band selection for oxygenation estimation with multispectral/hyperspectral imaging

Multispectral imaging provides valuable information on tissue composition such as hemoglobin oxygen saturation. However, the real-time application of this technique in interventional medicine can be challenging due to the long acquisition times needed for large amounts of hyperspectral data with hundreds of bands. While this challenge can partially be addressed by choosing a discriminative subset of bands, the band selection methods proposed to date are mainly restricted by the availability of often hard to obtain reference measurements. We address this bottleneck with a new approach to band selection that leverages highly accurate Monte Carlo (MC) simulations. We hypothesize that a so chosen small subset of bands can reproduce or even improve upon the results of a quasi continuous spectral measurement. We further investigate whether novel domain adaptation techniques can address the inevitable domain shift stemming from the use of simulations. Initial results based on in silico and in vivo experiments suggest that 10-20 bands are sufficient to closely reproduce results from spectral measurements with 101 bands in the 500-700 nm range. The investigated domain adaptation technique, which only requires unlabeled in vivo measurements, yielded better results than the pure in silico band selection method. Overall, our method could guide development of fast multispectral imaging systems suited for interventional use without relying on complex hardware setups or manually labeled data.

An Evaluation Framework for Spectral Filter Array Cameras to Optimize Skin Diagnosis

Comparing and selecting an adequate spectral filter array (SFA) camera is application-specific and usually requires extensive prior measurements. An evaluation framework for SFA cameras is proposed and three cameras are tested in the context of skin analysis. The proposed framework does not require application-specific measurements and spectral sensitivities together with the number of bands are the main focus. An optical model of skin is used to generate a specialized training set to improve spectral reconstruction. The quantitative comparison of the cameras is based on reconstruction of measured skin spectra, colorimetric accuracy, and oxygenation level estimation differences. Specific spectral sensitivity shapes influence the results directly and a 9-channel camera performed best regarding the spectral reconstruction metrics. Sensitivities at key wavelengths influence the performance of oxygenation level estimation the strongest. The proposed framework allows to compare spectral filter array cameras and can guide their application-specific development.

Investigating the potential of Zernike polynomials to characterise spatial distribution of macular pigment

It has been postulated that particular patterns of macular pigment (MP) distribution may be associated with the risk for eye diseases such as age-related macular degeneration (AMD). This work investigates the potential of Zernike polynomials (ZP) to characterise the level and distribution of MP, and their suitability as a representation for analysis of the effects of age and AMD on MP patterns. As the case study, MP distribution maps computed using an experimental method based on fundus reflectance (MRIA) were obtained for ninety volunteers representing three groups: under-fifty without AMD, fifty and over without AMD, and fifty and over with AMD. ZP with 105 coefficients were fitted to the maps using least-squares optimisation and found to represent MP maps accurately (RMSE<10⁻¹). One-way MANOVA analysis carried out on ZP representations showed that the three subject groups have significantly different means (Wilk’s Lambda 0.125, p<0.0001). Linear discriminant analysis with leave-one-out scheme resulted in accuracy, sensitivity and specificity of classification according to, respectively, disease status regardless of age (81% all); disease status in the age-matched groups (87%, 88%, 86%); age irrespective of disease status (81%, 83%, 73%); and age for subjects without AMD (83%, 88%, 80%). Mean MP distributions computed from ZP coefficients for the three groups showed more elevated and more peaked MP for the healthy under-fifty group; more irregular and more elevated peripheral levels in over-fifty AMD group than in over-fifty non-AMD group; and moderate radial asymmetry in non-AMD over-50 group. The results suggest that ZP coefficients are capable of accurately representing MP in a way that captures certain spatial patterns of its distribution. Using the ZP representation MP maps could be classified according to both age and disease status with accuracy significantly greater than chance, with peak elevation, pattern irregularity and radial asymmetry identified as important features.

Novel Application of Cultured Epithelial Autografts (CEA) with Expanded Mesh Skin Grafting Over an Artificial Dermis or Dermal Wound Bed Preparation

Cultured epithelial autografts (CEA) with highly expanded mesh skin grafts were used for extensive adult burns covering more than 30% of the total body surface area. A prospective study on eight patients assessed subjective and objective findings up to a 12-month follow-up. The results of wound healing for over 1:6 mesh plus CEA, gap 1:6 mesh plus CEA, and 1:3 mesh were compared at 3, 6, and 12 months using extensibility, viscoelasticity, color, and transepidermal water loss by a generalized estimating equation (GEE) or generalized linear mixed model (GLMM). No significant differences were observed among the paired treatments at any time point. At 6 and 12 months, over 1:6 mesh plus CEA achieved significantly better expert evaluation scores by the Vancouver and Manchester Scar Scales (p < 0.01). Extended skin grafting plus CEA minimizes donor resources and the quality of scars is equal or similar to that with conventional low extended mesh slit-thickness skin grafting such as 1:3 mesh. A longitudinal analysis of scars may further clarify the molecular changes of scar formation and pathogenesis.

Silver Sulfadiazine–Impregnated Hydrocolloid Dressing Is Beneficial in Split-Thickness Skin-Graft Donor Wound Healing in a Small Randomized Controlled Study

Donor-site wound healing was tested with a silver sulfadiazine (SSD)-impregnated hydrocolloid dressing and hydrocolloid dressing applied manually by a physician on site. A total of 14 patients, 5 woman and 9 men (23-89 years old, average = 61.6 ± 18.70 years), were enrolled in this prospective controlled study. The degree of bleeding was significantly less with the SSD-impregnated than with the hydrocolloid dressing (P < .01). In the moisture meter analysis, the values of the effective contact coefficient and corneal thickness were significantly smaller with the SSD-impregnated dressing (P < .05). In the color analysis, the clarity was significantly lower with the hydrocolloid dressing after 3 months than that of intact neighboring skin (P < .01). Regarding red-green color, SSD-impregnated and hydrocolloid values were significantly greater than the intact skin value immediately after and at 3 months, and the hydrocolloid value was significantly greater than intact at 6 months (P < .01 immediately; P < .01 at 3 months and intact at 6 months) in redness. Regarding yellow-blue color, the hydrocolloid value was significantly lower than the intact skin value at 3 months (P < .05 and intact) in yellow. The extensibility was significantly lower with the hydrocolloid dressing than in intact skin immediately (P < .01), and viscoelasticity was significantly lower with the hydrocolloid dressing than in intact skin immediately and after 3 and 6 months (P < .01 immediately and at 6 months and P < .05 at 3 months). The SSD-impregnated hydrocolloid dressing led to superior wound healing, decreased the degree of bleeding, and demonstrated better corneal barrier function, clarity, color matching, and viscoelasticity in split-thickness donors.

Non-invasive estimation of skin thickness from hyperspectral imaging and validation using echography

BACKGROUND

The skin is the largest organ and is subject to the greatest exposure to outside elements throughout one׳s lifetime. Current data by the American Academy of Dermatology suggests that approximately ten people die each hour worldwide due to skin related conditions. Cancers such as melanoma are growths that originate in the epidermis. Therefore, an accurate and non-invasive method to estimate skin constitutive elements can play an important clinical role in detecting the early onset of skin tumors. It can also serve as a valuable tool for research and development in cosmetics and pharmaceuticals in general.

METHODS

In our prior work, we developed a method that combined a physics-based model of human skin with machine learning and Hyperspectral imaging to non-invasively estimate physiological skin parameters, including melanosomes, collagen, oxygen saturation, and blood volume. In this work, we extend that model to also estimate skin thickness. Moreover, for the first time, we develop a protocol to test our methodology for skin thickness estimation using Ultrasound to acquire a gold standard dataset.

RESULTS

We tested our methodology for skin thickness estimation on a dataset of 48 Hyperspectral signatures obtained in vivo from six patients under IRB at Johns Hopkins Hospital. We found mean absolute errors on the order of the Ultrasound resolution (i.e., our gold standard).

DISCUSSION

This is the first study of its kind to validate skin thickness estimates using a gold standard. Our preliminary results constitute a proof-of-concept that hyperspectral-based methods can accurately and non-invasively estimate skin thickness in clinical settings.

Model based inversion for deriving maps of histological parameters characteristic of cancer from ex-vivo multispectral images of the colon

A model-based inversion method was used to obtain quantitative estimates of histological parameters from multispectral images of the colon and to examine their potential for discriminating between normal and pathological tissues. Pixel-wise estimates of the mucosal blood volume fraction, density of the scattering particles and thickness were derived using a two-stage method. In the first (forward) stage reflectance spectra corresponding to given instances of the parameter values were computed using Monte Carlo simulation of photon propagation through a multi-layered tissue. In the second (inversion) stage the parameter values were obtained via optimization using an iterated conditional modes algorithm based on Discrete Markov Random Fields. The method was validated on computer generated data contaminated with noise giving a mean normalized root mean square deviation (NRMSD) of 2.04. Validation on ex vivo images demonstrated that parametric maps show gross correspondence with histological features of mucosa characteristic of cancerous, precancerous and noncancerous colon lesions. The key signs of abnormality were shown to be the increase in the blood volume fraction and decrease in the density of scattering particles.

Skin parameter map retrieval from a dedicated multispectral imaging system applied to dermatology/cosmetology

In vivo quantitative assessment of skin lesions is an important step in the evaluation of skin condition. An objective measurement device can help as a valuable tool for skin analysis. We propose an explorative new multispectral camera specifically developed for dermatology/cosmetology applications. The multispectral imaging system provides images of skin reflectance at different wavebands covering visible and near-infrared domain. It is coupled with a neural network-based algorithm for the reconstruction of reflectance cube of cutaneous data. This cube contains only skin optical reflectance spectrum in each pixel of the bidimensional spatial information. The reflectance cube is analyzed by an algorithm based on a Kubelka-Munk model combined with evolutionary algorithm. The technique allows quantitative measure of cutaneous tissue and retrieves five skin parameter maps: melanin concentration, epidermis/dermis thickness, haemoglobin concentration, and the oxygenated hemoglobin. The results retrieved on healthy participants by the algorithm are in good accordance with the data from the literature. The usefulness of the developed technique was proved during two experiments: a clinical study based on vitiligo and melasma skin lesions and a skin oxygenation experiment (induced ischemia) with healthy participant where normal tissues are recorded at normal state and when temporary ischemia is induced.

Optimization of excitation-emission band-pass filter for visualization of viable bacteria distribution on the surface of pork meat

A novel method of optically reducing the dimensionality of an excitation-emission matrix (EEM) by optimizing the excitation and emission band-pass filters was proposed and applied to the visualization of viable bacteria on pork. Filters were designed theoretically using an EEM data set for evaluating colony-forming units on pork samples assuming signal-to-noise ratios of 100, 316, or 1000. These filters were evaluated using newly measured EEM images. The filters designed for S/N = 100 performed the best and allowed the visualization of viable bacteria distributions. The proposed method is expected to be a breakthrough in the application of EEM imaging.

Selection of optimal combinations of band-pass filters for ice detection by hyperspectral imaging

Hyperspectral imaging captures rich information in spatial and spectral domains but involves high costs and complex data processing. The use of a set of optical band-pass filters (BPFs) in the acquisition of spectral images is proposed for reducing dimensionality of spectral data while maintaining target detection and/or categorization performance. A set of BPFs that could distinguish ice from surrounding water on various materials (e.g., asphalt), was designed as an example. Relatively high accuracy (90.28%) was achieved with only two BPFs, showing the potential of this approach for accurate target detection with lesser complexity than conventional methods.

New closed-form approximation for skin chromophore mapping

The concentrations of blood and melanin in skin can be estimated based on the reflectance of light. Many models for this estimation have been built, such as Monte Carlo simulation, diffusion models, and the differential modified Beer-Lambert law. The optimization-based methods are too slow for chromophore mapping of high-resolution spectral images, and the differential modified Beer-Lambert is not often accurate enough. Optimal coefficients for the differential Beer-Lambert model are calculated by differentiating the diffusion model, optimized to the normal skin spectrum. The derivatives are then used in predicting the difference in chromophore concentrations from the difference in absorption spectra. The accuracy of the method is tested both computationally and experimentally using a Monte Carlo multilayer simulation model, and the data are measured from the palm of a hand during an Allen's test, which modulates the blood content of skin. The correlations of the given and predicted blood, melanin, and oxygen saturation levels are correspondingly r = 0.94, r = 0.99, and r = 0.73. The prediction of the concentrations for all pixels in a 1-megapixel image would take ∼ 20 min, which is orders of magnitude faster than the methods based on optimization during the prediction.

Optical filter highlighting spectral features part II: quantitative measurements of cosmetic foundation and assessment of their spatial distributions under realistic facial conditions

We previously proposed a filter that could detect cosmetic foundations with high discrimination accuracy [Opt. Express 19, 6020 (2011)]. This study extends the filter's functionality to the quantification of the amount of foundation and applies the filter for the assessment of spatial distributions of foundation under realistic facial conditions. Human faces that are applied with quantitatively controlled amounts of cosmetic foundations were measured using the filter. A calibration curve between pixel values of the image and the amount of foundation was created. The optical filter was applied to visualize spatial foundation distributions under realistic facial conditions, which clearly indicated areas on the face where foundation remained even after cleansing. Results confirm that the proposed filter could visualize and nondestructively inspect the foundation distributions.

Optical filter for highlighting spectral features part I: design and development of the filter for discrimination of human skin with and without an application of cosmetic foundation

Light reflected from an object's surface contains much information about its physical and chemical properties. Changes in the physical properties of an object are barely detectable in spectra. Conventional trichromatic systems, on the other hand, cannot detect most spectral features because spectral information is compressively represented as trichromatic signals forming a three-dimensional subspace. We propose a method for designing a filter that optically modulates a camera's spectral sensitivity to find an alternative subspace highlighting an object's spectral features more effectively than the original trichromatic space. We designed and developed a filter that detects cosmetic foundations on human face. Results confirmed that the filter can visualize and nondestructively inspect the foundation distribution.

Basic fibroblast growth factor is beneficial for postoperative color uniformity in split-thickness skin grafting

Color changes of visible and exposed body surfaces, such as the face and extremities, after burn injury or surgery, such as skin grafting, flap, or sclerotherapy for vascular malformations, are sometimes a concern. The consequences reduce the satisfaction of both patients and physicians. An easy and reproducible method has not yet been established for an objective analysis of color changes; therefore, we tested a hand-held color analyzer (NF-333; Nippon Denshoku Co. Ltd) with data transport to a computer database and analysis software for posttreatment skin color change. The parameters included L, a, and b, which measure clarity, red, and yellow, respectively. Two groups were prospectively divided with 20 (11 females and nine males) patients per group. One group received skin grafting plus basic fibroblast growth factor (bFGF) spray daily and the other group received only skin grafting. The patients were randomized by the date of their first visit to our hospital. Patients were treated with bFGF on odd days, while patients who came on even days were included in the non-bFGF-treated group. The donor site for skin grafting was the lateral thighs and the thickness was similar in both groups. The results were compared at 1-year posttreatment follow-up. Clinical and objective assessments of the scars were performed 1 to years after complete healing. Color change differentials in comparison with the surrounding skin were lower with bFGF treatment in all parameters (p<0.01), along with clinical assessment with the Vancouver Scar Scale; therefore, the treatment contribute to a better color match with skin grafting postoperatively.

Selection of optimal filters for multispectral imaging

Preece and Claridge [IEEE Trans. Pattern Anal. Mach. Intell. 26, 913 (2004)] have proposed a technique for selecting filters for the maximally accurate recovery of object parameters such as chromophore concentrations from a multispectral image of an object. Their selection criteria are derived from an analysis of a model of light propagation in the object and take into account both errors in the modeling process and errors in the image acquisition process, as well as the inherent behavior and structure of the model. We investigate their method on simulated image data and show that filters selected according to their criteria are demonstrably superior to other choices.

Quantitative analysis of multi-spectral fundus images

We have developed a new technique for extracting histological parameters from multi-spectral images of the ocular fundus. The new method uses a Monte Carlo simulation of the reflectance of the fundus to model how the spectral reflectance of the tissue varies with differing tissue histology. The model is parameterised by the concentrations of the five main absorbers found in the fundus: retinal haemoglobins, choroidal haemoglobins, choroidal melanin, RPE melanin and macular pigment. These parameters are shown to give rise to distinct variations in the tissue colouration. We use the results of the Monte Carlo simulations to construct an inverse model which maps tissue colouration onto the model parameters. This allows the concentration and distribution of the five main absorbers to be determined from suitable multi-spectral images. We propose the use of "image quotients" to allow this information to be extracted from uncalibrated image data. The filters used to acquire the images are selected to ensure a one-to-one mapping between model parameters and image quotients. To recover five model parameters uniquely, images must be acquired in six distinct spectral bands. Theoretical investigations suggest that retinal haemoglobins and macular pigment can be recovered with RMS errors of less than 10%. We present parametric maps showing the variation of these parameters across the posterior pole of the fundus. The results are in agreement with known tissue histology for normal healthy subjects. We also present an early result which suggests that, with further development, the technique could be used to successfully detect retinal haemorrhages.

Model-based parameter recovery from uncalibrated optical images

We propose a novel method for quantitative interpretation of uncalibrated optical images which is derived explicitly from an analysis of the image formation model. Parameters characterising the tissue are recovered from images acquired using filters optimised to minimise the error. Preliminary results are shown for the skin, where the technique was successfully applied to aid the diagnosis and interpretation of non-melanocytic skin cancers and acne; and for the more challenging ocular fundus, for mapping of the pigment xanthophyll.