A Spectral Filter Array Camera for Clinical Monitoring and Diagnosis: Proof of Concept for Skin Oxygenation Imaging

Artificial neural networks trained on simulated multispectral data for real-time imaging of skin microcirculatory blood oxygen saturation

Significance

Imaging blood oxygen saturation (SO 2 ) in the skin can be of clinical value when studying ischemic tissue. Emerging multispectral snapshot cameras enable real-time imaging but are limited by slow analysis when using inverse Monte Carlo (MC), the gold standard for analyzing multispectral data. Using artificial neural networks (ANNs) facilitates a significantly faster analysis but requires a large amount of high-quality training data from a wide range of tissue types for a precise estimation ofSO 2 .

Aim

We aim to develop a framework for training ANNs that estimatesSO 2 in real time from multispectral data with a precision comparable to inverse MC.

Approach

ANNs are trained using synthetic data from a model that includes MC simulations of light propagation in tissue and hardware characteristics. The model includes physiologically relevant variations in optical properties, unique sensor characteristics, variations in illumination spectrum, and detector noise. This approach enables a rapid way of generating high-quality training data that covers different tissue types and skin pigmentation.

Results

The ANN implementation analyzes an image in 0.11 s, which is at least 10,000 times faster than inverse MC. The hardware modeling is significantly improved by an in-house calibration of the sensor spectral response. An in-vivo example shows that inverse MC and ANN give almost identicalSO 2 values with a mean absolute deviation of 1.3%-units.

Conclusions

ANN can replace inverse MC and enable real-time imaging of microcirculatorySO 2 in the skin if detailed and precise modeling of both tissue and hardware is used when generating training data.

Unsupervised Spectral Demosaicing With Lightweight Spectral Attention Networks

This paper presents a deep learning-based spectral demosaicing technique trained in an unsupervised manner. Many existing deep learning-based techniques relying on supervised learning with synthetic images, often underperform on real-world images, especially as the number of spectral bands increases. This paper presents a comprehensive unsupervised spectral demosaicing (USD) framework based on the characteristics of spectral mosaic images. This framework encompasses a training method, model structure, transformation strategy, and a well-fitted model selection strategy. To enable the network to dynamically model spectral correlation while maintaining a compact parameter space, we reduce the complexity and parameters of the spectral attention module. This is achieved by dividing the spectral attention tensor into spectral attention matrices in the spatial dimension and spectral attention vector in the channel dimension. This paper also presents Mosaic 25 , a real 25-band hyperspectral mosaic image dataset featuring various objects, illuminations, and materials for benchmarking purposes. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed method outperforms conventional unsupervised methods in terms of spatial distortion suppression, spectral fidelity, robustness, and computational cost. Our code and dataset are publicly available at https://github.com/polwork/Unsupervised-Spectral-Demosaicing.

Multispectral indices for real-time and non-invasive tissue ischemia monitoring using snapshot cameras

An adequate supply of oxygen-rich blood is vital to maintain cell homeostasis, cellular metabolism, and overall tissue health. While classical methods of measuring tissue ischemia are often invasive, localized and require skin contact or contrast agents, spectral imaging shows promise as a non-invasive, wide field, and contrast-free approach. We evaluate three novel reflectance-based spectral indices from the 460 - 840 nm spectral range. With the aim of enabling real time visualization of tissue ischemia, information is extracted from only 2-3 spectral bands. Video-rate spectral data was acquired from arm occlusion experiments in 27 healthy volunteers. The performance of the indices was evaluated against binary Support Vector Machine (SVM) classification of healthy versus ischemic skin tissue, two other indices from literature, and tissue oxygenation estimated using spectral unmixing. Robustness was tested by evaluating these under various lighting conditions and on both the dorsal and palmar sides of the hand. A novel index with real-time capabilities using reflectance information only from 547 nm and 556 nm achieves an average classification accuracy of 88.48, compared to 92.65 using an SVM trained on all available wavelengths. Furthermore, the index has a higher accuracy compared to reference methods and its time dynamics compare well against the expected clinical responses. This holds promise for robust real-time detection of tissue ischemia, possibly contributing to improved patient care and clinical outcomes.

Near-Infrared Spectral Similarity between Ex Vivo Porcine and In Vivo Human Tissue

BACKGROUND

In vivo diffuse reflectance spectroscopy provides additional contrast in discriminating nerves embedded in adipose tissue during surgery. However, large datasets are required to achieve clinically acceptable classification levels. This study assesses the spectral similarity between ex vivo porcine and in vivo human spectral data of nerve and adipose tissue, as porcine tissue could contribute to generate large datasets.

METHODS

Porcine diffuse reflectance spectra were measured at 124 nerve and 151 adipose locations. A previously recorded dataset of 32 in vivo human nerve and 23 adipose tissue locations was used for comparison. In total, 36 features were extracted from the raw porcine to generate binary logistic regression models for all combinations of two, three, four and five features. Feature selection was performed by assessing similar means between normalized features of nerve and of adipose tissue (Kruskal-Wallis test, p < 0.05) and for models performing best on the porcine cross validation set. The human test set was used to assess classification performance.

RESULTS

The binary logistic regression models with selected features showed an accuracy of 60% on the test set.

CONCLUSIONS

Spectral similarity between ex vivo porcine and in vivo human adipose and nerve tissue was present, but further research is required.

Circadian effects on UV-induced damage and mutations

Skin cancer is the most diagnosed type of cancer in the United States, and while most of these malignancies are highly treatable, treatment costs still exceed $8 billion annually. Over the last 50 years, the annual incidence of skin cancer has steadily grown; therefore, understanding the environmental factors driving these types of cancer is a prominent research-focus. A causality between ultraviolet radiation (UVR) exposure and skin cancer is well-established, but exposure to UVR alone is not necessarily sufficient to induce carcinogenesis. The emerging field of circadian biology intersects strongly with the physiological systems of the mammalian body and introduces a unique opportunity for analyzing mechanisms of homeostatic disruption. The circadian clock refers to the approximate 24-hour cycle, in which protein levels of specific clock-controlled genes (CCGs) fluctuate based on the time of day. Though these CCGs are tissue specific, the skin has been observed to have a robust circadian clock that plays a role in its response to UVR exposure. This in-depth review will detail the mechanisms of the circadian clock and its role in cellular homeostasis. Next, the skin's response to UVR exposure and its induction of DNA damage and mutations will be covered - with an additional focus placed on how the circadian clock influences this response through nucleotide excision repair. Lastly, this review will discuss current models for studying UVR-induced skin lesions and perturbations of the circadian clock, as well as the impact of these factors on human health.

Multispectral Imaging for Hemoglobin Estimation by PCA

Tissular blood perfusion is helpful to assess the health condition of a subject and even monitor superficial lesions. Current state of the art is focused on developing non-invasive, quantitative and accessible methods for blood flow monitoring in large areas. This paper presents an approach based on multispectral images on the VIS-NIR range to quantify blood perfusion. Our goal is to estimate the changes in deoxygenated hemoglobin. To do so, we employ principal component analysis followed by a linear regression model. The proposal was evaluated using in-vivo data from a vascular occlusion protocol, and the results were validated against photoplethysmography measurements. Although the number of subjects in the protocol was limited, our model made a prediction with an average similarity of 91.53% with a mean R-squared adjusted of 0.8104.

Color-Ratio Maps Enhanced Optical Filter Design and Its Application in Green Pepper Segmentation

There is a growing demand for developing image sensor systems to aid fruit and vegetable harvesting, and crop growth prediction in precision agriculture. In this paper, we present an end-to-end optimization approach for the simultaneous design of optical filters and green pepper segmentation neural networks. Our optimization method modeled the optical filter as one learnable neural network layer and attached it to the subsequent camera spectral response (CSR) layer and segmentation neural network for green pepper segmentation. We used not only the standard red–green–blue output from the CSR layer but also the color-ratio maps as additional cues in the visible wavelength and to augment the feature maps as the input for segmentation. We evaluated how well our proposed color-ratio maps enhanced optical filter design methods in our collected dataset. We find that our proposed method can yield a better performance than both an optical filter RGB system without color-ratio maps and a raw RGB camera (without an optical filter) system. The proposed learning-based framework can potentially build better image sensor systems for green pepper segmentation.

Estimation of skin microcirculatory hemoglobin oxygen saturation and red blood cell tissue fraction using a multispectral snapshot imaging system: a validation study

SIGNIFICANCE

Hemoglobin oxygen saturation and red blood cell (RBC) tissue fraction are important parameters when assessing microvascular status. Functional information can be attained using temporally resolved measurements performed during stimulus-response protocols. Pointwise assessments can currently be conducted with probe-based systems. However, snapshot multispectral imaging (MSI) can be used for spatial-temporal measurements.

AIM

To validate if hemoglobin oxygen saturation and RBC tissue fraction can be quantified using a snapshot MSI system and an inverse Monte Carlo algorithm.

APPROACH

Skin tissue measurements from the MSI system were compared to those from a validated probe-based system during arterial and venous occlusion provocation on 24 subjects in the wavelength interval 450 to 650 nm, to evaluate a wide range of hemoglobin oxygen saturation and RBC tissue fraction levels.

RESULTS

Arterial occlusion results show a mean linear regression R2  =  0.958 for hemoglobin oxygen saturation. Comparing relative RBC tissue fraction during venous occlusion results in R2  =  0.925. The MSI system shows larger dynamic changes than the reference system, which might be explained by a deeper sampling including more capacitance vessels.

CONCLUSIONS

The snapshot MSI system estimates hemoglobin oxygen saturation and RBC tissue fraction in skin microcirculation showing a high correlation (R2  >  0.9 in most subjects) with those measured by the reference method.

An Evaluation Framework for Spectral Filter Array Cameras to Optimize Skin Diagnosis

Comparing and selecting an adequate spectral filter array (SFA) camera is application-specific and usually requires extensive prior measurements. An evaluation framework for SFA cameras is proposed and three cameras are tested in the context of skin analysis. The proposed framework does not require application-specific measurements and spectral sensitivities together with the number of bands are the main focus. An optical model of skin is used to generate a specialized training set to improve spectral reconstruction. The quantitative comparison of the cameras is based on reconstruction of measured skin spectra, colorimetric accuracy, and oxygenation level estimation differences. Specific spectral sensitivity shapes influence the results directly and a 9-channel camera performed best regarding the spectral reconstruction metrics. Sensitivities at key wavelengths influence the performance of oxygenation level estimation the strongest. The proposed framework allows to compare spectral filter array cameras and can guide their application-specific development.