Improved accuracy in colorectal cancer tissue decomposition through refinement of established deep learning solutions

Hematoxylin and eosin-stained biopsy slides are regularly available for colorectal cancer patients. These slides are often not used to define objective biomarkers for patient stratification and treatment selection. Standard biomarkers often pertain to costly and slow genetic tests. However, recent work has shown that relevant biomarkers can be extracted from these images using convolutional neural networks (CNNs). The CNN-based biomarkers predicted colorectal cancer patient outcomes comparably to gold standards. Extracting CNN-biomarkers is fast, automatic, and of minimal cost. CNN-based biomarkers rely on the ability of CNNs to recognize distinct tissue types from microscope whole slide images. The quality of these biomarkers (coined 'Deep Stroma') depends on the accuracy of CNNs in decomposing all relevant tissue classes. Improving tissue decomposition accuracy is essential for improving the prognostic potential of CNN-biomarkers. In this study, we implemented a novel training strategy to refine an established CNN model, which then surpassed all previous solutions . We obtained a 95.6% average accuracy in the external test set and 99.5% in the internal test set. Our approach reduced errors in biomarker-relevant classes, such as Lymphocytes, and was the first to include interpretability methods. These methods were used to better apprehend our model's limitations and capabilities.

Domain-specific transfer learning in the automated scoring of tumor-stroma ratio from histopathological images of colorectal cancer

Tumor-stroma ratio (TSR) is a prognostic factor for many types of solid tumors. In this study, we propose a method for automated estimation of TSR from histopathological images of colorectal cancer. The method is based on convolutional neural networks which were trained to classify colorectal cancer tissue in hematoxylin-eosin stained samples into three classes: stroma, tumor and other. The models were trained using a data set that consists of 1343 whole slide images. Three different training setups were applied with a transfer learning approach using domain-specific data i.e. an external colorectal cancer histopathological data set. The three most accurate models were chosen as a classifier, TSR values were predicted and the results were compared to a visual TSR estimation made by a pathologist. The results suggest that classification accuracy does not improve when domain-specific data are used in the pre-training of the convolutional neural network models in the task at hand. Classification accuracy for stroma, tumor and other reached 96.1% on an independent test set. Among the three classes the best model gained the highest accuracy (99.3%) for class tumor. When TSR was predicted with the best model, the correlation between the predicted values and values estimated by an experienced pathologist was 0.57. Further research is needed to study associations between computationally predicted TSR values and other clinicopathological factors of colorectal cancer and the overall survival of the patients.

Reflectance Measurement Method Based on Sensor Fusion of Frame-Based Hyperspectral Imager and Time-of-Flight Depth Camera

Hyperspectral imaging and distance data have previously been used in aerial, forestry, agricultural, and medical imaging applications. Extracting meaningful information from a combination of different imaging modalities is difficult, as the image sensor fusion requires knowing the optical properties of the sensors, selecting the right optics and finding the sensors' mutual reference frame through calibration. In this research we demonstrate a method for fusing data from Fabry-Perot interferometer hyperspectral camera and a Kinect V2 time-of-flight depth sensing camera. We created an experimental application to demonstrate utilizing the depth augmented hyperspectral data to measure emission angle dependent reflectance from a multi-view inferred point cloud. We determined the intrinsic and extrinsic camera parameters through calibration, used global and local registration algorithms to combine point clouds from different viewpoints, created a dense point cloud and determined the angle dependent reflectances from it. The method could successfully combine the 3D point cloud data and hyperspectral data from different viewpoints of a reference colorchecker board. The point cloud registrations gained 0.29-0.36 fitness for inlier point correspondences and RMSE was approx. 2, which refers a quite reliable registration result. The RMSE of the measured reflectances between the front view and side views of the targets varied between 0.01 and 0.05 on average and the spectral angle between 1.5 and 3.2 degrees. The results suggest that changing emission angle has very small effect on the surface reflectance intensity and spectrum shapes, which was expected with the used colorchecker.

DeepFake knee osteoarthritis X-rays from generative adversarial neural networks deceive medical experts and offer augmentation potential to automatic classification

Recent developments in deep learning have impacted medical science. However, new privacy issues and regulatory frameworks have hindered medical data sharing and collection. Deep learning is a very data-intensive process for which such regulatory limitations limit the potential for new breakthroughs and collaborations. However, generating medically accurate synthetic data can alleviate privacy issues and potentially augment deep learning pipelines. This study presents generative adversarial neural networks capable of generating realistic images of knee joint X-rays with varying osteoarthritis severity. We offer 320,000 synthetic (DeepFake) X-ray images from training with 5,556 real images. We validated our models regarding medical accuracy with 15 medical experts and for augmentation effects with an osteoarthritis severity classification task. We devised a survey of 30 real and 30 DeepFake images for medical experts. The result showed that on average, more DeepFakes were mistaken for real than the reverse. The result signified sufficient DeepFake realism for deceiving the medical experts. Finally, our DeepFakes improved classification accuracy in an osteoarthritis severity classification task with scarce real data and transfer learning. In addition, in the same classification task, we replaced all real training data with DeepFakes and suffered only a [Formula: see text] loss from baseline accuracy in classifying real osteoarthritis X-rays.

FPI Based Hyperspectral Imager for the Complex Surfaces—Calibration, Illumination and Applications

Hyperspectral imaging (HSI) applications for biomedical imaging and dermatological applications have been recently under research interest. Medical HSI applications are non-invasive methods with high spatial and spectral resolution. HS imaging can be used to delineate malignant tumours, detect invasions, and classify lesion types. Typical challenges of these applications relate to complex skin surfaces, leaving some skin areas unreachable. In this study, we introduce a novel spectral imaging concept and conduct a clinical pre-test, the findings of which can be used to develop the concept towards a clinical application. The SICSURFIS spectral imager concept combines a piezo-actuated Fabry–Pérot interferometer (FPI) based hyperspectral imager, a specially designed LED module and several sizes of stray light protection cones for reaching and adapting to the complex skin surfaces. The imager is designed for the needs of photometric stereo imaging for providing the skin surface models (3D) for each captured wavelength. The captured HS images contained 33 selected wavelengths (ranging from 477 nm to 891 nm), which were captured simultaneously with accordingly selected LEDs and three specific angles of light. The pre-test results show that the data collected with the new SICSURFIS imager enable the use of the spectral and spatial domains with surface model information. The imager can reach complex skin surfaces. Healthy skin, basal cell carcinomas and intradermal nevi lesions were classified and delineated pixel-wise with promising results, but further studies are needed. The results were obtained with a convolutional neural network.

Differentiating Malignant from Benign Pigmented or Non-Pigmented Skin Tumours-A Pilot Study on 3D Hyperspectral Imaging of Complex Skin Surfaces and Convolutional Neural Networks

Several optical imaging techniques have been developed to ease the burden of skin cancer disease on our health care system. Hyperspectral images can be used to identify biological tissues by their diffuse reflected spectra. In this second part of a three-phase pilot study, we used a novel hand-held SICSURFIS Spectral Imager with an adaptable field of view and target-wise selectable wavelength channels to provide detailed spectral and spatial data for lesions on complex surfaces. The hyperspectral images (33 wavelengths, 477–891 nm) provided photometric data through individually controlled illumination modules, enabling convolutional networks to utilise spectral, spatial, and skin-surface models for the analyses. In total, 42 lesions were studied: 7 melanomas, 13 pigmented and 7 intradermal nevi, 10 basal cell carcinomas, and 5 squamous cell carcinomas. All lesions were excised for histological analyses. A pixel-wise analysis provided map-like images and classified pigmented lesions with a sensitivity of 87% and a specificity of 93%, and 79% and 91%, respectively, for non-pigmented lesions. A majority voting analysis, which provided the most probable lesion diagnosis, diagnosed 41 of 42 lesions correctly. This pilot study indicates that our non-invasive hyperspectral imaging system, which involves shape and depth data analysed by convolutional neural networks, is feasible for differentiating between malignant and benign pigmented and non-pigmented skin tumours, even on complex skin surfaces.

Hyperspectral Imaging Reveals Spectral Differences and Can Distinguish Malignant Melanoma from Pigmented Basal Cell Carcinomas: A Pilot Study

Pigmented basal cell carcinomas can be difficult to distinguish from melanocytic tumours. Hyperspectral imaging is a non-invasive imaging technique that measures the reflectance spectra of skin in vivo. The aim of this prospective pilot study was to use a convolutional neural network classifier in hyperspectral images for differential diagnosis between pigmented basal cell carcinomas and melanoma. A total of 26 pigmented lesions (10 pigmented basal cell carcinomas, 12 melanomas in situ, 4 invasive melanomas) were imaged with hyperspectral imaging and excised for histopathological diagnosis. For 2-class classifier (melanocytic tumours vs pigmented basal cell carcinomas) using the majority of the pixels to predict the class of the whole lesion, the results showed a sensitivity of 100% (95% confidence interval 81–100%), specificity of 90% (95% confidence interval 60–98%) and positive predictive value of 94% (95% confidence interval 73–99%). These results indicate that a convolutional neural network classifier can differentiate melanocytic tumours from pigmented basal cell carcinomas in hyperspectral images. Further studies are warranted in order to confirm these preliminary results, using larger samples and multiple tumour types, including all types of melanocytic lesions.

A Do-It-Yourself Hyperspectral Imager Brought to Practice with Open-Source Python

Commercial hyperspectral imagers (HSIs) are expensive and thus unobtainable for large audiences or research groups with low funding. In this study, we used an existing do-it-yourself push-broom HSI design for which we provide software to correct for spectral smile aberration without using an optical laboratory. The software also corrects an aberration which we call tilt. The tilt is specific for the particular imager design used, but correcting it may be beneficial for other similar devices. The tilt and spectral smile were reduced to zero in terms of used metrics. The software artifact is available as an open-source Github repository. We also present improved casing for the imager design, and, for those readers interested in building their own HSI, we provide print-ready and modifiable versions of the 3D-models required in manufacturing the imager. To our best knowledge, solving the spectral smile correction problem without an optical laboratory has not been previously reported. This study re-solved the problem with simpler and cheaper tools than those commonly utilized. We hope that this study will promote easier access to hyperspectral imaging for all audiences regardless of their financial status and availability of an optical laboratory.

Hexyl aminolevulinate, 5-aminolevulinic acid nanoemulsion and methyl aminolevulinate in photodynamic therapy of non-aggressive basal cell carcinomas: A non-sponsored, randomized, prospective and double-blinded trial

BACKGROUND

In the photodynamic therapy (PDT) of non-aggressive basal cell carcinomas (BCCs), 5-aminolevulinic acid nanoemulsion (BF-200ALA) has shown non-inferior efficacy when compared with methyl aminolevulinate (MAL), a widely used photosensitizer. Hexyl aminolevulinate (HAL) is an interesting alternative photosensitizer. To our knowledge, this is the first study using HAL-PDT in the treatment of BCCs.

OBJECTIVES

To compare the histological clearance, tolerability (pain and post-treatment reaction) and cosmetic outcome of MAL, BF-200 ALA and low-concentration HAL in the PDT of non-aggressive BCCs.

METHODS

Ninety-eight histologically verified non-aggressive BCCs met the inclusion criteria, and 54 patients with 95 lesions completed the study. The lesions were randomized to receive LED-PDT in two repeated treatments with MAL, BF-200 ALA or HAL. Efficacy was assessed both clinically and confirmed histologically at three months by blinded observers. Furthermore, cosmetic outcome, pain, post-treatment reactions fluorescence and photobleaching were evaluated.

RESULTS

According to intention-to-treat analyses, the histologically confirmed lesion clearance was 93.8% (95% confidence interval [CI] = 79.9-98.3) for MAL, 90.9% (95% CI = 76.4-96.9) for BF-200 ALA and 87.9% (95% CI = 72.7-95.2) for HAL, with no differences between the arms (P = 0.84). There were no differences between the arms as regards pain, post-treatment reactions or cosmetic outcome.

CONCLUSIONS

Photodynamic therapy with low-concentration HAL and BF-200 ALA has a similar efficacy, tolerability and cosmetic outcome compared to MAL. HAL is an interesting new option in dermatological PDT, since good efficacy is achieved with a low concentration.

Erratum: Multilabel segmentation of cancer cell culture on vascular structures with deep neural networks (Erratum)

Corrections to the article, “Multilabel segmentation of cancer cell culture on vascular structures with deep neural networks,” by S. Rahkonen et al.

Multilabel segmentation of cancer cell culture on vascular structures with deep neural networks

New increasingly complex in vitro cancer cell models are being developed. These new models seem to represent the cell behavior in vivo more accurately and have better physiological relevance than prior models. An efficient testing method for selecting the most optimal drug treatment does not exist to date. One proposed solution to the problem involves isolation of cancer cells from the patients' cancer tissue, after which they are exposed to potential drugs alone or in combinations to find the most optimal medication. To achieve this goal, methods that can efficiently quantify and analyze changes in tested cell are needed. Our study aimed to detect and segment cells and structures from cancer cell cultures grown on vascular structures in phase-contrast microscope images using U-Net neural networks to enable future drug efficacy assessments. We cultivated prostate carcinoma cell lines PC3 and LNCaP on the top of a matrix containing vascular structures. The cells were imaged with a Cell-IQ phase-contrast microscope. Automatic analysis of microscope images could assess the efficacy of tested drugs. The dataset included 36 RGB images and ground-truth segmentations with mutually not exclusive classes. The used method could distinguish vascular structures, cells, spheroids, and cell matter around spheroids in the test images. Some invasive spikes were also detected, but the method could not distinguish the invasive cells in the test images.

Ablative fractional laser-assisted photodynamic therapy for lentigo maligna: a prospective pilot study

BACKGROUND

Lentigo maligna (LM) is an in situ form of melanoma carrying a risk of progression to invasive lentigo maligna melanoma (LMM). LM poses a clinical challenge, with subclinical extension and high recurrence rates after incomplete surgery. Alternative treatment methods have been investigated with varying results. Photodynamic therapy (PDT) with methylaminolaevulinate (MAL) has already proved promising in this respect.

OBJECTIVES

To investigate the efficacy of ablative fractional laser (AFL)-assisted PDT with 5-aminolaevulinic acid nanoemulsion (BF-200 ALA) for treating LM.

METHODS

In this non-sponsored prospective pilot study, ten histologically verified LMs were treated with AFL-assisted PDT three times at 2-week intervals using a light dose of 90 J/cm² per treatment session. Local anaesthesia with ropivacaine was used. Four weeks after the last PDT treatment the lesions were treated surgically with a wide excision and sent for histopathological examination. The primary outcome was complete histopathological clearance of the LM from the surgical specimen. Patient-reported pain during illumination and the severity of the skin reaction after the PDT treatments were monitored as secondary outcomes.

RESULTS

The complete histopathological clearance rate was 7 out of 10 LMs (70%). The pain during illumination was tolerable, with the mean pain scores for the PDT sessions on a visual assessment scale ranging from 2.9 to 3.8. Some severe skin reactions occurred during the treatment period, however.

CONCLUSIONS

Ablative fractional laser-assisted PDT showed moderate efficacy in terms of histological clearance. It could constitute an alternative treatment for LM but due to the side effects it should only be considered in inoperable cases.

Hyperspectral imaging system in the delineation of Ill-defined basal cell carcinomas: a pilot study

BACKGROUND

Basal cell carcinoma (BCC) is the most common skin cancer in the Caucasian population. Eighty per cent of BCCs are located on the head and neck area. Clinically ill-defined BCCs often represent histologically aggressive subtypes, and they can have subtle subclinical extensions leading to recurrence and the need for re-excisions.

OBJECTIVES

The aim of this pilot study was to test the feasibility of a hyperspectral imaging system (HIS) in vivo in delineating the preoperatively lateral margins of ill-defined BCCs on the head and neck area.

METHODS

Ill-defined BCCs were assessed clinically with a dermatoscope, photographed and imaged with HIS. This was followed by surgical procedures where the BCCs were excised at the clinical border and the marginal strip separately. HIS, with a 12-cm² field of view and fast data processing, records a hyperspectral graph for every pixel in the imaged area, thus creating a data cube. With automated computational modelling, the spectral data are converted into localization maps showing the tumour borders. Interpretation of these maps was compared to the histologically verified tumour borders.

RESULTS

Sixteen BCCs were included. Of these cases, 10 of 16 were the aggressive subtype of BCC and 6 of 16 were nodular, superficial or a mixed type. HIS delineated the lesions more accurately in 12 of 16 of the BCCs compared to the clinical evaluation (4 of 16 wider and 8 of 16 smaller by HIS). In 2 of 16 cases, the HIS-delineated lesion was wider without histopathological confirmation. In 2 of 16 cases, HIS did not detect the histopathologically confirmed subclinical extension.

CONCLUSIONS

HIS has the potential to be an easy and fast aid in the preoperative delineation of ill-defined BCCs, but further adjustment and larger studies are warranted for an optimal outcome.

UAS BASED TREE SPECIES IDENTIFICATION USING THE NOVEL FPI BASED HYPERSPECTRAL CAMERAS IN VISIBLE, NIR AND SWIR SPECTRAL RANGES

Unmanned airborne systems (UAS) based remote sensing offers flexible tool for environmental monitoring. Novel lightweight Fabry-Perot interferometer (FPI) based, frame format, hyperspectral imaging in the spectral range from 400 to 1600 nm was used for identifying different species of trees in a forest area. To the best of the authors’ knowledge, this was the first research where stereoscopic, hyperspectral VIS, NIR, SWIR data is collected for tree species identification using UAS. The first results of the analysis based on fusion of two FPI-based hyperspectral imagers and RGB camera showed that the novel FPI hyperspectral technology provided accurate geometric, radiometric and spectral information in a forested scene and is operational for environmental remote sensing applications.

Delineating margins of lentigo maligna using a hyperspectral imaging system

Lentigo maligna (LM) is an in situ form of melanoma which can progress into invasive lentigo maligna melanoma (LMM). Variations in the pigmentation and thus visibility of the tumour make assessment of lesion borders challenging. We tested hyperspectral imaging system (HIS) in in vivo preoperative delineation of LM and LMM margins. We compared lesion margins delineated by HIS with those estimated clinically, and confirmed histologically. A total of 14 LMs and 5 LMMs in 19 patients were included. HIS analysis matched the histo-pathological analysis in 18/19 (94.7%) cases while in 1/19 (5.3%) cases HIS showed lesion extension not confirmed by histopathology (false positives). Compared to clinical examination, HIS defined lesion borders more accurately in 10/19 (52.6%) of cases (wider, n = 7 or smaller, n = 3) while in 8/19 (42.1%) cases lesion borders were the same as delineated clinically as confirmed histologically. Thus, HIS is useful for the detection of subclinical LM/LMM borders.

Detecting field cancerization using a hyperspectral imaging system

BACKGROUND

Field cancerization denotes subclinical abnormalities in a tissue chronically exposed to UV radiation. These abnormalities can be found surrounding the clinically visible actinic keratoses.

OBJECTIVES

The aim of this study was to test the feasibility of a hyperspectral imaging system in the detection of multiple clinical and subclinical AKs for early treatment of the affected areas.

MATERIALS AND METHODS

Altogether 52 clinical AKs in 12 patients were included in this study. In six patients digital photos were taken of the naive AKs, and again after methylaminolevulinate(MAL)-fluorescence diagnosis which was used to teach HIS to find subclinical lesions. After 2-3 days when the MAL had vanished, the hyperspectral images were taken. Biopsies were taken from clinical AKs, healthy-looking skin and several suspected subclinical AKs. In the other six patients digital and hyperspectral images were taken of the naive AKs followed by one biopsy per patient.

RESULTS

HIS detected all clinically visible 52 AKs and numerous subclinical lesions. The histopathology of the 33 biopsied lesions were concordant with the HIS results showing either AK (n = 28) or photodamage (n = 5). Of the 28 histopathologically confirmed AKs, 16 were subclinical. A specific diffuse reflectance spectrum of an AK and healthy skin was defined.

CONCLUSION

The hyperspectral imaging system offers a new, non-invasive method for early detection of field cancerization. Lasers Surg. Med. 45:410-417, 2013. © 2013 Wiley Periodicals, Inc.

Fur animal botulism hazard due to feed

To assess the botulism hazard in fur animal feed production, 236 fur animal feed components and feed samples were analysed for Clostridium botulinum by detecting BoNT-encoding genes (botA, botB, botC, botE or botF) by PCR and for sulphite-reducing clostridia (SRC) by iron sulphite agar. The quality of the hazard analysis of critical control points (HACCP) -based in-house control system (IHCS) was evaluated with respect to botulism risk in feed plants (n=32). The overall prevalence of C. botulinum was 13% in different feed components and 5% in feed. The estimated MPN count of C. botulinum in feed components was 6.4 × 10(3)/kg at the highest and was shown to poorly correlate with SRC count. The critical control points in IHCSs were variable, and control limits were improperly set in most feed-producing plants. C. botulinum possesses a persistent safety hazard for fur animals by feed production, and control practices should be reassessed.