Targeted transfer learning to improve performance in small medical physics datasets

Predicting Osteoporosis and Osteopenia by Fusing Deep Transfer Learning Features and Classical Radiomics Features Based on Single-Source Dual-energy CT Imaging

RATIONALE AND OBJECTIVES

To develop and validate a predictive model for osteoporosis and osteopenia prediction by fusing deep transfer learning (DTL) features and classical radiomics features based on single-source dual-energy computed tomography (CT) virtual monochromatic imaging.

METHODS

A total of 606 lumbar vertebrae with dual-energy CT imaging and quantitative CT (QCT) evaluation were included in the retrospective study and randomly divided into the training (n = 424) and validation (n = 182) cohorts. Radiomics features and DTL features were extracted from 70-keV monochromatic CT images, followed by feature selection and model construction, radiomics and DTL features models were established. Then, we integrated the selected two types of features into a features fusion model. We developed a two-level classifier for the hierarchical pairwise classification of each vertebra. All the vertebrae were first classified into osteoporosis and non-osteoporosis groups, then non-osteoporosis group was classified into osteopenia and normal groups. QCT was used as reference. The predictive performance and clinical usefulness of three models were evaluated and compared.

RESULTS

The area under the curve (AUC) of the features fusion, radiomics and DTL models for the classification between osteoporosis and non-osteoporosis were 0.981, 0.999, 0.997 in the training cohort and 0.979, 0.943, 0.848 in the validation cohort. Furthermore, the AUCs of the previously mentioned models for the differentiation between osteopenia and normal were 0.994, 0.971, 0.996 in the training cohort and 0.990, 0.968, 0.908 in the validation cohort. The overall accuracy of the previously mentioned models for two-level classifications was 0.979, 0.955, 0.908 in the training cohort and 0.918, 0.885, 0.841 in the validation cohort. Decision curve analysis showed that all models had high clinical value.

CONCLUSION

The feature fusion model can be used for osteoporosis and osteopenia prediction with improved predictive ability over a radiomics model or a DTL model alone.

Omics-imaging signature-based nomogram to predict the progression-free survival of patients with hepatocellular carcinoma after transcatheter arterial chemoembolization

BACKGROUND

Enhanced magnetic resonance imaging (MRI) is widely used in the diagnosis, treatment and prognosis of hepatocellular carcinoma (HCC), but it can not effectively reflect the heterogeneity within the tumor and evaluate the effect after treatment. Preoperative imaging analysis of voxel changes can effectively reflect the internal heterogeneity of the tumor and evaluate the progression-free survival (PFS).

AIM

To predict the PFS of patients with HCC before operation by building a model with enhanced MRI images.

METHODS

Delineate the regions of interest (ROI) in arterial phase, portal venous phase and delayed phase of enhanced MRI. After extracting the combinatorial features of ROI, the features are fused to obtain deep learning radiomics (DLR)_Sig. DeLong's test was used to evaluate the diagnostic performance of different typological features. K-M analysis was applied to assess PFS in different risk groups, and the discriminative ability of the model was evaluated using the C-index.

RESULTS

Tumor diameter and diolame were independent factors influencing the prognosis of PFS. Delong's test revealed multi-phase combined radiomic features had significantly greater area under the curve values than did those of the individual phases (P < 0.05).In deep transfer learning (DTL) and DLR, significant differences were observed between the multi-phase and individual phases feature sets (P < 0.05). K-M survival analysis revealed a median survival time of high risk group and low risk group was 12.8 and 14.2 months, respectively, and the predicted probabilities of 6 months, 1 year and 2 years were 92%, 60%, 40% and 98%, 90%,73%, respectively. The C-index was 0.764, indicating relatively good consistency between the predicted and observed results. DTL and DLR have higher predictive value for 2-year PFS in nomogram.

CONCLUSION

Based on the multi-temporal characteristics of enhanced MRI and the constructed Nomograph, it provides a new strategy for predicting the PFS of transarterial chemoembolization treatment of HCC.

Contrastive Learning vs. Self-Learning vs. Deformable Data Augmentation in Semantic Segmentation of Medical Images

To develop a robust segmentation model, encoding the underlying features/structures of the input data is essential to discriminate the target structure from the background. To enrich the extracted feature maps, contrastive learning and self-learning techniques are employed, particularly when the size of the training dataset is limited. In this work, we set out to investigate the impact of contrastive learning and self-learning on the performance of the deep learning-based semantic segmentation. To this end, three different datasets were employed used for brain tumor and hippocampus delineation from MR images (BraTS and Decathlon datasets, respectively) and kidney segmentation from CT images (Decathlon dataset). Since data augmentation techniques are also aimed at enhancing the performance of deep learning methods, a deformable data augmentation technique was proposed and compared with contrastive learning and self-learning frameworks. The segmentation accuracy for the three datasets was assessed with and without applying data augmentation, contrastive learning, and self-learning to individually investigate the impact of these techniques. The self-learning and deformable data augmentation techniques exhibited comparable performance with Dice indices of 0.913 ± 0.030 and 0.920 ± 0.022 for kidney segmentation, 0.890 ± 0.035 and 0.898 ± 0.027 for hippocampus segmentation, and 0.891 ± 0.045 and 0.897 ± 0.040 for lesion segmentation, respectively. These two approaches significantly outperformed the contrastive learning and the original model with Dice indices of 0.871 ± 0.039 and 0.868 ± 0.042 for kidney segmentation, 0.872 ± 0.045 and 0.865 ± 0.048 for hippocampus segmentation, and 0.870 ± 0.049 and 0.860 ± 0.058 for lesion segmentation, respectively. The combination of self-learning with deformable data augmentation led to a robust segmentation model with no outliers in the outcomes. This work demonstrated the beneficial impact of self-learning and deformable data augmentation on organ and lesion segmentation, where no additional training datasets are needed.

AI Applications to Breast MRI: Today and Tomorrow

In breast imaging, there is an unrelenting increase in the demand for breast imaging services, partly explained by continuous expanding imaging indications in breast diagnosis and treatment. As the human workforce providing these services is not growing at the same rate, the implementation of artificial intelligence (AI) in breast imaging has gained significant momentum to maximize workflow efficiency and increase productivity while concurrently improving diagnostic accuracy and patient outcomes. Thus far, the implementation of AI in breast imaging is at the most advanced stage with mammography and digital breast tomosynthesis techniques, followed by ultrasound, whereas the implementation of AI in breast magnetic resonance imaging (MRI) is not moving along as rapidly due to the complexity of MRI examinations and fewer available dataset. Nevertheless, there is persisting interest in AI-enhanced breast MRI applications, even as the use of and indications of breast MRI continue to expand. This review presents an overview of the basic concepts of AI imaging analysis and subsequently reviews the use cases for AI-enhanced MRI interpretation, that is, breast MRI triaging and lesion detection, lesion classification, prediction of treatment response, risk assessment, and image quality. Finally, it provides an outlook on the barriers and facilitators for the adoption of AI in breast MRI. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 6.

Automated Measurement of Ovary Development in Atlantic Salmon Using Deep Learning

OBJECTIVE

Salmon breeding companies control the egg stripping period through environmental change, which triggers the need to identify the state of maturation. Ultrasound imaging of the salmon ovary is a proven non-invasive tool for this purpose; however, the process is laborious, and the interpretation of the ultrasound scans is subjective. Real-time ultrasound image segmentation of Atlantic salmon ovary provides an opportunity to overcome these limitations. However, several application challenges need to be addressed to achieve this goal. These challenges include the potential for false-positive and false-negative predictions, accurate prediction of attenuated lower ovary parts and resolution of inconsistencies in predicted ovary shape.

METHODS

We describe an approach designed to tackle these obstacles by employing targeted pre-training of a modified U-Net, capable of performing both segmentation and classification. In addition, a variational autoencoder (VAE) and generative adversarial network (GAN) were incorporated to rectify shape inconsistencies in the segmentation output. To train the proposed model, a data set of Atlantic salmon ovaries throughout two maturation periods was recorded.

RESULTS

We then tested our model and compared its performance with that of conventional and novel U-Nets. The method was also tested in a salmon on-site ultrasound examination setting. The results of our application indicate that our method is able to efficiently segment salmon ovary with an average Dice score of 0.885 per individual in real-time.

CONCLUSION

These results represent a competitive performance for this specific application, which enables us to design an automated system for smart monitoring of maturation state in Atlantic salmon.

A 3D transfer learning approach for identifying multiple simultaneous errors during radiotherapy

Objective. Deep learning models, such as convolutional neural networks (CNNs), can take full dose comparison images as input and have shown promising results for error identification during treatment. Clinically, complex scenarios should be considered, with the risk of multiple anatomical and/or mechanical errors occurring simultaneously during treatment. The purpose of this study was to evaluate the capability of CNN-based error identification in this more complex scenario.Approach. For 40 lung cancer patients, clinically realistic ranges of combinations of various treatment errors within treatment plans and/or computed tomography (CT) images were simulated. Modified CT images and treatment plans were used to predict 2580 3D dose distributions, which were compared to dose distributions without errors using various gamma analysis criteria and relative dose difference as dose comparison methods. A 3D CNN capable of multilabel classification was trained to identify treatment errors at two classification levels, using dose comparison volumes as input: Level 1 (main error type, e.g. anatomical change, mechanical error) and Level 2 (error subtype, e.g. tumor regression, patient rotation). For training the CNNs, a transfer learning approach was employed. An ensemble model was also evaluated, which consisted of three separate CNNs each taking a region of interest of the dose comparison volume as input. Model performance was evaluated by calculating sample F1-scores for training and validation sets.Main results. The model had high F1-scores for Level 1 classification, but performance for Level 2 was lower, and overfitting became more apparent. Using relative dose difference instead of gamma volumes as input improved performance for Level 2 classification, whereas using an ensemble model additionally reduced overfitting. The models obtained F1-scores of 0.86 and 0.62 on an independent test set for Level 1 and Level 2, respectively.Significance. This study shows that it is possible to identify multiple errors occurring simultaneously in 3D dose verification data.

Diagnosis of Salivary Gland Tumors Using Transfer Learning with Fine-Tuning and Gradual Unfreezing

Ultrasound is the primary tool for evaluating salivary gland tumors (SGTs); however, tumor diagnosis currently relies on subjective features. This study aimed to establish an objective ultrasound diagnostic method using deep learning. We collected 446 benign and 223 malignant SGT ultrasound images in the training/validation set and 119 benign and 44 malignant SGT ultrasound images in the testing set. We trained convolutional neural network (CNN) models from scratch and employed transfer learning (TL) with fine-tuning and gradual unfreezing to classify malignant and benign SGTs. The diagnostic performances of these models were compared. By utilizing the pretrained ResNet50V2 with fine-tuning and gradual unfreezing, we achieved a 5-fold average validation accuracy of 0.920. The diagnostic performance on the testing set demonstrated an accuracy of 89.0%, a sensitivity of 81.8%, a specificity of 91.6%, a positive predictive value of 78.3%, and a negative predictive value of 93.2%. This performance surpasses that of other models in our study. The corresponding Grad-CAM visualizations were also presented to provide explanations for the diagnosis. This study presents an effective and objective ultrasound method for distinguishing between malignant and benign SGTs, which could assist in preoperative evaluation.

Sensor-Location-Specific Joint Acquisition of Peripheral Artery Bioimpedance and Photoplethysmogram for Wearable Applications

BACKGROUND

Cardiovascular diseases (CVDs), being the culprit for one-third of deaths globally, constitute a challenge for biomedical instrumentation development, especially for early disease detection. Pulsating arterial blood flow, providing access to cardiac-related parameters, involves the whole body. Unobtrusive and continuous acquisition of electrical bioimpedance (EBI) and photoplethysmography (PPG) constitute important techniques for monitoring the peripheral arteries, requiring novel approaches and clever means.

METHODS

In this work, five peripheral arteries were selected for EBI and PPG signal acquisition. The acquisition sites were evaluated based on the signal morphological parameters. A small-data-based deep learning model, which increases the data by dividing them into cardiac periods, was proposed to evaluate the continuity of the signals.

RESULTS

The highest sensitivity of EBI was gained for the carotid artery (0.86%), three times higher than that for the next best, the posterior tibial artery (0.27%). The excitation signal parameters affect the measured EBI, confirming the suitability of classical 100 kHz frequency (average probability of 52.35%). The continuity evaluation of the EBI signals confirmed the advantage of the carotid artery (59.4%), while the posterior tibial artery (49.26%) surpasses the radial artery (48.17%). The PPG signal, conversely, commends the location of the posterior tibial artery (97.87%).

CONCLUSIONS

The peripheral arteries are highly suitable for non-invasive EBI and PPG signal acquisition. The posterior tibial artery constitutes a candidate for the joint acquisition of EBI and PPG signals in sensor-fusion-based wearable devices-an important finding of this research.

Linear fine-tuning: a linear transformation based transfer strategy for deep MRI reconstruction

Introduction

Fine-tuning (FT) is a generally adopted transfer learning method for deep learning-based magnetic resonance imaging (MRI) reconstruction. In this approach, the reconstruction model is initialized with pre-trained weights derived from a source domain with ample data and subsequently updated with limited data from the target domain. However, the direct full-weight update strategy can pose the risk of "catastrophic forgetting" and overfitting, hindering its effectiveness. The goal of this study is to develop a zero-weight update transfer strategy to preserve pre-trained generic knowledge and reduce overfitting.

Methods

Based on the commonality between the source and target domains, we assume a linear transformation relationship of the optimal model weights from the source domain to the target domain. Accordingly, we propose a novel transfer strategy, linear fine-tuning (LFT), which introduces scaling and shifting (SS) factors into the pre-trained model. In contrast to FT, LFT only updates SS factors in the transfer phase, while the pre-trained weights remain fixed.

Results

To evaluate the proposed LFT, we designed three different transfer scenarios and conducted a comparative analysis of FT, LFT, and other methods at various sampling rates and data volumes. In the transfer scenario between different contrasts, LFT outperforms typical transfer strategies at various sampling rates and considerably reduces artifacts on reconstructed images. In transfer scenarios between different slice directions or anatomical structures, LFT surpasses the FT method, particularly when the target domain contains a decreasing number of training images, with a maximum improvement of up to 2.06 dB (5.89%) in peak signal-to-noise ratio.

Discussion

The LFT strategy shows great potential to address the issues of "catastrophic forgetting" and overfitting in transfer scenarios for MRI reconstruction, while reducing the reliance on the amount of data in the target domain. Linear fine-tuning is expected to shorten the development cycle of reconstruction models for adapting complicated clinical scenarios, thereby enhancing the clinical applicability of deep MRI reconstruction.

Extensive deep learning model to enhance electrocardiogram application via latent cardiovascular feature extraction from identity identification

BACKGROUND AND OBJECTIVE

Deep learning models (DLMs) have been successfully applied in biomedicine primarily using supervised learning with large, annotated databases. However, scarce training resources limit the potential of DLMs for electrocardiogram (ECG) analysis.

METHODS

We have developed a novel pre-training strategy for unsupervised identity identification with an area under the receiver operating characteristic curve (AUC) >0.98. Accordingly, a DLM pre-trained with identity identification can be applied to 70 patient characteristic predictions using transfer learning (TL). These ECG-based patient characteristics were then used for cardiovascular disease (CVD) risk prediction. The DLMs were trained using 507,729 ECGs from 222,473 patients and validated using two independent validation sets (n = 27,824/31,925).

RESULTS

The DLMs using our method exhibited better performance than directly trained DLMs. Additionally, our DLM performed better than those of previous studies in terms of gender (AUC [internal/external] = 0.982/0.968), age (correlation = 0.886/0.892), low ejection fraction (AUC = 0.942/0.951), and critical markers not addressed previously, including high B-type natriuretic peptide (AUC = 0.921/0.899). Additionally, approximately 50% of the ECG-based characteristics provided significantly more prediction information for cardiovascular risk than real characteristics.

CONCLUSIONS

This is the first study to use identity identification as a pre-training task for TL in ECG analysis. An extensive exploration of the relationship between ECG and 70 patient characteristics was conducted. Our DLM-enhanced ECG interpretation system extensively advanced ECG-related patient characteristic prediction and mortality risk management for cardiovascular diseases.

Differentiation of acute and chronic vertebral compression fractures using conventional CT based on deep transfer learning features and hand-crafted radiomics features

BACKGROUND

We evaluated the diagnostic efficacy of deep learning radiomics (DLR) and hand-crafted radiomics (HCR) features in differentiating acute and chronic vertebral compression fractures (VCFs).

METHODS

A total of 365 patients with VCFs were retrospectively analysed based on their computed tomography (CT) scan data. All patients completed MRI examination within 2 weeks. There were 315 acute VCFs and 205 chronic VCFs. Deep transfer learning (DTL) features and HCR features were extracted from CT images of patients with VCFs using DLR and traditional radiomics, respectively, and feature fusion was performed to establish the least absolute shrinkage and selection operator. The MRI display of vertebral bone marrow oedema was used as the gold standard for acute VCF, and the model performance was evaluated using the receiver operating characteristic (ROC).To separately evaluate the effectiveness of DLR, traditional radiomics and feature fusion in the differential diagnosis of acute and chronic VCFs, we constructed a nomogram based on the clinical baseline data to visualize the classification evaluation. The predictive power of each model was compared using the Delong test, and the clinical value of the nomogram was evaluated using decision curve analysis (DCA).

RESULTS

Fifty DTL features were obtained from DLR, 41 HCR features were obtained from traditional radiomics, and 77 features fusion were obtained after feature screening and fusion of the two. The area under the curve (AUC) of the DLR model in the training cohort and test cohort were 0.992 (95% confidence interval (CI), 0.983-0.999) and 0.871 (95% CI, 0.805-0.938), respectively. While the AUCs of the conventional radiomics model in the training cohort and test cohort were 0.973 (95% CI, 0.955-0.990) and 0.854 (95% CI, 0.773-0.934), respectively. The AUCs of the features fusion model in the training cohort and test cohort were 0.997 (95% CI, 0.994-0.999) and 0.915 (95% CI, 0.855-0.974), respectively. The AUCs of nomogram constructed by the features fusion in combination with clinical baseline data were 0.998 (95% CI, 0.996-0.999) and 0.946 (95% CI, 0.906-0.987) in the training cohort and test cohort, respectively. The Delong test showed that the differences between the features fusion model and the nomogram in the training cohort and the test cohort were not statistically significant (P values were 0.794 and 0.668, respectively), and the differences in the other prediction models in the training cohort and the test cohort were statistically significant (P < 0.05). DCA showed that the nomogram had high clinical value.

CONCLUSION

The features fusion model can be used for the differential diagnosis of acute and chronic VCFs, and its differential diagnosis ability is improved when compared with that when either radiomics is used alone. At the same time, the nomogram has a high predictive value for acute and chronic VCFs and can be a potential decision-making tool to assist clinicians, especially when a patient is unable to undergo spinal MRI examination.

A comprehensive survey of deep learning research on medical image analysis with focus on transfer learning

This survey aims to identify commonly used methods, datasets, future trends, knowledge gaps, constraints, and limitations in the field to provide an overview of current solutions used in medical image analysis in parallel with the rapid developments in transfer learning (TL). Unlike previous studies, this survey grouped the last five years of current studies for the period between January 2017 and February 2021 according to different anatomical regions and detailed the modality, medical task, TL method, source data, target data, and public or private datasets used in medical imaging. Also, it provides readers with detailed information on technical challenges, opportunities, and future research trends. In this way, an overview of recent developments is provided to help researchers to select the most effective and efficient methods and access widely used and publicly available medical datasets, research gaps, and limitations of the available literature.

Transfer learning as an AI-based solution to address limited datasets in space medicine

The advent of artificial intelligence (AI) has a promising role in the future long-duration spaceflight missions. Traditional AI algorithms rely on training and testing data from the same domain. However, astronaut medical data is naturally limited to a small sample size and often difficult to collect, leading to extremely limited datasets. This significantly limits the ability of traditional machine learning methodologies. Transfer learning is a potential solution to overcome this dataset size limitation and can help improve training time and performance of a neural networks. We discuss the unique challenges of space medicine in producing datasets and transfer learning as an emerging technique to address these issues.

Identifying Solitary Granulomatous Nodules from Solid Lung Adenocarcinoma: Exploring Robust Image Features with Cross-Domain Transfer Learning

PURPOSE

This study aimed to find suitable source domain data in cross-domain transfer learning to extract robust image features. Then, a model was built to preoperatively distinguish lung granulomatous nodules (LGNs) from lung adenocarcinoma (LAC) in solitary pulmonary solid nodules (SPSNs).

METHODS

Data from 841 patients with SPSNs from five centres were collected retrospectively. First, adaptive cross-domain transfer learning was used to construct transfer learning signatures (TLS) under different source domain data and conduct a comparative analysis. The Wasserstein distance was used to assess the similarity between the source domain and target domain data in cross-domain transfer learning. Second, a cross-domain transfer learning radiomics model (TLRM) combining the best performing TLS, clinical factors and subjective CT findings was constructed. Finally, the performance of the model was validated through multicentre validation cohorts.

RESULTS

Relative to other source domain data, TLS based on lung whole slide images as source domain data (TLS-LW) had the best performance in all validation cohorts (AUC range: 0.8228-0.8984). Meanwhile, the Wasserstein distance of TLS-LW was 1.7108, which was minimal. Finally, TLS-LW, age, spiculated sign and lobulated shape were used to build the TLRM. In all validation cohorts, The AUC ranges were 0.9074-0.9442. Compared with other models, decision curve analysis and integrated discrimination improvement showed that TLRM had better performance.

CONCLUSIONS

The TLRM could assist physicians in preoperatively differentiating LGN from LAC in SPSNs. Furthermore, compared with other images, cross-domain transfer learning can extract robust image features when using lung whole slide images as source domain data and has a better effect.

Classification of tumor from computed tomography images: A brain-inspired multisource transfer learning under probability distribution adaptation

Preoperative diagnosis of gastric cancer and primary gastric lymphoma is challenging and has important clinical significance. Inspired by the inductive reasoning learning of the human brain, transfer learning can improve diagnosis performance of target task by utilizing the knowledge learned from the other domains (source domain). However, most studies focus on single-source transfer learning and may lead to model performance degradation when a large domain shift exists between the single-source domain and target domain. By simulating the multi-modal information learning and transfer mechanism of human brain, this study designed a multisource transfer learning feature extraction and classification framework, which can enhance the prediction performance of the target model by using multisource medical data (domain). First, this manuscript designs a feature extraction network that takes the maximum mean difference based on the Wasserstein distance as an adaptive measure of probability distribution and extracts the domain-specific invariant representations between source and target domain data. Then, aiming at the random generation of parameters bringing uncertainties to prediction accuracy and generalization ability of extreme learning machine network, the 1-norm regularization is used to implement sparse constraints of the output weight matrix and improve the robustness of the model. Finally, some experiments are carried out on the data of two medical centers. The experimental results show that the area under curves (AUCs) of the method are 0.958 and 0.929 in the two validation cohorts, respectively. The method in this manuscript can provide doctors with a better diagnostic reference, which has certain practical significance.

A brain-like classification method for computed tomography images based on adaptive feature matching dual-source domain heterogeneous transfer learning

Transfer learning can improve the robustness of deep learning in the case of small samples. However, when the semantic difference between the source domain data and the target domain data is large, transfer learning easily introduces redundant features and leads to negative transfer. According the mechanism of the human brain focusing on effective features while ignoring redundant features in recognition tasks, a brain-like classification method based on adaptive feature matching dual-source domain heterogeneous transfer learning is proposed for the preoperative aided diagnosis of lung granuloma and lung adenocarcinoma for patients with solitary pulmonary solid nodule in the case of small samples. The method includes two parts: (1) feature extraction and (2) feature classification. In the feature extraction part, first, By simulating the feature selection mechanism of the human brain in the process of drawing inferences about other cases from one instance, an adaptive selected-based dual-source domain feature matching network is proposed to determine the matching weight of each pair of feature maps and each pair of convolution layers between the two source networks and the target network, respectively. These two weights can, respectively, adaptive select the features in the source network that are conducive to the learning of the target task, and the destination of feature transfer to improve the robustness of the target network. Meanwhile, a target network based on diverse branch block is proposed, which made the target network have different receptive fields and complex paths to further improve the feature expression ability of the target network. Second, the convolution kernel of the target network is used as the feature extractor to extract features. In the feature classification part, an ensemble classifier based on sparse Bayesian extreme learning machine is proposed that can automatically decide how to combine the output of base classifiers to improve the classification performance. Finally, the experimental results (the AUCs were 0.9542 and 0.9356, respectively) on the data of two center data show that this method can provide a better diagnostic reference for doctors.

Development and inter-institutional validation of an automatic vertebral-body misalignment error detector for Cone-Beam CT guided radiotherapy

BACKGROUND

In Cone-Beam Computed Tomography (CBCT) guided radiotherapy, off-by-one vertebral body misalignments are rare but serious errors which lead to wrong-site treatments.

PURPOSE

An automatic error detection algorithm was developed that uses a three-branch convolutional neural network error detection model (EDM) to detect off-by-one vertebral body misalignments using planning computed tomography (CT) images and setup CBCT images.

METHODS

Algorithm training and test data consisted of planning CTs and CBCTs from 480 patients undergoing radiotherapy treatment in the thoracic and abdominal regions at two radiotherapy clinics. The clinically applied registration was used to derive true-negative (no error) data. The setup and planning images were then misaligned by one vertebral body in both the superior and inferior directions, simulating the most likely misalignment scenarios. For each of the aligned and misaligned 3D image pairs, 2D slice pairs were automatically extracted in each anatomical plane about a point within the vertebral column. The three slice pairs obtained were then inputted to the EDM which returned a probability of vertebral misalignment. One model (EDM₁ ) was trained solely on data from institution #1. EDM₁ was further trained using a lower learning rate on a dataset from institution #2 to produce a fine-tuned model, EDM₂ . Another model, EDM₃ , was trained from scratch using a training dataset composed of data from both institutions. These three models were validated on a randomly selected and unseen dataset composed of images from both institutions, for a total of 303 image pairs. The model performances were quantified using a receiver operating characteristic analysis. Due to the rarity of vertebral body misalignments in the clinic, a minimum threshold value yielding a specificity of at least 99% was selected. Using this threshold, the sensitivity was calculated for each model, on each institution's test set separately.

RESULTS

When applied to the combined test set, EDM₁ , EDM₂ , and EDM₃ resulted in an area under curve of 99.5%, 99.4% and 99.5%, respectively. EDM₁ achieved a sensitivity of 96% and 88% on Institution #1 and Institution #2 test set, respectively. EDM₂ obtained a sensitivity of 95% on each institution's test set. EDM₃ achieved a sensitivity of 95% and 88% on Institution #1 and Institution #2 test set, respectively.

CONCLUSION

The proposed algorithm demonstrated accuracy in identifying off-by-one vertebral body misalignments in CBCT-guided radiotherapy that was sufficiently high to allow for practical implementation. It was found that fine-tuning the model on a multi-facility dataset can further enhance the generalizability of the algorithm. This article is protected by copyright. All rights reserved.

A deep learning and Monte Carlo based framework for bioluminescence imaging center of mass-guided glioblastoma targeting

Objective. Bioluminescence imaging (BLI) is a valuable tool for non-invasive monitoring of glioblastoma multiforme (GBM) tumor-bearing small animals without incurring x-ray radiation burden. However, the use of this imaging modality is limited due to photon scattering and lack of spatial information. Attempts at reconstructing bioluminescence tomography (BLT) using mathematical models of light propagation show limited progress. Approach. This paper employed a different approach by using a deep convolutional neural network (CNN) to predict the tumor’s center of mass (CoM). Transfer-learning with a sizeable artificial database is employed to facilitate the training process for, the much smaller, target database including Monte Carlo (MC) simulations of real orthotopic glioblastoma models. Predicted CoM was then used to estimate a BLI-based planning target volume (bPTV), by using the CoM as the center of a sphere, encompassing the tumor. The volume of the encompassing target sphere was estimated based on the total number of photons reaching the skin surface. Main results. Results show sub-millimeter accuracy for CoM prediction with a median error of 0.59 mm. The proposed method also provides promising performance for BLI-based tumor targeting with on average 94% of the tumor inside the bPTV while keeping the average healthy tissue coverage below 10%. Significance. This work introduced a framework for developing and using a CNN for targeted radiation studies for GBM based on BLI. The framework will enable biologists to use BLI as their main image-guidance tool to target GBM tumors in rat models, avoiding delivery of high x-ray imaging dose to the animals.

Machine Learning Prediction Models for Neurodevelopmental Outcome After Preterm Birth: A Scoping Review and New Machine Learning Evaluation Framework

BACKGROUND AND OBJECTIVES

Outcome prediction of preterm birth is important for neonatal care, yet prediction performance using conventional statistical models remains insufficient. Machine learning has a high potential for complex outcome prediction. In this scoping review, we provide an overview of the current applications of machine learning models in the prediction of neurodevelopmental outcomes in preterm infants, assess the quality of the developed models, and provide guidance for future application of machine learning models to predict neurodevelopmental outcomes of preterm infants.

METHODS

A systematic search was performed using PubMed. Studies were included if they reported on neurodevelopmental outcome prediction in preterm infants using predictors from the neonatal period and applying machine learning techniques. Data extraction and quality assessment were independently performed by 2 reviewers.

RESULTS

Fourteen studies were included, focusing mainly on very or extreme preterm infants, predicting neurodevelopmental outcome before age 3 years, and mostly assessing outcomes using the Bayley Scales of Infant Development. Predictors were most often based on MRI. The most prevalent machine learning techniques included linear regression and neural networks. None of the studies met all newly developed quality assessment criteria. Studies least prone to inflated performance showed promising results, with areas under the curve up to 0.86 for classification and R2 values up to 91% in continuous prediction. A limitation was that only 1 data source was used for the literature search.

CONCLUSIONS

Studies least prone to inflated prediction results are the most promising. The provided evaluation framework may contribute to improved quality of future machine learning models.

Transfer learning for medical image classification: a literature review

BACKGROUND

Transfer learning (TL) with convolutional neural networks aims to improve performances on a new task by leveraging the knowledge of similar tasks learned in advance. It has made a major contribution to medical image analysis as it overcomes the data scarcity problem as well as it saves time and hardware resources. However, transfer learning has been arbitrarily configured in the majority of studies. This review paper attempts to provide guidance for selecting a model and TL approaches for the medical image classification task.

METHODS

425 peer-reviewed articles were retrieved from two databases, PubMed and Web of Science, published in English, up until December 31, 2020. Articles were assessed by two independent reviewers, with the aid of a third reviewer in the case of discrepancies. We followed the PRISMA guidelines for the paper selection and 121 studies were regarded as eligible for the scope of this review. We investigated articles focused on selecting backbone models and TL approaches including feature extractor, feature extractor hybrid, fine-tuning and fine-tuning from scratch.

RESULTS

The majority of studies (n = 57) empirically evaluated multiple models followed by deep models (n = 33) and shallow (n = 24) models. Inception, one of the deep models, was the most employed in literature (n = 26). With respect to the TL, the majority of studies (n = 46) empirically benchmarked multiple approaches to identify the optimal configuration. The rest of the studies applied only a single approach for which feature extractor (n = 38) and fine-tuning from scratch (n = 27) were the two most favored approaches. Only a few studies applied feature extractor hybrid (n = 7) and fine-tuning (n = 3) with pretrained models.

CONCLUSION

The investigated studies demonstrated the efficacy of transfer learning despite the data scarcity. We encourage data scientists and practitioners to use deep models (e.g. ResNet or Inception) as feature extractors, which can save computational costs and time without degrading the predictive power.

A Transfer Learning Radiomics Nomogram for Preoperative Prediction of Borrmann Type IV Gastric Cancer From Primary Gastric Lymphoma

Objective

This study aims to differentiate preoperative Borrmann type IV gastric cancer (GC) from primary gastric lymphoma (PGL) by transfer learning radiomics nomogram (TLRN) with whole slide images of GC as source domain data.

Materials and Methods

This study retrospectively enrolled 438 patients with histopathologic diagnoses of Borrmann type IV GC and PGL. They received CT examinations from three hospitals. Quantitative transfer learning features were extracted by the proposed transfer learning radiopathomic network and used to construct transfer learning radiomics signatures (TLRS). A TLRN, which integrates TLRS, clinical factors, and CT subjective findings, was developed by multivariate logistic regression. The diagnostic TLRN performance was assessed by clinical usefulness in the independent validation set.

Results

The TLRN was built by TLRS and a high enhanced serosa sign, which showed good agreement by the calibration curve. The TLRN performance was superior to the clinical model and TLRS. Its areas under the curve (AUC) were 0.958 (95% confidence interval [CI], 0.883–0.991), 0.867 (95% CI, 0.794–0.922), and 0.921 (95% CI, 0.860–0.960) in the internal and two external validation cohorts, respectively. Decision curve analysis (DCA) showed that the TLRN was better than any other model. TLRN has potential generalization ability, as shown in the stratification analysis.

Conclusions

The proposed TLRN based on gastric WSIs may help preoperatively differentiate PGL from Borrmann type IV GC.

Borrmann type IV gastric cancer, primary gastric lymphoma, transfer learning, whole slide image, deep learning.

Application of Artificial Intelligence in Diagnosis of Craniopharyngioma

Craniopharyngioma is a congenital brain tumor with clinical characteristics of hypothalamic-pituitary dysfunction, increased intracranial pressure, and visual field disorder, among other injuries. Its clinical diagnosis mainly depends on radiological examinations (such as Computed Tomography, Magnetic Resonance Imaging). However, assessing numerous radiological images manually is a challenging task, and the experience of doctors has a great influence on the diagnosis result. The development of artificial intelligence has brought about a great transformation in the clinical diagnosis of craniopharyngioma. This study reviewed the application of artificial intelligence technology in the clinical diagnosis of craniopharyngioma from the aspects of differential classification, prediction of tissue invasion and gene mutation, prognosis prediction, and so on. Based on the reviews, the technical route of intelligent diagnosis based on the traditional machine learning model and deep learning model were further proposed. Additionally, in terms of the limitations and possibilities of the development of artificial intelligence in craniopharyngioma diagnosis, this study discussed the attentions required in future research, including few-shot learning, imbalanced data set, semi-supervised models, and multi-omics fusion.

Segmentation and classification on chest radiography: a systematic survey

Chest radiography (X-ray) is the most common diagnostic method for pulmonary disorders. A trained radiologist is required for interpreting the radiographs. But sometimes, even experienced radiologists can misinterpret the findings. This leads to the need for computer-aided detection diagnosis. For decades, researchers were automatically detecting pulmonary disorders using the traditional computer vision (CV) methods. Now the availability of large annotated datasets and computing hardware has made it possible for deep learning to dominate the area. It is now the modus operandi for feature extraction, segmentation, detection, and classification tasks in medical imaging analysis. This paper focuses on the research conducted using chest X-rays for the lung segmentation and detection/classification of pulmonary disorders on publicly available datasets. The studies performed using the Generative Adversarial Network (GAN) models for segmentation and classification on chest X-rays are also included in this study. GAN has gained the interest of the CV community as it can help with medical data scarcity. In this study, we have also included the research conducted before the popularity of deep learning models to have a clear picture of the field. Many surveys have been published, but none of them is dedicated to chest X-rays. This study will help the readers to know about the existing techniques, approaches, and their significance.

Deep learning segmentation of glomeruli on kidney donor frozen sections

Purpose: Recent advances in computational image analysis offer the opportunity to develop automatic quantification of histologic parameters as aid tools for practicing pathologists. We aim to develop deep learning (DL) models to quantify nonsclerotic and sclerotic glomeruli on frozen sections from donor kidney biopsies. Approach: A total of 258 whole slide images (WSI) from cadaveric donor kidney biopsies performed at our institution ( n = 123 ) and at external institutions ( n = 135 ) were used in this study. WSIs from our institution were divided at the patient level into training and validation datasets (ratio: 0.8:0.2), and external WSIs were used as an independent testing dataset. Nonsclerotic ( n = 22767 ) and sclerotic ( n = 1366 ) glomeruli were manually annotated by study pathologists on all WSIs. A nine-layer convolutional neural network based on the common U-Net architecture was developed and tested for the segmentation of nonsclerotic and sclerotic glomeruli. DL-derived, manual segmentation, and reported glomerular count (standard of care) were compared. Results: The average Dice similarity coefficient testing was 0.90 and 0.83. And the F 1 , recall, and precision scores were 0.93, 0.96, and 0.90, and 0.87, 0.93, and 0.81, for nonsclerotic and sclerotic glomeruli, respectively. DL-derived and manual segmentation-derived glomerular counts were comparable, but statistically different from reported glomerular count. Conclusions: DL segmentation is a feasible and robust approach for automatic quantification of glomeruli. We represent the first step toward new protocols for the evaluation of donor kidney biopsies.