Accurate and Feasible Deep Learning Based Semi-Automatic Segmentation in CT for Radiomics Analysis in Pancreatic Neuroendocrine Neoplasms

A novel model for predicting postoperative liver metastasis in R0 resected pancreatic neuroendocrine tumors: integrating computational pathology and deep learning-radiomics

BACKGROUND

Postoperative liver metastasis significantly impacts the prognosis of pancreatic neuroendocrine tumor (panNET) patients after R0 resection. Combining computational pathology and deep learning radiomics can enhance the detection of postoperative liver metastasis in panNET patients.

METHODS

Clinical data, pathology slides, and radiographic images were collected from 163 panNET patients post-R0 resection at Fudan University Shanghai Cancer Center (FUSCC) and FUSCC Pathology Consultation Center. Digital image analysis and deep learning identified liver metastasis-related features in Ki67-stained whole slide images (WSIs) and enhanced CT scans to create a nomogram. The model's performance was validated in both internal and external test cohorts.

RESULTS

Multivariate logistic regression identified nerve infiltration as an independent risk factor for liver metastasis (p < 0.05). The Pathomics score, which was based on a hotspot and the heterogeneous distribution of Ki67 staining, showed improved predictive accuracy for liver metastasis (AUC = 0.799). The deep learning-radiomics (DLR) score achieved an AUC of 0.875. The integrated nomogram, which combines clinical, pathological, and imaging features, demonstrated outstanding performance, with an AUC of 0.985 in the training cohort and 0.961 in the validation cohort. High-risk group had a median recurrence-free survival of 28.5 months compared to 34.7 months for the low-risk group, showing significant correlation with prognosis (p < 0.05).

CONCLUSION

A new predictive model that integrates computational pathologic scores and deep learning-radiomics can better predict postoperative liver metastasis in panNET patients, aiding clinicians in developing personalized treatments.

Advancing musculoskeletal tumor diagnosis: Automated segmentation and predictive classification using deep learning and radiomics

OBJECTIVES

Musculoskeletal (MSK) tumors, given their high mortality rate and heterogeneity, necessitate precise examination and diagnosis to guide clinical treatment effectively. Magnetic resonance imaging (MRI) is pivotal in detecting MSK tumors, as it offers exceptional image contrast between bone and soft tissue. This study aims to enhance the speed of detection and the diagnostic accuracy of MSK tumors through automated segmentation and grading utilizing MRI.

MATERIALS AND METHODS

The research included 170 patients (mean age, 58 years ±12 (standard deviation), 84 men) with MSK lesions, who underwent MRI scans from April 2021 to May 2023. We proposed a deep learning (DL) segmentation model MSAPN based on multi-scale attention and pixel-level reconstruction, and compared it with existing algorithms. Using MSAPN-segmented lesions to extract their radiomic features for the benign and malignant classification of tumors.

RESULTS

Compared to the most advanced segmentation algorithms, MSAPN demonstrates better performance. The Dice similarity coefficients (DSC) are 0.871 and 0.815 in the testing set and independent validation set, respectively. The radiomics model for classifying benign and malignant lesions achieves an accuracy of 0.890. Moreover, there is no statistically significant difference between the radiomics model based on manual segmentation and MSAPN segmentation.

CONCLUSION

This research contributes to the advancement of MSK tumor diagnosis through automated segmentation and predictive classification. The integration of DL algorithms and radiomics shows promising results, and the visualization analysis of feature maps enhances clinical interpretability.

PST-Radiomics: a PET/CT lymphoma classification method based on pseudo spatial-temporal radiomic features and structured atrous recurrent convolutional neural network

Objective.Existing radiomic methods tend to treat each isolated tumor as an inseparable whole, when extracting radiomic features. However, they may discard the critical intra-tumor metabolic heterogeneity (ITMH) information, that contributes to triggering tumor subtypes. To improve lymphoma classification performance, we propose a pseudo spatial-temporal radiomic method (PST-Radiomics) based on positron emission tomography computed tomography (PET/CT).Approach.Specifically, to enable exploitation of ITMH, we first present a multi-threshold gross tumor volume sequence (GTVS). Next, we extract 1D radiomic features based on PET images and each volume in GTVS and create a pseudo spatial-temporal feature sequence (PSTFS) tightly interwoven with ITMH. Then, we reshape PSTFS to create 2D pseudo spatial-temporal feature maps (PSTFM), of which the columns are elements of PSTFS. Finally, to learn from PSTFM in an end-to-end manner, we build a light-weighted pseudo spatial-temporal radiomic network (PSTR-Net), in which a structured atrous recurrent convolutional neural network serves as a PET branch to better exploit the strong local dependencies in PSTFM, and a residual convolutional neural network is used as a CT branch to exploit conventional radiomic features extracted from CT volumes.Main results.We validate PST-Radiomics based on a PET/CT lymphoma subtype classification task. Experimental results quantitatively demonstrate the superiority of PST-Radiomics, when compared to existing radiomic methods.Significance.Feature map visualization of our method shows that it performs complex feature selection while extracting hierarchical feature maps, which qualitatively demonstrates its superiority.

Pancreatic Ductal Adenocarcinoma: Update of CT-Based Radiomics Applications in the Pre-Surgical Prediction of the Risk of Post-Operative Fistula, Resectability Status and Prognosis

BACKGROUND

Pancreatic ductal adenocarcinoma (PDAC) is the seventh leading cause of cancer-related deaths worldwide. Surgical resection is the main driver to improving survival in resectable tumors, while neoadjuvant treatment based on chemotherapy (and radiotherapy) is the best option-treatment for a non-primally resectable disease. CT-based imaging has a central role in detecting, staging, and managing PDAC. As several authors have proposed radiomics for risk stratification in patients undergoing surgery for PADC, in this narrative review, we have explored the actual fields of interest of radiomics tools in PDAC built on pre-surgical imaging and clinical variables, to obtain more objective and reliable predictors.

METHODS

The PubMed database was searched for papers published in the English language no earlier than January 2018.

RESULTS

We found 301 studies, and 11 satisfied our research criteria. Of those included, four were on resectability status prediction, three on preoperative pancreatic fistula (POPF) prediction, and four on survival prediction. Most of the studies were retrospective.

CONCLUSIONS

It is possible to conclude that many performing models have been developed to get predictive information in pre-surgical evaluation. However, all the studies were retrospective, lacking further external validation in prospective and multicentric cohorts. Furthermore, the radiomics models and the expression of results should be standardized and automatized to be applicable in clinical practice.

Radiomics Prediction of Muscle Invasion in Bladder Cancer Using Semi-Automatic Lesion Segmentation of MRI Compared with Manual Segmentation

Conventional radiomics analysis requires the manual segmentation of lesions, which is time-consuming and subjective. This study aimed to assess the feasibility of predicting muscle invasion in bladder cancer (BCa) with radiomics using a semi-automatic lesion segmentation method on T2-weighted images. Cases of non-muscle-invasive BCa (NMIBC) and muscle-invasive BCa (MIBC) were pathologically identified in a training cohort and in internal and external validation cohorts. For bladder tumor segmentation, a deep learning-based semi-automatic model was constructed, while manual segmentation was performed by a radiologist. Semi-automatic and manual segmentation results were respectively used in radiomics analyses to distinguish NMIBC from MIBC. An equivalence test was used to compare the models' performance. The mean Dice similarity coefficients of the semi-automatic segmentation method were 0.836 and 0.801 in the internal and external validation cohorts, respectively. The area under the receiver operating characteristic curve (AUC) were 1.00 (0.991) and 0.892 (0.894) for the semi-automated model (manual) on the internal and external validation cohort, respectively (both p < 0.05). The average total processing time for semi-automatic segmentation was significantly shorter than that for manual segmentation (35 s vs. 92 s, p < 0.001). The BCa radiomics model based on semi-automatic segmentation method had a similar diagnostic performance as that of manual segmentation, while being less time-consuming and requiring fewer manual interventions.

Predicting Breast Cancer Subtypes Using Magnetic Resonance Imaging Based Radiomics With Automatic Segmentation

OBJECTIVE

The aim of the study is to demonstrate whether radiomics based on an automatic segmentation method is feasible for predicting molecular subtypes.

METHODS

This retrospective study included 516 patients with confirmed breast cancer. An automatic segmentation-3-dimensional UNet-based Convolutional Neural Networks, trained on our in-house data set-was applied to segment the regions of interest. A set of 1316 radiomics features per region of interest was extracted. Eighteen cross-combination radiomics methods-with 6 feature selection methods and 3 classifiers-were used for model selection. Model classification performance was assessed using the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, and specificity.

RESULTS

The average dice similarity coefficient value of the automatic segmentation was 0.89. The radiomics models were predictive of 4 molecular subtypes with the best average: AUC = 0.8623, accuracy = 0.6596, sensitivity = 0.6383, and specificity = 0.8775. For luminal versus nonluminal subtypes, AUC = 0.8788 (95% confidence interval [CI], 0.8505-0.9071), accuracy = 0.7756, sensitivity = 0.7973, and specificity = 0.7466. For human epidermal growth factor receptor 2 (HER2)-enriched versus non-HER2-enriched subtypes, AUC = 0.8676 (95% CI, 0.8370-0.8982), accuracy = 0.7737, sensitivity = 0.8859, and specificity = 0.7283. For triple-negative breast cancer versus non-triple-negative breast cancer subtypes, AUC = 0.9335 (95% CI, 0.9027-0.9643), accuracy = 0.9110, sensitivity = 0.4444, and specificity = 0.9865.

CONCLUSIONS

Radiomics based on automatic segmentation of magnetic resonance imaging can predict breast cancer of 4 molecular subtypes noninvasively and is potentially applicable in large samples.

Boundary-Sensitive Loss Function With Location Constraint for Hard Region Segmentation

In computer-aided diagnosis and treatment planning, accurate segmentation of medical images plays an essential role, especially for some hard regions including boundaries, small objects and background interference. However, existing segmentation loss functions including distribution-, region- and boundary-based losses cannot achieve satisfactory performances on these hard regions. In this paper, a boundary-sensitive loss function with location constraint is proposed for hard region segmentation in medical images, which provides three advantages: i) our Boundary-Sensitive loss (BS-loss) can automatically pay more attention to the hard-to-segment boundaries (e.g., thin structures and blurred boundaries), thus obtaining finer object boundaries; ii) BS-loss also can adjust its attention to small objects during training to segment them more accurately; and iii) our location constraint can alleviate the negative impact of the background interference, through the distribution matching of pixels between prediction and Ground Truth (GT) along each axis. By resorting to the proposed BS-loss and location constraint, the hard regions in both foreground and background are considered. Experimental results on three public datasets demonstrate the superiority of our method. Specifically, compared to the second-best method tested in this study, our method improves performance on hard regions in terms of Dice similarity coefficient (DSC) and 95% Hausdorff distance (95%HD) of up to 4.17% and 73% respectively. In addition, it also achieves the best overall segmentation performance. Hence, we can conclude that our method can accurately segment these hard regions and improve the overall segmentation performance in medical images.

Deep learning radiomic analysis of DCE-MRI combined with clinical characteristics predicts pathological complete response to neoadjuvant chemotherapy in breast cancer

Objective

The aim of this study was to develop and validate a deep learning-based radiomic (DLR) model combined with clinical characteristics for predicting pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) in breast cancer. For early prediction of pCR, the DLR model was based on pre-treatment and early treatment dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data.

Materials and methods

This retrospective study included 95 women (mean age, 48.1 years; range, 29-77 years) who underwent DCE-MRI before (pre-treatment) and after two or three cycles of NAC (early treatment) from 2018 to 2021. The patients in this study were randomly divided into a training cohort (n=67) and a validation cohort (n=28) at a ratio of 7:3. Deep learning and handcrafted features were extracted from pre- and early treatment DCE-MRI contoured lesions. These features contribute to the construction of radiomic signature RS1 and RS2 representing information from different periods. Mutual information and least absolute shrinkage and selection operator regression were used for feature selection. A combined model was then developed based on the DCE-MRI features and clinical characteristics. The performance of the models was assessed using the area under the receiver operating characteristic curve (AUC) and compared using the DeLong test.

Results

The overall pCR rate was 25.3% (24/95). One radiomic feature and three deep learning features in RS1, five radiomic features and 11 deep learning features in RS2, and five clinical characteristics remained in the feature selection. The performance of the DLR model combining pre- and early treatment information (AUC=0.900) was better than that of RS1 (AUC=0.644, P=0.068) and slightly higher that of RS2 (AUC=0.888, P=0.604) in the validation cohort. The combined model including pre- and early treatment information and clinical characteristics showed the best ability with an AUC of 0.925 in the validation cohort.

Conclusion

The combined model integrating pre-treatment, early treatment DCE-MRI data, and clinical characteristics showed good performance in predicting pCR to NAC in patients with breast cancer. Early treatment DCE-MRI and clinical characteristics may play an important role in evaluating the outcomes of NAC by predicting pCR.

A Semi-Unsupervised Segmentation Methodology Based on Texture Recognition for Radiomics: A Preliminary Study on Brain Tumours

Because of the intrinsic anatomic complexity of the brain structures, brain tumors have a high mortality and disability rate, and an early diagnosis is mandatory to contain damages. The commonly used biopsy is the diagnostic gold standard method, but it is invasive and, due to intratumoral heterogeneity, biopsies may lead to an incorrect result. Moreover, some tumors cannot be resectable if located in critical eloquent areas. On the other hand, medical imaging procedures can evaluate the entire tumor in a non-invasive and reproducible way. Radiomics is an emerging diagnosis technique based on quantitative medical image analyses, which makes use of data provided by non-invasive diagnosis techniques such as X-ray, computer-tomography (CT), magnetic resonance (MR), and proton emission tomography (PET). Radiomics techniques require the comprehensive analysis of huge numbers of medical images to extract a large and useful number of phenotypic features (usually called radiomics biomarkers). The goal is to explore and obtain the associations between features of tumors, diagnosis and patients’ prognoses to choose the best treatments and maximize the patient’s survival rate. Current radiomics techniques are not standardized in term of segmentation, feature extraction, and selection, moreover, the decision on suitable therapies still requires the supervision of an expert doctor. In this paper, we propose a semi-automatic methodology aimed to help the identification and segmentation of malignant tissues by using the combination of binary texture recognition, growing area algorithm, and machine learning techniques. In particular, the proposed method not only helps to better identify pathologic tissues but also permits to analyze in a fast way the huge amount of data, in Dicom format, provided by non-invasive diagnostic techniques. A preliminary experimental assessment has been conducted, considering a real MRI database of brain tumors. The method has been compared with the segmentation software’s tools “slicer 3D”. The obtained results are quite promising and demonstrate the potentialities of the proposed semi-unsupervised segmentation methodology.

Artificial Intelligence and Machine Learning in the Diagnosis and Management of Gastroenteropancreatic Neuroendocrine Neoplasms—A Scoping Review

Neuroendocrine neoplasms (NENs) and tumors (NETs) are rare neoplasms that may affect any part of the gastrointestinal system. In this scoping review, we attempt to map existing evidence on the role of artificial intelligence, machine learning and deep learning in the diagnosis and management of NENs of the gastrointestinal system. After implementation of inclusion and exclusion criteria, we retrieved 44 studies with 53 outcome analyses. We then classified the papers according to the type of studied NET (26 Pan-NETs, 59.1%; 3 metastatic liver NETs (6.8%), 2 small intestinal NETs, 4.5%; colorectal, rectal, non-specified gastroenteropancreatic and non-specified gastrointestinal NETs had from 1 study each, 2.3%). The most frequently used AI algorithms were Supporting Vector Classification/Machine (14 analyses, 29.8%), Convolutional Neural Network and Random Forest (10 analyses each, 21.3%), Random Forest (9 analyses, 19.1%), Logistic Regression (8 analyses, 17.0%), and Decision Tree (6 analyses, 12.8%). There was high heterogeneity on the description of the prediction model, structure of datasets, and performance metrics, whereas the majority of studies did not report any external validation set. Future studies should aim at incorporating a uniform structure in accordance with existing guidelines for purposes of reproducibility and research quality, which are prerequisites for integration into clinical practice.

A systematic review on application of deep learning in digestive system image processing

With the advent of the big data era, the application of artificial intelligence represented by deep learning in medicine has become a hot topic. In gastroenterology, deep learning has accomplished remarkable accomplishments in endoscopy, imageology, and pathology. Artificial intelligence has been applied to benign gastrointestinal tract lesions, early cancer, tumors, inflammatory bowel diseases, livers, pancreas, and other diseases. Computer-aided diagnosis significantly improve diagnostic accuracy and reduce physicians' workload and provide a shred of evidence for clinical diagnosis and treatment. In the near future, artificial intelligence will have high application value in the field of medicine. This paper mainly summarizes the latest research on artificial intelligence in diagnosing and treating digestive system diseases and discussing artificial intelligence's future in digestive system diseases. We sincerely hope that our work can become a stepping stone for gastroenterologists and computer experts in artificial intelligence research and facilitate the application and development of computer-aided image processing technology in gastroenterology.