Deep learning and medical imaging

Deep learning in magnetic resonance enterography for Crohn's disease assessment: a systematic review

Crohn's disease (CD) poses significant morbidity, underscoring the need for effective, non-invasive inflammatory assessment using magnetic resonance enterography (MRE). This literature review evaluates recent publications on the role of deep learning in improving MRE for CD assessment. We searched MEDLINE/PUBMED for studies that reported the use of deep learning algorithms for assessment of CD activity. The study was conducted according to the PRISMA guidelines. The risk of bias was evaluated using the QUADAS-2 tool. Five eligible studies, encompassing 468 subjects, were identified. Our study suggests that diverse deep learning applications, including image quality enhancement, bowel segmentation for disease burden quantification, and 3D reconstruction for surgical planning are useful and promising for CD assessment. However, most of the studies are preliminary, retrospective studies, and have a high risk of bias in at least one category. Future research is needed to assess how deep learning can impact CD patient diagnostics, particularly when considering the increasing integration of such models into hospital systems.

Assessing GPT-4 multimodal performance in radiological image analysis

OBJECTIVES

This study aims to assess the performance of a multimodal artificial intelligence (AI) model capable of analyzing both images and textual data (GPT-4V), in interpreting radiological images. It focuses on a range of modalities, anatomical regions, and pathologies to explore the potential of zero-shot generative AI in enhancing diagnostic processes in radiology.

METHODS

We analyzed 230 anonymized emergency room diagnostic images, consecutively collected over 1 week, using GPT-4V. Modalities included ultrasound (US), computerized tomography (CT), and X-ray images. The interpretations provided by GPT-4V were then compared with those of senior radiologists. This comparison aimed to evaluate the accuracy of GPT-4V in recognizing the imaging modality, anatomical region, and pathology present in the images.

RESULTS

GPT-4V identified the imaging modality correctly in 100% of cases (221/221), the anatomical region in 87.1% (189/217), and the pathology in 35.2% (76/216). However, the model's performance varied significantly across different modalities, with anatomical region identification accuracy ranging from 60.9% (39/64) in US images to 97% (98/101) and 100% (52/52) in CT and X-ray images (p < 0.001). Similarly, pathology identification ranged from 9.1% (6/66) in US images to 36.4% (36/99) in CT and 66.7% (34/51) in X-ray images (p < 0.001). These variations indicate inconsistencies in GPT-4V's ability to interpret radiological images accurately.

CONCLUSION

While the integration of AI in radiology, exemplified by multimodal GPT-4, offers promising avenues for diagnostic enhancement, the current capabilities of GPT-4V are not yet reliable for interpreting radiological images. This study underscores the necessity for ongoing development to achieve dependable performance in radiology diagnostics.

CLINICAL RELEVANCE STATEMENT

Although GPT-4V shows promise in radiological image interpretation, its high diagnostic hallucination rate (> 40%) indicates it cannot be trusted for clinical use as a standalone tool. Improvements are necessary to enhance its reliability and ensure patient safety.

KEY POINTS

GPT-4V's capability in analyzing images offers new clinical possibilities in radiology. GPT-4V excels in identifying imaging modalities but demonstrates inconsistent anatomy and pathology detection. Ongoing AI advancements are necessary to enhance diagnostic reliability in radiological applications.

Automatic detection and visualization of temporomandibular joint effusion with deep neural network

This study investigated the usefulness of deep learning-based automatic detection of temporomandibular joint (TMJ) effusion using magnetic resonance imaging (MRI) in patients with temporomandibular disorder and whether the diagnostic accuracy of the model improved when patients' clinical information was provided in addition to MRI images. The sagittal MR images of 2948 TMJs were collected from 1017 women and 457 men (mean age 37.19 ± 18.64 years). The TMJ effusion diagnostic performances of three convolutional neural networks (scratch, fine-tuning, and freeze schemes) were compared with those of human experts based on areas under the curve (AUCs) and diagnosis accuracies. The fine-tuning model with proton density (PD) images showed acceptable prediction performance (AUC = 0.7895), and the from-scratch (0.6193) and freeze (0.6149) models showed lower performances (p < 0.05). The fine-tuning model had excellent specificity compared to the human experts (87.25% vs. 58.17%). However, the human experts were superior in sensitivity (80.00% vs. 57.43%) (all p < 0.001). In gradient-weighted class activation mapping (Grad-CAM) visualizations, the fine-tuning scheme focused more on effusion than on other structures of the TMJ, and the sparsity was higher than that of the from-scratch scheme (82.40% vs. 49.83%, p < 0.05). The Grad-CAM visualizations agreed with the model learned through important features in the TMJ area, particularly around the articular disc. Two fine-tuning models on PD and T2-weighted images showed that the diagnostic performance did not improve compared with using PD alone (p < 0.05). Diverse AUCs were observed across each group when the patients were divided according to age (0.7083-0.8375) and sex (male:0.7576, female:0.7083). The prediction accuracy of the ensemble model was higher than that of the human experts when all the data were used (74.21% vs. 67.71%, p < 0.05). A deep neural network (DNN) was developed to process multimodal data, including MRI and patient clinical data. Analysis of four age groups with the DNN model showed that the 41-60 age group had the best performance (AUC = 0.8258). The fine-tuning model and DNN were optimal for judging TMJ effusion and may be used to prevent true negative cases and aid in human diagnostic performance. Assistive automated diagnostic methods have the potential to increase clinicians' diagnostic accuracy.

Advancing DIEP Flap Monitoring with Optical Imaging Techniques: A Narrative Review

OBJECTIVES

This review aims to explore recent advancements in optical imaging techniques for monitoring the viability of Deep Inferior Epigastric Perforator (DIEP) flap reconstruction. The objectives include highlighting the principles, applications, and clinical utility of optical imaging modalities such as near-infrared spectroscopy (NIRS), indocyanine green (ICG) fluorescence angiography, laser speckle contrast imaging (LSCI), hyperspectral imaging (HSI), dynamic infrared thermography (DIRT), and short-wave infrared thermography (SWIR) in assessing tissue perfusion and oxygenation. Additionally, this review aims to discuss the potential of these techniques in enhancing surgical outcomes by enabling timely intervention in cases of compromised flap perfusion.

MATERIALS AND METHODS

A comprehensive literature review was conducted to identify studies focusing on optical imaging techniques for monitoring DIEP flap viability. We searched PubMed, MEDLINE, and relevant databases, including Google Scholar, Web of Science, Scopus, PsycINFO, IEEE Xplore, and ProQuest Dissertations & Theses, among others, using specific keywords related to optical imaging, DIEP flap reconstruction, tissue perfusion, and surgical outcomes. This extensive search ensured we gathered comprehensive data for our analysis. Articles discussing the principles, applications, and clinical use of NIRS, ICG fluorescence angiography, LSCI, HSI, DIRT, and SWIR in DIEP flap monitoring were selected for inclusion. Data regarding the techniques' effectiveness, advantages, limitations, and potential impact on surgical decision-making were extracted and synthesized.

RESULTS

Optical imaging modalities, including NIRS, ICG fluorescence angiography, LSCI, HSI, DIRT, and SWIR offer a non- or minimal-invasive, real-time assessment of tissue perfusion and oxygenation in DIEP flap reconstruction. These techniques provide objective and quantitative data, enabling surgeons to monitor flap viability accurately. Studies have demonstrated the effectiveness of optical imaging in detecting compromised perfusion and facilitating timely intervention, thereby reducing the risk of flap complications such as partial or total loss. Furthermore, optical imaging modalities have shown promise in improving surgical outcomes by guiding intraoperative decision-making and optimizing patient care.

CONCLUSIONS

Recent advancements in optical imaging techniques present valuable tools for monitoring the viability of DIEP flap reconstruction. NIRS, ICG fluorescence angiography, LSCI, HSI, DIRT, and SWIR offer a non- or minimal-invasive, real-time assessment of tissue perfusion and oxygenation, enabling accurate evaluation of flap viability. These modalities have the potential to enhance surgical outcomes by facilitating timely intervention in cases of compromised perfusion, thereby reducing the risk of flap complications. Incorporating optical imaging into clinical practice can provide surgeons with objective and quantitative data, assisting in informed decision-making for optimal patient care in DIEP flap reconstruction surgeries.

Artificial intelligence for X-ray scaphoid fracture detection: a systematic review and diagnostic test accuracy meta-analysis

OBJECTIVES

Scaphoid fractures are usually diagnosed using X-rays, a low-sensitivity modality. Artificial intelligence (AI) using Convolutional Neural Networks (CNNs) has been explored for diagnosing scaphoid fractures in X-rays. The aim of this systematic review and meta-analysis is to evaluate the use of AI for detecting scaphoid fractures on X-rays and analyze its accuracy and usefulness.

MATERIALS AND METHODS

This study followed the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) and PRISMA-Diagnostic Test Accuracy. A literature search was conducted in the PubMed database for original articles published until July 2023. The risk of bias and applicability were evaluated using the QUADAS-2 tool. A bivariate diagnostic random-effects meta-analysis was conducted, and the results were analyzed using the Summary Receiver Operating Characteristic (SROC) curve.

RESULTS

Ten studies met the inclusion criteria and were all retrospective. The AI's diagnostic performance for detecting scaphoid fractures ranged from AUC 0.77 to 0.96. Seven studies were included in the meta-analysis, with a total of 3373 images. The meta-analysis pooled sensitivity and specificity were 0.80 and 0.89, respectively. The meta-analysis overall AUC was 0.88. The QUADAS-2 tool found high risk of bias and concerns about applicability in 9 out of 10 studies.

CONCLUSIONS

The current results of AI's diagnostic performance for detecting scaphoid fractures in X-rays show promise. The results show high overall sensitivity and specificity and a high SROC result. Further research is needed to compare AI's diagnostic performance to human diagnostic performance in a clinical setting.

CLINICAL RELEVANCE STATEMENT

Scaphoid fractures are prone to be missed secondary to assessment with a low sensitivity modality and a high occult fracture rate. AI systems can be beneficial for clinicians and radiologists to facilitate early diagnosis, and avoid missed injuries.

KEY POINTS

• Scaphoid fractures are common and some can be easily missed in X-rays. • Artificial intelligence (AI) systems demonstrate high diagnostic performance for the diagnosis of scaphoid fractures in X-rays. • AI systems can be beneficial in diagnosing both obvious and occult scaphoid fractures.

Parkinson's disease diagnosis using deep learning: A bibliometric analysis and literature review

Parkinson's Disease (PD) is a progressive neurodegenerative illness triggered by decreased dopamine secretion. Deep Learning (DL) has gained substantial attention in PD diagnosis research, with an increase in the number of published papers in this discipline. PD detection using DL has presented more promising outcomes as compared with common machine learning approaches. This article aims to conduct a bibliometric analysis and a literature review focusing on the prominent developments taking place in this area. To achieve the target of the study, we retrieved and analyzed the available research papers in the Scopus database. Following that, we conducted a bibliometric analysis to inspect the structure of keywords, authors, and countries in the surveyed studies by providing visual representations of the bibliometric data using VOSviewer software. The study also provides an in-depth review of the literature focusing on different indicators of PD, deployed approaches, and performance metrics. The outcomes indicate the firm development of PD diagnosis using DL approaches over time and a large diversity of studies worldwide. Additionally, the literature review presented a research gap in DL approaches related to incremental learning, particularly in relation to big data analysis.

Advancing radiology practice and research: harnessing the potential of large language models amidst imperfections

Large language models (LLMs) are transforming the field of natural language processing (NLP). These models offer opportunities for radiologists to make a meaningful impact in their field. NLP is a part of artificial intelligence (AI) that uses computer algorithms to study and understand text data. Recent advances in NLP include the Attention mechanism and the Transformer architecture. Transformer-based LLMs, such as GPT-4 and Gemini, are trained on massive amounts of data and generate human-like text. They are ideal for analysing large text data in academic research and clinical practice in radiology. Despite their promise, LLMs have limitations, including their dependency on the diversity and quality of their training data and the potential for false outputs. Albeit these limitations, the use of LLMs in radiology holds promise and is gaining momentum. By embracing the potential of LLMs, radiologists can gain valuable insights and improve the efficiency of their work. This can ultimately lead to improved patient care.

Medical image analysis using deep learning algorithms

In the field of medical image analysis within deep learning (DL), the importance of employing advanced DL techniques cannot be overstated. DL has achieved impressive results in various areas, making it particularly noteworthy for medical image analysis in healthcare. The integration of DL with medical image analysis enables real-time analysis of vast and intricate datasets, yielding insights that significantly enhance healthcare outcomes and operational efficiency in the industry. This extensive review of existing literature conducts a thorough examination of the most recent deep learning (DL) approaches designed to address the difficulties faced in medical healthcare, particularly focusing on the use of deep learning algorithms in medical image analysis. Falling all the investigated papers into five different categories in terms of their techniques, we have assessed them according to some critical parameters. Through a systematic categorization of state-of-the-art DL techniques, such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), Long Short-term Memory (LSTM) models, and hybrid models, this study explores their underlying principles, advantages, limitations, methodologies, simulation environments, and datasets. Based on our results, Python was the most frequent programming language used for implementing the proposed methods in the investigated papers. Notably, the majority of the scrutinized papers were published in 2021, underscoring the contemporaneous nature of the research. Moreover, this review accentuates the forefront advancements in DL techniques and their practical applications within the realm of medical image analysis, while simultaneously addressing the challenges that hinder the widespread implementation of DL in image analysis within the medical healthcare domains. These discerned insights serve as compelling impetuses for future studies aimed at the progressive advancement of image analysis in medical healthcare research. The evaluation metrics employed across the reviewed articles encompass a broad spectrum of features, encompassing accuracy, sensitivity, specificity, F-score, robustness, computational complexity, and generalizability.

Adversarial attacks in radiology - A systematic review

PURPOSE

The growing application of deep learning in radiology has raised concerns about cybersecurity, particularly in relation to adversarial attacks. This study aims to systematically review the literature on adversarial attacks in radiology.

METHODS

We searched for studies on adversarial attacks in radiology published up to April 2023, using MEDLINE and Google Scholar databases.

RESULTS

A total of 22 studies published between March 2018 and April 2023 were included, primarily focused on image classification algorithms. Fourteen studies evaluated white-box attacks, three assessed black-box attacks and five investigated both. Eleven of the 22 studies targeted chest X-ray classification algorithms, while others involved chest CT (6/22), brain MRI (4/22), mammography (2/22), abdominal CT (1/22), hepatic US (1/22), and thyroid US (1/22). Some attacks proved highly effective, reducing the AUC of algorithm performance to 0 and achieving success rates up to 100 %.

CONCLUSIONS

Adversarial attacks are a growing concern. Although currently the threats are more theoretical than practical, they still represent a potential risk. It is important to be alert to such attacks, reinforce cybersecurity measures, and influence the formulation of ethical and legal guidelines. This will ensure the safe use of deep learning technology in medicine.

A Deep Learning Approach for Atrial Fibrillation Classification Using Multi-Feature Time Series Data from ECG and PPG

Atrial fibrillation is a prevalent cardiac arrhythmia that poses significant health risks to patients. The use of non-invasive methods for AF detection, such as Electrocardiogram and Photoplethysmogram, has gained attention due to their accessibility and ease of use. However, there are challenges associated with ECG-based AF detection, and the significance of PPG signals in this context has been increasingly recognized. The limitations of ECG and the untapped potential of PPG are taken into account as this work attempts to classify AF and non-AF using PPG time series data and deep learning. In this work, we emploted a hybrid deep neural network comprising of 1D CNN and BiLSTM for the task of AF classification. We addressed the under-researched area of applying deep learning methods to transmissive PPG signals by proposing a novel approach. Our approach involved integrating ECG and PPG signals as multi-featured time series data and training deep learning models for AF classification. Our hybrid 1D CNN and BiLSTM model achieved an accuracy of 95% on test data in identifying atrial fibrillation, showcasing its strong performance and reliable predictive capabilities. Furthermore, we evaluated the performance of our model using additional metrics. The precision of our classification model was measured at 0.88, indicating its ability to accurately identify true positive cases of AF. The recall, or sensitivity, was measured at 0.85, illustrating the model's capacity to detect a high proportion of actual AF cases. Additionally, the F1 score, which combines both precision and recall, was calculated at 0.84, highlighting the overall effectiveness of our model in classifying AF and non-AF cases.

PulDi-COVID: Chronic obstructive pulmonary (lung) diseases with COVID-19 classification using ensemble deep convolutional neural network from chest X-ray images to minimize severity and mortality rates

Background and Objective

In the current COVID-19 outbreak, efficient testing of COVID-19 individuals has proven vital to limiting and arresting the disease's accelerated spread globally. It has been observed that the severity and mortality ratio of COVID-19 affected patients is at greater risk because of chronic pulmonary diseases. This study looks at radiographic examinations exploiting chest X-ray images (CXI), which have become one of the utmost feasible assessment approaches for pulmonary disorders, including COVID-19. Deep Learning(DL) remains an excellent image classification method and framework; research has been conducted to predict pulmonary diseases with COVID-19 instances by developing DL classifiers with nine class CXI. However, a few claim to have strong prediction results; because of noisy and small data, their recommended DL strategies may suffer from significant deviation and generality failures.

Methods

Therefore, a unique CNN model(PulDi-COVID) for detecting nine diseases (atelectasis, bacterial-pneumonia, cardiomegaly, covid19, effusion, infiltration, no-finding, pneumothorax, viral-Pneumonia) using CXI has been proposed using the SSE algorithm. Several transfer-learning models: VGG16, ResNet50, VGG19, DenseNet201, MobileNetV2, NASNetMobile, ResNet152V2, DenseNet169 are trained on CXI of chronic lung diseases and COVID-19 instances. Given that the proposed thirteen SSE ensemble models solved DL's constraints by making predictions with different classifiers rather than a single, we present PulDi-COVID, an ensemble DL model that combines DL with ensemble learning. The PulDi-COVID framework is created by incorporating various snapshots of DL models, which have spearheaded chronic lung diseases with COVID-19 cases identification process with a deep neural network produced CXI by applying a suggested SSE method. That is familiar with the idea of various DL perceptions on different classes.

Results

PulDi-COVID findings were compared to thirteen existing studies for nine-class classification using COVID-19. Test results reveal that PulDi-COVID offers impressive outcomes for chronic diseases with COVID-19 identification with a 99.70% accuracy, 98.68% precision, 98.67% recall, 98.67% F1 score, lowest 12 CXIs zero-one loss, 99.24% AUC-ROC score, and lowest 1.33% error rate. Overall test results are superior to the existing Convolutional Neural Network(CNN). To the best of our knowledge, the observed results for nine-class classification are significantly superior to the state-of-the-art approaches employed for COVID-19 detection. Furthermore, the CXI that we used to assess our algorithm is one of the larger datasets for COVID detection with pulmonary diseases.

Conclusion

The empirical findings of our suggested approach PulDi-COVID show that it outperforms previously developed methods. The suggested SSE method with PulDi-COVID can effectively fulfill the COVID-19 speedy detection needs with different lung diseases for physicians to minimize patient severity and mortality.

See Lung Cancer with an AI

A lot has happened in the field of lung cancer screening in recent months. The ongoing discussion and documentation published by the scientific community and policymakers are of great importance to the entire European community and perhaps beyond. Lung cancer is the main worldwide killer. Low-dose computed tomography-based screening, together with smoking cessation, is the only tool to fight lung cancer, as it has already been proven in the United States of America but also European randomized controlled trials. Screening requires a lot of well-organized specialized work, but it can be supported by artificial intelligence (AI). Here we discuss whether and how to use AI for patients, radiologists, pulmonologists, thoracic surgeons, and all hospital staff supporting screening process benefits.

A Mathematically Generated Noise Technique for Ultrasound Systems

Ultrasound systems have been widely used for consultation; however, they are susceptible to cyberattacks. Such ultrasound systems use random bits to protect patient information, which is vital to the stability of information-protecting systems used in ultrasound machines. The stability of the random bit must satisfy its unpredictability. To create a random bit, noise generated in hardware is typically used; however, extracting sufficient noise from systems is challenging when resources are limited. There are various methods for generating noises but most of these studies are based on hardware. Compared with hardware-based methods, software-based methods can be easily accessed by the software developer; therefore, we applied a mathematically generated noise function to generate random bits for ultrasound systems. Herein, we compared the performance of random bits using a newly proposed mathematical function and using the frequency of the central processing unit of the hardware. Random bits are generated using a raw bitmap image measuring 1000 × 663 bytes. The generated random bit analyzes the sampling data in generation time units as time-series data and then verifies the mean, median, and mode. To further apply the random bit in an ultrasound system, the image is randomized by applying exclusive mixing to a 1000 × 663 ultrasound phantom image; subsequently, the comparison and analysis of statistical data processing using hardware noise and the proposed algorithm were provided. The peak signal-to-noise ratio and mean square error of the images are compared to evaluate their quality. As a result of the test, the min entropy estimate (estimated value) was 7.156616/8 bit in the proposed study, which indicated a performance superior to that of GetSystemTime. These results show that the proposed algorithm outperforms the conventional method used in ultrasound systems.

Deep Learning Multi-Domain Model Provides Accurate Detection and Grading of Mucosal Ulcers in Different Capsule Endoscopy Types

Background and Aims: The aim of our study was to create an accurate patient-level combined algorithm for the identification of ulcers on CE images from two different capsules. Methods: We retrospectively collected CE images from PillCam-SB3′s capsule and PillCam-Crohn’s capsule. ML algorithms were trained to classify small bowel CE images into either normal or ulcerated mucosa: a separate model for each capsule type, a cross-domain model (training the model on one capsule type and testing on the other), and a combined model. Results: The dataset included 33,100 CE images: 20,621 PillCam-SB3 images and 12,479 PillCam-Crohn’s images, of which 3582 were colonic images. There were 15,684 normal mucosa images and 17,416 ulcerated mucosa images. While the separate model for each capsule type achieved excellent accuracy (average AUC 0.95 and 0.98, respectively), the cross-domain model achieved a wide range of accuracies (0.569–0.88) with an AUC of 0.93. The combined model achieved the best results with an average AUC of 0.99 and average mean patient accuracy of 0.974. Conclusions: A combined model for two different capsules provided high and consistent diagnostic accuracy. Creating a holistic AI model for automated capsule reading is an essential part of the refinement required in ML models on the way to adapting them to clinical practice.

Advantages of deep learning with convolutional neural network in detecting disc displacement of the temporomandibular joint in magnetic resonance imaging

This study investigated the usefulness of deep learning-based automatic detection of anterior disc displacement (ADD) from magnetic resonance imaging (MRI) of patients with temporomandibular joint disorder (TMD). Sagittal MRI images of 2520 TMJs were collected from 861 men and 399 women (average age 37.33 ± 18.83 years). A deep learning algorithm with a convolutional neural network was developed. Data augmentation and the Adam optimizer were applied to reduce the risk of overfitting the deep-learning model. The prediction performances were compared between the models and human experts based on areas under the curve (AUCs). The fine-tuning model showed excellent prediction performance (AUC = 0.8775) and acceptable accuracy (approximately 77%). Comparing the AUC values of the from-scratch (0.8269) and freeze models (0.5858) showed lower performances of the other models compared to the fine-tuning model. In Grad-CAM visualizations, the fine-tuning scheme focused more on the TMJ disc when judging ADD, and the sparsity was higher than that of the from-scratch scheme (84.69% vs. 55.61%, p < 0.05). The three fine-tuned ensemble models using different data augmentation techniques showed a prediction accuracy of 83%. Moreover, the AUC values of ADD were higher when patients with TMD were divided by age (0.8549–0.9275) and sex (male: 0.8483, female: 0.9276). While the accuracy of the ensemble model was higher than that of human experts, the difference was not significant (p = 0.1987–0.0671). Learning from pre-trained weights allowed the fine-tuning model to outperform the from-scratch model. Another benefit of the fine-tuning model for diagnosing ADD of TMJ in Grad-CAM analysis was the deactivation of unwanted gradient values to provide clearer visualizations compared to the from-scratch model. The Grad-CAM visualizations also agreed with the model learned through important features in the joint disc area. The accuracy was further improved by an ensemble of three fine-tuning models using diversified data. The main benefits of this model were the higher specificity compared to human experts, which may be useful for preventing true negative cases, and the maintenance of its prediction accuracy across sexes and ages, suggesting a generalized prediction.

Nursing Analysis Based on Medical Imaging Technology before and after Coronary Angiography in Cardiovascular Medicine

With the advancement of technology, medical imaging technology has been greatly improved. This article mainly studies the nursing before and after coronary angiography in cardiovascular medicine based on medical imaging technology. This paper proposes a multimodal medical image fusion algorithm based on multiscale decomposition and convolution sparse representation. The algorithm first decomposes the preregistered source medical image by NSST, takes the subimages of different scales as training images, and optimizes the subdictionaries of different scales; then convolution and sparse the subimages on each scale encoding to obtain the sparse coefficients of different subimages; secondly, the combination of improved L1 norm and improved spatial frequency (novel sum-modified SF (NMSF)) is used for high-frequency subimage coefficients, and the fusion of low-frequency subimages improved the rule of combining the L1 norm and the regional energy; finally, the final fused image is obtained by inverse NSST of the fused low-frequency subband and high-frequency subband. Experimental analysis found that the bifurcation angle has nothing to do with the damage of the branch vessels after the main branch stent is placed. The bifurcation angle greater than 50° is an independent predictor of MACE after stent extrusion for bifurcation lesions. Experimental results show that the proposed method has good performance in contrast enhancement, detail extraction, and information retention, and it improves the quality of the fusion image.

Application of Artificial Intelligence in Lung Cancer

Lung cancer is the leading cause of malignancy-related mortality worldwide due to its heterogeneous features and diagnosis at a late stage. Artificial intelligence (AI) is good at handling a large volume of computational and repeated labor work and is suitable for assisting doctors in analyzing image-dominant diseases like lung cancer. Scientists have shown long-standing efforts to apply AI in lung cancer screening via CXR and chest CT since the 1960s. Several grand challenges were held to find the best AI model. Currently, the FDA have approved several AI programs in CXR and chest CT reading, which enables AI systems to take part in lung cancer detection. Following the success of AI application in the radiology field, AI was applied to digitalized whole slide imaging (WSI) annotation. Integrating with more information, like demographics and clinical data, the AI systems could play a role in decision-making by classifying EGFR mutations and PD-L1 expression. AI systems also help clinicians to estimate the patient's prognosis by predicting drug response, the tumor recurrence rate after surgery, radiotherapy response, and side effects. Though there are still some obstacles, deploying AI systems in the clinical workflow is vital for the foreseeable future.

Artificial Intelligence for Interstitial Lung Disease Analysis on Chest Computed Tomography: A Systematic Review

RATIONALE AND OBJECTIVES

High-resolution computed tomography (HRCT) is paramount in the assessment of interstitial lung disease (ILD). Yet, HRCT interpretation of ILDs may be hampered by inter- and intra-observer variability. Recently, artificial intelligence (AI) has revolutionized medical image analysis. This technology has the potential to advance patient care in ILD. We aimed to systematically evaluate the application of AI for the analysis of ILD in HRCT.

MATERIALS AND METHODS

We searched MEDLINE/PubMed databases for original publications of deep learning for ILD analysis on chest CT. The search included studies published up to March 1, 2021. The risk of bias evaluation included tailored Quality Assessment of Diagnostic Accuracy Studies and the modified Joanna Briggs Institute Critical Appraisal checklist.

RESULTS

Data was extracted from 19 retrospective studies. Deep learning techniques included detection, segmentation, and classification of ILD on HRCT. Most studies focused on the classification of ILD into different morphological patterns. Accuracies of 78%-91% were achieved. Two studies demonstrated near-expert performance for the diagnosis of idiopathic pulmonary fibrosis (IPF). The Quality Assessment of Diagnostic Accuracy Studies tool identified a high risk of bias in 15/19 (78.9%) of the studies.

CONCLUSION

AI has the potential to contribute to the radiologic diagnosis and classification of ILD. However, the accuracy performance is still not satisfactory, and research is limited by a small number of retrospective studies. Hence, the existing published data may not be sufficiently reliable. Only well-designed prospective controlled studies can accurately assess the value of existing AI tools for ILD evaluation.

Medical image processing and COVID-19: A literature review and bibliometric analysis

COVID-19 crisis has placed medical systems over the world under unprecedented and growing pressure. Medical imaging processing can help in the diagnosis, treatment, and early detection of diseases. It has been considered as one of the modern technologies applied to fight against the COVID-19 crisis. Although several artificial intelligence, machine learning, and deep learning techniques have been deployed in medical image processing in the context of COVID-19 disease, there is a lack of research considering systematic literature review and categorization of published studies in this field. A systematic review locates, assesses, and interprets research outcomes to address a predetermined research goal to present evidence-based practical and theoretical insights. The main goal of this study is to present a literature review of the deployed methods of medical image processing in the context of the COVID-19 crisis. With this in mind, the studies available in reliable databases were retrieved, studied, evaluated, and synthesized. Based on the in-depth review of literature, this study structured a conceptual map that outlined three multi-layered folds: data gathering and description, main steps of image processing, and evaluation metrics. The main research themes were elaborated in each fold, allowing the authors to recommend upcoming research paths for scholars. The outcomes of this review highlighted that several methods have been adopted to classify the images related to the diagnosis and detection of COVID-19. The adopted methods have presented promising outcomes in terms of accuracy, cost, and detection speed.

Automatic Diagnosis Labeling of Cardiovascular MRI by Using Semisupervised Natural Language Processing of Text Reports

PURPOSE

To assess whether the semisupervised natural language processing (NLP) of text from clinical radiology reports could provide useful automated diagnosis categorization for ground truth labeling to overcome manual labeling bottlenecks in the machine learning pipeline.

MATERIALS AND METHODS

In this retrospective study, 1503 text cardiac MRI reports from 2016 to 2019 were manually annotated for five diagnoses by clinicians: normal, dilated cardiomyopathy (DCM), hypertrophic cardiomyopathy, myocardial infarction (MI), and myocarditis. A semisupervised method that uses bidirectional encoder representations from transformers (BERT) pretrained on 1.14 million scientific publications was fine-tuned by using the manually extracted labels, with a report dataset split into groups of 801 for training, 302 for validation, and 400 for testing. The model's performance was compared with two traditional NLP models: a rule-based model and a support vector machine (SVM) model. The models' F1 scores and receiver operating characteristic curves were used to analyze performance.

RESULTS

After 15 epochs, the F1 scores on the test set of 400 reports were as follows: normal, 84%; DCM, 79%; hypertrophic cardiomyopathy, 86%; MI, 91%; and myocarditis, 86%. The pooled F1 score and area under the receiver operating curve were 86% and 0.96, respectively. On the same test set, the BERT model had a higher performance than the rule-based model (F1 score, 42%) and SVM model (F1 score, 82%). Diagnosis categories classified by using the BERT model performed the labeling of 1000 MR images in 0.2 second.

CONCLUSION

The developed model used labels extracted from radiology reports to provide automated diagnosis categorization of MR images with a high level of performance.Keywords: Semisupervised Learning, Diagnosis/Classification/Application Domain, Named Entity Recognition, MRI Supplemental material is available for this article. © RSNA, 2021.

Machine learning for prediction of intra-abdominal abscesses in patients with Crohn's disease visiting the emergency department

BACKGROUND

Intra-abdominal abscess (IA) is an important clinical complication of Crohn's disease (CD). A high index of clinical suspicion is needed as imaging is not routinely used during hospital admission. This study aimed to identify clinical predictors of an IA among hospitalized patients with CD using machine learning.

METHODS

We created an electronic data repository of all patients with CD who visited the emergency department of our tertiary medical center between 2012 and 2018. We searched for the presence of an IA on abdominal imaging within 7 days from visit. Machine learning models were trained to predict the presence of an IA. A logistic regression model was compared with a random forest model.

RESULTS

Overall, 309 patients with CD were hospitalized and underwent abdominal imaging within 7 days. Forty patients (12.9%) were diagnosed with an IA. On multivariate analysis, high C-reactive protein (CRP) [above 65 mg/l, adjusted odds ratio (aOR): 16 (95% CI: 5.51-46.18)], leukocytosis [above 10.5 K/μl, aOR: 4.47 (95% CI: 1.91-10.45)], thrombocytosis [above 322.5 K/μl, aOR: 4.1 (95% CI: 2-8.73)], and tachycardia [over 97 beats per minute, aOR: 2.7 (95% CI: 1.37-5.3)] were independently associated with an IA. Random forest model showed an area under the curve of 0.817 ± 0.065 with six features (CRP, hemoglobin, WBC, age, current biologic therapy, and BUN).

CONCLUSION

In our large tertiary center cohort, the machine learning model identified the association of six clinical features (CRP, hemoglobin, WBC, age, BUN, and biologic therapy) with the presentation of an IA. These may assist as a decision support tool in triaging CD patients for imaging to exclude this potentially life-threatening complication.

Deep learning for noninvasive liver fibrosis classification: A systematic review

BACKGROUND AND AIMS

While biopsy is the gold standard for liver fibrosis staging, it poses significant risks. Noninvasive assessment of liver fibrosis is a growing field. Recently, deep learning (DL) technology has revolutionized medical image analysis. This technology has the potential to enhance noninvasive fibrosis assessment. We systematically examined the application of DL in noninvasive liver fibrosis imaging.

METHODS

Embase, MEDLINE, Web of Science, and IEEE Xplore databases were used to identify studies that reported on the accuracy of DL for classification of liver fibrosis on noninvasive imaging. The search keywords were "liver or hepatic," "fibrosis or cirrhosis," and "neural or DL networks." Risk of bias and applicability were evaluated using the QUADAS-2 tool.

RESULTS

Sixteen studies were retrieved. Imaging modalities included ultrasound (n = 10), computed tomography (n = 3), and magnetic resonance imaging (n = 3). The studies analyzed a total of 40 405 radiological images from 15 853 patients. All but two of the studies were retrospective. In most studies the "ground truth" reference was the METAVIR score for pathological staging (n = 9.56%). The majority of the studies reported an accuracy >85% when compared to histopathology. Fourteen studies (87.5%) had a high risk of bias and concerns regarding applicability.

CONCLUSIONS

Deep learning has the potential to play an emerging role in liver fibrosis classification. Yet, it is still limited by a relatively small number of retrospective studies. Clinicians should facilitate the use of this technology by sharing databases and standardized reports. This may optimize the noninvasive evaluation of liver fibrosis on a large scale.

Diameter-Based Volumetric Models May Inadequately Calculate Jugular Paraganglioma Volume Following Sub-Total Resection

BACKGROUND

As gross total resection of jugular paragangliomas (JPs) may result in cranial nerve deficits, JPs are increasingly managed with subtotal resection (STR) with postoperative radiological monitoring. However, the validity of commonly used diameter-based models that calculate postoperative volume to determine residual tumor growth is dubious. The purpose of this study was to assess the accuracy of these models compared to manual volumetric slice-by-slice segmentation.

METHODS

A senior neuroradiologist measured volumes via slice-by-slice segmentation of JPs pre- and postoperatively from patients who underwent STR from 2007 to 2019. Volumes from three linear-based models were calculated. Models with absolute percent error (APE) > 20% were considered unsatisfactory based on a common volumetric definition for residual growth. Bland-Altman plots were used to evaluate reproducibility, and Wilcoxon matched-pairs signed rank test evaluated model bias.

RESULTS

Twenty-one patients were included. Median postoperative APE exceeded the established 20% threshold for each of the volumetric models as cuboidal, ellipsoidal, and spherical model APE were 63%, 28%, and 27%, respectively. The postoperative cuboidal model had significant systematic bias overestimating volume (p = 0.002) whereas the postoperative ellipsoidal and spherical models lacked systematic bias (p = 0.11 and p = 0.82).

CONCLUSION

Cuboidal, ellipsoidal, and spherical models do not provide accurate assessments of postoperative JP tumor volume and may result in salvage therapies that are unnecessary or inappropriately withheld due to inaccurate assessment of residual tumor growth. While more time-consuming, slice-by-slice segmentation by an experienced neuroradiologist provides a substantially more accurate and precise measurement of tumor volume that may optimize clinical management.

Capsule Endoscopy in Crohn's Disease-From a Relative Contraindication to Habitual Monitoring Tool

Crohn’s disease (CD) is a chronic inflammatory disorder that may involve the gastrointestinal tract from the mouth to the anus. Habitual disease monitoring is highly important during disease management, aiming to identify and treat disease exacerbations, in order to avoid immediate and future complications. Currently, ilio-clonoscopy is the gold standard for mucosal assessment. However, the procedure is invasive, involves sedation and allows for visualization of the colon and only a small part of the terminal ileum, while most of the small bowel is not visualized. Since CD may involve the whole length of the small bowel, the disease extent might be underestimated. Capsule endoscopy (CE) provides a technology that can screen the entire bowel in a non-invasive procedure, with minimal side effects. In recent years, this technique has gained in popularity for CD evaluation and monitoring. When CE was first introduced, two decades ago, the fear of possible capsule retention in the narrowed inflamed bowel lumen limited its use in CD patients, and a known CD located at the small bowel was even regarded as a relative contraindication for capsule examination. However, at present, as experience using CE in CD patients has accumulated, this procedure has become one of the accepted tools for disease diagnosis and monitoring. In our current review, we summarize the historic change in the indications and contraindications for the usage of capsule endoscopy for the evaluation of CD, and discuss international recommendations regarding CE’s role in CD diagnosis and monitoring.

A Convolutional Neural Network Deep Learning Model Trained on CD Ulcers Images Accurately Identifies NSAID Ulcers

Background and Study Aims: Deep learning (DL) for video capsule endoscopy (VCE) is an emerging research field. It has shown high accuracy for the detection of Crohn's disease (CD) ulcers. Non-steroidal anti-inflammatory drugs (NSAIDS) are commonly used medications. In the small bowel, NSAIDs may cause a variety of gastrointestinal adverse events including NSAID-induced ulcers. These ulcers are the most important differential diagnosis for small bowel ulcers in patients evaluated for suspected CD. We evaluated a DL network that was trained using CD VCE ulcer images and evaluated its performance for NSAID ulcers. Patients and Methods: The network was trained using CD ulcers and normal mucosa from a large image bank created from VCE of diagnosed CD patients. NSAIDs-induced enteropathy images were extracted from the prospective Bifidobacterium breve (BIf95) trial dataset. All images were acquired from studies performed using PillCam SBIII. The area under the receiver operating curve (AUC) was used as a metric. We compared the network's AUC for detecting NSAID ulcers to that of detecting CD ulcers. Results: Overall, the CD training dataset included 17,640 CE images. The NSAIDs testing dataset included 1,605 CE images. The DL network exhibited an AUC of 0.97 (95% CI 0.97-0.98) for identifying images with NSAID mucosal ulcers. The diagnostic accuracy was similar to that obtained for CD related ulcers (AUC 0.94-0.99). Conclusions: A network trained on VCE CD ulcers similarly identified NSAID findings. As deep learning is transforming gastrointestinal endoscopy, this result should be taken into consideration in the future design and analysis of VCE deep learning applications.

Role of capsule endoscopy in inflammatory bowel disease: Anything new?

Capsule endoscopy (CE) is a recently developed diagnostic method for diseases of the small bowel that is non-invasive, safe, and highly tolerable. Its role in patients with inflammatory bowel disease has been widely validated in suspected and established Crohn’s disease (CD) due to its ability to assess superficial lesions not detected by cross-sectional imaging and proximal lesions of the small bowel not evaluable by ileocolonoscopy. Because CE is a highly sensitive but less specific technique, differential diagnoses that can simulate CD must be considered, and its interpretation should be supported by other clinical and laboratory indicators. The use of validated scoring systems to characterize and estimate lesion severity (Lewis score, Capsule Endoscopy Crohn’s Disease Activity Index), as well as the standardization of the language used to define the lesions (Delphi Consensus), have reduced the interobserver variability in CE reading observed in clinical practice, allowing for the optimization of diagnoses and clinical management strategies. The appearance of the panenteric CE, the incorporation of artificial intelligence, magnetically-guided capsules, and tissue biopsies are elements that contribute to CE being a promising, unique diagnostic tool in digestive tract diseases.

Convolution neural network for the diagnosis of wireless capsule endoscopy: a systematic review and meta-analysis

Background

Wireless capsule endoscopy (WCE) is considered to be a powerful instrument for the diagnosis of intestine diseases. Convolution neural network (CNN) is a type of artificial intelligence that has the potential to assist the detection of WCE images. We aimed to perform a systematic review of the current research progress to the CNN application in WCE.

Methods

A search in PubMed, SinoMed, and Web of Science was conducted to collect all original publications about CNN implementation in WCE. Assessment of the risk of bias was performed by Quality Assessment of Diagnostic Accuracy Studies-2 risk list. Pooled sensitivity and specificity were calculated by an exact binominal rendition of the bivariate mixed-effects regression model. I² was used for the evaluation of heterogeneity.

Results

16 articles with 23 independent studies were included. CNN application to WCE was divided into detection on erosion/ulcer, gastrointestinal bleeding (GI bleeding), and polyps/cancer. The pooled sensitivity of CNN for erosion/ulcer is 0.96 [95% CI 0.91, 0.98], for GI bleeding is 0.97 (95% CI 0.93–0.99), and for polyps/cancer is 0.97 (95% CI 0.82–0.99). The corresponding specificity of CNN for erosion/ulcer is 0.97 (95% CI 0.93–0.99), for GI bleeding is 1.00 (95% CI 0.99–1.00), and for polyps/cancer is 0.98 (95% CI 0.92–0.99).

Conclusion

Based on our meta-analysis, CNN-dependent diagnosis of erosion/ulcer, GI bleeding, and polyps/cancer approached a high-level performance because of its high sensitivity and specificity. Therefore, future perspective, CNN has the potential to become an important assistant for the diagnosis of WCE.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00464-021-08689-3.

Deep learning for pulmonary embolism detection on computed tomography pulmonary angiogram: a systematic review and meta-analysis

Computed tomographic pulmonary angiography (CTPA) is the gold standard for pulmonary embolism (PE) diagnosis. However, this diagnosis is susceptible to misdiagnosis. In this study, we aimed to perform a systematic review of current literature applying deep learning for the diagnosis of PE on CTPA. MEDLINE/PUBMED were searched for studies that reported on the accuracy of deep learning algorithms for PE on CTPA. The risk of bias was evaluated using the QUADAS-2 tool. Pooled sensitivity and specificity were calculated. Summary receiver operating characteristic curves were plotted. Seven studies met our inclusion criteria. A total of 36,847 CTPA studies were analyzed. All studies were retrospective. Five studies provided enough data to calculate summary estimates. The pooled sensitivity and specificity for PE detection were 0.88 (95% CI 0.803-0.927) and 0.86 (95% CI 0.756-0.924), respectively. Most studies had a high risk of bias. Our study suggests that deep learning models can detect PE on CTPA with satisfactory sensitivity and an acceptable number of false positive cases. Yet, these are only preliminary retrospective works, indicating the need for future research to determine the clinical impact of automated PE detection on patient care. Deep learning models are gradually being implemented in hospital systems, and it is important to understand the strengths and limitations of these algorithms.

Investigation of clinical target volume segmentation for whole breast irradiation using three-dimensional convolutional neural networks with gradient-weighted class activation mapping

This study aims to implement three-dimensional convolutional neural networks (3D-CNN) for clinical target volume (CTV) segmentation for whole breast irradiation and investigate the focus of 3D-CNNs during decision-making using gradient-weighted class activation mapping (Grad-CAM). A 3D-UNet CNN was adopted to conduct automatic segmentation of the CTV for breast cancer. The 3D-UNet was trained using three datasets of left-, right-, and both left- and right-sided breast cancer patients. Segmentation accuracy was evaluated using the Dice similarity coefficient (DSC). Grad-CAM was applied to trained CNNs. The DSCs for the datasets of the left-, right-, and both left- and right-sided breasts were on an average 0.88, 0.89, and 0.85, respectively. The Grad-CAM heatmaps showed that the 3D-UNet used for segmentation determined the CTV region from the target-side breast tissue and by referring to the opposite-side breast. Although the size of the dataset was limited, DSC ≥ 0.85 was achieved for the segmentation of breast CTV using the 3D-UNet. Grad-CAM indicates the applicable scope and limitations of using a CNN by indicating the focus of such networks during decision-making.

Differentiation Between Malignant and Benign Endoscopic Images of Gastric Ulcers Using Deep Learning

Background and Aim

Endoscopic differentiation between malignant and benign gastric ulcers (GU) affects further evaluation and prognosis. The aim of our study was to evaluate a deep learning algorithm for discrimination between benign and malignant GU in a database of endoscopic ulcer images.

Methods

We retrospectively collected consecutive upper gastrointestinal endoscopy images of GU performed between 2011 and 2019 at the Sheba Medical Center. All ulcers had a corresponding histopathology result of either benign peptic ulcer or gastric adenocarcinoma. A convolutional neural network (CNN) was trained to classify the images into either benign or malignant. Endoscopies from 2011 to 2017 were used for training (2011-2015) and validation (2016-2017). Hyper-parameters, image augmentation and pre-training on Google images obtained images were evaluated on the validation data. Held-out data from 2018 to 2019 was used for testing the final model.

Results

Overall, the Sheba dataset included 1978 GU images; 1894 images from benign GU and 84 images of malignant ulcers. The final CNN model showed an AUC 0.91 (95% CI 0.85-0.96) for detecting malignant ulcers. For cut-off probability 0.5, the network showed a sensitivity of 92% and specificity of 75% for malignant ulcers.

Conclusion

Our study displays the applicability of a CNN model for automated evaluation of gastric ulcers images for malignant potential. Following further research, the algorithm may improve accuracy of differentiating benign from malignant ulcers during endoscopies and assist in patients' stratification, allowing accelerated patients management and individualized approach towards surveillance endoscopy.

Automated Detection of Crohn's Disease Intestinal Strictures on Capsule Endoscopy Images Using Deep Neural Networks

BACKGROUND AND AIMS

Passable intestinal strictures are frequently detected on capsule endoscopy [CE]. Such strictures are a major component of inflammatory scores. Deep neural network technology for CE is emerging. However, the ability of deep neural networks to identify intestinal strictures on CE images of Crohn's disease [CD] patients has not yet been evaluated.

METHODS

We tested a state-of-the-art deep learning network for detecting CE images of strictures. Images of normal mucosa, mucosal ulcers, and strictures of Crohn's disease patients were retrieved from our previously described CE image bank. Ulcers were classified as per degree of severity. We performed 10 cross-validation experiments. A clear patient-level separation was maintained between training and testing sets.

RESULTS

Overall, the entire dataset included 27 892 CE images: 1942 stricture images, 14 266 normal mucosa images, and 11 684 ulcer images [mild: 7075, moderate: 2386, severe: 2223]. For classifying strictures versus non-strictures, the network exhibited an average accuracy of 93.5% [±6.7%]. The network achieved excellent differentiation between strictures and normal mucosa (area under the curve [AUC] 0.989), strictures and all ulcers [AUC 0.942], and between strictures and different grades of ulcers [for mild, moderate, and severe ulcers-AUCs 0.992, 0.975, and 0.889, respectively].

CONCLUSIONS

Deep neural networks are highly accurate in the detection of strictures on CE images in Crohn's disease. The network can accurately separate strictures from ulcers across the severity range. The current accuracy for the detection of ulcers and strictures by deep neural networks may allow for automated detection and grading of Crohn's disease-related findings on CE.

Machine Learning and Deep Neural Network Applications in the Thorax: Pulmonary Embolism, Chronic Thromboembolic Pulmonary Hypertension, Aorta, and Chronic Obstructive Pulmonary Disease

The radiologic community is rapidly integrating a revolution that has not fully entered daily practice. It necessitates a close collaboration between computer scientists and radiologists to move from concepts to practical applications. This article reviews the current littérature on machine learning and deep neural network applications in the field of pulmonary embolism, chronic thromboembolic pulmonary hypertension, aorta, and chronic obstructive pulmonary disease.

Predicting In-Hospital Mortality at Admission to the Medical Ward: A Big-Data Machine Learning Model

BACKGROUND

General medical wards admit high-risk patients. Artificial intelligence algorithms can use big data for developing models to assess patients' risk stratification. The aim of this study was to develop a mortality prediction machine learning model using data available at the time of admission to the medical ward.

METHODS

We included consecutive patients (ages 18-100) admitted to medical wards at a single medical center (January 1, 2013-December 31, 2018). We constructed a machine learning model using patient characteristics, comorbidities, laboratory tests, and patients' emergency department (ED) management. The model was trained on data from the years 2013 to 2017 and validated on data from the year 2018. The area under the curve (AUC) for mortality prediction was used as an outcome metric. Youden index was used to find an optimal sensitivity-specificity cutoff point.

RESULTS

Of the 118,262 patients admitted to the medical ward, 6311 died (5.3%). The single variables with the highest AUCs were medications administered in the ED (AUC = 0.74), ED diagnosis (AUC = 0.74), and albumin (AUC = 0.73). The machine learning model yielded an AUC of 0.924 (95% confidence interval [CI]: 0.917-0.930). For Youden index, a sensitivity of 0.88 (95% CI: 0.86-0.89) and specificity of 0.83 (95% CI: 0.83-0.83) were observed. This corresponds to a false-positive rate of 1:5.9 and negative predictive value of 0.99.

CONCLUSION

A machine learning model outperforms single variables predictions of in-hospital mortality at the time of admission to the medical ward. Such a decision support tool has the potential to augment clinical decision-making regarding level of care needed for admitted patients.

Alerting on mortality among patients discharged from the emergency department: a machine learning model

OBJECTIVES

Physicians continuously make tough decisions when discharging patients. Alerting on poor outcomes may help in this decision. This study evaluates a machine learning model for predicting 30-day mortality in emergency department (ED) discharged patients.

METHODS

We retrospectively analysed visits of adult patients discharged from a single ED (1/2014-12/2018). Data included demographics, evaluation and treatment in the ED, and discharge diagnosis. The data comprised of both structured and free-text fields. A gradient boosting model was trained to predict mortality within 30 days of release from the ED. The model was trained on data from the years 2014-2017 and validated on data from the year 2018. In order to reduce potential end-of-life bias, a subgroup analysis was performed for non-oncological patients.

RESULTS

Overall, 363 635 ED visits of discharged patients were analysed. The 30-day mortality rate was 0.8%. A majority of the mortality cases (65.3%) had a known oncological disease. The model yielded an area under the curve (AUC) of 0.97 (95% CI 0.96 to 0.97) for predicting 30-day mortality. For a sensitivity of 84% (95% CI 0.81 to 0.86), this model had a false positive rate of 1:20. For patients without a known malignancy, the model yielded an AUC of 0.94 (95% CI 0.92 to 0.95).

CONCLUSIONS

Although not frequent, patients may die following ED discharge. Machine learning-based tools may help ED physicians identify patients at risk. An optimised decision for hospitalisation or palliative management may improve patient care and system resource allocation.

Deep learning for wireless capsule endoscopy: a systematic review and meta-analysis

BACKGROUND AND AIMS

Deep learning is an innovative algorithm based on neural networks. Wireless capsule endoscopy (WCE) is considered the criterion standard for detecting small-bowel diseases. Manual examination of WCE is time-consuming and can benefit from automatic detection using artificial intelligence (AI). We aimed to perform a systematic review of the current literature pertaining to deep learning implementation in WCE.

METHODS

We conducted a search in PubMed for all original publications on the subject of deep learning applications in WCE published between January 1, 2016 and December 15, 2019. Evaluation of the risk of bias was performed using tailored Quality Assessment of Diagnostic Accuracy Studies-2. Pooled sensitivity and specificity were calculated. Summary receiver operating characteristic curves were plotted.

RESULTS

Of the 45 studies retrieved, 19 studies were included. All studies were retrospective. Deep learning applications for WCE included detection of ulcers, polyps, celiac disease, bleeding, and hookworm. Detection accuracy was above 90% for most studies and diseases. Pooled sensitivity and specificity for ulcer detection were .95 (95% confidence interval [CI], .89-.98) and .94 (95% CI, .90-.96), respectively. Pooled sensitivity and specificity for bleeding or bleeding source were .98 (95% CI, .96-.99) and .99 (95% CI, .97-.99), respectively.

CONCLUSIONS

Deep learning has achieved excellent performance for the detection of a range of diseases in WCE. Notwithstanding, current research is based on retrospective studies with a high risk of bias. Thus, future prospective, multicenter studies are necessary for this technology to be implemented in the clinical use of WCE.

Performance improvement of mediastinal lymph node severity detection using GAN and Inception network

BACKGROUND AND OBJECTIVE

In lung cancer, the determination of mediastinal lymph node (MLN) status as benign or malignant influence treatment planning and survival rate. Invasive pathological tests for the classification of MLNs into benign and malignant have various shortcomings like painfulness, the risk associated with anesthesia, and depends to a large extent on skillset and preferences of the surgeon performing the test. Hence, computer-aided system for MLNs severity detection has been explored widely by the researchers. Very recently, in our earlier concluded work on non-invasive method for MLNs differential diagnosis in computed tomography (CT) images, combination of different data augmentation approaches and state-of-art fully convolutional network (FCN) were implemented to enhance the performance of malignancy detection. However, the performance of FCN network were highly depended on the selection of appropriate data augmentation approach and control of their hyperparameters. Moreover, a standard practice to get hierarchical features in convolutional neural network (CNN) models requires deeper stacking of layers. This leads to an increase in number of trainable parameters which prone to overfitting of the network.

METHODS

In view of the above mention limitations, in this paper, authors have proposed an approach that includes: 1) Generative Adversarial Network (GAN) for data augmentation, and 2) Inception network for malignancy detection. Unlike conventional data augmentation strategy, GAN based augmentation approach generates data that correlates to original data distribution. In the case of Inception based model, it uses multiple size kernels with factorized convolution for hierarchical feature extraction. This helps to a significant reduction in trainable parameters and the problem of overfitting.

RESULTS

In this paper, experiments with different GAN approaches, as well as with different Inception architectures, are conducted to evaluate and justify the selection of appropriate GAN and Inception architecture, respectively for MLNs severity detection. The proposed approach achieves superior results with an average accuracy, sensitivity, specificity, and area under curve of 94.95%, 93.65%, 96.67%, and 95%, respectively.

CONCLUSION

The obtained results validate the usefulness of GANs for data augmentation in the differential diagnosis of benign and malignant MLNs. The proposed Inception network based classifier for malignancy detection shows promising results compared to all investigated methods presented in various literature.

Application of deep learning for fast detection of COVID-19 in X-Rays using nCOVnet

Presently, COVID-19 has posed a serious threat to researchers, scientists, health professionals, and administrations around the globe from its detection to its treatment. The whole world is witnessing a lockdown like situation because of COVID-19 pandemic. Persistent efforts are being made by the researchers to obtain the possible solutions to control this pandemic in their respective areas. One of the most common and effective methods applied by the researchers is the use of CT-Scans and X-rays to analyze the images of lungs for COVID-19. However, it requires several radiology specialists and time to manually inspect each report which is one of the challenging tasks in a pandemic. In this paper, we have proposed a deep learning neural network-based method nCOVnet, an alternative fast screening method that can be used for detecting the COVID-19 by analyzing the X-rays of patients which will look for visual indicators found in the chest radiography imaging of COVID-19 patients.

Reinventing polysomnography in the age of precision medicine

For almost 50 years, sleep laboratories around the world have been collecting massive amounts of polysomnographic (PSG) physiological data to diagnose sleep disorders, the majority of which are not utilized in the clinical setting. Only a small fraction of the information available within these signals is utilized to generate indices. For example, the apnea-hypopnea index (AHI) remains the primary tool for diagnostic and therapeutic decision-making for obstructive sleep apnea (OSA) despite repeated studies showing it to be inadequate in predicting clinical consequences. Today, there are many novel approaches to PSG signals, making it possible to extract more complex metrics and analyses that are potentially more clinically relevant for individual patients. However, the pathway to implement novel PSG metrics/analyses into routine clinical practice is unclear. Our goal with this review is to highlight some of the novel PSG metrics/analyses that are becoming available. We suggest that stronger academic-industry relationships would facilitate the development of state-of-the-art clinical research to establish the value of novel PSG metrics/analyses in clinical sleep medicine. Collectively, as a sleep community, it is time to reinvent how we utilize the polysomnography to move us towards Precision Sleep Medicine.

Creating Artificial Images for Radiology Applications Using Generative Adversarial Networks (GANs) - A Systematic Review

RATIONALE AND OBJECTIVES

Generative adversarial networks (GANs) are deep learning models aimed at generating fake realistic looking images. These novel models made a great impact on the computer vision field. Our study aims to review the literature on GANs applications in radiology.

MATERIALS AND METHODS

This systematic review followed the PRISMA guidelines. Electronic datasets were searched for studies describing applications of GANs in radiology. We included studies published up-to September 2019.

RESULTS

Data were extracted from 33 studies published between 2017 and 2019. Eighteen studies focused on CT images generation, ten on MRI, three on PET/MRI and PET/CT, one on ultrasound and one on X-ray. Applications in radiology included image reconstruction and denoising for dose and scan time reduction (fourteen studies), data augmentation (six studies), transfer between modalities (eight studies) and image segmentation (five studies). All studies reported that generated images improved the performance of the developed algorithms.

CONCLUSION

GANs are increasingly studied for various radiology applications. They enable the creation of new data, which can be used to improve clinical care, education and research.

Time Series Analysis and Forecasting with Automated Machine Learning on a National ICD-10 Database

The application of machine learning (ML) for use in generating insights and making predictions on new records continues to expand within the medical community. Despite this progress to date, the application of time series analysis has remained underexplored due to complexity of the underlying techniques. In this study, we have deployed a novel ML, called automated time series (AutoTS) machine learning, to automate data processing and the application of a multitude of models to assess which best forecasts future values. This rapid experimentation allows for and enables the selection of the most accurate model in order to perform time series predictions. By using the nation-wide ICD-10 (International Classification of Diseases, Tenth Revision) dataset of hospitalized patients of Romania, we have generated time series datasets over the period of 2008–2018 and performed highly accurate AutoTS predictions for the ten deadliest diseases. Forecast results for the years 2019 and 2020 were generated on a NUTS 2 (Nomenclature of Territorial Units for Statistics) regional level. This is the first study to our knowledge to perform time series forecasting of multiple diseases at a regional level using automated time series machine learning on a national ICD-10 dataset. The deployment of AutoTS technology can help decision makers in implementing targeted national health policies more efficiently.

Comparison of deep learning models for natural language processing-based classification of non-English head CT reports

PURPOSE

Natural language processing (NLP) can be used for automatic flagging of radiology reports. We assessed deep learning models for classifying non-English head CT reports.

METHODS

We retrospectively collected head CT reports (2011-2018). Reports were signed in Hebrew. Emergency department (ED) reports of adult patients from January to February for each year (2013-2018) were manually labeled. All other reports were used to pre-train an embedding layer. We explored two use cases: (1) general labeling use case, in which reports were labeled as normal vs. pathological; (2) specific labeling use case, in which reports were labeled as with and without intra-cranial hemorrhage. We tested long short-term memory (LSTM) and LSTM-attention (LSTM-ATN) networks for classifying reports. We also evaluated the improvement of adding Word2Vec word embedding. Deep learning models were compared with a bag-of-words (BOW) model.

RESULTS

We retrieved 176,988 head CT reports for pre-training. We manually labeled 7784 reports as normal (46.3%) or pathological (53.7%), and 7.1% with intra-cranial hemorrhage. For the general labeling, LSTM-ATN-Word2Vec showed the best results (AUC = 0.967 ± 0.006, accuracy 90.8% ± 0.01). For the specific labeling, all methods showed similar accuracies between 95.0 and 95.9%. Both LSTM-ATN-Word2Vec and BOW had the highest AUC (0.970).

CONCLUSION

For a general use case, word embedding using a large cohort of non-English head CT reports and ATN improves NLP performance. For a more specific task, BOW and deep learning showed similar results. Models should be explored and tailored to the NLP task.

Deep learning algorithms for automated detection of Crohn's disease ulcers by video capsule endoscopy

BACKGROUND AND AIMS

The aim of our study was to develop and evaluate a deep learning algorithm for the automated detection of small-bowel ulcers in Crohn's disease (CD) on capsule endoscopy (CE) images of individual patients.

METHODS

We retrospectively collected CE images of known CD patients and control subjects. Each image was labeled by an expert gastroenterologist as either normal mucosa or containing mucosal ulcers. A convolutional neural network was trained to classify images into either normal mucosa or mucosal ulcers. First, we trained the network on 5-fold randomly split images (each fold with 80% training images and 20% images testing). We then conducted 10 experiments in which images from n - 1 patients were used to train a network and images from a different individual patient were used to test the network. Results of the networks were compared for randomly split images and for individual patients. Area under the curves (AUCs) and accuracies were computed for each individual network.

RESULTS

Overall, our dataset included 17,640 CE images from 49 patients: 7391 images with mucosal ulcers and 10,249 images of normal mucosa. For randomly split images results were excellent, with AUCs of .99 and accuracies ranging from 95.4% to 96.7%. For individual patient-level experiments, the AUCs were also excellent (.94-.99).

CONCLUSIONS

Deep learning technology provides accurate and fast automated detection of mucosal ulcers on CE images. Individual patient-level analysis provided high and consistent diagnostic accuracy with shortened reading time; in the future, deep learning algorithms may augment and facilitate CE reading.

Deep Learning for Natural Language Processing in Radiology-Fundamentals and a Systematic Review

PURPOSE

Natural language processing (NLP) enables conversion of free text into structured data. Recent innovations in deep learning technology provide improved NLP performance. We aimed to survey deep learning NLP fundamentals and review radiology-related research.

METHODS

This systematic review was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. We searched for deep learning NLP radiology studies published up to September 2019. MEDLINE, Scopus, and Google Scholar were used as search databases.

RESULTS

Ten relevant studies published between 2018 and 2019 were identified. Deep learning models applied for NLP in radiology are convolutional neural networks, recurrent neural networks, long short-term memory networks, and attention networks. Deep learning NLP applications in radiology include flagging of diagnoses such as pulmonary embolisms and fractures, labeling follow-up recommendations, and automatic selection of imaging protocols. Deep learning NLP models perform as well as or better than traditional NLP models.

CONCLUSION

Research and use of deep learning NLP in radiology is increasing. Acquaintance with this technology can help prepare radiologists for the coming changes in their field.

Current Volumetric Models Overestimate Vestibular Schwannoma Size Following Stereotactic Radiosurgery

OBJECTIVE

Accurate volume assessment is essential for the management of vestibular schwannoma after stereotactic radiosurgery (SRS). A cuboidal approximation for volume is the standard surveillance method; however, this may overestimate tumor volume. We sought to evaluate several volumetric models and their suitability for post-SRS surveillance.

STUDY DESIGN

Retrospective cohort study.

SETTING

Tertiary referral center.

PATIENTS

We evaluated 54 patients with vestibular schwannoma before and after SRS.

INTERVENTION(S)

Gold-standard volumes were obtained by a radiation oncologist using contouring software. Volume was also calculated by cuboidal, ellipsoidal, and spherical formulae using tumor diameters obtained by a neuroradiologist.

MAIN OUTCOME MEASURE(S)

Percent error (PE) and absolute percent error (APE) were calculated. Paired t test evaluated bias, and the Bland-Altman method evaluated reproducibility. Linear regression evaluated predictors of model error.

RESULTS

All models overestimated volume compared with the gold standard. The cuboidal model was not reproducible before SRS (p < 0.001), and no model was reproducible after SRS (cuboidal p < 0.001; ellipsoidal p = 0.02; spherical p = 0.02). Significant bias was present before SRS for the cuboidal model (p < 0.001), and post-SRS for all models [cuboidal (p < 0.001), ellipsoidal (p < 0.02), and spherical (p = 0.005)]. Model error was negatively associated with pretreatment volume for the cuboidal (PE p = 0.03; APE p = 0.03), ellipsoidal (PE p = 0.03; APE p = 0.04), and spherical (PE p = 0.02; APE p = 0.03) methods and lost linearity post-SRS.

CONCLUSIONS

The standard cuboidal practice for following vestibular schwannoma tumor volume after SRS overestimates size. Ellipsoidal and spherical estimations have improved performance but also overestimate volume and lack reliability post-SRS. The development of other volumetric models or application of contouring software should be investigated.