A Review Paper about Deep Learning for Medical Image Analysis

Quantifying the Characteristics of Diabetic Retinopathy in Macular Optical Coherence Tomography Angiography Images: A Few-Shot Learning and Explainable Artificial Intelligence Approach

BACKGROUND

Early detection and accurate staging of diabetic retinopathy (DR) are important to prevent vision loss. Optical coherence tomography angiography (OCTA) images provide detailed insights into the retinal vasculature, revealing intricate changes that occur as DR progresses. However, interpreting these complex images requires significant expertise and is often time-intensive. Deep learning techniques have the potential to automate DR analysis. However, they typically require large datasets for effective training. To address the challenge of limited data in this emerging imaging field, a combined approach using few-shot learning (FSL) and self-attention mechanisms within explainable AI (XAI) was explored.

OBJECTIVE

To investigate and evaluate the potential of an FSL-self-attention XAI approach to improve the accuracy of DR staging classification using OCTA images.

METHODS

A total of 206 OCTA images, comprising 104 non-proliferative diabetic retinopathy (NPDR) and 102 proliferative diabetic retinopathy (PDR) cases, were analyzed using the FSL method. Three pre-trained networks (ResNet-50, DenseNet-161, and MobileNet-v2) were employed, with the top-performing model subsequently integrated with the Match-Them-Up Network (MTUNet) to provide explainable interpretations using a self-attention mechanism. The performance of the models was evaluated by applying standard metrics, including accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC-ROC). The performance of the MTUNet model is assessed by calculating pattern-matching scores for PDR and NPDR classes.

RESULTS

The ResNet-50 pre-trained model in FSL demonstrated the best overall performance, achieving an accuracy of 76.17%, a sensitivity of 81.83%, a specificity of 70.5%, and 0.82 AUC in classifying DR stages. MTUNet provided pattern-matching scores of 0.77 and 0.75 for PDR and NPDR classes, respectively.

CONCLUSIONS

FSL and self-attention mechanisms in XAI offer promising approaches for accurate DR stage classification, especially in data-limited scenarios. This could potentially facilitate early DR detection and inform clinical decision-making.

The Three-Class Annotation Method Improves the AI Detection of Early-Stage Osteosarcoma on Plain Radiographs: A Novel Approach for Rare Cancer Diagnosis

Background/Objectives: Developing high-performance artificial intelligence (AI) models for rare diseases is challenging owing to limited data availability. This study aimed to evaluate whether a novel three-class annotation method for preparing training data could enhance AI model performance in detecting osteosarcoma on plain radiographs compared to conventional single-class annotation. Methods: We developed two annotation methods for the same dataset of 468 osteosarcoma X-rays and 378 normal radiographs: a conventional single-class annotation (1C model) and a novel three-class annotation method (3C model) that separately labeled intramedullary, cortical, and extramedullary tumor components. Both models used identical U-Net-based architectures, differing only in their annotation approaches. Performance was evaluated using an independent validation dataset. Results: Although both models achieved high diagnostic accuracy (AUC: 0.99 vs. 0.98), the 3C model demonstrated superior operational characteristics. At a standardized cutoff value of 0.2, the 3C model maintained balanced performance (sensitivity: 93.28%, specificity: 92.21%), whereas the 1C model showed compromised specificity (83.58%) despite high sensitivity (98.88%). Notably, at the 25th percentile threshold, both models showed identical false-negative rates despite significantly different cutoff values (3C: 0.661 vs. 1C: 0.985), indicating the ability of the 3C model to maintain diagnostic accuracy at substantially lower thresholds. Conclusions: This study demonstrated that anatomically informed three-class annotation can enhance AI model performance for rare disease detection without requiring additional training data. The improved stability at lower thresholds suggests that thoughtful annotation strategies can optimize the AI model training, particularly in contexts where training data are limited.

Automatic plan selection using deep network-A prostate study

BACKGROUND

Recently, high-dose-rate (HDR) brachytherapy treatment plans generation was improved with the development of multicriteria optimization (MCO) algorithms that can generate thousands of pareto optimal plans within seconds. This brings a shift, from the objective of generating an acceptable plan to choosing the best plans out of thousands.

PURPOSE

In order to choose the best plans, new criteria beyond usual dosimetrics volumes histogram (DVH) metrics are introduced and a deep learning (DL) framework is added as an automatic plan selection algorithm.

METHODS

The new criteria are visual-like criteria implemented for the bladder, rectum, and urethra. One criterion also takes into account the cold spot in the prostate. Those criteria, along with commonly used DVH criteria, are used to form classes on which to train the algorithm. The algorithm is trained with an input of two 3D images, dose and mask of the anatomy, in order to rank and automatically select a plan. The confidence in the output is used for ranking and the automatic plan selection. The algorithm is trained on 835 previously treated prostate cancer patients and evaluated on a separated 20 patients cohort previously evaluated by two experts (clinical medical physicists) in an inter-observer MCO study.

RESULTS

The deep network takes 10 s to rank 2000 plans (vs. 5-10 min for experts to rank 4 preferred plans). A total of four different networks are trained which offer different trade-offs. The key trade-offs are the target coverage or the organs at risk (OAR) sparing. The algorithm with the best network achieves no statistical difference with the plans chosen by the two experts for 6 and 9 criteria, respectively, out of 13 criteria (paired t-test with p > $>$ 0.05) while the two experts have no statistical difference between them for 7 criteria.

CONCLUSIONS

The developed approach is flexible since it allows the modification or addition of criteria to obtain different trade-offs in plan quality, per the institution standard. The approach is fast and robust while adding negligible time to MCO planning. These results demonstrate potential for clinical use.

A comprehensive analysis and performance evaluation for osteoporosis prediction models

Medical data analysis is an expanding area of study that holds the promise of transforming the healthcare landscape. The use of available data by researchers gives guidelines to improve health practitioners' decision-making capacity, thus enhancing patients' lives. The study looks at using deep learning techniques to predict the onset of osteoporosis from the NHANES 2017-2020 dataset that was preprocessed and arranged into SpineOsteo and FemurOsteo datasets. Two feature selection methods, namely mutual information (MI) and recursive feature elimination (RFE), were applied to sequential deep neural network models, convolutional neural network models, and recurrent neural network models. It can be concluded from the models that the mutual information method achieved higher accuracy than recursive feature elimination, and the MI feature selection CNN model showed better performance by showing 99.15% accuracy for the SpineOsteo dataset and 99.94% classification accuracy for the FemurOsteo dataset. Key findings of this study include family medical history, cases of fractures in patients and parental hip fractures, and regular use of medications like prednisone or cortisone. The research underscores the potential for deep learning in medical data processing, which eventually opens the way for enhanced models for diagnosis and prognosis based on non-image medical data. The implications of the study shall then be important for healthcare providers to be more informed in their decision-making processes for patients' outcomes.

A Review of Datasets, Optimization Strategies, and Learning Algorithms for Analyzing Alzheimer's Dementia Detection

Alzheimer's Dementia (AD) is a progressive neurological disorder that affects memory and cognitive function, necessitating early detection for its effective management. This poses a significant challenge to global public health. The early and accurate detection of dementia is crucial for several reasons. First, timely detection facilitates early intervention and planning of treatment. Second, precise diagnostic methods are essential for distinguishing dementia from other cognitive disorders and medical conditions that may present with similar symptoms. Continuous analysis and improvements in detection methods have contributed to advancements in medical research. It helps to identify new biomarkers, refine existing diagnostic tools, and foster the development of innovative technologies, ultimately leading to more accurate and efficient diagnostic approaches for dementia. This paper presents a critical analysis of multimodal imaging datasets, learning algorithms, and optimisation techniques utilised in the context of Alzheimer's dementia detection. The focus is on understanding the advancements and challenges in employing diverse imaging modalities, such as MRI (Magnetic Resonance Imaging), PET (Positron Emission Tomography), and EEG (ElectroEncephaloGram). This study evaluated various machine learning algorithms, deep learning models, transfer learning techniques, and generative adversarial networks for the effective analysis of multi-modality imaging data for dementia detection. In addition, a critical examination of optimisation techniques encompassing optimisation algorithms and hyperparameter tuning strategies for processing and analysing images is presented in this study to discern their influence on model performance and generalisation. Thorough examination and enhancement of methods for dementia detection are fundamental for addressing the healthcare challenges posed by dementia, facilitating timely interventions, improving diagnostic accuracy, and advancing research in neurodegenerative diseases.

Enhancing MRI brain tumor classification: A comprehensive approach integrating real-life scenario simulation and augmentation techniques

Brain cancer poses a significant global health challenge, with mortality rates showing a concerning surge over recent decades. The incidence of brain cancer-related mortality has risen from 140,000 to 250,000, accompanied by a doubling in new diagnoses from 175,000 to 350,000. In response, magnetic resonance imaging (MRI) has emerged as a pivotal diagnostic tool, facilitating early detection and treatment planning. However, the translation of deep learning approaches to brain cancer diagnosis faces a critical obstacle: the scarcity of public clinical datasets reflecting real-world complexities. This study aims to bridge this gap through a comprehensive exploration and augmentation of training data. Initially, a battery of pre-trained deep models undergoes evaluation on a main brain cancer MRI "BT-MRI" dataset, yielding remarkable performance metrics, including 100% accuracy, precision, recall, and F1-Score, substantiated by the Score-CAM methodology. This initial success underscores the potential of deep learning in brain cancer diagnosis. Subsequently, the model's efficacy undergoes further scrutiny using a supplementary brain cancer MRI "BCD-MRI" dataset, affirming its robustness and applicability across diverse datasets. However, the ultimate litmus test lies in confronting the model with synthetic testing datasets crafted to emulate real-world scenarios. The synthetic testing datasets, a BCD-MRI testing sub-dataset enriched with noise, blur, and simulated patient motion, reveal a sobering reality: the model's performance plummets, exposing inherent limitations in generalization. To address this issue, a diverse set of optimization strategies and augmentation techniques, ranging from diverse optimizers to sophisticated data augmentation methods, are exhaustively explored. Despite these efforts, the problem of generalization persists. The breakthrough emerges with the integration of noise and blur as augmentation techniques during the training process. Leveraging Gaussian noise and Gaussian blur kernels, the model undergoes a transformative evolution, exhibiting newfound robustness and resilience. Retesting the refined model against the challenging synthetic datasets reveals a remarkable transformation, with performance metrics witnessing a notable ascent. This achievement underscores the important role of correct selection of data augmentation in fortifying the generalization of deep learning models for brain cancer diagnosis. This study not only advances the frontiers of diagnostic precision in brain cancer but also underscores the paramount importance of methodological rigor and innovation in confronting the complexities of real-world clinical scenarios.

Signaling pathways involved in colorectal cancer: pathogenesis and targeted therapy

Colorectal cancer (CRC) remains one of the leading causes of cancer-related mortality worldwide. Its complexity is influenced by various signal transduction networks that govern cellular proliferation, survival, differentiation, and apoptosis. The pathogenesis of CRC is a testament to the dysregulation of these signaling cascades, which culminates in the malignant transformation of colonic epithelium. This review aims to dissect the foundational signaling mechanisms implicated in CRC, to elucidate the generalized principles underpinning neoplastic evolution and progression. We discuss the molecular hallmarks of CRC, including the genomic, epigenomic and microbial features of CRC to highlight the role of signal transduction in the orchestration of the tumorigenic process. Concurrently, we review the advent of targeted and immune therapies in CRC, assessing their impact on the current clinical landscape. The development of these therapies has been informed by a deepening understanding of oncogenic signaling, leading to the identification of key nodes within these networks that can be exploited pharmacologically. Furthermore, we explore the potential of integrating AI to enhance the precision of therapeutic targeting and patient stratification, emphasizing their role in personalized medicine. In summary, our review captures the dynamic interplay between aberrant signaling in CRC pathogenesis and the concerted efforts to counteract these changes through targeted therapeutic strategies, ultimately aiming to pave the way for improved prognosis and personalized treatment modalities in colorectal cancer.

Predicting multiple sclerosis disease progression and outcomes with machine learning and MRI-based biomarkers: a review

Multiple sclerosis (MS) is a demyelinating neurological disorder with a highly heterogeneous clinical presentation and course of progression. Disease-modifying therapies are the only available treatment, as there is no known cure for the disease. Careful selection of suitable therapies is necessary, as they can be accompanied by serious risks and adverse effects such as infection. Magnetic resonance imaging (MRI) plays a central role in the diagnosis and management of MS, though MRI lesions have displayed only moderate associations with MS clinical outcomes, known as the clinico-radiological paradox. With the advent of machine learning (ML) in healthcare, the predictive power of MRI can be improved by leveraging both traditional and advanced ML algorithms capable of analyzing increasingly complex patterns within neuroimaging data. The purpose of this review was to examine the application of MRI-based ML for prediction of MS disease progression. Studies were divided into five main categories: predicting the conversion of clinically isolated syndrome to MS, cognitive outcome, EDSS-related disability, motor disability and disease activity. The performance of ML models is discussed along with highlighting the influential MRI-derived biomarkers. Overall, MRI-based ML presents a promising avenue for MS prognosis. However, integration of imaging biomarkers with other multimodal patient data shows great potential for advancing personalized healthcare approaches in MS.

DeepPlaq: Dental plaque indexing based on deep neural networks

OBJECTIVES

The selection of treatment for dental plaque is closely related to the condition of the plaque on different teeth. This study validated the ability of CNN models in assessing the dental plaque indices.

MATERIALS AND METHODS

In 70 (20 male and 50 female) healthy adults (18 to 55 years old), frontal and lateral view intraoral images (210) of plaque disclosing agent stained permanent and deciduous dentitions were obtained. A three-stage method was employed, where the You Look Only Once version 8 (YOLOv8) model was first used to detect the target teeth, followed by the prompt-based Segment Anything Model (SAM) segmentation algorithm to segment teeth. A new single-tooth dataset consisting of 1400 photographs was obtained after applying a two-stage method. Finally, a multi-class classification model DeepPlaq was trained and evaluated on the accuracy of dental plaque indexing based on the Quigley-Hein Index (QHI) scoring system. Classification performance was measured using accuracy, recall, precision, and F1-score.

RESULTS

The teeth detector exhibited an accuracy (mean average precision, mAP) of approximately 0.941 ± 0.005 in identifying teeth with plaque disclosing agents. The maximum accuracy attained in the plaque indexing through DeepPlaq was 0.84 (probability that DeepPlaq scored identical to experts), and the smallest average scoring error was less than 0.25 on a 0 to 5 scale for scoring.

CONCLUSIONS

A three-stage approach demonstrated excellent performance in detecting and segmenting target teeth, and DeepPlaq model also showed strong performance in assessing dental plaque indices.

CLINICAL RELEVANCE

Application of artificial intelligence to the evaluation of dental plaque distribution could enhance diagnostic accuracy and treatment efficiency and accuracy.

ChoroidSeg-ViT: A Transformer Model for Choroid Layer Segmentation Based on a Mixed Attention Feature Enhancement Mechanism

Purpose

To develop a Vision Transformer (ViT) model based on the mixed attention feature enhancement mechanism, ChoroidSeg-ViT, for choroid layer segmentation in optical coherence tomography (OCT) images.

Methods

This study included a dataset of 100 OCT B-scans images. Ground truths were carefully labeled by experienced ophthalmologists. An end-to-end local-enhanced Transformer model, ChoroidSeg-ViT, was designed to segment the choroid layer by integrating the local enhanced feature extraction and semantic feature fusion paths. Standard segmentation metrics were selected to evaluate ChoroidSeg-ViT.

Results

Experimental results demonstrate that ChoroidSeg-ViT exhibited superior segmentation performance (mDice: 98.31, mIoU: 96.62, mAcc: 98.29) compared to other deep learning approaches, thus indicating the effectiveness and superiority of this proposed model for the choroid layer segmentation task. Furthermore, ablation and generalization experiments validated the reasonableness of the module design.

Conclusions

We developed a novel Transformer model to precisely and automatically segment the choroid layer and achieved the state-of-the-art performance.

Translational Relevance

ChoroidSeg-ViT could segment precise and smooth choroid layers and form the basis of an automatic choroid analysis system that would facilitate future choroidal research in ophthalmology.

Detection and counting of Leishmania intracellular parasites in microscopy images

Problem

Leishmaniasis is a disease caused by protozoan parasites of the genus Leishmania and has a high prevalence and impact on global health. Currently, the available drugs for its treatment have drawbacks, such as high toxicity, resistance of the parasite, and high cost. Therefore, the search for new, more effective, and safe drugs is a priority. The effectiveness of an anti-leishmanial drug is analyzed through in vitro studies in which a technician manually counts the intracellular form of the parasite (amastigote) within macrophages, which is slow, laborious, and prone to errors.

Objectives

To develop a computational system that facilitates the detection and counting of amastigotes in microscopy images obtained from in vitro studies using image processing techniques.

Methodology

Segmentation of objects in the microscope image that might be Leishmania amastigotes was performed using the multilevel Otsu method on the saturation component of the hue, saturation, and intensity color model. In addition, morphological operations and the watershed transform combined with the weighted external distance transform were used to separate clustered objects. Then positive (amastigote) objects were detected (and consequently counted) using a classifier algorithm, the selection of which as well as the definition of the features to be used were also part of this research. MATLAB was used for the development of the system.

Results and discussion

The results were evaluated in terms of sensitivity, precision, and the F-measure and suggested a favorable effectiveness of the proposed method.

Conclusions

This system can help researchers by allowing large volumes of images of amastigotes to be counted using an automatic image analysis technique.

Convolutional neural network-assisted diagnosis of midpalatal suture maturation stage in cone-beam computed tomography

OBJECTIVES

The selection of treatment for maxillary expansion is closely related to the calcification degree of the midpalatal suture. A classification method for individual assessment of the morphology of midpalatal suture in cone-beam computed tomography (CBCT) is useful for evaluating the calcification degree. Currently, convolutional neural networks (CNNs) have been introduced into the field of oral and maxillofacial imaging diagnosis. This study validated the ability of CNN models in assessing the maturation stage of the midpalatal suture.

METHODS

The existing CNN model ResNet50 was trained to locate the CBCT transverse plane which contained a complete midpalatal suture. ResNet18, ResNet50, RessNet101, Inception-v3, and Efficientnetv2-s models were trained to evaluate the midpalatal suture maturation stage. Multi-class classification metrics, accuracy, recall, precision, F1-score, and area under the curve values from the receiver operating characteristic curve were used to evaluate the performance of the models, and gradient-weighted class activation map technology was utilised to visualise five midpalatal suture maturation stages for each model.

RESULTS

Resnet50 demonstrated an accuracy of 99.74 % in identifying the transverse plane that contained the complete midpalatal suture. The highest accuracies achieved on the two-stage, three-stage, and five-stage maturation classification tests were 95.15, 88.06, and 75.37 %, all of which exceeded the average accuracy of three experienced orthodontists.

CONCLUSIONS

The CNN model can locate the plane of the midpalatal suture in CBCT images and can assist clinicians in assessing the maturation stage of the midpalatal suture to select the means of maxillary expansion.

CLINICAL SIGNIFICANCE

The application of artificial intelligence on CBCT midpalatal suture plane localisation and maturation stage evaluation enhances diagnostic and treatment efficiency and accuracy of individual assessment of midpalatal suture calcification degree. Additionally, it assists the clinical palatal expansion technique in achieving ideal results.