Convolutional neural networks for medical image analysis: State-of-the-art, comparisons, improvement and perspectives

Classification of land lot shapes in real estate sector using a convolutional neural network

In the agriculture and real estate industries, land lot shapes have mostly been classified by visual inspection or hard-crafted rules. These conventional methods are time-consuming, resource-intensive, and subject to human bias. This study aims to fill this gap and alleviate problems inherent in traditional lot classification approaches. This study attempts to classify lot shapes automatically, using a convolutional neural network. A study area was chosen, image data of the lots in the area were collected and preprocessed, and an Xception neural network was specified to classify land lots according to their shapes. The test applied to a different area adjacent to the study area achieved an accuracy of 90.1% and area under the curve (AUC) of 0.96. Additionally, this study demonstrated that shape regularity can be quantified using the output scores from the neural network analysis. This is the first attempt to employ a deep learning algorithm for land management on a micro-spatial scale. The classification approach proposed in this study is expected to encourage the rapid and accurate classification of various lot shapes.

Cross-domain additive learning of new knowledge rather than replacement

In medical clinical scenarios for reasons such as patient privacy, information protection and data migration, when domain adaptation is needed for real scenarios, the source-domain data is often inaccessible and only the pre-trained source model on the source-domain is available. Existing solutions for this type of problem tend to forget the rich task experience previously learned on the source domain after adapting, which means that the model simply overfits the target-domain data when adapting and does not learn robust features that facilitate real task decisions. We address this problem by exploring the particular application of source-free domain adaptation in medical image segmentation and propose a two-stage additive source-free adaptation framework. We generalize the domain-invariant features by constraining the core pathological structure and semantic consistency between different perspectives. And we reduce the segmentation generated by locating and filtering elements that may have errors through Monte-Carlo uncertainty estimation. We conduct comparison experiments with some other methods on a cross-device polyp segmentation and a cross-modal brain tumor segmentation dataset, the results in both the target and source domains verify that the proposed method can effectively solve the domain offset problem and the model retains its dominance on the source domain after learning new knowledge of the target domain.This work provides valuable exploration for achieving additive learning on the target and source domains in the absence of source data and offers new ideas and methods for adaptation research in the field of medical image segmentation.

MELEP: A Novel Predictive Measure of Transferability in Multi-label ECG Diagnosis

In practical electrocardiography (ECG) interpretation, the scarcity of well-annotated data is a common challenge. Transfer learning techniques are valuable in such situations, yet the assessment of transferability has received limited attention. To tackle this issue, we introduce MELEP, which stands for Muti-label Expected Log of Empirical Predictions, a measure designed to estimate the effectiveness of knowledge transfer from a pre-trained model to a downstream multi-label ECG diagnosis task. MELEP is generic, working with new target data with different label sets, and computationally efficient, requiring only a single forward pass through the pre-trained model. To the best of our knowledge, MELEP is the first transferability metric specifically designed for multi-label ECG classification problems. Our experiments show that MELEP can predict the performance of pre-trained convolutional and recurrent deep neural networks, on small and imbalanced ECG data. Specifically, we observed strong correlation coefficients (with absolute values exceeding 0.6 in most cases) between MELEP and the actual average F1 scores of the fine-tuned models. Our work highlights the potential of MELEP to expedite the selection of suitable pre-trained models for ECG diagnosis tasks, saving time and effort that would otherwise be spent on fine-tuning these models.

Performance Metrics, Algorithms, and Applications of Artificial Intelligence in Vascular and Interventional Neurology: A Review of Basic Elements

Artificial intelligence (AI) is currently being used as a routine tool for day-to-day activity. Medicine is not an exception to the growing usage of AI in various scientific fields. Vascular and interventional neurology deal with diseases that require early diagnosis and appropriate intervention, which are crucial to saving patients' lives. In these settings, AI can be an extra pair of hands for physicians or in conditions where there is a shortage of clinical experts. In this article, the authors have reviewed the common metrics used in interpreting the performance of models and common algorithms used in this field.

Radiograph-based rheumatoid arthritis diagnosis via convolutional neural network

OBJECTIVES

Rheumatoid arthritis (RA) is a severe and common autoimmune disease. Conventional diagnostic methods are often subjective, error-prone, and repetitive works. There is an urgent need for a method to detect RA accurately. Therefore, this study aims to develop an automatic diagnostic system based on deep learning for recognizing and staging RA from radiographs to assist physicians in diagnosing RA quickly and accurately.

METHODS

We develop a CNN-based fully automated RA diagnostic model, exploring five popular CNN architectures on two clinical applications. The model is trained on a radiograph dataset containing 240 hand radiographs, of which 39 are normal and 201 are RA with five stages. For evaluation, we use 104 hand radiographs, of which 13 are normal and 91 RA with five stages.

RESULTS

The CNN model achieves good performance in RA diagnosis based on hand radiographs. For the RA recognition, all models achieve an AUC above 90% with a sensitivity over 98%. In particular, the AUC of the GoogLeNet-based model is 97.80%, and the sensitivity is 100.0%. For the RA staging, all models achieve over 77% AUC with a sensitivity over 80%. Specifically, the VGG16-based model achieves 83.36% AUC with 92.67% sensitivity.

CONCLUSION

The presented GoogLeNet-based model and VGG16-based model have the best AUC and sensitivity for RA recognition and staging, respectively. The experimental results demonstrate the feasibility and applicability of CNN in radiograph-based RA diagnosis. Therefore, this model has important clinical significance, especially for resource-limited areas and inexperienced physicians.

Predicting recovery following stroke: Deep learning, multimodal data and feature selection using explainable AI

Machine learning offers great potential for automated prediction of post-stroke symptoms and their response to rehabilitation. Major challenges for this endeavour include the very high dimensionality of neuroimaging data, the relatively small size of the datasets available for learning and interpreting the predictive features, as well as, how to effectively combine neuroimaging and tabular data (e.g. demographic information and clinical characteristics). This paper evaluates several solutions based on two strategies. The first is to use 2D images that summarise MRI scans. The second is to select key features that improve classification accuracy. Additionally, we introduce the novel approach of training a convolutional neural network (CNN) on images that combine regions-of-interests (ROIs) extracted from MRIs, with symbolic representations of tabular data. We evaluate a series of CNN architectures (both 2D and a 3D) that are trained on different representations of MRI and tabular data, to predict whether a composite measure of post-stroke spoken picture description ability is in the aphasic or non-aphasic range. MRI and tabular data were acquired from 758 English speaking stroke survivors who participated in the PLORAS study. Each participant was assigned to one of five different groups that were matched for initial severity of symptoms, recovery time, left lesion size and the months or years post-stroke that spoken description scores were collected. Training and validation were carried out on the first four groups. The fifth (lock-box/test set) group was used to test how well model accuracy generalises to new (unseen) data. The classification accuracy for a baseline logistic regression was 0.678 based on lesion size alone, rising to 0.757 and 0.813 when initial symptom severity and recovery time were successively added. The highest classification accuracy (0.854), area under the curve (0.899) and F1 score (0.901) were observed when 8 regions of interest were extracted from each MRI scan and combined with lesion size, initial severity and recovery time in a 2D Residual Neural Network (ResNet). This was also the best model when data were limited to the 286 participants with moderate or severe initial aphasia (with area under curve = 0.865), a group that would be considered more difficult to classify. Our findings demonstrate how imaging and tabular data can be combined to achieve high post-stroke classification accuracy, even when the dataset is small in machine learning terms. We conclude by proposing how the current models could be improved to achieve even higher levels of accuracy using images from hospital scanners.

Assessment of artificial intelligence to detect gasoline in fire debris using HS-SPME-GC/MS and transfer learning

Due to the complex nature of the chemical compositions of ignitable liquids (IL) and the interferences from fire debris matrices, interpreting chromatographic data poses challenges to analysts. In this work, artificial intelligence (AI) was developed by transfer learning in a convolutional neural network (CNN), GoogLeNet. The image classification AI was fine-tuned to create intelligent classification systems to discriminate samples containing gasoline residues from burned substrates. All ground truth samples were analyzed by headspace solid-phase microextraction (HS-SPME) coupled with a gas chromatograph and mass spectrometer (GC/MS). The HS-SPME-GC/MS data were transformed into three types of image presentations, that is, heatmaps, extracted ion heatmaps, and total ion chromatograms. The abundance and mass-to-charge ratios of each scan were converted into image patterns that are characteristic of the chemical profiles of gasoline. The transfer learning data were labeled as "gasoline present" and "gasoline absent" classes. The assessment results demonstrated that all AI models achieved 100 ± 0% accuracy in identifying neat gasoline. When the models were assessed using the spiked samples, the AI model developed using the extracted ion heatmap obtained the highest accuracy rate (95.9 ± 0.4%), which was greater than those obtained by other machine learning models, ranging from 17.3 ± 0.7% to 78.7 ± 0.7%. The proposed work demonstrated that the heatmaps created from GC/MS data can represent chemical features from the samples. Additionally, the pretrained CNN models are readily available in the transfer learning workflow to develop AI for GC/MS data interpretation in fire debris analysis.

Characterization of Heart Diseases per Single Lead Using ECG Images and CNN-2D

Cardiopathy has become one of the predominant global causes of death. The timely identification of different types of heart diseases significantly diminishes mortality risk and enhances the efficacy of treatment. However, fast and efficient recognition necessitates continuous monitoring, encompassing not only specific clinical conditions but also diverse lifestyles. Consequently, an increasing number of studies are striving to automate and progress in the identification of different cardiopathies. Notably, the assessment of electrocardiograms (ECGs) is crucial, given that it serves as the initial diagnostic test for patients, proving to be both the simplest and the most cost-effective tool. This research employs a customized architecture of Convolutional Neural Network (CNN) to forecast heart diseases by analyzing the images of both three bands of electrodes and of each single electrode signal of the ECG derived from four distinct patient categories, representing three heart-related conditions as well as a spectrum of healthy controls. The analyses are conducted on a real dataset, providing noteworthy performance (recall greater than 80% for the majority of the considered diseases and sometimes even equal to 100%) as well as a certain degree of interpretability thanks to the understanding of the importance a band of electrodes or even a single ECG electrode can have in detecting a specific heart-related pathology.

Deep learning and content-based filtering techniques for improving plant disease identification and treatment recommendations: A comprehensive review

The importance of identifying plant diseases has risen recently due to the adverse effect they have on agricultutal production. Plant diseases have been a big concern in agriculture, as they affect crop production, and constitute a major threat to global food security. In the domain of modern agriculture, effective plant disease management is vital to ensure healthy crop yields and sustainable practices. Traditional means of identifying plant disease are faced with lots of challenges and the need for better and efficient detection methods cannot be overemphazised. The emergence of advanced technologies, particularly deep learning and content-based filtering techniques, if integrated together can changed the way plant diseases are identified and treated. Such as speedy and correct identification of plant diseases and efficient treatment recommendations which are keys for sustainable food production. In this work, We try to investigate the current state of research, identified gaps and limitations in knowledge, and suggests future directions for researchers, experts and farmers that could help to provide better ways of mitigating plant disease problems.

Don't Fear the Artificial Intelligence: A Systematic Review of Machine Learning for Prostate Cancer Detection in Pathology

CONTEXT

Automated prostate cancer detection using machine learning technology has led to speculation that pathologists will soon be replaced by algorithms. This review covers the development of machine learning algorithms and their reported effectiveness specific to prostate cancer detection and Gleason grading.

OBJECTIVE

To examine current algorithms regarding their accuracy and classification abilities. We provide a general explanation of the technology and how it is being used in clinical practice. The challenges to the application of machine learning algorithms in clinical practice are also discussed.

DATA SOURCES

The literature for this review was identified and collected using a systematic search. Criteria were established prior to the sorting process to effectively direct the selection of studies. A 4-point system was implemented to rank the papers according to their relevancy. For papers accepted as relevant to our metrics, all cited and citing studies were also reviewed. Studies were then categorized based on whether they implemented binary or multi-class classification methods. Data were extracted from papers that contained accuracy, area under the curve (AUC), or κ values in the context of prostate cancer detection. The results were visually summarized to present accuracy trends between classification abilities.

CONCLUSIONS

It is more difficult to achieve high accuracy metrics for multiclassification tasks than for binary tasks. The clinical implementation of an algorithm that can assign a Gleason grade to clinical whole slide images (WSIs) remains elusive. Machine learning technology is currently not able to replace pathologists but can serve as an important safeguard against misdiagnosis.

Deep Learning in Breast Cancer Imaging: State of the Art and Recent Advancements in Early 2024

The rapid advancement of artificial intelligence (AI) has significantly impacted various aspects of healthcare, particularly in the medical imaging field. This review focuses on recent developments in the application of deep learning (DL) techniques to breast cancer imaging. DL models, a subset of AI algorithms inspired by human brain architecture, have demonstrated remarkable success in analyzing complex medical images, enhancing diagnostic precision, and streamlining workflows. DL models have been applied to breast cancer diagnosis via mammography, ultrasonography, and magnetic resonance imaging. Furthermore, DL-based radiomic approaches may play a role in breast cancer risk assessment, prognosis prediction, and therapeutic response monitoring. Nevertheless, several challenges have limited the widespread adoption of AI techniques in clinical practice, emphasizing the importance of rigorous validation, interpretability, and technical considerations when implementing DL solutions. By examining fundamental concepts in DL techniques applied to medical imaging and synthesizing the latest advancements and trends, this narrative review aims to provide valuable and up-to-date insights for radiologists seeking to harness the power of AI in breast cancer care.

Computer vision model for the detection of canine pododermatitis and neoplasia of the paw

BACKGROUND

Artificial intelligence (AI) has been used successfully in human dermatology. AI utilises convolutional neural networks (CNN) to accomplish tasks such as image classification, object detection and segmentation, facilitating early diagnosis. Computer vision (CV), a field of AI, has shown great results in detecting signs of human skin diseases. Canine paw skin diseases are a common problem in general veterinary practice, and computer vision tools could facilitate the detection and monitoring of disease processes. Currently, no such tool is available in veterinary dermatology.

ANIMALS

Digital images of paws from healthy dogs and paws with pododermatitis or neoplasia were used.

OBJECTIVES

We tested the novel object detection model Pawgnosis, a Tiny YOLOv4 image analysis model deployed on a microcomputer with a camera for the rapid detection of canine pododermatitis and neoplasia.

MATERIALS AND METHODS

The prediction performance metrics used to evaluate the models included mean average precision (mAP), precision, recall, average precision (AP) for accuracy and frames per second (FPS) for speed.

RESULTS

A large dataset labelled by a single individual (Dataset A) used to train a Tiny YOLOv4 model provided the best results with a mean mAP of 0.95, precision of 0.86, recall of 0.93 and 20 FPS.

CONCLUSIONS AND CLINICAL RELEVANCE

This novel object detection model has the potential for application in the field of veterinary dermatology.

Small bowel capsule endoscopy examination and open access database with artificial intelligence: The SEE-artificial intelligence project

OBJECTIVES

Artificial intelligence (AI) may be practical for image classification of small bowel capsule endoscopy (CE). However, creating a functional AI model is challenging. We attempted to create a dataset and an object detection CE AI model to explore modeling problems to assist in reading small bowel CE.

METHODS

We extracted 18,481 images from 523 small bowel CE procedures performed at Kyushu University Hospital from September 2014 to June 2021. We annotated 12,320 images with 23,033 disease lesions, combined them with 6161 normal images as the dataset, and examined the characteristics. Based on the dataset, we created an object detection AI model using YOLO v5 and we tested validation.

RESULTS

We annotated the dataset with 12 types of annotations, and multiple annotation types were observed in the same image. We test validated our AI model with 1396 images, and sensitivity for all 12 types of annotations was about 91%, with 1375 true positives, 659 false positives, and 120 false negatives detected. The highest sensitivity for individual annotations was 97%, and the highest area under the receiver operating characteristic curve was 0.98, but the quality of detection varied depending on the specific annotation.

CONCLUSIONS

Object detection AI model in small bowel CE using YOLO v5 may provide effective and easy-to-understand reading assistance. In this SEE-AI project, we open our dataset, the weights of the AI model, and a demonstration to experience our AI. We look forward to further improving the AI model in the future.

Automatedly identify dryland threatened species at large scale by using deep learning

Dryland biodiversity is decreasing at an alarming rate. Advanced intelligent tools are urgently needed to rapidly, automatedly, and precisely detect dryland threatened species on a large scale for biological conservation. Here, we explored the performance of three deep convolutional neural networks (Deeplabv3+, Unet, and Pspnet models) on the intelligent recognition of rare species based on high-resolution (0.3 m) satellite images taken by an unmanned aerial vehicle (UAV). We focused on a threatened species, Populus euphratica, in the Tarim River Basin (China), where there has been a severe population decline in the 1970s and restoration has been carried out since 2000. The testing results showed that Unet outperforms Deeplabv3+ and Pspnet when the training samples are lower, while Deeplabv3+ performs best as the dataset increases. Overall, when training samples are 80, Deeplabv3+ had the best overall performance for Populus euphratica identification, with mean pixel accuracy (MPA) between 87.31 % and 90.2 %, which, on average is 3.74 % and 11.29 % higher than Unet and Pspnet, respectively. Deeplabv3+ can accurately detect the boundaries of Populus euphratica even in areas of dense vegetation, with lower identification uncertainty for each pixel than other models. This study developed a UAV imagery-based identification framework using deep learning with high resolution in large-scale regions. This approach can accurately capture the variation in dryland threatened species, especially those in inaccessible areas, thereby fostering rapid and efficient conservation actions.

Shallow and deep learning classifiers in medical image analysis

An increasingly strong connection between artificial intelligence and medicine has enabled the development of predictive models capable of supporting physicians' decision-making. Artificial intelligence encompasses much more than machine learning, which nevertheless is its most cited and used sub-branch in the last decade. Since most clinical problems can be modeled through machine learning classifiers, it is essential to discuss their main elements. This review aims to give primary educational insights on the most accessible and widely employed classifiers in radiology field, distinguishing between "shallow" learning (i.e., traditional machine learning) algorithms, including support vector machines, random forest and XGBoost, and "deep" learning architectures including convolutional neural networks and vision transformers. In addition, the paper outlines the key steps for classifiers training and highlights the differences between the most common algorithms and architectures. Although the choice of an algorithm depends on the task and dataset dealing with, general guidelines for classifier selection are proposed in relation to task analysis, dataset size, explainability requirements, and available computing resources. Considering the enormous interest in these innovative models and architectures, the problem of machine learning algorithms interpretability is finally discussed, providing a future perspective on trustworthy artificial intelligence.Relevance statement The growing synergy between artificial intelligence and medicine fosters predictive models aiding physicians. Machine learning classifiers, from shallow learning to deep learning, are offering crucial insights for the development of clinical decision support systems in healthcare. Explainability is a key feature of models that leads systems toward integration into clinical practice. Key points • Training a shallow classifier requires extracting disease-related features from region of interests (e.g., radiomics).• Deep classifiers implement automatic feature extraction and classification.• The classifier selection is based on data and computational resources availability, task, and explanation needs.

MS-TCNet: An effective Transformer-CNN combined network using multi-scale feature learning for 3D medical image segmentation

Medical image segmentation is a fundamental research problem in the field of medical image processing. Recently, the Transformer have achieved highly competitive performance in computer vision. Therefore, many methods combining Transformer with convolutional neural networks (CNNs) have emerged for segmenting medical images. However, these methods cannot effectively capture the multi-scale features in medical images, even though texture and contextual information embedded in the multi-scale features are extremely beneficial for segmentation. To alleviate this limitation, we propose a novel Transformer-CNN combined network using multi-scale feature learning for three-dimensional (3D) medical image segmentation, which is called MS-TCNet. The proposed model utilizes a shunted Transformer and CNN to construct an encoder and pyramid decoder, allowing six different scale levels of feature learning. It captures multi-scale features with refinement at each scale level. Additionally, we propose a novel lightweight multi-scale feature fusion (MSFF) module that can fully fuse the different-scale semantic features generated by the pyramid decoder for each segmentation class, resulting in a more accurate segmentation output. We conducted experiments on three widely used 3D medical image segmentation datasets. The experimental results indicated that our method outperformed state-of-the-art medical image segmentation methods, suggesting its effectiveness, robustness, and superiority. Meanwhile, our model has a smaller number of parameters and lower computational complexity than conventional 3D segmentation networks. The results confirmed that the model is capable of effective multi-scale feature learning and that the learned multi-scale features are useful for improving segmentation performance. We open-sourced our code, which can be found at https://github.com/AustinYuAo/MS-TCNet.

Stratification of tumour cell radiation response and metabolic signatures visualization with Raman spectroscopy and explainable convolutional neural network

Reprogramming of cellular metabolism is a driving factor of tumour progression and radiation therapy resistance. Identifying biochemical signatures associated with tumour radioresistance may assist with the development of targeted treatment strategies to improve clinical outcomes. Raman spectroscopy (RS) can monitor post-irradiation biomolecular changes and signatures of radiation response in tumour cells in a label-free manner. Convolutional Neural Networks (CNN) perform feature extraction directly from data in an end-to-end learning manner, with high classification performance. Furthermore, recently developed CNN explainability techniques help visualize the critical discriminative features captured by the model. In this work, a CNN is developed to characterize tumour response to radiotherapy based on its degree of radioresistance. The model was trained to classify Raman spectra of three human tumour cell lines as radiosensitive (LNCaP) or radioresistant (MCF7, H460) over a range of treatment doses and data collection time points. Additionally, a method based on Gradient-Weighted Class Activation Mapping (Grad-CAM) was used to determine response-specific salient Raman peaks influencing the CNN predictions. The CNN effectively classified the cell spectra, with accuracy, sensitivity, specificity, and F1 score exceeding 99.8%. Grad-CAM heatmaps of H460 and MCF7 cell spectra (radioresistant) exhibited high contributions from Raman bands tentatively assigned to glycogen, amino acids, and nucleic acids. Conversely, heatmaps of LNCaP cells (radiosensitive) revealed activations at lipid and phospholipid bands. Finally, Grad-CAM variable importance scores were derived for glycogen, asparagine, and phosphatidylcholine, and we show that their trends over cell line, dose, and acquisition time agreed with previously established models. Thus, the CNN can accurately detect biomolecular differences in the Raman spectra of tumour cells of varying radiosensitivity without requiring manual feature extraction. Finally, Grad-CAM may help identify metabolic signatures associated with the observed categories, offering the potential for automated clinical tumour radiation response characterization.

Binary decisions of artificial intelligence to classify third molar development around the legal age thresholds of 14, 16 and 18 years

Third molar development is used for dental age estimation when all the other teeth are fully mature. In most medicolegal facilities, dental age estimation is an operator-dependent procedure. During the examination of unaccompanied and undocumented minors, this procedure may lead to binary decisions around age thresholds of legal interest, namely the ages of 14, 16 and 18 years. This study aimed to test the performance of artificial intelligence to classify individuals below and above the legal age thresholds of 14, 16 and 18 years using third molar development. The sample consisted of 11,640 panoramic radiographs (9680 used for training and 1960 used for validation) of males (n = 5400) and females (n = 6240) between 6 and 22.9 years. Computer-based image annotation was performed with V7 software (V7labs, London, UK). The region of interest was the mandibular left third molar (T38) outlined with a semi-automated contour. DenseNet121 was the Convolutional Neural Network (CNN) of choice and was used with Transfer Learning. After Receiver-operating characteristic curves, the area under the curve (AUC) was 0.87 and 0.86 to classify males and females below and above the age of 14, respectively. For the age threshold of 16, the AUC values were 0.88 (males) and 0.83 (females), while for the age of 18, AUC were 0.94 (males) and 0.83 (females). Specificity rates were always between 0.80 and 0.92. Artificial intelligence was able to classify male and females below and above the legal age thresholds of 14, 16 and 18 years with high accuracy.

A Preliminary Investigation into Search and Matching for Tumor Discrimination in World Health Organization Breast Taxonomy Using Deep Networks

Breast cancer is one of the most common cancers affecting women worldwide. It includes a group of malignant neoplasms with a variety of biological, clinical, and histopathologic characteristics. There are more than 35 different histologic forms of breast lesions that can be classified and diagnosed histologically according to cell morphology, growth, and architecture patterns. Recently, deep learning, in the field of artificial intelligence, has drawn a lot of attention for the computerized representation of medical images. Searchable digital atlases can provide pathologists with patch-matching tools, allowing them to search among evidently diagnosed and treated archival cases, a technology that may be regarded as computational second opinion. In this study, we indexed and analyzed the World Health Organization breast taxonomy (Classification of Tumors fifth ed.) spanning 35 tumor types. We visualized all tumor types using deep features extracted from a state-of-the-art deep-learning model, pretrained on millions of diagnostic histopathology images from the Cancer Genome Atlas repository. Furthermore, we tested the concept of a digital "atlas" as a reference for search and matching with rare test cases. The patch similarity search within the World Health Organization breast taxonomy data reached >88% accuracy when validating through "majority vote" and >91% accuracy when validating using top n tumor types. These results show for the first time that complex relationships among common and rare breast lesions can be investigated using an indexed digital archive.

Acquisition-invariant brain MRI segmentation with informative uncertainties

Combining multi-site data can strengthen and uncover trends, but is a task that is marred by the influence of site-specific covariates that can bias the data and, therefore, any downstream analyses. Post-hoc multi-site correction methods exist but have strong assumptions that often do not hold in real-world scenarios. Algorithms should be designed in a way that can account for site-specific effects, such as those that arise from sequence parameter choices, and in instances where generalisation fails, should be able to identify such a failure by means of explicit uncertainty modelling. This body of work showcases such an algorithm that can become robust to the physics of acquisition in the context of segmentation tasks while simultaneously modelling uncertainty. We demonstrate that our method not only generalises to complete holdout datasets, preserving segmentation quality but does so while also accounting for site-specific sequence choices, which also allows it to perform as a harmonisation tool.

[Short paths to diagnosis with artificial intelligence: systematic literature review on diagnostic decision support systems]

BACKGROUND

Rare diseases are often recognized late. Their diagnosis is particularly challenging due to the diversity, complexity and heterogeneity of clinical symptoms. Computer-aided diagnostic aids, often referred to as diagnostic decision support systems (DDSS), are promising tools for shortening the time to diagnosis. Despite initial positive evaluations, DDSS are not yet widely used, partly due to a lack of integration with existing clinical or practice information systems.

OBJECTIVE

This article provides an insight into currently existing diagnostic support systems that function without access to electronic patient records and only require information that is easily obtainable.

MATERIALS AND METHODS

A systematic literature search identified eight articles on DDSS that can assist in the diagnosis of rare diseases with no need for access to electronic patient records or other information systems in practices and hospitals. The main advantages and disadvantages of the identified rare disease diagnostic support systems were extracted and summarized.

RESULTS

Symptom checkers and DDSS based on portrait photos and pain drawings already exist. The degree of maturity of these applications varies.

CONCLUSION

DDSS currently still face a number of challenges, such as concerns about data protection and accuracy, and acceptance and awareness continue to be rather low. On the other hand, there is great potential for faster diagnosis, especially for rare diseases, which are easily overlooked due to their large number and the low awareness of them. The use of DDSS should therefore be carefully considered by doctors on a case-by-case basis.

Few-shot learning based oral cancer diagnosis using a dual feature extractor prototypical network

A large global health issue is cancer, wherein early diagnosis and treatment have proven to be life-saving. This holds true for oral cancer, thus emphasizing the significance of timely intervention. Deep learning techniques have gained traction in early cancer detection, exhibiting promising outcomes in accurate diagnosis. However, collecting a substantial amount of training data poses a challenge for deep learning models in cancer diagnosis. To address this limitation, this study proposes an oral cancer diagnosis approach based on a few-shot learning framework that circumvents the need for extensive training data. Specifically, a prototypical network is employed to construct a diagnostic model, wherein two feature extractors are utilized to extract prototypical features and query features respectively, departing from the conventional use of a single feature extraction function in prototypical networks. Moreover, a customized loss function is designed for the proposed method. Rigorous experimentation using a histopathological image dataset demonstrates the superior performance of our proposed approach over comparison methods.

Research in the application of artificial intelligence to lung cancer diagnosis

The morbidity and mortality rates in lung cancer are high worldwide. Early diagnosis and personalized treatment are important to manage this public health issue. In recent years, artificial intelligence (AI) has played increasingly important roles in early screening, auxiliary diagnosis, and prognostic assessment. AI uses algorithms to extract quantitative feature information from high-volume and high-latitude data and learn existing data to predict disease outcomes. In this review, we describe the current uses of AI in lung cancer-focused pathomics, imageomics, and genomics applications.

Nonspecific Orbital Inflammation (NSOI): Unraveling the Molecular Pathogenesis, Diagnostic Modalities, and Therapeutic Interventions

Nonspecific orbital inflammation (NSOI), colloquially known as orbital pseudotumor, sometimes presents a diagnostic and therapeutic challenge in ophthalmology. This review aims to dissect NSOI through a molecular lens, offering a comprehensive overview of its pathogenesis, clinical presentation, diagnostic methods, and management strategies. The article delves into the underpinnings of NSOI, examining immunological and environmental factors alongside intricate molecular mechanisms involving signaling pathways, cytokines, and mediators. Special emphasis is placed on emerging molecular discoveries and approaches, highlighting the significance of understanding molecular mechanisms in NSOI for the development of novel diagnostic and therapeutic tools. Various diagnostic modalities are scrutinized for their utility and limitations. Therapeutic interventions encompass medical treatments with corticosteroids and immunomodulatory agents, all discussed in light of current molecular understanding. More importantly, this review offers a novel molecular perspective on NSOI, dissecting its pathogenesis and management with an emphasis on the latest molecular discoveries. It introduces an integrated approach combining advanced molecular diagnostics with current clinical assessments and explores emerging targeted therapies. By synthesizing these facets, the review aims to inform clinicians and researchers alike, paving the way for molecularly informed, precision-based strategies for managing NSOI.

Multi-pose-based convolutional neural network model for diagnosis of patients with central lumbar spinal stenosis

Although the role of plain radiographs in diagnosing lumbar spinal stenosis (LSS) has declined in importance since the advent of magnetic resonance imaging (MRI), diagnostic ability of plain radiographs has improved dramatically when combined with deep learning. Previously, we developed a convolutional neural network (CNN) model using a radiograph for diagnosing LSS. In this study, we aimed to improve and generalize the performance of CNN models and overcome the limitation of the single-pose-based CNN (SP-CNN) model using multi-pose radiographs. Individuals with severe or no LSS, confirmed using MRI, were enrolled. Lateral radiographs of patients in three postures were collected. We developed a multi-pose-based CNN (MP-CNN) model using the encoders of the three SP-CNN model (extension, flexion, and neutral postures). We compared the validation results of the MP-CNN model using four algorithms pretrained with ImageNet. The MP-CNN model underwent additional internal and external validations to measure generalization performance. The ResNet50-based MP-CNN model achieved the largest area under the receiver operating characteristic curve (AUROC) of 91.4% (95% confidence interval [CI] 90.9-91.8%) for internal validation. The AUROC of the MP-CNN model were 91.3% (95% CI 90.7-91.9%) and 79.5% (95% CI 78.2-80.8%) for the extra-internal and external validation, respectively. The MP-CNN based heatmap offered a logical decision-making direction through optimized visualization. This model holds potential as a screening tool for LSS diagnosis, offering an explainable rationale for its prediction.

A survey on cancer detection via convolutional neural networks: Current challenges and future directions

Cancer is a condition in which abnormal cells uncontrollably split and damage the body tissues. Hence, detecting cancer at an early stage is highly essential. Currently, medical images play an indispensable role in detecting various cancers; however, manual interpretation of these images by radiologists is observer-dependent, time-consuming, and tedious. An automatic decision-making process is thus an essential need for cancer detection and diagnosis. This paper presents a comprehensive survey on automated cancer detection in various human body organs, namely, the breast, lung, liver, prostate, brain, skin, and colon, using convolutional neural networks (CNN) and medical imaging techniques. It also includes a brief discussion about deep learning based on state-of-the-art cancer detection methods, their outcomes, and the possible medical imaging data used. Eventually, the description of the dataset used for cancer detection, the limitations of the existing solutions, future trends, and challenges in this domain are discussed. The utmost goal of this paper is to provide a piece of comprehensive and insightful information to researchers who have a keen interest in developing CNN-based models for cancer detection.

Evaluating a Periapical Lesion Detection CNN on a Clinically Representative CBCT Dataset-A Validation Study

The aim of this validation study was to comprehensively evaluate the performance and generalization capability of a deep learning-based periapical lesion detection algorithm on a clinically representative cone-beam computed tomography (CBCT) dataset and test for non-inferiority. The evaluation involved 195 CBCT images of adult upper and lower jaws, where sensitivity and specificity metrics were calculated for all teeth, stratified by jaw, and stratified by tooth type. Furthermore, each lesion was assigned a periapical index score based on its size to enable a score-based evaluation. Non-inferiority tests were conducted with proportions of 90% for sensitivity and 82% for specificity. The algorithm achieved an overall sensitivity of 86.7% and a specificity of 84.3%. The non-inferiority test indicated the rejection of the null hypothesis for specificity but not for sensitivity. However, when excluding lesions with a periapical index score of one (i.e., very small lesions), the sensitivity improved to 90.4%. Despite the challenges posed by the dataset, the algorithm demonstrated promising results. Nevertheless, further improvements are needed to enhance the algorithm's robustness, particularly in detecting very small lesions and the handling of artifacts and outliers commonly encountered in real-world clinical scenarios.

Smart Endoscopy Is Greener Endoscopy: Leveraging Artificial Intelligence and Blockchain Technologies to Drive Sustainability in Digestive Health Care

The surge in the implementation of artificial intelligence (AI) in recent years has permeated many aspects of our life, and health care is no exception. Whereas this technology can offer clear benefits, some of the problems associated with its use have also been recognised and brought into question, for example, its environmental impact. In a similar fashion, health care also has a significant environmental impact, and it requires a considerable source of greenhouse gases. Whereas efforts are being made to reduce the footprint of AI tools, here, we were specifically interested in how employing AI tools in gastroenterology departments, and in particular in conjunction with capsule endoscopy, can reduce the carbon footprint associated with digestive health care while offering improvements, particularly in terms of diagnostic accuracy. We address the different ways that leveraging AI applications can reduce the carbon footprint associated with all types of capsule endoscopy examinations. Moreover, we contemplate how the incorporation of other technologies, such as blockchain technology, into digestive health care can help ensure the sustainability of this clinical speciality and by extension, health care in general.

Artificial intelligence assistance in deciding management strategies for polytrauma and trauma patients

<b><br>Introduction:</b> Artificial intelligence (AI) is an emerging technology with vast potential for use in several fields of medicine. However, little is known about the application of AI in treatment decisions for patients with polytrauma. In this systematic review, we investigated the benefits and performance of AI in predicting the management of patients with polytrauma and trauma.</br> <b><br>Methods:</b> This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Studies were extracted from the PubMed and Google Scholar databases from their inception until November 2022, using the search terms "Artificial intelligence," "polytrauma," and "decision." Seventeen articles were identified and screened for eligibility. Animal studies, review articles, systematic reviews, meta-analyses, and studies that did not involve polytrauma or severe trauma management decisions were excluded. Eight studies were eligible for final review.</br> <b><br>Results:</b> Eight studies focusing on patients with trauma, including two on military trauma, were included. The AI applications were mainly implemented for predictions and/or decisions on shock, bleeding, and blood transfusion. Few studies predicted death/survival. The identification of trauma patients using AI was proposed in a previous study. The overall performance of AI was good (six studies), excellent (one study), and acceptable (one study).</br> <b><br>Discussion:</b> AI demonstrated satisfactory performance in decision-making and management prediction in patients with polytrauma/severe trauma, especially in situations of shock/bleeding.</br> <b><br>Importance:</b> The present study serves as a basis for further research to develop practical AI applications for the management of patients with trauma.</br>.

Artificial intelligence-powered spatial analysis of tumor-infiltrating lymphocytes for prediction of prognosis in resected colon cancer

Tumor-infiltrating lymphocytes (TIL) have been suggested as an important prognostic marker in colorectal cancer, but assessment usually requires additional tissue processing and interpretational efforts. The aim of this study is to assess the clinical significance of artificial intelligence (AI)-powered spatial TIL analysis using only a hematoxylin and eosin (H&E)-stained whole-slide image (WSI) for the prediction of prognosis in stage II-III colon cancer treated with surgery and adjuvant therapy. In this retrospective study, we used Lunit SCOPE IO, an AI-powered H&E WSI analyzer, to assess intratumoral TIL (iTIL) and tumor-related stromal TIL (sTIL) densities from WSIs of 289 patients. The patients with confirmed recurrences had significantly lower sTIL densities (mean sTIL density 630.2/mm2 in cases with confirmed recurrence vs. 1021.3/mm2 in no recurrence, p < 0.001). Additionally, significantly higher recurrence rates were observed in patients having sTIL or iTIL in the lower quartile groups. Risk groups defined as high-risk (both iTIL and sTIL in the lowest quartile groups), low-risk (sTIL higher than the median), or intermediate-risk (not high- or low-risk) were predictive of recurrence and were independently associated with clinical outcomes after adjusting for other clinical factors. AI-powered TIL analysis can provide prognostic information in stage II/III colon cancer in a practical manner.

Enhancing Fetal Anomaly Detection in Ultrasonography Images: A Review of Machine Learning-Based Approaches

Fetal development is a critical phase in prenatal care, demanding the timely identification of anomalies in ultrasound images to safeguard the well-being of both the unborn child and the mother. Medical imaging has played a pivotal role in detecting fetal abnormalities and malformations. However, despite significant advances in ultrasound technology, the accurate identification of irregularities in prenatal images continues to pose considerable challenges, often necessitating substantial time and expertise from medical professionals. In this review, we go through recent developments in machine learning (ML) methods applied to fetal ultrasound images. Specifically, we focus on a range of ML algorithms employed in the context of fetal ultrasound, encompassing tasks such as image classification, object recognition, and segmentation. We highlight how these innovative approaches can enhance ultrasound-based fetal anomaly detection and provide insights for future research and clinical implementations. Furthermore, we emphasize the need for further research in this domain where future investigations can contribute to more effective ultrasound-based fetal anomaly detection.

Ensemble model for predicting chronic non-communicable diseases using Latin square extraction and fuzzy-artificial neural networks from 2013 to 2019

Background

The presented study tracks the increase or decrease in the prevalence of seventeen different chronic non-communicable diseases in Serbia. This analysis considers factors such as region, age, and gender and is based on data from two national cross-sectional studies conducted in 2013 and 2019. The research aims to accurately identify the regions with the highest percentage of affected individuals, as well as their respective age and gender groups. The ultimate goal is to facilitate organized, free preventive screenings for these population categories within a very short time-frame in the future.

Materials and methods

The study analyzed two cross-sectional studies conducted between 2013 and 2019, using data obtained from the Institute of Public Health of Serbia. Both studies involved a total of 27801 participants. The study compared the performance of Decision Tree and Support Vector Regressor models with artificial neural network (ANN) models that employed two encoding functions. The new methodology for the ANN-L36 model was based on artificial neural networks constructed using a Latin square (L36) design, incorporating Taguchi's robust design optimization.

Results

The results of the analysis from three different models have shown that cardiovascular diseases are the most prevalent illnesses among the population in Serbia, with hypertension as the leading condition in all regions, particularly among individuals aged 64 to 75 years, and more prevalent among females. In 2019, there was a decrease in the percentage of the leading disease, hypertension, compared to 2013, with a decrease from 34.0% to 32.2%. The ANN-L36 model with Fuzzy encoding function demonstrated the highest precision, achieving the smallest relative error of 0.1%.

Conclusion

To date, no studies have been conducted at the national level in Serbia to comprehensively track and identify chronic diseases in the manner proposed by this study. The model presented in this research will be implemented in practice and is set to significantly contribute to the future healthcare framework in Serbia, shaping and advancing the approach towards addressing these conditions. Furthermore, experimental evidence has shown that Taguchi's optimization approach yields the best results for identifying various chronic non-communicable diseases.

Unsupervised deep network for image texture transformation: Improving the quality of cross-correlation analysis and mechanical vortex visualisation during cardiac fibrillation

Visualisation of cardiac fibrillation plays a very considerable role in cardiophysiological study and clinical applications. One of the ways to obtain the image of these phenomena is the registration of mechanical displacement fields reflecting the track from electrical activity. In this work, we read these fields using cross-correlation analysis from the video of open pig's epicardium at the start of fibrillation recorded with electrocardiogram. However, the quality of obtained displacement fields remains low due to the weak pixels heterogeneity of the frames. It disables to see more clearly such interesting phenomena as mechanical vortexes that underline the mechanical dysfunction of fibrillation. The applying of chemical or mechanical markers to solve this problem can affect the course of natural processes and falsify the results. Therefore, we developed a novel scheme of an unsupervised deep neural network that is based on the state-of-art positional coding technology for a multilayer perceptron. This network enables to generate a couple of frames with a more heterogeneous pixel texture, that is more suitable for cross-correlation analysis methods, from two consecutive frames. The novel network scheme was tested on synthetic pairs of images with different texture heterogeneity and frequency of displacement fields and also it was compared with different filters on our cardiac tissue image dataset. The testing showed that the displacement fields obtained with our method are closer to the ground truth than those which were computed only with the cross-correlation analysis in low contrast images case where filtering is impossible. Moreover, our model showed the best results comparing with the one of the popular filter CLAHE on our dataset. As a result, using our approach, it was possible to register more clearly a mechanical vortex on the epicardium at the start of fibrillation continuously for several milliseconds for the first time.

Using deep learning for an automatic detection and classification of the vascular bifurcations along the Circle of Willis

Most of the intracranial aneurysms (ICA) occur on a specific portion of the cerebral vascular tree named the Circle of Willis (CoW). More particularly, they mainly arise onto fifteen of the major arterial bifurcations constituting this circular structure. Hence, for an efficient and timely diagnosis it is critical to develop some methods being able to accurately recognize each Bifurcation of Interest (BoI). Indeed, an automatic extraction of the bifurcations presenting the higher risk of developing an ICA would offer the neuroradiologists a quick glance at the most alarming areas. Due to the recent efforts on Artificial Intelligence, Deep Learning turned out to be the best performing technology for many pattern recognition tasks. Moreover, various methods have been particularly designed for medical image analysis purposes. This study intends to assist the neuroradiologists to promptly locate any bifurcation presenting a high risk of ICA occurrence. It can be seen as a Computer Aided Diagnosis scheme, where the Artificial Intelligence facilitates the access to the regions of interest within the MRI. In this work, we propose a method for a fully automatic detection and recognition of the bifurcations of interest forming the Circle of Willis. Several neural networks architectures have been tested, and we thoroughly evaluate the bifurcation recognition rate.

Hybrid-supervised bidirectional transfer networks for computer-aided diagnosis

Medical imaging techniques have been widely used for diagnosis of various diseases. However, the imaging-based diagnosis generally depends on the clinical skill of radiologists. Computer-aided diagnosis (CAD) can help radiologists improve diagnostic accuracy as well as the consistency and reproducibility. Although convolutional neural network (CNN) has shown its feasibility and effectiveness in CAD, it generally suffers from the problem of small sample size when training CAD models. Nowadays, self-supervised learning (SSL) has shown its effectiveness in the field of medical image analysis, especially when there are only limited training samples. However, the backbone of downstream task sometimes cannot be well pre-trained in the conventional SSL framework due to the limitation of the pretext task and fine-tuning mechanism. In this work, an improved SSL framework, named Hybrid-supervised Bidirectional Transfer Networks (HBTN), is proposed to improve the performance of CAD models. Specifically, a novel Gray-Scale Image Mapping (GSIM) task is developed, which still takes the widely used image restoration task in SSL as the pretext task, but further embeds the class label information into it to improve discriminative feature learning of its corresponding network model. The proposed HBTN then integrates two different network architectures, i.e. the image restoration network for the pretext task and the classification network for the downstream task, into a unified hybrid-supervised learning (HSL) framework. It jointly trains both networks and collaboratively transfers the knowledge between each other. Consequently, the performance of downstream network is thus improved. The proposed HBTN is evaluated on two medical image datasets for CAD tasks. The experimental results indicate that HBTN outperforms the conventional SSL algorithms for CAD with limited training samples.

RANet: a custom CNN model and quanvolutional neural network for the automated detection of rheumatoid arthritis in hand thermal images

Rheumatoid arthritis is an autoimmune disease which affects the small joints. Early prediction of RA is necessary for the treatment and management of the disease. The current work presents a deep learning and quantum computing-based automated diagnostic approach for RA in hand thermal imaging. The study's goals are (i) to develop a custom RANet model and compare its performance with the pretrained models and quanvolutional neural network (QNN) to distinguish between the healthy subjects and RA patients, (ii) To validate the performance of the custom model using feature selection method and classification using machine learning (ML) classifiers. The present study developed a custom RANet model and employed pre-trained models such as ResNet101V2, InceptionResNetV2, and DenseNet201 to classify the RA patients and normal subjects. The deep features extracted from the RA Net model are fed into the ML classifiers after the feature selection process. The RANet model, RA Net+ SVM, and QNN model produced an accuracy of 95%, 97% and 93.33% respectively in the classification of healthy groups and RA patients. The developed RANet and QNN models based on thermal imaging could be employed as an accurate automated diagnostic tool to differentiate between the RA and control groups.

Design and validation of a convolutional neural network for fast, model-free blood flow imaging with multiple exposure speckle imaging

Multiple exposure speckle imaging has demonstrated its improved accuracy compared to single exposure speckle imaging for relative quantitation of blood flow in vivo. However, the calculation of blood flow maps relies on a pixelwise non-linear fit of a multi-parametric model to the speckle contrasts. This approach has two major drawbacks. First, it is computer-intensive and prevents real time imaging and, second, the mathematical model is not universal and should in principle be adapted to the type of blood vessels. We evaluated a model-free machine learning approach based on a convolutional neural network as an alternative to the non-linear fit approach. A network was designed and trained with annotated speckle contrast data from microfluidic experiments. The neural network performances are then compared to the non-linear fit approach applied to in vitro and in vivo data. The study demonstrates the potential of convolutional networks to provide relative blood flow maps from multiple exposure speckle data in real time.

AI-Assisted Screening of Oral Potentially Malignant Disorders Using Smartphone-Based Photographic Images

The prevalence of oral potentially malignant disorders (OPMDs) and oral cancer is surging in low- and middle-income countries. A lack of resources for population screening in remote locations delays the detection of these lesions in the early stages and contributes to higher mortality and a poor quality of life. Digital imaging and artificial intelligence (AI) are promising tools for cancer screening. This study aimed to evaluate the utility of AI-based techniques for detecting OPMDs in the Indian population using photographic images of oral cavities captured using a smartphone. A dataset comprising 1120 suspicious and 1058 non-suspicious oral cavity photographic images taken by trained front-line healthcare workers (FHWs) was used for evaluating the performance of different deep learning models based on convolution (DenseNets) and Transformer (Swin) architectures. The best-performing model was also tested on an additional independent test set comprising 440 photographic images taken by untrained FHWs (set I). DenseNet201 and Swin Transformer (base) models show high classification performance with an F1-score of 0.84 (CI 0.79-0.89) and 0.83 (CI 0.78-0.88) on the internal test set, respectively. However, the performance of models decreases on test set I, which has considerable variation in the image quality, with the best F1-score of 0.73 (CI 0.67-0.78) obtained using DenseNet201. The proposed AI model has the potential to identify suspicious and non-suspicious oral lesions using photographic images. This simplified image-based AI solution can assist in screening, early detection, and prompt referral for OPMDs.

DRSN4mCPred: accurately predicting sites of DNA N4-methylcytosine using deep residual shrinkage network for diagnosis and treatment of gastrointestinal cancer in the precision medicine era

Introduction

The DNA N4-methylcytosine (4mC) site levels of those suffering from digestive system cancers were higher, and the pathogenesis of digestive system cancers may also be related to the changes in DNA 4mC levels. Identifying DNA 4mC sites is a very important step in studying the analysis of biological function and cancer prediction. Extracting accurate features from DNA sequences is the key to establishing a prediction model of effective DNA 4mC sites. This study sought to develop a new predictive model, DRSN4mCPred, which aimed to improve the performance of the predicting DNA 4mC sites.

Methods

The model adopted multi-scale channel attention to extract features and used attention feature fusion (AFF) to fuse features. In order to capture features information more accurately and effectively, this model utilized Deep Residual Shrinkage Network with Channel-Wise thresholds (DRSN-CW) to eliminate noise-related features and achieve a more precise feature representation, thereby, distinguishing the sites in DNA with 4mC and non-4mC. Additionally, the predictive model incorporated an inverted residual block, a Multi-scale Channel Attention Module (MS-CAM), a Bi-directional Long Short Term Memory Network (Bi-LSTM), AFF, and DRSN-CW.

Results and Discussion

The results indicated the predictive model DRSN4mCPred had extremely good performance in predicting the DNA 4mC sites across different species. This paper will potentially provide support for the diagnosis and treatment of gastrointestinal cancer based on artificial intelligence in the precise medical era.

An Instance Segmentation Model Based on Deep Learning for Intelligent Diagnosis of Uterine Myomas in MRI

Uterine myomas affect 70% of women of reproductive age, potentially impacting their fertility and health. Manual film reading is commonly used to identify uterine myomas, but it is time-consuming, laborious, and subjective. Clinical treatment requires the consideration of the positional relationship among the uterine wall, uterine cavity, and uterine myomas. However, due to their complex and variable shapes, the low contrast of adjacent tissues or organs, and indistinguishable edges, accurately identifying them in MRI is difficult. Our work addresses these challenges by proposing an instance segmentation network capable of automatically outputting the location, category, and masks of each organ and lesion. Specifically, we designed a new backbone that facilitates learning the shape features of object diversity, and filters out background noise interference. We optimized the anchor box generation strategy to provide better priors in order to enhance the process of bounding box prediction and regression. An adaptive iterative subdivision strategy ensures that the mask boundary details of objects are more realistic and accurate. We conducted extensive experiments to validate our network, which achieved better average precision (AP) results than those of state-of-the-art instance segmentation models. Compared to the baseline network, our model improved AP on the uterine wall, uterine cavity, and myomas by 8.8%, 8.4%, and 3.2%, respectively. Our work is the first to realize multiclass instance segmentation in uterine MRI, providing a convenient and objective reference for the clinical development of appropriate surgical plans, and has significant value in improving diagnostic efficiency and realizing the automatic auxiliary diagnosis of uterine myomas.

MR Template-Based Individual Brain PET Volumes-of-Interest Generation Neither Using MR nor Using Spatial Normalization

For more anatomically precise quantitation of mouse brain PET, spatial normalization (SN) of PET onto MR template and subsequent template volumes-of-interest (VOIs)-based analysis are commonly used. Although this leads to dependency on the corresponding MR and the process of SN, routine preclinical/clinical PET images cannot always afford corresponding MR and relevant VOIs. To resolve this issue, we propose a deep learning (DL)-based individual-brain-specific VOIs (i.e., cortex, hippocampus, striatum, thalamus, and cerebellum) directly generated from PET images using the inverse-spatial-normalization (iSN)-based VOI labels and deep convolutional neural network model (deep CNN). Our technique was applied to mutated amyloid precursor protein and presenilin-1 mouse model of Alzheimer's disease. Eighteen mice underwent T2-weighted MRI and 18F FDG PET scans before and after the administration of human immunoglobin or antibody-based treatments. To train the CNN, PET images were used as inputs and MR iSN-based target VOIs as labels. Our devised methods achieved decent performance in terms of not only VOI agreements (i.e., Dice similarity coefficient) but also the correlation of mean counts and SUVR, and CNN-based VOIs was highly accordant with ground-truth (the corresponding MR and MR template-based VOIs). Moreover, the performance metrics were comparable to that of VOI generated by MR-based deep CNN. In conclusion, we established a novel quantitative analysis method both MR-less and SN-less fashion to generate individual brain space VOIs using MR template-based VOIs for PET image quantification.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13139-022-00772-4.

Machine Learning for Brain MRI Data Harmonisation: A Systematic Review

BACKGROUND

Magnetic Resonance Imaging (MRI) data collected from multiple centres can be heterogeneous due to factors such as the scanner used and the site location. To reduce this heterogeneity, the data needs to be harmonised. In recent years, machine learning (ML) has been used to solve different types of problems related to MRI data, showing great promise.

OBJECTIVE

This study explores how well various ML algorithms perform in harmonising MRI data, both implicitly and explicitly, by summarising the findings in relevant peer-reviewed articles. Furthermore, it provides guidelines for the use of current methods and identifies potential future research directions.

METHOD

This review covers articles published through PubMed, Web of Science, and IEEE databases through June 2022. Data from studies were analysed based on the criteria of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). Quality assessment questions were derived to assess the quality of the included publications.

RESULTS

a total of 41 articles published between 2015 and 2022 were identified and analysed. In the review, MRI data has been found to be harmonised either in an implicit (n = 21) or an explicit (n = 20) way. Three MRI modalities were identified: structural MRI (n = 28), diffusion MRI (n = 7) and functional MRI (n = 6).

CONCLUSION

Various ML techniques have been employed to harmonise different types of MRI data. There is currently a lack of consistent evaluation methods and metrics used across studies, and it is recommended that the issue be addressed in future studies. Harmonisation of MRI data using ML shows promises in improving performance for ML downstream tasks, while caution should be exercised when using ML-harmonised data for direct interpretation.

Medical image data augmentation: techniques, comparisons and interpretations

Designing deep learning based methods with medical images has always been an attractive area of research to assist clinicians in rapid examination and accurate diagnosis. Those methods need a large number of datasets including all variations in their training stages. On the other hand, medical images are always scarce due to several reasons, such as not enough patients for some diseases, patients do not want to allow their images to be used, lack of medical equipment or equipment, inability to obtain images that meet the desired criteria. This issue leads to bias in datasets, overfitting, and inaccurate results. Data augmentation is a common solution to overcome this issue and various augmentation techniques have been applied to different types of images in the literature. However, it is not clear which data augmentation technique provides more efficient results for which image type since different diseases are handled, different network architectures are used, and these architectures are trained and tested with different numbers of data sets in the literature. Therefore, in this work, the augmentation techniques used to improve performances of deep learning based diagnosis of the diseases in different organs (brain, lung, breast, and eye) from different imaging modalities (MR, CT, mammography, and fundoscopy) have been examined. Also, the most commonly used augmentation methods have been implemented, and their effectiveness in classifications with a deep network has been discussed based on quantitative performance evaluations. Experiments indicated that augmentation techniques should be chosen carefully according to image types.

Innovation in Hyperinsulinemia Diagnostics with ANN-L(atin square) Models

Hyperinsulinemia is a condition characterized by excessively high levels of insulin in the bloodstream. It can exist for many years without any symptomatology. The research presented in this paper was conducted from 2019 to 2022 in cooperation with a health center in Serbia as a large cross-sectional observational study of adolescents of both genders using datasets collected from the field. Previously used analytical approaches of integrated and relevant clinical, hematological, biochemical, and other variables could not identify potential risk factors for developing hyperinsulinemia. This paper aims to present several different models using machine learning (ML) algorithms such as naive Bayes, decision tree, and random forest and compare them with a new methodology constructed based on artificial neural networks using Taguchi's orthogonal vector plans (ANN-L), a special extraction of Latin squares. Furthermore, the experimental part of this study showed that ANN-L models achieved an accuracy of 99.5% with less than seven iterations performed. Furthermore, the study provides valuable insights into the share of each risk factor contributing to the occurrence of hyperinsulinemia in adolescents, which is crucial for more precise and straightforward medical diagnoses. Preventing the risk of hyperinsulinemia in this age group is crucial for the well-being of the adolescents and society as a whole.

Deep learning model integrating positron emission tomography and clinical data for prognosis prediction in non-small cell lung cancer patients

BACKGROUND

Lung cancer is the leading cause of cancer-related deaths worldwide. The majority of lung cancers are non-small cell lung cancer (NSCLC), accounting for approximately 85% of all lung cancer types. The Cox proportional hazards model (CPH), which is the standard method for survival analysis, has several limitations. The purpose of our study was to improve survival prediction in patients with NSCLC by incorporating prognostic information from F-18 fluorodeoxyglucose positron emission tomography (FDG PET) images into a traditional survival prediction model using clinical data.

RESULTS

The multimodal deep learning model showed the best performance, with a C-index and mean absolute error of 0.756 and 399 days under a five-fold cross-validation, respectively, followed by ResNet3D for PET (0.749 and 405 days) and CPH for clinical data (0.747 and 583 days).

CONCLUSION

The proposed deep learning-based integrative model combining the two modalities improved the survival prediction in patients with NSCLC.

Multi-task deep learning for medical image computing and analysis: A review

The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.

Raman spectroscopy and convolutional neural networks for monitoring biochemical radiation response in breast tumour xenografts

Tumour cells exhibit altered metabolic pathways that lead to radiation resistance and disease progression. Raman spectroscopy (RS) is a label-free optical modality that can monitor post-irradiation biomolecular signatures in tumour cells and tissues. Convolutional Neural Networks (CNN) perform automated feature extraction directly from data, with classification accuracy exceeding that of traditional machine learning, in cases where data is abundant and feature extraction is challenging. We are interested in developing a CNN-based predictive model to characterize clinical tumour response to radiation therapy based on their degree of radiosensitivity or radioresistance. In this work, a CNN architecture is built for identifying post-irradiation spectral changes in Raman spectra of tumour tissue. The model was trained to classify irradiated versus non-irradiated tissue using Raman spectra of breast tumour xenografts. The CNN effectively classified the tissue spectra, with accuracies exceeding 92.1% for data collected 3 days post-irradiation, and 85.0% at day 1 post-irradiation. Furthermore, the CNN was evaluated using a leave-one-out- (mouse, section or Raman map) validation approach to investigate its generalization to new test subjects. The CNN retained good predictive accuracy (average accuracies 83.7%, 91.4%, and 92.7%, respectively) when little to no information for a specific subject was given during training. Finally, the classification performance of the CNN was compared to that of a previously developed model based on group and basis restricted non-negative matrix factorization and random forest (GBR-NMF-RF) classification. We found that CNN yielded higher classification accuracy, sensitivity, and specificity in mice assessed 3 days post-irradiation, as compared with the GBR-NMF-RF approach. Overall, the CNN can detect biochemical spectral changes in tumour tissue at an early time point following irradiation, without the need for previous manual feature extraction. This study lays the foundation for developing a predictive framework for patient radiation response monitoring.

A survey on multi-omics-based cancer diagnosis using machine learning with the potential application in gastrointestinal cancer

Gastrointestinal cancer is becoming increasingly common, which leads to over 3 million deaths every year. No typical symptoms appear in the early stage of gastrointestinal cancer, posing a significant challenge in the diagnosis and treatment of patients with gastrointestinal cancer. Many patients are in the middle and late stages of gastrointestinal cancer when they feel uncomfortable, unfortunately, most of them will die of gastrointestinal cancer. Recently, various artificial intelligence techniques like machine learning based on multi-omics have been presented for cancer diagnosis and treatment in the era of precision medicine. This paper provides a survey on multi-omics-based cancer diagnosis using machine learning with potential application in gastrointestinal cancer. Particularly, we make a comprehensive summary and analysis from the perspective of multi-omics datasets, task types, and multi-omics-based integration methods. Furthermore, this paper points out the remaining challenges of multi-omics-based cancer diagnosis using machine learning and discusses future topics.

A Similarity Measure-Based Approach Using RS-fMRI Data for Autism Spectrum Disorder Diagnosis

Autism spectrum disorder (ASD) is a lifelong neurological disease, which seriously reduces the patients' life quality. Generally, an early diagnosis is beneficial to improve ASD children's life quality. Current methods based on samples from multiple sites for ASD diagnosis perform poorly in generalization due to the heterogeneity of the data from multiple sites. To address this problem, this paper presents a similarity measure-based approach for ASD diagnosis. Specifically, the few-shot learning strategy is used to measure potential similarities in the RS-fMRI data distributions, and, furthermore, a similarity function for samples from multiple sites is trained to enhance the generalization. On the ABIDE database, the presented approach is compared to some representative methods, such as SVM and random forest, in terms of accuracy, precision, and F1 score. The experimental results show that the experimental indicators of the proposed method are better than those of the comparison methods to varying degrees. For example, the accuracy on the TRINITY site is more than 5% higher than that of the comparison method, which clearly proves that the presented approach achieves a better generalization performance than the compared methods.