Deep learning with noisy labels: Exploring techniques and remedies in medical image analysis

Learning a Single Network for Robust Medical Image Segmentation With Noisy Labels

Robust segmenting with noisy labels is an important problem in medical imaging due to the difficulty of acquiring high-quality annotations. Despite the enormous success of recent developments, these developments still require multiple networks to construct their frameworks and focus on limited application scenarios, which leads to inflexibility in practical applications. They also do not explicitly consider the coarse boundary label problem, which results in sub-optimal results. To overcome these challenges, we propose a novel Simultaneous Edge Alignment and Memory-Assisted Learning (SEAMAL) framework for noisy-label robust segmentation. It achieves single-network robust learning, which is applicable for both 2D and 3D segmentation, in both Set-HQ-knowable and Set-HQ-agnostic scenarios. Specifically, to achieve single-model noise robustness, we design a Memory-assisted Selection and Correction module (MSC) that utilizes predictive history consistency from the Prediction Memory Bank to distinguish between reliable and non-reliable labels pixel-wisely, and that updates the reliable ones at the superpixel level. To overcome the coarse boundary label problem, which is common in practice, and to better utilize shape-relevant information at the boundary, we propose an Edge Detection Branch (EDB) that explicitly learns the boundary via an edge detection layer with only slight additional computational cost, and we improve the sharpness and precision of the boundary with a thinning loss. Extensive experiments verify that SEAMAL outperforms previous works significantly.

Better Rough Than Scarce: Proximal Femur Fracture Segmentation With Rough Annotations

Proximal femoral fracture segmentation in computed tomography (CT) is essential in the preoperative planning of orthopedic surgeons. Recently, numerous deep learning-based approaches have been proposed for segmenting various structures within CT scans. Nevertheless, distinguishing various attributes between fracture fragments and soft tissue regions in CT scans frequently poses challenges, which have received comparatively limited research attention. Besides, the cornerstone of contemporary deep learning methodologies is the availability of annotated data, while detailed CT annotations remain scarce. To address the challenge, we propose a novel weakly-supervised framework, namely Rough Turbo Net (RT-Net), for the segmentation of proximal femoral fractures. We emphasize the utilization of human resources to produce rough annotations on a substantial scale, as opposed to relying on limited fine-grained annotations that demand a substantial time to create. In RT-Net, rough annotations pose fractured-region constraints, which have demonstrated significant efficacy in enhancing the accuracy of the network. Conversely, the fine annotations can provide more details for recognizing edges and soft tissues. Besides, we design a spatial adaptive attention module (SAAM) that adapts to the spatial distribution of the fracture regions and align feature in each decoder. Moreover, we propose a fine-edge loss which is applied through an edge discrimination network to penalize the absence or imprecision edge features. Extensive quantitative and qualitative experiments demonstrate the superiority of RT-Net to state-of-the-art approaches. Furthermore, additional experiments show that RT-Net has the capability to produce pseudo labels for raw CT images that can further improve fracture segmentation performance and has the potential to improve segmentation performance on public datasets. The code is available at: https://github.com/zyairelu/RT-Net.

Simplified detection of genetic background admixture using artificial intelligence

Admixture refers to the mixing of genetic ancestry from different populations. Admixture is important for genomic medicine because it can affect how an individual responds to certain medications, how they metabolize drugs, and susceptibility to certain diseases. For example, some genetic variants associated with drug metabolism and response may be more common in certain populations, and individuals with admixed ancestry may have a different frequency of these variants than individuals from the ancestral populations. Understanding the patterns of admixture in a population can also help researchers identify new genetic variants associated with diseases or traits and develop more personalized and targeted treatments. In this study, we compared and classified the known and self-reported genetic backgrounds from 1000 Genomes Project and admixed samples from GTEx projects using supervised, unsupervised and statistical classification methodologies. We developed a novel tool called Admix-AI that uses a one-dimensional convolutional neural network to understand and classify admixed genetic backgrounds using 213 DNA-marker based genetic background labels. Admix-AI can be used to discover admixed proportions in samples and ultimately aid personalized genomic medicine by identifying specific biomarker systems. We compared Admix-AI to the existing admixture categorization software and found our tool to be computationally faster with 2× speedup and streamlined usage. Admix-AI is available as open-source code under GPL version 3.0 license at https://github.com/rpauly/Admix-AI.

Assessing the Value of Imaging Data in Machine Learning Models to Predict Patient-Reported Outcome Measures in Knee Osteoarthritis Patients

Knee osteoarthritis (OA) affects over 650 million patients worldwide. Total knee replacement is aimed at end-stage OA to relieve symptoms of pain, stiffness and reduced mobility. However, the role of imaging modalities in monitoring symptomatic disease progression remains unclear. This study aimed to compare machine learning (ML) models, with and without imaging features, in predicting the two-year Western Ontario and McMaster Universities Arthritis Index (WOMAC) score for knee OA patients. We included 2408 patients from the Osteoarthritis Initiative (OAI) database, with 629 patients from the Multicenter Osteoarthritis Study (MOST) database. The clinical dataset included 18 clinical features, while the imaging dataset contained an additional 10 imaging features. Minimal Clinically Important Difference (MCID) was set to 24, reflecting meaningful physical impairment. Clinical and imaging dataset models produced similar area under curve (AUC) scores, highlighting low differences in performance AUC < 0.025). For both clinical and imaging datasets, Gradient Boosting Machine (GBM) models performed the best in the external validation, with a clinically acceptable AUC of 0.734 (95% CI 0.687-0.781) and 0.747 (95% CI 0.701-0.792), respectively. The five features identified included educational background, family history of osteoarthritis, co-morbidities, use of osteoporosis medications and previous knee procedures. This is the first study to demonstrate that ML models achieve comparable performance with and without imaging features.

A neural cell automated analysis system based on pathological specimens in a gerbil brain ischemia model

PURPOSE

Amid rising health awareness, natural products which has milder effects than medical drugs are becoming popular. However, only few systems can quantitatively assess their impact on living organisms. Therefore, we developed a deep-learning system to automate the counting of cells in a gerbil model, aiming to assess a natural product's effectiveness against ischemia.

METHODS

The image acquired from paraffin blocks containing gerbil brains was analyzed by a deep-learning model (fine-tuned Detectron2).

RESULTS

The counting system achieved a 79%-positive predictive value and 85%-sensitivity when visual judgment by an expert was used as ground truth.

CONCLUSIONS

Our system evaluated hydrogen water's potential against ischemia and found it potentially useful, which is consistent with expert assessment. Due to natural product's milder effects, large data sets are needed for evaluation, making manual measurement labor-intensive. Hence, our system offers a promising new approach for evaluating natural products.

Responsible development of clinical speech AI: Bridging the gap between clinical research and technology

This perspective article explores the challenges and potential of using speech as a biomarker in clinical settings, particularly when constrained by the small clinical datasets typically available in such contexts. We contend that by integrating insights from speech science and clinical research, we can reduce sample complexity in clinical speech AI models with the potential to decrease timelines to translation. Most existing models are based on high-dimensional feature representations trained with limited sample sizes and often do not leverage insights from speech science and clinical research. This approach can lead to overfitting, where the models perform exceptionally well on training data but fail to generalize to new, unseen data. Additionally, without incorporating theoretical knowledge, these models may lack interpretability and robustness, making them challenging to troubleshoot or improve post-deployment. We propose a framework for organizing health conditions based on their impact on speech and promote the use of speech analytics in diverse clinical contexts beyond cross-sectional classification. For high-stakes clinical use cases, we advocate for a focus on explainable and individually-validated measures and stress the importance of rigorous validation frameworks and ethical considerations for responsible deployment. Bridging the gap between AI research and clinical speech research presents new opportunities for more efficient translation of speech-based AI tools and advancement of scientific discoveries in this interdisciplinary space, particularly if limited to small or retrospective datasets.

A benchmarked comparison of software packages for time-lapse image processing of monolayer bacterial population dynamics

Time-lapse microscopy offers a powerful approach for analyzing cellular activity. In particular, this technique is valuable for assessing the behavior of bacterial populations, which can exhibit growth and intercellular interactions in a monolayer. Such time-lapse imaging typically generates large quantities of data, limiting the options for manual investigation. Several image-processing software packages have been developed to facilitate analysis. It can thus be a challenge to identify the software package best suited to a particular research goal. Here, we compare four software packages that support the analysis of 2D time-lapse images of cellular populations: CellProfiler, SuperSegger-Omnipose, DeLTA, and FAST. We compare their performance against benchmarked results on time-lapse observations of Escherichia coli populations. Performance varies across the packages, with each of the four outperforming the others in at least one aspect of the analysis. Not surprisingly, the packages that have been in development for longer showed the strongest performance. We found that deep learning-based approaches to object segmentation outperformed traditional approaches, but the opposite was true for frame-to-frame object tracking. We offer these comparisons, together with insight into usability, computational efficiency, and feature availability, as a guide to researchers seeking image-processing solutions.

IMPORTANCE

Time-lapse microscopy provides a detailed window into the world of bacterial behavior. However, the vast amount of data produced by these techniques is difficult to analyze manually. We have analyzed four software tools designed to process such data and compared their performance, using populations of commonly studied bacterial species as our test subjects. Our findings offer a roadmap to scientists, helping them choose the right tool for their research. This comparison bridges a gap between microbiology and computational analysis, streamlining research efforts.

The METRIC-framework for assessing data quality for trustworthy AI in medicine: a systematic review

The adoption of machine learning (ML) and, more specifically, deep learning (DL) applications into all major areas of our lives is underway. The development of trustworthy AI is especially important in medicine due to the large implications for patients' lives. While trustworthiness concerns various aspects including ethical, transparency and safety requirements, we focus on the importance of data quality (training/test) in DL. Since data quality dictates the behaviour of ML products, evaluating data quality will play a key part in the regulatory approval of medical ML products. We perform a systematic review following PRISMA guidelines using the databases Web of Science, PubMed and ACM Digital Library. We identify 5408 studies, out of which 120 records fulfil our eligibility criteria. From this literature, we synthesise the existing knowledge on data quality frameworks and combine it with the perspective of ML applications in medicine. As a result, we propose the METRIC-framework, a specialised data quality framework for medical training data comprising 15 awareness dimensions, along which developers of medical ML applications should investigate the content of a dataset. This knowledge helps to reduce biases as a major source of unfairness, increase robustness, facilitate interpretability and thus lays the foundation for trustworthy AI in medicine. The METRIC-framework may serve as a base for systematically assessing training datasets, establishing reference datasets, and designing test datasets which has the potential to accelerate the approval of medical ML products.

WONOEP 2022: Neurotechnology for the diagnosis of epilepsy

Epilepsy's myriad causes and clinical presentations ensure that accurate diagnoses and targeted treatments remain a challenge. Advanced neurotechnologies are needed to better characterize individual patients across multiple modalities and analytical techniques. At the XVIth Workshop on Neurobiology of Epilepsy: Early Onset Epilepsies: Neurobiology and Novel Therapeutic Strategies (WONOEP 2022), the session on "advanced tools" highlighted a range of approaches, from molecular phenotyping of genetic epilepsy models and resected tissue samples to imaging-guided localization of epileptogenic tissue for surgical resection of focal malformations. These tools integrate cutting edge research, clinical data acquisition, and advanced computational methods to leverage the rich information contained within increasingly large datasets. A number of common challenges and opportunities emerged, including the need for multidisciplinary collaboration, multimodal integration, potential ethical challenges, and the multistage path to clinical translation. Despite these challenges, advanced epilepsy neurotechnologies offer the potential to improve our understanding of the underlying causes of epilepsy and our capacity to provide patient-specific treatment.

Optimization of Fast Non-Local Means Noise Reduction Algorithm Parameter in Computed Tomographic Phantom Images Using 3D Printing Technology

Noise in computed tomography (CT) is inevitably generated, which lowers the accuracy of disease diagnosis. The non-local means approach, a software technique for reducing noise, is widely used in medical imaging. In this study, we propose a noise reduction algorithm based on fast non-local means (FNLMs) and apply it to CT images of a phantom created using 3D printing technology. The self-produced phantom was manufactured using filaments with similar density to human brain tissues. To quantitatively evaluate image quality, the contrast-to-noise ratio (CNR), coefficient of variation (COV), and normalized noise power spectrum (NNPS) were calculated. The results demonstrate that the optimized smoothing factors of FNLMs are 0.08, 0.16, 0.22, 0.25, and 0.32 at 0.001, 0.005, 0.01, 0.05, and 0.1 of noise intensities, respectively. In addition, we compared the optimized FNLMs with noisy, local filters and total variation algorithms. As a result, FNLMs showed superior performance compared to various denoising techniques. Particularly, comparing the optimized FNLMs to the noisy images, the CNR improved by 6.53 to 16.34 times, COV improved by 6.55 to 18.28 times, and the NNPS improved by 10-2 mm2 on average. In conclusion, our approach shows significant potential in enhancing CT image quality with anthropomorphic phantoms, thus addressing the noise issue and improving diagnostic accuracy.

G-T correcting: an improved training of image segmentation under noisy labels

Data-driven medical image segmentation networks require expert annotations, which are hard to obtain. Non-expert annotations are often used instead, but these can be inaccurate (referred to as "noisy labels"), misleading the network's training and causing a decline in segmentation performance. In this study, we focus on improving the segmentation performance of neural networks when trained with noisy annotations. Specifically, we propose a two-stage framework named "G-T correcting," consisting of "G" stage for recognizing noisy labels and "T" stage for correcting noisy labels. In the "G" stage, a positive feedback method is proposed to automatically recognize noisy samples, using a Gaussian mixed model to classify clean and noisy labels through the per-sample loss histogram. In the "T" stage, a confident correcting strategy and early learning strategy are adopted to allow the segmentation network to receive productive guidance from noisy labels. Experiments on simulated and real-world noisy labels show that this method can achieve over 90% accuracy in recognizing noisy labels, and improve the network's DICE coefficient to 91%. The results demonstrate that the proposed method can enhance the segmentation performance of the network when trained with noisy labels, indicating good clinical application prospects.

Better performance of deep learning pulmonary nodule detection using chest radiography with pixel level labels in reference to computed tomography: data quality matters

Labeling errors can significantly impact the performance of deep learning models used for screening chest radiographs. The deep learning model for detecting pulmonary nodules is particularly vulnerable to such errors, mainly because normal chest radiographs and those with nodules obscured by ribs appear similar. Thus, high-quality datasets referred to chest computed tomography (CT) are required to prevent the misclassification of nodular chest radiographs as normal. From this perspective, a deep learning strategy employing chest radiography data with pixel-level annotations referencing chest CT scans may improve nodule detection and localization compared to image-level labels. We trained models using a National Institute of Health chest radiograph-based labeling dataset and an AI-HUB CT-based labeling dataset, employing DenseNet architecture with squeeze-and-excitation blocks. We developed four models to assess whether CT versus chest radiography and pixel-level versus image-level labeling would improve the deep learning model's performance to detect nodules. The models' performance was evaluated using two external validation datasets. The AI-HUB dataset with image-level labeling outperformed the NIH dataset (AUC 0.88 vs 0.71 and 0.78 vs. 0.73 in two external datasets, respectively; both p < 0.001). However, the AI-HUB data annotated at the pixel level produced the best model (AUC 0.91 and 0.86 in external datasets), and in terms of nodule localization, it significantly outperformed models trained with image-level annotation data, with a Dice coefficient ranging from 0.36 to 0.58. Our findings underscore the importance of accurately labeled data in developing reliable deep learning algorithms for nodule detection in chest radiography.

Using multi-label ensemble CNN classifiers to mitigate labelling inconsistencies in patch-level Gleason grading

This paper presents a novel approach to enhance the accuracy of patch-level Gleason grading in prostate histopathology images, a critical task in the diagnosis and prognosis of prostate cancer. This study shows that the Gleason grading accuracy can be improved by addressing the prevalent issue of label inconsistencies in the SICAPv2 prostate dataset, which employs a majority voting scheme for patch-level labels. We propose a multi-label ensemble deep-learning classifier that effectively mitigates these inconsistencies and yields more accurate results than the state-of-the-art works. Specifically, our approach leverages the strengths of three different one-vs-all deep learning models in an ensemble to learn diverse features from the histopathology images to individually indicate the presence of one or more Gleason grades (G3, G4, and G5) in each patch. These deep learning models have been trained using transfer learning to fine-tune a variant of the ResNet18 CNN classifier chosen after an extensive ablation study. Experimental results demonstrate that our multi-label ensemble classifier significantly outperforms traditional single-label classifiers reported in the literature by at least 14% and 4% on accuracy and f1-score metrics respectively. These results underscore the potential of our proposed machine learning approach to improve the accuracy and consistency of prostate cancer grading.

Weakly Supervised Deep Learning in Radiology

Deep learning (DL) is currently the standard artificial intelligence tool for computer-based image analysis in radiology. Traditionally, DL models have been trained with strongly supervised learning methods. These methods depend on reference standard labels, typically applied manually by experts. In contrast, weakly supervised learning is more scalable. Weak supervision comprises situations in which only a portion of the data are labeled (incomplete supervision), labels refer to a whole region or case as opposed to a precisely delineated image region (inexact supervision), or labels contain errors (inaccurate supervision). In many applications, weak labels are sufficient to train useful models. Thus, weakly supervised learning can unlock a large amount of otherwise unusable data for training DL models. One example of this is using large language models to automatically extract weak labels from free-text radiology reports. Here, we outline the key concepts in weakly supervised learning and provide an overview of applications in radiologic image analysis. With more fundamental and clinical translational work, weakly supervised learning could facilitate the uptake of DL in radiology and research workflows by enabling large-scale image analysis and advancing the development of new DL-based biomarkers.

Federated learning for medical image analysis: A survey

Machine learning in medical imaging often faces a fundamental dilemma, namely, the small sample size problem. Many recent studies suggest using multi-domain data pooled from different acquisition sites/centers to improve statistical power. However, medical images from different sites cannot be easily shared to build large datasets for model training due to privacy protection reasons. As a promising solution, federated learning, which enables collaborative training of machine learning models based on data from different sites without cross-site data sharing, has attracted considerable attention recently. In this paper, we conduct a comprehensive survey of the recent development of federated learning methods in medical image analysis. We have systematically gathered research papers on federated learning and its applications in medical image analysis published between 2017 and 2023. Our search and compilation were conducted using databases from IEEE Xplore, ACM Digital Library, Science Direct, Springer Link, Web of Science, Google Scholar, and PubMed. In this survey, we first introduce the background of federated learning for dealing with privacy protection and collaborative learning issues. We then present a comprehensive review of recent advances in federated learning methods for medical image analysis. Specifically, existing methods are categorized based on three critical aspects of a federated learning system, including client end, server end, and communication techniques. In each category, we summarize the existing federated learning methods according to specific research problems in medical image analysis and also provide insights into the motivations of different approaches. In addition, we provide a review of existing benchmark medical imaging datasets and software platforms for current federated learning research. We also conduct an experimental study to empirically evaluate typical federated learning methods for medical image analysis. This survey can help to better understand the current research status, challenges, and potential research opportunities in this promising research field.

A survey of label-noise deep learning for medical image analysis

Several factors are associated with the success of deep learning. One of the most important reasons is the availability of large-scale datasets with clean annotations. However, obtaining datasets with accurate labels in the medical imaging domain is challenging. The reliability and consistency of medical labeling are some of these issues, and low-quality annotations with label noise usually exist. Because noisy labels reduce the generalization performance of deep neural networks, learning with noisy labels is becoming an essential task in medical image analysis. Literature on this topic has expanded in terms of volume and scope. However, no recent surveys have collected and organized this knowledge, impeding the ability of researchers and practitioners to utilize it. In this work, we presented an up-to-date survey of label-noise learning for medical image domain. We reviewed extensive literature, illustrated some typical methods, and showed unified taxonomies in terms of methodological differences. Subsequently, we conducted the methodological comparison and demonstrated the corresponding advantages and disadvantages. Finally, we discussed new research directions based on the characteristics of medical images. Our survey aims to provide researchers and practitioners with a solid understanding of existing medical label-noise learning, such as the main algorithms developed over the past few years, which could help them investigate new methods to combat with the negative effects of label noise.

A comprehensive survey on deep active learning in medical image analysis

Deep learning has achieved widespread success in medical image analysis, leading to an increasing demand for large-scale expert-annotated medical image datasets. Yet, the high cost of annotating medical images severely hampers the development of deep learning in this field. To reduce annotation costs, active learning aims to select the most informative samples for annotation and train high-performance models with as few labeled samples as possible. In this survey, we review the core methods of active learning, including the evaluation of informativeness and sampling strategy. For the first time, we provide a detailed summary of the integration of active learning with other label-efficient techniques, such as semi-supervised, self-supervised learning, and so on. We also summarize active learning works that are specifically tailored to medical image analysis. Additionally, we conduct a thorough comparative analysis of the performance of different AL methods in medical image analysis with experiments. In the end, we offer our perspectives on the future trends and challenges of active learning and its applications in medical image analysis. An accompanying paper list and code for the comparative analysis is available in https://github.com/LightersWang/Awesome-Active-Learning-for-Medical-Image-Analysis.

The effect of editing clinical contours on deep-learning segmentation accuracy of the gross tumor volume in glioblastoma

Background and purpose

Deep-learning (DL) models for segmentation of the gross tumor volume (GTV) in radiotherapy are generally based on clinical delineations which suffer from inter-observer variability. The aim of this study was to compare performance of a DL-model based on clinical glioblastoma GTVs to a model based on a single-observer edited version of the same GTVs.

Materials and methods

The dataset included imaging data (Computed Tomography (CT), T1, contrast-T1 (T1C), and fluid-attenuated-inversion-recovery (FLAIR)) of 259 glioblastoma patients treated with post-operative radiotherapy between 2012 and 2019 at a single institute. The clinical GTVs were edited using all imaging data. The dataset was split into 207 cases for training/validation and 52 for testing.GTV segmentation models (nnUNet) were trained on clinical and edited GTVs separately and compared using Surface Dice with 1 mm tolerance (sDSC1mm). We also evaluated model performance with respect to extent of resection (EOR), and different imaging combinations (T1C/T1/FLAIR/CT, T1C/FLAIR/CT, T1C/FLAIR, T1C/CT, T1C/T1, T1C). A Wilcoxon test was used for significance testing.

Results

The median (range) sDSC1mm of the clinical-GTV-model and edited-GTV-model both evaluated with the edited contours, was 0.76 (0.43-0.94) vs. 0.92 (0.60-0.98) respectively (p < 0.001). sDSC1mm was not significantly different between patients with a biopsy, partial, and complete resection. T1C as single input performed as good as use of imaging combinations.

Conclusions

High segmentation accuracy was obtained by the DL-models. Editing of the clinical GTVs significantly increased DL performance with a relevant effect size. DL performance was robust for EOR and highly accurate using only T1C.

Impact of imperfect annotations on CNN training and performance for instance segmentation and classification in digital pathology

Segmentation and classification of large numbers of instances, such as cell nuclei, are crucial tasks in digital pathology for accurate diagnosis. However, the availability of high-quality datasets for deep learning methods is often limited due to the complexity of the annotation process. In this work, we investigate the impact of noisy annotations on the training and performance of a state-of-the-art CNN model for the combined task of detecting, segmenting and classifying nuclei in histopathology images. In this context, we investigate the conditions for determining an appropriate number of training epochs to prevent overfitting to annotation noise during training. Our results indicate that the utilisation of a small, correctly annotated validation set is instrumental in avoiding overfitting and maintaining model performance to a large extent. Additionally, our findings underscore the beneficial role of pre-training.

Surgical optomics: hyperspectral imaging and deep learning towards precision intraoperative automatic tissue recognition-results from the EX-MACHYNA trial

BACKGROUND

Hyperspectral imaging (HSI), combined with machine learning, can help to identify characteristic tissue signatures enabling automatic tissue recognition during surgery. This study aims to develop the first HSI-based automatic abdominal tissue recognition with human data in a prospective bi-center setting.

METHODS

Data were collected from patients undergoing elective open abdominal surgery at two international tertiary referral hospitals from September 2020 to June 2021. HS images were captured at various time points throughout the surgical procedure. Resulting RGB images were annotated with 13 distinct organ labels. Convolutional Neural Networks (CNNs) were employed for the analysis, with both external and internal validation settings utilized.

RESULTS

A total of 169 patients were included, 73 (43.2%) from Strasbourg and 96 (56.8%) from Verona. The internal validation within centers combined patients from both centers into a single cohort, randomly allocated to the training (127 patients, 75.1%, 585 images) and test sets (42 patients, 24.9%, 181 images). This validation setting showed the best performance. The highest true positive rate was achieved for the skin (100%) and the liver (97%). Misclassifications included tissues with a similar embryological origin (omentum and mesentery: 32%) or with overlaying boundaries (liver and hepatic ligament: 22%). The median DICE score for ten tissue classes exceeded 80%.

CONCLUSION

To improve automatic surgical scene segmentation and to drive clinical translation, multicenter accurate HSI datasets are essential, but further work is needed to quantify the clinical value of HSI. HSI might be included in a new omics science, namely surgical optomics, which uses light to extract quantifiable tissue features during surgery.

HAITCH: A Framework for Distortion and Motion Correction in Fetal Multi-Shell Diffusion-Weighted MRI

Diffusion magnetic resonance imaging (dMRI) is pivotal for probing the microstructure of the rapidly-developing fetal brain. However, fetal motion during scans and its interaction with magnetic field inhomogeneities result in artifacts and data scattering across spatial and angular domains. The effects of those artifacts are more pronounced in high-angular resolution fetal dMRI, where signal-to-noise ratio is very low. Those effects lead to biased estimates and compromise the consistency and reliability of dMRI analysis. This work presents HAITCH, the first and the only publicly available tool to correct and reconstruct multi-shell high-angular resolution fetal dMRI data. HAITCH offers several technical advances that include a blip-reversed dual-echo acquisition for dynamic distortion correction, advanced motion correction for model-free and robust reconstruction, optimized multi-shell design for enhanced information capture and increased tolerance to motion, and outlier detection for improved reconstruction fidelity. The framework is open-source, flexible, and can be used to process any type of fetal dMRI data including single-echo or single-shell acquisitions, but is most effective when used with multi-shell multi-echo fetal dMRI data that cannot be processed with any of the existing tools. Validation experiments on real fetal dMRI scans demonstrate significant improvements and accurate correction across diverse fetal ages and motion levels. HAITCH successfully removes artifacts and reconstructs high-fidelity fetal dMRI data suitable for advanced diffusion modeling, including fiber orientation distribution function estimation. These advancements pave the way for more reliable analysis of the fetal brain microstructure and tractography under challenging imaging conditions.

Deep learning with noisy labels in medical prediction problems: a scoping review

OBJECTIVES

Medical research faces substantial challenges from noisy labels attributed to factors like inter-expert variability and machine-extracted labels. Despite this, the adoption of label noise management remains limited, and label noise is largely ignored. To this end, there is a critical need to conduct a scoping review focusing on the problem space. This scoping review aims to comprehensively review label noise management in deep learning-based medical prediction problems, which includes label noise detection, label noise handling, and evaluation. Research involving label uncertainty is also included.

METHODS

Our scoping review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We searched 4 databases, including PubMed, IEEE Xplore, Google Scholar, and Semantic Scholar. Our search terms include "noisy label AND medical/healthcare/clinical," "uncertainty AND medical/healthcare/clinical," and "noise AND medical/healthcare/clinical."

RESULTS

A total of 60 papers met inclusion criteria between 2016 and 2023. A series of practical questions in medical research are investigated. These include the sources of label noise, the impact of label noise, the detection of label noise, label noise handling techniques, and their evaluation. Categorization of both label noise detection methods and handling techniques are provided.

DISCUSSION

From a methodological perspective, we observe that the medical community has been up to date with the broader deep-learning community, given that most techniques have been evaluated on medical data. We recommend considering label noise as a standard element in medical research, even if it is not dedicated to handling noisy labels. Initial experiments can start with easy-to-implement methods, such as noise-robust loss functions, weighting, and curriculum learning.

BadLabel: A Robust Perspective on Evaluating and Enhancing Label-Noise Learning

Label-noise learning (LNL) aims to increase the model's generalization given training data with noisy labels. To facilitate practical LNL algorithms, researchers have proposed different label noise types, ranging from class-conditional to instance-dependent noises. In this paper, we introduce a novel label noise type called BadLabel, which can significantly degrade the performance of existing LNL algorithms by a large margin. BadLabel is crafted based on the label-flipping attack against standard classification, where specific samples are selected and their labels are flipped to other labels so that the loss values of clean and noisy labels become indistinguishable. To address the challenge posed by BadLabel, we further propose a robust LNL method that perturbs the labels in an adversarial manner at each epoch to make the loss values of clean and noisy labels again distinguishable. Once we select a small set of (mostly) clean labeled data, we can apply the techniques of semi-supervised learning to train the model accurately. Empirically, our experimental results demonstrate that existing LNL algorithms are vulnerable to the newly introduced BadLabel noise type, while our proposed robust LNL method can effectively improve the generalization performance of the model under various types of label noise. The new dataset of noisy labels and the source codes of robust LNL algorithms are available at https://github.com/zjfheart/BadLabels.

Suppressing label noise in medical image classification using mixup attention and self-supervised learning

Deep neural networks (DNNs) have been widely applied in medical image classification and achieve remarkable classification performance. These achievements heavily depend on large-scale accurately annotated training data. However, label noise is inevitably introduced in the medical image annotation, as the labeling process heavily relies on the expertise and experience of annotators. Meanwhile, DNNs suffer from overfitting noisy labels, degrading the performance of models. Therefore, in this work, we innovatively devise a noise-robust training approach to mitigate the adverse effects of noisy labels in medical image classification. Specifically, we incorporate contrastive learning and intra-group mixup attention strategies into vanilla supervised learning. The contrastive learning for feature extractor helps to enhance visual representation of DNNs. The intra-group mixup attention module constructs groups and assigns self-attention weights for group-wise samples, and subsequently interpolates massive noisy-suppressed samples through weighted mixup operation. We conduct comparative experiments on both synthetic and real-world noisy medical datasets under various noise levels. Rigorous experiments validate that our noise-robust method with contrastive learning and mixup attention can effectively handle with label noise, and is superior to state-of-the-art methods. An ablation study also shows that both components contribute to boost model performance. The proposed method demonstrates its capability of curb label noise and has certain potential toward real-world clinic applications.

Pulse2AI: An Adaptive Framework to Standardize and Process Pulsatile Wearable Sensor Data for Clinical Applications

Goal: To establish Pulse2AI as a reproducible data preprocessing framework for pulsatile signals that generate high-quality machine-learning-ready datasets from raw wearable recordings. Methods: We proposed an end-to-end data preprocessing framework that adapts multiple pulsatile signal modalities and generates machine-learning-ready datasets agnostic to downstream medical tasks. Results: a dataset preprocessed by Pulse2AI improved systolic blood pressure estimation by 29.58%, from 11.41 to 8.03 mmHg in root-mean-square-error (RMSE) and its diastolic counterpart by 26.01%, from 7.93 to 5.87 mmHg in RMSE. For respiration rate (RR) estimation, Pulse2AI boosted performance by 19.69%, from 1.47 to 1.18 breaths per minute (BrPM) in mean-absolute-error (MAE). Conclusion: Pulse2AI turns pulsatile signals into machine learning (ML) ready datasets for arbitrary remote health monitoring tasks. We tested Pulse2AI on multiple pulsatile modalities and demonstrated its efficacy in two medical applications. This work bridges valuable assets in remote sensing and internet of medical things to ML-ready datasets for medical modeling.

Typicality- and instance-dependent label noise-combating: a novel framework for simulating and combating real-world noisy labels for endoscopic polyp classification

Learning with noisy labels aims to train neural networks with noisy labels. Current models handle instance-independent label noise (IIN) well; however, they fall short with real-world noise. In medical image classification, atypical samples frequently receive incorrect labels, rendering instance-dependent label noise (IDN) an accurate representation of real-world scenarios. However, the current IDN approaches fail to consider the typicality of samples, which hampers their ability to address real-world label noise effectively. To alleviate the issues, we introduce typicality- and instance-dependent label noise (TIDN) to simulate real-world noise and establish a TIDN-combating framework to combat label noise. Specifically, we use the sample's distance to decision boundaries in the feature space to represent typicality. The TIDN is then generated according to typicality. We establish a TIDN-attention module to combat label noise and learn the transition matrix from latent ground truth to the observed noisy labels. A recursive algorithm that enables the network to make correct predictions with corrections from the learned transition matrix is proposed. Our experiments demonstrate that the TIDN simulates real-world noise more closely than the existing IIN and IDN. Furthermore, the TIDN-combating framework demonstrates superior classification performance when training with simulated TIDN and actual real-world noise.

Deep-NCA: A deep learning methodology for performing noncompartmental analysis of pharmacokinetic data

Noncompartmental analysis (NCA) is a model-independent approach for assessing pharmacokinetics (PKs). Although the existing NCA algorithms are very well-established and widely utilized, they suffer from low accuracies in the setting of sparse PK samples. In response, we developed Deep-NCA, a deep learning (DL) model to improve the prediction of key noncompartmental PK parameters. Our methodology utilizes synthetic PK data for model training and uses an innovative patient-specific normalization method for data preprocessing. Deep-NCA demonstrated adequate performance across six previously unseen simulated drugs under multiple dosing, showcasing effective generalization. Compared to traditional NCA, Deep-NCA exhibited superior performance for sparse PK data. This study advances the application of DL to PK studies and introduces an effective method for handling sparse PK data. With further validation and refinement, Deep-NCA could significantly enhance the efficiency of drug development by providing more accurate NCA estimates while requiring fewer PK samples.

Automated Diagnosis of Major Depressive Disorder With Multi-Modal MRIs Based on Contrastive Learning: A Few-Shot Study

Depression ranks among the most prevalent mood-related psychiatric disorders. Existing clinical diagnostic approaches relying on scale interviews are susceptible to individual and environmental variations. In contrast, the integration of neuroimaging techniques and computer science has provided compelling evidence for the quantitative assessment of major depressive disorder (MDD). However, one of the major challenges in computer-aided diagnosis of MDD is to automatically and effectively mine the complementary cross-modal information from limited datasets. In this study, we proposed a few-shot learning framework that integrates multi-modal MRI data based on contrastive learning. In the upstream task, it is designed to extract knowledge from heterogeneous data. Subsequently, the downstream task is dedicated to transferring the acquired knowledge to the target dataset, where a hierarchical fusion paradigm is also designed to integrate features across inter- and intra-modalities. Lastly, the proposed model was evaluated on a set of multi-modal clinical data, achieving average scores of 73.52% and 73.09% for accuracy and AUC, respectively. Our findings also reveal that the brain regions within the default mode network and cerebellum play a crucial role in the diagnosis, which provides further direction in exploring reproducible biomarkers for MDD diagnosis.

Precise detection of awareness in disorders of consciousness using deep learning framework

Diagnosis of disorders of consciousness (DOC) remains a formidable challenge. Deep learning methods have been widely applied in general neurological and psychiatry disorders, while limited in DOC domain. Considering the successful use of resting-state functional MRI (rs-fMRI) for evaluating patients with DOC, this study seeks to explore the conjunction of deep learning techniques and rs-fMRI in precisely detecting awareness in DOC. We initiated our research with a benchmark dataset comprising 140 participants, including 76 unresponsive wakefulness syndrome (UWS), 25 minimally conscious state (MCS), and 39 Controls, from three independent sites. We developed a cascade 3D EfficientNet-B3-based deep learning framework tailored for discriminating MCS from UWS patients, referred to as "DeepDOC", and compared its performance against five state-of-the-art machine learning models. We also included an independent dataset consists of 11 DOC patients to test whether our model could identify patients with cognitive motor dissociation (CMD), in which DOC patients were behaviorally diagnosed unconscious but could be detected conscious by brain computer interface (BCI) method. Our results demonstrate that DeepDOC outperforms the five machine learning models, achieving an area under curve (AUC) value of 0.927 and accuracy of 0.861 for distinguishing MCS from UWS patients. More importantly, DeepDOC excels in CMD identification, achieving an AUC of 1 and accuracy of 0.909. Using gradient-weighted class activation mapping algorithm, we found that the posterior cortex, encompassing the visual cortex, posterior middle temporal gyrus, posterior cingulate cortex, precuneus, and cerebellum, as making a more substantial contribution to classification compared to other brain regions. This research offers a convenient and accurate method for detecting covert awareness in patients with MCS and CMD using rs-fMRI data.

An Automated Deep Learning-Based Framework for Uptake Segmentation and Classification on PSMA PET/CT Imaging of Patients with Prostate Cancer

Uptake segmentation and classification on PSMA PET/CT are important for automating whole-body tumor burden determinations. We developed and evaluated an automated deep learning (DL)-based framework that segments and classifies uptake on PSMA PET/CT. We identified 193 [18F] DCFPyL PET/CT scans of patients with biochemically recurrent prostate cancer from two institutions, including 137 [18F] DCFPyL PET/CT scans for training and internally testing, and 56 scans from another institution for external testing. Two radiologists segmented and labelled foci as suspicious or non-suspicious for malignancy. A DL-based segmentation was developed with two independent CNNs. An anatomical prior guidance was applied to make the DL framework focus on PSMA-avid lesions. Segmentation performance was evaluated by Dice, IoU, precision, and recall. Classification model was constructed with multi-modal decision fusion framework evaluated by accuracy, AUC, F1 score, precision, and recall. Automatic segmentation of suspicious lesions was improved under prior guidance, with mean Dice, IoU, precision, and recall of 0.700, 0.566, 0.809, and 0.660 on the internal test set and 0.680, 0.548, 0.749, and 0.740 on the external test set. Our multi-modal decision fusion framework outperformed single-modal and multi-modal CNNs with accuracy, AUC, F1 score, precision, and recall of 0.764, 0.863, 0.844, 0.841, and 0.847 in distinguishing suspicious and non-suspicious foci on the internal test set and 0.796, 0.851, 0.865, 0.814, and 0.923 on the external test set. DL-based lesion segmentation on PSMA PET is facilitated through our anatomical prior guidance strategy. Our classification framework differentiates suspicious foci from those not suspicious for cancer with good accuracy.

V2IED: Dual-view learning framework for detecting events of interictal epileptiform discharges

Interictal epileptiform discharges (IED) as large intermittent electrophysiological events are associated with various severe brain disorders. Automated IED detection has long been a challenging task, and mainstream methods largely focus on singling out IEDs from backgrounds from the perspective of waveform, leaving normal sharp transients/artifacts with similar waveforms almost unattended. An open issue still remains to accurately detect IED events that directly reflect the abnormalities in brain electrophysiological activities, minimizing the interference from irrelevant sharp transients with similar waveforms only. This study then proposes a dual-view learning framework (namely V2IED) to detect IED events from multi-channel EEG via aggregating features from the two phases: (1) Morphological Feature Learning: directly treating the EEG as a sequence with multiple channels, a 1D-CNN (Convolutional Neural Network) is applied to explicitly learning the deep morphological features; and (2) Spatial Feature Learning: viewing the EEG as a 3D tensor embedding channel topology, a CNN captures the spatial features at each sampling point followed by an LSTM (Long Short-Term Memories) to learn the evolution of these features. Experimental results from a public EEG dataset against the state-of-the-art counterparts indicate that: (1) compared with the existing optimal models, V2IED achieves a larger area under the receiver operating characteristic (ROC) curve in detecting IEDs from normal sharp transients with a 5.25% improvement in accuracy; (2) the introduction of spatial features improves performance by 2.4% in accuracy; and (3) V2IED also performs excellently in distinguishing IEDs from background signals especially benign variants.

Methods and Annotated Data Sets Used to Predict the Gender and Age of Twitter Users: Scoping Review

BACKGROUND

Patient health data collected from a variety of nontraditional resources, commonly referred to as real-world data, can be a key information source for health and social science research. Social media platforms, such as Twitter (Twitter, Inc), offer vast amounts of real-world data. An important aspect of incorporating social media data in scientific research is identifying the demographic characteristics of the users who posted those data. Age and gender are considered key demographics for assessing the representativeness of the sample and enable researchers to study subgroups and disparities effectively. However, deciphering the age and gender of social media users poses challenges.

OBJECTIVE

This scoping review aims to summarize the existing literature on the prediction of the age and gender of Twitter users and provide an overview of the methods used.

METHODS

We searched 15 electronic databases and carried out reference checking to identify relevant studies that met our inclusion criteria: studies that predicted the age or gender of Twitter users using computational methods. The screening process was performed independently by 2 researchers to ensure the accuracy and reliability of the included studies.

RESULTS

Of the initial 684 studies retrieved, 74 (10.8%) studies met our inclusion criteria. Among these 74 studies, 42 (57%) focused on predicting gender, 8 (11%) focused on predicting age, and 24 (32%) predicted a combination of both age and gender. Gender prediction was predominantly approached as a binary classification task, with the reported performance of the methods ranging from 0.58 to 0.96 F1-score or 0.51 to 0.97 accuracy. Age prediction approaches varied in terms of classification groups, with a higher range of reported performance, ranging from 0.31 to 0.94 F1-score or 0.43 to 0.86 accuracy. The heterogeneous nature of the studies and the reporting of dissimilar performance metrics made it challenging to quantitatively synthesize results and draw definitive conclusions.

CONCLUSIONS

Our review found that although automated methods for predicting the age and gender of Twitter users have evolved to incorporate techniques such as deep neural networks, a significant proportion of the attempts rely on traditional machine learning methods, suggesting that there is potential to improve the performance of these tasks by using more advanced methods. Gender prediction has generally achieved a higher reported performance than age prediction. However, the lack of standardized reporting of performance metrics or standard annotated corpora to evaluate the methods used hinders any meaningful comparison of the approaches. Potential biases stemming from the collection and labeling of data used in the studies was identified as a problem, emphasizing the need for careful consideration and mitigation of biases in future studies. This scoping review provides valuable insights into the methods used for predicting the age and gender of Twitter users, along with the challenges and considerations associated with these methods.

Advances in artificial intelligence for the diagnosis and treatment of ovarian cancer (Review)

Artificial intelligence (AI) has emerged as a crucial technique for extracting high‑throughput information from various sources, including medical images, pathological images, and genomics, transcriptomics, proteomics and metabolomics data. AI has been widely used in the field of diagnosis, for the differentiation of benign and malignant ovarian cancer (OC), and for prognostic assessment, with favorable results. Notably, AI‑based radiomics has proven to be a non‑invasive, convenient and economical approach, making it an essential asset in a gynecological setting. The present study reviews the application of AI in the diagnosis, differentiation and prognostic assessment of OC. It is suggested that AI‑based multi‑omics studies have the potential to improve the diagnostic and prognostic predictive ability in patients with OC, thereby facilitating the realization of precision medicine.

Detailed Image Data Quality and Cleaning Practices for Artificial Intelligence Tools for Breast Cancer

Standardizing image-data preparation practices to improve accuracy/consistency of AI diagnostic tools.

Learning from crowds for automated histopathological image segmentation

Automated semantic segmentation of histopathological images is an essential task in Computational Pathology (CPATH). The main limitation of Deep Learning (DL) to address this task is the scarcity of expert annotations. Crowdsourcing (CR) has emerged as a promising solution to reduce the individual (expert) annotation cost by distributing the labeling effort among a group of (non-expert) annotators. Extracting knowledge in this scenario is challenging, as it involves noisy annotations. Jointly learning the underlying (expert) segmentation and the annotators' expertise is currently a commonly used approach. Unfortunately, this approach is frequently carried out by learning a different neural network for each annotator, which scales poorly when the number of annotators grows. For this reason, this strategy cannot be easily applied to real-world CPATH segmentation. This paper proposes a new family of methods for CR segmentation of histopathological images. Our approach consists of two coupled networks: a segmentation network (for learning the expert segmentation) and an annotator network (for learning the annotators' expertise). We propose to estimate the annotators' behavior with only one network that receives the annotator ID as input, achieving scalability on the number of annotators. Our family is composed of three different models for the annotator network. Within this family, we propose a novel modeling of the annotator network in the CR segmentation literature, which considers the global features of the image. We validate our methods on a real-world dataset of Triple Negative Breast Cancer images labeled by several medical students. Our new CR modeling achieves a Dice coefficient of 0.7827, outperforming the well-known STAPLE (0.7039) and being competitive with the supervised method with expert labels (0.7723). The code is available at https://github.com/wizmik12/CRowd_Seg.

RobOCTNet: Robotics and Deep Learning for Referable Posterior Segment Pathology Detection in an Emergency Department Population

Purpose

To evaluate the diagnostic performance of a robotically aligned optical coherence tomography (RAOCT) system coupled with a deep learning model in detecting referable posterior segment pathology in OCT images of emergency department patients.

Methods

A deep learning model, RobOCTNet, was trained and internally tested to classify OCT images as referable versus non-referable for ophthalmology consultation. For external testing, emergency department patients with signs or symptoms warranting evaluation of the posterior segment were imaged with RAOCT. RobOCTNet was used to classify the images. Model performance was evaluated against a reference standard based on clinical diagnosis and retina specialist OCT review.

Results

We included 90,250 OCT images for training and 1489 images for internal testing. RobOCTNet achieved an area under the curve (AUC) of 1.00 (95% confidence interval [CI], 0.99-1.00) for detection of referable posterior segment pathology in the internal test set. For external testing, RAOCT was used to image 72 eyes of 38 emergency department patients. In this set, RobOCTNet had an AUC of 0.91 (95% CI, 0.82-0.97), a sensitivity of 95% (95% CI, 87%-100%), and a specificity of 76% (95% CI, 62%-91%). The model's performance was comparable to two human experts' performance.

Conclusions

A robotically aligned OCT coupled with a deep learning model demonstrated high diagnostic performance in detecting referable posterior segment pathology in a cohort of emergency department patients.

Translational Relevance

Robotically aligned OCT coupled with a deep learning model may have the potential to improve emergency department patient triage for ophthalmology referral.

Learning Better Registration to Learn Better Few-Shot Medical Image Segmentation: Authenticity, Diversity, and Robustness

In this work, we address the task of few-shot medical image segmentation (MIS) with a novel proposed framework based on the learning registration to learn segmentation (LRLS) paradigm. To cope with the limitations of lack of authenticity, diversity, and robustness in the existing LRLS frameworks, we propose the better registration better segmentation (BRBS) framework with three main contributions that are experimentally shown to have substantial practical merit. First, we improve the authenticity in the registration-based generation program and propose the knowledge consistency constraint strategy that constrains the registration network to learn according to the domain knowledge. It brings the semantic-aligned and topology-preserved registration, thus allowing the generation program to output new data with great space and style authenticity. Second, we deeply studied the diversity of the generation process and propose the space-style sampling program, which introduces the modeling of the transformation path of style and space change between few atlases and numerous unlabeled images into the generation program. Therefore, the sampling on the transformation paths provides much more diverse space and style features to the generated data effectively improving the diversity. Third, we first highlight the robustness in the learning of segmentation in the LRLS paradigm and propose the mix misalignment regularization, which simulates the misalignment distortion and constrains the network to reduce the fitting degree of misaligned regions. Therefore, it builds regularization for these regions improving the robustness of segmentation learning. Without any bells and whistles, our approach achieves a new state-of-the-art performance in few-shot MIS on two challenging tasks that outperform the existing LRLS-based few-shot methods. We believe that this novel and effective framework will provide a powerful few-shot benchmark for the field of medical image and efficiently reduce the costs of medical image research. All of our code will be made publicly available online.

Leveraging Multi-Annotator Label Uncertainties as Privileged Information for Acute Respiratory Distress Syndrome Detection in Chest X-ray Images

Acute Respiratory Distress Syndrome (ARDS) is a life-threatening lung injury for which early diagnosis and evidence-based treatment can improve patient outcomes. Chest X-rays (CXRs) play a crucial role in the identification of ARDS; however, their interpretation can be difficult due to non-specific radiological features, uncertainty in disease staging, and inter-rater variability among clinical experts, thus leading to prominent label noise issues. To address these challenges, this study proposes a novel approach that leverages label uncertainty from multiple annotators to enhance ARDS detection in CXR images. Label uncertainty information is encoded and supplied to the model as privileged information, a form of information exclusively available during the training stage and not during inference. By incorporating the Transfer and Marginalized (TRAM) network and effective knowledge transfer mechanisms, the detection model achieved a mean testing AUROC of 0.850, an AUPRC of 0.868, and an F1 score of 0.797. After removing equivocal testing cases, the model attained an AUROC of 0.973, an AUPRC of 0.971, and an F1 score of 0.921. As a new approach to addressing label noise in medical image analysis, the proposed model has shown superiority compared to the original TRAM, Confusion Estimation, and mean-aggregated label training. The overall findings highlight the effectiveness of the proposed methods in addressing label noise in CXRs for ARDS detection, with potential for use in other medical imaging domains that encounter similar challenges.

Translating prognostic quantification of c-MYC and BCL2 from tissue microarrays to whole slide images in diffuse large B-cell lymphoma using deep learning

BACKGROUND

c-MYC and BCL2 positivity are important prognostic factors for diffuse large B-cell lymphoma. However, manual quantification is subject to significant intra- and inter-observer variability. We developed an automated method for quantification in whole-slide images of tissue sections where manual quantification requires evaluating large areas of tissue with possibly heterogeneous staining. We train this method using annotations of tumor positivity in smaller tissue microarray cores where expression and staining are more homogeneous and then translate this model to whole-slide images.

METHODS

Our method applies a technique called attention-based multiple instance learning to regress the proportion of c-MYC-positive and BCL2-positive tumor cells from pathologist-scored tissue microarray cores. This technique does not require annotation of individual cell nuclei and is trained instead on core-level annotations of percent tumor positivity. We translate this model to scoring of whole-slide images by tessellating the slide into smaller core-sized tissue regions and calculating an aggregate score. Our method was trained on a public tissue microarray dataset from Stanford and applied to whole-slide images from a geographically diverse multi-center cohort produced by the Lymphoma Epidemiology of Outcomes study.

RESULTS

In tissue microarrays, the automated method had Pearson correlations of 0.843 and 0.919 with pathologist scores for c-MYC and BCL2, respectively. When utilizing standard clinical thresholds, the sensitivity/specificity of our method was 0.743 / 0.963 for c-MYC and 0.938 / 0.951 for BCL2. For double-expressors, sensitivity and specificity were 0.720 and 0.974. When translated to the external WSI dataset scored by two pathologists, Pearson correlation was 0.753 & 0.883 for c-MYC and 0.749 & 0.765 for BCL2, and sensitivity/specificity was 0.857/0.991 & 0.706/0.930 for c-MYC, 0.856/0.719 & 0.855/0.690 for BCL2, and 0.890/1.00 & 0.598/0.952 for double-expressors. Survival analysis demonstrates that for progression-free survival, model-predicted TMA scores significantly stratify double-expressors and non double-expressors (p = 0.0345), whereas pathologist scores do not (p = 0.128).

CONCLUSIONS

We conclude that proportion of positive stains can be regressed using attention-based multiple instance learning, that these models generalize well to whole slide images, and that our models can provide non-inferior stratification of progression-free survival outcomes.

Optimizing the Synergistic Potential of Pseudo-Labels from Radiology Notes and Annotated Ground Truth in Identifying Pulmonary Opacities on Chest Radiographs for Early Detection of Acute Respiratory Distress Syndrome

Acute Respiratory Distress Syndrome (ARDS) is a life-threatening lung injury, hallmarks of which are bilateral radiographic opacities. Studies have shown that early recognition of ARDS could reduce severity and lethal clinical sequela. A Convolutional Neural Network (CNN) model that can identify bilateral pulmonary opacities on chest x-ray (CXR) images can aid early ARDS recognition. Obtaining large datasets with ground truth labels to train CNNs is challenging, as medical image annotation requires clinical expertise and meticulous consideration. In this work, we implement a natural language processing pipeline that extracts pseudo-labels CXR images by parsing radiology notes for abnormal findings. We obtain ground-truth annotations from clinicians for the presence of pulmonary opacities for a subset of these images. A knowledge distillation-based teacher-student training framework is implemented to leverage the larger dataset with noisy pseudo-labels. Our results show an AUC of 0.93 (95%CI 0.92-0.94) for the prediction of bilateral opacities on chest radiographs.

Automated non-contact measurement of the spine curvature at the sagittal plane using a deep neural network

BACKGROUND

Non-radiographical techniques have been suggested to measure the spine curvature at the sagittal plane. However, a neural network has not been used to measure the curvature.

METHODS

A single video camera captured images of a standing posture at the sagittal plane from twenty healthy males. Six marker positions along the spine's contour in each image were identified for measuring inclination, thoracic kyphosis, and lumbar lordosis angles. We estimated three inflection points around the neck, hip, and between the neck and hip, followed by identifying two adjacent marker positions per inflection point to compute its tangent. The angular deviation of each tangent line from the horizontal was computed to measure inclination angles. Thoracic kyphosis and lumbar lordosis angles were computed by the angular difference between the two adjacent tangents. A deep neural network was trained with 500,000 iterations using the labeled images from 18 participants (388 and 44 images for training and test set) and then evaluated using the unseen images (2 participants, 48 images; evaluation set).

FINDINGS

The mean total training and test errors were <2 pixels (∼ 0.6 cm). The total error in the evaluation set was qualitatively comparable (∼ 3 pixels = ∼ 0.9 cm), suggesting the model performance was maintained in the unseen data. The angle values between labeled and network-predicted marker positions were similar in the evaluation set.

INTERPRETATION

The network training with a relatively small number of images was successful based on the small error values observed in the evaluation set. The model may be an affordable, automated, and non-contact measurement tool for the human spine curvature.

Uncertainty-aware network for fine-grained and imbalanced reflux esophagitis grading

Computer-aided diagnosis (CAD) assists endoscopists in analyzing endoscopic images, reducing misdiagnosis rates and enabling timely treatment. A few studies have focused on CAD for gastroesophageal reflux disease, but CAD studies on reflux esophagitis (RE) are still inadequate. This paper presents a CAD study on RE using a dataset collected from hospital, comprising over 3000 images. We propose an uncertainty-aware network with handcrafted features, utilizing representation and classifier decoupling with metric learning to address class imbalance and achieve fine-grained RE classification. To enhance interpretability, the network estimates uncertainty through test time augmentation. The experimental results demonstrate that the proposed network surpasses previous methods, achieving an accuracy of 90.2% and an F1 score of 90.1%.

An artificial intelligence model for embryo selection in preimplantation DNA methylation screening in assisted reproductive technology

Embryo quality is a critical determinant of clinical outcomes in assisted reproductive technology (ART). A recent clinical trial investigating preimplantation DNA methylation screening (PIMS) revealed that whole genome DNA methylation level is a novel biomarker for assessing ART embryo quality. Here, we reinforced and estimated the clinical efficacy of PIMS. We introduce PIMS-AI, an innovative artificial intelligence (AI) based model, to predict the probability of an embryo producing live birth and subsequently assist ART embryo selection. Our model demonstrated robust performance, achieving an area under the curve (AUC) of 0.90 in cross-validation and 0.80 in independent testing. In simulated embryo selection, PIMS-AI attained an accuracy of 81% in identifying viable embryos for patients. Notably, PIMS-AI offers significant advantages over conventional preimplantation genetic testing for aneuploidy (PGT-A), including enhanced embryo discriminability and the potential to benefit a broader patient population. In conclusion, our approach holds substantial promise for clinical application and has the potential to significantly improve the ART success rate.

Strengthening deep-learning models for intracranial hemorrhage detection: strongly annotated computed tomography images and model ensembles

Background and purpose

Multiple attempts at intracranial hemorrhage (ICH) detection using deep-learning techniques have been plagued by clinical failures. We aimed to compare the performance of a deep-learning algorithm for ICH detection trained on strongly and weakly annotated datasets, and to assess whether a weighted ensemble model that integrates separate models trained using datasets with different ICH improves performance.

Methods

We used brain CT scans from the Radiological Society of North America (27,861 CT scans, 3,528 ICHs) and AI-Hub (53,045 CT scans, 7,013 ICHs) for training. DenseNet121, InceptionResNetV2, MobileNetV2, and VGG19 were trained on strongly and weakly annotated datasets and compared using independent external test datasets. We then developed a weighted ensemble model combining separate models trained on all ICH, subdural hemorrhage (SDH), subarachnoid hemorrhage (SAH), and small-lesion ICH cases. The final weighted ensemble model was compared to four well-known deep-learning models. After external testing, six neurologists reviewed 91 ICH cases difficult for AI and humans.

Results

InceptionResNetV2, MobileNetV2, and VGG19 models outperformed when trained on strongly annotated datasets. A weighted ensemble model combining models trained on SDH, SAH, and small-lesion ICH had a higher AUC, compared with a model trained on all ICH cases only. This model outperformed four deep-learning models (AUC [95% C.I.]: Ensemble model, 0.953[0.938-0.965]; InceptionResNetV2, 0.852[0.828-0.873]; DenseNet121, 0.875[0.852-0.895]; VGG19, 0.796[0.770-0.821]; MobileNetV2, 0.650[0.620-0.680]; p < 0.0001). In addition, the case review showed that a better understanding and management of difficult cases may facilitate clinical use of ICH detection algorithms.

Conclusion

We propose a weighted ensemble model for ICH detection, trained on large-scale, strongly annotated CT scans, as no model can capture all aspects of complex tasks.

CheXNet and feature pyramid network: a fusion deep learning architecture for multilabel chest X-Ray clinical diagnoses classification

The existing multilabel X-Ray image learning tasks generally contain much information on pathology co-occurrence and interdependency, which is very important for clinical diagnosis. However, the challenging part of this subject is to accurately diagnose multiple diseases that occurred in a single X-Ray image since multiple levels of features are generated in the images, and create different features as in single label detection. Various works were developed to address this challenge with proposed deep learning architectures to improve classification performance and enrich diagnosis results with multi-probability disease detection. The objective is to create an accurate result and a faster inference system to support a quick diagnosis in the medical system. To contribute to this state-of-the-art, we designed a fusion architecture, CheXNet and Feature Pyramid Network (FPN), to classify and discriminate multiple thoracic diseases from chest X-Rays. This concept enables the model to extract while creating a pyramid of feature maps with different spatial resolutions that capture low-level and high-level semantic information to encounter multiple features. The model's effectiveness is evaluated using the NIH ChestXray14 dataset, with the Area Under Curve (AUC) and accuracy metrics used to compare the results against other cutting-edge approaches. The overall results demonstrate that our method outperforms other approaches and has become promising for multilabel disease classification in chest X-Rays, with potential applications in clinical practice. The result demonstrated that we achieved an average AUC of 0.846 and an accuracy of 0.914. Further, our proposed architecture diagnoses images in 0.013 s, faster than the latest approaches.

Deep learning in bioinformatics

Deep learning is a powerful machine learning technique that can learn from large amounts of data using multiple layers of artificial neural networks. This paper reviews some applications of deep learning in bioinformatics, a field that deals with analyzing and interpreting biological data. We first introduce the basic concepts of deep learning and then survey the recent advances and challenges of applying deep learning to various bioinformatics problems, such as genome sequencing, gene expression analysis, protein structure prediction, drug discovery, and disease diagnosis. We also discuss future directions and opportunities for deep learning in bioinformatics. We aim to provide an overview of deep learning so that bioinformaticians applying deep learning models can consider all critical technical and ethical aspects. Thus, our target audience is biomedical informatics researchers who use deep learning models for inference. This review will inspire more bioinformatics researchers to adopt deep-learning methods for their research questions while considering fairness, potential biases, explainability, and accountability.

Reproducible and clinically translatable deep neural networks for cervical screening

Cervical cancer is a leading cause of cancer mortality, with approximately 90% of the 250,000 deaths per year occurring in low- and middle-income countries (LMIC). Secondary prevention with cervical screening involves detecting and treating precursor lesions; however, scaling screening efforts in LMIC has been hampered by infrastructure and cost constraints. Recent work has supported the development of an artificial intelligence (AI) pipeline on digital images of the cervix to achieve an accurate and reliable diagnosis of treatable precancerous lesions. In particular, WHO guidelines emphasize visual triage of women testing positive for human papillomavirus (HPV) as the primary screen, and AI could assist in this triage task. In this work, we implemented a comprehensive deep-learning model selection and optimization study on a large, collated, multi-geography, multi-institution, and multi-device dataset of 9462 women (17,013 images). We evaluated relative portability, repeatability, and classification performance. The top performing model, when combined with HPV type, achieved an area under the Receiver Operating Characteristics (ROC) curve (AUC) of 0.89 within our study population of interest, and a limited total extreme misclassification rate of 3.4%, on held-aside test sets. Our model also produced reliable and consistent predictions, achieving a strong quadratic weighted kappa (QWK) of 0.86 and a minimal %2-class disagreement (% 2-Cl. D.) of 0.69%, between image pairs across women. Our work is among the first efforts at designing a robust, repeatable, accurate and clinically translatable deep-learning model for cervical screening.

Medical federated learning with joint graph purification for noisy label learning

In terms of increasing privacy issues, Federated Learning (FL) has received extensive attention in medical imaging. Through collaborative training, FL can produce superior diagnostic models with global knowledge, while preserving private data locally. In practice, medical diagnosis suffers from intra-/inter-observer variability, thus label noise is inevitable in dataset preparation. Different from existing studies on centralized datasets, the label noise problem in FL scenarios confronts more challenges, due to data inaccessibility and even noise heterogeneity. In this work, we propose a federated framework with joint Graph Purification (FedGP) to address the label noise in FL through server and clients collaboration. Specifically, to overcome the impact of label noise on local training, we first devise a noisy graph purification on the client side to generate reliable pseudo labels by progressively expanding the purified graph with topological knowledge. Then, we further propose a graph-guided negative ensemble loss to exploit the topology of the client-side purified graph with robust complementary supervision against label noise. Moreover, to address the FL label noise with data silos, we propose a global centroid aggregation on the server side to produce a robust classifier with global knowledge, which can be optimized collaboratively in the FL framework. Extensive experiments are conducted on endoscopic and pathological images with the comparison under the homogeneous, heterogeneous, and real-world label noise for medical FL. Among these diverse noisy FL settings, our FedGP framework significantly outperforms denoising and noisy FL state-of-the-arts by a large margin. The source code is available at https://github.com/CUHK-AIM-Group/FedGP.

USE-Evaluator: Performance metrics for medical image segmentation models supervised by uncertain, small or empty reference annotations in neuroimaging

Performance metrics for medical image segmentation models are used to measure the agreement between the reference annotation and the predicted segmentation. Usually, overlap metrics, such as the Dice, are used as a metric to evaluate the performance of these models in order for results to be comparable. However, there is a mismatch between the distributions of cases and the difficulty level of segmentation tasks in public data sets compared to clinical practice. Common metrics used to assess performance fail to capture the impact of this mismatch, particularly when dealing with datasets in clinical settings that involve challenging segmentation tasks, pathologies with low signal, and reference annotations that are uncertain, small, or empty. Limitations of common metrics may result in ineffective machine learning research in designing and optimizing models. To effectively evaluate the clinical value of such models, it is essential to consider factors such as the uncertainty associated with reference annotations, the ability to accurately measure performance regardless of the size of the reference annotation volume, and the classification of cases where reference annotations are empty. We study how uncertain, small, and empty reference annotations influence the value of metrics on a stroke in-house data set regardless of the model. We examine metrics behavior on the predictions of a standard deep learning framework in order to identify suitable metrics in such a setting. We compare our results to the BRATS 2019 and Spinal Cord public data sets. We show how uncertain, small, or empty reference annotations require a rethinking of the evaluation. The evaluation code was released to encourage further analysis of this topic https://github.com/SophieOstmeier/UncertainSmallEmpty.git.

Efficient automated error detection in medical data using deep-learning and label-clustering

Medical datasets inherently contain errors from subjective or inaccurate test results, or from confounding biological complexities. It is difficult for medical experts to detect these elusive errors manually, due to lack of contextual information, limiting data privacy regulations, and the sheer scale of data to be reviewed. Current methods for training robust artificial intelligence (AI) models on data containing mislabeled examples generally fall into one of several categories-attempting to improve the robustness of the model architecture, the regularization techniques used, the loss function used during training, or selecting a subset of data that contains cleaner labels. This last category requires the ability to efficiently detect errors either prior to or during training, either relabeling them or removing them completely. More recent progress in error detection has focused on using multi-network learning to minimize deleterious effects of errors on training, however, using many neural networks to reach a consensus on which data should be removed can be computationally intensive and inefficient. In this work, a deep-learning based algorithm was used in conjunction with a label-clustering approach to automate error detection. For dataset with synthetic label flips added, these errors were identified with an accuracy of up to 85%, while requiring up to 93% less computing resources to complete compared to a previous model consensus approach developed previously. The resulting trained AI models exhibited greater training stability and up to a 45% improvement in accuracy, from 69 to over 99% compared to the consensus approach, at least 10% improvement on using noise-robust loss functions in a binary classification problem, and a 51% improvement for multi-class classification. These results indicate that practical, automated a priori detection of errors in medical data is possible, without human oversight.