Addressing class imbalance in deep learning for small lesion detection on medical images

Fully automated 3D machine learning model for HPV status characterization in oropharyngeal squamous cell carcinomas based on CT images

BACKGROUND

Human papillomavirus (HPV) status plays a major role in predicting oropharyngeal squamous cell carcinoma (OPSCC) survival. This study assesses the accuracy of a fully automated 3D convolutional neural network (CNN) in predicting HPV status using CT images.

METHODS

Pretreatment CT images from OPSCC patients were used to train a 3D DenseNet-121 model to predict HPV-p16 status. Performance was evaluated by the ROC Curve (AUC), sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1 score.

RESULTS

The network achieved a mean AUC of 0.80 ± 0.06. The best-preforming fold had a sensitivity of 0.86 and specificity of 0.92 at the Youden's index. The PPV, NPV, and F1 scores are 0.97, 0.71, and 0.82, respectively.

CONCLUSIONS

A fully automated CNN can characterize the HPV status of OPSCC patients with high sensitivity and specificity. Further refinement of this algorithm has the potential to provide a non-invasive tool to guide clinical management.

Predictive health monitoring: Leveraging artificial intelligence for early detection of infectious diseases in nursing home residents through discontinuous vital signs analysis

This research addresses the problem of detecting acute respiratory, urinary tract, and other infectious diseases in elderly nursing home residents using machine learning algorithms. The study analyzes data extracted from multiple vital signs and other contextual information for diagnostic purposes. The daily data collection process encounters sampling constraints due to weekends, holidays, shift changes, staff turnover, and equipment breakdowns, resulting in numerous nulls, repeated readings, outliers, and meaningless values. The short time series generated also pose a challenge to analysis, preventing the extraction of seasonal information or consistent trends. Blind data collection results in most of the data coming from periods when residents are healthy, resulting in excessively imbalanced data. This study proposes a data cleaning process and then builds a mechanism that reproduces the basal activity of the residents to improve the classification of the disease. The results show that the proposed basal module-assisted machine learning techniques allow anticipating diagnostics 2, 3 or 4 days before doctors decide to start treatment with antibiotics, achieving a performance measured by the area-under-the-curve metric of 0.857. The contributions of this work are: (1) a new data cleaning process; (2) the analysis of contextual information to improve data quality; (3) the generation of a baseline measure for relative comparison; and (4) the use of either binary (disease/no disease) or multiclass classification, differentiating among types of infections and showing the advantages of multiclass versus binary classification. From a medical point of view, the anticipated detection of infectious diseases in institutionalized individuals is brand new.

GravityNet for end-to-end small lesion detection

This paper introduces a novel one-stage end-to-end detector specifically designed to detect small lesions in medical images. Precise localization of small lesions presents challenges due to their appearance and the diverse contextual backgrounds in which they are found. To address this, our approach introduces a new type of pixel-based anchor that dynamically moves towards the targeted lesion for detection. We refer to this new architecture as GravityNet, and the novel anchors as gravity points since they appear to be "attracted" by the lesions. We conducted experiments on two well-established medical problems involving small lesions to evaluate the performance of the proposed approach: microcalcifications detection in digital mammograms and microaneurysms detection in digital fundus images. Our method demonstrates promising results in effectively detecting small lesions in these medical imaging tasks.

A robust model training strategy using hard negative mining in a weakly labeled dataset for lymphatic invasion in gastric cancer

Gastric cancer is a significant public health concern, emphasizing the need for accurate evaluation of lymphatic invasion (LI) for determining prognosis and treatment options. However, this task is time-consuming, labor-intensive, and prone to intra- and interobserver variability. Furthermore, the scarcity of annotated data presents a challenge, particularly in the field of digital pathology. Therefore, there is a demand for an accurate and objective method to detect LI using a small dataset, benefiting pathologists. In this study, we trained convolutional neural networks to classify LI using a four-step training process: (1) weak model training, (2) identification of false positives, (3) hard negative mining in a weakly labeled dataset, and (4) strong model training. To overcome the lack of annotated datasets, we applied a hard negative mining approach in a weakly labeled dataset, which contained only final diagnostic information, resembling the typical data found in hospital databases, and improved classification performance. Ablation studies were performed to simulate the lack of datasets and severely unbalanced datasets, further confirming the effectiveness of our proposed approach. Notably, our results demonstrated that, despite the small number of annotated datasets, efficient training was achievable, with the potential to extend to other image classification approaches used in medicine.

Reduction of overfitting on the highly imbalanced ISIC-2019 skin dataset using deep learning frameworks

BACKGROUND

With the rapid growth of Deep Neural Networks (DNN) and Computer-Aided Diagnosis (CAD), more significant works have been analysed for cancer related diseases. Skin cancer is the most hazardous type of cancer that cannot be diagnosed in the early stages.

OBJECTIVE

The diagnosis of skin cancer is becoming a challenge to dermatologists as an abnormal lesion looks like an ordinary nevus at the initial stages. Therefore, early identification of lesions (origin of skin cancer) is essential and helpful for treating skin cancer patients effectively. The enormous development of automated skin cancer diagnosis systems significantly supports dermatologists.

METHODS

This paper performs a classification of skin cancer by utilising various deep-learning frameworks after resolving the class Imbalance problem in the ISIC-2019 dataset. A fine-tuned ResNet-50 model is used to evaluate the performance of original data, augmented data, and after by adding the focal loss. Focal loss is the best technique to solve overfitting problems by assigning weights to hard misclassified images.

RESULTS

Finally, augmented data with focal loss is given a good classification performance with 98.85% accuracy, 95.52% precision, and 95.93% recall. Matthews Correlation coefficient (MCC) is the best metric to evaluate the quality of multi-class images. It has given outstanding performance by using augmented data and focal loss.

Application of cloud server-based machine learning for assisting pathological structure recognition in IgA nephropathy

BACKGROUND

Machine learning (ML) models can help assisting diagnosis by rapidly localising and classifying regions of interest (ROIs) within whole slide images (WSIs). Effective ML models for clinical decision support require a substantial dataset of 'real' data, and in reality, it should be robust, user-friendly and universally applicable.

METHODS

WSIs of primary IgAN were collected and annotated. The H-AI-L algorithm which could facilitate direct WSI viewing and potential ROI detection for clinicians was built on the cloud server of matpool, a shared internet-based service platform. Model performance was evaluated using F1-score, precision, recall and Matthew's correlation coefficient (MCC).

RESULTS

The F1-score of glomerular localisation in WSIs was 0.85 and 0.89 for the initial and pretrained models, respectively, with corresponding recall values of 0.79 and 0.83, and precision scores of 0.92 and 0.97. Dichotomous differentiation between global sclerotic (GS) and other glomeruli revealed F1-scores of 0.70 and 0.91, and MCC values of 0.55 and 0.87, for the initial and pretrained models, respectively. The overall F1-score of multiclassification was 0.81 for the pretrained models. The total glomerular recall rate was 0.96, with F1-scores of 0.68, 0.56 and 0.26 for GS, segmental glomerulosclerosis and crescent (C), respectively. Interstitial fibrosis/tubular atrophy lesion similarity between the true label and model predictions was 0.75.

CONCLUSIONS

Our results underscore the efficacy of the ML integration algorithm in segmenting ROIs in IgAN WSIs, and the internet-based model deployment is in favour of widespread adoption and utilisation across multiple centres and increased volumes of WSIs.

Learnable DoG convolutional filters for microcalcification detection

Difference of Gaussians (DoG) convolutional filters are one of the earliest image processing methods employed for detecting microcalcifications on mammogram images before machine and deep learning methods became widespread. DoG is a blob enhancement filter that consists in subtracting one Gaussian-smoothed version of an image from another less Gaussian-smoothed version of the same image. Smoothing with a Gaussian kernel suppresses high-frequency spatial information, thus DoG can be regarded as a band-pass filter. However, due to their small size and overimposed breast tissue, microcalcifications vary greatly in contrast-to-noise ratio and sharpness. This makes it difficult to find a single DoG configuration that enhances all microcalcifications. In this work, we propose a convolutional network, named DoG-MCNet, where the first layer automatically learns a bank of DoG filters parameterized by their associated standard deviations. We experimentally show that when employed for microcalcification detection, our DoG layer acts as a learnable bank of band-pass preprocessing filters and improves detection performance by 4.86% AUFROC over baseline MCNet and 1.53% AUFROC over state-of-the-art multicontext ensemble of CNNs.

A universal lesion detection method based on partially supervised learning

Partially supervised learning (PSL) is urgently necessary to explore to construct an efficient universal lesion detection (ULD) segmentation model. An annotated dataset is crucial but hard to acquire because of too many Computed tomography (CT) images and the lack of professionals in computer-aided detection/diagnosis (CADe/CADx). To address this problem, we propose a novel loss function to reduce the proportion of negative anchors which is extremely likely to classify the lesion area (positive samples) as a negative bounding box, further leading to an unexpected performance. Before calculating loss, we generate a mask to intentionally choose fewer negative anchors which will backward wrongful loss to the network. During the process of loss calculation, we set a parameter to reduce the proportion of negative samples, and it significantly reduces the adverse effect of misclassification on the model. Our experiments are implemented in a 3D framework by feeding a partially annotated dataset named DeepLesion, a large-scale public dataset for universal lesion detection from CT. We implement a lot of experiments to choose the most suitable parameter, and the result shows that the proposed method has greatly improved the performance of a ULD detector. Our code can be obtained at https://github.com/PLuld0/PLuldl.

Vessel Delineation Using U-Net: A Sparse Labeled Deep Learning Approach for Semantic Segmentation of Histological Images

Semantic segmentation is an important imaging analysis method enabling the identification of tissue structures. Histological image segmentation is particularly challenging, having large structural information while providing only limited training data. Additionally, labeling these structures to generate training data is time consuming. Here, we demonstrate the feasibility of a semantic segmentation using U-Net with a novel sparse labeling technique. The basic U-Net architecture was extended by attention gates, residual and recurrent links, and dropout regularization. To overcome the high class imbalance, which is intrinsic to histological data, under- and oversampling and data augmentation were used. In an ablation study, various architectures were evaluated, and the best performing model was identified. This model contains attention gates, residual links, and a dropout regularization of 0.125. The segmented images show accurate delineations of the vascular structures (with a precision of 0.9088 and an AUC-ROC score of 0.9717), and the segmentation algorithm is robust to images containing staining variations and damaged tissue. These results demonstrate the feasibility of sparse labeling in combination with the modified U-Net architecture.

Deep learning algorithm for predicting subacromial motion trajectory: Dynamic shoulder ultrasound analysis

Subacromial motion metrics can be extracted from dynamic shoulder ultrasonography, which is useful for identifying abnormal motion patterns in painful shoulders. However, frame-by-frame manual labeling of anatomical landmarks in ultrasound images is time consuming. The present study aims to investigate the feasibility of a deep learning algorithm for extracting subacromial motion metrics from dynamic ultrasonography. Dynamic ultrasound imaging was retrieved by asking 17 participants to perform cyclic shoulder abduction and adduction along the scapular plane, whereby the trajectory of the humeral greater tubercle (in relation to the lateral acromion) was depicted by the deep learning algorithm. Extraction of the subacromial motion metrics was conducted using a convolutional neural network (CNN) or a self-transfer learning-based (STL)-CNN with or without an autoencoder (AE). The mean absolute error (MAE) compared with the manually-labeled data (ground truth) served as the main outcome variable. Using eight-fold cross-validation, the average MAE was proven to be significantly higher in the group using CNN than in those using STL-CNN or STL-CNN+AE for the relative difference between the greater tubercle and lateral acromion on the horizontal axis. The MAE for the localization of the two aforementioned landmarks on the vertical axis also seemed to be enlarged in those using CNN compared with those using STL-CNN. In the testing dataset, the errors in relation to the ground truth for the minimal vertical acromiohumeral distance were 0.081-0.333 cm using CNN, compared with 0.002-0.007 cm using STL-CNN. We successfully demonstrated the feasibility of a deep learning algorithm for automatic detection of the greater tubercle and lateral acromion during dynamic shoulder ultrasonography. Our framework also demonstrated the capability of capturing the minimal vertical acromiohumeral distance, which is the most important indicator of subacromial motion metrics in daily clinical practice.

A feasibility study of enhanced prompt gamma imaging for range verification in proton therapy using deep learning

Background and objective. Range uncertainty is a major concern affecting the delivery precision in proton therapy. The Compton camera (CC)-based prompt-gamma (PG) imaging is a promising technique to provide 3Din vivorange verification. However, the conventional back-projected PG images suffer from severe distortions due to the limited view of the CC, significantly limiting its clinical utility. Deep learning has demonstrated effectiveness in enhancing medical images from limited-view measurements. But different from other medical images with abundant anatomical structures, the PGs emitted along the path of a proton pencil beam take up an extremely low portion of the 3D image space, presenting both the attention and the imbalance challenge for deep learning. To solve these issues, we proposed a two-tier deep learning-based method with a novel weighted axis-projection loss to generate precise 3D PG images to achieve accurate proton range verification.Materials and methods: the proposed method consists of two models: first, a localization model is trained to define a region-of-interest (ROI) in the distorted back-projected PG image that contains the proton pencil beam; second, an enhancement model is trained to restore the true PG emissions with additional attention on the ROI. In this study, we simulated 54 proton pencil beams (energy range: 75-125 MeV, dose level: 1 × 109protons/beam and 3 × 108protons/beam) delivered at clinical dose rates (20 kMU min-1and 180 kMU min-1) in a tissue-equivalent phantom using Monte-Carlo (MC). PG detection with a CC was simulated using the MC-Plus-Detector-Effects model. Images were reconstructed using the kernel-weighted-back-projection algorithm, and were then enhanced by the proposed method.Results. The method effectively restored the 3D shape of the PG images with the proton pencil beam range clearly visible in all testing cases. Range errors were within 2 pixels (4 mm) in all directions in most cases at a higher dose level. The proposed method is fully automatic, and the enhancement takes only ∼0.26 s.Significance. Overall, this preliminary study demonstrated the feasibility of the proposed method to generate accurate 3D PG images using a deep learning framework, providing a powerful tool for high-precisionin vivorange verification of proton therapy.

Image Turing test and its applications on synthetic chest radiographs by using the progressive growing generative adversarial network

The generative adversarial network (GAN) is a promising deep learning method for generating images. We evaluated the generation of highly realistic and high-resolution chest radiographs (CXRs) using progressive growing GAN (PGGAN). We trained two PGGAN models using normal and abnormal CXRs, solely relying on normal CXRs to demonstrate the quality of synthetic CXRs that were 1000 × 1000 pixels in size. Image Turing tests were evaluated by six radiologists in a binary fashion using two independent validation sets to judge the authenticity of each CXR, with a mean accuracy of 67.42% and 69.92% for the first and second trials, respectively. Inter-reader agreements were poor for the first (κ = 0.10) and second (κ = 0.14) Turing tests. Additionally, a convolutional neural network (CNN) was used to classify normal or abnormal CXR using only real images and/or synthetic images mixed datasets. The accuracy of the CNN model trained using a mixed dataset of synthetic and real data was 93.3%, compared to 91.0% for the model built using only the real data. PGGAN was able to generate CXRs that were identical to real CXRs, and this showed promise to overcome imbalances between classes in CNN training.

Convolutional Networks and Transformers for Mammography Classification: An Experimental Study

Convolutional Neural Networks (CNN) have received a large share of research in mammography image analysis due to their capability of extracting hierarchical features directly from raw data. Recently, Vision Transformers are emerging as viable alternative to CNNs in medical imaging, in some cases performing on par or better than their convolutional counterparts. In this work, we conduct an extensive experimental study to compare the most recent CNN and Vision Transformer architectures for whole mammograms classification. We selected, trained and tested 33 different models, 19 convolutional- and 14 transformer-based, on the largest publicly available mammography image database OMI-DB. We also performed an analysis of the performance at eight different image resolutions and considering all the individual lesion categories in isolation (masses, calcifications, focal asymmetries, architectural distortions). Our findings confirm the potential of visual transformers, which performed on par with traditional CNNs like ResNet, but at the same time show a superiority of modern convolutional networks like EfficientNet.

Multiple-level thresholding for breast mass detection

Detection of breast mass plays a very important role in making the diagnosis of breast cancer. For faster detection of breast cancer caused by breast mass, we developed a novel and efficient patch-based breast mass detection system for mammography images. The proposed framework is comprised of three modules, including pre-processing, multiple-level breast tissue segmentation, and final breast mass detection. An improved Deeplabv3+ model for pectoral muscle removal is deployed in pre-processing. We then proposed a multiple-level thresholding segmentation method to segment breast mass and obtained the connected components (ConCs), where the corresponding image patch to each ConC is extracted for mass detection. In the final detection stage, each image patch is classified into breast mass and breast tissue background by trained deep learning models. The patches that are classified as breast mass are then taken as the candidates for breast mass. To reduce the false positive rate in the detection results, we applied the non-maximum suppression algorithm to combine the overlapped detection results. Once an image patch is considered a breast mass, the accurate detection result can then be retrieved from the corresponding ConC in the segmented images. Moreover, a coarse segmentation result can be simultaneously retrieved after detection. Compared to the state-of-the-art methods, the proposed method achieved comparable performance. On CBIS-DDSM, the proposed method achieved a detection sensitivity of 0.87 at 2.86 FPI (False Positive rate per Image), while the sensitivity reached 0.96 on INbreast with an FPI of only 1.29.

A Comparison of Techniques for Class Imbalance in Deep Learning Classification of Breast Cancer

Tools based on deep learning models have been created in recent years to aid radiologists in the diagnosis of breast cancer from mammograms. However, the datasets used to train these models may suffer from class imbalance, i.e., there are often fewer malignant samples than benign or healthy cases, which can bias the model towards the healthy class. In this study, we systematically evaluate several popular techniques to deal with this class imbalance, namely, class weighting, over-sampling, and under-sampling, as well as a synthetic lesion generation approach to increase the number of malignant samples. These techniques are applied when training on three diverse Full-Field Digital Mammography datasets, and tested on in-distribution and out-of-distribution samples. The experiments show that a greater imbalance is associated with a greater bias towards the majority class, which can be counteracted by any of the standard class imbalance techniques. On the other hand, these methods provide no benefit to model performance with respect to Area Under the Curve of the Recall Operating Characteristic (AUC-ROC), and indeed under-sampling leads to a reduction of 0.066 in AUC in the case of a 19:1 benign to malignant imbalance. Our synthetic lesion methodology leads to better performance in most cases, with increases of up to 0.07 in AUC on out-of-distribution test sets over the next best experiment.

Multi-objective data enhancement for deep learning-based ultrasound analysis

Recently, Deep Learning based automatic generation of treatment recommendation has been attracting much attention. However, medical datasets are usually small, which may lead to over-fitting and inferior performances of deep learning models. In this paper, we propose multi-objective data enhancement method to indirectly scale up the medical data to avoid over-fitting and generate high quantity treatment recommendations. Specifically, we define a main and several auxiliary tasks on the same dataset and train a specific model for each of these tasks to learn different aspects of knowledge in limited data scale. Meanwhile, a Soft Parameter Sharing method is exploited to share learned knowledge among models. By sharing the knowledge learned by auxiliary tasks to the main task, the proposed method can take different semantic distributions into account during the training process of the main task. We collected an ultrasound dataset of thyroid nodules that contains Findings, Impressions and Treatment Recommendations labeled by professional doctors. We conducted various experiments on the dataset to validate the proposed method and justified its better performance than existing methods.

Recent developments on computer aided systems for diagnosis of diabetic retinopathy: a review

Diabetes is a long-term condition in which the pancreas quits producing insulin or the body's insulin isn't utilised properly. One of the signs of diabetes is Diabetic Retinopathy. Diabetic retinopathy is the most prevalent type of diabetes, if remains unaddressed, diabetic retinopathy can affect all diabetics and become very serious, raising the chances of blindness. It is a chronic systemic condition that affects up to 80% of patients for more than ten years. Many researchers believe that if diabetes individuals are diagnosed early enough, they can be rescued from the condition in 90% of cases. Diabetes damages the capillaries, which are microscopic blood vessels in the retina. On images, blood vessel damage is usually noticeable. Therefore, in this study, several traditional, as well as deep learning-based approaches, are reviewed for the classification and detection of this particular diabetic-based eye disease known as diabetic retinopathy, and also the advantage of one approach over the other is also described. Along with the approaches, the dataset and the evaluation metrics useful for DR detection and classification are also discussed. The main finding of this study is to aware researchers about the different challenges occurs while detecting diabetic retinopathy using computer vision, deep learning techniques. Therefore, a purpose of this review paper is to sum up all the major aspects while detecting DR like lesion identification, classification and segmentation, security attacks on the deep learning models, proper categorization of datasets and evaluation metrics. As deep learning models are quite expensive and more prone to security attacks thus, in future it is advisable to develop a refined, reliable and robust model which overcomes all these aspects which are commonly found while designing deep learning models.

Image segmentation using transfer learning and Fast R-CNN for diabetic foot wound treatments

Diabetic foot ulcers (DFUs) are considered the most challenging forms of chronic ulcerations to handle their multifactorial nature. It is necessary to establish a comprehensive treatment plan, accurate, and systematic evaluation of a patient with a DFU. This paper proposed an image recognition of diabetic foot wounds to support the effective execution of the treatment plan. In the severity of a diabetic foot ulcer, we refer to the current qualitative evaluation method commonly used in clinical practice, developed by the International Working Group on the Diabetic Foot: PEDIS index, and the evaluation made by physicians. The deep neural network, convolutional neural network, object recognition, and other technologies are applied to analyze the classification, location, and size of wounds by image analysis technology. The image features are labeled with the help of the physician. The Object Detection Fast R-CNN method is applied to these wound images to build and train machine learning modules and evaluate their effectiveness. In the assessment accuracy, it can be indicated that the wound image detection data can be as high as 90%.

Impact of the Volume and Distribution of Training Datasets in the Development of Deep-Learning Models for the Diagnosis of Colorectal Polyps in Endoscopy Images

Background: Establishment of an artificial intelligence model in gastrointestinal endoscopy has no standardized dataset. The optimal volume or class distribution of training datasets has not been evaluated. An artificial intelligence model was previously created by the authors to classify endoscopic images of colorectal polyps into four categories, including advanced colorectal cancer, early cancers/high-grade dysplasia, tubular adenoma, and nonneoplasm. The aim of this study was to evaluate the impact of the volume and distribution of training dataset classes in the development of deep-learning models for colorectal polyp histopathology prediction from endoscopic images. Methods: The same 3828 endoscopic images that were used to create earlier models were used. An additional 6838 images were used to find the optimal volume and class distribution for a deep-learning model. Various amounts of data volume and class distributions were tried to establish deep-learning models. The training of deep-learning models uniformly used no-code platform Neuro-T. Accuracy was the primary outcome on four-class prediction. Results: The highest internal-test classification accuracy in the original dataset, doubled dataset, and tripled dataset was commonly shown by doubling the proportion of data for fewer categories (2:2:1:1 for advanced colorectal cancer: early cancers/high-grade dysplasia: tubular adenoma: non-neoplasm). Doubling the proportion of data for fewer categories in the original dataset showed the highest accuracy (86.4%, 95% confidence interval: 85.0–97.8%) compared to that of the doubled or tripled dataset. The total required number of images in this performance was only 2418 images. Gradient-weighted class activation mapping confirmed that the part that the deep-learning model pays attention to coincides with the part that the endoscopist pays attention to. Conclusion: As a result of a data-volume-dependent performance plateau in the classification model of colonoscopy, a dataset that has been doubled or tripled is not always beneficial to training. Deep-learning models would be more accurate if the proportion of fewer category lesions was increased.

A Priori Determining the Performance of the Customized Naïve Associative Classifier for Business Data Classification Based on Data Complexity Measures

In the supervised classification area, the algorithm selection problem (ASP) refers to determining the a priori performance of a given classifier in some specific problem, as well as the finding of which is the most suitable classifier for some tasks. Recently, this topic has attracted the attention of international research groups because a very promising vein of research has emerged: the application of some measures of data complexity in the pattern classification algorithms. This paper aims to analyze the response of the Customized Naïve Associative Classifier (CNAC) in data taken from the business area when some measures of data complexity are introduced. To perform this analysis, we used classification datasets from real-world related to business, 22 in total; then, we computed the value of nine measures of data complexity to compare the performance of the CNAC against other algorithms of the state of the art. A very important aspect of performing this task is the creation of an artificial dataset for meta-learning purposes, in which we considered the performance of CNAC, and then we trained a decision tree as meta learner. As shown, the CNAC classifier obtained the best results for 10 out of 22 datasets of the experimental study.

Application of deep learning methods: From molecular modelling to patient classification

We are now well into the information driven age with complex, heterogeneous, datasets in the biological sciences continuing to grow at a rapid pace. Moreover, distilling of such datasets, to find new governing principles, are underway. Leading the surge are new and exciting algorithmic developments in computer simulation and machine learning, most notably for the latter, those centred on deep learning. However, practical applications of cell centric computations within the biological sciences, even when carefully benchmarked against existing experimental datasets, remain challenging. Here we discuss the application of deep learning methodologies to support our understanding of cell functionality and as an aid to patient classification. Whilst comprehensive end-to-end deep learning approaches that utilise knowledge of the cell and its molecular components to aid human disease classification are yet to be implemented, important for opening the door to more effective molecular and cell-based therapies, we illustrate that many deep learning applications have been developed to tackle components of such an ambitious pipeline. We end our discussion on what the future may hold, especially how an integrated framework of computer simulations and deep learning, in conjunction with wet-bench experimentation, could enable to reveal the governing principles underlying cell functionalities within the tissue environments cells operate.

Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency

The interest in machine learning (ML) has grown tremendously in recent years, partly due to the performance leap that occurred with new techniques of deep learning, convolutional neural networks for images, increased computational power, and wider availability of large datasets. Most fields of medicine follow that popular trend and, notably, radiation oncology is one of those that are at the forefront, with already a long tradition in using digital images and fully computerized workflows. ML models are driven by data, and in contrast with many statistical or physical models, they can be very large and complex, with countless generic parameters. This inevitably raises two questions, namely, the tight dependence between the models and the datasets that feed them, and the interpretability of the models, which scales with its complexity. Any problems in the data used to train the model will be later reflected in their performance. This, together with the low interpretability of ML models, makes their implementation into the clinical workflow particularly difficult. Building tools for risk assessment and quality assurance of ML models must involve then two main points: interpretability and data-model dependency. After a joint introduction of both radiation oncology and ML, this paper reviews the main risks and current solutions when applying the latter to workflows in the former. Risks associated with data and models, as well as their interaction, are detailed. Next, the core concepts of interpretability, explainability, and data-model dependency are formally defined and illustrated with examples. Afterwards, a broad discussion goes through key applications of ML in workflows of radiation oncology as well as vendors' perspectives for the clinical implementation of ML.

Deep Learning Using CT Images to Grade Clear Cell Renal Cell Carcinoma: Development and Validation of a Prediction Model

Simple Summary

Clear cell renal cell carcinoma (ccRCC) pathologic grade identification is essential to both monitoring patients’ conditions and constructing individualized subsequent treatment strategies. However, biopsies are typically used to obtain the pathological grade, entailing tremendous physical and mental suffering as well as heavy economic burden, not to mention the increased risk of complications. Our study explores a new way to provide grade assessment of ccRCC on the basis of the individual’s appearance on CT images. A deep learning (DL) method that includes self-supervised learning is constructed to identify patients with high grade for ccRCC. We confirmed that our grading network can accurately differentiate between different grades of CT scans of ccRCC patients using a cohort of 706 patients from West China Hospital. The promising diagnostic performance indicates that our DL framework is an effective, non-invasive and labor-saving method for decoding CT images, offering a valuable means for ccRCC grade stratification and individualized patient treatment.

Abstract

This retrospective study aimed to develop and validate deep-learning-based models for grading clear cell renal cell carcinoma (ccRCC) patients. A cohort enrolling 706 patients (n = 706) with pathologically verified ccRCC was used in this study. A temporal split was applied to verify our models: the first 83.9% of the cases (years 2010–2017) for development and the last 16.1% (year 2018–2019) for validation (development cohort: n = 592; validation cohort: n = 114). Here, we demonstrated a deep learning(DL) framework initialized by a self-supervised pre-training method, developed with the addition of mixed loss strategy and sample reweighting to identify patients with high grade for ccRCC. Four types of DL networks were developed separately and further combined with different weights for better prediction. The single DL model achieved up to an area under curve (AUC) of 0.864 in the validation cohort, while the ensembled model yielded the best predictive performance with an AUC of 0.882. These findings confirms that our DL approach performs either favorably or comparably in terms of grade assessment of ccRCC with biopsies whilst enjoying the non-invasive and labor-saving property.

State-of-the-art retinal vessel segmentation with minimalistic models

The segmentation of retinal vasculature from eye fundus images is a fundamental task in retinal image analysis. Over recent years, increasingly complex approaches based on sophisticated Convolutional Neural Network architectures have been pushing performance on well-established benchmark datasets. In this paper, we take a step back and analyze the real need of such complexity. We first compile and review the performance of 20 different techniques on some popular databases, and we demonstrate that a minimalistic version of a standard U-Net with several orders of magnitude less parameters, carefully trained and rigorously evaluated, closely approximates the performance of current best techniques. We then show that a cascaded extension (W-Net) reaches outstanding performance on several popular datasets, still using orders of magnitude less learnable weights than any previously published work. Furthermore, we provide the most comprehensive cross-dataset performance analysis to date, involving up to 10 different databases. Our analysis demonstrates that the retinal vessel segmentation is far from solved when considering test images that differ substantially from the training data, and that this task represents an ideal scenario for the exploration of domain adaptation techniques. In this context, we experiment with a simple self-labeling strategy that enables moderate enhancement of cross-dataset performance, indicating that there is still much room for improvement in this area. Finally, we test our approach on Artery/Vein and vessel segmentation from OCTA imaging problems, where we again achieve results well-aligned with the state-of-the-art, at a fraction of the model complexity available in recent literature. Code to reproduce the results in this paper is released.

MSAL-Net: improve accurate segmentation of nuclei in histopathology images by multiscale attention learning network

BACKGROUND

The digital pathology images obtain the essential information about the patient's disease, and the automated nuclei segmentation results can help doctors make better decisions about diagnosing the disease. With the speedy advancement of convolutional neural networks in image processing, deep learning has been shown to play a significant role in the various analysis of medical images, such as nuclei segmentation, mitosis detection and segmentation etc. Recently, several U-net based methods have been developed to solve the automated nuclei segmentation problems. However, these methods fail to deal with the weak features representation from the initial layers and introduce the noise into the decoder path. In this paper, we propose a multiscale attention learning network (MSAL-Net), where the dense dilated convolutions block captures more comprehensive nuclei context information, and a newly modified decoder part is introduced, which integrates with efficient channel attention and boundary refinement modules to effectively learn spatial information for better prediction and further refine the nuclei cell of boundaries.

RESULTS

Both qualitative and quantitative results are obtained on the publicly available MoNuseg dataset. Extensive experiment results verify that our proposed method significantly outperforms state-of-the-art methods as well as the vanilla Unet method in the segmentation task. Furthermore, we visually demonstrate the effect of our modified decoder part.

CONCLUSION

The MSAL-Net shows superiority with a novel decoder to segment the touching and blurred background nuclei cells obtained from histopathology images with better performance for accurate decoding.

A Review on Computer Aided Diagnosis of Acute Brain Stroke

Amongst the most common causes of death globally, stroke is one of top three affecting over 100 million people worldwide annually. There are two classes of stroke, namely ischemic stroke (due to impairment of blood supply, accounting for ~70% of all strokes) and hemorrhagic stroke (due to bleeding), both of which can result, if untreated, in permanently damaged brain tissue. The discovery that the affected brain tissue (i.e., 'ischemic penumbra') can be salvaged from permanent damage and the bourgeoning growth in computer aided diagnosis has led to major advances in stroke management. Abiding to the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines, we have surveyed a total of 177 research papers published between 2010 and 2021 to highlight the current status and challenges faced by computer aided diagnosis (CAD), machine learning (ML) and deep learning (DL) based techniques for CT and MRI as prime modalities for stroke detection and lesion region segmentation. This work concludes by showcasing the current requirement of this domain, the preferred modality, and prospective research areas.

Study on Data Partition for Delimitation of Masses in Mammography

Mammography is the primary medical imaging method used for routine screening and early detection of breast cancer in women. However, the process of manually inspecting, detecting, and delimiting the tumoral massess in 2D images is a very time-consuming task, subject to human errors due to fatigue. Therefore, integrated computer-aided detection systems have been proposed, based on modern computer vision and machine learning methods. In the present work, mammogram images from the publicly available Inbreast dataset are first converted to pseudo-color and then used to train and test a Mask R-CNN deep neural network. The most common approach is to start with a dataset and split the images into train and test set randomly. However, since there are often two or more images of the same case in the dataset, the way the dataset is split may have an impact on the results. Our experiments show that random partition of the data can produce unreliable training, so the dataset must be split using case-wise partition for more stable results. In experimental results, the method achieves an average true positive rate of 0.936 with 0.063 standard deviation using random partition and 0.908 with 0.002 standard deviation using case-wise partition, showing that case-wise partition must be used for more reliable results.

The use of artificial intelligence for the diagnosis of bladder cancer: a review and perspectives

PURPOSE OF REVIEW

White light cystoscopy is the current standard for primary diagnosis and surveillance of bladder cancer. However, cancer changes can be subtle and may be easily missed. With the advancement of deep learning (DL), image recognition by artificial intelligence (AI) proves a high accuracy for image-based diagnosis. AI can be a solution to enhance bladder cancer diagnosis on cystoscopy.

RECENT FINDINGS

An algorithm that classifies cystoscopic images into normal and tumour images is essential for AI cystoscopy. To develop this AI-based system requires a training dataset, an appropriate type of DL algorithm for the learning process and a specific outcome classification. A large data volume with minimal class imbalance, data accuracy and representativeness are pre-requisite for a good dataset. Algorithms developed during the past two years to detect bladder tumour achieved high performance with a pooled sensitivity of 89.7% and specificity of 96.1%. The area under the curve ranged from 0.960 to 0.980, and the accuracy ranged from 85.6 to 96.9%. There were also favourable results in the various attempts to enhance detection of flat lesions or carcinoma-in-situ.

SUMMARY

AI cystoscopy is a possible solution in clinical practice to enhance bladder cancer diagnosis, improve tumour clearance during transurethral resection of bladder tumour and detect recurrent tumours upon surveillance.

Deep learning-based automated detection for diabetic retinopathy and diabetic macular oedema in retinal fundus photographs

OBJECTIVES

To present and validate a deep ensemble algorithm to detect diabetic retinopathy (DR) and diabetic macular oedema (DMO) using retinal fundus images.

METHODS

A total of 8739 retinal fundus images were collected from a retrospective cohort of 3285 patients. For detecting DR and DMO, a multiple improved Inception-v4 ensembling approach was developed. We measured the algorithm's performance and made a comparison with that of human experts on our primary dataset, while its generalization was assessed on the publicly available Messidor-2 dataset. Also, we investigated systematically the impact of the size and number of input images used in training on model's performance, respectively. Further, the time budget of training/inference versus model performance was analyzed.

RESULTS

On our primary test dataset, the model achieved an 0.992 (95% CI, 0.989-0.995) AUC corresponding to 0.925 (95% CI, 0.916-0.936) sensitivity and 0.961 (95% CI, 0.950-0.972) specificity for referable DR, while the sensitivity and specificity for ophthalmologists ranged from 0.845 to 0.936, and from 0.912 to 0.971, respectively. For referable DMO, our model generated an AUC of 0.994 (95% CI, 0.992-0.996) with a 0.930 (95% CI, 0.919-0.941) sensitivity and 0.971 (95% CI, 0.965-0.978) specificity, whereas ophthalmologists obtained sensitivities ranging between 0.852 and 0.946, and specificities ranging between 0.926 and 0.985.

CONCLUSION

This study showed that the deep ensemble model exhibited excellent performance in detecting DR and DMO, and had good robustness and generalization, which could potentially help support and expand DR/DMO screening programs.

MDC-net: A new convolutional neural network for nucleus segmentation in histopathology images with distance maps and contour information

Accurate segmentation of nuclei in digital pathology images can assist doctors in diagnosing diseases and evaluating subsequent treatments. Manual segmentation of nuclei from pathology images is time-consuming because of the large number of nuclei and is also error-prone. Therefore, accurate and automatic nucleus segmentation methods are required. Owing to the large variations in the characterization of nuclei, it is difficult to accurately segment nuclei using traditional methods. In this study, we propose a new method for nucleus segmentation. The proposed method uses a deep fully convolutional neural network to perform end-to-end segmentation on pathological tissue slices. Multiple short residual connections were used to fuse feature maps from different scales to better utilize the context information. Dilated convolutions with different dilation ratios were used to increase the receptive fields. In addition, we incorporated the distance map and contour information into the segmentation method to segment touching nuclei, which is difficult via traditional segmentation methods. Finally, post-processing was used to improve the segmentation results. The results demonstrate that our segmentation method can obtain comparable or better performance than other state-of-the-art methods on the public nuclei histopathology datasets.

Leveraging the Bhattacharyya coefficient for uncertainty quantification in deep neural networks

Modern deep learning models achieve state-of-the-art results for many tasks in computer vision, such as image classification and segmentation. However, its adoption into high-risk applications, e.g. automated medical diagnosis systems, happens at a slow pace. One of the main reasons for this is that regular neural networks do not capture uncertainty. To assess uncertainty in classification, several techniques have been proposed casting neural network approaches in a Bayesian setting. Amongst these techniques, Monte Carlo dropout is by far the most popular. This particular technique estimates the moments of the output distribution through sampling with different dropout masks. The output uncertainty of a neural network is then approximated as the sample variance. In this paper, we highlight the limitations of such a variance-based uncertainty metric and propose an novel approach. Our approach is based on the overlap between output distributions of different classes. We show that our technique leads to a better approximation of the inter-class output confusion. We illustrate the advantages of our method using benchmark datasets. In addition, we apply our metric to skin lesion classification—a real-world use case—and show that this yields promising results.

Convolutional neural networks for breast cancer detection in mammography: A survey

Despite its proven record as a breast cancer screening tool, mammography remains labor-intensive and has recognized limitations, including low sensitivity in women with dense breast tissue. In the last ten years, Neural Network advances have been applied to mammography to help radiologists increase their efficiency and accuracy. This survey aims to present, in an organized and structured manner, the current knowledge base of convolutional neural networks (CNNs) in mammography. The survey first discusses traditional Computer Assisted Detection (CAD) and more recently developed CNN-based models for computer vision in mammography. It then presents and discusses the literature on available mammography training datasets. The survey then presents and discusses current literature on CNNs for four distinct mammography tasks: (1) breast density classification, (2) breast asymmetry detection and classification, (3) calcification detection and classification, and (4) mass detection and classification, including presenting and comparing the reported quantitative results for each task and the pros and cons of the different CNN-based approaches. Then, it offers real-world applications of CNN CAD algorithms by discussing current Food and Drug Administration (FDA) approved models. Finally, this survey highlights the potential opportunities for future work in this field. The material presented and discussed in this survey could serve as a road map for developing CNN-based solutions to improve mammographic detection of breast cancer further.