Medical image segmentation automatic quality control: A multi-dimensional approach

Inline automatic quality control of 2D phase-contrast flow MRI for subject-specific scan time adaptation

PURPOSE

To develop an inline automatic quality control to achieve consistent diagnostic image quality with subject-specific scan time, and to demonstrate this method for 2D phase-contrast flow MRI to reach a predetermined SNR.

METHODS

We designed a closed-loop feedback framework between image reconstruction and data acquisition to intermittently check SNR (every 20 s) and automatically stop the acquisition when a target SNR is achieved. A free-breathing 2D pseudo-golden-angle spiral phase-contrast sequence was modified to listen for image-quality messages from the reconstructions. Ten healthy volunteers and 1 patient were imaged at 0.55 T. Target SNR was selected based on retrospective analysis of cardiac output error, and performance of the automatic SNR-driven "stop" was assessed inline.

RESULTS

SNR calculation and automated segmentation was feasible within 20 s with inline deployment. The SNR-driven acquisition time was 2 min 39 s ± 67 s (aorta) and 3 min ± 80 s (main pulmonary artery) with a min/max acquisition time of 1 min 43 s/4 min 52 s (aorta) and 1 min 43 s/5 min 50 s (main pulmonary artery) across 6 healthy volunteers, while ensuring a diagnostic measurement with relative absolute error in quantitative flow measurement lower than 2.1% (aorta) and 6.3% (main pulmonary artery).

CONCLUSION

The inline quality control enables subject-specific optimized scan times while ensuring consistent diagnostic image quality. The distribution of automated stopping times across the population revealed the value of a subject-specific scan time.

Deep learning-based hyperspectral technique identifies metastatic lymph nodes in oral squamous cell carcinoma-A pilot study

AIMS

To establish a system based on hyperspectral imaging and deep learning for the detection of cancer cells in metastatic lymph nodes.

MAIN METHODS

The continuous sections of metastatic lymph nodes from 45 oral squamous cell carcinoma (OSCC) patients were collected. An improved ResUNet algorithm was established for deep learning to analyze the spectral curve differences between cancer cells and lymphocytes, and that between tumor tissue and normal tissue.

KEY FINDINGS

It was found that cancer cells, lymphocytes, and erythrocytes in the metastatic lymph nodes could be distinguished basing hyperspectral image, with overall accuracy (OA) as 87.30% and average accuracy (AA) as 85.46%. Cancerous area could be recognized by hyperspectral image and deep learning, and the average intersection over union (IOU) and accuracy were 0.6253 and 0.7692, respectively.

SIGNIFICANCE

This study indicated that deep learning-based hyperspectral techniques can identify tumor tissue in OSCC metastatic lymph nodes, achieving high accuracy of pathological diagnosis, high work efficiency, and reducing work burden. But these are preliminary results limited to a small sample.

Improving the efficiency and accuracy of cardiovascular magnetic resonance with artificial intelligence-review of evidence and proposition of a roadmap to clinical translation

BACKGROUND

Cardiovascular magnetic resonance (CMR) is an important imaging modality for the assessment of heart disease; however, limitations of CMR include long exam times and high complexity compared to other cardiac imaging modalities. Recently advancements in artificial intelligence (AI) technology have shown great potential to address many CMR limitations. While the developments are remarkable, translation of AI-based methods into real-world CMR clinical practice remains at a nascent stage and much work lies ahead to realize the full potential of AI for CMR.

METHODS

Herein we review recent cutting-edge and representative examples demonstrating how AI can advance CMR in areas such as exam planning, accelerated image reconstruction, post-processing, quality control, classification and diagnosis.

RESULTS

These advances can be applied to speed up and simplify essentially every application including cine, strain, late gadolinium enhancement, parametric mapping, 3D whole heart, flow, perfusion and others. AI is a unique technology based on training models using data. Beyond reviewing the literature, this paper discusses important AI-specific issues in the context of CMR, including (1) properties and characteristics of datasets for training and validation, (2) previously published guidelines for reporting CMR AI research, (3) considerations around clinical deployment, (4) responsibilities of clinicians and the need for multi-disciplinary teams in the development and deployment of AI in CMR, (5) industry considerations, and (6) regulatory perspectives.

CONCLUSIONS

Understanding and consideration of all these factors will contribute to the effective and ethical deployment of AI to improve clinical CMR.

Left Ventricular Trabeculations at Cardiac MRI: Reference Ranges and Association with Cardiovascular Risk Factors in UK Biobank

Background The extent of left ventricular (LV) trabeculation and its relationship with cardiovascular (CV) risk factors is unclear. Purpose To apply automated segmentation to UK Biobank cardiac MRI scans to (a) assess the association between individual characteristics and CV risk factors and trabeculated LV mass (LVM) and (b) establish normal reference ranges in a selected group of healthy UK Biobank participants. Materials and Methods In this cross-sectional secondary analysis, prospectively collected data from the UK Biobank (2006 to 2010) were retrospectively analyzed. Automated segmentation of trabeculations was performed using a deep learning algorithm. After excluding individuals with known CV diseases, White adults without CV risk factors (reference group) and those with preexisting CV risk factors (hypertension, hyperlipidemia, diabetes mellitus, or smoking) (exposed group) were compared. Multivariable regression models, adjusted for potential confounders (age, sex, and height), were fitted to evaluate the associations between individual characteristics and CV risk factors and trabeculated LVM. Results Of 43 038 participants (mean age, 64 years ± 8 [SD]; 22 360 women), 28 672 individuals (mean age, 66 years ± 7; 14 918 men) were included in the exposed group, and 7384 individuals (mean age, 60 years ± 7; 4729 women) were included in the reference group. Higher body mass index (BMI) (β = 0.66 [95% CI: 0.63, 0.68]; P < .001), hypertension (β = 0.42 [95% CI: 0.36, 0.48]; P < .001), and higher physical activity level (β = 0.15 [95% CI: 0.12, 0.17]; P < .001) were associated with higher trabeculated LVM. In the reference group, the median trabeculated LVM was 6.3 g (IQR, 4.7-8.5 g) for men and 4.6 g (IQR, 3.4-6.0 g) for women. Median trabeculated LVM decreased with age for men from 6.5 g (IQR, 4.8-8.7 g) at age 45-50 years to 5.9 g (IQR, 4.3-7.8 g) at age 71-80 years (P = .03). Conclusion Higher trabeculated LVM was observed with hypertension, higher BMI, and higher physical activity level. Age- and sex-specific reference ranges of trabeculated LVM in a healthy middle-aged White population were established. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Kawel-Boehm in this issue.

A Deep Learning-Based Integrated Framework for Quality-Aware Undersampled Cine Cardiac MRI Reconstruction and Analysis

Cine cardiac magnetic resonance (CMR) imaging is considered the gold standard for cardiac function evaluation. However, cine CMR acquisition is inherently slow and in recent decades considerable effort has been put into accelerating scan times without compromising image quality or the accuracy of derived results. In this article, we present a fully-automated, quality-controlled integrated framework for reconstruction, segmentation and downstream analysis of undersampled cine CMR data. The framework produces high quality reconstructions and segmentations, leading to undersampling factors that are optimised on a scan-by-scan basis. This results in reduced scan times and automated analysis, enabling robust and accurate estimation of functional biomarkers. To demonstrate the feasibility of the proposed approach, we perform simulations of radial k-space acquisitions using in-vivo cine CMR data from 270 subjects from the UK Biobank (with synthetic phase) and in-vivo cine CMR data from 16 healthy subjects (with real phase). The results demonstrate that the optimal undersampling factor varies for different subjects by approximately 1 to 2 seconds per slice. We show that our method can produce quality-controlled images in a mean scan time reduced from 12 to 4 seconds per slice, and that image quality is sufficient to allow clinically relevant parameters to be automatically estimated to lie within 5% mean absolute difference.

FD-Net: Feature Distillation Network for Oral Squamous Cell Carcinoma Lymph Node Segmentation in Hyperspectral Imagery

Oral squamous cell carcinoma (OSCC) has the characteristics of early regional lymph node metastasis. OSCC patients often have poor prognoses and low survival rates due to cervical lymph metastases. Therefore, it is necessary to rely on a reasonable screening method to quickly judge the cervical lymph metastastic condition of OSCC patients and develop appropriate treatment plans. In this study, the widely used pathological sections with hematoxylin-eosin (H&E) staining are taken as the target, and combined with the advantages of hyperspectral imaging technology, a novel diagnostic method for identifying OSCC lymph node metastases is proposed. The method consists of a learning stage and a decision-making stage, focusing on cancer and non-cancer nuclei, gradually completing the lesions' segmentation from coarse to fine, and achieving high accuracy. In the learning stage, the proposed feature distillation-Net (FD-Net) network is developed to segment the cancerous and non-cancerous nuclei. In the decision-making stage, the segmentation results are post-processed, and the lesions are effectively distinguished based on the prior. Experimental results demonstrate that the proposed FD-Net is very competitive in the OSCC hyperspectral medical image segmentation task. The proposed FD-Net method performs best on the seven segmentation evaluation indicators: MIoU, OA, AA, SE, CSI, GDR, and DICE. Among these seven evaluation indicators, the proposed FD-Net method is 1.75%, 1.27%, 0.35%, 1.9%, 0.88%, 4.45%, and 1.98% higher than the DeepLab V3 method, which ranks second in performance, respectively. In addition, the proposed diagnosis method of OSCC lymph node metastasis can effectively assist pathologists in disease screening and reduce the workload of pathologists.

Deep learning in mesoscale brain image analysis: A review

Mesoscale microscopy images of the brain contain a wealth of information which can help us understand the working mechanisms of the brain. However, it is a challenging task to process and analyze these data because of the large size of the images, their high noise levels, the complex morphology of the brain from the cellular to the regional and anatomical levels, the inhomogeneous distribution of fluorescent labels in the cells and tissues, and imaging artifacts. Due to their impressive ability to extract relevant information from images, deep learning algorithms are widely applied to microscopy images of the brain to address these challenges and they perform superiorly in a wide range of microscopy image processing and analysis tasks. This article reviews the applications of deep learning algorithms in brain mesoscale microscopy image processing and analysis, including image synthesis, image segmentation, object detection, and neuron reconstruction and analysis. We also discuss the difficulties of each task and possible directions for further research.

Interobserver Agreement in Automatic Segmentation Annotation of Prostate Magnetic Resonance Imaging

We aimed to compare the performance and interobserver agreement of radiologists manually segmenting images or those assisted by automatic segmentation. We further aimed to reduce interobserver variability and improve the consistency of radiomics features. This retrospective study included 327 patients diagnosed with prostate cancer from September 2016 to June 2018; images from 228 patients were used for automatic segmentation construction, and images from the remaining 99 were used for testing. First, four radiologists with varying experience levels retrospectively segmented 99 axial prostate images manually using T2-weighted fat-suppressed magnetic resonance imaging. Automatic segmentation was performed after 2 weeks. The Pyradiomics software package v3.1.0 was used to extract the texture features. The Dice coefficient and intraclass correlation coefficient (ICC) were used to evaluate segmentation performance and the interobserver consistency of prostate radiomics. The Wilcoxon rank sum test was used to compare the paired samples, with the significance level set at p < 0.05. The Dice coefficient was used to accurately measure the spatial overlap of manually delineated images. In all the 99 prostate segmentation result columns, the manual and automatic segmentation results of the senior group were significantly better than those of the junior group (p < 0.05). Automatic segmentation was more consistent than manual segmentation (p < 0.05), and the average ICC reached >0.85. The automatic segmentation annotation performance of junior radiologists was similar to that of senior radiologists performing manual segmentation. The ICC of radiomics features increased to excellent consistency (0.925 [0.888~0.950]). Automatic segmentation annotation provided better results than manual segmentation by radiologists. Our findings indicate that automatic segmentation annotation helps reduce variability in the perception and interpretation between radiologists with different experience levels and ensures the stability of radiomics features.

An artificial intelligence tool for automated analysis of large-scale unstructured clinical cine cardiac magnetic resonance databases

Aims

Artificial intelligence (AI) techniques have been proposed for automating analysis of short-axis (SAX) cine cardiac magnetic resonance (CMR), but no CMR analysis tool exists to automatically analyse large (unstructured) clinical CMR datasets. We develop and validate a robust AI tool for start-to-end automatic quantification of cardiac function from SAX cine CMR in large clinical databases.

Methods and results

Our pipeline for processing and analysing CMR databases includes automated steps to identify the correct data, robust image pre-processing, an AI algorithm for biventricular segmentation of SAX CMR and estimation of functional biomarkers, and automated post-analysis quality control to detect and correct errors. The segmentation algorithm was trained on 2793 CMR scans from two NHS hospitals and validated on additional cases from this dataset (n = 414) and five external datasets (n = 6888), including scans of patients with a range of diseases acquired at 12 different centres using CMR scanners from all major vendors. Median absolute errors in cardiac biomarkers were within the range of inter-observer variability: <8.4 mL (left ventricle volume), <9.2 mL (right ventricle volume), <13.3 g (left ventricular mass), and <5.9% (ejection fraction) across all datasets. Stratification of cases according to phenotypes of cardiac disease and scanner vendors showed good performance across all groups.

Conclusion

We show that our proposed tool, which combines image pre-processing steps, a domain-generalizable AI algorithm trained on a large-scale multi-domain CMR dataset and quality control steps, allows robust analysis of (clinical or research) databases from multiple centres, vendors, and cardiac diseases. This enables translation of our tool for use in fully automated processing of large multi-centre databases.

Radiomics-Based Quality Control System for Automatic Cardiac Segmentation: A Feasibility Study

PURPOSE

In the past decade, there has been a rapid increase in the development of automatic cardiac segmentation methods. However, the automatic quality control (QC) of these segmentation methods has received less attention. This study aims to address this gap by developing an automatic pipeline that incorporates DL-based cardiac segmentation and radiomics-based quality control.

METHODS

In the DL-based localization and segmentation part, the entire heart was first located and cropped. Then, the cropped images were further utilized for the segmentation of the right ventricle cavity (RVC), myocardium (MYO), and left ventricle cavity (LVC). As for the radiomics-based QC part, a training radiomics dataset was created with segmentation tasks of various quality. This dataset was used for feature extraction, selection, and QC model development. The model performance was then evaluated using both internal and external testing datasets.

RESULTS

In the internal testing dataset, the segmentation model demonstrated a great performance with a dice similarity coefficient (DSC) of 0.954 for whole heart segmentations. Images were then appropriately cropped to 160 × 160 pixels. The models also performed well for cardiac substructure segmentations. The DSC values were 0.863, 0.872, and 0.940 for RVC, MYO, and LVC for 2D masks and 0.928, 0.886, and 0.962 for RVC, MYO, and LVC for 3D masks with an attention-UNet. After feature selection with the radiomics dataset, we developed a series of models to predict the automatic segmentation quality and its DSC value for the RVC, MYO, and LVC structures. The mean absolute values for our best prediction models were 0.060, 0.032, and 0.021 for 2D segmentations and 0.027, 0.017, and 0.011 for 3D segmentations, respectively. Additionally, the radiomics-based classification models demonstrated a high negative detection rate of >0.85 in all 2D groups. In the external dataset, models showed similar results.

CONCLUSIONS

We developed a pipeline including cardiac substructure segmentation and QC at both the slice (2D) and subject (3D) levels. Our results demonstrate that the radiomics method possesses great potential for the automatic QC of cardiac segmentation.

Cross-convolutional transformer for automated multi-organs segmentation in a variety of medical images

Objective.It is a huge challenge for multi-organs segmentation in various medical images based on a consistent algorithm with the development of deep learning methods. We therefore develop a deep learning method based on cross-convolutional transformer for these automated- segmentation to obtain better generalization and accuracy.Approach.We propose a cross-convolutional transformer network (C²Former) to solve the segmentation problem. Specifically, we first redesign a novel cross-convolutional self-attention mechanism in terms of the algorithm to integrate local and global contexts and model long-distance and short-distance dependencies to enhance the semantic feature understanding of images. Then multi-scale feature edge fusion module is proposed to combine the image edge features, which effectively form multi-scale feature streams and establish reliable relational connections in the global context. Finally, we use three different modalities, imaging three different anatomical regions to train and test multi organs and evaluate segmentation performance.Main results.We use the evaluation metrics of Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD95) for each dataset. Experiments showed the average DSC of 83.22% and HD95 of 17.55 mm on the Synapse dataset (CT images of abdominal multi-organ), the average DSC of 91.42% and HD95 of 1.06 mm on the ACDC dataset (MRI of cardiac substructures) and the average DSC of 86.78% and HD95 of 16.85 mm on the ISIC 2017 dataset (skin cancer images). In each dataset, our proposed method consistently outperforms the compared networks.Significance.The proposed deep learning network provides a generalized and accurate solution method for multi-organ segmentation in the three different datasets. It has the potential to be applied to a variety of medical datasets for structural segmentation.

Cerebral Angiography under Artificial Intelligence Algorithm in the Design of Nursing Cooperation Plan for Intracranial Aneurysm Patients in Craniotomy Clipping

This research was to investigate the value of indocyanine green angiography (ICGA) based on maximum interclass variance (Otsu) method in the nursing plan of intracranial aneurysm clipping (ICAC) for intracranial aneurysm patients. An Otsu algorithm was selected to optimize the original images with the optimal threshold. In addition, the algorithm was applied to ICGA images of 86 patients with intracranial aneurysms, who were randomly divided into an experimental group (using ICGA + ICAC+ perioperative nursing) and a control group (ICAC + conventional nursing), to observe the clinical indicators, treatment, complications, nursing satisfaction, and quality of life of patients in two groups. The results showed that the mean square error (MSE), structural similarity (SSIM), and shape error (SE) were 3.71, 0.84, and 0.47, respectively. The length of hospital stay in the experimental group (19.9 ± 3.5 days) was significantly shorter than that in the control group (23.2 ± 3.0 days), the rate of excellent treatment was significantly higher than that in the control group, and the incidence of complications was lower. WHOQOL-BREF scores of the two groups after nursing intervention were higher than before, and the score in the experimental group was higher than the control group. In addition, the nursing satisfaction was also significantly higher in the experimental group, and the difference was statistically significant (P < 0.05). In conclusion, ICGA based on the Otsu method could effectively evaluate the cerebrovascular morphology during craniotomy and ICAP and improve the surgical efficacy. Combined with perioperative nursing intervention, it could greatly reduce the incidence of postoperative complications, improve the treatment effect and quality of life, and enhance the long-term prognosis.

From Accuracy to Reliability and Robustness in Cardiac Magnetic Resonance Image Segmentation: A Review

Since the rise of deep learning (DL) in the mid-2010s, cardiac magnetic resonance (CMR) image segmentation has achieved state-of-the-art performance. Despite achieving inter-observer variability in terms of different accuracy performance measures, visual inspections reveal errors in most segmentation results, indicating a lack of reliability and robustness of DL segmentation models, which can be critical if a model was to be deployed into clinical practice. In this work, we aim to bring attention to reliability and robustness, two unmet needs of cardiac image segmentation methods, which are hampering their translation into practice. To this end, we first study the performance accuracy evolution of CMR segmentation, illustrate the improvements brought by DL algorithms and highlight the symptoms of performance stagnation. Afterwards, we provide formal definitions of reliability and robustness. Based on the two definitions, we identify the factors that limit the reliability and robustness of state-of-the-art deep learning CMR segmentation techniques. Finally, we give an overview of the current set of works that focus on improving the reliability and robustness of CMR segmentation, and we categorize them into two families of methods: quality control methods and model improvement techniques. The first category corresponds to simpler strategies that only aim to flag situations where a model may be incurring poor reliability or robustness. The second one, instead, directly tackles the problem by bringing improvements into different aspects of the CMR segmentation model development process. We aim to bring the attention of more researchers towards these emerging trends regarding the development of reliable and robust CMR segmentation frameworks, which can guarantee the safe use of DL in clinical routines and studies.