Statistical validation metric for accuracy assessment in medical image segmentation

FMD-UNet: fine-grained feature squeeze and multiscale cascade dilated semantic aggregation dual-decoder UNet for COVID-19 lung infection segmentation from CT images

With the advancement of computer-aided diagnosis, the automatic segmentation of COVID-19 infection areas holds great promise for assisting in the timely diagnosis and recovery of patients in clinical practice. Currently, methods relying on U-Net face challenges in effectively utilizing fine-grained semantic information from input images and bridging the semantic gap between the encoder and decoder. To address these issues, we propose an FMD-UNet dual-decoder U-Net network for COVID-19 infection segmentation, which integrates a Fine-grained Feature Squeezing (FGFS) decoder and a Multi-scale Dilated Semantic Aggregation (MDSA) decoder. The FGFS decoder produces fine feature maps through the compression of fine-grained features and a weighted attention mechanism, guiding the model to capture detailed semantic information. The MDSA decoder consists of three hierarchical MDSA modules designed for different stages of input information. These modules progressively fuse different scales of dilated convolutions to process the shallow and deep semantic information from the encoder, and use the extracted feature information to bridge the semantic gaps at various stages, this design captures extensive contextual information while decoding and predicting segmentation, thereby suppressing the increase in model parameters. To better validate the robustness and generalizability of the FMD-UNet, we conducted comprehensive performance evaluations and ablation experiments on three public datasets, and achieved leading Dice Similarity Coefficient (DSC) scores of 84.76, 78.56 and 61.99% in COVID-19 infection segmentation, respectively. Compared to previous methods, the FMD-UNet has fewer parameters and shorter inference time, which also demonstrates its competitiveness.

Deep-learning-based pelvic automatic segmentation in pelvic fractures

With the recent increase in traffic accidents, pelvic fractures are increasing, second only to skull fractures, in terms of mortality and risk of complications. Research is actively being conducted on the treatment of intra-abdominal bleeding, the primary cause of death related to pelvic fractures. Considerable preliminary research has also been performed on segmenting tumors and organs. However, studies on clinically useful algorithms for bone and pelvic segmentation, based on developed models, are limited. In this study, we explored the potential of deep-learning models presented in previous studies to accurately segment pelvic regions in X-ray images. Data were collected from X-ray images of 940 patients aged 18 or older at Gachon University Gil Hospital from January 2015 to December 2022. To segment the pelvis, Attention U-Net, Swin U-Net, and U-Net were trained, thereby comparing and analyzing the results using five-fold cross-validation. The Swin U-Net model displayed relatively high performance compared to Attention U-Net and U-Net models, achieving an average sensitivity, specificity, accuracy, and dice similarity coefficient of 96.77%, of 98.50%, 98.03%, and 96.32%, respectively.

Towards clinically applicable automated mandibular canal segmentation on CBCT

OBJECTIVES

To develop a deep learning-based system for precise, robust, and fully automated segmentation of the mandibular canal on cone beam computed tomography (CBCT) images.

METHODS

The system was developed on 536 CBCT scans (training set: 376, validation set: 80, testing set: 80) from one center and validated on an external dataset of 89 CBCT scans from 3 centers. Each scan was annotated using a multi-stage annotation method and refined by oral and maxillofacial radiologists. We proposed a three-step strategy for the mandibular canal segmentation: extraction of the region of interest based on 2D U-Net, global segmentation of the mandibular canal, and segmentation refinement based on 3D U-Net.

RESULTS

The system consistently achieved accurate mandibular canal segmentation in the internal set (Dice similarity coefficient [DSC], 0.952; intersection over union [IoU], 0.912; average symmetric surface distance [ASSD], 0.046 mm; 95% Hausdorff distance [HD95], 0.325 mm) and the external set (DSC, 0.960; IoU, 0.924; ASSD, 0.040 mm; HD95, 0.288 mm).

CONCLUSIONS

These results demonstrated the potential clinical application of this AI system in facilitating clinical workflows related to mandibular canal localization.

CLINICAL SIGNIFICANCE

Accurate delineation of the mandibular canal on CBCT images is critical for implant placement, mandibular third molar extraction, and orthognathic surgery. This AI system enables accurate segmentation across different models, which could contribute to more efficient and precise dental automation systems.

A deep learning model for automated kidney calculi detection on non-contrast computed tomography scans in dogs

Nephrolithiasis is one of the most common urinary disorders in dogs. Although a majority of kidney calculi are non-obstructive and are likely to be asymptomatic, they can lead to parenchymal loss and obstruction as they progress. Thus, early diagnosis of kidney calculi is important for patient monitoring and better prognosis. However, detecting kidney calculi and monitoring changes in the sizes of the calculi from computed tomography (CT) images is time-consuming for clinicians. This study, in a first of its kind, aims to develop a deep learning model for automatic kidney calculi detection using pre-contrast CT images of dogs. A total of 34,655 transverseimage slices obtained from 76 dogs with kidney calculi were used to develop the deep learning model. Because of the differences in kidney location and calculi sizes in dogs compared to humans, several processing methods were used. The first stage of the models, based on the Attention U-Net (AttUNet), was designed to detect the kidney for the coarse feature map. Five different models-AttUNet, UTNet, TransUNet, SwinUNet, and RBCANet-were used in the second stage to detect the calculi in the kidneys, and the performance of the models was evaluated. Compared with a previously developed model, all the models developed in this study yielded better dice similarity coefficients (DSCs) for the automatic segmentation of the kidney. To detect kidney calculi, RBCANet and SwinUNet yielded the best DSC, which was 0.74. In conclusion, the deep learning model developed in this study can be useful for the automated detection of kidney calculi.

The use of deep learning in medical imaging to improve spine care: A scoping review of current literature and clinical applications

Background

Artificial intelligence is a revolutionary technology that promises to assist clinicians in improving patient care. In radiology, deep learning (DL) is widely used in clinical decision aids due to its ability to analyze complex patterns and images. It allows for rapid, enhanced data, and imaging analysis, from diagnosis to outcome prediction. The purpose of this study was to evaluate the current literature and clinical utilization of DL in spine imaging.

Methods

This study is a scoping review and utilized the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology to review the scientific literature from 2012 to 2021. A search in PubMed, Web of Science, Embased, and IEEE Xplore databases with syntax specific for DL and medical imaging in spine care applications was conducted to collect all original publications on the subject. Specific data was extracted from the available literature, including algorithm application, algorithms tested, database type and size, algorithm training method, and outcome of interest.

Results

A total of 365 studies (total sample of 232,394 patients) were included and grouped into 4 general applications: diagnostic tools, clinical decision support tools, automated clinical/instrumentation assessment, and clinical outcome prediction. Notable disparities exist in the selected algorithms and the training across multiple disparate databases. The most frequently used algorithms were U-Net and ResNet. A DL model was developed and validated in 92% of included studies, while a pre-existing DL model was investigated in 8%. Of all developed models, only 15% of them have been externally validated.

Conclusions

Based on this scoping review, DL in spine imaging is used in a broad range of clinical applications, particularly for diagnosing spinal conditions. There is a wide variety of DL algorithms, database characteristics, and training methods. Future studies should focus on external validation of existing models before bringing them into clinical use.

MISM: A Medical Image Segmentation Metric for Evaluation of Weak Labeled Data

Performance measures are an important tool for assessing and comparing different medical image segmentation algorithms. Unfortunately, the current measures have their weaknesses when it comes to assessing certain edge cases. These limitations arise when images with a very small region of interest or without a region of interest at all are assessed. As a solution to these limitations, we propose a new medical image segmentation metric: MISm. This metric is a composition of the Dice similarity coefficient and the weighted specificity. MISm was investigated for definition gaps, an appropriate scoring gradient, and different weighting coefficients used to propose a constant value. Furthermore, an evaluation was performed by comparing the popular metrics in the medical image segmentation and MISm using images of magnet resonance tomography from several fictitious prediction scenarios. Our analysis shows that MISm can be applied in a general way and thus also covers the mentioned edge cases, which are not covered by other metrics, in a reasonable way. In order to allow easy access to MISm and therefore widespread application in the community, as well as reproducibility of experimental results, we included MISm in the publicly available evaluation framework MISeval.

Automated segmentation of the mandibular canal and its anterior loop by deep learning

Accurate mandibular canal (MC) detection is crucial to avoid nerve injury during surgical procedures. Moreover, the anatomic complexity of the interforaminal region requires a precise delineation of anatomical variations such as the anterior loop (AL). Therefore, CBCT-based presurgical planning is recommended, even though anatomical variations and lack of MC cortication make canal delineation challenging. To overcome these limitations, artificial intelligence (AI) may aid presurgical MC delineation. In the present study, we aim to train and validate an AI-driven tool capable of performing accurate segmentation of the MC even in the presence of anatomical variation such as AL. Results achieved high accuracy metrics, with 0.997 of global accuracy for both MC with and without AL. The anterior and middle sections of the MC, where most surgical interventions are performed, presented the most accurate segmentation compared to the posterior section. The AI-driven tool provided accurate segmentation of the mandibular canal, even in the presence of anatomical variation such as an anterior loop. Thus, the presently validated dedicated AI tool may aid clinicians in automating the segmentation of neurovascular canals and their anatomical variations. It may significantly contribute to presurgical planning for dental implant placement, especially in the interforaminal region.

Eliminating the need for manual segmentation to determine size and volume from MRI. A proof of concept on segmenting the lateral ventricles

Manual segmentation, which is tedious, time-consuming, and operator-dependent, is currently used as the gold standard to validate automatic and semiautomatic methods that quantify geometries from 2D and 3D MR images. This study examines the accuracy of manual segmentation and generalizes a strategy to eliminate its use. Trained individuals manually measured MR lateral ventricles images of normal and hydrocephalus infants from 1 month to 9.5 years of age. We created 3D-printed models of the lateral ventricles from the MRI studies and accurately estimated their volume by water displacement. MRI phantoms were made from the 3D models and images obtained. Using a previously developed artificial intelligence (AI) algorithm that employs four features extracted from the images, we estimated the ventricular volume of the phantom images. The algorithm was certified when discrepancies between the volumes-gold standards-yielded by the water displacement device and those measured by the automation were smaller than 2%. Then, we compared volumes after manual segmentation with those obtained with the certified automation. As determined by manual segmentation, lateral ventricular volume yielded an inter and intra-operator variation up to 50% and 48%, respectively, while manually segmenting saggital images generated errors up to 71%. These errors were determined by direct comparisons with the volumes yielded by the certified automation. The errors induced by manual segmentation are large enough to adversely affect decisions that may lead to less-than-optimal treatment; therefore, we suggest avoiding manual segmentation whenever possible.

ACU2E-Net: A novel predict-refine attention network for segmentation of soft-tissue structures in ultrasound images

Segmentation of anatomical structures in ultrasound images is a challenging task due to existence of artifacts inherit to the modality such as speckle noise, attenuation, shadowing, uneven textures and blurred boundaries. This paper presents a novel attention-based predict-refine network, called ACU2E-Net, for segmentation of soft-tissue structures in ultrasound images. The network consists of two modules: a predict module, which is built upon our newly proposed attentive coordinate convolution; and a novel multi-head residual refinement module, which consists of three parallel residual refinement modules. The attentive coordinate convolution is designed to improve the segmentation accuracy by perceiving the shape and positional information of the target anatomy. The proposed multi-head residual refinement module reduces both segmentation biases and variances by integrating residual refinement and ensemble strategies. Moreover, it avoids multi-pass training and inference commonly seen in ensemble methods. To show the effectiveness of our method, we collect a comprehensive dataset of thyroid ultrasound scans from 12 different imaging centers, and evaluate our proposed network against state-of-the-art segmentation methods. Comparisons against state-of-the-art models demonstrate the competitive performance of our newly designed network on both the transverse and sagittal thyroid images. Ablation studies show that proposed modules improve the segmentation Dice score of the baseline model from 79.62% to 80.97% and 82.92% while reducing the variance from 6.12% to 4.67% and 3.21% in transverse and sagittal views, respectively.

MSeg-Net: A Melanoma Mole Segmentation Network Using CornerNet and Fuzzy K-Means Clustering

Melanoma is a dangerous form of skin cancer that results in the demise of patients at the developed stage. Researchers have attempted to develop automated systems for the timely recognition of this deadly disease. However, reliable and precise identification of melanoma moles is a tedious and complex activity as there exist huge differences in the mass, structure, and color of the skin lesions. Additionally, the incidence of noise, blurring, and chrominance changes in the suspected images further enhance the complexity of the detection procedure. In the proposed work, we try to overcome the limitations of the existing work by presenting a deep learning (DL) model. Descriptively, after accomplishing the preprocessing task, we have utilized an object detection approach named CornerNet model to detect melanoma lesions. Then the localized moles are passed as input to the fuzzy K-means (FLM) clustering approach to perform the segmentation task. To assess the segmentation power of the proposed approach, two standard databases named ISIC-2017 and ISIC-2018 are employed. Extensive experimentation has been conducted to demonstrate the robustness of the proposed approach through both numeric and pictorial results. The proposed approach is capable of detecting and segmenting the moles of arbitrary shapes and orientations. Furthermore, the presented work can tackle the presence of noise, blurring, and brightness variations as well. We have attained the segmentation accuracy values of 99.32% and 99.63% over the ISIC-2017 and ISIC-2018 databases correspondingly which clearly depicts the effectiveness of our model for the melanoma mole segmentation.

nnUNet-based Multi-modality Breast MRI Segmentation and Tissue-Delineating Phantom for Robotic Tumor Surgery Planning

Segmentation of the thoracic region and breast tissues is crucial for analyzing and diagnosing the presence of breast masses. This paper introduces a medical image segmentation architecture that aggregates two neural networks based on the state-of-the-art nnU-Net. Additionally, this study proposes a polyvinyl alcohol cryogel (PVA-C) breast phantom, based on its automated segmentation approach, to enable planning and navigation experiments for robotic breast surgery. The dataset consists of multimodality breast MRI of T2W and STIR images obtained from 10 patients. A statistical analysis of segmentation tasks emphasizes the Dice Similarity Coefficient (DSC), segmentation accuracy, sensitivity, and specificity. We first use a single class labeling to segment the breast region and then exploit it as an input for three-class labeling to segment fatty, fibroglandular (FGT), and tumorous tissues. The first network has a 0.95 DCS, while the second network has a 0.95, 0.83, and 0.41 for fat, FGT, and tumor classes, respectively. Clinical Relevance-This research is relevant to the breast surgery community as it establishes a deep learning-based (DL) algorithmic and phantomic foundation for surgical planning and navigation that will exploit preoperative multimodal MRI and intraoperative ultrasound to achieve highly cosmetic breast surgery. In addition, the planning and navigation will guide a robot that can cut, resect, bag, and grasp a tissue mass that encapsulates breast tumors and positive tissue margins. This image-guided robotic approach promises to potentiate the accuracy of breast surgeons and improve patient outcomes.

Towards a guideline for evaluation metrics in medical image segmentation

In the last decade, research on artificial intelligence has seen rapid growth with deep learning models, especially in the field of medical image segmentation. Various studies demonstrated that these models have powerful prediction capabilities and achieved similar results as clinicians. However, recent studies revealed that the evaluation in image segmentation studies lacks reliable model performance assessment and showed statistical bias by incorrect metric implementation or usage. Thus, this work provides an overview and interpretation guide on the following metrics for medical image segmentation evaluation in binary as well as multi-class problems: Dice similarity coefficient, Jaccard, Sensitivity, Specificity, Rand index, ROC curves, Cohen's Kappa, and Hausdorff distance. Furthermore, common issues like class imbalance and statistical as well as interpretation biases in evaluation are discussed. As a summary, we propose a guideline for standardized medical image segmentation evaluation to improve evaluation quality, reproducibility, and comparability in the research field.

Explainable artificial intelligence-based edge fuzzy images for COVID-19 detection and identification

The COVID-19 pandemic continues to wreak havoc on the world’s population’s health and well-being. Successful screening of infected patients is a critical step in the fight against it, with radiology examination using chest radiography being one of the most important screening methods. For the definitive diagnosis of COVID-19 disease, reverse-transcriptase polymerase chain reaction remains the gold standard. Currently available lab tests may not be able to detect all infected individuals; new screening methods are required. We propose a Multi-Input Transfer Learning COVID-Net fuzzy convolutional neural network to detect COVID-19 instances from torso X-ray, motivated by the latter and the open-source efforts in this research area. Furthermore, we use an explainability method to investigate several Convolutional Networks COVID-Net forecasts in an effort to not only gain deeper insights into critical factors associated with COVID-19 instances, but also to aid clinicians in improving screening. We show that using transfer learning and pre-trained models, we can detect it with a high degree of accuracy. Using X-ray images, we chose four neural networks to predict its probability. Finally, in order to achieve better results, we considered various methods to verify the techniques proposed here. As a result, we were able to create a model with an AUC of 1.0 and accuracy, precision, and recall of 0.97. The model was quantized for use in Internet of Things devices and maintained a 0.95 percent accuracy.

Development and validation of a novel artificial intelligence driven tool for accurate mandibular canal segmentation on CBCT

OBJECTIVES

The objective of this study is the development and validation of a novel artificial intelligence driven tool for fast and accurate mandibular canal segmentation on cone beam computed tomography (CBCT).

METHODS

A total of 235 CBCT scans from dentate subjects needing oral surgery were used in this study, allowing for development, training and validation of a deep learning algorithm for automated mandibular canal (MC) segmentation on CBCT. Shape, diameter and direction of the MC were adjusted on all CBCT slices using a voxel-wise approach. Validation was then performed on a random set of 30 CBCTs - previously unseen by the algorithm - where voxel-level annotations allowed for assessment of all MC segmentations.

RESULTS

Primary results show successful implementation of the AI algorithm for segmentation of the MC with a mean IoU of 0.636 (± 0.081), a median IoU of 0.639 (± 0.081), a mean Dice Similarity Coefficient of 0.774 (± 0.062). Precision, recall and accuracy had mean values of 0.782 (± 0.121), 0.792 (± 0.108) and 0.99 (± 7.64×10^-05) respectively. The total time for automated AI segmentation was 21.26 s (±2.79), which is 107 times faster than accurate manual segmentation.

CONCLUSIONS

This study demonstrates a novel, fast and accurate AI-driven module for MC segmentation on CBCT.

CLINICAL SIGNIFICANCE

Given the importance of adequate pre-operative mandibular canal assessment, Artificial Intelligence could help relieve practitioners from the delicate and time-consuming task of manually tracing and segmenting this structure, helping prevent per- and post-operative neurovascular complications.

New Hybrid Method for Left Ventricular Ejection Fraction Assessment from Radionuclide Ventriculography Images

BACKGROUND

In this paper, we propose a new efficient method of radionuclide ventriculography image segmentation to estimate the left ventricular ejection fraction. This parameter is an important prognostic factor for diagnosing abnormal cardiac function.

METHODS

The proposed method combines the Chan-Vese and the mathematical morphology algorithms. It was applied to diastolic and systolic images obtained from the Nuclear Medicine Department of Salah AZAIEZ Institute. In order to validate our proposed method, we compare the obtained results to those of two methods present in the literature. The first one is based on mathematical morphology, while the second one uses the basic Chan-Vese algorithm. To evaluate the quality of segmentation, we compute accuracy, positive predictive value and area under the ROC curve. We also compare the left ventricle ejection fraction estimated by our method to that of the reference given by the software of the gamma-camera and validated by the expert, using Pearson's correlation coefficient, ANOVA test and linear regression.

RESULTS

Static results show that the proposed method is very efficient for the detection of the left ventricle. The accuracy was 98.60%, higher than that of the other two methods (95.52% and 98.50%).

CONCLUSION

Likewise, the positive predictive value was the highest (86.40% vs. 83.63% 71.82%). The area under the ROC curve was also the most important (0.998% vs. 0.926% 0.919%). On the other hand, Pearson's correlation coefficient was the highest (99% vs. 98% 37%). The correlation was significantly positive (p<0.001).

Comparison of metrics for the evaluation of medical segmentations using prostate MRI dataset

Nine previously proposed segmentation evaluation metrics, targeting medical relevance, accounting for holes, and added regions or differentiating over- and under-segmentation, were compared with 24 traditional metrics to identify those which better capture the requirements for clinical segmentation evaluation. Evaluation was first performed using 2D synthetic shapes to highlight features and pitfalls of the metrics with known ground truths (GTs) and machine segmentations (MSs). Clinical evaluation was then performed using publicly-available prostate images of 20 subjects with MSs generated by 3 different deep learning networks (DenseVNet, HighRes3DNet, and ScaleNet) and GTs drawn by 2 readers. The same readers also performed the 2D visual assessment of the MSs using a dual negative-positive grading of -5 to 5 to reflect over- and under-estimation. Nine metrics that correlated well with visual assessment were selected for further evaluation using 3 different network ranking methods - based on a single metric, normalizing the metric using 2 GTs, and ranking the network based on a metric then averaging, including leave-one-out evaluation. These metrics yielded consistent ranking with HighRes3DNet ranked first then DenseVNet and ScaleNet using all ranking methods. Relative volume difference yielded the best positivity-agreement and correlation with dual visual assessment, and thus is better for providing over- and under-estimation. Interclass Correlation yielded the strongest correlation with the absolute visual assessment (0-5). Symmetric-boundary dice consistently yielded good discrimination of the networks for all three ranking methods with relatively small variations within network. Good rank discrimination may be an additional metric feature required for better network performance evaluation.

Semantic segmentation of the multiform proximal femur and femoral head bones with the deep convolutional neural networks in low quality MRI sections acquired in different MRI protocols

Medical image segmentation is one of the most crucial issues in medical image processing and analysis. In general, segmentation of the various structures in medical images is performed for the further image analyzes such as quantification, assessment, diagnosis, prognosis and classification. In this paper, a research study for the 2D semantic segmentation of the multiform, both spheric and aspheric, femoral head and proximal femur bones in magnetic resonance imaging (MRI) sections of the patients with Legg-Calve-Perthes disease (LCPD) with the deep convolutional neural networks (CNNs) is presented. In the scope of the proposed study, bilateral hip MRI sections acquired in coronal plane were used. The main characteristic of the MRI sections that were used is to be low quality images which were obtained in different MRI protocols by using 3 different MRI scanners with 1.5 T imaging capability. In performance evaluations, promising segmentation results were achieved with deep CNNs in low quality MRI sections acquired in different MRI protocols. A success rate about 90% was observed in semantic segmentation of the multiform femoral head and proximal femur bones in a total of 194 MRI sections obtained from 33 MRI sequences of 13 patients with deep CNNs.

Bone-Cancer Assessment and Destruction Pattern Analysis in Long-Bone X-ray Image

Bone cancer originates from bone and rapidly spreads to the rest of the body affecting the patient. A quick and preliminary diagnosis of bone cancer begins with the analysis of bone X-ray or MRI image. Compared to MRI, an X-ray image provides a low-cost diagnostic tool for diagnosis and visualization of bone cancer. In this paper, a novel technique for the assessment of cancer stage and grade in long bones based on X-ray image analysis has been proposed. Cancer-affected bone images usually appear with a variation in bone texture in the affected region. A fusion of different methodologies is used for the purpose of our analysis. In the proposed approach, we extract certain features from bone X-ray images and use support vector machine (SVM) to discriminate healthy and cancerous bones. A technique based on digital geometry is deployed for localizing cancer-affected regions. Characterization of the present stage and grade of the disease and identification of the underlying bone-destruction pattern are performed using a decision tree classifier. Furthermore, the method leads to the development of a computer-aided diagnostic tool that can readily be used by paramedics and doctors. Experimental results on a number of test cases reveal satisfactory diagnostic inferences when compared with ground truth known from clinical findings.

Validation of parametric mesh generation for subject-specific cerebroarterial trees using modified Hausdorff distance metrics

Accurate subject-specific vascular network reconstruction is a critical task for the hemodynamic analysis of cerebroarterial circulation. Vascular skeletonization and computational mesh generation for large sections of cerebrovascular trees from magnetic resonance angiography (MRA) is an error-prone, operator-dependent, and very time-consuming task. Validation of reconstructed computational models is essential to ascertain their accuracy and precision, which directly relates to the confidence of CFD computations performed on these meshes. The aim of this study is to generate an imaging segmentation pipeline to validate and quantify the spatial accuracy of computational models of subject-specific cerebral arterial trees. We used a recently introduced parametric structured mesh (PSM) generation method to automatically reconstruct six subject-specific cerebral arterial trees containing 1364 vessels and 571 bifurcations. By automatically extracting sampling frames for all vascular segments and bifurcations, we quantify the spatial accuracy of PSM against the original MRA images. Our comprehensive study correlates lumen area, pixel-based statistical analysis, area overlap and centerline accuracy measurements. In addition, we propose a new metric, the pointwise offset surface distance metric (PSD), to quantify the spatial alignment between dimensions of reconstructed arteries and bifurcations with in-vivo data with the ability to quantify the over- and under-approximation of the reconstructed models. Accurate reconstruction of vascular trees can a practical process tool for morphological analysis of large patient data banks, such as medical record files in hospitals, or subject-specific hemodynamic simulations of the cerebral arterial circulation.

Enhanced particle-filtering framework for vessel segmentation and tracking

BACKGROUND AND OBJECTIVES

A robust vessel segmentation and tracking method based on a particle-filtering framework is proposed to cope with increasing demand for a method that can detect and track vessel anomalies.

METHODS

We apply the level set method to segment the vessel boundary and a particle filter to track the position and shape variations in the vessel boundary between two adjacent slices. To enhance the segmentation and tracking performances, the importance density of the particle filter is localized by estimating the translation of an object's boundary. In addition, to minimize problems related to degeneracy and sample impoverishment in the particle filter, a newly proposed weighting policy is investigated.

RESULTS

Compared to conventional methods, the proposed algorithm demonstrates better segmentation and tracking performances. Moreover, the stringent weighting policy we proposed demonstrates a tendency of suppressing degeneracy and sample impoverishment, and higher tracking accuracy can be obtained.

CONCLUSIONS

The proposed method is expected to be applied to highly valuable applications for more accurate three-dimensional vessel tracking and rendering.

Active appearance pyramids for object parametrisation and fitting

Object class representation is one of the key problems in various medical image analysis tasks. We propose a part-based parametric appearance model we refer to as an Active Appearance Pyramid (AAP). The parts are delineated by multi-scale Local Feature Pyramids (LFPs) for superior spatial specificity and distinctiveness. An AAP models the variability within a population with local translations of multi-scale parts and linear appearance variations of the assembly of the parts. It can fit and represent new instances by adjusting the shape and appearance parameters. The fitting process uses a two-step iterative strategy: local landmark searching followed by shape regularisation. We present a simultaneous local feature searching and appearance fitting algorithm based on the weighted Lucas and Kanade method. A shape regulariser is derived to calculate the maximum likelihood shape with respect to the prior and multiple landmark candidates from multi-scale LFPs, with a compact closed-form solution. We apply the 2D AAP on the modelling of variability in patients with lumbar spinal stenosis (LSS) and validate its performance on 200 studies consisting of routine axial and sagittal MRI scans. Intervertebral sagittal and parasagittal cross-sections are typically used for the diagnosis of LSS, we therefore build three AAPs on L3/4, L4/5 and L5/S1 axial cross-sections and three on parasagittal slices. Experiments show significant improvement in convergence range, robustness to local minima and segmentation precision compared with Constrained Local Models (CLMs), Active Shape Models (ASMs) and Active Appearance Models (AAMs), as well as superior performance in appearance reconstruction compared with AAMs. We also validate the performance on 3D CT volumes of hip joints from 38 studies. Compared to AAMs, AAPs achieve a higher segmentation and reconstruction precision. Moreover, AAPs have a significant improvement in efficiency, consuming about half the memory and less than 10% of the training time and 15% of the testing time.

Adaptive Kalman snake for semi-autonomous 3D vessel tracking

In this paper, we propose a robust semi-autonomous algorithm for 3D vessel segmentation and tracking based on an active contour model and a Kalman filter. For each computed tomography angiography (CTA) slice, we use the active contour model to segment the vessel boundary and the Kalman filter to track position and shape variations of the vessel boundary between slices. For successful segmentation via active contour, we select an adequate number of initial points from the contour of the first slice. The points are set manually by user input for the first slice. For the remaining slices, the initial contour position is estimated autonomously based on segmentation results of the previous slice. To obtain refined segmentation results, an adaptive control spacing algorithm is introduced into the active contour model. Moreover, a block search-based initial contour estimation procedure is proposed to ensure that the initial contour of each slice can be near the vessel boundary. Experiments were performed on synthetic and real chest CTA images. Compared with the well-known Chan-Vese (CV) model, the proposed algorithm exhibited better performance in segmentation and tracking. In particular, receiver operating characteristic analysis on the synthetic and real CTA images demonstrated the time efficiency and tracking robustness of the proposed model. In terms of computational time redundancy, processing time can be effectively reduced by approximately 20%.

Big Data Analytics in Healthcare

The rapidly expanding field of big data analytics has started to play a pivotal role in the evolution of healthcare practices and research. It has provided tools to accumulate, manage, analyze, and assimilate large volumes of disparate, structured, and unstructured data produced by current healthcare systems. Big data analytics has been recently applied towards aiding the process of care delivery and disease exploration. However, the adoption rate and research development in this space is still hindered by some fundamental problems inherent within the big data paradigm. In this paper, we discuss some of these major challenges with a focus on three upcoming and promising areas of medical research: image, signal, and genomics based analytics. Recent research which targets utilization of large volumes of medical data while combining multimodal data from disparate sources is discussed. Potential areas of research within this field which have the ability to provide meaningful impact on healthcare delivery are also examined.

Computer Vision Techniques for Transcatheter Intervention

Minimally invasive transcatheter technologies have demonstrated substantial promise for the diagnosis and the treatment of cardiovascular diseases. For example, transcatheter aortic valve implantation is an alternative to aortic valve replacement for the treatment of severe aortic stenosis, and transcatheter atrial fibrillation ablation is widely used for the treatment and the cure of atrial fibrillation. In addition, catheter-based intravascular ultrasound and optical coherence tomography imaging of coronary arteries provides important information about the coronary lumen, wall, and plaque characteristics. Qualitative and quantitative analysis of these cross-sectional image data will be beneficial to the evaluation and the treatment of coronary artery diseases such as atherosclerosis. In all the phases (preoperative, intraoperative, and postoperative) during the transcatheter intervention procedure, computer vision techniques (e.g., image segmentation and motion tracking) have been largely applied in the field to accomplish tasks like annulus measurement, valve selection, catheter placement control, and vessel centerline extraction. This provides beneficial guidance for the clinicians in surgical planning, disease diagnosis, and treatment assessment. In this paper, we present a systematical review on these state-of-the-art methods. We aim to give a comprehensive overview for researchers in the area of computer vision on the subject of transcatheter intervention. Research in medical computing is multi-disciplinary due to its nature, and hence, it is important to understand the application domain, clinical background, and imaging modality, so that methods and quantitative measurements derived from analyzing the imaging data are appropriate and meaningful. We thus provide an overview on the background information of the transcatheter intervention procedures, as well as a review of the computer vision techniques and methodologies applied in this area.

Using preoperative imaging for intraoperative guidance: a case of mistaken identity

BACKGROUND

Surgical image guidance systems to date have tended to rely on reconstructions of preoperative datasets. This paper assesses the accuracy of these reconstructions to establish whether they are appropriate for use in image guidance platforms.

METHODS

Nine raters (two experts in image interpretation and preparation, three in image interpretation, and four in neither interpretation nor preparation) were asked to perform a segmentation of ten renal tumours (four cystic and six solid tumours). These segmentations were compared with a gold standard consensus segmentation generated using a previously validated algorithm.

RESULTS

Average sensitivity and positive predictive value (PPV) were 0.902 and 0.891, respectively. When assessing for variability between raters, significant differences were seen in the PPV, sensitivity and incursions and excursions from consensus tumour boundary.

CONCLUSIONS

This paper has demonstrated that the interpretation required for the segmentation of preoperative imaging of renal tumours introduces significant inconsistency and inaccuracy. Copyright © 2015 John Wiley & Sons, Ltd.

Segmentation editing improves efficiency while reducing inter-expert variation and maintaining accuracy for normal brain tissues in the presence of space-occupying lesions

Image segmentation has become a vital and often rate-limiting step in modern radiotherapy treatment planning. In recent years, the pace and scope of algorithm development, and even introduction into the clinic, have far exceeded evaluative studies. In this work we build upon our previous evaluation of a registration driven segmentation algorithm in the context of 8 expert raters and 20 patients who underwent radiotherapy for large space-occupying tumours in the brain. In this work we tested four hypotheses concerning the impact of manual segmentation editing in a randomized single-blinded study. We tested these hypotheses on the normal structures of the brainstem, optic chiasm, eyes and optic nerves using the Dice similarity coefficient, volume, and signed Euclidean distance error to evaluate the impact of editing on inter-rater variance and accuracy. Accuracy analyses relied on two simulated ground truth estimation methods: simultaneous truth and performance level estimation and a novel implementation of probability maps. The experts were presented with automatic, their own, and their peers' segmentations from our previous study to edit. We found, independent of source, editing reduced inter-rater variance while maintaining or improving accuracy and improving efficiency with at least 60% reduction in contouring time. In areas where raters performed poorly contouring from scratch, editing of the automatic segmentations reduced the prevalence of total anatomical miss from approximately 16% to 8% of the total slices contained within the ground truth estimations. These findings suggest that contour editing could be useful for consensus building such as in developing delineation standards, and that both automated methods and even perhaps less sophisticated atlases could improve efficiency, inter-rater variance, and accuracy.

Automatic segmentation of cell nuclei in Feulgen-stained histological sections of prostate cancer and quantitative evaluation of segmentation results

Digital image analysis of cell nuclei is useful to obtain quantitative information for the diagnosis and prognosis of cancer. However, the lack of a reliable automatic nuclear segmentation is a limiting factor for high-throughput nuclear image analysis. We have developed a method for automatic segmentation of nuclei in Feulgen-stained histological sections of prostate cancer. A local adaptive thresholding with an object perimeter gradient verification step detected the nuclei and was combined with an active contour model that featured an optimized initialization and worked within a restricted region to improve convergence of the segmentation of each nucleus. The method was tested on 30 randomly selected image frames from three cases, comparing the results from the automatic algorithm to a manual delineation of 924 nuclei. The automatic method segmented a few more nuclei compared to the manual method, and about 73% of the manually segmented nuclei were also segmented by the automatic method. For each nucleus segmented both manually and automatically, the accuracy (i.e., agreement with manual delineation) was estimated. The mean segmentation sensitivity/specificity were 95%/96%. The results from the automatic method were not significantly different from the ground truth provided by manual segmentation. This opens the possibility for large-scale nuclear analysis based on automatic segmentation of nuclei in Feulgen-stained histological sections.

Robust white matter lesion segmentation in FLAIR MRI

This paper discusses a white matter lesion (WML) segmentation scheme for fluid attenuation inversion recovery (FLAIR) MRI. The method computes the volume of lesions with subvoxel precision by accounting for the partial volume averaging (PVA) artifact. As WMLs are related to stroke and carotid disease, accurate volume measurements are most important. Manual volume computation is laborious, subjective, time consuming, and error prone. Automated methods are a nice alternative since they quantify WML volumes in an objective, efficient, and reliable manner. PVA is initially modeled with a localized edge strength measure since PVA resides in the boundaries between tissues. This map is computed in 3-D and is transformed to a global representation to increase robustness to noise. Significant edges correspond to PVA voxels, which are used to find the PVA fraction α (amount of each tissue present in mixture voxels). Results on simulated and real FLAIR images show high WML segmentation performance compared to ground truth (98.9% and 83% overlap, respectively), which outperforms other methods. Lesion load studies are included that automatically analyze WML volumes for each brain hemisphere separately. This technique does not require any distributional assumptions/parameters or training samples and is applied on a single MR modality, which is a major advantage compared to the traditional methods.

Active volume models for medical image segmentation

In this paper, we propose a novel predictive model, active volume model (AVM), for object boundary extraction. It is a dynamic "object" model whose manifestation includes a deformable curve or surface representing a shape, a volumetric interior carrying appearance statistics, and an embedded classifier that separates object from background based on current feature information. The model focuses on an accurate representation of the foreground object's attributes, and does not explicitly represent the background. As we will show, however, the model is capable of reasoning about the background statistics thus can detect when is change sufficient to invoke a boundary decision. When applied to object segmentation, the model alternates between two basic operations: 1) deforming according to current region of interest (ROI), which is a binary mask representing the object region predicted by the current model, and 2) predicting ROI according to current appearance statistics of the model. To further improve robustness and accuracy when segmenting multiple objects or an object with multiple parts, we also propose multiple-surface active volume model (MSAVM), which consists of several single-surface AVM models subject to high-level geometric spatial constraints. An AVM's deformation is derived from a linear system based on finite element method (FEM). To keep the model's surface triangulation optimized, surface remeshing is derived from another linear system based on Laplacian mesh optimization (LMO). Thus efficient optimization and fast convergence of the model are achieved by solving two linear systems. Segmentation, validation and comparison results are presented from experiments on a variety of 2-D and 3-D medical images.

Evaluation of methods for detection of fluorescence labeled subcellular objects in microscope images

Background

Several algorithms have been proposed for detecting fluorescently labeled subcellular objects in microscope images. Many of these algorithms have been designed for specific tasks and validated with limited image data. But despite the potential of using extensive comparisons between algorithms to provide useful information to guide method selection and thus more accurate results, relatively few studies have been performed.

Results

To better understand algorithm performance under different conditions, we have carried out a comparative study including eleven spot detection or segmentation algorithms from various application fields. We used microscope images from well plate experiments with a human osteosarcoma cell line and frames from image stacks of yeast cells in different focal planes. These experimentally derived images permit a comparison of method performance in realistic situations where the number of objects varies within image set. We also used simulated microscope images in order to compare the methods and validate them against a ground truth reference result. Our study finds major differences in the performance of different algorithms, in terms of both object counts and segmentation accuracies.

Conclusions

These results suggest that the selection of detection algorithms for image based screens should be done carefully and take into account different conditions, such as the possibility of acquiring empty images or images with very few spots. Our inclusion of methods that have not been used before in this context broadens the set of available detection methods and compares them against the current state-of-the-art methods for subcellular particle detection.

An evaluation of four automatic methods of segmenting the subcortical structures in the brain

The automation of segmentation of subcortical structures in the brain is an active research area. We have comprehensively evaluated four novel methods of fully automated segmentation of subcortical structures using volumetric, spatial overlap and distance-based measures. Two methods are atlas-based - classifier fusion and labelling (CFL) and expectation-maximisation segmentation using a brain atlas (EMS), and two incorporate statistical models of shape and appearance - profile active appearance models (PAM) and Bayesian appearance models (BAM). Each method was applied to the segmentation of 18 subcortical structures in 270 subjects from a diverse pool varying in age, disease, sex and image acquisition parameters. Our results showed that all four methods perform on par with recently published methods. CFL performed better than the others according to all three classes of metrics. In summary over all structures, the ranking by the Dice coefficient was CFL, BAM, joint EMS and PAM. The Hausdorff distance ranked the methods as CFL, joint PAM and BAM, EMS, whilst percentage absolute volumetric difference ranked them as joint CFL and PAM, joint BAM and EMS. Furthermore, as we had four methods of performing segmentation, we investigated whether the results obtained by each method were more similar to each other than to the manual segmentations using Williams' Index. Reassuringly, the Williams' Index was close to 1 for most subjects (mean=1.02, sd=0.05), indicating better agreement of each method with the gold standard than with the other methods. However, 2% of cases (mainly amygdala and nucleus accumbens) had values outside 3 standard deviations of the mean.

Active volume models with probabilistic object boundary prediction module

We propose a novel Active Volume Model (AVM) which deforms in a free-form manner to minimize energy. Unlike Snakes and level-set active contours which only consider curves or surfaces, the AVM is a deforming object model that has both boundary and an interior area. When applied to object segmentation and tracking, the model alternates between two basic operations: deform according to current object prediction, and predict according to current appearance statistics of the model. The probabilistic object prediction module relies on the Bayesian Decision Rule to separate foreground (i.e., object represented by the model) and background. Optimization of the model is a natural extension of the Snakes model so that region information becomes part of the external forces. The AVM thus has the efficiency of Snakes while having adaptive region-based constraints. Segmentation results, validation, and comparison with GVF Snakes and level set methods are presented for experiments on noisy 2D/3D medical images.