Robust Registration of Multi-modal Images: Towards Real-Time Clinical Applications

Automated breast tumor ultrasound image segmentation with hybrid UNet and classification using fine-tuned CNN model

Introduction

Breast cancer stands as the second most deadly form of cancer among women worldwide. Early diagnosis and treatment can significantly mitigate mortality rates.

Purpose

The study aims to classify breast ultrasound images into benign and malignant tumors. This approach involves segmenting the breast's region of interest (ROI) employing an optimized UNet architecture and classifying the ROIs through an optimized shallow CNN model utilizing an ablation study.

Method

Several image processing techniques are utilized to improve image quality by removing text, artifacts, and speckle noise, and statistical analysis is done to check the enhanced image quality is satisfactory. With the processed dataset, the segmentation of breast tumor ROI is carried out, optimizing the UNet model through an ablation study where the architectural configuration and hyperparameters are altered. After obtaining the tumor ROIs from the fine-tuned UNet model (RKO-UNet), an optimized CNN model is employed to classify the tumor into benign and malignant classes. To enhance the CNN model's performance, an ablation study is conducted, coupled with the integration of an attention unit. The model's performance is further assessed by classifying breast cancer with mammogram images.

Result

The proposed classification model (RKONet-13) results in an accuracy of 98.41 %. The performance of the proposed model is further compared with five transfer learning models for both pre-segmented and post-segmented datasets. K-fold cross-validation is done to assess the proposed RKONet-13 model's performance stability. Furthermore, the performance of the proposed model is compared with previous literature, where the proposed model outperforms existing methods, demonstrating its effectiveness in breast cancer diagnosis. Lastly, the model demonstrates its robustness for breast cancer classification, delivering an exceptional performance of 96.21 % on a mammogram dataset.

Conclusion

The efficacy of this study relies on image pre-processing, segmentation with hybrid attention UNet, and classification with fine-tuned robust CNN model. This comprehensive approach aims to determine an effective technique for detecting breast cancer within ultrasound images.

ISLES 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset

Magnetic resonance imaging (MRI) is an important imaging modality in stroke. Computer based automated medical image processing is increasingly finding its way into clinical routine. The Ischemic Stroke Lesion Segmentation (ISLES) challenge is a continuous effort to develop and identify benchmark methods for acute and sub-acute ischemic stroke lesion segmentation. Here we introduce an expert-annotated, multicenter MRI dataset for segmentation of acute to subacute stroke lesions ( https://doi.org/10.5281/zenodo.7153326 ). This dataset comprises 400 multi-vendor MRI cases with high variability in stroke lesion size, quantity and location. It is split into a training dataset of n = 250 and a test dataset of n = 150. All training data is publicly available. The test dataset will be used for model validation only and will not be released to the public. This dataset serves as the foundation of the ISLES 2022 challenge ( https://www.isles-challenge.org/ ) with the goal of finding algorithmic methods to enable the development and benchmarking of automatic, robust and accurate segmentation methods for ischemic stroke.

Population-specific brain [18F]-FDG PET templates of Chinese subjects for statistical parametric mapping

Statistical Parametric Mapping (SPM) is a computational approach for analysing functional brain images like Positron Emission Tomography (PET). When performing SPM analysis for different patient populations, brain PET template images representing population-specific brain morphometry and metabolism features are helpful. However, most currently available brain PET templates were constructed using the Caucasian data. To enrich the family of publicly available brain PET templates, we created Chinese-specific template images based on 116 [¹⁸F]-fluorodeoxyglucose ([¹⁸F]-FDG) PET images of normal participants. These images were warped into a common averaged space, in which the mean and standard deviation templates were both computed. We also developed the SPM analysis programmes to facilitate easy use of the templates. Our templates were validated through the SPM analysis of Alzheimer’s and Parkinson’s patient images. The resultant SPM t-maps accurately depicted the disease-related brain regions with abnormal [¹⁸F]-FDG uptake, proving the templates’ effectiveness in brain function impairment analysis.

Measurement(s)	brain metabolism measurement
Technology Type(s)	FDG-Positron Emission Tomography
Factor Type(s)	age • sex
Sample Characteristic - Organism	Homo sapiens
Sample Characteristic - Location	China

Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.16382418

A new machine learning approach for predicting likelihood of recurrence following ablation for atrial fibrillation from CT

OBJECTIVE

To investigate left atrial shape differences on CT scans of atrial fibrillation (AF) patients with (AF+) versus without (AF-) post-ablation recurrence and whether these shape differences predict AF recurrence.

METHODS

This retrospective study included 68 AF patients who had pre-catheter ablation cardiac CT scans with contrast. AF recurrence was defined at 1 year, excluding a 3-month post-ablation blanking period. After creating atlases of atrial models from segmented AF+ and AF- CT images, an atlas-based implicit shape differentiation method was used to identify surface of interest (SOI). After registering the SOI to each patient model, statistics of the deformation on the SOI were used to create shape descriptors. The performance in predicting AF recurrence using shape features at and outside the SOI and eight clinical factors (age, sex, left atrial volume, left ventricular ejection fraction, body mass index, sinus rhythm, and AF type [persistent vs paroxysmal], catheter-ablation type [Cryoablation vs Irrigated RF]) were compared using 100 runs of fivefold cross validation.

RESULTS

Differences in atrial shape were found surrounding the pulmonary vein ostia and the base of the left atrial appendage. In the prediction of AF recurrence, the area under the receiver-operating characteristics curve (AUC) was 0.67 for shape features from the SOI, 0.58 for shape features outside the SOI, 0.71 for the clinical parameters, and 0.78 combining shape and clinical features.

CONCLUSION

Differences in left atrial shape were identified between AF recurrent and non-recurrent patients using pre-procedure CT scans. New radiomic features corresponding to the differences in shape were found to predict post-ablation AF recurrence.

Voxel-based identification of local recurrence sub-regions from pre-treatment PET/CT for locally advanced head and neck cancers

Background

Overall, 40% of patients with a locally advanced head and neck cancer (LAHNC) treated by chemoradiotherapy (CRT) present local recurrence within 2 years after the treatment. The aims of this study were to characterize voxel-wise the sub-regions where tumor recurrence appear and to predict their location from pre-treatment ¹⁸F-fluorodeoxyglucose (FDG) positron emission tomography (PET) images.

Materials and methods

Twenty-six patients with local failure after treatment were included in this study. Local recurrence volume was identified by co-registering pre-treatment and recurrent PET/CT images using a customized rigid registration algorithm. A large set of voxel-wise features were extracted from pre-treatment PET to train a random forest model allowing to predict local recurrence at the voxel level.

Results

Out of 26 expert-assessed registrations, 15 provided enough accuracy to identify recurrence volumes and were included for further analysis. Recurrence volume represented on average 23% of the initial tumor volume. The MTV with a threshold of 50% of SUVmax plus a 3D margin of 10 mm covered on average 89.8% of the recurrence and 96.9% of the initial tumor. SUV and MTV alone were not sufficient to identify the area of recurrence. Using a random forest model, 15 parameters, combining radiomics and spatial location, were identified, allowing to predict the recurrence sub-regions with a median area under the receiver operating curve of 0.71 (range 0.14–0.91).

Conclusion

As opposed to regional comparisons which do not bring enough evidence for accurate prediction of recurrence volume, a voxel-wise analysis of FDG-uptake features suggested a potential to predict recurrence with enough accuracy to consider tailoring CRT by dose escalation within likely radioresistant regions.

VERDICT-AMICO: Ultrafast fitting algorithm for non-invasive prostate microstructure characterization

VERDICT (vascular, extracellular and restricted diffusion for cytometry in tumours) estimates and maps microstructural features of cancerous tissue non-invasively using diffusion MRI. The main purpose of this study is to address the high computational time of microstructural model fitting for prostate diagnosis, while retaining utility in terms of tumour conspicuity and repeatability. In this work, we adapt the accelerated microstructure imaging via convex optimization (AMICO) framework to linearize the estimation of VERDICT parameters for the prostate gland. We compare the original non-linear fitting of VERDICT with the linear fitting, quantifying accuracy with synthetic data, and computational time and reliability (performance and precision) in eight patients. We also assess the repeatability (scan-rescan) of the parameters. Comparison of the original VERDICT fitting versus VERDICT-AMICO showed that the linearized fitting (1) is more accurate in simulation for a signal-to-noise ratio of 20 dB; (2) reduces the processing time by three orders of magnitude, from 6.55 seconds/voxel to 1.78 milliseconds/voxel; (3) estimates parameters more precisely; (4) produces similar parametric maps and (5) produces similar estimated parameters with a high Pearson correlation between implementations, r2  > 0.7. The VERDICT-AMICO estimates also show high levels of repeatability. Finally, we demonstrate that VERDICT-AMICO can estimate an extra diffusivity parameter without losing tumour conspicuity and retains the fitting advantages. VERDICT-AMICO provides microstructural maps for prostate cancer characterization in seconds.

Cryo-Imaging and Software Platform for Analysis of Molecular MR Imaging of Micrometastases

We created and evaluated a preclinical, multimodality imaging, and software platform to assess molecular imaging of small metastases. This included experimental methods (e.g., GFP-labeled tumor and high resolution multispectral cryo-imaging), nonrigid image registration, and interactive visualization of imaging agent targeting. We describe technological details earlier applied to GFP-labeled metastatic tumor targeting by molecular MR (CREKA-Gd) and red fluorescent (CREKA-Cy5) imaging agents. Optimized nonrigid cryo-MRI registration enabled nonambiguous association of MR signals to GFP tumors. Interactive visualization of out-of-RAM volumetric image data allowed one to zoom to a GFP-labeled micrometastasis, determine its anatomical location from color cryo-images, and establish the presence/absence of targeted CREKA-Gd and CREKA-Cy5. In a mouse with >160 GFP-labeled tumors, we determined that in the MR images every tumor in the lung >0.3 mm² had visible signal and that some metastases as small as 0.1 mm² were also visible. More tumors were visible in CREKA-Cy5 than in CREKA-Gd MRI. Tape transfer method and nonrigid registration allowed accurate (<11 μm error) registration of whole mouse histology to corresponding cryo-images. Histology showed inflammation and necrotic regions not labeled by imaging agents. This mouse-to-cells multiscale and multimodality platform should uniquely enable more informative and accurate studies of metastatic cancer imaging and therapy.

Classifying Glioblastoma Multiforme Follow-Up Progressive vs. Responsive Forms Using Multi-Parametric MRI Features

Purpose: The purpose of this paper is discriminating between tumor progression and response to treatment based on follow-up multi-parametric magnetic resonance imaging (MRI) data retrieved from glioblastoma multiforme (GBM) patients.

Materials and Methods: Multi-parametric MRI data consisting of conventional MRI (cMRI) and advanced MRI [i.e., perfusion weighted MRI (PWI) and diffusion kurtosis MRI (DKI)] were acquired from 29 GBM patients treated with adjuvant therapy after surgery. We propose an automatic pipeline for processing advanced MRI data and extracting intensity-based histogram features and 3-D texture features using manually and semi-manually delineated regions of interest (ROIs). Classifiers are trained using a leave-one-patient-out cross validation scheme on complete MRI data. Balanced accuracy rate (BAR)–values are computed and compared between different ROIs, MR modalities, and classifiers, using non-parametric multiple comparison tests.

Results: Maximum BAR–values using manual delineations are 0.956, 0.85, 0.879, and 0.932, for cMRI, PWI, DKI, and all three MRI modalities combined, respectively. Maximum BAR–values using semi-manual delineations are 0.932, 0.894, 0.885, and 0.947, for cMRI, PWI, DKI, and all three MR modalities combined, respectively. After statistical testing using Kruskal-Wallis and post-hoc Dunn-Šidák analysis we conclude that training a RUSBoost classifier on features extracted using semi-manual delineations on cMRI or on all MRI modalities combined performs best.

Conclusions: We present two main conclusions: (1) using T1 post-contrast (T1pc) features extracted from manual total delineations, AdaBoost achieves the highest BAR–value, 0.956; (2) using T1pc-average, T1pc-90th percentile, and Cerebral Blood Volume (CBV) 90th percentile extracted from semi-manually delineated contrast enhancing ROIs, SVM-rbf, and RUSBoost achieve BAR–values of 0.947 and 0.932, respectively. Our findings show that AdaBoost, SVM-rbf, and RUSBoost trained on T1pc and CBV features can differentiate progressive from responsive GBM patients with very high accuracy.

A logarithmic opinion pool based STAPLE algorithm for the fusion of segmentations with associated reliability weights

Pelvic floor dysfunction is common in women after childbirth and precise segmentation of magnetic resonance images (MRI) of the pelvic floor may facilitate diagnosis and treatment of patients. However, because of the complexity of its structures, manual segmentation of the pelvic floor is challenging and suffers from high inter and intra-rater variability of expert raters. Multiple template fusion algorithms are promising segmentation techniques for these types of applications, but they have been limited by imperfections in the alignment of templates to the target, and by template segmentation errors. A number of algorithms sought to improve segmentation performance by combining image intensities and template labels as two independent sources of information, carrying out fusion through local intensity weighted voting schemes. This class of approach is a form of linear opinion pooling, and achieves unsatisfactory performance for this application. We hypothesized that better decision fusion could be achieved by assessing the contribution of each template in comparison to a reference standard segmentation of the target image and developed a novel segmentation algorithm to enable automatic segmentation of MRI of the female pelvic floor. The algorithm achieves high performance by estimating and compensating for both imperfect registration of the templates to the target image and template segmentation inaccuracies. A local image similarity measure is used to infer a local reliability weight, which contributes to the fusion through a novel logarithmic opinion pooling. We evaluated our new algorithm in comparison to nine state-of-the-art segmentation methods and demonstrated our algorithm achieves the highest performance.

Is your system calibrated? MRI gradient system calibration for pre-clinical, high-resolution imaging

High-field, pre-clinical MRI systems are widely used to characterise tissue structure and volume in small animals, using high resolution imaging. Both applications rely heavily on the consistent, accurate calibration of imaging gradients, yet such calibrations are typically only performed during maintenance sessions by equipment manufacturers, and potentially with acceptance limits that are inadequate for phenotyping. To overcome this difficulty, we present a protocol for gradient calibration quality assurance testing, based on a 3D-printed, open source, structural phantom that can be customised to the dimensions of individual scanners and RF coils. In trials on a 9.4 T system, the gradient scaling errors were reduced by an order of magnitude, and displacements of greater than 100 µm, caused by gradient non-linearity, were corrected using a post-processing technique. The step-by-step protocol can be integrated into routine pre-clinical MRI quality assurance to measure and correct for these errors. We suggest that this type of quality assurance is essential for robust pre-clinical MRI experiments that rely on accurate imaging gradients, including small animal phenotyping and diffusion MR.

Semi-automatic brain tumor segmentation by constrained MRFs using structural trajectories

Quantifying volume and growth of a brain tumor is a primary prognostic measure and hence has received much attention in the medical imaging community. Most methods have sought a fully automatic segmentation, but the variability in shape and appearance of brain tumor has limited their success and further adoption in the clinic. In reaction, we present a semi-automatic brain tumor segmentation framework for multi-channel magnetic resonance (MR) images. This framework does not require prior model construction and only requires manual labels on one automatically selected slice. All other slices are labeled by an iterative multi-label Markov random field optimization with hard constraints. Structural trajectories-the medical image analog to optical flow and 3D image over-segmentation are used to capture pixel correspondences between consecutive slices for pixel labeling. We show robustness and effectiveness through an evaluation on the 2012 MICCAI BRATS Challenge Dataset; our results indicate superior performance to baselines and demonstrate the utility of the constrained MRF formulation.

Robust nonrigid registration to capture brain shift from intraoperative MRI

We present a new algorithm to register 3-D preoperative magnetic resonance (MR) images to intraoperative MR images of the brain which have undergone brain shift. This algorithm relies on a robust estimation of the deformation from a sparse noisy set of measured displacements. We propose a new framework to compute the displacement field in an iterative process, allowing the solution to gradually move from an approximation formulation (minimizing the sum of a regularization term and a data error term) to an interpolation formulation (least square minimization of the data error term). An outlier rejection step is introduced in this gradual registration process using a weighted least trimmed squares approach, aiming at improving the robustness of the algorithm. We use a patient-specific model discretized with the finite element method in order to ensure a realistic mechanical behavior of the brain tissue. To meet the clinical time constraint, we parallelized the slowest step of the algorithm so that we can perform a full 3-D image registration in 35 s (including the image update time) on a heterogeneous cluster of 15 personal computers. The algorithm has been tested on six cases of brain tumor resection, presenting a brain shift of up to 14 mm. The results show a good ability to recover large displacements, and a limited decrease of accuracy near the tumor resection cavity.

Registration of two-dimensional cardiac images to preprocedural three-dimensional images for interventional applications

PURPOSE

To evaluate the accuracy and efficiency of rigid-body registration of two-dimensional fast cine and real-time cardiac images to high-resolution and SNR three-dimensional preprocedural reference volumes for application during MRI-guided interventional procedures.

MATERIALS AND METHODS

Mutual information (MI) and correlation ratio (CR) similarity measures were evaluated. The dependence of registration accuracy and efficiency on different resolution and SNR parameters, and also on cardiac-phase differences was evaluated in a porcine model. Two-dimensional images were initially misoriented at distances (d) of 2-10 mm, and rotations of +/-5 degrees about all axes. Registration error and computation time were evaluated, and performance was also assessed visually.

RESULTS

The maximum registration error using MI (<2.7 mm and <3.6 degrees ) occurred for d = 10 mm, misrotation of +/-5 degrees , and relative SNR = 1. The computation time was 15 seconds for MI and 10 seconds for CR.

CONCLUSION

Registration accuracy was not highly dependent on the relative timing, within the cycle, between the two-dimensional and three-dimensional images. Registration using CR was faster than that using MI, although accuracy was marginally higher with MI. J.

Strategies for brain shift evaluation

For the analysis of the brain shift phenomenon different strategies were applied. In 32 glioma cases pre- and intraoperative MR datasets were acquired in order to evaluate the maximum displacement of the brain surface and the deep tumor margin. After rigid registration using the software of the neuronavigation system, a direct comparison was made with 2D- and 3D visualizations. As a result, a great variability of the brain shift was observed ranging up to 24 mm for cortical displacement and exceeding 3 mm for the deep tumor margin in 66% of all cases. Following intraoperative imaging the neuronavigation system was updated in eight cases providing reliable guidance. For a more comprehensive analysis a voxel-based nonlinear registration was applied. Aiming at improved speed of alignment we performed all interpolation operations with 3D texture mapping based on OpenGL functions supported in graphics hardware. Further acceleration was achieved with an adaptive refinement of the underlying control point grid focusing on the main deformation areas. For a quick overview the registered datasets were evaluated with different 3D visualization approaches. Finally, the results were compared to the initial measurements contributing to a better understanding of the brain shift phenomenon. Overall, the experiments clearly demonstrate that deformations of the brain surface and deeper brain structures are uncorrelated.