A neural network-based 2D/3D image registration quality evaluator for pediatric patient setup in external beam radiotherapy

Ultrasound Frame-to-Volume Registration via Deep Learning for Interventional Guidance

Fusing intraoperative 2-D ultrasound (US) frames with preoperative 3-D magnetic resonance (MR) images for guiding interventions has become the clinical gold standard in image-guided prostate cancer biopsy. However, developing an automatic image registration system for this application is challenging because of the modality gap between US/MR and the dimensionality gap between 2-D/3-D data. To overcome these challenges, we propose a novel US frame-to-volume registration (FVReg) pipeline to bridge the dimensionality gap between 2-D US frames and 3-D US volume. The developed pipeline is implemented using deep neural networks, which are fully automatic without requiring external tracking devices. The framework consists of three major components, including one) a frame-to-frame registration network (Frame2Frame) that estimates the current frame's 3-D spatial position based on previous video context, two) a frame-to-slice correction network (Frame2Slice) adjusting the estimated frame position using the 3-D US volumetric information, and three) a similarity filtering (SF) mechanism selecting the frame with the highest image similarity with the query frame. We validated our method on a clinical dataset with 618 subjects and tested its potential on real-time 2-D-US to 3-D-MR fusion navigation tasks. The proposed FVReg achieved an average target navigation error of 1.93 mm at 5-14 fps. Our source code is publicly available at https://github.com/DIAL-RPI/Frame-to-Volume-Registration.

Image preprocessing to improve the accuracy and robustness of mutual-information-based automatic image registration in proton therapy

PURPOSE

We propose a method that potentially improves the outcome of mutual-information-based automatic image registration by using the contrast enhancement filter (CEF).

METHODS

Seventy-six pairs of two-dimensional X-ray images and digitally reconstructed radiographs for 20 head and neck and nine lung cancer patients were analyzed retrospectively. Automatic image registration was performed using the mutual-information-based algorithm in VeriSuite®. Images were preprocessed using the CEF in VeriSuite®. The correction vector for translation and rotation error was calculated and manual image registration was compared with automatic image registration, with and without CEF. In addition, the normalized mutual information (NMI) distribution between two-dimensional images was compared, with and without CEF.

RESULTS

In the correction vector comparison between manual and automatic image registration, the average differences in translation error were < 1 mm in most cases in the head and neck region. The average differences in rotation error were 0.71 and 0.16 degrees without and with CEF, respectively, in the head and neck region; they were 2.67 and 1.64 degrees, respectively, in the chest region. When used with oblique projection, the average rotation error was 0.39 degrees with CEF. CEF improved the NMI by 17.9 % in head and neck images and 18.2 % in chest images.

CONCLUSIONS

CEF preprocessing improved the NMI and registration accuracy of mutual-information-based automatic image registration on the medical images. The proposed method achieved accuracy equivalent to that achieved by experienced therapists and it will significantly contribute to the standardization of image registration quality.

Estimating medical image registration error and confidence: A taxonomy and scoping review

Given that image registration is a fundamental and ubiquitous task in both clinical and research domains of the medical field, errors in registration can have serious consequences. Since such errors can mislead clinicians during image-guided therapies or bias the results of a downstream analysis, methods to estimate registration error are becoming more popular. To give structure to this new heterogenous field we developed a taxonomy and performed a scoping review of methods that quantitatively and automatically provide a dense estimation of registration error. The taxonomy breaks down error estimation methods into Approach (Image- or Transformation-based), Framework (Machine Learning or Direct) and Measurement (error or confidence) components. Following the PRISMA guidelines for scoping reviews, the 570 records found were reduced to twenty studies that met inclusion criteria, which were then reviewed according to the proposed taxonomy. Trends in the field, advantages and disadvantages of the methods, and potential sources of bias are also discussed. We provide suggestions for best practices and identify areas of future research.

Artificial Intelligence Applications in Pediatric Brain Tumor Imaging: A Systematic Review

OBJECTIVE

Artificial intelligence (AI) has facilitated the analysis of medical imaging given increased computational capacity and medical data availability in recent years. Although many applications for AI in the imaging of brain tumors have been proposed, their potential clinical impact remains to be explored. A systematic review was performed to examine the role of AI in the analysis of pediatric brain tumor imaging.

METHODS

PubMed, Embase, and Scopus were searched for relevant articles up to January 27, 2021.

RESULTS

Literature search identified 298 records, of which 22 studies were included. The most commonly studied tumors were posterior fossa tumors including brainstem glioma, ependymoma, medulloblastoma, and pilocytic astrocytoma (15, 68%). Tumor diagnosis was the most frequently performed task (14, 64%), followed by tumor segmentation (3, 14%) and tumor detection (3, 14%). Of the 6 studies comparing AI to clinical experts, 5 demonstrated superiority of AI for tumor diagnosis. Other tasks including tumor segmentation, attenuation correction of positron emission tomography scans, image registration for patient positioning, and dose calculation for radiotherapy were performed with high accuracy comparable with clinical experts. No studies described use of the AI tool in routine clinical practice.

CONCLUSIONS

AI methods for analysis of pediatric brain tumor imaging have increased exponentially in recent years. However, adoption of these methods in clinical practice requires further characterization of validity and utility. Implementation of these methods may streamline clinical workflows by improving diagnostic accuracy and automating basic imaging analysis tasks.

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective

Image-based navigation is widely considered the next frontier of minimally invasive surgery. It is believed that image-based navigation will increase the access to reproducible, safe, and high-precision surgery as it may then be performed at acceptable costs and effort. This is because image-based techniques avoid the need of specialized equipment and seamlessly integrate with contemporary workflows. Furthermore, it is expected that image-based navigation techniques will play a major role in enabling mixed reality environments, as well as autonomous and robot-assisted workflows. A critical component of image guidance is 2D/3D registration, a technique to estimate the spatial relationships between 3D structures, e.g., preoperative volumetric imagery or models of surgical instruments, and 2D images thereof, such as intraoperative X-ray fluoroscopy or endoscopy. While image-based 2D/3D registration is a mature technique, its transition from the bench to the bedside has been restrained by well-known challenges, including brittleness with respect to optimization objective, hyperparameter selection, and initialization, difficulties in dealing with inconsistencies or multiple objects, and limited single-view performance. One reason these challenges persist today is that analytical solutions are likely inadequate considering the complexity, variability, and high-dimensionality of generic 2D/3D registration problems. The recent advent of machine learning-based approaches to imaging problems that, rather than specifying the desired functional mapping, approximate it using highly expressive parametric models holds promise for solving some of the notorious challenges in 2D/3D registration. In this manuscript, we review the impact of machine learning on 2D/3D registration to systematically summarize the recent advances made by introduction of this novel technology. Grounded in these insights, we then offer our perspective on the most pressing needs, significant open problems, and possible next steps.