Survey of Non-Rigid Registration Tools in Medicine

An operator-independent quality assurance system for automatically generated structure sets

Objective. This study describes geometry-based and intensity-based tools for quality assurance (QA) of automatically generated structures for online adaptive radiotherapy, and designs an operator-independent traffic light system that identifies erroneous structure sets.Approach.A cohort of eight head and neck (HN) patients with daily CBCTs was selected for test development. Radiotherapy contours were propagated from planning computed tomography (CT) to daily cone beam CT (CBCT) using deformable image registration. These propagated structures were visually verified for acceptability. For each CBCT, several error scenarios were used to generate what were judged unacceptable structures. Ten additional HN patients with daily CBCTs and different error scenarios were selected for validation. A suite of tests based on image intensity, intensity gradient, and structure geometry was developed using acceptable and unacceptable HN planning structures. Combinations of one test applied to one structure, referred to as structure-test combinations, were selected for inclusion in the QA system based on their discriminatory power. A traffic light system was used to aggregate the structure-test combinations, and the system was evaluated on all fractions of the ten validation HN patients.Results.The QA system distinguished between acceptable and unacceptable fractions with high accuracy, labeling 294/324 acceptable fractions as green or yellow and 19/20 unacceptable fractions as yellow or red.Significance.This study demonstrates a system to supplement manual review of radiotherapy planning structures. Automated QA is performed by aggregating results from multiple intensity- and geometry-based tests.

MRI non-rigid registration with tumor contraction correction for ablative margin assessment after microwave ablation of hepatocellular carcinomas

Objective. This study aims to develop and assess a tumor contraction model, enhancing the precision of ablative margin (AM) evaluation after microwave ablation (MWA) treatment for hepatocellular carcinomas (HCCs).Approach. We utilize a probabilistic method called the coherent point drift algorithm to align pre-and post-ablation MRI images. Subsequently, a nonlinear regression method quantifies local tumor contraction induced by MWA, utilizing data from 47 HCC with viable ablated tumors in post-ablation MRI. After automatic non-rigid registration, correction for tumor contraction involves contracting the 3D contour of the warped tumor towards its center in all orientations.Main results. We evaluate the performance of our proposed method on 30 HCC patients who underwent MWA. The Dice similarity coefficient between the post-ablation liver and the warped pre-ablation livers is found to be 0.95 ± 0.01, with a mean corresponding distance between the corresponding landmarks measured at 3.25 ± 0.62 mm. Additionally, we conduct a comparative analysis of clinical outcomes assessed through MRI over a 3 month follow-up period, noting that the AM, as evaluated by our proposed method, accurately detects residual tumor after MWA.Significance. Our proposed method showcases a high level of accuracy in MRI liver registration and AM assessment following ablation treatment. It introduces a potentially approach for predicting incomplete ablations and gauging treatment success.

Comparison of consecutive and restained sections for image registration in histopathology

Significance

Although the registration of restained sections allows nucleus-level alignment that enables a direct analysis of interacting biomarkers, consecutive sections only allow the transfer of region-level annotations. The latter can be achieved at low computational cost using coarser image resolutions.

Purpose

In digital histopathology, virtual multistaining is important for diagnosis and biomarker research. Additionally, it provides accurate ground truth for various deep-learning tasks. Virtual multistaining can be obtained using different stains for consecutive sections or by restaining the same section. Both approaches require image registration to compensate for tissue deformations, but little attention has been devoted to comparing their accuracy.

Approach

We compared affine and deformable variational image registration of consecutive and restained sections and analyzed the effect of the image resolution that influences accuracy and required computational resources. The registration was applied to the automatic nonrigid histological image registration (ANHIR) challenge data (230 consecutive slide pairs) and the hyperparameters were determined. Then without changing the parameters, the registration was applied to a newly published hybrid dataset of restained and consecutive sections (HyReCo, 86 slide pairs, 5404 landmarks).

Results

We obtain a median landmark error after registration of 6.5    μ m (HyReCo) and 24.1    μ m (ANHIR) between consecutive sections. Between restained sections, the median registration error is 2.2 and 0.9    μ m in the two subsets of the HyReCo dataset. We observe that deformable registration leads to lower landmark errors than affine registration in both cases (p < 0.001 ), though the effect is smaller in restained sections.

Conclusion

Deformable registration of consecutive and restained sections is a valuable tool for the joint analysis of different stains.

An image-based modeling framework for predicting spatiotemporal brain cancer biology within individual patients

Imaging is central to the clinical surveillance of brain tumors yet it provides limited insight into a tumor's underlying biology. Machine learning and other mathematical modeling approaches can leverage paired magnetic resonance images and image-localized tissue samples to predict almost any characteristic of a tumor. Image-based modeling takes advantage of the spatial resolution of routine clinical scans and can be applied to measure biological differences within a tumor, changes over time, as well as the variance between patients. This approach is non-invasive and circumvents the intrinsic challenges of inter- and intratumoral heterogeneity that have historically hindered the complete assessment of tumor biology and treatment responsiveness. It can also reveal tumor characteristics that may guide both surgical and medical decision-making in real-time. Here we describe a general framework for the acquisition of image-localized biopsies and the construction of spatiotemporal radiomics models, as well as case examples of how this approach may be used to address clinically relevant questions.

Non-Rigid Registration for High-Resolution Retinal Imaging

Adaptive optics provides improved resolution in ophthalmic imaging when retinal microstructures need to be identified, counted, and mapped. In general, multiple images are averaged to improve the signal-to-noise ratio or analyzed for temporal dynamics. Image registration by cross-correlation is straightforward for small patches; however, larger images require more sophisticated registration techniques. Strip-based registration has been used successfully for photoreceptor mosaic alignment in small patches; however, if the deformations along strips are not simple displacements, averaging can degrade the final image. We have applied a non-rigid registration technique that improves the quality of processed images for mapping cones over large image patches. In this approach, correction of local deformations compensates for local image stretching, compressing, bending, and twisting due to a number of causes. The main result of this procedure is improved definition of retinal microstructures that can be better identified and segmented. Derived metrics such as cone density, wall-to-lumen ratio, and quantification of structural modification of blood vessel walls have diagnostic value in many retinal diseases, including diabetic retinopathy and age-related macular degeneration, and their improved evaluations may facilitate early diagnostics of retinal diseases.

Serial intravital 2-photon microscopy and analysis of the kidney using upright microscopes

Serial intravital 2-photon microscopy of the kidney and other abdominal organs is a powerful technique to assess tissue function and structure simultaneously and over time. Thus, serial intravital microscopy can capture dynamic tissue changes during health and disease and holds great potential to characterize (patho-) physiological processes with subcellular resolution. However, successful image acquisition and analysis require significant expertise and impose multiple potential challenges. Abdominal organs are rhythmically displaced by breathing movements which hamper high-resolution imaging. Traditionally, kidney intravital imaging is performed on inverted microscopes where breathing movements are partly compensated by the weight of the animal pressing down. Here, we present a custom and easy-to-implement setup for intravital imaging of the kidney and other abdominal organs on upright microscopes. Furthermore, we provide image processing protocols and a new plugin for the free image analysis software FIJI to process multichannel fluorescence microscopy data. The proposed image processing pipelines cover multiple image denoising algorithms, sample drift correction using 2D registration, and alignment of serial imaging data collected over several weeks using landmark-based 3D registration. The provided tools aim to lower the barrier of entry to intravital microscopy of the kidney and are readily applicable by biomedical practitioners.

Zebrafish neuromesodermal progenitors undergo a critical state transition in vivo

The transition state model of cell differentiation proposes that a transient window of gene expression stochasticity precedes entry into a differentiated state. Here, we assess this theoretical model in zebrafish neuromesodermal progenitors (NMps) in vivo during late somitogenesis stages. We observed an increase in gene expression variability at the 24 somite stage (24ss) before their differentiation into spinal cord and paraxial mesoderm. Analysis of a published 18ss scRNA-seq dataset showed that the NMp population is noisier than its derivatives. By building in silico composite gene expression maps from image data, we assigned an 'NM index' to in silico NMps based on the expression of neural and mesodermal markers and demonstrated that cell population heterogeneity peaked at 24ss. Further examination revealed cells with gene expression profiles incongruent with their prospective fate. Taken together, our work supports the transition state model within an endogenous cell fate decision making event.

A Novel Method for Observing Tumor Margin in Hepatoblastoma Based on Microstructure 3D Reconstruction

Objective: We investigated three-dimensional (3 D) reconstruction for the assessment of the tumor margin microstructure of hepatoblastoma (HB). Methods: Eleven surgical resections of childhood hepatoblastomas obtained between September 2018 and December 2019 were formalin-fixed, paraffin-embedded, serially sectioned at 4 μm, stained with hematoxylin and eosin (every 19th and 20th section stained with alpha-fetoprotein and glypican 3), and the digital images of all sections were acquired at 100× followed by image registration using the B-spline based method with modified residual complexity. Reconstruction was performed using 3 D Slicer software. Results: The reconstructed orthogonal 3 D images clearly presented the internal microstructure of the tumor margin. The rendered 3 D image could be rotated at any angle. Conclusions: Microstructure 3 D reconstruction is feasible for observing the pathological structure of the HB tumor margin.

Dynamic changes in liver volume calculated using a three-dimensional visualization system after microwave ablation of hepatocellular carcinomas

OBJECTIVES

To investigate the changes in liver volume and function after microwave ablation (MWA) of hepatocellular carcinomas (HCCs).

MATERIALS AND METHODS

We retrospectively analyzed 76 patients with 106 nodules who underwent MWA for HCCs ≤5 cm between January 2015 and September 2017. Liver and ablation volumes were calculated using a three-dimensional visualization system on MRI. Multiple regression analysis was used to estimate the association between the ablation volume and liver volume changes. Deformable image registration (DIR) was performed to confirm the influence of liver volume changes on curative effect evaluation after ablation.

RESULTS

The initial liver and tumor volumes were 1262.1±259.91 cm³ (range: 864.9∼1966.8) and 2.5 cm³ (interquartile range [IQR]: 1.3∼8.8), respectively. Compared to the initial liver volumes, the entire live volume (ELV) increased by 10.1%±8.93% (range: -4.9%∼46.68%) on the 3rd day after ablation. Subsequently, it recovered to initial level at the 3rd month and maintained its level during the 1-year follow-up. The median total ablation volume was 34.9 cm³ (IQR: 20.4∼65.4) on the 3rd day after ablation, which decreased by 71.2% (IQR: 57.4%∼78.1%) one year after ablation. Alanine aminotransferase (ALT), aspartate aminotransferase (AST), and total bilirubin (T-Bil) peaked within 3 days after MWA and recovered to normal within 1 month. The ablation volume proportion of the ELV was an independent risk factor for the increase in the ELV and AST, ALT, and T-Bil levels within 3 days after ablation. When DIR was conducted to fuse ablation zone and tumor, the reshaped tumor volumes were enlarged by 40% because of the increase in ELV.

CONCLUSIONS

MWA of HCCs based on the Milan criteria could induce temporary increases in ELV and RLV within 3 days after ablation, but both parameters recovered to initial levels 3 months after ablation. This indicates that MWA of early-stage HCCs would not lead to liver volume loss and could potentially protect liver function. The liver cannot be treated as an incompressible organ after ablation, and the appropriate deformation constraint should be designed for DIR to evaluate ablation margin accurately. This article is protected by copyright. All rights reserved.

A Baboon Brain Atlas for Magnetic Resonance Imaging and Positron Emission Tomography Image Analysis

The olive baboon (Papio anubis) is phylogenetically proximal to humans. Investigation into the baboon brain has shed light on the function and organization of the human brain, as well as on the mechanistic insights of neurological disorders such as Alzheimer's and Parkinson's. Non-invasive brain imaging, including positron emission tomography (PET) and magnetic resonance imaging (MRI), are the primary outcome measures frequently used in baboon studies. PET functional imaging has long been used to study cerebral metabolic processes, though it lacks clear and reliable anatomical information. In contrast, MRI provides a clear definition of soft tissue with high resolution and contrast to distinguish brain pathology and anatomy, but lacks specific markers of neuroreceptors and/or neurometabolites. There is a need to create a brain atlas that combines the anatomical and functional/neurochemical data independently available from MRI and PET. For this purpose, a three-dimensional atlas of the olive baboon brain was developed to enable multimodal imaging analysis. The atlas was created on a population-representative template encompassing 89 baboon brains. The atlas defines 24 brain regions, including the thalamus, cerebral cortex, putamen, corpus callosum, and insula. The atlas was evaluated with four MRI images and 20 PET images employing the radiotracers for [11C]benzamide, [11C]metergoline, [18F]FAHA, and [11C]rolipram, with and without structural aids like [18F]flurodeoxyglycose images. The atlas-based analysis pipeline includes automated segmentation, registration, quantification of region volume, the volume of distribution, and standardized uptake value. Results showed that, in comparison to PET analysis utilizing the "gold standard" manual quantification by neuroscientists, the performance of the atlas-based analysis was at >80 and >70% agreement for MRI and PET, respectively. The atlas can serve as a foundation for further refinement, and incorporation into a high-throughput workflow of baboon PET and MRI data. The new atlas is freely available on the Figshare online repository (https://doi.org/10.6084/m9.figshare.16663339), and the template images are available from neuroImaging tools & resources collaboratory (NITRC) (https://www.nitrc.org/projects/haiko89/).

Collection and Reporting of Patient-reported Outcome Measures in Arthroplasty Registries: Multinational Survey and Recommendations

BACKGROUND

Patient-reported outcome measures (PROMs) are validated questionnaires that are completed by patients. Arthroplasty registries vary in PROM collection and use. Current information about registry collection and use of PROMs is important to help improve methods of PROM data analysis, reporting, comparison, and use toward improving clinical practice.

QUESTIONS/PURPOSES

To characterize PROM collection and use by registries, we asked: (1) What is the current practice of PROM collection by arthroplasty registries that are current or former members of the International Society of Arthroplasty Registries, and are there sufficient similarities in PROM collection between registries to enable useful international comparisons that could inform the improvement of arthroplasty care? (2) How do registries differ in PROM administration and demographic, clinical, and comorbidity index variables collected for case-mix adjustment in data analysis and reporting? (3) What quality assurance methods are used for PROMs, and how are PROM results reported and used by registries? (4) What recommendations to arthroplasty registries may improve PROM reporting and facilitate international comparisons?

METHODS

An electronic survey was developed with questions about registry structure and collection, analysis, reporting, and use of PROM data and distributed to directors or senior administrators of 39 arthroplasty registries that were current or former members of the International Society of Arthroplasty Registries. In all, 64% (25 of 39) of registries responded and completed the survey. Missing responses from incomplete surveys were captured by contacting the registries, and up to three reminder emails were sent to nonresponding registries. Recommendations about PROM collection were drafted, revised, and approved by the International Society of Arthroplasty Registries PROMs Working Group members.

RESULTS

Of the 25 registries that completed the survey, 15 collected generic PROMs, most frequently the EuroQol-5 Dimension survey; 16 collected joint-specific PROMs, most frequently the Knee Injury and Osteoarthritis Outcome Score and Hip Disability and Osteoarthritis Outcome Score; and 11 registries collected a satisfaction item. Most registries administered PROM questionnaires within 3 months before and 1 year after surgery. All 16 registries that collected PROM data collected patient age, sex or gender, BMI, indication for the primary arthroplasty, reason for revision arthroplasty, and a comorbidity index, most often the American Society of Anesthesiologists classification. All 16 registries performed regular auditing and reporting of data quality, and most registries reported PROM results to hospitals and linked PROM data to other data sets such as hospital, medication, billing, and emergency care databases. Recommendations for transparent reporting of PROMs were grouped into four categories: demographic and clinical, survey administration, data analysis, and results.

CONCLUSION

Although registries differed in PROM collection and use, there were sufficient similarities that may enable useful data comparisons. The International Society of Arthroplasty Registries PROMs Working Group recommendations identify issues that may be important to most registries such as the need to make decisions about survey times and collection methods, as well as how to select generic and joint-specific surveys, handle missing data and attrition, report data, and ensure representativeness of the sample.

CLINICAL RELEVANCE

By collecting PROMs, registries can provide patient-centered data to surgeons, hospitals, and national entities to improve arthroplasty care.

CT-Based Pelvic T1-Weighted MR Image Synthesis Using UNet, UNet++ and Cycle-Consistent Generative Adversarial Network (Cycle-GAN)

BACKGROUND

Computed tomography (CT) and magnetic resonance imaging (MRI) are the mainstay imaging modalities in radiotherapy planning. In MR-Linac treatment, manual annotation of organs-at-risk (OARs) and clinical volumes requires a significant clinician interaction and is a major challenge. Currently, there is a lack of available pre-annotated MRI data for training supervised segmentation algorithms. This study aimed to develop a deep learning (DL)-based framework to synthesize pelvic T1-weighted MRI from a pre-existing repository of clinical planning CTs.

METHODS

MRI synthesis was performed using UNet++ and cycle-consistent generative adversarial network (Cycle-GAN), and the predictions were compared qualitatively and quantitatively against a baseline UNet model using pixel-wise and perceptual loss functions. Additionally, the Cycle-GAN predictions were evaluated through qualitative expert testing (4 radiologists), and a pelvic bone segmentation routine based on a UNet architecture was trained on synthetic MRI using CT-propagated contours and subsequently tested on real pelvic T1 weighted MRI scans.

RESULTS

In our experiments, Cycle-GAN generated sharp images for all pelvic slices whilst UNet and UNet++ predictions suffered from poorer spatial resolution within deformable soft-tissues (e.g. bladder, bowel). Qualitative radiologist assessment showed inter-expert variabilities in the test scores; each of the four radiologists correctly identified images as acquired/synthetic with 67%, 100%, 86% and 94% accuracy. Unsupervised segmentation of pelvic bone on T1-weighted images was successful in a number of test cases.

CONCLUSION

Pelvic MRI synthesis is a challenging task due to the absence of soft-tissue contrast on CT. Our study showed the potential of deep learning models for synthesizing realistic MR images from CT, and transferring cross-domain knowledge which may help to expand training datasets for 21 development of MR-only segmentation models.

MRI and CT Fusion in Stereotactic Electroencephalography: A Literature Review

Epilepsy is a common neurological disease characterized by spontaneous recurrent seizures. Resection of the epileptogenic tissue may be needed in approximately 25% of all cases due to ineffective treatment with anti-epileptic drugs. The surgical intervention depends on the correct detection of epileptogenic zones. The detection relies on invasive diagnostic techniques such as Stereotactic Electroencephalography (SEEG), which uses multi-modal fusion to aid localizing electrodes, using pre-surgical magnetic resonance and intra-surgical computer tomography as the input images. Moreover, it is essential to know how to measure the performance of fusion methods in the presence of external objects, such as electrodes. In this paper, a literature review is presented, applying the methodology proposed by Kitchenham to determine the main techniques of multi-modal brain image fusion, the most relevant performance metrics, and the main fusion tools. The search was conducted using the databases and search engines of Scopus, IEEE, PubMed, Springer, and Google Scholar, resulting in 15 primary source articles. The literature review found that rigid registration was the most used technique when electrode localization in SEEG is required, which was the proposed method in nine of the found articles. However, there is a lack of standard validation metrics, which makes the performance measurement difficult when external objects are presented, caused primarily by the absence of a gold-standard dataset for comparison.

Registration and Fusion of Close-Range Multimodal Wheat Images in Field Conditions

Multimodal images fusion has the potential to enrich the information gathered by multi-sensor plant phenotyping platforms. Fusion of images from multiple sources is, however, hampered by the technical lock of image registration. The aim of this paper is to provide a solution to the registration and fusion of multimodal wheat images in field conditions and at close range. Eight registration methods were tested on nadir wheat images acquired by a pair of red, green and blue (RGB) cameras, a thermal camera and a multispectral camera array. The most accurate method, relying on a local transformation, aligned the images with an average error of 2 mm but was not reliable for thermal images. More generally, the suggested registration method and the preprocesses necessary before fusion (plant mask erosion, pixel intensity averaging) would depend on the application. As a consequence, the main output of this study was to identify four registration-fusion strategies: (i) the REAL-TIME strategy solely based on the cameras’ positions, (ii) the FAST strategy suitable for all types of images tested, (iii) and (iv) the ACCURATE and HIGHLY ACCURATE strategies handling local distortion but unable to deal with images of very different natures. These suggestions are, however, limited to the methods compared in this study. Further research should investigate how recent cutting-edge registration methods would perform on the specific case of wheat canopy.

3D reconstructions, 4D imaging and postprocessing with CT in musculoskeletal disorders: Past, present and future

Three-dimensional (3D) imaging and post processing are common tasks used daily in many disciplines. The purpose of this article is to review the new postprocessing tools available. Although 3D imaging can be applied to all anatomical regions and used with all imaging techniques, its most varied and relevant applications are found with computed tomography (CT) data in musculoskeletal imaging. These new applications include global illumination rendering (GIR), unfolded rib reformations, subtracted CT angiography for bone analysis, dynamic studies, temporal subtraction and image fusion. In all of these tasks, registration and segmentation are two basic processes that affect the quality of the results. GIR simulates the complete interaction of photons with the scanned object, providing photorealistic volume rendering. Reformations to unfold the rib cage allow more accurate and faster diagnosis of rib lesions. Dynamic CT can be applied to cinematic joint evaluations a well as to perfusion and angiographic studies. Finally, more traditional techniques, such as minimum intensity projection, might find new applications for bone evaluation with the advent of ultra-high-resolution CT scanners. These tools can be used synergistically to provide morphologic, topographic and functional information and increase the versatility of CT.

Balanced multi-image demons for non-rigid registration of magnetic resonance images

A new approach is introduced for non-rigid registration of a pair of magnetic resonance images (MRI). It is a generalization of the demons algorithm with low computational cost, based on local information augmentation (by integrating multiple images) and balanced implementation. Specifically, a single deformation that best registers more pairs of images is estimated. All these images are extracted by applying different operators to the two original ones, processing local neighbors of each pixel. The following five images were found to be appropriate for MRI registration: the raw image and those obtained by contrast-limited adaptive histogram equalization, local median, local entropy and phase symmetry. Thus, each local point in the images is supplemented by augmented information coming by processing its neighbor. Moreover, image pairs are processed in alternation for each iteration of the algorithm (in a balanced way), computing both a forward and a backward registration. The new method (called balanced multi-image demons) is tested on sagittal MRIs from 10 patients, both in simulated and experimental conditions, improving the performances over the classical demons approach with minimal increase of the computational cost (processing time around twice that of standard demons). Specifically, a simulated deformation was applied to the MRIs (either original or corrupted by additive Gaussian or speckle noises). In all tested cases, the new algorithm improved the estimation of the simulated deformation (squared estimation error decreased by about 65% in the average). Moreover, statistically significant improvements were obtained in experimental tests, in which different brain regions (i.e., brain, posterior fossa and cerebellum) were identified by the atlas approach and compared to those manually delineated (in the average, Dice coefficient increased of about 6%). The conclusion is that a balanced method applied to multiple information extracted from neighboring pixels is a low cost approach to improve registration of MRIs.

One-Shot Learning for Deformable Medical Image Registration and Periodic Motion Tracking

Deformable image registration is a very important field of research in medical imaging. Recently multiple deep learning approaches were published in this area showing promising results. However, drawbacks of deep learning methods are the need for a large amount of training datasets and their inability to register unseen images different from the training datasets. One shot learning comes without the need of large training datasets and has already been proven to be applicable to 3D data. In this work we present a one shot registration approach for periodic motion tracking in 3D and 4D datasets. When applied to a 3D dataset the algorithm calculates the inverse of the registration vector field simultaneously. For registration we employed a U-Net combined with a coarse to fine approach and a differential spatial transformer module. The algorithm was thoroughly tested with multiple 4D and 3D datasets publicly available. The results show that the presented approach is able to track periodic motion and to yield a competitive registration accuracy. Possible applications are the use as a stand-alone algorithm for 3D and 4D motion tracking or in the beginning of studies until enough datasets for a separate training phase are available.

Free-to-use DIR solutions in radiotherapy: Benchmark against commercial platforms through a contour-propagation study

PURPOSE

A contour propagation study has been conducted to benchmark three algorithms for Deformable Image Registration (DIR) freely available online against well-established commercial solutions.

METHODS

ElastiX, BRAINS and Plastimach, available as modules in the open source platform 3DSlicer, were tested as the recent AAPM Task group 132 guidelines proposes. The overlap of the DIR-mapped ROIs in four computational anthropomorphic phantoms was measured. To avoid bias every algorithm was left to run without any human interaction nor particular registration strategy. The accuracy of the algorithms was measured using the Dice Similarity Coefficient (DSC) and Mean Distance to Conformity (MDC) metrics. The registration quality was compared to the recommended geometrical accuracy suggested by AAPM TG132 and to the results of a large population-based study performed with commercial DIR solutions.

RESULTS

The considered free-to-use DIR solutions generally meet acceptable accuracy and good overlap (DSC > 0.85). Mild failures (DSC < 0.75) were detected only for the smallest structures. In case of extremely severe deformations acceptable accuracy was not met (MDC > 3 mm). The morphing capability of the tested algorithms equals those of commercial systems when the user interaction is avoided. Underperformances were detected only in cases where a specific registration strategy is mandatory to obtain a satisfying match.

CONCLUSIONS

All of the considered algorithms show performances not inferior to previously published data and have the potential to be good candidates for use in the clinical routine. The results and conclusions only apply to the considered phantoms and should not be considered to be generally applicable and extendable to patient cases.

Rigid Shape Registration Based on Extended Hamiltonian Learning

Shape registration, finding the correct alignment of two sets of data, plays a significant role in computer vision such as objection recognition and image analysis. The iterative closest point (ICP) algorithm is one of well known and widely used algorithms in this area. The main purpose of this paper is to incorporate ICP with the fast convergent extended Hamiltonian learning (EHL), so called EHL-ICP algorithm, to perform planar and spatial rigid shape registration. By treating the registration error as the potential for the extended Hamiltonian system, the rigid shape registration is modelled as an optimization problem on the special Euclidean group SE(n)(n=2,3). Our method is robust to initial values and parameters. Compared with some state-of-art methods, our approach shows better efficiency and accuracy by simulation experiments.

Anatomical brain structures normalization for deep brain stimulation in movement disorders

•

Non-linear iterative structural normalization method focused on the deep brain.

•

Multi-modality image data from deep brain stimulation patients.

•

Comparison of ANTS, FNIRT and DRAMMS for the non-linear registrations using different settings for each.

•

Evaluation of the registration tools based on the analysis of 58 structures of the deep brain segmented manually by a single expert.

•

ANTS was identified as the best performing non-linear registration tool.

Deep brain stimulation (DBS) therapy requires extensive patient-specific planning prior to implantation to achieve optimal clinical outcomes. Collective analysis of patient’s brain images is promising in order to provide more systematic planning assistance. In this paper the design of a normalization pipeline using a group specific multi-modality iterative template creation process is presented. The focus was to compare the performance of a selection of freely available registration tools and select the best combination. The workflow was applied on 19 DBS patients with T1 and WAIR modality images available. Non-linear registrations were computed with ANTS, FNIRT and DRAMMS, using several settings from the literature. Registration accuracy was measured using single-expert labels of thalamic and subthalamic structures and their agreement across the group. The best performance was provided by ANTS using the High Variance settings published elsewhere. Neither FNIRT nor DRAMMS reached the level of performance of ANTS. The resulting normalized definition of anatomical structures were used to propose an atlas of the diencephalon region defining 58 structures using data from 19 patients.

Non-rigid image registration of 4D-MRI data for improved delineation of moving tumors

BACKGROUND

To increase the image quality of end-expiratory and end-inspiratory phases of retrospective respiratory self-gated 4D MRI data sets using non-rigid image registration for improved target delineation of moving tumors.

METHODS

End-expiratory and end-inspiratory phases of volunteer and patient 4D MRI data sets are used as targets for non-rigid image registration of all other phases using two different registration schemes: In the first, all phases are registered directly (dir-Reg) while next neighbors are successively registered until the target is reached in the second (nn-Reg). Resulting data sets are quantitatively compared using diaphragm and tumor sharpness and the coefficient of variation of regions of interest in the lung, liver, and heart. Qualitative assessment of the patient data regarding noise level, tumor delineation, and overall image quality was performed by blinded reading based on a 4 point Likert scale.

RESULTS

The median coefficient of variation was lower for both registration schemes compared to the target. Median dir-Reg coefficient of variation of all ROIs was 5.6% lower for expiration and 7.0% lower for inspiration compared with nn-Reg. Statistical significant differences between the two schemes were found in all comparisons. Median sharpness in inspiration is lower compared to expiration sharpness in all cases. Registered data sets were rated better compared to the targets in all categories. Over all categories, mean expiration scores were 2.92 ± 0.18 for the target, 3.19 ± 0.22 for nn-Reg and 3.56 ± 0.14 for dir-Reg and mean inspiration scores 2.25 ± 0.12 for the target, 2.72 ± 215 0.04 for nn-Reg and 3.78 ± 0.04 for dir-Reg.

CONCLUSIONS

In this work, end-expiratory and inspiratory phases of a 4D MRI data sets are used as targets for non-rigid image registration of all other phases. It is qualitatively and quantitatively shown that image quality of the targets can be significantly enhanced leading to improved target delineation of moving tumors.

Deep learning applied to hyperspectral endoscopy for online spectral classification

Hyperspectral imaging (HSI) is being explored in endoscopy as a tool to extract biochemical information that may improve contrast for early cancer detection in the gastrointestinal tract. Motion artefacts during medical endoscopy have traditionally limited HSI application, however, recent developments in the field have led to real-time HSI deployments. Unfortunately, traditional HSI analysis methods remain unable to rapidly process the volume of hyperspectral data in order to provide real-time feedback to the operator. Here, a convolutional neural network (CNN) is proposed to enable online classification of data obtained during HSI endoscopy. A five-layered CNN was trained and fine-tuned on a dataset of 300 hyperspectral endoscopy images acquired from a planar Macbeth ColorChecker chart and was able to distinguish between its 18 constituent colors with an average accuracy of 94.3% achieved at 8.8 fps. Performance was then tested on a set of images simulating an endoscopy environment, consisting of color charts warped inside a rigid tube mimicking a lumen. The algorithm proved robust to such variations, with classification accuracies over 90% being obtained despite the variations, with an average drop in accuracy of 2.4% being registered at the points of longest working distance and most inclination. For further validation of the color-based classification system, ex vivo videos of a methylene blue dyed pig esophagus and images of different disease stages in the human esophagus were analyzed, showing spatially distinct color classifications. These results suggest that the CNN has potential to provide color-based classification during real-time HSI in endoscopy.

Improved co-registration of ex-vivo and in-vivo cardiovascular magnetic resonance images using heart-specific flexible 3D printed acrylic scaffold combined with non-rigid registration

BACKGROUND

Ex-vivo cardiovascular magnetic resonance (CMR) imaging has played an important role in the validation of in-vivo CMR characterization of pathological processes. However, comparison between in-vivo and ex-vivo imaging remains challenging due to shape changes occurring between the two states, which may be non-uniform across the diseased heart. A novel two-step process to facilitate registration between ex-vivo and in-vivo CMR was developed and evaluated in a porcine model of chronic myocardial infarction (MI).

METHODS

Seven weeks after ischemia-reperfusion MI, 12 swine underwent in-vivo CMR imaging with late gadolinium enhancement followed by ex-vivo CMR 1 week later. Five animals comprised the control group, in which ex-vivo imaging was undertaken without any support in the LV cavity, 7 animals comprised the experimental group, in which a two-step registration optimization process was undertaken. The first step involved a heart specific flexible 3D printed scaffold generated from in-vivo CMR, which was used to maintain left ventricular (LV) shape during ex-vivo imaging. In the second step, a non-rigid co-registration algorithm was applied to align in-vivo and ex-vivo data. Tissue dimension changes between in-vivo and ex-vivo imaging were compared between the experimental and control group. In the experimental group, tissue compartment volumes and thickness were compared between in-vivo and ex-vivo data before and after non-rigid registration. The effectiveness of the alignment was assessed quantitatively using the DICE similarity coefficient.

RESULTS

LV cavity volume changed more in the control group (ratio of cavity volume between ex-vivo and in-vivo imaging in control and experimental group 0.14 vs 0.56, p < 0.0001) and there was a significantly greater change in the short axis dimensions in the control group (ratio of short axis dimensions in control and experimental group 0.38 vs 0.79, p < 0.001). In the experimental group, prior to non-rigid co-registration the LV cavity contracted isotropically in the ex-vivo condition by less than 20% in each dimension. There was a significant proportional change in tissue thickness in the healthy myocardium (change = 29 ± 21%), but not in dense scar (change = - 2 ± 2%, p = 0.034). Following the non-rigid co-registration step of the process, the DICE similarity coefficients for the myocardium, LV cavity and scar were 0.93 (±0.02), 0.89 (±0.01) and 0.77 (±0.07) respectively and the myocardial tissue and LV cavity volumes had a ratio of 1.03 and 1.00 respectively.

CONCLUSIONS

The pattern of the morphological changes seen between the in-vivo and the ex-vivo LV differs between scar and healthy myocardium. A 3D printed flexible scaffold based on the in-vivo shape of the LV cavity is an effective strategy to minimize morphological changes in the ex-vivo LV. The subsequent non-rigid registration step further improved the co-registration and local comparison between in-vivo and ex-vivo data.

An overview of deep learning in medical imaging focusing on MRI

What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge potential for medical imaging technology, medical data analysis, medical diagnostics and healthcare in general, slowly being realized. We provide a short overview of recent advances and some associated challenges in machine learning applied to medical image processing and image analysis. As this has become a very broad and fast expanding field we will not survey the entire landscape of applications, but put particular focus on deep learning in MRI. Our aim is threefold: (i) give a brief introduction to deep learning with pointers to core references; (ii) indicate how deep learning has been applied to the entire MRI processing chain, from acquisition to image retrieval, from segmentation to disease prediction; (iii) provide a starting point for people interested in experimenting and perhaps contributing to the field of deep learning for medical imaging by pointing out good educational resources, state-of-the-art open-source code, and interesting sources of data and problems related medical imaging.

MITT: Medical Image Tracking Toolbox

Over the years, medical image tracking has gained considerable attention from both medical and research communities due to its widespread utility in a multitude of clinical applications, from functional assessment during diagnosis and therapy planning to structure tracking or image fusion during image-guided interventions. Despite the ever-increasing number of image tracking methods available, most still consist of independent implementations with specific target applications, lacking the versatility to deal with distinct end-goals without the need for methodological tailoring and/or exhaustive tuning of numerous parameters. With this in mind, we have developed the medical image tracking toolbox (MITT)-a software package designed to ease customization of image tracking solutions in the medical field. While its workflow principles make it suitable to work with 2-D or 3-D image sequences, its modules offer versatility to set up computationally efficient tracking solutions, even for users with limited programming skills. MITT is implemented in both C/C++ and MATLAB, including several variants of an object-based image tracking algorithm and allowing to track multiple types of objects (i.e., contours, multi-contours, surfaces, and multi-surfaces) with several customization features. In this paper, the toolbox is presented, its features discussed, and illustrative examples of its usage in the cardiology field provided, demonstrating its versatility, simplicity, and time efficiency.

Quantifying Tensor Field Similarity With Global Distributions and Optimal Transport

Strain tensor fields quantify tissue deformation and are important for functional analysis of moving organs such as the heart and the tongue. Strain data can be readily obtained using medical imaging. However, quantification of similarity between different data sets is difficult. Strain patterns vary in space and time, and are inherently multidimensional. Also, the same type of mechanical deformation can be applied to different shapes; hence, automatic quantification of similarity should be unaffected by the geometry of the objects being deformed. This work introduces the application of global distributions used to classify shapes and vector fields in the pattern recognition literature, in the context of tensorial strain data. In particular, the distribution of mechanical properties of a field are approximated using a 3D histogram, and the Wasserstein distance from optimal transport theory is used to measure the similarity between histograms. To measure the method's consistency in matching deformations across different objects, the proposed approach was evaluated by sorting strain fields according to their similarity. Performance was compared to sorting via maximum shear distribution (a 1D histogram) and tensor residual magnitude (in perfectly registered objects). The technique was also applied to correlate muscle activation to muscular contraction observed via tagged MRI. The results show that the proposed approach accurately matches deformation regardless of the shape of the object being deformed. Sorting accuracy surpassed 1D shear distribution and was on par with residual magnitude, but without the need for registration between objects.

Normalizing brain activity across individuals using functional reference mapping

Neural activity can be mapped across individuals using brain atlases, but when spatial relationships are not equal, these techniques collapse. We map activity across individuals using functional registration, based on physiological responses to predetermined reference stimuli. Data from several individuals are integrated into a common multidimensional stimulus space, where dimensionality and axes are defined by these reference stimuli. We used this technique to discriminate volatile compounds with a cohort of Drosophila flies, by recording odor responses in receptor neurons on the flies’ antennae. We propose this technique for the development of reliable biological sensors when activity raw data cannot be calibrated. In particular, this technique will be useful for evaluating physiological measurements in natural chemosensory systems, and therefore will allow to exploit the sensitivity and selectivity of olfactory receptors present in the animal kingdom for analytical purposes.

Recent advances in radiation oncology

Radiotherapy (RT) is very much a technology-driven treatment modality in the management of cancer. RT techniques have changed significantly over the past few decades, thanks to improvements in engineering and computing. We aim to highlight the recent developments in radiation oncology, focusing on the technological and biological advances. We will present state-of-the-art treatment techniques, employing photon beams, such as intensity-modulated RT, volumetric-modulated arc therapy, stereotactic body RT and adaptive RT, which make possible a highly tailored dose distribution with maximum normal tissue sparing. We will analyse all the steps involved in the treatment: imaging, delineation of the tumour and organs at risk, treatment planning and finally image-guidance for accurate tumour localisation before and during treatment delivery. Particular attention will be given to the crucial role that imaging plays throughout the entire process. In the case of adaptive RT, the precise identification of target volumes as well as the monitoring of tumour response/modification during the course of treatment is mainly based on multimodality imaging that integrates morphological, functional and metabolic information. Moreover, real-time imaging of the tumour is essential in breathing adaptive techniques to compensate for tumour motion due to respiration. Brief reference will be made to the recent spread of particle beam therapy, in particular to the use of protons, but also to the yet limited experience of using heavy particles such as carbon ions. Finally, we will analyse the latest biological advances in tumour targeting. Indeed, the effectiveness of RT has been improved not only by technological developments but also through the integration of radiobiological knowledge to produce more efficient and personalised treatment strategies.

Mesh-to-raster region-of-interest-based nonrigid registration of multimodal images

Region of interest (RoI) alignment in medical images plays a crucial role in diagnostics, procedure planning, treatment, and follow-up. Frequently, a model is represented as triangulated mesh while the patient data is provided from computed axial tomography scanners as pixel or voxel data. Previously, we presented a 2-D method for curve-to-pixel registration. This paper contributes (i) a general mesh-to-raster framework to register RoIs in multimodal images; (ii) a 3-D surface-to-voxel application, and (iii) a comprehensive quantitative evaluation in 2-D using ground truth (GT) provided by the simultaneous truth and performance level estimation (STAPLE) method. The registration is formulated as a minimization problem, where the objective consists of a data term, which involves the signed distance function of the RoI from the reference image and a higher order elastic regularizer for the deformation. The evaluation is based on quantitative light-induced fluoroscopy (QLF) and digital photography (DP) of decalcified teeth. STAPLE is computed on 150 image pairs from 32 subjects, each showing one corresponding tooth in both modalities. The RoI in each image is manually marked by three experts (900 curves in total). In the QLF-DP setting, our approach significantly outperforms the mutual information-based registration algorithm implemented with the Insight Segmentation and Registration Toolkit and Elastix.