Medical image registration

Metadata-conditioned generative models to synthesize anatomically-plausible 3D brain MRIs

Recent advances in generative models have paved the way for enhanced generation of natural and medical images, including synthetic brain MRIs. However, the mainstay of current AI research focuses on optimizing synthetic MRIs with respect to visual quality (such as signal-to-noise ratio) while lacking insights into their relevance to neuroscience. To generate high-quality T1-weighted MRIs relevant for neuroscience discovery, we present a two-stage Diffusion Probabilistic Model (called BrainSynth) to synthesize high-resolution MRIs conditionally-dependent on metadata (such as age and sex). We then propose a novel procedure to assess the quality of BrainSynth according to how well its synthetic MRIs capture macrostructural properties of brain regions and how accurately they encode the effects of age and sex. Results indicate that more than half of the brain regions in our synthetic MRIs are anatomically plausible, i.e., the effect size between real and synthetic MRIs is small relative to biological factors such as age and sex. Moreover, the anatomical plausibility varies across cortical regions according to their geometric complexity. As is, the MRIs generated by BrainSynth significantly improve the training of a predictive model to identify accelerated aging effects in an independent study. These results indicate that our model accurately capture the brain's anatomical information and thus could enrich the data of underrepresented samples in a study. The code of BrainSynth will be released as part of the MONAI project at https://github.com/Project-MONAI/GenerativeModels.

Tools and recommendations for commissioning and quality assurance of deformable image registration in radiotherapy

Multiple tools are available for commissioning and quality assurance of deformable image registration (DIR), each with their own advantages and disadvantages in the context of radiotherapy. The selection of appropriate tools should depend on the DIR application with its corresponding available input, desired output, and time requirement. Discussions were hosted by the ESTRO Physics Workshop 2021 on Commissioning and Quality Assurance for DIR in Radiotherapy. A consensus was reached on what requirements are needed for commissioning and quality assurance for different applications, and what combination of tools is associated with this. For commissioning, we recommend the target registration error of manually annotated anatomical landmarks or the distance-to-agreement of manually delineated contours to evaluate alignment. These should be supplemented by the distance to discordance and/or biomechanical criteria to evaluate consistency and plausibility. Digital phantoms can be useful to evaluate DIR for dose accumulation but are currently only available for a limited range of anatomies, image modalities and types of deformations. For quality assurance of DIR for contour propagation, we recommend at least a visual inspection of the registered image and contour. For quality assurance of DIR for warping quantitative information such as dose, Hounsfield units or positron emission tomography-data, we recommend visual inspection of the registered image together with image similarity to evaluate alignment, supplemented by an inspection of the Jacobian determinant or bending energy to evaluate plausibility, and by the dose (gradient) to evaluate relevance. We acknowledge that some of these metrics are still missing in currently available commercial solutions.

Complementary information mutual learning for multimodality medical image segmentation

Radiologists must utilize medical images of multiple modalities for tumor segmentation and diagnosis due to the limitations of medical imaging technology and the diversity of tumor signals. This has led to the development of multimodal learning in medical image segmentation. However, the redundancy among modalities creates challenges for existing subtraction-based joint learning methods, such as misjudging the importance of modalities, ignoring specific modal information, and increasing cognitive load. These thorny issues ultimately decrease segmentation accuracy and increase the risk of overfitting. This paper presents the complementary information mutual learning (CIML) framework, which can mathematically model and address the negative impact of inter-modal redundant information. CIML adopts the idea of addition and removes inter-modal redundant information through inductive bias-driven task decomposition and message passing-based redundancy filtering. CIML first decomposes the multimodal segmentation task into multiple subtasks based on expert prior knowledge, minimizing the information dependence between modalities. Furthermore, CIML introduces a scheme in which each modality can extract information from other modalities additively through message passing. To achieve non-redundancy of extracted information, the redundant filtering is transformed into complementary information learning inspired by the variational information bottleneck. The complementary information learning procedure can be efficiently solved by variational inference and cross-modal spatial attention. Numerical results from the verification task and standard benchmarks indicate that CIML efficiently removes redundant information between modalities, outperforming SOTA methods regarding validation accuracy and segmentation effect. To emphasize, message-passing-based redundancy filtering allows neural network visualization techniques to visualize the knowledge relationship among different modalities, which reflects interpretability.

Deep learning for rapid virtual H&E staining of label-free glioma tissue from hyperspectral images

Hematoxylin and eosin (H&E) staining is a crucial technique for diagnosing glioma, allowing direct observation of tissue structures. However, the H&E staining workflow necessitates intricate processing, specialized laboratory infrastructures, and specialist pathologists, rendering it expensive, labor-intensive, and time-consuming. In view of these considerations, we combine the deep learning method and hyperspectral imaging technique, aiming at accurately and rapidly converting the hyperspectral images into virtual H&E staining images. The method overcomes the limitations of H&E staining by capturing tissue information at different wavelengths, providing comprehensive and detailed tissue composition information as the realistic H&E staining. In comparison with various generator structures, the Unet exhibits substantial overall advantages, as evidenced by a mean structure similarity index measure (SSIM) of 0.7731 and a peak signal-to-noise ratio (PSNR) of 23.3120, as well as the shortest training and inference time. A comprehensive software system for virtual H&E staining, which integrates CCD control, microscope control, and virtual H&E staining technology, is developed to facilitate fast intraoperative imaging, promote disease diagnosis, and accelerate the development of medical automation. The platform reconstructs large-scale virtual H&E staining images of gliomas at a high speed of 3.81 mm2/s. This innovative approach will pave the way for a novel, expedited route in histological staining.

The influence of metal artifact reduction on the trueness of registration of a cone-beam computed tomography scan with an intraoral scan in the presence of severe restoration artifact

PURPOSE

When planning guided implant surgery, highly radiopaque materials such as metals or zirconia produce streaking artifacts ('metal artifact') on cone-beam computed tomography scans, which can impair registration of the intraoral scan. This study aimed to determine the effect of metal artifact reduction on the trueness of registration in the presence of multiple full-coverage zirconia crowns.

MATERIALS AND METHODS

A 3D-printed maxillary study model was restored with 12 full-coverage zirconia crowns and scanned with an intraoral scanner. Cone-beam computed tomography scans of the study model were acquired, with and without activation of the metal artifact reduction algorithm. Registration of the optical scans was performed using initial point-based registration with surface-based refinement, and the deviation was measured at four pre-defined dental landmarks. Welch's t-test was used to compare the registration error for the metal artifact reduction group with the control group.

RESULTS

The average registration error was 0.519 mm (95% CI 0.507 to 0.531) with metal artifact reduction deactivated, compared to 0.478 mm (95% CI 0.460 to 0.496) without metal artifact reduction. Therefore, activation of the metal artifact reduction algorithm was associated with a 0.041 mm (95% CI 0.020 to 0.061, p < 0.001) increase in average registration error.

CONCLUSIONS

The use of the metal artifact reduction algorithm slightly reduced trueness in this in vitro study. Clinicians are advised not to rely on a metal artifact reduction (MAR) algorithm for registration of a cone-beam computed tomography scan with an intraoral scan when planning guided implant surgery in the presence of restoration artifacts.

Evaluation of Progressive Architectural Distortion in Idiopathic Pulmonary Fibrosis Using Deformable Registration of Sequential CT Images

BACKGROUND

Monitoring the progression of idiopathic pulmonary fibrosis (IPF) using CT primarily focuses on assessing the extent of fibrotic lesions, without considering the distortion of lung architecture.

OBJECTIVES

To evaluate three-dimensional average displacement (3D-AD) quantification of lung structures using deformable registration of serial CT images as a parameter of local lung architectural distortion and predictor of IPF prognosis.

MATERIALS AND METHODS

Patients with IPF evaluated between January 2016 and March 2017 who had undergone CT at least twice were retrospectively included (n = 114). The 3D-AD was obtained by deformable registration of baseline and follow-up CT images. A computer-aided quantification software measured the fibrotic lesion volume. Cox regression analysis evaluated these variables to predict mortality.

RESULTS

The 3D-AD and the fibrotic lesion volume change were significantly larger in the subpleural lung region (5.2 mm (interquartile range (IQR): 3.6-7.1 mm) and 0.70% (IQR: 0.22-1.60%), respectively) than those in the inner region (4.7 mm (IQR: 3.0-6.4 mm) and 0.21% (IQR: 0.004-1.12%), respectively). Multivariable logistic analysis revealed that subpleural region 3D-AD and fibrotic lesion volume change were independent predictors of mortality (hazard ratio: 1.12 and 1.23; 95% confidence interval: 1.02-1.22 and 1.10-1.38; p = 0.01 and p < 0.001, respectively).

CONCLUSIONS

The 3D-AD quantification derived from deformable registration of serial CT images serves as a marker of lung architectural distortion and a prognostic predictor in patients with IPF.

Free-breathing MRI techniques for fat and R2* quantification in the liver

OBJECTIVE

To review the recent advancements in free-breathing MRI techniques for proton-density fat fraction (PDFF) and R2* quantification in the liver, and discuss the current challenges and future opportunities.

MATERIALS AND METHODS

This work focused on recent developments of different MRI pulse sequences, motion management strategies, and reconstruction approaches that enable free-breathing liver PDFF and R2* quantification.

RESULTS

Different free-breathing liver PDFF and R2* quantification techniques have been evaluated in various cohorts, including healthy volunteers and patients with liver diseases, both in adults and children. Initial results demonstrate promising performance with respect to reference measurements. These techniques have a high potential impact on providing a solution to the clinical need of accurate liver fat and iron quantification in populations with limited breath-holding capacity.

DISCUSSION

As these free-breathing techniques progress toward clinical translation, studies of the linearity, bias, and repeatability of free-breathing PDFF and R2* quantification in a larger cohort are important. Scan acceleration and improved motion management also hold potential for further enhancement.

Quantification of Resection Margin following Sublobar Resection in Lung Cancer Patients through Pre- and Post-Operative CT Image Comparison: Utilizing a CT-Based 3D Reconstruction Algorithm

Sublobar resection has emerged as a standard treatment option for early-stage peripheral non-small cell lung cancer. Achieving an adequate resection margin is crucial to prevent local tumor recurrence. However, gross measurement of the resection margin may lack accuracy due to the elasticity of lung tissue and interobserver variability. Therefore, this study aimed to develop an objective measurement method, the CT-based 3D reconstruction algorithm, to quantify the resection margin following sublobar resection in lung cancer patients through pre- and post-operative CT image comparison. An automated subvascular matching technique was first developed to ensure accuracy and reproducibility in the matching process. Following the extraction of matched feature points, another key technique involves calculating the displacement field within the image. This is particularly important for mapping discontinuous deformation fields around the surgical resection area. A transformation based on thin-plate spline is used for medical image registration. Upon completing the final step of image registration, the distance at the resection margin was measured. After developing the CT-based 3D reconstruction algorithm, we included 12 cases for resection margin distance measurement, comprising 4 right middle lobectomies, 6 segmentectomies, and 2 wedge resections. The outcomes obtained with our method revealed that the target registration error for all cases was less than 2.5 mm. Our method demonstrated the feasibility of measuring the resection margin following sublobar resection in lung cancer patients through pre- and post-operative CT image comparison. Further validation with a multicenter, large cohort, and analysis of clinical outcome correlation is necessary in future studies.

Sub-Diffraction Correlation of Quantum Emitters and Local Strain Fields in Strain-Engineered WSe2 Monolayers

Strain-engineering in atomically thin metal dichalcogenides is a useful method for realizing single-photon emitters (SPEs) for quantum technologies. Correlating SPE position with local strain topography is challenging due to localization inaccuracies from the diffraction limit. Currently, SPEs are assumed to be positioned at the highest strained location and are typically identified by randomly screening narrow-linewidth emitters, of which only a few are spectrally pure. In this work, hyperspectral quantum emitter localization microscopy is used to locate 33 SPEs in nanoparticle-strained WSe2 monolayers with sub-diffraction-limit resolution (≈30 nm) and correlate their positions with the underlying strain field via image registration. In this system, spectrally pure emitters are not concentrated at the highest strain location due to spectral contamination; instead, isolable SPEs are distributed away from points of peak strain with an average displacement of 240 nm. These observations point toward a need for a change in the design rules for strain-engineered SPEs and constitute a key step toward realizing next-generation quantum optical architectures.

Model of zonular forces on the lens capsule during accommodation

How the human eye focuses for near; i.e. accommodates, is still being evaluated after more than 165 years. The mechanism of accommodation is essential for understanding the etiology and potential treatments for myopia, glaucoma and presbyopia. Presbyopia affects 100% of the population in the fifth decade of life. The lens is encased in a semi-elastic capsule with attached ligaments called zonules that mediate ciliary muscle forces to alter lens shape. The zonules are attached at the lens capsule equator. The fundamental issue is whether during accommodation all the zonules relax causing the central and peripheral lens surfaces to steepen, or the equatorial zonules are under increased tension while the anterior and posterior zonules relax causing the lens surface to peripherally flatten and centrally steepen while maintaining lens stability. Here we show with a balloon capsule zonular force model that increased equatorial zonular tension with relaxation of the anterior and posterior zonules replicates the topographical changes observed during in vivo rhesus and human accommodation of the lens capsule without lens stroma. The zonular forces required to simulate lens capsule configuration during in vivo accommodation are inconsistent with the general belief that all the zonules relax during accommodation.

Affine medical image registration with fusion feature mapping in local and global

Objective. Medical image affine registration is a crucial basis before using deformable registration. On the one hand, the traditional affine registration methods based on step-by-step optimization are very time-consuming, so these methods are not compatible with most real-time medical applications. On the other hand, convolutional neural networks are limited in modeling long-range spatial relationships of the features due to inductive biases, such as weight sharing and locality. This is not conducive to affine registration tasks. Therefore, the evolution of real-time and high-accuracy affine medical image registration algorithms is necessary for registration applications.Approach. In this paper, we propose a deep learning-based coarse-to-fine global and local feature fusion architecture for fast affine registration, and we use an unsupervised approach for end-to-end training. We use multiscale convolutional kernels as our elemental convolutional blocks to enhance feature extraction. Then, to learn the long-range spatial relationships of the features, we propose a new affine registration framework with weighted global positional attention that fuses global feature mapping and local feature mapping. Moreover, the fusion regressor is designed to generate the affine parameters.Main results. The additive fusion method can be adaptive to global mapping and local mapping, which improves affine registration accuracy without the center of mass initialization. In addition, the max pooling layer and the multiscale convolutional kernel coding module increase the ability of the model in affine registration.Significance. We validate the effectiveness of our method on the OASIS dataset with 414 3D MRI brain maps. Comprehensive results demonstrate that our method achieves state-of-the-art affine registration accuracy and very efficient runtimes.

Motion-correction strategies for enhancing whole-body PET imaging

Positron Emission Tomography (PET) is a powerful medical imaging technique widely used for detection and monitoring of disease. However, PET imaging can be adversely affected by patient motion, leading to degraded image quality and diagnostic capability. Hence, motion gating schemes have been developed to monitor various motion sources including head motion, respiratory motion, and cardiac motion. The approaches for these techniques have commonly come in the form of hardware-driven gating and data-driven gating, where the distinguishing aspect is the use of external hardware to make motion measurements vs. deriving these measures from the data itself. The implementation of these techniques helps correct for motion artifacts and improves tracer uptake measurements. With the great impact that these methods have on the diagnostic and quantitative quality of PET images, much research has been performed in this area, and this paper outlines the various approaches that have been developed as applied to whole-body PET imaging.

Radiomics of metastatic brain tumor as a predictive image biomarker of progression-free survival in patients with non-small-cell lung cancer with brain metastasis receiving tyrosine kinase inhibitors

BACKGROUND AND OBJECTIVE

Epidermal growth factor receptor (EGFR)-targeted tyrosine kinase inhibitors (TKIs) are the first-line therapy for EGFR-mutant non-small-cell lung cancer (NSCLC). Early prediction of treatment failure in patients with brain metastases treated with EGFR-TKIs may help in making decisions for systemic drug therapy or local brain tumor control. This study examined the predictive power of the radiomics of both brain metastasis tumors and primary lung tumors. We propose a deep learning based CoxCC model based on quantitative brain magnetic resonance imaging (MRI), a prognostic index and clinical data; the model can be used to predict progression-free survival (PFS) after EGFR-TKI therapy in advanced EGFR-mutant NSCLC.

METHODS

This retrospective single-center study included 271 patients receiving first-line EGFR-TKI targeted therapy in 2018-2019. Among them, 72 patients who had brain metastases before receiving first-line EGFR-TKI treatment. Three radiomic features were extracted from pretreatment brain MRI images. A CoxCC model for the progression risk stratification of EGFR-TKI treatment was proposed on the basis of MRI radiomics, clinical features, and a prognostic index. We performed time-dependent PFS predictions to evaluate the performance of the CoxCC model.

RESULTS

The CoxCC model based on a prognostic index, clinical features, and radiomic features of brain metastasis exhibited higher performance than clinical features combined with indexes previously proposed for determining the prognosis of brain metastasis, including recursive partitioning analysis, diagnostic-specific graded prognostic assessment, graded prognostic assessment for lung cancer using molecular markers (lung-molGPA), and modified lung-molGPA, with c-index values of 0.75, 0.67, 0.66, 0.65, and 0.65, respectively. The model achieved areas under the curve of 0.88, 0.73, 0.92, and 0.90 for predicting PFS at 3, 6, 9 and 12 months, respectively. PFS significantly differed between the high- and low-risk groups (p < 0.001).

CONCLUSIONS

For patients with advanced-stage NSCLC with brain metastasis, MRI radiomics of brain metastases may predict PFS. The CoxCC model integrating brain metastasis radiomics, clinical features, and a prognostic index provided reliable multi-time-point PFS predictions for patients with advanced NSCLC and brain metastases receiving EGFR-TKI treatment.

Review and recommendations on deformable image registration uncertainties for radiotherapy applications

Deformable image registration (DIR) is a versatile tool used in many applications in radiotherapy (RT). DIR algorithms have been implemented in many commercial treatment planning systems providing accessible and easy-to-use solutions. However, the geometric uncertainty of DIR can be large and difficult to quantify, resulting in barriers to clinical practice. Currently, there is no agreement in the RT community on how to quantify these uncertainties and determine thresholds that distinguish a good DIR result from a poor one. This review summarises the current literature on sources of DIR uncertainties and their impact on RT applications. Recommendations are provided on how to handle these uncertainties for patient-specific use, commissioning, and research. Recommendations are also provided for developers and vendors to help users to understand DIR uncertainties and make the application of DIR in RT safer and more reliable.

A robust and interpretable deep learning framework for multi-modal registration via keypoints

We present KeyMorph, a deep learning-based image registration framework that relies on automatically detecting corresponding keypoints. State-of-the-art deep learning methods for registration often are not robust to large misalignments, are not interpretable, and do not incorporate the symmetries of the problem. In addition, most models produce only a single prediction at test-time. Our core insight which addresses these shortcomings is that corresponding keypoints between images can be used to obtain the optimal transformation via a differentiable closed-form expression. We use this observation to drive the end-to-end learning of keypoints tailored for the registration task, and without knowledge of ground-truth keypoints. This framework not only leads to substantially more robust registration but also yields better interpretability, since the keypoints reveal which parts of the image are driving the final alignment. Moreover, KeyMorph can be designed to be equivariant under image translations and/or symmetric with respect to the input image ordering. Finally, we show how multiple deformation fields can be computed efficiently and in closed-form at test time corresponding to different transformation variants. We demonstrate the proposed framework in solving 3D affine and spline-based registration of multi-modal brain MRI scans. In particular, we show registration accuracy that surpasses current state-of-the-art methods, especially in the context of large displacements. Our code is available at https://github.com/alanqrwang/keymorph.

Comparative analysis of alignment algorithms for macular optical coherence tomography imaging

BACKGROUND

Optical coherence tomography (OCT) is the most important and commonly utilized imaging modality in ophthalmology and is especially crucial for the diagnosis and management of macular diseases. Each OCT volume is typically only available as a series of cross-sectional images (B-scans) that are accessible through proprietary software programs which accompany the OCT machines. To maximize the potential of OCT imaging for machine learning purposes, each OCT image should be analyzed en bloc as a 3D volume, which requires aligning all the cross-sectional images within a particular volume.

METHODS

A dataset of OCT B-scans obtained from 48 age-related macular degeneration (AMD) patients and 50 normal controls was used to evaluate five registration algorithms. After alignment of B-scans from each patient, an en face surface map was created to measure the registration quality, based on an automatically generated Laplace difference of the surface map-the smoother the surface map, the smaller the average Laplace difference. To demonstrate the usefulness of B-scan alignment, we trained a 3D convolutional neural network (CNN) to detect age-related macular degeneration (AMD) on OCT images and compared the performance of the model with and without B-scan alignment.

RESULTS

The mean Laplace difference of the surface map before registration was 27 ± 4.2 pixels for the AMD group and 26.6 ± 4 pixels for the control group. After alignment, the smoothness of the surface map was improved, with a mean Laplace difference of 5.5 ± 2.7 pixels for Advanced Normalization Tools Symmetric image Normalization (ANTs-SyN) registration algorithm in the AMD group and a mean Laplace difference of 4.3 ± 1.4.2 pixels for ANTs in the control group. Our 3D CNN achieved superior performance in detecting AMD, when aligned OCT B-scans were used (AUC 0.95 aligned vs. 0.89 unaligned).

CONCLUSIONS

We introduced a novel metric to quantify OCT B-scan alignment and compared the effectiveness of five alignment algorithms. We confirmed that alignment could be improved in a statistically significant manner with readily available alignment algorithms that are available to the public, and the ANTs algorithm provided the most robust performance overall. We further demonstrated that alignment of OCT B-scans will likely be useful for training 3D CNN models.

Advances in TEE-Centric Intraprocedural Multimodal Image Guidance for Congenital and Structural Heart Disease

Percutaneous interventions are gaining rapid acceptance in cardiology and revolutionizing the treatment of structural heart disease (SHD). As new percutaneous procedures of SHD are being developed, their associated complexity and anatomical variability demand a high-resolution special understanding for intraprocedural image guidance. During the last decade, three-dimensional (3D) transesophageal echocardiography (TEE) has become one of the most accessed imaging methods for structural interventions. Although 3D-TEE can assess cardiac structures and functions in real-time, its limitations (e.g., limited field of view, image quality at a large depth, etc.) must be addressed for its universal adaptation, as well as to improve the quality of its imaging and interventions. This review aims to present the role of TEE in the intraprocedural guidance of percutaneous structural interventions. We also focus on the current and future developments required in a multimodal image integration process when using TEE to enhance the management of congenital and SHD treatments.

MICDIR: Multi-scale inverse-consistent deformable image registration using UNetMSS with self-constructing graph latent

Image registration is the process of bringing different images into a common coordinate system - a technique widely used in various applications of computer vision, such as remote sensing, image retrieval, and, most commonly, medical imaging. Deep learning based techniques have been applied successfully to tackle various complex medical image processing problems, including medical image registration. Over the years, several image registration techniques have been proposed using deep learning. Deformable image registration techniques such as Voxelmorph have been successful in capturing finer changes and providing smoother deformations. However, Voxelmorph, as well as ICNet and FIRE, do not explicitly encode global dependencies (i.e. the overall anatomical view of the supplied image) and, therefore, cannot track large deformations. In order to tackle the aforementioned problems, this paper extends the Voxelmorph approach in three different ways. To improve the performance in case of small as well as large deformations, supervision of the model at different resolutions has been integrated using a multi-scale UNet. To support the network to learn and encode the minute structural co-relations of the given image-pairs, a self-constructing graph network (SCGNet) has been used as the latent of the multi-scale UNet - which can improve the learning process of the model and help the model to generalise better. And finally, to make the deformations inverse-consistent, cycle consistency loss has been employed. On the task of registration of brain MRIs, the proposed method achieved significant improvements over ANTs and VoxelMorph, obtaining a Dice score of 0.8013 ± 0.0243 for intramodal and 0.6211 ± 0.0309 for intermodal, while VoxelMorph achieved 0.7747 ± 0.0260 and 0.6071 ± 0.0510, respectively.

Quantitative longitudinal mapping of radiation-treated prostate cancer using MR fingerprinting with radial acquisition and subspace reconstruction

MR fingerprinting (MRF) enables fast multiparametric quantitative imaging with a single acquisition and has been shown to improve diagnosis of prostate cancer. However, most prostate MRF studies were performed with spiral acquisitions that are sensitive to B0 inhomogeneities and consequent blurring. In this work, a radial MRF acquisition with a novel subspace reconstruction technique was developed to enable fast T1/T2 mapping in the prostate in under 4 min. The subspace reconstruction exploits the extensive temporal correlations in the MRF dictionary to pre-compute a low dimensional space for the solution and thus reduce the number of radial spokes to accelerate the acquisition. Iterative reconstruction with the subspace model and additional regularization of the signal representation in the subspace is performed to minimize the number of spokes and maintain matching quality and SNR. Reconstruction accuracy was assessed using the ISMRM NIST phantom. In-vivo validation was performed on two healthy subjects and two prostate cancer patients undergoing radiation therapy. The longitudinal repeatability was quantified using the concordance correlation coefficient (CCC) in one of the healthy subjects by repeated scans over 1 year. One prostate cancer patient was scanned at three time points, before initiating therapy and following brachytherapy and external beam radiation. Changes in the T1/T2 maps obtained with the proposed method were quantified. The prostate, peripheral and transitional zones, and visible dominant lesion were delineated for each study, and the statistics and distribution of the quantitative mapping values were analyzed. Significant image quality improvements compared with standard reconstruction methods were obtained with the proposed subspace reconstruction method. A notable decrease in the spread of the T1/T2 values without biasing the estimated mean values was observed with the subspace reconstruction and agreed with reported literature values. The subspace reconstruction enabled visualization of small differences in T1/T2 values in the tumor region within the peripheral zone. Longitudinal imaging of a volunteer subject yielded CCC of 0.89 for MRF T1, and 0.81 for MRF T2 in the prostate gland. Longitudinal imaging of the prostate patient confirmed the feasibility of capturing radiation treatment related changes. This work is a proof-of-concept for a high resolution and fast quantitative mapping using golden-angle radial MRF combined with a subspace reconstruction technique for longitudinal treatment response assessment in subjects undergoing radiation treatment.

Image Analysis Techniques for In Vivo Quantification of Cerebrospinal Fluid Flow

Over the last decade, there has been a tremendously increased interest in understanding the neurophysiology of cerebrospinal fluid (CSF) flow, which plays a crucial role in clearing metabolic waste from the brain. This growing interest was largely initiated by two significant discoveries: the glymphatic system (a pathway for solute exchange between interstitial fluid deep within the brain and the CSF surrounding the brain) and meningeal lymphatic vessels (lymphatic vessels in the layer of tissue surrounding the brain that drain CSF). These two CSF systems work in unison, and their disruption has been implicated in several neurological disorders including Alzheimer's disease, stoke, and traumatic brain injury. Here, we present experimental techniques for in vivo quantification of CSF flow via direct imaging of fluorescent microspheres injected into the CSF. We discuss detailed image processing methods, including registration and masking of stagnant particles, to improve the quality of measurements. We provide guidance for quantifying CSF flow through particle tracking and offer tips for optimizing the process. Additionally, we describe techniques for measuring changes in arterial diameter, which is an hypothesized CSF pumping mechanism. Finally, we outline how these same techniques can be applied to cervical lymphatic vessels, which collect fluid downstream from meningeal lymphatic vessels. We anticipate that these fluid mechanical techniques will prove valuable for future quantitative studies aimed at understanding mechanisms of CSF transport and disruption, as well as for other complex biophysical systems.

A brain CT-based approach for predicting and analyzing stroke-associated pneumonia from intracerebral hemorrhage

Introduction

Stroke-associated pneumonia (SAP) is a common complication of stroke that can increase the mortality rate of patients and the burden on their families. In contrast to prior clinical scoring models that rely on baseline data, we propose constructing models based on brain CT scans due to their accessibility and clinical universality.

Methods

Our study aims to explore the mechanism behind the distribution and lesion areas of intracerebral hemorrhage (ICH) in relation to pneumonia, we utilized an MRI atlas that could present brain structures and a registration method in our program to extract features that may represent this relationship. We developed three machine learning models to predict the occurrence of SAP using these features. Ten-fold cross-validation was applied to evaluate the performance of models. Additionally, we constructed a probability map through statistical analysis that could display which brain regions are more frequently impacted by hematoma in patients with SAP based on four types of pneumonia.

Results

Our study included a cohort of 244 patients, and we extracted 35 features that captured the invasion of ICH to different brain regions for model development. We evaluated the performance of three machine learning models, namely, logistic regression, support vector machine, and random forest, in predicting SAP, and the AUCs for these models ranged from 0.77 to 0.82. The probability map revealed that the distribution of ICH varied between the left and right brain hemispheres in patients with moderate and severe SAP, and we identified several brain structures, including the left-choroid-plexus, right-choroid-plexus, right-hippocampus, and left-hippocampus, that were more closely related to SAP based on feature selection. Additionally, we observed that some statistical indicators of ICH volume, such as mean and maximum values, were proportional to the severity of SAP.

Discussion

Our findings suggest that our method is effective in classifying the development of pneumonia based on brain CT scans. Furthermore, we identified distinct characteristics, such as volume and distribution, of ICH in four different types of SAP.

Volumetric registration framework for multimodal functional magnetic resonance and optoacoustic tomography of the rodent brain

Optoacoustic tomography (OAT) provides a non-invasive means to characterize cerebral hemodynamics across an entire murine brain while attaining multi-parametric readouts not available with other modalities. This unique capability can massively impact our understanding of brain function. However, OAT largely lacks the soft tissue contrast required for unambiguous identification of brain regions. Hence, its accurate registration to a reference brain atlas is paramount for attaining meaningful functional readings. Herein, we capitalized on the simultaneously acquired bi-modal data from the recently-developed hybrid magnetic resonance optoacoustic tomography (MROT) scanner in order to devise an image coregistration paradigm that facilitates brain parcellation and anatomical referencing. We evaluated the performance of the proposed methodology by coregistering OAT data acquired with a standalone system using different registration methods. The enhanced performance is further demonstrated for functional OAT data analysis and characterization of stimulus-evoked brain responses. The suggested approach enables better consolidation of the research findings thus facilitating wider acceptance of OAT as a powerful neuroimaging tool to study brain functions and diseases.

Optimizing 3DCT image registration for interfractional changes in carbon-ion prostate radiotherapy

To perform setup procedures including both positional and dosimetric information, we developed a CT-CT rigid image registration algorithm utilizing water equivalent pathlength (WEPL)-based image registration and compared the resulting dose distribution with those of two other algorithms, intensity-based image registration and target-based image registration, in prostate cancer radiotherapy using the carbon-ion pencil beam scanning technique. We used the data of the carbon ion therapy planning CT and the four-weekly treatment CTs of 19 prostate cancer cases. Three CT-CT registration algorithms were used to register the treatment CTs to the planning CT. Intensity-based image registration uses CT voxel intensity information. Target-based image registration uses target position on the treatment CTs to register it to that on the planning CT. WEPL-based image registration registers the treatment CTs to the planning CT using WEPL values. Initial dose distributions were calculated using the planning CT with the lateral beam angles. The treatment plan parameters were optimized to administer the prescribed dose to the PTV on the planning CT. Weekly dose distributions using the three different algorithms were calculated by applying the treatment plan parameters to the weekly CT data. Dosimetry, including the dose received by 95% of the clinical target volume (CTV-D95), rectal volumes receiving > 20 Gy (RBE) (V20), > 30 Gy (RBE) (V30), and > 40 Gy (RBE) (V40), were calculated. Statistical significance was assessed using the Wilcoxon signed-rank test. Interfractional CTV displacement over all patients was 6.0 ± 2.7 mm (19.3 mm maximum standard amount). WEPL differences between the planning CT and the treatment CT were 1.2 ± 0.6 mm-H₂O (< 3.9 mm-H₂O), 1.7 ± 0.9 mm-H₂O (< 5.7 mm-H₂O) and 1.5 ± 0.7 mm-H₂O (< 3.6 mm-H₂O maxima) with the intensity-based image registration, target-based image registration, and WEPL-based image registration, respectively. For CTV coverage, the D95 values on the planning CT were > 95% of the prescribed dose in all cases. The mean CTV-D95 values were 95.8 ± 11.5% and 98.8 ± 1.7% with the intensity-based image registration and target-based image registration, respectively. The WEPL-based image registration was CTV-D95 to 99.0 ± 0.4% and rectal Dmax to 51.9 ± 1.9 Gy (RBE) compared to 49.4 ± 9.1 Gy (RBE) with intensity-based image registration and 52.2 ± 1.8 Gy (RBE) with target-based image registration. The WEPL-based image registration algorithm improved the target coverage from the other algorithms and reduced rectal dose from the target-based image registration, even though the magnitude of the interfractional variation was increased.

uRP: An integrated research platform for one-stop analysis of medical images

Introduction

Medical image analysis is of tremendous importance in serving clinical diagnosis, treatment planning, as well as prognosis assessment. However, the image analysis process usually involves multiple modality-specific software and relies on rigorous manual operations, which is time-consuming and potentially low reproducible.

Methods

We present an integrated platform - uAI Research Portal (uRP), to achieve one-stop analyses of multimodal images such as CT, MRI, and PET for clinical research applications. The proposed uRP adopts a modularized architecture to be multifunctional, extensible, and customizable.

Results and Discussion

The uRP shows 3 advantages, as it 1) spans a wealth of algorithms for image processing including semi-automatic delineation, automatic segmentation, registration, classification, quantitative analysis, and image visualization, to realize a one-stop analytic pipeline, 2) integrates a variety of functional modules, which can be directly applied, combined, or customized for specific application domains, such as brain, pneumonia, and knee joint analyses, 3) enables full-stack analysis of one disease, including diagnosis, treatment planning, and prognosis assessment, as well as full-spectrum coverage for multiple disease applications. With the continuous development and inclusion of advanced algorithms, we expect this platform to largely simplify the clinical scientific research process and promote more and better discoveries.

Quality assurance assessment of intra-acquisition diffusion-weighted and T2-weighted magnetic resonance imaging registration and contour propagation for head and neck cancer radiotherapy

BACKGROUND/PURPOSE

Adequate image registration of anatomical and functional magnetic resonance imaging (MRI) scans is necessary for MR-guided head and neck cancer (HNC) adaptive radiotherapy planning. Despite the quantitative capabilities of diffusion-weighted imaging (DWI) MRI for treatment plan adaptation, geometric distortion remains a considerable limitation. Therefore, we systematically investigated various deformable image registration (DIR) methods to co-register DWI and T2-weighted (T2W) images.

MATERIALS/METHODS

We compared three commercial (ADMIRE, Velocity, Raystation) and three open-source (Elastix with default settings [Elastix Default], Elastix with parameter set 23 [Elastix 23], Demons) post-acquisition DIR methods applied to T2W and DWI MRI images acquired during the same imaging session in twenty immobilized HNC patients. In addition, we used the non-registered images (None) as a control comparator. Ground-truth segmentations of radiotherapy structures (tumour and organs at risk) were generated by a physician expert on both image sequences. For each registration approach, structures were propagated from T2W to DWI images. These propagated structures were then compared with ground-truth DWI structures using the Dice similarity coefficient and mean surface distance.

RESULTS

19 left submandibular glands, 18 right submandibular glands, 20 left parotid glands, 20 right parotid glands, 20 spinal cords, and 12 tumours were delineated. Most DIR methods took <30 s to execute per case, with the exception of Elastix 23 which took ∼458 s to execute per case. ADMIRE and Elastix 23 demonstrated improved performance over None for all metrics and structures (Bonferroni-corrected p < 0.05), while the other methods did not. Moreover, ADMIRE and Elastix 23 significantly improved performance in individual and pooled analysis compared to all other methods.

CONCLUSIONS

The ADMIRE DIR method offers improved geometric performance with reasonable execution time so should be favoured for registering T2W and DWI images acquired during the same scan session in HNC patients. These results are important to ensure the appropriate selection of registration strategies for MR-guided radiotherapy.

Rapid Morphological Measurement Method of Aortic Dissection Stent Based on Spatial Observation Point Set

OBJECTIVES

Post-operative stent morphology of aortic dissection patients is important for performing clinical diagnosis and prognostic assessment. However, stent morphologies still need to be manually measured, which is a process prone to errors, high time consumption and difficulty in exploiting inter-data associations. Herein, we propose a method based on the stepwise combination of basic, non-divisible data sets to quickly obtain morphological parameters with high accuracy.

METHODS

We performed the 3D reconstruction of 109 post-operative follow-up CT image data from 26 patients using mimics software. By extracting the spatial locations of the basic morphological observation points on the stent, we defined a basic and non-reducible set of observation points. Further, we implemented a fully automatic stent segmentation and an observation point extraction algorithm. We analyzed the stability and accuracy of the algorithms on a test set containing 8 cases and 408 points. Based on this dataset, we calculated three morphological parameters of different complexity for the different spatial structural features exhibited by the stent. Finally, we compared the two measurement schemes in four aspects: data variability, data stability, statistical process complexity and algorithmic error.

RESULTS

The statistical results of the two methods on two low-complexity morphological parameters (spatial position of stent end and vascular stent end-slip volume) show good agreement (n = 26, P₁, P₂ < 0.001, r₁ = 0.992, r₂ = 0.988). The statistics of the proposed method for the morphological parameters of medium complexity (proximal support ring feature diameter and distal support ring feature diameter) avoid the errors caused by manual extraction, and the magnitude of this correction to the traditional method does not exceed 4 mm with an average correction of 1.38 mm. Meanwhile, our proposed automatic observation point extraction method has only 2.2% error rate on the test set, and the average spatial distance from the manually marked observation points is 0.73 mm. Thus, the proposed method is able to rapidly and accurately measure the stent circumferential deflection angle, which is highly complex and cannot be measured using traditional methods.

CONCLUSIONS

The proposed method can significantly reduce the statistical observation time and information processing cost compared to the traditional morphological observation methods. Moreover, when new morphological parameters are required, one can quickly and accurately obtain the target parameters by new "combinatorial functions." Iterative modification of the data set itself is avoided.

Reliable and stable fundus image registration based on brain-inspired spatially-varying adaptive pyramid context aggregation network

The task of fundus image registration aims to find matching keypoints between an image pair. Traditional methods detect the keypoint by hand-designed features, which fail to cope with complex application scenarios. Due to the strong feature learning ability of deep neural network, current image registration methods based on deep learning directly learn to align the geometric transformation between the reference image and test image in an end-to-end manner. Another mainstream of this task aims to learn the displacement vector field between the image pair. In this way, the image registration has achieved significant advances. However, due to the complicated vascular morphology of retinal image, such as texture and shape, current widely used image registration methods based on deep learning fail to achieve reliable and stable keypoint detection and registration results. To this end, in this paper, we aim to bridge this gap. Concretely, since the vessel crossing and branching points can reliably and stably characterize the key components of fundus image, we propose to learn to detect and match all the crossing and branching points of the input images based on a single deep neural network. Moreover, in order to accurately locate the keypoints and learn discriminative feature embedding, a brain-inspired spatially-varying adaptive pyramid context aggregation network is proposed to incorporate the contextual cues under the supervision of structured triplet ranking loss. Experimental results show that the proposed method achieves more accurate registration results with significant speed advantage.

Food Image Segmentation Using Multi-Modal Imaging Sensors with Color and Thermal Data

Sensor-based food intake monitoring has become one of the fastest-growing fields in dietary assessment. Researchers are exploring imaging-sensor-based food detection, food recognition, and food portion size estimation. A major problem that is still being tackled in this field is the segmentation of regions of food when multiple food items are present, mainly when similar-looking foods (similar in color and/or texture) are present. Food image segmentation is a relatively under-explored area compared with other fields. This paper proposes a novel approach to food imaging consisting of two imaging sensors: color (Red-Green-Blue) and thermal. Furthermore, we propose a multi-modal four-Dimensional (RGB-T) image segmentation using a k-means clustering algorithm to segment regions of similar-looking food items in multiple combinations of hot, cold, and warm (at room temperature) foods. Six food combinations of two food items each were used to capture RGB and thermal image data. RGB and thermal data were superimposed to form a combined RGB-T image and three sets of data (RGB, thermal, and RGB-T) were tested. A bootstrapped optimization of within-cluster sum of squares (WSS) was employed to determine the optimal number of clusters for each case. The combined RGB-T data achieved better results compared with RGB and thermal data, used individually. The mean ± standard deviation (std. dev.) of the F1 score for RGB-T data was 0.87 ± 0.1 compared with 0.66 ± 0.13 and 0.64 ± 0.39, for RGB and Thermal data, respectively.

The Systematic Review of Artificial Intelligence Applications in Breast Cancer Diagnosis

Several studies have demonstrated the value of artificial intelligence (AI) applications in breast cancer diagnosis. The systematic review of AI applications in breast cancer diagnosis includes several studies that compare breast cancer diagnosis and AI. However, they lack systematization, and each study appears to be conducted uniquely. The purpose and contributions of this study are to offer elaborative knowledge on the applications of AI in the diagnosis of breast cancer through citation analysis in order to categorize the main area of specialization that attracts the attention of the academic community, as well as thematic issue analysis to identify the species being researched in each category. In this study, a total number of 17,900 studies addressing breast cancer and AI published between 2012 and 2022 were obtained from these databases: IEEE, Embase: Excerpta Medica Database Guide-Ovid, PubMed, Springer, Web of Science, and Google Scholar. We applied inclusion and exclusion criteria to the search; 36 studies were identified. The vast majority of AI applications used classification models for the prediction of breast cancer. Howbeit, accuracy (99%) has the highest number of performance metrics, followed by specificity (98%) and area under the curve (0.95). Additionally, the Convolutional Neural Network (CNN) was the best model of choice in several studies. This study shows that the quantity and caliber of studies that use AI applications in breast cancer diagnosis will continue to rise annually. As a result, AI-based applications are viewed as a supplement to doctors' clinical reasoning, with the ultimate goal of providing quality healthcare that is both affordable and accessible to everyone worldwide.

A unified framework for focal intensity change detection and deformable image registration. Application to the monitoring of multiple sclerosis lesions in longitudinal 3D brain MRI

Registration is a crucial step in the design of automatic change detection methods dedicated to longitudinal brain MRI. Even small registration inaccuracies can significantly deteriorate the detection performance by introducing numerous spurious detections. Rigid or affine registration are usually considered to align baseline and follow-up scans, as a pre-processing step before applying a change detection method. In the context of multiple sclerosis, using deformable registration can be required to capture the complex deformations due to brain atrophy. However, non-rigid registration can alter the shape of appearing and evolving lesions while minimizing the dissimilarity between the two images. To overcome this issue, we consider registration and change detection as intertwined problems that should be solved jointly. To this end, we formulate these two separate tasks as a single optimization problem involving a unique energy that models their coupling. We focus on intensity-based change detection and registration, but the approach is versatile and could be extended to other modeling choices. We show experimentally on synthetic and real data that the proposed joint approach overcomes the limitations of the sequential scheme.

Computational Analysis of Cardiac Contractile Function

PURPOSE OF REVIEW

Heart failure results in the high incidence and mortality all over the world. Mechanical properties of myocardium are critical determinants of cardiac function, with regional variations in myocardial contractility demonstrated within infarcted ventricles. Quantitative assessment of cardiac contractile function is therefore critical to identify myocardial infarction for the early diagnosis and therapeutic intervention.

RECENT FINDINGS

Current advancement of cardiac functional assessments is in pace with the development of imaging techniques. The methods tailored to advanced imaging have been widely used in cardiac magnetic resonance, echocardiography, and optical microscopy. In this review, we introduce fundamental concepts and applications of representative methods for each imaging modality used in both fundamental research and clinical investigations. All these methods have been designed or developed to quantify time-dependent 2-dimensional (2D) or 3D cardiac mechanics, holding great potential to unravel global or regional myocardial deformation and contractile function from end-systole to end-diastole. Computational methods to assess cardiac contractile function provide a quantitative insight into the analysis of myocardial mechanics during cardiac development, injury, and remodeling.

Is image-to-image translation the panacea for multimodal image registration? A comparative study

Despite current advancement in the field of biomedical image processing, propelled by the deep learning revolution, multimodal image registration, due to its several challenges, is still often performed manually by specialists. The recent success of image-to-image (I2I) translation in computer vision applications and its growing use in biomedical areas provide a tempting possibility of transforming the multimodal registration problem into a, potentially easier, monomodal one. We conduct an empirical study of the applicability of modern I2I translation methods for the task of rigid registration of multimodal biomedical and medical 2D and 3D images. We compare the performance of four Generative Adversarial Network (GAN)-based I2I translation methods and one contrastive representation learning method, subsequently combined with two representative monomodal registration methods, to judge the effectiveness of modality translation for multimodal image registration. We evaluate these method combinations on four publicly available multimodal (2D and 3D) datasets and compare with the performance of registration achieved by several well-known approaches acting directly on multimodal image data. Our results suggest that, although I2I translation may be helpful when the modalities to register are clearly correlated, registration of modalities which express distinctly different properties of the sample are not well handled by the I2I translation approach. The evaluated representation learning method, which aims to find abstract image-like representations of the information shared between the modalities, manages better, and so does the Mutual Information maximisation approach, acting directly on the original multimodal images. We share our complete experimental setup as open-source (https://github.com/MIDA-group/MultiRegEval), including method implementations, evaluation code, and all datasets, for further reproducing and benchmarking.

A Systematic Literature Review on Applications of GAN-Synthesized Images for Brain MRI

With the advances in brain imaging, magnetic resonance imaging (MRI) is evolving as a popular radiological tool in clinical diagnosis. Deep learning (DL) methods can detect abnormalities in brain images without an extensive manual feature extraction process. Generative adversarial network (GAN)-synthesized images have many applications in this field besides augmentation, such as image translation, registration, super-resolution, denoising, motion correction, segmentation, reconstruction, and contrast enhancement. The existing literature was reviewed systematically to understand the role of GAN-synthesized dummy images in brain disease diagnosis. Web of Science and Scopus databases were extensively searched to find relevant studies from the last 6 years to write this systematic literature review (SLR). Predefined inclusion and exclusion criteria helped in filtering the search results. Data extraction is based on related research questions (RQ). This SLR identifies various loss functions used in the above applications and software to process brain MRIs. A comparative study of existing evaluation metrics for GAN-synthesized images helps choose the proper metric for an application. GAN-synthesized images will have a crucial role in the clinical sector in the coming years, and this paper gives a baseline for other researchers in the field.

Zebrafish neuromesodermal progenitors undergo a critical state transition in vivo

The transition state model of cell differentiation proposes that a transient window of gene expression stochasticity precedes entry into a differentiated state. Here, we assess this theoretical model in zebrafish neuromesodermal progenitors (NMps) in vivo during late somitogenesis stages. We observed an increase in gene expression variability at the 24 somite stage (24ss) before their differentiation into spinal cord and paraxial mesoderm. Analysis of a published 18ss scRNA-seq dataset showed that the NMp population is noisier than its derivatives. By building in silico composite gene expression maps from image data, we assigned an 'NM index' to in silico NMps based on the expression of neural and mesodermal markers and demonstrated that cell population heterogeneity peaked at 24ss. Further examination revealed cells with gene expression profiles incongruent with their prospective fate. Taken together, our work supports the transition state model within an endogenous cell fate decision making event.

A New Approach to Automatically Calibrate and Detect Building Cracks

Timely crack detection plays an important role in building damage assessment. In this study, an automatic crack detection method based on image registration and pixel-level segmentation (improved DeepLab_v3+) is proposed. Firstly, the moving images are calibrated by image registration, and the similarity method is adopted to evaluate the calibrated results. Secondly, the DeepLab_v3+ is improved and used to segment the fixed images and the calibrated images. Finally, the difference of crack pixels between the fixed and calibrated images is estimated, and the key parameter is investigated to find the optimal optimizer and learning rate. The results illustrate that: (1) the image registration technology shows excellent calibration achievement and the average error is only 4%; (2) with the resnet50 being selected as the backbone network of improved Deeplab_v3+, the automatic detection method proposed in this study is more efficient in comparison with other common pixel-level segmentation algorithms; (3) the best network optimizer of improved Deeplab_v3+ and learning rate of crack segmentation task are sgdm and 0.001, respectively. The crack detection method proposed in this study can significantly improves the technical level of crack detection in practical projects.

Semi-Automatic Multiparametric MR Imaging Classification Using Novel Image Input Sequences and 3D Convolutional Neural Networks

The role of multi-parametric magnetic resonance imaging (mp-MRI) is becoming increasingly important in the diagnosis of the clinical severity of prostate cancer (PCa). However, mp-MRI images usually contain several unaligned 3D sequences, such as DWI image sequences and T2-weighted image sequences, and there are many images among the entirety of 3D sequence images that do not contain cancerous tissue, which affects the accuracy of large-scale prostate cancer detection. Therefore, there is a great need for a method that uses accurate computer-aided detection of mp-MRI images and minimizes the influence of useless features. Our proposed PCa detection method is divided into three stages: (i) multimodal image alignment, (ii) automatic cropping of the sequence images to the entire prostate region, and, finally, (iii) combining multiple modal images of each patient into novel 3D sequences and using 3D convolutional neural networks to learn the newly composed 3D sequences with different modal alignments. We arrange the different modal methods to make the model fully learn the cancerous tissue features; then, we predict the clinical severity of PCa and generate a 3D cancer response map for the 3D sequence images from the last convolution layer of the network. The prediction results and 3D response map help to understand the features that the model focuses on during the process of 3D-CNN feature learning. We applied our method to Toho hospital prostate cancer patient data; the AUC (=0.85) results were significantly higher than those of other methods.

MRI Radiogenomics in Precision Oncology: New Diagnosis and Treatment Method

Precision medicine for cancer affords a new way for the most accurate and effective treatment to each individual cancer. Given the high time-evolving intertumor and intratumor heterogeneity features of personal medicine, there are still several obstacles hindering its diagnosis and treatment in clinical practice regardless of extensive exploration on it over the past years. This paper is to investigate radiogenomics methods in the literature for precision medicine for cancer focusing on the heterogeneity analysis of tumors. Based on integrative analysis of multimodal (parametric) imaging and molecular data in bulk tumors, a comprehensive analysis and discussion involving the characterization of tumor heterogeneity in imaging and molecular expression are conducted. These investigations are intended to (i) fully excavate the multidimensional spatial, temporal, and semantic related information regarding high-dimensional breast magnetic resonance imaging data, with integration of the highly specific structured data of genomics and combination of the diagnosis and cognitive process of doctors, and (ii) establish a radiogenomics data representation model based on multidimensional consistency analysis with multilevel spatial-temporal correlations.

Rigid registration algorithm based on the minimization of the total variation of the difference map

Image registration is broadly used in various scenarios in which similar scenes in different images are to be aligned. However, image registration becomes challenging when the contrasts and backgrounds in the images are vastly different. This work proposes using the total variation of the difference map between two images (TVDM) as a dissimilarity metric in rigid registration. A method based on TVDM minimization is implemented for image rigid registration. The method is tested with both synthesized and real experimental data that have various noise and background conditions. The performance of the proposed method is compared with the results of other rigid registration methods. It is demonstrated that the proposed method is highly accurate and robust and outperforms other methods in all of the tests. The new algorithm provides a robust option for image registrations that are critical to many nano-scale X-ray imaging and microscopy applications.

Medical Image Registration Algorithm Based on Bounded Generalized Gaussian Mixture Model

In this paper, a method for medical image registration based on the bounded generalized Gaussian mixture model is proposed. The bounded generalized Gaussian mixture model is used to approach the joint intensity of source medical images. The mixture model is formulated based on a maximum likelihood framework, and is solved by an expectation-maximization algorithm. The registration performance of the proposed approach on different medical images is verified through extensive computer simulations. Empirical findings confirm that the proposed approach is significantly better than other conventional ones.

MIRSIG position paper: the use of image registration and fusion algorithms in radiotherapy

The report of the American Association of Physicists in Medicine (AAPM) Task Group No. 132 published in 2017 reviewed rigid image registration and deformable image registration (DIR) approaches and solutions to provide recommendations for quality assurance and quality control of clinical image registration and fusion techniques in radiotherapy. However, that report did not include the use of DIR for advanced applications such as dose warping or warping of other matrices of interest. Considering that DIR warping tools are now readily available, discussions were hosted by the Medical Image Registration Special Interest Group (MIRSIG) of the Australasian College of Physical Scientists & Engineers in Medicine in 2018 to form a consensus on best practice guidelines. This position statement authored by MIRSIG endorses the recommendations of the report of AAPM task group 132 and expands on the best practice advice from the 'Deforming to Best Practice' MIRSIG publication to provide guidelines on the use of DIR for advanced applications.

Overview of radiomics in prostate imaging and future directions

Recent advancements in imaging technology and analysis methods have led to an analytic framework known as radiomics. This framework extracts comprehensive high-dimensional features from imaging data and performs data mining to build analytical models for improved decision-support. Its features include many categories spanning texture and shape; thus, it can provide abundant information for precision medicine. Many studies of prostate radiomics have shown promising results in the assessment of pathological features, prediction of treatment response, and stratification of risk groups. Herein, we aimed to provide a general overview of radiomics procedures, discuss technical issues, explain various clinical applications, and suggest future research directions, especially for prostate imaging.

Early Melanoma Diagnosis With Sequential Dermoscopic Images

Dermatologists often diagnose or rule out early melanoma by evaluating the follow-up dermoscopic images of skin lesions. However, existing algorithms for early melanoma diagnosis are developed using single time-point images of lesions. Ignoring the temporal, morphological changes of lesions can lead to misdiagnosis in borderline cases. In this study, we propose a framework for automated early melanoma diagnosis using sequential dermoscopic images. To this end, we construct our method in three steps. First, we align sequential dermoscopic images of skin lesions using estimated Euclidean transformations, extract the lesion growth region by computing image differences among the consecutive images, and then propose a spatio-temporal network to capture the dermoscopic changes from aligned lesion images and the corresponding difference images. Finally, we develop an early diagnosis module to compute probability scores of malignancy for lesion images over time. We collected 179 serial dermoscopic imaging data from 122 patients to verify our method. Extensive experiments show that the proposed model outperforms other commonly used sequence models. We also compared the diagnostic results of our model with those of seven experienced dermatologists and five registrars. Our model achieved higher diagnostic accuracy than clinicians (63.69% vs. 54.33%, respectively) and provided an earlier diagnosis of melanoma (60.7% vs. 32.7% of melanoma correctly diagnosed on the first follow-up images). These results demonstrate that our model can be used to identify melanocytic lesions that are at high-risk of malignant transformation earlier in the disease process and thereby redefine what is possible in the early detection of melanoma.

Registration of IRT and visible light images in neurosurgery: analysis and comparison of automatic intensity-based registration approaches

PURPOSE

The purpose of this study is to analyze and compare six automatic intensity-based registration methods for intraoperative infrared thermography (IRT) and visible light imaging (VIS/RGB). The practical requirement is to get a good performance of Euclidean distance between manually set landmarks in reference and target images as well as to achieve a high structural similarity index metric (SSIM) and peak signal-to-noise ratio (PSNR) with respect to the reference image.

METHODS

In this study, preprocessing is applied to bring both image types to a similar intensity. Similarity transformation is employed to align roughly IRT and visible light images. Two optimizers and two measures are used in this process. Thereafter, due to locally different displacement of the brain surface through respiration and heartbeat, two non-rigid transformations are applied, and finally, a bicubic interpolation is carried out to compensate for the resulting estimated transformation. Performance was assessed using eleven image datasets. The registration accuracy of the different computational approaches was assessed based on SSIM and PSNR. Additionally, five concise landmarks for each dataset were selected manually in reference and target images and the Euclidean distance between the corresponding landmarks was compared.

RESULTS

The results are showing that the combination of normalized intensity, mutual information measure with one-plus-one evolutionary optimizer in combination with Demon registration results in improved accuracy and performance as compared to all other methods tested here. Furthermore, the obtained results led to [Formula: see text], [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] registrations for datasets 1, 2, 5, 7, and 8 with respect to the second best result by calculating the mean Euclidean distance of five landmarks.

CONCLUSIONS

We conclude that the mutual information measure with one-plus-one evolutionary optimizer in combination with Demon registration can achieve better accuracy and performance to those other methods mentioned here for automatic registration of IRT and visible light images in neurosurgery.

Anatomical variability, multi-modal coordinate systems, and precision targeting in the marmoset brain

Localising accurate brain regions needs careful evaluation in each experimental species due to their individual variability. However, the function and connectivity of brain areas is commonly studied using a single-subject cranial landmark-based stereotactic atlas in animal neuroscience. Here, we address this issue in a small primate, the common marmoset, which is increasingly widely used in systems neuroscience. We developed a non-invasive multi-modal neuroimaging-based targeting pipeline, which accounts for intersubject anatomical variability in cranial and cortical landmarks in marmosets. This methodology allowed creation of multi-modal templates (MarmosetRIKEN20) including head CT and brain MR images, embedded in coordinate systems of anterior and posterior commissures (AC-PC) and CIFTI grayordinates. We found that the horizontal plane of the stereotactic coordinate was significantly rotated in pitch relative to the AC-PC coordinate system (10 degrees, frontal downwards), and had a significant bias and uncertainty due to positioning procedures. We also found that many common cranial and brain landmarks (e.g., bregma, intraparietal sulcus) vary in location across subjects and are substantial relative to average marmoset cortical area dimensions. Combining the neuroimaging-based targeting pipeline with robot-guided surgery enabled proof-of-concept targeting of deep brain structures with an accuracy of 0.2 mm. Altogether, our findings demonstrate substantial intersubject variability in marmoset brain and cranial landmarks, implying that subject-specific neuroimaging-based localization is needed for precision targeting in marmosets. The population-based templates and atlases in grayordinates, created for the first time in marmoset monkeys, should help bridging between macroscale and microscale analyses.