Generalized 3D nonlinear transformations for medical imaging: an object-oriented implementation in VTK

Prior information guided auto-contouring of breast gland for deformable image registration in postoperative breast cancer radiotherapy

Background

Contouring of breast gland in planning CT is important to postoperative radiotherapy of patients after breast conserving surgery (BCS). However, the contouring task is difficult because of the poorer contrast of breast gland in planning CT. To improve its efficiency and accuracy, prior information was introduced in a 3D U-Net model to predict the contour of breast gland automatically.

Methods

The preoperative CT was first aligned to the planning CT via affine registration. The resulting transform was then applied to the contour of breast gland in preoperative CT, and the corresponding contour in planning CT was obtained. This transformed contour was a preliminary estimation of breast gland in planning CT and was used as prior information in a 3D U-Net model to obtain a more accurate contour. For evaluation, the dice similarity coefficient (DSC) and Hausdorff distance (HD) were used to assess the deep learning (DL) model's prediction accuracy.

Results

The average DSC and HD of the prediction model were 0.775±0.065 and 44.979±20.565 for breast gland without the input of prior information, while the average values were 0.830±0.038 and 17.896±5.737 with the input of prior information (0.775 vs. 0.830, P=0.0014<0.05; 44.979 vs. 17.896, P=0.002<0.05).

Conclusions

The prediction accuracy was increased significantly with the introduction of prior information, which provided valuable geometrical distribution of target for model training. This method provides an effective way to identify low-contrast targets from surrounding tissues in CT and will be useful in other image modalities.

An unsupervised 2D-3D deformable registration network (2D3D-RegNet) for cone-beam CT estimation

Acquiring CBCTs from a limited scan angle can help to reduce the imaging time, save the imaging dose, and allow continuous target localizations through arc-based treatments with high temporal resolution. However, insufficient scan angle sampling leads to severe distortions and artifacts in the reconstructed CBCT images, limiting their clinical applicability. 2D-3D deformable registration can map a prior fully-sampled CT/CBCT volume to estimate a new CBCT, based on limited-angle on-board cone-beam projections. The resulting CBCT images estimated by 2D-3D deformable registration can successfully suppress the distortions and artifacts, and reflect up-to-date patient anatomy. However, traditional iterative 2D-3D deformable registration algorithm is very computationally expensive and time-consuming, which takes hours to generate a high quality deformation vector field (DVF) and the CBCT. In this work, we developed an unsupervised, end-to-end, 2D-3D deformable registration framework using convolutional neural networks (2D3D-RegNet) to address the speed bottleneck of the conventional iterative 2D-3D deformable registration algorithm. The 2D3D-RegNet was able to solve the DVFs within 5 seconds for 90 orthogonally-arranged projections covering a combined 90° scan angle, with DVF accuracy superior to 3D-3D deformable registration, and on par with the conventional 2D-3D deformable registration algorithm. We also performed a preliminary robustness analysis of 2D3D-RegNet towards projection angular sampling frequency variations, as well as scan angle offsets. The synergy of 2D3D-RegNet with biomechanical modeling was also evaluated, and demonstrated that 2D3D-RegNet can function as a fast DVF solution core for further DVF refinement.

Modeling Tool for Rapid Virtual Planning of the Intracardiac Baffle in Double-Outlet Right Ventricle

PURPOSE

Biventricular repair of double-outlet right ventricle (DORV) necessitates the creation of a complex intracardiac baffle. Creation of the optimal baffle design and placement thereof can be challenging to conceptualize, even with 2-dimensional and 3-dimensional images. This report describes a recently developed methodology for creating virtual baffles to inform intraoperative repair.

DESCRIPTION

A total of 3 heart models of DORV were created from cardiac magnetic resonance images. Baffles were created and visualized using custom software.

EVALUATION

This report demonstrates application of the tool to virtual planning of the baffle for repair of DORV in 3 cases. Models were examined by a multidisciplinary team, on screen and in virtual reality. Baffles could be rapidly created and revised to facilitate planning of the surgical procedure.

CONCLUSIONS

Virtual modeling of the baffle pathway by using cardiac magnetic resonance, creation of physical templates for the baffle, and visualization in virtual reality are feasible and may be beneficial for preoperative planning of complex biventricular repairs in DORV. Further work is needed to demonstrate clinical benefit or improvement in outcomes.

Dynamic CTA-Derived Perfusion Maps Predict Final Infarct Volume: The Simple Perfusion Reconstruction Algorithm

BACKGROUND AND PURPOSE

Infarct core volume measurement using CTP (CT perfusion) is a mainstay paradigm for stroke treatment decision-making. Yet, there are several downfalls with cine CTP technology that can be overcome by adopting the simple perfusion reconstruction algorithm (SPIRAL) derived from multiphase CTA. We compare SPIRAL with CTP parameters for the prediction of 24-hour infarction.

MATERIALS AND METHODS

Seventy-two patients had admission NCCT, multiphase CTA, CTP, and 24-hour DWI. All patients had successful/quality reperfusion. Patient-level and cohort-level receiver operator characteristic curves were generated to determine accuracy. A 10-fold cross-validation was performed on the cohort-level data. Infarct core volume was compared for SPIRAL, CTP-time-to-maximum, and final DWI by Bland-Altman analysis.

RESULTS

When we compared the accuracy in patients with early and late reperfusion for cortical GM and WM, there was no significant difference at the patient level (0.83 versus 0.84, respectively), cohort level (0.82 versus 0.81, respectively), or the cross-validation (0.77 versus 0.74, respectively). In the patient-level receiver operating characteristic analysis, the SPIRAL map had a slightly higher, though nonsignificant (P < .05), average receiver operating characteristic area under the curve (cortical GM/WM, r = 0.82; basal ganglia  = 0.79, respectively) than both the CTP-time-to-maximum (cortical GM/WM = 0.82; basal ganglia = 0.78, respectively) and CTP-CBF (cortical GM/WM = 0.74; basal ganglia = 0.78, respectively) parameter maps. The same relationship was observed at the cohort level. The Bland-Altman plot limits of agreement for SPIRAL and time-to-maximum infarct volume were similar compared with 24-hour DWI.

CONCLUSIONS

We have shown that perfusion maps generated from a temporally sampled helical CTA are an accurate surrogate for infarct core.

Deep Brain Stimulation of the Hypothalamus Leads to Increased Metabolic Rate in Refractory Obesity

OBJECTIVE

Obesity has become a worldwide epidemic, with very few long-term successful treatment options for refractory disease. Deep brain stimulation (DBS) of the bilateral lateral hypothalamus (LH) in refractory obesity has been performed safely. However, questions remain regarding the optimal settings and its effects on metabolic rate. The goals of our experiment were to determine the optimal DBS settings and the actual effect of optimal stimulation on energy expenditure.

METHODS

After bilateral LH DBS implantation, 2 subjects with treatment refractory obesity underwent 4 days of metabolic testing. The subjects slept overnight in a respiratory chamber to measure their baseline sleep energy expenditure, followed by 4 consecutive days of resting metabolic rate (RMR) testing at different stimulation settings. On day 4, the optimized DBS settings were used, and sleep energy expenditure was measured again overnight in the room calorimeter.

RESULTS

During daily testing, the RMR fluctuated acutely with changes in stimulation settings and returned to baseline immediately after turning off the stimulation. Optimal stimulation settings selected for participants showed a 20% and 16% increase in RMR for the 2 participants. Overnight sleep energy expenditure measurements at these optimized settings on day 4 yielded a 10.4% and 4.8% increase over the baseline measurements for the 2 participants.

CONCLUSIONS

These findings have demonstrated the efficacy of optimized DBS of the LH on increasing the RMR acutely and maintaining this increase during overnight sleep. These promising preliminary findings have laid the groundwork for the possible treatment of refractory obesity with DBS.

A Biomechanical Modeling Guided CBCT Estimation Technique

Two-dimensional-to-three-dimensional (2D-3D) deformation has emerged as a new technique to estimate cone-beam computed tomography (CBCT) images. The technique is based on deforming a prior high-quality 3D CT/CBCT image to form a new CBCT image, guided by limited-view 2D projections. The accuracy of this intensity-based technique, however, is often limited in low-contrast image regions with subtle intensity differences. The solved deformation vector fields (DVFs) can also be biomechanically unrealistic. To address these problems, we have developed a biomechanical modeling guided CBCT estimation technique (Bio-CBCT-est) by combining 2D-3D deformation with finite element analysis (FEA)-based biomechanical modeling of anatomical structures. Specifically, Bio-CBCT-est first extracts the 2D-3D deformation-generated displacement vectors at the high-contrast anatomical structure boundaries. The extracted surface deformation fields are subsequently used as the boundary conditions to drive structure-based FEA to correct and fine-tune the overall deformation fields, especially those at low-contrast regions within the structure. The resulting FEA-corrected deformation fields are then fed back into 2D-3D deformation to form an iterative loop, combining the benefits of intensity-based deformation and biomechanical modeling for CBCT estimation. Using eleven lung cancer patient cases, the accuracy of the Bio-CBCT-est technique has been compared to that of the 2D-3D deformation technique and the traditional CBCT reconstruction techniques. The accuracy was evaluated in the image domain, and also in the DVF domain through clinician-tracked lung landmarks.

Lateral hypothalamic area deep brain stimulation for refractory obesity: a pilot study with preliminary data on safety, body weight, and energy metabolism

OBJECT

Deep brain stimulation (DBS) of the lateral hypothalamic area (LHA) has been suggested as a potential treatment for intractable obesity. The authors present the 2-year safety results as well as early efficacy and metabolic effects in 3 patients undergoing bilateral LHA DBS in the first study of this approach in humans.

METHODS

Three patients meeting strict criteria for intractable obesity, including failed bariatric surgery, underwent bilateral implantation of LHA DBS electrodes as part of an institutional review board- and FDA-approved pilot study. The primary focus of the study was safety; however, the authors also received approval to collect data on early efficacy including weight change and energy metabolism.

RESULTS

No serious adverse effects, including detrimental psychological consequences, were observed with continuous LHA DBS after a mean follow-up of 35 months (range 30-39 months). Three-dimensional nonlinear transformation of postoperative imaging superimposed onto brain atlas anatomy was used to confirm and study DBS contact proximity to the LHA. No significant weight loss trends were seen when DBS was programmed using standard settings derived from movement disorder DBS surgery. However, promising weight loss trends have been observed when monopolar DBS stimulation has been applied via specific contacts found to increase the resting metabolic rate measured in a respiratory chamber.

CONCLUSIONS

Deep brain stimulation of the LHA may be applied safely to humans with intractable obesity. Early evidence for some weight loss under metabolically optimized settings provides the first "proof of principle" for this novel antiobesity strategy. A larger follow-up study focused on efficacy along with a more rigorous metabolic analysis is planned to further explore the benefits and therapeutic mechanism behind this investigational therapy.

The Virtual Physiological Human ToolKit

The Virtual Physiological Human (VPH) is a major European e-Science initiative intended to support the development of patient-specific computer models and their application in personalized and predictive healthcare. The VPH Network of Excellence (VPH-NoE) project is tasked with facilitating interaction between the various VPH projects and addressing issues of common concern. A key deliverable is the 'VPH ToolKit'--a collection of tools, methodologies and services to support and enable VPH research, integrating and extending existing work across Europe towards greater interoperability and sustainability. Owing to the diverse nature of the field, a single monolithic 'toolkit' is incapable of addressing the needs of the VPH. Rather, the VPH ToolKit should be considered more as a 'toolbox' of relevant technologies, interacting around a common set of standards. The latter apply to the information used by tools, including any data and the VPH models themselves, and also to the naming and categorizing of entities and concepts involved. Furthermore, the technologies and methodologies available need to be widely disseminated, and relevant tools and services easily found by researchers. The VPH-NoE has thus created an online resource for the VPH community to meet this need. It consists of a database of tools, methods and services for VPH research, with a Web front-end. This has facilities for searching the database, for adding or updating entries, and for providing user feedback on entries. Anyone is welcome to contribute.

Utility of neuronavigation and neuromonitoring in epilepsy surgery

The management of medically refractory epilepsy poses both a valuable therapeutic opportunity and a formidable technical challenge to epilepsy surgeons. Recent decades have produced significant advancements in the capabilities and availability of adjunctive tools in epilepsy surgery. In particular, image-based neuronavigation and electrophysiological neuromonitoring represent versatile and informative modalities that can assist a surgeon in performing safe and effective resections. In the present article the authors discuss these 2 subjects with reference to how they can be applied and what evidence supports their use. As technologies evolve with demonstrated and potential utility, it is important for all clinicians who deal with epilepsy to understand where neuronavigation and neuromonitoring stand in the present and what avenues for improvement exist for the future.

Non-rigid alignment of pre-operative MRI, fMRI, and DT-MRI with intra-operative MRI for enhanced visualization and navigation in image-guided neurosurgery

OBJECTIVE

The usefulness of neurosurgical navigation with current visualizations is seriously compromised by brain shift, which inevitably occurs during the course of the operation, significantly degrading the precise alignment between the pre-operative MR data and the intra-operative shape of the brain. Our objectives were (i) to evaluate the feasibility of non-rigid registration that compensates for the brain deformations within the time constraints imposed by neurosurgery, and (ii) to create augmented reality visualizations of critical structural and functional brain regions during neurosurgery using pre-operatively acquired fMRI and DT-MRI.

MATERIALS AND METHODS

Eleven consecutive patients with supratentorial gliomas were included in our study. All underwent surgery at our intra-operative MR imaging-guided therapy facility and have tumors in eloquent brain areas (e.g. precentral gyrus and cortico-spinal tract). Functional MRI and DT-MRI, together with MPRAGE and T2w structural MRI were acquired at 3 T prior to surgery. SPGR and T2w images were acquired with a 0.5 T magnet during each procedure. Quantitative assessment of the alignment accuracy was carried out and compared with current state-of-the-art systems based only on rigid registration.

RESULTS

Alignment between pre-operative and intra-operative datasets was successfully carried out during surgery for all patients. Overall, the mean residual displacement remaining after non-rigid registration was 1.82 mm. There is a statistically significant improvement in alignment accuracy utilizing our non-rigid registration in comparison to the currently used technology (p<0.001).

CONCLUSIONS

We were able to achieve intra-operative rigid and non-rigid registration of (1) pre-operative structural MRI with intra-operative T1w MRI; (2) pre-operative fMRI with intra-operative T1w MRI, and (3) pre-operative DT-MRI with intra-operative T1w MRI. The registration algorithms as implemented were sufficiently robust and rapid to meet the hard real-time constraints of intra-operative surgical decision making. The validation experiments demonstrate that we can accurately compensate for the deformation of the brain and thus can construct an augmented reality visualization to aid the surgeon.

A feasibility study to investigate the use of thin-plate splines to account for prostate deformation

Image registration is an important step in the radiotherapy treatment planning process. It provides a method of fusing different types of diagnostic imaging information. One such application is to combine magnetic resonance spectroscopic images (MRSI) of the prostate with anatomical MRI and/or computed tomography images that are routinely used in the radiation treatment planning of prostate cancer. MRSI provides in vivo information related to the underlying metabolic activity of tissues, and can be related to the presence of cancer. However, the inflated endorectal coil required during MRS imaging poses a potential problem by deforming the prostate when it is filled with approximately 100 cm3 of air during image acquisition. This pushes the prostate superiorly/anteriorly, deforming the prostate and consequently the spectroscopic imaging data in a nonlinear manner. In this application, the coil-deformed MRS images are warped back to a non-deformed state, using a single data set. A nonlinear warping algorithm is presented to achieve this. Results indicate that the algorithm attains an accuracy of 97% (4 cm3 difference) when reproducing the total prostate volume compared to a Radiation Oncologist defined prostate volume. This difference is slightly smaller than the measured intra-operator variance of +/-1.5 cm3 (deflated coil) and the measured algorithm variance of +/-1.0 cm3. Additionally, intraprostatic nodules were used to assess the accuracy of the warping algorithm in regions inside the prostate. While choosing anatomical tie points along the external prostate surface, analysis of the nodules revealed the algorithm accuracy reduced to 63-93%.

Acceleration of 3D, nonlinear warping using standard video graphics hardware: implementation and initial validation

Thin Plate Spline (TPS) transformations are used in many medical imaging algorithms, but are time-prohibitive for iterative or interactive use. We have utilized current generation consumer 3D graphics cards to accelerate the application of the TPS nonlinear transformation by combining hardware accelerated 3D textures, vertex shaders and trilinear interpolation. Our hardware accelerated algorithm warped a 512x512x173 Computed Tomography (CT) dataset in 2.3 s, and the same study, scaled to 256x256x173, in 0.5 s, using a set of 92 landmarks. An accelerated software implementation of the TPS transformation warped the 512x512x173 study in 32.9 s and the 256x256x173 study in 9.5 s. Subtracted images were used for qualitative analysis of the warp and the Mutual Information (MI) and Sum of Absolute Difference (SAD) similarity metrics were used to quantitatively compare the output of the hardware accelerated algorithm to that of the software algorithm. The hardware accelerated algorithm provided a 7-65 times performance increase when compared to the software algorithm, and produced output of comparable quality (MI > 300 and SAD < 1.75%).