Comparison of physics-based deformable registration methods for image-guided neurosurgery

This paper compares three finite element-based methods used in a physics-based non-rigid registration approach and reports on the progress made over the last 15 years. Large brain shifts caused by brain tumor removal affect registration accuracy by creating point and element outliers. A combination of approximation- and geometry-based point and element outlier rejection improves the rigid registration error by 2.5 mm and meets the real-time constraints (4 min). In addition, the paper raises several questions and presents two open problems for the robust estimation and improvement of registration error in the presence of outliers due to sparse, noisy, and incomplete data. It concludes with preliminary results on leveraging Quantum Computing, a promising new technology for computationally intensive problems like Feature Detection and Block Matching in addition to finite element solver; all three account for 75% of computing time in deformable registration.

Robust automatic hexahedral cartilage meshing framework enables population-based computational studies of the knee

Osteoarthritis of the knee is increasingly prevalent as our population ages, representing an increasing financial burden, and severely impacting quality of life. The invasiveness of in vivo procedures and the high cost of cadaveric studies has left computational tools uniquely suited to study knee biomechanics. Developments in deep learning have great potential for efficiently generating large-scale datasets to enable researchers to perform population-sized investigations, but the time and effort associated with producing robust hexahedral meshes has been a limiting factor in expanding finite element studies to encompass a population. Here we developed a fully automated pipeline capable of taking magnetic resonance knee images and producing a working finite element simulation. We trained an encoder-decoder convolutional neural network to perform semantic image segmentation on the Imorphics dataset provided through the Osteoarthritis Initiative. The Imorphics dataset contained 176 image sequences with varying levels of cartilage degradation. Starting from an open-source swept-extrusion meshing algorithm, we further developed this algorithm until it could produce high quality meshes for every sequence and we applied a template-mapping procedure to automatically place soft-tissue attachment points. The meshing algorithm produced simulation-ready meshes for all 176 sequences, regardless of the use of provided (manually reconstructed) or predicted (automatically generated) segmentation labels. The average time to mesh all bones and cartilage tissues was less than 2 min per knee on an AMD Ryzen 5600X processor, using a parallel pool of three workers for bone meshing, followed by a pool of four workers meshing the four cartilage tissues. Of the 176 sequences with provided segmentation labels, 86% of the resulting meshes completed a simulated flexion-extension activity. We used a reserved testing dataset of 28 sequences unseen during network training to produce simulations derived from predicted labels. We compared tibiofemoral contact mechanics between manual and automated reconstructions for the 24 pairs of successful finite element simulations from this set, resulting in mean root-mean-squared differences under 20% of their respective min-max norms. In combination with further advancements in deep learning, this framework represents a feasible pipeline to produce population sized finite element studies of the natural knee from subject-specific models.

PIMesh: An automatic point cloud and unstructured mesh generation algorithm for meshless methods and finite element analysis-with applications in surgical simulations

We propose a point cloud and mesh generation algorithm, particle injection mesh generator (PIMesh), that can be used to generate optimized high-quality point clouds and unstructured meshes for domains in any shape with minimum (or even no) user intervention. The domains can be scanned images in OBJ format in 2D and 3D or just a line drawing in 2D. Mesh grading can also be easily controlled. The PIMesh is robust and easy to be implemented and is useful for a variety of applications, ranging from generating point clouds for meshless methods, mesh generation for finite element methods, computer graphics applications and surgical simulators. The core idea of the PIMesh is that a mesh domain is considered as an "airtight container" into which particles are "injected" at one or multiple selected interior points. The motion of the particles is controlled by a pseudo-molecular dynamics (PMD) formulation with a pairwise purely repelling "force" moderated by an absolute velocity dependent drag force. The particles repel each other and occupy the whole domain somewhat like blowing up a balloon. When the container is full of particles and the motion is stopped (the particles can be considered as a point cloud), a Delaunay triangulation algorithm is employed to link the particles together to generate an unstructured mesh. The performance of the PIMesh and the comparison with other unstructured mesh generation approaches are demonstrated through generating node distributions and meshes for several 2D and 3D object domains including a scanned image of bones and others.

BlenderPhotonics: an integrated open-source software environment for three-dimensional meshing and photon simulations in complex tissues

SIGNIFICANCE

Rapid advances in biophotonics techniques require quantitative, model-based computational approaches to obtain functional and structural information from increasingly complex and multiscaled anatomies. The lack of efficient tools to accurately model tissue structures and subsequently perform quantitative multiphysics modeling greatly impedes the clinical translation of these modalities.

AIM

Although the mesh-based Monte Carlo (MMC) method expands our capabilities in simulating complex tissues using tetrahedral meshes, the generation of such domains often requires specialized meshing tools, such as Iso2Mesh. Creating a simplified and intuitive interface for tissue anatomical modeling and optical simulations is essential toward making these advanced modeling techniques broadly accessible to the user community.

APPROACH

We responded to the above challenge by combining the powerful, open-source three-dimensional (3D) modeling software, Blender, with state-of-the-art 3D mesh generation and MC simulation tools, utilizing the interactive graphical user interface in Blender as the front-end to allow users to create complex tissue mesh models and subsequently launch MMC light simulations.

RESULTS

Here, we present a tutorial to our Python-based Blender add-on-BlenderPhotonics-to interface with Iso2Mesh and MMC, which allows users to create, configure and refine complex simulation domains and run hardware-accelerated 3D light simulations with only a few clicks. We provide a comprehensive introduction to this tool and walk readers through five examples, ranging from simple shapes to sophisticated realistic tissue models.

CONCLUSIONS

BlenderPhotonics is user friendly and open source, and it leverages the vastly rich ecosystem of Blender. It wraps advanced modeling capabilities within an easy-to-use and interactive interface. The latest software can be downloaded at http://mcx.space/bp.

Adaptive Physics-Based Non-Rigid Registration for Immersive Image-Guided Neuronavigation Systems

Objective: In image-guided neurosurgery, co-registered preoperative anatomical, functional, and diffusion tensor imaging can be used to facilitate a safe resection of brain tumors in eloquent areas of the brain. However, the brain deforms during surgery, particularly in the presence of tumor resection. Non-Rigid Registration (NRR) of the preoperative image data can be used to create a registered image that captures the deformation in the intraoperative image while maintaining the quality of the preoperative image. Using clinical data, this paper reports the results of a comparison of the accuracy and performance among several non-rigid registration methods for handling brain deformation. A new adaptive method that automatically removes mesh elements in the area of the resected tumor, thereby handling deformation in the presence of resection is presented. To improve the user experience, we also present a new way of using mixed reality with ultrasound, MRI, and CT.

Materials and methods: This study focuses on 30 glioma surgeries performed at two different hospitals, many of which involved the resection of significant tumor volumes. An Adaptive Physics-Based Non-Rigid Registration method (A-PBNRR) registers preoperative and intraoperative MRI for each patient. The results are compared with three other readily available registration methods: a rigid registration implemented in 3D Slicer v4.4.0; a B-Spline non-rigid registration implemented in 3D Slicer v4.4.0; and PBNRR implemented in ITKv4.7.0, upon which A-PBNRR was based. Three measures were employed to facilitate a comprehensive evaluation of the registration accuracy: (i) visual assessment, (ii) a Hausdorff Distance-based metric, and (iii) a landmark-based approach using anatomical points identified by a neurosurgeon.

Results: The A-PBNRR using multi-tissue mesh adaptation improved the accuracy of deformable registration by more than five times compared to rigid and traditional physics based non-rigid registration, and four times compared to B-Spline interpolation methods which are part of ITK and 3D Slicer. Performance analysis showed that A-PBNRR could be applied, on average, in <2 min, achieving desirable speed for use in a clinical setting.

Conclusions: The A-PBNRR method performed significantly better than other readily available registration methods at modeling deformation in the presence of resection. Both the registration accuracy and performance proved sufficient to be of clinical value in the operating room. A-PBNRR, coupled with the mixed reality system, presents a powerful and affordable solution compared to current neuronavigation systems.

3D Biomimetic Tongue-Emulating Surfaces for Tribological Applications

Oral friction on the tongue surface plays a pivotal role in mechanics of food transport, speech, sensing, and hedonic responses. The highly specialized biophysical features of the human tongue such as micropapillae-dense topology, optimum wettability, and deformability present architectural challenges in designing artificial tongue surfaces, and the absence of such a biomimetic surface impedes the fundamental understanding of tongue-food/fluid interaction. Herein, we fabricate for the first time, a 3D soft biomimetic surface that replicates the topography and wettability of a real human tongue. The 3D-printed fabrication contains a Poisson point process-based (random) papillae distribution and is employed to micromold soft silicone surfaces with wettability modifications. We demonstrate the unprecedented capability of these surfaces to replicate the theoretically defined and simulated collision probability of papillae and to closely resemble the tribological performances of human tongue masks. These de novo biomimetic surfaces pave the way for accurate quantification of mechanical interactions in the soft oral mucosa.

Reliability and accuracy assessment of morphometric measurements obtained with software for three-dimensional reconstruction of brain aneurysms relative to cerebral angiography measures

OBJECTIVE

To analyze the reliability and accuracy of morphological measurements of software employed to three-dimensionally reconstruct aneurysms and vessels (VMTKlab, version 1.6.1,) with computed tomography angiography (CTA) as the source of images. Agreement with measurements from three-dimensional digital subtraction angiography (3 D-DSA) was evaluated.

METHODS

We evaluated 40 patients presenting with aneurysmal subarachnoid hemorrhage (aSAH). We analyzed four main variables of the aneurysm morphology: absolute height (size), neck (maximum neck width), perpendicular height, and maximum width. The CTA images were uploaded to the software and then segmented to reconstruct the aneurysm. This new method was compared to the current gold standard-3D reconstruction of pretreatment cerebral angiography. We used intraclass correlation coefficient (ICC) and Bland-Altman plot analyses to evaluate the agreement between these methods.

RESULTS

The ICCs obtained for absolute height, neck, perpendicular height, and maximum width were 0.85, 0.57, 0.85, and 0.89, respectively. This implied good agreement except for the neck of the aneurysm (moderate agreement). Bland-Altman plots are presented for the four indexes. The average of the differences was not significant in terms of absolute height, perpendicular height, and maximum width indicating good agreement. However, it was significant for the neck of the aneurysm.

CONCLUSIONS

We report good agreement between the values generated using VMTKlab and cerebral angiography for three of the four main variables. Discrepancies in neck diameter are not surprising and its underestimation with a traditional delineation from cerebral angiography has been reported before.

WingMesh: A Matlab-Based Application for Finite Element Modeling of Insect Wings

The finite element (FE) method is one of the most widely used numerical techniques for the simulation of the mechanical behavior of engineering and biological objects. Although very efficient, the use of the FE method relies on the development of accurate models of the objects under consideration. The development of detailed FE models of often complex-shaped objects, however, can be a time-consuming and error-prone procedure in practice. Hence, many researchers aim to reach a compromise between the simplicity and accuracy of their developed models. In this study, we adapted Distmesh2D, a popular meshing tool, to develop a powerful application for the modeling of geometrically complex objects, such as insect wings. The use of the burning algorithm (BA) in digital image processing (DIP) enabled our method to automatically detect an arbitrary domain and its subdomains in a given image. This algorithm, in combination with the mesh generator Distmesh2D, was used to develop detailed FE models of both planar and out-of-plane (i.e., three-dimensionally corrugated) domains containing discontinuities and consisting of numerous subdomains. To easily implement the method, we developed an application using the Matlab App Designer. This application, called WingMesh, was particularly designed and applied for rapid numerical modeling of complicated insect wings but is also applicable for modeling purposes in the earth, engineering, mathematical, and physical sciences.

Dual-grid mesh-based Monte Carlo algorithm for efficient photon transport simulations in complex three-dimensional media

The mesh-based Monte Carlo (MMC) method is an efficient algorithm to model light propagation inside tissues with complex boundaries, but choosing appropriate mesh density can be challenging. A fine mesh improves the spatial resolution of the output but requires more computation. We propose an improved MMC-dual-grid mesh-based Monte Carlo (DMMC)-to accelerate photon simulations using a coarsely tessellated tetrahedral mesh for ray-tracing computation and an independent voxelated grid for output data storage. The decoupling between ray-tracing and data storage grids allows us to simultaneously achieve faster simulations and improved output spatial accuracy. Furthermore, we developed an optimized ray-tracing technique to eliminate unnecessary ray-tetrahedron intersection tests in optically thick mesh elements. We validate the proposed algorithms using a complex heterogeneous domain and compare the solutions with those from MMC and voxel-based Monte Carlo. We found that DMMC with an unrefined constrained Delaunay tessellation of the boundary nodes yielded the highest speedup, ranging from 1.3  ×   to 2.9  ×   for various scattering settings, with nearly no loss in accuracy. In addition, the optimized ray-tracing technique offers excellent acceleration in high-scattering media, reducing the ray-tetrahedron test count by over 100-fold. Our DMMC software can be downloaded at http://mcx.space/mmc.

Numerical study on the hemodynamics of patient-specific carotid bifurcation using a new mesh approach

The definition of a suitable mesh to simulate blood flow in the human carotid bifurcation has been investigated. In this research, a novel mesh generation method is proposed: hexahedral cells at the center of the vessel and a fine grid of tetrahedral cells near the artery wall, in order to correctly simulate the large blood velocity gradients associated with specific locations. The selected numerical examples to show the pertinence of the novel generation method are supported by carotid ultrasound image data of a patient-specific case. Doppler systolic blood velocities measured during ultrasound examination are compared with simulated velocities using 4 different combinations of hexahedral and tetrahedral meshes and different fluid dynamic simulators. The Lin's test was applied to show the concordance of the results. Wall shear stress-based descriptors and localized normalized helicity descriptor emphasize the performance of the new method. Another feature is the reduced computation time required by the developed methodology. With the accurate combined mesh, different flow rate partitions, between the internal carotid artery and external carotid artery, were studied. The overall effect of the partitions is mainly in the blood flow patterns and in the hot-spot modulation of atherosclerosis-susceptible regions, rather than in their distribution along the bifurcation.

Left Ventricular Diastolic and Systolic Material Property Estimation from Image Data: LV Mechanics Challenge

Cardiovascular simulations using patient-specific geometries can help researchers understand the mechanical behavior of the heart under different loading or disease conditions. However, to replicate the regional mechanics of the heart accurately, both the nonlinear passive and active material properties must be estimated reliably. In this paper, automated methods were used to determine passive material properties while simultaneously computing the unloaded reference geometry of the ventricles for stress analysis. Two different approaches were used to model systole. In the first, a physiologically-based active contraction model [1] coupled to a hemodynamic three-element Windkessel model of the circulation was used to simulate ventricular ejection. In the second, developed active tension was directly adjusted to match ventricular volumes at end-systole while prescribing the known end-systolic pressure. These methods were tested in four normal dogs using the data provided for the LV mechanics challenge [2]. The resulting end-diastolic and end-systolic geometry from the simulation were compared with measured image data.

Assessment of Surgical Effects on Patients with Obstructive Sleep Apnea Syndrome Using Computational Fluid Dynamics Simulations

Obstructive sleep apnea syndrome is one of the most common sleep disorders. To treat patients with this health problem, it is important to detect the severity of this syndrome and occlusion sites in each patient. The goal of this study is to test the hypothesis that the cure of obstructive sleep apnea syndrome by maxillomandibular advancement surgery can be predicted by analyzing the effect of anatomical airway changes on the pressure effort required for normal breathing using a high-fidelity, 3-D numerical model. The employed numerical model consists of: 1) 3-D upper airway geometry construction from patient-specific computed tomographic scans using an image segmentation technique, 2) mixed-element mesh generation of the numerically constructed airway geometry for discretizing the domain of interest, and 3) computational fluid dynamics simulations for predicting the flow field within the airway and the degree of severity of breathing obstruction. In the present study, both laminar and turbulent flow simulations were performed to predict the flow field in the upper airway of the selected patients before and after maxillomandibular advancement surgery. Patients of different body mass indices were also studied to assess their effects. The numerical results were analyzed to evaluate the pressure gradient along the upper airway. The magnitude of the pressure gradient is regarded as the pressure effort required for breathing, and the extent of reduction of the pressure effort is taken to measure the success of the surgery. The description of the employed numerical model, numerical results from simulations of various patients, and suggestion for future work are detailed in this paper.

Generating smooth surface meshes from multi-region medical images

Thanks to advances in medical imaging technologies and numerical methods, patient-specific modelling is more and more used to improve diagnosis and to estimate the outcome of surgical interventions. It requires the extraction of the domain of interest from the medical scans of the patient, as well as the discretisation of this geometry. However, extracting smooth multi-material meshes that conform to the tissue boundaries described in the segmented image is still an active field of research. We propose to solve this issue by combining an implicit surface reconstruction method with a multi-region mesh extraction scheme. The surface reconstruction algorithm is based on multi-level partition of unity implicit surfaces, which we extended to the multi-material case. The mesh generation algorithm consists in a novel multi-domain version of the marching tetrahedra. It generates multi-region meshes as a set of triangular surface patches consistently joining each other at material junctions. This paper presents this original meshing strategy, starting from boundary points extraction from the segmented data to heterogeneous implicit surface definition, multi-region surface triangulation and mesh adaptation. Results indicate that the proposed approach produces smooth and high-quality triangular meshes with a reasonable geometric accuracy. Hence, the proposed method is well suited for subsequent volume mesh generation and finite element simulations.

Patient-Specific Geometry Modeling and Mesh Generation for Simulating Obstructive Sleep Apnea Syndrome Cases by Maxillomandibular Advancement

The objective of this paper is the reconstruction of upper airway geometric models as hybrid meshes from clinically used Computed Tomography (CT) data sets in order to understand the dynamics and behaviors of the pre- and postoperative upper airway systems of Obstructive Sleep Apnea Syndrome (OSAS) patients by viscous Computational Fluid Dynamics (CFD) simulations. The selection criteria for OSAS cases studied are discussed because two reasonable pre- and postoperative upper airway models for CFD simulations may not be created for every case without a special protocol for CT scanning. The geometry extraction and manipulation methods are presented with technical barriers that must be overcome so that they can be used along with computational simulation software as a daily clinical evaluation tool. Eight cases are presented in this paper, and each case consists of pre- and postoperative configurations. The results of computational simulations of two cases are included in this paper as demonstration.

Robust adaptive 3-D segmentation of vessel laminae from fluorescence confocal microscope images and parallel GPU implementation

This paper presents robust 3-D algorithms to segment vasculature that is imaged by labeling laminae, rather than the lumenal volume. The signal is weak, sparse, noisy, nonuniform, low-contrast, and exhibits gaps and spectral artifacts, so adaptive thresholding and Hessian filtering based methods are not effective. The structure deviates from a tubular geometry, so tracing algorithms are not effective. We propose a four step approach. The first step detects candidate voxels using a robust hypothesis test based on a model that assumes Poisson noise and locally planar geometry. The second step performs an adaptive region growth to extract weakly labeled and fine vessels while rejecting spectral artifacts. To enable interactive visualization and estimation of features such as statistical confidence, local curvature, local thickness, and local normal, we perform the third step. In the third step, we construct an accurate mesh representation using marching tetrahedra, volume-preserving smoothing, and adaptive decimation algorithms. To enable topological analysis and efficient validation, we describe a method to estimate vessel centerlines using a ray casting and vote accumulation algorithm which forms the final step of our algorithm. Our algorithm lends itself to parallel processing, and yielded an 8 x speedup on a graphics processor (GPU). On synthetic data, our meshes had average error per face (EPF) values of (0.1-1.6) voxels per mesh face for peak signal-to-noise ratios from (110-28 dB). Separately, the error from decimating the mesh to less than 1% of its original size, the EPF was less than 1 voxel/face. When validated on real datasets, the average recall and precision values were found to be 94.66% and 94.84%, respectively.

Variational Generation of Prismatic Boundary-Layer Meshes for Biomedical Computing

Boundary-layer meshes are important for numerical simulations in computational fluid dynamics, including computational biofluid dynamics of air flow in lungs and blood flow in hearts. Generating boundary-layer meshes is challenging for complex biological geometries. In this paper, we propose a novel technique for generating prismatic boundary-layer meshes for such complex geometries. Our method computes a feature size of the geometry, adapts the surface mesh based on the feature size, and then generates the prismatic layers by propagating the triangulated surface using the face-offsetting method. We derive a new variational method to optimize the prismatic layers to improve the triangle shapes and edge orthogonality of the prismatic elements and also introduce simple and effective measures to guarantee the validity of the mesh. Coupled with a high-quality tetrahedral mesh generator for the interior of the domain, our method generates high-quality hybrid meshes for accurate and efficient numerical simulations. We present comparative study to demonstrate the robustness and quality of our method for complex biomedical geometries.

Subject-specific, multiscale simulation of electrophysiology: a software pipeline for image-based models and application examples

Many simulation studies in biomedicine are based on a similar sequence of processing steps, starting from images and running through geometric model generation, assignment of tissue properties, numerical simulation and visualization of the results--a process known as image-based geometric modelling and simulation. We present an overview of software systems for implementing such a sequence both within highly integrated problem-solving environments and in the form of loosely integrated pipelines. Loose integration in this case indicates that individual programs function largely independently but communicate through files of a common format and support simple scripting, so as to automate multiple executions wherever possible. We then describe three specific applications of such pipelines to translational biomedical research in electrophysiology.

Encoding probabilistic brain atlases using Bayesian inference

This paper addresses the problem of creating probabilistic brain atlases from manually labeled training data. Probabilistic atlases are typically constructed by counting the relative frequency of occurrence of labels in corresponding locations across the training images. However, such an "averaging" approach generalizes poorly to unseen cases when the number of training images is limited, and provides no principled way of aligning the training datasets using deformable registration. In this paper, we generalize the generative image model implicitly underlying standard "average" atlases, using mesh-based representations endowed with an explicit deformation model. Bayesian inference is used to infer the optimal model parameters from the training data, leading to a simultaneous group-wise registration and atlas estimation scheme that encompasses standard averaging as a special case. We also use Bayesian inference to compare alternative atlas models in light of the training data, and show how this leads to a data compression problem that is intuitive to interpret and computationally feasible. Using this technique, we automatically determine the optimal amount of spatial blurring, the best deformation field flexibility, and the most compact mesh representation. We demonstrate, using 2-D training datasets, that the resulting models are better at capturing the structure in the training data than conventional probabilistic atlases. We also present experiments of the proposed atlas construction technique in 3-D, and show the resulting atlases' potential in fully-automated, pulse sequence-adaptive segmentation of 36 neuroanatomical structures in brain MRI scans.

IA-FEMesh: an open-source, interactive, multiblock approach to anatomic finite element model development

Finite element (FE) analysis is a valuable tool in musculoskeletal research. The demands associated with mesh development, however, often prove daunting. In an effort to facilitate anatomic FE model development we have developed an open-source software toolkit (IA-FEMesh). IA-FEMesh employs a multiblock meshing scheme aimed at hexahedral mesh generation. An emphasis has been placed on making the tools interactive, in an effort to create a user friendly environment. The goal is to provide an efficient and reliable method for model development, visualization, and mesh quality evaluation. While these tools have been developed, initially, in the context of skeletal structures they can be applied to countless applications.