1
|
Ringel MJ, Heiselman JS, Richey WL, Meszoely IM, Jarnagin WR, Miga MI. Comparing regularized Kelvinlet functions and the finite element method for registration of medical images to sparse organ data. Med Image Anal 2024; 96:103221. [PMID: 38824864 DOI: 10.1016/j.media.2024.103221] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Revised: 05/06/2024] [Accepted: 05/25/2024] [Indexed: 06/04/2024]
Abstract
Image-guided surgery collocates patient-specific data with the physical environment to facilitate surgical decision making. Unfortunately, these guidance systems commonly become compromised by intraoperative soft-tissue deformations. Nonrigid image-to-physical registration methods have been proposed to compensate for deformations, but clinical utility requires compatibility of these techniques with data sparsity and temporal constraints in the operating room. While finite element models can be effective in sparse data scenarios, computation time remains a limitation to widespread deployment. This paper proposes a registration algorithm that uses regularized Kelvinlets, which are analytical solutions to linear elasticity in an infinite domain, to overcome these barriers. This algorithm is demonstrated and compared to finite element-based registration on two datasets: a phantom liver deformation dataset and an in vivo breast deformation dataset. The regularized Kelvinlets algorithm resulted in a significant reduction in computation time compared to the finite element method. Accuracy as evaluated by target registration error was comparable between methods. Average target registration errors were 4.6 ± 1.0 and 3.2 ± 0.8 mm on the liver dataset and 5.4 ± 1.4 and 6.4 ± 1.5 mm on the breast dataset for the regularized Kelvinlets and finite element method, respectively. Limitations of regularized Kelvinlets include the lack of organ-specific geometry and the assumptions of linear elasticity and infinitesimal strain. Despite limitations, this work demonstrates the generalizability of regularized Kelvinlets registration on two soft-tissue elastic organs. This method may improve and accelerate registration for image-guided surgery, and it shows the potential of using regularized Kelvinlets on medical imaging data.
Collapse
Affiliation(s)
- Morgan J Ringel
- Vanderbilt University, Department of Biomedical Engineering, Nashville, TN, USA; Vanderbilt Institute for Surgery and Engineering, Nashville, TN, USA.
| | - Jon S Heiselman
- Vanderbilt University, Department of Biomedical Engineering, Nashville, TN, USA; Vanderbilt Institute for Surgery and Engineering, Nashville, TN, USA; Memorial Sloan-Kettering Cancer Center, Department of Surgery, New York, NY, USA
| | - Winona L Richey
- Vanderbilt University, Department of Biomedical Engineering, Nashville, TN, USA; Vanderbilt Institute for Surgery and Engineering, Nashville, TN, USA
| | - Ingrid M Meszoely
- Vanderbilt University Medical Center, Division of Surgical Oncology, Nashville, TN, USA
| | - William R Jarnagin
- Memorial Sloan-Kettering Cancer Center, Department of Surgery, New York, NY, USA
| | - Michael I Miga
- Vanderbilt University, Department of Biomedical Engineering, Nashville, TN, USA; Vanderbilt Institute for Surgery and Engineering, Nashville, TN, USA
| |
Collapse
|
2
|
Fang X, Kim D, Xu X, Kuang T, Lampen N, Lee J, Deng HH, Liebschner MAK, Xia JJ, Gateno J, Yan P. Correspondence attention for facial appearance simulation. Med Image Anal 2024; 93:103094. [PMID: 38306802 DOI: 10.1016/j.media.2024.103094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Revised: 12/02/2023] [Accepted: 01/22/2024] [Indexed: 02/04/2024]
Abstract
In orthognathic surgical planning for patients with jaw deformities, it is crucial to accurately simulate the changes in facial appearance that follow the bony movement. Compared with the traditional biomechanics-based methods like the finite-element method (FEM), which are both labor-intensive and computationally inefficient, deep learning-based methods offer an efficient and robust modeling alternative. However, current methods do not account for the physical relationship between facial soft tissue and bony structure, causing them to fall short in accuracy compared to FEM. In this work, we propose an Attentive Correspondence assisted Movement Transformation network (ACMT-Net) to predict facial changes by correlating facial soft tissue changes with bony movement through a point-to-point attentive correspondence matrix. To ensure efficient training, we also introduce a contrastive loss for self-supervised pre-training of the ACMT-Net with a k-Nearest Neighbors (k-NN) based clustering. Experimental results on patients with jaw deformities show that our proposed solution can achieve significantly improved computational efficiency over the state-of-the-art FEM-based method with comparable facial change prediction accuracy.
Collapse
Affiliation(s)
- Xi Fang
- Department of Biomedical Engineering and Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY 12180, USA
| | - Daeseung Kim
- Department of Oral and Maxillofacial Surgery, Houston Methodist Research Institute, Houston, TX 77030, USA
| | - Xuanang Xu
- Department of Biomedical Engineering and Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY 12180, USA
| | - Tianshu Kuang
- Department of Oral and Maxillofacial Surgery, Houston Methodist Research Institute, Houston, TX 77030, USA
| | - Nathan Lampen
- Department of Biomedical Engineering and Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY 12180, USA
| | - Jungwook Lee
- Department of Biomedical Engineering and Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY 12180, USA
| | - Hannah H Deng
- Department of Oral and Maxillofacial Surgery, Houston Methodist Research Institute, Houston, TX 77030, USA
| | | | - James J Xia
- Department of Oral and Maxillofacial Surgery, Houston Methodist Research Institute, Houston, TX 77030, USA; Weill Medical College, Cornell University, New York, NY, 10021, USA
| | - Jaime Gateno
- Department of Oral and Maxillofacial Surgery, Houston Methodist Research Institute, Houston, TX 77030, USA; Weill Medical College, Cornell University, New York, NY, 10021, USA.
| | - Pingkun Yan
- Department of Biomedical Engineering and Center for Biotechnology and Interdisciplinary Studies, Rensselaer Polytechnic Institute, Troy, NY 12180, USA.
| |
Collapse
|
3
|
Kobeissi H, Jilberto J, Karakan MÇ, Gao X, DePalma SJ, Das SL, Quach L, Urquia J, Baker BM, Chen CS, Nordsletten D, Lejeune E. MicroBundleCompute: Automated segmentation, tracking, and analysis of subdomain deformation in cardiac microbundles. PLoS One 2024; 19:e0298863. [PMID: 38530829 DOI: 10.1371/journal.pone.0298863] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 02/01/2024] [Indexed: 03/28/2024] Open
Abstract
Advancing human induced pluripotent stem cell derived cardiomyocyte (hiPSC-CM) technology will lead to significant progress ranging from disease modeling, to drug discovery, to regenerative tissue engineering. Yet, alongside these potential opportunities comes a critical challenge: attaining mature hiPSC-CM tissues. At present, there are multiple techniques to promote maturity of hiPSC-CMs including physical platforms and cell culture protocols. However, when it comes to making quantitative comparisons of functional behavior, there are limited options for reliably and reproducibly computing functional metrics that are suitable for direct cross-system comparison. In addition, the current standard functional metrics obtained from time-lapse images of cardiac microbundle contraction reported in the field (i.e., post forces, average tissue stress) do not take full advantage of the available information present in these data (i.e., full-field tissue displacements and strains). Thus, we present "MicroBundleCompute," a computational framework for automatic quantification of morphology-based mechanical metrics from movies of cardiac microbundles. Briefly, this computational framework offers tools for automatic tissue segmentation, tracking, and analysis of brightfield and phase contrast movies of beating cardiac microbundles. It is straightforward to implement, runs without user intervention, requires minimal input parameter setting selection, and is computationally inexpensive. In this paper, we describe the methods underlying this computational framework, show the results of our extensive validation studies, and demonstrate the utility of exploring heterogeneous tissue deformations and strains as functional metrics. With this manuscript, we disseminate "MicroBundleCompute" as an open-source computational tool with the aim of making automated quantitative analysis of beating cardiac microbundles more accessible to the community.
Collapse
Affiliation(s)
- Hiba Kobeissi
- Department of Mechanical Engineering, Boston University, Boston, MA, United States of America
- Center for Multiscale and Translational Mechanobiology, Boston University, Boston, MA, United States of America
| | - Javiera Jilberto
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States of America
| | - M Çağatay Karakan
- Department of Mechanical Engineering, Boston University, Boston, MA, United States of America
- Photonics Center, Boston University, Boston, MA, United States of America
- Department of Biomedical Engineering, Boston University, Boston, MA, United States of America
| | - Xining Gao
- Department of Biomedical Engineering, Boston University, Boston, MA, United States of America
- Harvard-MIT Program in Health Sciences and Technology, Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, United States of America
- Wyss Institute for Biologically Inspired Engineering, Harvard University, Boston, MA, United States of America
| | - Samuel J DePalma
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States of America
| | - Shoshana L Das
- Department of Biomedical Engineering, Boston University, Boston, MA, United States of America
- Harvard-MIT Program in Health Sciences and Technology, Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, United States of America
- Wyss Institute for Biologically Inspired Engineering, Harvard University, Boston, MA, United States of America
| | - Lani Quach
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States of America
| | - Jonathan Urquia
- Department of Electrical and Computer Engineering, New York Institute of Technology, New York, NY, United States of America
| | - Brendon M Baker
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States of America
| | - Christopher S Chen
- Department of Biomedical Engineering, Boston University, Boston, MA, United States of America
- Wyss Institute for Biologically Inspired Engineering, Harvard University, Boston, MA, United States of America
| | - David Nordsletten
- Department of Biomedical Engineering, University of Michigan, Ann Arbor, MI, United States of America
- Department of Cardiac Surgery, University of Michigan, Ann Arbor, MI, United States of America
- Department of Biomedical Engineering, School of Imaging Sciences and Biomedical Engineering, King's Health Partners, King's College London, King's Health Partners, London, United Kingdom
| | - Emma Lejeune
- Department of Mechanical Engineering, Boston University, Boston, MA, United States of America
- Center for Multiscale and Translational Mechanobiology, Boston University, Boston, MA, United States of America
| |
Collapse
|
4
|
Safdar S, Zwick BF, Yu Y, Bourantas GC, Joldes GR, Warfield SK, Hyde DE, Frisken S, Kapur T, Kikinis R, Golby A, Nabavi A, Wittek A, Miller K. SlicerCBM: automatic framework for biomechanical analysis of the brain. Int J Comput Assist Radiol Surg 2023; 18:1925-1940. [PMID: 37004646 PMCID: PMC10497672 DOI: 10.1007/s11548-023-02881-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 03/17/2023] [Indexed: 04/04/2023]
Abstract
PURPOSE Brain shift that occurs during neurosurgery disturbs the brain's anatomy. Prediction of the brain shift is essential for accurate localisation of the surgical target. Biomechanical models have been envisaged as a possible tool for such predictions. In this study, we created a framework to automate the workflow for predicting intra-operative brain deformations. METHODS We created our framework by uniquely combining our meshless total Lagrangian explicit dynamics (MTLED) algorithm for computing soft tissue deformations, open-source software libraries and built-in functions within 3D Slicer, an open-source software package widely used for medical research. Our framework generates the biomechanical brain model from the pre-operative MRI, computes brain deformation using MTLED and outputs results in the form of predicted warped intra-operative MRI. RESULTS Our framework is used to solve three different neurosurgical brain shift scenarios: craniotomy, tumour resection and electrode placement. We evaluated our framework using nine patients. The average time to construct a patient-specific brain biomechanical model was 3 min, and that to compute deformations ranged from 13 to 23 min. We performed a qualitative evaluation by comparing our predicted intra-operative MRI with the actual intra-operative MRI. For quantitative evaluation, we computed Hausdorff distances between predicted and actual intra-operative ventricle surfaces. For patients with craniotomy and tumour resection, approximately 95% of the nodes on the ventricle surfaces are within two times the original in-plane resolution of the actual surface determined from the intra-operative MRI. CONCLUSION Our framework provides a broader application of existing solution methods not only in research but also in clinics. We successfully demonstrated the application of our framework by predicting intra-operative deformations in nine patients undergoing neurosurgical procedures.
Collapse
Affiliation(s)
- Saima Safdar
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia.
| | - Benjamin F Zwick
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
| | - Yue Yu
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
| | - George C Bourantas
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
- Department of Agriculture, University of Patras Nea Ktiria, 30200, Campus Mesologhi, Greece
| | - Grand R Joldes
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
| | - Simon K Warfield
- Computational Radiology Laboratory, Boston Children's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Damon E Hyde
- Computational Radiology Laboratory, Boston Children's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Sarah Frisken
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Tina Kapur
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Ron Kikinis
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Alexandra Golby
- Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Arya Nabavi
- Department of Neurosurgery, KRH Klinikum Nordstadt, Hannover, Germany
| | - Adam Wittek
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
| | - Karol Miller
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
- Harvard Medical School, Boston, MA, USA
| |
Collapse
|
5
|
Zwick BF, Safdar S, Bourantas GC, Joldes GR, Hyde DE, Warfield SK, Wittek A, Miller K. Image data and computational grids for computing brain shift and solving the electrocorticography forward problem. Data Brief 2023; 48:109122. [PMID: 37128587 PMCID: PMC10147975 DOI: 10.1016/j.dib.2023.109122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/23/2022] [Revised: 03/30/2023] [Accepted: 03/31/2023] [Indexed: 05/03/2023] Open
Abstract
This article describes the dataset applied in the research reported in NeuroImage article "Patient-specific solution of the electrocorticography forward problem in deforming brain" [1] that is available for download from the Zenodo data repository (https://zenodo.org/record/7687631) [2]. Preoperative structural and diffusion-weighted magnetic resonance (MR) and postoperative computed tomography (CT) images of a 12-year-old female epilepsy patient under evaluation for surgical intervention were obtained retrospectively from Boston Children's Hospital. We used these images to conduct the analysis at The University of Western Australia's Intelligent Systems for Medicine Laboratory using SlicerCBM [3], our open-source software extension for the 3D Slicer medical imaging platform. As part of the analysis, we processed the images to extract the patient-specific brain geometry; created computational grids, including a tetrahedral grid for the meshless solution of the biomechanical model and a regular hexahedral grid for the finite element solution of the electrocorticography forward problem; predicted the postoperative MRI and DTI that correspond to the brain configuration deformed by the placement of subdural electrodes using biomechanics-based image warping; and solved the patient-specific electrocorticography forward problem to compute the electric potential distribution within the patient's head using the original preoperative and predicted postoperative image data. The well-established and open-source file formats used in this dataset, including Nearly Raw Raster Data (NRRD) files for images, STL files for surface geometry, and Visualization Toolkit (VTK) files for computational grids, allow other research groups to easily reuse the data presented herein to solve the electrocorticography forward problem accounting for the brain shift caused by implantation of subdural grid electrodes.
Collapse
Affiliation(s)
- Benjamin F. Zwick
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
- Corresponding author.
| | - Saima Safdar
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
| | - George C. Bourantas
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
| | - Grand R. Joldes
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
| | - Damon E. Hyde
- Computational Radiology Laboratory, Boston Children's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Simon K. Warfield
- Computational Radiology Laboratory, Boston Children's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Adam Wittek
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
| | - Karol Miller
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, 35 Stirling Highway, Perth, WA, Australia
- Harvard Medical School, Boston, MA, USA
| |
Collapse
|
6
|
Song J, Xie H, Zhong Y, Gu C, Choi KS. Maximum likelihood-based extended Kalman filter for soft tissue modelling. J Mech Behav Biomed Mater 2023; 137:105553. [PMID: 36375275 DOI: 10.1016/j.jmbbm.2022.105553] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Revised: 10/14/2022] [Accepted: 10/31/2022] [Indexed: 11/09/2022]
Abstract
Realistic modelling of human soft tissue is very important in medical applications. This paper proposes a novel method by dynamically incorporating soft tissue characterisation in the process of soft tissue modelling to increase the modelling fidelity. This method defines nonlinear tissue deformation with unknown mechanical properties as a problem of nonlinear filtering identification to dynamically identify mechanical properties and further estimate nonlinear deformation behaviour of soft tissue. It combines maximum likelihood theory, nonlinear filtering and nonlinear finite element method (NFEM) for modelling of nonlinear tissue deformation behaviour based on dynamic identification of homogeneous tissue properties. On the basis of hyperelasticity, a nonlinear state-space equation is established by discretizing tissue deformation through NFEM for dynamic filtering. A maximum likelihood algorithm is also established to dynamically identify tissue mechanical properties during the deformation process. Upon above, a maximum likelihood-based extended Kalman filter is further developed for dynamically estimating tissue nonlinear deformation based on dynamic identification of tissue mechanical properties. Simulation and experimental analyses reveal that the proposed method not only overcomes the NFEM limitation of expensive computations, but also absorbs the NFEM merit of high accuracy for modelling of homogeneous tissue deformation. Further, the proposed method also effectively identifies tissue mechanical properties during the deformation modelling process.
Collapse
Affiliation(s)
- Jialu Song
- School of Engineering, RMIT University, Australia.
| | - Hujin Xie
- School of Engineering, RMIT University, Australia
| | | | - Chengfan Gu
- Centre of Smart Health, The Hong Kong Polytechnic University, Hong Kong, China
| | - Kup-Sze Choi
- Centre of Smart Health, The Hong Kong Polytechnic University, Hong Kong, China
| |
Collapse
|
7
|
Patient-specific solution of the electrocorticography forward problem in deforming brain. Neuroimage 2022; 263:119649. [PMID: 36167268 DOI: 10.1016/j.neuroimage.2022.119649] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Revised: 08/25/2022] [Accepted: 09/23/2022] [Indexed: 11/22/2022] Open
Abstract
Invasive intracranial electroencephalography (iEEG), or electrocorticography (ECoG), measures electric potential directly on the surface of the brain and can be used to inform treatment planning for epilepsy surgery. Combined with numerical modeling it can further improve accuracy of epilepsy surgery planning. Accurate solution of the iEEG forward problem, which is a crucial prerequisite for solving the iEEG inverse problemin epilepsy seizure onset zone localization, requires accurate representation of the patient's brain geometry and tissue electrical conductivity after implantation of electrodes. However, implantation of subdural grid electrodes causes the brain to deform, which invalidates preoperatively acquired image data. Moreover, postoperative magnetic resonance imaging (MRI) is incompatible with implanted electrodes and computed tomography (CT) has insufficient range of soft tissue contrast, which precludes both MRI and CT from being used to obtain the deformed postoperative geometry. In this paper, we present a biomechanics-based image warping procedure using preoperative MRI for tissue classification and postoperative CT for locating implanted electrodes to perform non-rigid registration of the preoperative image data to the postoperative configuration. We solve the iEEG forward problem on the predicted postoperative geometry using the finite element method (FEM) which accounts for patient-specific inhomogeneity and anisotropy of tissue conductivity. Results for the simulation of a current source in the brain show large differences in electric potential predicted by the models based on the original images and the deformed images corresponding to the brain geometry deformed by placement of invasive electrodes. Computation of the lead field matrix (useful for solution of the iEEG inverse problem) also showed significant differences between the different models. The results suggest that rapid and accurate solution of the forward problem in a deformed brain for a given patient is achievable.
Collapse
|
8
|
Zhang X, Zhang W, Sun W, Song A. A new soft tissue deformation model based on Runge-Kutta: Application in lung. Comput Biol Med 2022; 148:105811. [PMID: 35834968 DOI: 10.1016/j.compbiomed.2022.105811] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2022] [Revised: 06/25/2022] [Accepted: 07/03/2022] [Indexed: 11/30/2022]
Abstract
Flexible body deformation model is the most critical research in the field of telemedicine, which decides whether the deformation process of tissues and organs can be simulated in real time and realistically. Basically, the improvement of model accuracy often means the loss of real-time performance. In order to effectively balance between accuracy and real-time performance, this paper proposes a new model, which uses the step-variable fourth-order Runge-Kutta method for the first time to solve the relationship problem between the external force and displacement of the nodes in the finite element deformation of the lung. To improve the real-time performance of the model, a one-stage solution optimization algorithm is proposed to optimize the step size selection problem. Finally, an accelerated two-level node update algorithm is proposed to further improve the real-time performance. A lung surgery simulation platform with 3DMax2020 and OpenGL4.5 is built to verify the accuracy, efficiency, realism and applicability of the model. Experimental results show that the proposed lung model can achieve real-world visual reproduction during remote surgery, and exceeds other 13 reference models in real-time performance, with natural deformation effect, high simulation accuracy, which is suitable for modeling normal lung and four types of lungs suffering from diseases.
Collapse
Affiliation(s)
- Xiaorui Zhang
- Wuxi Research Institute, Nanjing University of Information Science & Technology, Wuxi, 214100, China; Engineering Research Center of Digital Forensics, Ministry of Education, Jiangsu Engineering Center of Network Monitoring, School of Computer and Software, Nanjing University of Information Science & Technology, Nanjing, 210044, China
| | - Wenzheng Zhang
- Engineering Research Center of Digital Forensics, Ministry of Education, Jiangsu Engineering Center of Network Monitoring, School of Computer and Software, Nanjing University of Information Science & Technology, Nanjing, 210044, China.
| | - Wei Sun
- School of Automation, Nanjing University of Information Science & Technology, Nanjing, 210044, China
| | - Aiguo Song
- State Key Laboratory of Bioelectronics, Jiangsu Key Lab of Remote Measurement and Control, School of Instrument Science and Engineering, Southeast University, Nanjing, 210096, China
| |
Collapse
|
9
|
Kobeissi H, Mohammadzadeh S, Lejeune E. Enhancing Mechanical Metamodels with a Generative Model-Based Augmented Training Dataset. J Biomech Eng 2022; 144:1141932. [PMID: 35767343 DOI: 10.1115/1.4054898] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Indexed: 11/08/2022]
Abstract
Modeling biological soft tissue is complex in part due to material heterogeneity. Microstructural patterns, which play a major role in defining the mechanical behavior of these tissues, are both challenging to characterize, and difficult to simulate. Recently, machine learning (ML)-based methods to predict the mechanical behavior of heterogeneous materials have made it possible to more thoroughly explore the massive input parameter space associated with heterogeneous blocks of material. Specifically, we can train ML models to closely approximate computationally expensive heterogeneous material simulations where the ML model is trained on datasets of simulations with relevant spatial heterogeneity. However, when it comes to applying these techniques to tissue, there is a major limitation: the number of useful examples available to characterize the input domain under study is often limited. In this work, we investigate the efficacy of both ML-based generative models and procedural methods as tools for augmenting limited input pattern datasets. We find that a Style-based Generative Adversarial Network with an adaptive discriminator augmentation mechanism is able to successfully leverage just 1,000 example patterns to create authentic generated patterns. And, we find that diverse generated patterns with adequate resemblance to real patterns can be used as inputs to finite element simulations to meaningfully augment the training dataset. To enable this methodological contribution, we have created an open access Finite Element Analysis simulation dataset based on Cahn-Hilliard patterns. We anticipate that future researchers will be able to leverage this dataset and build on the work presented here.
Collapse
Affiliation(s)
- Hiba Kobeissi
- Department of Mechanical Engineering, Boston University, Boston, MA 02215
| | | | - Emma Lejeune
- Department of Mechanical Engineering, Boston University, Boston, MA 02215
| |
Collapse
|
10
|
Yu Y, Safdar S, Bourantas G, Zwick B, Joldes G, Kapur T, Frisken S, Kikinis R, Nabavi A, Golby A, Wittek A, Miller K. Automatic framework for patient-specific modelling of tumour resection-induced brain shift. Comput Biol Med 2022; 143:105271. [PMID: 35123136 PMCID: PMC9389918 DOI: 10.1016/j.compbiomed.2022.105271] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Revised: 01/09/2022] [Accepted: 01/24/2022] [Indexed: 11/25/2022]
Abstract
Our motivation is to enable non-biomechanical engineering specialists to use sophisticated biomechanical models in the clinic to predict tumour resection-induced brain shift, and subsequently know the location of the residual tumour and its boundary. To achieve this goal, we developed a framework for automatically generating and solving patient-specific biomechanical models of the brain. This framework automatically determines patient-specific brain geometry from MRI data, generates patient-specific computational grid, assigns material properties, defines boundary conditions, applies external loads to the anatomical structures, and solves differential equations of nonlinear elasticity using Meshless Total Lagrangian Explicit Dynamics (MTLED) algorithm. We demonstrated the effectiveness and appropriateness of our framework on real clinical cases of tumour resection-induced brain shift.
Collapse
Affiliation(s)
- Yue Yu
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth 6009, Australia.
| | - Saima Safdar
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth 6009, Australia
| | - George Bourantas
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth 6009, Australia
| | - Benjamin Zwick
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth 6009, Australia
| | - Grand Joldes
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth 6009, Australia
| | - Tina Kapur
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Sarah Frisken
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Ron Kikinis
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Arya Nabavi
- Department of Neurosurgery, KRH Klinikum Nordstadt, Hannover, Germany
| | - Alexandra Golby
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Adam Wittek
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth 6009, Australia
| | - Karol Miller
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth 6009, Australia; Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
11
|
Yu Y, Bourantas G, Zwick B, Joldes G, Kapur T, Frisken S, Kikinis R, Nabavi A, Golby A, Wittek A, Miller K. Computer simulation of tumour resection-induced brain deformation by a meshless approach. INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN BIOMEDICAL ENGINEERING 2022; 38:e3539. [PMID: 34647427 PMCID: PMC8881972 DOI: 10.1002/cnm.3539] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/15/2021] [Revised: 09/06/2021] [Accepted: 10/03/2021] [Indexed: 06/13/2023]
Abstract
Tumour resection requires precise planning and navigation to maximise tumour removal while simultaneously protecting nearby healthy tissues. Neurosurgeons need to know the location of the remaining tumour after partial tumour removal before continuing with the resection. Our approach to the problem uses biomechanical modelling and computer simulation to compute the brain deformations after the tumour is resected. In this study, we use meshless Total Lagrangian explicit dynamics as the solver. The problem geometry is extracted from the patient-specific magnetic resonance imaging (MRI) data and includes the parenchyma, tumour, cerebrospinal fluid and skull. The appropriate non-linear material formulation is used. Loading is performed by imposing intra-operative conditions of gravity and reaction forces between the tumour and surrounding healthy parenchyma tissues. A finite frictionless sliding contact is enforced between the skull (rigid) and parenchyma. The meshless simulation results are compared to intra-operative MRI sections. We also calculate Hausdorff distances between the computed deformed surfaces (ventricles and tumour cavities) and surfaces observed intra-operatively. Over 80% of points on the ventricle surface and 95% of points on the tumour cavity surface were successfully registered (results within the limits of two times the original in-plane resolution of the intra-operative image). Computed results demonstrate the potential for our method in estimating the tissue deformation and tumour boundary during the resection.
Collapse
Affiliation(s)
- Yue Yu
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - George Bourantas
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - Benjamin Zwick
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - Grand Joldes
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - Tina Kapur
- Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Sarah Frisken
- Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Ron Kikinis
- Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Arya Nabavi
- Department of Neurosurgery, KRH Klinikum Nordstadt, Hannover, Germany
| | - Alexandra Golby
- Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Adam Wittek
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - Karol Miller
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
- Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
12
|
Xie H, Song J, Zhong Y, Li J, Gu C, Choi KS. Extended Kalman Filter Nonlinear Finite Element Method for Nonlinear Soft Tissue Deformation. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 200:105828. [PMID: 33199083 DOI: 10.1016/j.cmpb.2020.105828] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/29/2020] [Accepted: 10/31/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND AND OBJECTIVE Soft tissue modelling is crucial to surgery simulation. This paper introduces an innovative approach to realistic simulation of nonlinear deformation behaviours of biological soft tissues in real time. METHODS This approach combines the traditional nonlinear finite-element method (NFEM) and nonlinear Kalman filtering to address both physical fidelity and real-time performance for soft tissue modelling. It defines tissue mechanical deformation as a nonlinear filtering process for dynamic estimation of nonlinear deformation behaviours of biological tissues. Tissue mechanical deformation is discretized in space using NFEM in accordance with nonlinear elastic theory and in time using the central difference scheme to establish the nonlinear state-space models for dynamic filtering. RESULTS An extended Kalman filter is established to dynamically estimate nonlinear mechanical deformation of biological tissues. Interactive deformation of biological soft tissues with haptic feedback is accomplished as well for surgery simulation. CONCLUSIONS The proposed approach conquers the NFEM limitation of step computation but without trading off the modelling accuracy. It not only has a similar level of accuracy as NFEM, but also meets the real-time requirement for soft tissue modelling.
Collapse
Affiliation(s)
- Hujin Xie
- School of Engineering, RMIT University, Australia.
| | - Jialu Song
- School of Engineering, RMIT University, Australia
| | | | - Jiankun Li
- School of Engineering, RMIT University, Australia
| | - Chengfan Gu
- School of Nursing, The Hong Kong Polytechnic University, Hong Kong
| | - Kup-Sze Choi
- School of Nursing, The Hong Kong Polytechnic University, Hong Kong
| |
Collapse
|
13
|
Wittek A, Bourantas G, Zwick BF, Joldes G, Esteban L, Miller K. Mathematical modeling and computer simulation of needle insertion into soft tissue. PLoS One 2020; 15:e0242704. [PMID: 33351854 PMCID: PMC7755224 DOI: 10.1371/journal.pone.0242704] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2020] [Accepted: 11/08/2020] [Indexed: 01/25/2023] Open
Abstract
In this study we present a kinematic approach for modeling needle insertion into soft tissues. The kinematic approach allows the presentation of the problem as Dirichlet-type (i.e. driven by enforced motion of boundaries) and therefore weakly sensitive to unknown properties of the tissues and needle-tissue interaction. The parameters used in the kinematic approach are straightforward to determine from images. Our method uses Meshless Total Lagrangian Explicit Dynamics (MTLED) method to compute soft tissue deformations. The proposed scheme was validated against experiments of needle insertion into silicone gel samples. We also present a simulation of needle insertion into the brain demonstrating the method's insensitivity to assumed mechanical properties of tissue.
Collapse
Affiliation(s)
- Adam Wittek
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - George Bourantas
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - Benjamin F Zwick
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - Grand Joldes
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| | - Lionel Esteban
- Commonwealth Science and Industry Research Organization CSIRO, Medical XCT Facility, Kensington, Western Australia, Australia
| | - Karol Miller
- Intelligent Systems for Medicine Laboratory, The University of Western Australia, Perth, Western Australia, Australia
| |
Collapse
|
14
|
|
15
|
Miller K, Joldes GR, Bourantas G, Warfield S, Hyde DE, Kikinis R, Wittek A. Biomechanical modeling and computer simulation of the brain during neurosurgery. INTERNATIONAL JOURNAL FOR NUMERICAL METHODS IN BIOMEDICAL ENGINEERING 2019; 35:e3250. [PMID: 31400252 PMCID: PMC6785376 DOI: 10.1002/cnm.3250] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/21/2019] [Revised: 06/28/2019] [Accepted: 08/08/2019] [Indexed: 06/10/2023]
Abstract
Computational biomechanics of the brain for neurosurgery is an emerging area of research recently gaining in importance and practical applications. This review paper presents the contributions of the Intelligent Systems for Medicine Laboratory and its collaborators to this field, discussing the modeling approaches adopted and the methods developed for obtaining the numerical solutions. We adopt a physics-based modeling approach and describe the brain deformation in mechanical terms (such as displacements, strains, and stresses), which can be computed using a biomechanical model, by solving a continuum mechanics problem. We present our modeling approaches related to geometry creation, boundary conditions, loading, and material properties. From the point of view of solution methods, we advocate the use of fully nonlinear modeling approaches, capable of capturing very large deformations and nonlinear material behavior. We discuss finite element and meshless domain discretization, the use of the total Lagrangian formulation of continuum mechanics, and explicit time integration for solving both time-accurate and steady-state problems. We present the methods developed for handling contacts and for warping 3D medical images using the results of our simulations. We present two examples to showcase these methods: brain shift estimation for image registration and brain deformation computation for neuronavigation in epilepsy treatment.
Collapse
Affiliation(s)
- K. Miller
- Intelligent Systems for Medicine Laboratory, Department of Mechanical Engineering, The University of Western Australia, 35 Stirling Highway, Perth, WA 6009, Australia
| | - G. R. Joldes
- Intelligent Systems for Medicine Laboratory, Department of Mechanical Engineering, The University of Western Australia, 35 Stirling Highway, Perth, WA 6009, Australia
| | - G. Bourantas
- Intelligent Systems for Medicine Laboratory, Department of Mechanical Engineering, The University of Western Australia, 35 Stirling Highway, Perth, WA 6009, Australia
| | - S.K. Warfield
- Computational Radiology Laboratory, Department of Radiology, Boston Children’s Hospital and Harvard Medical School, 300 Longwood Avenue, Boston MA 02115
| | - D. E. Hyde
- Computational Radiology Laboratory, Department of Radiology, Boston Children’s Hospital and Harvard Medical School, 300 Longwood Avenue, Boston MA 02115
| | - R. Kikinis
- Surgical Planning Laboratory, Brigham and Women’s Hospital and Harvard Medical School, 45 Francis St, Boston, MA 02115
- Medical Image Computing, University of Bremen, Germany
- Fraunhofer MEVIS, Bremen, Germany
| | - A. Wittek
- Intelligent Systems for Medicine Laboratory, Department of Mechanical Engineering, The University of Western Australia, 35 Stirling Highway, Perth, WA 6009, Australia
| |
Collapse
|