1
|
Wodzinski M, Kwarciak K, Daniol M, Hemmerling D. Improving deep learning-based automatic cranial defect reconstruction by heavy data augmentation: From image registration to latent diffusion models. Comput Biol Med 2024; 182:109129. [PMID: 39265478 DOI: 10.1016/j.compbiomed.2024.109129] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2024] [Revised: 08/28/2024] [Accepted: 09/06/2024] [Indexed: 09/14/2024]
Abstract
Modeling and manufacturing of personalized cranial implants are important research areas that may decrease the waiting time for patients suffering from cranial damage. The modeling of personalized implants may be partially automated by the use of deep learning-based methods. However, this task suffers from difficulties with generalizability into data from previously unseen distributions that make it difficult to use the research outcomes in real clinical settings. Due to difficulties with acquiring ground-truth annotations, different techniques to improve the heterogeneity of datasets used for training the deep networks have to be considered and introduced. In this work, we present a large-scale study of several augmentation techniques, varying from classical geometric transformations, image registration, variational autoencoders, and generative adversarial networks, to the most recent advances in latent diffusion models. We show that the use of heavy data augmentation significantly increases both the quantitative and qualitative outcomes, resulting in an average Dice Score above 0.94 for the SkullBreak and above 0.96 for the SkullFix datasets. The results show that latent diffusion models combined with vector quantized variational autoencoder outperform other generative augmentation strategies. Moreover, we show that the synthetically augmented network successfully reconstructs real clinical defects, without the need to acquire costly and time-consuming annotations. The findings of the work will lead to easier, faster, and less expensive modeling of personalized cranial implants. This is beneficial to numerous people suffering from cranial injuries. The work constitutes a considerable contribution to the field of artificial intelligence in the automatic modeling of personalized cranial implants.
Collapse
Affiliation(s)
- Marek Wodzinski
- AGH University of Krakow, Department of Measurement and Electronics, Kraków, al. Mickiewicza 30, PL32059, Poland; University of Applied Sciences Western Switzerland (HES-SO Valais), Information Systems Institute, Sierre, Rue de Technopôle 3, 3960, Switzerland.
| | - Kamil Kwarciak
- AGH University of Krakow, Department of Measurement and Electronics, Kraków, al. Mickiewicza 30, PL32059, Poland
| | - Mateusz Daniol
- AGH University of Krakow, Department of Measurement and Electronics, Kraków, al. Mickiewicza 30, PL32059, Poland
| | - Daria Hemmerling
- AGH University of Krakow, Department of Measurement and Electronics, Kraków, al. Mickiewicza 30, PL32059, Poland
| |
Collapse
|
2
|
Xu J, Wei Y, Jiang S, Zhou H, Li Y, Chen X. Intelligent surgical planning for automatic reconstruction of orbital blowout fracture using a prior adversarial generative network. Med Image Anal 2024; 99:103332. [PMID: 39321669 DOI: 10.1016/j.media.2024.103332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 08/27/2024] [Accepted: 08/29/2024] [Indexed: 09/27/2024]
Abstract
Orbital blowout fracture (OBF) is a disease that can result in herniation of orbital soft tissue, enophthalmos, and even severe visual dysfunction. Given the complex and diverse types of orbital wall fractures, reconstructing the orbital wall presents a significant challenge in OBF repair surgery. Accurate surgical planning is crucial in addressing this issue. However, there is currently a lack of efficient and precise surgical planning methods. Therefore, we propose an intelligent surgical planning method for automatic OBF reconstruction based on a prior adversarial generative network (GAN). Firstly, an automatic generation method of symmetric prior anatomical knowledge (SPAK) based on spatial transformation is proposed to guide the reconstruction of fractured orbital wall. Secondly, a reconstruction network based on SPAK-guided GAN is proposed to achieve accurate and automatic reconstruction of fractured orbital wall. Building upon this, a new surgical planning workflow based on the proposed reconstruction network and 3D Slicer software is developed to simplify the operational steps. Finally, the proposed surgical planning method is successfully applied in OBF repair surgery, verifying its reliability. Experimental results demonstrate that the proposed reconstruction network achieves relatively accurate automatic reconstruction of the orbital wall, with an average DSC of 92.35 ± 2.13% and a 95% Hausdorff distance of 0.59 ± 0.23 mm, markedly outperforming the compared state-of-the-art networks. Additionally, the proposed surgical planning workflow reduces the traditional planning time from an average of 25 min and 17.8 s to just 1 min and 35.1 s, greatly enhancing planning efficiency. In the future, the proposed surgical planning method will have good application prospects in OBF repair surgery.
Collapse
Affiliation(s)
- Jiangchang Xu
- Institute of Biomedical Manufacturing and Life Quality Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200241, China; Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China
| | - Yining Wei
- Department of Ophthalmology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200011, China; Shanghai Key Laboratory of Orbital Diseases and Ocular Oncology, Shanghai 200011, China
| | - Shuanglin Jiang
- Institute of Biomedical Manufacturing and Life Quality Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200241, China
| | - Huifang Zhou
- Department of Ophthalmology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200011, China; Shanghai Key Laboratory of Orbital Diseases and Ocular Oncology, Shanghai 200011, China
| | - Yinwei Li
- Department of Ophthalmology, Shanghai Ninth People's Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200011, China; Shanghai Key Laboratory of Orbital Diseases and Ocular Oncology, Shanghai 200011, China.
| | - Xiaojun Chen
- Institute of Biomedical Manufacturing and Life Quality Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200241, China; Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai 200241, China.
| |
Collapse
|
3
|
Mazzocchetti S, Spezialetti R, Bevini M, Badiali G, Lisanti G, Salti S, Di Stefano L. Neural shape completion for personalized Maxillofacial surgery. Sci Rep 2024; 14:19810. [PMID: 39191797 DOI: 10.1038/s41598-024-68084-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2024] [Accepted: 07/19/2024] [Indexed: 08/29/2024] Open
Abstract
In this paper, we investigate the effectiveness of shape completion neural networks as clinical aids in maxillofacial surgery planning. We present a pipeline to apply shape completion networks to automatically reconstruct complete eumorphic 3D meshes starting from a partial input mesh, easily obtained from CT data routinely acquired for surgery planning. Most of the existing works introduced solutions to aid the design of implants for cranioplasty, i.e. all the defects are located in the neurocranium. In this work, we focus on reconstructing defects localized on both neurocranium and splanchnocranium. To this end, we introduce a new dataset, specifically designed for this task, derived from publicly available CT scans and subjected to a comprehensive pre-processing procedure. All the scans in the dataset have been manually cleaned and aligned to a common reference system. In addition, we devised a pre-processing stage to automatically extract point clouds from the scans and enrich them with virtual defects. We experimentally compare several state-of-the-art point cloud completion networks and identify the two most promising models. Finally, expert surgeons evaluated the best-performing network on a clinical case. Our results show how casting the creation of personalized implants as a problem of shape completion is a promising approach for automatizing this complex task.
Collapse
Affiliation(s)
- Stefano Mazzocchetti
- eDIMES Lab - Laboratory of Bioengineering, Department of Medical and Surgical Sciences, University of Bologna, Bologna, Italy.
| | - Riccardo Spezialetti
- Department of Computer Science and Engineering (DISI), University of Bologna, Bologna, Italy
| | - Mirko Bevini
- Oral and Maxillo-Facial Surgery Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
| | - Giovanni Badiali
- Oral and Maxillo-Facial Surgery Unit, IRCCS Azienda Ospedaliero-Universitaria di Bologna, Bologna, Italy
- Department of Biomedical and Neuromotoric Science (DIBINEM), University of Bologna, Bologna, Italy
| | - Giuseppe Lisanti
- Department of Computer Science and Engineering (DISI), University of Bologna, Bologna, Italy
| | - Samuele Salti
- Department of Computer Science and Engineering (DISI), University of Bologna, Bologna, Italy
| | - Luigi Di Stefano
- Department of Computer Science and Engineering (DISI), University of Bologna, Bologna, Italy
| |
Collapse
|
4
|
Li J, Ellis DG, Pepe A, Gsaxner C, Aizenberg MR, Kleesiek J, Egger J. Back to the Roots: Reconstructing Large and Complex Cranial Defects using an Image-based Statistical Shape Model. J Med Syst 2024; 48:55. [PMID: 38780820 PMCID: PMC11116219 DOI: 10.1007/s10916-024-02066-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2023] [Accepted: 04/11/2024] [Indexed: 05/25/2024]
Abstract
Designing implants for large and complex cranial defects is a challenging task, even for professional designers. Current efforts on automating the design process focused mainly on convolutional neural networks (CNN), which have produced state-of-the-art results on reconstructing synthetic defects. However, existing CNN-based methods have been difficult to translate to clinical practice in cranioplasty, as their performance on large and complex cranial defects remains unsatisfactory. In this paper, we present a statistical shape model (SSM) built directly on the segmentation masks of the skulls represented as binary voxel occupancy grids and evaluate it on several cranial implant design datasets. Results show that, while CNN-based approaches outperform the SSM on synthetic defects, they are inferior to SSM when it comes to large, complex and real-world defects. Experienced neurosurgeons evaluate the implants generated by the SSM to be feasible for clinical use after minor manual corrections. Datasets and the SSM model are publicly available at https://github.com/Jianningli/ssm .
Collapse
Affiliation(s)
- Jianning Li
- Institute for Artificial Intelligence in Medicine (IKIM), Essen University Hospital, Girardetstraße 2, 45131, Essen, Germany.
| | - David G Ellis
- Department of Neurosurgery, University of Nebraska Medical Center, Omaha, NE, 68198, USA
| | - Antonio Pepe
- Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, Graz, 8010, Austria
| | - Christina Gsaxner
- Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, Graz, 8010, Austria
| | - Michele R Aizenberg
- Department of Neurosurgery, University of Nebraska Medical Center, Omaha, NE, 68198, USA
| | - Jens Kleesiek
- Institute for Artificial Intelligence in Medicine (IKIM), Essen University Hospital, Girardetstraße 2, 45131, Essen, Germany
| | - Jan Egger
- Institute for Artificial Intelligence in Medicine (IKIM), Essen University Hospital, Girardetstraße 2, 45131, Essen, Germany.
| |
Collapse
|
5
|
Fishman Z, Mainprize JG, Edwards G, Antonyshyn O, Hardisty M, Whyne CM. Thickness and design features of clinical cranial implants-what should automated methods strive to replicate? Int J Comput Assist Radiol Surg 2024:10.1007/s11548-024-03068-4. [PMID: 38430381 DOI: 10.1007/s11548-024-03068-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Accepted: 01/24/2024] [Indexed: 03/03/2024]
Abstract
PURPOSE New deep learning and statistical shape modelling approaches aim to automate the design process for patient-specific cranial implants, as highlighted by the MICCAI AutoImplant Challenges. To ensure applicability, it is important to determine if the training data used in developing these algorithms represent the geometry of implants designed for clinical use. METHODS Calavera Surgical Design provided a dataset of 206 post-craniectomy skull geometries and their clinically used implants. The MUG500+ dataset includes 29 post-craniectomy skull geometries and implants designed for automating design. For both implant and skull shapes, the inner and outer cortical surfaces were segmented, and the thickness between them was measured. For the implants, a 'rim' was defined that transitions from the repaired defect to the surrounding skull. For unilateral defect cases, skull implants were mirrored to the contra-lateral side and thickness differences were quantified. RESULTS The average thickness of the clinically used implants was 6.0 ± 0.5 mm, which approximates the thickness on the contra-lateral side of the skull (relative difference of -0.3 ± 1.4 mm). The average thickness of the MUG500+ implants was 2.9 ± 1.0 mm, significantly thinner than the intact skull thickness (relative difference of 2.9 ± 1.2 mm). Rim transitions in the clinical implants (average width of 8.3 ± 3.4 mm) were used to cap and create a smooth boundary with the skull. CONCLUSIONS For implant modelers or manufacturers, this shape analysis quantified differences of cranial implants (thickness, rim width, surface area, and volume) to help guide future automated design algorithms. After skull completion, a thicker implant can be more versatile for cases involving muscle hollowing or thin skulls, and wider rims can smooth over the defect margins to provide more stability. For clinicians, the differing measurements and implant designs can help inform the options available for their patient specific treatment.
Collapse
Affiliation(s)
- Z Fishman
- Orthopaedic Biomechanics Laboratory, Sunnybrook Research Institute, Toronto, ON, Canada.
| | | | | | - Oleh Antonyshyn
- Calavera Surgical Design Inc., Toronto, ON, Canada
- Division of Plastic Surgery, Sunnybrook Health Sciences Centre, Toronto, ON, Canada
| | - Michael Hardisty
- Orthopaedic Biomechanics Laboratory, Sunnybrook Research Institute, Toronto, ON, Canada
- Department of Surgery, University of Toronto, Toronto, ON, Canada
| | - C M Whyne
- Orthopaedic Biomechanics Laboratory, Sunnybrook Research Institute, Toronto, ON, Canada
- Department of Surgery, University of Toronto, Toronto, ON, Canada
- Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
6
|
Li J, Gsaxner C, Pepe A, Schmalstieg D, Kleesiek J, Egger J. Sparse convolutional neural network for high-resolution skull shape completion and shape super-resolution. Sci Rep 2023; 13:20229. [PMID: 37981641 PMCID: PMC10658170 DOI: 10.1038/s41598-023-47437-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Accepted: 11/14/2023] [Indexed: 11/21/2023] Open
Abstract
Traditional convolutional neural network (CNN) methods rely on dense tensors, which makes them suboptimal for spatially sparse data. In this paper, we propose a CNN model based on sparse tensors for efficient processing of high-resolution shapes represented as binary voxel occupancy grids. In contrast to a dense CNN that takes the entire voxel grid as input, a sparse CNN processes only on the non-empty voxels, thus reducing the memory and computation overhead caused by the sparse input data. We evaluate our method on two clinically relevant skull reconstruction tasks: (1) given a defective skull, reconstruct the complete skull (i.e., skull shape completion), and (2) given a coarse skull, reconstruct a high-resolution skull with fine geometric details (shape super-resolution). Our method outperforms its dense CNN-based counterparts in the skull reconstruction task quantitatively and qualitatively, while requiring substantially less memory for training and inference. We observed that, on the 3D skull data, the overall memory consumption of the sparse CNN grows approximately linearly during inference with respect to the image resolutions. During training, the memory usage remains clearly below increases in image resolution-an [Formula: see text] increase in voxel number leads to less than [Formula: see text] increase in memory requirements. Our study demonstrates the effectiveness of using a sparse CNN for skull reconstruction tasks, and our findings can be applied to other spatially sparse problems. We prove this by additional experimental results on other sparse medical datasets, like the aorta and the heart. Project page at https://github.com/Jianningli/SparseCNN .
Collapse
Affiliation(s)
- Jianning Li
- Institute for AI in Medicine (IKIM), University Medicine Essen (AöR), Girardetstraße 2, 45131, Essen, Germany.
| | - Christina Gsaxner
- Institute of computer graphics and vision, Graz University of Technology, Graz, Austria
| | - Antonio Pepe
- Institute of computer graphics and vision, Graz University of Technology, Graz, Austria
| | - Dieter Schmalstieg
- Institute of computer graphics and vision, Graz University of Technology, Graz, Austria
| | - Jens Kleesiek
- Institute for AI in Medicine (IKIM), University Medicine Essen (AöR), Girardetstraße 2, 45131, Essen, Germany
| | - Jan Egger
- Institute for AI in Medicine (IKIM), University Medicine Essen (AöR), Girardetstraße 2, 45131, Essen, Germany.
- Institute of computer graphics and vision, Graz University of Technology, Graz, Austria.
| |
Collapse
|
7
|
Li J, Ellis DG, Kodym O, Rauschenbach L, Rieß C, Sure U, Wrede KH, Alvarez CM, Wodzinski M, Daniol M, Hemmerling D, Mahdi H, Clement A, Kim E, Fishman Z, Whyne CM, Mainprize JG, Hardisty MR, Pathak S, Sindhura C, Gorthi RKSS, Kiran DV, Gorthi S, Yang B, Fang K, Li X, Kroviakov A, Yu L, Jin Y, Pepe A, Gsaxner C, Herout A, Alves V, Španěl M, Aizenberg MR, Kleesiek J, Egger J. Towards clinical applicability and computational efficiency in automatic cranial implant design: An overview of the AutoImplant 2021 cranial implant design challenge. Med Image Anal 2023; 88:102865. [PMID: 37331241 DOI: 10.1016/j.media.2023.102865] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Revised: 05/23/2023] [Accepted: 06/02/2023] [Indexed: 06/20/2023]
Abstract
Cranial implants are commonly used for surgical repair of craniectomy-induced skull defects. These implants are usually generated offline and may require days to weeks to be available. An automated implant design process combined with onsite manufacturing facilities can guarantee immediate implant availability and avoid secondary intervention. To address this need, the AutoImplant II challenge was organized in conjunction with MICCAI 2021, catering for the unmet clinical and computational requirements of automatic cranial implant design. The first edition of AutoImplant (AutoImplant I, 2020) demonstrated the general capabilities and effectiveness of data-driven approaches, including deep learning, for a skull shape completion task on synthetic defects. The second AutoImplant challenge (i.e., AutoImplant II, 2021) built upon the first by adding real clinical craniectomy cases as well as additional synthetic imaging data. The AutoImplant II challenge consisted of three tracks. Tracks 1 and 3 used skull images with synthetic defects to evaluate the ability of submitted approaches to generate implants that recreate the original skull shape. Track 3 consisted of the data from the first challenge (i.e., 100 cases for training, and 110 for evaluation), and Track 1 provided 570 training and 100 validation cases aimed at evaluating skull shape completion algorithms at diverse defect patterns. Track 2 also made progress over the first challenge by providing 11 clinically defective skulls and evaluating the submitted implant designs on these clinical cases. The submitted designs were evaluated quantitatively against imaging data from post-craniectomy as well as by an experienced neurosurgeon. Submissions to these challenge tasks made substantial progress in addressing issues such as generalizability, computational efficiency, data augmentation, and implant refinement. This paper serves as a comprehensive summary and comparison of the submissions to the AutoImplant II challenge. Codes and models are available at https://github.com/Jianningli/Autoimplant_II.
Collapse
Affiliation(s)
- Jianning Li
- Institute for AI in Medicine (IKIM), University Medicine Essen, Girardetstraße 2, 45131 Essen, Germany; Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, 8010 Graz, Austria; Computer Algorithms for Medicine Laboratory, Graz, Austria.
| | - David G Ellis
- Department of Neurosurgery, University of Nebraska Medical Center, Omaha, NE, 68198, USA
| | - Oldřich Kodym
- Graph@FIT, Brno University of Technology, Brno, Czech Republic
| | - Laurèl Rauschenbach
- Department of Neurosurgery and Spine Surgery, University Hospital Essen, Hufelandstrasse 55, 45147 Essen, Germany
| | - Christoph Rieß
- Department of Neurosurgery and Spine Surgery, University Hospital Essen, Hufelandstrasse 55, 45147 Essen, Germany
| | - Ulrich Sure
- Department of Neurosurgery and Spine Surgery, University Hospital Essen, Hufelandstrasse 55, 45147 Essen, Germany
| | - Karsten H Wrede
- Department of Neurosurgery and Spine Surgery, University Hospital Essen, Hufelandstrasse 55, 45147 Essen, Germany
| | - Carlos M Alvarez
- Department of Neurosurgery, University of Nebraska Medical Center, Omaha, NE, 68198, USA
| | - Marek Wodzinski
- AGH University of Science and Technology, Department of Measurement and Electronics, Krakow, Poland; University of Applied Sciences Western Switzerland (HES-SO Valais), Information Systems Institute, Sierre, Switzerland
| | - Mateusz Daniol
- AGH University of Science and Technology, Department of Measurement and Electronics, Krakow, Poland
| | - Daria Hemmerling
- AGH University of Science and Technology, Department of Measurement and Electronics, Krakow, Poland
| | - Hamza Mahdi
- Sunnybrook Research Institute, Toronto, ON, Canada
| | | | - Evan Kim
- Sunnybrook Research Institute, Toronto, ON, Canada
| | | | - Cari M Whyne
- Sunnybrook Research Institute, Toronto, ON, Canada; Division of Orthopaedic Surgery, University of Toronto, Toronto, ON, M5T 1P5, Canada
| | - James G Mainprize
- Sunnybrook Research Institute, Toronto, ON, Canada; Calavera Surgical Design Inc., Toronto, ON, Canada
| | - Michael R Hardisty
- Sunnybrook Research Institute, Toronto, ON, Canada; Division of Orthopaedic Surgery, University of Toronto, Toronto, ON, M5T 1P5, Canada
| | - Shashwat Pathak
- Department of Electrical Engineering, Indian Institute of Technology, Tirupati, India
| | - Chitimireddy Sindhura
- Department of Electrical Engineering, Indian Institute of Technology, Tirupati, India
| | | | - Degala Venkata Kiran
- Department of Mechanical Engineering, Indian Institute of Technology, Tirupati, India
| | - Subrahmanyam Gorthi
- Department of Electrical Engineering, Indian Institute of Technology, Tirupati, India
| | - Bokai Yang
- Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 2R3, Canada
| | - Ke Fang
- Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 2R3, Canada
| | - Xingyu Li
- Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB T6G 2R3, Canada
| | - Artem Kroviakov
- Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, 8010 Graz, Austria
| | - Lei Yu
- Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, 8010 Graz, Austria
| | - Yuan Jin
- Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, 8010 Graz, Austria; Computer Algorithms for Medicine Laboratory, Graz, Austria
| | - Antonio Pepe
- Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, 8010 Graz, Austria; Computer Algorithms for Medicine Laboratory, Graz, Austria
| | - Christina Gsaxner
- Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, 8010 Graz, Austria; Computer Algorithms for Medicine Laboratory, Graz, Austria
| | - Adam Herout
- Graph@FIT, Brno University of Technology, Brno, Czech Republic
| | - Victor Alves
- ALGORITMI Research Centre/LASI, University of Minho, Braga, Portugal
| | | | - Michele R Aizenberg
- Department of Neurosurgery, University of Nebraska Medical Center, Omaha, NE, 68198, USA
| | - Jens Kleesiek
- Institute for AI in Medicine (IKIM), University Medicine Essen, Girardetstraße 2, 45131 Essen, Germany
| | - Jan Egger
- Institute for AI in Medicine (IKIM), University Medicine Essen, Girardetstraße 2, 45131 Essen, Germany; Institute of Computer Graphics and Vision, Graz University of Technology, Inffeldgasse 16, 8010 Graz, Austria; Computer Algorithms for Medicine Laboratory, Graz, Austria.
| |
Collapse
|
8
|
Synergy between artificial intelligence and precision medicine for computer-assisted oral and maxillofacial surgical planning. Clin Oral Investig 2023; 27:897-906. [PMID: 36323803 DOI: 10.1007/s00784-022-04706-4] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Accepted: 08/29/2022] [Indexed: 11/06/2022]
Abstract
OBJECTIVES The aim of this review was to investigate the application of artificial intelligence (AI) in maxillofacial computer-assisted surgical planning (CASP) workflows with the discussion of limitations and possible future directions. MATERIALS AND METHODS An in-depth search of the literature was undertaken to review articles concerned with the application of AI for segmentation, multimodal image registration, virtual surgical planning (VSP), and three-dimensional (3D) printing steps of the maxillofacial CASP workflows. RESULTS The existing AI models were trained to address individual steps of CASP, and no single intelligent workflow was found encompassing all steps of the planning process. Segmentation of dentomaxillofacial tissue from computed tomography (CT)/cone-beam CT imaging was the most commonly explored area which could be applicable in a clinical setting. Nevertheless, a lack of generalizability was the main issue, as the majority of models were trained with the data derived from a single device and imaging protocol which might not offer similar performance when considering other devices. In relation to registration, VSP and 3D printing, the presence of inadequate heterogeneous data limits the automatization of these tasks. CONCLUSION The synergy between AI and CASP workflows has the potential to improve the planning precision and efficacy. However, there is a need for future studies with big data before the emergent technology finds application in a real clinical setting. CLINICAL RELEVANCE The implementation of AI models in maxillofacial CASP workflows could minimize a surgeon's workload and increase efficiency and consistency of the planning process, meanwhile enhancing the patient-specific predictability.
Collapse
|
9
|
Thimukonda Jegadeesan J, Baldia M, Basu B. Next-generation personalized cranioplasty treatment. Acta Biomater 2022; 154:63-82. [PMID: 36272686 DOI: 10.1016/j.actbio.2022.10.030] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Revised: 10/10/2022] [Accepted: 10/13/2022] [Indexed: 12/14/2022]
Abstract
Decompressive craniectomy (DC) is a surgical procedure, that is followed by cranioplasty surgery. DC is usually performed to treat patients with traumatic brain injury, intracranial hemorrhage, cerebral infarction, brain edema, skull fractures, etc. In many published clinical case studies and systematic reviews, cranioplasty surgery is reported to restore cranial symmetry with good cosmetic outcomes and neurophysiologically relevant functional outcomes in hundreds of patients. In this review article, we present a number of key issues related to the manufacturing of patient-specific implants, clinical complications, cosmetic outcomes, and newer alternative therapies. While discussing alternative therapeutic treatments for cranioplasty, biomolecules and cellular-based approaches have been emphasized. The current clinical practices in the restoration of cranial defects involve 3D printing to produce patient-specific prefabricated cranial implants, that provide better cosmetic outcomes. Regardless of the advancements in image processing and 3D printing, the complete clinical procedure is time-consuming and requires significant costs. To reduce manual intervention and to address unmet clinical demands, it has been highlighted that automated implant fabrication by data-driven methods can accelerate the design and manufacturing of patient-specific cranial implants. The data-driven approaches, encompassing artificial intelligence (machine learning/deep learning) and E-platforms, such as publicly accessible clinical databases will lead to the development of the next generation of patient-specific cranial implants, which can provide predictable clinical outcomes. STATEMENT OF SIGNIFICANCE: Cranioplasty is performed to reconstruct cranial defects of patients who have undergone decompressive craniectomy. Cranioplasty surgery improves the aesthetic and functional outcomes of those patients. To meet the clinical demands of cranioplasty surgery, accelerated designing and manufacturing of 3D cranial implants are required. This review provides an overview of biomaterial implants and bone flap manufacturing methods for cranioplasty surgery. In addition, tissue engineering and regenerative medicine-based approaches to reduce clinical complications are also highlighted. The potential use of data-driven computer applications and data-driven artificial intelligence-based approaches are emphasized to accelerate the clinical protocols of cranioplasty treatment with less manual intervention and shorter intraoperative time.
Collapse
Affiliation(s)
| | - Manish Baldia
- Department of Neurosurgery, Jaslok Hospital and Research Centre, Mumbai, Maharashtra 400026, India
| | - Bikramjit Basu
- Materials Research Centre, Indian Institute of Science, CV Raman Road, Bangalore, Karnataka 560012, India; Centre for Biosystems Science and Engineering, Indian Institute of Science, Bangalore, Karnataka 560012, India.
| |
Collapse
|
10
|
Wodzinski M, Daniol M, Socha M, Hemmerling D, Stanuch M, Skalski A. Deep learning-based framework for automatic cranial defect reconstruction and implant modeling. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 226:107173. [PMID: 36257198 DOI: 10.1016/j.cmpb.2022.107173] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 08/19/2022] [Accepted: 10/02/2022] [Indexed: 06/16/2023]
Abstract
BACKGROUND AND OBJECTIVE This article presents a robust, fast, and fully automatic method for personalized cranial defect reconstruction and implant modeling. METHODS We propose a two-step deep learning-based method using a modified U-Net architecture to perform the defect reconstruction, and a dedicated iterative procedure to improve the implant geometry, followed by an automatic generation of models ready for 3-D printing. We propose a cross-case augmentation based on imperfect image registration combining cases from different datasets. Additional ablation studies compare different augmentation strategies and other state-of-the-art methods. RESULTS We evaluate the method on three datasets introduced during the AutoImplant 2021 challenge, organized jointly with the MICCAI conference. We perform the quantitative evaluation using the Dice and boundary Dice coefficients, and the Hausdorff distance. The Dice coefficient, boundary Dice coefficient, and the 95th percentile of Hausdorff distance averaged across all test sets, are 0.91, 0.94, and 1.53 mm respectively. We perform an additional qualitative evaluation by 3-D printing and visualization in mixed reality to confirm the implant's usefulness. CONCLUSION The article proposes a complete pipeline that enables one to create the cranial implant model ready for 3-D printing. The described method is a greatly extended version of the method that scored 1st place in all AutoImplant 2021 challenge tasks. We freely release the source code, which together with the open datasets, makes the results fully reproducible. The automatic reconstruction of cranial defects may enable manufacturing personalized implants in a significantly shorter time, possibly allowing one to perform the 3-D printing process directly during a given intervention. Moreover, we show the usability of the defect reconstruction in a mixed reality that may further reduce the surgery time.
Collapse
Affiliation(s)
- Marek Wodzinski
- Department of Measurement and Electronics, AGH University of Science and Technology, Krakow, Poland; MedApp S.A., Krakow, Poland; Information Systems Institute, University of Applied Sciences Western Switzerland, Sierre, Switzerland.
| | - Mateusz Daniol
- Department of Measurement and Electronics, AGH University of Science and Technology, Krakow, Poland; MedApp S.A., Krakow, Poland
| | - Miroslaw Socha
- Department of Measurement and Electronics, AGH University of Science and Technology, Krakow, Poland
| | - Daria Hemmerling
- Department of Measurement and Electronics, AGH University of Science and Technology, Krakow, Poland
| | - Maciej Stanuch
- Department of Measurement and Electronics, AGH University of Science and Technology, Krakow, Poland; MedApp S.A., Krakow, Poland
| | - Andrzej Skalski
- Department of Measurement and Electronics, AGH University of Science and Technology, Krakow, Poland; MedApp S.A., Krakow, Poland
| |
Collapse
|
11
|
Xiong YT, Zeng W, Xu L, Guo JX, Liu C, Chen JT, Du XY, Tang W. Virtual reconstruction of midfacial bone defect based on generative adversarial network. Head Face Med 2022; 18:19. [PMID: 35761334 PMCID: PMC9235085 DOI: 10.1186/s13005-022-00325-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Accepted: 05/19/2022] [Indexed: 02/08/2023] Open
Abstract
BACKGROUND The study aims to evaluate the accuracy of the generative adversarial networks (GAN) for reconstructing bony midfacial defects. METHODS According to anatomy, the bony midface was divided into five subunit structural regions and artificial defects are manually created on the corresponding CT images. GAN is trained to reconstruct artificial defects to their previous normal shape and tested. The clinical defects are reconstructed by the trained GAN, where the midspan defects were used for qualitative evaluation and the unilateral defects were used for quantitative evaluation. The cosine similarity and the mean error are used to evaluate the accuracy of reconstruction. The Mann-Whitney U test is used to detect whether reconstruction errors were consistent in artificial and unilateral clinical defects. RESULTS This study included 518 normal CT data, with 415 in training set and 103 in testing set, and 17 real patient data, with 2 midspan defects and 15 unilateral defects. Reconstruction of midspan clinical defects assessed by experts is acceptable. The cosine similarity in the reconstruction of artificial defects and unilateral clinical defects is 0.97 ± 0.01 and 0.96 ± 0.01, P = 0.695. The mean error in the reconstruction of artificial defects and unilateral clinical defects is 0.59 ± 0.31 mm and 0.48 ± 0.08 mm, P = 0.09. CONCLUSION GAN-based virtual reconstruction technology has reached a high accuracy in testing set, and statistical tests suggest that it can achieve similar results in real patient data. This study has preliminarily solved the problem of bony midfacial defect without reference.
Collapse
Affiliation(s)
- Yu-Tao Xiong
- State Key Laboratory of Oral Diseases and National Clinical Research Centre for Oral Diseases and Department of Oral and Maxillofacial Surgery, West China Hospital of Stomatology, Sichuan University, No.14, 3rd section of Ren Min Nan Road, Chengdu, 610041, China
| | - Wei Zeng
- State Key Laboratory of Oral Diseases and National Clinical Research Centre for Oral Diseases and Department of Oral and Maxillofacial Surgery, West China Hospital of Stomatology, Sichuan University, No.14, 3rd section of Ren Min Nan Road, Chengdu, 610041, China
| | - Lei Xu
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu, 610065, China
| | - Ji-Xiang Guo
- Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu, 610065, China
| | - Chang Liu
- State Key Laboratory of Oral Diseases and National Clinical Research Centre for Oral Diseases and Department of Oral and Maxillofacial Surgery, West China Hospital of Stomatology, Sichuan University, No.14, 3rd section of Ren Min Nan Road, Chengdu, 610041, China
| | - Jun-Tian Chen
- State Key Laboratory of Oral Diseases and National Clinical Research Centre for Oral Diseases and Department of Oral and Maxillofacial Surgery, West China Hospital of Stomatology, Sichuan University, No.14, 3rd section of Ren Min Nan Road, Chengdu, 610041, China
| | - Xin-Ya Du
- Department of Stomatology, the People's Hospital of Longhua, Shenzhen, 518109, China
| | - Wei Tang
- State Key Laboratory of Oral Diseases and National Clinical Research Centre for Oral Diseases and Department of Oral and Maxillofacial Surgery, West China Hospital of Stomatology, Sichuan University, No.14, 3rd section of Ren Min Nan Road, Chengdu, 610041, China.
| |
Collapse
|