Design, development and clinical validation of computer-aided surgical simulation system for streamlined orthognathic surgical planning

Improving Image Segmentation with Contextual and Structural Similarity

Deep learning models for medical image segmentation are usually trained with voxel-wise losses, e.g., cross-entropy loss, focusing on unary supervision without considering inter-voxel relationships. This oversight potentially leads to semantically inconsistent predictions. Here, we propose a contextual similarity loss (CSL) and a structural similarity loss (SSL) to explicitly and efficiently incorporate inter-voxel relationships for improved performance. The CSL promotes consistency in predicted object categories for each image sub-region compared to ground truth. The SSL enforces compatibility between the predictions of voxel pairs by computing pair-wise distances between them, ensuring that voxels of the same class are close together whereas those from different classes are separated by a wide margin in the distribution space. The effectiveness of the CSL and SSL is evaluated using a clinical cone-beam computed tomography (CBCT) dataset of patients with various craniomaxillofacial (CMF) deformities and a public pancreas dataset. Experimental results show that the CSL and SSL outperform state-of-the-art regional loss functions in preserving segmentation semantics.

Predicting optimal patient-specific postoperative facial landmarks for patients with craniomaxillofacial deformities

Orthognathic surgery primarily corrects skeletal anomalies and malocclusion to enhance facial aesthetics, aiming for an improved facial appearance. However, this traditional skeletal-driven approach may result in undesirable residual asymmetry. To address this issue, a soft tissue-driven planning methodology has been proposed. This technique estimates bone movements based on the envisioned optimal facial appearance, thereby enhancing surgical accuracy and effectiveness. This study investigates the initial implementation phase of the soft tissue-driven approach, simulating the patient's ideal appearance by realigning distorted facial landmarks to an ideal state. The algorithm employs symmetrization and weighted optimization strategies, aligning projected optimal landmarks with standard cephalometric values for both facial symmetry and form, which are essential in orthognathic surgery for facial aesthetics. It also incorporates regularization to preserve the patient's facial characteristics. Validation through retrospective analysis of preoperative patients and normal subjects demonstrates this method's efficacy in achieving facial symmetry, particularly in the lower face, and promoting a natural, harmonious contour. Adhering to soft tissue-driven principles, this novel approach shows promise in surpassing traditional methods, potentially leading to enhanced facial outcomes and patient satisfaction in orthognathic surgery.

Machine Learning Effectively Diagnoses Mandibular Deformity Using Three-Dimensional Landmarks

BACKGROUND

Jaw deformity diagnosis requires objective tests. Current methods, like cephalometry, have limitations. However, recent studies have shown that machine learning can diagnose jaw deformities in two dimensions. Therefore, we hypothesized that a multilayer perceptron (MLP) could accurately diagnose jaw deformities in three dimensions (3D).

PURPOSE

Examine the hypothesis by focusing on anomalous mandibular position. We aimed to: (1) create a machine learning model to diagnose mandibular retrognathism and prognathism; and (2) compare its performance with traditional cephalometric methods.

STUDY DESIGN, SETTING, SAMPLE

An in-silico experiment on deidentified retrospective data. The study was conducted at the Houston Methodist Research Institute and Rensselaer Polytechnic Institute. Included were patient records with jaw deformities and preoperative 3D facial models. Patients with significant jaw asymmetry were excluded.

PREDICTOR VARIABLES

The tests used to diagnose mandibular anteroposterior position are: (1) SNB angle; (2) facial angle; (3) mandibular unit length (MdUL); and (4) MLP model.

MAIN OUTCOME VARIABLE

The resultant diagnoses: normal, prognathic, or retrognathic.

COVARIATES

None.

ANALYSES

A senior surgeon labeled the patients' mandibles as prognathic, normal, or retrognathic, creating a gold standard. Scientists at Rensselaer Polytechnic Institute developed an MLP model to diagnose mandibular prognathism and retrognathism using the 3D coordinates of 50 landmarks. The performance of the MLP model was compared with three traditional cephalometric measurements: (1) SNB, (2) facial angle, and (3) MdUL. The primary metric used to assess the performance was diagnostic accuracy. McNemar's exact test tested the difference between traditional cephalometric measurement and MLP. Cohen's Kappa measured inter-rater agreement between each method and the gold standard.

RESULTS

The sample included 101 patients. The diagnostic accuracy of SNB, facial angle, MdUL, and MLP were 74.3, 74.3, 75.3, and 85.2%, respectively. McNemar's test shows that our MLP performs significantly better than the SNB (P = .027), facial angle (P = .019), and MdUL (P = .031). The agreement between the traditional cephalometric measurements and the surgeon's diagnosis was fair. In contrast, the agreement between the MLP and the surgeon was moderate.

CONCLUSION AND RELEVANCE

The performance of the MLP is significantly better than that of the traditional cephalometric measurements.

Helical distraction is superior to linear and circular distraction in mandibular distraction osteogenesis: an in silico study

Helical mandibular distraction is theoretically better than linear or circular distraction. However, it is not known whether this more complex treatment will result in unquestionably better outcomes. Therefore, the best attainable outcomes of mandibular distraction osteogenesis were evaluated in silico, given the constraints of linear, circular, and helical motion. This cross-sectional kinematic study included 30 patients with mandibular hypoplasia who had been treated with distraction, or to whom this treatment had been recommended. Demographic information and the computed tomography (CT) scans showing the baseline deformity were collected. The CT scans of each patient were segmented and three-dimensional models of the face created. Then, the ideal distraction outcomes were simulated. Next, the most favorable helical, circular, and linear distraction movements were calculated. Finally, errors were measured: misalignment of key mandibular landmarks, misalignment of the occlusion, and changes in intercondylar distance. Helical distraction produced trivial errors. In contrast, circular and linear distractions resulted in errors that were statistically and clinically significant. Helical distraction also preserved the planned intercondylar distance, while circular and linear distractions led to unwanted changes in the intercondylar distance. It is now evident that helical distraction offers a new strategy to improve the outcomes of mandibular distraction osteogenesis.

Accuracy of Surgical Outcome Using Computer-Aided Surgical Simulation in Fronto-Orbital Advancement for Craniosynostosis: A Pilot Study

BACKGROUND AND OBJECTIVE

Computer-aided surgical simulation (CASS) can be used to virtually plan ideal outcomes of craniosynostosis surgery. Our purpose was to create a workflow analyzing the accuracy of surgical outcomes relative to virtually planned fronto-orbital advancement (FOA).

METHODS

Patients who underwent FOA using CASS between October 1, 2017, and February 28, 2022, at our center and had postoperative computed tomography within 6 months of surgery were included. Virtual 3-dimensional (3D) models were created and coregistered using each patient's preoperative and postoperative computed tomography data. Three points on each bony segment were used to define the object in 3D space. Each planned bony segment was manipulated to match the actual postoperative outcome. The change in position of the 3D object was measured in translational (X, Y, Z) and rotational (roll, pitch, yaw) aspects to represent differences between planned and actual postoperative positions. The difference in the translational position of several bony landmarks was also recorded. Wilcoxon signed-rank tests were performed to measure significance of these differences from the ideal value of 0, which would indicate no difference between preoperative plan and postoperative outcome.

RESULTS

Data for 63 bony segments were analyzed from 8 patients who met the inclusion criteria. Median differences between planned and actual outcomes of the segment groups ranged from -0.3 to -1.3 mm in the X plane; 1.4 to 5.6 mm in the Y plane; 0.9 to 2.7 mm in the Z plane; -1.2° to -4.5° in pitch; -0.1° to 1.0° in roll; and -2.8° to 1.0° in yaw. No significant difference from 0 was found in 21 of 24 segment region/side combinations. Translational differences of bony landmarks ranged from -2.7 to 3.6 mm.

CONCLUSION

A high degree of accuracy was observed relative to the CASS plan. Virtual analysis of surgical accuracy in FOA using CASS was feasible.

Predicting Optimal Patient-Specific Postoperative Facial Landmarks for Patients with Craniomaxillofacial Deformities

Orthognathic surgery traditionally focuses on correcting skeletal abnormalities and malocclusion, with the expectation that an optimal facial appearance will naturally follow. However, this skeletal-driven approach can lead to undesirable facial aesthetics and residual asymmetry. To address these issues, a soft-tissue-driven planning method has been proposed. This innovative method bases bone movement estimates on the targeted ideal facial appearance, thus increasing the surgical plan's accuracy and effectiveness. This study explores the initial phase of implementing a soft-tissue-driven approach, simulating the patient's optimal facial look by repositioning deformed facial landmarks to an ideal state. The algorithm incorporates symmetrization and weighted optimization strategies, aligning projected optimal landmarks with standard cephalometric values for both facial symmetry and form, which are integral to facial aesthetics in orthognathic surgery. It also includes regularization to preserve the patient's original facial characteristics. Validated using retrospective analysis of data from both preoperative patients and normal subjects, this approach effectively achieves not only facial symmetry, particularly in the lower face, but also a more natural and normalized facial form. This novel approach, aligning with soft-tissue-driven planning principles, shows promise in surpassing traditional methods, potentially leading to enhanced facial outcomes and patient satisfaction in orthognathic surgery.

Predictability of the virtual surgical plan for orthognathic surgery with the mandible surgery first sequence

The aim of this study was to compare the virtually planned position to the postoperative position of the maxilla, having performed the maxilla-first sequence or mandible-first sequence orthognathic surgery. An audit of 64 patients who underwent bimaxillary surgery between 2017 and 2020 was performed. Thirty patients had maxilla-first surgery and 34 had mandible-first surgery. The planned and post-surgical positions were analyzed using specific skeletal landmarks. Differences were calculated and the two-sample t-test was used to compare the groups. Measured differences between the planned and postoperative results differed significantly between the mandible-first and maxillary-first surgery groups (P < 0.001). The maxillary central incisors were under-advanced in the anterior-posterior direction in both groups. Most data points showed deviation from the surgical plan ≤ 2 mm and ≤ 4°. Secondarily, maxillary under-advancement in the mandible-first cohort was evaluated; these patients were subdivided into rigid and non-rigid fixation groups. The non-rigid fixation group showed less accuracy compared to the rigid fixation group, which was statistically significant (P = 0.014). The findings of this study demonstrate that virtual surgical planning can be less accurate in predicting the maxillary incisor position when performing mandible-first surgery, but this inaccuracy is within the acceptable range and can be mitigated by rigid fixation of the mandible.

Synergy between artificial intelligence and precision medicine for computer-assisted oral and maxillofacial surgical planning

OBJECTIVES

The aim of this review was to investigate the application of artificial intelligence (AI) in maxillofacial computer-assisted surgical planning (CASP) workflows with the discussion of limitations and possible future directions.

MATERIALS AND METHODS

An in-depth search of the literature was undertaken to review articles concerned with the application of AI for segmentation, multimodal image registration, virtual surgical planning (VSP), and three-dimensional (3D) printing steps of the maxillofacial CASP workflows.

RESULTS

The existing AI models were trained to address individual steps of CASP, and no single intelligent workflow was found encompassing all steps of the planning process. Segmentation of dentomaxillofacial tissue from computed tomography (CT)/cone-beam CT imaging was the most commonly explored area which could be applicable in a clinical setting. Nevertheless, a lack of generalizability was the main issue, as the majority of models were trained with the data derived from a single device and imaging protocol which might not offer similar performance when considering other devices. In relation to registration, VSP and 3D printing, the presence of inadequate heterogeneous data limits the automatization of these tasks.

CONCLUSION

The synergy between AI and CASP workflows has the potential to improve the planning precision and efficacy. However, there is a need for future studies with big data before the emergent technology finds application in a real clinical setting.

CLINICAL RELEVANCE

The implementation of AI models in maxillofacial CASP workflows could minimize a surgeon's workload and increase efficiency and consistency of the planning process, meanwhile enhancing the patient-specific predictability.

Simulation of Postoperative Facial Appearances via Geometric Deep Learning for Efficient Orthognathic Surgical Planning

Orthognathic surgery corrects jaw deformities to improve aesthetics and functions. Due to the complexity of the craniomaxillofacial (CMF) anatomy, orthognathic surgery requires precise surgical planning, which involves predicting postoperative changes in facial appearance. To this end, most conventional methods involve simulation with biomechanical modeling methods, which are labor intensive and computationally expensive. Here we introduce a learning-based framework to speed up the simulation of postoperative facial appearances. Specifically, we introduce a facial shape change prediction network (FSC-Net) to learn the nonlinear mapping from bony shape changes to facial shape changes. FSC-Net is a point transform network weakly-supervised by paired preoperative and postoperative data without point-wise correspondence. In FSC-Net, a distance-guided shape loss places more emphasis on the jaw region. A local point constraint loss restricts point displacements to preserve the topology and smoothness of the surface mesh after point transformation. Evaluation results indicate that FSC-Net achieves 15× speedup with accuracy comparable to a state-of-the-art (SOTA) finite-element modeling (FEM) method.

Bidirectional prediction of facial and bony shapes for orthognathic surgical planning

This paper proposes a deep learning framework to encode subject-specific transformations between facial and bony shapes for orthognathic surgical planning. Our framework involves a bidirectional point-to-point convolutional network (P2P-Conv) to predict the transformations between facial and bony shapes. P2P-Conv is an extension of the state-of-the-art P2P-Net and leverages dynamic point-wise convolution (i.e., PointConv) to capture local-to-global spatial information. Data augmentation is carried out in the training of P2P-Conv with multiple point subsets from the facial and bony shapes. During inference, network outputs generated for multiple point subsets are combined into a dense transformation. Finally, non-rigid registration using the coherent point drift (CPD) algorithm is applied to generate surface meshes based on the predicted point sets. Experimental results on real-subject data demonstrate that our method substantially improves the prediction of facial and bony shapes over state-of-the-art methods.

Localization of Craniomaxillofacial Landmarks on CBCT Images Using 3D Mask R-CNN and Local Dependency Learning

Cephalometric analysis relies on accurate detection of craniomaxillofacial (CMF) landmarks from cone-beam computed tomography (CBCT) images. However, due to the complexity of CMF bony structures, it is difficult to localize landmarks efficiently and accurately. In this paper, we propose a deep learning framework to tackle this challenge by jointly digitalizing 105 CMF landmarks on CBCT images. By explicitly learning the local geometrical relationships between the landmarks, our approach extends Mask R-CNN for end-to-end prediction of landmark locations. Specifically, we first apply a detection network on a down-sampled 3D image to leverage global contextual information to predict the approximate locations of the landmarks. We subsequently leverage local information provided by higher-resolution image patches to refine the landmark locations. On patients with varying non-syndromic jaw deformities, our method achieves an average detection accuracy of 1.38± 0.95mm, outperforming a related state-of-the-art method.

Personal guide assisted orthognathic surgery: application of a T-shaped tooth bone combined supporting osteotomy guide plate and positioning guide plate

ABSTRACT

The precise movement of the maxilla is particularly important for orthognathic surgery, especially for patients with maxillary segmentation. In this preliminary study, the authors present a new tooth bone combined with a supporting osteotomy guide and positioning guide to guide the osteotomy and reduction of the maxilla. Through our preoperative simulation and postoperative image fusion, the authors found that the overlapping area is more than 90%. According to compare of the virtual plans and the postoperative results based on distances from the maxillary land- marks to the horizontal plane, sagittal plane, and coronal plane, the surgical error was about 2mm. Our T-shaped guide provides a reliable method for patients with maxillary segmental osteotomy, which may be a useful alternative to the intermediate.

Computer Aided Orthognathic Surgery: A General Method for Designing and Manufacturing Personalized Cutting/Repositioning Templates

Orthognathic surgery allows broad-spectrum deformity correction involving both aesthetic and functional aspects on the TMJ (temporo-mandibular joint) and on the facial skull district. The combination of Reverse Engineering (RE), Virtual Surgery Planning (VSP), Computer Aided Design (CAD), Additive Manufacturing (AM), and 3D visualization allows surgeons to plan, virtually, manipulations and the translation of the human parts in the operating room. This work’s aim was to define a methodology, in the form of a workflow, for surgery planning and for designing and manufacturing templates for orthognathic surgery. Along the workflow, the error chain was checked and the maximum error in virtual planning was evaluated. The three-dimensional reconstruction of the mandibular shape and bone fragment movements after segmentation allow complete planning of the surgery and, following the proposed method, the introduction of both the innovative evaluation of the transversal intercondylar distance variation after mandibular arch advancement/set and the possibility of use of standard plates to plan and realize a customized surgery. The procedure was adopted in one clinical case on a patient affected by a class III malocclusion with an associated open bite and right deviation of the mandible with expected good results. Compared with the methods from most recent literature, the presented method introduces two elements of novelty and improves surgery results by optimizing costs and operating time. A new era of collaboration among surgeons and engineer has begun and is now bringing several benefits in personalized surgery.

Shaping the Lower Jaw Border with Customized Cutting Guides: Development, Validation, and Application in Facial Gender-Affirming Surgery

Importance: Three-dimensional planning software is not standardized in facial gender-affirming surgery. Objective: To develop and validate surgical planning software to create cutting guides to contour the lower jaw border. Design, Setting, and Participants: A 3-year prospective case series study done in three phases: software development, validation, and surgical guide application. Ethics committee approval was obtained to enroll the patients (Clinical Research Ethics Committee, Hospital Costa del Sol, Marbella, Spain). Main Outcomes and Measures: Validation phase: degree of agreement between the planned and obtained results, modification of cephalometric parameters, and surgical times. Application phase: surgical technique description, complications, and patient-reported outcome measures. Results: The degree of agreement between the planned and obtained results was inframillimetric (0.31 ± 0.70 mm). The guides reduced the mandible to within feminine parameters (p < 0.05). Surgical times decreased by 10.96% with chin ostectomies (p < 0.05) and 23.06% with lower jaw border (angle-to-angle) surgeries (p < 0.001). In the application phase, revision surgery was required for 11 patients out of 260 (4.23%). Conclusions and Relevance: The use of cutting guides on the lower jaw border is effective, helps reach standard feminine parameters, and decreases surgical times.

Clinical feasibility evaluation of digital dental articulation for three-piece maxillary orthognathic surgery: a proof-of-concept study

Digital dental articulation for three-piece maxillary orthognathic surgery is challenging. The purpose of this proof-of-concept study was to evaluate the clinical feasibility of a newly developed mathematical algorithm to digitally establish the final occlusion for three-piece maxillary surgery. Five patients with jaw deformities who had undergone a three-piece double-jaw surgery that was planned virtually were randomly selected for this study. The final occlusion had been hand-articulated using stone casts, scanned into the computer and used in the surgery. These hand-articulated occlusions served as the control group. To form the experimental group, the three-piece maxillary dental arch was articulated again automatically from the patient's original occlusion using the mathematical algorithm. The hand- and algorithm-articulated occlusions were then evaluated qualitatively by two experienced orthodontists. A quantitative evaluation was also performed. The results of the qualitative evaluation showed that all of the three-piece occlusions, hand- and algorithm-articulated, were clinically acceptable based on the American Board of Orthodontics grading system. When compared, two of the algorithm-articulated occlusions were clearly better (40%), one was the same (20%), and two were slightly worse (40%) than the hand-articulated occlusions. All of the quantitative measurements were comparable between the two articulation methods. In conclusion, the results of this study demonstrate that it is clinically feasible to digitally articulate the three-piece maxillary arch to the intact mandibular dental arch.

Physical Versus Digital Orthognathic Surgical Planning

OBJECTIVE

Orthognathic occlusal repositioning wafers could be constructed virtually and 3D printed. This paper assessed the accuracy of a suggested virtual model to the conventionally established Glasgow model surgery.

DESIGN

Prospective study of the orthognathic surgery models digitally.

PARTICIPANTS

Seven patients who received bi-maxillary orthognathic surgeries for correction of dentofacial deformities.

METHODS

The patients were clinically assessed and their cone beam cmputerized tomography (CBCT) studied. Model surgery of each patient was performed conventionally using face-bow and semi-adjustable articulator. Same plan was executed virtually using Mimics (Materialise, Leuven, Belgium) and 3Matic (Materialise, Leuven, Belgium). Conventionally fabricated acrylic wafers as well as 3D printed wafers were CBCT scanned with the casts reflecting the archived repositioning dictated by the wafers. Paired sample t test was performed to compare accuracy between intermediate and final occlusal repositioning wafers within conventional and virtual technique groups.

RESULTS

The mean deviation in intermediate wafer group was 0.64 ± 0.33 mm; whereas the mean deviation in final wafer group was 0.53 ± 0.10 mm. Paired sample t test showed that there was no statistically significant difference in mean deviation between both groups (P = 0.403).

CONCLUSIONS

This virtual surgical wafer achieves a similar level of accuracy to the conventional Glasgow model surgery.

Fast and Accurate Craniomaxillofacial Landmark Detection via 3D Faster R-CNN

Automatic craniomaxillofacial (CMF) landmark localization from cone-beam computed tomography (CBCT) images is challenging, considering that 1) the number of landmarks in the images may change due to varying deformities and traumatic defects, and 2) the CBCT images used in clinical practice are typically large. In this paper, we propose a two-stage, coarse-to-fine deep learning method to tackle these challenges with both speed and accuracy in mind. Specifically, we first use a 3D faster R-CNN to roughly locate landmarks in down-sampled CBCT images that have varying numbers of landmarks. By converting the landmark point detection problem to a generic object detection problem, our 3D faster R-CNN is formulated to detect virtual, fixed-size objects in small boxes with centers indicating the approximate locations of the landmarks. Based on the rough landmark locations, we then crop 3D patches from the high-resolution images and send them to a multi-scale UNet for the regression of heatmaps, from which the refined landmark locations are finally derived. We evaluated the proposed approach by detecting up to 18 landmarks on a real clinical dataset of CMF CBCT images with various conditions. Experiments show that our approach achieves state-of-the-art accuracy of 0.89 ± 0.64mm in an average time of 26.2 seconds per volume.

Midsagittal Plane First: Building a Strong Facial Reference Frame for Computer-Aided Surgical Simulation

PURPOSE

A facial reference frame is a 3-dimensional Cartesian coordinate system that includes 3 perpendicular planes: midsagittal, axial, and coronal. The order in which one defines the planes matters. The purposes of this study are to determine the following: 1) what sequence (axial-midsagittal-coronal vs midsagittal-axial-coronal) produced more appropriate reference frames and 2) whether orbital or auricular dystopia influenced the outcomes.

METHODS

This study is an ambispective cross-sectional study. Fifty-four subjects with facial asymmetry were included. The facial reference frames of each subject (outcome variable) were constructed using 2 methods (independent variable): axial plane first and midsagittal plane first. Two board-certified orthodontists together blindly evaluated the results using a 3-point categorical scale based on their careful inspection and expert intuition. The covariant for stratification was the existence of orbital or auricular dystopia. Finally, Wilcoxon signed rank tests were performed.

RESULTS

The facial reference frames defined by the midsagittal plane first method was statistically significantly different from ones defined by the axial plane first method (P = .001). Using the midsagittal plane first method, the reference frames were more appropriately defined in 22 (40.7%) subjects, equivalent in 26 (48.1%) and less appropriately defined in 6 (11.1%). After stratified by orbital or auricular dystopia, the results also showed that the reference frame computed using midsagittal plane first method was statistically significantly more appropriate in both subject groups regardless of the existence of orbital or auricular dystopia (27 with orbital or auricular dystopia and 27 without, both P < .05).

CONCLUSIONS

The midsagittal plane first sequence improves the facial reference frames compared with the traditional axial plane first approach. However, regardless of the sequence used, clinicians need to judge the correctness of the reference frame before diagnosis or surgical planning.

The Use of Surgical Splints in Orthognathic Surgery: A Bibliometric Study

Introduction  Orthognathic surgeries require the use of surgical splints (SS) to stabilize the occlusion and the segments fixed with plates and screws. Technological advances in the field of computing and the possibility of generating three-dimensional (3D) images have brought different possibilities for making SS, which has generated greater predictability and customization of surgical plans. The bibliometric study can have a qualitative character through the scope of articles in a certain area of knowledge. It is a selection process that can track a topic or scientific production.

Methods  The present study aimed to carry out a bibliometric literature review, in order to assess the evolution of the use of SS and the different planning protocols in orthognathic surgery. The Scopus database was used, with the terms “splint” and “orthognathic surgery.”

Results  A total of 331 articles were found. These were exported to Rayyan for application of the inclusion and exclusion criteria and selection of articles. A total of 76 references were selected and exported to the VOSviewer application for the analysis of bibliometric data.

Conclusions  Orthognathic surgery was initially not associated with any computerized technological resource; however, it underwent updates between the years 2010 to 2012. These advances allowed surgical planning to become faster, cheaper, and more accurate.

SkullEngine: A Multi-Stage CNN Framework for Collaborative CBCT Image Segmentation and Landmark Detection

Accurate bone segmentation and landmark detection are two essential preparation tasks in computer-aided surgical planning for patients with craniomaxillofacial (CMF) deformities. Surgeons typically have to complete the two tasks manually, spending ~12 hours for each set of CBCT or ~5 hours for CT. To tackle these problems, we propose a multi-stage coarse-to-fine CNN-based framework, called SkullEngine, for high-resolution segmentation and large-scale landmark detection through a collaborative, integrated, and scalable JSD model and three segmentation and landmark detection refinement models. We evaluated our framework on a clinical dataset consisting of 170 CBCT/CT images for the task of segmenting 2 bones (midface and mandible) and detecting 175 clinically common landmarks on bones, teeth, and soft tissues. Experimental results show that SkullEngine significantly improves segmentation quality, especially in regions where the bone is thin. In addition, SkullEngine also efficiently and accurately detect all of the 175 landmarks. Both tasks were completed simultaneously within 3 minutes regardless of CBCT or CT with high segmentation quality. Currently, SkullEngine has been integrated into a clinical workflow to further evaluate its clinical efficiency.

A Self-Supervised Deep Framework for Reference Bony Shape Estimation in Orthognathic Surgical Planning

Virtual orthognathic surgical planning involves simulating surgical corrections of jaw deformities on 3D facial bony shape models. Due to the lack of necessary guidance, the planning procedure is highly experience-dependent and the planning results are often suboptimal. A reference facial bony shape model representing normal anatomies can provide an objective guidance to improve planning accuracy. Therefore, we propose a self-supervised deep framework to automatically estimate reference facial bony shape models. Our framework is an end-to-end trainable network, consisting of a simulator and a corrector. In the training stage, the simulator maps jaw deformities of a patient bone to a normal bone to generate a simulated deformed bone. The corrector then restores the simulated deformed bone back to normal. In the inference stage, the trained corrector is applied to generate a patient-specific normal-looking reference bone from a real deformed bone. The proposed framework was evaluated using a clinical dataset and compared with a state-of-the-art method that is based on a supervised point-cloud network. Experimental results show that the estimated shape models given by our approach are clinically acceptable and significantly more accurate than that of the competing method.

Deep Simulation of Facial Appearance Changes Following Craniomaxillofacial Bony Movements in Orthognathic Surgical Planning

Facial appearance changes with the movements of bony segments in orthognathic surgery of patients with craniomaxillofacial (CMF) deformities. Conventional bio-mechanical methods, such as finite element modeling (FEM), for simulating such changes, are labor intensive and computationally expensive, preventing them from being used in clinical settings. To overcome these limitations, we propose a deep learning framework to predict post-operative facial changes. Specifically, FC-Net, a facial appearance change simulation network, is developed to predict the point displacement vectors associated with a facial point cloud. FC-Net learns the point displacements of a pre-operative facial point cloud from the bony movement vectors between pre-operative and simulated post-operative bony models. FC-Net is a weakly-supervised point displacement network trained using paired data with strict point-to-point correspondence. To preserve the topology of the facial model during point transform, we employ a local-point-transform loss to constrain the local movements of points. Experimental results on real patient data reveal that the proposed framework can predict post-operative facial appearance changes remarkably faster than a state-of-the-art FEM method with comparable prediction accuracy.

Mixed reality and three dimensional printed models for resection of maxillary tumor: a case report

In the field of oral and maxillofacial surgery, many institutions have recently begun using three-dimensional printers to create three-dimensional models and mixed reality in a variety of diseases. Here, we report the actual situation model which we made using three-dimensional printer from virtual operation data and the resection that was performed while grasping a maxillary benign tumor and neighboring three-dimensional structure by designing an application for Microsoft® HoloLens, and using Mixed Reality surgery support during the procedure.

Error Analysis of Robot-Assisted Orthognathic Surgery

OBJECTIVE

Orthognathic surgery is an effective method to correct the dentomaxillofacial deformities. The aim of the study is to introduce the robot-assisted orthognathic surgery and demonstrate the accuracy and feasibility of robot-assisted osteotomy in transferring the preoperative virtual surgical planning (VSP) into the intraoperative phase.

METHODS

The CMF robot system, a craniomaxillofacial surgical robot system was developed, consisted of a robotic arm with 6 degrees of freedom, a self-developed end-effector, and an optical localizer. The individualized end-effector was installed with reciprocating saw so that it could perform osteotomy. The study included control and experimental groups. In control group, under the guidance of navigation system, surgeon performed the osteotomies on 3 skull models. In experimental group, according to the preoperative VSP, the robot completed the osteotomies on 3 skull models automatically with assistance of navigation. Statistical analysis was carried out to evaluate the accuracy and feasibility of robot-assisted orthognathic surgery and compare the errors between robot-assisted automatic osteotomy and navigation-assisted manual osteotomy.

RESULTS

All the osteotomies were successfully completed. The overall osteotomy error was 1.07 ± 0.19 mm in the control group, and 1.12 ± 0.20 mm in the experimental group. No significant difference in osteotomy errors was found in the robot-assisted osteotomy groups (P = 0.353). There was consistence of errors between robot-assisted automatic osteotomy and navigation-assisted manual osteotomy.

CONCLUSION

In robot-assisted orthognathic surgery, the robot can complete an osteotomy according to the preoperative VSP and transfer a preoperative VSP into the actual surgical operation with good accuracy and feasibility.

Estimating Reference Shape Model for Personalized Surgical Reconstruction of Craniomaxillofacial Defects

OBJECTIVE

To estimate a patient-specific reference bone shape model for a patient with craniomaxillofacial (CMF) defects due to facial trauma.

METHODS

We proposed an automatic facial bone shape estimation framework using pre-traumatic conventional portrait photos and post-traumatic head computed tomography (CT) scans via a 3D face reconstruction and a deformable shape model. Specifically, a three-dimensional (3D) face was first reconstructed from the patient's pre-traumatic portrait photos. Second, a correlation model between the skin and bone surfaces was constructed using a sparse representation based on the CT images of training normal subjects. Third, by feeding the reconstructed 3D face into the correlation model, an initial reference shape model was generated. In addition, we refined the initial estimation by applying non-rigid surface matching between the initially estimated shape and the patient's post-traumatic bone based on the adaptive-focus deformable shape model (AFDSM). Furthermore, a statistical shape model, built from the training normal subjects, was utilized to constrain the deformation process to avoid overfitting.

RESULTS AND CONCLUSION

The proposed method was evaluated using both synthetic and real patient data. Experimental results show that the patient's abnormal facial bony structure can be recovered using our method, and the estimated reference shape model is considered clinically acceptable by an experienced CMF surgeon.

SIGNIFICANCE

The proposed method is more suitable to the complex CMF defects for CMF reconstructive surgical planning.

Automatic Localization of Landmarks in Craniomaxillofacial CBCT Images Using a Local Attention-Based Graph Convolution Network

Landmark localization is an important step in quantifying craniomaxillofacial (CMF) deformities and designing treatment plans of reconstructive surgery. However, due to the severity of deformities and defects (partially missing anatomy), it is difficult to automatically and accurately localize a large set of landmarks simultaneously. In this work, we propose two cascaded networks for digitizing 60 anatomical CMF landmarks in cone-beam computed tomography (CBCT) images. The first network is a U-Net that outputs heatmaps for landmark locations and landmark features extracted with a local attention mechanism. The second network is a graph convolution network that takes the features extracted by the first network as input and determines whether each landmark exists via binary classification. We evaluated our approach on 50 sets of CBCT scans of patients with CMF deformities and compared them with state-of-the-art methods. The results indicate that our approach can achieve an average detection error of 1.47mm with a false positive rate of 19%, outperforming related methods.

Design and manufacture of dental-supported surgical guide for genioplasty

Background/purpose

Genioplasty were used widely to correct chin deformities. The purpose of this study was to design and manufacture a dental-supported surgical guide for genioplasty surgery and assess for surgical accuracy.

Materials and methods

eleven patients with chin deformities were treated in this study. The computed tomography (CT) data of the patient's skull and the digital dental models of stone dental models were acquired preoperatively. For each patient, a virtual three-dimensional (3D) model of the skull was constructed and enhanced with digital dental models. A surgical simulation was then performed using computer-aided surgical simulation (CASS) technology based on clinical examination and 3D cephalometry. The surgery was simulated preoperatively which allowed the design of a cutting guide and a dental-supported repositioning guide for genioplasty, which was then 3D-printed and used during operation after disinfection. After surgery, the outcome was evaluated by superimposing the postoperative CT model onto the preoperative model, recording the linear and angular deviation of landmarks and plane, then measuring the differences between the planned and actual outcomes.

Results

The osteotomy and repositioning were successfully performed as planned using surgical guides. No inferior alveolar nerve damage was seen in this study. The dental-supported surgical guide showed excellent accuracy, with the largest differences between the planned and the postoperative chin segment being 0.9 mm and 3.2°.

Conclusion

The dental-supported surgical guide designed preoperatively provided a reliable method of transfer genioplasty planning. This can assist surgeons in accurately performing osteotomy and repositioning bone segments during a genioplasty.

Clinical Evaluation of Digital Dental Articulation for One-Piece Maxillary Surgery

PURPOSE

Methods for digital dental alignment are not readily available to automatically articulate the upper and lower jaw models. The purpose of the present study was to assess the accuracy of our newly developed 3-stage automatic digital articulation approach by comparing it with the reference standard of orthodontist-articulated occlusion.

MATERIALS AND METHODS

Thirty pairs of stone dental models from double-jaw orthognathic surgery patients who had undergone 1-piece Le Fort I osteotomy were used. Two experienced orthodontists manually articulated the models to their perceived final occlusion for surgery. Each pair of models was then scanned twice-while in the orthodontist-determined occlusion and again with the upper and lower models separated and positioned randomly. The separately scanned models were automatically articulated to the final occlusion using our 3-stage algorithm, resulting in an algorithm-articulated occlusion (experimental group). The models scanned together represented the manually articulated occlusion (control group). A qualitative evaluation was completed using a 3-point categorical scale by the same orthodontists, who were unaware of the methods used to articulate the models. A quantitative evaluation was also completed to determine whether any differences were present in the midline, canine, and molar relationships between the algorithm-determined and manually articulated occlusions using repeated measures analysis of variance (ANOVA). Finally, the mean ± standard deviation values were computed to determine the differences between the 2 methods.

RESULTS

The results of the qualitative evaluation revealed that all the algorithm-articulated occlusions were as good as the manually articulated ones. The results of the repeated measures ANOVA found no statistically significant differences between the 2 methods [F(1,28) = 0.03; P = .87]. The mean differences between the 2 methods were all within 0.2 mm.

CONCLUSIONS

The results of our study have demonstrated that dental models can be accurately, reliably, and automatically articulated using our 3-stage algorithm approach, meeting the reference standard of orthodontist-articulated occlusion.

Maxillary reconstruction using tunneling flap technique with 3D custom-made titanium mesh plate and particulate cancellous bone and marrow graft: a case report

Background

Reconstructive surgery is often required for tumors of the oral and maxillofacial region, irrespective of whether they are benign or malignant, the area involved, and the tumor size. Recently, three-dimensional (3D) models are increasingly used in reconstructive surgery. However, these models have rarely been adapted for the fabrication of custom-made reconstruction materials. In this report, we present a case of maxillary reconstruction using a laboratory-engineered, custom-made mesh plate from a 3D model.

Case presentation

The patient was a 56-year-old female, who had undergone maxillary resection in 2011 for intraoral squamous cell carcinoma that presented as a swelling of the anterior maxillary gingiva. Five years later, there was no recurrence of the malignant tumor and a maxillary reconstruction was planned. Computed tomography (CT) revealed a large bony defect in the dental-alveolar area of the anterior maxilla. Using the CT data, a 3D model of the maxilla was prepared, and the site of reconstruction determined. A custom-made mesh plate was fabricated using the 3D model (Okada Medical Supply, Tokyo, Japan). We performed the reconstruction using the custom-made titanium mesh plate and the particulate cancellous bone and marrow graft from her iliac bone. We employed the tunneling flap technique without alveolar crest incision, to prevent surgical wound dehiscence, mesh exposure, and alveolar bone loss. Ten months later, three dental implants were inserted in the graft. Before the final crown setting, we performed a gingivoplasty with palate mucosal graft. The patient has expressed total satisfaction with both the functional and esthetic outcomes of the procedure.

Conclusion

We have successfully performed a maxillary and dental reconstruction using a custom-made, pre-bent titanium mesh plate.

A machine learning framework for automated diagnosis and computer-assisted planning in plastic and reconstructive surgery

Current computational tools for planning and simulation in plastic and reconstructive surgery lack sufficient precision and are time-consuming, thus resulting in limited adoption. Although computer-assisted surgical planning systems help to improve clinical outcomes, shorten operation time and reduce cost, they are often too complex and require extensive manual input, which ultimately limits their use in doctor-patient communication and clinical decision making. Here, we present the first large-scale clinical 3D morphable model, a machine-learning-based framework involving supervised learning for diagnostics, risk stratification, and treatment simulation. The model, trained and validated with 4,261 faces of healthy volunteers and orthognathic (jaw) surgery patients, diagnoses patients with 95.5% sensitivity and 95.2% specificity, and simulates surgical outcomes with a mean accuracy of 1.1 ± 0.3 mm. We demonstrate how this model could fully-automatically aid diagnosis and provide patient-specific treatment plans from a 3D scan alone, to help efficient clinical decision making and improve clinical understanding of face shape as a marker for primary and secondary surgery.

An average three-dimensional virtual human skull for a template-assisted maxillofacial surgery

INTRODUCTION

Although many advances have been made in three-dimensional virtual planning in maxillofacial surgery, facial harmony is still difficult to achieve and is heavily dependent on the surgeon's experience. The aim of the study is to present a method to build up an average three-dimensional virtual human skull to be used as a reference template for bone repositioning and reconstruction during maxillofacial surgical interventions.

METHODS

A total of 20 patients (10 females and 10 males) were selected for the optimal outcome after orthognathic surgery. Postoperative cone-beam computed tomography scans were collected and processed in order to obtain three-dimensional digital models of each skull. For male and female subgroups, the three-dimensional skull models were registered and an average three-dimensional virtual skull model was computed. Deviation color maps were calculated to show differences between each postoperative skull model in the population and the obtained average three-dimensional skull. A clinical use case of genioplasty treatment assisted by the provided average three-dimensional skull template was presented.

RESULTS

The overall mean deviation from the average three-dimensional skull model was 1.3 ± 0.6 and 1.6 ± 0.5 mm in male and female subgroups, respectively. For both groups, the greatest deviations were at the area of the mandible, while almost no deviation was found at the zygomatic and orbital areas. In the presented use case, the female average three-dimensional skull model was effectively used for guiding surgical planning.

CONCLUSION

The presented method of obtaining an average three-dimensional virtual human skull may offer the interesting perspective of performing an innovative template-assisted maxillofacial surgery.

Both the Observer's Expertise and the Subject's Facial Symmetry Can Affect Anatomical Position of the Head

PURPOSE

It is easier to judge facial deformity when the patient's head is in anatomic position. The purposes of this study were to determine 1) whether a group of expert observers would agree more than a group of nonexperts on what is the correct anatomic position of the head, 2) whether there would be more variation in the alignment of an asymmetrical face compared with a symmetrical one, and 3) whether the alignments of experts would be more repeatable than those of nonexperts.

MATERIALS AND METHODS

Thirty-one orthodontists (experts) and 31 dental students (nonexperts) were recruited for this mixed-model study. They were shown randomly oriented 3-dimensional head photographs of an adult with a symmetrical face and an adolescent with an asymmetrical face. In viewing software, the observers oriented the images into anatomic position. They repeated the orientations 4 weeks later. Data were analyzed using a generalized linear model and Bland-Altman plots. The primary predictor variables were experience and symmetry status. The outcome variable was the anatomic position of the head. The other variables of interest included time and orientation direction.

RESULTS

There was a statistically significant difference between measurements completed by experts and nonexperts (F_1,60 = 14.83; P < .01). The interaction between expertise and symmetrical status showed a statistically significant difference between symmetrical and asymmetrical faces in the expert and nonexpert groups (F_1,60 = 9.93; P = .003). The interaction between expertise and time showed a statistically significant difference in measurement over time in the expert and nonexpert groups (F_1,60 = 4.66; P = .03).

CONCLUSIONS

The study shows that experts can set a head into anatomic position better than nonexperts. In addition, facial asymmetry has a profound effect on the ability of an observer to align a head in the correct anatomic position. Moreover, observer-guided alignment is not reproducible.

New frontiers in Dental Technology: An Overview

The pace of technological advance across science is staggeringly fast. Our ability to translate some of the potential developments in technology into concepts/products/devices that can assist dentists in caring for patients is key to ensuring that both the profession and the people we care for derive full benefit from these new technologies. This overview will focus in four areas: research and how we gather and interpret data to inform health care; the diagnosis and prevention of disease; planning care; and new concepts in terms of achieving desired health outcomes for patients. Some of the technological advances will be in their infancy and others close to or indeed clinical reality. The objective of this overview is to show where we are in terms of the cutting edge of technology and to whet the appetite for things to come.

Digital platform for planning facial asymmetry orthodontic-surgical treatment preparation

Dentofacial deformities usually are surgically treated, and 3D virtual planning has been used to favor accurate outcomes. Cases reported in the present article show that orthognathic surgery carried out to correct facial asymmetries does not comprise only one treatment protocol. 3D virtual planning might be used for surgical planning, but it should also be used to diagnose the deformity, thus allowing for an analysis of the best-recommended possibilities for the orthodontic preparation that suits each individual case.

Novel condylar repositioning method for 3D-printed models

Background

Along with the advances in technology of three-dimensional (3D) printer, it became a possible to make more precise patient-specific 3D model in the various fields including oral and maxillofacial surgery. When creating 3D models of the mandible and maxilla, it is easier to make a single unit with a fused temporomandibular joint, though this results in poor operability of the model. However, while models created with a separate mandible and maxilla have operability, it can be difficult to fully restore the position of the condylar after simulation. The purpose of this study is to introduce and asses the novel condylar repositioning method in 3D model preoperational simulation.

Methods

Our novel condylar repositioning method is simple to apply two irregularities in 3D models. Three oral surgeons measured and evaluated one linear distance and two angles in 3D models.

Results

This study included two patients who underwent sagittal split ramus osteotomy (SSRO) and two benign tumor patients who underwent segmental mandibulectomy and immediate reconstruction. For each SSRO case, the mandibular condyles were designed to be convex and the glenoid cavities were designed to be concave. For the benign tumor cases, the margins on the resection side, including the joint portions, were designed to be convex, and the resection margin was designed to be concave. The distance from the mandibular ramus to the tip of the maxillary canine, the angle created by joining the inferior edge of the orbit to the tip of the maxillary canine and the ramus, the angle created by the lines from the base of the mentum to the endpoint of the condyle, and the angle between the most lateral point of the condyle and the most medial point of the condyle were measured before and after simulations. Near-complete matches were observed for all items measured before and after model simulations of surgery in all jaw deformity and reconstruction cases.

Conclusions

We demonstrated that 3D models manufactured using our method can be applied to simulations and fully restore the position of the condyle without the need for special devices.