Accuracy of automated identification of lateral cephalometric landmarks using cascade convolutional neural networks on lateral cephalograms from nationwide multi-centres

Cephalometric analysis performance discrepancy between orthodontists and an artificial intelligence model using lateral cephalometric radiographs

PURPOSE

The purpose of the present clinical study was to compare the Ricketts and Steiner cephalometric analysis obtained by two experienced orthodontists and artificial intelligence (AI)-based software program and measure the orthodontist variability.

MATERIALS AND METHODS

A total of 50 lateral cephalometric radiographs from 50 patients were obtained. Two groups were created depending on the operator performing the cephalometric analysis: orthodontists (Orthod group) and an AI software program (AI group). In the Orthod group, two independent experienced orthodontists performed the measurements by performing a manual identification of the cephalometric landmarks and a software program (NemoCeph; Nemotec) to calculate the measurements. In the AI group, an AI software program (CephX; ORCA Dental AI) was selected for both the automatic landmark identification and cephalometric measurements. The Ricketts and Steiner cephalometric analyses were assessed in both groups including a total of 24 measurements. The Shapiro-Wilk test showed that the data was normally distributed. The t-test was used to analyze the data (α = 0.05).

RESULTS

The t-test analysis showed significant measurement discrepancies between the Orthod and AI group in seven of the 24 cephalometric parameters tested, namely the corpus length (p = 0.003), mandibular arc (p < 0.001), lower face height (p = 0.005), overjet (p = 0.019), and overbite (p = 0.022) in the Ricketts cephalometric analysis and occlusal to SN (p = 0.002) and GoGn-SN (p < 0.001) in the Steiner cephalometric analysis. The intraclass correlation coefficient (ICC) between both orthodontists of the Orthod group for each cephalometric measurement was calculated.

CONCLUSIONS

Significant discrepancies were found in seven of the 24 cephalometric measurements tested between the orthodontists and the AI-based program assessed. The intra-operator reliability analysis showed reproducible measurements between both orthodontists, except for the corpus length measurement.

CLINICAL SIGNIFICANCE

The artificial intelligence software program tested has the potential to automatically obtain cephalometric analysis using lateral cephalometric radiographs; however, additional studies are needed to further evaluate the accuracy of this AI-based system.

Landmark annotation through feature combinations: a comparative study on cephalometric images with in-depth analysis of model's explainability

OBJECTIVES

The objectives of this study are to explore and evaluate the automation of anatomical landmark localization in cephalometric images using machine learning techniques, with a focus on feature extraction and combinations, contextual analysis, and model interpretability through Shapley Additive exPlanations (SHAP) values.

METHODS

We conducted extensive experimentation on a private dataset of 300 lateral cephalograms to thoroughly study the annotation results obtained using pixel feature descriptors including raw pixel, gradient magnitude, gradient direction, and histogram-oriented gradient (HOG) values. The study includes evaluation and comparison of these feature descriptions calculated at different contexts namely local, pyramid, and global. The feature descriptor obtained using individual combinations is used to discern between landmark and nonlandmark pixels using classification method. Additionally, this study addresses the opacity of LGBM ensemble tree models across landmarks, introducing SHAP values to enhance interpretability.

RESULTS

The performance of feature combinations was assessed using metrics like mean radial error, standard deviation, success detection rate (SDR) (2 mm), and test time. Remarkably, among all the combinations explored, both the HOG and gradient direction operations demonstrated significant performance across all context combinations. At the contextual level, the global texture outperformed the others, although it came with the trade-off of increased test time. The HOG in the local context emerged as the top performer with an SDR of 75.84% compared to others.

CONCLUSIONS

The presented analysis enhances the understanding of the significance of different features and their combinations in the realm of landmark annotation but also paves the way for further exploration of landmark-specific feature combination methods, facilitated by explainability.

Accuracy of artificial intelligence-assisted growth prediction in skeletal Class I preadolescent patients using serial lateral cephalograms for a 2-year growth interval

OBJECTIVE

To investigate the accuracy of artificial intelligence-assisted growth prediction using a convolutional neural network (CNN) algorithm and longitudinal lateral cephalograms (Lat-cephs).

MATERIALS AND METHODS

A total of 198 Japanese preadolescent children, who had skeletal Class I malocclusion and whose Lat-cephs were available at age 8 years (T0) and 10 years (T1), were allocated into the training, validation, and test phases (n = 161, n = 17, n = 20). Orthodontists and the CNN model identified 28 hard-tissue landmarks (HTL) and 19 soft-tissue landmarks (STL). The mean prediction error values were defined as 'excellent,' 'very good,' 'good,' 'acceptable,' and 'unsatisfactory' (criteria: 0.5 mm, 1.0 mm, 1.5 mm, and 2.0 mm, respectively). The degree of accurate prediction percentage (APP) was defined as 'very high,' 'high,' 'medium,' and 'low' (criteria: 90%, 70%, and 50%, respectively) according to the percentage of subjects that showed the error range within 1.5 mm.

RESULTS

All HTLs showed acceptable-to-excellent mean PE values, while the STLs Pog', Gn', and Me' showed unsatisfactory values, and the rest showed good-to-acceptable values. Regarding the degree of APP, HTLs Ba, ramus posterior, Pm, Pog, B-point, Me, and mandibular first molar root apex exhibited low APPs. The STLs labrale superius, lower embrasure, lower lip, point of lower profile, B', Pog,' Gn' and Me' also exhibited low APPs. The remainder of HTLs and STLs showed medium-to-very high APPs.

CONCLUSION

Despite the possibility of using the CNN model to predict growth, further studies are needed to improve the prediction accuracy in HTLs and STLs of the chin area.

Accuracy and clinical validity of automated cephalometric analysis using convolutional neural networks

BACKGROUND

This study aimed to assess the error range of cephalometric measurements based on the landmarks detected using cascaded CNNs and determine how horizontal and vertical positional errors of individual landmarks affect lateral cephalometric measurements.

METHODS

In total, 120 lateral cephalograms were obtained consecutively from patients (mean age, 32.5 ± 11.6) who visited the Asan Medical Center, Seoul, Korea, for orthodontic treatment between 2019 and 2021. An automated lateral cephalometric analysis model previously developed from a nationwide multi-centre database was used to digitize the lateral cephalograms. The horizontal and vertical landmark position error attributable to the AI model was defined as the distance between the landmark identified by the human and that identified by the AI model on the x- and y-axes. The differences between the cephalometric measurements based on the landmarks identified by the AI model vs those identified by the human examiner were assessed. The association between the lateral cephalometric measurements and the positioning errors in the landmarks comprising the cephalometric measurement was assessed.

RESULTS

The mean difference in the angular and linear measurements based on AI vs human landmark localization was .99 ± 1.05°, and .80 ± .82 mm, respectively. Significant differences between the measurements derived from AI-based and human localization were observed for all cephalometric variables except SNA, pog-Nperp, facial angle, SN-GoGn, FMA, Bjork sum, U1-SN, U1-FH, IMPA, L1-NB (angular) and interincisal angle.

CONCLUSIONS

The errors in landmark positions, especially those that define reference planes, may significantly affect cephalometric measurements. The possibility of errors generated by automated lateral cephalometric analysis systems should be considered when using such systems for orthodontic diagnoses.

Applications of artificial intelligence in dentomaxillofacial imaging-a systematic review

BACKGROUND

Artificial intelligence (AI) technology has been increasingly developed in oral and maxillofacial imaging. The aim of this systematic review was to assess the applications and performance of the developed algorithms in different dentomaxillofacial imaging modalities.

STUDY DESIGN

A systematic search of PubMed and Scopus databases was performed. The search strategy was set as a combination of the following keywords: "Artificial Intelligence," "Machine Learning," "Deep Learning," "Neural Networks," "Head and Neck Imaging," and "Maxillofacial Imaging." Full-text screening and data extraction were independently conducted by two independent reviewers; any mismatch was resolved by discussion. The risk of bias was assessed by one reviewer and validated by another.

RESULTS

The search returned a total of 3,392 articles. After careful evaluation of the titles, abstracts, and full texts, a total number of 194 articles were included. Most studies focused on AI applications for tooth and implant classification and identification, 3-dimensional cephalometric landmark detection, lesion detection (periapical, jaws, and bone), and osteoporosis detection.

CONCLUSION

Despite the AI models' limitations, they showed promising results. Further studies are needed to explore specific applications and real-world scenarios before confidently integrating these models into dental practice.

Artificial Intelligence and Its Clinical Applications in Orthodontics: A Systematic Review

This review aims to analyze different strategies that make use of artificial intelligence to enhance diagnosis, treatment planning, and monitoring in orthodontics. Orthodontics has seen significant technological advancements with the introduction of digital equipment, including cone beam computed tomography, intraoral scanners, and software coupled to these devices. The use of deep learning in software has sped up image processing processes. Deep learning is an artificial intelligence technology that trains computers to analyze data like the human brain does. Deep learning models are capable of recognizing complex patterns in photos, text, audio, and other data to generate accurate information and predictions.

MATERIALS AND METHODS

Pubmed, Scopus, and Web of Science were used to discover publications from 1 January 2013 to 18 October 2023 that matched our topic. A comparison of various artificial intelligence applications in orthodontics was generated.

RESULTS

A final number of 33 studies were included in the review for qualitative analysis.

CONCLUSIONS

These studies demonstrate the effectiveness of AI in enhancing orthodontic diagnosis, treatment planning, and assessment. A lot of articles emphasize the integration of artificial intelligence into orthodontics and its potential to revolutionize treatment monitoring, evaluation, and patient outcomes.

Application of Artificial Intelligence in Orthodontics: Current State and Future Perspectives

In recent years, there has been the notable emergency of artificial intelligence (AI) as a transformative force in multiple domains, including orthodontics. This review aims to provide a comprehensive overview of the present state of AI applications in orthodontics, which can be categorized into the following domains: (1) diagnosis, including cephalometric analysis, dental analysis, facial analysis, skeletal-maturation-stage determination and upper-airway obstruction assessment; (2) treatment planning, including decision making for extractions and orthognathic surgery, and treatment outcome prediction; and (3) clinical practice, including practice guidance, remote care, and clinical documentation. We have witnessed a broadening of the application of AI in orthodontics, accompanied by advancements in its performance. Additionally, this review outlines the existing limitations within the field and offers future perspectives.

Assessment of the quality of different commercial providers using artificial intelligence for automated cephalometric analysis compared to human orthodontic experts

PURPOSE

The aim of this investigation was to evaluate the accuracy of various skeletal and dental cephalometric parameters as produced by different commercial providers that make use of artificial intelligence (AI)-assisted automated cephalometric analysis and to compare their quality to a gold standard established by orthodontic experts.

METHODS

Twelve experienced orthodontic examiners pinpointed 15 radiographic landmarks on a total of 50 cephalometric X‑rays. The landmarks were used to generate 9 parameters for orthodontic treatment planning. The "humans' gold standard" was defined by calculating the median value of all 12 human assessments for each parameter, which in turn served as reference values for comparisons with results given by four different commercial providers of automated cephalometric analyses (DentaliQ.ortho [CellmatiQ GmbH, Hamburg, Germany], WebCeph [AssembleCircle Corp, Seongnam-si, Korea], AudaxCeph [Audax d.o.o., Ljubljana, Slovenia], CephX [Orca Dental AI, Herzliya, Israel]). Repeated measures analysis of variances (ANOVAs) were calculated and Bland-Altman plots were generated for comparisons.

RESULTS

The results of the repeated measures ANOVAs indicated significant differences between the commercial providers' predictions and the humans' gold standard for all nine investigated parameters. However, the pairwise comparisons also demonstrate that there were major differences among the four commercial providers. While there were no significant mean differences between the values of DentaliQ.ortho and the humans' gold standard, the predictions of AudaxCeph showed significant deviations in seven out of nine parameters. Also, the Bland-Altman plots demonstrate that a reduced precision of AI predictions must be expected especially for values attributed to the inclination of the incisors.

CONCLUSION

Fully automated cephalometric analyses are promising in terms of timesaving and avoidance of individual human errors. At present, however, they should only be used under supervision of experienced clinicians.

A Novel Machine Learning Model for Predicting Orthodontic Treatment Duration

In the field of orthodontics, providing patients with accurate treatment time estimates is of utmost importance. As orthodontic practices continue to evolve and embrace new advancements, incorporating machine learning (ML) methods becomes increasingly valuable in improving orthodontic diagnosis and treatment planning. This study aimed to develop a novel ML model capable of predicting the orthodontic treatment duration based on essential pre-treatment variables. Patients who completed comprehensive orthodontic treatment at the Indiana University School of Dentistry were included in this retrospective study. Fifty-seven pre-treatment variables were collected and used to train and test nine different ML models. The performance of each model was assessed using descriptive statistics, intraclass correlation coefficients, and one-way analysis of variance tests. Random Forest, Lasso, and Elastic Net were found to be the most accurate, with a mean absolute error of 7.27 months in predicting treatment duration. Extraction decision, COVID, intermaxillary relationship, lower incisor position, and additional appliances were identified as important predictors of treatment duration. Overall, this study demonstrates the potential of ML in predicting orthodontic treatment duration using pre-treatment variables.

Short- and Long-Term Prediction of the Post-Pubertal Mandibular Length and Y-Axis in Females Utilizing Machine Learning

The aim of this study was to create a novel machine learning (ML) algorithm for predicting the post-pubertal mandibular length and Y-axis in females. Cephalometric data from 176 females with Angle Class I occlusion were used to train and test seven ML algorithms. For all ML methods tested, the mean absolute errors (MAEs) for the 2-year prediction ranged from 2.78 to 5.40 mm and 0.88 to 1.48 degrees, respectively. For the 4-year prediction, MAEs of mandibular length and Y-axis ranged from 3.21 to 4.00 mm and 1.19 to 5.12 degrees, respectively. The most predictive factors for post-pubertal mandibular length were mandibular length at previous timepoints, age, sagittal positions of the maxillary and mandibular skeletal bases, mandibular plane angle, and anterior and posterior face heights. The most predictive factors for post-pubertal Y-axis were Y-axis at previous timepoints, mandibular plane angle, and sagittal positions of the maxillary and mandibular skeletal bases. ML methods were identified as capable of predicting mandibular length within 3 mm and Y-axis within 1 degree. Compared to each other, all of the ML algorithms were similarly accurate, with the exception of multilayer perceptron regressor.

Prediction of Pubertal Mandibular Growth in Males with Class II Malocclusion by Utilizing Machine Learning

The goal of this study was to create a novel machine learning (ML) model that can predict the magnitude and direction of pubertal mandibular growth in males with Class II malocclusion. Lateral cephalometric radiographs of 123 males at three time points (T1: 12; T2: 14; T3: 16 years old) were collected from an online database of longitudinal growth studies. Each radiograph was traced, and seven different ML models were trained using 38 data points obtained from 92 subjects. Thirty-one subjects were used as the test group to predict the post-pubertal mandibular length and y-axis, using input data from T1 and T2 combined (2 year prediction), and T1 alone (4 year prediction). Mean absolute errors (MAEs) were used to evaluate the accuracy of each model. For all ML methods tested using the 2 year prediction, the MAEs for post-pubertal mandibular length ranged from 2.11-6.07 mm to 0.85-2.74° for the y-axis. For all ML methods tested with 4 year prediction, the MAEs for post-pubertal mandibular length ranged from 2.32-5.28 mm to 1.25-1.72° for the y-axis. Besides its initial length, the most predictive factors for mandibular length were found to be chronological age, upper and lower face heights, upper and lower incisor positions, and inclinations. For the y-axis, the most predictive factors were found to be y-axis at earlier time points, SN-MP, SN-Pog, SNB, and SNA. Although the potential of ML techniques to accurately forecast future mandibular growth in Class II cases is promising, a requirement for more substantial sample sizes exists to further enhance the precision of these predictions.

Automated diagnosis of flatfoot using cascaded convolutional neural network for angle measurements in weight-bearing lateral radiographs

OBJECTIVES

Diagnosis of flatfoot using a radiograph is subject to intra- and inter-observer variabilities. Here, we developed a cascade convolutional neural network (CNN)-based deep learning model (DLM) for an automated angle measurement for flatfoot diagnosis using landmark detection.

METHODS

We used 1200 weight-bearing lateral foot radiographs from young adult Korean males for the model development. An experienced orthopedic surgeon identified 22 radiographic landmarks and measured three angles for flatfoot diagnosis that served as the ground truth (GT). Another orthopedic surgeon (OS) and a general physician (GP) independently identified the landmarks of the test dataset and measured the angles using the same method. External validation was performed using 100 and 17 radiographs acquired from a tertiary referral center and a public database, respectively.

RESULTS

The DLM showed smaller absolute average errors from the GT for the three angle measurements for flatfoot diagnosis compared with both human observers. Under the guidance of the DLM, the average errors of observers OS and GP decreased from 2.35° ± 3.01° to 1.55° ± 2.09° and from 1.99° ± 2.76° to 1.56° ± 2.19°, respectively (both p < 0.001). The total measurement time decreased from 195 to 135 min in observer OS and from 205 to 155 min in observer GP. The absolute average errors of the DLM in the external validation sets were similar or superior to those of human observers in the original test dataset.

CONCLUSIONS

Our CNN model had significantly better accuracy and reliability than human observers in diagnosing flatfoot, and notably improved the accuracy and reliability of human observers.

KEY POINTS

• Development of deep learning model (DLM) that allows automated angle measurements for landmark detection based on 1200 weight-bearing lateral radiographs for diagnosing flatfoot. • Our DLM showed smaller absolute average errors for flatfoot diagnosis compared with two human observers. • Under the guidance of the model, the average errors of two human observers decreased and total measurement time also decreased from 195 to 135 min and from 205 to 155 min.

Artificial intelligence system for automated landmark localization and analysis of cephalometry

OBJECTIVES

Cephalometric analysis is essential for diagnosis, treatment planning and outcome assessment of orthodontics and orthognathic surgery. Utilizing artificial intelligence (AI) to achieve automated landmark localization has proved feasible and convenient. However, current systems remain insufficient for clinical application, as patients exhibit various malocclusions in cephalograms produced by different manufacturers while limited cephalograms were applied to train AI in these systems.

METHODS

A robust and clinically applicable AI system was proposed for automatic cephalometric analysis. First, 9870 cephalograms taken by different radiography machines with various malocclusions of patients were collected from 20 medical institutions. Then 30 landmarks of all these cephalogram samples were manually annotated to train an AI system, composed of a two-stage convolutional neural network and a software-as-a-service system. Further, more than 100 orthodontists participated to refine the AI-output landmark localizations and retrain this system.

RESULTS

The average landmark prediction error of this system was as low as 0.94 ± 0.74 mm and the system achieved an average classification accuracy of 89.33%.

CONCLUSIONS

An automatic cephalometric analysis system based on convolutional neural network was proposed, which can realize automatic landmark location and cephalometric measurements classification. This system showed promise in improving diagnostic efficiency in clinical circumstances.

Effectiveness of cone-beam computed tomography-generated cephalograms using artificial intelligence cephalometric analysis

Lateral cephalograms and related analysis constitute representative methods for orthodontic treatment. However, since conventional cephalometric radiographs display a three-dimensional structure on a two-dimensional plane, inaccuracies may be produced when quantitative evaluation is required. Cone-beam computed tomography (CBCT) has minimal image distortion, and important parts can be observed without overlapping. It provides a high-resolution three-dimensional image at a relatively low dose and cost, but still shows a higher dose than a lateral cephalogram. It is especially true for children who are more susceptible to radiation doses and often have difficult diagnoses. A conventional lateral cephalometric radiograph can be obtained by reconstructing the Digital Imaging and Communications in Medicine data obtained from CBCT. This study evaluated the applicability and consistency of lateral cephalograms generated by CBCT using an artificial intelligence analysis program. Group I comprised conventional lateral cephalometric radiographs, group II comprised lateral cephalometric radiographs generated from CBCT using OnDemand 3D, and group III comprised lateral cephalometric radiographs generated from CBCT using Invivo5. All measurements in the three groups showed non-significant results. Therefore, a CBCT scan and artificial intelligence programs are efficient means when performing orthodontic analysis on pediatric or orthodontic patients for orthodontic diagnosis and planning.

Accuracy of artificial intelligence-assisted landmark identification in serial lateral cephalograms of Class III patients who underwent orthodontic treatment and two-jaw orthognathic surgery

Objective

To investigate the pattern of accuracy change in artificial intelligence-assisted landmark identification (LI) using a convolutional neural network (CNN) algorithm in serial lateral cephalograms (Lat-cephs) of Class III (C-III) patients who underwent two-jaw orthognathic surgery.

Methods

A total of 3,188 Lat-cephs of C-III patients were allocated into the training and validation sets (3,004 Lat-cephs of 751 patients) and test set (184 Lat-cephs of 46 patients; subdivided into the genioplasty and non-genioplasty groups, n = 23 per group) for LI. Each C-III patient in the test set had four Lat-cephs initial (T0), pre-surgery (T1, presence of orthodontic brackets [OBs]), post-surgery (T2, presence of OBs and surgical plates and screws [S-PS]), and debonding (T3, presence of S-PS and fixed retainers [FR]). After mean errors of 20 landmarks between human gold standard and the CNN model were calculated, statistical analysis was performed.

Results

The total mean error was 1.17 mm without significant difference among the four time-points (T0, 1.20 mm; T1, 1.14 mm; T2, 1.18 mm; T3, 1.15 mm). In comparison of two time-points ([T0, T1] vs. [T2, T3]), ANS, A point, and B point showed an increase in error (p < 0.01, 0.05, 0.01, respectively), while Mx6D and Md6D showeda decrease in error (all p < 0.01). No difference in errors existed at B point, Pogonion, Menton, Md1C, and Md1R between the genioplasty and non-genioplasty groups.

Conclusions

The CNN model can be used for LI in serial Lat-cephs despite the presence of OB, S-PS, FR, genioplasty, and bone remodeling.