Systematic comparison of machine learning algorithms to develop and validate predictive models for periodontitis

Accuracy of Artificial Intelligence Models in the Prediction of Periodontitis: A Systematic Review

INTRODUCTION

Periodontitis is the main cause of tooth loss and is related to many systemic diseases. Artificial intelligence (AI) in periodontics has the potential to improve the accuracy of risk assessment and provide personalized treatment planning for patients with periodontitis. This systematic review aims to examine the actual evidence on the accuracy of various AI models in predicting periodontitis.

METHODS

Using a mix of MeSH keywords and free text words pooled by Boolean operators ('AND', 'OR'), a search strategy without a time frame setting was conducted on the following databases: Web of Science, ProQuest, PubMed, Scopus, and IEEE Explore. The QUADAS-2 risk of bias assessment was then performed.

RESULTS

From a total of 961 identified records screened, 8 articles were included for qualitative analysis: 4 studies showed an overall low risk of bias, 2 studies an unclear risk, and the remaining 2 studies a high risk. The most employed algorithms for periodontitis prediction were artificial neural networks, followed by support vector machines, decision trees, logistic regression, and random forest. The models showed good predictive performance for periodontitis according to different evaluation metrics, but the presented methods were heterogeneous.

CONCLUSIONS

AI algorithms may improve in the future the accuracy and reliability of periodontitis prediction. However, to date, most of the studies had a retrospective design and did not consider the most modern deep learning networks. Although the available evidence is limited by a lack of standardized data collection and protocols, the potential benefits of using AI in periodontics are significant and warrant further research and development in this area.

KNOWLEDGE TRANSFER STATEMENT

The use of AI in periodontics can lead to more accurate diagnosis and treatment planning, as well as improved patient education and engagement. Despite the current challenges and limitations of the available evidence, particularly the lack of standardized data collection and analysis protocols, the potential benefits of using AI in periodontics are significant and warrant further research and development in this area.

Automated Machine Learning to Develop Predictive Models of Metabolic Syndrome in Patients with Periodontal Disease

Metabolic syndrome is experiencing a concerning and escalating rise in prevalence today. The link between metabolic syndrome and periodontal disease is a highly relevant area of research. Some studies have suggested a bidirectional relationship between metabolic syndrome and periodontal disease, where one condition may exacerbate the other. Furthermore, the existence of periodontal disease among these individuals significantly impacts overall health management. This research focuses on the relationship between periodontal disease and metabolic syndrome, while also incorporating data on general health status and overall well-being. We aimed to develop advanced machine learning models that efficiently identify key predictors of metabolic syndrome, a significant emphasis being placed on thoroughly explaining the predictions generated by the models. We studied a group of 296 patients, hospitalized in SCJU Sibiu, aged between 45-79 years, of which 57% had metabolic syndrome. The patients underwent dental consultations and subsequently responded to a dedicated questionnaire, along with a standard EuroQol 5-Dimensions 5-Levels (EQ-5D-5L) questionnaire. The following data were recorded: DMFT (Decayed, Missing due to caries, and Filled Teeth), CPI (Community Periodontal Index), periodontal pockets depth, loss of epithelial insertion, bleeding after probing, frequency of tooth brushing, regular dental control, cardiovascular risk, carotid atherosclerosis, and EQ-5D-5L score. We used Automated Machine Learning (AutoML) frameworks to build predictive models in order to determine which of these risk factors exhibits the most robust association with metabolic syndrome. To gain confidence in the results provided by the machine learning models provided by the AutoML pipelines, we used SHapley Additive exPlanations (SHAP) values for the interpretability of these models, from a global and local perspective. The obtained results confirm that the severity of periodontal disease, high cardiovascular risk, and low EQ-5D-5L score have the greatest impact in the occurrence of metabolic syndrome.

Suitability of machine learning models for prediction of clinically defined Stage III/IV periodontitis from questionnaires and demographic data in Danish cohorts

AIM

To evaluate if, and to what extent, machine learning models can capture clinically defined Stage III/IV periodontitis from self-report questionnaires and demographic data.

MATERIALS AND METHODS

Self-reported measures of periodontitis, demographic data and clinically established Stage III/IV periodontitis status were extracted from two Danish population-based cohorts (The Copenhagen Aging and Midlife Biobank [CAMB] and The Danish Health Examination Survey [DANHES]) and used to develop cross-validated machine learning models for the prediction of clinically established Stage III/IV periodontitis. Models were trained using 10-fold cross-validations repeated three times on the CAMB dataset (n = 1476), and the resulting models were validated in the DANHES dataset (n = 3585).

RESULTS

The prevalence of Stage III/IV periodontitis was 23.2% (n = 342) in the CAMB dataset and 9.3% (n = 335) in the DANHES dataset. For the prediction of clinically established Stage III/IV periodontitis in the CAMB cohort, models reached area under the receiver operating characteristics (AUROCs) of 0.67-0.69, sensitivities of 0.58-0.64 and specificities of 0.71-0.80. In the DANHES cohort, models derived from the CAMB cohort achieved AUROCs of 0.64-0.70, sensitivities of 0.44-0.63 and specificities of 0.75-0.84.

CONCLUSIONS

Applying cross-validated machine learning algorithms to demographic data and self-reported measures of periodontitis resulted in models with modest capabilities for the prediction of Stage III/IV periodontitis in two Danish cohorts.

Development and international validation of logistic regression and machine-learning models for the prediction of 10-year molar loss

AIM

To develop and validate models based on logistic regression and artificial intelligence for prognostic prediction of molar survival in periodontally affected patients.

MATERIALS AND METHODS

Clinical and radiographic data from four different centres across four continents (two in Europe, one in the United States, and one in China) including 515 patients and 3157 molars were collected and used to train and test different types of machine-learning algorithms for their prognostic ability of molar loss over 10 years. The following models were trained: logistic regression, support vector machine, K-nearest neighbours, decision tree, random forest, artificial neural network, gradient boosting, and naive Bayes. In addition, different models were aggregated by means of the ensembled stacking method. The primary outcome of the study was related to the prediction of overall molar loss (MLO) in patients after active periodontal treatment.

RESULTS

The general performance in the external validation settings (aggregating three cohorts) revealed that the ensembled model, which combined neural network and logistic regression, showed the best performance among the different models for the prediction of MLO with an area under the curve (AUC) = 0.726. The neural network model showed the best AUC of 0.724 for the prediction of periodontitis-related molar loss. In addition, the ensembled model showed the best calibration performance.

CONCLUSIONS

Through a multi-centre collaboration, both prognostic models for the prediction of molar loss were developed and externally validated. The ensembled model showed the best performance in terms of both discrimination and validation, and it is made freely available to clinicians for widespread use in clinical practice.