Strategies for developing evidence-based clinical practice guidelines to foster implementation into dental practice

Large language models in periodontology: Assessing their performance in clinically relevant questions

STATEMENT OF PROBLEM

Although the use of artificial intelligence (AI) seems promising and may assist dentists in clinical practice, the consequences of inaccurate or even harmful responses are paramount. Research is required to examine whether large language models (LLMs) can be used in accessing periodontal content reliably.

PURPOSE

The purpose of this study was to evaluate and compare the evidence-based potential of answers provided by 4 LLMs to common clinical questions in the field of periodontology.

MATERIAL AND METHODS

A total of 10 open-ended questions pertinent to periodontology were posed to 4 distinct LLMs: ChatGPT model GPT 4.0, Google Gemini, Google Gemini Advanced, and Microsoft Copilot. The answers to each question were evaluated independently by 2 periodontists against robust scientific evidence based on a predefined rubric assessing the comprehensiveness, scientific accuracy, clarity, and relevance. Each response received a score ranging from 0 (minimum) to 10 (maximum). After a period of 2 weeks from initial evaluation, the answers were re-graded independently to gauge intra-evaluator reliability. Inter-evaluator reliability was assessed using correlation tests, while Cronbach alpha and interclass correlation coefficient were used to measure overall reliability. The Kruskal-Wallis test was employed to compare the scores given by different LLMs.

RESULTS

The scores provided by the 2 evaluators for both evaluations were statistically similar (P values ranging from .083 to >;.999), therefore an average score was calculated for each LLM. Both evaluators gave the highest scores to the answers generated by ChatGPT 4.0, while Google Gemini had the lowest scores. ChatGPT 4.0 received the highest average score, while significant differences were detected between ChatGPT 4.0 and Google Gemini (P=.042). ChatGPT 4.0 answers were found to be highly comprehensive, with scientific accuracy, clarity, and relevance.

CONCLUSIONS

Professionals need to be aware of the limitations of LLMs when utilizing them. These models must not replace dental professionals as improper use may negatively impact patient care. Chat GPT 4.0, Google Gemini, Google Gemini Advanced, and Microsoft CoPilot performed relatively well with Chat GPT 4.0 demonstrating the highest performance.

'Feeling more like a mechanic' - A qualitative study on experiences of caries prevention to patients with recurrent cavities among experienced dentists

OBJECTIVE

To explore experiences of caries prevention in adult patients with recurrent cavities among experienced dentists.  Method: Five focus group discussions consisting of seven men and nine women, 38-61 years of age, and with working experience as dentists between 5 and 35 years, were conducted. The participants represented Public Dental Health Service clinics and private practitioners. Qualitative content analysis was used to analyze data.  Results: The participants emphasized the importance of effective communication and patient engagement in caries prevention. They described their experiences as an endless trail, making fillings. They expressed their inability to take necessary responsibility and being stuck in the dental care system due to various circumstances. The understanding of caries was contradictory, and an inadequate mandate to control time to fulfill their preventive work was evident. They felt responsible to do the best for their patients, but how to share responsibility with colleagues and patients and having enough time for this seemed difficult and unclear. These problems did not motivate to further education in cariology.

CONCLUSION

The findings underscore the urgent need for improvement in preventive caries treatment and the necessity of allocating sufficient time for dentists to engage in this crucial aspect of their work.

Identification of Swedish caries active individuals aged 30-90 years using a life course perspective and SKaPa longitudinal national registry data over a 10-year period

OBJECTIVE

To describe the occurrence of caries disease from a life course perspective using longitudinal data from the Swedish Quality Registry for Caries and Periodontal Disease (SKaPa).  Material and Methods: Data from seven age cohorts (ages 30-90 years), each followed over 10 years, were retrieved from the SKaPa. Using a three-trajectory model, individuals were divided into three trajectories according to their caries development over time: high (15%), moderate (45%), or low (40%). Caries experience was expressed as the mean decayed, missing, and filled surfaces (DMFS) index.

RESULTS

Significant differences were found for all three trajectories and in all age groups over the 10 years. The mean DMFS index increase was significantly larger for the high trajectory group than for the moderate and low trajectory groups across all age cohorts. An increase in caries experience was observed for the older cohorts across all trajectories.

CONCLUSIONS

A three-trajectory model appears useful for identifying and quantifying caries experiences in longitudinal studies. Increased caries disease occurs over time, especially in the highest trajectory group and among older cohorts. These findings emphasise the need for greater attention and more efficient caries prevention methods.

Painless cost control as a central strategy for universal oral health coverage: A critical review with policy guide

AIM

This study aimed to critically review the methods used to control the significantly increasing costs of dental care.

METHODS

Through a comprehensive search of the available literature, the cost control (CC) mechanisms for health services were identified from a healthcare system perspective. The probable applicability of each CC method was evaluated mainly based on its potential contribution to oral health promotion. Each mechanism was then classified and discussed under any of the two headings of financing and service provision. An operational guide was finally presented for policy-making in each of the three main models of healthcare systems, including National Health Services, social/public health insurance and private insurance.

RESULTS

From a total of 142 articles/reports retrieved in PubMed, 73 in Scopus and 791 in Google Scholar, 35 were included in the final review after eliminating the duplicates and screening process. Totally ten mechanisms were identified for CC of dental care. Seven were discussed under the financing function, including cost sharing, preauthorization, mixed payment method and an evidence-based approach to benefit package definition, among others. Three further methods were classified under the service provision function, including workforce skill mix with emphasis on primary oral healthcare providers, development of primary healthcare (PHC) network and an appropriate use of tele-dentistry.

CONCLUSION

Painless control of dental expenditures requires a smart integration of prevention into the CC plans. The suggested policy guide emphasizes organizational factors; particularly including the development of PHC-based networks with midlevel providers (desirably extended-duty dental hygienists) as the frontline oral healthcare providers.

Learning from data in dentistry: Summary of the third annual OpenWide conference

The overarching goal of the third scientific oral health symposium was to introduce the concept of a learning health system to the dental community and to identify and discuss cutting-edge research and strategies using data for improving the quality of dental care and patient safety. Conference participants included clinically active dentists, dental researchers, quality improvement experts, informaticians, insurers, EHR vendors/developers, and members of dental professional organizations and dental service organizations. This report summarizes the main outputs of the third annual OpenWide conference held in Houston, Texas, on October 12, 2022, as an affiliated meeting of the American Dental Association (ADA) 2022 annual conference.

Implementation of guidelines in clinical practice; barriers and strategies

PURPOSE OF REVIEW

Published clinical practice guidelines frequently have difficulties for implementation of the recommendations and adherence in daily clinical practice. The present review summarizes the current knowledge on the barriers encountered when implementing clinical practice guideline and the strategies proposed to address these barriers.

RECENT FINDINGS

Studies on strategies for implementation of clinical guidelines are scarce. Evidence indicates that a multidisciplinary policy is necessary in order to address the barriers at various levels. Continuous education and motivation of the stakeholders, together with structural adaptations are key elements in the process.

SUMMARY

The barriers for implementation of guidelines involve different levels, including the healthcare system, organizational, societal and cultural specificities, and individual attitudes. All of these should be addressed with policy-driven strategies. Such strategies could include optimization of resources allocations, and establishing well coordinated multidisciplinary networks, finally, future studies should also evaluate the effectiveness of the potential strategies.

Evaluation of the Performance of Generative AI Large Language Models ChatGPT, Google Bard, and Microsoft Bing Chat in Supporting Evidence-Based Dentistry: Comparative Mixed Methods Study

BACKGROUND

The increasing application of generative artificial intelligence large language models (LLMs) in various fields, including dentistry, raises questions about their accuracy.

OBJECTIVE

This study aims to comparatively evaluate the answers provided by 4 LLMs, namely Bard (Google LLC), ChatGPT-3.5 and ChatGPT-4 (OpenAI), and Bing Chat (Microsoft Corp), to clinically relevant questions from the field of dentistry.

METHODS

The LLMs were queried with 20 open-type, clinical dentistry-related questions from different disciplines, developed by the respective faculty of the School of Dentistry, European University Cyprus. The LLMs' answers were graded 0 (minimum) to 10 (maximum) points against strong, traditionally collected scientific evidence, such as guidelines and consensus statements, using a rubric, as if they were examination questions posed to students, by 2 experienced faculty members. The scores were statistically compared to identify the best-performing model using the Friedman and Wilcoxon tests. Moreover, the evaluators were asked to provide a qualitative evaluation of the comprehensiveness, scientific accuracy, clarity, and relevance of the LLMs' answers.

RESULTS

Overall, no statistically significant difference was detected between the scores given by the 2 evaluators; therefore, an average score was computed for every LLM. Although ChatGPT-4 statistically outperformed ChatGPT-3.5 (P=.008), Bing Chat (P=.049), and Bard (P=.045), all models occasionally exhibited inaccuracies, generality, outdated content, and a lack of source references. The evaluators noted instances where the LLMs delivered irrelevant information, vague answers, or information that was not fully accurate.

CONCLUSIONS

This study demonstrates that although LLMs hold promising potential as an aid in the implementation of evidence-based dentistry, their current limitations can lead to potentially harmful health care decisions if not used judiciously. Therefore, these tools should not replace the dentist's critical thinking and in-depth understanding of the subject matter. Further research, clinical validation, and model improvements are necessary for these tools to be fully integrated into dental practice. Dental practitioners must be aware of the limitations of LLMs, as their imprudent use could potentially impact patient care. Regulatory measures should be established to oversee the use of these evolving technologies.

Evidence-Informed Oral Health Policy Making: Opportunities and Challenges

Despite a clear need for improvement in oral health systems, progress in oral health systems transformation has been slow. Substantial gaps persist in leveraging evidence and stakeholder values for collective problem solving. To truly enable evidence-informed oral health policy making, substantial "know-how" and "know-do" gaps still need to be overcome. However, there is a unique opportunity for the oral health community to learn and evolve from previous successes and failures in evidence-informed health policy making. As stated by the Global Commission on Evidence to Address Societal Challenges, COVID-19 has created a once-in-a-generation focus on evidence, which has fast-tracked collaboration among decision makers, researchers, and evidence intermediaries. In addition, this has led to a growing recognition of the need to formalize and strengthen evidence-support systems. This article provides an overview of recent advancements in evidence-informed health policy making, including normative goals and a health systems taxonomy, the role of evidence-support and evidence-implementation systems to improve context-specific decision-making processes, the evolution of learning health systems, and the important role of citizen deliberations. The article also highlights opportunities for evidence-informed policy making to drive change in oral health systems. All in all, strengthening capacities for evidence-informed health policy making is critical to enable and enact improvements in oral health systems.

An Integrated Process for Co-Developing and Implementing Written and Computable Clinical Practice Guidelines

The goal of this article is to describe an integrated parallel process for the co-development of written and computable clinical practice guidelines (CPGs) to accelerate adoption and increase the impact of guideline recommendations in clinical practice. From February 2018 through December 2021, interdisciplinary work groups were formed after an initial Kaizen event and using expert consensus and available literature, produced a 12-phase integrated process (IP). The IP includes activities, resources, and iterative feedback loops for developing, implementing, disseminating, communicating, and evaluating CPGs. The IP incorporates guideline standards and informatics practices and clarifies how informaticians, implementers, health communicators, evaluators, and clinicians can help guideline developers throughout the development and implementation cycle to effectively co-develop written and computable guidelines. More efficient processes are essential to create actionable CPGs, disseminate and communicate recommendations to clinical end users, and evaluate CPG performance. Pilot testing is underway to determine how this IP expedites the implementation of CPGs into clinical practice and improves guideline uptake and health outcomes.

Dissemination and implementation research for oral and craniofacial health: Background, a review of literature and future directions

Oral conditions are highly prevalent globally and have profound consequence on individuals and communities. Clinical (e.g. dental treatments, behavioural counselling) and non-clinical (e.g. community-based programming, water fluoridation, oral health policy) evidence-based interventions have been identified, recommended and applied at the clinic, community and policy levels. Still, the burden of oral conditions persists, with inequitable distribution across populations. A major driver of this lack of progress is poor translation of research findings, which results in an evidence-to-practice gap. Dissemination and implementation science (DIS) has emerged to address this gap. A relatively new field, application of DIS represents an important avenue for achieving good dental, oral and craniofacial health for all. The goal of this introductory article is to provide a brief background on DIS relevant to researchers in dentistry and oral health. The problem of knowledge translation, basic concepts and terminology in DIS, and approaches to doing dissemination and implementation research-including implementation strategies, key outcomes, and implementation theories, models and frameworks-are discussed. Additionally, the article reviews literature applying DIS to dentistry and oral health. Results of published studies and their implications for the field are presented. Drawing on the literature review and contemporary thinking in DIS, current gaps, opportunities and future directions are discussed. Resources for understanding and applying DIS are provided throughout. This article serves as a primer on DIS for dental and oral health researchers of all types working across a range of contexts; it also serves as a call to action for increased application of DIS to address the burden of oral conditions globally.