Evaluating quality of dental care among patients with diabetes: Adaptation and testing of a dental quality measure in electronic health records

Learning from data in dentistry: Summary of the third annual OpenWide conference

The overarching goal of the third scientific oral health symposium was to introduce the concept of a learning health system to the dental community and to identify and discuss cutting-edge research and strategies using data for improving the quality of dental care and patient safety. Conference participants included clinically active dentists, dental researchers, quality improvement experts, informaticians, insurers, EHR vendors/developers, and members of dental professional organizations and dental service organizations. This report summarizes the main outputs of the third annual OpenWide conference held in Houston, Texas, on October 12, 2022, as an affiliated meeting of the American Dental Association (ADA) 2022 annual conference.

Current state of dental informatics in the field of health information systems: a scoping review

Background

Over the past 50 years, dental informatics has developed significantly in the field of health information systems. Accordingly, several studies have been conducted on standardized clinical coding systems, data capture, and clinical data reuse in dentistry.

Methods

Based on the definition of health information systems, the literature search was divided into three specific sub-searches: “standardized clinical coding systems,” “data capture,” and “reuse of routine patient care data.” PubMed and Web of Science were searched for peer-reviewed articles. The review was conducted following the PRISMA-ScR protocol.

Results

A total of 44 articles were identified for inclusion in the review. Of these, 15 were related to “standardized clinical coding systems,” 15 to “data capture,” and 14 to “reuse of routine patient care data.” Articles related to standardized clinical coding systems focused on the design and/or development of proposed systems, on their evaluation and validation, on their adoption in academic settings, and on user perception. Articles related to data capture addressed the issue of data completeness, evaluated user interfaces and workflow integration, and proposed technical solutions. Finally, articles related to reuse of routine patient care data focused on clinical decision support systems centered on patient care, institutional or population-based health monitoring support systems, and clinical research.

Conclusions

While the development of health information systems, and especially standardized clinical coding systems, has led to significant progress in research and quality measures, most reviewed articles were published in the US. Clinical decision support systems that reuse EDR data have been little studied. Likewise, few studies have examined the working environment of dental practitioners or the pedagogical value of using health information systems in dentistry.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12903-022-02163-9.

BigMouth: development and maintenance of a successful dental data repository

Few clinical datasets exist in dentistry to conduct secondary research. Hence, a novel dental data repository called BigMouth was developed, which has grown to include 11 academic institutions contributing Electronic Health Record data on over 4.5 million patients. The primary purpose for BigMouth is to serve as a high-quality resource for rapidly conducting oral health-related research. BigMouth allows for assessing the oral health status of a diverse US patient population; provides rationale and evidence for new oral health care delivery modes; and embraces the specific oral health research education mission. A data governance framework that encouraged data sharing while controlling contributed data was initially developed. This transformed over time into a mature framework, including a fee schedule for data requests and allowing access to researchers from noncontributing institutions. Adoption of BigMouth helps to foster new collaborations between clinical, epidemiological, statistical, and informatics experts and provides an additional venue for professional development.

Caries Risk Documentation And Prevention: eMeasures For Dental Electronic Health Records

BACKGROUND

Longitudinal patient level data available in the electronic health record (EHR) allows for the development, implementation, and validations of dental quality measures (eMeasures).

OBJECTIVE

We report the feasibility and validity of implementing two eMeasures. The eMeasures determined the proportion of patients receiving a caries risk assessment (eCRA) and corresponding appropriate risk-based preventative treatments for patients at elevated risk of caries (appropriateness of care [eAoC]) in two academic institutions and one accountable care organization, in the 2019 reporting year.

METHODS

Both eMeasures define the numerator and denominator beginning at the patient level, populations' specifications, and validated the automated queries. For eCRA, patients who completed a comprehensive or periodic oral evaluation formed the denominator, and patients of any age who received a CRA formed the numerator. The eAoC evaluated the proportion of patients at elevated caries risk who received the corresponding appropriate risk-based preventative treatments.

RESULTS

EHR automated queries identified in three sites 269,536 patients who met the inclusion criteria for receiving a CRA. The overall proportion of patients who received a CRA was 94.4% (eCRA). In eAoC, patients at elevated caries risk levels (moderate, high, or extreme) received fluoride preventive treatment ranging from 56 to 93.8%. For patients at high and extreme risk, antimicrobials were prescribed more frequently site 3 (80.6%) than sites 2 (16.7%) and 1 (2.9%).

CONCLUSION

Patient-level data available in the EHRs can be used to implement process-of-care dental eCRA and AoC, eAoC measures identify gaps in clinical practice. EHR-based measures can be useful in improving delivery of evidence-based preventative treatments to reduce risk, prevent tooth decay, and improve oral health.

ESTABLISHING A QUALITY IMPROVEMENT CULTURE WITHIN A LARGE INTEGRATED MEDICAL-DENTAL HEALTH SYSTEM WITH A POPULATION BASED FOCUS

OBJECTIVES

Quality improvement strategies have been an integral part of healthcare to attain improved care delivery and effective health outcomes. The dental quality initiative improvement (DQII) presented in this manuscript represents a case study of successful implementation of a quality improvement culture within a large integrated-medical-dental health system serving a largely rural population.

METHODS

The key elements of DQII included steering committee establishment, definition or dental quality measures and development/implementation of a dental quality analytics dashboard (DQAD) that provides relevant data on dental quality measures. Qualitative metrics were applied to look at the improvement in performance for the various measures relative to quality benchmarks.

RESULTS

DQII facilitated improved oversight of care continuity and provider performance surrounding quality measures at granular and/or institutional level. Improvement associated with care delivery performance relative to benchmarks was observed.

CONCLUSIONS

DQII further advanced the quality improvement culture prevalent in our learning healthcare environment with its focus on value-based care delivery. DQII initiative and establishment of DQAD provided ability to track performance in operational care delivery for dental providers in a clinical setting in real time.

Assessing the completeness of periodontal disease documentation in the EHR: a first step in measuring the quality of care

Background

Our objective was to measure the proportion of patients for which comprehensive periodontal charting, periodontal disease risk factors (diabetes status, tobacco use, and oral home care compliance), and periodontal diagnoses were documented in the electronic health record (EHR). We developed an EHR-based quality measure to assess how well four dental institutions documented periodontal disease-related information. An automated database script was developed and implemented in the EHR at each institution. The measure was validated by comparing the findings from the measure with a manual review of charts.

Results

The overall measure scores varied significantly across the four institutions (institution 1 = 20.47%, institution 2 = 0.97%, institution 3 = 22.27% institution 4 = 99.49%, p-value < 0.0001). The largest gaps in documentation were related to periodontal diagnoses and capturing oral homecare compliance. A random sample of 1224 charts were manually reviewed and showed excellent validity when compared with the data generated from the EHR-based measure (Sensitivity, Specificity, PPV, and NPV > 80%).

Conclusion

Our results demonstrate the feasibility of developing automated data extraction scripts using structured data from EHRs, and successfully implementing these to identify and measure the periodontal documentation completeness within and across different dental institutions.

Measuring sealant placement in children at the dental practice level

Background.

Although sealants are an established and recommended caries-preventive treatment, many children still fail to receive them. In addition, research has shown that existing measures underestimate care by overlooking the sealable potential of teeth before evaluating care. To address this, the authors designed and evaluated 3 novel dental electronic health record–based clinical quality measures that evaluate sealant care only after assessing the sealable potential of teeth.

Methods.

Measure I recorded the proportion of patients with sealable teeth who received sealants. Measure II recorded the proportion of patients who had at least 1 of their sealable teeth sealed. Measure III recorded the proportion of patients who received sealant on all of their sealable teeth.

Results.

On average, 48.1% of 6- through 9-year-old children received 1 or more sealants compared with 32.4% of 10- through 14-year-olds (measure I). The average measure score decreased for patients who received sealants for at least 1 of their sealable teeth (measure II) (43.2% for 6- through 9-year-olds and 28.4% for 10- through 14-year-olds). Fewer children received sealants on all eligible teeth (measure III) (35.5% of 6- through 9-year-olds and 21% of 10- through 14-year-olds received sealant on all eligible teeth). Among the 48.5% who were at elevated caries risk, the sealant rates were higher across all 3 measures.

Conclusions.

A valid and actionable practice-based sealant electronic measure that evaluates sealant treatment among the eligible population, both at the patient level and the tooth level, has been developed.

Practical Implications.

The measure developed in this work provides practices with patient-centered and actionable sealant quality measures that aim to improve oral health outcomes.

Usage and Scientific Properties of the ADVOCATE Oral Health Care Measures

BACKGROUND

Feedback information about the quality of oral health care is important for reflective learning by oral health care professionals and the wider health system. To this end, a list of 48 topics describing oral health and oral health care was recently agreed as part of the EU H2020 ADVOCATE project.

OBJECTIVE

This article reports on the formulation of measures based on the ADVOCATE topics and provides information on usage, reporting, validity, and reliability of the measures.

METHODS

The AIRE instrument was used to guide the methodological approach adopted. The appropriateness of the measures was tested among 39 general dental practitioners (GDPs) in Denmark, Germany, and the Netherlands. Data were collected from a convenience sample of patients via a questionnaire deployed in an online application in their practice. Feasibility, acceptability, and usefulness of the measures were evaluated through focus group interviews with GDPs. Face validity and test-retest reliability of the measures were assessed.

RESULTS

For 46 of the 48 topics, a measure could be defined by constructing a definition and a numerator and denominator. Data collection for all 46 measures was feasible and acceptable for patients using the online questionnaire. The practicalities of using claims data for the purpose of giving feedback to individual and groups of GDPs proved to be challenging in terms of timely access of such data, the granularity of the data, and matching the content of the data with the consented items on quality of oral health care. Face validity was considered appropriate, as the patients found the questionnaire easy to understand. Test-retest reliability was found to be acceptable for 36 of 46 measures.

CONCLUSION

The broad range of the ADVOCATE oral health care measures could make a useful contribution to a more transparent, evidence-based, and patient-centered oral health care system.

KNOWLEDGE TRANSFER STATEMENT

This study shows the usage, reliability, and validity of 46 oral health care measures. The measures, which include patient experience and health behaviors, were found to be useful to stimulate discussions about clinical practice. The measures can provide essential information for quality improvement strategies and useful and relevant feedback information for GDPs.

Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records

Electronic medical records (EMRs) support the development of machine learning algorithms for predicting disease incidence, patient response to treatment, and other healthcare events. But so far most algorithms have been centralized, taking little account of the decentralized, non-identically independently distributed (non-IID), and privacy-sensitive characteristics of EMRs that can complicate data collection, sharing and learning. To address this challenge, we introduced a community-based federated machine learning (CBFL) algorithm and evaluated it on non-IID ICU EMRs. Our algorithm clustered the distributed data into clinically meaningful communities that captured similar diagnoses and geographical locations, and learnt one model for each community. Throughout the learning process, the data was kept local at hospitals, while locally-computed results were aggregated on a server. Evaluation results show that CBFL outperformed the baseline federated machine learning (FL) algorithm in terms of Area Under the Receiver Operating Characteristic Curve (ROC AUC), Area Under the Precision-Recall Curve (PR AUC), and communication cost between hospitals and the server. Furthermore, communities' performance difference could be explained by how dissimilar one community was to others.

Number of Pregnant Women at Four Dental Clinics and the Care They Received: A Dental Quality eMeasure Evaluation

Process-of-care quality measure research can be used to identify gaps in the delivery of dental services to pregnant patients. The aim of this study was to evaluate the types of dental services that pregnant patients received in four dental clinics over five years as documented in the electronic health record (EHR). To accomplish this objective, the authors modified and validated a previously published claims-based dental quality measure for EHR use. After the electronic dental quality measure specifications were defined, the number of pregnant patients was calculated at three academic dental institutions and one large accountable care organization, and the types of dental care services they received over a five-year period (2013-17) were determined. Calibrated reviewers at each institution independently reviewed a sub-sample of patient charts to validate the information obtained from EHR queries, and the concordance between manual chart reviews and EHR query reports was analyzed. Of the 335,078 women aged 15-44 years who received care at the four clinics for the five reporting years, 3.9% (n=13,026) were pregnant. Among these pregnant patients, 48.9% (n=6,366) received a periodic dental examination; 30.0% (n=3,909) received a comprehensive dental exam; and 21.5% (n=2,799) received additional dental services, irrespective of comprehensive or periodic oral evaluations. Overall, the mean proportion of pregnant patients seeking care in these academic dental and group practice clinics was low, but 78.9% of them received either a periodic or comprehensive oral evaluation. Given the importance of oral health care during pregnancy, these findings suggest a need for curriculum development to incorporate prenatal oral health education in the training of dental students.

Tobacco screening and cessation efforts by dental providers: A quality measure evaluation

OBJECTIVES

The purpose of this study was to adapt, test, and evaluate the implementation of a primary care "Preventive care and Screening" meaningful use quality measure for tobacco use, in dental institutions. We determined the percentage of dental patients screened for tobacco use, and the percentage of tobacco users who received cessation counseling.

METHODS

We implemented the dental quality measure (DQM), in three dental schools and a large dental accountable care organization. An automated electronic health record (EHR) query identified patients 18 years and older who were screened for tobacco use one or more times within 24 months, and who received cessation counseling intervention if identified as a tobacco user. We evaluated EHR query performance with a manual review of a subsample of charts.

RESULTS

Across all four sites, in the reporting calendar year of 2015, a total of 143,675 patients met the inclusion criteria for the study. Within 24 months, including 2014 and 2015 calendar years, percentages of tobacco screening ranged from 79.7 to 99.9 percent, while cessation intervention percentages varied from 1 to 81 percent among sites. By employing DQM research methodology, we identified intervention gaps in clinical practice.

CONCLUSIONS

We demonstrated the successful implementation of a DQM to evaluate screening rates for tobacco use and cessation intervention. There is substantial variation in the cessation intervention rates across sites, and these results are a call for action for the dental profession to employ tobacco evidence-based cessation strategies to improve oral health and general health outcomes.

Unintended consequences and challenges of quality measurements in dentistry

BACKGROUND

In recent years, several state dental programs, researchers and the Dental Quality Alliance (DQA) have sought to develop baseline quality measures for dentistry as a way to improve health outcomes, reduce costs and enhance patient experiences. Some of these measures have been tested and validated for various population groups. However, there are some unintended consequences and challenges with quality measurement in dentistry as observed from our previous work on refining and transforming dental quality measures into e-measures.

MAIN BODY

Some examples of the unintended consequences and challenges associated with implementing dental quality measures include: a de-emphasis on patient-centeredness with process-based quality measures, an incentivization of unethical behavior due to fee-for-service reimbursement systems, the risk of compromising patient and provider autonomy with plan-level measures, a disproportionate benefits of dental quality measurement going toward payers, and the risk of alienating smaller dental offices due to the resource-intensive nature of quality measurement.

CONCLUSION

As our medical counterparts have embraced quality measurement for improved health outcomes, so too must the dental profession. Our ultimate goal is to ensure the delivery of high quality, patient-centered dental care and effective quality measurement is the first step. By continuously monitoring the performance of dental quality measures and their continued refinement when unintended consequences are observed, we can improve patient and population health outcomes.

Quality measures for dental care: A systematic review

Objectives

This systematic review aimed to (a) provide an overview of existing quality measures in the field of oral health care, and to (b) evaluate the scientific soundness and applicability of these quality measures.

Methods

A systematic search was conducted in three electronic databases MEDLINE (via PubMed), EMBASE (via OVID) and LILACS (via BIREME). The search was restricted to articles published between 2002 and 2018. Publications reporting on the development process or clinimetric properties of oral health care quality measures for outpatient oral health care in dental practices were included. The identified publications reporting on oral health care quality measures were critically appraised with the Appraisal of Indicators through Research and Evaluation 2.0 (AIRE 2.0) instrument to evaluate the soundness and applicability of the measures.

Results

The search strategy resulted in 2541 unique and potentially relevant articles. In total, 24 publications were included yielding 215 quality measures. The critical appraisal showed a large variation in the quality of the included publications (AIRE scores ranging from 38 to 78 out of 80 possible points). The majority of measures (n = 71) referred to treatment and preventive services. Comparably, few measures referred to the domain patient safety (n = 3). The development process of measures often exhibited a lack of involvement of patients and dental professionals. Few projects reported on the validity (n = 2) and reliability (n = 3) of the measures. Four projects piloted the measures for implementation in practice.

Conclusions

This systematic review provides an overview of the status quo with respect to existing quality measures in oral health care. Potential opportunities include the piloting and testing of quality measures and the establishment of suitable information systems that allow the provision of transparent routine feedback on the quality of oral health care.

Assessing the validity of existing dental sealant quality measures

BACKGROUND

Although sealants are highly effective in preventing caries in children, placement rates continue to be low. The authors' goals were to implement and assess the performance of 2 existing sealant quality measures against a manual audit of charts at 4 dental institutions and to identify measurement gaps that may be filled by using data from electronic health records.

METHODS

The authors evaluated the performance of 2 quality measures designed for claims-based data: the Dental Quality Alliance (DQA) sealant measure, which includes patients at risk of developing elevated caries, and the Oregon Health Authority (OHA) sealant measure (irrespective of caries risk). The authors adapted and validated these measures at 4 sites: 3 dental schools and 1 large dental accountable care organization.

RESULTS

The overall modified DQA and modified OHA measure scores in the 6- through 9-year-old age group were 37.0% and 31.6% and in the 10- through 14-year-old age group were 15.8% and 6.6%, respectively. Results from the manual review of charts showed that 67.6% of children who did not receive sealants did not have any teeth to seal because their molars had not yet erupted, had been extracted, had been sealed previously, or had existing caries or restorations.

CONCLUSIONS

Both the DQA and OHA measures, which rely mainly on Current Dental Terminology procedure codes, led to underestimation of the care delivered from a practice perspective. Future sealant quality measures should exclude patients whose teeth cannot be sealed.

PRACTICAL IMPLICATIONS

This study's results support the suitability of using electronic health record data for assessing the quality of oral health care, particularly for measuring sealant placement in children.