How to Make Feedback More Effective? Qualitative Findings from Pilot Testing of an Audit and Feedback Report for Endoscopists

Pediatric endoscopy: how can we improve patient outcomes and ensure best practices?

INTRODUCTION

Strategies to promote high-quality endoscopy in children require consensus around pediatric-specific quality standards and indicators. Using a rigorous guideline development process, the international Pediatric Endoscopy Quality Improvement Network (PEnQuIN) was developed to support continuous quality improvement efforts within and across pediatric endoscopy services.

AREAS COVERED

This review presents a framework, informed by the PEnQuIN guidelines, for assessing endoscopist competence, granting procedural privileges, audit and feedback, and for skill remediation, when required. As is critical for promoting quality, PEnQuIN indicators can be benchmarked at the individual endoscopist, endoscopy facility, and endoscopy community levels. Furthermore, efforts to incorporate technologies, including electronic medical records and artificial intelligence, into endoscopic quality improvement processes can aid in creation of large-scale networks to facilitate comparison and standardization of quality indicator reporting across sites.

EXPERT OPINION

PEnQuIN quality standards and indicators provide a framework for continuous quality improvement in pediatric endoscopy, benefiting individual endoscopists, endoscopy facilities, and the broader endoscopy community. Routine and reliable measurement of data, facilitated by technology, is required to identify and drive improvements in care. Engaging all stakeholders in endoscopy quality improvement processes is crucial to enhancing patient outcomes and establishing best practices for safe, efficient, and effective pediatric endoscopic care.

An exploration into physician and surgeon data sensemaking: a qualitative systematic review using thematic synthesis

Providing electronic health data to medical practitioners to reflect on their performance can lead to improved clinical performance and quality of care. Understanding the sensemaking process that is enacted when practitioners are presented with such data is vital to ensure an improvement in performance. Thus, the primary objective of this research was to explore physician and surgeon sensemaking when presented with electronic health data associated with their clinical performance. A systematic literature review was conducted to analyse qualitative research that explored physicians and surgeons experiences with electronic health data associated with their clinical performance published between January 2010 and March 2022. Included articles were assessed for quality, thematically synthesised, and discussed from the perspective of sensemaking. The initial search strategy for this review returned 8,829 articles that were screened at title and abstract level. Subsequent screening found 11 articles that met the eligibility criteria and were retained for analyses. Two articles met all of the standards within the chosen quality assessment (Standards for Reporting Qualitative Research, SRQR). Thematic synthesis generated five overarching themes: data communication, performance reflection, infrastructure, data quality, and risks. The confidence of such findings is reported using CERQual (Confidence in the Evidence from Reviews of Qualitative research). The way the data is communicated can impact sensemaking which has implications on what is learned and has impact on future performance. Many factors including data accuracy, validity, infrastructure, culture can also impact sensemaking and have ramifications on future practice. Providing data in order to support performance reflection is not without risks, both behavioural and affective. The latter of which can impact the practitioner's ability to effectively make sense of the data. An important consideration when data is presented with the intent to improve performance.Registration This systematic review was registered with Prospero, registration number: CRD42020197392.

A Pragmatic Randomized Controlled Trial of an Endoscopist Audit and Feedback Report for Colonoscopy

INTRODUCTION

Variation in endoscopist performance contributes to poor-quality colonoscopy. Audit and feedback (A/F) can be used to improve physician performance, particularly among lower performing physicians. In this large pragmatic randomized controlled trial, we compared A/F to improve endoscopists' colonoscopy performance to usual practice.

METHODS

Endoscopists practicing in Ontario, Canada, in 2014 were randomly assigned in October 2015 (index date) to receive (intervention group, n = 417) or not receive (control group, n = 416) an A/F report generated centrally using health administrative data. Colonoscopy performance was measured in both groups over two 12-month periods: prereport and postreport (relative to the index date). The primary outcome was polypectomy rate (PR). Secondary outcomes were cecal intubation rate, bowel preparation, and premature repeat after normal colonoscopy. A post hoc analysis used adenoma detection rate as the outcome. Outcomes were compared between groups for all endoscopists and for lower performing endoscopists using Poisson regression analyses under a difference-in-difference framework.

RESULTS

Among all endoscopists, PR did not significantly improve from prereport to postreport periods for those receiving the intervention (relative rate [RR], intervention vs control: 1.07 vs 1.05, P = 0.09). Among lower performing endoscopists, PR improved significantly (RR, intervention vs control 1.34 vs 1.11, P = 0.02) in the intervention group compared with controls. In this subgroup, adenoma detection rate also improved but not significantly (RR, intervention vs control 1.12 vs 1.04, P = 0.12). There was no significant improvement in secondary outcomes between the intervention and control groups.

DISCUSSION

A/F reports for colonoscopy improve performance in lower performing endoscopists (ClinicalTrials.gov: NCT02595775).

How well do critical care audit and feedback interventions adhere to best practice? Development and application of the REFLECT-52 evaluation tool

BACKGROUND

Healthcare Audit and Feedback (A&F) interventions have been shown to be an effective means of changing healthcare professional behavior, but work is required to optimize them, as evidence suggests that A&F interventions are not improving over time. Recent published guidance has suggested an initial set of best practices that may help to increase intervention effectiveness, which focus on the "Nature of the desired action," "Nature of the data available for feedback," "Feedback display," and "Delivering the feedback intervention." We aimed to develop a generalizable evaluation tool that can be used to assess whether A&F interventions conform to these suggestions for best practice and conducted initial testing of the tool through application to a sample of critical care A&F interventions.

METHODS

We used a consensus-based approach to develop an evaluation tool from published guidance and subsequently applied the tool to conduct a secondary analysis of A&F interventions. To start, the 15 suggestions for improved feedback interventions published by Brehaut et al. were deconstructed into rateable items. Items were developed through iterative consensus meetings among researchers. These items were then piloted on 12 A&F studies (two reviewers met for consensus each time after independently applying the tool to four A&F intervention studies). After each consensus meeting, items were modified to improve clarity and specificity, and to help increase the reliability between coders. We then assessed the conformity to best practices of 17 critical care A&F interventions, sourced from a systematic review of A&F interventions on provider ordering of laboratory tests and transfusions in the critical care setting. Data for each criteria item was extracted by one coder and confirmed by a second; results were then aggregated and presented graphically or in a table and described narratively.

RESULTS

In total, 52 criteria items were developed (38 ratable items and 14 descriptive items). Eight studies targeted lab test ordering behaviors, and 10 studies targeted blood transfusion ordering. Items focused on specifying the "Nature of the Desired Action" were adhered to most commonly-feedback was often presented in the context of an external priority (13/17), showed or described a discrepancy in performance (14/17), and in all cases it was reasonable for the recipients to be responsible for the change in behavior (17/17). Items focused on the "Nature of the Data Available for Feedback" were adhered to less often-only some interventions provided individual (5/17) or patient-level data (5/17), and few included aspirational comparators (2/17), or justifications for specificity of feedback (4/17), choice of comparator (0/9) or the interval between reports (3/13). Items focused on the "Nature of the Feedback Display" were reported poorly-just under half of interventions reported providing feedback in more than one way (8/17) and interventions rarely included pilot-testing of the feedback (1/17 unclear) or presentation of a visual display and summary message in close proximity of each other (1/13). Items focused on "Delivering the Feedback Intervention" were also poorly reported-feedback rarely reported use of barrier/enabler assessments (0/17), involved target members in the development of the feedback (0/17), or involved explicit design to be received and discussed in a social context (3/17); however, most interventions clearly indicated who was providing the feedback (11/17), involved a facilitator (8/12) or involved engaging in self-assessment around the target behavior prior to receipt of feedback (12/17).

CONCLUSIONS

Many of the theory-informed best practice items were not consistently applied in critical care and can suggest clear ways to improve interventions. Standardized reporting of detailed intervention descriptions and feedback templates may also help to further advance research in this field. The 52-item tool can serve as a basis for reliably assessing concordance with best practice guidance in existing A&F interventions trialed in other healthcare settings, and could be used to inform future A&F intervention development.

TRIAL REGISTRATION

Not applicable.

Colonoscopy quality in community hospitals and nonhospital facilities in Korea

BACKGROUND/AIMS

High-quality colonoscopy is essential to reduce colorectal cancer-related deaths. Little is known about colonoscopy quality in non-academic practice settings. We aimed to evaluate the quality of colonoscopies performed in community hospitals and nonhospital facilities.

METHODS

Colonoscopy data were collected from patients referred to six tertiary care centers after receiving colonoscopies at community hospitals and nonhospital facilities. Based on their photographs, we measured quality indicators including cecal intubation rate, withdrawal time, adequacy of bowel preparation, and number of polyps.

RESULTS

Data from a total of 1,064 colonoscopies were analyzed. The overall cecal intubation rate was 93.1%. The median withdrawal time was 8.3 minutes, but 31.3% of colonoscopies were withdrawn within 6 minutes. Community hospitals had longer withdrawal time and more polyps than nonhospital facilities (median withdrawal time: 9.9 minutes vs. 7.5 minutes, p < 0.001; mean number of polyps: 3.1 vs. 2.3, p = 0.001). Board-certified endoscopists had a higher rate of cecal intubation than non-board-certified endoscopists (93.2% vs. 85.2%, p = 0.006). A total of 819 follow-up colonoscopies were performed at referral centers with a median interval of 28 days. In total, 2,546 polyps were detected at baseline, and 1,088 were newly identified (polyp miss rate, 29.9%). Multivariable analysis revealed that older age (odds ratio [OR], 1.032; 95% confidence interval [CI], 1.020 to 1.044) and male sex (OR, 1.719; 95% CI, 1.281 to 2.308) were associated with increased risk of missed polyps.

CONCLUSION

The quality of colonoscopies performed in community hospitals and nonhospital facilities was suboptimal. Systematic reporting, auditing, and feedback are needed for quality improvement.