Evaluating the impact of a very low-cost intervention to increase practices' engagement with data and change prescribing behaviour: a randomized trial in English primary care

Development and application of Breadth-Depth-Context (BDC), a conceptual framework for measuring technology engagement with a qualified clinical data registry

Objectives

Despite the proliferation of dashboards that display performance data derived from Qualified Clinical Data Registries (QCDR), the degree to which clinicians and practices engage with such dashboards has not been well described. We aimed to develop a conceptual framework for assessing user engagement with dashboard technology and to demonstrate its application to a rheumatology QCDR.

Materials and Methods

We developed the BDC (Breadth-Depth-Context) framework, which included concepts of breadth (derived from dashboard sessions), depth (derived from dashboard actions), and context (derived from practice characteristics). We demonstrated its application via user log data from the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) registry to define engagement profiles and characterize practice-level factors associated with different profiles.

Results

We applied the BDC framework to 213 ambulatory practices from the RISE registry in 2020-2021, and classified practices into 4 engagement profiles: not engaged (8%), minimally engaged (39%), moderately engaged (34%), and most engaged (19%). Practices with more patients and with specific electronic health record vendors (eClinicalWorks and eMDs) had a higher likelihood of being in the most engaged group, even after adjusting for other factors.

Discussion

We developed the BDC framework to characterize user engagement with a registry dashboard and demonstrated its use in a specialty QCDR. The application of the BDC framework revealed a wide range of breadth and depth of use and that specific contextual factors were associated with nature of engagement.

Conclusion

Going forward, the BDC framework can be used to study engagement with similar dashboards.

Evaluation of different landing pages on behavioural engagement with the CARA dashboard: A user research protocol

BACKGROUND

CARA set out to develop a data-visualisation platform to facilitate general practitioners to develop a deeper understanding of their patient population, disease management and prescribing through dashboards. To support the continued use and sustainability of the CARA dashboards, dashboard performance and user engagement have to be optimised. User research places people at the centre of the design process and aims to evaluate the needs, behaviours and attitudes of users to inform the design, development and impact of a product.

OBJECTIVE

To explore how different initial key messages impact the level of behavioural engagement with a CARA dashboard.

METHODS

Participating general practices can upload their practice data for analysis and visualisation in CARA dashboards. Practices will be randomised to one of three different initial landing pages: the full dashboard or one of two key messages: a between comparison (their practice prescribing with the average of all other practices) or within comparison (with practice data of the same month the previous year) with subsequent continuation to the full dashboard. Analysis will determine which of the three landing pages encourages user interaction, as measured by the number of 'clicks', 'viewings' and 'sessions'. Dashboard usage data will be collected through Google analytics.

DISCUSSION

This study will provide evidence of behavioural engagement and its metrics during the implementation of the CARA dashboards to optimise and sustain interaction.

TRIAL REGISTRATION

ISRCTN32783644 (Registration date: 02/01/2024).

Selection of proxy indicators estimating the appropriateness of antibiotic prescriptions in general practice: a national consensus procedure in France

Background

GPs are responsible for more than 70% of outpatient antibiotic prescriptions in France. Metrics are important antibiotic stewardship tools that can be used to set targets for improvement and to give feedback to professionals and stakeholders.

Objectives

The primary objective of the present study was to select a set of proxy indicators (PIs) based on 10 previously developed PIs, to estimate the appropriateness of antibiotic prescriptions by GPs. The secondary objective was to evaluate the clinimetric properties of the selected PIs.

Methods

A RAND-modified Delphi consensus procedure was conducted with a multidisciplinary panel of stakeholders. This procedure consisted of two successive online surveys with a consensus meeting in between. Clinimetric properties (measurability, applicability and potential room for improvement) were evaluated for the PIs selected through the consensus procedure, using 2022 Regional Health Insurance data.

Results

Seventeen experts participated in the first-round survey and 14 in the second-round. A final set of 12 PIs was selected. Among the 10 initial PIs, 3 were selected without modification and 7 were modified and selected. Moreover, two newly suggested PIs were selected. Ten of the 12 PIs presented good clinimetric properties.

Conclusions

The 12 selected PIs cover the main situations responsible for inappropriate and unnecessary use of antibiotics in general practice. These PIs, easily calculable using routinely collected health insurance reimbursement data, might be used to give feedback to prescribers and stakeholders and help improve antibiotic prescriptions in primary care.

Sustainability, spread, and scale in trials using audit and feedback: a theory-informed, secondary analysis of a systematic review

BACKGROUND

Audit and feedback (A&F) is a widely used implementation strategy to influence health professionals' behavior that is often tested in implementation trials. This study examines how A&F trials describe sustainability, spread, and scale.

METHODS

This is a theory-informed, descriptive, secondary analysis of an update of the Cochrane systematic review of A&F trials, including all trials published since 2011. Keyword searches related to sustainability, spread, and scale were conducted. Trials with at least one keyword, and those identified from a forward citation search, were extracted to examine how they described sustainability, spread, and scale. Results were qualitatively analyzed using the Integrated Sustainability Framework (ISF) and the Framework for Going to Full Scale (FGFS).

RESULTS

From the larger review, n = 161 studies met eligibility criteria. Seventy-eight percent (n = 126) of trials included at least one keyword on sustainability, and 49% (n = 62) of those studies (39% overall) frequently mentioned sustainability based on inclusion of relevant text in multiple sections of the paper. For spread/scale, 62% (n = 100) of trials included at least one relevant keyword and 51% (n = 51) of those studies (31% overall) frequently mentioned spread/scale. A total of n = 38 studies from the forward citation search were included in the qualitative analysis. Although many studies mentioned the need to consider sustainability, there was limited detail on how this was planned, implemented, or assessed. The most frequent sustainability period duration was 12 months. Qualitative results mapped to the ISF, but not all determinants were represented. Strong alignment was found with the FGFS for phases of scale-up and support systems (infrastructure), but not for adoption mechanisms. New spread/scale themes included (1) aligning affordability and scalability; (2) balancing fidelity and scalability; and (3) balancing effect size and scalability.

CONCLUSION

A&F trials should plan for sustainability, spread, and scale so that if the trial is effective, the benefits can continue. A deeper empirical understanding of the factors impacting A&F sustainability is needed. Scalability planning should go beyond cost and infrastructure to consider other adoption mechanisms, such as leadership, policy, and communication, that may support further scalability.

TRIAL REGISTRATION

Registered with Prospero in May 2022. CRD42022332606.

Nudge interventions to reduce unnecessary antibiotic prescribing in primary care: a systematic review

OBJECTIVES

Antibiotic prescribing in primary care contributes significantly to antibiotic overuse. Nudge interventions alter the decision-making environment to achieve behaviour change without restricting options. Our objectives were to conduct a systematic review to describe the types of nudge interventions used to reduce unnecessary antibiotic prescribing in primary care, their key features, and their effects on antibiotic prescribing overall.

METHODS

Medline, Embase and grey literature were searched for randomised trials or regression discontinuity studies in April 2021. Risk of bias was assessed independently by two researchers using the Cochrane Effective Practice and Organisation of Care group's tool. Results were synthesised to report the percentage of studies demonstrating a reduction in overall antibiotic prescribing for different types of nudges. Effects of social norm nudges were examined for features that may enhance effectiveness.

RESULTS

Nineteen studies were included, testing 23 nudge interventions. Four studies were rated as having a high risk of bias, nine as moderate risk of bias and six as at low risk. Overall, 78.3% (n=18, 95% CI 58.1 to 90.3) of the nudges evaluated resulted in a reduction in overall antibiotic prescribing. Social norm feedback was the most frequently applied nudge (n=17), with 76.5% (n=13; 95% CI 52.7 to 90.4) of these studies reporting a reduction. Other nudges applied were changing option consequences (n=3; with 2 reporting a reduction), providing reminders (n=2; 2 reporting a reduction) and facilitating commitment (n=1; reporting a reduction). Successful social norm nudges typically either included an injunctive norm, compared prescribing to physicians with the lowest prescribers or targeted high prescribers.

CONCLUSIONS

Nudge interventions are effective for improving antibiotic prescribing in primary care. Expanding the use of nudge interventions beyond social norm nudges could reap further improvements in antibiotic prescribing practices. Policy-makers and managers need to be mindful of how social norm nudges are implemented to enhance intervention effects.

The Effectiveness of Interactive Dashboards to Optimise Antibiotic Prescribing in Primary Care: A Systematic Review

Governments and healthcare organisations collect data on antibiotic prescribing (AP) for surveillance. This data can support tools for visualisations and feedback to GPs using dashboards that may prompt a change in prescribing behaviour. The objective of this systematic review was to assess the effectiveness of interactive dashboards to optimise AP in primary care. Six electronic databases were searched for relevant studies up to August 2022. A narrative synthesis of findings was conducted to evaluate the intervention processes and results. Two independent reviewers assessed the relevance, risk of bias and quality of the evidence. A total of ten studies were included (eight RCTs and two non-RCTs). Overall, seven studies showed a slight reduction in AP. However, this reduction in AP when offering a dashboard may not in itself result in reductions but only when combined with educational components, public commitment or behavioural strategies. Only one study recorded dashboard engagement and showed a difference of 10% (95% CI 5% to 15%) between intervention and control. None of the studies reported on the development, pilot or implementation of dashboards or the involvement of stakeholders in design and testing. Interactive dashboards may reduce AP in primary care but most likely only when combined with other educational or behavioural intervention strategies.

Feedback of Antibiotic Prescribing in Primary Care (FAPPC) trial: results of a real-world cluster randomized controlled trial in Scotland, UK

OBJECTIVES

To evaluate the effect of general practice-level prescribing feedback on antibiotic prescribing in a real-world pragmatic cluster randomized controlled trial.

METHODS

Three hundred and forty general practices in four territorial Health Boards in NHS Scotland were randomized in Quarter 1, 2016 to receive four quarterly antibiotic-prescribing feedback reports or not, from Quarter 2, 2016 to Quarter 1, 2017. Reports included different clinical topics, benchmarking against national and health board rates, and behavioural messaging with improvement actions. The primary outcome was total antibiotic prescribing rate. There were 16 secondary prescribing outcomes and 5 hospital admission outcomes (potential adverse effects of reduced prescribing). The main evaluation timepoint was 1 year after the final report (Quarter 1, 2018), with an additional evaluation in the quarter after the final report (Quarter 2, 2017). Routine administrative NHS data were used to generate the feedback reports and analyse the effects.

RESULTS

Total antibiotic prescribing rates were lower at the main evaluation timepoint in both intervention (1.83 versus baseline 1.93 prescriptions/1000 patients/day) and control (1.90 versus baseline 1.98) practices, with no evidence of intervention effect [adjusted rate ratio (ARR) 0.98 (95% CI 0.94-1.02; P = 0.35)]. At the additional timepoint, adjusted total antibiotic prescribing rates were 1.67 and 1.73 prescriptions/1000 patients/day, with evidence of a small intervention effect, ARR 0.99 (0.98-1.00; P = 0.03).

CONCLUSIONS

This well-designed, practice-level antibiotic-prescribing feedback had limited evidence of additional effects in the context of decreasing antibiotic prescribing and an established national stewardship programme.