Exploring the content and delivery of feedback facilitation co-interventions: a systematic review

BACKGROUND

Policymakers and researchers recommend supporting the capabilities of feedback recipients to increase the quality of care. There are different ways to support capabilities. We aimed to describe the content and delivery of feedback facilitation interventions delivered alongside audit and feedback within randomised controlled trials.

METHODS

We included papers describing feedback facilitation identified by the latest Cochrane review of audit and feedback. The piloted extraction proforma was based upon a framework to describe intervention content, with additional prompts relating to the identification of influences, selection of improvement actions and consideration of priorities and implications. We describe the content and delivery graphically, statistically and narratively.

RESULTS

We reviewed 146 papers describing 104 feedback facilitation interventions. Across included studies, feedback facilitation contained 26 different implementation strategies. There was a median of three implementation strategies per intervention and evidence that the number of strategies per intervention is increasing. Theory was used in 35 trials, although the precise role of theory was poorly described. Ten studies provided a logic model and six of these described their mechanisms of action. Both the exploration of influences and the selection of improvement actions were described in 46 of the feedback facilitation interventions; we describe who undertook this tailoring work. Exploring dose, there was large variation in duration (15-1800 min), frequency (1 to 42 times) and number of recipients per site (1 to 135). There were important gaps in reporting, but some evidence that reporting is improving over time.

CONCLUSIONS

Heterogeneity in the design of feedback facilitation needs to be considered when assessing the intervention's effectiveness. We describe explicit feedback facilitation choices for future intervention developers based upon choices made to date. We found the Expert Recommendations for Implementing Change to be valuable when describing intervention components, with the potential for some minor clarifications in terms and for greater specificity by intervention providers. Reporting demonstrated extensive gaps which hinder both replication and learning. Feedback facilitation providers are recommended to close reporting gaps that hinder replication. Future work should seek to address the 'opportunity' for improvement activity, defined as factors that lie outside the individual that make care or improvement behaviour possible.

REVIEW REGISTRATION

The study protocol was published at: https://www.protocols.io/private/4DA5DE33B68E11ED9EF70A58A9FEAC02 .

Potentiality of algorithms and artificial intelligence adoption to improve medication management in primary care: a systematic review

OBJECTIVES

The aim of this study is to investigate the effect of artificial intelligence (AI) and/or algorithms on drug management in primary care settings comparing AI and/or algorithms with standard clinical practice. Second, we evaluated what is the most frequently reported type of medication error and the most used AI machine type.

METHODS

A systematic review of literature was conducted querying PubMed, Cochrane and ISI Web of Science until November 2021. The search strategy and the study selection were conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses and the Population, Intervention, Comparator, Outcome framework. Specifically, the Population chosen was general population of all ages (ie, including paediatric patients) in primary care settings (ie, home setting, ambulatory and nursery homes); the Intervention considered was the analysis AI and/or algorithms (ie, intelligent programs or software) application in primary care for reducing medications errors, the Comparator was the general practice and, lastly, the Outcome was the reduction of preventable medication errors (eg, overprescribing, inappropriate medication, drug interaction, risk of injury, dosing errors or in an increase in adherence to therapy). The methodological quality of included studies was appraised adopting the Quality Assessment of Controlled Intervention Studies of the National Institute of Health for randomised controlled trials.

RESULTS

Studies reported in different ways the effective reduction of medication error. Ten out of 14 included studies, corresponding to 71% of articles, reported a reduction of medication errors, supporting the hypothesis that AI is an important tool for patient safety.

CONCLUSION

This study highlights how a proper application of AI in primary care is possible, since it provides an important tool to support the physician with drug management in non-hospital environments.

Costs of Physician Continuous Professional Development: A Systematic Review

PURPOSE

An essential yet oft-neglected step in cost evaluations is the selection of resources (ingredients) to include in cost estimates. The ingredients that most influence the cost of physician continuous professional development (CPD) are unknown, as are the relative costs of instructional modalities. This study's purpose was to estimate the costs of cost ingredients and instructional modalities in physician CPD.

METHOD

The authors conducted a systematic review in April 2020, searching MEDLINE, Embase, PsycInfo, and the Cochrane Library for comparative cost evaluations of CPD for practicing physicians. Two reviewers, working independently, screened articles for inclusion and extracted information on costs (converted to 2021 U.S. dollars) for each intervention overall, each ingredient, and each modality.

RESULTS

Of 3,338 eligible studies, 62 were included, enumerating costs for 86 discrete training interventions or instructional modalities. The most frequently reported ingredients were faculty time (25 of 86 interventions), materials (24), administrator/staff time (23), and travel (20). Ingredient costs varied widely, ranging from a per-physician median of $4 for postage (10 interventions) to $525 for learner time (13); equipment (9) and faculty time were also relatively expensive (median > $170). Among instructional modalities (≤ 11 interventions per modality), audit and feedback performed by physician learners, computer-based modules, computer-based virtual patients, in-person lectures, and experiences with real patients were relatively expensive (median > $1,000 per physician). Mailed paper materials, video clips, and audit and feedback performed by others were relatively inexpensive (median ≤ $62 per physician). Details regarding ingredient selection (10 of 62 studies), quantitation (10), and pricing (26) were reported infrequently.

CONCLUSIONS

Some ingredients, including time, are more important (i.e., contribute more to total costs) than others and should be prioritized in cost evaluations. Data on the relative costs of instructional modalities are insightful but limited. The methods and reporting of cost valuations merit improvement.

Economic evaluations of audit and feedback interventions: a systematic review

BACKGROUND

The effectiveness of audit and feedback (A&F) interventions to improve compliance to healthcare guidelines is supported by randomised controlled trials (RCTs) and meta-analyses of RCTs. However, there is currently a knowledge gap on their cost-effectiveness.

OBJECTIVE

We aimed to assess whether A&F interventions targeting improvements in compliance to recommended care are economically favourable.

METHODS

We conducted a systematic review including experimental, observational and simulation-based economic evaluation studies of A&F interventions targeting healthcare providers. Comparators were a 'do nothing' strategy, or any other intervention not involving A&F or involving a subset of A&F intervention components. We searched MEDLINE, CINAHL, CENTRAL, Econlit, EMBASE, Health Technology Assessment Database, MEDLINE, NHS Economic Evaluation Database, ABI/INFORM, Web of Science, ProQuest and websites of healthcare quality associations to December 2021. Outcomes were incremental cost-effectiveness ratios, incremental cost-utility ratios, incremental net benefit and incremental cost-benefit ratios. Pairs of reviewers independently selected eligible studies and extracted relevant data. Reporting quality was evaluated using CHEERS (Consolidated Health Economic Evaluation Reporting Standards). Results were synthesised using permutation matrices for all studies and predefined subgroups.

RESULTS

Of 13 221 unique citations, 35 studies met our inclusion criteria. The A&F intervention was dominant (ie, at least as effective with lower cost) in 7 studies, potentially cost-effective in 26 and was dominated (ie, the same or less effectiveness and higher costs) in 2 studies. A&F interventions were more likely to be economically favourable in studies based on health outcomes rather than compliance to recommended practice, considering medical costs in addition to intervention costs, published since 2010, and with high reporting quality.

DISCUSSION

Results suggest that A&F interventions may have a high potential to be cost-effective. However, as is common in systematic reviews of economic evaluations, publication bias could have led to an overestimation of their economic value.

Ambulatory Medication Safety in Primary Care: A Systematic Review

PURPOSE

To review the literature on medication safety in primary care in the electronic health record era.

METHODS

Included studies measured rates and outcomes of medication safety in patients whose prescriptions were written in primary care clinics with electronic prescribing. Four investigators independently reviewed titles and analyzed abstracts with dual-reviewer review for eligibility, characteristics, and risk of bias.

RESULTS

Of 1464 articles identified, 56 met the inclusion criteria. Forty-three studies were noninterventional and 13 included an intervention. The majority of the studies (30) used their own definition of error. The most common outcomes were potentially inappropriate prescribing/medications (PIPs), adverse drug events (ADEs), and potential prescribing omissions (PPOs). Most of the studies only included high-risk subpopulations (39), usually older adults taking > 4 medications. The rate of PIPs varied widely (0.19% to 98.2%). The rate of ADEs was lower (0.47% to 14.7%). There was poor correlation of PIP and PPO with documented ADEs leading to physical harm.

CONCLUSIONS

This literature is limited by its inconsistent and highly variable outcomes. The majority of medication safety studies in primary care were in high-risk populations and measured potential harms rather than actual harms. Applying algorithms to primary care medication lists significantly overestimates rate of actual harms.

Cost-effectiveness and Economic Benefit of Continuous Professional Development for Drug Prescribing: A Systematic Review

Importance

The economic impact of continuous professional development (CPD) education is incompletely understood.

Objective

To systematically identify and synthesize published research examining the costs associated with physician CPD for drug prescribing.

Evidence Review

MEDLINE, Embase, PsycInfo, and the Cochrane Database were searched from inception to April 23, 2020, for comparative studies that evaluated the cost of CPD focused on drug prescribing. Two reviewers independently screened all articles for inclusion and reviewed all included articles to extract data on participants, educational interventions, study designs, and outcomes (costs and effectiveness). Results were synthesized for educational costs, health care costs, and cost-effectiveness.

Findings

Of 3338 articles screened, 38 were included in this analysis. These studies included at least 15 659 health care professionals and 1 963 197 patients. Twelve studies reported on educational costs, ranging from $281 to $183 554 (median, $15 664). When economic outcomes were evaluated, 31 of 33 studies (94%) comparing CPD with no intervention found that CPD was associated with reduced health care costs (drug costs), ranging from $4731 to $6 912 000 (median, $79 373). Four studies found reduced drug costs for 1-on-1 outreach compared with other CPD approaches. Regarding cost-effectiveness, among 5 studies that compared CPD with no intervention, the incremental cost-effectiveness ratio for a 10% improvement in prescribing ranged from $15 390 to $437 027 to train all program participants. Four comparisons of alternative CPD approaches found that 1-on-1 educational outreach was more effective but more expensive than group education or mailed materials (incremental cost-effectiveness ratio, $18-$4105 per physician trained).

Conclusions and Relevance

In this systematic review, CPD for drug prescribing was associated with reduced health care (drug) costs. The educational costs and cost-effectiveness of CPD varied widely. Several CPD instructional approaches (including educational outreach) were more effective but more costly than comparators.

Safer prescribing and care for the elderly (SPACE): cluster randomised controlled trial in general practice

Background

Safer prescribing in general practice may help to decrease preventable adverse drug events (ADE) and related hospitalisations.

Aim

To test the effect of the Safer Prescribing and Care for the Elderly (SPACE) intervention on high-risk prescribing of non-steroidal anti-inflammatory drugs (NSAIDs) and/or antiplatelet medicines and related hospitalisations.

Design & setting

A pragmatic cluster randomised controlled trial in general practice. Participants were patients at increased risk of ADEs from NSAIDs and/or antiplatelet medicines at baseline. SPACE comprises automated search to generate for each GP a list of patients with high-risk prescribing; pharmacist outreach to provide education and one-on-one review of list with GP; and automated letter inviting patients to seek medication review with their GP.

Method

The primary outcome was the difference in high-risk prescribing of NSAIDs and/or antiplatelet medicines at 6 months. Secondary outcomes were high-risk prescribing for gastrointestinal, renal, or cardiac ADEs separately, 12-month outcomes, and related ADE hospitalisations.

Results

Thirty-nine practices were recruited with 205 GPs and 191 593 patients, of which 21 877 (11.4%) were participants. Of the participants, 1479 (6.8%) had high-risk prescribing. High-risk prescribing improved in both groups at 6 and 12 months compared with baseline. At 6 months, there was no significant difference between groups (odds ratio [OR] 0.99; 95% confidence intervals [CI] = 0.87 to 1.13) although SPACE improved more for gastrointestinal ADEs (OR 0.81; 95% CI = 0.68 to 0.96). At 12 months, the control group improved more (OR 1.29; 95% CI = 1.11 to 1.49). There was no significant difference for related hospitalisations.

Conclusion

Further work is needed to identify scalable interventions that support safer prescribing in general practice. The use of automated search and feedback plus letter to patient warrants further exploration.

Economic Evaluations Informed Exclusively by Real World Data: A Systematic Review

Economic evaluations using Real World Data (RWD) has been increasing in the very recent years, however, this source of information has several advantages and limitations. The aim of this review was to assess the quality of full economic evaluations (EE) developed using RWD. A systematic review was carried out through articles from the following databases: PubMed, Embase, Web of Science and Centre for Reviews and Dissemination. Included were studies that employed RWD for both costs and effectiveness. Methodological quality of the studies was assessed using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Of the 14,011 studies identified, 93 were included. Roughly half of the studies were carried out in a hospital setting. The most frequently assessed illnesses were neoplasms while the most evaluated interventions were pharmacological. The main source of costs and effects of RWD were information systems. The most frequent clinical outcome was survival. Some 47% of studies met at least 80% of CHEERS criteria. Studies were conducted with samples of 100-1000 patients or more, were randomized, and those that reported bias controls were those that fulfilled most CHEERS criteria. In conclusion, fewer than half the studies met 80% of the CHEERS checklist criteria.

Using Electronic Health Records to Identify Adverse Drug Events in Ambulatory Care: A Systematic Review

OBJECTIVE

We identified the methods used and determined the roles of electronic health records (EHRs) in detecting and assessing adverse drug events (ADEs) in the ambulatory setting.

METHODS

We performed a systematic literature review by searching PubMed and Google Scholar for studies on ADEs detected in the ambulatory setting involving any EHR use published before June 2017. We extracted study characteristics from included studies related to ADE detection methods for analysis.

RESULTS

We identified 30 studies that evaluated ADEs in an ambulatory setting with an EHR. In 27 studies, EHRs were used only as the data source for ADE identification. In two studies, the EHR was used as both a data source and to deliver decision support to providers during order entry. In one study, the EHR was a source of data and generated patient safety reports that researchers used in the process of identifying ADEs. Methods of identification included manual chart review by trained nurses, pharmacists, and/or physicians; prescription review; computer monitors; electronic triggers; International Classification of Diseases codes; natural language processing of clinical notes; and patient phone calls and surveys. Seven studies provided examples of search phrases, laboratory values, and rules used to identify ADEs.

CONCLUSION

The majority of studies examined used EHRs as sources of data for ADE detection. This retrospective approach is appropriate to measure incidence rates of ADEs but not adequate to detect preventable ADEs before patient harm occurs. New methods involving computer monitors and electronic triggers will enable researchers to catch preventable ADEs and take corrective action.

A survey of attitudes, practices, and knowledge regarding drug-drug interactions among medical residents in Iran

Background When prescribing medications, physicians should recognize clinically relevant potential drug-drug interactions (DDIs). To improve medication safety, it is important to understand prescribers' knowledge and opinions pertaining to DDIs. Objective To determine the current DDI information sources used by medical residents, their knowledge of DDIs, their opinions about performance feedback on co-prescription of interacting drugs. Setting Academic hospitals of Mashhad University of Medical Sciences (MUMS) in Iran. Methods A questionnaire containing questions regarding demographic and practice characteristics, DDI information sources, ability to recognize DDIs, and opinions about performance feedback was distributed to medical residents of 22 specialties in eight academic hospitals in Iran. We analyzed their perception pertaining to DDIs, their performance on classifying drug pairs, and we used a linear regression model to assess the association of potential determinants on their DDI knowledge. Main Outcome Measure prescribers' knowledge and opinions pertaining to DDIs. Results The overall response rate and completion rate for 315 distributed questionnaires were 90% (n = 295) and 86% (n = 281), respectively. Among DDI information sources, books, software on mobile phone or tablet, and Internet were the most commonly-used references. Residents could correctly classify only 41% (5.7/14) of the drug pairs. The regression model showed no significant association between residents' characteristics and their DDI knowledge. An overwhelming majority of the respondents (n = 268, 95.4%) wished to receive performance feedback on co-prescription of interacting drugs in their prescriptions. They mostly selected information technology-based tools (i.e. short text message and email) as their preferred method of receiving feedback. Conclusion Our findings indicate that prescribers may have poor ability to prevent clinically relevant potential DDI occurrence, and they perceive the need for performance feedback. These findings underline the importance of well-designed computerized alerting systems and delivering performance feedback to improve patient safety.

Safety of medication use in primary care

BACKGROUND

Medication errors are one of the leading causes of harmin health care. Review and analysis of errors have often emphasized their preventable nature and potential for reoccurrence. Of the few error studies conducted in primary care to date, most have focused on evaluating individual parts of the medicines management system. Studying individual parts of the system does not provide a complete perspective and may further weaken the evidence and undermine interventions.

AIM AND OBJECTIVES

The aim of this review is to estimate the scale of medication errors as a problem across the medicines management system in primary care. Objectives were: To review studies addressing the rates of medication errors, and To identify studies on interventions to prevent medication errors in primary care.

METHODS

A systematic search of the literature was performed in PubMed (MEDLINE), International Pharmaceutical Abstracts (IPA), Embase, PsycINFO, PASCAL, Science Direct, Scopus, Web of Knowledge, and CINAHL PLUS from 1999 to November, 2012. Bibliographies of relevant publications were searched for additional studies.

KEY FINDINGS

Thirty-three studies estimating the incidence of medication errors and thirty-six studies evaluating the impact of error-prevention interventions in primary care were reviewed. This review demonstrated that medication errors are common, with error rates between <1% and >90%, depending on the part of the system studied, and the definitions and methods used. The prescribing stage is the most susceptible, and that the elderly (over 65 years), and children (under 18 years) are more likely to experience significant errors. Individual interventions demonstrated marginal improvements in medication safety when implemented on their own.

CONCLUSION

Targeting the more susceptible population groups and the most dangerous aspects of the system may be a more effective approach to error management and prevention. Co-implementation of existing interventions at points within the system may offer time- and cost-effective options to improving medication safety in primary care.

Audit and feedback: effects on professional practice and healthcare outcomes

BACKGROUND

Audit and feedback is widely used as a strategy to improve professional practice either on its own or as a component of multifaceted quality improvement interventions. This is based on the belief that healthcare professionals are prompted to modify their practice when given performance feedback showing that their clinical practice is inconsistent with a desirable target. Despite its prevalence as a quality improvement strategy, there remains uncertainty regarding both the effectiveness of audit and feedback in improving healthcare practice and the characteristics of audit and feedback that lead to greater impact.

OBJECTIVES

To assess the effects of audit and feedback on the practice of healthcare professionals and patient outcomes and to examine factors that may explain variation in the effectiveness of audit and feedback.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) 2010, Issue 4, part of The Cochrane Library. www.thecochranelibrary.com, including the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register (searched 10 December 2010); MEDLINE, Ovid (1950 to November Week 3 2010) (searched 09 December 2010); EMBASE, Ovid (1980 to 2010 Week 48) (searched 09 December 2010); CINAHL, Ebsco (1981 to present) (searched 10 December 2010); Science Citation Index and Social Sciences Citation Index, ISI Web of Science (1975 to present) (searched 12-15 September 2011).

SELECTION CRITERIA

Randomised trials of audit and feedback (defined as a summary of clinical performance over a specified period of time) that reported objectively measured health professional practice or patient outcomes. In the case of multifaceted interventions, only trials in which audit and feedback was considered the core, essential aspect of at least one intervention arm were included.

DATA COLLECTION AND ANALYSIS

All data were abstracted by two independent review authors. For the primary outcome(s) in each study, we calculated the median absolute risk difference (RD) (adjusted for baseline performance) of compliance with desired practice compliance for dichotomous outcomes and the median percent change relative to the control group for continuous outcomes. Across studies the median effect size was weighted by number of health professionals involved in each study. We investigated the following factors as possible explanations for the variation in the effectiveness of interventions across comparisons: format of feedback, source of feedback, frequency of feedback, instructions for improvement, direction of change required, baseline performance, profession of recipient, and risk of bias within the trial itself. We also conducted exploratory analyses to assess the role of context and the targeted clinical behaviour. Quantitative (meta-regression), visual, and qualitative analyses were undertaken to examine variation in effect size related to these factors.

MAIN RESULTS

We included and analysed 140 studies for this review. In the main analyses, a total of 108 comparisons from 70 studies compared any intervention in which audit and feedback was a core, essential component to usual care and evaluated effects on professional practice. After excluding studies at high risk of bias, there were 82 comparisons from 49 studies featuring dichotomous outcomes, and the weighted median adjusted RD was a 4.3% (interquartile range (IQR) 0.5% to 16%) absolute increase in healthcare professionals' compliance with desired practice. Across 26 comparisons from 21 studies with continuous outcomes, the weighted median adjusted percent change relative to control was 1.3% (IQR = 1.3% to 28.9%). For patient outcomes, the weighted median RD was -0.4% (IQR -1.3% to 1.6%) for 12 comparisons from six studies reporting dichotomous outcomes and the weighted median percentage change was 17% (IQR 1.5% to 17%) for eight comparisons from five studies reporting continuous outcomes. Multivariable meta-regression indicated that feedback may be more effective when baseline performance is low, the source is a supervisor or colleague, it is provided more than once, it is delivered in both verbal and written formats, and when it includes both explicit targets and an action plan. In addition, the effect size varied based on the clinical behaviour targeted by the intervention.

AUTHORS' CONCLUSIONS

Audit and feedback generally leads to small but potentially important improvements in professional practice. The effectiveness of audit and feedback seems to depend on baseline performance and how the feedback is provided. Future studies of audit and feedback should directly compare different ways of providing feedback.