Randomised comparison of a simple warfarin dosing algorithm versus a computerised anticoagulation management system for control of warfarin maintenance therapy

Optimizing warfarin dosing for patients with atrial fibrillation using machine learning

While novel oral anticoagulants are increasingly used to reduce risk of stroke in patients with atrial fibrillation, vitamin K antagonists such as warfarin continue to be used extensively for stroke prevention across the world. While effective in reducing the risk of strokes, the complex pharmacodynamics of warfarin make it difficult to use clinically, with many patients experiencing under- and/or over- anticoagulation. In this study we employed a novel implementation of deep reinforcement learning to provide clinical decision support to optimize time in therapeutic International Normalized Ratio (INR) range. We used a novel semi-Markov decision process formulation of the Batch-Constrained deep Q-learning algorithm to develop a reinforcement learning model to dynamically recommend optimal warfarin dosing to achieve INR of 2.0-3.0 for patients with atrial fibrillation. The model was developed using data from 22,502 patients in the warfarin treated groups of the pivotal randomized clinical trials of edoxaban (ENGAGE AF-TIMI 48), apixaban (ARISTOTLE) and rivaroxaban (ROCKET AF). The model was externally validated on data from 5730 warfarin-treated patients in a fourth trial of dabigatran (RE-LY) using multilevel regression models to estimate the relationship between center-level algorithm consistent dosing, time in therapeutic INR range (TTR), and a composite clinical outcome of stroke, systemic embolism or major hemorrhage. External validation showed a positive association between center-level algorithm-consistent dosing and TTR (R2 = 0.56). Each 10% increase in algorithm-consistent dosing at the center level independently predicted a 6.78% improvement in TTR (95% CI 6.29, 7.28; p < 0.001) and a 11% decrease in the composite clinical outcome (HR 0.89; 95% CI 0.81, 1.00; p = 0.015). These results were comparable to those of a rules-based clinical algorithm used for benchmarking, for which each 10% increase in algorithm-consistent dosing independently predicted a 6.10% increase in TTR (95% CI 5.67, 6.54, p < 0.001) and a 10% decrease in the composite outcome (HR 0.90; 95% CI 0.83, 0.98, p = 0.018). Our findings suggest that a deep reinforcement learning algorithm can optimize time in therapeutic range for patients taking warfarin. A digital clinical decision support system to promote algorithm-consistent warfarin dosing could optimize time in therapeutic range and improve clinical outcomes in atrial fibrillation globally.

Telemedicine-based management of oral anticoagulation therapy: a systematic review and meta-analysis (Preprint)

BACKGROUND

Oral anticoagulation is the cornerstone treatment of several diseases. Its management is often challenging, and different telemedicine strategies have been implemented to support it.

OBJECTIVE

The aim of the study is to systematically review the evidence on the impact of telemedicine-based oral anticoagulation management compared to usual care on thromboembolic and bleeding events.

METHODS

Randomized controlled trials were searched in 5 databases from inception to September 2021. Two independent reviewers performed study selection and data extraction. Total thromboembolic events, major bleeding, mortality, and time in therapeutic range were assessed. Results were pooled using random effect models.

RESULTS

In total, 25 randomized controlled trials were included (n=25,746 patients) and classified as moderate to high risk of bias by the Cochrane tool. Telemedicine resulted in lower rates of thromboembolic events, though not statistically significant (n=13 studies, relative risk [RR] 0.75, 95% CI 0.53-1.07; I²=42%), comparable rates of major bleeding (n=11 studies, RR 0.94, 95% CI 0.82-1.07; I²=0%) and mortality (n=12 studies, RR 0.96, 95% CI 0.78-1.20; I²=11%), and an improved time in therapeutic range (n=16 studies, mean difference 3.38, 95% CI 1.12-5.65; I²=90%). In the subgroup of the multitasking intervention, telemedicine resulted in an important reduction of thromboembolic events (RR 0.20, 95% CI 0.08-0.48).

CONCLUSIONS

Telemedicine-based oral anticoagulation management resulted in similar rates of major bleeding and mortality, a trend for fewer thromboembolic events, and better anticoagulation quality compared to standard care. Given the potential benefits of telemedicine-based care, such as greater access to remote populations or people with ambulatory restrictions, these findings may encourage further implementation of eHealth strategies for anticoagulation management, particularly as part of multifaceted interventions for integrated care of chronic diseases. Meanwhile, researchers should develop higher-quality evidence focusing on hard clinical outcomes, cost-effectiveness, and quality of life.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42020159208; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=159208.

[Improved patient safety through a clinical decision support system in laboratory medicine]

BACKGROUND

Laboratory diagnostics are essential for diagnosis, initiation of therapy, and monitoring of patients. Laboratory results that are overlooked or incorrectly interpreted lead to adverse events and endanger patient safety. Clinical decision support systems (CDSSs) may facilitate appropriate interpretation of results and subsequent medical response.

OBJECTIVES

The research project on digital laboratory medicine (AMPEL) aims at developing a CDSS based on laboratory diagnostics, which supports practitioners in ensuring the necessary medical consequences.

MATERIALS AND METHODS

A literature review of CDSSs describes the current state of research. The research project AMPEL is presented with its objectives, challenges, and first results. Furthermore, the development of a framework and reporting system is illustrated through the clinical example of severe hypokalemia.

RESULTS AND CONCLUSION

Through interdisciplinary development and constant optimization, a specific CDSS with high acceptance among clinicians was developed. Initial results in the case of severe hypokalemia show a positive effect on patient care. Thereby, more complex frameworks such as sepsis diagnostics or acute coronary syndrome are implemented. The limited availability of standardized and digital clinical data is challenging. In addition to the application of classic decision trees in CDSS, the use of machine learning offers a promising perspective for future developments.

Warfarin Dosing Algorithms Underpredict Dose Requirements in Patients Requiring ≥7 mg Daily: A Systematic Review and Meta-analysis

There is preliminary evidence to suggest that some published warfarin dosing algorithms produce biased maintenance dose predictions in patients who require higher than average doses. We conducted a meta-analysis of warfarin dosing algorithms to determine if there exists a systematic under- or overprediction of dose requirements for patients requiring ≥7 mg/day across published algorithms. Medline and Embase databases were searched up to September 2015. We quantified the proportion of over- and underpredicted doses in patients whose observed maintenance dose was ≥7 mg/day. The meta-analysis included 47 evaluations of 22 different warfarin dosing algorithms from 16 studies. The meta-analysis included data from 1,492 patients who required warfarin doses of ≥7 mg/day. All 22 algorithms were found to underpredict warfarin dosing requirements in patients who required ≥7 mg/day by an average of 2.3 mg/day with a pooled estimate of underpredicted doses of 92.3% (95% confidence interval 90.3-94.1, I² = 24%).

The Effects of Computerized Clinical Decision Support Systems on Laboratory Test Ordering: A Systematic Review

CONTEXT

- Inappropriate laboratory test ordering has been shown to be as high as 30%. This can have an important impact on quality of care and costs because of downstream consequences such as additional diagnostics, repeat testing, imaging, prescriptions, surgeries, or hospital stays.

OBJECTIVE

- To evaluate the effect of computerized clinical decision support systems on appropriateness of laboratory test ordering.

DATA SOURCES

- We used MEDLINE, Embase, CINAHL, MEDLINE In-Process and Other Non-Indexed Citations, Clinicaltrials.gov, Cochrane Library, and Inspec through December 2015. Investigators independently screened articles to identify randomized trials that assessed a computerized clinical decision support system aimed at improving laboratory test ordering by providing patient-specific information, delivered in the form of an on-screen management option, reminder, or suggestion through a computerized physician order entry using a rule-based or algorithm-based system relying on an evidence-based knowledge resource. Investigators extracted data from 30 papers about study design, various study characteristics, study setting, various intervention characteristics, involvement of the software developers in the evaluation of the computerized clinical decision support system, outcome types, and various outcome characteristics.

CONCLUSIONS

- Because of heterogeneity of systems and settings, pooled estimates of effect could not be made. Data showed that computerized clinical decision support systems had little or no effect on clinical outcomes but some effect on compliance. Computerized clinical decision support systems targeted at laboratory test ordering for multiple conditions appear to be more effective than those targeted at a single condition.

Guidance for the practical management of warfarin therapy in the treatment of venous thromboembolism

Venous thromboembolism (VTE) is a serious and often fatal medical condition with an increasing incidence. The treatment of VTE is undergoing tremendous changes with the introduction of the new direct oral anticoagulants and clinicians need to understand new treatment paradigms. This article, initiated by the Anticoagulation Forum, provides clinical guidance based on existing guidelines and consensus expert opinion where guidelines are lacking. Well-managed warfarin therapy remains an important anticoagulant option and it is hoped that anticoagulation providers will find the guidance contained in this article increases their ability to achieve optimal outcomes for their patients with VTE Pivotal practical questions pertaining to this topic were developed by consensus of the authors and were derived from evidence-based consensus statements whenever possible. The medical literature was reviewed and summarized using guidance statements that reflect the consensus opinion(s) of all authors and the endorsement of the Anticoagulation Forum’s Board of Directors. In an effort to provide practical and implementable information about VTE and its treatment, guidance statements pertaining to choosing good candidates for warfarin therapy, warfarin initiation, optimizing warfarin control, invasive procedure management, excessive anticoagulation, subtherapeutic anticoagulation, drug interactions, switching between anticoagulants, and care transitions are provided.

Effects of anticoagulation provider continuity on time in therapeutic range for warfarin patients

Anticoagulation (AC) clinics use the percentage of time in the therapeutic INR range (%TTR) to characterize the quality of management for patients treated with warfarin. In order to guide policy and procedure changes, the purpose of this quality improvement (QI) study was to characterize the AC patient population at The Johns Hopkins Hospital (JHH). We set out to investigate the impact of AC clinic provider continuity on the quality of anticoagulation management. This QI study is a retrospective chart review of 525 warfarin patients managed by pharmacists in the Hematology AC Management Clinic at JHH from June 28, 2013 to November 1, 2014. We recorded patient demographic and clinical characteristics and the quality of AC management using %TTR, and compared these parameters between patients with (Group A) and without a primary AC (Group B). Group A patients had a significantly higher %TTR than Group B patients (53.2 vs. 46.5 %, p = 0.008). In conclusion, we found that patients with a primary AC clinic provider had a higher %TTR than patients with multiple providers. If confirmed prospectively, this approach to warfarin management could represent one technique for AC clinics to optimize patient management and clinical outcomes.

Optimization of vitamin K antagonist drug dose finding by replacement of the international normalized ratio by a bidirectional factor: validation of a new algorithm

ESSENTIALS: We developed a new algorithm to optimize vitamin K antagonist dose finding. Validation was by comparing actual dosing to algorithm predictions. Predicted and actual dosing of well performing centers were highly associated. The method is promising and should be tested in a randomized trial.

SUMMARY

BACKGROUND

Oral vitamin K antagonists (VKAs) have a narrow therapeutic window and thus require frequent monitoring of its intensity by the international normalized ratio (INR). Improvement of VKA dosing defined as more time in therapeutic range (TTR) can reduce thrombotic disease and bleeding. Computerized decision support programs (CDSs) are used to optimize VKA dosing, but the effects are heterogeneous. CDSs significantly improve the proportion of time in the therapeutic INR range for initiation therapy but not the quality of anticoagulant management in an outpatient setting. One of the major problems of VKA dose finding is that the INR is a ratio and does not present linearity. We developed a new dose-finding algorithm, based on a novel bidirectional factor (BF). This BF is linear transformation of the nonlinear INR.

METHODS

We compared the outcomes of the new algorithm, called BF-N, with dose finding performed at three highly ranked Dutch anticoagulation centers, using both acenocoumarol and phenprocoumon.

RESULTS

The outcomes of the BF-N algorithm showed a linear correlation with VKA doses of the three centers (y = 1.001x, r(2) 0.999 for acenocoumarol and y = 0.999x, r(2) 0.999 for phenprocoumon), with a standard deviation of 3.83%. The rate of automated dosage proposals increased to 100%.

CONCLUSION

The BF-N algorithm performs well in real-life settings and increases the rate of automated dosage proposals. The algorithm can be easily built into existing CDSs. Experienced staff remains necessary for complicated situations. The new algorithm needs to be evaluated in a prospective trial.

Features of electronic health records necessary for the delivery of optimized anticoagulant therapy: consensus of the EHR Task Force of the New York State Anticoagulation Coalition

BACKGROUND

Oral anticoagulants are prescribed to millions of Americans, and consequently are among the medications most likely to contribute to emergency department visits and hospitalizations. Although guidelines and consensus statements promote systematic approaches to therapy, anticoagulation (AC) management is often suboptimal. Electronic health records (EHRs) have the potential to improve safety and quality but have not yet incorporated specialized features necessary to optimize therapy.

OBJECTIVE

To generate a comprehensive, consensus-based list of EHR features clinically necessary to deliver optimized AC management, provide a "language bridge" to accelerate incorporation of features into EHR systems, and suggest mechanisms for the objective evaluation of available EHRs.

METHODS

A multidisciplinary panel of AC specialists utilized the framework of a previously published consensus statement to map outpatient AC management and developed a comprehensive array of sequential computer logic steps using a restricted language scheme. Logic steps were then translated into narrative descriptions of potential EHR features, which were refined through multiple group evaluations. A finalized list of proposed features was ranked according to perceived clinical necessity by physician, pharmacist, and nurse panelists in a blinded manner using a 5-point Likert scale. Features receiving no more than 1 dissenting opinion were included in a finalized list of clinically necessary features.

RESULTS

The task force generated 78 recommended EHR features across 20 key discrete areas and 425 individual logic steps. All recommended features received Strongly Agree or Agree rankings regarding their perceived clinical necessity, and no feature received more than a single Disagree response.

CONCLUSION

The incorporation of key AC-related features into existing EHRs or specialized AC management systems has the potential to systematize the delivery of optimal AC care by health care professionals at the point of care. Optimized AC management has the potential to reduce adverse drug events associated with anticoagulant therapy in the outpatient setting.

Computerized clinical decision support improves warfarin management and decreases recurrent venous thromboembolism

BACKGROUND

An explicit approach to warfarin dose adjustment using computerized clinical decision support (CDS) improves warfarin management. We report metrics of quality for warfarin management before and after implementation of CDS in a large health care system.

METHODS

A total of 2591 chronically anticoagulated patients were eligible for inclusion. We compared interpatient time in therapeutic range (TTR) and international normalized ratio (INR) variability before and after implementation of CDS. We report outcomes of major bleeding, thrombosis, and health care utilization.

RESULTS

Implementation of CDS significantly improved TTR (from 63.99% to 65.13%; P = .04) and reduced out-of-range INRs (from 42.39% to 39.97%; P < .001). Venous thromboembolism (relative risk [RR] 0.41; P < .001) emergency department utilization (RR 0.62; P < .001), and hospitalization (RR 0.62; P < .001) were reduced after CDS implementation. Major hemorrhage was more frequent after CDS implementation (RR 1.42; P = .01).

CONCLUSION

The CDS warfarin management was associated with improved TTR and decreased INR variability in a large cohort of chronically anticoagulated patients. Clinically relevant outcomes were broadly improved, although more bleeding events were observed.

An audit of anticoagulant management to assess anticoagulant control using decision support software

OBJECTIVE

To evaluate the effectiveness of a computerised self-adjusting anticoagulant algorithm to predict appropriate warfarin dosing and to assess its use in clinical practice.

DESIGN

A 3-year audit of anticoagulant control in patients managed by doctors and pharmacists using computer decision support and an evaluation of the impact of dose adjustments made by the users.

PARTICIPANTS

3660 patients on oral anticoagulants; one-third of patients managed by doctors and two-thirds by pharmacists.

SETTING

Anticoagulant supervision in primary care and pharmacies at 60 sites in New Zealand.

MAIN OUTCOME MEASURES

The time in the therapeutic range (TTR), the outcome of adherence to the computer dosing algorithm, the percentage of time the clinicians over-ride the algorithm and the impact of their intervention on anticoagulant control.

RESULTS

A TTR of 72.9% was achieved for all patients. The TTR was significantly better in patients managed by pharmacists than doctors (75.1% versus 67.4%, p<0.0001). The computer algorithm provides appropriate dose recommendations for INR results from 1.5 to 4. Users administered a dose that differed from the computer recommendation 23.3% of the time. The doctors adjusted the dose more frequently (28.2% versus 21.1% of tests) and made larger dose changes than the pharmacists.

CONCLUSIONS

The clinicians predominantly change the dose when the INR is below the therapeutic range. The changes are not necessary to correct for inaccuracies in the algorithm. The most likely explanation is the clinician's belief that their own dose adjustment would achieve better control; however, in practice, their changes tend to underdose patients. The doctors achieved poorer control than the pharmacists; this is in part due to the action of the doctors over-riding the algorithm. Our results imply that clinicians could achieve better anticoagulant control if they more closely followed the computer algorithm.

Improvement of anticoagulant treatment using a dynamic decision support algorithm: a Danish Cohort study

INTRODUCTION

Warfarin is the most widely prescribed vitamin K antagonist and in the United States and Europe more than 10 million people are currently in long-term oral anticoagulant treatment. This study aims to retrospectively validate a dynamic statistical model providing dosage suggestions to patients in warfarin treatment.

MATERIALS AND METHODS

The model was validated on a cohort of 553 patients with a mean TTR of 83%. Patients in the cohort were self-monitoring and managed by a highly specialised anticoagulation clinic. The predictive model essentially consists of three parts handling INR history, warfarin dosage and biological noise, which allows for prediction of future INR values and optimal warfarin dose to stay on INR target. Further, the model is based on parameters initially being set to population values and gradually individualised during monitoring of patients.

PRIMARY OUTCOME

Time in therapeutic range was used as surrogate quality measure of the treatment, and model-suggested dosage of warfarin was used to assess the accuracy of the model performance.

RESULTS

The accuracy of the model predictions measured as median absolute error was 0.53 mg/day (interquartile range from 0.25 to 1.0). The model performance was evaluated by the difference between observed and predicted warfarin intake in the preceding week of an INR measurement. In more than 70% of the cases where INR measurements were outside the therapeutic range, the model suggested a more reasonable dose than the observed intake.

CONCLUSION

Applying the proposed dosing algorithm can potentially further increase the time in INR target range beyond 83%.

Computerized advice on drug dosage to improve prescribing practice

BACKGROUND

Maintaining therapeutic concentrations of drugs with a narrow therapeutic window is a complex task. Several computer systems have been designed to help doctors determine optimum drug dosage. Significant improvements in health care could be achieved if computer advice improved health outcomes and could be implemented in routine practice in a cost-effective fashion. This is an updated version of an earlier Cochrane systematic review, first published in 2001 and updated in 2008.

OBJECTIVES

To assess whether computerized advice on drug dosage has beneficial effects on patient outcomes compared with routine care (empiric dosing without computer assistance).

SEARCH METHODS

The following databases were searched from 1996 to January 2012: EPOC Group Specialized Register, Reference Manager; Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Ovid; EMBASE, Ovid; and CINAHL, EbscoHost. A "top up" search was conducted for the period January 2012 to January 2013; these results were screened by the authors and potentially relevant studies are listed in Studies Awaiting Classification. The review authors also searched reference lists of relevant studies and related reviews.

SELECTION CRITERIA

We included randomized controlled trials, non-randomized controlled trials, controlled before-and-after studies and interrupted time series analyses of computerized advice on drug dosage. The participants were healthcare professionals responsible for patient care. The outcomes were any objectively measured change in the health of patients resulting from computerized advice (such as therapeutic drug control, clinical improvement, adverse reactions).

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data and assessed study quality. We grouped the results from the included studies by drug used and the effect aimed at for aminoglycoside antibiotics, amitriptyline, anaesthetics, insulin, anticoagulants, ovarian stimulation, anti-rejection drugs and theophylline. We combined the effect sizes to give an overall effect for each subgroup of studies, using a random-effects model. We further grouped studies by type of outcome when appropriate (i.e. no evidence of heterogeneity).

MAIN RESULTS

Forty-six comparisons (from 42 trials) were included (as compared with 26 comparisons in the last update) including a wide range of drugs in inpatient and outpatient settings. All were randomized controlled trials except two studies. Interventions usually targeted doctors, although some studies attempted to influence prescriptions by pharmacists and nurses. Drugs evaluated were anticoagulants, insulin, aminoglycoside antibiotics, theophylline, anti-rejection drugs, anaesthetic agents, antidepressants and gonadotropins. Although all studies used reliable outcome measures, their quality was generally low.This update found similar results to the previous update and managed to identify specific therapeutic areas where the computerized advice on drug dosage was beneficial compared with routine care:1. it increased target peak serum concentrations (standardized mean difference (SMD) 0.79, 95% CI 0.46 to 1.13) and the proportion of people with plasma drug concentrations within the therapeutic range after two days (pooled risk ratio (RR) 4.44, 95% CI 1.94 to 10.13) for aminoglycoside antibiotics;2. it led to a physiological parameter more often within the desired range for oral anticoagulants (SMD for percentage of time spent in target international normalized ratio +0.19, 95% CI 0.06 to 0.33) and insulin (SMD for percentage of time in target glucose range: +1.27, 95% CI 0.56 to 1.98);3. it decreased the time to achieve stabilization for oral anticoagulants (SMD -0.56, 95% CI -1.07 to -0.04);4. it decreased the thromboembolism events (rate ratio 0.68, 95% CI 0.49 to 0.94) and tended to decrease bleeding events for anticoagulants although the difference was not significant (rate ratio 0.81, 95% CI 0.60 to 1.08). It tended to decrease unwanted effects for aminoglycoside antibiotics (nephrotoxicity: RR 0.67, 95% CI 0.42 to 1.06) and anti-rejection drugs (cytomegalovirus infections: RR 0.90, 95% CI 0.58 to 1.40);5. it tended to reduce the length of time spent in the hospital although the difference was not significant (SMD -0.15, 95% CI -0.33 to 0.02) and to achieve comparable or better cost-effectiveness ratios than usual care;6. there was no evidence of differences in mortality or other clinical adverse events for insulin (hypoglycaemia), anaesthetic agents, anti-rejection drugs and antidepressants.For all outcomes, statistical heterogeneity quantified by I(2) statistics was moderate to high.

AUTHORS' CONCLUSIONS

This review update suggests that computerized advice for drug dosage has some benefits: it increases the serum concentrations for aminoglycoside antibiotics and improves the proportion of people for which the plasma drug is within the therapeutic range for aminoglycoside antibiotics.It leads to a physiological parameter more often within the desired range for oral anticoagulants and insulin. It decreases the time to achieve stabilization for oral anticoagulants. It tends to decrease unwanted effects for aminoglycoside antibiotics and anti-rejection drugs, and it significantly decreases thromboembolism events for anticoagulants. It tends to reduce the length of hospital stay compared with routine care while comparable or better cost-effectiveness ratios were achieved.However, there was no evidence that decision support had an effect on mortality or other clinical adverse events for insulin (hypoglycaemia), anaesthetic agents, anti-rejection drugs and antidepressants. In addition, there was no evidence to suggest that some decision support technical features (such as its integration into a computer physician order entry system) or aspects of organization of care (such as the setting) could optimize the effect of computerized advice.Taking into account the high risk of bias of, and high heterogeneity between, studies, these results must be interpreted with caution.

The effects of on-screen, point of care computer reminders on processes and outcomes of care

BACKGROUND

The opportunity to improve care by delivering decision support to clinicians at the point of care represents one of the main incentives for implementing sophisticated clinical information systems. Previous reviews of computer reminder and decision support systems have reported mixed effects, possibly because they did not distinguish point of care computer reminders from e-mail alerts, computer-generated paper reminders, and other modes of delivering 'computer reminders'.

OBJECTIVES

To evaluate the effects on processes and outcomes of care attributable to on-screen computer reminders delivered to clinicians at the point of care.

SEARCH STRATEGY

We searched the Cochrane EPOC Group Trials register, MEDLINE, EMBASE and CINAHL and CENTRAL to July 2008, and scanned bibliographies from key articles.

SELECTION CRITERIA

Studies of a reminder delivered via a computer system routinely used by clinicians, with a randomised or quasi-randomised design and reporting at least one outcome involving a clinical endpoint or adherence to a recommended process of care.

DATA COLLECTION AND ANALYSIS

Two authors independently screened studies for eligibility and abstracted data. For each study, we calculated the median improvement in adherence to target processes of care and also identified the outcome with the largest such improvement. We then calculated the median absolute improvement in process adherence across all studies using both the median outcome from each study and the best outcome.

MAIN RESULTS

Twenty-eight studies (reporting a total of thirty-two comparisons) were included. Computer reminders achieved a median improvement in process adherence of 4.2% (interquartile range (IQR): 0.8% to 18.8%) across all reported process outcomes, 3.3% (IQR: 0.5% to 10.6%) for medication ordering, 3.8% (IQR: 0.5% to 6.6%) for vaccinations, and 3.8% (IQR: 0.4% to 16.3%) for test ordering. In a sensitivity analysis using the best outcome from each study, the median improvement was 5.6% (IQR: 2.0% to 19.2%) across all process measures and 6.2% (IQR: 3.0% to 28.0%) across measures of medication ordering. In the eight comparisons that reported dichotomous clinical endpoints, intervention patients experienced a median absolute improvement of 2.5% (IQR: 1.3% to 4.2%). Blood pressure was the most commonly reported clinical endpoint, with intervention patients experiencing a median reduction in their systolic blood pressure of 1.0 mmHg (IQR: 2.3 mmHg reduction to 2.0 mmHg increase).

AUTHORS' CONCLUSIONS

Point of care computer reminders generally achieve small to modest improvements in provider behaviour. A minority of interventions showed larger effects, but no specific reminder or contextual features were significantly associated with effect magnitude. Further research must identify design features and contextual factors consistently associated with larger improvements in provider behaviour if computer reminders are to succeed on more than a trial and error basis.