Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care

Electronic Health Record Nudges and Health Care Quality and Outcomes in Primary Care: A Systematic Review

Importance

Nudges have been increasingly studied as a tool for facilitating behavior change and may represent a novel way to modify the electronic health record (EHR) to encourage evidence-based care.

Objective

To evaluate the association between EHR nudges and health care outcomes in primary care settings and describe implementation facilitators and barriers.

Evidence Review

On June 9, 2023, an electronic search was performed in PubMed, Embase, PsycINFO, CINAHL, and Web of Science for all articles about clinician-facing EHR nudges. After reviewing titles, abstracts, and full texts, the present review was restricted to articles that used a randomized clinical trial (RCT) design, focused on primary care settings, and evaluated the association between EHR nudges and health care quality and patient outcome measures. Two reviewers abstracted the following elements: country, targeted clinician types, medical conditions studied, length of evaluation period, study design, sample size, intervention conditions, nudge mechanisms, implementation facilitators and barriers encountered, and major findings. The findings were qualitatively reported by type of health care quality and patient outcome and type of primary care condition targeted. The Risk of Bias 2.0 tool was adapted to evaluate the studies based on RCT design (cluster, parallel, crossover). Studies were scored from 0 to 5 points, with higher scores indicating lower risk of bias.

Findings

Fifty-four studies met the inclusion criteria. Overall, most studies (79.6%) were assessed to have a moderate risk of bias. Most or all descriptive (eg, documentation patterns) (30 of 38) or patient-centeredness measures (4 of 4) had positive associations with EHR nudges. As for other measures of health care quality and patient outcomes, few had positive associations between EHR nudges and patient safety (4 of 12), effectiveness (19 of 48), efficiency (0 of 4), patient-reported outcomes (0 of 3), patient adherence (1 of 2), or clinical outcome measures (1 of 7).

Conclusions and Relevance

This systematic review found low- and moderate-quality evidence that suggested that EHR nudges were associated with improved descriptive measures (eg, documentation patterns). Meanwhile, it was unclear whether EHR nudges were associated with improvements in other areas of health care quality, such as effectiveness and patient safety outcomes. Future research is needed using longer evaluation periods, a broader range of primary care conditions, and in deimplementation contexts.

Randomized Controlled Comparative Effectiveness Trial of Risk Model-Guided Clinical Decision Support for Suicide Screening

Suicide prevention requires risk identification, appropriate intervention, and follow-up. Traditional risk identification relies on patient self-reporting, support network reporting, or face-to-face screening with validated instruments or history and physical exam. In the last decade, statistical risk models have been studied and more recently deployed to augment clinical judgment. Models have generally been found to be low precision or problematic at scale due to low incidence. Few have been tested in clinical practice, and none have been tested in clinical trials to our knowledge.

Methods

We report the results of a pragmatic randomized controlled trial (RCT) in three outpatient adult Neurology clinic settings. This two-arm trial compared the effectiveness of Interruptive and Non-Interruptive Clinical Decision Support (CDS) to prompt further screening of suicidal ideation for those predicted to be high risk using a real-time, validated statistical risk model of suicide attempt risk, with the decision to screen as the primary end point. Secondary outcomes included rates of suicidal ideation and attempts in both arms. Manual chart review of every trial encounter was used to determine if suicide risk assessment was subsequently documented.

Results

From August 16, 2022, through February 16, 2023, our study randomized 596 patient encounters across 561 patients for providers to receive either Interruptive or Non-Interruptive CDS in a 1:1 ratio. Adjusting for provider cluster effects, Interruptive CDS led to significantly higher numbers of decisions to screen (42%=121/289 encounters) compared to Non-Interruptive CDS (4%=12/307) (odds ratio=17.7, p-value <0.001). Secondarily, no documented episodes of suicidal ideation or attempts occurred in either arm. While the proportion of documented assessments among those noting the decision to screen was higher for providers in the Non-Interruptive arm (92%=11/12) than in the Interruptive arm (52%=63/121), the interruptive CDS was associated with more frequent documentation of suicide risk assessment (63/289 encounters compared to 11/307, p-value<0.001).

Conclusions

In this pragmatic RCT of real-time predictive CDS to guide suicide risk assessment, Interruptive CDS led to higher numbers of decisions to screen and documented suicide risk assessments. Well-powered large-scale trials randomizing this type of CDS compared to standard of care are indicated to measure effectiveness in reducing suicidal self-harm. ClinicalTrials.gov Identifier: NCT05312437.

Interventions to improve appropriateness of laboratory testing in the intensive care unit: a narrative review

Healthcare expenses are increasing, as is the utilization of laboratory resources. Despite this, between 20% and 40% of requested tests are deemed inappropriate. Improper use of laboratory resources leads to unwanted consequences such as hospital-acquired anemia, infections, increased costs, staff workload and patient stress and discomfort. The most unfavorable consequences result from unnecessary follow-up tests and treatments (overuse) and missed or delayed diagnoses (underuse). In this context, several interventions have been carried out to improve the appropriateness of laboratory testing. To date, there have been few published assessments of interventions specific to the intensive care unit. We reviewed the literature for interventions implemented in the ICU to improve the appropriateness of laboratory testing. We searched literature from 2008 to 2023 in PubMed, Embase, Scopus, and Google Scholar databases between April and June 2023. Five intervention categories were identified: education and guidance (E&G), audit and feedback, gatekeeping, computerized physician order entry (including reshaping of ordering panels), and multifaceted interventions (MFI). We included a sixth category exploring the potential role of artificial intelligence and machine learning (AI/ML)-based assisting tools in such interventions. E&G-based interventions and MFI are the most frequently used approaches. MFI is the most effective type of intervention, and shows the strongest persistence of effect over time. AI/ML-based tools may offer valuable assistance to the improvement of appropriate laboratory testing in the near future. Patient safety outcomes are not impaired by interventions to reduce inappropriate testing. The literature focuses mainly on reducing overuse of laboratory tests, with only one intervention mentioning underuse. We highlight an overall poor quality of methodological design and reporting and argue for standardization of intervention methods. Collaboration between clinicians and laboratory staff is key to improve appropriate laboratory utilization. This article offers practical guidance for optimizing the effectiveness of an intervention protocol designed to limit inappropriate use of laboratory resources.

Monitoring for adverse drug events of high-risk medications with a computerized clinical decision support system: a prospective cohort study

Monitoring is recommended to prevent severe adverse drug events, but such examinations are often missed. To increase the number of monitoring that should be ordered for high-risk medications, we introduced a clinical decision support system (CDSS) that alerts and orders the monitoring for high-risk medications in an outpatient setting. We conducted a 2-year prospective cohort study at a tertiary care teaching hospital before (phase 1) and after (phase 2) the activation of a CDSS. The CDSS automatically provided alerts for liver function tests for vildagliptin, thyroid function tests for immune checkpoint inhibitors (ICIs) and multikinase inhibitors (MKIs), and a slit-lamp examination of the eyes for oral amiodarone when outpatients were prescribed the medications but not examined for a fixed period. The order of laboratory tests automatically appeared if alert was accepted. The alerts were hidden and did not appear on the display before activation of the CDSS. The outcomes were the number of prescriptions with alerts and examinations. During the study period, 330 patients in phase 1 and 307 patients in phase 2 were prescribed vildagliptin, 20 patients in phase 1 and 19 patients in phase 2 were prescribed ICIs or MKIs, and 72 patients in phase 1 and 66 patients in phase 2 were prescribed oral amiodarone. The baseline characteristics were similar between the phases. In patients prescribed vildagliptin, the proportion of alerts decreased significantly (38% vs 27%, P < 0.0001), and the proportion of examinations increased significantly (0.9% vs 4.0%, P < 0.0001) after activation of the CDSS. In patients prescribed ICIs or MKIs, the proportion of alerts decreased significantly (43% vs 11%, P < 0.0001), and the proportion of examinations increased numerically, but not significantly (2.6% vs 7.0%, P = 0.13). In patients prescribed oral amiodarone, the proportion of alerts decreased (86% vs 81%, P = 0.055), and the proportion of examinations increased (2.2% and 3.0%, P = 0.47); neither was significant. The CDSS has potential to increase the monitoring for high-risk medications. Our study also highlighted the limited acceptance rate of monitoring by CDSS. Further studies are needed to explore the generalizability to other medications and the cause of the limited acceptance rates among physicians.

Meaningful time-related aspects of alerts in Clinical Decision Support Systems. A unified framework

Alerts are a common functionality of clinical decision support systems (CDSSs). Although they have proven to be useful in clinical practice, the alert burden can lead to alert fatigue and significantly reduce their usability and acceptance. Based on a literature review, we propose a unified framework consisting of a set of meaningful timestamps that allows the use of state-of-the-art measures for alert burden, such as alert dwell time, alert think time, and response time. In addition, it can be used to investigate other measures that could be relevant as regards dealing with this problem. Furthermore, we provide a case study concerning three different types of alerts to which the framework was successfully applied. We consider that our framework can easily be adapted to other CDSSs and that it could be useful for dealing with alert burden measurement thus contributing to its appropriate management.

Using the Electronic Medical Record to Increase Laboratory Test Monitoring in Ocular Inflammation Patients: A Quality Improvement Study

BACKGROUND AND OBJECTIVES

Treatment of chronic, noninfectious ocular inflammation includes corticosteroids, disease-modifying antirheumatic medications, and biologics. To mitigate adverse effects associated with the use of these medications, routine laboratory test monitoring is recommended throughout treatment. We evaluated the effectiveness of an alert added to the electronic medical record (EMR) to aid in laboratory test monitoring for patients prescribed these high-risk medications.

METHODS

A prospective, interventional study assessed the effect of the alert within the EMR on laboratory test ordering at the Division of Ocular Immunology at the Wilmer Eye Institute. The primary outcome measure was the change in number of ordered laboratory tests at 3, 6, and 12 months after the alert activation compared with pre-intervention levels and overall through the study period. The laboratory tests that were monitored included complete blood count, comprehensive metabolic panel, dual-energy x-ray absorptiometry (DXA) scanning, fasting lipid panel, and interferon gamma release assays.

RESULTS

The laboratory test orders for 153 patients on high risk medications were analyzed. Only the frequency of ordering the DXA and interferon gamma release assays increased significantly, compared with baseline, throughout the study. Conversely, there was a significant decrease in the frequency of ordering of fasting lipid profiles and hemoglobin A 1c at each time point and for complete blood count and comprehensive metabolic panel at the 6-month time point.

CONCLUSION

An EMR alert results in increased laboratory test ordering initially for tests drawn on a yearly basis, but the effect on more frequently ordered tests wanes with time if the alert can be silenced by the provider. Nonetheless, it provides a novel mechanism to increase laboratory ordering in patients on high-risk medications that can be adapted for use in other EMR software. Future studies are needed to assess whether physician laboratory test ordering behavior is altered throughout the study period with the use of a non-silencable alert.

Lessons in machine learning model deployment learned from sepsis

In three recent and related publications, researchers from Johns Hopkins University and Bayesian Health report results from implementing and prospectively evaluating the Targeted Real-time Early Warning System (TREWS) for sepsis at five hospitals.^1-3.

Impact of Kidney Failure Risk Prediction Clinical Decision Support on Monitoring and Referral in Primary Care Management of CKD: A Randomized Pragmatic Clinical Trial

Rationale & Objective

To design and implement clinical decision support incorporating a validated risk prediction estimate of kidney failure in primary care clinics and to evaluate the impact on stage-appropriate monitoring and referral.

Study Design

Block-randomized, pragmatic clinical trial.

Setting & Participants

Ten primary care clinics in the greater Boston area. Patients with stage 3-5 chronic kidney disease (CKD) were included. Patients were randomized within each primary care physician panel through a block randomization approach. The trial occurred between December 4, 2015, and December 3, 2016.

Intervention

Point-of-care noninterruptive clinical decision support that delivered the 5-year kidney failure risk equation as well as recommendations for stage-appropriate monitoring and referral to nephrology.

Outcomes

The primary outcome was as follows: Urine and serum laboratory monitoring test findings measured at one timepoint 6 months after the initial primary care visit and analyzed only in patients who had not undergone the recommended monitoring test in the preceding 12 months. The secondary outcome was nephrology referral in patients with a calculated kidney failure risk equation value of >10% measured at one timepoint 6 months after the initial primary care visit.

Results

The clinical decision support application requested and processed 569,533 Continuity of Care Documents during the study period. Of these, 41,842 (7.3%) documents led to a diagnosis of stage 3, 4, or 5 CKD by the clinical decision support application. A total of 5,590 patients with stage 3, 4, or 5 CKD were randomized and included in the study. The link to the clinical decision support application was clicked 122 times by 57 primary care physicians. There was no association between the clinical decision support intervention and the primary outcome. There was a small but statistically significant difference in nephrology referral, with a higher rate of referral in the control arm.

Limitations

Contamination within provider and clinic may have attenuated the impact of the intervention and may have biased the result toward null.

Conclusions

The noninterruptive design of the clinical decision support was selected to prevent cognitive overload; however, the design led to a very low rate of use and ultimately did not improve stage-appropriate monitoring.

Funding

Research reported in this publication was supported by the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health under award K23DK097187.

Trial Registration

ClinicalTrials.gov Identifier: NCT02990897.

Clinical Decision Support for Laboratory Testing

BACKGROUND

As technology enables new and increasingly complex laboratory tests, test utilization presents a growing challenge for healthcare systems. Clinical decision support (CDS) refers to digital tools that present providers with clinically relevant information and recommendations, which have been shown to improve test utilization. Nevertheless, individual CDS applications often fail, and implementation remains challenging.

CONTENT

We review common classes of CDS tools grounded in examples from the literature as well as our own institutional experience. In addition, we present a practical framework and specific recommendations for effective CDS implementation.

SUMMARY

CDS encompasses a rich set of tools that have the potential to drive significant improvements in laboratory testing, especially with respect to test utilization. Deploying CDS effectively requires thoughtful design and careful maintenance, and structured processes focused on quality improvement and change management play an important role in achieving these goals.

Implementation and evaluation of sepsis surveillance and decision support in medical ICU and emergency department

OBJECTIVE

To improve the timely diagnosis and treatment of sepsis many institutions implemented automated sepsis alerts. Poor specificity, time delays, and a lack of actionable information lead to limited adoption by bedside clinicians and no change in practice or clinical outcomes. We aimed to compare sepsis care compliance before and after a multi-year implementation of a sepsis surveillance coupled with decision support in a tertiary care center.

DESIGN

Single center before and after study.

SETTING

Large academic Medical Intensive Care Unit (MICU) and Emergency Department (ED).

POPULATION

Patients 18 years of age or older admitted to *** Hospital MICU and ED from 09/4/2011 to 05/01/2018 with severe sepsis or septic shock.

INTERVENTIONS

Electronic medical record-based sepsis surveillance system augmented by clinical decision support and completion feedback.

MEASUREMENTS AND MAIN RESULTS

There were 1950 patients admitted to the MICU with the diagnosis of severe sepsis or septic shock during the study period. The baseline characteristics were similar before (N = 854) and after (N = 1096) implementation of sepsis surveillance. The performance of the alert was modest with a sensitivity of 79.9%, specificity of 76.9%, positive predictive value (PPV) 27.9%, and negative predictive value (NPV) 97.2%. There were 3424 unique alerts and 1131 confirmed sepsis patients after the sniffer implementation. During the study period average care bundle compliance was higher; however after taking into account improvements in compliance leading up to the intervention, there was no association between intervention and improved care bundle compliance (Odds ratio: 1.16; 95% CI: 0.71 to 1.89; p-value 0.554). Similarly, the intervention was not associated with improvement in hospital mortality (Odds ratio: 1.55; 95% CI: 0.95 to 2.52; p-value: 0.078).

CONCLUSIONS

A sepsis surveillance system incorporating decision support or completion feedback was not associated with improved sepsis care and patient outcomes.

Efficiency over thoroughness in laboratory testing decision making in primary care: findings from a realist review

Background

Existing research demonstrates significant variation in test-ordering practice, and growth in the use of laboratory tests in primary care. Reviews of interventions designed to change test-ordering practice report heterogeneity in design and effectiveness. Improving understanding of clinicians’ decision making in relation to laboratory testing is an important means of understanding practice patterns and developing theory-informed interventions.

Aim

To develop explanations for the underlying causes of patterns of variation and increasing use of laboratory tests in primary care, and make recommendations for future research and intervention design.

Design & setting

Realist review of secondary data from primary care.

Method

Diverse evidence, including data from qualitative and quantitative studies, was gathered via systematic and iterative searching processes. Data were synthesised according to realist principles to develop explanations accounting for clinicians’ decision making in relation to laboratory tests.

Results

A total of 145 documents contributed data to the synthesis. Laboratory test ordering can fulfil many roles in primary care. Decisions about tests are incorporated into practice heuristics and tests are deployed as a tool to manage patient interactions. Ordering tests may be easier than not ordering tests in existing systems. Alongside high workloads and limited time to devote to decision making, there is a common perception that laboratory tests are relatively inconsequential interventions. Clinicians prioritise efficiency over thoroughness in decision making about laboratory tests.

Conclusion

Interventions to change test-ordering practice can be understood as aiming to preserve efficiency or encourage thoroughness in decision making. Intervention designs and evaluations should consider how testing decisions are made in real-world clinical practice.

Designing, Conducting, and Reporting Clinical Decision Support Studies: Recommendations and Call to Action

By enabling more efficient and effective medical decision making, computer-based clinical decision support (CDS) could unlock widespread benefits from the significant investment in electronic health record (EHR) systems in the United States. Evidence from high-quality CDS studies is needed to enable and support this vision of CDS-facilitated care optimization, but limited guidance is available in the literature for designing and reporting CDS studies. To address this research gap, this article provides recommendations for designing, conducting, and reporting CDS studies to: 1) ensure that EHR data to inform the CDS are available; 2) choose decision rules that are consistent with local care processes; 3) target the right users and workflows; 4) make the CDS easy to access and use; 5) minimize the burden placed on users; 6) incorporate CDS success factors identified in the literature, in particular the automatic provision of CDS as a part of clinician workflow; 7) ensure that the CDS rules are adequately tested; 8) select meaningful evaluation measures; 9) use as rigorous a study design as is feasible; 10) think about how to deploy the CDS beyond the original host organization; 11) report the study in context; 12) help the audience understand why the intervention succeeded or failed; and 13) consider the financial implications. If adopted, these recommendations should help advance the vision of more efficient, effective care facilitated by useful and widely available CDS.

Primary care provider adherence to an alert for intensification of diabetes blood pressure medications before and after the addition of a "chart closure" hard stop

Objective

To evaluate provider responses to a narrowly targeted "Best Practice Advisory" (BPA) alert for the intensification of blood pressure medications for persons with diabetes before and after implementation of a "chart closure" hard stop, which is non-interruptive but demands an action or dismissal before the chart can be closed.

Materials and Methods

We designed a BPA that fired alerts within an electronic health record (EHR) system during outpatient encounters for patients with diabetes when they had elevated blood pressures and were not on angiotensin receptor blocking medications. The BPA alerts were implemented in eight primary care practices within UCLA Health. We compared data on provider responses to the alerts before and after implementing a "chart closure" hard stop, and we conducted chart reviews to adjudicate each alert's appropriateness.

Results

Providers responded to alerts more often after the "chart closure" hard stop was implemented (P < .001). Among 284 alert firings over 16 months, we judged 107 (37.7%) to be clinically unnecessary or inappropriate based on chart review. Among the remainder, which represent clear opportunities for treatment, providers ordered the indicated medication more often (41% vs 75%) after the "chart closure" hard stop was implemented (P = .001).

Discussion

The BPA alerts for diabetes and blood pressure control achieved relatively high specificity. The "chart closure" hard stop improved provider attention to the alerts and was effective at getting patients treated when they needed it.

Conclusion

Targeting specific omitted medication classes can produce relatively specific alerts that may reduce alert fatigue, and using a "chart closure" hard stop may prompt providers to take action without excessively disrupting their workflow.

Use of Electronic Health Record Access and Audit Logs to Identify Physician Actions Following Noninterruptive Alert Opening: Descriptive Study

Background

Electronic health record (EHR) access and audit logs record behaviors of providers as they navigate the EHR. These data can be used to better understand provider responses to EHR–based clinical decision support (CDS), shedding light on whether and why CDS is effective.

Objective

This study aimed to determine the feasibility of using EHR access and audit logs to track primary care physicians’ (PCPs’) opening of and response to noninterruptive alerts delivered to EHR InBaskets.

Methods

We conducted a descriptive study to assess the use of EHR log data to track provider behavior. We analyzed data recorded following opening of 799 noninterruptive alerts sent to 75 PCPs’ InBaskets through a prior randomized controlled trial. Three types of alerts highlighted new medication concerns for older patients’ posthospital discharge: information only (n=593), medication recommendations (n=37), and test recommendations (n=169). We sought log data to identify the person opening the alert and the timing and type of PCPs’ follow-up EHR actions (immediate vs by the end of the following day). We performed multivariate analyses examining associations between alert type, patient characteristics, provider characteristics, and contextual factors and likelihood of immediate or subsequent PCP action (general, medication-specific, or laboratory-specific actions). We describe challenges and strategies for log data use.

Results

We successfully identified the required data in EHR access and audit logs. More than three-quarters of alerts (78.5%, 627/799) were opened by the PCP to whom they were directed, allowing us to assess immediate PCP action; of these, 208 alerts were followed by immediate action. Expanding on our analyses to include alerts opened by staff or covering physicians, we found that an additional 330 of the 799 alerts demonstrated PCP action by the end of the following day. The remaining 261 alerts showed no PCP action. Compared to information-only alerts, the odds ratio (OR) of immediate action was 4.03 (95% CI 1.67-9.72) for medication-recommendation and 2.14 (95% CI 1.38-3.32) for test-recommendation alerts. Compared to information-only alerts, ORs of medication-specific action by end of the following day were significantly greater for medication recommendations (5.59; 95% CI 2.42-12.94) and test recommendations (1.71; 95% CI 1.09-2.68). We found a similar pattern for OR of laboratory-specific action. We encountered 2 main challenges: (1) Capturing a historical snapshot of EHR status (number of InBasket messages at time of alert delivery) required incorporation of data generated many months prior with longitudinal follow-up. (2) Accurately interpreting data elements required iterative work by a physician/data manager team taking action within the EHR and then examining audit logs to identify corresponding documentation.

Conclusions

EHR log data could inform future efforts and provide valuable information during development and refinement of CDS interventions. To address challenges, use of these data should be planned before implementing an EHR–based study.

Electronic alerts, comparative practitioner metrics, and education improve thromboprophylaxis and reduce venous thrombosis in community hospitals

BACKGROUND

Venous thromboembolism prophylaxis remains underutilized in hospitalized medical patients at high risk for venous thromboembolism. We previously reported that a multifaceted intervention was associated with a sustained increase in appropriate thromboprophylaxis and reduced symptomatic venous thromboembolism among medical patients hospitalized in two urban teaching hospitals. The effectiveness of this intervention in community hospitals is unknown.

METHODS

We performed a prospective multicenter cohort study in three community hospitals. All medical patients admitted from February 1, 2011 to January 31, 2014 were eligible. Consecutive eligible patients were enrolled into the 12-month "control," 12-month "intervention," or 12-month "maintenance" group. We provided electronic alerts, physician performance feedback, and targeted medical education for the intervention group. Only the alert component of the intervention continued in the maintenance group. The primary outcome was the rate of appropriate thromboprophylaxis among patients at high risk for venous thromboembolism defined as the prescription of guideline recommended chemoprophylaxis, or identification of a chemoprophylaxis contraindication. Secondary outcomes included rates of symptomatic venous thromboembolism, major bleeding, all-cause mortality, heparin-induced thrombocytopenia, physician satisfaction, and alert fatigue.

RESULTS

Appropriate thromboprophylaxis when compared to the control group rate of 67% was higher for the intervention group (85%) and for the maintenance group (77%; P < .001 for each comparison). A reduction of 90-day symptomatic venous thromboembolism accompanied the intervention (control 4.5%, intervention 3.4%, maintenance 3.0%, P = .04).

CONCLUSIONS

This multifaceted intervention was associated with an overall increase in appropriate thromboprophylaxis of medical patients compared with the control period. Hospital-associated venous thrombosis rates decreased.

Impact of nonintrusive clinical decision support systems on laboratory test utilization in a large academic centre

BACKGROUND

The near-universal prevalence of electronic health records (EHRs) has made the utilization of clinical decision support systems (CDSS) an integral strategy for improving the value of laboratory ordering. Few studies have examined the effectiveness of nonintrusive CDSS on inpatient laboratory utilization in large academic centres.

METHODS

Red blood cell folate, hepatitis C virus viral loads and genotypes, and type and screens were selected for study. We incorporated the appropriate indications for these labs into text that accompanied the laboratory orders in our hospital's EHR. Providers could proceed with the order without additional clicks. An interrupted time-series analysis was performed, and the primary outcome was the rate of tests ordered on all inpatient medicine floors.

RESULTS

The rate of folate tests ordered per monthly admissions showed no significant level change at the time of the intervention with only a slight decrease in rate of 0.0109 (P = .07). There was a 43% decrease in the rate of hepatitis C virus tests per monthly admissions immediately after the intervention with a decrease of 0.0135 tests per monthly admissions (P = .02). The rate of type and screens orders per patient days each month had a significant downward trend by 0.114 before the intervention (P = .04) but no significant level change at the time of the intervention or significant change in rate after the intervention.

DISCUSSION

Our study suggests that nonintrusive CDSS should be evaluated for individual laboratory tests to ensure only effective alerts continue to be used so as to avoid increasing EHR fatigue.

The Effects of Computerized Clinical Decision Support Systems on Laboratory Test Ordering: A Systematic Review

CONTEXT

- Inappropriate laboratory test ordering has been shown to be as high as 30%. This can have an important impact on quality of care and costs because of downstream consequences such as additional diagnostics, repeat testing, imaging, prescriptions, surgeries, or hospital stays.

OBJECTIVE

- To evaluate the effect of computerized clinical decision support systems on appropriateness of laboratory test ordering.

DATA SOURCES

- We used MEDLINE, Embase, CINAHL, MEDLINE In-Process and Other Non-Indexed Citations, Clinicaltrials.gov, Cochrane Library, and Inspec through December 2015. Investigators independently screened articles to identify randomized trials that assessed a computerized clinical decision support system aimed at improving laboratory test ordering by providing patient-specific information, delivered in the form of an on-screen management option, reminder, or suggestion through a computerized physician order entry using a rule-based or algorithm-based system relying on an evidence-based knowledge resource. Investigators extracted data from 30 papers about study design, various study characteristics, study setting, various intervention characteristics, involvement of the software developers in the evaluation of the computerized clinical decision support system, outcome types, and various outcome characteristics.

CONCLUSIONS

- Because of heterogeneity of systems and settings, pooled estimates of effect could not be made. Data showed that computerized clinical decision support systems had little or no effect on clinical outcomes but some effect on compliance. Computerized clinical decision support systems targeted at laboratory test ordering for multiple conditions appear to be more effective than those targeted at a single condition.

Electronic Alerts, Comparative Practitioner Metrics, and Education Improves Thromboprophylaxis and Reduces Thrombosis

BACKGROUND

Venous thromboembolism chemoprophylaxis remains underutilized in hospitalized medical patients at high risk for venous thromboembolism. We assessed the effect of a health care quality-improvement initiative comprised of a targeted electronic alert, comparative practitioner metrics, and practitioner-specific continuing medical education on the rate of appropriate venous thromboembolism chemoprophylaxis provided to medical inpatients at high risk for venous thromboembolism.

METHODS

We performed a multicenter prospective observational cohort study in an urban Utah hospital system. All medical patients admitted to 1 of 2 participating hospitals from April 1, 2010 to December 31, 2012 were eligible. Patients were members of the "control" (April 1, 2010 to December 31, 2010), "intervention" (January 1, 2011 to December 31, 2011), or "subsequent year" (January 1, 2012 to December 31, 2012) group. The primary outcome was the rate of appropriate chemoprophylaxis among patients at high risk for venous thromboembolism. Secondary outcomes included rates of symptomatic venous thromboembolism, major bleeding, all-cause mortality, heparin-induced thrombocytopenia, physician satisfaction, and alert fatigue.

RESULTS

The rate of appropriate chemoprophylaxis among patients at high risk for venous thromboembolism increased (66.1% control period vs 81.0% intervention period vs 88.1% subsequent year; P <.001 for each comparison). A significant reduction of 90-day symptomatic venous thromboembolism accompanied the quality initiative (9.3% control period, 9.7% intervention period, 6.7% subsequent year; P = .009); 30-day venous thromboembolism rates also significantly decreased.

CONCLUSIONS

A multifaceted intervention was associated with increased appropriate venous thromboembolism chemoprophylaxis among medical inpatients at high risk for venous thromboembolism and reduced symptomatic venous thromboembolism. The effect of the intervention was sustained.

Interventions to Educate Family Physicians to Change Test Ordering: Systematic Review of Randomized Controlled Trials

The purpose is to systematically review randomised controlled trials (RCTs) to change family physicians’ laboratory test-ordering. We searched 15 electronic databases (no language/date limitations). We identified 29 RCTs (4,111 physicians, 175,563 patients). Six studies specifically focused on reducing unnecessary tests, 23 on increasing screening tests. Using Cochrane methodology 48.5% of studies were low risk-of-bias for randomisation, 7% concealment of randomisation, 17% blinding of participants/personnel, 21% blinding outcome assessors, 27.5% attrition, 93% selective reporting. Only six studies were low risk for both randomisation and attrition. Twelve studies performed a power computation, three an intention-to-treat analysis and 13 statistically controlled clustering. Unweighted averages were computed to compare intervention/control groups for tests assessed by >5 studies. The results were that fourteen studies assessed lipids (average 10% more tests than control), 14 diabetes (average 8% > control), 5 cervical smears, 2 INR, one each thyroid, fecal occult-blood, cotinine, throat-swabs, testing after prescribing, and urine-cultures. Six studies aimed to decrease test groups (average decrease 18%), and two to increase test groups. Intervention strategies: one study used education (no change): two feedback (one 5% increase, one 27% desired decrease); eight education + feedback (average increase in desired direction >control 4.9%), ten system change (average increase 14.9%), one system change + feedback (increases 5-44%), three education + system change (average increase 6%), three education + system change + feedback (average 7.7% increase), one delayed testing. The conclusions are that only six RCTs were assessed at low risk of bias from both randomisation and attrition. Nevertheless, despite methodological shortcomings studies that found large changes (e.g. >20%) probably obtained real change.

A randomized controlled trial to provide adherence information and motivational interviewing to improve diabetes and lipid control

PURPOSE

The purpose of this study was to assess whether providing medication adherence information with or without motivational interviewing improves diabetes and lipid control.

METHODS

Study participants were adult members of a health system in southeast Michigan, were using both oral diabetes and lipid-lowering medications, and had glycated hemoglobin (A1C) or low-density lipoprotein cholesterol (LDL-C) levels not at goal. Participants were randomly assigned to receive usual care (UC), n = 567; have medication adherence information (AI) provided to their physician, n = 569; or have AI and receive motivational interviewing (MI) though trained staff (AI + MI), n = 556. Primary outcomes were A1C and LDL-C levels at 18 months post randomization.

RESULTS

Primary outcomes were not significantly different between patients in the AI or AI + MI study arms when compared with UC. Similarly, neither oral diabetes nor lipid-lowering medication adherence was significantly different between groups. Patient participation in the AI + MI arm was low and limit the interpretation of the study results, but post hoc analysis of the AI + MI study arm showed that the number of MI sessions received was positively associated with only oral diabetes medication adherence.

CONCLUSION

Neither AI nor MI significantly improved diabetes and lipid control when compared with UC. Moreover, patient participation appeared to be a particular barrier for MI.

Factors associated with ordering laboratory monitoring of high-risk medications

BACKGROUND

Knowledge about factors associated with provider ordering of appropriate testing is limited.

OBJECTIVE

To determine physician factors associated with ordering recommended laboratory monitoring tests for high-risk medications.

METHODS

Retrospective cohort study of patients prescribed a high-risk medication requiring laboratory monitoring in a large multispecialty group practice between 1 January 2008 and 31 December 2008. Analyses are based on administrative claims and electronic medical records. The outcome is a physician order for each recommended laboratory test for each prescribed medication. Key predictor variables are physician characteristics, including age, gender, specialty training, years since completing training, and prescribing volume. Additional variables are patient characteristics such as age, gender, comorbidity burden, whether the medication requiring monitoring is new or chronic, and drug-test characteristics such as inclusion in black box warnings. We used multivariable logistic regression, accounting for clustering of drugs within patients and patients within providers.

RESULTS

Physician orders for laboratory testing varied across drug-test pairs and ranged from 9% (Primidone-Phenobarbital level) to 97% (Azathioprine-CBC), with half of the drug-test pairs in the 85-91% ordered range. Test ordering was associated with higher provider prescribing volume for study drugs and specialist status (primary care providers were less likely to order tests than specialists). Patients with higher comorbidity burden and older patients were more likely to have appropriate tests ordered. Drug-test combinations with black box warnings were more likely to have tests ordered.

CONCLUSIONS

Interventions to improve laboratory monitoring should focus on areas with the greatest potential for improvement: providers with lower frequencies of prescribing medications with monitoring recommendations and those prescribing these medications for healthier and younger patients; patients with less interaction with the health care system are at particular risk of not having tests ordered. Black box warnings were associated with higher ordering rates and may be a tool to increase appropriate test ordering.

Clinical decision support for colon and rectal surgery: an overview

Clinical decision support (CDS) has been shown to improve clinical processes, promote patient safety, and reduce costs in healthcare settings, and it is now a requirement for clinicians as part of the Meaningful Use Regulation. However, most evidence for CDS has been evaluated primarily in internal medicine care settings, and colon and rectal surgery (CRS) has unique needs with CDS that are not frequently described in the literature. The authors reviewed published literature in informatics and medical journals, combined with expert opinion to define CDS, describe the evidence for CDS, outline the implementation process for CDS, and present applications of CDS in CRS.CDS functionalities such as order sets, documentation templates, and order facilitation aids are most often described in the literature and most likely to be beneficial in CRS. Further research is necessary to identify and better evaluate additional CDS systems in the setting of CRS.

The effectiveness of computerized drug-lab alerts: a systematic review and meta-analysis

BACKGROUND

Inadequate lab monitoring of drugs is a potential cause of ADEs (adverse drug events) which is remediable.

OBJECTIVES

To determine the effectiveness of computerized drug-lab alerts to improve medication-related outcomes.

DATA SOURCES

Citations from the Computerized Clinical Decision Support System Systematic Review (CCDSSR) and MMIT (Medications Management through Health Information Technology) databases, which had searched MEDLINE, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, International Pharmaceutical Abstracts from 1974 to March 27, 2013.

STUDY SELECTION

Randomized controlled trials (RCTs) of clinician-targeted computerized drug lab alerts conducted in any healthcare setting. Two reviewers performed full text review to determine study eligibility.

DATA ABSTRACTION

A single reviewer abstracted data and evaluated validity of included studies using Cochrane handbook domains.

DATA SYNTHESIS

Thirty-six studies met the inclusion criteria (25 single drug studies with 22,504 participants, 14 targeting anticoagulation; 11 multi-drug studies with 56,769 participants). ADEs were reported as an outcome in only four trials, all targeting anticoagulants. Computerized drug-lab alerts did not reduce ADEs (OR 0.89, 95% CI 0.79-1.00, p=0.05), length of hospital stay (SMD 0.00, 95%CI -0.93 to 0.93, p=0.055, 1 study), likelihood of hypoglycemia (OR 1.29, 95% CI 0.31-5.37) or likelihood of bleeding, but were associated with increased likelihood of prescribing changes (OR 1.73, 95% CI 1.21-2.47) or lab monitoring (OR 1.47, 95% confidence interval 1.12-1.94) in accordance with the alert.

CONCLUSIONS

There is no evidence that computerized drug-lab alerts are associated with important clinical benefits, but there is evidence of improvement in selected clinical surrogate outcomes (time in therapeutic range for vitamin K antagonists), and changes in process outcomes (lab monitoring and prescribing decisions).

Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation

OBJECTIVE

To apply human factors engineering principles to improve alert interface design. We hypothesized that incorporating human factors principles into alerts would improve usability, reduce workload for prescribers, and reduce prescribing errors.

MATERIALS AND METHODS

We performed a scenario-based simulation study using a counterbalanced, crossover design with 20 Veterans Affairs prescribers to compare original versus redesigned alerts. We redesigned drug-allergy, drug-drug interaction, and drug-disease alerts based upon human factors principles. We assessed usability (learnability of redesign, efficiency, satisfaction, and usability errors), perceived workload, and prescribing errors.

RESULTS

Although prescribers received no training on the design changes, prescribers were able to resolve redesigned alerts more efficiently (median (IQR): 56 (47) s) compared to the original alerts (85 (71) s; p=0.015). In addition, prescribers rated redesigned alerts significantly higher than original alerts across several dimensions of satisfaction. Redesigned alerts led to a modest but significant reduction in workload (p=0.042) and significantly reduced the number of prescribing errors per prescriber (median (range): 2 (1-5) compared to original alerts: 4 (1-7); p=0.024).

DISCUSSION

Aspects of the redesigned alerts that likely contributed to better prescribing include design modifications that reduced usability-related errors, providing clinical data closer to the point of decision, and displaying alert text in a tabular format. Displaying alert text in a tabular format may help prescribers extract information quickly and thereby increase responsiveness to alerts.

CONCLUSIONS

This simulation study provides evidence that applying human factors design principles to medication alerts can improve usability and prescribing outcomes.

The association between use of a clinical decision support tool and adherence to monitoring for medication-laboratory guidelines in the ambulatory setting

BACKGROUND

Stage 2 Meaningful Use criteria require the use of clinical decision support systems (CDSS) on high priority health conditions to improve clinical quality measures. Although CDSS hold great promise, implementation has been fraught with challenges, evidence of their impact is mixed, and the optimal method of content delivery is unknown.

OBJECTIVE

The authors investigated whether implementation of a simple clinical decision support (CDS) tool was associated with improved prescriber adherence to national medication-laboratory monitoring guidelines for safety (hepatic function, renal function, myalgias/rhabdomyolysis) and intermediate outcomes for antidiabetic (Hemoglobin A(1c); HbA(1c)) and antihyperlipidemic (low density lipoprotein; LDL) medications prescribed within a diabetes registry.

METHODS

This was a retrospective observational study conducted in three phases of CDS implementation (2008-2009): pre-, transition-, and post-Prescriptions evaluated were ordered from an electronic health record within a multispecialty medical group. Adherence was evaluated within and without applying guideline-imposed time constraints.

RESULTS

Forty-thousand prescriptions were ordered over three timeframes. For hepatic and renal function, the proportion of prescriptions for which labs were monitored at any time increased from 52% to 65% (p<0.001); those that met time guidelines, from 14% to 21% (p<0.001). Only 6% of required labs were drawn to monitor for myalgias/rhabdomyolysis, regardless of timeframe. Over 90% of safety labs were within normal limits. The proportion of labs monitored at any time for LDL increased from 56% to 64% (p<0.001); those that met time guidelines from 11% to 17% (p<0.001). The proportion of labs monitored at any time for HbA(1c) remained the same (72%); those that met time guidelines decreased from 45% to 41% (p<0.001).

CONCLUSION

A simple CDS tool may be associated with improved adherence to guidelines. Efforts are needed to confirm findings and improve the timeliness of monitoring; investigations to optimize alerts should be ongoing.

Use of health information technology to reduce diagnostic errors

BACKGROUND

Health information technology (HIT) systems have the potential to reduce delayed, missed or incorrect diagnoses. We describe and classify the current state of diagnostic HIT and identify future research directions.

METHODS

A multi-pronged literature search was conducted using PubMed, Web of Science, backwards and forwards reference searches and contributions from domain experts. We included HIT systems evaluated in clinical and experimental settings as well as previous reviews, and excluded radiology computer-aided diagnosis, monitor alerts and alarms, and studies focused on disease staging and prognosis. Articles were organised within a conceptual framework of the diagnostic process and areas requiring further investigation were identified.

RESULTS

HIT approaches, tools and algorithms were identified and organised into 10 categories related to those assisting: (1) information gathering; (2) information organisation and display; (3) differential diagnosis generation; (4) weighing of diagnoses; (5) generation of diagnostic plan; (6) access to diagnostic reference information; (7) facilitating follow-up; (8) screening for early detection in asymptomatic patients; (9) collaborative diagnosis; and (10) facilitating diagnostic feedback to clinicians. We found many studies characterising potential interventions, but relatively few evaluating the interventions in actual clinical settings and even fewer demonstrating clinical impact.

CONCLUSIONS

Diagnostic HIT research is still in its early stages with few demonstrations of measurable clinical impact. Future efforts need to focus on: (1) improving methods and criteria for measurement of the diagnostic process using electronic data; (2) better usability and interfaces in electronic health records; (3) more meaningful incorporation of evidence-based diagnostic protocols within clinical workflows; and (4) systematic feedback of diagnostic performance.

Validity of a clinical decision rule-based alert system for drug dose adjustment in patients with renal failure intended to improve pharmacists' analysis of medication orders in hospitals

OBJECTIVE

The main objective of this study was to assess the diagnostic performances of an alert system integrated into the CPOE/EMR system for renally cleared drug dosing control. The generated alerts were compared with the daily routine practice of pharmacists as part of the analysis of medication orders.

MATERIALS AND METHODS

The pharmacists performed their analysis of medication orders as usual and were not aware of the alert system interventions that were not displayed for the purpose of the study neither to the physician nor to the pharmacist but kept with associate recommendations in a log file. A senior pharmacist analyzed the results of medication order analysis with and without the alert system. The unit of analysis was the drug prescription line. The primary study endpoints were the detection of drug dose prescription errors and inter-rater reliability (Kappa coefficient) between the alert system and the pharmacists in the detection of drug dose error.

RESULTS

The alert system fired alerts in 8.41% (421/5006) of cases: 5.65% (283/5006) "exceeds max daily dose" alerts and 2.76% (138/5006) "under-dose" alerts. The alert system and the pharmacists showed a relatively poor concordance: 0.106 (CI 95% [0.068-0.144]). According to the senior pharmacist review, the alert system fired more appropriate alerts than pharmacists, and made fewer errors than pharmacists in analyzing drug dose prescriptions: 143 for the alert system and 261 for the pharmacists. Unlike the alert system, most diagnostic errors made by the pharmacists were 'false negatives'. The pharmacists were not able to analyze a significant number (2097; 25.42%) of drug prescription lines because understaffing.

CONCLUSION

This study strongly suggests that an alert system would be complementary to the pharmacists' activity and contribute to drug prescription safety.

A web-based system for clinical decision support and knowledge maintenance for deterioration monitoring of hemato-oncological patients

We introduce a web-based clinical decision support system (CDSS) and knowledge maintenance based on rules and a set covering method focusing on the problem of detecting serious comorbidities in hemato-oncological patients who are at high risk of developing serious infections and life threatening complications. We experienced that diagnostic problems which are characterized by fuzzy, uncertain knowledge and overlapping signs, still reveal some kind of patterns that can be transferred into a computer-based decision model. We applied a multi-stage evaluation process to assess the system's diagnostic performance. Depending on how system behavior was compared to presumably correct judgment of a case the correctness rate for closed cases with all data available varied between 58% and 71%, the overall rate after critical review was 84%. However, the real time behavior of our approach which data becoming available as time passes still has to be evaluated and observational studies need to be conducted.

Patient completion of laboratory tests to monitor medication therapy: a mixed-methods study

BACKGROUND

Little is known about the contribution of patient behavior to incomplete laboratory monitoring, and the reasons for patient non-completion of ordered laboratory tests remain unclear.

OBJECTIVE

To describe factors, including patient-reported reasons, associated with non-completion of ordered laboratory tests.

DESIGN

Mixed-Methods study including a quantitative assessment of the frequency of patient completion of ordered monitoring tests combined with qualitative, semi-structured, patient interviews.

PARTICIPANTS

Quantitative assessment included patients 18 years or older from a large multispecialty group practice, who were prescribed a medication requiring monitoring. Qualitative interviews included a subset of show and no-show patients prescribed a cardiovascular, anticonvulsant, or thyroid replacement medication.

MAIN MEASURES

Proportion of recommended monitoring tests for each medication not completed, factors associated with patient non-completion, and patient-reported reasons for non-completion.

KEY RESULTS

Of 27,802 patients who were prescribed one of 34 medications, patient non-completion of ordered tests varied (range: 0-24 %, by drug-test pair). Factors associated with higher odds of test non-completion included: younger patient age (< 40 years vs. ≥ 80 years, adjusted odds ratio [AOR] 1.52, 95 % confidence interval [95 % CI] 1.27-1.83); lower medication burden (one medication vs. more than one drug, AOR for non-completion 1.26, 95 % CI 1.15-1.37), and lower visit frequency (0-5 visits/year vs. ≥ 19 visits/year, AOR 1.41, 95 % CI 1.25 to 1.59). Drug-test pairs with black box warning status were associated with greater odds of non-completion, compared to drugs without a black box warning or other guideline for testing (AOR 1.91, 95 % CI 1.66-2.19). Qualitative interviews, with 16 no-show and seven show patients, identified forgetting as the main cause of non-completion of ordered tests.

CONCLUSIONS

Patient non-completion contributed to missed opportunities to monitor medications, and was associated with younger patient age, lower medication burden and black box warning status. Interventions to improve laboratory monitoring should target patients as well as physicians.

Computer-generated reminders delivered on paper to healthcare professionals; effects on professional practice and health care outcomes

BACKGROUND

Clinical practice does not always reflect best practice and evidence, partly because of unconscious acts of omission, information overload, or inaccessible information. Reminders may help clinicians overcome these problems by prompting the doctor to recall information that they already know or would be expected to know and by providing information or guidance in a more accessible and relevant format, at a particularly appropriate time.

OBJECTIVES

To evaluate the effects of reminders automatically generated through a computerized system and delivered on paper to healthcare professionals on processes of care (related to healthcare professionals' practice) and outcomes of care (related to patients' health condition).

SEARCH METHODS

For this update the EPOC Trials Search Co-ordinator searched the following databases between June 11-19, 2012: The Cochrane Central Register of Controlled Trials (CENTRAL) and Cochrane Library (Economics, Methods, and Health Technology Assessment sections), Issue 6, 2012; MEDLINE, OVID (1946- ), Daily Update, and In-process; EMBASE, Ovid (1947- ); CINAHL, EbscoHost (1980- ); EPOC Specialised Register, Reference Manager, and INSPEC, Engineering Village. The authors reviewed reference lists of related reviews and studies. 

SELECTION CRITERIA

We included individual or cluster-randomized controlled trials (RCTs) and non-randomized controlled trials (NRCTs) that evaluated the impact of computer-generated reminders delivered on paper to healthcare professionals on processes and/or outcomes of care.

DATA COLLECTION AND ANALYSIS

Review authors working in pairs independently screened studies for eligibility and abstracted data. We contacted authors to obtain important missing information for studies that were published within the last 10 years. For each study, we extracted the primary outcome when it was defined or calculated the median effect size across all reported outcomes. We then calculated the median absolute improvement and interquartile range (IQR) in process adherence across included studies using the primary outcome or median outcome as representative outcome.

MAIN RESULTS

In the 32 included studies, computer-generated reminders delivered on paper to healthcare professionals achieved moderate improvement in professional practices, with a median improvement of processes of care of 7.0% (IQR: 3.9% to 16.4%). Implementing reminders alone improved care by 11.2% (IQR 6.5% to 19.6%) compared with usual care, while implementing reminders in addition to another intervention improved care by 4.0% only (IQR 3.0% to 6.0%) compared with the other intervention. The quality of evidence for these comparisons was rated as moderate according to the GRADE approach. Two reminder features were associated with larger effect sizes: providing space on the reminder for provider to enter a response (median 13.7% versus 4.3% for no response, P value = 0.01) and providing an explanation of the content or advice on the reminder (median 12.0% versus 4.2% for no explanation, P value = 0.02). Median improvement in processes of care also differed according to the behaviour the reminder targeted: for instance, reminders to vaccinate improved processes of care by 13.1% (IQR 12.2% to 20.7%) compared with other targeted behaviours. In the only study that had sufficient power to detect a clinically significant effect on outcomes of care, reminders were not associated with significant improvements.

AUTHORS' CONCLUSIONS

There is moderate quality evidence that computer-generated reminders delivered on paper to healthcare professionals achieve moderate improvement in process of care. Two characteristics emerged as significant predictors of improvement: providing space on the reminder for a response from the clinician and providing an explanation of the reminder's content or advice. The heterogeneity of the reminder interventions included in this review also suggests that reminders can improve care in various settings under various conditions.

Efficacy of a clinical decision-support system in an HIV practice: a randomized trial

BACKGROUND

Data to support improved patient outcomes from clinical decision-support systems (CDSSs) are lacking in HIV care.

OBJECTIVE

To test the efficacy of a CDSS in improving HIV outcomes in an outpatient clinic.

DESIGN

Randomized, controlled trial. (ClinicalTrials.gov registration number: NCT00678600)

SETTING

Massachusetts General Hospital HIV Clinic.

PARTICIPANTS

HIV care providers and their patients.

INTERVENTION

Computer alerts were generated for virologic failure (HIV RNA level >400 copies/mL after a previous HIV RNA level ≤400 copies/mL), evidence of suboptimal follow-up, and 11 abnormal laboratory test results. Providers received interactive computer alerts, facilitating appointment rescheduling and repeated laboratory testing, for half of their patients and static alerts for the other half.

MEASUREMENTS

The primary end point was change in CD4 cell count. Other end points included time to clinical event, 6-month suboptimal follow-up, and severe laboratory toxicity.

RESULTS

Thirty-three HIV care providers followed 1011 patients with HIV. In the intervention group, the mean increase in CD4 cell count was greater (0.0053 vs. 0.0032 × 109 cells/L per month; difference, 0.0021 × 109 cells/L per month [95% CI, 0.0001 to 0.004]; P = 0.040) and the rate of 6-month suboptimal follow-up was lower (20.6 vs. 30.1 events per 100 patient-years; P = 0.022) than those in the control group. Median time to next scheduled appointment was shorter in the intervention group than in the control group after a suboptimal follow-up alert (1.71 vs. 3.48 months; P < 0.001) and after a toxicity alert (2.79 vs. >6 months; P = 0.072). More than 90% of providers supported adopting the CDSS as part of standard care.

LIMITATION

This was a 1-year informatics study conducted at a single hospital subspecialty clinic.

CONCLUSION

A CDSS using interactive provider alerts improved CD4 cell counts and clinic follow-up for patients with HIV. Wider implementation of such systems can provide important clinical benefits.

PRIMARY FUNDING SOURCE

National Institute of Allergy and Infectious Diseases.

A clinical data warehouse-based process for refining medication orders alerts

The objective of this case report is to evaluate the use of a clinical data warehouse coupled with a clinical information system to test and refine alerts for medication orders control before they were fully implemented. A clinical decision rule refinement process was used to assess alerts. The criteria assessed were the frequencies of alerts for initial prescriptions of 10 medications whose dosage levels depend on renal function thresholds. In the first iteration of the process, the frequency of the 'exceeds maximum daily dose' alerts was 7.10% (617/8692), while that of the 'under dose' alerts was 3.14% (273/8692). Indicators were presented to the experts. During the different iterations of the process, 45 (16.07%) decision rules were removed, 105 (37.5%) were changed and 136 new rules were introduced. Extensive retrospective analysis of physicians' medication orders stored in a clinical data warehouse facilitates alert optimization toward the goal of maximizing the safety of the patient and minimizing overridden alerts.

Changing clinical practice through patient specific reminders available at the time of the clinical encounter: systematic review and meta-analysis

OBJECTIVE

To synthesise current evidence for the influence on clinical behaviour of patient-specific electronically generated reminders available at the time of the clinical encounter.

DATA SOURCES

PubMed, Cochrane library of systematic reviews; Science Citation Index Expanded; Social Sciences Citation Index; ASSIA; EMBASE; CINAHL; DARE; HMIC were searched for relevant articles.

STUDY ELIGIBILITY CRITERIA, PARTICIPANTS AND INTERVENTIONS

We included controlled trials of reminder interventions if the intervention was: directed at clinician behaviour; available during the clinical encounter; computer generated (including computer generated paper-based reminders); and generated by patient-specific (rather than condition specific or drug specific) data.

STUDY APPRAISAL AND SYNTHESIS METHODS

Systematic review and meta-analysis of controlled trials published since 1970. A random effects model was used to derive a pooled odds ratio for adherence to recommended care or achievement of target outcome. Subgroups were examined based on area of care and study design. Odds ratios were derived for each sub-group. We examined the designs, settings and other features of reminders looking for factors associated with a consistent effect.

RESULTS

Altogether, 42 papers met the inclusion criteria. The studies were of variable quality and some were affected by unit of analysis errors due to a failure to account for clustering. An overall odds ratio of 1.79 [95% confidence interval 1.56, 2.05] in favour of reminders was derived. Heterogeneity was high and factors predicting effect size were difficult to identify.

LIMITATIONS

Methodological diversity added to statistical heterogeneity as an obstacle to meta-analysis. The quality of included studies was variable and in some reports procedural details were lacking.

CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS

The analysis suggests a moderate effect of electronically generated, individually tailored reminders on clinician behaviour during the clinical encounter. Future research should concentrate on identifying the features of reminder interventions most likely to result in the target behaviour.

Electronic health record-based monitoring of primary care patients at risk of medication-related toxicity

BACKGROUND

Timely laboratory monitoring may reduce the potential harm associated with chronic medication use. A study was conducted to determine the proportion of patients receiving National Committee for Quality Assurance (NCQA)-recommended laboratory medication monitoring in a primary care setting and to assess the effect of electronic health record (EHR)-derived, paper-based, provider-specific feedback bulletins on subsequent patient receipt of medication monitoring.

METHODS

In a single-arm, pre-post intervention in two federally qualified community health centers in Baltimore, patients targeted were adults prescribed at least 6 months (in the preceding year) for at least one index medication (digoxin, statins, diuretics, angiotensin-converting enzyme inhibitors/ angiotensin II-receptor blockers) in a 12-month period (August 2008-July 2009).

RESULTS

Among the 2,013 patients for whom medication monitoring was recommended, 42% were overdue for monitoring at some point during the study. As the number of index medications the patient was prescribed increased, the likelihood of ever being overdue for monitoring decreased. Being listed on the provider-specific monitoring bulletin doubled the odds of a patient receiving recommended laboratory monitoring before the next measurement period (1-2 months). Limiting the intervention to the most overdue patients, however, mitigated its overall impact.

CONCLUSIONS

Recommended laboratory monitoring of chronic medications appears to be inconsistent in primary care, resulting in potential harm for individuals at risk for medication-related toxicity. EHRs may be an important component of systems designed to improve medication monitoring, but multimodal interventions will likely be needed to achieve high reliability.

A novel strategy for evaluating the effects of an electronic test ordering alert message: Optimizing cardiac marker use

Background:

Laboratory ordering functions within computerized provider order entry (CPOE) systems typically support the display of electronic alert messages to improve test utilization or implement new ordering policies. However, alert strategies have been shown to vary considerably in their success and the characteristics contributing to an alert's success are poorly understood. Improved methodologies are needed to evaluate alerts and their mechanisms of action.

Materials and Methods:

Clinicians order inpatient and emergency department laboratory tests using our institutional CPOE system. We analyzed user interaction data captured by our CPOE system to evaluate how clinicians responded to an alert. We evaluated an alert designed to implement an institutional policy restricting the indications for ordering creatine kinase-MB (CKMB).

Results:

Within 2 months of alert implementation, CKMB-associated searches declined by 79% with a corresponding decline in CKMB orders. Furthermore, while prior to alert implementation, clinicians searching for CKMB ultimately ordered this test 99% of the time, following implementation, only 60% of CKMB searches ultimately led to CKMB test orders. This difference presumably represents clinicians who reconsidered the need for CKMB in response to the alert, demonstrating the alert's just-in-time advisory capability. In addition, as clinicians repeatedly viewed the alert, there was a “dose-dependant” decrease in the fraction of searches without orders. This presumably reflects the alerting strategy's long-term educational component, as clinicians aware of the new policy will not search for CKMB when not indicated.

Conclusions:

Our analytic approach provides insight into the mechanism of a CPOE alert and demonstrates that alerts may act through a combination of just-in-time advice and longer term education. Use of this approach when implementing alerts may prove useful to improve the success of a given alerting strategy.

Biochemical monitoring of patients treated with antihypertensive therapy for adverse drug reactions: a systematic review

Biochemical monitoring of patients treated with antihypertensive therapy is recommended in order to identify potential adverse reactions to treatment. We aimed to review the literature investigating the nature of biochemical monitoring in adults treated in primary care with antihypertensive drugs. Specifically, we wished to establish (i) the proportion of patients with biochemical baseline testing prior to the initiation of antihypertensive therapy; (ii) the proportion of patients with biochemical monitoring after initiation of antihypertensive therapy; (iii) the patient characteristics associated with biochemical monitoring; (iv) the frequency of biochemical monitoring after the initiation of antihypertensive therapy; and (v) the relationship, if any, between biochemical monitoring and adverse patient outcomes. We searched MEDLINE, EMBASE and Google Scholar from 1948 to 31 December 2010 using a combination of text words and search terms. Retrospective and prospective cohort studies, cross-sectional studies, randomized controlled trials or quasi-randomized controlled trials, and audits of current clinical practice were included. Clinical trials, case reports and case series were excluded. Studies were included if they provided data on the proportion of patients treated with antihypertensive therapy in primary care who had any biochemical monitoring before or after the initiation of therapy. In total, 15 studies were included in our review, which used a wide variety of definitions of monitoring prior to and after the initiation of antihypertensive therapy. From 17% to 81% of patients treated with antihypertensive drugs had a baseline biochemical test and from 20% to 79% had any follow-up monitoring. In only 7 of the 12 studies that examined follow-up monitoring did more than half of the patients have any monitoring. Overall, this systematic review provides evidence that monitoring as recommended by published guidelines is not commonly undertaken. Only two studies were identified that examined patients with both baseline testing and follow-up monitoring. Omission of one or the other limits the ability to analyse the effect of treatment on electrolyte concentrations or renal function. There is limited research on the patient factors associated with monitoring and further work is required to determine the impact of monitoring on adverse patient outcomes. Important barriers to effective monitoring exist and this review emphasizes that these have not yet been overcome.

The effectiveness of integrated health information technologies across the phases of medication management: a systematic review of randomized controlled trials

OBJECTIVE

The US Agency for Healthcare Research and Quality funded an evidence report to address seven questions on multiple aspects of the effectiveness of medication management information technology (MMIT) and its components (prescribing, order communication, dispensing, administering, and monitoring).

MATERIALS AND METHODS

Medline and 11 other databases without language or date limitations to mid-2010. Randomized controlled trials (RCTs) assessing integrated MMIT were selected by two independent reviewers. Reviewers assessed study quality and extracted data. Senior staff checked accuracy.

RESULTS

Most of the 87 RCTs focused on clinical decision support and computerized provider order entry systems, were performed in hospitals and clinics, included primarily physicians and sometimes nurses but not other health professionals, and studied process changes related to prescribing and monitoring medication. Processes of care improved for prescribing and monitoring mostly in hospital settings, but the few studies measuring clinical outcomes showed small or no improvements. Studies were performed most frequently in the USA (n=63), Europe (n=16), and Canada (n=6).

DISCUSSION

Many studies had limited description of systems, installations, institutions, and targets of the intervention. Problems with methods and analyses were also found. Few studies addressed order communication, dispensing, or administering, non-physician prescribers or pharmacists and their MMIT tools, or patients and caregivers. Other study methods are also needed to completely understand the effects of MMIT.

CONCLUSIONS

Almost half of MMIT interventions improved the process of care, but few studies measured clinical outcomes. This large body of literature, although instructive, is not uniformly distributed across settings, people, medication phases, or outcomes.

Interventions to safeguard system effectiveness during periods of emergency department crowding

This article summarizes the proceedings of a breakout session, "Interventions to Safeguard System Effectiveness," at the 2011 Academic Emergency Medicine consensus conference, "Interventions to Assure Quality in the Crowded Emergency Department." Key definitions fundamental to understanding the effectiveness of emergency care during periods of emergency department (ED) crowding are outlined. Next, a proposed research agenda to evaluate interventions directed at improving emergency care effectiveness is outlined, and the paper concludes with a prioritization of those interventions based on breakout session participant discussion and evaluation.

Provider and pharmacist responses to warfarin drug-drug interaction alerts: a study of healthcare downstream of CPOE alerts

OBJECTIVE

To categorize the appropriateness of provider and pharmacist responses to warfarin critical drug-drug interaction (cDDI) alerts, assess responses and actions to the cDDI, and determine the occurrence of warfarin adverse drug events (ADE) after alerts.

DESIGN

An 18-month, retrospective study of acute care admissions at a single Veterans Affairs medical center using computerized provider order entry (CPOE).

MEASUREMENTS

Patients included had at least one warfarin cDDI alert. Chart reviews included baseline laboratory values and demographics, provider actions, patient outcomes, and associated factors, including other interacting medications and number of simultaneously processed alerts.

RESULTS

137 admissions were included (133 unique patients). Amiodarone, vitamin E in a multivitamin, sulfamethoxazole, and levothyroxine accounted for 75% of warfarin cDDI. Provider responses were clinically appropriate in 19.7% of admissions and pharmacist responses were appropriate in 9.5% of admissions. There were 50 ADE (36.6% of admissions) with warfarin; 80% were rated as having no or mild clinical effect. An increased number of non-critical alerts at the time of the reference cDDI alert was the only variable associated with an inappropriate provider response (p=0.01).

LIMITATIONS

This study was limited by being a retrospective review and the possibility of confounding variables, such as other interacting medications.

CONCLUSION

The large number of CPOE alerts may lead to inappropriate responses by providers and pharmacists. The high rate of ADE suggests a need for improved medication management systems for patients on warfarin. This study highlights the possibility of alert fatigue contributing to the high prevalence of inappropriate alert over-ride text responses.

A clinical decision support needs assessment of community-based physicians

OBJECTIVE

To conduct a grounded needs assessment to elicit community-based physicians' current views on clinical decision support (CDS) and its desired capabilities that may assist future CDS design and development for community-based practices.

MATERIALS AND METHODS

To gain insight into community-based physicians' goals, environments, tasks, and desired support tools, we used a human-computer interaction model that was based in grounded theory. We conducted 30 recorded interviews with, and 25 observations of, primary care providers within 15 urban and rural community-based clinics across Oregon. Participants were members of three healthcare organizations with different commercial electronic health record systems. We used a grounded theory approach to analyze data and develop a user-centered definition of CDS and themes related to desired CDS functionalities.

RESULTS

Physicians viewed CDS as a set of software tools that provide alerts, prompts, and reference tools, but not tools to support patient management, clinical operations, or workflow, which they would like. They want CDS to enhance physician-patient relationships, redirect work among staff, and provide time-saving tools. Participants were generally dissatisfied with current CDS capabilities and overall electronic health record usability.

DISCUSSION

Physicians identified different aspects of decision-making in need of support: clinical decision-making such as medication administration and treatment, and cognitive decision-making that enhances relationships and interactions with patients and staff.

CONCLUSION

Physicians expressed a need for decision support that extended beyond their own current definitions. To meet this requirement, decision support tools must integrate functions that align time and resources in ways that assist providers in a broad range of decisions.

Adopting real-time surveillance dashboards as a component of an enterprisewide medication safety strategy

BACKGROUND

High-alert medications are frequently responsible for adverse drug events and present significant hazards to inpatients, despite technical improvements in the way they are ordered, dispensed, and administered.

METHODS

A real-time surveillance application was designed and implemented to enable pharmacy review of high-alert medication orders to complement existing computerized provider order entry and integrated clinical decision support systems in a tertiary care hospital. The surveillance tool integrated real-time data from multiple clinical systems and applied logical criteria to highlight potentially high-risk scenarios. Use of the surveillance system for adult inpatients was analyzed for warfarin, heparin and enoxaparin, and aminoglycoside antibiotics.

RESULTS

Among 28,929 hospitalizations during the study period, patients eligible to appear on a dashboard included 2224 exposed to warfarin, 8383 to heparin or enoxaparin, and 893 to aminoglycosides. Clinical pharmacists reviewed the warfarin and aminoglycoside dashboards during 100% of the days in the study period-and the heparinlenoxaparin dashboard during 71% of the days. Displayed alert conditions ranged from common events, such as 55% of patients receiving aminoglycosides were missing a baseline creatinine, to rare events, such as 0.1% of patients exposed to heparin were given a bolus greater than 10,000 units. On the basis of interpharmacist communication and electronic medical record notes recorded within the dashboards, interventions to prevent further patient harm were frequent.

CONCLUSIONS

Even in an environment with sophisticated computerized provider order entry and clinical decision support systems, real-time pharmacy surveillance of high-alert medications provides an important platform for intercepting medication errors and optimizing therapy.

Computerized provider order entry in the clinical laboratory

Clinicians have traditionally ordered laboratory tests using paper-based orders and requisitions. However, paper orders are becoming increasingly incompatible with the complexities, challenges, and resource constraints of our modern healthcare systems and are being replaced by electronic order entry systems. Electronic systems that allow direct provider input of diagnostic testing or medication orders into a computer system are known as Computerized Provider Order Entry (CPOE) systems. Adoption of laboratory CPOE systems may offer institutions many benefits, including reduced test turnaround time, improved test utilization, and better adherence to practice guidelines. In this review, we outline the functionality of various CPOE implementations, review the reported benefits, and discuss strategies for using CPOE to improve the test ordering process. Further, we discuss barriers to the implementation of CPOE systems that have prevented their more widespread adoption.

Making electronic prescribing alerts more effective: scenario-based experimental study in junior doctors

OBJECTIVE

Expert authorities recommend clinical decision support systems to reduce prescribing error rates, yet large numbers of insignificant on-screen alerts presented in modal dialog boxes persistently interrupt clinicians, limiting the effectiveness of these systems. This study compared the impact of modal and non-modal electronic (e-) prescribing alerts on prescribing error rates, to help inform the design of clinical decision support systems.

DESIGN

A randomized study of 24 junior doctors each performing 30 simulated prescribing tasks in random order with a prototype e-prescribing system. Using a within-participant design, doctors were randomized to be shown one of three types of e-prescribing alert (modal, non-modal, no alert) during each prescribing task.

MEASUREMENTS

The main outcome measure was prescribing error rate. Structured interviews were performed to elicit participants' preferences for the prescribing alerts and their views on clinical decision support systems.

RESULTS

Participants exposed to modal alerts were 11.6 times less likely to make a prescribing error than those not shown an alert (OR 11.56, 95% CI 6.00 to 22.26). Those shown a non-modal alert were 3.2 times less likely to make a prescribing error (OR 3.18, 95% CI 1.91 to 5.30) than those not shown an alert. The error rate with non-modal alerts was 3.6 times higher than with modal alerts (95% CI 1.88 to 7.04).

CONCLUSIONS

Both kinds of e-prescribing alerts significantly reduced prescribing error rates, but modal alerts were over three times more effective than non-modal alerts. This study provides new evidence about the relative effects of modal and non-modal alerts on prescribing outcomes.

Can computerized clinical decision support systems improve practitioners' diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review

Background

Underuse and overuse of diagnostic tests have important implications for health outcomes and costs. Decision support technology purports to optimize the use of diagnostic tests in clinical practice. The objective of this review was to assess whether computerized clinical decision support systems (CCDSSs) are effective at improving ordering of tests for diagnosis, monitoring of disease, or monitoring of treatment. The outcome of interest was effect on the diagnostic test-ordering behavior of practitioners.

Methods

We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews database, Inspec, and reference lists for eligible articles published up to January 2010. We included randomized controlled trials comparing the use of CCDSSs to usual practice or non-CCDSS controls in clinical care settings. Trials were eligible if at least one component of the CCDSS gave suggestions for ordering or performing a diagnostic procedure. We considered studies 'positive' if they showed a statistically significant improvement in at least 50% of test ordering outcomes.

Results

Thirty-five studies were identified, with significantly higher methodological quality in those published after the year 2000 (p = 0.002). Thirty-three trials reported evaluable data on diagnostic test ordering, and 55% (18/33) of CCDSSs improved testing behavior overall, including 83% (5/6) for diagnosis, 63% (5/8) for treatment monitoring, 35% (6/17) for disease monitoring, and 100% (3/3) for other purposes. Four of the systems explicitly attempted to reduce test ordering rates and all succeeded. Factors of particular interest to decision makers include costs, user satisfaction, and impact on workflow but were rarely investigated or reported.

Conclusions

Some CCDSSs can modify practitioner test-ordering behavior. To better inform development and implementation efforts, studies should describe in more detail potentially important factors such as system design, user interface, local context, implementation strategy, and evaluate impact on user satisfaction and workflow, costs, and unintended consequences.

Computerized clinical decision support systems for drug prescribing and management: a decision-maker-researcher partnership systematic review

Background

Computerized clinical decision support systems (CCDSSs) for drug therapy management are designed to promote safe and effective medication use. Evidence documenting the effectiveness of CCDSSs for improving drug therapy is necessary for informed adoption decisions. The objective of this review was to systematically review randomized controlled trials assessing the effects of CCDSSs for drug therapy management on process of care and patient outcomes. We also sought to identify system and study characteristics that predicted benefit.

Methods

We conducted a decision-maker-researcher partnership systematic review. We updated our earlier reviews (1998, 2005) by searching MEDLINE, EMBASE, EBM Reviews, Inspec, and other databases, and consulting reference lists through January 2010. Authors of 82% of included studies confirmed or supplemented extracted data. We included only randomized controlled trials that evaluated the effect on process of care or patient outcomes of a CCDSS for drug therapy management compared to care provided without a CCDSS. A study was considered to have a positive effect (i.e., CCDSS showed improvement) if at least 50% of the relevant study outcomes were statistically significantly positive.

Results

Sixty-five studies met our inclusion criteria, including 41 new studies since our previous review. Methodological quality was generally high and unchanged with time. CCDSSs improved process of care performance in 37 of the 59 studies assessing this type of outcome (64%, 57% of all studies). Twenty-nine trials assessed patient outcomes, of which six trials (21%, 9% of all trials) reported improvements.

Conclusions

CCDSSs inconsistently improved process of care measures and seldomly improved patient outcomes. Lack of clear patient benefit and lack of data on harms and costs preclude a recommendation to adopt CCDSSs for drug therapy management.

Electronic medical record prompts for lab orders in patients initiating statins

BACKGROUND

The Institute of Medicine (IOM) reports that at least a fourth of all medication related injuries are preventable. Therefore, the IOM recommends healthcare organizations and providers implement electronic prescribing and clinical decision support systems in practices to aid in medication error prevention.

OBJECTIVE

To assess the impact of noninstrusive-intrusive prompts from an electronic medical record on recommended baseline and follow up laboratory monitoring, CK and liver transaminase levels (AST and ALT), in patients initiated on statin therapy.

METHODS

Hybrid nonintrusive-intrusive prompts for laboratory monitoring specific for statin initiation were implemented in the electronic medical record system in a community based, university affiliated family medicine residency program. A retrospective chart review was conducted to compare and assess laboratory monitoring in patients initiated on statin therapy from two specific time periods: a six month period prior to initiation of the prompts and a six month period after initiation of the prompts.

RESULTS

One hundred seventy three patients met inclusion criteria. There were no significant differences in assessment of baseline liver transaminases and CK levels from the initial study period to the follow up study period. There were significant differences in follow up liver transaminase levels (18% vs 33%, p = 0.035) and CK levels (none vs 7%, p = 0.03) from the initial study period to the activated prompt interval.

CONCLUSION

A hybrid nonintrusive-intrusive specific prompts for laboratory monitoring triggered by statin initiation within an electronic medical record improved follow up lab assessments for liver transaminases and CK but did not improve baseline assessments of CK or liver transaminases.

Impact of health information technology interventions to improve medication laboratory monitoring for ambulatory patients: a systematic review

Medication errors are a major source of morbidity and mortality. Inadequate laboratory monitoring of high-risk medications after initial prescription is a medical error that contributes to preventable adverse drug events. Health information technology (HIT)-based clinical decision support may improve patient safety by improving the laboratory monitoring of high-risk medications, but the effectiveness of such interventions is unclear. Therefore, the authors conducted a systematic review to identify studies that evaluate the independent effect of HIT interventions on improving laboratory monitoring for high-risk medications in the ambulatory setting using a Medline search from January 1, 1980 through January 1, 2009 and a manual review of relevant bibliographies. All anticoagulation monitoring studies were excluded. Eight articles met the inclusion criteria, including six randomized controlled trials and two pre-post intervention studies. Six of the studies were conducted in two large, integrated healthcare delivery systems in the USA. Overall, five of the eight studies reported statistically significant, but small, improvements in laboratory monitoring; only half of the randomized controlled trials reported statistically significant improvements. Studies that found no improvement were more likely to have used analytic strategies that addressed clustering and confounding. Whether HIT improves laboratory monitoring of certain high-risk medications for ambulatory patients remains unclear, and further research is needed to clarify this important question.