Clinical reminder alert fatigue in healthcare: a systematic literature review protocol using qualitative evidence

Views and Uses of Sepsis Digital Alerts in National Health Service Trusts in England: Qualitative Study With Health Care Professionals

BACKGROUND

Sepsis is a common cause of serious illness and death. Sepsis management remains challenging and suboptimal. To support rapid sepsis diagnosis and treatment, screening tools have been embedded into hospital digital systems to appear as digital alerts. The implementation of digital alerts to improve the management of sepsis and deterioration is a complex intervention that has to fit with team workflow and the views and practices of hospital staff. Despite the importance of human decision-making and behavior in optimal implementation, there are limited qualitative studies that explore the views and experiences of health care professionals regarding digital alerts as sepsis or deterioration computerized clinician decision support systems (CCDSSs).

OBJECTIVE

This study aims to explore the views and experiences of health care professionals on the use of sepsis or deterioration CCDSSs and to identify barriers and facilitators to their implementation and use in National Health Service (NHS) hospitals.

METHODS

We conducted a qualitative, multisite study with unstructured observations and semistructured interviews with health care professionals from emergency departments, outreach teams, and intensive or acute units in 3 NHS hospital trusts in England. Data from both interviews and observations were analyzed together inductively using thematic analysis.

RESULTS

A total of 22 health care professionals were interviewed, and 12 observation sessions were undertaken. A total of four themes regarding digital alerts were identified: (1) support decision-making as nested in electronic health records, but never substitute professionals' knowledge and experience; (2) remind to take action according to the context, such as the hospital unit and the job role; (3) improve the alerts and their introduction, by making them more accessible, easy to use, not intrusive, more accurate, as well as integrated across the whole health care system; and (4) contextual factors affecting views and use of alerts in the NHS trusts. Digital alerts are more optimally used in general hospital units with a lower senior decision maker:patient ratio and by health care professionals with experience of a similar technology. Better use of the alerts was associated with quality improvement initiatives and continuous sepsis training. The trusts' features, such as the presence of a 24/7 emergency outreach team, good technological resources, and staffing and teamwork, favored a more optimal use.

CONCLUSIONS

Trust implementation of sepsis or deterioration CCDSSs requires support on multiple levels and at all phases of the intervention, starting from a prego-live analysis addressing organizational needs and readiness. Advancements toward minimally disruptive and smart digital alerts as sepsis or deterioration CCDSSs, which are more accurate and specific but at the same time scalable and accessible, require policy changes and investments in multidisciplinary research.

Impact of an Electronic Health Record-Based Interruptive Alert Among Patients With Headaches Seen in Primary Care: Cluster Randomized Controlled Trial

Background

Headaches, including migraines, are one of the most common causes of disability and account for nearly 20%-30% of referrals from primary care to neurology. In primary care, electronic health record-based alerts offer a mechanism to influence health care provider behaviors, manage neurology referrals, and optimize headache care.

Objective

This project aimed to evaluate the impact of an electronic alert implemented in primary care on patients' overall headache management.

Methods

We conducted a stratified cluster-randomized study across 38 primary care clinic sites between December 2021 to December 2022 at a large integrated health care delivery system in the United States. Clinics were stratified into 6 blocks based on region and patient-to-health care provider ratios and then 1:1 randomized within each block into either the control or intervention. Health care providers practicing at intervention clinics received an interruptive alert in the electronic health record. The primary end point was a change in headache burden, measured using the Headache Impact Test 6 scale, from baseline to 6 months. Secondary outcomes included changes in headache frequency and intensity, access to care, and resource use. We analyzed the difference-in-differences between the arms at follow-up at the individual patient level.

Results

We enrolled 203 adult patients with a confirmed headache diagnosis. At baseline, the average Headache Impact Test 6 scores in each arm were not significantly different (intervention: mean 63, SD 6.9; control: mean 61.8, SD 6.6; P=.21). We observed a significant reduction in the headache burden only in the intervention arm at follow-up (3.5 points; P=.009). The reduction in the headache burden was not statistically different between groups (difference-in-differences estimate -1.89, 95% CI -5 to 1.31; P=.25). Similarly, secondary outcomes were not significantly different between groups. Only 11.32% (303/2677) of alerts were acted upon.

Conclusions

The use of an interruptive electronic alert did not significantly improve headache outcomes. Low use of alerts by health care providers prompts future alterations of the alert and exploration of alternative approaches.

A Discount Approach to Reducing Nursing Alert Burden

BACKGROUND

Numerous programs have arisen to address interruptive clinical decision support (CDS) with the goals of reducing alert burden and alert fatigue. These programs often have standing committees with broad stakeholder representation, significant governance efforts, and substantial analyst hours to achieve reductions in alert burden which can be difficult for hospital systems to replicate.

OBJECTIVE

 This study aimed to reduce nursing alert burden with a primary nurse informaticist and small support team through a quality-improvement approach focusing on high-volume alerts.

METHODS

 Target alerts were identified from the period of January 2022 to April 2022 and four of the highest firing alerts were chosen initially, which accounted for 43% of all interruptive nursing alerts and an estimated 86 hours per month of time across all nurses occupied resolving these alerts per month. Work was done concurrently for each alert with design changes based on the Five Rights of CDS and following a quality-improvement framework. Priority for work was based on operational engagement for design review and approval. Once initial design changes were approved, alerts were taken for in situ usability testing and additional changes were made as needed. Final designs were presented to stakeholders for approval prior to implementation.

RESULTS

 The total number of interruptive nursing alert firings decreased by 58% from preintervention period (1 January 2022-30 June 2022) to postintervention period (July 1, 2022-December 31, 2022). Action taken on alerts increased from 8.1 to 17.3%. The estimated time spent resolving interruptive alerts summed across all nurses in the system decreased from 197 hours/month to 114 hours/month.

CONCLUSION

 While CDS may improve use of evidence-based practices, implementation without a clear framework for evaluation and monitoring often results in alert burden and fatigue without clear benefits. An alert burden reduction effort spearheaded by a single empowered nurse informaticist efficiently reduced nursing alert burden substantially.

Temporary Financial Assistance for Housing Expenditures and Mortality and Suicide Outcomes Among US Veterans

INTRODUCTION

It is unclear whether interventions designed to increase housing stability can also lead to improved health outcomes such as reduced risk of death and suicide morbidity. The objective of this study was to estimate the potential impact of temporary financial assistance (TFA) for housing-related expenses from the US Department of Veterans Affairs (VA) on health outcomes including all-cause mortality, suicide attempt, and suicidal ideation.

METHODS

We conducted a retrospective national cohort study of Veterans who entered the VA Supportive Services for Veteran Families (SSVF) program between 10/2015 and 9/2018. We assessed the association between TFA and health outcomes using a multivariable Cox proportional hazards regression approach with inverse probability of treatment weighting. We conducted these analyses on our overall cohort as well as separately for those in the rapid re-housing (RRH) and homelessness prevention (HP) components of SSVF. Outcomes were all-cause mortality, suicide attempt, and suicidal ideation at 365 and 730 days following enrollment in SSVF.

RESULTS

Our analysis cohort consisted of 41,969 unique Veterans with a mean (SD) duration of 87.6 (57.4) days in the SSVF program. At 365 days following SSVF enrollment, TFA was associated with a decrease in the risk of all-cause mortality (HR: 0.696, p < 0.001) and suicidal ideation (HR: 0.788, p < 0.001). We found similar results at 730 days (HR: 0.811, p = 0.007 for all-cause mortality and HR: 0.881, p = 0.037 for suicidal ideation). These results were driven primarily by individuals enrolled in the RRH component of SSVF. We found no association between TFA and suicide attempts.

CONCLUSION

We find that providing housing-related financial assistance to individuals facing housing instability is associated with improvements in important health outcomes such as all-cause mortality and suicidal ideation. If causal, these results suggest that programs to provide housing assistance have positive spillover effects into other important aspects of individuals' lives.

Resident-Driven Clinical Decision Support Governance to Improve the Utility of Clinical Decision Support

OBJECTIVES

 This resident-driven quality improvement project aimed to better understand the known problem of a misaligned clinical decision support (CDS) strategy and improve CDS utilization.

METHODS

 An internal survey was sent to all internal medicine (IM) residents to identify the most bothersome CDS alerts. Survey results were supported by electronic health record (EHR) data of CDS firing rates and response rates which were collected for each of the three most bothersome CDS tools. Changes to firing criteria were created to increase utilization and to better align with the five rights of CDS. Findings and proposed changes were presented to our institution's CDS Governance Committee. Changes were approved and implemented. Postintervention firing rates were then collected for 1 week.

RESULTS

 Twenty nine residents participated in the CDS survey and identified sepsis alerts, lipid profile reminders, and telemetry renewals to be the most bothersome alerts. EHR data showed action rates for these CDS as low as 1%. We implemented changes to focus emergency department (ED)-based sepsis alerts to the right provider, better address the right information for lipid profile reminders, and select the right time in workflow for telemetry renewals to be most effective. With these changes we successfully eliminated ED-based sepsis CDS reminders for IM providers, saw a 97% reduction in firing rates for the lipid profile CDS, and noted a 55% reduction in firing rates for telemetry CDS.

CONCLUSION

 This project highlighted that alert improvements spearheaded by resident teams can be completed successfully using robust CDS governance strategies and can effectively optimize interruptive alerts.

Evaluation of an emergency department opt-out provider-driven HIV and syphilis screening and linkage-to-care program

OBJECTIVE

While the effectiveness of emergency departments (ED) in screening for HIV and syphilis is understood, less is known about dual screening programs. We aim to evaluate the impact of an opt-out provider-initiated HIV and syphilis program on screening, diagnosis, and linkage to care outcomes.

METHODS

We performed a retrospective review of patients screened pre (2014-2017) and post (2017-2021) program implementation. Primary outcomes include HIV and syphilis screening, incidence of positive tests, and proportion of patients linked to care. Secondary outcomes included pre-exposure prophylaxis (PrEP) referral and successful linkage rates for HIV-negative syphilis-positive patients.

RESULTS

Pre-implementation, 882 HIV tests were performed, of which 22 (2.49%) were new cases and 18 (81.82%) were linked to care; 754 syphilis tests were performed, of which 33 (4.38%) were active infections and 30 (90.91%) were treated. No eligible patients received PrEP referral. Post-implementation, 12,999 HIV tests were performed, of which 73 (0.56%) were new cases and 55 (75.34%) were linked to care; 10,885 syphilis tests were performed, of which 216 (1.98%) were active infections and 188 (87.04%) were treated. 25 (9.09%) eligible patients were referred for PrEP, and four (16.0%) attended their appointment.

CONCLUSIONS

Post-implementation, there was a 1373.81% and 1343.63% increase in screening, and a 231.82% and 554.55% increase in positive cases of HIV and syphilis, respectively. Dual screening programs can be successfully implemented within the existing ED framework to increase screening and early detection for HIV and syphilis.

A novel fracture liaison service using digital health: impact on mortality in hospitalized elderly osteoporotic fracture patients

We examined the performance of an intelligent fracture liaison service (FLS) assisted by digital health (DH) to reduce all-cause mortality (ACM) risk. According to our findings, the new FLS reduced ACM by 36%.

INTRODUCTION

A well-designed secondary prevention program known as FLS enhances the bone densitometry-based assessment rate as well as osteoporosis (OP) medication usage following a fracture. However, there are only a few reports on FLS incorporating DH, and it remains unclear whether this integration has influenced patient ACM, which refers to the overall death rate from any cause during the study period.

METHODS

This retrospective observational study was conducted on data from the Fragility Fracture Registration System database linked to the Regional Health Registration Platform of Kunshan City and the Population Death Registration System of Jiangsu Province for one tertiary-level A hospital in China. Patients aged ≥ 50 years, who experienced an OP fracture between January 1, 2017, and July 27, 2022, requiring hospitalization, were selected for analysis. We compared the outcomes of patients who received routine fragility fracture management (the no-FLS group) or FLS (the FLS group). We employed multivariable Cox regression with inverse probability weighting based on the propensity score (PS).

RESULTS

Of 2317 patients, 756 (32.6%) received FLS and 1561 (67.4%) did not. Using PS matching, we minimized the baseline characteristic differences between the two groups in the propensity score-matched samples, relative to the unmatched samples. Based on our analysis, the new FLS reduced ACM by 36% (hazard ratio [HR], 0.64; 95% confidence interval [CI], 0.47 to 0.87; P-value = 0.004). Moreover, FLS patients experienced further reductions in fall-related mortality, refracture rate, and total refracture-related hospital costs, and had increased dual-energy X-ray absorptiometry (DXA) testing and treatment initiation rates, relative to the no-FLS patients.

CONCLUSIONS

A new FLS model implementation assisted by DH can effectively reduce ACM among elderly patients with OP fractures requiring surgery. In future investigations, we recommend examining the scalability of this model.

Impact of Predictive Text Clinical Decision Support on Imaging Order Entry in the Emergency Department

PURPOSE

Imaging clinical decision support (CDS) is designed to assist providers in selecting appropriate imaging studies and is now federally required. The aim of this study was to understand the effect of CDS on decisions and workflows in the emergency department (ED).

METHODS

The authors' institution's order entry platform serves up structured indications for imaging orders. Imaging orders are scored by CDS on the basis of appropriate use criteria (AUC). CDS triggers alerts for imaging orders with low AUC scores. Because free text alone cannot be scored by CDS, an artificial intelligence predictive text (AIPT) module was implemented to guide the selection of structured indications when free-text indications are entered. A total of 17,355 imaging orders in the ED over 6 months were retrospectively analyzed.

RESULTS

CDS alerts for low AUC scores were triggered for 3% of all imaging study orders (522 of 17,355). Providers spent an average of 24 seconds interacting with alerts. In 18 of 522 imaging orders with alerts, alternative studies were ordered. After AIPT implementation, the percentage of unscored studies significantly decreased from 81% to 45% (P < .001).

CONCLUSIONS

In a quaternary academic ED, CDS alerts triggered by low AUC scores caused minimal increase in time spent on imaging order entry but had a relatively marginal impact on imaging study selection. AIPT implementation increased the number of scored studies and could potentially enhance CDS effects. CDS implementation enables the collection of novel data regarding which imaging studies receive low AUC scores. Future work could include exploring alternative models of CDS implementation to maximize its impact.

Reducing low-value ED coags across 11 hospitals in a safety net setting

BACKGROUND

Prothrombin/international normalized ratio and activated partial thromboplastin time (PT/INR and aPTT) are frequently ordered in emergency departments (EDs), but rarely affect management. They offer limited utility outside of select indications. Several quality improvement initiatives have shown reduction in ED use of PT/INR and aPTT using multifaceted interventions in well-resourced settings. Successful reduction of these low-value tests has not yet been shown using a single intervention across a large hospital system in a safety net setting. This study aims to determine if an intervention of two BPAs is associated with a reduction in PT/INR and aPTT usage across a large safety net system.

METHODS

This initiative was set at a large safety net system in the United States with 11 acute care hospitals. Two Best Practice Advisories (BPAs) discouraging inappropriate PT/INR and aPTT use were implemented from March 16, 2022-August 30, 2022. Order rate per 100 ED patients during the pre-intervention period was compared to the post-intervention period on both the system and individual hospital level. Complete blood count (CBC) testing served as a control, and packed red blood cell transfusions served as a balancing measure. An interrupted time series regression analysis was performed to capture immediate and temporal changes in ordering for all tests in the pre and post-intervention periods.

RESULTS

PT/INR tests exhibited an absolute decline of 4.11 tests per 100 ED encounters (95% confidence interval -5.17 to -3.05; relative reduction of 18.9%). aPTT tests exhibited absolute decline of 4.03 tests per 100 ED encounters (95% CI -5.10 to -2.97; relative reduction of 19.8%). The control measure, CBC, did not significantly change (-0.43, 95% CI -2.83 to 1.96). Individual hospitals showed variable response, with absolute reductions from 2.02 to 9.6 tests per 100 ED encounters for PT/INR (relative reduction 12.1%-30.5%) and 2.07 to 10.04 for aPTT (relative reduction 12.1%-31.4%). Regression analysis showed that the intervention caused an immediate 25.7% decline in PT/INR and 24.7% decline in aPTT tests compared to the control measure. The slope differences (rate of order increase pre vs post intervention) did not significantly decline compared to the control.

CONCLUSIONS

This BPA intervention reduced PT/INR and aPTT use across 11 EDs in a large, urban, safety net system. Further study is needed in implementation to other non-safety net settings.

An Intraoperative Telemedicine Program to Improve Perioperative Quality Measures: The ACTFAST-3 Randomized Clinical Trial

Importance

Telemedicine for clinical decision support has been adopted in many health care settings, but its utility in improving intraoperative care has not been assessed.

Objective

To pilot the implementation of a real-time intraoperative telemedicine decision support program and evaluate whether it reduces postoperative hypothermia and hyperglycemia as well as other quality of care measures.

Design, Setting, and Participants

This single-center pilot randomized clinical trial (Anesthesiology Control Tower-Feedback Alerts to Supplement Treatments [ACTFAST-3]) was conducted from April 3, 2017, to June 30, 2019, at a large academic medical center in the US. A total of 26 254 adult surgical patients were randomized to receive either usual intraoperative care (control group; n = 12 980) or usual care augmented by telemedicine decision support (intervention group; n = 13 274). Data were initially analyzed from April 22 to May 19, 2021, with updates in November 2022 and February 2023.

Intervention

Patients received either usual care (medical direction from the anesthesia care team) or intraoperative anesthesia care monitored and augmented by decision support from the Anesthesiology Control Tower (ACT), a real-time, live telemedicine intervention. The ACT incorporated remote monitoring of operating rooms by a team of anesthesia clinicians with customized analysis software. The ACT reviewed alerts and electronic health record data to inform recommendations to operating room clinicians.

Main Outcomes and Measures

The primary outcomes were avoidance of postoperative hypothermia (defined as the proportion of patients with a final recorded intraoperative core temperature >36 °C) and hyperglycemia (defined as the proportion of patients with diabetes who had a blood glucose level ≤180 mg/dL on arrival to the postanesthesia recovery area). Secondary outcomes included intraoperative hypotension, temperature monitoring, timely antibiotic redosing, intraoperative glucose evaluation and management, neuromuscular blockade documentation, ventilator management, and volatile anesthetic overuse.

Results

Among 26 254 participants, 13 393 (51.0%) were female and 20 169 (76.8%) were White, with a median (IQR) age of 60 (47-69) years. There was no treatment effect on avoidance of hyperglycemia (7445 of 8676 patients [85.8%] in the intervention group vs 7559 of 8815 [85.8%] in the control group; rate ratio [RR], 1.00; 95% CI, 0.99-1.01) or hypothermia (7602 of 11 447 patients [66.4%] in the intervention group vs 7783 of 11 672 [66.7.%] in the control group; RR, 1.00; 95% CI, 0.97-1.02). Intraoperative glucose measurement was more common among patients with diabetes in the intervention group (RR, 1.07; 95% CI, 1.01-1.15), but other secondary outcomes were not significantly different.

Conclusions and Relevance

In this randomized clinical trial, anesthesia care quality measures did not differ between groups, with high confidence in the findings. These results suggest that the intervention did not affect the targeted care practices. Further streamlining of clinical decision support and workflows may help the intraoperative telemedicine program achieve improvement in targeted clinical measures.

Trial Registration

ClinicalTrials.gov Identifier: NCT02830126.

Electronic alert outpatient protocol improves the quality of care for the risk of postcontrast acute kidney injury following computed tomography

BACKGROUND

Prevention and diagnosis of postcontrast acute kidney injury (AKI) after contrast-enhanced computed tomography is burdensome in outpatient department. We investigated whether an electronic alert system could improve prevention and diagnosis of postcontrast AKI.

METHODS

In March 2018, we launched an electronic alert system that automatically identifies patients with a baseline estimated glomerular filtration rate of <45 mL/min/1.73 m2, provides a prescription of fluid regimen, and recommends a follow-up for serum creatinine measurement. Participants prescribed contrast-enhanced computed tomography at outpatient department before and after the launch of the system were categorized as historical and alert group, respectively. Propensity for the surveillance of postcontrast AKI was compared using logistic regression. Risks of AKI, admission, mortality, and renal replacement therapy were analyzed.

RESULTS

The historical and alert groups included 289 and 309 participants, respectively. The alert group was more likely to be men and take diuretics. The most frequent volume of prophylactic fluid in historical and alert group was 1,000 and 750 mL, respectively. Follow-up for AKI was more common in the alert group (adjusted odds ratio, 6.00; p < 0.001). Among them, incidence of postcontrast AKI was not statistically different. The two groups did not differ in risks of admission, mortality, or renal replacement therapy.

CONCLUSION

The electronic alert system could assist in the detection of high-risk patients, prevention with reduced fluid volume, and proper diagnosis of postcontrast AKI, while limiting the prescribing clinicians' burden. Whether the system can improve long-term outcomes remains unclear.

Technology-Assisted Stepped-Care to Promote Adherence in Adolescents with Asthma: A Pilot Study

To examine the feasibility, acceptability, and preliminary efficacy of a technology-assisted stepped-care behavioral intervention to improve adherence in adolescents with asthma. Thirty adolescents (Mage = 14.66, 53% male) with moderate to severe-persistent asthma completed daily adherence monitoring and medication reminders via a mobile app (Step 1). Participants with < 68% adherence during Step 1 received a telehealth behavioral intervention (Step 2). Twenty-six of 30 participants (87%) completed Step 1. Step 2 was indicated for 18 participants and was completed by 17. Participants favorably rated their experience in the study. Improvements in adherence (40-58%, p = .048) and decreases in asthma composite severity scores (CASI 6.08-5.08, p = .023) were observed for the full sample. Technology-assisted stepped-care is feasible and acceptable. Participants demonstrated improved adherence and asthma composite severity scores once they received the appropriate step of the intervention. Future studies should include a control group, a longer time-frame and an intermediate intervention step.

Practical issues in pragmatic trials: the implementation of the Diuretic Comparison Project

BACKGROUND/AIMS

The US Department of Veterans Affairs Point of Care Clinical Trial Program conducts studies that utilize informatics infrastructure to integrate clinical trial protocols into routine care delivery. The Diuretic Comparison Project compared hydrochlorothiazide to chlorthalidone in reduction of major cardiovascular events in subjects with hypertension. Here we describe the cultural, technical, regulatory, and logistical challenges and solutions that enabled successful implementation of this large pragmatic comparative effectiveness Point of Care clinical trial.

METHODS

Patients were recruited from 72 Veterans Affairs Healthcare Systems using centralized processes for subject identification, obtaining informed consent, data collection, safety monitoring, site communication, and endpoint identification with minimal perturbation of the local clinical care ecosystem. Patients continued to be managed exclusively by their clinical care providers without protocol specified study visits, treatment recommendations, or data collection extraneous to routine care. Centralized study processes were operationalized through the application layer of the electronic health record via a data coordinating center staffed by clinical nurses, data scientists, and statisticians without site-based research coordinators. Study data was collected from the Veterans Affairs electronic health record supplemented by Medicare and National Death Index data.

RESULTS

The study exceeded its enrolled goal (13,523 subjects) and followed subjects for the 5-year study duration. The key determinant of program success was collaboration between researchers, regulators, clinicians, and administrative staff at the site level to customize study procedures to align with local clinical practice. This flexibility was enabled by designation of the study as minimal risk and determination that clinical care providers were not engaged in research by the Veterans Affairs Central Institutional Review Board. Cultural, regulatory, technical, and logistical problems were identified and solved through iterative collaboration between clinical and research entities. Paramount among these problems was customization of the Veterans Affairs electronic health record and data systems to accommodate study procedures.

CONCLUSIONS

Leveraging clinical care for large-scale clinical trials is feasible but requires a rethinking of traditional clinical trial design (and regulation) to better meet requirements of clinical care ecosystems. Study designs must accommodate site-specific practice variation to reduce the impact on clinical care. A tradeoff thus exists between designing trial processes tailored to expedite local study implementation versus those to produce a more refined response to the research question. The availability of a uniform and flexible electronic health record in the Department of Veterans Affairs played a major role in the success of the trial. Conducting Point of Care research in other healthcare systems without such research-friendly infrastructure presents a more formidable challenge.

Human-Centered Design of a Clinical Decision Support for Anemia Screening in Children with Inflammatory Bowel Disease

BACKGROUND

Inflammatory bowel disease (IBD) commonly leads to iron deficiency anemia (IDA). Rates of screening and treatment of IDA are often low. A clinical decision support system (CDSS) embedded in an electronic health record could improve adherence to evidence-based care. Rates of CDSS adoption are often low due to poor usability and fit with work processes. One solution is to use human-centered design (HCD), which designs CDSS based on identified user needs and context of use and evaluates prototypes for usefulness and usability.

OBJECTIVES

this study aimed to use HCD to design a CDSS tool called the IBD Anemia Diagnosis Tool, IADx.

METHODS

Interviews with IBD practitioners informed creation of a process map of anemia care that was used by an interdisciplinary team that used HCD principles to create a prototype CDSS. The prototype was iteratively tested with "Think Aloud" usability evaluation with clinicians as well as semi-structured interviews, a survey, and observations. Feedback was coded and informed redesign.

RESULTS

Process mapping showed that IADx should function at in-person encounters and asynchronous laboratory review. Clinicians desired full automation of clinical information acquisition such as laboratory trends and analysis such as calculation of iron deficit, less automation of clinical decision selection such as laboratory ordering, and no automation of action implementation such as signing medication orders. Providers preferred an interruptive alert over a noninterruptive reminder.

CONCLUSION

Providers preferred an interruptive alert, perhaps due to the low likelihood of noticing a noninterruptive advisory. High levels of desire for automation of information acquisition and analysis with less automation of decision selection and action may be generalizable to other CDSSs designed for chronic disease management. This underlines the ways in which CDSSs have the potential to augment rather than replace provider cognitive work.

A pilot study to predict cardiac arrest in the pediatric intensive care unit

BACKGROUND

Cardiac arrest is a leading cause of mortality prior to discharge for children admitted to the pediatric intensive care unit. To address this problem, we used machine learning to predict cardiac arrest up to three hours in advance.

METHODS

Our data consists of 240 Hz ECG waveform data, 0.5 Hz physiological time series data, medications, and demographics from 1,145 patients in the pediatric intensive care unit at the Johns Hopkins Hospital, 15 of whom experienced a cardiac arrest. The data were divided into training, validating, and testing sets, and features were generated every five minutes. 23 heart rate variability (HRV) metrics were determined from ECG waveforms. 96 summary statistics were calculated for 12 vital signs, such as respiratory rate and blood pressure. Medications were classified into 42 therapeutic drug classes. Binary features were generated to indicate the administration of these different drugs. Next, six machine learning models were evaluated: logistic regression, support vector machine, random forest, XGBoost, LightGBM, and a soft voting ensemble.

RESULTS

XGBoost performed the best, with 0.971 auROC, 0.797 auPRC, 99.5% sensitivity, and 69.6% specificity on an independent test set.

CONCLUSION

We have created high-performing models that identify signatures of in-hospital cardiac arrest (IHCA) that may not be evident to clinicians. These signatures include a combination of heart rate variability metrics, vital signs data, and therapeutic drug classes. These machine learning models can predict IHCA up to three hours prior to onset with high performance, allowing clinicians to intervene earlier, improving patient outcomes.

Improving paediatric antimicrobial stewardship in remote and regional Queensland hospitals: development and qualitative evaluation of a tailored intervention for intravenous-to-oral antibiotic switching

OBJECTIVES

Timely intravenous-to-oral antibiotic switching for children is important for paediatric antimicrobial stewardship (AMS). However, low decision-making confidence and fragmentation of patient care can hamper implementation, with difficulties heightened regionally where AMS programmes for children are lacking. The aim of this study was to develop and evaluate user-led creation and implementation of an intervention package for early intravenous-to-oral switching at regional hospitals in Queensland, Australia.

DESIGN

Guided by theory, a four-phase approach was used to: (1) develop multifaceted intervention materials; (2) review materials and their usage through stakeholders; (3) adapt materials based on user-feedback and (4) qualitatively evaluate health workers experiences at 6 months postintervention.

SETTING

Seven regional hospitals in Queensland, Australia.

PARTICIPANTS

Phase 2 included 15 stakeholders; health workers and patient representatives (patient-guardians and Indigenous liaison officers). Phase 4 included 20 health workers across the seven intervention sites.

RESULTS

Content analysis of health worker and parent/guardian reviews identified the 'perceived utility of materials' and 'possible barriers to use'. 'Recommendations and strategies for improvement' provided adjustments for the materials that were able to be tailored to individual practice. Postintervention interviews generated three overarching themes that combined facilitators and barriers to switching: (1) application of materials, (2) education and support, and (3) team dynamics. Overall, despite difficulties with turnover and problems with the medical hierarchy, interventions aided and empowered antibiotic therapy decision-making and enhanced education and self-reflection.

CONCLUSIONS

Despite structural barriers to AMS for switching from intravenous-to-oral antibiotics in paediatric patients, offering a tailored multifaceted intervention was reported to provide support and confidence to adjust practice across a diverse set of health workers in regional areas. Future AMS activities should be guided by users and provide opportunities for tailoring tools to practice setting and patients' requirements.

Usability and Utility of Human Immunodeficiency Virus Pre-exposure Prophylaxis Clinical Decision Support to Increase Knowledge and Pre-exposure Prophylaxis Initiations among Pediatric Providers

OBJECTIVES

 An effective clinical decision support system (CDSS) may address the current provider training barrier to offering preexposure prophylaxis (PrEP) to youth at risk for human immunodeficiency virus (HIV) infection. This study evaluated change in provider knowledge and the likelihood to initiate PrEP after exposure to a PrEP CDSS. A secondary objective explored perceived provider utility of the CDSS and suggestions for improving CDSS effectiveness.

METHODS

 This was a prospective study using survey responses from a convenience sample of pediatric providers who launched the interruptive PrEP CDSS when ordering an HIV test. McNemar's test evaluated change in provider PrEP knowledge and likelihood to initiate PrEP. Qualitative responses on CDSS utility and suggested improvements were analyzed using framework analysis and were connected to quantitative analysis elements using the merge approach.

RESULTS

 Of the 73 invited providers, 43 had available outcome data and were included in the analysis. Prior to using the CDSS, 86% of participants had never been prescribed PrEP. Compared to before CDSS exposure, there were significant increases in the proportion of providers who were knowledgeable about PrEP (p = 0.0001), likely to prescribe PrEP (p < 0.0001) and likely to refer their patient for PrEP (p < 0.0001). Suggestions for improving the CDSS included alternative "triggers" for the CDSS earlier in visit workflows, having a noninterruptive CDSS, additional provider educational materials, access to patient-facing PrEP materials, and additional CDSS support for adolescent confidentiality and navigating financial implications of PrEP.

CONCLUSION

 Our findings suggest that an interruptive PrEP CDSS attached to HIV test orders can be an effective tool to increase knowledge and likelihood to initiate PrEP among pediatric providers. Continual improvement of the PrEP CDSS based on provider feedback is required to optimize usability, effectiveness, and adoption. A highly usable PrEP CDSS may be a powerful tool to close the gap in youth PrEP access and uptake.

Embedded point of care randomisation for evaluating comparative effectiveness questions: PROSPECTOR-critical care feasibility study protocol

INTRODUCTION

Many routinely administered treatments lack evidence as to their effectiveness. When treatments lack evidence, patients receive varying care based on the preferences of clinicians. Standard randomised controlled trials are unsuited to comparisons of different routine treatment strategies, and there remains little economic incentive for change.Integrating clinical trial infrastructure into electronic health record systems offers the potential for routine treatment comparisons at scale, through reduced trial costs. To date, embedded trials have automated data collection, participant identification and eligibility screening, but randomisation and consent remain manual and therefore costly tasks.This study will investigate the feasibility of using computer prompts to allow flexible randomisation at the point of clinical decision making. It will compare the effectiveness of two prompt designs through the lens of a candidate research question-comparing liberal or restrictive magnesium supplementation practices for critical care patients. It will also explore the acceptability of two consent models for conducting comparative effectiveness research.

METHODS AND ANALYSIS

We will conduct a single centre, mixed-methods feasibility study, aiming to recruit 50 patients undergoing elective surgery requiring postoperative critical care admission. Participants will be randomised to either 'Nudge' or 'Preference' designs of electronic point-of-care randomisation prompt, and liberal or restrictive magnesium supplementation.We will judge feasibility through a combination of study outcomes. The primary outcome will be the proportion of prompts displayed resulting in successful randomisation events (compliance with the allocated magnesium strategy). Secondary outcomes will evaluate the acceptability of both prompt designs to clinicians and ascertain the acceptability of pre-emptive and opt-out consent models to patients.

ETHICS AND DISSEMINATION

This study was approved by Riverside Research Ethics Committee (Ref: 21/LO/0785) and will be published on completion.

TRIAL REGISTRATION NUMBER

NCT05149820.

A Qualitative Description of Clinician Free-Text Rationales Entered within Accountable Justification Interventions

BACKGROUND

Requiring accountable justifications-visible, clinician-recorded explanations for not following a clinical decision support (CDS) alert-has been used to steer clinicians away from potentially guideline-discordant decisions. Understanding themes from justifications across clinical content areas may reveal how clinicians rationalize decisions and could help inform CDS alerts.

METHODS

We conducted a qualitative evaluation of the free-text justifications entered by primary care physicians from three pilot interventions designed to reduce opioid prescribing and, in older adults, high-risk polypharmacy and overtesting. Clinicians encountered alerts when triggering conditions were met within the chart. Clinicians were asked to change their course of action or enter a justification for the action that would be displayed in the chart. We extracted all justifications and grouped justifications with common themes. Two authors independently coded each justification and resolved differences via discussion. Three physicians used a modified Delphi technique to rate the clinical appropriateness of the justifications.

RESULTS

There were 560 justifications from 50 unique clinicians. We grouped these into three main themes used to justify an action: (1) report of a particular diagnosis or symptom (e.g., for "anxiety" or "acute pain"); (2) provision of further contextual details about the clinical case (e.g., tried and failed alternatives, short-term supply, or chronic medication); and (3) noting communication between clinician and patient (e.g., "risks and benefits discussed"). Most accountable justifications (65%) were of uncertain clinical appropriateness.

CONCLUSION

Most justifications clinicians entered across three separate clinical content areas fit within a small number of themes, and these common rationales may aid in the design of effective accountable justification interventions. Justifications varied in terms of level of clinical detail. On their own, most justifications did not clearly represent appropriate clinical decision making.

Early diagnosis of symptomatic ovarian cancer in primary care in the UK: opportunities and challenges

Background:

Ovarian cancer is the sixth most common cause of cancer-related death in the UK amongst women. Ovarian cancer presents particular challenges for general practitioners (GPs) to diagnose due to its rarity and presentation with non-specific symptoms.

Methods:

A narrative overview of the literature was conducted by searching PubMed and Researchgate for relevant articles, using keywords such as “ovarian cancer,” “primary care” and “diagnosis.”

Results and Discussion:

Studies have shown that in the UK, GPs have a lower readiness to refer and investigate potential cancer symptoms compared with their international counterparts; and this has been correlated with reduced survival. Early diagnosis can be facilitated through a people-focussed and system-based approach which involves both educating GPs and using risk algorithms, rapid diagnostic centres/multi-disciplinary centres and being data-driven through the identification of best practice from national audits. Further research is required into the best evidence-based early investigations for ovarian cancer and more effective biomarkers.

Using the Electronic Health Record User Context in Clinical Decision Support Criteria

BACKGROUND

Computerized clinical decision support (CDS) used in electronic health record systems (EHRs) has led to positive outcomes as well as unintended consequences, such as alert fatigue. Characteristics of the EHR session can be used to restrict CDS tools and increase their relevance, but implications of this approach are not rigorously studied.

OBJECTIVES

To assess the utility of using "login location" of EHR users-that is, the location they chose on the login screen-as a variable in the CDS logic.

METHODS

We measured concordance between user's login location and the location of the patients they placed orders for and conducted stratified analyses by user groups. We also estimated how often login location data may be stale or inaccurate.

RESULTS

One in five CDS alerts incorporated the EHR users' login location into their logic. Analysis of nearly 2 million orders placed by nearly 8,000 users showed that concordance between login location and patient location was high for nurses, nurse practitioners, and physician assistance (all >95%), but lower for fellows (77%) and residents (55%). When providers switched between patients in the EHR, they usually did not update their login location accordingly.

CONCLUSION

CDS alerts commonly incorporate user's login location into their logic. User's login location is often the same as the location of the patient the user is providing care for, but substantial discordance can be observed for certain user groups. While this may provide additional information that could be useful to the CDS logic, a substantial amount of discordance happened in specific user groups or when users appeared not to change their login location across different sessions. Those who design CDS alerts should consider a data-driven approach to evaluate the appropriateness of login location for each use case.

Digital health interventions for osteoporosis and post-fragility fracture care

The growing burden from osteoporosis and fragility fractures highlights a need to improve osteoporosis management across healthcare systems. Sub-optimal management of osteoporosis is an area suitable for digital health interventions. While fracture liaison services (FLSs) are proven to greatly improve care for people with osteoporosis, such services might benefit from technologies that enhance automation. The term 'Digital Health' covers a variety of different tools including clinical decision support systems, electronic medical record tools, patient decision aids, patient apps, education tools, and novel artificial intelligence (AI) algorithms. Within the scope of this review are AI solutions that use algorithms within health system registries to target interventions. Clinician-targeted, patient-targeted, or system-targeted digital health interventions could be used to improve management and prevent fragility fractures. This review was commissioned by The Royal Osteoporosis Society and Bone Research Academy during the production of the 2020 Research Roadmap (https://theros.org.uk), with the intention of identifying gaps where targeted research funding could lead to improved patient health. We explore potential uses of digital technology in the general management of osteoporosis. Evidence suggests that digital technologies can support multidisciplinary teams to provide the best possible patient care based on current evidence and to support patients in self-management. However, robust randomised controlled studies are still needed to assess the effectiveness and cost-effectiveness of these technologies.

Emerging digital technologies in cancer treatment, prevention, and control

The very first issue of the journal of Translational Behavioral Medicine (TBM) was dedicated, in part, to the theme of Health Information Technology as a platform for evidence implementation. The topic was timely: legislation in the USA was passed with the intent of stimulating the adoption of electronic health records; mobile smartphones, tablets, and other devices were gaining traction in the consumer market, while members within the Society of Behavioral Medicine were gaining scientific understanding on how to use these tools to effect healthy behavior change. For the anniversary issue of TBM, we evaluated the progress and problems associated with deploying digital health technologies to support cancer treatment, prevention, and control over the last decade. We conducted a narrative review of published literature to identify the role that emerging digital technologies may take in achieving national and international objectives in the decade to come. We tracked our evaluation of the literature across three phases in the cancer control continuum: (a) prevention, (b) early detection/screening, and (c) treatment/survivorship. From our targeted review and analyses, we noted that significant progress had been made in the adoption of digital health technologies in the cancer space over the past decade but that significant work remains to be done to integrate these technologies effectively into the cancer control systems needed to improve outcomes equitably across populations. The challenge for the next 10 years is inherently translational.

Reflex factor coagulation testing in patients with an unexplained prolonged aPTT: An institutional retrospective review

BACKGROUND

We aim to determine the clinical utility of reflex coagulation investigations (RCI) for prolonged lupus insensitive activated partial thromboplastin time (aPTT) at our institution.

METHODS

We retrospectively reviewed all potential RCI (lupus insensitive aPTT of ≥32s) from April 2014 to June 2019. Our diagnostic algorithm requires completion of RCI only if samples had no interfering medications to explain a prolonged aPTT and were either from a preoperative sample or from a patient presenting with unexplained bleeding. Appropriate RCI samples undergo further investigations with one-stage factor activity testing for factors 8(FVIII), 9(FIX), and 11(FXI) reflexively. Data were obtained through electronic medical records to capture clinical characteristics, laboratory findings, prophylactic hemostatic replacement, and bleeding outcomes.

RESULTS

Three thousand and three hundreds seventeen samples from 2940 distinct patients were considered as potential RCI during the study period. 263/3317 (8%) samples had RCI completed. Of those, 55/263 (21%) had abnormal factor testing, with the majority from preoperative setting (43/55; 78%). 5/43 (12%) patients were referred to hematology for preoperative evaluation. 5/43 patients received preoperative hemostatic support. A total of 5 patients (5/43) developed postop bleeding. Six patients (6/55) had RCI for unexplained bleeding, and five patients (83%) had a newly identified clinically significant bleeding disorder.

CONCLUSION

Reflex coagulation investigations benefited patients presenting with unexplained bleeding as this expedited the diagnosis and management of clinically significant bleeding disorders. RCI for preoperative evaluation infrequently led to additional hemostatic support/referral to hematology. The lack of additional workup for an abnormal factor activity level suggests laboratory alert fatigue as a potential contributory factor.

Using machine learning to predict severe hypoglycaemia in hospital

AIM

To predict the risk of hypoglycaemia using machine-learning techniques in hospitalized patients.

METHODS

We conducted a retrospective cohort study of patients hospitalized under general internal medicine (GIM) and cardiovascular surgery (CV) at a tertiary care teaching hospital in Toronto, Ontario. Three models were generated using supervised machine learning: least absolute shrinkage and selection operator (LASSO) logistic regression; gradient-boosted trees; and a recurrent neural network. Each model included baseline patient data and time-varying data. Natural-language processing was used to incorporate text data from physician and nursing notes.

RESULTS

We included 8492 GIM admissions and 8044 CV admissions. Hypoglycaemia occurred in 16% of GIM admissions and 13% of CV admissions. The area under the curve for the models in the held-out validation set was approximately 0.80 on the GIM ward and 0.82 on the CV ward. When the threshold for hypoglycaemia was lowered to 2.9 mmol/L (52 mg/dL), similar results were observed. Among the patients at the highest decile of risk, the positive predictive value was approximately 50% and the sensitivity was 99%.

CONCLUSION

Machine-learning approaches can accurately identify patients at high risk of hypoglycaemia in hospital. Future work will involve evaluating whether implementing this model with targeted clinical interventions can improve clinical outcomes.

Impact of electronic medical records and COVID-19 on adult Goals-of-Care document completion and revision in hospitalised general medicine patients

BACKGROUND

Conversion from paper-based to electronic medical records (EMR) may affect the quality and timeliness of the completion of Goals-Of-Care (GOC) documents during hospital admissions, and the COVID-19 pandemic may have further impacted this.

AIMS

Determine the impact of EMR and COVID-19 on the proper completion of GOC forms, and the factors associated with inpatient changes in GOC.

METHODS

We conducted a cross-sectional study of adult general medicine admissions (Aug 2018-Sep 2020) at Dandenong Hospital (Victoria, Australia). We used interrupted time series to model the changes in the rates of proper GOC completion (adequate documented discussion, completed ≤2 days) after the introduction of EMR and arrival of COVID-19.

RESULTS

We included a total of 5147 patients. The pre-EMR GOC proper completion rate was 27.7% (overall completion, 86.5%). There was a decrease in the proper completion rate by 2.21% per month (95% CI: -2.83%, -1.58%) after EMR implementation despite an increase in overall completion rates (91.2%). The main reason for the negative trend was a decline in adequate documentation despite improvements in timeliness. COVID-19 arrival saw a reversal of this negative trend, with proper completion rates increasing by 2.25% per month (95% CI: 1.35%, 3.15%) compared to the EMR period, but also resulted in a higher proportion GOC changes within 2 days of admission.

CONCLUSION

EMR improved the timeliness and overall completion rates of GOC at the cost of a lower quality of documented discussion. COVID-19 reversed the negative trend in proper GOC completion but increased the number of early revisions. This article is protected by copyright. All rights reserved.

Uncovering Barriers to Screening for Distress in Patients With Cancer via Machine Learning

BACKGROUND

Psychologic distress and manifest mental disorders are overlooked in 30-50% of patients with cancer. Accordingly, international cancer treatment guidelines recommend routine screening for distress in order to provide psychologic support to those in need. Yet, institutional and patient-related factors continue to hinder implementation.

OBJECTIVE

This study aims to investigate factors, which are associated with no screening for distress in patients with cancer.

METHODS

Using machine learning, factors associated with lack of distress screening were explored in 6491 patients with cancer between 2011 and 2019 at a large cancer treatment center. Parameters were hierarchically ordered based on statistical relevance. Nested resampling and cross validation were performed to avoid overfitting and to comply with assumptions for machine learning approaches.

RESULTS

Patients unlikely to be screened were not discussed at a tumor board, had inpatient treatment of less than 28 days, did not consult with a psychiatrist or clinical psychologist, had no (primary) nervous system cancer, no head and neck cancer, and did have breast or skin cancer. The final validated model was optimized to maximize sensitivity at 83.9%, and achieved a balanced accuracy of 68.9, area under the curve of 0.80, and specificity of 53.9%.

CONCLUSION

Findings of this study may be relevant to stakeholders at both a clinical and institutional level in order to optimize distress screening rates.

Bridging the Chasm between Pregnancy and Health over the Life Course: A National Agenda for Research and Action

BACKGROUND

Many pregnant people find no bridge to ongoing specialty or primary care after giving birth, even when clinical and social complications of pregnancy signal need. Black, indigenous, and all other women of color are especially harmed by fragmented care and access disparities, coupled with impacts of racism over the life course and in health care.

METHODS

We launched the initiative "Bridging the Chasm between Pregnancy and Health across the Life Course" in 2018, bringing together patients, advocates, providers, researchers, policymakers, and systems innovators to create a National Agenda for Research and Action. We held a 2-day conference that blended storytelling, evidence analysis, and consensus building to identify key themes related to gaps in care and root causes of inequities. In 2019, more than 70 stakeholders joined six working groups to reach consensus on strategic priorities based on equity, innovation, effectiveness, and feasibility.

FINDINGS

Working groups identified six key strategic areas for bridging the chasm. These include: 1) progress toward eliminating institutional and interpersonal racism and bias as a requirement for accreditation of health care institutions, 2) infrastructure support for community-based organizations, 3) extension of holistic team-based care to the postpartum year and beyond, with integration of doulas and community health workers on the team, 4) extension of Medicaid coverage and new quality and pay-for-performance metrics to link maternity care and primary care, 5) systems to preserve maternal narratives and data across providers, and 6) alignment of research with women's lived experiences.

CONCLUSIONS

The resulting agenda presents a path forward to remedy the structural chasms in women's health care, with key roles for advocates, policymakers, researchers, health care leaders, educators, and the media.

A machine learning-based clinical decision support system to identify prescriptions with a high risk of medication error

Objective

To improve patient safety and clinical outcomes by reducing the risk of prescribing errors, we tested the accuracy of a hybrid clinical decision support system in prioritizing prescription checks.

Materials and Methods

Data from electronic health records were collated over a period of 18 months. Inferred scores at a patient level (probability of a patient’s set of active orders to require a pharmacist review) were calculated using a hybrid approach (machine learning and a rule-based expert system). A clinical pharmacist analyzed randomly selected prescription orders over a 2-week period to corroborate our findings. Predicted scores were compared with the pharmacist’s review using the area under the receiving-operating characteristic curve and area under the precision-recall curve. These metrics were compared with existing tools: computerized alerts generated by a clinical decision support (CDS) system and a literature-based multicriteria query prioritization technique. Data from 10 716 individual patients (133 179 prescription orders) were used to train the algorithm on the basis of 25 features in a development dataset.

Results

While the pharmacist analyzed 412 individual patients (3364 prescription orders) in an independent validation dataset, the areas under the receiving-operating characteristic and precision-recall curves of our digital system were 0.81 and 0.75, respectively, thus demonstrating greater accuracy than the CDS system (0.65 and 0.56, respectively) and multicriteria query techniques (0.68 and 0.56, respectively).

Discussion

Our innovative digital tool was notably more accurate than existing techniques (CDS system and multicriteria query) at intercepting potential prescription errors.

Conclusions

By primarily targeting high-risk patients, this novel hybrid decision support system improved the accuracy and reliability of prescription checks in a hospital setting.

Factors shaping the implementation and use of Clinical Cancer Decision Tools by GPs in primary care: a qualitative framework synthesis

OBJECTIVE

Clinical Cancer Decision Tools (CCDTs) aim to alert general practitioners (GPs) to signs and symptoms of cancer, supporting prompt investigation and onward referral. CCDTs are available in primary care in the UK but are not widely utilised. Qualitative research has highlighted the complexities and mechanisms surrounding their implementation and use; this has focused on specific cancer types, formats, systems or settings. This study aims to synthesise qualitative data of GPs' attitudes to and experience with a range of CCDTs to gain better understanding of the factors shaping their implementation and use.

DESIGN

A systematic search of the published (MEDLINE, CINAHL, Web of Science and EMBASE) and grey literature (July 2020). Following screening, selection and assessment of suitability, the data were analysed and synthesised using normalisation process theory.

RESULTS

Six studies (2011 to 2019), exploring the views of GPs were included for analysis. Studies focused on the use of several different types of CCDTs (Risk Assessment Tools (RAT) or electronic version of RAT (eRAT), QCancer and the 7-point checklist). GPs agreed CCDTs were useful to increase awareness of signs and symptoms of undiagnosed cancer. They had concerns about the impact on trust in their own clinical acumen, whether secondary care clinicians would consider referrals generated by CCDT as valid and whether integration of the CCDTs within existing systems was achievable.

CONCLUSIONS

CCDTs might be a helpful adjunct to clinical work in primary care, but without careful development to legitimise their use GPs are likely to give precedence to clinical acumen and gut instinct. Stakeholder consultation with secondary care clinicians and consideration of how the CCDTs fit into a GP consultation are crucial to successful uptake. The role and responsibilities of a GP as a clinician, gatekeeper, health promoter and resource manager affect the interaction with and implementation of innovations such as CCDTs.

Towards The Automated, Empirical Filtering of Drug-Drug Interaction Alerts in Clinical Decision Support Systems: Historical Cohort Study of Vitamin K Antagonists

Background

Drug-drug interactions (DDIs) involving vitamin K antagonists (VKAs) constitute an important cause of in-hospital morbidity and mortality. However, the list of potential DDIs is long; the implementation of all these interactions in a clinical decision support system (CDSS) results in over-alerting and alert fatigue, limiting the benefits provided by the CDSS.

Objective

To estimate the probability of occurrence of international normalized ratio (INR) changes for each DDI rule, via the reuse of electronic health records.

Methods

An 8-year, exhaustive, population-based, historical cohort study including a French community hospital, a group of Danish community hospitals, and a Bulgarian hospital. The study database included 156,893 stays. After filtering against two criteria (at least one VKA administration and at least one INR laboratory result), the final analysis covered 4047 stays. Exposure to any of the 145 drugs known to interact with VKA was tracked and analyzed if at least 3 patients were concerned. The main outcomes are VKA potentiation (defined as an INR≥5) and VKA inhibition (defined as an INR≤1.5). Groups were compared using the Fisher exact test and logistic regression, and the results were expressed as an odds ratio (95% confidence limits).

Results

The drugs known to interact with VKAs either did not have a statistically significant association regarding the outcome (47 drug administrations and 14 discontinuations) or were associated with significant reduction in risk of its occurrence (odds ratio<1 for 18 administrations and 21 discontinuations).

Conclusions

The probabilities of outcomes obtained were not those expected on the basis of our current body of pharmacological knowledge. The results do not cast doubt on our current pharmacological knowledge per se but do challenge the commonly accepted idea whereby this knowledge alone should be used to define when a DDI alert should be displayed. Real-life probabilities should also be considered during the filtration of DDI alerts by CDSSs, as proposed in SPC-CDSS (statistically prioritized and contextualized CDSS). However, these probabilities may differ from one hospital to another and so should probably be calculated locally.

Implementation of Electronic Decision Support for Diabetic Care in a Student-Run Clinic

Background and objectives Type 2 diabetes mellitus (T2DM) is a complex disease that can lead to complications. Electronic decision support in the electronic medical record (EMR) aids management. There is no study demonstrating the effectiveness of electronic decision support in assisting medical student providers in student-run free clinics. Methods There were 71 T2DM patients seen by medical students. Twenty-three encounters used a Diabetes Progress Note (DPN) that was created from consensus, opinion-based guidelines. Each note received a total composite score based on an eight-point scale for adherence to guidelines. Statistical comparisons between mean composite scores were performed using independent t-tests. Results The mean total composite score of DPN users was significantly greater than DPN non-users (5.35 vs. 4.23, p = 0.008), with a significant difference in the physical exam component (1.70 vs. 1.31, p = 0.002). Conclusions In this exploratory study, medical student providers at an attending-supervised, student-run free clinic that used electronic decision support during T2DM patient visits improved adherence to screening for diabetic complications and standard of care.

Implementation of Electronic Decision Support for Diabetic Care in a Student-Run Clinic

Background and objectives Type 2 diabetes mellitus (T2DM) is a complex disease that can lead to complications. Electronic decision support in the electronic medical record (EMR) aids management. There is no study demonstrating the effectiveness of electronic decision support in assisting medical student providers in student-run free clinics. Methods There were 71 T2DM patients seen by medical students. Twenty-three encounters used a Diabetes Progress Note (DPN) that was created from consensus, opinion-based guidelines. Each note received a total composite score based on an eight-point scale for adherence to guidelines. Statistical comparisons between mean composite scores were performed using independent t-tests. Results The mean total composite score of DPN users was significantly greater than DPN non-users (5.35 vs. 4.23, p = 0.008), with a significant difference in the physical exam component (1.70 vs. 1.31, p = 0.002). Conclusions In this exploratory study, medical student providers at an attending-supervised, student-run free clinic that used electronic decision support during T2DM patient visits improved adherence to screening for diabetic complications and standard of care.

Analysis of the cognitive demands of electronic health record use

The goal of this study was to elicit the cognitive demands facing clinicians when using an electronic health record (EHR) system and learn the cues and strategies expert clinicians rely on to manage those demands. This study differs from prior research by applying a joint cognitive systems perspective to examining the cognitive aspects of clinical work. We used a cognitive task analysis (CTA) method specifically tailored to elicit the cognitive demands of an EHR system from expert clinicians from different sites in a variety of inpatient and outpatient roles. The analysis of the interviews revealed 145 unique cognitive demands of using an EHR, which were organized into 22 distinct themes across seven broad categories. In addition to confirming previously published themes of cognitive demands, the main emergent themes of this study are: 1) The EHR does not help clinicians develop and maintain awareness of the big picture; 2) The EHR does not support clinicians' need to reason about patients' current and future states, including effects of potential treatments; and 3) The EHR limits agency of clinicians to work individually and collaboratively. Implications for theory and EHR design and evaluation are discussed.

Mobile Health and Inhaler-Based Monitoring Devices for Asthma Management

Mobile health and web applications (apps), wearables, and other personal monitoring devices have tremendous potential to improve the management of asthma. More than 500 asthma-related apps, whether standalone or paired with sensors on inhalers, are currently available for health education, symptom recording, tracking of inhaler use, displaying environmental alerts, and providing medication reminders. Benefits of these tools include the ability to longitudinally collect symptom, trigger, and inhaler usage data, allowing the detection of significant changes over time to help patients and their caregivers determine whether symptoms are worsening. In addition, data from external information sources, including weather, allergen, and air quality reports, can be integrated with user-specific data to enhance predictions on when patients may experience symptoms and/or need to avoid triggers. Barriers to adoption of asthma-related apps and inhaler-based devices include uncertain efficacy and effectiveness, potential high cost, sustained user engagement, and concerns about privacy. Moreover, ensuring the acceptability and utility of asthma management apps for individuals of all races/ethnicities, socioeconomic groups, ages, genders, and literacy levels is necessary. Based on studies thus far, mobile health apps and inhaler-based devices have great potential to serve as useful tools in the patient-doctor relationship and revolutionize asthma care.

Impact of Electronic Health Record Interface Design on Unsafe Prescribing of Ciclosporin, Tacrolimus, and Diltiazem: Cohort Study in English National Health Service Primary Care

BACKGROUND

In England, national safety guidance recommends that ciclosporin, tacrolimus, and diltiazem are prescribed by brand name due to their narrow therapeutic windows and, in the case of tacrolimus, to reduce the chance of organ transplantation rejection. Various small studies have shown that changes to electronic health record (EHR) system interfaces can affect prescribing choices.

OBJECTIVE

Our objectives were to assess variation by EHR systems in breach of safety guidance around prescribing of ciclosporin, tacrolimus, and diltiazem, and to conduct user-interface research into the causes of such breaches.

METHODS

We carried out a retrospective cohort study using prescribing data in English primary care. Participants were English general practices and their respective EHR systems. The main outcome measures were (1) the variation in ratio of safety breaches to adherent prescribing in all practices and (2) the description of observations of EHR system usage.

RESULTS

A total of 2,575,411 prescriptions were issued in 2018 for ciclosporin, tacrolimus, and diltiazem (over 60 mg); of these, 316,119 prescriptions breached NHS guidance (12.27%). Breaches were most common among users of the EMIS EHR system (breaches in 18.81% of ciclosporin and tacrolimus prescriptions and in 17.99% of diltiazem prescriptions), but breaches were observed in all EHR systems.

CONCLUSIONS

Design choices in EHR systems strongly influence safe prescribing of ciclosporin, tacrolimus, and diltiazem, and breaches are prevalent in general practices in England. We recommend that all EHR vendors review their systems to increase safe prescribing of these medicines in line with national guidance. Almost all clinical practice is now mediated through an EHR system; further quantitative research into the effect of EHR system design on clinical practice is long overdue.

Applying population health approaches to improve safe anticoagulant use in the outpatient setting: the DOAC Dashboard multi-cohort implementation evaluation study protocol

BACKGROUND

Use of direct oral anticoagulants (DOAC) is rapidly growing for treatment of atrial fibrillation and venous thromboembolism. However, incorrect dosing of these medications is common and puts patients at risk of adverse drug events. One way to improve safe prescribing is the use of population health tools, including interactive dashboards built into the electronic health record (EHR). As such tools become more common, exploring ways to understand which aspects are effective in specific settings and how to effectively adapt and implement in existing anticoagulation clinics across different health systems is vital.

METHODS

This three-phase project will evaluate a current nation-wide implementation effort of the DOAC Dashboard in the Veterans Health Administration (VHA) using both quantitative and qualitative methods. Informed by this evaluation, the DOAC Dashboard will be implemented in four new health systems using an implementation strategy derived from the VHA experience and interviews with providers in those new health systems. Quantitative evaluation of the VHA and non-VHA implementation will follow the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework. Qualitative interviews with stakeholders will be analyzed using the Consolidated Framework for Implementation Research and Technology Acceptance Models to identify key determinants of implementation success.

DISCUSSION

This study will (1) evaluate the implementation of an EHR-based population health tool for medication management within a large, nation-wide, highly integrated health system; (2) guide the adoption in a set of four different health systems; and (3) evaluation that multi-center implementation effort. These findings will help to inform future EHR-based implementation efforts in a wide variety of health care settings.

Triggered Palliative Care Consults: A Systematic Review of Interventions for Hospitalized and Emergency Department Patients

CONTEXT

Palliative care improves the quality of care and may reduce utilization, but delays or the absences of such services are common and costly in inpatient and emergency department settings. Triggered palliative care consults (PCCs) offer one way to identify patients who would benefit from palliative care and to connect them with services early in their course. Consensus reports recommend use of triggers to identify patients for PCC, but no standards exist to guide trigger design or implementation.

OBJECTIVES

To conduct a systematic review of published trigger tools for PCC.

METHODS

Studies included quality improvement and prospective analyses of triggers for PCC for adults in the emergency department and inpatient settings since 2008. Paired reviewers evaluated the studies for inclusion criteria and extracted data related to study demographics, trigger processes, trigger criteria, and study bias.

RESULTS

The search yielded 5773 citations. Twenty studies were included for final analysis with more than 17,000 patients represented. Trigger processes and composition were heterogeneous, although frequently used categories, such as cancer, dementia, and chronic comorbidities, were identified. Three-quarters of the studies were deemed to have moderate or high risk of bias.

CONCLUSION

We present a range of trigger tools spanning different hospital settings and patient populations. Common themes in implementation and content arose, but the limitations of these studies are notable, and further rigorous randomized comparisons are needed to generate standards of care. In addition, future studies should focus on developing triggers that identify patients requiring primary-level vs. specialty-level palliative care.

Medical Decision Support to Reduce Unwarranted Methylene Tetrahydrofolate Reductase (MTHFR) Genetic Testing

Most major national medical associations have advised against routine MTHFR testing since at least 2013. However, many providers continue to order this unwarranted genetic test. This study assessed the efficacy of an electronic best practice alert to aid ordering providers. We tracked the rate of MTHFR tests ordered per 1 million patients in the twelve months before and after the implementation of an alert that suggested an alternative test. Associated factors including the ordering department, diagnosis, patient sex, and patient age were also analyzed. Chi square analysis was used to compare the difference between pre- and post-alert test ordering rates. A total of 997 MTHFR analysis were ordered in Southern California Kaiser Permanente from January 2017 through December 2018. Overall, the average MTHFR monthly test ordering rates dropped significantly from 12.93 per million patients in 2017 to 7.08 per million patients in 2018 (p = 0.0056). However, testing rates in children were unchanged and, in some associated diagnoses, such as psychiatric illnesses and neurodevelopmental conditions, the testing rates increased. Recommending an alternate test in lieu of the unwarranted one significantly reduced the overall rate of MTHFR testing. The alert was most effective for specialties and diagnoses where MTHFR was historically medically indicated. This suggests such alerts are an effective intervention that health care systems can implement to serve as an educational update and to reduce unwarranted genetic testing.

Implementation of clinical decision support to manage acute kidney injury in secondary care: an ethnographic study

BACKGROUND

Over the past decade, acute kidney injury (AKI) has become a global priority for improving patient safety and health outcomes. In the UK, a confidential inquiry into AKI led to the publication of clinical guidance and a range of policy initiatives. National patient safety directives have focused on the mandatory establishment of clinical decision support systems (CDSSs) within all acute National Health Service (NHS) trusts to improve the detection, alerting and response to AKI. We studied the organisational work of implementing AKI CDSSs within routine hospital care.

METHODS

An ethnographic study comprising non-participant observation and interviews was conducted in two NHS hospitals, delivering AKI quality improvement programmes, located in one region of England. Three researchers conducted a total of 49 interviews and 150 hours of observation over an 18-month period. Analysis was conducted collaboratively and iteratively around emergent themes, relating to the organisational work of technology adoption.

RESULTS

The two hospitals developed and implemented AKI CDSSs using very different approaches. Nevertheless, both resulted in adaptive work and trade-offs relating to the technology, the users, the organisation and the wider system of care. A common tension was associated with attempts to maximise benefit while minimise additional burden. In both hospitals, resource pressures exacerbated the tensions of translating AKI recommendations into routine practice.

CONCLUSIONS

Our analysis highlights a conflicted relationship between external context (policy and resources), and organisational structure and culture (eg, digital capability, attitudes to quality improvement). Greater consideration is required to the long-term effectiveness of the approaches taken, particularly in light of the ongoing need for adaptation to incorporate new practices into routine work.

The problem of look-alike, sound-alike name errors: Drivers and solutions

Look-alike or sound-alike (LASA) medication names may be mistaken for each other, e.g. mercaptamine and mercaptopurine. If an error of this sort is not intercepted, it can reach the patient and may result in harm. LASA errors occur because of shared linguistic properties between names (phonetic or orthographic), and potential for error is compounded by similar packaging, tablet appearance, tablet strength, route of administration or therapeutic indication. Estimates of prevalence range from 0.00003 to 0.0022% of all prescriptions, 7% of near misses, and between 6.2 and 14.7% of all medication error events. Solutions to LASA errors can target people or systems, and include reducing interruptions or distractions during medication administration, typographic tweaks, such as selective capitalization (Tall Man letters) or boldface, barcoding, and computerized physician order entry.

Development and efficacy of a computerized decision support system for osteoporosis management in the community

Computerized alerts for primary care physicians, provided during visits of patients who met treatment guidelines based on their electronic medical records, are an efficient method to raise awareness to many otherwise missed cases, especially after fracture.

PURPOSE

Measure the efficacy of an automated real-time alert which was developed to assist osteoporosis management in the community.

METHODS

The study population included treatment naïve patients with T-score ≤ - 2.5 or hip or vertebral fracture in a 2 million member Israeli health fund. On each ambulatory visit to a primary care physician or endocrinologist, a pop-up screen reminded the caregiver to consider treatment initiation. A follow-up "smart-set" screen conveniently gathered links to common actions (namely, (a) issue first line therapy prescription, (b) referral to nutritionist consultation, (c) laboratory tests relevant for osteoporosis, and (d) printing an information page for the patient). Time till treatment initiation was compared between the 3 years prior to and following the intervention.

RESULTS

Within 2 years since alert activation, a total of n = 21,070 cases were alerted, 52% of which were long standing cases: untreated for over 6 months since the event. During this period, a total of 30% initiated treatment purchases. As compared with the 3 years prior to the intervention, time till treatment initiation decreased following the intervention with HR = 1.05, 1.94, 1.29 (p values = 0.020, < 0.001, 0.005) for T-score, hip, and vertebral cases respectively. Initiation rates within 6 months increased from 52.0 to 59.8%, from 12.3 to 27.7%, and from 17.4 to 27.1% among T-score, hip, and vertebral cases, respectively (p value < 0.001). Male sex, nursing home residence, having diabetes or a cardiovascular disease and age younger than 60 or older than 80 were associated with lower treatment rates.

CONCLUSIONS

A computerized decision support system can efficiently raise attention to many otherwise missed high-risk osteoporotic cases, particularly those after fractures.

Association between organizational characteristics and adequate pain management at the intensive care unit

PURPOSE

Half of the patients experience pain during their ICU stay which is known to influence their outcomes. Nurses and physicians encounter organizational barriers towards pain assessment and treatment. We aimed to evaluate the association between adequate pain management and nurse to patient ratio, bed occupancy rate, and fulltime presence of an intensivist.

MATERIALS AND METHODS

We performed unadjusted and case-mix adjusted mixed-effect logistic regression modeling on data from thirteen Dutch ICUs to investigate the association between ICU organizational characteristics and adequate pain management, i.e. patient-shift observations in which patients' pain was measured and acceptable, or unacceptable and normalized within 1 h. All ICU patients admitted between December 2017 and June 2018 were included, excluding patients who were delirious, comatose or had a Glasgow coma score < 8 at the first day of ICU admission.

RESULTS

Case-mix adjusted nurse to patient ratios of 0.70 to 0.80 and over 0.80 were significantly associated with adequate pain management (OR [95% confidence interval] of respectively 1.14 [1.07-1.21] and 1.16 [1.08-1.24]). Bed occupancy rate and intensivist presence showed no association.

CONCLUSION

Higher nurse to patient ratios increase the percentage of patients with adequate pain management especially in medical and mechanically ventilated patients.

[Efficacy of decision support systems to improve medication safety - results of the evaluation of the "Arzneimittelkonto NRW"]

Polypharmacy increases the risk of adverse drug reactions, especially in the elderly. Therefore, the reduction of potentially inadequate medication (PIM), an improvement in drug therapy safety and, in general, a more rational use of drugs is an objective of various interventions. The aim of this prospective single-arm interventional study is to investigate the potential of a decision support system (DSS; "Arzneimittelkonto NRW") to improve medication safety in outpatient care. 15 primary care physicians participating in the study recruited 874 patients. Prescription data and results of medication safety tests were available for 654 patients. Data of at least 12 months were available for 86% of these patients. PIM prevalence declined within 12 months (-11.3%), but not at a statistically significant level. The number of prescriptions after the introduction of the DSS is significantly below the prescription volume before the introduction of the DSS (-14.1%). Constantly high alteration rates of up to 85% were observed, for example, on drug interaction system warnings made by the DSS. Technical decision support systems have the potential to support a safer and cost-saving drug use. For the first time, this pilot study provides evidence for this in the context of standard outpatient care in Germany. However, further investigations are necessary to establish a robust body of evidence. A particular focus should be on the qualitative monitoring of the studies and the involvement of other actors in the care process.

Prevalence and Predictability of Low-Yield Inpatient Laboratory Diagnostic Tests

IMPORTANCE

Laboratory testing is an important target for high-value care initiatives, constituting the highest volume of medical procedures. Prior studies have found that up to half of all inpatient laboratory tests may be medically unnecessary, but a systematic method to identify these unnecessary tests in individual cases is lacking.

OBJECTIVE

To systematically identify low-yield inpatient laboratory testing through personalized predictions.

DESIGN, SETTING, AND PARTICIPANTS

In this retrospective diagnostic study with multivariable prediction models, 116 637 inpatients treated at Stanford University Hospital from January 1, 2008, to December 31, 2017, a total of 60 929 inpatients treated at University of Michigan from January 1, 2015, to December 31, 2018, and 13 940 inpatients treated at the University of California, San Francisco from January 1 to December 31, 2018, were assessed.

MAIN OUTCOMES AND MEASURES

Diagnostic accuracy measures, including sensitivity, specificity, negative predictive values (NPVs), positive predictive values (PPVs), and area under the receiver operating characteristic curve (AUROC), of machine learning models when predicting whether inpatient laboratory tests yield a normal result as defined by local laboratory reference ranges.

RESULTS

In the recent data sets (July 1, 2014, to June 30, 2017) from Stanford University Hospital (including 22 664 female inpatients with a mean [SD] age of 58.8 [19.0] years and 22 016 male inpatients with a mean [SD] age of 59.0 [18.1] years), among the top 20 highest-volume tests, 792 397 were repeats of orders within 24 hours, including tests that are physiologically unlikely to yield new information that quickly (eg, white blood cell differential, glycated hemoglobin, and serum albumin level). The best-performing machine learning models predicted normal results with an AUROC of 0.90 or greater for 12 stand-alone laboratory tests (eg, sodium AUROC, 0.92 [95% CI, 0.91-0.93]; sensitivity, 98%; specificity, 35%; PPV, 66%; NPV, 93%; lactate dehydrogenase AUROC, 0.93 [95% CI, 0.93-0.94]; sensitivity, 96%; specificity, 65%; PPV, 71%; NPV, 95%; and troponin I AUROC, 0.92 [95% CI, 0.91-0.93]; sensitivity, 88%; specificity, 79%; PPV, 67%; NPV, 93%) and 10 common laboratory test components (eg, hemoglobin AUROC, 0.94 [95% CI, 0.92-0.95]; sensitivity, 99%; specificity, 17%; PPV, 90%; NPV, 81%; creatinine AUROC, 0.96 [95% CI, 0.96-0.97]; sensitivity, 93%; specificity, 83%; PPV, 79%; NPV, 94%; and urea nitrogen AUROC, 0.95 [95% CI, 0.94, 0.96]; sensitivity, 87%; specificity, 89%; PPV, 77%; NPV 94%).

CONCLUSIONS AND RELEVANCE

The findings suggest that low-yield diagnostic testing is common and can be systematically identified through data-driven methods and patient context-aware predictions. Implementing machine learning models appear to be able to quantify the level of uncertainty and expected information gained from diagnostic tests explicitly, with the potential to encourage useful testing and discourage low-value testing that incurs direct costs and indirect harms.

Interrupting providers with clinical decision support to improve care for heart failure

BACKGROUND

Evidence-based therapy for heart failure remains underutilized at hospital discharge, particularly for patients with heart failure who are hospitalized for another cause. We developed clinical decision support (CDS) to recommend an angiotensin converting enzyme (ACE) inhibitor during hospitalization to promote its continuation at discharge. The CDS was designed to be implemented in both interruptive and non-interruptive versions.

OBJECTIVES

To compare the effectiveness and implementation of interruptive and non-interruptive versions of a CDS to improve care for heart failure.

METHODS

Hospitalizations of patients with reduced ejection fraction were pseudo-randomized to deliver interruptive or non-interruptive CDS alerts to providers based on even or odd medical record number. We compared discharge utilization of an ACE inhibitor or angiotensin receptor blocker (ARB) for these two implementation approaches. We also assessed adoption and implementation fidelity of the CDS.

RESULTS

Of 958 hospitalizations, interruptive alert hospitalizations had higher rates of discharge utilization of ACE inhibitors or ARBs than non-interruptive alert hospitalizations (79.6% vs. 74.2%, p = 0.05). Utilization was higher for interruptive alert versus non-interruptive alert hospitalizations which were principally for causes other than heart failure (79.8% vs. 73.4%; p = 0.05) but no difference was observed among hospitalizations with a principal heart failure diagnosis (85.9% vs.81.7%; p = 0.49). As compared to non-interruptive hospitalizations, interruptive alert hospitalizations were more likely to have had: an alert with any response (40.6% vs. 13.1%, p < 0.001), contraindications reported (33.1% vs 11.3%, p < 0.001), and an ACE inhibitor ordered within twelve hours of the alert (17.6% vs 10.3%, p < 0.01). The response rate for the interruptive alert was 1.7%, and a median (25^th, 75^th percentile) of 14 (5,32) alerts were triggered per hospitalization.

CONCLUSIONS

A CDS implemented as an interruptive alert was associated with improved quality of care for heart failure. Whether the potential benefits of CDS in improving cardiovascular care were worth the high burden of interruptive alerts deserves further consideration. CLINICALTRIALS.

GOV IDENTIFIER

NCT02858674.