Evaluation of appropriateness of alerts overrides and physicians' responses of the medication-related clinical decision support system in China, a hospital-based study

This study was designed to investigate the state quo of the appropriateness of alerts overrides of the medication-related clinical decision support system (MRCDSS) in China. The medication-related alerts in one hospital from Jan 2022 to Dec 2022 were acquired and sampled. Rates of alert overrides, appropriateness of alert generation and physicians' responses were observed. Total 14,612 medication-related alerts (≤ level 3) were recorded, of those, 12,659 (86.6%) alerts were overridden. The top 3 alert types were: drug and diagnosis contraindications (23.8%), drug and test value contraindications (23.3%), and compatibility issues (17.7%). Of all sampled 1,501 alerts, 80.2% of them were appropriately overridden by the physicians. The appropriate rate of alert generation was 57.9% and the inappropriate rate was 42.1%. The inappropriate rate of physicians' responses was 17.8%, and 2.0% physicians' responses were undetermined. A few medications accounted for over 10% of overrides, 88.3% of "overridden reasons" inputted by the physicians were meaningless characters or values, indicating an obvious "alert fatigue" in these physicians. Our results indicated that the overridden rate of MRCDSS in China was still high, and appropriateness of generation of alert was quite low. These data indicated that the MRCDSS currently using in China still needs constantly optimization and timely maintenance. Proper sensitivity to reduce triggering of useless alerts and generation of alert fatigue might play a vital role. We believed that these findings are helpful for better understanding the state quo of MRCDSS in China and providing useful insights for future developing and improving MRCDSS.

Impact of Visit Volume on the Effectiveness of Electronic Tools to Improve Heart Failure Care

BACKGROUND

Electronic health record (EHR) tools can improve prescribing of guideline-recommended therapies for heart failure with reduced ejection fraction (HFrEF), but their effectiveness may vary by physician workload.

OBJECTIVES

This paper aims to assess whether physician workload modifies the effectiveness of EHR tools for HFrEF.

METHODS

This was a prespecified subgroup analysis of the BETTER CARE-HF (Building Electronic Tools to Enhance and Reinforce Cardiovascular Recommendations for Heart Failure) cluster-randomized trial, which compared effectiveness of an alert vs message vs usual care on prescribing of mineralocorticoid antagonists (MRAs). The trial included adults with HFrEF seen in cardiology offices who were eligible for and not prescribed MRAs. Visit volume was defined at the cardiologist-level as number of visits per 6-month study period (high = upper tertile vs non-high = remaining). Analysis at the patient-level used likelihood ratio test for interaction with log-binomial models.

RESULTS

Among 2,211 patients seen by 174 cardiologists, 932 (42.2%) were seen by high-volume cardiologists (median: 1,853; Q1-Q3: 1,637-2,225 visits/6 mo; and median: 10; Q1-Q3: 9-12 visits/half-day). MRA was prescribed to 5.5% in the high-volume vs 14.8% in the non-high-volume groups in the usual care arm, 10.3% vs 19.6% in the message arm, and 31.2% vs 28.2% in the alert arm, respectively. Visit volume modified treatment effect (P for interaction = 0.02) such that the alert was more effective in the high-volume group (relative risk: 5.16; 95% CI: 2.57-10.4) than the non-high-volume group (relative risk: 1.93; 95% CI: 1.29-2.90).

CONCLUSIONS

An EHR-embedded alert increased prescribing by >5-fold among patients seen by high-volume cardiologists. Our findings support use of EHR alerts, especially in busy practice settings. (Building Electronic Tools to Enhance and Reinforce Cardiovascular Recommendations for Heart Failure [BETTER CARE-HF]; NCT05275920).

Monitoring of Remotely Reprogrammable Implantable Loop Recorders With Algorithms to Reduce False-Positive Alerts

BACKGROUND

Implantable loop recorders (ILRs) are increasingly placed for arrhythmia detection. However, historically, ≈75% of ILR alerts are false positives, requiring significant time and effort for adjudication. The LINQII and LUX-Dx are remotely reprogrammable ILRs with dual-stage algorithms using artificial intelligence to reduce false positives, but their utility in routine clinical practice has not been studied.

METHODS AND RESULTS

We identified patients with the LINQII and LUX-Dx who were monitored by the Veterans Affairs National Cardiac Device Surveillance Program between March and June 2022. ILR programming was customized on the basis of implant indication. All alerts and every 90-day scheduled transmissions were manually reviewed. ILRs were remotely reprogrammed, as appropriate, after false-positive alerts or 2 consecutive same-type alerts, unless there was ongoing clinical need for that alert. Outcomes were total number of transmissions and false positives. We performed medical record review to determine if patients experienced any adverse clinical events, including hospitalization and mortality. Among 117 LINQII patients, there were 239 total alerts, 43 (18.0%) of which were false positives. Among 105 LUX-Dx patients, there were 300 total alerts, 115 (38.3%) of which were false positives. LINQIIs were reprogrammed 22 times, resulting in a decrease in median alerts/day from 0.13 to 0.03. LUX-Dx ILRs were reprogrammed 52 times, resulting in a decrease from 0.15 to 0.01 median alerts/day. There were no adverse clinical events that could have been identified by superior or earlier arrhythmia detection.

CONCLUSIONS

ILRs with artificial intelligence algorithms and remote reprogramming ability are associated with reduced alert burden because of higher true-positive rates than prior ILRs, without missing potentially consequential arrhythmias.

Factors Influencing Medication Administration Outcomes Among New Graduate Nurses Using Bar Code-Assisted Medication Administration

Paramount to patient safety is the ability for nurses to make clinical decisions free from human error. Yet, the dynamic clinical environment in which nurses work is characterized by uncertainty, urgency, and high consequence, necessitating that nurses make quick and critical decisions. The aim of this study was to examine the influence of human and environmental factors on the decision to administer among new graduate nurses in response to alert generation during bar code-assisted medication administration. The design for this study was a descriptive, longitudinal, observational cohort design using EHR audit log and administrative data. The study was set at a large, urban medical center in the United States and included 132 new graduate nurses who worked on adult, inpatient units. Research variables included human and environmental factors. Data analysis included descriptive and inferential analyses. This study found that participants continued with administration of a medication in 90.75% of alert encounters. When considering the response to an alert, residency cohort, alert category, and previous exposure variables were associated with the decision to proceed with administration. It is important to continue to study factors that influence nurses' decision-making, particularly during the process of medication administration, to improve patient safety and outcomes.

Retrospective analysis of the impact of electronic medical record alerts on low value care in a pediatric hospital

OBJECTIVES

Hospital costs continue to rise unsustainably. Up to 20% of care is wasteful including low value care (LVC). This study aimed to understand whether electronic medical record (EMR) alerts are effective at reducing pediatric LVC and measure the impact on hospital costs.

MATERIALS AND METHODS

Using EMR data over a 76-month period, we evaluated changes in 4 LVC practices following the implementation of EMR alerts, using time series analysis to control for underlying time-based trends, in a large pediatric hospital in Australia. The main outcome measure was the change in rate of each LVC practice. Balancing measures included the rate of alert adherence as a proxy measure for risk of alert fatigue. Hospital costs were calculated by the volume of LVC avoided multiplied by the unit costs. Costs of the intervention were calculated from clinician and analyst time required.

RESULTS

All 4 LVC practices showed a statistically significant reduction following alert implementation. Two LVC practices (blood tests) showed an abrupt change, associated with high rates of alert adherence. The other 2 LVC practices (bronchodilator use in bronchiolitis and electrocardiogram ordering for sleeping bradycardia) showed an accelerated rate of improvement compared to baseline trends with lower rates of alert adherence. Hospital savings were $325 to $180 000 per alert.

DISCUSSION AND CONCLUSION

EMR alerts are effective in reducing pediatric LVC practices and offer a cost-saving opportunity to the hospital. Further efforts to leverage EMR alerts in pediatric settings to reduce LVC are likely to support future sustainable healthcare delivery.

Remote monitoring of implantable cardiac monitors in patients with unexplained syncope: Predictors of false-positive alert episodes

BACKGROUND

Remote monitoring is recommended for patients with implantable cardiac monitors (ICMs), but compared to other cardiac implantable devices, ICMs are less accurate and transmit a higher number of alerts.

OBJECTIVE

The aim of this study was to investigate the predictors of false-positive (FP) arrhythmic alerts in patients with unexplained syncope who were implanted with ICM and followed by an automatic remote monitoring system.

METHODS

We retrospectively evaluated all consecutive patients who received a long-sensing vector ICM for unexplained syncope between January 2019 to September 2021 at our Syncope Unit. The primary endpoint was the incidence of the first FP episode. The secondary endpoints included assessing the incidence of FP episodes for all types of algorhythms and indentifying the reasons for the misdetection of these episodes.

RESULTS

Among 105 patients (44.8% males, median age 51 years), 51 (48.6%) transmitted at least one FP alert during a median follow-up of 301 days. The presence of pre-ventricular complexes (PVCs) on the resting electrocardiogram was the only clinical characteristic associated with an increased risk of FP alerts (adjusted Hazard ratio [HR] 5.76 [2.66-12.4], p = 0.010). The other significant device-related variables were a low-frequency filter at 0.05 Hz versus the default 0.5 Hz (adjusted HR 3.82 [1.38-10.5], p = 0.010) and the R-wave amplitude (adjusted HR 0.35 [0.13-0.99], p = 0.049).

CONCLUSION

Patients who have PVCs are at higher risk of inappropriate ICM activations. To reduce the occurrence of FP alerts, it may be beneficial to target a large R-wave amplitude during device insertion and avoid programming a low-frequency filter at 0.05 Hz.

Optimizing Best Practice Advisory alerts in electronic medical records with a multi-pronged strategy at a tertiary care hospital in Singapore

Objective

Clinical decision support (CDS) alerts can aid in improving patient care. One CDS functionality is the Best Practice Advisory (BPA) alert notification system, wherein BPA alerts are automated alerts embedded in the hospital's electronic medical records (EMR). However, excessive alerts can change clinician behavior; redundant and repetitive alerts can contribute to alert fatigue. Alerts can be optimized through a multipronged strategy. Our study aims to describe these strategies adopted and evaluate the resultant BPA alert optimization outcomes.

Materials and Methods

This retrospective single-center study was done at Jurong Health Campus. Aggregated, anonymized data on patient demographics and alert statistics were collected from January 1, 2018 to December 31, 2021. "Preintervention" period was January 1-December 31, 2018, and "postintervention" period was January 1-December 31, 2021. The intervention period was the intervening period. Categorical variables were reported as frequencies and proportions and compared using the chi-square test. Continuous data were reported as median (interquartile range, IQR) and compared using the Wilcoxon rank-sum test. Statistical significance was defined at P < .05.

Results

There was a significant reduction of 59.6% in the total number of interruptive BPA alerts, despite an increase in the number of unique BPAs from 54 to 360 from pre- to postintervention. There was a 74% reduction in the number of alerts from the 7 BPAs that were optimized from the pre- to postintervention period. There was a significant increase in percentage of overall interruptive BPA alerts with action taken (8 [IQR 7.7-8.4] to 54.7 [IQR 52.5-58.9], P-value < .05) and optimized BPAs with action taken (32.6 [IQR 32.3-32.9] to 72.6 [IQR 64.3-73.4], P-value < .05). We estimate that the reduction in alerts saved 3600 h of providers' time per year.

Conclusions

A significant reduction in interruptive alert volume, and a significant increase in action taken rates despite manifold increase in the number of unique BPAs could be achieved through concentrated efforts focusing on governance, data review, and visualization using a system-embedded tool, combined with the CDS Five Rights framework, to optimize alerts. Improved alert compliance was likely multifactorial-due to decreased repeated alert firing for the same patient; better awareness due to stakeholders' involvement; and less fatigue since unnecessary alerts were removed. Future studies should prospectively focus on patients' clinical chart reviews to assess downstream effects of various actions taken, identify any possibility of harm, and collect end-user feedback regarding the utility of alerts.

Alerts and Collections for Automating Patients' Sensemaking and Organizing of Their Electronic Health Record Data for Reflection, Planning, and Clinical Visits: Qualitative Research-Through-Design Study

BACKGROUND

Electronic health record (EHR) data from multiple providers often exhibit important but convoluted and complex patterns that patients find hard and time-consuming to identify and interpret. However, existing patient-facing applications lack the capability to incorporate automatic pattern detection robustly and toward supporting making sense of the patient's EHR data. In addition, there is no means to organize EHR data in an efficient way that suits the patient's needs and makes them more actionable in real-life settings. These shortcomings often result in a skewed and incomplete picture of the patient's health status, which may lead to suboptimal decision-making and actions that put the patient at risk.

OBJECTIVE

Our main goal was to investigate patients' attitudes, needs, and use scenarios with respect to automatic support for surfacing important patterns in their EHR data and providing means for organizing them that best suit patients' needs.

METHODS

We conducted an inquisitive research-through-design study with 14 participants. Presented in the context of a cutting-edge application with strong emphasis on independent EHR data sensemaking, called Discovery, we used high-level mock-ups for the new features that were supposed to support automatic identification of important data patterns and offer recommendations-Alerts-and means for organizing the medical records based on patients' needs, much like photos in albums-Collections. The combined audio recording transcripts and in-study notes were analyzed using the reflexive thematic analysis approach.

RESULTS

The Alerts and Collections can be used for raising awareness, reflection, planning, and especially evidence-based patient-provider communication. Moreover, patients desired carefully designed automatic pattern detection with safe and actionable recommendations, which produced a well-tailored and scoped landscape of alerts for both potential threats and positive progress. Furthermore, patients wanted to contribute their own data (eg, progress notes) and log feelings, daily observations, and measurements to enrich the meaning and enable easier sensemaking of the alerts and collections. On the basis of the findings, we renamed Alerts to Reports for a more neutral tone and offered design implications for contextualizing the reports more deeply for increased actionability; automatically generating the collections for more expedited and exhaustive organization of the EHR data; enabling patient-generated data input in various formats to support coarser organization, richer pattern detection, and learning from experience; and using the reports and collections for efficient, reliable, and common-ground patient-provider communication.

CONCLUSIONS

Patients need to have a flexible and rich way to organize and annotate their EHR data; be introduced to insights from these data-both positive and negative; and share these artifacts with their physicians in clinical visits or via messaging for establishing shared mental models for clear goals, agreed-upon priorities, and feasible actions.

A Nationwide Digital Multidisciplinary Intervention Aimed at Promoting Pneumococcal Vaccination in Immunocompromised Patients

Immunocompromised patients (IPs) are at high risk for infections, some of which are vaccine-preventable. The Israeli Ministry of Health recommends pneumococcal conjugate vaccine 13 (PCV13) and pneumococcal polysaccharide vaccine 23 (PPSV23) for IP, but vaccine coverage is suboptimal. We assessed the project's effectiveness in improving the pneumococcal vaccination rate among IP. An automated population-based registry of IP was developed and validated at Maccabi Healthcare Services, an Israeli health maintenance organization serving over 2.6 million members. Included were transplant recipients, patients with asplenia, HIV or advanced kidney disease; or those receiving immunosuppressive therapy. A personalized electronic medical record alert was activated reminding clinicians to consider vaccination during IP encounters. Later, IP were invited to get vaccinated via their electronic patient health record. Pre- and post-intervention vaccination rates were compared. Between October 2019 and October 2021, overall PCV13 vaccination rates among 32,637 IP went up from 11.9% (n = 3882) to 52% (n = 16,955) (p < 0.0001). The PPSV23 vaccination rate went up from 39.4% (12,857) to 57.1% (18,652) (p < 0.0001). In conclusion, implementation of targeted automated patient- and clinician-facing alerts, a remarkable increase in pneumococcal vaccine uptake was observed among IP. The outlined approach may be applied to increase vaccination uptake in large health organizations.

Transitioning Cognitive Aids into Decision Support Platforms: Requirements and Design Guidelines

Digital cognitive aids have the potential to serve as clinical decision support platforms, triggering alerts about process delays and recommending interventions. In this mixed-methods study, we examined how a digital checklist for pediatric trauma resuscitation could trigger decision support alerts and recommendations. We identified two criteria that cognitive aids must satisfy to support these alerts: (1) context information must be entered in a timely, accurate, and standardized manner, and (2) task status must be accurately documented. Using co-design sessions and near-live simulations, we created two checklist features to satisfy these criteria: a form for entering the pre-hospital information and a progress slider for documenting the progression of a multi-step task. We evaluated these two features in the wild, contributing guidelines for designing these features on cognitive aids to support alerts and recommendations in time- and safety-critical scenarios.

Provider Interaction With an Electronic Health Record Notification to Identify Eligible Patients for a Cluster Randomized Trial of Advance Care Planning in Primary Care: Secondary Analysis

BACKGROUND

Advance care planning (ACP) improves patient-provider communication and aligns care to patient values, preferences, and goals. Within a multisite Meta-network Learning and Research Center ACP study, one health system deployed an electronic health record (EHR) notification and algorithm to alert providers about patients potentially appropriate for ACP and the clinical study.

OBJECTIVE

The aim of the study is to describe the implementation and usage of an EHR notification for referring patients to an ACP study, evaluate the association of notifications with study referrals and engagement in ACP, and assess provider interactions with and perspectives on the notifications.

METHODS

A secondary analysis assessed provider usage and their response to the notification (eg, acknowledge, dismiss, or engage patient in ACP conversation and refer patient to the clinical study). We evaluated all patients identified by the EHR algorithm during the Meta-network Learning and Research Center ACP study. Descriptive statistics compared patients referred to the study to those who were not referred to the study. Health care utilization, hospice referrals, and mortality as well as documentation and billing for ACP and related legal documents are reported. We evaluated associations between notifications with provider actions (ie, referral to study, ACP not documentation, and ACP billing). Provider free-text comments in the notifications were summarized qualitatively. Providers were surveyed on their satisfaction with the notification.

RESULTS

Among the 2877 patients identified by the EHR algorithm over 20 months, 17,047 unique notifications were presented to 45 providers in 6 clinics, who then referred 290 (10%) patients. Providers had a median of 269 (IQR 65-552) total notifications, and patients had a median of 4 (IQR 2-8). Patients with more (over 5) notifications were less likely to be referred to the study than those with fewer notifications (57/1092, 5.2% vs 233/1785, 13.1%; P<.001). The most common free-text comment on the notification was lack of time. Providers who referred patients to the study were more likely to document ACP and submit ACP billing codes (P<.001). In the survey, 11 providers would recommend the notification (n=7, 64%); however, the notification impacted clinical workflow (n=9, 82%) and was difficult to navigate (n=6, 55%).

CONCLUSIONS

An EHR notification can be implemented to remind providers to both perform ACP conversations and refer patients to a clinical study. There were diminishing returns after the fifth EHR notification where additional notifications did not lead to more trial referrals, ACP documentation, or ACP billing. Creation and optimization of EHR notifications for study referrals and ACP should consider the provider user, their workflow, and alert fatigue to improve implementation and adoption.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03577002; https://clinicaltrials.gov/ct2/show/NCT03577002.

G6 continuous glucose monitoring system feature use and its associations with glycaemia in Europe

AIMS

Current continuous glucose monitoring (CGM) devices provide features that alert individuals with diabetes about their current and impending adverse glycaemic events. The use of these features has been associated with glycaemic improvements. However, how these features are utilised under real-world conditions has not been well studied. We queried a large database to quantify utilisation of the Dexcom G6 system features and how utilisation impacted glycaemic outcomes within a cohort of European users.

METHODS

This 6-month retrospective, observational, large database analysis utilised anonymised data from a sample of 47,784 Europe-based G6 users. Primary outcome measures were associations between utilisation and customisation of High/Low threshold alerts, 'urgent low soon' (ULS) alert, and established CGM metrics.

RESULTS

Users in the Germany, Austria, Switzerland region (n = 20,257), the Nordic countries (n = 10,314), United Kingdom (n = 9006), Italy (n = 4747), France (n = 2130) and Spain (1330) were included. All alert features were utilised by >75% of the cohort across all regions/countries and age groups. Enabling the Low alert and ULS alert was associated with lower percentage of time below range compared to disabling the Low alert (p < 0.001). Enabling the High alert was associated with higher percentage of time in range (%TIR) and lower percentage of time above range (%TAR) %TAR compared to disabling the High alert (p < 0.001). Paediatric patients and older adults tended to set a higher threshold for High/Low alerts, while younger adults tended to use lower threshold values for High/Low alerts.

CONCLUSIONS

Individuals who utilised the Dexcom G6 features showed better glycaemic control, particularly among those who utilised more sensitive High alert and Low alert settings, than users who did not utilise the system features.

Remote Monitoring of Physiology in People Living With Dementia: An Observational Cohort Study

BACKGROUND

Internet of Things (IoT) technology enables physiological measurements to be recorded at home from people living with dementia and monitored remotely. However, measurements from people with dementia in this context have not been previously studied. We report on the distribution of physiological measurements from 82 people with dementia over approximately 2 years.

OBJECTIVE

Our objective was to characterize the physiology of people with dementia when measured in the context of their own homes. We also wanted to explore the possible use of an alerts-based system for detecting health deterioration and discuss the potential applications and limitations of this kind of system.

METHODS

We performed a longitudinal community-based cohort study of people with dementia using "Minder," our IoT remote monitoring platform. All people with dementia received a blood pressure machine for systolic and diastolic blood pressure, a pulse oximeter measuring oxygen saturation and heart rate, body weight scales, and a thermometer, and were asked to use each device once a day at any time. Timings, distributions, and abnormalities in measurements were examined, including the rate of significant abnormalities ("alerts") defined by various standardized criteria. We used our own study criteria for alerts and compared them with the National Early Warning Score 2 criteria.

RESULTS

A total of 82 people with dementia, with a mean age of 80.4 (SD 7.8) years, recorded 147,203 measurements over 958,000 participant-hours. The median percentage of days when any participant took any measurements (ie, any device) was 56.2% (IQR 33.2%-83.7%, range 2.3%-100%). Reassuringly, engagement of people with dementia with the system did not wane with time, reflected in there being no change in the weekly number of measurements with respect to time (1-sample t-test on slopes of linear fit, P=.45). A total of 45% of people with dementia met criteria for hypertension. People with dementia with α-synuclein-related dementia had lower systolic blood pressure; 30% had clinically significant weight loss. Depending on the criteria used, 3.03%-9.46% of measurements generated alerts, at 0.066-0.233 per day per person with dementia. We also report 4 case studies, highlighting the potential benefits and challenges of remote physiological monitoring in people with dementia. These include case studies of people with dementia developing acute infections and one of a person with dementia developing symptomatic bradycardia while taking donepezil.

CONCLUSIONS

We present findings from a study of the physiology of people with dementia recorded remotely on a large scale. People with dementia and their carers showed acceptable compliance throughout, supporting the feasibility of the system. Our findings inform the development of technologies, care pathways, and policies for IoT-based remote monitoring. We show how IoT-based monitoring could improve the management of acute and chronic comorbidities in this clinically vulnerable group. Future randomized trials are required to establish if a system like this has measurable long-term benefits on health and quality of life outcomes.

Monitoring the Implementation of Tobacco Cessation Support Tools: Using Novel Electronic Health Record Activity Metrics

BACKGROUND

Clinical decision support (CDS) tools in electronic health records (EHRs) are often used as core strategies to support quality improvement programs in the clinical setting. Monitoring the impact (intended and unintended) of these tools is crucial for program evaluation and adaptation. Existing approaches for monitoring typically rely on health care providers' self-reports or direct observation of clinical workflows, which require substantial data collection efforts and are prone to reporting bias.

OBJECTIVE

This study aims to develop a novel monitoring method leveraging EHR activity data and demonstrate its use in monitoring the CDS tools implemented by a tobacco cessation program sponsored by the National Cancer Institute's Cancer Center Cessation Initiative (C3I).

METHODS

We developed EHR-based metrics to monitor the implementation of two CDS tools: (1) a screening alert reminding clinic staff to complete the smoking assessment and (2) a support alert prompting health care providers to discuss support and treatment options, including referral to a cessation clinic. Using EHR activity data, we measured the completion (encounter-level alert completion rate) and burden (the number of times an alert was fired before completion and time spent handling the alert) of the CDS tools. We report metrics tracked for 12 months post implementation, comparing 7 cancer clinics (2 clinics implemented the screening alert and 5 implemented both alerts) within a C3I center, and identify areas to improve alert design and adoption.

RESULTS

The screening alert fired in 5121 encounters during the 12 months post implementation. The encounter-level alert completion rate (clinic staff acknowledged completion of screening in EHR: 0.55; clinic staff completed EHR documentation of screening results: 0.32) remained stable over time but varied considerably across clinics. The support alert fired in 1074 encounters during the 12 months. Providers acted upon (ie, not postponed) the support alert in 87.3% (n=938) of encounters, identified a patient ready to quit in 12% (n=129) of encounters, and ordered a referral to the cessation clinic in 2% (n=22) of encounters. With respect to alert burden, on average, both alerts fired over 2 times (screening alert: 2.7; support alert: 2.1) before completion; time spent postponing the screening alert was similar to completing (52 vs 53 seconds) the alert, and time spent postponing the support alert was more than completing (67 vs 50 seconds) the alert per encounter. These findings inform four areas where the alert design and use can be improved: (1) improving alert adoption and completion through local adaptation, (2) improving support alert efficacy by additional strategies including training in provider-patient communication, (3) improving the accuracy of tracking for alert completion, and (4) balancing alert efficacy with the burden.

CONCLUSIONS

EHR activity metrics were able to monitor the success and burden of tobacco cessation alerts, allowing for a more nuanced understanding of potential trade-offs associated with alert implementation. These metrics can be used to guide implementation adaptation and are scalable across diverse settings.

Potential Drug-Related Problems in Pediatric Patients-Describing the Use of a Clinical Decision Support System at Pharmacies in Sweden

The clinical support system Electronic Expert Support (EES) is available at all pharmacies in Sweden to examine electronic prescriptions when dispensing to prevent drug-related problems (DRPs). DRPs are common, and result in patient suffering and substantial costs for society. The aim of this research was to study the use of EES for the pediatric population (ages 0-12 years), by describing what types of alerts are generated for potential DRPs, how they are handled, and how the use of EES has changed over time. Data on the number and categories of EES analyses, alerts, and resolved alerts were provided by the Swedish eHealth Agency. The study shows that the use of EES has increased. The most common type of alert for a potential DRP among pediatric patients was regarding high doses in children (30.3% of all alerts generated). The most common type of alert for a potential DRP that was resolved among pediatrics was therapy duplication (4.6% of the alerts were resolved). The most common reason for closing an alert was dialogue with patient for verification of the treatment (66.3% of all closed alerts). Knowledge of which type of alerts are the most common may contribute to increased prescriber awareness of important potential DRPs.

Identification of sepsis in paediatric emergency departments: A scoping review

AIM

Sepsis is an acute illness associated with significant morbidity and mortality. Early detection and time-sensitive management of sepsis has been shown to improve outcomes. We report the results of a scoping review to explore methods evaluated for the identification of sepsis in children presenting to emergency departments.

METHODS

A systematic literature search was carried out on two databases, Medline and Web of Science, to identify relevant studies published from 1990 to 2022. Data were extracted for age groups including study design, reference standard used for comparison, sepsis identification method evaluated and study quality.

RESULTS

A total of 89 studies were identified from the literature search. There was significant heterogeneity in the age groups including study design and reference standards used for evaluating the performance of the sepsis identification methods. There has been a substantial increase in the number of published studies in the last 2 years.

CONCLUSION

Our scoping review identifies marked heterogeneity in approaches to identifying sepsis but demonstrates a recent focus of research on patient outcomes. Using appropriate core outcome sets, developing reference standards, monitoring sepsis prevalence via registries and continuously monitoring process measures will provide robust evidence to identify the best performing identification tools and the impact they have on patient-orientated outcomes.

Clinical decision support malfunctions related to medication routes: a case series

Objective

To identify common medication route-related causes of clinical decision support (CDS) malfunctions and best practices for avoiding them.

Materials and Methods

Case series of medication route-related CDS malfunctions from diverse healthcare provider organizations.

Results

Nine cases were identified and described, including both false-positive and false-negative alert scenarios. A common cause was the inclusion of nonsystemically available medication routes in value sets (eg, eye drops, ear drops, or topical preparations) when only systemically available routes were appropriate.

Discussion

These value set errors are common, occur across healthcare provider organizations and electronic health record (EHR) systems, affect many different types of medications, and can impact the accuracy of CDS interventions. New knowledge management tools and processes for auditing existing value sets and supporting the creation of new value sets can mitigate many of these issues. Furthermore, value set issues can adversely affect other aspects of the EHR, such as quality reporting and population health management.

Conclusion

Value set issues related to medication routes are widespread and can lead to CDS malfunctions. Organizations should make appropriate investments in knowledge management tools and strategies, such as those outlined in our recommendations.

Design Considerations for the Use of Patient-Generated Health Data in the Electronic Medical Records

Patient-generated health data (PGHD) is of growing interest to physicians, particularly if they are integrated in the electronic medical record (EMR). Concerns about how to manage vast amounts of PGHD and potential liability issues have limited their use. Based on interviews with specialists, we present types of PGHD, workflow processes and needs. We then discuss consideration for how to manage PGHD with approaches for analyses to detect abnormal results, and present implications for alert systems and visualization requirements in multi-patient views.

Impact of an electronic health record alert on inappropriate prescribing of high-risk medications to patients with concurrent "do not give" orders

DISCLAIMER

In an effort to expedite the publication of articles related to the COVID-19 pandemic, AJHP is posting these manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.

PURPOSE

To evaluate the effectiveness of clinical decision support (CDS) alerts tied to high-risk medications at a Michigan health system by determining the true prescriber action rate in response to select "do not give" (DNG) alerts.

METHODS

A retrospective review of prescriber actions in response to CDS alerts was conducted to evaluate the effectiveness of alerts designed to prevent prescribing of high-risk medications to patients with concurrent DNG orders. The primary endpoint was the overall action rate, determined by totaling orders cancelled within the alert display and orders modified shortly after an alert. The overall action rate was hypothesized to significantly exceed the action rate estimated on the basis of alert overrides alone. Following the initial review, changes were made to the alert format and preset documentation choices ("acknowledgement comments"), and it was hypothesized that these changes would increase the overall action rate. A repeat analysis was conducted to evaluate the impact of these changes.

RESULTS

Across a total of 506 CDS alerts over 14 months, 78% resulted in prescribers modifying orders to comply with alert recommendations. Prescribers cancelled orders in response to only 26% of alerts, often overriding alerts prior to modifying orders. Documentation of rationale or approval for overrides was inconsistent, and while requiring acknowledgement comments facilitated documentation of prescriber rationale, it did not consistently improve overall action rates.

CONCLUSION

These findings demonstrate that override rates alone are not good markers for the true effectiveness of CDS alerts and support the need for frequent evaluation of alerts at the institutional level. CDS alerts remain a valuable tool to prevent inappropriate prescribing of high-risk medications and for promoting patient safety.

Learning About the Effects of Alert Uncertainty in Attack and Defend Decisions via Cognitive Modeling

OBJECTIVE

We aim to learn about the cognitive mechanisms governing the decisions of attackers and defenders in cybersecurity involving intrusion detection systems (IDSs).

BACKGROUND

Prior research has experimentally studied the role of the presence and accuracy of IDS alerts on attacker's and defender's decisions using a game-theoretic approach. However, little is known about the cognitive mechanisms that govern these decisions.

METHOD

To investigate the cognitive mechanisms governing the attacker's and defender's decisions in the presence of IDSs of different accuracies, instance-based learning (IBL) models were developed. One model (NIDS) disregarded the IDS alerts and one model (IDS) considered them in the instance structure. Both the IDS and NIDS models were trained in an existing dataset where IDSs were either absent or present and they possessed different accuracies. The calibrated IDS model was tested in a newly collected test dataset where IDSs were present 50% of the time and they possessed different accuracies.

RESULTS

Both the IDS and NIDS models were able to account for human decisions in the training dataset, where IDS was absent or present and it possessed different accuracies. However, the IDS model could accurately predict the decision-making in only one of the several IDS accuracy conditions in the test dataset.

CONCLUSIONS

Cognitive models like IBL may provide some insights regarding the cognitive mechanisms governing the decisions of attackers and defenders in conditions not involving IDSs or IDSs of different accuracies.

APPLICATION

IBL models may be helpful for penetration testing exercises in scenarios involving IDSs of different accuracies.

Improving the specificity of drug-drug interaction alerts: Can it be done?

PURPOSE

Inaccurate and nonspecific medication alerts contribute to high override rates, alert fatigue, and ultimately patient harm. Drug-drug interaction (DDI) alerts often fail to account for factors that could reduce risk; further, drugs that trigger alerts are often inconsistently grouped into value sets. Toward improving the specificity of DDI alerts, the objectives of this study were to (1) highlight the inconsistency of drug value sets for triggering DDI alerts and (2) demonstrate a method of classifying factors that can be used to modify the risk of harm from a DDI.

METHODS

This was a proof-of-concept study focused on 15 well-known DDIs. Using 3 drug interaction references, we extracted 2 drug value sets and any available order- and patient-related factors for each DDI. Fleiss' kappa was used to measure the consistency of value sets among references. Risk-modifying factors were classified as order parameters (eg, route and dose) or patient characteristics (eg, comorbidities and laboratory results).

RESULTS

Seventeen value sets (56%) had nonsignificant agreement. Agreement among the remaining 13 value sets was on average moderate. Thirty-three factors that could reduce risk in 14 of 15 DDIs (93%) were identified. Most risk-modifying factors (67%) were classified as order parameters.

CONCLUSION

This study demonstrates the importance of increasing the consistency of drug value sets that trigger DDI alerts and how alert specificity and usefulness can be improved with risk-modifying factors obtained from drug references. It may be difficult to operationalize certain factors to reduce unnecessary alerts; however, factors can be used to support decisions by providing contextual information.

Results of the Second Phase of the GER-e-TEC Experiment concerning the Telemonitoring of Elderly Patients Affected by COVID-19 Disease to Detect the Exacerbation of Geriatric Syndromes

BACKGROUND

Coronavirus disease 2019 (COVID-19) has wreaked health and economic damage globally. This pandemic has created a difficult challenge for global public health. The coronavirus disease 2019 (COVID-19) pandemic has necessitated the use of new technologies and new processes to care for hospitalized patients, including elderly patients. Our team developed a telemonitoring program focused on the prevention of geriatric syndromes, the "GER-e-TEC COVID study".

METHODS

This second phase took place during the 3rd wave of the epidemic in France, between 14 December 2020 and 25 February 2021, conducted in the University Hospital of Strasbourg.

RESULTS

30 elderly patients affected by COVID-19 disease were monitored remotely; the mean age was 85.9 years and a male/female ratio of 1.5 to 1.11 (36.7%) died during the experiment. The patients used the telemedicine solution for an average of 27.3 days. 140,260 measurements were taken while monitoring the geriatric syndromes of the entire patient group. 4675 measurements were recorded per patient for geriatric disorders and risks. 319 measurements were recorded per patient per day. The telemedicine solution emitted a total of 1245 alerts while monitoring the geriatric syndromes of the entire patient group. In terms of sensitivity, the results were 100% for all geriatric risks and extremely satisfactory in terms of positive and negative predictive values. Survival analyses showed that gender played no role in the length of the hospital stay, regardless of the reason for the hospitalization (decompensated heart failure (p = 0.45), deterioration of general condition (p = 0.12), but significant for death (p = 0.028)). The analyses revealed that the length of the hospital stay was not affected by the number of alerts. The results concerning the predictive nature of alerts are satisfactory.

CONCLUSIONS

The MyPredi™ telemedicine system allows for the generation of automatic, non-intrusive alerts when the health of a COVID-19 elderly patient deteriorates due to risks associated with geriatric syndromes.

Uganda Mountain Community Health System-Perspectives and Capacities towards Emerging Infectious Disease Surveillance

In mountain communities like Sebei, Uganda, which are highly vulnerable to emerging and re-emerging infectious diseases, community-based surveillance plays an important role in the monitoring of public health hazards. In this survey, we explored capacities of village health teams (VHTs) in Sebei communities of Mount Elgon in undertaking surveillance tasks for emerging and re-emerging infectious diseases in the context of a changing climate. We used participatory epidemiology techniques to elucidate VHTs' perceptions on climate change and public health and assessed their capacities to conduct surveillance for emerging and re-emerging infectious diseases. Overall, VHTs perceived climate change to be occurring with wider impacts on public health. However, they had inadequate capacities in collecting surveillance data. The VHTs lacked transport to navigate through their communities and had insufficient capacities in using mobile phones for sending alerts. They did not engage in reporting other hazards related to the environment, wildlife, and domestic livestock that would accelerate infectious disease outbreaks. Records were not maintained for disease surveillance activities and the abilities of VHTs to analyze data were also limited. However, VHTs had access to platforms that could enable them to disseminate public health information. The VHTs thus need to be retooled to conduct their work effectively and efficiently through equipping them with adequate logistics and knowledge on collecting, storing, analyzing, and relaying data, which will improve infectious disease response and mitigation efforts.

Understanding User Acceptance of Clinical Decision Support Systems to Promote Increased Cancer Screening Rates in a Primary Care Practice

OBJECTIVE

Clinical decision support systems (CDDSs) in the electronic medical record (EMR) have been implemented in primary care settings to identify patients due for cancer screening tests, while functioning as a real time reminder system. There is little known about primary care providers (PCPs) perspective or user acceptance of CDSS. The purpose of this study was to investigate primary care provider perceptions of utilizing CDSS alerts in the EMR to promote increased screening rates for breast cancer, cervical cancer, and colorectal cancer.

METHODS

An electronic survey was administered to PCPs in a Midwest Health Institution community internal medicine practice from September 25, 2019 through November 27, 2019.

RESULTS

Among 37 participants (9 NP/Pas and 28 MD/DOs), the NP/PA group was more likely to agree that alerts were helpful (50%; P-value = .0335) and the number of alerts (89%; P = .0227) in the EMR was appropriate. The NP/PA group also was more likely to find alerts straightforward to use (78%, P = .0239). Both groups agreed about feeling comfortable using the health maintenance alerts (MD/DO = 79%; NP/PA = 100%).

CONCLUSION

CDSSs can promote and facilitate ordering of cancer screening tests. The use of technology can promptly identify patients due for a test and act as a reminder to the PCP. PCPs identify these alerts to be a beneficial tool in the EMR when they do not interrupt workflow and provide value to patient care. More work is needed to identify factors that could optimize alerts to be even more helpful, particularly to MD/DO groups.

Designing Interactive Alerts to Improve Recognition of Critical Events in Medical Emergencies

Vital sign values during medical emergencies can help clinicians recognize and treat patients with life-threatening injuries. Identifying abnormal vital signs, however, is frequently delayed and the values may not be documented at all. In this mixed-methods study, we designed and evaluated a two-phased visual alert approach for a digital checklist in trauma resuscitation that informs users about undocumented vital signs. Using an interrupted time series analysis, we compared documentation in the periods before (two years) and after (four months) the introduction of the alerts. We found that introducing alerts led to an increase in documentation throughout the post-intervention period, with clinicians documenting vital signs earlier. Interviews with users and video review of cases showed that alerts were ineffective when clinicians engaged less with the checklist or set the checklist down to perform another activity. From these findings, we discuss approaches to designing alerts for dynamic team-based settings.

Evaluating a digital sepsis alert in a London multisite hospital network: a natural experiment using electronic health record data

Objective

The study sought to determine the impact of a digital sepsis alert on patient outcomes in a UK multisite hospital network.

Materials and Methods

A natural experiment utilizing the phased introduction (without randomization) of a digital sepsis alert into a multisite hospital network. Sepsis alerts were either visible to clinicians (patients in the intervention group) or running silently and not visible (the control group). Inverse probability of treatment-weighted multivariable logistic regression was used to estimate the effect of the intervention on individual patient outcomes.

Outcomes

In-hospital 30-day mortality (all inpatients), prolonged hospital stay (≥7 days) and timely antibiotics (≤60 minutes of the alert) for patients who alerted in the emergency department.

Results

The introduction of the alert was associated with lower odds of death (odds ratio, 0.76; 95% confidence interval [CI], 0.70-0.84; n = 21 183), lower odds of prolonged hospital stay ≥7 days (OR, 0.93; 95% CI, 0.88-0.99; n = 9988), and in patients who required antibiotics, an increased odds of receiving timely antibiotics (OR, 1.71; 95% CI, 1.57-1.87; n = 4622).

Discussion

Current evidence that digital sepsis alerts are effective is mixed. In this large UK study, a digital sepsis alert has been shown to be associated with improved outcomes, including timely antibiotics. It is not known whether the presence of alerting is responsible for improved outcomes or whether the alert acted as a useful driver for quality improvement initiatives.

Conclusions

These findings strongly suggest that the introduction of a network-wide digital sepsis alert is associated with improvements in patient outcomes, demonstrating that digital based interventions can be successfully introduced and readily evaluated.

The effect of medication cost transparency alerts on prescriber behavior

OBJECTIVE

The purpose of this study was to determine if medication cost transparency alerts provided at time of prescribing led ambulatory prescribers to reduce their use of low-value medications.

MATERIALS AND METHODS

Provider-level alerts were deployed to ambulatory practices of a single health system from February 2018 through April 2018. Practice sites included 58 primary care and 152 specialty care clinics totaling 1896 attending physicians, residents, and advanced practice nurses throughout western Washington. Prescribers in the randomly assigned intervention arm received a computerized alert whenever they ordered a medication among 4 high-cost medication classes. For each class, a lower cost, equally effective, and safe alternative was available. The primary outcome was the change in prescribing volume for each of the 4 selected medication classes during the 12-week intervention period relative to a prior 24-week baseline.

RESULTS

A total of 15 456 prescriptions for high-cost medications were written during the baseline period including 7223 in the intervention arm and 8233 in the control arm. During the intervention period, a decrease in daily prescribing volume was noted for all high-cost medications including 33% for clobetasol propionate (p < .0001), 59% for doxycycline hyclate (p < .0001), 43% for fluoxetine tablets (p < .0001), and a non-significant 3% decrease for high-cost triptans (p = .65). Prescribing volume for the high-cost medications overall decreased by 32% (p < .0001).

CONCLUSION

Medication cost transparency alerts in an ambulatory setting lead to more cost-conscious prescribing. Future work is needed to predict which alerts will be most effective.

Smart Electronic Eyedrop Bottle for Unobtrusive Monitoring of Glaucoma Medication Adherence

Glaucoma, the leading cause of irreversible blindness, affects >70 million people worldwide. Lowering intraocular pressure via topical administration of eye drops is the most common first-line therapy for glaucoma. This treatment paradigm has notoriously high non-adherence rates: ranging from 30% to 80%. The advent of smart phone enabled technologies creates promise for improving eyedrop adherence. However, previous eyedrop electronic monitoring solutions had awkward medication bottle adjuncts and crude software for monitoring the administration of a drop that adversely affected their ability to foster sustainable improvements in adherence. The current work begins to address this unmet need for wireless technology by creating a “smart drop” bottle. This medication bottle is instrumented with sensing electronics that enable detection of each eyedrop administered while maintaining the shape and size of the bottle. This is achieved by a thin electronic force sensor wrapped around the bottle and underneath the label, interfaced with a thin electronic circuit underneath the bottle that allows for detection and wireless transmission to a smart-phone application. We demonstrate 100% success rate of wireless communication over 75 feet with <1% false positive and false negative rates of single drop deliveries, thus providing a viable solution for eyedrop monitoring for glaucoma patients.

Using journal alerts to support your continuing professional development

There are many ways to keep up to date with research that affects your role and personal development. You can regularly use PubMed or Scholar to find recent papers using keyword searches, you can rely on others to do the work for you with literature reviews, share the job with Journal clubs or using Journal alerts you can have the papers and research you want delivered to your inbox.

The Incidence of Paediatric Acute Kidney Injury Identified Using an AKI E-Alert Algorithm in Six English Hospitals

Objective: Acute kidney injury (AKI) is a significant cause of morbidity and mortality among hospitalised patients. The objectives in this study were (i) to investigate the incidence of AKI using the National Health Services (NHS) AKI e-alert algorithm as a means of identifying AKI; and (ii) in a randomly selected sub-group of children with AKI identified using the algorithm, to evaluate the recognition and management of AKI. Patients and Methods: Retrospective cross-sectional study with initial electronic retrieval of creatinine measurements at six hospitals in England over a six-month period. Results were evaluated using the NHS AKI e-alert algorithm with recognition and management of AKI stages 1, 2 and 3 reviewed in a sub-set of randomly selected patient case notes. Patients aged 29 to 17 years were included. AKI stage 1 was defined as a rise of 1.5 - ≤2x baseline creatinine level; AKI stage 2 a rise of ≤ 2.0 and < 3.0; AKI stage 3 a rise of ≥ 3.0. Urine output was not considered for AKI staging. Results: 57,278 creatinine measurements were analysed. 5,325 (10.8%) AKI alerts were noted in 1,112 patients with AKI 1 (62%), AKI 2 (16%) and AKI 3 (22%). There were 222 (20%) <1y, 432 (39%) 1 ≤ 6y, 192 (17%) 6 ≤ 11y, 207 (19%) 11 ≤ 16y, and 59 (5%) 16-17y. Case notes of 123 of 1,112 [11.1%] children with AKI alerts were reviewed. Confirmed AKI was recognised with a documented management plan following its identification in n = 32 [26%] patients only. Conclusions: In this first multicentre study of the incidence of AKI in children admitted to selected hospitals across England, the incidence of AKI was 10.8% with most patients under the age of 6 years and with AKI stage 1. Recognition and management of AKI was seen in just over 25% children. These data highlight the need to improve recognition of AKI in hospitalised children in the UK.

Real-World Hypoglycemia Avoidance With a Predictive Low Glucose Alert Does Not Depend on Frequent Screen Views

BACKGROUND

Frequent real-time continuous glucose monitoring (rtCGM) data viewing has been associated with reduced mean glucose and frequent scanning of an intermittently scanned continuous glucose monitoring (isCGM) system has been associated with reduced hypoglycemia for patients with diabetes. However, requiring patients to frequently interact with their glucose monitoring devices to detect actual or impending hypoglycemia is burdensome. We hypothesized that a predictive low glucose alert, which forecasts glucose ≤55 mg/dL within 20 minutes and is included in a new rtCGM system, could mitigate hypoglycemia without requiring frequent device interaction.

METHODS

We analyzed estimated glucose values (EGVs) from an anonymized convenience sample of 15,000 patients who used Dexcom G6 (Dexcom, Inc, San Diego, CA, USA) and its mobile app for at least 30 days with or without the "Urgent Low Soon" alert (ULS) enabled. Screen view frequency was determined as the frequency with which the trend screen was accessed on the app. Multiple screen views within any 5-minute interval were counted as one. Hypoglycemia exposure for patients in the top and bottom quartiles of screen view frequency (>8.25 and <3.30 per day, respectively) was calculated as the percentage of EGVs below various thresholds.

RESULTS

Over 93% of users enabled the ULS alert; its use was associated with significantly reduced hypoglycemia <55 and <70 mg/dL, independent of screen view frequency.

CONCLUSION

Use of the G6 ULS alert may disencumber rtCGM users by promoting significant reductions in hypoglycemia without requiring frequent device interactions.

Structured override reasons for drug-drug interaction alerts in electronic health records

Objective

The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records.

Materials and Methods

We collected data on DDI alerts and override reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multistage iterative card sort method to categorize the override reasons from all sites and identified best practices.

Results

Our methodology established 177 unique override reasons across the 10 sites. The number of coded override reasons at each site ranged from 3 to 100. Many sites offered override reasons not relevant to DDIs. Twelve categories of override reasons were identified. Three categories accounted for 78% of all overrides: “will monitor or take precautions,” “not clinically significant,” and “benefit outweighs risk.”

Discussion

We found wide variability in override reasons between sites and many opportunities to improve alerts. Some override reasons were irrelevant to DDIs. Many override reasons attested to a future action (eg, decreasing a dose or ordering monitoring tests), which requires an additional step after the alert is overridden, unless the alert is made actionable. Some override reasons deferred to another party, although override reasons often are not visible to other users. Many override reasons stated that the alert was inaccurate, suggesting that specificity of alerts could be improved.

Conclusions

Organizations should improve the options available to providers who choose to override DDI alerts. DDI alerting systems should be actionable and alerts should be tailored to the patient and drug pairs.

Do user preferences align with human factors assessment scores of drug-drug interaction alerts?

This study aimed to assess drug-drug interaction alert interfaces and to examine the relationship between compliance with human factors principles and user-preferences of alerts. Three reviewers independently evaluated drug-drug interaction alert interfaces in seven electronic systems using the Instrument-for-Evaluating-Human-Factors-Principles-in-Medication-Related-Decision-Support-Alerts (I-MeDeSA). Fifty-three doctors and pharmacists completed a survey to rate the alert interfaces from best to worst and reported on liked and disliked features. Human factors compliance and user-preferences of alerts were compared. Statistical analysis revealed no significant association between I-MeDeSA scores and user-preferences. However, the strengths and weaknesses of drug-drug interaction alerts from users' perspectives were in-line with the human factors constructs evaluated by the I-MeDeSA. I-MeDeSA in its current form, is unable to identify alerts that are preferred by the users. The design principles assessed by I-MeDeSA appear to be sound, but its arbitrary allocation of points to each human factors construct may not reflect the relative importance that the end-users place on different aspects of alert design.

Timely Interventions for Children with ADHD through Web-Based Monitoring Algorithms

The aim of this study was to evaluate an automated trigger algorithm designed to detect potentially adverse events in children with Attention-Deficit/Hyperactivity Disorder (ADHD), who were monitored remotely between visits. We embedded a trigger algorithm derived from parent-reported ADHD rating scales within an electronic patient monitoring system. We categorized clinicians’ alert resolution outcomes and compared Vanderbilt ADHD rating scale scores between patients who did or did not have triggered alerts. A total of 146 out of 1738 parent reports (8%) triggered alerts for 98 patients. One hundred and eleven alerts (76%) required immediate clinician review. Nurses successfully contacted parents for 68 (61%) of actionable alerts; 46% (31/68) led to a change in care plan prior to the next scheduled appointment. Compared to patients without alerts, patients with alerts demonstrated worsened ADHD severity (β = 5.8, 95% CI: 3.5–8.1 [p < 0.001] within 90 days prior to an alert. The trigger algorithm facilitated timely changes in the care plan in between face-to-face visits.

The Triad of Risk-Related Behaviors (TriRB): A Three-Dimensional Model of Cyber Risk Taking

OBJECTIVE

We identify three risk-related behaviors in coping with cyber threats-the exposure to risk a person chooses, use of security features, and responses to security indications. The combinations of behaviors that users choose determine how well they cope with threats and the severity of adverse events they experience.

BACKGROUND

End users' coping with risks is a major factor in cybersecurity. This behavior results from a combination of risk-related behaviors rather than from a single risk-taking tendency.

METHOD

In two experiments, participants played a Tetris-like game, attempting to maximize their gains, while exogenous occasional attacks could diminish earnings. An alerting system provided indications about possible attacks, and participants could take protective actions to limit the losses from attacks.

RESULTS

Variables such as the costs of protective actions, reliability of the alerting system, and attack severity affected the three behaviors differently. Also, users dynamically adjusted each of the three risk-related behaviors after gaining experience with the system.

CONCLUSION

The results demonstrate that users' risk taking is the complex combination of three behaviors rather than the expression of a general risk-taking tendency. The use of security features, exposure to risk, and responses to security indications reflect long-term strategy, short-term tactical decisions, and immediate maneuvering in coping with risks in dynamic environments.

APPLICATION

The results have implications for the analysis of cybersecurity-related decisions and actions as well as for the evaluation and design of systems and targeted interventions in other domains.

Mixed methods study of medication-related decision support alerts experienced during electronic prescribing for inpatients at an English hospital

Objectives

Electronic prescribing and medication administration systems are being introduced in many hospitals worldwide, with varying degrees of clinical decision support including pop-up alerts. Previous research suggests that prescribers override a high proportion of alerts, but little research has been carried out in the UK. Our objective was to explore rates of alert overriding in different prescribing situations and prescribers’ perceptions around the use of decision support alerts in a UK hospital.

Methods

We conducted a mixed methods study on three cardiology wards, directly observing medical and non-medical prescribers’ alert override rates during both ward round and non-ward round prescribing; observations were followed by semi-structured interviews with prescribers, which were then transcribed and analysed thematically.

Results

Overall, 69% of 199 observed alerts were overridden. Alerts experienced during ward rounds were significantly more likely to be overridden than those outside of ward rounds (80% of 56 vs 51% of 63; p=0.001, Χ² test). While respondents acknowledged that alerts could be useful, several also described negative unintended consequences. Many were of the view that usefulness of alerts was limited if the alert was reminding them to do something they would do anyway, or suggesting something they did not feel was relevant. Findings suggest that targeting, timing and additional features of alerts are critical factors in determining whether they are acted on or overridden.

Conclusion

The majority of alerts were overridden. Alerts may be less likely to be overridden if they are built into the prescribing workflow.

Asynchronous automated electronic laboratory result notifications: a systematic review

Objective

To systematically review the literature pertaining to asynchronous automated electronic notifications of laboratory results to clinicians.

Methods

PubMed, Web of Science, and the Cochrane Collaboration were queried for studies pertaining to automated electronic notifications of laboratory results. A title review was performed on the primary results, with a further abstract review and full review to produce the final set of included articles.

Results

The full review included 34 articles, representing 19 institutions. Of these, 19 reported implementation and design of systems, 11 reported quasi-experimental studies, 3 reported a randomized controlled trial, and 1 was a meta-analysis. Twenty-seven articles included alerts of critical results, while 5 focused on urgent notifications and 2 on elective notifications. There was considerable variability in clinical setting, system implementation, and results presented.

Conclusion

Several asynchronous automated electronic notification systems for laboratory results have been evaluated, most from >10 years ago. Further research on the effect of notifications on clinicians as well as the use of modern electronic health records and new methods of notification is warranted to determine their effects on workflow and clinical outcomes.

A systematic review of the types and causes of prescribing errors generated from using computerized provider order entry systems in primary and secondary care

Objective

To understand the different types and causes of prescribing errors associated with computerized provider order entry (CPOE) systems, and recommend improvements in these systems.

Materials and Methods

We conducted a systematic review of the literature published between January 2004 and June 2015 using three large databases: the Cumulative Index to Nursing and Allied Health Literature, Embase, and Medline. Studies that reported qualitative data about the types and causes of these errors were included. A narrative synthesis of all eligible studies was undertaken.

Results

A total of 1185 publications were identified, of which 34 were included in the review. We identified 8 key themes associated with CPOE-related prescribing errors: computer screen display, drop-down menus and auto-population, wording, default settings, nonintuitive or inflexible ordering, repeat prescriptions and automated processes, users' work processes, and clinical decision support systems. Displaying an incomplete list of a patient's medications on the computer screen often contributed to prescribing errors. Lack of system flexibility resulted in users employing error-prone workarounds, such as the addition of contradictory free-text comments. Users' misinterpretations of how text was presented in CPOE systems were also linked with the occurrence of prescribing errors.

Discussion and Conclusions

Human factors design is important to reduce error rates. Drop-down menus should be designed with safeguards to decrease the likelihood of selection errors. Development of more sophisticated clinical decision support, which can perform checks on free-text, may also prevent errors. Further research is needed to ensure that systems minimize error likelihood and meet users' workflow expectations.

Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial

Introduction

Several methods have been developed to electronically monitor patients for severe sepsis, but few provide predictive capabilities to enable early intervention; furthermore, no severe sepsis prediction systems have been previously validated in a randomised study. We tested the use of a machine learning-based severe sepsis prediction system for reductions in average length of stay and in-hospital mortality rate.

Methods

We conducted a randomised controlled clinical trial at two medical-surgical intensive care units at the University of California, San Francisco Medical Center, evaluating the primary outcome of average length of stay, and secondary outcome of in-hospital mortality rate from December 2016 to February 2017. Adult patients (18+) admitted to participating units were eligible for this factorial, open-label study. Enrolled patients were assigned to a trial arm by a random allocation sequence. In the control group, only the current severe sepsis detector was used; in the experimental group, the machine learning algorithm (MLA) was also used. On receiving an alert, the care team evaluated the patient and initiated the severe sepsis bundle, if appropriate. Although participants were randomly assigned to a trial arm, group assignments were automatically revealed for any patients who received MLA alerts.

Results

Outcomes from 75 patients in the control and 67 patients in the experimental group were analysed. Average length of stay decreased from 13.0 days in the control to 10.3 days in the experimental group (p=0.042). In-hospital mortality decreased by 12.4 percentage points when using the MLA (p=0.018), a relative reduction of 58.0%. No adverse events were reported during this trial.

Conclusion

The MLA was associated with improved patient outcomes. This is the first randomised controlled trial of a sepsis surveillance system to demonstrate statistically significant differences in length of stay and in-hospital mortality.

Trial registration

NCT03015454.

Clinical decisions support malfunctions in a commercial electronic health record

OBJECTIVES

Determine if clinical decision support (CDS) malfunctions occur in a commercial electronic health record (EHR) system, characterize their pathways and describe methods of detection.

METHODS

We retrospectively examined the firing rate for 226 alert type CDS rules for detection of anomalies using both expert visualization and statistical process control (SPC) methods over a five year period. Candidate anomalies were investigated and validated.

RESULTS

Twenty-one candidate CDS anomalies were identified from 8,300 alert-months. Of these candidate anomalies, four were confirmed as CDS malfunctions, eight as false-positives, and nine could not be classified. The four CDS malfunctions were a result of errors in knowledge management: 1) inadvertent addition and removal of a medication code to the electronic formulary list; 2) a seasonal alert which was not activated; 3) a change in the base data structures; and 4) direct editing of an alert related to its medications. 154 CDS rules (68%) were amenable to SPC methods and the test characteristics were calculated as a sensitivity of 95%, positive predictive value of 29% and F-measure 0.44.

DISCUSSION

CDS malfunctions were found to occur in our EHR. All of the pathways for these malfunctions can be described as knowledge management errors. Expert visualization is a robust method of detection, but is resource intensive. SPC-based methods, when applicable, perform reasonably well retrospectively.

CONCLUSION

CDS anomalies were found to occur in a commercial EHR and visual detection along with SPC analysis represents promising methods of malfunction detection.

More Than a Text Message: Dismantling Digital Triggers to Curate Behavior Change in Patient-Centered Health Interventions

Digital triggers such as text messages, emails, and push alerts are designed to focus an individual on a desired goal by prompting an internal or external reaction at the appropriate time. Triggers therefore have an essential role in engaging individuals with digital interventions delivered outside of traditional health care settings, where other events in daily lives and fluctuating motivation to engage in effortful behavior exist. There is an emerging body of literature examining the use of digital triggers for short-term action and longer-term behavior change. However, little attention has been given to understanding the components of digital triggers. Using tailoring as an overarching framework, we separated digital triggers into 5 primary components: (1) who (sender), (2) how (stimulus type, delivery medium, heterogeneity), (3) when (delivered), (4) how much (frequency, intensity), and (5) what (trigger's target, trigger's structure, trigger's narrative). We highlighted key considerations when tailoring each component and the pitfalls of ignoring common mistakes, such as alert fatigue and habituation. As evidenced throughout the paper, there is a broad literature base from which to draw when tailoring triggers to curate behavior change in health interventions. More research is needed, however, to examine differences in efficacy based on component tailoring, to best use triggers to facilitate behavior change over time, and to keep individuals engaged in physical and mental health behavior change efforts. Dismantling digital triggers into their component parts and reassembling them according to the gestalt of one's change goals is the first step in this development work.

Computerized pharmacy surveillance and alert system for drug-related problems

WHAT IS KNOWN AND OBJECTIVE

Because of the impact of drug-related problems (DRPs) on morbidity and mortality, there is a need for computerized strategies to increase drug safety. The detection and identification of the causes of potential DRPs can be facilitated by the incorporation of a pharmacy warning system (PWS) in the computerized prescriber order entry (CPOE) and its application in the routine validation of inpatient drug therapy. A limited number of studies have evaluated a clinical decision support system to monitor drug treatment. Most of these applications have utilized a small range of drugs with alerts and/or types of alert. The objective of this study was to describe the implementation of a PWS integrated in the electronic medical record (EMR).

METHODS

The PWS was developed in 2003-2004. Pharmacological information to generate drug alerts was entered on demographic data, drug dosage, laboratory tests related to the prescribed drug and drug combinations (interactions, duplications and necessary combinations). The PWS was applied in the prescription reviews conducted in patients admitted to the hospital in 2012.

RESULTS AND DISCUSSION

Information on 83% of the drugs included in the pharmacopeia was introduced into the PWS, allowing detection of 2808 potential DRPs, representing 79·1% of all potential DRPs detected during the study period. Twenty per cent of PWS DRPs were clinically relevant, requiring pharmacist intervention.

WHAT IS NEW AND CONCLUSION

The PWS detected most potential DRPs, thus increasing inpatient safety. The detection ability of the PWS was higher than that reported for other tools described in the literature.

Feedback and Training to Improve Use of an Electronic Prescribing System: A Randomised Controlled Trial

Excessive presentation of alerts in electronic prescribing systems (ePS) results in 'alert fatigue' which reduces alert effectiveness and frustrates users. Previous research at our study site showed high rates of duplication alerts, some of which were the result of doctors not using available short-cut functions in the ePS. This study aimed to improve uptake of short-cut functions and so reduce alert fatigue by trialing two interventions: feedback and training. Fifty doctors were randomised to one of three groups: Control, Feedback or Training. The Feedback group received an individualised feedback report via email and the Training group received brief face-to-face refresher training. Participants partook in informal interviews to discuss the training and the ePS in use. The proportion of orders which triggered a duplication alert was our primary outcome measure. Neither intervention had a significant impact on duplication alert rate (Feedback: 80.8% vs. 77.8% of orders, Training: 77.5% vs. 76.5% of orders; all p&gt;0.05). We identified a number of factors related to the intervention, ePS and prescribing environment that contributed to this result. Rather than focusing on changing prescribing behaviour, we suggest a more effective and appropriate approach is to redesign the ePS so that fewer and more meaningful alerts are presented.

The frequency of inappropriate nonformulary medication alert overrides in the inpatient setting

BACKGROUND

Experts suggest that formulary alerts at the time of medication order entry are the most effective form of clinical decision support to automate formulary management.

OBJECTIVE

Our objectives were to quantify the frequency of inappropriate nonformulary medication (NFM) alert overrides in the inpatient setting and provide insight on how the design of formulary alerts could be improved.

METHODS

Alert overrides of the top 11 (n = 206) most-utilized and highest-costing NFMs, from January 1 to December 31, 2012, were randomly selected for appropriateness evaluation. Using an empirically developed appropriateness algorithm, appropriateness of NFM alert overrides was assessed by 2 pharmacists via chart review. Appropriateness agreement of overrides was assessed with a Cohen's kappa. We also assessed which types of NFMs were most likely to be inappropriately overridden, the override reasons that were disproportionately provided in the inappropriate overrides, and the specific reasons the overrides were considered inappropriate.

RESULTS

Approximately 17.2% (n = 35.4/206) of NFM alerts were inappropriately overridden. Non-oral NFM alerts were more likely to be inappropriately overridden compared to orals. Alerts overridden with "blank" reasons were more likely to be inappropriate. The failure to first try a formulary alternative was the most common reason for alerts being overridden inappropriately.

CONCLUSION

Approximately 1 in 5 NFM alert overrides are overridden inappropriately. Future research should evaluate the impact of mandating a valid override reason and adding a list of formulary alternatives to each NFM alert; we speculate these NFM alert features may decrease the frequency of inappropriate overrides.

The vulnerabilities of computerized physician order entry systems: a qualitative study

OBJECTIVE

To test the vulnerabilities of a wide range of computerized physician order entry (CPOE) systems to different types of medication errors, and develop a more comprehensive qualitative understanding of how their design could be improved.

MATERIALS AND METHODS

The authors reviewed a random sample of 63,040 medication error reports from the US Pharmacopeia (USP) MEDMARX reporting system where CPOE systems were considered a "contributing factor" to errors and flagged test scenarios that could be tested in current CPOE systems. Testers entered these orders in 13 commercial and homegrown CPOE systems across 16 different sites in the United States and Canada, using both usual practice and where-needed workarounds. Overarching themes relevant to interface design and usability/workflow issues were identified.

RESULTS

CPOE systems often failed to detect and prevent important medication errors. Generation of electronic alert warnings varied widely between systems, and depended on a number of factors, including how the order information was entered. Alerts were often confusing, with unrelated warnings appearing on the same screen as those more relevant to the current erroneous entry. Dangerous drug-drug interaction warnings were displayed only after the order was placed rather than at the time of ordering. Testers illustrated various workarounds that allowed them to enter these erroneous orders.

DISCUSSION AND CONCLUSION

The authors found high variability in ordering approaches between different CPOE systems, with major deficiencies identified in some systems. It is important that developers reflect on these findings and build in safeguards to ensure safer prescribing for patients.

Using a mobile health application to support self-management in chronic obstructive pulmonary disease: a six-month cohort study

BACKGROUND

Self-management strategies have the potential to support patients with chronic obstructive pulmonary disease (COPD). Telehealth interventions may have a role in delivering this support along with the opportunity to monitor symptoms and physiological variables. This paper reports findings from a six-month, clinical, cohort study of COPD patients' use of a mobile telehealth based (mHealth) application and how individually determined alerts in oxygen saturation levels, pulse rate and symptoms scores related to patient self-initiated treatment for exacerbations.

METHODS

The development of the mHealth intervention involved a patient focus group and multidisciplinary team of researchers, engineers and clinicians. Individual data thresholds to set alerts were determined, and the relationship to exacerbations, defined by the initiation of stand-by medications, was measured. The sample comprised 18 patients (age range of 50-85 years) with varied levels of computer skills.

RESULTS

Patients identified no difficulties in using the mHealth application and used all functions available. 40% of exacerbations had an alert signal during the three days prior to a patient starting medication. Patients were able to use the mHealth application to support self- management, including monitoring of clinical data. Within three months, 95% of symptom reporting sessions were completed in less than 100 s.

CONCLUSIONS

Home based, unassisted, daily use of the mHealth platform is feasible and acceptable to people with COPD for reporting daily symptoms and medicine use, and to measure physiological variables such as pulse rate and oxygen saturation. These findings provide evidence for integrating telehealth interventions with clinical care pathways to support self-management in COPD.

The MOnitoring Resynchronization dEvices and CARdiac patiEnts (MORE-CARE) randomized controlled trial: phase 1 results on dynamics of early intervention with remote monitoring

Background

Remote monitoring (RM) in patients with advanced heart failure and cardiac resynchronization therapy defibrillators (CRT-D) may reduce delays in clinical decisions by transmitting automatic alerts. However, this strategy has never been tested specifically in this patient population, with alerts for lung fluid overload, and in a European setting.

Objective

The main objective of Phase 1 (presented here) is to evaluate if RM strategy is able to reduce time from device-detected events to clinical decisions.

Methods

In this multicenter randomized controlled trial, patients with moderate to severe heart failure implanted with CRT-D devices were randomized to a Remote group (with remote follow-up and wireless automatic alerts) or to a Control group (with standard follow-up without alerts). The primary endpoint of Phase 1 was the delay between an alert event and clinical decisions related to the event in the first 154 enrolled patients followed for 1 year.

Results

The median delay from device-detected events to clinical decisions was considerably shorter in the Remote group compared to the Control group: 2 (25^th-75^th percentile, 1-4) days vs 29 (25^th-75^th percentile, 3-51) days respectively, P=.004. In-hospital visits were reduced in the Remote group (2.0 visits/patient/year vs 3.2 visits/patient/year in the Control group, 37.5% relative reduction, P<.001). Automatic alerts were successfully transmitted in 93% of events occurring outside the hospital in the Remote group. The annual rate of all-cause hospitalizations per patient did not differ between the two groups (P=.65).

Conclusions

RM in CRT-D patients with advanced heart failure allows physicians to promptly react to clinically relevant automatic alerts and significantly reduces the burden of in-hospital visits.

Trial Registration

Clinicaltrials.gov NCT00885677; http://clinicaltrials.gov/show/NCT00885677 (Archived by WebCite at http://www.webcitation.org/6IkcCJ7NF).