An Observational Study to Evaluate the Usability and Intent to Adopt an Artificial Intelligence-Powered Medication Reconciliation Tool

Medication reconciliationis it possible to speed up without compromising quality? A before-after study in the emergency department

OBJECTIVE

The aim of this study was to investigate whether it was possible to decrease the time used for medication reconciliation (MR) in the emergency department without compromising quality. A more efficient method will enable more patients to receive MR as early as possible after admission to hospital.

METHODS

Potential key factors for improvement of the standard method of MR by clinical pharmacists were identified through an observational period. A revised method was developed, focusing on decreasing time spent on the patient interview by use of a condensed checklist and probing questions based on information from a prescription database. Non-inferior quality (proportion of patients with at least one identified medication discrepancy and number of identified medication discrepancies per patient) of the revised method was evaluated using a before-after study design with 200 individuals in each group. Non-inferiority limit was set at 10%. The Mann-Whitney U test was used for statistical evaluation of the difference in time use per patient in the MR process between the before and after group.

RESULTS

Mean age of the included patients was 78 years in both groups. The time used for MR in the after group was 34% shorter (37 min vs 56 min, p<0.0001) compared with the before group. The revised method was shown to be non-inferior compared with the original method with respect to the proportion of patients with at least one identified discrepancy (81%, 95% CI 76% to 86% vs 79%, 95% CI 73% to 84%). Also, non-inferiority was shown for the number of identified discrepancies per patient, where the average number of discrepancies per patient was 1.9 (95% CI 1.7 to 2.1) in both groups.

CONCLUSION

This study showed that it was possible to speed up the MR process without compromising its effectiveness in identifying medication discrepancies.

Inclusion of Clinicians in the Development and Evaluation of Clinical Artificial Intelligence Tools: A Systematic Literature Review

The application of machine learning (ML) and artificial intelligence (AI) in healthcare domains has received much attention in recent years, yet significant questions remain about how these new tools integrate into frontline user workflow, and how their design will impact implementation. Lack of acceptance among clinicians is a major barrier to the translation of healthcare innovations into clinical practice. In this systematic review, we examine when and how clinicians are consulted about their needs and desires for clinical AI tools. Forty-five articles met criteria for inclusion, of which 24 were considered design studies. The design studies used a variety of methods to solicit and gather user feedback, with interviews, surveys, and user evaluations. Our findings show that tool designers consult clinicians at various but inconsistent points during the design process, and most typically at later stages in the design cycle (82%, 19/24 design studies). We also observed a smaller amount of studies adopting a human-centered approach and where clinician input was solicited throughout the design process (22%, 5/24). A third (15/45) of all studies reported on clinician trust in clinical AI algorithms and tools. The surveyed articles did not universally report validation against the “gold standard” of clinical expertise or provide detailed descriptions of the algorithms or computational methods used in their work. To realize the full potential of AI tools within healthcare settings, our review suggests there are opportunities to more thoroughly integrate frontline users’ needs and feedback in the design process.

Artificial Intelligence Solutions to Increase Medication Adherence in Patients With Non-communicable Diseases

Artificial intelligence (AI) tools are increasingly being used within healthcare for various purposes, including helping patients to adhere to drug regimens. The aim of this narrative review was to describe: (1) studies on AI tools that can be used to measure and increase medication adherence in patients with non-communicable diseases (NCDs); (2) the benefits of using AI for these purposes; (3) challenges of the use of AI in healthcare; and (4) priorities for future research. We discuss the current AI technologies, including mobile phone applications, reminder systems, tools for patient empowerment, instruments that can be used in integrated care, and machine learning. The use of AI may be key to understanding the complex interplay of factors that underly medication non-adherence in NCD patients. AI-assisted interventions aiming to improve communication between patients and physicians, monitor drug consumption, empower patients, and ultimately, increase adherence levels may lead to better clinical outcomes and increase the quality of life of NCD patients. However, the use of AI in healthcare is challenged by numerous factors; the characteristics of users can impact the effectiveness of an AI tool, which may lead to further inequalities in healthcare, and there may be concerns that it could depersonalize medicine. The success and widespread use of AI technologies will depend on data storage capacity, processing power, and other infrastructure capacities within healthcare systems. Research is needed to evaluate the effectiveness of AI solutions in different patient groups and establish the barriers to widespread adoption, especially in light of the COVID-19 pandemic, which has led to a rapid increase in the use and development of digital health technologies.

Patients' perceptions of using artificial intelligence (AI)-based technology to comprehend radiology imaging data

Results of radiology imaging studies are not typically comprehensible to patients. With the advances in artificial intelligence (AI) technology in recent years, it is expected that AI technology can aid patients' understanding of radiology imaging data. The aim of this study is to understand patients' perceptions and acceptance of using AI technology to interpret their radiology reports. We conducted semi-structured interviews with 13 participants to elicit reflections pertaining to the use of AI technology in radiology report interpretation. A thematic analysis approach was employed to analyze the interview data. Participants have a generally positive attitude toward using AI-based systems to comprehend their radiology reports. AI is perceived to be particularly useful in seeking actionable information, confirming the doctor's opinions, and preparing for the consultation. However, we also found various concerns related to the use of AI in this context, such as cyber-security, accuracy, and lack of empathy. Our results highlight the necessity of providing AI explanations to promote people's trust and acceptance of AI. Designers of patient-centered AI systems should employ user-centered design approaches to address patients' concerns. Such systems should also be designed to promote trust and deliver concerning health results in an empathetic manner to optimize the user experience.

Effect of AI Explanations on Human Perceptions of Patient-Facing AI-Powered Healthcare Systems

Ongoing research efforts have been examining how to utilize artificial intelligence technology to help healthcare consumers make sense of their clinical data, such as diagnostic radiology reports. How to promote the acceptance of such novel technology is a heated research topic. Recent studies highlight the importance of providing local explanations about AI prediction and model performance to help users determine whether to trust AI's predictions. Despite some efforts, limited empirical research has been conducted to quantitatively measure how AI explanations impact healthcare consumers' perceptions of using patient-facing, AI-powered healthcare systems. The aim of this study is to evaluate the effects of different AI explanations on people's perceptions of AI-powered healthcare system. In this work, we designed and deployed a large-scale experiment (N = 3,423) on Amazon Mechanical Turk (MTurk) to evaluate the effects of AI explanations on people's perceptions in the context of comprehending radiology reports. We created four groups based on two factors-the extent of explanations for the prediction (High vs. Low Transparency) and the model performance (Good vs. Weak AI Model)-and randomly assigned participants to one of the four conditions. Participants were instructed to classify a radiology report as describing a normal or abnormal finding, followed by completing a post-study survey to indicate their perceptions of the AI tool. We found that revealing model performance information can promote people's trust and perceived usefulness of system outputs, while providing local explanations for the rationale of a prediction can promote understandability but not necessarily trust. We also found that when model performance is low, the more information the AI system discloses, the less people would trust the system. Lastly, whether human agrees with AI predictions or not and whether the AI prediction is correct or not could also influence the effect of AI explanations. We conclude this paper by discussing implications for designing AI systems for healthcare consumers to interpret diagnostic report.

Root causes of adverse drug events in hospitals and artificial intelligence capabilities for prevention

AIMS

To identify and prioritize the root causes of adverse drug events (ADEs) in hospitals and to assess the ability of artificial intelligence (AI) capabilities to prevent ADEs.

DESIGN

A mixed method design was used.

METHODS

A cross-sectional study for hospitals in Spain was carried out between February and April 2019 to identify and prioritize the root causes of ADEs. A nominal group technique was also used to assess the ability of AI capabilities to prevent ADEs.

RESULTS

The main root cause of ADEs was a lack of adherence to safety protocols (64.8%), followed by identification errors (57.4%), and fragile and polymedicated patients (44.4%). An analysis of the AI capabilities to prevent the root causes of ADEs showed that identification and reading are two potentially useful capabilities.

CONCLUSION

Identification error is one of the main root causes of drug adverse events and AI capabilities could potentially prevent drug adverse events.

IMPACT

This study highlights the role of AI capabilities in safely identifying both patients and drugs, which is a crucial part of the medication administration process, and how this can prevent ADEs in hospitals.

Medication Reconciliation: The Foundation of Medication Safety for Patients Requiring Dialysis

Medication-related problems are a leading cause of morbidity and mortality. Patients requiring dialysis are at heightened risk for adverse drug reactions because of the prevalence of polypharmacy, multiple chronic conditions, and altered (but not well understood) medication pharmacokinetics and pharmacodynamics inherent to kidney failure. To minimize preventable medication-related problems, health care providers need to prioritize medication safety for this population. The cornerstone of medication safety is medication reconciliation. We present a case highlighting adverse outcomes when medication reconciliation is insufficient at care transitions. We review available literature on the prevalence of medication discrepancies worldwide. We also explain effective medication reconciliation and the practical considerations for implementation of effective medication reconciliation in dialysis units. In light of the addition of medication reconciliation requirements to the Centers for Medicare & Medicaid Services End-Stage Renal Disease Quality Incentive Program, this review also provides guidance to dialysis unit leadership for improving current medication reconciliation practices. Prioritization of medication reconciliation has the potential to positively affect rates of medication-related problems, as well as medication adherence, health care costs, and quality of life.

Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review

BACKGROUND

Artificial intelligence (AI) provides opportunities to identify the health risks of patients and thus influence patient safety outcomes.

OBJECTIVE

The purpose of this systematic literature review was to identify and analyze quantitative studies utilizing or integrating AI to address and report clinical-level patient safety outcomes.

METHODS

We restricted our search to the PubMed, PubMed Central, and Web of Science databases to retrieve research articles published in English between January 2009 and August 2019. We focused on quantitative studies that reported positive, negative, or intermediate changes in patient safety outcomes using AI apps, specifically those based on machine-learning algorithms and natural language processing. Quantitative studies reporting only AI performance but not its influence on patient safety outcomes were excluded from further review.

RESULTS

We identified 53 eligible studies, which were summarized concerning their patient safety subcategories, the most frequently used AI, and reported performance metrics. Recognized safety subcategories were clinical alarms (n=9; mainly based on decision tree models), clinical reports (n=21; based on support vector machine models), and drug safety (n=23; mainly based on decision tree models). Analysis of these 53 studies also identified two essential findings: (1) the lack of a standardized benchmark and (2) heterogeneity in AI reporting.

CONCLUSIONS

This systematic review indicates that AI-enabled decision support systems, when implemented correctly, can aid in enhancing patient safety by improving error detection, patient stratification, and drug management. Future work is still needed for robust validation of these systems in prospective and real-world clinical environments to understand how well AI can predict safety outcomes in health care settings.

A web application to involve patients in the medication reconciliation process: a user-centered usability and usefulness study

Objective

Medication reconciliation (MedRec) can improve patient safety by resolving medication discrepancies. Because information technology (IT) and patient engagement are promising approaches to optimizing MedRec, the SEAMPAT project aims to develop a MedRec IT platform based on two applications: the "patient app" and the "MedRec app." This study evaluates three dimensions of the usability (efficiency, satisfaction, and effectiveness) and usefulness of the patient app.

Methods

We performed a four-month user-centered observational study. Quantitative and qualitative data were collected. Participants completed the system usability scale (SUS) questionnaire and a second questionnaire on usefulness. Effectiveness was assessed by measuring the completeness of the medication list generated by the patient application and its correctness (ie medication discrepancies between the patient list and the best possible medication history). Qualitative data were collected from semi-structured interviews, observations and comments, and questions raised by patients.

Results

Forty-two patients completed the study. Sixty-nine percent of patients considered the patient app to be acceptable (SUS Score ≥ 70) and usefulness was high. The medication list was complete for a quarter of the patients (7/28) and there was a discrepancy for 21.7% of medications (21/97). The qualitative data enabled the identification of several barriers (related to functional and non-functional aspects) to the optimization of usability and usefulness.

Conclusions

Our findings highlight the importance and value of user-centered usability testing of a patient application implemented in "real-world" conditions. To achieve adoption and sustained use by patients, the app should meet patients' needs while also efficiently improving the quality of MedRec.

Using a Machine Learning System to Identify and Prevent Medication Prescribing Errors: A Clinical and Cost Analysis Evaluation

BACKGROUND

Clinical decision support (CDS) alerting tools can identify and reduce medication errors. However, they are typically rule-based and can identify only the errors previously programmed into their alerting logic. Machine learning holds promise for improving medication error detection and reducing costs associated with adverse events. This study evaluates the ability of a machine learning system (MedAware) to generate clinically valid alerts and estimates the cost savings associated with potentially prevented adverse events.

METHODS

Alerts were generated retrospectively by the MedAware system on outpatient data from two academic medical centers between 2009 and 2013. MedAware alerts were compared to alerts in an existing CDS system. A random sample of 300 alerts was selected for medical record review. Frequency and severity of potential outcomes of alerted medication errors of medium and high clinical value were estimated, along with associated health care costs of these potentially prevented adverse events.

RESULTS

A total of 10,668 alerts were generated. Overall, 68.2% of MedAware alerts would not have been generated by the existing CDS system. Ninety-two percent of a random sample of the chart-reviewed alerts were accurate based on structured data available in the record, and 79.7% were clinically valid. Estimated cost of adverse events potentially prevented in an outpatient setting was more than $60 per drug alert and $1.3 million when extrapolating study findings to the full patient population.

CONCLUSION

A machine learning system identified clinically valid medication error alerts that might otherwise be missed with existing CDS systems. Estimates show potential for cost savings associated with potentially prevented adverse events.

Impact of Computer-Based and Pharmacist-Assisted Medication Review Initiated in the Emergency Department

OBJECTIVES

Whether early medication reconciliation and integration can reduce polypharmacy and potentially inappropriate medication (PIM) in the emergency department (ED) remains unclear. Polypharmacy and PIM have been recognized as significant causes of adverse drug events in older adults. Therefore, this pilot study was conducted to delineate this issue.

DESIGN

An interventional study.

SETTING

A medical center in Taiwan.

PARTICIPANTS

Older ED patients (aged ≥65 years) awaiting hospitalization between December 1, 2017, and October 31, 2018 were recruited in this study. A multidisciplinary team and a computer-based and pharmacist-assisted medication reconciliation and integration system were implemented.

MEASUREMENTS

The reduced proportions of major polypharmacy (≥10 medications) and PIM at hospital discharge were compared with those on admission to the ED between pre- and post-intervention periods.

RESULTS

A total of 911 patients (pre-intervention = 243 vs post-intervention = 668) were recruited. The proportions of major polypharmacy and PIM were lower in the post-intervention than in the pre-intervention period (-79.4% vs -65.3%; P < .001, and - 67.5% vs -49.1%; P < .001, respectively). The number of medications was reduced from 12.5 ± 2.7 to 6.9 ± 3.0 in the post-intervention period in patients with major polypharmacy (P < .001).

CONCLUSION

Early initiation of computer-based and pharmacist-assisted intervention in the ED for reducing major polypharmacy and PIM is a promising method for improving geriatric care and reducing medical expenditures. J Am Geriatr Soc 67:2298-2304, 2019.