Implementation costs of hospital-based computerised decision support systems: a systematic review

How intervention studies measure the effectiveness of medication safety-related clinical decision support systems in primary and long-term care: a systematic review

BACKGROUND

Medication errors and associated adverse drug events (ADE) are a major cause of morbidity and mortality worldwide. In recent years, the prevention of medication errors has become a high priority in healthcare systems. In order to improve medication safety, computerized Clinical Decision Support Systems (CDSS) are increasingly being integrated into the medication process. Accordingly, a growing number of studies have investigated the medication safety-related effectiveness of CDSS. However, the outcome measures used are heterogeneous, leading to unclear evidence. The primary aim of this study is to summarize and categorize the outcomes used in interventional studies evaluating the effects of CDSS on medication safety in primary and long-term care.

METHODS

We systematically searched PubMed, Embase, CINAHL, and Cochrane Library for interventional studies evaluating the effects of CDSS targeting medication safety and patient-related outcomes. We extracted methodological characteristics, outcomes and empirical findings from the included studies. Outcomes were assigned to three main categories: process-related, harm-related, and cost-related. Risk of bias was assessed using the Evidence Project risk of bias tool.

RESULTS

Thirty-two studies met the inclusion criteria. Almost all studies (n = 31) used process-related outcomes, followed by harm-related outcomes (n = 11). Only three studies used cost-related outcomes. Most studies used outcomes from only one category and no study used outcomes from all three categories. The definition and operationalization of outcomes varied widely between the included studies, even within outcome categories. Overall, evidence on CDSS effectiveness was mixed. A significant intervention effect was demonstrated by nine of fifteen studies with process-related primary outcomes (60%) but only one out of five studies with harm-related primary outcomes (20%). The included studies faced a number of methodological problems that limit the comparability and generalizability of their results.

CONCLUSIONS

Evidence on the effectiveness of CDSS is currently inconclusive due in part to inconsistent outcome definitions and methodological problems in the literature. Additional high-quality studies are therefore needed to provide a comprehensive account of CDSS effectiveness. These studies should follow established methodological guidelines and recommendations and use a comprehensive set of harm-, process- and cost-related outcomes with agreed-upon and consistent definitions.

PROSPERO REGISTRATION

CRD42023464746.

Is ChatGPT reliable and accurate in answering pharmacotherapy-related inquiries in both Turkish and English?

INTRODUCTION

Artificial intelligence (AI), particularly ChatGPT, is becoming more and more prevalent in the healthcare field for tasks such as disease diagnosis and medical record analysis. The objective of this study is to evaluate the proficiency and accuracy of ChatGPT in different domains of clinical pharmacy cases and queries.

METHODS

The study NAPLEX® Review Questions, 4th edition, pertaining to 10 different chronic conditions compared ChatGPT's responses to pharmacotherapy cases and questions obtained from McGraw Hill's, alongside the answers provided by the book's authors. The proportion of correct responses was collected and analyzed using the Statistical Package for the Social Sciences (SPSS) version 29.

RESULTS

When tested in English, ChatGPT had substantially higher mean scores than when tested in Turkish. The average accurate score for English and Turkish was 0.41 ± 0.49 and 0.32 ± 0.46, respectively, p = 0.18. Responses to queries beginning with "Which of the following is correct?" are considerably more precise than those beginning with "Mark all the incorrect answers?" 0.66 ± 0.47 as opposed to 0.16 ± 0.36; p = 0.01 in English language and 0.50 ± 0.50 as opposed to 0.14 ± 0.34; p < 0.05in Turkish language.

CONCLUSION

ChatGPT displayed a moderate level of accuracy while responding to English inquiries, but it displayed a slight level of accuracy when responding to Turkish inquiries, contingent upon the question format. Improving the accuracy of ChatGPT in languages other than English requires the incorporation of several components. The integration of the English version of ChatGPT into clinical practice has the potential to improve the effectiveness, precision, and standard of patient care provision by supplementing personal expertise and professional judgment. However, it is crucial to utilize technology as an adjunct and not a replacement for human decision-making and critical thinking.

We Ask and Listen: A Group-Wide Retrospective Survey on Satisfaction with Digital Medication Software

Purpose

Computerized physician order entry (CPOE) and clinical decision support systems (CDSS) are used internationally since the 1980s. These systems reduce costs, enhance drug therapy safety, and improve quality of care. A few years ago, there was a growing effort to digitize the healthcare sector in Germany. Implementing such systems like CPOE-CDSS requires training for effective adoption and, more important, acceptance by the users. Potential improvements for the software and implementation process can be derived from the users' perspective. The implementation process is globally relevant and applicable across professions due to the constant advancement of digitalization. The study assessed the implementation of medication software and overall satisfaction.

Methods

In an anonymous voluntary online survey, physicians and nursing staff were asked about their satisfaction with the new CPOE-CDSS. The survey comprised single-choice queries on a Likert scale, categorizing into general information, digital medication administration, drug safety, and software introduction. In addition multiple-choice questions are mentioned. Data analysis was performed using Microsoft Office Excel 2016 and GraphPad PRISM 9.5.0.

Results

Nurses and physicians' satisfaction with the new software increased with usage hours. The software's performance and loading times have clearly had a negative impact, which leads to a low satisfaction of only 20% among physicians and 17% among nurses. 53% of nurses find the program's training period unsuitable for their daily use, while 57% of physicians approve the training's scope for their professional group. Both professions agree that drug-related problems are easier to detect using CPOE-CDSS, with 76% of nurses and 75% of physicians agreeing. The study provides unbiased feedback on software implementation.

Conclusion

In conclusion, digitizing healthcare requires managing change, effective training, and addressing software functionality concerns to ensure improved medication safety and streamlined processes. Interfaces, performance optimization, and training remain crucial for software acceptance and effectiveness.

Effectiveness of ChatGPT in clinical pharmacy and the role of artificial intelligence in medication therapy management

BACKGROUND

The use of artificial intelligence (AI) to optimize medication therapy management (MTM) in identifying drug interactions may potentially improve MTM efficiency. ChatGPT, an AI language model, may be applied to identify medication interventions by integrating patient and drug databases. ChatGPT has been shown to be effective in other areas of clinical medicine, from diagnosis to patient management. However, ChatGPT's ability to manage MTM related activities is little known.

OBJECTIVES

To evaluate the effectiveness of ChatGPT in MTM services in simple, complex, and very complex cases to understand AI contributions in MTM.

METHODS

Two clinical pharmacists rated and validated the difficulty of patient cases from simple, complex, and very complex. ChatGPT's response to the cases was assessed based on 3 criteria: the ability to identify drug interactions, precision in recommending alternatives, and appropriateness in devising management plans. Two clinical pharmacists validated the accuracy of ChatGPT's responses and compared them to actual answers for each complexity level.

RESULTS

ChatGPT 4.0 accurately solved 39 out of 39 (100 %) patient cases. ChatGPT successfully identified drug interactions, provided therapy recommendations and formulated general management plans, but it did not recommend specific dosages. Results suggest it can assist pharmacists in formulating MTM plans to improve overall efficiency.

CONCLUSION

The application of ChatGPT in MTM has the potential to enhance patient safety and involvement, lower healthcare costs, and assist healthcare providers in medication management and identifying drug interactions. Future pharmacists can utilize AI models such as ChatGPT to improve patient care. The future of the pharmacy profession will depend on how the field responds to the changing need for patient care optimized by AI and automation.

Achieving Value by Risk Stratification With Machine Learning Model or Clinical Risk Score in Acute Upper Gastrointestinal Bleeding: A Cost Minimization Analysis

INTRODUCTION

We estimate the economic impact of applying risk assessment tools to identify very low-risk patients with upper gastrointestinal bleeding who can be safely discharged from the emergency department using a cost minimization analysis.

METHODS

We compare triage strategies (Glasgow-Blatchford score = 0/0-1 or validated machine learning model) with usual care using a Markov chain model from a US health care payer perspective.

RESULTS

Over 5 years, the Glasgow-Blatchford score triage strategy produced national cumulative savings over usual care of more than $2.7 billion and the machine learning strategy of more than $3.4 billion.

DISCUSSION

Implementing risk assessment models for upper gastrointestinal bleeding reduces costs, thereby increasing value.

Financial Impact of a Radiology Safety Net Program for Resolution of Clinically Necessary Follow-up Imaging Recommendations

OBJECTIVE

Health care safety net (SN) programs can potentially improve patient safety and decrease risk associated with missed or delayed follow-up care, although they require financial resources. This study aimed to assess whether the revenue generated from completion of clinically necessary recommendations for additional imaging (RAI) made possible by an IT-enabled SN program could fund the required additional labor resources.

METHODS

Clinically necessary RAI generated October 21, 2019, to September 24, 2021, were tracked to resolution as of April 13, 2023. A new radiology SN team worked with existing schedulers and care coordinators, performing chart review and patient and provider outreach to ensure RAI resolution. We applied relevant Current Procedural Terminology, version 4 codes of the completed imaging examinations to estimate total revenue. Coprimary outcomes included revenue generated by total performed examinations and estimated revenue attributed to SN involvement. We used Student's t test to compare the secondary outcome, RAI time interval, for higher versus lower revenue-generating modalities.

RESULTS

In all, 24% (3,243) of eligible follow-up recommendations (13,670) required SN involvement. Total estimated revenue generated by performed recommended examinations was $6,116,871, with $980,628 attributed to SN. Net SN-generated revenue per 1.0 full-time equivalent was an estimated $349,768. Greatest proportion of performed examinations were cross-sectional modalities (CT, MRI, PET/CT), which were higher revenue-generating than non-cross-sectional modalities (x-ray, ultrasound, mammography), and had shorter recommendation time frames (153 versus 180 days, P < .001).

DISCUSSION

The revenue generated from completion of RAI facilitated by an IT-enabled quality and safety program supplemented by an SN team can fund the required additional labor resources to improve patient safety. Realizing early revenue may require 5 to 6 months postimplementation.

Cost-effectiveness of left atrial appendage closure for stroke prevention in atrial fibrillation: a systematic review appraising the methodological quality

BACKGROUND

The increasing global prevalence of atrial fibrillation (AF) has led to a growing demand for stroke prevention strategies, resulting in higher healthcare costs. High-quality economic evaluations of stroke prevention strategies can play a crucial role in maximising efficient allocation of resources. In this systematic review, we assessed the methodological quality of such economic evaluations.

METHODS

We searched electronic databases of PubMed, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, and Econ Lit to identify model-based economic evaluations comparing the left atrial appendage closure procedure (LAAC) and oral anticoagulants published in English since 2000. Data on study characteristics, model-based details, and analyses were collected. The methodological quality was evaluated using the modified Economic Evaluations Bias (ECOBIAS) checklist. For each of the 22 biases listed in this checklist, studies were categorised into one of four groups: low risk, partial risk, high risk due to inadequate reporting, or high risk. To gauge the overall quality of each study, we computed a composite score by assigning + 2, 0, - 1 and - 2 to each risk category, respectively.

RESULTS

In our analysis of 12 studies, majority adopted a healthcare provider or payer perspective and employed Markov Models with the number of health states varying from 6 to 16. Cost-effectiveness results varied across studies. LAAC displayed a probability exceeding 50% of being the cost-effective option in six out of nine evaluations compared to warfarin, six out of eight evaluations when compared to dabigatran, in three out of five evaluations against apixaban, and in two out of three studies compared to rivaroxaban. The methodological quality scores for individual studies ranged from 10 to - 12 out of a possible 24. Most high-risk ratings were due to inadequate reporting, which was prevalent across various biases, including those related to data identification, baseline data, treatment effects, and data incorporation. Cost measurement omission bias and inefficient comparator bias were also common.

CONCLUSIONS

While most studies concluded LAAC to be the cost-effective strategy for stroke prevention in AF, shortcomings in methodological quality raise concerns about reliability and validity of results. Future evaluations, free of these shortcomings, can yield stronger policy evidence.

Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review

BACKGROUND

Successful implementation and utilization of Computerized Clinical Decision Support Systems (CDSS) in hospitals is complex and challenging. Implementation science, and in particular the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework, may offer a systematic approach for identifying and addressing these challenges. This review aimed to identify, categorize, and describe barriers and facilitators to CDSS implementation in hospital settings and map them to the NASSS framework. Exploring the applicability of the NASSS framework to CDSS implementation was a secondary aim.

METHODS

Electronic database searches were conducted (21 July 2020; updated 5 April 2022) in Ovid MEDLINE, Embase, Scopus, PyscInfo, and CINAHL. Original research studies reporting on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS in hospital settings, or attitudes of healthcare professionals towards CDSS were included. Articles with a primary focus on CDSS development were excluded. No language or date restrictions were applied. We used qualitative content analysis to identify determinants and organize them into higher-order themes, which were then reflexively mapped to the NASSS framework.

RESULTS

Forty-four publications were included. These comprised a range of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. A total of 227 individual barriers and 130 individual facilitators were identified across the included studies. The most commonly reported influences on implementation were fit of CDSS with workflows (19 studies), the usefulness of the CDSS output in practice (17 studies), CDSS technical dependencies and design (16 studies), trust of users in the CDSS input data and evidence base (15 studies), and the contextual fit of the CDSS with the user's role or clinical setting (14 studies). Most determinants could be appropriately categorized into domains of the NASSS framework with barriers and facilitators in the "Technology," "Organization," and "Adopters" domains most frequently reported. No determinants were assigned to the "Embedding and Adaptation Over Time" domain.

CONCLUSIONS

This review identified the most common determinants which could be targeted for modification to either remove barriers or facilitate the adoption and use of CDSS within hospitals. Greater adoption of implementation theory should be encouraged to support CDSS implementation.