Factors That Impact the Adoption of Clinical Decision Support Systems (CDSS) for Antibiotic Management

Development and application of Breadth-Depth-Context (BDC), a conceptual framework for measuring technology engagement with a qualified clinical data registry

Objectives

Despite the proliferation of dashboards that display performance data derived from Qualified Clinical Data Registries (QCDR), the degree to which clinicians and practices engage with such dashboards has not been well described. We aimed to develop a conceptual framework for assessing user engagement with dashboard technology and to demonstrate its application to a rheumatology QCDR.

Materials and Methods

We developed the BDC (Breadth-Depth-Context) framework, which included concepts of breadth (derived from dashboard sessions), depth (derived from dashboard actions), and context (derived from practice characteristics). We demonstrated its application via user log data from the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) registry to define engagement profiles and characterize practice-level factors associated with different profiles.

Results

We applied the BDC framework to 213 ambulatory practices from the RISE registry in 2020-2021, and classified practices into 4 engagement profiles: not engaged (8%), minimally engaged (39%), moderately engaged (34%), and most engaged (19%). Practices with more patients and with specific electronic health record vendors (eClinicalWorks and eMDs) had a higher likelihood of being in the most engaged group, even after adjusting for other factors.

Discussion

We developed the BDC framework to characterize user engagement with a registry dashboard and demonstrated its use in a specialty QCDR. The application of the BDC framework revealed a wide range of breadth and depth of use and that specific contextual factors were associated with nature of engagement.

Conclusion

Going forward, the BDC framework can be used to study engagement with similar dashboards.

Correlation between antifungal clinical practices and a new clinical decision support system ANTIFON-CLIC® for the treatment of invasive candidiasis: a retrospective multicentre study

BACKGROUND

Invasive candidiasis is still recognized as a major cause of morbidity and mortality. To support clinicians in the optimal use of antifungals for the treatment of invasive candidiasis, a computerized decision support system (CDSS) was developed based on institutional guidelines.

OBJECTIVES

To evaluate the correlation of this newly developed CDSS with clinical practices, we set-up a retrospective multicentre cohort study with the aim of providing the concordance rate between the CDSS recommendation and the medical prescription (NCT05656157).

PATIENTS AND METHODS

Adult patients who received caspofungin or fluconazole for the treatment of an invasive candidiasis were included. The analysis of factors associated with concordance was performed using mixed logistic regression models with department as a random effect.

RESULTS

From March to November 2022, 190 patients were included from three centres and eight departments: 70 patients from centre A, 84 from centre B and 36 from centre C. Overall, 100 patients received caspofungin and 90 received fluconazole, mostly (59%; 112/190) for empirical/pre-emptive treatment. The overall percentage of concordance between the CDSS and medical prescriptions was 91% (173/190) (confidence interval 95%: 82%-96%). No significant difference in concordance was observed considering the centres (P > 0.99), the department of inclusion (P = 0.968), the antifungal treatment (P = 0.656) or the indication of treatment (P = 0.997). In most cases of discordance (n = 13/17, 76%), the CDSS recommended fluconazole whereas caspofungin was prescribed. The clinical usability evaluated by five clinicians was satisfactory.

CONCLUSIONS

Our results demonstrated the high correlation between current antifungal clinical practice and this user-friendly and institutional guidelines-based CDSS.

Perceptions of infection control professionals toward electronic surveillance software supporting inpatient infections: A mixed methods study

BACKGROUND

Electronic surveillance software (ESS) collects multiple patient data from hospital software to assist infection control professionals in the prevention and control of hospital-associated infections. This study aimed to understand the perceptions of end users (i.e., infection control professionals) and the facilitators and barriers related to a commercial ESS named ZINC and to assess its usability.

METHODS

A mixed-method research approach was adopted among infection control professionals 10 months after the implementation of commercial ESS in the university hospital of Nancy, France. A qualitative analysis based on individual semistructured interviews was conducted to collect professionals' perceptions of ESS and to understand barriers and facilitators. Qualitative data were systematically coded and thematically analyzed. A quantitative analysis was performed using the System Usability Scale (SUS).

RESULTS

Thirteen infection control professionals were included. Qualitative analysis revealed technical, organizational and human barriers to the installation and use stages and five significant facilitators: the relevant design of the ESS, the improvement of infection prevention and control practices, the designation of a champion/superuser among professionals, training, and collaboration with the developer team. Quantitative analysis indicated that the evaluated ESS was a "good" system in terms of perceived ease of use, with an overall median SUS score of 85/100.

CONCLUSIONS

This study shows the value of ESS to support inpatient infections as perceived by infection control professionals. It reveals barriers and facilitators to the implementation and adoption of ESS. These barriers and facilitators should be considered to facilitate the installation of the software in other hospitals.

Decision support systems for antibiotic prescription in hospitals: a survey with hospital managers on factors for implementation

BACKGROUND

Inappropriate antimicrobial use, such as antibiotic intake in viral infections, incorrect dosing and incorrect dosing cycles, has been shown to be an important determinant of the emergence of antimicrobial resistance. Artificial intelligence-based decision support systems represent a potential solution for improving antimicrobial prescribing and containing antimicrobial resistance by supporting clinical decision-making thus optimizing antibiotic use and improving patient outcomes.

OBJECTIVE

The aim of this research was to examine implementation factors of artificial intelligence-based decision support systems for antibiotic prescription in hospitals from the perspective of the hospital managers, who have decision-making authority for the organization.

METHODS

An online survey was conducted between December 2022 and May 2023 with managers of German hospitals on factors for decision support system implementation. Survey responses were analyzed from 118 respondents through descriptive statistics.

RESULTS

Survey participants reported openness towards the use of artificial intelligence-based decision support systems for antibiotic prescription in hospitals but little self-perceived knowledge in this field. Artificial intelligence-based decision support systems appear to be a promising opportunity to improve quality of care and increase treatment safety. Along with the Human-Organization-Technology-fit model attitudes were presented. In particular, user-friendliness of the system and compatibility with existing technical structures are considered to be important for implementation. The uptake of decision support systems also depends on the ability of an organization to create a facilitating environment that helps to address the lack of user knowledge as well as trust in and skepticism towards these systems. This includes the training of user groups and support of the management level. Besides, it has been assessed to be important that potential users are open towards change and perceive an added value of the use of artificial intelligence-based decision support systems.

CONCLUSION

The survey has revealed the perspective of hospital managers on different factors that may help to address implementation challenges for artificial intelligence-based decision support systems in antibiotic prescribing. By combining factors of user perceptions about the systems´ perceived benefits with external factors of system design requirements and contextual conditions, the findings highlight the need for a holistic implementation framework of artificial intelligence-based decision support systems.

Exploring the role of professional identity in the implementation of clinical decision support systems-a narrative review

BACKGROUND

Clinical decision support systems (CDSSs) have the potential to improve quality of care, patient safety, and efficiency because of their ability to perform medical tasks in a more data-driven, evidence-based, and semi-autonomous way. However, CDSSs may also affect the professional identity of health professionals. Some professionals might experience these systems as a threat to their professional identity, as CDSSs could partially substitute clinical competencies, autonomy, or control over the care process. Other professionals may experience an empowerment of the role in the medical system. The purpose of this study is to uncover the role of professional identity in CDSS implementation and to identify core human, technological, and organizational factors that may determine the effect of CDSSs on professional identity.

METHODS

We conducted a systematic literature review and included peer-reviewed empirical studies from two electronic databases (PubMed, Web of Science) that reported on key factors to CDSS implementation and were published between 2010 and 2023. Our explorative, inductive thematic analysis assessed the antecedents of professional identity-related mechanisms from the perspective of different health care professionals (i.e., physicians, residents, nurse practitioners, pharmacists).

RESULTS

One hundred thirty-one qualitative, quantitative, or mixed-method studies from over 60 journals were included in this review. The thematic analysis found three dimensions of professional identity-related mechanisms that influence CDSS implementation success: perceived threat or enhancement of professional control and autonomy, perceived threat or enhancement of professional skills and expertise, and perceived loss or gain of control over patient relationships. At the technological level, the most common issues were the system's ability to fit into existing clinical workflows and organizational structures, and its ability to meet user needs. At the organizational level, time pressure and tension, as well as internal communication and involvement of end users were most frequently reported. At the human level, individual attitudes and emotional responses, as well as familiarity with the system, most often influenced the CDSS implementation. Our results show that professional identity-related mechanisms are driven by these factors and influence CDSS implementation success. The perception of the change of professional identity is influenced by the user's professional status and expertise and is improved over the course of implementation.

CONCLUSION

This review highlights the need for health care managers to evaluate perceived professional identity threats to health care professionals across all implementation phases when introducing a CDSS and to consider their varying manifestations among different health care professionals. Moreover, it highlights the importance of innovation and change management approaches, such as involving health professionals in the design and implementation process to mitigate threat perceptions. We provide future areas of research for the evaluation of the professional identity construct within health care.

Enhancing AI-CDSS with U-AnoGAN: Tackling data imbalance

BACKGROUND AND OBJECTIVE

Clinical Decision Support Systems (CDSS) have substantially evolved, aiding healthcare professionals in informed patient care decision-making. The integration of AI, encompassing machine learning and natural language processing, has notably enhanced the capabilities of CDSS. However, a significant challenge remains in addressing data imbalance and the black box nature of AI algorithms, particularly for rare diseases or underrepresented demographic groups. This study aims to propose a model, U-AnoGAN, designed to overcome these hurdles and augment the diagnostic accuracy of AI-integrated CDSS.

METHODS

The U-AnoGAN was trained using masks derived from normal data, focusing on the Covid-19 and pneumonia datasets. Anomaly scores were calculated to assess the model's performance compared to existing AnoGAN-related algorithms. The study also evaluated the model's interpretability through the visualization of abnormal regions.

RESULTS

The results indicated that U-AnoGAN surpassed its counterparts in performance and interpretability. It effectively addressed the data imbalance problem by necessitating only normal data and showcased enhanced diagnostic accuracy. Precision, sensitivity, and specificity values reflected U-AnoGAN's superior capability in accurate disease prediction, diagnosis, treatment recommendations, and adverse event detection.

CONCLUSIONS

U-AnoGAN significantly bolsters the predictive power of AI-integrated CDSS, enabling more precise and timely diagnoses while providing better visualization to potentially overcome the black box problem. This model presents tremendous potential in elevating patient care with advanced AI tools and fostering more accurate and effective decision-making in healthcare environments. As the healthcare sector grapples with escalating data complexity and volume, the importance of models like U-AnoGAN in enhancing CDSS cannot be overstated.

Recent advancement in integrating artificial intelligence and information technology with real-world data for clinical decision-making in China: A scoping review

OBJECTIVE

Striking innovations and advancements have been achieved with the use of artificial intelligence and healthcare information technology being integrated into clinical real-world data. The current scoping review aimed to provide an overview of the current status of artificial intelligence-/information technology-based clinical decision support tools in China.

METHODS

PubMed/MEDLINE, Embase, China National Knowledge Internet, and Wanfang data were searched for both English and Chinese literature. The gray literature search was conducted for commercially available tools. Original studies that focused on clinical decision support tools driven by artificial intelligence or information technology in China and were published between 2010 and February 2022 were included. Information extracted from each article was further synthesized by themes based on three types of clinical decision-making.

RESULTS

A total of 37 peer-reviewed publications and 13 commercially available tools were included in the final analysis. Among them, 32.0% were developed for disease diagnosis, 54.0% for risk prediction and classification, and 14.0% for disease management. Chronic diseases were the most popular therapeutic areas of exploration, with particular emphasis on cardiovascular and cerebrovascular diseases. Single-center electronic medical records were the mainstream data sources leveraged to inform clinical decision-making, with internal validation being predominately used for model evaluation.

CONCLUSIONS

To effectively promote the extensive use of real-world data and drive a paradigm shift in clinical decision-making in China, multidisciplinary collaboration of key stakeholders is urgently needed.

Informing the design of a digital intervention to support sexually transmissible infection care in general practice: a qualitative study exploring the views of clinicians

BACKGROUND

Strengthening sexually transmissible infection (STI) management in general practice is prioritised in Australian STI strategy. Digital interventions incorporating clinical decision support offer a mechanism to assist general practitioners (GPs) in STI care. This study explored clinicians' views towards a proposed digital intervention for supporting STI care in Australian general practice as a first step in the tool's design.

METHODS

Semi-structured one-to-one interviews were conducted during 2021 with sexual health physicians (n =2) and GPs (n =7) practicing in the state of Victoria, Australia. Interviews explored views on a proposed STI digital intervention for general practice. We applied the Theoretical Domains Framework (TDF), a behaviour change framework to our analysis. This involved: (1) directed content analysis of transcripts into TDF domains; and (2) thematic analysis to identify sub-themes within relevant TDF domains. Subthemes were subsequently categorised into enablers and barriers to the use and implementation of a STI computerised clinical decision support system (CDSS).

RESULTS

All interviewees viewed a digital intervention for STI care favourably, expressing confidence in its potential to improve care and support management. Within the relevant TDF domains (e.g. environmental context and resources), subthemes emerged as barriers (e.g. lack of sensitivity to patient context) or enablers (e.g. clear communication and guidance) to the use and implementation of a STI CDSS in primary care. Multiple subthemes (e.g. time constraints) have the potential to be a barrier or an enabler, and is largely dependent on end-user needs being met and clinical context being appropriately addressed.

CONCLUSIONS

A digital intervention incorporating clinical decision support was viewed favourably, indicating a possible role for such a tool in Australian general practice. Co-design with end-users and prototype evaluation with health consumers is recommended to ensure relevance and usefulness.

Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review

BACKGROUND

Successful implementation and utilization of Computerized Clinical Decision Support Systems (CDSS) in hospitals is complex and challenging. Implementation science, and in particular the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework, may offer a systematic approach for identifying and addressing these challenges. This review aimed to identify, categorize, and describe barriers and facilitators to CDSS implementation in hospital settings and map them to the NASSS framework. Exploring the applicability of the NASSS framework to CDSS implementation was a secondary aim.

METHODS

Electronic database searches were conducted (21 July 2020; updated 5 April 2022) in Ovid MEDLINE, Embase, Scopus, PyscInfo, and CINAHL. Original research studies reporting on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS in hospital settings, or attitudes of healthcare professionals towards CDSS were included. Articles with a primary focus on CDSS development were excluded. No language or date restrictions were applied. We used qualitative content analysis to identify determinants and organize them into higher-order themes, which were then reflexively mapped to the NASSS framework.

RESULTS

Forty-four publications were included. These comprised a range of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. A total of 227 individual barriers and 130 individual facilitators were identified across the included studies. The most commonly reported influences on implementation were fit of CDSS with workflows (19 studies), the usefulness of the CDSS output in practice (17 studies), CDSS technical dependencies and design (16 studies), trust of users in the CDSS input data and evidence base (15 studies), and the contextual fit of the CDSS with the user's role or clinical setting (14 studies). Most determinants could be appropriately categorized into domains of the NASSS framework with barriers and facilitators in the "Technology," "Organization," and "Adopters" domains most frequently reported. No determinants were assigned to the "Embedding and Adaptation Over Time" domain.

CONCLUSIONS

This review identified the most common determinants which could be targeted for modification to either remove barriers or facilitate the adoption and use of CDSS within hospitals. Greater adoption of implementation theory should be encouraged to support CDSS implementation.

Analysis of Publication Activity and Research Trends in the Field of AI Medical Applications: Network Approach

Artificial intelligence (AI) has revolutionized numerous industries, including medicine. In recent years, the integration of AI into medical practices has shown great promise in enhancing the accuracy and efficiency of diagnosing diseases, predicting patient outcomes, and personalizing treatment plans. This paper aims at the exploration of the AI-based medicine research using network approach and analysis of existing trends based on PubMed. Our findings are based on the results of PubMed search queries and analysis of the number of papers obtained by the different search queries. Our goal is to explore how are the AI-based methods used in healthcare research, which approaches and techniques are the most popular, and to discuss the potential reasoning behind the obtained results. Using analysis of the co-occurrence network constructed using VOSviewer software, we detected the main clusters of interest in AI-based healthcare research. Then, we proceeded with the thorough analysis of publication activity in various categories of medical AI research, including research on different AI-based methods applied to different types of medical data. We analyzed the results of query processing in the PubMed database over the past 5 years obtained via a specifically designed strategy for generating search queries based on the thorough selection of keywords from different categories of interest. We provide a comprehensive analysis of existing applications of AI-based methods to medical data of different modalities, including the context of various medical fields and specific diseases that carry the greatest danger to the human population.

Healthcare Professionals’ Perceptions, Barriers, and Facilitators towards Adopting Computerised Clinical Decision Support Systems in Antimicrobial Stewardship in Jordanian Hospitals

Understanding healthcare professionals’ perceptions towards a computerised decision support system (CDSS) may provide a platform for the determinants of the successful adoption and implementation of CDSS. This cross-sectional study examined healthcare professionals’ perceptions, barriers, and facilitators to adopting a CDSS for antibiotic prescribing in Jordanian hospitals. This study was conducted among healthcare professionals in Jordan’s two tertiary and teaching hospitals over four weeks (June–July 2021). Data were collected in a paper-based format from senior and junior prescribers and non-prescribers (n = 254) who agreed to complete a questionnaire. The majority (n = 184, 72.4%) were aware that electronic prescribing and electronic health record systems could be used specifically to facilitate antibiotic use and prescribing. The essential facilitator made CDSS available in a portable format (n = 224, 88.2%). While insufficient training to use CDSS was the most significant barrier (n = 175, 68.9%). The female providers showed significantly lower awareness (p = 0.006), and the nurses showed significantly higher awareness (p = 0.041) about using electronic prescribing and electronic health record systems. This study examined healthcare professionals’ perceptions of adopting CDSS in antimicrobial stewardship (AMS) and shed light on the perceived barriers and facilitators to adopting CDSS in AMS, reducing antibiotic resistance, and improving patient safety. Furthermore, results would provide a framework for other hospital settings concerned with implementing CDSS in AMS and inform policy decision-makers to react by implementing the CDSS system in Jordan and globally. Future studies should concentrate on establishing policies and guidelines and a framework to examine the adoption of the CDSS for AMS.

Where Do Real-Time Prescription Benefit Tools Fit in the Landscape of High US Prescription Medication Costs? A Narrative Review

The problem of unaffordable prescription medications in the United States is complex and can result in poor patient adherence to therapy, worse clinical outcomes, and high costs to the healthcare system. While providers are aware of the financial burden of healthcare for patients, there is a lack of actionable price transparency at the point of prescribing. Real-time prescription benefit (RTPB) tools are new electronic clinical decision support tools that retrieve patient- and medication-specific out-of-pocket cost information and display it to clinicians at the point of prescribing. The rise in US healthcare costs has been a major driver for efforts to increase medication price transparency, and mandates from the Centers for Medicare & Medicaid Services for Medicare Part D sponsors to adopt RTPB tools may spur integration of such tools into electronic health records. Although multiple factors affect the implementation of RTPB tools, there is limited evidence on outcomes. Further research will be needed to understand the impact of RTPB tools on end results such as prescribing behavior, out-of-pocket medication costs for patients, and adherence to pharmacologic treatment. We review the terminology and concepts essential in understanding the landscape of RTPB tools, implementation considerations, barriers to adoption, and directions for future research that will be important to patients, prescribers, health systems, and insurers.

Workload and workflow implications associated with the use of electronic clinical decision support tools used by health professionals in general practice: a scoping review

BACKGROUND

Electronic clinical decision support tools (eCDS) are increasingly available to assist General Practitioners (GP) with the diagnosis and management of a range of health conditions. It is unclear whether the use of eCDS tools has an impact on GP workload. This scoping review aimed to identify the available evidence on the use of eCDS tools by health professionals in general practice in relation to their impact on workload and workflow.

METHODS

A scoping review was carried out using the Arksey and O'Malley methodological framework. The search strategy was developed iteratively, with three main aspects: general practice/primary care contexts, risk assessment/decision support tools, and workload-related factors. Three databases were searched in 2019, and updated in 2021, covering articles published since 2009: Medline (Ovid), HMIC (Ovid) and Web of Science (TR). Double screening was completed by two reviewers, and data extracted from included articles were analysed.

RESULTS

The search resulted in 5,594 references, leading to 95 full articles, referring to 87 studies, after screening. Of these, 36 studies were based in the USA, 21 in the UK and 11 in Australia. A further 18 originated from Canada or Europe, with the remaining studies conducted in New Zealand, South Africa and Malaysia. Studies examined the use of eCDS tools and reported some findings related to their impact on workload, including on consultation duration. Most studies were qualitative and exploratory in nature, reporting health professionals' subjective perceptions of consultation duration as opposed to objectively-measured time spent using tools or consultation durations. Other workload-related findings included impacts on cognitive workload, "workflow" and dialogue with patients, and clinicians' experience of "alert fatigue".

CONCLUSIONS

The published literature on the impact of eCDS tools in general practice showed that limited efforts have focused on investigating the impact of such tools on workload and workflow. To gain an understanding of this area, further research, including quantitative measurement of consultation durations, would be useful to inform the future design and implementation of eCDS tools.

Learning meaningful latent space representations for patient risk stratification: Model development and validation for dengue and other acute febrile illness

Background

Increased data availability has prompted the creation of clinical decision support systems. These systems utilise clinical information to enhance health care provision, both to predict the likelihood of specific clinical outcomes or evaluate the risk of further complications. However, their adoption remains low due to concerns regarding the quality of recommendations, and a lack of clarity on how results are best obtained and presented.

Methods

We used autoencoders capable of reducing the dimensionality of complex datasets in order to produce a 2D representation denoted as latent space to support understanding of complex clinical data. In this output, meaningful representations of individual patient profiles are spatially mapped in an unsupervised manner according to their input clinical parameters. This technique was then applied to a large real-world clinical dataset of over 12,000 patients with an illness compatible with dengue infection in Ho Chi Minh City, Vietnam between 1999 and 2021. Dengue is a systemic viral disease which exerts significant health and economic burden worldwide, and up to 5% of hospitalised patients develop life-threatening complications.

Results

The latent space produced by the selected autoencoder aligns with established clinical characteristics exhibited by patients with dengue infection, as well as features of disease progression. Similar clinical phenotypes are represented close to each other in the latent space and clustered according to outcomes broadly described by the World Health Organisation dengue guidelines. Balancing distance metrics and density metrics produced results covering most of the latent space, and improved visualisation whilst preserving utility, with similar patients grouped closer together. In this case, this balance is achieved by using the sigmoid activation function and one hidden layer with three neurons, in addition to the latent dimension layer, which produces the output (Pearson, 0.840; Spearman, 0.830; Procrustes, 0.301; GMM 0.321).

Conclusion

This study demonstrates that when adequately configured, autoencoders can produce two-dimensional representations of a complex dataset that conserve the distance relationship between points. The output visualisation groups patients with clinically relevant features closely together and inherently supports user interpretability. Work is underway to incorporate these findings into an electronic clinical decision support system to guide individual patient management.

Barriers and Facilitators to the Use of Clinical Decision Support Systems in Primary Care: A Mixed-Methods Systematic Review

PURPOSE

To identify and quantify the barriers and facilitators to the use of clinical decision support systems (CDSSs) by primary care professionals (PCPs).

METHODS

A mixed-methods systematic review was conducted using a sequential synthesis design. PubMed/MEDLINE, PsycInfo, Embase, CINAHL, and the Cochrane library were searched in July 2021. Studies that evaluated CDSSs providing recommendations to PCPs and intended for use during a consultation were included. We excluded CDSSs used only by patients, described as concepts or prototypes, used with simulated cases, and decision supports not considered as CDSSs. A framework synthesis was performed according to the HOT-fit framework (Human, Organizational, Technology, Net Benefits), then a quantitative synthesis evaluated the impact of the HOT-fit categories on CDSS use.

RESULTS

A total of 48 studies evaluating 45 CDSSs were included, and 186 main barriers or facilitators were identified. Qualitatively, barriers and facilitators were classified as human (eg, perceived usefulness), organizational (eg, disruption of usual workflow), and technological (eg, CDSS user-friendliness), with explanatory elements. The greatest barrier to using CDSSs was an increased workload. Quantitatively, the human and organizational factors had negative impacts on CDSS use, whereas the technological factor had a neutral impact and the net benefits dimension a positive impact.

CONCLUSIONS

Our findings emphasize the need for CDSS developers to better address human and organizational issues, in addition to technological challenges. We inferred core CDSS features covering these 3 factors, expected to improve their usability in primary care.

Clinician Adoption of an Artificial Intelligence Algorithm to Detect Left Ventricular Systolic Dysfunction in Primary Care

OBJECTIVE

To compare the clinicians' characteristics of "high adopters" and "low adopters" of an artificial intelligence (AI)-enabled electrocardiogram (ECG) algorithm that alerted for possible low left ventricular ejection fraction (EF) and the subsequent effectiveness of detecting patients with low EF.

METHODS

Clinicians in 48 practice sites of a US Midwest health system were cluster-randomized by the care team to usual care or to receive a notification that suggested ordering an echocardiogram in patients flagged as potentially having low EF based on an AI-ECG algorithm. Enrollment was between June 26, 2019, and July 30, 2019; participation concluded on March 31, 2020. This report is focused on those clinicians randomized to receive the notification of the AI-ECG algorithm. At the patient level, data were analyzed for the proportion of patients with positive AI-ECG results. Adoption was defined as the clinician order of an echocardiogram after prompted by the alert.

RESULTS

A total of 165 clinicians and 11,573 patients were included in this analysis. Among patients with positive AI-ECG, high adopters (n=41) were twice as likely to diagnose patients with low EF (33.9%) vs low adopters, n=124, (16.9%); odds ratio, 1.62; 95% CI, 1.21 to 2.17). High adopters were more often advanced practice providers (eg, nurse practitioners and physician assistants) vs physicians, Family Medicine vs Internal Medicine specialty, and tended to have less complex patients.

CONCLUSION

Clinicians who most frequently followed the recommendations of an AI tool were twice as likely to diagnose low EF. Those clinicians with less complex patients were more likely to be high adopters.

TRIAL REGISTRATION

Clinicaltrials.gov Identifier: NCT04000087.

Improving Antibiotic Stewardship for Diarrheal Disease With Probability-Based Electronic Clinical Decision Support: A Randomized Crossover Trial

IMPORTANCE

Inappropriate use of antibiotics for diarrheal illness can result in adverse effects and increase in antimicrobial resistance.

OBJECTIVE

To determine whether the diarrheal etiology prediction (DEP) algorithm, which uses patient-specific and location-specific features to estimate the probability that diarrhea etiology is exclusively viral, impacts antibiotic prescriptions in patients with acute diarrhea.

DESIGN, SETTING, AND PARTICIPANTS

A randomized crossover study was conducted to evaluate the DEP incorporated into a smartphone-based electronic clinical decision-support (eCDS) tool. The DEP calculated the probability of viral etiology of diarrhea, based on dynamic patient-specific and location-specific features. Physicians were randomized in the first 4-week study period to the intervention arm (eCDS with the DEP) or control arm (eCDS without the DEP), followed by a 1-week washout period before a subsequent 4-week crossover period. The study was conducted at 3 sites in Bangladesh from November 17, 2021, to January 21, 2021, and at 4 sites in Mali from January 6, 2021, to March 5, 2021. Eligible physicians were those who treated children with diarrhea. Eligible patients were children between ages 2 and 59 months with acute diarrhea and household access to a cell phone for follow-up.

INTERVENTIONS

Use of the eCDS with the DEP (intervention arm) vs use of the eCDS without the DEP (control arm).

MAIN OUTCOMES AND MEASURES

The primary outcome was the proportion of children prescribed an antibiotic.

RESULTS

A total of 30 physician participants and 941 patient participants (57.1% male; median [IQR] age, 12 [8-18] months) were enrolled. There was no evidence of a difference in the proportion of children prescribed antibiotics by physicians using the DEP (risk difference [RD], -4.2%; 95% CI, -10.7% to 1.0%). In a post hoc analysis that accounted for the predicted probability of a viral-only etiology, there was a statistically significant difference in risk of antibiotic prescription between the DEP and control arms (RD, -0.056; 95% CI, -0.128 to -0.01). No known adverse effects of the DEP were detected at 10-day postdischarge.

CONCLUSIONS AND RELEVANCE

Use of a tool that provides an estimate of etiological likelihood did not result in a significant change in overall antibiotic prescriptions. Post hoc analysis suggests that a higher predicted probability of viral etiology was linked to reductions in antibiotic use.

TRIAL REGISTRATION

Clinicaltrials.gov Identifier: NCT04602676.

Providing Care: Intrinsic Human-Machine Teams and Data

Despite the many successes of artificial intelligence in healthcare applications where human-machine teaming is an intrinsic characteristic of the environment, there is little work that proposes methods for adapting quantitative health data-features with human expertise insights. A method for incorporating qualitative expert perspectives in machine learning training data is proposed. The method implements an entropy-based consensus construct that minimizes the challenges of qualitative-scale data such that they can be combined with quantitative measures in a critical clinical event (CCE) vector. Specifically, the CCE vector minimizes the effects where (a) the sample size is too small, (b) the data may not be normally distributed, or (c) The data are from Likert scales, which are ordinal, so parametric statistics cannot be used. The incorporation of human perspectives in machine learning training data provides encoding of human considerations in the subsequent machine learning model. This encoding provides a basis for increasing explainability, understandability, and ultimately trust in AI-based clinical decision support system (CDSS), thereby improving human-machine teaming concerns. A discussion of applying the CCE vector in a CDSS regime and implications for machine learning are also presented.

Impact of Clinical Decision Support System Implementation at a Community Hospital with an Existing Tele-Antimicrobial Stewardship Program

Background

Lack of on-site antimicrobial stewardship expertise is a barrier to establishing successful programs. Tele-antimicrobial stewardship programs (TASPs) utilizing a clinical decision support system (CDSS) can address these challenges.

Methods

This interrupted time series study reports the impact of CDSS implementation (February 2020) within an existing TASP on antimicrobial usage in a community hospital. Segmented regression analysis was used to assess differences in antimicrobial usage from January 2018 through December 2021. Pre- and post-CDSS frequencies of intravenous vs oral antimicrobials, time to optimal therapy (TTOT), pharmacist efficiency (number of documented interventions per month), and percentage of hospitalized patients receiving antimicrobials were compared with descriptive statistics.

Results

Implementation of a CDSS into an existing TASP was associated with an immediate 11% reduction in antimicrobial usage (level change, P < .0001). Antimicrobial usage was already trending down by 0.25% per month (pre-CDSS slope, P < .0001) and continued to trend down at a similar rate after implementation (post-CDSS slope, P = .0129). Frequency of use of select oral agents increased from 38% to 57%. Median TTOT was 1 day faster (2.9 days pre-CDSS vs 1.9 days post-CDSS). On average, pharmacists documented 2.2-fold more interventions per month (198 vs 90) and patients received 1.03 fewer days of antimicrobials per admission post-CDSS.

Conclusions

Implementation of a CDSS within an established TASP at a community hospital resulted in decreased antimicrobial usage, higher rates of oral usage, faster TTOT, and improved pharmacist efficiency.

Clinical Decision Support System Based on Hybrid Knowledge Modeling: A Case Study of Chronic Kidney Disease-Mineral and Bone Disorder Treatment

Clinical decision support systems (CDSSs) represent the latest technological transformation in healthcare for assisting clinicians in complex decision-making. Several CDSSs are proposed to deal with a range of clinical tasks such as disease diagnosis, prescription management, and medication ordering. Although a small number of CDSSs have focused on treatment selection, areas such as medication selection and dosing selection remained under-researched. In this regard, this study represents one of the first studies in which a CDSS is proposed for clinicians who manage patients with end-stage renal disease undergoing maintenance hemodialysis, almost all of whom have some manifestation of chronic kidney disease-mineral and bone disorder (CKD-MBD). The primary objective of the system is to aid clinicians in dosage prescription by levering medical domain knowledge as well existing practices. The proposed CDSS is evaluated with a real-world hemodialysis patient dataset acquired from Kyung Hee University Hospital, South Korea. Our evaluation demonstrates overall high compliance based on the concordance metric between the proposed CKD-MBD CDSS recommendations and the routine clinical practice. The concordance rate of overall medication dosing selection is 78.27%. Furthermore, the usability aspects of the system are also evaluated through the User Experience Questionnaire method to highlight the appealing aspects of the system for clinicians. The overall user experience dimension scores for pragmatic, hedonic, and attractiveness are 1.53, 1.48, and 1.41, respectively. A service reliability for the Cronbach's alpha coefficient greater than 0.7 is achieved using the proposed system, whereas a dependability coefficient of the value 0.84 reveals a significant effect.

Human Factors and Technological Characteristics Influencing the Interaction with AI-enabled Clinical Decision Support Systems: A Literature Review (Preprint)

Background

The digitization and automation of diagnostics and treatments promise to alter the quality of health care and improve patient outcomes, whereas the undersupply of medical personnel, high workload on medical professionals, and medical case complexity increase. Clinical decision support systems (CDSSs) have been proven to help medical professionals in their everyday work through their ability to process vast amounts of patient information. However, comprehensive adoption is partially disrupted by specific technological and personal characteristics. With the rise of artificial intelligence (AI), CDSSs have become an adaptive technology with human-like capabilities and are able to learn and change their characteristics over time. However, research has not reflected on the characteristics and factors essential for effective collaboration between human actors and AI-enabled CDSSs.

Objective

Our study aims to summarize the factors influencing effective collaboration between medical professionals and AI-enabled CDSSs. These factors are essential for medical professionals, management, and technology designers to reflect on the adoption, implementation, and development of an AI-enabled CDSS.

Methods

We conducted a literature review including 3 different meta-databases, screening over 1000 articles and including 101 articles for full-text assessment. Of the 101 articles, 7 (6.9%) met our inclusion criteria and were analyzed for our synthesis.

Results

We identified the technological characteristics and human factors that appear to have an essential effect on the collaboration of medical professionals and AI-enabled CDSSs in accordance with our research objective, namely, training data quality, performance, explainability, adaptability, medical expertise, technological expertise, personality, cognitive biases, and trust. Comparing our results with those from research on non-AI CDSSs, some characteristics and factors retain their importance, whereas others gain or lose relevance owing to the uniqueness of human-AI interactions. However, only a few (1/7, 14%) studies have mentioned the theoretical foundations and patient outcomes related to AI-enabled CDSSs.

Conclusions

Our study provides a comprehensive overview of the relevant characteristics and factors that influence the interaction and collaboration between medical professionals and AI-enabled CDSSs. Rather limited theoretical foundations currently hinder the possibility of creating adequate concepts and models to explain and predict the interrelations between these characteristics and factors. For an appropriate evaluation of the human-AI collaboration, patient outcomes and the role of patients in the decision-making process should be considered.