Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success

Big data research is everyone's research-Making epilepsy data science accessible to the global community: Report of the ILAE big data commission

Epilepsy care generates multiple sources of high-dimensional data, including clinical, imaging, electroencephalographic, genomic, and neuropsychological information, that are collected routinely to establish the diagnosis and guide management. Thanks to high-performance computing, sophisticated graphics processing units, and advanced analytics, we are now on the cusp of being able to use these data to significantly improve individualized care for people with epilepsy. Despite this, many clinicians, health care providers, and people with epilepsy are apprehensive about implementing Big Data and accompanying technologies such as artificial intelligence (AI). Practical, ethical, privacy, and climate issues represent real and enduring concerns that have yet to be completely resolved. Similarly, Big Data and AI-related biases have the potential to exacerbate local and global disparities. These are highly germane concerns to the field of epilepsy, given its high burden in developing nations and areas of socioeconomic deprivation. This educational paper from the International League Against Epilepsy's (ILAE) Big Data Commission aims to help clinicians caring for people with epilepsy become familiar with how Big Data is collected and processed, how they are applied to studies using AI, and outline the immense potential positive impact Big Data can have on diagnosis and management.

Enhancing nursing care through technology and standardized nursing language: The TEC-MED multilingual platform

PURPOSE

This study describes the design, integration, and semantic interoperability process of a minimum data set using standardized nursing language in the caring module of the TEC-MED care platform.

METHODS

The caring module was developed in three phases (2020-2022): platform concept, functional design and construction, and testing and evaluation. Phases involved collaboration among academics, information technology developers, and social/healthcare professionals. Nursing taxonomies (NANDA-I, NOC, NIC) were integrated to support the nursing process. The platform was piloted in six Mediterranean countries.

FINDINGS

The final platform features an assessment module with eight dimensions for data collection on older adults and their caregivers. A clinical decision support system links assessment data with nursing diagnoses, outcomes, and interventions. The platform is available in six languages (English, Spanish, French, Italian, Greek, and Arabic). Usability testing identified the need for improved Arabic language support.

CONCLUSIONS

The TEC-MED platform is a pioneering tool using standardized nursing language to improve care for older adults in the Mediterranean. The platform's multilingualism promotes accessibility. Limitations include offline use and mobile app functionality. Pilot testing is underway to evaluate effectiveness and facilitate cross-cultural validation of nursing taxonomies.

IMPLICATIONS FOR NURSING PRACTICE

The TEC-MED platform offers standardized nursing care for older adults across the Mediterranean, promoting consistent communication and evidence-based practice. This approach has the potential to improve care quality and accessibility for a vulnerable population.

Enhancing medication management of older adults in Qatar: healthcare professionals' perspectives on challenges, barriers and enabling solutions

Background

Polypharmacy and potentially inappropriate medications are significant challenges in older adults' medication management. The Consolidated Framework for Implementation Research (CFIR) is a comprehensive approach used to explore barriers and enablers to the healthcare system in guiding the effective implementation of evidence-based practices.

Objectives

This study examines the barriers and enablers to promote safe medication management among older adults in Qatar from healthcare professionals' perspectives. This includes identifying critical factors within the healthcare system influencing medication management and suggesting practical solutions to improve it.

Design

The study employs a qualitative design. Focus Groups (FGs) were conducted with healthcare professionals from the geriatric, mental health and medicine departments of Hamad Medical Corporation (HMC), the leading governmental sector in Qatar serving the older adult population.

Methods

Utilising the CFIR, this study analysed feedback from healthcare professionals through FGs at HMC. A combined inductive and deductive thematic analysis was applied to transcripts from five FGs, focusing on identifying barriers and enablers to safe medication management among older adults. Two researchers transcribed the audio-recorded FG discussions verbatim, and two researchers analysed the data using a mixed inductive and deductive thematic analysis approach utilising CFIR constructs.

Results

We engaged 53 healthcare professionals (31 physicians, 10 nurses and 12 clinical pharmacists) in FGs. The analysis identified current barriers and enabler themes under different CFIR constructs, including inner settings, outer settings, individual characteristics and intervention characteristics. We identified 44 themes, with 25 classifieds as barriers and 19 as enablers. The findings revealed that barriers and enablers within the inner settings were primarily related to structural characteristics, resources, policies, communication and culture. On the other hand, barriers and enablers from the outer settings included patients and caregivers, care coordination, policies and laws, and resources.

Conclusion

This study identified several barriers and enablers to promote medication management for older adults using the CFIR constructs from the perspective of healthcare professionals. The multifaceted findings emphasise involving stakeholders like clinical leaders, policymakers and decision-makers to address medication safety factors. A robust action plan, continuously monitored under Qatar's national strategy, is vital. Further research is needed to implement recommended interventions.

Comparative Study to Evaluate the Accuracy of Differential Diagnosis Lists Generated by Gemini Advanced, Gemini, and Bard for a Case Report Series Analysis: Cross-Sectional Study

BACKGROUND

Generative artificial intelligence (GAI) systems by Google have recently been updated from Bard to Gemini and Gemini Advanced as of December 2023. Gemini is a basic, free-to-use model after a user's login, while Gemini Advanced operates on a more advanced model requiring a fee-based subscription. These systems have the potential to enhance medical diagnostics. However, the impact of these updates on comprehensive diagnostic accuracy remains unknown.

OBJECTIVE

This study aimed to compare the accuracy of the differential diagnosis lists generated by Gemini Advanced, Gemini, and Bard across comprehensive medical fields using case report series.

METHODS

We identified a case report series with relevant final diagnoses published in the American Journal Case Reports from January 2022 to March 2023. After excluding nondiagnostic cases and patients aged 10 years and younger, we included the remaining case reports. After refining the case parts as case descriptions, we input the same case descriptions into Gemini Advanced, Gemini, and Bard to generate the top 10 differential diagnosis lists. In total, 2 expert physicians independently evaluated whether the final diagnosis was included in the lists and its ranking. Any discrepancies were resolved by another expert physician. Bonferroni correction was applied to adjust the P values for the number of comparisons among 3 GAI systems, setting the corrected significance level at P value <.02.

RESULTS

In total, 392 case reports were included. The inclusion rates of the final diagnosis within the top 10 differential diagnosis lists were 73% (286/392) for Gemini Advanced, 76.5% (300/392) for Gemini, and 68.6% (269/392) for Bard. The top diagnoses matched the final diagnoses in 31.6% (124/392) for Gemini Advanced, 42.6% (167/392) for Gemini, and 31.4% (123/392) for Bard. Gemini demonstrated higher diagnostic accuracy than Bard both within the top 10 differential diagnosis lists (P=.02) and as the top diagnosis (P=.001). In addition, Gemini Advanced achieved significantly lower accuracy than Gemini in identifying the most probable diagnosis (P=.002).

CONCLUSIONS

The results of this study suggest that Gemini outperformed Bard in diagnostic accuracy following the model update. However, Gemini Advanced requires further refinement to optimize its performance for future artificial intelligence-enhanced diagnostics. These findings should be interpreted cautiously and considered primarily for research purposes, as these GAI systems have not been adjusted for medical diagnostics nor approved for clinical use.

Discrepancy-based diffusion models for lesion detection in brain MRI

Diffusion probabilistic models (DPMs) have exhibited significant effectiveness in computer vision tasks, particularly in image generation. However, their notable performance heavily relies on labelled datasets, which limits their application in medical images due to the associated high-cost annotations. Current DPM-related methods for lesion detection in medical imaging, which can be categorized into two distinct approaches, primarily rely on image-level annotations. The first approach, based on anomaly detection, involves learning reference healthy brain representations and identifying anomalies based on the difference in inference results. In contrast, the second approach, resembling a segmentation task, employs only the original brain multi-modalities as prior information for generating pixel-level annotations. In this paper, our proposed model - discrepancy distribution medical diffusion (DDMD) - for lesion detection in brain MRI introduces a novel framework by incorporating distinctive discrepancy features, deviating from the conventional direct reliance on image-level annotations or the original brain modalities. In our method, the inconsistency in image-level annotations is translated into distribution discrepancies among heterogeneous samples while preserving information within homogeneous samples. This property retains pixel-wise uncertainty and facilitates an implicit ensemble of segmentation, ultimately enhancing the overall detection performance. Thorough experiments conducted on the BRATS2020 benchmark dataset containing multimodal MRI scans for brain tumour detection demonstrate the great performance of our approach in comparison to state-of-the-art methods.

Stepped-Wedge Trial of Decision Support for Acute Kidney Injury on Surgical Units

Introduction

Acute kidney injury (AKI) is common in the perioperative setting and associated with poor outcomes. Whether clinical decision support improves early management and outcomes of AKI on surgical units is uncertain.

Methods

In this cluster-randomized, stepped-wedge trial, 8 surgical units in Alberta, Canada were randomized to various start dates to receive an education and clinical decision support intervention for recognition and early management of AKI. Eligible patients were aged ≥18 years, receiving care on a surgical unit, not already receiving dialysis, and with AKI.

Results

There were 2135 admissions of 2038 patients who met the inclusion criteria; mean (SD) age was 64.3 (16.2) years, and 885 (41.4%) were females. The proportion of patients who experienced the composite primary outcome of progression of AKI to a higher stage, receipt of dialysis, or death was 16.0% (178 events/1113 admissions) in the intervention group; and 17.5% (179 events/1022 admissions) in the control group (time-adjusted odds ratio, 0.76; 95% confidence interval [CI], 0.53-1.08; P = 0.12). There were no significant differences between groups in process of care outcomes within 48 hours of AKI onset, including administration of i.v. fluids, or withdrawal of medications affecting kidney function. Both groups experienced similar lengths of stay in hospital after AKI and change in estimated glomerular filtration rate (eGFR) at 3 months.

Conclusion

An education and clinical decision support intervention did not significantly improve processes of care or reduce progression of AKI, length of hospital stays, or recovery of kidney function in patients with AKI on surgical units.

Challenges and Facilitation Approaches for the Participatory Design of AI-Based Clinical Decision Support Systems: Protocol for a Scoping Review

BACKGROUND

In the last few years, there has been an increasing interest in the development of artificial intelligence (AI)-based clinical decision support systems (CDSS). However, there are barriers to the successful implementation of such systems in practice, including the lack of acceptance of these systems. Participatory approaches aim to involve future users in designing applications such as CDSS to be more acceptable, feasible, and fundamentally more relevant for practice. The development of technologies based on AI, however, challenges the process of user involvement and related methods.

OBJECTIVE

The aim of this review is to summarize and present the main approaches, methods, practices, and specific challenges for participatory research and development of AI-based decision support systems involving clinicians.

METHODS

This scoping review will follow the Joanna Briggs Institute approach to scoping reviews. The search for eligible studies was conducted in the databases MEDLINE via PubMed; ACM Digital Library; Cumulative Index to Nursing and Allied Health; and PsycInfo. The following search filters, adapted to each database, were used: Period January 01, 2012, to October 31, 2023, English and German studies only, abstract available. The scoping review will include studies that involve the development, piloting, implementation, and evaluation of AI-based CDSS (hybrid and data-driven AI approaches). Clinical staff must be involved in a participatory manner. Data retrieval will be accompanied by a manual gray literature search. Potential publications will then be exported into reference management software, and duplicates will be removed. Afterward, the obtained set of papers will be transferred into a systematic review management tool. All publications will be screened, extracted, and analyzed: title and abstract screening will be carried out by 2 independent reviewers. Disagreements will be resolved by involving a third reviewer. Data will be extracted using a data extraction tool prepared for the study.

RESULTS

This scoping review protocol was registered on March 11, 2023, at the Open Science Framework. The full-text screening had already started at that time. Of the 3,118 studies screened by title and abstract, 31 were included in the full-text screening. Data collection and analysis as well as manuscript preparation are planned for the second and third quarter of 2024. The manuscript should be submitted towards the end of 2024.

CONCLUSIONS

This review will describe the current state of knowledge on participatory development of AI-based decision support systems. The aim is to identify knowledge gaps and provide research impetus. It also aims to provide relevant information for policy makers and practitioners.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/58185.

Designing and validating a clinical decision support algorithm for diabetic nephroprotection in older patients

BACKGROUND

Older patients with diabetic kidney disease (DKD) often do not receive optimal pharmacological treatment. Current clinical practice guidelines (CPGs) do not incorporate the concept of personalised care. Clinical decision support (CDS) algorithms that consider both evidence and personalised care to improve patient outcomes can improve the care of older adults. The aim of this research is to design and validate a CDS algorithm for prescribing renin-angiotensin-aldosterone system inhibitors (RAASi) for older patients with diabetes.

METHODS

The design of the CDS tool included the following phases: (1) gathering evidence from systematic reviews and meta-analyses of randomised clinical trials to determine the number needed to treat (NNT) and time-to-benefit (TTB) values applicable to our target population for use in the algorithm. (2) Building a list of potential cases that addressed different prescribing scenarios (starting, adding or switching to RAASi). (3) Reviewing relevant guidelines and extracting all recommendations related to prescribing RAASi for DKD. (4) Matching NNT and TTB with specific clinical cases. (5) Validating the CDS algorithm using Delphi technique.

RESULTS

We created a CDS algorithm that covered 15 possible scenarios and we generated 36 personalised and nine general recommendations based on the calculated and matched NNT and TTB values and considering the patient's life expectancy and functional capacity. The algorithm was validated by experts in three rounds of Delphi study.

CONCLUSION

We designed an evidence-informed CDS algorithm that integrates considerations often overlooked in CPGs. The next steps include testing the CDS algorithm in a clinical trial.

Artificial Intelligence, the Digital Surgeon: Unravelling Its Emerging Footprint in Healthcare - The Narrative Review

Background

Artificial Intelligence (AI) holds transformative potential for the healthcare industry, offering innovative solutions for diagnosis, treatment planning, and improving patient outcomes. As AI continues to be integrated into healthcare systems, it promises advancements across various domains. This review explores the diverse applications of AI in healthcare, along with the challenges and limitations that need to be addressed. The aim is to provide a comprehensive overview of AI's impact on healthcare and to identify areas for further development and focus.

Main Applications

The review discusses the broad range of AI applications in healthcare. In medical imaging and diagnostics, AI enhances the accuracy and efficiency of diagnostic processes, aiding in early disease detection. AI-powered clinical decision support systems assist healthcare professionals in patient management and decision-making. Predictive analytics using AI enables the prediction of patient outcomes and identification of potential health risks. AI-driven robotic systems have revolutionized surgical procedures, improving precision and outcomes. Virtual assistants and chatbots enhance patient interaction and support, providing timely information and assistance. In the pharmaceutical industry, AI accelerates drug discovery and development by identifying potential drug candidates and predicting their efficacy. Additionally, AI improves administrative efficiency and operational workflows in healthcare, streamlining processes and reducing costs. AI-powered remote monitoring and telehealth solutions expand access to healthcare, particularly in underserved areas.

Challenges and Limitations

Despite the significant promise of AI in healthcare, several challenges persist. Ensuring the reliability and consistency of AI-driven outcomes is crucial. Privacy and security concerns must be navigated carefully, particularly in handling sensitive patient data. Ethical considerations, including bias and fairness in AI algorithms, need to be addressed to prevent unintended consequences. Overcoming these challenges is critical for the ethical and successful integration of AI in healthcare.

Conclusion

The integration of AI into healthcare is advancing rapidly, offering substantial benefits in improving patient care and operational efficiency. However, addressing the associated challenges is essential to fully realize the transformative potential of AI in healthcare. Future efforts should focus on enhancing the reliability, transparency, and ethical standards of AI technologies to ensure they contribute positively to global health outcomes.

Mental Health Care Support in Rural India: A Cluster Randomized Clinical Trial

Importance

More than 150 million people in India need mental health care but few have access to affordable care, especially in rural areas.

Objective

To determine whether a multifaceted intervention involving a digital health care model along with a community-based antistigma campaign leads to reduced depression risk and lower mental health-related stigma among adults residing in rural India.

Design, Setting, and Participants

This parallel, cluster randomized, usual care-controlled trial was conducted from September 2020 to December 2021 with blinded follow-up assessments at 3, 6, and 12 months at 44 rural primary health centers across 3 districts in Haryana and Andhra Pradesh states in India. Adults aged 18 years and older at high risk of depression or self-harm defined by either a Patient Health Questionnaire-9 item (PHQ-9) score of 10 or greater, a Generalized Anxiety Disorder-7 item (GAD-7) score of 10 or greater, or a score of 2 or greater on the self-harm/suicide risk question on the PHQ-9. A second cohort of adults not at high risk were selected randomly from the remaining screened population. Data were cleaned and analyzed from April 2022 to February 2023.

Interventions

The 12-month intervention included a community-based antistigma campaign involving all participants and a digital mental health intervention involving only participants at high risk. Primary health care workers were trained to identify and manage participants at high risk using the Mental Health Gap Action Programme guidelines from the World Health Organization.

Main Outcomes and Measures

The 2 coprimary outcomes assessed at 12 months were mean PHQ-9 scores in the high-risk cohort and mean behavior scores in the combined high-risk and non-high-risk cohorts using the Mental Health Knowledge, Attitude, and Behavior scale.

Results

Altogether, 9928 participants were recruited (3365 at high risk and 6563 not at high risk; 5638 [57%] female and 4290 [43%] male; mean [SD] age, 43 [16] years) with 9057 (91.2%) followed up at 12 months. Mean PHQ-9 scores at 12 months for the high-risk cohort were lower in the intervention vs control groups (2.77 vs 4.48; mean difference, -1.71; 95% CI, -2.53 to -0.89; P < .001). The remission rate in the high-risk cohort (PHQ-9 and GAD-7 scores <5 and no risk of self-harm) was higher in the intervention vs control group (74.7% vs 50.6%; odds ratio [OR], 2.88; 95% CI, 1.53 to 5.42; P = .001). Across both cohorts, there was no difference in 12-month behavior scores in the intervention vs control group (17.39 vs 17.74; mean difference, -0.35; 95% CI, -1.11 to 0.41; P = .36).

Conclusions and Relevance

A multifaceted intervention was effective in reducing depression risk but did not improve intended help-seeking behaviors for mental illness.

Trial Registration

Clinical Trial Registry India: CTRI/2018/08/015355.

Understanding Users' Acceptance of Artificial Intelligence Applications: A Literature Review

In recent years, with the continuous expansion of artificial intelligence (AI) application forms and fields, users' acceptance of AI applications has attracted increasing attention from scholars and business practitioners. Although extant studies have extensively explored user acceptance of different AI applications, there is still a lack of understanding of the roles played by different AI applications in human-AI interaction, which may limit the understanding of inconsistent findings about user acceptance of AI. This study addresses this issue by conducting a systematic literature review on AI acceptance research in leading journals of Information Systems and Marketing disciplines from 2020 to 2023. Based on a review of 80 papers, this study made contributions by (i) providing an overview of methodologies and theoretical frameworks utilized in AI acceptance research; (ii) summarizing the key factors, potential mechanisms, and theorization of users' acceptance response to AI service providers and AI task substitutes, respectively; and (iii) proposing opinions on the limitations of extant research and providing guidance for future research.

Clinical Decision Support Tool to Promote Adoption of New Neonatal Hyperbilirubinemia Guidelines

OBJECTIVE

 This study aimed to increase the adoption of revised newborn hyperbilirubinemia guidelines by building a clinical decision support (CDS) tool into templated notes.

METHODS

 We created a rule-based CDS tool that correctly populates the phototherapy threshold from more than 2,700 possible values directly into the note and guides clinicians to an appropriate follow-up plan consistent with the new recommendations. We manually reviewed notes before and after CDS tool implementation to evaluate new guidelines adherence, and surveys were used to assess clinicians' perceptions.

RESULTS

 Postintervention documentation showed a decrease in old risk stratification methods (48 to 0.4%, p < 0.01) and an increase in new phototherapy threshold usage (39 to 95%, p < 0.01) and inclusion of follow-up guidance (28 to 79%, p < 0.01). Survey responses on workflow efficiency and satisfaction did not significantly change after CDS tool implementation.

CONCLUSION

 Our study details an innovative CDS tool that contributed to increased adoption of newly revised guidelines after the addition of this tool to templated notes.

Using the Electronic Health Record to Implement Expedited Partner Therapy in the Pediatric Emergency Department

OBJECTIVES

Expedited partner therapy (EPT) is a partner treatment strategy for sexually transmitted infections (STIs) including gonorrhea and chlamydia as well as trichomoniasis in some states. The process allows healthcare providers to write prescriptions for STI treatment among partners of infected patients without a previous medical evaluation. The Centers for Disease Control (CDC) has recommended EPT as a useful option to facilitate partner treatment, particularly male partners of women with chlamydia or gonorrhea infections. Our institution implemented EPT in 2016 after Ohio legislation was passed to authorize its use. We aim to describe the implementation process and descriptive outcomes of EPT adoption in a pediatric emergency department.

METHODS

This study describes use of the electronic health record for implementation of EPT in our institution. We conducted a retrospective review of EPT utilization from implementation. Electronic records from the implementation date of January 1, 2017, through December 31, 2021, were reviewed. We describe basic demographics and overall uptake of the intervention. Fisher exact tests were used for categorical variables and two-sample t-tests for continuous variables.

RESULTS

There was a total of 3275 positive test results and 739 EPT prescriptions written. Adolescent patients who received prescriptions for EPT were more likely to be female (78.7% of all EPT prescriptions, P = 0.007) and older than other patients (average age 17.7 vs 17.4 years, P = 0.004). There was no significant difference in race, insurance, or ethnicity among adolescent patients receiving and not receiving EPT. The percentage of positive STI tests associated with an EPT prescription ranged between 11.4% and 18.2%. Metronidazole was the most prescribed EPT medication.

CONCLUSIONS

The use of the electronic health record provides a platform for implementation of EPT. Our study highlights a potential strategy for increasing treatments of STIs through EPT prescribing in the emergency department setting.

Evaluating the Accuracy and Impact of the ESR-iGuide Decision Support Tool in Optimizing CT Imaging Referral Appropriateness

Radiology referral quality impacts patient care, yet factors influencing quality are poorly understood. This study assessed the quality of computed tomography (CT) referrals, identified associated characteristics, and evaluated the ESR-iGuide clinical decision support tool's ability to optimize referrals. A retrospective review analyzed 300 consecutive CT referrals from an acute care hospital. Referral quality was evaluated on a 5-point scale by three expert reviewers (inter-rater reliability κ = 0.763-0.97). The ESR-iGuide tool provided appropriateness scores and estimated radiation exposure levels for the actual referred exams and recommended exams. Scores were compared between actual and recommended exams. Associations between ESR-iGuide scores and referral characteristics, including the specialty of the ordering physician (surgical vs. non-surgical), were explored. Of the referrals, 67.1% were rated as appropriate. The most common exams were head and abdomen/pelvis CTs. The ESR-iGuide deemed 70% of the actual referrals "usually appropriate" and found that the recommended exams had lower estimated radiation exposure compared to the actual exams. Logistic regression analysis showed that non-surgical physicians were more likely to order inappropriate exams compared to surgical physicians. Over one-third of the referrals showed suboptimal quality in the unstructured system. The ESR-iGuide clinical decision support tool identified opportunities to optimize appropriateness and reduce radiation exposure. Implementation of such a tool warrants consideration to improve communication and maximize patient care quality.

Clinical decision support systems in addiction and concurrent disorders: A systematic review and meta-analysis

INTRODUCTION

This review aims to synthesise the literature on the efficacy, evolution, and challenges of implementing Clincian Decision Support Systems (CDSS) in the realm of mental health, addiction, and concurrent disorders.

METHODS

Following PRISMA guidelines, a systematic review and meta-analysis were performed. Searches conducted in databases such as MEDLINE, Embase, CINAHL, PsycINFO, and Web of Science through 25 May 2023, yielded 27,344 records. After necessary exclusions, 69 records were allocated for detailed synthesis. In the examination of patient outcomes with a focus on metrics such as therapeutic efficacy, patient satisfaction, and treatment acceptance, meta-analytic techniques were employed to synthesise data from randomised controlled trials.

RESULTS

A total of 69 studies were included, revealing a shift from knowledge-based models pre-2017 to a rise in data-driven models post-2017. The majority of models were found to be in Stage 2 or 4 of maturity. The meta-analysis showed an effect size of -0.11 for addiction-related outcomes and a stronger effect size of -0.50 for patient satisfaction and acceptance of CDSS.

DISCUSSION

The results indicate a shift from knowledge-based to data-driven CDSS approaches, aligned with advances in machine learning and big data. Although the immediate impact on addiction outcomes is modest, higher patient satisfaction suggests promise for wider CDSS use. Identified challenges include alert fatigue and opaque AI models.

CONCLUSION

CDSS shows promise in mental health and addiction treatment but requires a nuanced approach for effective and ethical implementation. The results emphasise the need for continued research to ensure optimised and equitable use in healthcare settings.

How intervention studies measure the effectiveness of medication safety-related clinical decision support systems in primary and long-term care: a systematic review

BACKGROUND

Medication errors and associated adverse drug events (ADE) are a major cause of morbidity and mortality worldwide. In recent years, the prevention of medication errors has become a high priority in healthcare systems. In order to improve medication safety, computerized Clinical Decision Support Systems (CDSS) are increasingly being integrated into the medication process. Accordingly, a growing number of studies have investigated the medication safety-related effectiveness of CDSS. However, the outcome measures used are heterogeneous, leading to unclear evidence. The primary aim of this study is to summarize and categorize the outcomes used in interventional studies evaluating the effects of CDSS on medication safety in primary and long-term care.

METHODS

We systematically searched PubMed, Embase, CINAHL, and Cochrane Library for interventional studies evaluating the effects of CDSS targeting medication safety and patient-related outcomes. We extracted methodological characteristics, outcomes and empirical findings from the included studies. Outcomes were assigned to three main categories: process-related, harm-related, and cost-related. Risk of bias was assessed using the Evidence Project risk of bias tool.

RESULTS

Thirty-two studies met the inclusion criteria. Almost all studies (n = 31) used process-related outcomes, followed by harm-related outcomes (n = 11). Only three studies used cost-related outcomes. Most studies used outcomes from only one category and no study used outcomes from all three categories. The definition and operationalization of outcomes varied widely between the included studies, even within outcome categories. Overall, evidence on CDSS effectiveness was mixed. A significant intervention effect was demonstrated by nine of fifteen studies with process-related primary outcomes (60%) but only one out of five studies with harm-related primary outcomes (20%). The included studies faced a number of methodological problems that limit the comparability and generalizability of their results.

CONCLUSIONS

Evidence on the effectiveness of CDSS is currently inconclusive due in part to inconsistent outcome definitions and methodological problems in the literature. Additional high-quality studies are therefore needed to provide a comprehensive account of CDSS effectiveness. These studies should follow established methodological guidelines and recommendations and use a comprehensive set of harm-, process- and cost-related outcomes with agreed-upon and consistent definitions.

PROSPERO REGISTRATION

CRD42023464746.

Multimodal Data Integration Enhance Longitudinal Prediction of New-Onset Systemic Arterial Hypertension Patients with Suspected Obstructive Sleep Apnea

Background

It is crucial to accurately predict the disease progression of systemic arterial hypertension in order to determine the most effective therapeutic strategy. To achieve this, we have employed a multimodal data-integration approach to predict the longitudinal progression of new-onset systemic arterial hypertension patients with suspected obstructive sleep apnea (OSA) at the individual level.

Methods

We developed and validated a predictive nomogram model that utilizes multimodal data, consisting of clinical features, laboratory tests, and sleep monitoring data. We assessed the probabilities of major adverse cardiac and cerebrovascular events (MACCEs) as scores for participants in longitudinal cohorts who have systemic arterial hypertension and suspected OSA. In this cohort study, MACCEs were considered as a composite of cardiac mortality, acute coronary syndrome and nonfatal stroke. The least absolute shrinkage and selection operator (LASSO) regression and multiple Cox regression analyses were performed to identify independent risk factors for MACCEs among these patients.

Results

448 patients were randomly assigned to the training cohort while 189 were assigned to the verification cohort. Four clinical variables were enrolled in the constructed nomogram: age, diabetes mellitus, triglyceride, and apnea-hypopnea index (AHI). This model accurately predicted 2-year and 3-year MACCEs, achieving an impressive area under the receiver operating characteristic (ROC) curve of 0.885 and 0.784 in the training cohort, respectively. In the verification cohort, the performance of the nomogram model had good discriminatory power, with an area under the ROC curve of 0.847 and 0.729 for 2-year and 3-year MACCEs, respectively. The correlation between predicted and actual observed MACCEs was high, provided by a calibration plot, for training and verification cohorts.

Conclusions

Our study yielded risk stratification for systemic arterial hypertension patients with suspected OSA, which can be quantified through the integration of multimodal data, thus highlighting OSA as a spectrum of disease. This prediction nomogram could be instrumental in defining the disease state and long-term clinical outcomes.

Impact of Clinical Decision Support with Mandatory versus Voluntary Venous Thromboembolism Risk Assessment in Hospitalized Patients

Background  Venous thromboembolism (VTE) causes significant preventable morbidity and mortality in hospitalized patients. Assessing VTE risk is essential to initiating appropriate prophylaxis and reducing VTE outcomes. Studies show that computerized clinical decision support (CDS) can improve VTE risk assessment (RA), prophylaxis, and outcomes but few examined the effectiveness of specific design features. From 2008 to 2016, University of Michigan Health implemented CDS for VTE prevention in four stages, which alternated between voluntary and mandatory RA using the 2005 Caprini model and generated inpatient orders for risk-appropriate prophylaxis based on CHEST guidelines. This cross-sectional study evaluated the impact of mandatory versus voluntary RA on VTE prophylaxis and outcomes for adult medical and surgical patients admitted to the health system. Methods  Interrupted time series analysis was conducted to evaluate the trend in smart order set-recommended VTE prophylaxis by CDS stage. Logistic regression with CDS stage as the primary independent variable was used in pairwise comparisons of VTE during hospitalization and within 90 days post-discharge for mandatory versus voluntary RA. Adjusted odd ratios (ORs) were calculated for total, in-hospital, and post-discharge VTE. Results  In this study of 223,405 inpatients over 8 years, smart order set-recommended prophylaxis increased from 65 to 79%; it increased significantly when voluntary RA in Stage 1 became mandatory in Stage 2 (10.59%, p  < 0.001) and decreased significantly when it returned to voluntary in Stage 3 (-11.24%, p  < 0.001). The rate increased slightly when mandatory RA was reestablished in Stage 4 (0.23%, p  = 0.935). Adjusted ORs for VTE were lower for mandatory RA versus adjacent stages with voluntary RA. The adjusted OR for Stage 2 versus Stage 1 was 14% lower ( p  < 0.05) and versus Stage 3 was 11% lower ( p  < 0.05). The adjusted OR for Stage 4 versus Stage 3 was 4% lower ( p  = 0.60). These results were driven by changes in in-hospital VTE. By contrast, the incidence of post-discharge VTE increased in each successive stage. Conclusion  Mandatory RA was more effective in improving smart order set-recommended prophylaxis and VTE outcomes, particularly in-hospital VTE. Post-discharge VTE increased despite high adherence to risk-appropriate prophylaxis, indicating that guidelines for extended, post-discharge prophylaxis are needed to further reduce VTE for hospitalized patients.

Deploying an asthma dashboard to support quality improvement across a nationally representative sentinel network of 7.6 million people in England

Every year, there are ~100,000 hospital admissions for asthma in the UK, many of which are potentially preventable. Evidence suggests that carefully conceptualised and implemented audit and feedback (A&F) cycles have the potential to improve clinical outcomes for those with chronic conditions. We wanted to investigate the technical feasibility of developing a near-real time asthma dashboard to support A&F interventions for asthma management in primary care. We extracted cross-sectional data on asthma from 756 participating GP practices in the Oxford-Royal College of General Practitioners Research and Surveillance Centre (RCGP RSC) database in England comprising 7.6 million registered people. Summary indicators for a GP practice were compared to all participating RCGP RSC practices using practice-level data, for the week 6-12th-Mar-2023. A weekly, automated asthma dashboard with features that can support electronic-A&F cycles that compared key asthma indicators for a GP practice to RCGP RSC could be created ( https://tinyurl.com/3ydtrt85 ): 12-weeks-incidence 0.4% vs 0.4%, annual prevalence 6.1% vs 6.7%, inhaled relievers to preventer 1.2 vs 1.1, self-management plan given 83.4% vs 60.8%, annual reviews 36.8% vs 57.3%, prednisolone prescriptions 2.0% vs 3.2%, influenza vaccination 56.6% vs 55.5%, pneumococcal vaccination ever (aged ≥65 years) 90.2% vs 84.1% and current smokers 14.9% vs 14.8%. Across the RCGP RSC, the rate of hospitalisations was 0.024%; comparative data had to be suppressed for the study practice because of small numbers. We have successfully created an automated near real-time asthma dashboard that can be used to support A&F initiatives to improve asthma care and outcomes in primary care.

Strengthening the use of artificial intelligence within healthcare delivery organizations: balancing regulatory compliance and patient safety

OBJECTIVES

Surface the urgent dilemma that healthcare delivery organizations (HDOs) face navigating the US Food and Drug Administration (FDA) final guidance on the use of clinical decision support (CDS) software.

MATERIALS AND METHODS

We use sepsis as a case study to highlight the patient safety and regulatory compliance tradeoffs that 6129 hospitals in the United States must navigate.

RESULTS

Sepsis CDS remains in broad, routine use. There is no commercially available sepsis CDS system that is FDA cleared as a medical device. There is no public disclosure of an HDO turning off sepsis CDS due to regulatory compliance concerns. And there is no public disclosure of FDA enforcement action against an HDO for using sepsis CDS that is not cleared as a medical device.

DISCUSSION AND CONCLUSION

We present multiple policy interventions that would relieve the current tension to enable HDOs to utilize artificial intelligence to improve patient care while also addressing FDA concerns about product safety, efficacy, and equity.

The role of information systems in emergency department decision-making-a literature review

OBJECTIVES

Healthcare providers employ heuristic and analytical decision-making to navigate the high-stakes environment of the emergency department (ED). Despite the increasing integration of information systems (ISs), research on their efficacy is conflicting. Drawing on related fields, we investigate how timing and mode of delivery influence IS effectiveness. Our objective is to reconcile previous contradictory findings, shedding light on optimal IS design in the ED.

MATERIALS AND METHODS

We conducted a systematic review following PRISMA across PubMed, Scopus, and Web of Science. We coded the ISs' timing as heuristic or analytical, their mode of delivery as active for automatic alerts and passive when requiring user-initiated information retrieval, and their effect on process, economic, and clinical outcomes.

RESULTS

Our analysis included 83 studies. During early heuristic decision-making, most active interventions were ineffective, while passive interventions generally improved outcomes. In the analytical phase, the effects were reversed. Passive interventions that facilitate information extraction consistently improved outcomes.

DISCUSSION

Our findings suggest that the effectiveness of active interventions negatively correlates with the amount of information received during delivery. During early heuristic decision-making, when information overload is high, physicians are unresponsive to alerts and proactively consult passive resources. In the later analytical phases, physicians show increased receptivity to alerts due to decreased diagnostic uncertainty and information quantity. Interventions that limit information lead to positive outcomes, supporting our interpretation.

CONCLUSION

We synthesize our findings into an integrated model that reveals the underlying reasons for conflicting findings from previous reviews and can guide practitioners in designing ISs in the ED.

Expanding access to veterinary clinical decision support in resource-limited settings: a scoping review of clinical decision support tools in medicine and antimicrobial stewardship

Introduction

Digital clinical decision support (CDS) tools are of growing importance in supporting healthcare professionals in understanding complex clinical problems and arriving at decisions that improve patient outcomes. CDS tools are also increasingly used to improve antimicrobial stewardship (AMS) practices in healthcare settings. However, far fewer CDS tools are available in lowerand middle-income countries (LMICs) and in animal health settings, where their use in improving diagnostic and treatment decision-making is likely to have the greatest impact. The aim of this study was to evaluate digital CDS tools designed as a direct aid to support diagnosis and/or treatment decisionmaking, by reviewing their scope, functions, methodologies, and quality. Recommendations for the development of veterinary CDS tools in LMICs are then provided.

Methods

The review considered studies and reports published between January 2017 and October 2023 in the English language in peer-reviewed and gray literature.

Results

A total of 41 studies and reports detailing CDS tools were included in the final review, with 35 CDS tools designed for human healthcare settings and six tools for animal healthcare settings. Of the tools reviewed, the majority were deployed in high-income countries (80.5%). Support for AMS programs was a feature in 12 (29.3%) of the tools, with 10 tools in human healthcare settings. The capabilities of the CDS tools varied when reviewed against the GUIDES checklist.

Discussion

We recommend a methodological approach for the development of veterinary CDS tools in LMICs predicated on securing sufficient and sustainable funding. Employing a multidisciplinary development team is an important first step. Developing standalone CDS tools using Bayesian algorithms based on local expert knowledge will provide users with rapid and reliable access to quality guidance on diagnoses and treatments. Such tools are likely to contribute to improved disease management on farms and reduce inappropriate antimicrobial use, thus supporting AMS practices in areas of high need.

Association of collaborative care intervention features with depression and metabolic outcomes in the INDEPENDENT study: A mixed methods study

AIMS

The INtegrating DEPrEssioN and Diabetes treatmENT (INDEPENDENT) trial tested a collaborative care model including electronic clinical decision support (CDS) for treating diabetes and depression in India. We aimed to assess which features of this clinically and cost-effective intervention were associated with improvements in diabetes and depression measures.

METHODS

Post-hoc analysis of the INDEPENDENT trial data (189 intervention participants) was conducted to determine each intervention feature's effect: 1. Collaborative case reviews between expert psychiatrists and the care team; 2. Patient care-coordinator contacts; and 3. Clinicians' CDS prompt modifications. Primary outcome was baseline-to-12-months improvements in diabetes control, blood pressure, cholesterol, and depression. Implementer interviews revealed barriers and facilitators of intervention success. Joint displays integrated mixed methods' results.

RESULTS

High baseline HbA1c≥ 74.9 mmol/mol (9%) was associated with 5.72 fewer care-coordinator contacts than those with better baseline HbA1c (76.8 mmol/mol, 9.18%, p < 0.001). Prompt modification proportions varied from 38.3% (diabetes) to 1.3% (LDL). Interviews found that providers' and participants' visit frequencies were preference dependent. Qualitative data elucidated patient-level factors that influenced number of clinical contacts and prompt modifications explaining their lack of association with clinical outcomes.

CONCLUSION

Our mixed methods approach underlines the importance of the complementarity of different intervention features. Qualitative findings further illuminate reasons for variations in fidelity from the core model.

The role of artificial intelligence for the application of integrating electronic health records and patient-generated data in clinical decision support

This narrative review aims to identify and understand the role of artificial intelligence in the application of integrated electronic health records (EHRs) and patient-generated health data (PGHD) in clinical decision support. We focused on integrated data that combined PGHD and EHR data, and we investigated the role of artificial intelligence (AI) in the application. We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to search articles in six databases: PubMed, Embase, Web of Science, Scopus, ACM Digital Library, and IEEE Computer Society Digital Library. In addition, we also synthesized seminal sources, including other systematic reviews, reports, and white papers, to inform the context, history, and development of this field. Twenty-six publications met the review criteria after screening. The EHR-integrated PGHD introduces benefits to health care, including empowering patients and families to engage via shared decision-making, improving the patient-provider relationship, and reducing the time and cost of clinical visits. AI's roles include cleaning and management of heterogeneous datasets, assisting in identifying dynamic patterns to improve clinical care processes, and providing more sophisticated algorithms to better predict outcomes and propose precise recommendations based on the integrated data. Challenges mainly stem from the large volume of integrated data, data standards, data exchange and interoperability, security and privacy, interpretation, and meaningful use. The use of PGHD in health care is at a promising stage but needs further work for widespread adoption and seamless integration into health care systems. AI-driven, EHR-integrated PGHD systems can greatly improve clinicians' abilities to diagnose patients' health issues, classify risks at the patient level by drawing on the power of integrated data, and provide much-needed support to clinics and hospitals. With EHR-integrated PGHD, AI can help transform health care by improving diagnosis, treatment, and the delivery of clinical care, thus improving clinical decision support.

An Infrastructure Framework for Remote Patient Monitoring Interventions and Research

Remote patient monitoring (RPM) enables clinicians to maintain and adjust their patients' plan of care by using remotely gathered data, such as vital signs, to proactively make medical decisions about a patient's care. RPM interventions have been touted as a means to improve patient care and well-being while reducing costs and resource needs within the health care ecosystem. However, multiple interworking components must be successfully implemented for an RPM intervention to yield the desired outcomes, and the design and key driver of each component can vary depending on the medical context. This viewpoint and perspective paper presents a 4-component RPM infrastructure framework based on a synthesis of existing literature and practice related to RPM. Specifically, these components are identified and considered: (1) data collection, (2) data transmission and storage, (3) data analysis, and (4) information presentation. Interaction points to consider between components include transmission, interoperability, accessibility, workflow integration, and transparency. Within each of the 4 components, questions affecting research and practice emerge that can affect the outcomes of RPM interventions. This framework provides a holistic perspective of the technologies involved in RPM interventions and how these core elements interact to provide an appropriate infrastructure for deploying RPM in health systems. Further, it provides a common vocabulary to compare and contrast RPM solutions across health contexts and may stimulate new research and intervention opportunities.

Clinical decision support systems to optimize adherence to anticoagulant guidelines in patients with atrial fibrillation: a systematic review and meta-analysis of randomized controlled trials

BACKGROUND

Clinical decision support systems (CDSS) have been utilized as a low-cost intervention to improve healthcare process measures. Thus, we aim to estimate CDSS efficacy to optimize adherence to oral anticoagulant guidelines in eligible patients with atrial fibrillation (AF).

METHODS

A systematic review and meta-analysis of randomized controlled trials (RCTs) retrieved from PubMed, WOS, SCOPUS, EMBASE, and CENTRAL through August 2023. We used RevMan V. 5.4 to pool dichotomous data using risk ratio (RR) with a 95% confidence interval (CI).

PROSPERO ID

CRD42023471806.

RESULTS

We included nine RCTs with a total of 25,573 patients. There was no significant difference, with the use of CDSS compared to routine care, in the number of patients prescribed anticoagulants (RR: 1.06, 95% CI [0.98, 1.14], P = 0.16), the number of patients prescribed antiplatelets (RR: 1.01 with 95% CI [0.97, 1.06], P = 0.59), all-cause mortality (RR: 1.19, 95% CI [0.31, 4.50], P = 0.80), major bleeding (RR: 0.84, 95% CI [0.21, 3.45], P = 0.81), and clinically relevant non-major bleeding (RR: 1.05, 95% CI [0.52, 2.16], P = 0.88). However, CDSS was significantly associated with reduced incidence of myocardial infarction (RR: 0.18, 95% CI [0.06, 0.54], P = 0.002) and cerebral or systemic embolic event (RR: 0.11, 95% CI [0.01, 0.83], P = 0.03).

CONCLUSION

We report no significant difference with the use of CDSS compared to routine care in anticoagulant or antiplatelet prescription in eligible patients with AF. CDSS was associated with a reduced incidence of myocardial infarction and cerebral or systemic embolic events.

User-Centered Framework for Implementation of Technology (UFIT): Development of an Integrated Framework for Designing Clinical Decision Support Tools Packaged With Tailored Implementation Strategies

BACKGROUND

Electronic health record-based clinical decision support (CDS) tools can facilitate the adoption of evidence into practice. Yet, the impact of CDS beyond single-site implementation is often limited by dissemination and implementation barriers related to site- and user-specific variation in workflows and behaviors. The translation of evidence-based CDS from initial development to implementation in heterogeneous environments requires a framework that assures careful balancing of fidelity to core functional elements with adaptations to ensure compatibility with new contexts.

OBJECTIVE

This study aims to develop and apply a framework to guide tailoring and implementing CDS across diverse clinical settings.

METHODS

In preparation for a multisite trial implementing CDS for pediatric overweight or obesity in primary care, we developed the User-Centered Framework for Implementation of Technology (UFIT), a framework that integrates principles from user-centered design (UCD), human factors/ergonomics theories, and implementation science to guide both CDS adaptation and tailoring of related implementation strategies. Our transdisciplinary study team conducted semistructured interviews with pediatric primary care clinicians and a diverse group of stakeholders from 3 health systems in the northeastern, midwestern, and southeastern United States to inform and apply the framework for our formative evaluation.

RESULTS

We conducted 41 qualitative interviews with primary care clinicians (n=21) and other stakeholders (n=20). Our workflow analysis found 3 primary ways in which clinicians interact with the electronic health record during primary care well-child visits identifying opportunities for decision support. Additionally, we identified differences in practice patterns across contexts necessitating a multiprong design approach to support a variety of workflows, user needs, preferences, and implementation strategies.

CONCLUSIONS

UFIT integrates theories and guidance from UCD, human factors/ergonomics, and implementation science to promote fit with local contexts for optimal outcomes. The components of UFIT were used to guide the development of Improving Pediatric Obesity Practice Using Prompts, an integrated package comprising CDS for obesity or overweight treatment with tailored implementation strategies.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05627011; https://clinicaltrials.gov/study/NCT05627011.

Is the European Crohn's and Colitis organisation (ECCO) e-guide an acceptable and feasible tool for increasing gastroenterologists' guideline adherence? A mixed methods evaluation

BACKGROUND AND AIMS

Management of inflammatory bowel disease is constantly evolving, increasing the importance for gastroenterologists to keep up to date with guidelines. Traditional implementation strategies have had only small positive impacts on clinical practice. eHealth strategies such as the European Crohn's and Colitis Organisation e-guide may be beneficial for clinician decision making in keeping with guidelines. The aim of this study was to evaluate the feasibility and acceptability of the e-guide.

METHODS

A mixed methods approach was used to evaluate feasibility and acceptability. Cognitive (think-aloud) interviews were conducted with Australian gastroenterologists while using the e-guide. Two clinical scenarios were developed to allow evaluation of various aspects of the e-guide. Content analysis was applied to the qualitative interview data and descriptive analysis to the quantitative and observational data.

RESULTS

Seventeen participants completed the study. Data saturation were reached. The ECCO e-guide was largely feasible and acceptable, as demonstrated by most clinical questions answered correctly, 87% reaching the answer within 3 min, and most feeling it was useful, would be beneficial to their practice and would use it again. Issues raised included difficulties with website navigation, layout of the e-guide and difficulties with access (network firewalls, paid subscription required).

CONCLUSIONS

The ECCO e-guide is largely acceptable and feasible for gastroenterologists to use. Aspects of the e-guide could be modified to improve user experience. This study highlights the importance of engaging end-users in the development and evaluation of clinician educational tools.

Evolving Adult ADHD Care: Preparatory Evaluation of a Prototype Digital Service Model Innovation for ADHD Care

BACKGROUND

Given the prevalence of ADHD and the gaps in ADHD care in Australia, this study investigates the critical barriers and driving forces for innovation. It does so by conducting a preparatory evaluation of an ADHD prototype digital service innovation designed to help streamline ADHD care and empower individual self-management.

METHODS

Semi-structured interviews with ADHD care consumers/participants and practitioners explored their experiences and provided feedback on a mobile self-monitoring app and related service innovations. Interview transcripts were double coded to explore thematic barriers and the enablers for better ADHD care.

RESULTS

Fifteen interviews (9 consumers, 6 practitioners) revealed barriers to better ADHD care for consumers (ignorance and prejudice, trust, impatience) and for practitioners (complexity, sustainability). Enablers for consumers included validation/empowerment, privacy, and security frameworks, tailoring, and access. Practitioners highlighted the value of transparency, privacy and security frameworks, streamlined content, connected care between services, and the tailoring of broader metrics.

CONCLUSIONS

A consumer-centred approach to digital health service innovation, featuring streamlined, private, and secure solutions with enhanced mobile tools proves instrumental in bridging gaps in ADHD care in Australia. These innovations should help to address the gaps in ADHD care in Australia. These innovations should encompass integrated care, targeted treatment outcome data, and additional lifestyle support, whilst recognising the tensions between customised functionalities and streamlined displays.

Applying Machine Learning Techniques to Implementation Science

Machine learning (ML) approaches could expand the usefulness and application of implementation science methods in clinical medicine and public health settings. The aim of this viewpoint is to introduce a roadmap for applying ML techniques to address implementation science questions, such as predicting what will work best, for whom, under what circumstances, and with what predicted level of support, and what and when adaptation or deimplementation are needed. We describe how ML approaches could be used and discuss challenges that implementation scientists and methodologists will need to consider when using ML throughout the stages of implementation.

Genetically guided precision medicine clinical decision support tools: a systematic review

OBJECTIVES

Patient care using genetics presents complex challenges. Clinical decision support (CDS) tools are a potential solution because they provide patient-specific risk assessments and/or recommendations at the point of care. This systematic review evaluated the literature on CDS systems which have been implemented to support genetically guided precision medicine (GPM).

MATERIALS AND METHODS

A comprehensive search was conducted in MEDLINE and Embase, encompassing January 1, 2011-March 14, 2023. The review included primary English peer-reviewed research articles studying humans, focused on the use of computers to guide clinical decision-making and delivering genetically guided, patient-specific assessments, and/or recommendations to healthcare providers and/or patients.

RESULTS

The search yielded 3832 unique articles. After screening, 41 articles were identified that met the inclusion criteria. Alerts and reminders were the most common form of CDS used. About 27 systems were integrated with the electronic health record; 2 of those used standards-based approaches for genomic data transfer. Three studies used a framework to analyze the implementation strategy.

DISCUSSION

Findings include limited use of standards-based approaches for genomic data transfer, system evaluations that do not employ formal frameworks, and inconsistencies in the methodologies used to assess genetic CDS systems and their impact on patient outcomes.

CONCLUSION

We recommend that future research on CDS system implementation for genetically GPM should focus on implementing more CDS systems, utilization of standards-based approaches, user-centered design, exploration of alternative forms of CDS interventions, and use of formal frameworks to systematically evaluate genetic CDS systems and their effects on patient care.

Academic Detailing as a Health Information Technology Implementation Method: Supporting the Design and Implementation of an Emergency Department-Based Clinical Decision Support Tool to Prevent Future Falls

BACKGROUND

Clinical decision support (CDS) tools that incorporate machine learning-derived content have the potential to transform clinical care by augmenting clinicians' expertise. To realize this potential, such tools must be designed to fit the dynamic work systems of the clinicians who use them. We propose the use of academic detailing-personal visits to clinicians by an expert in a specific health IT tool-as a method for both ensuring the correct understanding of that tool and its evidence base and identifying factors influencing the tool's implementation.

OBJECTIVE

This study aimed to assess academic detailing as a method for simultaneously ensuring the correct understanding of an emergency department-based CDS tool to prevent future falls and identifying factors impacting clinicians' use of the tool through an analysis of the resultant qualitative data.

METHODS

Previously, our team designed a CDS tool to identify patients aged 65 years and older who are at the highest risk of future falls and prompt an interruptive alert to clinicians, suggesting the patient be referred to a mobility and falls clinic for an evidence-based preventative intervention. We conducted 10-minute academic detailing interviews (n=16) with resident emergency medicine physicians and advanced practice providers who had encountered our CDS tool in practice. We conducted an inductive, team-based content analysis to identify factors that influenced clinicians' use of the CDS tool.

RESULTS

The following categories of factors that impacted clinicians' use of the CDS were identified: (1) aspects of the CDS tool's design (2) clinicians' understanding (or misunderstanding) of the CDS or referral process, (3) the busy nature of the emergency department environment, (4) clinicians' perceptions of the patient and their associated fall risk, and (5) the opacity of the referral process. Additionally, clinician education was done to address any misconceptions about the CDS tool or referral process, for example, demonstrating how simple it is to place a referral via the CDS and clarifying which clinic the referral goes to.

CONCLUSIONS

Our study demonstrates the use of academic detailing for supporting the implementation of health information technologies, allowing us to identify factors that impacted clinicians' use of the CDS while concurrently educating clinicians to ensure the correct understanding of the CDS tool and intervention. Thus, academic detailing can inform both real-time adjustments of a tool's implementation, for example, refinement of the language used to introduce the tool, and larger scale redesign of the CDS tool to better fit the dynamic work environment of clinicians.

De-imFAR phase II project: a study protocol for a cluster randomised implementation trial to evaluate the effectiveness of de-implementation strategies to reduce low-value statin prescribing in the primary prevention of cardiovascular disease

INTRODUCTION

This study aims to reduce potentially inappropriate prescribing (PIP) of statins and foster healthy lifestyle promotion in cardiovascular disease (CVD) primary prevention in low-risk patients. To this end, we will compare the effectiveness and feasibility of several de-implementation strategies developed following the structured design process of the Behaviour Change Wheel targeting key determinants of the clinical decision-making process in CVD prevention.

METHODS AND ANALYSIS

A cluster randomised implementation trial, with an additional control group, will be launched, involving family physicians (FPs) from 13 Integrated Healthcare Organisations (IHOs) of Osakidetza-Basque Health Service with non-zero incidence rates of PIP of statins in 2021. All FPs will be exposed to a non-reflective decision assistance strategy based on reminders and decision support tools. Additionally, FPs from two of the IHOs will be randomly assigned to one of two increasingly intensive de-implementation strategies: adding a decision information strategy based on knowledge dissemination and a reflective decision structure strategy through audit/feedback. The target population comprises women aged 45-74 years and men aged 40-74 years with moderately elevated cholesterol levels but no diagnosed CVD and low cardiovascular risk (REGICOR<7.5%), who attend at least one appointment with any of the participating FPs (May 2022-May 2023), and will be followed until May 2024. We use the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework to evaluate outcomes. The main outcome will be the change in the incidence rate of PIP of statins and healthy lifestyle counselling in the study population 12 and 24 months after FPs' exposure to the strategies. Moreover, FPs' perception of their feasibility and acceptability, and patient experience regarding the quality of care received will be evaluated.

ETHICS AND DISSEMINATION

The study was approved by the Basque Country Clinical Research Ethics Committee and was registered in ClinicalTrials.gov (NCT04022850). Results will be disseminated in scientific peer-reviewed journals.

TRIAL REGISTRATION NUMBER

NCT04022850.

Evaluating ChatGPT-4's Diagnostic Accuracy: Impact of Visual Data Integration

BACKGROUND

In the evolving field of health care, multimodal generative artificial intelligence (AI) systems, such as ChatGPT-4 with vision (ChatGPT-4V), represent a significant advancement, as they integrate visual data with text data. This integration has the potential to revolutionize clinical diagnostics by offering more comprehensive analysis capabilities. However, the impact on diagnostic accuracy of using image data to augment ChatGPT-4 remains unclear.

OBJECTIVE

This study aims to assess the impact of adding image data on ChatGPT-4's diagnostic accuracy and provide insights into how image data integration can enhance the accuracy of multimodal AI in medical diagnostics. Specifically, this study endeavored to compare the diagnostic accuracy between ChatGPT-4V, which processed both text and image data, and its counterpart, ChatGPT-4, which only uses text data.

METHODS

We identified a total of 557 case reports published in the American Journal of Case Reports from January 2022 to March 2023. After excluding cases that were nondiagnostic, pediatric, and lacking image data, we included 363 case descriptions with their final diagnoses and associated images. We compared the diagnostic accuracy of ChatGPT-4V and ChatGPT-4 without vision based on their ability to include the final diagnoses within differential diagnosis lists. Two independent physicians evaluated their accuracy, with a third resolving any discrepancies, ensuring a rigorous and objective analysis.

RESULTS

The integration of image data into ChatGPT-4V did not significantly enhance diagnostic accuracy, showing that final diagnoses were included in the top 10 differential diagnosis lists at a rate of 85.1% (n=309), comparable to the rate of 87.9% (n=319) for the text-only version (P=.33). Notably, ChatGPT-4V's performance in correctly identifying the top diagnosis was inferior, at 44.4% (n=161), compared with 55.9% (n=203) for the text-only version (P=.002, χ2 test). Additionally, ChatGPT-4's self-reports showed that image data accounted for 30% of the weight in developing the differential diagnosis lists in more than half of cases.

CONCLUSIONS

Our findings reveal that currently, ChatGPT-4V predominantly relies on textual data, limiting its ability to fully use the diagnostic potential of visual information. This study underscores the need for further development of multimodal generative AI systems to effectively integrate and use clinical image data. Enhancing the diagnostic performance of such AI systems through improved multimodal data integration could significantly benefit patient care by providing more accurate and comprehensive diagnostic insights. Future research should focus on overcoming these limitations, paving the way for the practical application of advanced AI in medicine.

Implementation of an Electronic Clinical Decision Support System for the Early Recognition and Management of Dysglycemia in an Inpatient Mental Health Setting Using CogStack: Protocol for a Pilot Hybrid Type 3 Effectiveness-Implementation Randomized Controlled Cluster Trial

BACKGROUND

Severe mental illnesses (SMIs), including schizophrenia, bipolar affective disorder, and major depressive disorder, are associated with an increased risk of physical health comorbidities and premature mortality from conditions including cardiovascular disease and diabetes. Digital technologies such as electronic clinical decision support systems (eCDSSs) could play a crucial role in improving the clinician-led management of conditions such as dysglycemia (deranged blood sugar levels) and associated conditions such as diabetes in people with a diagnosis of SMI in mental health settings.

OBJECTIVE

We have developed a real-time eCDSS using CogStack, an information retrieval and extraction platform, to automatically alert clinicians with National Health Service Trust-approved, guideline-based recommendations for dysglycemia monitoring and management in secondary mental health care. This novel system aims to improve the management of dysglycemia and associated conditions, such as diabetes, in SMI. This protocol describes a pilot study to explore the acceptability, feasibility, and evaluation of its implementation in a mental health inpatient setting.

METHODS

This will be a pilot hybrid type 3 effectiveness-implementation randomized controlled cluster trial in inpatient mental health wards. A ward will be the unit of recruitment, where it will be randomly allocated to receive either access to the eCDSS plus usual care or usual care alone over a 4-month period. We will measure implementation outcomes, including the feasibility and acceptability of the eCDSS to clinicians, as primary outcomes, alongside secondary outcomes relating to the process of care measures such as dysglycemia screening rates. An evaluation of other implementation outcomes relating to the eCDSS will be conducted, identifying facilitators and barriers based on established implementation science frameworks.

RESULTS

Enrollment of wards began in April 2022, after which clinical staff were recruited to take part in surveys and interviews. The intervention period of the trial began in February 2023, and subsequent data collection was completed in August 2023. Data are currently being analyzed, and results are expected to be available in June 2024.

CONCLUSIONS

An eCDSS can have the potential to improve clinician-led management of dysglycemia in inpatient mental health settings. If found to be feasible and acceptable, then, in combination with the results of the implementation evaluation, the system can be refined and improved to support future successful implementation. A larger and more definitive effectiveness trial should then be conducted to assess its impact on clinical outcomes and to inform scalability and application to other conditions in wider mental health care settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04792268; https://clinicaltrials.gov/study/NCT04792268.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/49548.

Leveraging explainable artificial intelligence to optimize clinical decision support

OBJECTIVE

To develop and evaluate a data-driven process to generate suggestions for improving alert criteria using explainable artificial intelligence (XAI) approaches.

METHODS

We extracted data on alerts generated from January 1, 2019 to December 31, 2020, at Vanderbilt University Medical Center. We developed machine learning models to predict user responses to alerts. We applied XAI techniques to generate global explanations and local explanations. We evaluated the generated suggestions by comparing with alert's historical change logs and stakeholder interviews. Suggestions that either matched (or partially matched) changes already made to the alert or were considered clinically correct were classified as helpful.

RESULTS

The final dataset included 2 991 823 firings with 2689 features. Among the 5 machine learning models, the LightGBM model achieved the highest Area under the ROC Curve: 0.919 [0.918, 0.920]. We identified 96 helpful suggestions. A total of 278 807 firings (9.3%) could have been eliminated. Some of the suggestions also revealed workflow and education issues.

CONCLUSION

We developed a data-driven process to generate suggestions for improving alert criteria using XAI techniques. Our approach could identify improvements regarding clinical decision support (CDS) that might be overlooked or delayed in manual reviews. It also unveils a secondary purpose for the XAI: to improve quality by discovering scenarios where CDS alerts are not accepted due to workflow, education, or staffing issues.

Systematic development and validation of a predictive model for major postoperative complications in the Peri-operative Quality Improvement Project (PQIP) dataset

Complications are common following major surgery and are associated with increased use of healthcare resources, disability and mortality. Continued reliance on mortality estimates risks harming patients and health systems, but existing tools for predicting complications are unwieldy and inaccurate. We aimed to systematically construct an accurate pre-operative model for predicting major postoperative complications; compare its performance against existing tools; and identify sources of inaccuracy in predictive models more generally. Complete patient records from the UK Peri-operative Quality Improvement Programme dataset were analysed. Major complications were defined as Clavien-Dindo grade ≥ 2 for novel models. In a 75% train:25% test split cohort, we developed a pipeline of increasingly complex models, prioritising pre-operative predictors using Least Absolute Shrinkage and Selection Operators (LASSO). We defined the best model in the training cohort by the lowest Akaike's information criterion, balancing accuracy and simplicity. Of the 24,983 included cases, 6389 (25.6%) patients developed major complications. Potentially modifiable risk factors (pain, reduced mobility and smoking) were retained. The best-performing model was highly complex, specifying individual hospital complication rates and 11 patient covariates. This novel model showed substantially superior performance over generic and specific prediction models and scores. We have developed a novel complications model with good internal accuracy, re-prioritised predictor variables and identified hospital-level variation as an important, but overlooked, source of inaccuracy in existing tools. The complexity of the best-performing model does, however, highlight the need for a step-change in clinical risk prediction to automate the delivery of informative risk estimates in clinical systems.

Multisite development and validation of machine learning models to predict severe outcomes and guide decision-making for emergency department patients with influenza

Objective

Millions of Americans are infected by influenza annually. A minority seek care in the emergency department (ED) and, of those, only a limited number experience severe disease or death. ED clinicians must distinguish those at risk for deterioration from those who can be safely discharged.

Methods

We developed random forest machine learning (ML) models to estimate needs for critical care within 24 h and inpatient care within 72 h in ED patients with influenza. Predictor data were limited to those recorded prior to ED disposition decision: demographics, ED complaint, medical problems, vital signs, supplemental oxygen use, and laboratory results. Our study population was comprised of adults diagnosed with influenza at one of five EDs in our university health system between January 1, 2017 and May 18, 2022; visits were divided into two cohorts to facilitate model development and validation. Prediction performance was assessed by the area under the receiver operating characteristic curve (AUC) and the Brier score.

Results

Among 8032 patients with laboratory-confirmed influenza, incidence of critical care needs was 6.3% and incidence of inpatient care needs was 19.6%. The most common reasons for ED visit were symptoms of respiratory tract infection, fever, and shortness of breath. Model AUCs were 0.89 (95% CI 0.86-0.93) for prediction of critical care and 0.90 (95% CI 0.88-0.93) for inpatient care needs; Brier scores were 0.026 and 0.042, respectively. Importantpredictors included shortness of breath, increasing respiratory rate, and a high number of comorbid diseases.

Conclusions

ML methods can be used to accurately predict clinical deterioration in ED patients with influenza and have potential to support ED disposition decision-making.

Evaluating the use of clinical decision aids in an Australian emergency department: A cross-sectional survey

OBJECTIVE

To identify healthcare professionals' knowledge, self-reported use, and documentation of clinical decision aids (CDAs) in a large ED in Australia, to identify behavioural determinants influencing the use of CDAs, and healthcare professionals preferences for integrating CDAs into the electronic medical record (EMR) system.

METHODS

Healthcare professionals (doctors, nurses and physiotherapists) working in the ED at the Gold Coast Hospital, Queensland were invited to complete an online survey. Quantitative data were analysed using descriptive statistics, and where appropriate, mapped to the theoretical domains framework to identify potential barriers to the use of CDAs. Qualitative data were analysed using content analysis.

RESULTS

Seventy-four healthcare professionals (34 medical officers, 31 nurses and nine physiotherapists) completed the survey. Healthcare professionals' knowledge and self-reported use of 21 validated CDAs was low but differed considerably across CDAs. Only 4 out of 21 CDAs were reported to be used 'sometimes' or 'always' by the majority of respondents (Ottawa Ankle Rule for ankle injury, Wells' criteria for pulmonary embolism, Wells' criteria for deep vein thrombosis and PERC rule for pulmonary embolism). Most respondents wanted to increase their use of valid and reliable CDAs and supported the integration of CDAs into the EMR to facilitate their use and support documentation. Potential barriers impacting the use of CDAs represented three theoretical domains of knowledge, social/professional role and identity, and social influences.

CONCLUSIONS

CDAs are used variably by healthcare professionals and are inconsistently applied in the clinical encounter. Preferences of healthcare professionals need to be considered to allow the successful integration of CDAs into the EMR.

Trust and acceptability of data-driven clinical recommendations in everyday practice: A scoping review

BACKGROUND

Increasing attention is being given to the analysis of large health datasets to derive new clinical decision support systems (CDSS). However, few data-driven CDSS are being adopted into clinical practice. Trust in these tools is believed to be fundamental for acceptance and uptake but to date little attention has been given to defining or evaluating trust in clinical settings.

OBJECTIVES

A scoping review was conducted to explore how and where acceptability and trustworthiness of data-driven CDSS have been assessed from the health professional's perspective.

METHODS

Medline, Embase, PsycInfo, Web of Science, Scopus, ACM Digital, IEEE Xplore and Google Scholar were searched in March 2022 using terms expanded from: "data-driven" AND "clinical decision support" AND "acceptability". Included studies focused on healthcare practitioner-facing data-driven CDSS, relating directly to clinical care. They included trust or a proxy as an outcome, or in the discussion. The preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (PRISMA-ScR) is followed in the reporting of this review.

RESULTS

3291 papers were screened, with 85 primary research studies eligible for inclusion. Studies covered a diverse range of clinical specialisms and intended contexts, but hypothetical systems (24) outnumbered those in clinical use (18). Twenty-five studies measured trust, via a wide variety of quantitative, qualitative and mixed methods. A further 24 discussed themes of trust without it being explicitly evaluated, and from these, themes of transparency, explainability, and supporting evidence were identified as factors influencing healthcare practitioner trust in data-driven CDSS.

CONCLUSION

There is a growing body of research on data-driven CDSS, but few studies have explored stakeholder perceptions in depth, with limited focused research on trustworthiness. Further research on healthcare practitioner acceptance, including requirements for transparency and explainability, should inform clinical implementation.

PillHarmonics: An Orchestrated Pharmacogenetics Medication Clinical Decision Support Service

OBJECTIVES

 Pharmacogenetics (PGx) is increasingly important in individualizing therapeutic management plans, but is often implemented apart from other types of medication clinical decision support (CDS). The lack of integration of PGx into existing CDS may result in incomplete interaction information, which may pose patient safety concerns. We sought to develop a cloud-based orchestrated medication CDS service that integrates PGx with a broad set of drug screening alerts and evaluate it through a clinician utility study.

METHODS

 We developed the PillHarmonics service for implementation per the CDS Hooks protocol, algorithmically integrating a wide range of drug interaction knowledge using cloud-based screening services from First Databank (drug-drug/allergy/condition), PharmGKB (drug-gene), and locally curated content (drug-renal/hepatic/race). We performed a user study, presenting 13 clinicians and pharmacists with a prototype of the system's usage in synthetic patient scenarios. We collected feedback via a standard questionnaire and structured interview.

RESULTS

 Clinician assessment of PillHarmonics via the Technology Acceptance Model questionnaire shows significant evidence of perceived utility. Thematic analysis of structured interviews revealed that aggregated knowledge, concise actionable summaries, and information accessibility were highly valued, and that clinicians would use the service in their practice.

CONCLUSION

 Medication safety and optimizing efficacy of therapy regimens remain significant issues. A comprehensive medication CDS system that leverages patient clinical and genomic data to perform a wide range of interaction checking and presents a concise and holistic view of medication knowledge back to the clinician is feasible and perceived as highly valuable for more informed decision-making. Such a system can potentially address many of the challenges identified with current medication-related CDS.

Global Trends and Hotspots in Nursing Research on Decision Support Systems: A Bibliometric Analysis in CiteSpace

Decision support systems have been widely used in healthcare in recent years; however, there is lack of evidence on global trends and hotspots. This descriptive bibliometric study aimed to analyze bibliometric patterns of decision support systems in nursing. Data were extracted from the Web of Science Core Collection. Published research articles on decision support systems in nursing were identified. Co-occurrence and co-citation analysis was performed using CiteSpace version 6.1.R2. In total, 165 articles were analyzed. A total of 358 authors and 257 institutions from 20 countries contributed to this research field. The most productive authors were Andrew Johnson, Suzanne Bakken, Alessandro Febretti, Eileen S. O'Neill, and Kathryn H. Bowles. The most productive country and institution were the United States and Duke University, respectively. The top 10 keywords were "care," "clinical decision support," "clinical decision support system," "decision support system," "electronic health record," "system," "nursing informatics," "guideline," "decision support," and "outcomes." Common themes on keywords were planning intervention, national health information infrastructure, and methodological challenge. This study will help to find potential partners, countries, and institutions for future researchers, practitioners, and scholars. Additionally, it will contribute to health policy development, evidence-based practice, and further studies for researchers, practitioners, and scholars.

Formative evaluation of an emergency department clinical decision support system for agitation symptoms: a study protocol

INTRODUCTION

The burden of mental health-related visits to emergency departments (EDs) is growing, and agitation episodes are prevalent with such visits. Best practice guidance from experts recommends early assessment of at-risk populations and pre-emptive intervention using de-escalation techniques to prevent agitation. Time pressure, fluctuating work demands, and other systems-related factors pose challenges to efficient decision-making and adoption of best practice recommendations during an unfolding behavioural crisis. As such, we propose to design, develop and evaluate a computerised clinical decision support (CDS) system, Early Detection and Treatment to Reduce Events with Agitation Tool (ED-TREAT). We aim to identify patients at risk of agitation and guide ED clinicians through appropriate risk assessment and timely interventions to prevent agitation with a goal of minimising restraint use and improving patient experience and outcomes.

METHODS AND ANALYSIS

This study describes the formative evaluation of the health record embedded CDS tool. Under aim 1, the study will collect qualitative data to design and develop ED-TREAT using a contextual design approach and an iterative user-centred design process. Participants will include potential CDS users, that is, ED physicians, nurses, technicians, as well as patients with lived experience of restraint use for behavioural crisis management during an ED visit. We will use purposive sampling to ensure the full spectrum of perspectives until we reach thematic saturation. Next, under aim 2, the study will conduct a pilot, randomised controlled trial of ED-TREAT at two adult ED sites in a regional health system in the Northeast USA to evaluate the feasibility, fidelity and bedside acceptability of ED-TREAT. We aim to recruit a total of at least 26 eligible subjects under the pilot trial.

ETHICS AND DISSEMINATION

Ethical approval by the Yale University Human Investigation Committee was obtained in 2021 (HIC# 2000030893 and 2000030906). All participants will provide informed verbal consent prior to being enrolled in the study. Results will be disseminated through publications in open-access, peer-reviewed journals, via scientific presentations or through direct email notifications.

TRIAL REGISTRATION NUMBER

NCT04959279; Pre-results.

The Role of Artificial Intelligence in Improving Patient Outcomes and Future of Healthcare Delivery in Cardiology: A Narrative Review of the Literature

Cardiovascular diseases exert a significant burden on the healthcare system worldwide. This narrative literature review discusses the role of artificial intelligence (AI) in the field of cardiology. AI has the potential to assist healthcare professionals in several ways, such as diagnosing pathologies, guiding treatments, and monitoring patients, which can lead to improved patient outcomes and a more efficient healthcare system. Moreover, clinical decision support systems in cardiology have improved significantly over the past decade. The addition of AI to these clinical decision support systems can improve patient outcomes by processing large amounts of data, identifying subtle associations, and providing a timely, evidence-based recommendation to healthcare professionals. Lastly, the application of AI allows for personalized care by utilizing predictive models and generating patient-specific treatment plans. However, there are several challenges associated with the use of AI in healthcare. The application of AI in healthcare comes with significant cost and ethical considerations. Despite these challenges, AI will be an integral part of healthcare delivery in the near future, leading to personalized patient care, improved physician efficiency, and anticipated better outcomes.

Retrospective evaluation of clinical decision support for within-laboratory optimization of SARS-CoV-2 NAAT workflow

The unprecedented demand for severe acute respiratory syndrome coronavirus 2 (SARS‑CoV‑2) testing led to challenges in prioritizing and processing specimens efficiently. We describe and evaluate a novel workflow using provider- and patient-facing ask at order entry (AOE) questions to generate distinctive icons on specimen labels for within-laboratory clinical decision support (CDS) for specimen triaging. A multidisciplinary committee established target turnaround times (TATs) for SARS-CoV-2 nucleic acid amplification test (NAAT) based on common clinical scenarios. A set of AOE questions was used to collect relevant clinical information that prompted icon generation for triaging SARS-CoV-2 NAAT specimens. We assessed the collect-to-verify TATs among relevant clinical scenarios. Our study included a total of 1,385,813 SARS-CoV-2 NAAT conducted from March 2020 to June 2022. Most testing met the TAT targets established by institutional committees, but deviations from target TATs occurred during periods of high demand and supply shortages. Median TATs for emergency department (ED) and inpatient specimens and ambulatory pre-procedure populations were stable over the pandemic. However, healthcare worker and other ambulatory test TATs varied substantially, depending on testing volume and community transmission rates. Median TAT significantly differed throughout the pandemic for ED and inpatient clinical scenarios, and there were significant differences in TAT among label icon-signified ambulatory clinical scenarios. We describe a novel approach to CDS for triaging specimens within the laboratory. The use of CDS tools could help clinical laboratories prioritize and process specimens efficiently, especially during times of high demand. Further studies are needed to evaluate the impact of our CDS tool on overall laboratory efficiency and patient outcomes. IMPORTANCE We describe a novel approach to clinical decision support (CDS) for triaging specimens within the clinical laboratory for severe acute respiratory syndrome coronavirus 2 (SARS‑CoV‑2) nucleic acid amplification tests (NAAT). The use of our CDS tool could help clinical laboratories prioritize and process specimens efficiently, especially during times of high demand. There were significant differences in the turnaround time for specimens differentiated by icons on specimen labels. Further studies are needed to evaluate the impact of our CDS tool on overall laboratory efficiency and patient outcomes.

Deployment of a human-centred clinical decision support system for pulmonary embolism: evaluation of impact on quality of diagnostic decisions

Pulmonary embolism (PE) is a serious condition that presents a diagnostic challenge for which diagnostic errors often happen. The literature suggests that a gap remains between PE diagnostic guidelines and adherence in healthcare practice. While system-level decision support tools exist, the clinical impact of a human-centred design (HCD) approach of PE diagnostic tool design is unknown.

DESIGN

Before-after (with a preintervention period as non-concurrent control) design study.

SETTING

Inpatient units at two tertiary care hospitals.

PARTICIPANTS

General internal medicine physicians and their patients who underwent PE workups.

INTERVENTION

After a 6-month preintervention period, a clinical decision support system (CDSS) for diagnosis of PE was deployed and evaluated over 6 months. A CDSS technical testing phase separated the two time periods.

MEASUREMENTS

PE workups were identified in both the preintervention and CDSS intervention phases, and data were collected from medical charts. Physician reviewers assessed workup summaries (blinded to the study period) to determine adherence to evidence-based recommendations. Adherence to recommendations was quantified with a score ranging from 0 to 1.0 (the primary study outcome). Diagnostic tests ordered for PE workups were the secondary outcomes of interest.

RESULTS

Overall adherence to diagnostic pathways was 0.63 in the CDSS intervention phase versus 0.60 in the preintervention phase (p=0.18), with fewer workups in the CDSS intervention phase having very low adherence scores. Further, adherence was significantly higher when PE workups included the Wells prediction rule (median adherence score=0.76 vs 0.59, p=0.002). This difference was even more pronounced when the analysis was limited to the CDSS intervention phase only (median adherence score=0.80 when Wells was used vs 0.60 when Wells was not used, p=0.001). For secondary outcomes, using both the D-dimer blood test (42.9% vs 55.7%, p=0.014) and CT pulmonary angiogram imaging (61.9% vs 75.4%, p=0.005) was lower during the CDSS intervention phase.

CONCLUSION

A clinical decision support intervention with an HCD improves some aspects of the diagnostic decision, such as the selection of diagnostic tests and the use of the Wells probabilistic prediction rule for PE.

Electronic Integrated Management of Childhood Illness (eIMCI): a randomized controlled trial to evaluate an electronic clinical decision-making support system for management of sick children in primary health care facilities in South Africa

BACKGROUND

Electronic clinical decision-making support systems (eCDSS) aim to assist clinicians making complex patient management decisions and improve adherence to evidence-based guidelines. Integrated management of Childhood Illness (IMCI) provides guidelines for management of sick children attending primary health care clinics and is widely implemented globally. An electronic version of IMCI (eIMCI) was developed in South Africa.

METHODS

We conducted a cluster randomized controlled trial comparing management of sick children with eIMCI to the management when using paper-based IMCI (pIMCI) in one district in KwaZulu-Natal. From 31 clinics in the district, 15 were randomly assigned to intervention (eIMCI) or control (pIMCI) groups. Computers were deployed in eIMCI clinics, and one IMCI trained nurse was randomly selected to participate from each clinic. eIMCI participants received a one-day computer training, and all participants received a similar three-day IMCI update and two mentoring visits. A quantitative survey was conducted among mothers and sick children attending participating clinics to assess the quality of care provided by IMCI practitioners. Sick child assessments by participants in eIMCI and pIMCI groups were compared to assessment by an IMCI expert.

RESULTS

Self-reported computer skills were poor among all nurse participants. IMCI knowledge was similar in both groups. Among 291 enrolled children: 152 were in the eIMCI group; 139 in the pIMCI group. The mean number of enrolled children was 9.7 per clinic (range 7-12). IMCI implementation was sub-optimal in both eIMCI and pIMCI groups. eIMCI consultations took longer than pIMCI consultations (median duration 28 minutes vs 25 minutes; p = 0.02). eIMCI participants were less likely than pIMCI participants to correctly classify children for presenting symptoms, but were more likely to correctly classify for screening conditions, particularly malnutrition. eIMCI participants were less likely to provide all required medications (124/152; 81.6% vs 126/139; 91.6%, p= 0.026), and more likely to prescribe unnecessary medication (48/152; 31.6% vs 20/139; 14.4%, p = 0.004) compared to pIMCI participants.

CONCLUSIONS

Implementation of eIMCI failed to improve management of sick children, with poor IMCI implementation in both groups. Further research is needed to understand barriers to comprehensive implementation of both pIMCI and eIMCI. (349) CLINICAL TRIALS REGISTRATION: Clinicaltrials.gov ID: BFC157/19, August 2019.

Clinical decision support systems (CDSS) in assistance to COVID-19 diagnosis: A scoping review on types and evaluation methods

Background and Aims

Due to the COVID-19 pandemic, a precise and reliable diagnosis of this disease is critical. The use of clinical decision support systems (CDSS) can help facilitate the diagnosis of COVID-19. This scoping review aimed to investigate the role of CDSS in diagnosing COVID-19.

Methods

We searched four databases (Web of Science, PubMed, Scopus, and Embase) using three groups of keywords related to CDSS, COVID-19, and diagnosis. To collect data from studies, we utilized a data extraction form that consisted of eight fields. Three researchers selected relevant articles and extracted data using a data collection form. To resolve any disagreements, we consulted with a fourth researcher.

Results

A search of the databases retrieved 2199 articles, of which 68 were included in this review after removing duplicates and irrelevant articles. The studies used nonknowledge-based CDSS (n = 52) and knowledge-based CDSS (n = 16). Convolutional Neural Networks (CNN) (n = 33) and Support Vector Machine (SVM) (n = 8) were employed to design the CDSS in most of the studies. Accuracy (n = 43) and sensitivity (n = 35) were the most common metrics for evaluating CDSS.

Conclusion

CDSS for COVID-19 diagnosis have been developed mainly through machine learning (ML) methods. The greater use of these techniques can be due to their availability of public data sets about chest imaging. Although these studies indicate high accuracy for CDSS based on ML, their novelty and data set biases raise questions about replacing these systems as clinician assistants in decision-making. Further studies are needed to improve and compare the robustness and reliability of nonknowledge-based and knowledge-based CDSS in COVID-19 diagnosis.