Reasons For Physicians Not Adopting Clinical Decision Support Systems: Critical Analysis

Development and application of Breadth-Depth-Context (BDC), a conceptual framework for measuring technology engagement with a qualified clinical data registry

Objectives

Despite the proliferation of dashboards that display performance data derived from Qualified Clinical Data Registries (QCDR), the degree to which clinicians and practices engage with such dashboards has not been well described. We aimed to develop a conceptual framework for assessing user engagement with dashboard technology and to demonstrate its application to a rheumatology QCDR.

Materials and Methods

We developed the BDC (Breadth-Depth-Context) framework, which included concepts of breadth (derived from dashboard sessions), depth (derived from dashboard actions), and context (derived from practice characteristics). We demonstrated its application via user log data from the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) registry to define engagement profiles and characterize practice-level factors associated with different profiles.

Results

We applied the BDC framework to 213 ambulatory practices from the RISE registry in 2020-2021, and classified practices into 4 engagement profiles: not engaged (8%), minimally engaged (39%), moderately engaged (34%), and most engaged (19%). Practices with more patients and with specific electronic health record vendors (eClinicalWorks and eMDs) had a higher likelihood of being in the most engaged group, even after adjusting for other factors.

Discussion

We developed the BDC framework to characterize user engagement with a registry dashboard and demonstrated its use in a specialty QCDR. The application of the BDC framework revealed a wide range of breadth and depth of use and that specific contextual factors were associated with nature of engagement.

Conclusion

Going forward, the BDC framework can be used to study engagement with similar dashboards.

Piloting a Mobile Clinical Decision Support Application for Pediatric Clinical Guidelines

Mobile Clinical Decision Support Systems (CDSSs) represent an increasingly utilized technology to promote clinical guideline use. We sought to explore clinician guideline use and access preferences during implementation of a mobile guideline app at a free-standing children's hospital integrating 23 guidelines. Surveys included demographic variables and access preferences among anonymous onboarded clinicians in January 2022. Response rate was 21.8% (57/261) among onboarded users, mostly attending (59.6%) and resident/fellow physicians (21.1%) in inpatient (42.1%) and emergency department (31.6%) settings. Onboarded users accessed guidelines on over half of shifts (68.4%) and quickly (80.7%, <1 minute). Overall, most users reported favorable patterns for adoption of mobile CDSSs as useful adjuncts to existing formats. Users reported more ease of access and frequent guideline usage, particularly for younger clinicians. Guidelines related to antibiotic decision-making or newer disease processes were most useful. Further study is needed on electronic health record incorporation, adherence, and patient outcomes.

An Expert-Knowledge-Based Graph Convolutional Network for Skeleton- Based Physical Rehabilitation Exercises Assessment

Physical therapists play a crucial role in guiding patients through effective and safe rehabilitation processes according to medical guidelines. However, due to the therapist-patient imbalance, it is neither economical nor feasible for therapists to provide guidance to every patient during recovery sessions. Automated assessment of physical rehabilitation can help with this problem, but accurately quantifying patients' training movements and providing meaningful feedback poses a challenge. In this paper, an Expert-knowledge-based Graph Convolutional approach is proposed to automate the assessment of the quality of physical rehabilitation exercises. This approach utilizes experts' knowledge to improve the spatial feature extraction ability of the Graph Convolutional module and a Gated pooling module for feature aggregation. Additionally, a Transformer module is employed to capture long-range temporal dependencies in the movements. The attention scores and weight matrix obtained through this approach can serve as interpretability tools to help therapists understand the assessment model and assist patients in improving their exercises. The effectiveness of the proposed method is verified on the KIMORE dataset, achieving state-of-the-art performance compared to existing models. Experimental results also illustrate the interpretability of the method in both spatial and temporal dimensions.

Harnessing innovation to help meet the needs of elders: field testing an electronic tool to streamline geriatric assessments across healthcare settings

BACKGROUND

As populations age globally, effectively managing geriatric health poses challenges for primary care. Comprehensive geriatric assessments (CGAs) aim to address these challenges through multidisciplinary screening and coordinated care planning. However, most CGA tools and workflows have not been optimised for routine primary care delivery.

OBJECTIVE

This study aimed to evaluate the impact of a computerised CGA tool, called the Golden Age Visit, implemented in primary care in Israel.

METHODS

This study employed a quasiexperimental mixed-methods design to evaluate outcomes associated with the Golden Age electronic health assessment tool. Quantitative analysis used electronic medical records data from Maccabi Healthcare Services, the second largest health management organisation (HMO) in Israel. Patients aged 75 and older were included in analyses from January 2017 to December 2019 and January 2021 to December 2022. For patients, data were also collected on controls who did not participate in the Golden Age Visit programme during the same time period, to allow for comparison of outcomes. For physicians, qualitative data were collected via surveys and interviews with primary care physicians who used the Golden Age Visit SMARTEST e-assessment tool.

RESULTS

A total of 9022 community-dwelling adults aged 75 and older were included in the study: 1421 patients received a Golden Age Visit CGA (intervention group), and 7601 patients did not receive the assessment (control group). After CGAs, diagnosis rates increased significantly for neuropsychiatric conditions and falls. Referrals to physiotherapy, occupational therapy, dietetics and geriatric outpatient clinics also rose substantially. However, no differences were found in rates of hip fracture or relocation to long-term care between groups. Surveys among physicians (n=151) found high satisfaction with the programme.

CONCLUSION

Implementation of a large-scale primary care CGA programme was associated with improved diagnosis and management of geriatric conditions. Physicians were also satisfied, suggesting good uptake and feasibility within usual care. Further high-quality studies are still needed but these results provide real-world support for proactively addressing geriatric health needs through structured screening models.

Clinician views on how clinical decision support systems can help diagnose asthma in primary care: a qualitative study

OBJECTIVE

Asthma can be difficult to diagnose in primary care. Clinical decision support systems (CDSS) can assist clinicians when making diagnostic decisions, but the perspectives of intended users need to be incorporated into the software if the CDSS is to be clinically useful. Therefore, we aimed to understand health professional views on the value of an asthma diagnosis CDSS and the barriers and facilitators for use in UK primary care.

METHODS

We recruited doctors and nurses working in UK primary care who had experience of assessing respiratory symptoms and diagnosing asthma. Qualitative interviews were used to explore clinicians' experiences of making a diagnosis of asthma and understand views on a CDSS to support asthma diagnosis. Interviews were audio-recorded, transcribed verbatim and analyzed thematically.

RESULTS

16 clinicians (nine doctors, seven nurses) including 13 participants with over 10 years experience, contributed interviews. Participants saw the potential for a CDSS to support asthma diagnosis in primary care by structuring consultations, identifying relevant information from health records, and having visuals to communicate findings to patients. Being evidence based, regularly updated, integrated with software, quick and easy to use were considered important for a CDSS to be successfully implemented. Experienced clinicians were unsure a CDSS would help their routine practice, particularly in straightforward diagnostic scenarios, but thought a CDSS would be useful for trainees or less experienced colleagues.

CONCLUSIONS

To be adopted into clinical practice, clinicians were clear that a CDSS must be validated, integrated with existing software, and quick and easy to use.

Evaluating Pilot Implementation of 'PenCS Flu Topbar' App in Medical Practices to Improve National Immunisation Program-Funded Seasonal Influenza Vaccination in Central Queensland, Australia

BACKGROUND

The 'PenCS Flu Topbar' app was deployed in Central Queensland (CQ), Australia, medical practices through a pilot programme in March 2021.

METHODS

We evaluated the app's user experience and examined whether the introduction of 'PenCS Flu Topbar' in medical practices could improve the coverage of NIP-funded influenza vaccinations. We conducted a mixed-method study including a qualitative analysis of in-depth interviews with key end-users and a quantitative analysis of influenza vaccine administrative data.

RESULTS

'PenCS Flu Topbar' app users reported positive experiences identifying patients eligible for NIP-funded seasonal influenza vaccination. A total of 3606 NIP-funded influenza vaccinations was administered in the eight intervention practices, 14% higher than the eight control practices. NIP-funded vaccination coverage within practices was significantly higher in the intervention practices (31.2%) than in the control practices (27.3%) (absolute difference: 3.9%; 95% CI: 2.9%-5.0%; p < 0.001). The coverage was substantially higher in Aboriginal and Torres Strait Islander people aged more than 6 months, pregnant women and children aged 6 months to less than 5 years for the practices where the app was introduced when compared to control practices: incidence rate ratio (IRR) 2.4 (95% CI: 1.8-3.2), IRR 2.7 (95% CI: 1.8-4.2) and IRR 2.3 (1.8-2.9) times higher, respectively.

CONCLUSIONS

Our evaluation indicated that the 'PenCS Flu Topbar' app is useful for identifying the patients eligible for NIP-funded influenza vaccination and is likely to increase NIP-funded influenza vaccine coverage in the eligible populations. Future impact evaluation including a greater number of practices and a wider geographical area is essential.

Clinical Decision Support Requirements for Ventricular Tachycardia Diagnosis Within the Frameworks of Knowledge and Practice: Survey Study

BACKGROUND

Ventricular tachycardia (VT) diagnosis is challenging due to the similarity between VT and some forms of supraventricular tachycardia, complexity of clinical manifestations, heterogeneity of underlying diseases, and potential for life-threatening hemodynamic instability. Clinical decision support systems (CDSSs) have emerged as promising tools to augment the diagnostic capabilities of cardiologists. However, a requirements analysis is acknowledged to be vital for the success of a CDSS, especially for complex clinical tasks such as VT diagnosis.

OBJECTIVE

The aims of this study were to analyze the requirements for a VT diagnosis CDSS within the frameworks of knowledge and practice and to determine the clinical decision support (CDS) needs.

METHODS

Our multidisciplinary team first conducted semistructured interviews with seven cardiologists related to the clinical challenges of VT and expected decision support. A questionnaire was designed by the multidisciplinary team based on the results of interviews. The questionnaire was divided into four sections: demographic information, knowledge assessment, practice assessment, and CDS needs. The practice section consisted of two simulated cases for a total score of 10 marks. Online questionnaires were disseminated to registered cardiologists across China from December 2022 to February 2023. The scores for the practice section were summarized as continuous variables, using the mean, median, and range. The knowledge and CDS needs sections were assessed using a 4-point Likert scale without a neutral option. Kruskal-Wallis tests were performed to investigate the relationship between scores and practice years or specialty.

RESULTS

Of the 687 cardiologists who completed the questionnaire, 567 responses were eligible for further analysis. The results of the knowledge assessment showed that 383 cardiologists (68%) lacked knowledge in diagnostic evaluation. The overall average score of the practice assessment was 6.11 (SD 0.55); the etiological diagnosis section had the highest overall scores (mean 6.74, SD 1.75), whereas the diagnostic evaluation section had the lowest scores (mean 5.78, SD 1.19). A majority of cardiologists (344/567, 60.7%) reported the need for a CDSS. There was a significant difference in practice competency scores between general cardiologists and arrhythmia specialists (P=.02).

CONCLUSIONS

There was a notable deficiency in the knowledge and practice of VT among Chinese cardiologists. Specific knowledge and practice support requirements were identified, which provide a foundation for further development and optimization of a CDSS. Moreover, it is important to consider clinicians' specialization levels and years of practice for effective and personalized support.

Clinical Decision Support System Used in Spinal Disorders: Scoping Review

BACKGROUND

Spinal disorders are highly prevalent worldwide with high socioeconomic costs. This cost is associated with the demand for treatment and productivity loss, prompting the exploration of technologies to improve patient outcomes. Clinical decision support systems (CDSSs) are computerized systems that are increasingly used to facilitate safe and efficient health care. Their applications range in depth and can be found across health care specialties.

OBJECTIVE

This scoping review aims to explore the use of CDSSs in patients with spinal disorders.

METHODS

We used the Joanna Briggs Institute methodological guidance for this scoping review and reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) statement. Databases, including PubMed, Embase, Cochrane, CINAHL, Web of Science, Scopus, ProQuest, and PsycINFO, were searched from inception until October 11, 2022. The included studies examined the use of digitalized CDSSs in patients with spinal disorders.

RESULTS

A total of 4 major CDSS functions were identified from 31 studies: preventing unnecessary imaging (n=8, 26%), aiding diagnosis (n=6, 19%), aiding prognosis (n=11, 35%), and recommending treatment options (n=6, 20%). Most studies used the knowledge-based system. Logistic regression was the most commonly used method, followed by decision tree algorithms. The use of CDSSs to aid in the management of spinal disorders was generally accepted over the threat to physicians' clinical decision-making autonomy.

CONCLUSIONS

Although the effectiveness was frequently evaluated by examining the agreement between the decisions made by the CDSSs and the health care providers, comparing the CDSS recommendations with actual clinical outcomes would be preferable. In addition, future studies on CDSS development should focus on system integration, considering end user's needs and preferences, and external validation and impact studies to assess effectiveness and generalizability.

TRIAL REGISTRATION

OSF Registries osf.io/dyz3f; https://osf.io/dyz3f.

Implementation Requires Evaluation of Adoption: Lessons From a Multimodal Pain Regimen Order Set

INTRODUCTION

Multimodal pain regimen (MMPR) protocols are the standard of care per the 2020 Trauma Quality Improvement Program guidelines. MMPR implementation methodology in trauma services has not been reported. The primary objective of this study was to evaluate the adoption of an MMPR order set at a level 1 trauma center and to describe its implementation. We hypothesized that order set utilization would be about 50%, and barriers to adoption would be related to personal biases.

METHODS

This was a mixed-methods study at a level 1 trauma center. We retrospectively evaluated MMPR utilization from July 1, 2021 to February 28, 2022. Agile implementation was the method used to implement a clinical decision support tool for the MMPR: a flow chart order set in the electronic medical record. This methodology utilizes short experiment sprints during which data are collected to guide the next iterations. During this process quantitative as well as qualitative data were collected. This included end user testing of the order set and a survey distributed to surgical residents about the order set. Manual thematic network analysis was employed to identify basic and organizing themes from the survey responses.

RESULTS

A total of 587 trauma patients were admitted during the study period and 95 patients (16.2%) had MMPR ordered through the order set. The survey response rate was 19% (13/68). We identified ease of use, desire for options, inadequate education, and assumption of personal expertise as the four basic themes from the survey. These basic themes were further analyzed to two organizing themes: heuristics and overconfidence bias.

CONCLUSIONS

The MMPR order set was easy to use but had low adoption at our center in the first 8 months of implementation. Agile implementation methodology provided an ideal framework to identify reasons for low adoption and guide the next sprint to address personal biases, improve heuristics, and provide effective education and dissemination. Evaluation of utilization and qualitative analysis are key components to ensuring clinical decision support tool adoption.

'Knowledge exchange' workshops to optimise development of a risk prediction tool to assist conveyance decisions for suspected seizures - Part of the Risk of ADverse Outcomes after a Suspected Seizure (RADOSS) project

PURPOSE

Suspected seizures present challenges for ambulance services, with paramedics reporting uncertainty over whether or not to convey individuals to emergency departments. The Risk of ADverse Outcomes after a Suspected Seizure (RADOSS) project aims to address this by developing a risk assessment tool utilizing structured patient care record and dispatch data. It proposes a tool that would provide estimates of an individual's likelihood of death and/or recontact with emergency care within 3 days if conveyed compared to not conveyed, and the likelihood of an 'avoidable attendance' occurring if conveyed. Knowledge Exchange workshops engaged stakeholders to resolve key design uncertainties before model derivation.

METHOD

Six workshops involved 26 service users and their significant others (epilepsy or nonepileptic attack disorder), and 25 urgent and emergency care clinicians from different English ambulance regions. Utilizing Nominal Group Techniques, participants shared views of the proposed tool, benefits and concerns, suggested predictors, critiqued outcome measures, and expressed functionality preferences. Data were analysed using Hamilton's Rapid Analysis.

RESULTS

Stakeholders supported tool development, proposing 10 structured variables for predictive testing. Emphasis was placed on the tool supporting, not dictating, care decisions. Participants highlighted some reasons why RADOSS might struggle to derive a predictive model based on structured data alone and suggested some non-structured variables for future testing. Feedback on prediction timeframes for service recontact was received, along with advice on amending the 'avoidable attendance' definition to prevent the tool's predictions being undermined by potential overuse of certain investigations in hospital.

CONCLUSION

Collaborative stakeholder engagement provided crucial insights that can guide RADOSS to develop a user-aligned, optimized tool.

CURATE.AI-assisted dose titration for anti-hypertensive personalized therapy: study protocol for a multi-arm, randomized, pilot feasibility trial using CURATE.AI (CURATE.AI ADAPT trial)

Aims

Artificial intelligence-driven small data platforms such as CURATE.AI hold potential for personalized hypertension care by assisting physicians in identifying personalized anti-hypertensive doses for titration. This trial aims to assess the feasibility of a larger randomized controlled trial (RCT), evaluating the efficacy of CURATE.AI-assisted dose titration intervention. We will also collect preliminary efficacy and safety data and explore stakeholder feedback in the early design process.

Methods and results

In this open-label, randomized, pilot feasibility trial, we aim to recruit 45 participants with primary hypertension. Participants will be randomized in 1:1:1 ratio into control (no intervention), home blood pressure monitoring (active control; HBPM), or CURATE.AI arms (intervention; HBPM and CURATE.AI-assisted dose titration). The home treatments include 1 month of two-drug anti-hypertensive regimens. Primary endpoints assess the logistical (e.g. dose adherence) and scientific (e.g. percentage of participants for which CURATE.AI profiles can be generated) feasibility, and define the progression criteria for the RCT in a 'traffic light system'. Secondary endpoints assess preliminary efficacy [e.g. mean change in office blood pressures (BPs)] and safety (e.g. hospitalization events) associated with each treatment protocol. Participants with both baseline and post-treatment BP measurements will form the intent-to-treat analysis. Following their involvement with the CURATE.AI intervention, feedback from CURATE.AI participants and healthcare providers will be collected via exit survey and interviews.

Conclusion

Findings from this study will inform about potential refinements of the current treatment protocols before proceeding with a larger RCT, or potential expansion to collect additional information. Positive results may suggest the potential efficacy of CURATE.AI to improve BP control.

Trial registration number

NCT05376683.

Ability of machine-learning based clinical decision support system to reduce alert fatigue, wrong-drug errors, and alert users about look alike, sound alike medication

BACKGROUND AND OBJECTIVE

The overall benefits of using clinical decision support systems (CDSSs) can be restrained if physicians inadvertently ignore clinically useful alerts due to "alert fatigue" caused by an excessive number of clinically irrelevant warnings. Moreover, inappropriate drug errors, look-alike/sound-alike (LASA) drug errors, and problem list documentation are common, costly, and potentially harmful. This study sought to evaluate the overall performance of a machine learning-based CDSS (MedGuard) for triggering clinically relevant alerts, acceptance rate, and to intercept inappropriate drug errors as well as LASA drug errors.

METHODS

We conducted a retrospective study that evaluated MedGuard alerts, the alert acceptance rate, and the rate of LASA alerts between July 1, 2019, and June 31, 2021, from outpatient settings at an academic hospital. An expert pharmacist checked the suitability of the alerts, rate of acceptance, wrong-drug errors, and confusing drug pairs.

RESULTS

Over the two-year study period, 1,206,895 prescriptions were ordered and a total of 28,536 alerts were triggered (alert rate: 2.36 %). Of the 28,536 alerts presented to physicians, 13,947 (48.88 %) were accepted. A total of 8,014 prescriptions were changed/modified (28.08 %, 8,014/28,534) with the most common reasons being adding and/or deleting diseases (52.04 %, 4,171/8,014), adding and/or deleting drugs (21.89 %, 1,755/8,014) and others (35.48 %, 2,844/ 8,014). However, the rate of drug error interception was 1.64 % (470 intercepted errors out of 28,536 alerts), which equates to 16.4 intercepted errors per 1000 alerted orders.

CONCLUSION

This study shows that machine learning based CDSS, MedGuard, has an ability to improve patients' safety by triggering clinically valid alerts. This system can also help improve problem list documentation and intercept inappropriate drug errors and LASA drug errors, which can improve medication safety. Moreover, high acceptance of alert rates can help reduce clinician burnout and adverse events.

Acceptance and use of a clinical decision support system in musculoskeletal pain disorders - the SupportPrim project

BACKGROUND

We have developed a clinical decision support system (CDSS) based on methods from artificial intelligence to support physiotherapists and patients in the decision-making process of managing musculoskeletal (MSK) pain disorders in primary care. The CDSS finds the most similar successful patients from the past to give treatment recommendations for a new patient. Using previous similar patients with successful outcomes to advise treatment moves management of MSK pain patients from one-size fits all recommendations to more individually tailored treatment. This study aimed to summarise the development and explore the acceptance and use of the CDSS for MSK pain patients.

METHODS

This qualitative study was carried out in the Norwegian physiotherapy primary healthcare sector between October and November 2020, ahead of a randomised controlled trial. We included four physiotherapists and three of their patients, in total 12 patients, with musculoskeletal pain in the neck, shoulder, back, hip, knee or complex pain. We conducted semi-structured telephone interviews with all participants. The interviews were analysed using the Framework Method.

RESULTS

Overall, both the physiotherapists and patients found the system acceptable and usable. Important findings from the analysis of the interviews were that the CDSS was valued as a preparatory and exploratory tool, facilitating the therapeutic relationship. However, the physiotherapists used the system mainly to support their previous and current practice rather than involving patients to a greater extent in decisions and learning from previous successful patients.

CONCLUSIONS

The CDSS was acceptable and usable to both the patients and physiotherapists. However, the system appeared not to considerably influence the physiotherapists' clinical reasoning and choice of treatment based on information from most similar successful patients. This could be due to a smaller than optimal number of previous patients in the CDSS or insufficient clinical implementation. Extensive training of physiotherapists should not be underestimated to build understanding and trust in CDSSs.

Provider perspectives on a point-of-care tool to facilitate patient-centered contraceptive care among Latina/x patients in Baltimore, MD

Objectives

To explore clinician perspectives on the development, utility, and feasibility of a provider-facing point-of-care tool to assist in provision of patient-centered contraceptive care for Latina/x patients in Baltimore, MD.

Methods

We conducted 25 semi-structured qualitative interviews with a sample of clinicians who provide contraceptive care to Latina/x patients. An interview guide was developed based on prior research related to patient-centered care and extant point-of-care tools. Transcripts were independently coded by two study team members and analyzed using a directed content analysis approach.

Results

Four themes emerged from the data: (1) clinician perception of a need for a tool to facilitate patient-centered contraceptive care, (2) concern for tool burden and burnout, (3) desire for tool ease of use, and (4) a need for cultural awareness during tool development to avoid bias and typecasting.

Conclusions

A provider-facing, point-of-care tool to facilitate patient-centered contraceptive counseling was acceptable among providers, provided the tool is easy to use and promotes cultural awareness.

Innovation

In the current era of more limited reproductive choice across the U.S., the need and support for non-coercive, patient-centered contraceptive care is timely. A provider-facing, point-of-care tool can facilitate the provision of patient-centered care among clinicians proving contraceptive counseling to Latina/s.

Advancing acceptance: assessing acceptance of the ESR iGuide clinical decision support system for improved computed tomography test justification

Background

A clinical decision support system (CDSS), the European Society of Radiologists (ESR) iGuide, was developed to address gaps in the availability and use of effective imaging referral guidelines.

Aim

This study aimed to assess the appropriateness of computed tomography (CT) exams with and without ESR iGuide use, as well as the usability and acceptance of the physician systems.

Methods

A retrospective single-center study was conducted in which data from 278 consecutive CT tests referred by physicians were collected in the first phase (T1), and physicians used the ESR iGuide system for imaging referrals in the second phase (T2; n = 85). The appropriateness of imaging referrals in each phase was assessed by two experts, and physicians completed the System Usability Scale.

Results

The mean appropriateness level on a scale of 0-9 was 6.62 ± 2.69 at T1 and 7.88 ± 1.4 at T2. When using a binary variable (0-6 = non-appropriate; 7-9 = appropriate), 70.14% of cases were found appropriate at T1 and 96.47% at T2. Surgery physician specialty and post-intervention phase showed a higher likelihood of ordering an appropriate test (p = 0.0045 and p = 0.0003, respectively). However, the questionnaire results indicated low system trust and minimal clinical value, with all physicians indicating they would not recommend collegial use (100%).

Conclusion

The study suggests that ESR iGuide can effectively guide the selection of appropriate imaging tests. However, physicians showed low system trust and use, indicating a need for further understanding of CDSS acceptance properties. Maximizing CDSS potential could result in crucial decision-support compliance and promotion of appropriate imaging.

Multidisciplinary Development and Initial Validation of a Clinical Knowledge Base on Chronic Respiratory Diseases for mHealth Decision Support Systems

Most mobile health (mHealth) decision support systems currently available for chronic obstructive respiratory diseases (CORDs) are not supported by clinical evidence or lack clinical validation. The development of the knowledge base that will feed the clinical decision support system is a crucial step that involves the collection and systematization of clinical knowledge from relevant scientific sources and its representation in a human-understandable and computer-interpretable way. This work describes the development and initial validation of a clinical knowledge base that can be integrated into mHealth decision support systems developed for patients with CORDs. A multidisciplinary team of health care professionals with clinical experience in respiratory diseases, together with data science and IT professionals, defined a new framework that can be used in other evidence-based systems. The knowledge base development began with a thorough review of the relevant scientific sources (eg, disease guidelines) to identify the recommendations to be implemented in the decision support system based on a consensus process. Recommendations were selected according to predefined inclusion criteria: (1) applicable to individuals with CORDs or to prevent CORDs, (2) directed toward patient self-management, (3) targeting adults, and (4) within the scope of the knowledge domains and subdomains defined. Then, the selected recommendations were prioritized according to (1) a harmonized level of evidence (reconciled from different sources); (2) the scope of the source document (international was preferred); (3) the entity that issued the source document; (4) the operability of the recommendation; and (5) health care professionals' perceptions of the relevance, potential impact, and reach of the recommendation. A total of 358 recommendations were selected. Next, the variables required to trigger those recommendations were defined (n=116) and operationalized into logical rules using Boolean logical operators (n=405). Finally, the knowledge base was implemented in an intelligent individualized coaching component and pretested with an asthma use case. Initial validation of the knowledge base was conducted internally using data from a population-based observational study of individuals with or without asthma or rhinitis. External validation of the appropriateness of the recommendations with the highest priority level was conducted independently by 4 physicians. In addition, a strategy for knowledge base updates, including an easy-to-use rules editor, was defined. Using this process, based on consensus and iterative improvement, we developed and conducted preliminary validation of a clinical knowledge base for CORDs that translates disease guidelines into personalized patient recommendations. The knowledge base can be used as part of mHealth decision support systems. This process could be replicated in other clinical areas.

Medical practitioner perspectives on AI in emergency triage

Introduction

A proposed Diagnostic AI System for Robot-Assisted Triage ("DAISY") is under development to support Emergency Department ("ED") triage following increasing reports of overcrowding and shortage of staff in ED care experienced within National Health Service, England ("NHS") but also globally. DAISY aims to reduce ED patient wait times and medical practitioner overload. The objective of this study was to explore NHS health practitioners' perspectives and attitudes towards the future use of AI-supported technologies in ED triage.

Methods

Between July and August 2022 a qualitative-exploratory research study was conducted to collect and capture the perceptions and attitudes of nine NHS healthcare practitioners to better understand the challenges and benefits of a DAISY deployment. The study was based on a thematic analysis of semi-structured interviews. The study involved qualitative data analysis of the interviewees' responses. Audio-recordings were transcribed verbatim, and notes included into data documents. The transcripts were coded line-by-line, and data were organised into themes and sub-themes. Both inductive and deductive approaches to thematic analysis were used to analyse such data.

Results

Based on a qualitative analysis of coded interviews with the practitioners, responses were categorised into broad main thematic-types, namely: trust; current practice; social, legal, ethical, and cultural concerns; and empathetic practice. Sub-themes were identified for each main theme. Further quantitative analyses explored the vocabulary and sentiments of the participants when talking generally about NHS ED practices compared to discussing DAISY. Limitations include a small sample size and the requirement that research participants imagine a prototype AI-supported system still under development. The expectation is that such a system would work alongside the practitioner. Findings can be generalisable to other healthcare AI-supported systems and to other domains.

Discussion

This study highlights the benefits and challenges for an AI-supported triage healthcare solution. The study shows that most NHS ED practitioners interviewed were positive about such adoption. Benefits cited were a reduction in patient wait times in the ED, assistance in the streamlining of the triage process, support in calling for appropriate diagnostics and for further patient examination, and identification of those very unwell and requiring more immediate and urgent attention. Words used to describe the system were that DAISY is a "good idea", "help", helpful, "easier", "value", and "accurate". Our study demonstrates that trust in the system is a significant driver of use and a potential barrier to adoption. Participants emphasised social, legal, ethical, and cultural considerations and barriers to DAISY adoption and the importance of empathy and non-verbal cues in patient interactions. Findings demonstrate how DAISY might support and augment human medical performance in ED care, and provide an understanding of attitudinal barriers and considerations for the development and implementation of future triage AI-supported systems.

Are physicians ready for precision antibiotic prescribing? A qualitative analysis of the acceptance of artificial intelligence-enabled clinical decision support systems in India and Singapore

OBJECTIVES

Artificial intelligence (AI)-driven clinical decision support systems (CDSSs) can augment antibiotic decision-making capabilities, but physicians' hesitancy in adopting them may undermine their utility. We conducted a cross-country comparison of physician perceptions on the barriers and facilitators in accepting an AI-enabled CDSS for antibiotic prescribing.

METHODS

We conducted in-depth interviews with physicians from the National Centre for Infectious Diseases (NCID), Singapore, and Christian Medical College Vellore (CMCV), India, between April and December 2022. Our semi-structured in-depth interview guides were anchored on Venkatesh's UTAUT model. We used clinical vignettes to illustrate the application of AI in clinical decision support for antibiotic prescribing and explore medico-legal concerns.

RESULTS

Most NCID physicians felt that an AI-enabled CDSS could facilitate antibiotic prescribing, while most CMCV physicians were sceptical about the tool's utility. The hesitancy in adopting an AI-enabled CDSS stems from concerns about the lack of validated and successful examples, fear of losing autonomy and clinical skills, difficulty of use, and impediment in work efficiency. Physicians from both sites felt that a user-friendly interface, integration with workflow, transparency of output, a guiding medico-legal framework, and training and technical support would improve the uptake of an AI-enabled CDSS.

CONCLUSION

In conclusion, the acceptance of AI-enabled CDSSs depends on the physician's confidence with the tool's recommendations, perceived ease of use, familiarity with AI, the organisation's digital culture and support, and the presence of medico-legal governance of AI. Progressive implementation and continuous feedback are essential to allay scepticism around the utility of AI-enabled CDSSs.

[Factors influencing the implementation of AI-based decision support systems for antibiotic prescription in hospitals: a qualitative analysis from the perspective of health professionals]

BACKGROUND

Decision support systems based on artificial intelligence might optimize antibiotic prescribing in hospitals and prevent the development of antimicrobial resistance. The aim of this study was to identify impeding and facilitating factors for successful implementation from the perspective of health professionals.

METHODS

Problem-centered individual interviews were conducted with health professionals working in hospitals. Data evaluation was based on the structured qualitative content analysis according Kuckartz.

RESULTS

Attitudes of health professionals were presented along the Human-Organization -Technology-fit model. Technological and organizational themes were the most important factors for system implementation. Especially, compatibility with existing systems and user-friendliness were seen to play a major role in successful implementation. Additionally, the training of potential users and the technical equipment of the organization were considered essential. Finally, the importance of promoting technical skills of potential users in the long term and creating trust in the benefits of the system were highlighted.

CONCLUSION

The identified factors provide a basis for prioritizing and quantifying needs and attitudes in a next step. It becomes clear that, beside technological factors, attention to context-specific and user-related conditions are of fundamental importance to ensure successful implementation and system trust in the long term.

Insights on the Current State and Future Outlook of AI in Health Care: Expert Interview Study

BACKGROUND

Artificial intelligence (AI) is often promoted as a potential solution for many challenges health care systems face worldwide. However, its implementation in clinical practice lags behind its technological development.

OBJECTIVE

This study aims to gain insights into the current state and prospects of AI technology from the stakeholders most directly involved in its adoption in the health care sector whose perspectives have received limited attention in research to date.

METHODS

For this purpose, the perspectives of AI researchers and health care IT professionals in North America and Western Europe were collected and compared for profession-specific and regional differences. In this preregistered, mixed methods, cross-sectional study, 23 experts were interviewed using a semistructured guide. Data from the interviews were analyzed using deductive and inductive qualitative methods for the thematic analysis along with topic modeling to identify latent topics.

RESULTS

Through our thematic analysis, four major categories emerged: (1) the current state of AI systems in health care, (2) the criteria and requirements for implementing AI systems in health care, (3) the challenges in implementing AI systems in health care, and (4) the prospects of the technology. Experts discussed the capabilities and limitations of current AI systems in health care in addition to their prevalence and regional differences. Several criteria and requirements deemed necessary for the successful implementation of AI systems were identified, including the technology's performance and security, smooth system integration and human-AI interaction, costs, stakeholder involvement, and employee training. However, regulatory, logistical, and technical issues were identified as the most critical barriers to an effective technology implementation process. In the future, our experts predicted both various threats and many opportunities related to AI technology in the health care sector.

CONCLUSIONS

Our work provides new insights into the current state, criteria, challenges, and outlook for implementing AI technology in health care from the perspective of AI researchers and IT professionals in North America and Western Europe. For the full potential of AI-enabled technologies to be exploited and for them to contribute to solving current health care challenges, critical implementation criteria must be met, and all groups involved in the process must work together.

Physicians' Perspectives on AI in Clinical Decision Support Systems: Interview Study of the CURATE.AI Personalized Dose Optimization Platform

BACKGROUND

Physicians play a key role in integrating new clinical technology into care practices through user feedback and growth propositions to developers of the technology. As physicians are stakeholders involved through the technology iteration process, understanding their roles as users can provide nuanced insights into the workings of these technologies that are being explored. Therefore, understanding physicians' perceptions can be critical toward clinical validation, implementation, and downstream adoption. Given the increasing prevalence of clinical decision support systems (CDSSs), there remains a need to gain an in-depth understanding of physicians' perceptions and expectations toward their downstream implementation. This paper explores physicians' perceptions of integrating CURATE.AI, a novel artificial intelligence (AI)-based and clinical stage personalized dosing CDSSs, into clinical practice.

OBJECTIVE

This study aims to understand physicians' perspectives of integrating CURATE.AI for clinical work and to gather insights on considerations of the implementation of AI-based CDSS tools.

METHODS

A total of 12 participants completed semistructured interviews examining their knowledge, experience, attitudes, risks, and future course of the personalized combination therapy dosing platform, CURATE.AI. Interviews were audio recorded, transcribed verbatim, and coded manually. The data were thematically analyzed.

RESULTS

Overall, 3 broad themes and 9 subthemes were identified through thematic analysis. The themes covered considerations that physicians perceived as significant across various stages of new technology development, including trial, clinical implementation, and mass adoption.

CONCLUSIONS

The study laid out the various ways physicians interpreted an AI-based personalized dosing CDSS, CURATE.AI, for their clinical practice. The research pointed out that physicians' expectations during the different stages of technology exploration can be nuanced and layered with expectations of implementation that are relevant for technology developers and researchers.

A prospective observational concordance study to evaluate computational model-driven clinical practice guidelines for Type 2 diabetes mellitus

BACKGROUND

Clinical Practice Guidelines (CPGs) provide healthcare professionals with performance and decision-making support during the treatment of patients. Sometimes, however, they are poorly implemented. The IDE4ICDS platform was developed and validated with CPGs for type 2 diabetes mellitus (T2DM).

OBJECTIVE

The main objective of this paper is to present the results of the clinical validation of the IDE4ICDS platform in a real clinical environment at two health clinics in the Andalusian Public Health System (SSPA) in the southern Spanish region of Andalusia.

METHODS

National and international knowledge sources on T2DM were selected and reviewed and used to define a diabetes CPG model on the IDE4ICDS platform. Once the diabetes CPG was configured and deployed, it was validated. A total of 506 patients were identified as meeting the inclusion criteria, of whom 130 could be recruited and 89 attended the appointment.

RESULTS

A concordance analysis was performed with the kappa value. Overall agreement between the recommendations provided by the system and those recorded in each patient's EHR was good (0.61 - 0.80) with a total kappa index of 0.701, leading to the conclusion that the system provided appropriate recommendations for each patient and was therefore well-functioning.

CONCLUSIONS

A series of possible improvements were identified based on the limitations for the recovery of variables related to the quality of these recolected variables, the detection of duplicate recommendations based on different input variables for the same patient, and clinical usability, such as the capacity to generate reports based on the recommendations generated. Nevertheless, the project resulted in the IDE4ICDS platform: a Clinical Decision Support System (CDSS) capable of providing appropriate recommendations for improving the management and quality of patient care and optimizing health outcomes. The result of this validation is a safe and effective pathway for developing and adopting digital transformation at the regional scale of the use of biomedical knowledge in real healthcare.

Examining explainable clinical decision support systems with think aloud protocols

Machine learning tools are increasingly used to improve the quality of care and the soundness of a treatment plan. Explainable AI (XAI) helps users in understanding the inner mechanisms of opaque machine learning models and is a driver of trust and adoption. Explanation methods for black-box models exist, but there is a lack of user studies on the interpretability of the provided explanations. We used a Think Aloud Protocol (TAP) to explore oncologists' assessment of a lung cancer relapse prediction system with the aim of refining the purpose-built explanation model for better credibility and utility. Novel to this context, TAP is used as a neutral methodology to elicit experts' thought processes and judgements of the AI system, without explicit prompts. TAP aims to elicit the factors which influenced clinicians' perception of credibility and usefulness of the system. Ten oncologists took part in the study. We conducted a thematic analysis of their verbalized responses, generating five themes that help us to understand the context within which oncologists' may (or may not) integrate an explainable AI system into their working day.

Acceptance of a digital therapy recommender system for psoriasis

BACKGROUND

About 2% of the German population are affected by psoriasis. A growing number of cost-intensive systemic treatments are available. Surveys have shown high proportions of patients with moderate to severe psoriasis are not adequately treated despite a high disease burden. Digital therapy recommendation systems (TRS) may help implement guideline-based treatment. However, little is known about the acceptance of such clinical decision support systems (CDSSs). Therefore, the aim of the study was to access the acceptance of a prototypical TRS demonstrator.

METHODS

Three scenarios (potential test patients with psoriasis but different sociodemographic and clinical characteristics, previous treatments, desire to have children, and multiple comorbidities) were designed in the demonstrator. The TRS demonstrator and test patients were presented to a random sample of 76 dermatologists attending a national dermatology conference in a cross-sectional face-to-face survey with case vignettes. The dermatologist were asked to rate the demonstrator by system usability scale (SUS), whether they would use it for certain patients populations and barriers of usage. Reasons for potential usage of the TRS demonstrator were tested via a Poisson regression with robust standard errors.

RESULTS

Acceptance of the TRS was highest for patients eligible for systemic therapy (82%). 50% of participants accepted the system for patients with additional comorbidities and 43% for patients with special subtypes of psoriasis. Dermatologists in the outpatient sector or with many patients per week were less willing to use the TRS for patients with special psoriasis-subtypes. Dermatologists rated the demonstrator as acceptable with an mean SUS of 76.8. Participants whose SUS was 10 points above average were 27% more likely to use TRS for special psoriasis-subtypes. The main barrier in using the TRS was time demand (47.4%). Participants who perceived time as an obstacle were 22.3% less willing to use TRS with systemic therapy patients. 27.6% of physicians stated that they did not understand exactly how the recommendation was generated by the TRS, with no effect on the preparedness to use the system.

CONCLUSION

The considerably high acceptance and the preparedness to use the psoriasis CDSS suggests that a TRS appears to be implementable in routine healthcare and may improve clinical care. Main barrier is the additional time demand posed on dermatologists in a busy clinical setting. Therefore, it will be a major challenge to identify a limited set of variables that still allows a valid recommendation with precise prediction of the patient-individual benefits and harms.

A novel, evidence-based, comprehensive clinical decision support system improves outcomes for patients with traumatic rib fractures

BACKGROUND

Traumatic rib fractures are associated with high morbidity and mortality. Clinical decision support systems (CDSS) have been shown to improve adherence to evidence-based (EB) practice and improve clinical outcomes. The objective of this study was to investigate if a rib fracture CDSS reduced hospital length of stay (LOS), 90-day and 1-year mortality, unplanned ICU transfer, and the need for mechanical ventilation. The independent association of two process measures, an admission EB order set and a pain-inspiratory-cough score early warning system, with LOS were investigated.

METHODS

The CDSS was scaled across nine US trauma centers. Following multiple imputation, multivariable regression models were fit to evaluate the association of the CDSS on primary and secondary outcomes. As a sensitivity analysis, propensity score matching was also performed to confirm regression findings.

RESULTS

Overall, 3,279 patients met inclusion criteria. Rates of EB practices increased following implementation. On risk-adjusted analysis, in-hospital LOS preintervention versus postintervention was unchanged (incidence rate ratio [IRR], 1.06; 95% confidence interval [CI], 0.97-1.15, p = 0.2) but unplanned transfer to the ICU was reduced (odds ratio, 0.28; 95% CI, 0.09-0.84, p = 0.024), as was 1-year mortality (hazard ratio, 0.6; 95% CI, 0.4-0.89, p = 0.01). Provider utilization of the admission order bundle was 45.3%. Utilization was associated with significantly reduced LOS (IRR, 0.87; 95% CI, 0.77-0.98; p = 0.019). The early warning system triggered on 34.4% of patients; however, was not associated with a significant reduction in hospital LOS (IRR, 0.76; 95% CI, 0.55-1.06; p = 0.1).

CONCLUSION

A novel, user-centered, comprehensive CDSS improves adherence to EB practice and is associated with a significant reduction in unplanned ICU admissions and possibly mortality, but not hospital LOS.

LEVEL OF EVIDENCE

Therapeutic/Care Management; Level III.

Requirements analysis for an AI-based clinical decision support system for general practitioners: a user-centered design process

BACKGROUND

As the first point of contact for patients with health issues, general practitioners (GPs) are frequently confronted with patients presenting with non-specific symptoms of unclear origin. This can result in delayed, prolonged or false diagnoses. To accelerate and improve the diagnosis of diseases, clinical decision support systems would appear to be an appropriate tool. The objective of the project 'Smart physician portal for patients with unclear disease' (SATURN) is to employ a user-centered design process based on the requirements analysis presented in this paper to develop an artificial Intelligence (AI)-based diagnosis support system that specifically addresses the needs of German GPs.

METHODS

Requirements analysis for a GP-specific diagnosis support system was conducted in an iterative process with five GPs. First, interviews were conducted to analyze current workflows and the use of digital applications in cases of diagnostic uncertainty (as-is situation). Second, we focused on collecting and prioritizing tasks to be performed by an ideal smart physician portal (to-be situation) in a workshop. We then developed a task model with corresponding user requirements.

RESULTS

Numerous GP-specific user requirements were identified concerning the tasks and subtasks: performing data entry (open system, enter patient data), reviewing results (receiving and evaluating results), discussing results (with patients and colleagues), scheduling further diagnostic procedures, referring to specialists (select, contact, make appointments), and case closure. Suggested features particularly concerned the process of screening and assessing results: e.g., the system should focus more on atypical patterns of common diseases than on rare diseases only, display probabilities of differential diagnoses, ensure sources and results are transparent, and mark diagnoses that have already been ruled out. Moreover, establishing a means of using the platform to communicate with colleagues and transferring patient data directly from electronic patient records to the system was strongly recommended.

CONCLUSIONS

Essential user requirements to be considered in the development and design of a diagnosis system for primary care could be derived from the analysis. They form the basis for mockup-development and system engineering.

An Ontology-Based Approach to Improving Medication Appropriateness in Older Patients: Algorithm Development and Validation Study

Background: Inappropriate medication in older patients with multimorbidity results in a greater risk of adverse drug events. Clinical decision support systems (CDSSs) are intended to improve medication appropriateness. One approach to improving CDSSs is to use ontologies instead of relational databases. Previously, we developed OntoPharma-an ontology-based CDSS for reducing medication prescribing errors. Objective: The primary aim was to model a domain for improving medication appropriateness in older patients (chronic patient domain). The secondary aim was to implement the version of OntoPharma containing the chronic patient domain in a hospital setting. Methods: A 4-step process was proposed. The first step was defining the domain scope. The chronic patient domain focused on improving medication appropriateness in older patients. A group of experts selected the following three use cases: medication regimen complexity, anticholinergic and sedative drug burden, and the presence of triggers for identifying possible adverse events. The second step was domain model representation. The implementation was conducted by medical informatics specialists and clinical pharmacists using Protégé-OWL (Stanford Center for Biomedical Informatics Research). The third step was OntoPharma-driven alert module adaptation. We reused the existing framework based on SPARQL to query ontologies. The fourth step was implementing the version of OntoPharma containing the chronic patient domain in a hospital setting. Alerts generated from July to September 2022 were analyzed. Results: We proposed 6 new classes and 5 new properties, introducing the necessary changes in the ontologies previously created. An alert is shown if the Medication Regimen Complexity Index is ≥40, if the Drug Burden Index is ≥1, or if there is a trigger based on an abnormal laboratory value. A total of 364 alerts were generated for 107 patients; 154 (42.3%) alerts were accepted. Conclusions: We proposed an ontology-based approach to provide support for improving medication appropriateness in older patients with multimorbidity in a scalable, sustainable, and reusable way. The chronic patient domain was built based on our previous research, reusing the existing framework. OntoPharma has been implemented in clinical practice and generates alerts, considering the following use cases: medication regimen complexity, anticholinergic and sedative drug burden, and the presence of triggers for identifying possible adverse events.

Clinical Decision Support Systems for Brain Tumour Diagnosis and Prognosis: A Systematic Review

CDSSs are being continuously developed and integrated into routine clinical practice as they assist clinicians and radiologists in dealing with an enormous amount of medical data, reduce clinical errors, and improve diagnostic capabilities. They assist detection, classification, and grading of brain tumours as well as alert physicians of treatment change plans. The aim of this systematic review is to identify various CDSSs that are used in brain tumour diagnosis and prognosis and rely on data captured by any imaging modality. Based on the 2020 preferred reporting items for systematic reviews and meta-analyses (PRISMA) protocol, the literature search was conducted in PubMed and Engineering Village Compendex databases. Different types of CDSSs identified through this review include Curiam BT, FASMA, MIROR, HealthAgents, and INTERPRET, among others. This review also examines various CDSS tool types, system features, techniques, accuracy, and outcomes, to provide the latest evidence available in the field of neuro-oncology. An overview of such CDSSs used to support clinical decision-making in the management and treatment of brain tumours, along with their benefits, challenges, and future perspectives has been provided. Although a CDSS improves diagnostic capabilities and healthcare delivery, there is lack of specific evidence to support these claims. The absence of empirical data slows down both user acceptance and evaluation of the actual impact of CDSS on brain tumour management. Instead of emphasizing the advantages of implementing CDSS, it is important to address its potential drawbacks and ethical implications. By doing so, it can promote the responsible use of CDSS and facilitate its faster adoption in clinical settings.

Assessing the decision quality of artificial intelligence and oncologists of different experience in different regions in breast cancer treatment

Background

AI-based clinical decision support system (CDSS) has important prospects in overcoming the current informational challenges that cancer diseases faced, promoting the homogeneous development of standardized treatment among different geographical regions, and reforming the medical model. However, there are still a lack of relevant indicators to comprehensively assess its decision-making quality and clinical impact, which greatly limits the development of its clinical research and clinical application. This study aims to develop and application an assessment system that can comprehensively assess the decision-making quality and clinical impacts of physicians and CDSS.

Methods

Enrolled adjuvant treatment decision stage early breast cancer cases were randomly assigned to different decision-making physician panels (each panel consisted of three different seniority physicians in different grades hospitals), each physician made an independent "Initial Decision" and then reviewed the CDSS report online and made a "Final Decision". In addition, the CDSS and guideline expert groups independently review all cases and generate "CDSS Recommendations" and "Guideline Recommendations" respectively. Based on the design framework, a multi-level multi-indicator system including "Decision Concordance", "Calibrated Concordance", " Decision Concordance with High-level Physician", "Consensus Rate", "Decision Stability", "Guideline Conformity", and "Calibrated Conformity" were constructed.

Results

531 cases containing 2124 decision points were enrolled; 27 different seniority physicians from 10 different grades hospitals have generated 6372 decision opinions before and after referring to the "CDSS Recommendations" report respectively. Overall, the calibrated decision concordance was significantly higher for CDSS and provincial-senior physicians (80.9%) than other physicians. At the same time, CDSS has a higher " decision concordance with high-level physician" (76.3%-91.5%) than all physicians. The CDSS had significantly higher guideline conformity than all decision-making physicians and less internal variation, with an overall guideline conformity variance of 17.5% (97.5% vs. 80.0%), a standard deviation variance of 6.6% (1.3% vs. 7.9%), and a mean difference variance of 7.8% (1.5% vs. 9.3%). In addition, provincial-middle seniority physicians had the highest decision stability (54.5%). The overall consensus rate among physicians was 64.2%.

Conclusions

There are significant internal variation in the standardization treatment level of different seniority physicians in different geographical regions in the adjuvant treatment of early breast cancer. CDSS has a higher standardization treatment level than all physicians and has the potential to provide immediate decision support to physicians and have a positive impact on standardizing physicians' treatment behaviors.

Clinician Perspectives on Clinical Decision Support for Familial Hypercholesterolemia

Familial Hypercholesterolemia (FH) is underdiagnosed in the United States. Clinical decision support (CDS) could increase FH detection once implemented in clinical workflows. We deployed CDS for FH at an academic medical center and sought clinician insights using an implementation survey. In November 2020, the FH CDS was deployed in the electronic health record at all Mayo Clinic sites in two formats: a best practice advisory (BPA) and an in-basket alert. Over three months, 104 clinicians participated in the survey (response rate 11.1%). Most clinicians (81%) agreed that CDS implementation was a good option for identifying FH patients; 78% recognized the importance of implementing the tool in practice, and 72% agreed it would improve early diagnosis of FH. In comparing the two alert formats, clinicians found the in-basket alert more acceptable (p = 0.036) and more feasible (p = 0.042) than the BPA. Overall, clinicians favored implementing the FH CDS in clinical practice and provided feedback that led to iterative refinement of the tool. Such a tool can potentially increase FH detection and optimize patient management.

The Application of Knowledge-Based Clinical Decision Support Systems to Enhance Adherence to Evidence-Based Medicine in Chronic Disease

Among the technology-based solutions, clinical decision support systems (CDSSs) have the ability to keep up with clinicians with the latest evidence in a smart way. Hence, the main objective of our study was to investigate the applicability and characteristics of CDSSs regarding chronic disease. The Web of Science, Scopus, OVID, and PubMed databases were searched using keywords from January 2000 to February 2023. The review was completed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist. Then, an analysis was done to determine the characteristics and applicability of CDSSs. The quality of the appraisal was assessed using the Mixed Methods Appraisal Tool checklist (MMAT). A systematic database search yielded 206 citations. Eventually, 38 articles from sixteen countries met the inclusion criteria and were accepted for final analysis. The main approaches of all studies can be classified into adherence to evidence-based medicine (84.2%), early and accurate diagnosis (81.6%), identifying high-risk patients (50%), preventing medical errors (47.4%), providing up-to-date information to healthcare providers (36.8%), providing patient care remotely (21.1%), and standardizing care (71.1%). The most common features among the knowledge-based CDSSs included providing guidance and advice for physicians (92.11%), generating patient-specific recommendations (84.21%), integrating into electronic medical records (60.53%), and using alerts or reminders (60.53%). Among thirteen different methods to translate the knowledge of evidence into machine-interpretable knowledge, 34.21% of studies utilized the rule-based logic technique while 26.32% of studies used rule-based decision tree modeling. For CDSS development and translating knowledge, diverse methods and techniques were applied. Therefore, the development of a standard framework for the development of knowledge-based decision support systems should be considered by informaticians.

Impact of an automated large vessel occlusion detection tool on clinical workflow and patient outcomes

Purpose

Automated large vessel occlusion (LVO) tools allow for prompt identification of positive LVO cases, but little is known about their role in acute stroke triage when implemented in a real-world setting. The purpose of this study was to evaluate the automated LVO detection tool's impact on acute stroke workflow and clinical outcomes.

Materials and methods

Consecutive patients with a computed tomography angiography (CTA) presenting with suspected acute ischemic stroke were compared before and after the implementation of an AI tool, RAPID LVO (RAPID 4.9, iSchemaView, Menlo Park, CA). Radiology CTA report turnaround times (TAT), door-to-treatment times, and the NIH stroke scale (NIHSS) after treatment were evaluated.

Results

A total of 439 cases in the pre-AI group and 321 cases in the post-AI group were included, with 62 (14.12%) and 43 (13.40%) cases, respectively, receiving acute therapies. The AI tool demonstrated a sensitivity of 0.96, a specificity of 0.85, a negative predictive value of 0.99, and a positive predictive value of 0.53. Radiology CTA report TAT significantly improved post-AI (mean 30.58 min for pre-AI vs. 22 min for post-AI, p < 0.0005), notably at the resident level (p < 0.0003) but not at higher levels of expertise. There were no differences in door-to-treatment times, but the NIHSS at discharge was improved for the pre-AI group adjusted for confounders (parameter estimate = 3.97, p < 0.01).

Conclusion

Implementation of an automated LVO detection tool improved radiology TAT but did not translate to improved stroke metrics and outcomes in a real-world setting.

Identifying Barriers to Post-Acute Care Referral and Characterizing Negative Patient Preferences Among Hospitalized Older Adults Using Natural Language Processing

Our objective was to detect common barriers to post-acute care (B2PAC) among hospitalized older adults using natural language processing (NLP) of clinical notes from patients discharged home when a clinical decision support system recommended post-acute care. We annotated B2PAC sentences from discharge planning notes and developed an NLP classifier to identify the highest-value B2PAC class (negative patient preferences). Thirteen machine learning models were compared with Amazon's AutoGluon deep learning model. The study included 594 acute care notes from 100 patient encounters (1156 sentences contained 11 B2PAC) in a large academic health system. The most frequent and modifiable B2PAC class was negative patient preferences (18.3%). The best supervised model was Extreme Gradient Boosting (F1: 0.859), but the deep learning model performed better (F1: 0.916). Alerting clinicians of negative patient preferences early in the hospitalization can prompt interventions such as patient education to ensure patients receive the right level of care and avoid negative outcomes.

Healthcare Professionals’ Perceptions, Barriers, and Facilitators towards Adopting Computerised Clinical Decision Support Systems in Antimicrobial Stewardship in Jordanian Hospitals

Understanding healthcare professionals’ perceptions towards a computerised decision support system (CDSS) may provide a platform for the determinants of the successful adoption and implementation of CDSS. This cross-sectional study examined healthcare professionals’ perceptions, barriers, and facilitators to adopting a CDSS for antibiotic prescribing in Jordanian hospitals. This study was conducted among healthcare professionals in Jordan’s two tertiary and teaching hospitals over four weeks (June–July 2021). Data were collected in a paper-based format from senior and junior prescribers and non-prescribers (n = 254) who agreed to complete a questionnaire. The majority (n = 184, 72.4%) were aware that electronic prescribing and electronic health record systems could be used specifically to facilitate antibiotic use and prescribing. The essential facilitator made CDSS available in a portable format (n = 224, 88.2%). While insufficient training to use CDSS was the most significant barrier (n = 175, 68.9%). The female providers showed significantly lower awareness (p = 0.006), and the nurses showed significantly higher awareness (p = 0.041) about using electronic prescribing and electronic health record systems. This study examined healthcare professionals’ perceptions of adopting CDSS in antimicrobial stewardship (AMS) and shed light on the perceived barriers and facilitators to adopting CDSS in AMS, reducing antibiotic resistance, and improving patient safety. Furthermore, results would provide a framework for other hospital settings concerned with implementing CDSS in AMS and inform policy decision-makers to react by implementing the CDSS system in Jordan and globally. Future studies should concentrate on establishing policies and guidelines and a framework to examine the adoption of the CDSS for AMS.

Monitoring the Implementation of Tobacco Cessation Support Tools: Using Novel Electronic Health Record Activity Metrics

BACKGROUND

Clinical decision support (CDS) tools in electronic health records (EHRs) are often used as core strategies to support quality improvement programs in the clinical setting. Monitoring the impact (intended and unintended) of these tools is crucial for program evaluation and adaptation. Existing approaches for monitoring typically rely on health care providers' self-reports or direct observation of clinical workflows, which require substantial data collection efforts and are prone to reporting bias.

OBJECTIVE

This study aims to develop a novel monitoring method leveraging EHR activity data and demonstrate its use in monitoring the CDS tools implemented by a tobacco cessation program sponsored by the National Cancer Institute's Cancer Center Cessation Initiative (C3I).

METHODS

We developed EHR-based metrics to monitor the implementation of two CDS tools: (1) a screening alert reminding clinic staff to complete the smoking assessment and (2) a support alert prompting health care providers to discuss support and treatment options, including referral to a cessation clinic. Using EHR activity data, we measured the completion (encounter-level alert completion rate) and burden (the number of times an alert was fired before completion and time spent handling the alert) of the CDS tools. We report metrics tracked for 12 months post implementation, comparing 7 cancer clinics (2 clinics implemented the screening alert and 5 implemented both alerts) within a C3I center, and identify areas to improve alert design and adoption.

RESULTS

The screening alert fired in 5121 encounters during the 12 months post implementation. The encounter-level alert completion rate (clinic staff acknowledged completion of screening in EHR: 0.55; clinic staff completed EHR documentation of screening results: 0.32) remained stable over time but varied considerably across clinics. The support alert fired in 1074 encounters during the 12 months. Providers acted upon (ie, not postponed) the support alert in 87.3% (n=938) of encounters, identified a patient ready to quit in 12% (n=129) of encounters, and ordered a referral to the cessation clinic in 2% (n=22) of encounters. With respect to alert burden, on average, both alerts fired over 2 times (screening alert: 2.7; support alert: 2.1) before completion; time spent postponing the screening alert was similar to completing (52 vs 53 seconds) the alert, and time spent postponing the support alert was more than completing (67 vs 50 seconds) the alert per encounter. These findings inform four areas where the alert design and use can be improved: (1) improving alert adoption and completion through local adaptation, (2) improving support alert efficacy by additional strategies including training in provider-patient communication, (3) improving the accuracy of tracking for alert completion, and (4) balancing alert efficacy with the burden.

CONCLUSIONS

EHR activity metrics were able to monitor the success and burden of tobacco cessation alerts, allowing for a more nuanced understanding of potential trade-offs associated with alert implementation. These metrics can be used to guide implementation adaptation and are scalable across diverse settings.

Implementation of indication-based antibiotic order sentences improves antibiotic use in emergency departments

INTRODUCTION

Prior data have suggested that suboptimal antibiotic prescribing in the emergency department (ED) is common for uncomplicated lower respiratory tract infections (LRTI), urinary tract infections (UTI), and acute bacterial skin and skin structure infections (ABSSSI). The objective of this study was to measure the effect of indication-based antibiotic order sentences (AOS) on optimal antibiotic prescribing in the ED.

METHODS

This was an IRB-approved quasi-experiment of adults prescribed antibiotics in EDs for uncomplicated LRTI, UTI, or ABSSSI from January to June 2019 (pre-implementation) and September to December 2021 (post-implementation). AOS implementation occurred in July 2021. AOS are lean process, electronic discharge prescriptions retrievable by name or indication within the discharge order field. The primary outcome was optimal prescribing, defined as correct antibiotic selection, dose, and duration per local and national guidelines. Descriptive and bivariate statistics were performed; multivariable logistic regression was used to determine variables associated with optimal prescribing.

RESULTS

A total of 294 patients were included: 147 pre-group and 147 post-group. Overall optimal prescribing improved from 12 (8%) to 34 (23%) (P < 0.001). Individual components of optimal prescribing were optimal selection at 90 (61%) vs 117 (80%) (P < 0.001), optimal dose at 99 (67%) vs 115 (78%) (P = 0.036), and optimal duration at 38 (26%) vs 50 (34%) (P = 0.13) for pre- and post-group, respectively. AOS was independently associated with optimal prescribing after multivariable logistic regression analysis (adjOR, 3.6; 95%CI,1.7-7.2). A post-hoc analysis showed low uptake of AOS by ED prescribers.

CONCLUSIONS

AOS are an efficient and promising strategy to enhance antimicrobial stewardship in the ED.

User-centred design of a clinical decision support system for palliative care: Insights from healthcare professionals

Objective

Although clinical decision support systems (CDSS) have many benefits for clinical practice, they also have several barriers to their acceptance by professionals. Our objective in this study was to design and validate The Aleph palliative care (PC) CDSS through a user-centred method, considering the predictions of the artificial intelligence (AI) core, usability and user experience (UX).

Methods

We performed two rounds of individual evaluation sessions with potential users. Each session included a model evaluation, a task test and a usability and UX assessment.

Results

The machine learning (ML) predictive models outperformed the participants in the three predictive tasks. System Usability Scale (SUS) reported 62.7  ±  14.1 and 65  ±  26.2 on a 100-point rating scale for both rounds, respectively, while User Experience Questionnaire - Short Version (UEQ-S) scores were 1.42 and 1.5 on the -3 to 3 scale.

Conclusions

The think-aloud method and including the UX dimension helped us to identify most of the workflow implementation issues. The system has good UX hedonic qualities; participants were interested in the tool and responded positively to it. Performance regarding usability was modest but acceptable.

Invisible clinical labor driving the successful integration of AI in healthcare

Artificial Intelligence and Machine Learning (AI/ML) tools are changing the landscape of healthcare decision-making. Vast amounts of data can lead to efficient triage and diagnosis of patients with the assistance of ML methodologies. However, more research has focused on the technological challenges of developing AI, rather than the system integration. As a result, clinical teams' role in developing and deploying these tools has been overlooked. We look to three case studies from our research to describe the often invisible work that clinical teams do in driving the successful integration of clinical AI tools. Namely, clinical teams support data labeling, identifying algorithmic errors and accounting for workflow exceptions, translating algorithmic output to clinical next steps in care, and developing team awareness of how the tool is used once deployed. We call for detailed and extensive documentation strategies (of clinical labor, workflows, and team structures) to ensure this labor is valued and to promote sharing of sociotechnical implementation strategies.

Physician Compliance with Computerized Clinical Decision Support System is a Complete Intermediate Factor in the Anemia Management of Patients with End-Stage Kidney Disease on Hemodialysis: A Retrospective Electronic Health Record Observational Study (Preprint)

BACKGROUND

Previous studies on clinical decision support systems (CDSSs) for the management of renal anemia in patients with end-stage kidney disease undergoing hemodialysis have previously focused solely on the effects of the CDSS. However, the role of physician compliance in the efficacy of the CDSS remains ill-defined.

OBJECTIVE

We aimed to investigate whether physician compliance was an intermediate variable between the CDSS and the management outcomes of renal anemia.

METHODS

We extracted the electronic health records of patients with end-stage kidney disease on hemodialysis at the Far Eastern Memorial Hospital Hemodialysis Center (FEMHHC) from 2016 to 2020. FEMHHC implemented a rule-based CDSS for the management of renal anemia in 2019. We compared the clinical outcomes of renal anemia between the pre- and post-CDSS periods using random intercept models. Hemoglobin levels of 10 to 12 g/dL were defined as the on-target range. Physician compliance was defined as the concordance of adjustments of the erythropoietin-stimulating agent (ESA) between the CDSS recommendations and the actual physician prescriptions.

RESULTS

We included 717 eligible patients on hemodialysis (mean age 62.9, SD 11.6 years; male n=430, 59.9%) with a total of 36,091 hemoglobin measurements (average hemoglobin and on-target rate were 11.1, SD 1.4, g/dL and 59.9%, respectively). The on-target rate decreased from 61.3% (pre-CDSS) to 56.2% (post-CDSS) owing to a high hemoglobin percentage of >12 g/dL (pre: 21.5%; post: 29%). The failure rate (hemoglobin <10 g/dL) decreased from 17.2% (pre-CDSS) to 14.8% (post-CDSS). The average weekly ESA use of 5848 (SD 4211) units per week did not differ between phases. The overall concordance between CDSS recommendations and physician prescriptions was 62.3%. The CDSS concordance increased from 56.2% to 78.6%. In the adjusted random intercept model, the post-CDSS phase showed increased hemoglobin by 0.17 (95% CI 0.14-0.21) g/dL, weekly ESA by 264 (95% CI 158-371) units per week, and 3.4-fold (95% CI 3.1-3.6) increased concordance rate. However, the on-target rate (29%; odds ratio 0.71, 95% CI 0.66-0.75) and failure rate (16%; odds ratio 0.84, 95% CI 0.76-0.92) were reduced. After additional adjustments for concordance in the full models, increased hemoglobin and decreased on-target rate tended toward attenuation (from 0.17 to 0.13 g/dL and 0.71 to 0.73 g/dL, respectively). Increased ESA and decreased failure rate were completely mediated by physician compliance (from 264 to 50 units and 0.84 to 0.97, respectively).

CONCLUSIONS

Our results confirmed that physician compliance was a complete intermediate factor accounting for the efficacy of the CDSS. The CDSS reduced failure rates of anemia management through physician compliance. Our study highlights the importance of optimizing physician compliance in the design and implementation of CDSSs to improve patient outcomes.

Meta-analysis of pharmacogenetic clinical decision support systems for the treatment of major depressive disorder

The study aimed to conduct a meta-analysis of studies comparing pharmacogenetically guided dosing of antidepressants with empiric standard of care. Publications referring to genotype-guided antidepressant therapy were identified via PubMed, Google Scholar, Scopus, Web of Science, Embase, and Cochrane databases from the inception of the databases to 2021. In addition, bibliographies of all articles were manually searched for additional references not identified in primary searches. Studies comparing clinical outcomes between two groups of patients who received antidepressant treatment were included in meta-analysis. Analysis of the data revealed statistically significant differences between the experimental group receiving pharmacogenetically guided dosing and the empirically treated controls. Specifically, genotype-guided treatment significantly improved response and remission of patients after both eight and twelve weeks of therapy, whereas no effect on the development of adverse drug reactions was observed. This meta-analysis indicates that the use of preemptive genotyping to guide dosing of antidepressants might increase treatment efficacy.

Machine learning and artificial intelligence: applications in healthcare epidemiology

Artificial intelligence (AI) refers to the performance of tasks by machines ordinarily associated with human intelligence. Machine learning (ML) is a subtype of AI; it refers to the ability of computers to draw conclusions (ie, learn) from data without being directly programmed. ML builds from traditional statistical methods and has drawn significant interest in healthcare epidemiology due to its potential for improving disease prediction and patient care. This review provides an overview of ML in healthcare epidemiology and practical examples of ML tools used to support healthcare decision making at 4 stages of hospital-based care: triage, diagnosis, treatment, and discharge. Examples include model-building efforts to assist emergency department triage, predicting time before septic shock onset, detecting community-acquired pneumonia, and classifying COVID-19 disposition risk level. Increasing availability and quality of electronic health record (EHR) data as well as computing power provides opportunities for ML to increase patient safety, improve the efficiency of clinical management, and reduce healthcare costs.

OntoPharma: ontology based clinical decision support system to reduce medication prescribing errors

Background

Clinical decision support systems (CDSS) have been shown to reduce medication errors. However, they are underused because of different challenges. One approach to improve CDSS is to use ontologies instead of relational databases. The primary aim was to design and develop OntoPharma, an ontology based CDSS to reduce medication prescribing errors. Secondary aim was to implement OntoPharma in a hospital setting.

Methods

A four-step process was proposed. (1) Defining the ontology domain. The ontology scope was the medication domain. An advisory board selected four use cases: maximum dosage alert, drug-drug interaction checker, renal failure adjustment, and drug allergy checker. (2) Implementing the ontology in a formal representation. The implementation was conducted by Medical Informatics specialists and Clinical Pharmacists using Protégé-OWL. (3) Developing an ontology-driven alert module. Computerised Physician Order Entry (CPOE) integration was performed through a REST API. SPARQL was used to query ontologies. (4) Implementing OntoPharma in a hospital setting. Alerts generated between July 2020/ November 2021 were analysed.

Results

The three ontologies developed included 34,938 classes, 16,672 individuals and 82 properties. The domains addressed by ontologies were identification data of medicinal products, appropriateness drug data, and local concepts from CPOE. When a medication prescribing error is identified an alert is shown. OntoPharma generated 823 alerts in 1046 patients. 401 (48.7%) of them were accepted.

Conclusions

OntoPharma is an ontology based CDSS implemented in clinical practice which generates alerts when a prescribing medication error is identified. To gain user acceptance OntoPharma has been designed and developed by a multidisciplinary team. Compared to CDSS based on relational databases, OntoPharma represents medication knowledge in a more intuitive, extensible and maintainable manner.

Enabling Fairness in Healthcare Through Machine Learning

The use of machine learning systems for decision-support in healthcare may exacerbate health inequalities. However, recent work suggests that algorithms trained on sufficiently diverse datasets could in principle combat health inequalities. One concern about these algorithms is that their performance for patients in traditionally disadvantaged groups exceeds their performance for patients in traditionally advantaged groups. This renders the algorithmic decisions unfair relative to the standard fairness metrics in machine learning. In this paper, we defend the permissible use of affirmative algorithms; that is, algorithms trained on diverse datasets that perform better for traditionally disadvantaged groups. Whilst such algorithmic decisions may be unfair, the fairness of algorithmic decisions is not the appropriate locus of moral evaluation. What matters is the fairness of final decisions, such as diagnoses, resulting from collaboration between clinicians and algorithms. We argue that affirmative algorithms can permissibly be deployed provided the resultant final decisions are fair.

Combined Expert and User-Driven Usability Assessment of Trauma Decision Support Systems Improves User-Centered Design

BACKGROUND

Trauma clinical decision support systems improve adherence with evidence-based practice but suffer from poor usability and the lack of a user-centered design. The objective of this study was to compare the effectiveness of user and expert-driven usability testing methods to detect usability issues in a rib fracture clinical decision support system and identify guiding principles for trauma clinical decision support systems.

METHODS

A user-driven and expert-driven usability investigation was conducted using a clinical decision support system developed for patients with rib fractures. The user-driven usability evaluation was as follows: 10 clinicians were selected for simulation-based usability testing using snowball sampling, and each clinician completed 3 simulations using a video-conferencing platform. End-users participated in a novel team-based approach that simulated realistic clinical workflows. The expert-driven heuristic evaluation was as follows: 2 usability experts conducted a heuristic evaluation of the clinical decision support system using 10 common usability heuristics. Usability issues were identified, cataloged, and ranked for severity using a 4-level ordinal scale. Thematic analysis was utilized to categorize the identified usability issues.

RESULTS

Seventy-nine usability issues were identified; 63% were identified by experts and 48% by end-users. Notably, 58% of severe usability issues were identified by experts alone. Only 11% of issues were identified by both methods. Five themes were identified that could guide the design of clinical decision support systems-transparency, functionality and integration into workflow, automated and noninterruptive, flexibility, and layout and appearance. Themes were preferentially identified by different methods.

CONCLUSION

We found that a dual-method usability evaluation involving usability experts and end-users drastically improved detection of usability issues over single-method alone. We identified 5 themes to guide trauma clinical decision support system design. Performing usability testing via a remote video-conferencing platform facilitated multi-site involvement despite a global pandemic.

Acceptance of clinical decision support system to prevent venous thromboembolism among nurses: an extension of the UTAUT model

Background

Venous thromboembolism has been a major public health problem and caused a heavy disease burden. Venous thromboembolism clinical decision support system was proved to have a positive influence on the prevention and management of venous thromboembolism. As the direct users, nurses' acceptance of this system is of great importance to support the successful implementation of it. However, there are few relevant studies to investigate nurses' acceptance and the associated factors are still unclear.

Objective

To investigate the determinant factors of nurses' acceptance of venous thromboembolism clinical decision support system with the modified Unified Theory of Acceptance and Use of Technology.

Methods

We designed a questionnaire based on the modified Unified Theory of Acceptance and Use of Technology and then a cross-sectional survey was conducted among nurses in a tertiary hospital in Nanjing, China. Statistically, a Structural Equation Modeling -Partial Least Squares path modeling approach was applied to examine the research model.

Results

A total of 1100 valid questionnaires were recycled. The modified model explained 74.7%, 83.0% and 86% of the variance in user satisfaction, behavioral intention and user behavior, respectively. The results showed that performance expectancy (β = 0.254, p = 0.000), social influence (β = 0.136, p = 0.047), facilitating conditions (β = 0.245, p = 0.000), self-efficacy (β = 0.121, p = 0.048) and user satisfaction (β = 0.193, p = 0.001) all had significant effects on nurses' intention. Although effort expectancy (β = 0.010, p = 0.785) did not have a direct effect on nurses' intention, it could indirectly influence nurses' intention with user satisfaction as the mediator (β = 0.296, p = 0.000). User behavior was significantly predicted by facilitating conditions (β = 0.298, p = 0.000) and user intention (β = 0.654, p = 0.001).

Conclusion

The research enhances our understanding of the determinants of nurses' acceptance of venous thromboembolism clinical decision support system. Among these factors, performance expectancy was considered as the top priority. It highlights the importance of optimizing system performance to fit the users' needs. Generally, the findings in our research provide clinical technology designers and administrators with valuable information to better meet users' requirements and promote the implementation of venous thromboembolism clinical decision support system.

The Interplay of Work, Digital Health Usage, and the Perceived Effects of Digitalization on Physicians' Work: Network Analysis Approach

Background

In health care, the benefits of digitalization need to outweigh the risks, but there is limited knowledge about the factors affecting this balance in the work environment of physicians. To achieve the benefits of digitalization, a more comprehensive understanding of this complex phenomenon related to the digitalization of physicians’ work is needed.

Objective

The aim of this study was to examine physicians’ perceptions of the effects of health care digitalization on their work and to analyze how these perceptions are associated with multiple factors related to work and digital health usage.

Methods

A representative sample of 4630 (response rate 24.46%) Finnish physicians (2960/4617, 64.11% women) was used. Statements measuring the perceived effects of digitalization on work included the patients’ active role, preventive work, interprofessional cooperation, decision support, access to patient information, and faster consultations. Network analysis of the perceived effects of digitalization and factors related to work and digital health usage was conducted using mixed graphical modeling. Adjusted and standardized regression coefficients are denoted by b. Centrality statistics were examined to evaluate the relative influence of each variable in terms of node strength.

Results

Nearly half of physicians considered that digitalization has promoted an active role for patients in their own care (2104/4537, 46.37%) and easier access to patient information (1986/4551, 43.64%), but only 1 in 10 (445/4529, 9.82%) felt that the impact has been positive on consultation times with patients. Almost half of the respondents estimated that digitalization has neither increased nor decreased the possibilities for preventive work (2036/4506, 45.18%) and supportiveness of clinical decision support systems (1941/4458, 43.54%). When all variables were integrated into the network, the most influential variables were purpose of using health information systems, employment sector, and specialization status. However, the grade given to the electronic health record (EHR) system that was primarily used had the strongest direct links to faster consultations (b=0.32) and facilitated access to patient information (b=0.28). At least 6 months of use of the main EHR was associated with facilitated access to patient information (b=0.18).

Conclusions

The results highlight the complex interdependence of multiple factors associated with the perceived effects of digitalization on physicians’ work. It seems that a high-quality EHR system is critical for promoting smooth clinical practice. In addition, work-related factors may influence other factors that affect digital health success. These factors should be considered when developing and implementing new digital health technologies or services for physicians’ work. The adoption of digital health is not just a technological project but a project that changes existing work practices.

Clinical Decision Support in the PICU: Implications for Design and Evaluation

OBJECTIVES

To assess the current landscape of clinical decision support (CDS) tools in PICUs in order to identify priority areas of focus in this field.

DESIGN

International, quantitative, cross-sectional survey.

SETTING

Role-specific, web-based survey administered in November and December 2020.

SUBJECTS

Medical directors, bedside nurses, attending physicians, and residents/advanced practice providers at Pediatric Acute Lung Injury and Sepsis Network-affiliated PICUs.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The survey was completed by 109 respondents from 45 institutions, primarily attending physicians from university-affiliated PICUs in the United States. The most commonly used CDS tools were people-based resources (93% used always or most of the time) and laboratory result highlighting (86%), with order sets, order-based alerts, and other electronic CDS tools also used frequently. The most important goal providers endorsed for CDS tools were a proven impact on patient safety and an evidence base for their use. Negative perceptions of CDS included concerns about diminished critical thinking and the burden of intrusive processes on providers. Routine assessment of existing CDS was rare, with infrequent reported use of observation to assess CDS impact on workflows or measures of individual alert burden.

CONCLUSIONS

Although providers share some consensus over CDS utility, we identified specific priority areas of research focus. Consensus across practitioners exists around the importance of evidence-based CDS tools having a proven impact on patient safety. Despite broad presence of CDS tools in PICUs, practitioners continue to view them as intrusive and with concern for diminished critical thinking. Deimplementing ineffective CDS may mitigate this burden, though postimplementation evaluation of CDS is rare.

Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing

Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments at early time points, which is critical for improving sepsis outcomes. In view of the increasing use of such systems, better understanding of how they are adopted and used by healthcare providers is needed. Here, we analyzed provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System), which was deployed at five hospitals over a 2-year period. Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. Adjusting for patient presentation and severity, patients with sepsis whose alert was confirmed by a provider within 3 h had a 1.85-h (95% CI 1.66-2.00) reduction in median time to first antibiotic order compared to patients with sepsis whose alert was either dismissed, confirmed more than 3 h after the alert or never addressed in the system. Finally, we found that emergency department providers and providers who had previous interactions with an alert were more likely to interact with alerts, as well as to confirm alerts on retrospectively identified patients with sepsis. Beyond efforts to improve the performance of early warning systems, efforts to improve adoption are essential to their clinical impact and should focus on understanding providers' knowledge of, experience with and attitudes toward such systems.

Integrated antibiotic clinical decision support system (CDSS) for appropriate choice and dosage: an analysis of retrospective data

Introduction

Decision-making for inpatient antibiotic prescribing is complex due to many considerations to be taken. So far, clinical decision support systems (CDSS) have been rarely used in antibiotic stewardship (ABS) and even less integrated in computerized physician order entry systems (CPOE).

Methods

We developed a guideline-based, CPOE-integrated CDSS (ID ANTIBIOTICS) to support antibiotic selection and dosing. We compared routine antibiotic inpatient prescribing data with CDSS-generated recommendations in the initial antibiotic selection, the duration of therapies, and costs. Finally, we assessed possible benefits of the CDSS by its performance in German ABS-guideline quality indicators (ABS-QIs).

Results

The requirements of several ABS-QIs can be supported with ID ANTIBIOTICS: electronic local guidelines, electronic decision-support, renal dosage adjustments, local guideline-based initial selection (all not quantified), and therapy durations for the treatment of pneumonia (significantly) without increasing costs. Performance in ABS-QIs for extensive therapies for community-acquired pneumonia could be improved with the CDSS by 20.2% (OR 0.134; 95% CI: 0.101-0.178); for hospital-acquired pneumonia by 3.7% (OR 0.742; 95% CI: 0.629-0.877). There was no difference in median daily drug costs between real-world prescriptions and CDSS recommendations (both: € 4.78, p=0.081).

Conclusions

In retrospective analyses, antibiotic CDSS can show possible performance in antibiotic stewardship through quality indicators (ABS-QIs). Further research and pilot testing of the software are needed to provide more insights into ABS-QI evaluation, user acceptance, and real-world effectiveness. Deep integration of antibiotic CDSS into existing medication processes without using multiple systems could contribute to the necessary acceptance of clinical practitioners.