1
|
Scholich T, Raj S, Lee J, Newman MW. Augmenting clinicians' analytical workflow through task-based integration of data visualizations and algorithmic insights: a user-centered design study. J Am Med Inform Assoc 2024:ocae183. [PMID: 39003519 DOI: 10.1093/jamia/ocae183] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 06/07/2024] [Accepted: 07/03/2024] [Indexed: 07/15/2024] Open
Abstract
OBJECTIVES To understand healthcare providers' experiences of using GlucoGuide, a mockup tool that integrates visual data analysis with algorithmic insights to support clinicians' use of patientgenerated data from Type 1 diabetes devices. MATERIALS AND METHODS This qualitative study was conducted in three phases. In Phase 1, 11 clinicians reviewed data using commercial diabetes platforms in a think-aloud data walkthrough activity followed by semistructured interviews. In Phase 2, GlucoGuide was developed. In Phase 3, the same clinicians reviewed data using GlucoGuide in a think-aloud activity followed by semistructured interviews. Inductive thematic analysis was used to analyze transcripts of Phase 1 and Phase 3 think-aloud activity and interview. RESULTS 3 high level tasks, 8 sub-tasks, and 4 challenges were identified in Phase 1. In Phase 2, 3 requirements for GlucoGuide were identified. Phase 3 results suggested that clinicians found GlucoGuide easier to use and experienced a lower cognitive burden as compared to the commercial diabetes data reports that were used in Phase 1. Additionally, GlucoGuide addressed the challenges experienced in Phase 1. DISCUSSION The study suggests that the knowledge of analytical tasks and task-specific visualization strategies in implementing features of data interfaces can result in tools that lower the perceived burden of engaging with data. Additionally, supporting clinicians in contextualizing algorithmic insights by visual analysis of relevant data can positively influence clinicians' willingness to leverage algorithmic support. CONCLUSION Task-aligned tools that combine multiple data-driven approaches, such as visualization strategies and algorithmic insights, can improve clinicians' experience in reviewing device data.
Collapse
Affiliation(s)
- Till Scholich
- School of Information, University of Michigan, Ann Arbor, MI 48109, United States
| | - Shriti Raj
- Department of Medicine, Center for Biomedical Informatics Research, Stanford University, Stanford, CA 94305, United States
- Institute for Human-Centered AI, Stanford University, Stanford, CA 94305, United States
| | - Joyce Lee
- Susan B. Meister Child Health Evaluation and Research Center (CHEAR), University of Michigan, Ann Arbor, MI 48109, United States
- Division of Pediatric Endocrinology, University of Michigan, Ann Arbor, MI 48109, United States
| | - Mark W Newman
- School of Information, University of Michigan, Ann Arbor, MI 48109, United States
- Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI 48109, United States
| |
Collapse
|
2
|
Wieben AM, Alreshidi BG, Douthit BJ, Sileo M, Vyas P, Steege L, Gilmore-Bykovskyi A. Nurses' perceptions of the design, implementation, and adoption of machine learning clinical decision support: A descriptive qualitative study. J Nurs Scholarsh 2024. [PMID: 38898636 DOI: 10.1111/jnu.13001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2024] [Revised: 05/06/2024] [Accepted: 06/07/2024] [Indexed: 06/21/2024]
Abstract
INTRODUCTION The purpose of this study was to explore nurses' perspectives on Machine Learning Clinical Decision Support (ML CDS) design, development, implementation, and adoption. DESIGN Qualitative descriptive study. METHODS Nurses (n = 17) participated in semi-structured interviews. Data were transcribed, coded, and analyzed using Thematic analysis methods as described by Braun and Clarke. RESULTS Four major themes and 14 sub-themes highlight nurses' perspectives on autonomy in decision-making, the influence of prior experience in shaping their preferences for use of novel CDS tools, the need for clarity in why ML CDS is useful in improving practice/outcomes, and their desire to have nursing integrated in design and implementation of these tools. CONCLUSION This study provided insights into nurse perceptions regarding the utility and usability of ML CDS as well as the influence of previous experiences with technology and CDS, change management strategies needed at the time of implementation of ML CDS, the importance of nurse-perceived engagement in the development process, nurse information needs at the time of ML CDS deployment, and the perceived impact of ML CDS on nurse decision making autonomy. CLINICAL RELEVANCE This study contributes to the body of knowledge about the use of AI and machine learning (ML) in nursing practice. Through generation of insights drawn from nurses' perspectives, these findings can inform successful design and adoption of ML Clinical Decision Support.
Collapse
Affiliation(s)
- Ann M Wieben
- University of Wisconsin-Madison School of Nursing, Madison, Wisconsin, USA
| | - Bader G Alreshidi
- Department of Medical Surgical Nursing, University of Hail College of Nursing, Hail, Saudi Arabia
| | - Brian J Douthit
- United States Department of Veterans Affairs, Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA
| | - Marisa Sileo
- Boston Children's Hospital, Boston, Massachusetts, USA
| | | | - Linsey Steege
- University of Wisconsin-Madison School of Nursing, Madison, Wisconsin, USA
| | - Andrea Gilmore-Bykovskyi
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison School of Medicine & Public Health, Madison, Wisconsin, USA
| |
Collapse
|
3
|
Sundermann M, Clendon O, McNeill R, Doogue M, Chin PKL. Optimising interruptive clinical decision support alerts for antithrombotic duplicate prescribing in hospital. Int J Med Inform 2024; 186:105418. [PMID: 38518676 DOI: 10.1016/j.ijmedinf.2024.105418] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 03/05/2024] [Accepted: 03/17/2024] [Indexed: 03/24/2024]
Abstract
INTRODUCTION Duplicate prescribing clinical decision support alerts can prevent important prescribing errors but are frequently the cause of much alert fatigue. Stat dose prescriptions are a known reason for overriding these alerts. This study aimed to evaluate the effect of excluding stat dose prescriptions from duplicate prescribing alerts for antithrombotic medicines on alert burden, prescriber adherence, and prescribing. MATERIALS AND METHODS A before (January 1st, 2017 to August 31st, 2022) and after (October 5th, 2022 to September 30th, 2023) study was undertaken of antithrombotic duplicate prescribing alerts and prescribing following a change in alert settings. Alert and prescribing data for antithrombotic medicines were joined, processed, and analysed to compare alert rates, adherence, and prescribing. Alert burden was assessed as alerts per 100 prescriptions. Adherence was measured at the point of the alert as whether the prescriber accepted the alert and following the alert as whether a relevant prescription was ceased within an hour. Co-prescribing of antithrombotic stat dose prescriptions was assessed pre- and post-alert reconfiguration. RESULTS Reconfiguration of the alerts reduced the alert rate by 29 % (p < 0.001). The proportion of alerts associated with cessation of antithrombotic duplication significantly increased (32.8 % to 44.5 %, p < 0.001). Adherence at the point of the alert increased 1.2 % (4.8 % to 6.0 %, p = 0.012) and 11.5 % (29.4 % to 40.9 %, p < 0.001) within one hour of the alert. When ceased after the alert over 80 % of duplicate prescriptions were ceased within 2 min of overriding. Antithrombotic stat dose co-prescribing was unchanged for 4 out of 5 antithrombotic duplication alert rules. CONCLUSION By reconfiguring our antithrombotic duplicate prescribing alerts, we reduced alert burden and increased alert adherence. Many prescribers ceased duplicate prescribing within 2 min of alert override highlighting the importance of incorporating post-alert measures in accurately determining prescriber alert adherence.
Collapse
Affiliation(s)
- Milan Sundermann
- Department of Medicine, University of Otago, Christchurch, New Zealand
| | - Olivia Clendon
- Department of Clinical Pharmacology, Te Whatu Ora Health New Zealand - Waitaha Canterbury, New Zealand
| | - Richard McNeill
- Department of Medicine, University of Otago, Christchurch, New Zealand
| | - Matthew Doogue
- Department of Medicine, University of Otago, Christchurch, New Zealand; Department of Clinical Pharmacology, Te Whatu Ora Health New Zealand - Waitaha Canterbury, New Zealand
| | - Paul K L Chin
- Department of Medicine, University of Otago, Christchurch, New Zealand; Department of Clinical Pharmacology, Te Whatu Ora Health New Zealand - Waitaha Canterbury, New Zealand.
| |
Collapse
|
4
|
Gencturk M, Laleci Erturkmen GB, Akpinar AE, Pournik O, Ahmad B, Arvanitis TN, Schmidt-Barzynski W, Robbins T, Alcantud Corcoles R, Abizanda P. Transforming evidence-based clinical guidelines into implementable clinical decision support services: the CAREPATH study for multimorbidity management. Front Med (Lausanne) 2024; 11:1386689. [PMID: 38860204 PMCID: PMC11163046 DOI: 10.3389/fmed.2024.1386689] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2024] [Accepted: 05/13/2024] [Indexed: 06/12/2024] Open
Abstract
Introduction The CAREPATH Project aims to develop a patient-centered integrated care platform tailored to older adults with multimorbidity, including mild cognitive impairment (MCI) or mild dementia. Our goal is to empower multidisciplinary care teams to craft personalized holistic care plans while adhering to evidence-based guidelines. This necessitates the creation of clear specifications for clinical decision support (CDS) services, consolidating guidance from multiple evidence-based clinical guidelines. Thus, a co-creation approach involving both clinical and technical experts is essential. Methods This paper outlines a robust methodology for generating implementable specifications for CDS services to automate clinical guidelines. We have established a co-creation framework to facilitate collaborative exploration of clinical guidelines between clinical experts and software engineers. We have proposed an open, repeatable, and traceable method for translating evidence-based guideline narratives into implementable specifications of CDS services. Our approach, based on international standards such as CDS-Hooks and HL7 FHIR, enhances interoperability and potential adoption of CDS services across diverse healthcare systems. Results This methodology has been followed to create implementable specifications for 65 CDS services, automating CAREPATH consensus guideline consolidating guidance from 25 selected evidence-based guidelines. A total of 296 CDS rules have been formally defined, with input parameters defined as clinical concepts bound to FHIR resources and international code systems. Outputs include 346 well-defined CDS Cards, offering clear guidance for care plan activities and goal suggestions. These specifications have led to the implementation of 65 CDS services integrated into the CAREPATH Adaptive Integrated Care Platform. Discussion Our methodology offers a systematic, replicable process for generating CDS specifications, ensuring consistency and reliability across implementation. By fostering collaboration between clinical expertise and technical proficiency, we enhance the quality and relevance of generated specifications. Clear traceability enables stakeholders to track the development process and ensure adherence to guideline recommendations.
Collapse
Affiliation(s)
- Mert Gencturk
- SRDC Software Research & Development and Consultancy Corporation, Ankara, Türkiye
| | | | - A. Emre Akpinar
- SRDC Software Research & Development and Consultancy Corporation, Ankara, Türkiye
- Department of Computer Engineering, Middle East Technical University, Ankara, Türkiye
| | - Omid Pournik
- Department of Electronic, Electrical and Systems Engineering, School of Engineering, University of Birmingham, Birmingham, United Kingdom
| | - Bilal Ahmad
- Department of Electronic, Electrical and Systems Engineering, School of Engineering, University of Birmingham, Birmingham, United Kingdom
| | - Theodoros N. Arvanitis
- Department of Electronic, Electrical and Systems Engineering, School of Engineering, University of Birmingham, Birmingham, United Kingdom
- Digital & Data Driven Research Unit, University Hospitals Coventry & Warwickshire NHS Trust, Coventry, United Kingdom
| | | | - Tim Robbins
- Digital & Data Driven Research Unit, University Hospitals Coventry & Warwickshire NHS Trust, Coventry, United Kingdom
| | - Ruben Alcantud Corcoles
- Geriatrics Department, Complejo Hospitalario Universitario de Albacete, Albacete, Spain
- CIBER de Fragilidad y Envejecimiento Saludable (CIBERFES), Instituto de Salud Carlos III, Madrid, Spain
| | - Pedro Abizanda
- Geriatrics Department, Complejo Hospitalario Universitario de Albacete, Albacete, Spain
- CIBER de Fragilidad y Envejecimiento Saludable (CIBERFES), Instituto de Salud Carlos III, Madrid, Spain
- Facultad de Medicina de Albacete, Universidad de Castilla-La Mancha, Albacete, Spain
| |
Collapse
|
5
|
Ray J, Finn EB, Tyrrell H, Aloe CF, Perrin EM, Wood CT, Miner DS, Grout R, Michel JJ, Damschroder LJ, Sharifi M. User-Centered Framework for Implementation of Technology (UFIT): Development of an Integrated Framework for Designing Clinical Decision Support Tools Packaged With Tailored Implementation Strategies. J Med Internet Res 2024; 26:e51952. [PMID: 38771622 PMCID: PMC11150893 DOI: 10.2196/51952] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/17/2023] [Revised: 11/30/2023] [Accepted: 02/17/2024] [Indexed: 05/22/2024] Open
Abstract
BACKGROUND Electronic health record-based clinical decision support (CDS) tools can facilitate the adoption of evidence into practice. Yet, the impact of CDS beyond single-site implementation is often limited by dissemination and implementation barriers related to site- and user-specific variation in workflows and behaviors. The translation of evidence-based CDS from initial development to implementation in heterogeneous environments requires a framework that assures careful balancing of fidelity to core functional elements with adaptations to ensure compatibility with new contexts. OBJECTIVE This study aims to develop and apply a framework to guide tailoring and implementing CDS across diverse clinical settings. METHODS In preparation for a multisite trial implementing CDS for pediatric overweight or obesity in primary care, we developed the User-Centered Framework for Implementation of Technology (UFIT), a framework that integrates principles from user-centered design (UCD), human factors/ergonomics theories, and implementation science to guide both CDS adaptation and tailoring of related implementation strategies. Our transdisciplinary study team conducted semistructured interviews with pediatric primary care clinicians and a diverse group of stakeholders from 3 health systems in the northeastern, midwestern, and southeastern United States to inform and apply the framework for our formative evaluation. RESULTS We conducted 41 qualitative interviews with primary care clinicians (n=21) and other stakeholders (n=20). Our workflow analysis found 3 primary ways in which clinicians interact with the electronic health record during primary care well-child visits identifying opportunities for decision support. Additionally, we identified differences in practice patterns across contexts necessitating a multiprong design approach to support a variety of workflows, user needs, preferences, and implementation strategies. CONCLUSIONS UFIT integrates theories and guidance from UCD, human factors/ergonomics, and implementation science to promote fit with local contexts for optimal outcomes. The components of UFIT were used to guide the development of Improving Pediatric Obesity Practice Using Prompts, an integrated package comprising CDS for obesity or overweight treatment with tailored implementation strategies. TRIAL REGISTRATION ClinicalTrials.gov NCT05627011; https://clinicaltrials.gov/study/NCT05627011.
Collapse
Affiliation(s)
- Jessica Ray
- Department of Health Outcomes and Biomedical Informatics, University of Florida College of Medicine, Gainesville, FL, United States
| | - Emily Benjamin Finn
- Department of Pediatrics, Yale University School of Medicine, New Haven, CT, United States
| | | | - Carlin F Aloe
- Department of Pediatrics, Yale University School of Medicine, New Haven, CT, United States
| | - Eliana M Perrin
- Department of Pediatrics, Johns Hopkins University School of Medicine and School of Nursing, Baltimore, MD, United States
| | - Charles T Wood
- Department of Pediatrics, Duke University School of Medicine, Durham, NC, United States
| | - Dean S Miner
- Departments of Pediatrics and Internal Medicine, East Carolina University, Greenville, NC, United States
| | - Randall Grout
- Department of Pediatrics, Indiana University School of Medicine and Regenstrief Institute, Indianapolis, IN, United States
| | - Jeremy J Michel
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Laura J Damschroder
- Implementation Pathways, LLC and Veterans Affairs Center for Clinical Management Research, Ann Arbor, MI, United States
| | - Mona Sharifi
- Department of Pediatrics, Yale University School of Medicine, New Haven, CT, United States
| |
Collapse
|
6
|
Grijalvo M, Ordieres-Meré J, Villalba-Díez J, Aladro-Benito Y, Martín-Ávila G, Simon-Hurtado A, Vivaracho-Pascual C. Sufficiency for PSS tracking gait disorders in multiple sclerosis: A managerial perspective. Heliyon 2024; 10:e30001. [PMID: 38707444 PMCID: PMC11066638 DOI: 10.1016/j.heliyon.2024.e30001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2023] [Revised: 04/15/2024] [Accepted: 04/18/2024] [Indexed: 05/07/2024] Open
Abstract
This study primarily aimed to explore the capabilities of digitalisation in the healthcare context, focusing on a specific disease. In this case, the study examined the potential of remote monitoring of gait to address the sensitivity of multiple sclerosis progression to gait characteristics by adopting a non-invasive approach to remotely quantify gait disturbances in a patient's daily life. To better understand the managerial aspects associated with this approach, the researchers conducted a literature review along with a set of semi-structured interviews. The target population included MS patients as well as the key agents involved in their care: patients' family members, neurologists, MS nurses, physiotherapists, medical directors, and pharmacist. The study identifies the perceived barriers and drivers that could contribute to the successful deployment of PSS remote gait monitoring as a healthcare service: i) At mega-level governance. Implications on privacy and security data are notable barriers missing on the speech. ii) At macro level, funding is highlighted as main barrier. The cost and lack of health system subsidies may render initiatives unsustainable, as emphasised by the interviewees. iii) At meso level, useable data is recognised as a driver. The data collection process can align with diverse interests to create value and business opportunities for the ecosystem actors, enhance care, attract stakeholders, such as insurers and pharma, and form partnerships. iv) At micro-level processes, we find two potential barriers: wearable device and app usability (comfort, navigation, efficiency) and organisational/behavioural aspects (training, digital affinity, skills), which are crucial for value creation in innovation ecosystems among patients and healthcare professionals. Finally, we find an interesting gap in the literature and interviews. Stakeholders' limited awareness of technological demands, especially from information technologies, for a successful long-term service, can be consider two key barriers for PSS.
Collapse
Affiliation(s)
- Mercedes Grijalvo
- Department of Organizational Engineering, Business Administration and Statistics, Universidad Politécnica de Madrid, Madrid, Spain
| | - Joaquín Ordieres-Meré
- Department of Organizational Engineering, Business Administration and Statistics, Universidad Politécnica de Madrid, Madrid, Spain
| | | | - Yolanda Aladro-Benito
- Department of Neurology, Getafe University Hospital, Madrid, Spain
- Faculty of Biomedical and Health Sciences, European University of Madrid, Madrid, Spain
| | | | - Arancha Simon-Hurtado
- Departamento de Informática, Escuela de Ingeniería Informática de Valladolid, Universidad de Valladolid, Paseo de Belén 15, 47011, Valladolid, Spain
| | - Carlos Vivaracho-Pascual
- Departamento de Informática, Escuela de Ingeniería Informática de Valladolid, Universidad de Valladolid, Paseo de Belén 15, 47011, Valladolid, Spain
| |
Collapse
|
7
|
Barton HJ, Maru A, Leaf MA, Hekman DJ, Wiegmann DA, Shah MN, Patterson BW. Academic Detailing as a Health Information Technology Implementation Method: Supporting the Design and Implementation of an Emergency Department-Based Clinical Decision Support Tool to Prevent Future Falls. JMIR Hum Factors 2024; 11:e52592. [PMID: 38635318 PMCID: PMC11066751 DOI: 10.2196/52592] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 02/08/2024] [Accepted: 03/02/2024] [Indexed: 04/19/2024] Open
Abstract
BACKGROUND Clinical decision support (CDS) tools that incorporate machine learning-derived content have the potential to transform clinical care by augmenting clinicians' expertise. To realize this potential, such tools must be designed to fit the dynamic work systems of the clinicians who use them. We propose the use of academic detailing-personal visits to clinicians by an expert in a specific health IT tool-as a method for both ensuring the correct understanding of that tool and its evidence base and identifying factors influencing the tool's implementation. OBJECTIVE This study aimed to assess academic detailing as a method for simultaneously ensuring the correct understanding of an emergency department-based CDS tool to prevent future falls and identifying factors impacting clinicians' use of the tool through an analysis of the resultant qualitative data. METHODS Previously, our team designed a CDS tool to identify patients aged 65 years and older who are at the highest risk of future falls and prompt an interruptive alert to clinicians, suggesting the patient be referred to a mobility and falls clinic for an evidence-based preventative intervention. We conducted 10-minute academic detailing interviews (n=16) with resident emergency medicine physicians and advanced practice providers who had encountered our CDS tool in practice. We conducted an inductive, team-based content analysis to identify factors that influenced clinicians' use of the CDS tool. RESULTS The following categories of factors that impacted clinicians' use of the CDS were identified: (1) aspects of the CDS tool's design (2) clinicians' understanding (or misunderstanding) of the CDS or referral process, (3) the busy nature of the emergency department environment, (4) clinicians' perceptions of the patient and their associated fall risk, and (5) the opacity of the referral process. Additionally, clinician education was done to address any misconceptions about the CDS tool or referral process, for example, demonstrating how simple it is to place a referral via the CDS and clarifying which clinic the referral goes to. CONCLUSIONS Our study demonstrates the use of academic detailing for supporting the implementation of health information technologies, allowing us to identify factors that impacted clinicians' use of the CDS while concurrently educating clinicians to ensure the correct understanding of the CDS tool and intervention. Thus, academic detailing can inform both real-time adjustments of a tool's implementation, for example, refinement of the language used to introduce the tool, and larger scale redesign of the CDS tool to better fit the dynamic work environment of clinicians.
Collapse
Affiliation(s)
- Hanna J Barton
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Apoorva Maru
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Margaret A Leaf
- Department of Information Services, UW Health, Madison, WI, United States
| | - Daniel J Hekman
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Douglas A Wiegmann
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI, United States
| | - Manish N Shah
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Brian W Patterson
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI, United States
| |
Collapse
|
8
|
Michaleff ZA, Hattingh L, Greenwood H, Mickan S, Jones M, van der Merwe M, Thomas R, Carlini J, Henry D, Stehlik P, Glasziou P, Keijzers G. Evaluating the use of clinical decision aids in an Australian emergency department: A cross-sectional survey. Emerg Med Australas 2024; 36:221-230. [PMID: 37963836 DOI: 10.1111/1742-6723.14338] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 10/10/2023] [Accepted: 10/12/2023] [Indexed: 11/16/2023]
Abstract
OBJECTIVE To identify healthcare professionals' knowledge, self-reported use, and documentation of clinical decision aids (CDAs) in a large ED in Australia, to identify behavioural determinants influencing the use of CDAs, and healthcare professionals preferences for integrating CDAs into the electronic medical record (EMR) system. METHODS Healthcare professionals (doctors, nurses and physiotherapists) working in the ED at the Gold Coast Hospital, Queensland were invited to complete an online survey. Quantitative data were analysed using descriptive statistics, and where appropriate, mapped to the theoretical domains framework to identify potential barriers to the use of CDAs. Qualitative data were analysed using content analysis. RESULTS Seventy-four healthcare professionals (34 medical officers, 31 nurses and nine physiotherapists) completed the survey. Healthcare professionals' knowledge and self-reported use of 21 validated CDAs was low but differed considerably across CDAs. Only 4 out of 21 CDAs were reported to be used 'sometimes' or 'always' by the majority of respondents (Ottawa Ankle Rule for ankle injury, Wells' criteria for pulmonary embolism, Wells' criteria for deep vein thrombosis and PERC rule for pulmonary embolism). Most respondents wanted to increase their use of valid and reliable CDAs and supported the integration of CDAs into the EMR to facilitate their use and support documentation. Potential barriers impacting the use of CDAs represented three theoretical domains of knowledge, social/professional role and identity, and social influences. CONCLUSIONS CDAs are used variably by healthcare professionals and are inconsistently applied in the clinical encounter. Preferences of healthcare professionals need to be considered to allow the successful integration of CDAs into the EMR.
Collapse
Affiliation(s)
- Zoe A Michaleff
- Northern New South Wales Local Health District, Lismore, New South Wales, Australia
- Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Queensland, Australia
| | - Laetitia Hattingh
- Gold Coast Hospital and Health Service, Gold Coast, Queensland, Australia
- School of Pharmacy, The University of Queensland, Brisbane, Queensland, Australia
| | - Hannah Greenwood
- Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Queensland, Australia
| | - Sharon Mickan
- Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Queensland, Australia
| | - Mark Jones
- Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Queensland, Australia
- Gold Coast Hospital and Health Service, Gold Coast, Queensland, Australia
| | - Madeleen van der Merwe
- Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Queensland, Australia
- Gold Coast Hospital and Health Service, Gold Coast, Queensland, Australia
| | - Rae Thomas
- Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Queensland, Australia
- Tropical Australian Academic Health Centre, Townsville, Queensland, Australia
| | - Joan Carlini
- Consumer Advisory Group, Gold Coast Health, Gold Coast, Queensland, Australia
- Department of Marketing, Griffith University, Gold Coast, Queensland, Australia
| | - David Henry
- Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Queensland, Australia
- Gold Coast Hospital and Health Service, Gold Coast, Queensland, Australia
| | - Paulina Stehlik
- Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Queensland, Australia
- Gold Coast Hospital and Health Service, Gold Coast, Queensland, Australia
| | - Paul Glasziou
- Institute for Evidence-Based Healthcare, Bond University, Gold Coast, Queensland, Australia
- Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Queensland, Australia
- School of Medicine, Griffith University, Gold Coast, Queensland, Australia
| | - Gerben Keijzers
- Faculty of Health Sciences and Medicine, Bond University, Gold Coast, Queensland, Australia
- School of Medicine, Griffith University, Gold Coast, Queensland, Australia
- Department of Emergency Medicine, Gold Coast University Hospital, Gold Coast, Queensland, Australia
| |
Collapse
|
9
|
Domínguez J, Prociuk D, Marović B, Čyras K, Cocarascu O, Ruiz F, Mi E, Mi E, Ramtale C, Rago A, Darzi A, Toni F, Curcin V, Delaney B. ROAD2H: Development and evaluation of an open-source explainable artificial intelligence approach for managing co-morbidity and clinical guidelines. Learn Health Syst 2024; 8:e10391. [PMID: 38633019 PMCID: PMC11019374 DOI: 10.1002/lrh2.10391] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2023] [Revised: 07/29/2023] [Accepted: 08/07/2023] [Indexed: 04/19/2024] Open
Abstract
Introduction Clinical decision support (CDS) systems (CDSSs) that integrate clinical guidelines need to reflect real-world co-morbidity. In patient-specific clinical contexts, transparent recommendations that allow for contraindications and other conflicts arising from co-morbidity are a requirement. In this work, we develop and evaluate a non-proprietary, standards-based approach to the deployment of computable guidelines with explainable argumentation, integrated with a commercial electronic health record (EHR) system in Serbia, a middle-income country in West Balkans. Methods We used an ontological framework, the Transition-based Medical Recommendation (TMR) model, to represent, and reason about, guideline concepts, and chose the 2017 International global initiative for chronic obstructive lung disease (GOLD) guideline and a Serbian hospital as the deployment and evaluation site, respectively. To mitigate potential guideline conflicts, we used a TMR-based implementation of the Assumptions-Based Argumentation framework extended with preferences and Goals (ABA+G). Remote EHR integration of computable guidelines was via a microservice architecture based on HL7 FHIR and CDS Hooks. A prototype integration was developed to manage chronic obstructive pulmonary disease (COPD) with comorbid cardiovascular or chronic kidney diseases, and a mixed-methods evaluation was conducted with 20 simulated cases and five pulmonologists. Results Pulmonologists agreed 97% of the time with the GOLD-based COPD symptom severity assessment assigned to each patient by the CDSS, and 98% of the time with one of the proposed COPD care plans. Comments were favourable on the principles of explainable argumentation; inclusion of additional co-morbidities was suggested in the future along with customisation of the level of explanation with expertise. Conclusion An ontological model provided a flexible means of providing argumentation and explainable artificial intelligence for a long-term condition. Extension to other guidelines and multiple co-morbidities is needed to test the approach further.
Collapse
Affiliation(s)
- Jesús Domínguez
- Department of Population Health SciencesKing's College LondonLondonUK
| | | | | | | | | | - Francis Ruiz
- London School of Hygiene and Tropical MedicineLondonUK
| | - Ella Mi
- University of OxfordOxfordUK
| | - Emma Mi
- University of OxfordOxfordUK
| | | | | | | | | | - Vasa Curcin
- Department of Population Health SciencesKing's College LondonLondonUK
| | | |
Collapse
|
10
|
Hu Z, Wang M, Zheng S, Xu X, Zhang Z, Ge Q, Li J, Yao Y. Clinical Decision Support Requirements for Ventricular Tachycardia Diagnosis Within the Frameworks of Knowledge and Practice: Survey Study. JMIR Hum Factors 2024; 11:e55802. [PMID: 38530337 PMCID: PMC11005434 DOI: 10.2196/55802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2023] [Revised: 02/15/2024] [Accepted: 03/02/2024] [Indexed: 03/27/2024] Open
Abstract
BACKGROUND Ventricular tachycardia (VT) diagnosis is challenging due to the similarity between VT and some forms of supraventricular tachycardia, complexity of clinical manifestations, heterogeneity of underlying diseases, and potential for life-threatening hemodynamic instability. Clinical decision support systems (CDSSs) have emerged as promising tools to augment the diagnostic capabilities of cardiologists. However, a requirements analysis is acknowledged to be vital for the success of a CDSS, especially for complex clinical tasks such as VT diagnosis. OBJECTIVE The aims of this study were to analyze the requirements for a VT diagnosis CDSS within the frameworks of knowledge and practice and to determine the clinical decision support (CDS) needs. METHODS Our multidisciplinary team first conducted semistructured interviews with seven cardiologists related to the clinical challenges of VT and expected decision support. A questionnaire was designed by the multidisciplinary team based on the results of interviews. The questionnaire was divided into four sections: demographic information, knowledge assessment, practice assessment, and CDS needs. The practice section consisted of two simulated cases for a total score of 10 marks. Online questionnaires were disseminated to registered cardiologists across China from December 2022 to February 2023. The scores for the practice section were summarized as continuous variables, using the mean, median, and range. The knowledge and CDS needs sections were assessed using a 4-point Likert scale without a neutral option. Kruskal-Wallis tests were performed to investigate the relationship between scores and practice years or specialty. RESULTS Of the 687 cardiologists who completed the questionnaire, 567 responses were eligible for further analysis. The results of the knowledge assessment showed that 383 cardiologists (68%) lacked knowledge in diagnostic evaluation. The overall average score of the practice assessment was 6.11 (SD 0.55); the etiological diagnosis section had the highest overall scores (mean 6.74, SD 1.75), whereas the diagnostic evaluation section had the lowest scores (mean 5.78, SD 1.19). A majority of cardiologists (344/567, 60.7%) reported the need for a CDSS. There was a significant difference in practice competency scores between general cardiologists and arrhythmia specialists (P=.02). CONCLUSIONS There was a notable deficiency in the knowledge and practice of VT among Chinese cardiologists. Specific knowledge and practice support requirements were identified, which provide a foundation for further development and optimization of a CDSS. Moreover, it is important to consider clinicians' specialization levels and years of practice for effective and personalized support.
Collapse
Affiliation(s)
- Zhao Hu
- Arrhythmia Center, Fuwai Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing, China
| | - Min Wang
- Institute of Medical Information, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Si Zheng
- Institute of Medical Information, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Xiaowei Xu
- Institute of Medical Information, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Zhuxin Zhang
- Arrhythmia Center, Fuwai Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing, China
| | - Qiaoyue Ge
- West China School of Public Health, West China Fourth Hospital, Sichuan University, Chengdu, China
| | - Jiao Li
- Institute of Medical Information, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
| | - Yan Yao
- Arrhythmia Center, Fuwai Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College/National Center for Cardiovascular Diseases, Beijing, China
| |
Collapse
|
11
|
Sloss EA, Jones TL, Baker K, Robins JLW, Thacker LR. Factors Influencing Medication Administration Outcomes Among New Graduate Nurses Using Bar Code-Assisted Medication Administration. Comput Inform Nurs 2024; 42:199-206. [PMID: 38206171 PMCID: PMC10925919 DOI: 10.1097/cin.0000000000001083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2024]
Abstract
Paramount to patient safety is the ability for nurses to make clinical decisions free from human error. Yet, the dynamic clinical environment in which nurses work is characterized by uncertainty, urgency, and high consequence, necessitating that nurses make quick and critical decisions. The aim of this study was to examine the influence of human and environmental factors on the decision to administer among new graduate nurses in response to alert generation during bar code-assisted medication administration. The design for this study was a descriptive, longitudinal, observational cohort design using EHR audit log and administrative data. The study was set at a large, urban medical center in the United States and included 132 new graduate nurses who worked on adult, inpatient units. Research variables included human and environmental factors. Data analysis included descriptive and inferential analyses. This study found that participants continued with administration of a medication in 90.75% of alert encounters. When considering the response to an alert, residency cohort, alert category, and previous exposure variables were associated with the decision to proceed with administration. It is important to continue to study factors that influence nurses' decision-making, particularly during the process of medication administration, to improve patient safety and outcomes.
Collapse
Affiliation(s)
- Elizabeth A Sloss
- Author Affiliation: School of Nursing, Virginia Commonwealth University (Dr Sloss), Richmond; College of Nursing, University of Utah (Dr Sloss), Salt Lake City; Department of Adult Health and Nursing Systems, School of Nursing, Virginia Commonwealth University (Dr Jones and Robins), Richmond, Virginia; UVA Health (Dr Baker), Charlottesville, Virginia; and Department of Biostatistics, School of Medicine, Virginia Commonwealth University (Dr Thacker)
| | | | | | | | | |
Collapse
|
12
|
Calderon Y, Sandigan G, Tan-Lim CSC, De Mesa RYH, Fabian NMC, Rey MP, Sanchez JT, Dans LF, Galingana CLT, Bernal-Sundiang N, Casile RU, Aquino MRN, Poblete KE, Lopez JFE, Zabala H, Dans AL. Feasibility, acceptability and impact of a clinical decision support tool among primary care providers in an urban, rural and remote site in the Philippines. BMJ Open Qual 2024; 13:e002526. [PMID: 38423587 PMCID: PMC10910488 DOI: 10.1136/bmjoq-2023-002526] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2023] [Accepted: 02/22/2024] [Indexed: 03/02/2024] Open
Abstract
INTRODUCTION Strengthening primary care helps address health inequities that continue to persist in the Philippines. The Philippine Primary Care Studies pilot-tested interventions to improve the primary care system. One intervention was the provision of a free subscription to an electronic decision support application called UpToDate (UTD) for primary care providers (PCPs), including doctors, nurses, midwives and community health workers (CHWs). This study aimed to (1) assess the feasibility of using UTD as information source for PCPs in urban, rural and remote settings, (2) determine the acceptability of UTD as an information source for PCPs and (3) examine the impact of UTD access on PCP clinical decision-making. METHODS Four focus group discussions (FGDs) and two key informant interviews (KII) were conducted to gather insights from 30 PCPs. Thematic analysis through coding in NVivo V.12 was done using the technology acceptance model (TAM) as a guiding framework. RESULTS All PCPs had positive feedback regarding UTD use because of its comprehensiveness, accessibility, mobility and general design. The participants relayed UTD's benefit for point-of-contact use, capacity-building and continuing professional development. PCPs across the three sites, including CHWs with no formal medical education, were able to provide evidence-based medical advice to patients through UTD. However, external factors in these settings impeded the full integration of UTD in the PCPs' workflow, including poor internet access, unstable sources of electricity, lack of compatible mobile devices and the need for translation to the local language. CONCLUSION UTD was a feasible and acceptable clinical decision support tool for the PCPs. Factors affecting the feasibility of using UTD include technological and environmental factors (ie, internet access and the lack of translation to the local language), as well as the organisational structure of the primary care facility which determines the roles of the PCPs. Despite the difference in roles and responsibilities of the PCPs, UTD positively impacted decision-making and patient education for all PCPs through its use as a point-of-contact tool and a tool for capacity-building.
Collapse
Affiliation(s)
- Ysabela Calderon
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Gillian Sandigan
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Carol Stephanie C Tan-Lim
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
- Department of Clinical Epidemiology, University of the Philippines Manila, Manila, Philippines
| | - Regine Ynez H De Mesa
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Noleen Marie C Fabian
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Mia P Rey
- Department of Accounting and Finance, Cesar E.A. Virata School of Business, University of the Philippines Diliman, Quezon City, Philippines
| | - Josephine T Sanchez
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Leonila F Dans
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
- Department of Clinical Epidemiology, University of the Philippines Manila, Manila, Philippines
| | - Cara Lois T Galingana
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Nannette Bernal-Sundiang
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Ray U Casile
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Maria Rhodora N Aquino
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Karl Engelene Poblete
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Johanna Faye E Lopez
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Herbert Zabala
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
| | - Antonio L Dans
- Center for Integrative and Development Studies, University of the Philippines Diliman, Quezon City, Metro Manila, Philippines
- National Institutes of Health, University of the Philippines Manila, Manila, Metro Manila, Philippines
| |
Collapse
|
13
|
Liao Z, Wang J, Shi Z, Lu L, Tabata H. Revolutionary Potential of ChatGPT in Constructing Intelligent Clinical Decision Support Systems. Ann Biomed Eng 2024; 52:125-129. [PMID: 37332008 DOI: 10.1007/s10439-023-03288-w] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Accepted: 06/13/2023] [Indexed: 06/20/2023]
Abstract
Recently, Chatbot Generative Pre-trained Transformer (ChatGPT) is recognized as a promising clinical decision support system (CDSS) in the medical field owing to its advanced text analysis capabilities and interactive design. However, ChatGPT primarily focuses on learning text semantics rather than learning complex data structures and conducting real-time data analysis, which typically necessitate the development of intelligent CDSS employing specialized machine learning algorithms. Although ChatGPT cannot directly execute specific algorithms, it aids in algorithm design for intelligent CDSS at the textual level. In this study, besides discussing the types of CDSS and their relationship with ChatGPT, we mainly investigate the benefits and drawbacks of employing ChatGPT as an auxiliary design tool for intelligent CDSS. Our findings indicate that by collaborating with human expertise, ChatGPT has the potential to revolutionize the development of robust and effective intelligent CDSS.
Collapse
Affiliation(s)
- Zhiqiang Liao
- Department of Electrical Engineering and Information Systems, Graduate School of Engineering, The University of Tokyo, Tokyo, 113-8656, Japan.
| | - Jian Wang
- Department of Orthopaedics, Qilu Hospital of Shandong University, Jinan, 250012, People's Republic of China
| | - Zhuozheng Shi
- Department of Bioengineering, Graduate School of Engineering, The University of Tokyo, Tokyo, 113-8656, Japan
| | - Lintao Lu
- Department of Orthopaedics, Qilu Hospital of Shandong University, Jinan, 250012, People's Republic of China.
- Department of Orthopaedics, Qilu Hospital of Shandong University Dezhou Hospital, Dezhou, 253000, People's Republic of China.
| | - Hitoshi Tabata
- Department of Electrical Engineering and Information Systems, Graduate School of Engineering, The University of Tokyo, Tokyo, 113-8656, Japan
- Department of Bioengineering, Graduate School of Engineering, The University of Tokyo, Tokyo, 113-8656, Japan
| |
Collapse
|
14
|
García-García JA, Carrero M, Escalona MJ, Lizcano D. Evaluation of clinical practice guideline-derived clinical decision support systems using a novel quality model. J Biomed Inform 2024; 149:104573. [PMID: 38081565 DOI: 10.1016/j.jbi.2023.104573] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2022] [Revised: 11/26/2023] [Accepted: 12/08/2023] [Indexed: 01/22/2024]
Abstract
Over the last decade, clinical practice guidelines (CPGs) have become an important asset for daily life in healthcare organizations. Efficient management and digitization of CPGs help achieve organizational objectives and improve patient care and healthcare quality by reducing variability. However, digitizing CPGs is a difficult, complex task because they are usually expressed as text, and this often leads to the development of partial software solutions. At present, different research proposals and CPG-derived CDSS (clinical decision support system) do exist for managing CPG digitalization lifecycles (from modeling to deployment and execution), but they do not all provide full lifecycle support, making it more difficult to choose solutions or proposals that fully meet the needs of a healthcare organization. This paper proposes a method based on quality models to uniformly compare and evaluate technological tools, providing a rigorous method that uses qualitative and quantitative analysis of technological aspects. In addition, this paper also presents how this method has been instantiated to evaluate and compare CPG-derived CDSS by highlighting each phase of the CPG digitization lifecycle. Finally, discussion and analysis of currently available tools are presented, identifying gaps and limitations.
Collapse
Affiliation(s)
- Julián Alberto García-García
- ES3 Group (Engineering and Science for Software Systems group). Computer Languages and Systems Department. Escuela Técnica Superior de Ingeniería Informática., Avda. Reina Mercedes s/n. 41012 Seville, Spain.
| | - Manuel Carrero
- ES3 Group (Engineering and Science for Software Systems group). Computer Languages and Systems Department. Escuela Técnica Superior de Ingeniería Informática., Avda. Reina Mercedes s/n. 41012 Seville, Spain.
| | - María José Escalona
- ES3 Group (Engineering and Science for Software Systems group). Computer Languages and Systems Department. Escuela Técnica Superior de Ingeniería Informática., Avda. Reina Mercedes s/n. 41012 Seville, Spain.
| | - David Lizcano
- School of Computer Science, Madrid Open University, UDIMA., Campus Collado Villalba, Madrid, Spain.
| |
Collapse
|
15
|
Hong WS, Rudas A, Bell EJ, Chiang JN. Association of red blood cell distribution width with hospital admission and in-hospital mortality across all-cause adult emergency department visits. JAMIA Open 2023; 6:ooad053. [PMID: 37501917 PMCID: PMC10368803 DOI: 10.1093/jamiaopen/ooad053] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 07/07/2023] [Accepted: 07/12/2023] [Indexed: 07/29/2023] Open
Abstract
Objectives To test the association between the initial red blood cell distribution width (RDW) value in the emergency department (ED) and hospital admission and, among those admitted, in-hospital mortality. Materials and Methods We perform a retrospective analysis of 210 930 adult ED visits with complete blood count results from March 2013 to February 2022. Primary outcomes were hospital admission and in-hospital mortality. Variables for each visit included demographics, comorbidities, vital signs, basic metabolic panel, complete blood count, and final diagnosis. The association of each outcome with the initial RDW value was calculated across 3 age groups (<45, 45-65, and >65) as well as across 374 diagnosis categories. Logistic regression (LR) and XGBoost models using all variables excluding final diagnoses were built to test whether RDW was a highly weighted and informative predictor for each outcome. Finally, simplified models using only age, sex, and vital signs were built to test whether RDW had additive predictive value. Results Compared to that of discharged visits (mean [SD]: 13.8 [2.03]), RDW was significantly elevated in visits that resulted in admission (15.1 [2.72]) and, among admissions, those resulting in intensive care unit stay (15.3 [2.88]) and/or death (16.8 [3.25]). This relationship held across age groups as well as across various diagnosis categories. An RDW >16 achieved 90% specificity for hospital admission, while an RDW >18.5 achieved 90% specificity for in-hospital mortality. LR achieved a test area under the curve (AUC) of 0.77 (95% confidence interval [CI] 0.77-0.78) for hospital admission and 0.85 (95% CI 0.81-0.88) for in-hospital mortality, while XGBoost achieved a test AUC of 0.90 (95% CI 0.89-0.90) for hospital admission and 0.96 (95% CI 0.94-0.97) for in-hospital mortality. RDW had high scaled weights and information gain for both outcomes and had additive value in simplified models predicting hospital admission. Discussion Elevated RDW, previously associated with mortality in myocardial infarction, pulmonary embolism, heart failure, sepsis, and COVID-19, is associated with hospital admission and in-hospital mortality across all-cause adult ED visits. Used alone, elevated RDW may be a specific, but not sensitive, test for both outcomes, with multivariate LR and XGBoost models showing significantly improved test characteristics. Conclusions RDW, a component of the complete blood count panel routinely ordered as the initial workup for the undifferentiated patient, may be a generalizable biomarker for acuity in the ED.
Collapse
Affiliation(s)
- Woo Suk Hong
- Department of Emergency Medicine, University of California Los Angeles David Geffen School of Medicine, Los Angeles, California, USA
| | - Akos Rudas
- Department of Computational Medicine, University of California Los Angeles David Geffen School of Medicine, Los Angeles, California, USA
| | - Elijah J Bell
- Department of Emergency Medicine, University of California Los Angeles David Geffen School of Medicine, Los Angeles, California, USA
| | - Jeffrey N Chiang
- Corresponding Author: Jeffrey N. Chiang, PhD, Department of Computational Medicine, University of California Los Angeles David Geffen School of Medicine, 621 Charles E Young Dr S, Room 5217 Life Sciences Bldg., Los Angeles, CA 90095, USA;
| |
Collapse
|
16
|
Scott P, Heigl M, McCay C, Shepperdson P, Lima‐Walton E, Andrikopoulou E, Brunnhuber K, Cornelius G, Faulding S, McAlister B, Rowark S, South M, Thomas MR, Whatling J, Williams J, Wyatt JC, Greaves F. Modelling clinical narrative as computable knowledge: The NICE computable implementation guidance project. Learn Health Syst 2023; 7:e10394. [PMID: 37860056 PMCID: PMC10582221 DOI: 10.1002/lrh2.10394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 08/21/2023] [Accepted: 08/22/2023] [Indexed: 10/21/2023] Open
Abstract
Introduction Translating narrative clinical guidelines to computable knowledge is a long-standing challenge that has seen a diverse range of approaches. The UK National Institute for Health and Care Excellence (NICE) Content Advisory Board (CAB) aims ultimately to (1) guide clinical decision support and other software developers to increase traceability, fidelity and consistency in supporting clinical use of NICE recommendations, (2) guide local practice audit and intervention to reduce unwarranted variation, (3) provide feedback to NICE on how future recommendations should be developed. Objectives The first phase of work was to explore a range of technical approaches to transition NICE toward the production of natively digital content. Methods Following an initial 'collaborathon' in November 2022, the NICE Computable Implementation Guidance project (NCIG) was established. We held a series of workstream calls approximately fortnightly, focusing on (1) user stories and trigger events, (2) information model and definitions, (3) horizon-scanning and output format. A second collaborathon was held in March 2023 to consolidate progress across the workstreams and agree residual actions to complete. Results While we initially focussed on technical implementation standards, we decided that an intermediate logical model was a more achievable first step in the journey from narrative to fully computable representation. NCIG adopted the WHO Digital Adaptation Kit (DAK) as a technology-agnostic method to model user scenarios, personae, processes and workflow, core data elements and decision-support logic. Further work will address indicators, such as prescribing compliance, and implementation in document templates for primary care patient record systems. Conclusions The project has shown that the WHO DAK, with some modification, is a promising approach to build technology-neutral logical specifications of NICE recommendations. Implementation of concurrent computable modelling by multidisciplinary teams during guideline development poses methodological and cultural questions that are complex but tractable given suitable will and leadership.
Collapse
Affiliation(s)
- Philip Scott
- Institute of Management & HealthUniversity of Wales Trinity Saint DavidCarmarthenWalesUK
| | | | - Charles McCay
- Ramsey SystemsShrewsburyUK
- Professional Record Standards BodyLondonUK
| | | | | | | | | | - Gary Cornelius
- Computer & Information SciencesUniversity of StrathclydeGlasgowScotlandUK
| | | | | | | | - Matthew South
- Open Clinical CICLoughtonUK
- Institute of Applied Health ResearchUniversity of BirminghamBirminghamUK
| | - Mark R. Thomas
- Institute of Applied Health ResearchUniversity of BirminghamBirminghamUK
| | | | | | | | | |
Collapse
|
17
|
Sommers S, Tolle H, Napier C, Hoppe J. Targeted messaging to improve the adoption of clinical decision support for prescription drug monitoring program use. J Am Med Inform Assoc 2023; 30:1711-1716. [PMID: 37433582 PMCID: PMC10531197 DOI: 10.1093/jamia/ocad127] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 06/08/2023] [Accepted: 06/30/2023] [Indexed: 07/13/2023] Open
Abstract
Clinical decision support (CDS) can prevent medical errors and improve patient outcomes. Electronic health record (EHR)-based CDS, designed to facilitate prescription drug monitoring program (PDMP) review, has reduced inappropriate opioid prescribing. However, the pooled effectiveness of CDS has exhibited substantial heterogeneity and current literature does not adequately detail why certain CDS are more successful than others. Clinicians regularly override CDS, limiting its impact. No studies recommend how to help nonadopters recognize and recover from CDS misuse. We hypothesized that a targeted educational intervention would improve CDS adoption and effectiveness for nonadopters. Over 10 months, we identified 478 providers consistently overriding CDS (nonadopters) and sent each up to 3 educational message(s) via email or EHR-based chat. One hundred sixty-one (34%) nonadopters stopped consistently overriding CDS and started reviewing the PDMP after contact. We concluded that targeted messaging is a low-resource way to disseminate CDS education and improve CDS adoption and best practice delivery.
Collapse
Affiliation(s)
- Stuart Sommers
- Department of Emergency Medicine, University of Colorado Denver School of Medicine, Aurora, Colorado, USA
| | - Heather Tolle
- Department of Emergency Medicine, University of Colorado Denver School of Medicine, Aurora, Colorado, USA
| | - Cheryl Napier
- Department of Emergency Medicine, University of Colorado Denver School of Medicine, Aurora, Colorado, USA
| | - Jason Hoppe
- Department of Emergency Medicine, University of Colorado Denver School of Medicine, Aurora, Colorado, USA
| |
Collapse
|
18
|
Matson-Koffman DM, Robinson SJ, Jakhmola P, Fochtmann LJ, Willett D, Lubin IM, Burton MM, Tailor A, Pitts DL, Casey DE, Opelka FG, Mullins R, Elder R, Michaels M. An Integrated Process for Co-Developing and Implementing Written and Computable Clinical Practice Guidelines. Am J Med Qual 2023; 38:S12-S34. [PMID: 37668271 PMCID: PMC10476601 DOI: 10.1097/jmq.0000000000000137] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/06/2023]
Abstract
The goal of this article is to describe an integrated parallel process for the co-development of written and computable clinical practice guidelines (CPGs) to accelerate adoption and increase the impact of guideline recommendations in clinical practice. From February 2018 through December 2021, interdisciplinary work groups were formed after an initial Kaizen event and using expert consensus and available literature, produced a 12-phase integrated process (IP). The IP includes activities, resources, and iterative feedback loops for developing, implementing, disseminating, communicating, and evaluating CPGs. The IP incorporates guideline standards and informatics practices and clarifies how informaticians, implementers, health communicators, evaluators, and clinicians can help guideline developers throughout the development and implementation cycle to effectively co-develop written and computable guidelines. More efficient processes are essential to create actionable CPGs, disseminate and communicate recommendations to clinical end users, and evaluate CPG performance. Pilot testing is underway to determine how this IP expedites the implementation of CPGs into clinical practice and improves guideline uptake and health outcomes.
Collapse
Affiliation(s)
| | | | | | - Laura J. Fochtmann
- Department of Psychiatry, Pharmacological Sciences and Biomedical Informatics, Stony Brook University
| | - DuWayne Willett
- Department of Internal Medicine’s Division of Cardiology at University of Texas, Southwestern Medical Center
| | - Ira M. Lubin
- Centers for Disease Control and Prevention (CDC)
| | | | | | | | - Donald E. Casey
- Department of Internal Medicine, Rush Medical College, Jefferson College of Population Health, Institute for Healthcare Informatics, University of Minnesota
| | | | | | - Randy Elder
- Centers for Disease Control and Prevention (CDC)
| | | |
Collapse
|
19
|
Sittig DF, Boxwala A, Wright A, Zott C, Desai P, Dhopeshwarkar R, Swiger J, Lomotan EA, Dobes A, Dullabh P. A lifecycle framework illustrates eight stages necessary for realizing the benefits of patient-centered clinical decision support. J Am Med Inform Assoc 2023; 30:1583-1589. [PMID: 37414544 PMCID: PMC10436138 DOI: 10.1093/jamia/ocad122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2022] [Revised: 06/06/2023] [Accepted: 06/23/2023] [Indexed: 07/08/2023] Open
Abstract
The design, development, implementation, use, and evaluation of high-quality, patient-centered clinical decision support (PC CDS) is necessary if we are to achieve the quintuple aim in healthcare. We developed a PC CDS lifecycle framework to promote a common understanding and language for communication among researchers, patients, clinicians, and policymakers. The framework puts the patient, and/or their caregiver at the center and illustrates how they are involved in all the following stages: Computable Clinical Knowledge, Patient-specific Inference, Information Delivery, Clinical Decision, Patient Behaviors, Health Outcomes, Aggregate Data, and patient-centered outcomes research (PCOR) Evidence. Using this idealized framework reminds key stakeholders that developing, deploying, and evaluating PC-CDS is a complex, sociotechnical challenge that requires consideration of all 8 stages. In addition, we need to ensure that patients, their caregivers, and the clinicians caring for them are explicitly involved at each stage to help us achieve the quintuple aim.
Collapse
Affiliation(s)
- Dean F Sittig
- McWilliams School of Biomedical Informatics, University of Texas Health Science Center at Houston, Houston, Texas, USA
| | | | - Adam Wright
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Courtney Zott
- NORC at the University of Chicago, Bethesda, Maryland, USA
| | - Priyanka Desai
- NORC at the University of Chicago, Bethesda, Maryland, USA
| | | | - James Swiger
- Center for Evidence and Practice Improvement, Agency for Healthcare Research and Quality, Rockville, Maryland, USA
| | - Edwin A Lomotan
- Center for Evidence and Practice Improvement, Agency for Healthcare Research and Quality, Rockville, Maryland, USA
| | - Angela Dobes
- Crohn’s & Colitis Foundation, New York, New York, USA
| | | |
Collapse
|
20
|
Hekman DJ, Cochran AL, Maru AP, Barton HJ, Shah MN, Wiegmann D, Smith MA, Liao F, Patterson BW. Effectiveness of an Emergency Department-Based Machine Learning Clinical Decision Support Tool to Prevent Outpatient Falls Among Older Adults: Protocol for a Quasi-Experimental Study. JMIR Res Protoc 2023; 12:e48128. [PMID: 37535416 PMCID: PMC10436111 DOI: 10.2196/48128] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2023] [Revised: 05/04/2023] [Accepted: 05/23/2023] [Indexed: 08/04/2023] Open
Abstract
BACKGROUND Emergency department (ED) providers are important collaborators in preventing falls for older adults because they are often the first health care providers to see a patient after a fall and because at-home falls are often preceded by previous ED visits. Previous work has shown that ED referrals to falls interventions can reduce the risk of an at-home fall by 38%. Screening patients at risk for a fall can be time-consuming and difficult to implement in the ED setting. Machine learning (ML) and clinical decision support (CDS) offer the potential of automating the screening process. However, it remains unclear whether automation of screening and referrals can reduce the risk of future falls among older patients. OBJECTIVE The goal of this paper is to describe a research protocol for evaluating the effectiveness of an automated screening and referral intervention. These findings will inform ongoing discussions about the use of ML and artificial intelligence to augment medical decision-making. METHODS To assess the effectiveness of our program for patients receiving the falls risk intervention, our primary analysis will be to obtain referral completion rates at 3 different EDs. We will use a quasi-experimental design known as a sharp regression discontinuity with regard to intent-to-treat, since the intervention is administered to patients whose risk score falls above a threshold. A conditional logistic regression model will be built to describe 6-month fall risk at each site as a function of the intervention, patient demographics, and risk score. The odds ratio of a return visit for a fall and the 95% CI will be estimated by comparing those identified as high risk by the ML-based CDS (ML-CDS) and those who were not but had a similar risk profile. RESULTS The ML-CDS tool under study has been implemented at 2 of the 3 EDs in our study. As of April 2023, a total of 1326 patient encounters have been flagged for providers, and 339 unique patients have been referred to the mobility and falls clinic. To date, 15% (45/339) of patients have scheduled an appointment with the clinic. CONCLUSIONS This study seeks to quantify the impact of an ML-CDS intervention on patient behavior and outcomes. Our end-to-end data set allows for a more meaningful analysis of patient outcomes than other studies focused on interim outcomes, and our multisite implementation plan will demonstrate applicability to a broad population and the possibility to adapt the intervention to other EDs and achieve similar results. Our statistical methodology, regression discontinuity design, allows for causal inference from observational data and a staggered implementation strategy allows for the identification of secular trends that could affect causal associations and allow mitigation as necessary. TRIAL REGISTRATION ClinicalTrials.gov NCT05810064; https://www.clinicaltrials.gov/study/NCT05810064. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) DERR1-10.2196/48128.
Collapse
Affiliation(s)
- Daniel J Hekman
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Amy L Cochran
- Department of Population Health, University of Wisconsin-Madison, Madison, WI, United States
| | - Apoorva P Maru
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Hanna J Barton
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI, United States
| | - Manish N Shah
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| | - Douglas Wiegmann
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI, United States
| | - Maureen A Smith
- Health Innovation Program, University of Wisconsin-Madison, Madison, WI, United States
| | - Frank Liao
- Department of Applied Data Science, UWHealth Hospitals and Clinics, University of Wisconsin-Madison, Madison, WI, United States
| | - Brian W Patterson
- BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, Madison, WI, United States
| |
Collapse
|
21
|
Shegog R, Savas LS, Frost EL, Thormaehlen LC, Teague T, Steffy J, Healy CM, Shay LA, Preston S, Vernon SW. Adaptation and Formative Evaluation of Online Decision Support to Implement Evidence-Based Strategies to Increase HPV Vaccination Rates in Pediatric Clinics. Vaccines (Basel) 2023; 11:1270. [PMID: 37515085 PMCID: PMC10383429 DOI: 10.3390/vaccines11071270] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 07/11/2023] [Accepted: 07/19/2023] [Indexed: 07/30/2023] Open
Abstract
Human papilloma virus (HPV) vaccination rates remain below national goals in the United States despite the availability of evidence-based strategies to increase rates. The Adolescent Vaccination Program (AVP) is a multi-component intervention demonstrated to increase HPV vaccination rates in pediatric clinics through the implementation of six evidence-based strategies. The purpose of this study, conducted in Houston, Texas, from 2019-2021, was to adapt the AVP into an online decision support implementation tool for standalone use and to evaluate its feasibility for use in community clinics. Phase 1 (Adaptation) comprised clinic interviews (n = 23), literature review, Adolescent Vaccination Program Implementation Tool (AVP-IT) design documentation, and AVP-IT development. Phase 2 (Evaluation) comprised usability testing with healthcare providers (HCPs) (n = 5) and feasibility testing in community-based clinics (n = 2). AVP-IT decision support provides an Action Plan with tailored guidance on implementing six evidence-based strategies (immunization champions, assessment and feedback, continuing education, provider prompts, parent reminders, and parent education). HCPs rated the AVP-IT as acceptable, credible, easy, helpful, impactful, and appealing (≥80% agreement). They rated AVP-IT supported implementation as easier and more effective compared to usual practice (p ≤ 0.05). The clinic-based AVP-IT uses facilitated strategy implementation by 3-month follow-up. The AVP-IT promises accessible, utilitarian, and scalable decision support on strategies to increase HPV vaccination rates in pediatric clinic settings. Further feasibility and efficacy testing is indicated.
Collapse
Affiliation(s)
- Ross Shegog
- Department of Health Promotion and Behavioral Sciences, University of Texas School of Public Health, Houston, TX 77030, USA
| | - Lara S Savas
- Department of Health Promotion and Behavioral Sciences, University of Texas School of Public Health, Houston, TX 77030, USA
| | - Erica L Frost
- Department of Health Promotion and Behavioral Sciences, University of Texas School of Public Health, Houston, TX 77030, USA
| | - Laura C Thormaehlen
- Department of Health Promotion and Behavioral Sciences, University of Texas School of Public Health, Houston, TX 77030, USA
| | - Travis Teague
- Department of Health Promotion and Behavioral Sciences, University of Texas School of Public Health, Houston, TX 77030, USA
| | - Jack Steffy
- Department of Anthropology, Washington University in St. Louis, St. Louis, MO 63130, USA
| | - Catherine Mary Healy
- Department of Pediatrics, Infectious Diseases Section, Baylor College of Medicine, Houston, TX 77030, USA
| | - Laura Aubree Shay
- Department of Health Promotion and Behavioral Sciences, University of Texas School of Public Health, Houston, TX 77030, USA
| | - Sharice Preston
- Department of Health Promotion and Behavioral Sciences, University of Texas School of Public Health, Houston, TX 77030, USA
| | - Sally W Vernon
- Department of Health Promotion and Behavioral Sciences, University of Texas School of Public Health, Houston, TX 77030, USA
| |
Collapse
|
22
|
Kavya R, Kala A, Christopher J, Panda S, Lazarus B. DAAR: Drift Adaption and Alternatives Ranking approach for interpretable clinical decision support systems. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2023.104793] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/12/2023]
|
23
|
Wong R, Mehta T, Very B, Luo J, Feterik K, Crotty BH, Epstein JA, Fliotsos MJ, Kashyap N, Smith E, Woreta FA, Schwartz JI. Where Do Real-Time Prescription Benefit Tools Fit in the Landscape of High US Prescription Medication Costs? A Narrative Review. J Gen Intern Med 2023; 38:1038-1045. [PMID: 36441366 PMCID: PMC10039141 DOI: 10.1007/s11606-022-07945-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/01/2022] [Accepted: 11/08/2022] [Indexed: 11/29/2022]
Abstract
The problem of unaffordable prescription medications in the United States is complex and can result in poor patient adherence to therapy, worse clinical outcomes, and high costs to the healthcare system. While providers are aware of the financial burden of healthcare for patients, there is a lack of actionable price transparency at the point of prescribing. Real-time prescription benefit (RTPB) tools are new electronic clinical decision support tools that retrieve patient- and medication-specific out-of-pocket cost information and display it to clinicians at the point of prescribing. The rise in US healthcare costs has been a major driver for efforts to increase medication price transparency, and mandates from the Centers for Medicare & Medicaid Services for Medicare Part D sponsors to adopt RTPB tools may spur integration of such tools into electronic health records. Although multiple factors affect the implementation of RTPB tools, there is limited evidence on outcomes. Further research will be needed to understand the impact of RTPB tools on end results such as prescribing behavior, out-of-pocket medication costs for patients, and adherence to pharmacologic treatment. We review the terminology and concepts essential in understanding the landscape of RTPB tools, implementation considerations, barriers to adoption, and directions for future research that will be important to patients, prescribers, health systems, and insurers.
Collapse
Affiliation(s)
- Rachel Wong
- Department of Biomedical Informatics, Renaissance School of Medicine at Stony Brook, Stony Brook, USA.
| | - Tanvi Mehta
- Duke University School of Medicine, Durham, USA
| | - Bradley Very
- Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, USA
| | - Jing Luo
- Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, USA
| | - Kristian Feterik
- Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, USA
| | - Bradley H Crotty
- Froedtert & the Medical College of Wisconsin Health Network, Milwaukee, WI, USA
| | - Jeremy A Epstein
- Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Michael J Fliotsos
- Department of Ophthalmology and Visual Science, Yale School of Medicine, New Haven, CT, USA
| | - Nitu Kashyap
- Joint Data Analytics Team, Yale New Haven Hospital, New Haven, CT, USA
- Internal Medicine and Information Technology, Yale New Haven Health and Yale School of Medicine, New Haven, CT, USA
| | - Erika Smith
- Froedtert & the Medical College of Wisconsin Health Network, Milwaukee, WI, USA
| | - Fasika A Woreta
- Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Jeremy I Schwartz
- Section of General Internal Medicine, Yale University School of Medicine, New Haven, CT, USA
| |
Collapse
|
24
|
Harper J, Hunt T, Choudry M, Kapron AL, Cooney KA, Martin C, Ambrose J, O'Neil B. Clinician interest in clinical decision support for PSA-based prostate cancer screening. Urol Oncol 2023; 41:145.e17-145.e23. [PMID: 36610816 PMCID: PMC9992103 DOI: 10.1016/j.urolonc.2022.11.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2022] [Revised: 11/13/2022] [Accepted: 11/21/2022] [Indexed: 01/07/2023]
Abstract
OBJECTIVE To evaluate the interest of primary care clinicians in utilizing CDS for PSA screening. Evidence suggests that electronic clinical decision support (CDS) may decrease low-value prostate-specific antigen (PSA) testing. However, physician attitudes towards CDS for PSA screening are largely unknown. METHODS A survey was sent to 201 primary care clinicians, including both physicians and Advanced Practice Providers (APP), within a large academic health system. Eligible clinicians cared for male patients aged 40 to 80 years and ordered ≥5 PSA tests in the past year. Respondents were stratified into 3 groups, appropriate screeners, low-value screeners, or rare-screeners, based on responses to survey questions assessing PSA screening practices. The degree of interest in electronic CDS was determined via a composite Likert score comprising relevant survey items. RESULTS Survey response rate was 29% (59/201) consisting of 85% MD/DO and 15% APP respondents. All clinicians surveyed were interested in CDS (P < 0.001) without significant difference between screener groups. Clinicians agreed most uniformly that CDS be evidence-based. Clinicians disagreed on whether CDS would decrease professional discretion over patient decisions. CONCLUSIONS Primary care clinicians are interested in CDS for PSA screening regardless of their current screening practices. Prioritizing CDS features that clinicians value, such as ensuring CDS recommendations are evidence-based, may increase the likelihood of successful implementation, whereas perceived threat to autonomy may be a hinderance to utilization.
Collapse
Affiliation(s)
- Jonathan Harper
- Division of Urology, Department of Surgery, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT
| | - Trevor Hunt
- Division of Urology, Department of Surgery, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT; Department of Urology, University of Rochester Medical Center, Rochester, NY
| | - Mouneeb Choudry
- Division of Urology, Department of Surgery, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT
| | - Ashley L Kapron
- Utah Clinical & Translational Science Institute, University of Utah Health, Salt Lake City, UT
| | - Kathleen A Cooney
- Department of Medicine, Duke Cancer Institute, Duke University School of Medicine, Durham, NC
| | - Christopher Martin
- Division of Urology, Department of Surgery, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT
| | - Jacob Ambrose
- Division of Urology, Department of Surgery, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT
| | - Brock O'Neil
- Division of Urology, Department of Surgery, Huntsman Cancer Institute, University of Utah, Salt Lake City, UT.
| |
Collapse
|
25
|
APLUS: A Python library for usefulness simulations of machine learning models in healthcare. J Biomed Inform 2023; 139:104319. [PMID: 36791900 DOI: 10.1016/j.jbi.2023.104319] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 02/09/2023] [Accepted: 02/10/2023] [Indexed: 02/16/2023]
Abstract
Despite the creation of thousands of machine learning (ML) models, the promise of improving patient care with ML remains largely unrealized. Adoption into clinical practice is lagging, in large part due to disconnects between how ML practitioners evaluate models and what is required for their successful integration into care delivery. Models are just one component of care delivery workflows whose constraints determine clinicians' abilities to act on models' outputs. However, methods to evaluate the usefulness of models in the context of their corresponding workflows are currently limited. To bridge this gap we developed APLUS, a reusable framework for quantitatively assessing via simulation the utility gained from integrating a model into a clinical workflow. We describe the APLUS simulation engine and workflow specification language, and apply it to evaluate a novel ML-based screening pathway for detecting peripheral artery disease at Stanford Health Care.
Collapse
|
26
|
Cross DA, Adler-Milstein J, Holmgren AJ. Management Opportunities and Challenges After Achieving Widespread Health System Digitization. Adv Health Care Manag 2022; 21:67-87. [PMID: 36437617 DOI: 10.1108/s1474-823120220000021004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
The adoption of electronic health records (EHRs) and digitization of health data over the past decade is ushering in the next generation of digital health tools that leverage artificial intelligence (AI) to improve varied aspects of health system performance. The decade ahead is therefore shaping up to be one in which digital health becomes even more at the forefront of health care delivery - demanding the time, attention, and resources of health care leaders and frontline staff, and becoming inextricably linked with all dimensions of health care delivery. In this chapter, we look back and look ahead. There are substantive lessons learned from the first era of large-scale adoption of enterprise EHRs and ongoing challenges that organizations are wrestling with - particularly related to the tension between standardization and flexibility/customization of EHR systems and the processes they support. Managing this tension during efforts to implement and optimize enterprise systems is perhaps the core challenge of the past decade, and one that has impeded consistent realization of value from initial EHR investments. We describe these challenges, how they manifest, and organizational strategies to address them, with a specific focus on alignment with broader value-based care transformation. We then look ahead to the AI wave - the massive number of applications of AI to health care delivery, the expected benefits, the risks and challenges, and approaches that health systems can consider to realize the benefits while avoiding the risks.
Collapse
|
27
|
Chen W, O’Bryan CM, Gorham G, Howard K, Balasubramanya B, Coffey P, Abeyaratne A, Cass A. Barriers and enablers to implementing and using clinical decision support systems for chronic diseases: a qualitative systematic review and meta-aggregation. Implement Sci Commun 2022; 3:81. [PMID: 35902894 PMCID: PMC9330991 DOI: 10.1186/s43058-022-00326-x] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Accepted: 07/10/2022] [Indexed: 11/17/2022] Open
Abstract
Background Clinical decision support (CDS) is increasingly used to facilitate chronic disease care. Despite increased availability of electronic health records and the ongoing development of new CDS technologies, uptake of CDS into routine clinical settings is inconsistent. This qualitative systematic review seeks to synthesise healthcare provider experiences of CDS—exploring the barriers and enablers to implementing, using, evaluating, and sustaining chronic disease CDS systems. Methods A search was conducted in Medline, CINAHL, APA PsychInfo, EconLit, and Web of Science from 2011 to 2021. Primary research studies incorporating qualitative findings were included if they targeted healthcare providers and studied a relevant chronic disease CDS intervention. Relevant CDS interventions were electronic health record-based and addressed one or more of the following chronic diseases: cardiovascular disease, diabetes, chronic kidney disease, hypertension, and hypercholesterolaemia. Qualitative findings were synthesised using a meta-aggregative approach. Results Thirty-three primary research articles were included in this qualitative systematic review. Meta-aggregation of qualitative data revealed 177 findings and 29 categories, which were aggregated into 8 synthesised findings. The synthesised findings related to clinical context, user, external context, and technical factors affecting CDS uptake. Key barriers to uptake included CDS systems that were simplistic, had limited clinical applicability in multimorbidity, and integrated poorly into existing workflows. Enablers to successful CDS interventions included perceived usefulness in providing relevant clinical knowledge and structured chronic disease care; user confidence gained through training and post training follow-up; external contexts comprised of strong clinical champions, allocated personnel, and technical support; and CDS technical features that are both highly functional, and attractive. Conclusion This systematic review explored healthcare provider experiences, focussing on barriers and enablers to CDS use for chronic diseases. The results provide an evidence-base for designing, implementing, and sustaining future CDS systems. Based on the findings from this review, we highlight actionable steps for practice and future research. Trial registration PROSPERO CRD42020203716 Supplementary Information The online version contains supplementary material available at 10.1186/s43058-022-00326-x.
Collapse
|
28
|
Iglesias N, Juarez JM, Campos M. Business Process Model and Notation and openEHR Task Planning for Clinical Pathway Standards in Infections: Critical Analysis. J Med Internet Res 2022; 24:e29927. [PMID: 36107480 PMCID: PMC9523526 DOI: 10.2196/29927] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2021] [Revised: 08/23/2021] [Accepted: 03/31/2022] [Indexed: 11/13/2022] Open
Abstract
Background Clinical pathways (CPs) are usually expressed by means of workflow formalisms, providing health care personnel with an easy-to-understand, high-level conceptual model of medical steps in specific patient conditions, thereby improving overall health care process quality in clinical practice. From a standardized perspective, the business process model and notation (BPMN), a widely spread general-purpose process formalism, has been used for conceptual modeling in clinical domains, mainly because of its easy-to-use graphical notation, facilitating the common understanding and communication of the parties involved in health care. However, BPMN is not particularly oriented toward the peculiarities of complex clinical processes such as infection diagnosis and treatment, in which time plays a critical role, which is why much of the BPMN clinical-oriented research has revolved around how to extend the standard to address these special needs. The shift from an agnostic, general-purpose BPMN notation to a natively clinical-oriented notation such as openEHR Task Planning (TP) could constitute a major step toward clinical process improvement, enhancing the representation of CPs for infection treatment and other complex scenarios. Objective Our work aimed to analyze the suitability of a clinical-oriented formalism (TP) to successfully represent typical process patterns in infection treatment, identifying domain-specific improvements to the standard that could help enhance its modeling capabilities, thereby promoting the widespread adoption of CPs to improve medical practice and overall health care quality. Methods Our methodology consisted of 4 major steps: identification of key features of infection CPs through literature review, clinical guideline analysis, and BPMN extensions; analysis of the presence of key features in TP; modeling of relevant process patterns of catheter-related bloodstream infection as a case study; and analysis and proposal of extensions in view of the results. Results We were able to easily represent the same logic applied in the extended BPMN-based process models in our case study using out-of-the-box standard TP primitives. However, we identified possible improvements to the current version of TP to allow for simpler conceptual models of infection CPs and possibly of other complex clinical scenarios. Conclusions Our study showed that the clinical-oriented TP specification is able to successfully represent the most complex catheter-related bloodstream infection process patterns depicted in our case study and identified possible extensions that can help increase its adequacy for modeling infection CPs and possibly other complex clinical conditions.
Collapse
Affiliation(s)
- Natalia Iglesias
- Instituto de Investigación de Tecnologías de la Información y las Comunicaciones Orientadas, University of Murcia, Murcia, Spain
| | - Jose M Juarez
- Instituto de Investigación de Tecnologías de la Información y las Comunicaciones Orientadas, University of Murcia, Murcia, Spain
| | - Manuel Campos
- Instituto de Investigación de Tecnologías de la Información y las Comunicaciones Orientadas, University of Murcia, Murcia, Spain
- Instituto Murciano de Investigación Biosanitaria - Arrixaca, Murcia, Spain
| |
Collapse
|
29
|
Abouzahra M, Guenter D, Tan J. Exploring physicians’ continuous use of clinical decision support systems. EUR J INFORM SYST 2022. [DOI: 10.1080/0960085x.2022.2119172] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022]
Affiliation(s)
| | - Dale Guenter
- Department of Family Medicine, McMaster University
| | - Joseph Tan
- DeGroote School of Medicine, McMaster University
| |
Collapse
|
30
|
Gao E, Radparvar I, Dieu H, Ross MK. User Experience Design for Adoption of Asthma Clinical Decision Support Tools. Appl Clin Inform 2022; 13:971-982. [PMID: 36223869 PMCID: PMC9556170 DOI: 10.1055/s-0042-1757292] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Affiliation(s)
- Emily Gao
- University of California Los Angeles, Los Angeles, California, United States
| | - Ilana Radparvar
- University of California Los Angeles, Los Angeles, California, United States
| | - Holly Dieu
- Department of Pediatrics, University of California Los Angeles, David Geffen School of Medicine, Los Angeles, California, United States
| | - Mindy K Ross
- Department of Pediatrics, University of California Los Angeles, David Geffen School of Medicine, Los Angeles, California, United States
| |
Collapse
|
31
|
Henry KE, Kornfield R, Sridharan A, Linton RC, Groh C, Wang T, Wu A, Mutlu B, Saria S. Human-machine teaming is key to AI adoption: clinicians' experiences with a deployed machine learning system. NPJ Digit Med 2022; 5:97. [PMID: 35864312 PMCID: PMC9304371 DOI: 10.1038/s41746-022-00597-7] [Citation(s) in RCA: 50] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Accepted: 03/09/2022] [Indexed: 12/23/2022] Open
Abstract
While a growing number of machine learning (ML) systems have been deployed in clinical settings with the promise of improving patient care, many have struggled to gain adoption and realize this promise. Based on a qualitative analysis of coded interviews with clinicians who use an ML-based system for sepsis, we found that, rather than viewing the system as a surrogate for their clinical judgment, clinicians perceived themselves as partnering with the technology. Our findings suggest that, even without a deep understanding of machine learning, clinicians can build trust with an ML system through experience, expert endorsement and validation, and systems designed to accommodate clinicians’ autonomy and support them across their entire workflow.
Collapse
Affiliation(s)
- Katharine E Henry
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - Rachel Kornfield
- Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA.,Center for Behavioral Intervention Technologies, Northwestern University, Chicago, IL, USA
| | | | | | - Catherine Groh
- Department of Industrial Engineering, University of Wisconsin-Madison, Madison, WI, USA
| | - Tony Wang
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA
| | - Albert Wu
- Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Bilge Mutlu
- Department of Industrial Engineering, University of Wisconsin-Madison, Madison, WI, USA. .,Department of Computer Sciences, University of Wisconsin-Madison, Madison, WI, USA.
| | - Suchi Saria
- Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA. .,Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA. .,Department of Applied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD, USA. .,Bayesian Health, New York, NY, 10005, USA.
| |
Collapse
|
32
|
Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing. Nat Med 2022; 28:1447-1454. [PMID: 35864251 DOI: 10.1038/s41591-022-01895-z] [Citation(s) in RCA: 25] [Impact Index Per Article: 12.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2021] [Accepted: 06/08/2022] [Indexed: 01/04/2023]
Abstract
Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments at early time points, which is critical for improving sepsis outcomes. In view of the increasing use of such systems, better understanding of how they are adopted and used by healthcare providers is needed. Here, we analyzed provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System), which was deployed at five hospitals over a 2-year period. Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. Adjusting for patient presentation and severity, patients with sepsis whose alert was confirmed by a provider within 3 h had a 1.85-h (95% CI 1.66-2.00) reduction in median time to first antibiotic order compared to patients with sepsis whose alert was either dismissed, confirmed more than 3 h after the alert or never addressed in the system. Finally, we found that emergency department providers and providers who had previous interactions with an alert were more likely to interact with alerts, as well as to confirm alerts on retrospectively identified patients with sepsis. Beyond efforts to improve the performance of early warning systems, efforts to improve adoption are essential to their clinical impact and should focus on understanding providers' knowledge of, experience with and attitudes toward such systems.
Collapse
|
33
|
Fortmann J, Lutz M, Spreckelsen C. System for Context-Specific Visualization of Clinical Practice Guidelines (GuLiNav): Concept and Software Implementation. JMIR Form Res 2022; 6:e28013. [PMID: 35731571 PMCID: PMC9260532 DOI: 10.2196/28013] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2021] [Revised: 06/14/2021] [Accepted: 03/17/2022] [Indexed: 12/04/2022] Open
Abstract
Background Clinical decision support systems often adopt and operationalize existing clinical practice guidelines leading to higher guideline availability, increased guideline adherence, and data integration. Most of these systems use an internal state-based model of a clinical practice guideline to derive recommendations but do not provide the user with comprehensive insight into the model. Objective Here we present a novel approach based on dynamic guideline visualization that incorporates the individual patient’s current treatment context. Methods We derived multiple requirements to be fulfilled by such an enhanced guideline visualization. Using business process and model notation as the representation format for computer-interpretable guidelines, a combination of graph-based representation and logical inferences is adopted for guideline processing. A context-specific guideline visualization is inferred using a business rules engine. Results We implemented and piloted an algorithmic approach for guideline interpretation and processing. As a result of this interpretation, a context-specific guideline is derived and visualized. Our implementation can be used as a software library but also provides a representational state transfer interface. Spring, Camunda, and Drools served as the main frameworks for implementation. A formative usability evaluation of a demonstrator tool that uses the visualization yielded high acceptance among clinicians. Conclusions The novel guideline processing and visualization concept proved to be technically feasible. The approach addresses known problems of guideline-based clinical decision support systems. Further research is necessary to evaluate the applicability of the approach in specific medical use cases.
Collapse
Affiliation(s)
- Jonas Fortmann
- Institute of Medical Informatics, Medical Faculty, Rheinisch-Westfälische Technische Hochschule Aachen University, Aachen, Germany
- Smart Medical Technology for Healthcare Consortium of the German Medical Informatics Initiative, Leipzig, Germany
| | - Marlene Lutz
- Institute of Medical Informatics, Medical Faculty, Rheinisch-Westfälische Technische Hochschule Aachen University, Aachen, Germany
| | - Cord Spreckelsen
- Smart Medical Technology for Healthcare Consortium of the German Medical Informatics Initiative, Leipzig, Germany
- Institute of Medical Statistics, Computer and Data Sciences, Jena University Hospital, Jena, Germany
| |
Collapse
|
34
|
Allen KS, Danielson EC, Downs SM, Mazurenko O, Diiulio J, Salloum RG, Mamlin BW, Harle CA. Evaluating a Prototype Clinical Decision Support Tool for Chronic Pain Treatment in Primary Care. Appl Clin Inform 2022; 13:602-611. [PMID: 35649500 DOI: 10.1055/s-0042-1749332] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022] Open
Abstract
OBJECTIVES The Chronic Pain Treatment Tracker (Tx Tracker) is a prototype decision support tool to aid primary care clinicians when caring for patients with chronic noncancer pain. This study evaluated clinicians' perceived utility of Tx Tracker in meeting information needs and identifying treatment options, and preferences for visual design. METHODS We conducted 12 semi-structured interviews with primary care clinicians from four health systems in Indiana. The interviews were conducted in two waves, with prototype and interview guide revisions after the first six interviews. The interviews included exploration of Tx Tracker using a think-aloud approach and a clinical scenario. Clinicians were presented with a patient scenario and asked to use Tx Tracker to make a treatment recommendation. Last, participants answered several evaluation questions. Detailed field notes were collected, coded, and thematically analyzed by four analysts. RESULTS We identified several themes: the need for clinicians to be presented with a comprehensive patient history, the usefulness of Tx Tracker in patient discussions about treatment planning, potential usefulness of Tx Tracker for patients with high uncertainty or risk, potential usefulness of Tx Tracker in aggregating scattered information, variability in expectations about workflows, skepticism about underlying electronic health record data quality, interest in using Tx Tracker to annotate or update information, interest in using Tx Tracker to translate information to clinical action, desire for interface with visual cues for risks, warnings, or treatment options, and desire for interactive functionality. CONCLUSION Tools like Tx Tracker, by aggregating key information about past, current, and potential future treatments, may help clinicians collaborate with their patients in choosing the best pain treatments. Still, the use and usefulness of Tx Tracker likely relies on continued improvement of its functionality, accurate and complete underlying data, and tailored integration with varying workflows, care team roles, and user preferences.
Collapse
Affiliation(s)
- Katie S Allen
- Health Policy and Management, Richard M. Fairbanks School of Public Health, IUPUI, Indianapolis, Indiana, United States.,Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, United States
| | - Elizabeth C Danielson
- Center for Education in Health Sciences, Northwestern University Feinberg School of Medicine, Chicago, Illinois, United States
| | - Sarah M Downs
- Division of Internal Medicine, Indiana University School of Medicine, Indianapolis, Indiana, United States
| | - Olena Mazurenko
- Health Policy and Management, Richard M. Fairbanks School of Public Health, IUPUI, Indianapolis, Indiana, United States
| | - Julie Diiulio
- Health Outcomes and Biomedical Informatics, Applied Decision Science, LLC, Dayton, Ohio, United States
| | | | - Burke W Mamlin
- Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, United States.,Division of Internal Medicine, Indiana University School of Medicine, Indianapolis, Indiana, United States
| | - Christopher A Harle
- Center for Biomedical Informatics, Regenstrief Institute, Inc., Indianapolis, Indiana, United States.,University of Florida, Gainesville, Florida, United States
| |
Collapse
|
35
|
Chaparro JD, Beus JM, Dziorny AC, Hagedorn PA, Hernandez S, Kandaswamy S, Kirkendall ES, McCoy AB, Muthu N, Orenstein EW. Clinical Decision Support Stewardship: Best Practices and Techniques to Monitor and Improve Interruptive Alerts. Appl Clin Inform 2022; 13:560-568. [PMID: 35613913 PMCID: PMC9132737 DOI: 10.1055/s-0042-1748856] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022] Open
Abstract
Interruptive clinical decision support systems, both within and outside of electronic health records, are a resource that should be used sparingly and monitored closely. Excessive use of interruptive alerting can quickly lead to alert fatigue and decreased effectiveness and ignoring of alerts. In this review, we discuss the evidence for effective alert stewardship as well as practices and methods we have found useful to assess interruptive alert burden, reduce excessive firings, optimize alert effectiveness, and establish quality governance at our institutions. We also discuss the importance of a holistic view of the alerting ecosystem beyond the electronic health record.
Collapse
Affiliation(s)
- Juan D Chaparro
- Division of Clinical Informatics, Nationwide Children's Hospital, Columbus, Ohio, United States.,Departments of Pediatrics and Biomedical Informatics, The Ohio State University College of Medicine, Columbus, Ohio, United States
| | - Jonathan M Beus
- Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia, United States.,Children's Healthcare of Atlanta, Atlanta, Georgia, United States
| | - Adam C Dziorny
- Department of Pediatrics, University of Rochester School of Medicine, Rochester, New York, United States
| | - Philip A Hagedorn
- Department of Pediatrics, University of Cincinnati, College of Medicine, Cincinnati, Ohio, United States.,Division of Hospital Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, United States
| | - Sean Hernandez
- Center for Healthcare Innovation, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States.,Department of General Internal Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States
| | - Swaminathan Kandaswamy
- Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia, United States
| | - Eric S Kirkendall
- Center for Healthcare Innovation, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States.,Department of Pediatrics, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States.,Center for Biomedical Informatics, Wake Forest School of Medicine, Winston-Salem NC, United States
| | - Allison B McCoy
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States
| | - Naveen Muthu
- Department of Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, United States.,Department of Biomedical and Health Informatics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, United States
| | - Evan W Orenstein
- Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia, United States.,Children's Healthcare of Atlanta, Atlanta, Georgia, United States
| |
Collapse
|
36
|
Rastogi R, Lattimore CM, Mehaffey JH, Turrentine FE, Maitland HS, Zaydfudim VM. Electronic Health Record Risk-Stratification Tool Reduces Venous Thromboembolism Events in Surgical Patients. Surg Open Sci 2022; 9:34-40. [PMID: 35620709 PMCID: PMC9127397 DOI: 10.1016/j.sopen.2022.04.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2022] [Accepted: 04/13/2022] [Indexed: 11/28/2022] Open
Abstract
Background Venous thromboembolism is a preventable cause of morbidity and mortality after surgery. To ensure that patients receive appropriate venous thromboembolism chemoprophylaxis, a nonmandatory risk-stratification tool based on patient clinical condition was implemented through the electronic health record to stratify patient risk and recommend chemoprophylaxis. We hypothesized that implementing this tool would reduce postoperative venous thromboembolism events in general surgery as well as across all surgical services. Methods All adult patients undergoing inpatient surgical operations (January 2012–December 2019) at a single quaternary care center and Level 1 trauma center were abstracted from institutional electronic health record database and stratified into patients admitted before and after venous thromboembolism risk-stratification tool implementation. Bivariable analyses compared venous thromboembolism chemoprophylaxis prescription and venous thromboembolism events with implementation and screening among all surgical patients as well as in general surgery patient subset. Results A total of 64,377 adults underwent operations: 27,819 preimplementation and 36,558 postimplementation. A significant reduction in venous thromboembolism events occurred from pre- to post-tool implementation for all cases (0.77% vs 0.47%, P < .001). General surgery patients (n = 15,723) had a significant increase in chemoprophylaxis prescription (81.9% vs 86.0%, P < .001) and a significant reduction in venous thromboembolism events (1.41% vs 0.59%, P < .001). After tool implementation, use of extended postdischarge chemoprophylaxis was greater among general surgery patient subset than the entire patient cohort (46.7% vs 29.6%, P < .001). Conclusion The integration of a nonmandatory electronic health record risk-stratification tool was associated with a significant reduction in venous thromboembolism events. Extended chemoprophylaxis was prescribed in nearly half of general surgery patients at very high risk for postdischarge events. Implementing an electronic VTE risk-stratification tool reduced surgical VTE events. Even as a nonmandatory tool, risk stratification led to overall fewer VTE events. Postoperative VTE events were reduced by 39% after the tool was integrated in EHR. With the tool, general surgery had 58% less VTE events and improved prophylaxis use.
Collapse
Affiliation(s)
- Radhika Rastogi
- Department of Surgery, University of Virginia, Charlottesville, VA 22908
| | - Courtney M. Lattimore
- Department of Surgery, University of Virginia, Charlottesville, VA 22908
- Surgical Outcomes Research Center, University of Virginia, Charlottesville, VA 22908
| | - J. Hunter Mehaffey
- Department of Surgery, University of Virginia, Charlottesville, VA 22908
| | - Florence E. Turrentine
- Department of Surgery, University of Virginia, Charlottesville, VA 22908
- Surgical Outcomes Research Center, University of Virginia, Charlottesville, VA 22908
| | - Hillary S. Maitland
- Department of Medicine, Hematology/Oncology, University of Virginia, Charlottesville, VA 22908
| | - Victor M. Zaydfudim
- Department of Surgery, University of Virginia, Charlottesville, VA 22908
- Surgical Outcomes Research Center, University of Virginia, Charlottesville, VA 22908
- Corresponding author at: Division of Surgical Oncology, Department of Surgery, PO Box 800709, Charlottesville, VA, 22908-0709. Tel.: + 1-434-924-2839; fax: + 1 434-982-4778. @vz_surgery
| |
Collapse
|
37
|
Kim MS, Ryu JM, Choi BK. Development and Effectiveness of a Clinical Decision Support System for Pressure Ulcer Prevention Care Using Machine Learning: A Quasi-experimental Study. Comput Inform Nurs 2022; 41:00024665-900000000-99171. [PMID: 35266901 DOI: 10.1097/cin.0000000000000899] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
This study was conducted to develop and evaluate the effectiveness of a clinical decision support system for pressure ulcer prevention on clinical (performance, visual discrimination ability, and decision-making ability) and cognitive (knowledge and attitude) workflow. After developing a clinical decision support system using machine learning, a quasi-experimental study was used. Data were collected between January and April 2020. Forty-nine RNs who met the inclusion criteria and worked at seven tertiary and five secondary hospitals participated. A clinical decision support system was provided to the intervention group during the same period. Differences in outcome variables between the two groups were analyzed using t tests. The level of pressure ulcer prevention nursing performance and visual differentiation ability of skin pressure and oral mucosa pressure ulcer showed significantly greater improvement in the experimental group compared with the control group, whereas clinical decision making did not differ significantly. A clinical decision support system using machine learning was partially successful in performance of skin pressure ulcer prevention, attitude, and visual differentiation ability for skin and oral mucosa pressure ulcer prevention. These findings indicated that a clinical decision support system using machine learning needs to be implemented for pressure ulcer prevention.
Collapse
Affiliation(s)
- Myoung Soo Kim
- Author Affiliations: Department of Nursing, Pukyong National University (Dr Kim); Department of Nursing, Busan Institute of Science and Technology (Dr Ryu); and Department of Neurosurgery, College of Medicine, Pusan National University (Dr Choi), Busan, South Korea
| | | | | |
Collapse
|
38
|
Ebben KCWJ, Hendriks MP, Markus L, Kos M, De Hingh IHJT, Oddens JR, Rothbarth J, De wilt H, Strobbe LJA, Bessems M, Mellema CT, Siesling S, Verbeek XAAM. Using Guideline-Based Clinical Decision Support in Oncological Multidisciplinary Team Meetings: A Prospective, Multicenter Concordance Study. Int J Qual Health Care 2022; 34:6523785. [PMID: 35137091 PMCID: PMC8934031 DOI: 10.1093/intqhc/mzac007] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2021] [Revised: 01/21/2022] [Accepted: 02/07/2022] [Indexed: 11/14/2022] Open
Abstract
Background Multidisciplinary team meetings formulate guideline-based individual treatment plans based on patient and disease characteristics and motivate reasons for deviation. Clinical decision trees could support multidisciplinary teams to adhere more accurately to guidelines. Every clinical decision tree is tailored to a specific decision moment in a care pathway and is composed of patient and disease characteristics leading to a guideline recommendation. Objective This study investigated (1) the concordance between multidisciplinary team and clinical decision tree recommendations and (2) the completeness of patient and disease characteristics available during multidisciplinary team meetings to apply clinical decision trees such that it results in a guideline recommendation. Methods This prospective, multicenter, observational concordance study evaluated 17 selected clinical decision trees, based on the prevailing Dutch guidelines for breast, colorectal and prostate cancers. In cases with sufficient data, concordance between multidisciplinary team and clinical decision tree recommendations was classified as concordant, conditional concordant (multidisciplinary team specified a prerequisite for the recommendation) and non-concordant. Results Fifty-nine multidisciplinary team meetings were attended in 8 different hospitals, and 355 cases were included. For 296 cases (83.4%), all patient data were available for providing an unconditional clinical decision tree recommendation. In 59 cases (16.6%), insufficient data were available resulting in provisional clinical decision tree recommendations. From the 296 successfully generated clinical decision tree recommendations, the multidisciplinary team recommendations were concordant in 249 (84.1%) cases, conditional concordant in 24 (8.1%) cases and non-concordant in 23 (7.8%) cases of which in 7 (2.4%) cases the reason for deviation from the clinical decision tree generated guideline recommendation was not motivated. Conclusion The observed concordance of recommendations between multidisciplinary teams and clinical decision trees and data completeness during multidisciplinary team meetings in this study indicate a potential role for implementation of clinical decision trees to support multidisciplinary team decision-making.
Collapse
Affiliation(s)
- Kees C W J Ebben
- Address reprint requests to: Kees C.W.J. Ebben, Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL), Godebaldkwartier 419, Utrecht 3511 DT, The Netherlands. Tel: +31 6 1179 0131; E-mail:
| | | | - Lieke Markus
- Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL), Utrecht, The Netherlands
| | - Milan Kos
- Department of Medical Oncology, Amsterdam University Medical Centers, University of Amsterdam, Meibergdreef 9, Amsterdam 1105AZ, Noord-Holland, The Netherlands
| | - Ignace H J T De Hingh
- Department of Surgical Oncology, Catharina Hospital, Michelangelolaan 2, Eindhoven 5623EJ, The Netherlands
| | - Jorg R Oddens
- Department of Urology, Amsterdam University Medical Centers, University of Amsterdam, Meibergdreef 9, Amsterdam 1105AZ, Noord-Holland, The Netherlands
| | - Joost Rothbarth
- Department of Surgical Oncology and Gastrointestinal Surgery, Erasmus MC Cancer Institute, Doctor Molewaterplein 40, Rotterdam 3015GD, The Netherlands
| | - Hans De wilt
- Department of Surgical Oncology, Radboud University Medical Center, Geert Grooteplein Zuid 10, Nijmegen 6525GA, The Netherlands
| | - Luc J A Strobbe
- Department of Surgical Oncology, Canisius Wilhelmina Hospital, Weg door Jonkerbos 100, Nijmegen 6532SZ, The Netherlands
| | - Maud Bessems
- Department of Surgical Oncology, Jeroen Bosch Hospital, Henri Dunantstraat 1, ‘s-Hertogenbosch 5223 GZ, The Netherlands
| | - Carsten T Mellema
- Department of Urology, Spaarne Hospital, Boerhavelaan 22, Haarlem 2035RC, The Netherlands
| | - Sabine Siesling
- Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL), Utrecht, The Netherlands
- Department of Health Technology and Services Research, Technical Medical Center, University of Twente, Hallenweg 5, Enschede 7522NH, Overijssel, The Netherlands
| | - Xander A A M Verbeek
- Department of Research and Development, Netherlands Comprehensive Cancer Organization (IKNL), Utrecht, The Netherlands
| |
Collapse
|
39
|
Ekeberg OM, Pedersen SJ, Natvig B, Brox JI, Biringer EK, Endresen Reme S, Engebretsen KB, Joranger P, Mdala I, Juel NG. Making should er p ain simple in g eneral p ractice: implementing an evidence-based guideline for shoulder pain, protocol for a hybrid design stepped-wedge cluster randomised study (EASIER study). BMJ Open 2022; 12:e051656. [PMID: 34996788 PMCID: PMC8744118 DOI: 10.1136/bmjopen-2021-051656] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/04/2022] Open
Abstract
INTRODUCTION Research suggests that current care for shoulder pain is not in line with the best available evidence. This project aims to assess the effectiveness, cost-effectiveness and the implementation of an evidence-based guideline for shoulder pain in general practice in Norway. METHODS AND ANALYSIS A stepped-wedge, cluster-randomised trial with a hybrid design assessing clinical effectiveness, cost-effectiveness and the effect of the implementation strategy of a guideline-based intervention in general practice. We will recruit at least 36 general practitioners (GPs) and randomise the time of cross-over from treatment as usual to the implemented intervention. The intervention includes an educational outreach visit to the GPs, a computerised decision tool for GPs and a self-management application for patients. We will measure outcomes at patient and GP levels using self-report questionnaires, focus group interviews and register based data. The primary outcome measure is the patient-reported Shoulder Pain and Disability Index measured at 12 weeks. Secondary outcomes include the EuroQol Quality of Life Measure (EQ5D-5L), direct and indirect costs, patient's global perceived effect of treatment outcome, Pain Self-Efficacy and Brief Illness Perception Questionnaire. We will evaluate the implementation process with focus on adherence to guideline treatment. We will do a cost-minimisation analysis based on direct and selected indirect costs and a cost-utility analysis based on EQ5D-5L. We will use mixed effect models to analyse primary and secondary outcomes. ETHICS AND DISSEMINATION Ethics approval was granted by the Regional Committee for Medical and Health Research Ethics-South East Norway (ref. no: 2019/104). Trial results will be submitted for publication in a peer-reviewed medical journal in accordance with Consolidated Standards of Reporting Trials. TRIAL REGISTRATION NUMBER NCT04806191.
Collapse
Affiliation(s)
- Ole Marius Ekeberg
- Department of Research and innovation, Helse Fonna HF, Haugesund, Norway
| | | | - Bård Natvig
- Department of General Practice, University of Oslo, Oslo, Norway
| | - Jens Ivar Brox
- Department of Physical Medicine and rehabilitation, Oslo University Hospital, Oslo, Norway
- Medical Faculty, Institute for Clinical Medicine, University of Oslo, Oslo, Norway
| | | | | | - Kaia Beck Engebretsen
- Department of Physical Medicine and rehabilitation, Oslo University Hospital, Oslo, Norway
| | - Pål Joranger
- Department of Nursing and Health Promotion, Oslo Metropolitan University, Oslo, Norway
| | - Ibrahimu Mdala
- Department of General Practice, University of Oslo, Oslo, Norway
| | - Niels Gunnar Juel
- Department of General Practice, University of Oslo, Oslo, Norway
- Department of Physical Medicine and rehabilitation, Oslo University Hospital, Oslo, Norway
| |
Collapse
|
40
|
Chen Y, Harris S, Rogers Y, Ahmad T, Asselbergs FW. OUP accepted manuscript. Eur Heart J 2022; 43:1296-1306. [PMID: 35139182 PMCID: PMC8971005 DOI: 10.1093/eurheartj/ehac030] [Citation(s) in RCA: 12] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/18/2021] [Revised: 01/11/2022] [Accepted: 01/17/2022] [Indexed: 12/15/2022] Open
Abstract
The increasing volume and richness of healthcare data collected during routine clinical
practice have not yet translated into significant numbers of actionable insights that have
systematically improved patient outcomes. An evidence-practice gap continues to exist in
healthcare. We contest that this gap can be reduced by assessing the use of nudge theory
as part of clinical decision support systems (CDSS). Deploying nudges to modify clinician
behaviour and improve adherence to guideline-directed therapy represents an underused tool
in bridging the evidence-practice gap. In conjunction with electronic health records
(EHRs) and newer devices including artificial intelligence algorithms that are
increasingly integrated within learning health systems, nudges such as CDSS alerts should
be iteratively tested for all stakeholders involved in health decision-making: clinicians,
researchers, and patients alike. Not only could they improve the implementation of known
evidence, but the true value of nudging could lie in areas where traditional randomized
controlled trials are lacking, and where clinical equipoise and variation dominate. The
opportunity to test CDSS nudge alerts and their ability to standardize behaviour in the
face of uncertainty may generate novel insights and improve patient outcomes in areas of
clinical practice currently without a robust evidence base.
Collapse
Affiliation(s)
- Yang Chen
- Institute of Health Informatics, University College London,
222 Euston Road, London NW1 2DA, UK
- Clinical Research Informatics Unit, University College London Hospitals NHS
Healthcare Trust, London, UK
- Barts Heart Centre, St Bartholomew’s Hospital, London,
UK
| | - Steve Harris
- Institute of Health Informatics, University College London,
222 Euston Road, London NW1 2DA, UK
| | - Yvonne Rogers
- UCL Interaction Centre, University College London, London,
UK
| | - Tariq Ahmad
- Section of Cardiovascular Medicine, School of Medicine, Yale
University, New Haven, CT, USA
| | | |
Collapse
|
41
|
Brunner J, Farmer MM, Bean-Mayberry B, Chanfreau-Coffinier C, Than CT, Hamilton AB, Finley EP. Implementing clinical decision support for reducing women Veterans' cardiovascular risk in VA: A mixed-method, longitudinal study of context, adaptation, and uptake. FRONTIERS IN HEALTH SERVICES 2022; 2:946802. [PMID: 36925876 PMCID: PMC10012802 DOI: 10.3389/frhs.2022.946802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Accepted: 08/29/2022] [Indexed: 11/13/2022]
Abstract
Evaluations of clinical decision support (CDS) implementation often struggle to measure and explain heterogeneity in uptake over time and across settings, and to account for the impact of context and adaptation on implementation success. In 2017-2020, the EMPOWER QUERI implemented a cardiovascular toolkit using a computerized template aimed at reducing women Veterans' cardiovascular risk across five Veterans Healthcare Administration (VA) sites, using an enhanced Replicating Effective Programs (REP) implementation approach. In this study, we used longitudinal joint displays of qualitative and quantitative findings to explore (1) how contextual factors emerged across sites, (2) how the template and implementation strategies were adapted in response to contextual factors, and (3) how contextual factors and adaptations coincided with template uptake across sites and over time. We identified site structure, staffing changes, relational authority of champions, and external leadership as important contextual factors. These factors gave rise to adaptations such as splitting the template into multiple parts, pairing the template with a computerized reminder, conducting academic detailing, creating cheat sheets, and using small-scale pilot testing. All five sites exhibited variability in utilization over the months of implementation, though later sites exhibited higher template utilization immediately post-launch, possibly reflecting a "preloading" of adaptations from previous sites. These findings underscore the importance of adaptive approaches to implementation, with intentional shifts in intervention and strategy to meet the needs of individual sites, as well as the value of integrating mixed-method data sources in conducting longitudinal evaluation of implementation efforts.
Collapse
Affiliation(s)
- Julian Brunner
- Center for the Study of Healthcare Innovation, Implementation, and Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA, United States
| | - Melissa M Farmer
- Center for the Study of Healthcare Innovation, Implementation, and Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA, United States
| | - Bevanne Bean-Mayberry
- Center for the Study of Healthcare Innovation, Implementation, and Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA, United States.,Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, United States
| | | | - Claire T Than
- Center for the Study of Healthcare Innovation, Implementation, and Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA, United States
| | - Alison B Hamilton
- Center for the Study of Healthcare Innovation, Implementation, and Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA, United States.,Department of Medicine, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, United States
| | - Erin P Finley
- Center for the Study of Healthcare Innovation, Implementation, and Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA, United States.,Long School of Medicine, University of Texas Health Science Center San Antonio, San Antonio, TX, United States
| |
Collapse
|
42
|
Sassoon I, Kökciyan N, Modgil S, Parsons S. Argumentation schemes for clinical decision support. ARGUMENT & COMPUTATION 2021. [DOI: 10.3233/aac-200550] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
This paper demonstrates how argumentation schemes can be used in decision support systems that help clinicians in making treatment decisions. The work builds on the use of computational argumentation, a rigorous approach to reasoning with complex data that places strong emphasis on being able to justify and explain the decisions that are recommended. The main contribution of the paper is to present a novel set of specialised argumentation schemes that can be used in the context of a clinical decision support system to assist in reasoning about what treatments to offer. These schemes provide a mechanism for capturing clinical reasoning in such a way that it can be handled by the formal reasoning mechanisms of formal argumentation. The paper describes how the integration between argumentation schemes and formal argumentation may be carried out, sketches how this is achieved by an implementation that we have created and illustrates the overall process on a small set of case studies.
Collapse
Affiliation(s)
- Isabel Sassoon
- Department of Computer Science, Brunel University London, UK. E-mail:
| | - Nadin Kökciyan
- School of Informatics, University of Edinburgh, UK. E-mail:
| | - Sanjay Modgil
- Department of Informatics, King’s College London, UK. E-mail:
| | - Simon Parsons
- Department of Computer Science, University of Lincoln, UK. E-mail:
| |
Collapse
|
43
|
Salwei ME, Carayon P, Hoonakker PLT, Hundt AS, Wiegmann D, Pulia M, Patterson BW. Workflow integration analysis of a human factors-based clinical decision support in the emergency department. APPLIED ERGONOMICS 2021; 97:103498. [PMID: 34182430 PMCID: PMC8474147 DOI: 10.1016/j.apergo.2021.103498] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/01/2020] [Revised: 06/03/2021] [Accepted: 06/07/2021] [Indexed: 05/27/2023]
Abstract
Numerous challenges with the implementation, acceptance, and use of health IT are related to poor usability and a lack of integration of the technologies into clinical workflow, and have, therefore, limited the potential of these technologies to improve patient safety. We propose a definition and conceptual model of health IT workflow integration. Using interviews of 12 emergency department (ED) physicians, we identify 134 excerpts of barriers and facilitators to workflow integration of a human factors (HF)-based clinical decision support (CDS) implemented in the ED. Using data on these 134 barriers and facilitators, we distinguish 25 components of workflow integration of the CDS, which are described according to four dimensions of workflow integration: time, flow, scope of patient journey, and level. The proposed definition and conceptual model of workflow integration can be used to inform health IT design; this is the purpose of the proposed checklist that can help to ensure consideration of workflow integration during the development of health IT.
Collapse
Affiliation(s)
- Megan E Salwei
- Department of Biomedical Informatics, Vanderbilt University Medical Center, USA; Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, USA.
| | - Pascale Carayon
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, USA; Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, USA
| | - Peter L T Hoonakker
- Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, USA
| | - Ann Schoofs Hundt
- Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, USA
| | - Douglas Wiegmann
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, USA; Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, USA
| | - Michael Pulia
- Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, USA; BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, USA
| | - Brian W Patterson
- Department of Industrial and Systems Engineering, University of Wisconsin-Madison, USA; Wisconsin Institute for Healthcare Systems Engineering, University of Wisconsin-Madison, USA; BerbeeWalsh Department of Emergency Medicine, University of Wisconsin-Madison, USA
| |
Collapse
|
44
|
Wolff J, Pauling J, Keck A, Baumbach J. Success Factors of Artificial Intelligence Implementation in Healthcare. Front Digit Health 2021; 3:594971. [PMID: 34713083 PMCID: PMC8521923 DOI: 10.3389/fdgth.2021.594971] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2020] [Accepted: 04/22/2021] [Indexed: 01/31/2023] Open
Abstract
Background: Artificial Intelligence (AI) in healthcare has demonstrated high efficiency in academic research, while only few, and predominantly small, real-world AI applications exist in the preventive, diagnostic and therapeutic contexts. Our identification and analysis of success factors for the implementation of AI aims to close the gap between recent years' significant academic AI advancements and the comparably low level of practical application in healthcare. Methods: A literature and real life cases analysis was conducted in Scopus and OpacPlus as well as the Google advanced search database. The according search queries have been defined based on success factor categories for AI implementation derived from a prior World Health Organization survey about barriers of adoption of Big Data within 125 countries. The eligible publications and real life cases were identified through a catalog of in- and exclusion criteria focused on concrete AI application cases. These were then analyzed to deduct and discuss success factors that facilitate or inhibit a broad-scale implementation of AI in healthcare. Results: The analysis revealed three categories of success factors, namely (1) policy setting, (2) technological implementation, and (3) medical and economic impact measurement. For each of them a set of recommendations has been deducted: First, a risk adjusted policy frame is required that distinguishes between precautionary and permissionless principles, and differentiates among accountability, liability, and culpability. Second, a "privacy by design" centered technology infrastructure shall be applied that enables practical and legally compliant data access. Third, the medical and economic impact need to be quantified, e.g., through the measurement of quality-adjusted life years while applying the CHEERS and PRISMA reporting criteria. Conclusions: Private and public institutions can already today leverage AI implementation based on the identified results and thus drive the translation from scientific development to real world application. Additional success factors could include trust-building measures, data categorization guidelines, and risk level assessments and as the success factors are interlinked, future research should elaborate on their optimal interaction to utilize the full potential of AI in real world application.
Collapse
Affiliation(s)
- Justus Wolff
- Chair of Experimental Bioinformatics, TUM School of Life Sciences, Technical University of Munich, Munich, Germany.,Syte - Strategy Institute for Digital Health, Hamburg, Germany
| | - Josch Pauling
- LipiTUM, Chair of Experimental Bioinformatics, TUM School of Life Sciences, Technical University of Munich, Munich, Germany
| | - Andreas Keck
- Syte - Strategy Institute for Digital Health, Hamburg, Germany
| | - Jan Baumbach
- Chair of Experimental Bioinformatics, TUM School of Life Sciences, Technical University of Munich, Munich, Germany.,Chair of Computational Systems Biology, University of Hamburg, Hamburg, Germany
| |
Collapse
|
45
|
Sperl-Hillen JM, Crain AL, Chumba L, Ekstrom HL, Appana D, Kopski KM, Wetmore JB, Wheeler J, Ishani A, O'Connor PJ. Pragmatic clinic randomized trial to improve chronic kidney disease care: Design and adaptation due to COVID disruptions. Contemp Clin Trials 2021; 109:106501. [PMID: 34271175 PMCID: PMC8276567 DOI: 10.1016/j.cct.2021.106501] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2021] [Revised: 06/24/2021] [Accepted: 06/30/2021] [Indexed: 11/08/2022]
Abstract
BACKGROUND We describe a clinic-randomized trial to improve chronic kidney disease (CKD) care through a CKD-clinical decision support (CKD-CDS) intervention in primary care clinics and the challenges we encountered due to COVID-19 care disruption. METHODS/DESIGN Primary care clinics (N = 32) were randomized to usual care (UC) or to CKD-CDS. Between April 17, 2019 and March 14, 2020, more than 7000 patients had accrued for analysis by meeting study-eligibility criteria at an index office visit: age 18-75, laboratory criteria for stage 3 or 4 CKD (eGFR 15-59 mL/min/1.73 m2), and one or more opportunities algorithmically identified to improve CKD care such as blood pressure (BP) or glucose control, angiotensin converting enzyme inhibitor (ACEI) or angiotensin receptor blocker (ARB) use, discontinuance of a nonsteroidal anti-inflammatory drug (NSAID), or nephrology referral. At CKD-CDS clinics, CDS provided individualized treatment suggestions that were printed for patients and clinicians at the start of office encounters and were viewable within the electronic health record. By initial design, the impact of the CKD-CDS intervention on care gaps was to be assessed 12 months after the index date, but COVID-19 caused major disruptions to care delivery during the intervention period. In response to disruptions, the intervention was temporarily suspended while we expanded CDS use for telehealth encounters and programmed new criteria for displaying the CKD-CDS to intervention patients due to clinic closures and scheduling changes. DISCUSSION We describe a NIH-funded pragmatic trial of web-based EHR-integrated CKD-CDS and modifications necessary mid-study to complete the study as intended in the face of COVID-19 pandemic challenges.
Collapse
Affiliation(s)
| | - A Lauren Crain
- HealthPartners Institute, Minneapolis, MN, United States of America
| | - Lilian Chumba
- HealthPartners Institute, Minneapolis, MN, United States of America
| | - Heidi L Ekstrom
- HealthPartners Institute, Minneapolis, MN, United States of America
| | - Deepika Appana
- HealthPartners Institute, Minneapolis, MN, United States of America
| | - Kristen M Kopski
- Park Nicollet Medical Group, Minneapolis, MN, United States of America
| | - James B Wetmore
- Division of Nephrology, Hennepin County Medical Center, Minneapolis, MN, United States of America
| | - James Wheeler
- Park Nicollet Medical Group, Minneapolis, MN, United States of America
| | - Areef Ishani
- Minneapolis Veterans Affairs Health Care System and the University of Minnesota, Minneapolis, MN, United States of America
| | | |
Collapse
|
46
|
Taber P, Radloff C, Del Fiol G, Staes C, Kawamoto K. New Standards for Clinical Decision Support: A Survey of The State of Implementation. Yearb Med Inform 2021; 30:159-171. [PMID: 34479387 PMCID: PMC8416232 DOI: 10.1055/s-0041-1726502] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/07/2023] Open
Abstract
Objectives:
To review the current state of research on designing and implementing clinical decision support (CDS) using four current interoperability standards: Fast Healthcare Interoperability Resources (FHIR); Substitutable Medical Applications and Reusable Technologies (SMART); Clinical Quality Language (CQL); and CDS Hooks.
Methods:
We conducted a review of original studies describing development of specific CDS tools or infrastructures using one of the four targeted standards, regardless of implementation stage. Citations published any time before the literature search was executed on October 21, 2020 were retrieved from PubMed. Two reviewers independently screened articles and abstracted data according to a protocol designed by team consensus.
Results:
Of 290 articles identified via PubMed search, 44 were included in this study. More than three quarters were published since 2018. Forty-three (98%) used FHIR; 22 (50%) used SMART; two (5%) used CQL; and eight (18%) used CDS Hooks. Twenty-four (55%) were in the design stage, 15 (34%) in the piloting stage, and five (11%) were deployed in a real-world setting. Only 12 (27%) of the articles reported an evaluation of the technology under development. Three of the four articles describing a deployed technology reported an evaluation. Only two evaluations with randomized study components were identified.
Conclusion:
The diversity of topics and approaches identified in the literature highlights the utility of these standards. The infrequency of reported evaluations, as well as the high number of studies in the design or piloting stage, indicate that these technologies are still early in their life cycles. Informaticists will require a stronger evidence base to understand the implications of using these standards in CDS design and implementation.
Collapse
Affiliation(s)
- Peter Taber
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA
| | | | - Guilherme Del Fiol
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA
| | - Catherine Staes
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA.,College of Nursing, University of Utah, Salt Lake City, UT, USA
| | - Kensaku Kawamoto
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA
| |
Collapse
|
47
|
Taber P, Weir C, Butler JM, Graber CJ, Jones MM, Madaras-Kelly K, Zhang Y, Chou AF, Samore MH, Goetz MB, Glassman PA. Social dynamics of a population-level dashboard for antimicrobial stewardship: A qualitative analysis. Am J Infect Control 2021; 49:862-867. [PMID: 33515622 DOI: 10.1016/j.ajic.2021.01.015] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2020] [Revised: 01/15/2021] [Accepted: 01/19/2021] [Indexed: 12/21/2022]
Abstract
OBJECTIVE To evaluate antimicrobial stewards' experiences of using a dashboard display integrating local and national antibiotic use data implemented in the U.S. Department of Veterans Affairs (VA). This paper reports early formative evaluation. DESIGN Qualitative interviewing. SETTING Eight VA hospitals participated with established antimicrobial stewardship (AS) programs participated in the pilot. PARTICIPANTS Six infectious disease physicians and eight clinical pharmacists agreed to be interviewed (n = 14). METHODS A 3-part qualitative interview script was used involving a description of local stewardship activities, a Critical Incident description of dashboard use, and general questions regarding attitudes towards the tool. An inductive open coding approach was used for analysis. RESULTS We found 4 themes showing the complexities of using stewardship tools: (1) Data validity is socially negotiated; (2) Performance feedback motivates and persuades social goals when situated in an empirical distribution; (3) Shared problem awareness is aided by authoritative data; and (4) The AS dashboard encourages connections with local quality improvement culture. CONCLUSIONS Social dimensions of AS tool use emerged as distinct from, and equally important as decision support provided by the dashboard. Successful stewardship tools should be designed to support both the social and cognitive needs of users.
Collapse
Affiliation(s)
- Peter Taber
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT.
| | - Charlene Weir
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT; IDEAS Center, VA Salt Lake City Healthcare System, Salt Lake City, UT.
| | - Jorie M Butler
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT; IDEAS Center, VA Salt Lake City Healthcare System, Salt Lake City, UT; Geriatric Education and Clinical Center, VA Salt Lake City Healthcare System, Salt Lake City, UT; Department of Medicine, University of Utah, Salt Lake City, UT
| | - Christopher J Graber
- Center for Healthcare Innovation, Implementation, and Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA; Department of Medicine, David Geffen School of Medicine at the University of California-Los Angeles, Los Angeles, CA; Infectious Diseases Section, VA Greater Los Angeles Healthcare System, Los Angeles, CA
| | - Makoto M Jones
- IDEAS Center, VA Salt Lake City Healthcare System, Salt Lake City, UT; Geriatric Education and Clinical Center, VA Salt Lake City Healthcare System, Salt Lake City, UT; Division of Epidemiology, University of Utah, Salt Lake City, UT
| | - Karl Madaras-Kelly
- Department of Pharmacy Boise VA Medical Center, Boise, ID; College of Pharmacy, Idaho State University, Meridian, ID
| | - Yue Zhang
- IDEAS Center, VA Salt Lake City Healthcare System, Salt Lake City, UT; Geriatric Education and Clinical Center, VA Salt Lake City Healthcare System, Salt Lake City, UT; Division of Epidemiology, University of Utah, Salt Lake City, UT
| | - Ann F Chou
- Department of Family and Preventive Medicine, College of Medicine, University of Oklahoma Health Sciences Center, Oklahoma City, OK
| | - Matthew H Samore
- IDEAS Center, VA Salt Lake City Healthcare System, Salt Lake City, UT; Geriatric Education and Clinical Center, VA Salt Lake City Healthcare System, Salt Lake City, UT; Division of Epidemiology, University of Utah, Salt Lake City, UT
| | - Matthew Bidwell Goetz
- Center for Healthcare Innovation, Implementation, and Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA; Department of Medicine, David Geffen School of Medicine at the University of California-Los Angeles, Los Angeles, CA; Infectious Diseases Section, VA Greater Los Angeles Healthcare System, Los Angeles, CA
| | - Peter A Glassman
- Center for Healthcare Innovation, Implementation, and Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA; VA Pharmacy Benefits Management Services, Department of Veterans Affairs, Washington, DC; Department of Medicine, VA Greater Los Angeles Healthcare System, Los Angeles, CA
| |
Collapse
|
48
|
Ford E, Edelman N, Somers L, Shrewsbury D, Lopez Levy M, van Marwijk H, Curcin V, Porat T. Barriers and facilitators to the adoption of electronic clinical decision support systems: a qualitative interview study with UK general practitioners. BMC Med Inform Decis Mak 2021; 21:193. [PMID: 34154580 PMCID: PMC8215812 DOI: 10.1186/s12911-021-01557-z] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2020] [Accepted: 05/31/2021] [Indexed: 11/29/2022] Open
Abstract
Background Well-established electronic data capture in UK general practice means that algorithms, developed on patient data, can be used for automated clinical decision support systems (CDSSs). These can predict patient risk, help with prescribing safety, improve diagnosis and prompt clinicians to record extra data. However, there is persistent evidence of low uptake of CDSSs in the clinic. We interviewed UK General Practitioners (GPs) to understand what features of CDSSs, and the contexts of their use, facilitate or present barriers to their use. Methods We interviewed 11 practicing GPs in London and South England using a semi-structured interview schedule and discussed a hypothetical CDSS that could detect early signs of dementia. We applied thematic analysis to the anonymised interview transcripts. Results We identified three overarching themes: trust in individual CDSSs; usability of individual CDSSs; and usability of CDSSs in the broader practice context, to which nine subthemes contributed. Trust was affected by CDSS provenance, perceived threat to autonomy and clear management guidance. Usability was influenced by sensitivity to the patient context, CDSS flexibility, ease of control, and non-intrusiveness. CDSSs were more likely to be used by GPs if they did not contribute to alert proliferation and subsequent fatigue, or if GPs were provided with training in their use. Conclusions Building on these findings we make a number of recommendations for CDSS developers to consider when bringing a new CDSS into GP patient records systems. These include co-producing CDSS with GPs to improve fit within clinic workflow and wider practice systems, ensuring a high level of accuracy and a clear clinical pathway, and providing CDSS training for practice staff. These recommendations may reduce the proliferation of unhelpful alerts that can result in important decision-support being ignored. Supplementary Information The online version contains supplementary material available at 10.1186/s12911-021-01557-z.
Collapse
Affiliation(s)
- Elizabeth Ford
- Department of Primary Care and Public Health, Brighton and Sussex Medical School, Watson Building, Village Way, Falmer, Brighton, BN1 9PH, UK.
| | - Natalie Edelman
- Department of Primary Care and Public Health, Brighton and Sussex Medical School, Watson Building, Village Way, Falmer, Brighton, BN1 9PH, UK.,School of Sport and Health Sciences, University of Brighton, Brighton, UK
| | - Laura Somers
- Department of Primary Care and Public Health, Brighton and Sussex Medical School, Watson Building, Village Way, Falmer, Brighton, BN1 9PH, UK
| | - Duncan Shrewsbury
- Department of Primary Care and Public Health, Brighton and Sussex Medical School, Watson Building, Village Way, Falmer, Brighton, BN1 9PH, UK
| | - Marcela Lopez Levy
- Psychosocial Department, Centre for Researching and Embedding Human Rights (CREHR), Birkbeck College, London, UK
| | - Harm van Marwijk
- Department of Primary Care and Public Health, Brighton and Sussex Medical School, Watson Building, Village Way, Falmer, Brighton, BN1 9PH, UK
| | - Vasa Curcin
- School of Population Health and Environmental Sciences, King's College London, London, UK
| | - Talya Porat
- Dyson School of Design Engineering, Imperial College London, London, UK
| |
Collapse
|
49
|
van Baalen S, Boon M, Verhoef P. From clinical decision support to clinical reasoning support systems. J Eval Clin Pract 2021; 27:520-528. [PMID: 33554432 PMCID: PMC8248191 DOI: 10.1111/jep.13541] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Revised: 12/15/2020] [Accepted: 01/07/2021] [Indexed: 12/19/2022]
Abstract
Despite the great promises that artificial intelligence (AI) holds for health care, the uptake of such technologies into medical practice is slow. In this paper, we focus on the epistemological issues arising from the development and implementation of a class of AI for clinical practice, namely clinical decision support systems (CDSS). We will first provide an overview of the epistemic tasks of medical professionals, and then analyse which of these tasks can be supported by CDSS, while also explaining why some of them should remain the territory of human experts. Clinical decision making involves a reasoning process in which clinicians combine different types of information into a coherent and adequate 'picture of the patient' that enables them to draw explainable and justifiable conclusions for which they bear epistemological responsibility. Therefore, we suggest that it is more appropriate to think of a CDSS as clinical reasoning support systems (CRSS). Developing CRSS that support clinicians' reasoning process therefore requires that: (a) CRSSs are developed on the basis of relevant and well-processed data; and (b) the system facilitates an interaction with the clinician. Therefore, medical experts must collaborate closely with AI experts developing the CRSS. In addition, responsible use of an CRSS requires that the data generated by the CRSS is empirically justified through an empirical link with the individual patient. In practice, this means that the system indicates what factors contributed to arriving at an advice, allowing the user (clinician) to evaluate whether these factors are medically plausible and applicable to the patient. Finally, we defend that proper implementation of CRSS allows combining human and artificial intelligence into hybrid intelligence, were both perform clearly delineated and complementary empirical tasks. Whereas CRSSs can assist with statistical reasoning and finding patterns in complex data, it is the clinicians' task to interpret, integrate and contextualize.
Collapse
Affiliation(s)
| | - Mieke Boon
- Department of PhilosophyUniversity of TwenteEnschedeThe Netherlands
| | | |
Collapse
|
50
|
Sloss EA, Jones TL. Nurse Cognition, Decision Support, and Barcode Medication Administration: A Conceptual Framework for Research, Practice, and Education. Comput Inform Nurs 2021; 39:851-857. [PMID: 33935198 DOI: 10.1097/cin.0000000000000724] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
This article synthesizes theoretical perspectives related to nurse cognition. We present a conceptual model that can be used by multiple stakeholders to study and contemplate how nurses use clinical decision support systems, and specifically, Barcode-Assisted Medication Administration, to make decisions during the delivery of care. Theoretical perspectives integrated into the model include dual process theory, the Cognitive Continuum Theory, human factors engineering, and the Recognition-Primed Decision model. The resulting framework illustrates the process of nurse cognition during Barcode-Assisted Medication Administration. Additionally, the model includes individual or human and environmental factors that may influence nurse cognition and decision making. It is important to consider the influence of individual, human, and environmental factors on the process of nurse cognition and decision making. Specifically, it is necessary to explore the impact of heuristics and biases on clinician decision making, particularly related to the development of alarm and alert fatigue. Aided by the proposed framework, stakeholders may begin to identify heuristics and cognitive biases that influence the decision of clinicians to accept or override a clinical decision support system alert and whether heuristics and biases are associated with inappropriate alert override.
Collapse
Affiliation(s)
- Elizabeth Ann Sloss
- Author Affiliations: Department of Professional Nursing Practice, Georgetown University (Ms Sloss), Washington, DC; and Department of Adult Health and Nursing Systems, Virginia Commonwealth University (Dr Jones), Richmond
| | | |
Collapse
|