1
|
Reese TJ, Domenico HJ, Hernandez A, Byrne DW, Moore RP, Williams JB, Douthit BJ, Russo E, McCoy AB, Ivory CH, Steitz BD, Wright A. Implementable Prediction of Pressure Injuries in Hospitalized Adults: Model Development and Validation. JMIR Med Inform 2024; 12:e51842. [PMID: 38722209 PMCID: PMC11094428 DOI: 10.2196/51842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Revised: 03/08/2024] [Accepted: 03/10/2024] [Indexed: 05/18/2024] Open
Abstract
Background Numerous pressure injury prediction models have been developed using electronic health record data, yet hospital-acquired pressure injuries (HAPIs) are increasing, which demonstrates the critical challenge of implementing these models in routine care. Objective To help bridge the gap between development and implementation, we sought to create a model that was feasible, broadly applicable, dynamic, actionable, and rigorously validated and then compare its performance to usual care (ie, the Braden scale). Methods We extracted electronic health record data from 197,991 adult hospital admissions with 51 candidate features. For risk prediction and feature selection, we used logistic regression with a least absolute shrinkage and selection operator (LASSO) approach. To compare the model with usual care, we used the area under the receiver operating curve (AUC), Brier score, slope, intercept, and integrated calibration index. The model was validated using a temporally staggered cohort. Results A total of 5458 HAPIs were identified between January 2018 and July 2022. We determined 22 features were necessary to achieve a parsimonious and highly accurate model. The top 5 features included tracheostomy, edema, central line, first albumin measure, and age. Our model achieved higher discrimination than the Braden scale (AUC 0.897, 95% CI 0.893-0.901 vs AUC 0.798, 95% CI 0.791-0.803). Conclusions We developed and validated an accurate prediction model for HAPIs that surpassed the standard-of-care risk assessment and fulfilled necessary elements for implementation. Future work includes a pragmatic randomized trial to assess whether our model improves patient outcomes.
Collapse
Affiliation(s)
- Thomas J Reese
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Henry J Domenico
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Antonio Hernandez
- Department of Anesthesiology, Division of Critical Care Medicine, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Daniel W Byrne
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Ryan P Moore
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Jessica B Williams
- Department of Nursing, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Brian J Douthit
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Elise Russo
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Allison B McCoy
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Catherine H Ivory
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Bryan D Steitz
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Adam Wright
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States
| |
Collapse
|
2
|
Wurhofer D, Neunteufel J, Strumegger EM, Höppchen I, Mayr B, Egger A, Sareban M, Reich B, Neudorfer M, Niebauer J, Smeddinck JD, Kulnik ST. Investigating shared decision-making during the use of a digital health tool for physical activity planning in cardiac rehabilitation. Front Digit Health 2024; 5:1324488. [PMID: 38239278 PMCID: PMC10794499 DOI: 10.3389/fdgth.2023.1324488] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Accepted: 12/05/2023] [Indexed: 01/22/2024] Open
Abstract
Background Shared decision making (SDM) between healthcare professionals and persons with CVD can have a positive impact on motivation, adherence, or sustainability regarding long-term goals and integration of cardiovascular disease (CVD) rehabilitation in the everyday lives of persons with CVD. SDM can foster the transition between regular heart-healthy activity at rehabilitation facilities and more independent activity at home, but it is often challenging to implement SDM given limited time and resources, e.g., in the daily practice of rehabilitation. Digital tools can help but must be appropriately tailored for situated use and user needs. Objective We aimed to (1) describe in how far SDM is manifested in the situated context when using a digital tool developed by our group, and, based on that, (2) reflect on how digital health tools can be designed to facilitate and improve the SDM process. Methods In the context of a field study, we investigated how SDM is already naturally applied and manifested when using a digital tool for joint physical activity planning in cardiac rehabilitation in clinical practice. In a two-week qualitative study, we collected data on expectations, experiences and interactions during the use of a digital health tool by seven persons with CVD and five healthcare professionals. Data was collected by means of observations, interviews, questionnaires and a self-reported diary, and analysed with a particular focus on episodes related to SDM. Results We found that SDM was manifested in the situated context to limited extent. For example, we identified high improvement potential for more structured goal-setting and more explicit consideration of preferences and routines. Based on mapping our findings to temporal phases where SDM can be adopted, we highlight implications for design to further support SDM in clinical practice. We consider this as "SDM supportive design in digital health apps," suggesting for example step-by-step guidance to be used during the actual consultation. Conclusion This study contributes to further understanding and integration of SDM in digital health tools with a focus on rehabilitation, to empower and support both persons with CVD and healthcare professionals.
Collapse
Affiliation(s)
- Daniela Wurhofer
- Ludwig Boltzmann Institute for Digital Health and Prevention, Salzburg, Austria
| | - Julia Neunteufel
- Ludwig Boltzmann Institute for Digital Health and Prevention, Salzburg, Austria
| | | | - Isabel Höppchen
- Ludwig Boltzmann Institute for Digital Health and Prevention, Salzburg, Austria
| | - Barbara Mayr
- University Institute of Sports Medicine, Prevention and Rehabilitation and Research Institute of Molecular Sports Medicine and Rehabilitation, Paracelsus Medical University, Salzburg, Austria
| | - Andreas Egger
- University Institute of Sports Medicine, Prevention and Rehabilitation and Research Institute of Molecular Sports Medicine and Rehabilitation, Paracelsus Medical University, Salzburg, Austria
| | - Mahdi Sareban
- Ludwig Boltzmann Institute for Digital Health and Prevention, Salzburg, Austria
- University Institute of Sports Medicine, Prevention and Rehabilitation and Research Institute of Molecular Sports Medicine and Rehabilitation, Paracelsus Medical University, Salzburg, Austria
| | - Bernhard Reich
- University Institute of Sports Medicine, Prevention and Rehabilitation and Research Institute of Molecular Sports Medicine and Rehabilitation, Paracelsus Medical University, Salzburg, Austria
| | - Michael Neudorfer
- University Institute of Sports Medicine, Prevention and Rehabilitation and Research Institute of Molecular Sports Medicine and Rehabilitation, Paracelsus Medical University, Salzburg, Austria
| | - Josef Niebauer
- Ludwig Boltzmann Institute for Digital Health and Prevention, Salzburg, Austria
- University Institute of Sports Medicine, Prevention and Rehabilitation and Research Institute of Molecular Sports Medicine and Rehabilitation, Paracelsus Medical University, Salzburg, Austria
| | - Jan David Smeddinck
- Ludwig Boltzmann Institute for Digital Health and Prevention, Salzburg, Austria
| | - Stefan Tino Kulnik
- Ludwig Boltzmann Institute for Digital Health and Prevention, Salzburg, Austria
| |
Collapse
|
3
|
Gomez Lumbreras A, Reese TJ, Del Fiol G, Tan MS, Butler JM, Hurwitz JT, Brown M, Kawamoto K, Thiess H, Wright M, Malone DC. Shared Decision-Making for Drug-Drug Interactions: Formative Evaluation of an Anticoagulant Drug Interaction. JMIR Form Res 2022; 6:e40018. [PMID: 36260377 PMCID: PMC9631167 DOI: 10.2196/40018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2022] [Revised: 08/21/2022] [Accepted: 09/22/2022] [Indexed: 11/07/2022] Open
Abstract
Background Warnings about drug-drug interactions (DDIs) between warfarin and nonsteroidal anti-inflammatory drugs (NSAIDs) within electronic health records indicate potential harm but fail to account for contextual factors and preferences. We developed a tool called DDInteract to enhance and support shared decision-making (SDM) between patients and physicians when both warfarin and NSAIDs are used concurrently. DDInteract was designed to be integrated into electronic health records using interoperability standards. Objective The purpose of this study was to conduct a formative evaluation of a DDInteract that incorporates patient and product contextual factors to estimate the risk of bleeding. Methods A randomized formative evaluation was conducted to compare DDInteract to usual care (UC) using physician-patient dyads. Using case vignettes, physicians and patients on warfarin participated in simulated virtual clinical encounters where they discussed the use of taking ibuprofen and warfarin concurrently and determined an appropriate therapeutic plan based on the patient’s individualized risk. Dyads were randomized to either DDInteract or UC. Participants completed a postsession interview and survey of the SDM process. This included the 9-item Shared Decision-Making Questionnaire (SDM-Q-9), tool usability and workload National Aeronautics and Space Administration (NASA) Task Load Index, Unified Theory of Acceptance and Use of Technology (UTAUT), Perceived Behavioral Control (PBC) scale, System Usability Scale (SUS), and Decision Conflict Scale (DCS). They also were interviewed after the session to obtain perceptions on DDInteract and UC resources for DDIs. Results Twelve dyad encounters were performed using virtual software. Most (n=11, 91.7%) patients were over 50 years of age, and 9 (75%) had been taking warfarin for more than 2 years (75%). Regarding scores on the SDM-Q-9, participants rated DDInteract higher than UC for questions pertaining to helping patients clarify the decision (P=.03), involving patients in the decision (P=.01), displaying treatment options (P<.001), identifying advantages and disadvantages (P=.01), and facilitating patient understanding (P=.01) and discussion of preferences (P=.01). Five of the 8 UTAUT constructs showed differences between the 2 groups, favoring DDInteract (P<.05). Usability ratings from the SUS were significantly higher (P<.05) for physicians using DDInteract compared to those in the UC group but showed no differences from the patient’s perspective. No differences in patient responses were observed between groups using the DCS. During the session debrief, physicians indicated little concern for the additional time or workload entailed by DDInteract use. Both clinicians and patients indicated that the tool was beneficial in simulated encounters to understand and mitigate the risk of harm from this DDI. Conclusions Overall, DDInteract may improve encounters where there is a risk of bleeding due to a potential drug-drug interaction involving anticoagulants. Participants rated DDInteract as logical and useful for enhancing SDM. They reported that they would be willing to use the tool for an interaction involving warfarin and NSAIDs.
Collapse
Affiliation(s)
- Ainhoa Gomez Lumbreras
- Department of Pharmacotherapy, Skaggs College of Pharmacy, University of Utah, Salt Lake City, UT, United States
| | - Thomas J Reese
- Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, United States
| | - Guilherme Del Fiol
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States
| | - Malinda S Tan
- Department of Pharmacotherapy, Skaggs College of Pharmacy, University of Utah, Salt Lake City, UT, United States
| | - Jorie M Butler
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States
| | - Jason T Hurwitz
- Center for Health Outcomes and Pharmacoeconomic Research, University of Arizona, Tucson, AZ, United States
| | - Mary Brown
- University of Arizona, Tucson, AZ, United States
| | - Kensaku Kawamoto
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States
| | | | - Maria Wright
- Department of Pharmacotherapy, Skaggs College of Pharmacy, University of Utah, Salt Lake City, UT, United States
| | - Daniel C Malone
- Department of Pharmacotherapy, Skaggs College of Pharmacy, University of Utah, Salt Lake City, UT, United States
| |
Collapse
|
4
|
Reese TJ, Liu S, Steitz B, McCoy A, Russo E, Koh B, Ancker J, Wright A. Conceptualizing clinical decision support as complex interventions: a meta-analysis of comparative effectiveness trials. J Am Med Inform Assoc 2022; 29:1744-1756. [PMID: 35652167 PMCID: PMC9471719 DOI: 10.1093/jamia/ocac089] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Revised: 04/26/2022] [Accepted: 05/23/2022] [Indexed: 12/26/2022] Open
Abstract
OBJECTIVES Complex interventions with multiple components and behavior change strategies are increasingly implemented as a form of clinical decision support (CDS) using native electronic health record functionality. Objectives of this study were, therefore, to (1) identify the proportion of randomized controlled trials with CDS interventions that were complex, (2) describe common gaps in the reporting of complexity in CDS research, and (3) determine the impact of increased complexity on CDS effectiveness. MATERIALS AND METHODS To assess CDS complexity and identify reporting gaps for characterizing CDS interventions, we used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting tool for complex interventions. We evaluated the effect of increased complexity using random-effects meta-analysis. RESULTS Most included studies evaluated a complex CDS intervention (76%). No studies described use of analytical frameworks or causal pathways. Two studies discussed use of theory but only one fully described the rationale and put it in context of a behavior change. A small but positive effect (standardized mean difference, 0.147; 95% CI, 0.039-0.255; P < .01) in favor of increasing intervention complexity was observed. DISCUSSION While most CDS studies should classify interventions as complex, opportunities persist for documenting and providing resources in a manner that would enable CDS interventions to be replicated and adapted. Unless reporting of the design, implementation, and evaluation of CDS interventions improves, only slight benefits can be expected. CONCLUSION Conceptualizing CDS as complex interventions may help convey the careful attention that is needed to ensure these interventions are contextually and theoretically informed.
Collapse
Affiliation(s)
- Thomas J Reese
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Siru Liu
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Bryan Steitz
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Allison McCoy
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Elise Russo
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Brian Koh
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Jessica Ancker
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Adam Wright
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| |
Collapse
|
5
|
Jones EK, Hultman G, Schmoke K, Ninkovic I, Dodge S, Bahr M, Melton GB, Marquard J, Tignanelli CJ. Combined Expert and User-Driven Usability Assessment of Trauma Decision Support Systems Improves User-Centered Design. Surgery 2022; 172:1537-1548. [PMID: 36031451 DOI: 10.1016/j.surg.2022.05.037] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 05/11/2022] [Accepted: 05/30/2022] [Indexed: 11/20/2022]
Abstract
BACKGROUND Trauma clinical decision support systems improve adherence with evidence-based practice but suffer from poor usability and the lack of a user-centered design. The objective of this study was to compare the effectiveness of user and expert-driven usability testing methods to detect usability issues in a rib fracture clinical decision support system and identify guiding principles for trauma clinical decision support systems. METHODS A user-driven and expert-driven usability investigation was conducted using a clinical decision support system developed for patients with rib fractures. The user-driven usability evaluation was as follows: 10 clinicians were selected for simulation-based usability testing using snowball sampling, and each clinician completed 3 simulations using a video-conferencing platform. End-users participated in a novel team-based approach that simulated realistic clinical workflows. The expert-driven heuristic evaluation was as follows: 2 usability experts conducted a heuristic evaluation of the clinical decision support system using 10 common usability heuristics. Usability issues were identified, cataloged, and ranked for severity using a 4-level ordinal scale. Thematic analysis was utilized to categorize the identified usability issues. RESULTS Seventy-nine usability issues were identified; 63% were identified by experts and 48% by end-users. Notably, 58% of severe usability issues were identified by experts alone. Only 11% of issues were identified by both methods. Five themes were identified that could guide the design of clinical decision support systems-transparency, functionality and integration into workflow, automated and noninterruptive, flexibility, and layout and appearance. Themes were preferentially identified by different methods. CONCLUSION We found that a dual-method usability evaluation involving usability experts and end-users drastically improved detection of usability issues over single-method alone. We identified 5 themes to guide trauma clinical decision support system design. Performing usability testing via a remote video-conferencing platform facilitated multi-site involvement despite a global pandemic.
Collapse
Affiliation(s)
- Emma K Jones
- Department of Surgery, University of Minnesota, Minneapolis, MN.
| | - Gretchen Hultman
- Institute for Health Informatics, University of Minnesota, Minneapolis, MN
| | - Kristine Schmoke
- Veterans Health Administration, Department of Veterans Affairs, Washington, DC
| | | | - Sarah Dodge
- Fairview Health Services IT, Minneapolis, MN
| | - Matthew Bahr
- Trauma Services, Fairview Health Services, Minneapolis, MN
| | - Genevieve B Melton
- Department of Surgery, University of Minnesota, Minneapolis, MN; Institute for Health Informatics, University of Minnesota, Minneapolis, MN; Fairview Health Services IT, Minneapolis, MN; Center for Learning Health System Sciences, University of Minnesota, Minneapolis, MN
| | - Jenna Marquard
- School of Nursing, University of Minnesota, Minneapolis, MN
| | - Christopher J Tignanelli
- Department of Surgery, University of Minnesota, Minneapolis, MN; Institute for Health Informatics, University of Minnesota, Minneapolis, MN; Center for Learning Health System Sciences, University of Minnesota, Minneapolis, MN
| |
Collapse
|
6
|
Thiess H, Del Fiol G, Malone DC, Cornia R, Sibilla M, Rhodes B, Boyce RD, Kawamoto K, Reese T. Coordinated use of Health Level 7 standards to support clinical decision support: Case study with shared decision making and drug-drug interactions. Int J Med Inform 2022; 162:104749. [PMID: 35358893 PMCID: PMC9703934 DOI: 10.1016/j.ijmedinf.2022.104749] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2021] [Revised: 02/22/2022] [Accepted: 03/17/2022] [Indexed: 11/29/2022]
Abstract
BACKGROUND Despite advances in interoperability standards, it remains challenging and often costly to share clinical decision support (CDS) across healthcare organizations. This is due in part to limited coordination among CDS components. To improve coordination of CDS components, Health Level 7 (HL7) has developed a suite of interoperability standards with Fast Health Interoperability Resources (FHIR) specification as a common information model. Evidence is needed to determine the feasibility of implementing these CDS components; therefore, the objective of this study was to investigate the coordination of emerging HL7 standards with modular CDS architecture components. METHODS We used a modular, standards-based architecture consisting of four components: data, logic, services, and applications. The implementation use-case was an application to support shared decision making in the context of drug-drug interactions (DDInteract). RESULTS DDInteract uses FHIR as the data representation model, Clinical Quality Language for logic representation, CDS Hooks for the services layer, and Substitutable Medical Apps Reusable Technologies for application integration. DDInteract was first implemented in a sandbox environment and then in an electronic health record (Epic®) test environment. DDInteract can be integrated in clinical workflows through on-demand access from a menu or through CDS Hooks upon opening a patient's record or placing a medication order. CONCLUSION In the context of drug interactions, DDInteract is the first application to leverage a full stack of emerging interoperability standards for each component of modular CDS architecture. The demonstrated feasibility of interoperable components can be generalized to other modular CDS applications.
Collapse
Affiliation(s)
| | | | | | - Ryan Cornia
- University of Utah, Department of Biomedical Informatics, United States
| | - Max Sibilla
- University of Pittsburgh, Department of Biomedical Informatics, United States
| | - Bryn Rhodes
- Alphora, Chief Technology Officer, United States
| | - Richard D Boyce
- University of Pittsburgh, Department of Biomedical Informatics, United States
| | - Kensaku Kawamoto
- University of Utah, Department of Biomedical Informatics, United States
| | - Thomas Reese
- Vanderbilt University Medical Center, Department of Biomedical Informatics, Nashville, TN.
| |
Collapse
|