1
|
Lin CS, Liu WT, Tsai DJ, Lou YS, Chang CH, Lee CC, Fang WH, Wang CC, Chen YY, Lin WS, Cheng CC, Lee CC, Wang CH, Tsai CS, Lin SH, Lin C. AI-enabled electrocardiography alert intervention and all-cause mortality: a pragmatic randomized clinical trial. Nat Med 2024; 30:1461-1470. [PMID: 38684860 DOI: 10.1038/s41591-024-02961-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2023] [Accepted: 03/29/2024] [Indexed: 05/02/2024]
Abstract
The early identification of vulnerable patients has the potential to improve outcomes but poses a substantial challenge in clinical practice. This study evaluated the ability of an artificial intelligence (AI)-enabled electrocardiogram (ECG) to identify hospitalized patients with a high risk of mortality in a multisite randomized controlled trial involving 39 physicians and 15,965 patients. The AI-ECG alert intervention included an AI report and warning messages delivered to the physicians, flagging patients predicted to be at high risk of mortality. The trial met its primary outcome, finding that implementation of the AI-ECG alert was associated with a significant reduction in all-cause mortality within 90 days: 3.6% patients in the intervention group died within 90 days, compared to 4.3% in the control group (4.3%) (hazard ratio (HR) = 0.83, 95% confidence interval (CI) = 0.70-0.99). A prespecified analysis showed that reduction in all-cause mortality associated with the AI-ECG alert was observed primarily in patients with high-risk ECGs (HR = 0.69, 95% CI = 0.53-0.90). In analyses of secondary outcomes, patients in the intervention group with high-risk ECGs received increased levels of intensive care compared to the control group; for the high-risk ECG group of patients, implementation of the AI-ECG alert was associated with a significant reduction in the risk of cardiac death (0.2% in the intervention arm versus 2.4% in the control arm, HR = 0.07, 95% CI = 0.01-0.56). While the precise means by which implementation of the AI-ECG alert led to decreased mortality are to be fully elucidated, these results indicate that such implementation assists in the detection of high-risk patients, prompting timely clinical care and reducing mortality. ClinicalTrials.gov registration: NCT05118035 .
Collapse
Affiliation(s)
- Chin-Sheng Lin
- Division of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
- Medical Technology Education Center, School of Medicine, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Wei-Ting Liu
- Division of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Dung-Jang Tsai
- Medical Technology Education Center, School of Medicine, National Defense Medical Center, Taipei, Taiwan, Republic of China
- Department of Artificial Intelligence and Internet of Things, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
- Department of Statistics and Information Science, Fu Jen Catholic University, New Taipei City, Taiwan, Republic of China
| | - Yu-Sheng Lou
- Department of Artificial Intelligence and Internet of Things, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Chiao-Hsiang Chang
- Division of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Chiao-Chin Lee
- Division of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Wen-Hui Fang
- Department of Artificial Intelligence and Internet of Things, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
- Department of Family and Community Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Chih-Chia Wang
- Department of Family and Community Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Yen-Yuan Chen
- Department and Graduate Institute of Medical Education and Bioethics, National Taiwan University College of Medicine, Taipei, Taiwan, Republic of China
| | - Wei-Shiang Lin
- Division of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Cheng-Chung Cheng
- Division of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Chia-Cheng Lee
- Department of Medical Informatics, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
- Division of Colorectal Surgery, Department of Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Chih-Hung Wang
- Department of Otolaryngology-Head and Neck Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
- Graduate Institute of Medical Sciences, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Chien-Sung Tsai
- Division of Cardiovascular Surgery, Department of Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Shih-Hua Lin
- Division of Nephrology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China
| | - Chin Lin
- Medical Technology Education Center, School of Medicine, National Defense Medical Center, Taipei, Taiwan, Republic of China.
- Department of Artificial Intelligence and Internet of Things, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.
- School of Public Health, National Defense Medical Center, Taipei, Taiwan, Republic of China.
- Graduate Institute of Aerospace and Undersea Medicine, National Defense Medical Center, Taipei, Taiwan, Republic of China.
| |
Collapse
|
2
|
Hom GL, Kuo BL, Ross JH, Chapman GC, Sharma N, Sastry R, Muste JC, Greenlee TE, Conti TF, Singh RP, Sharma S. Characterization of pentosan polysulfate patients for development of an alert and screening system for ophthalmic monitoring. CANADIAN JOURNAL OF OPHTHALMOLOGY 2024; 59:128-136. [PMID: 36878265 DOI: 10.1016/j.jcjo.2023.01.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/16/2022] [Revised: 12/27/2022] [Accepted: 01/30/2023] [Indexed: 03/06/2023]
Abstract
OBJECTIVE Pentosan polysulfate (PPS; ELMIRON, Janssen Pharmaceuticals, Titusville, NJ) is a U.S. Food and Drug Administration-approved oral medication for interstitial cystitis. Numerous reports have been published detailing retinal toxicity with the use of PPS. Studies characterizing this condition are primarily retrospective, and consequently, alert and screening systems need to be developed to actively screen for this disease. The goal of this study was to characterize ophthalmic monitoring trends of a PPS-using patient sample to construct an alert and screening system for monitoring this condition. METHODS A single-institution retrospective chart review was conducted between January 2005 and November 2020 to characterize PPS use. An electronic medical record (EMR) alert was constructed to trigger based on new PPS prescriptions and renewals offering ophthalmology referral. RESULTS A total of 1407 PPS users over 15 years was available for characterization, with 1220 (86.7%) being female, the average duration of exposure being 71.2 ± 62.6 months, and the average medication cumulative exposure being 669.7 ± 569.2 g. A total of 151 patients (10.7%) had a recorded visit with an ophthalmologist, with 71 patients (5.0%) having optical coherence tomography imaging. The EMR alert fired for 88 patients over 1 year, with 34 patients (38.6%) either already being screened by an ophthalmologist or having been referred for screening. CONCLUSIONS An EMR support tool can improve referral rates of PPS maculopathy screening with an ophthalmologist and may serve as an efficient method for longitudinal screening of this condition with the added benefit of informing pentosan polysulfate prescribers about this condition. Effective screening and detection may help determine which patients are at high risk for this condition.
Collapse
Affiliation(s)
- Grant L Hom
- Case Western Reserve University School of Medicine, Cleveland, OH
| | - Blanche L Kuo
- Case Western Reserve University School of Medicine, Cleveland, OH
| | - James H Ross
- Department of Obstetrics and Gynecology, Women's Health Institute, Cleveland Clinic, Cleveland, Ohio
| | - Graham C Chapman
- Department of Obstetrics and Gynecology, Women's Health Institute, Cleveland Clinic, Cleveland, Ohio
| | - Neha Sharma
- Case Western Reserve University School of Medicine, Cleveland, OH
| | - Resya Sastry
- Center for Ophthalmic Bioinformatics, Cole Eye Institute, Cleveland Clinic, Cleveland, OH
| | - Justin C Muste
- Center for Ophthalmic Bioinformatics, Cole Eye Institute, Cleveland Clinic, Cleveland, OH
| | - Tyler E Greenlee
- Center for Ophthalmic Bioinformatics, Cole Eye Institute, Cleveland Clinic, Cleveland, OH
| | - Thais F Conti
- Center for Ophthalmic Bioinformatics, Cole Eye Institute, Cleveland Clinic, Cleveland, OH
| | - Rishi P Singh
- Center for Ophthalmic Bioinformatics, Cole Eye Institute, Cleveland Clinic, Cleveland, OH
| | - Sumit Sharma
- Center for Ophthalmic Bioinformatics, Cole Eye Institute, Cleveland Clinic, Cleveland, OH.
| |
Collapse
|
3
|
Sheehan KN, Cioci AL, Lucioni TM, Hernandez SM. Resident-Driven Clinical Decision Support Governance to Improve the Utility of Clinical Decision Support. Appl Clin Inform 2024; 15:335-341. [PMID: 38692282 PMCID: PMC11062759 DOI: 10.1055/s-0044-1786682] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2023] [Accepted: 03/12/2024] [Indexed: 05/03/2024] Open
Abstract
OBJECTIVES This resident-driven quality improvement project aimed to better understand the known problem of a misaligned clinical decision support (CDS) strategy and improve CDS utilization. METHODS An internal survey was sent to all internal medicine (IM) residents to identify the most bothersome CDS alerts. Survey results were supported by electronic health record (EHR) data of CDS firing rates and response rates which were collected for each of the three most bothersome CDS tools. Changes to firing criteria were created to increase utilization and to better align with the five rights of CDS. Findings and proposed changes were presented to our institution's CDS Governance Committee. Changes were approved and implemented. Postintervention firing rates were then collected for 1 week. RESULTS Twenty nine residents participated in the CDS survey and identified sepsis alerts, lipid profile reminders, and telemetry renewals to be the most bothersome alerts. EHR data showed action rates for these CDS as low as 1%. We implemented changes to focus emergency department (ED)-based sepsis alerts to the right provider, better address the right information for lipid profile reminders, and select the right time in workflow for telemetry renewals to be most effective. With these changes we successfully eliminated ED-based sepsis CDS reminders for IM providers, saw a 97% reduction in firing rates for the lipid profile CDS, and noted a 55% reduction in firing rates for telemetry CDS. CONCLUSION This project highlighted that alert improvements spearheaded by resident teams can be completed successfully using robust CDS governance strategies and can effectively optimize interruptive alerts.
Collapse
Affiliation(s)
- Kristin N. Sheehan
- Department of Internal Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States
| | - Anthony L. Cioci
- Department of Internal Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States
| | - Tomas M. Lucioni
- Department of Internal Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States
| | - Sean M. Hernandez
- Department of Internal Medicine, Wake Forest School of Medicine, Winston-Salem, North Carolina, United States
| |
Collapse
|
4
|
Musser RC, Senior R, Havrilesky LJ, Buuck J, Casarett DJ, Ibrahim S, Davidson BA. Randomized Comparison of Electronic Health Record Alert Types in Eliciting Responses about Prognosis in Gynecologic Oncology Patients. Appl Clin Inform 2024; 15:204-211. [PMID: 38232748 PMCID: PMC10937092 DOI: 10.1055/a-2247-9355] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2023] [Accepted: 01/16/2024] [Indexed: 01/19/2024] Open
Abstract
OBJECTIVES To compare the ability of different electronic health record alert types to elicit responses from users caring for cancer patients benefiting from goals of care (GOC) conversations. METHODS A validated question asking if the user would be surprised by the patient's 6-month mortality was built as an Epic BestPractice Advisory (BPA) alert in three versions-(1) Required on Open chart (pop-up BPA), (2) Required on Close chart (navigator BPA), and (3) Optional Persistent (Storyboard BPA)-randomized using patient medical record number. Meaningful responses were defined as "Yes" or "No," rather than deferral. Data were extracted over 6 months. RESULTS Alerts appeared for 685 patients during 1,786 outpatient encounters. Measuring encounters where a meaningful response was elicited, rates were highest for Required on Open (94.8% of encounters), compared with Required on Close (90.1%) and Optional Persistent (19.7%) (p < 0.001). Measuring individual alerts to which responses were given, they were most likely meaningful with Optional Persistent (98.3% of responses) and least likely with Required on Open (68.0%) (p < 0.001). Responses of "No," suggesting poor prognosis and prompting GOC, were more likely with Optional Persistent (13.6%) and Required on Open (10.3%) than with Required on Close (7.0%) (p = 0.028). CONCLUSION Required alerts had response rates almost five times higher than optional alerts. Timing of alerts affects rates of meaningful responses and possibly the response itself. The alert with the most meaningful responses was also associated with the most interruptions and deferral responses. Considering tradeoffs in these metrics is important in designing clinical decision support to maximize success.
Collapse
Affiliation(s)
- Robert Clayton Musser
- Department of Medicine, Duke University Health System, Durham, North Carolina, United States
- Duke Health Technology Solutions, Durham, North Carolina, United States
| | - Rashaud Senior
- Duke Health Technology Solutions, Durham, North Carolina, United States
- Duke Primary Care, Duke University Health System, Durham, North Carolina, United States
| | - Laura J. Havrilesky
- Division of Gynecologic Oncology, Department of Obstetrics and Gynecology, Duke University Health System, Durham, North Carolina, United States
| | - Jordan Buuck
- Duke Health Technology Solutions, Durham, North Carolina, United States
| | - David J. Casarett
- Section of Palliative Care, Department of Medicine, Duke University Health System, Durham, North Carolina, United States
| | - Salam Ibrahim
- Duke Health Performance Services, Duke University Health System, Durham, North Carolina, United States
| | - Brittany A. Davidson
- Division of Gynecologic Oncology, Department of Obstetrics and Gynecology, Duke University Health System, Durham, North Carolina, United States
| |
Collapse
|
5
|
Virtanen L, Kaihlanen AM, Saukkonen P, Reponen J, Lääveri T, Vehko T, Saastamoinen P, Viitanen J, Heponiemi T. Associations of perceived changes in work due to digitalization and the amount of digital work with job strain among physicians: a national representative sample. BMC Med Inform Decis Mak 2023; 23:252. [PMID: 37940995 PMCID: PMC10631156 DOI: 10.1186/s12911-023-02351-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2023] [Accepted: 10/24/2023] [Indexed: 11/10/2023] Open
Abstract
BACKGROUND Physicians' work is often stressful. The digitalization of healthcare aims to streamline work, but not all physicians have experienced its realization. We examined associations of perceived changes in work due to digitalization and the amount of digital work with job strain among physicians. The moderating role of the length of work experience was investigated for these associations. METHODS We used representative survey data on Finnish physicians' (N = 4271) experiences of digitalization from 2021. The independent variables included perceptions on statements about work transformations aligned with digitalization goals, and the extent that information systems and teleconsultations were utilized. Stress related to information systems (SRIS), time pressure, and psychological stress were the dependent variables. We analyzed the associations using multivariable linear and logistic regressions. RESULTS Respondents had a mean SRIS score of 3.5 and a mean time pressure score of 3.7 on a scale of 1-5. Psychological stress was experienced by 60%. Perceptions associated with higher SRIS comprised disagreements with statements asserting that digitalization accelerates clinical encounters (b = .23 [95% CI: .16-.30]), facilitates access to patient information (b = .15 [.07-.23]), and supports decision-making (b = .11 [.05-.18]). Disagreement with accelerated clinical encounters (b = .12 [.04-.20]), and agreements with patients' more active role in care (b = .11 [.04-.19]) and interprofessional collaboration (b = .10 [.02-.18]) were opinions associated with greater time pressure. Disagreeing with supported decision-making (OR = 1.26 [1.06-1.48]) and agreeing with patients' active role (OR = 1.19 [1.02-1.40]) were associated with greater psychological stress. However, perceiving improvements in the pace of clinical encounters and access to patient information appeared to alleviate job strain. Additionally, extensive digital work was consistently linked to higher strain. Those respondents who held teleconsultations frequently and had less than 6 years of work experience reported the greatest levels of time pressure. CONCLUSIONS Physicians seem to be strained by frequent teleconsultations and work that does not meet the goals of digitalization. Improving physicians' satisfaction with digitalization through training specific to the stage of career and system development can be crucial for their well-being. Schedules for digital tasks should be planned and allocated to prevent strain related to achieving the digitalization goals.
Collapse
Affiliation(s)
- Lotta Virtanen
- Department of Public Health and Welfare, Finnish Institute for Health and Welfare, P.O.Box 30, 00271, Helsinki, Finland.
| | - Anu-Marja Kaihlanen
- Department of Public Health and Welfare, Finnish Institute for Health and Welfare, P.O.Box 30, 00271, Helsinki, Finland
| | - Petra Saukkonen
- Department of Public Health and Welfare, Finnish Institute for Health and Welfare, P.O.Box 30, 00271, Helsinki, Finland
| | - Jarmo Reponen
- Research Unit of Health Sciences and Technology, University of Oulu, P.O.Box 5000, 90014, Oulu, Finland
- Medical Research Center, Oulu University Hospital and University of Oulu, P.O.Box 8000, 90014, Oulu, Finland
| | - Tinja Lääveri
- Department of Infectious Diseases, University of Helsinki and Helsinki University Hospital, P.O.Box 700, 00029, Helsinki, Finland
- Department of Computer Science, Aalto University, P.O.Box 15400, 00076, Espoo, Finland
| | - Tuulikki Vehko
- Department of Public Health and Welfare, Finnish Institute for Health and Welfare, P.O.Box 30, 00271, Helsinki, Finland
| | | | - Johanna Viitanen
- Department of Computer Science, Aalto University, P.O.Box 15400, 00076, Espoo, Finland
| | - Tarja Heponiemi
- Department of Public Health and Welfare, Finnish Institute for Health and Welfare, P.O.Box 30, 00271, Helsinki, Finland
| |
Collapse
|
6
|
Joshi RN, Kalaminsky S, Feemster AA, Hill J, Leiman J, Evelyn D, Duncan R. A Data-Driven Approach to Evaluate Barcode-Assisted Medication Preparation Alerts at a Large Academic Medical Center. Jt Comm J Qual Patient Saf 2023; 49:599-603. [PMID: 37429757 DOI: 10.1016/j.jcjq.2023.06.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Revised: 06/12/2023] [Accepted: 06/19/2023] [Indexed: 07/12/2023]
Abstract
BACKGROUND The purpose of this study was to develop a data-driven process to analyze barcode-assisted medication preparation alert data with a goal of minimizing inaccurate alerts. METHODS Medication preparation data for the prior three-month period was obtained from an electronic health record system. A dashboard was developed to identify recurrent, high-volume alerts and associated medication records. A randomization tool was used to obtain a prespecified proportion of the alerts to review for appropriateness. Alert root causes were identified by chart review. Depending on the alert's cause(s), targeted informatics build changes, workflow and purchasing changes, and/or staff education were implemented. The rate of alerts was measured postintervention for select drugs. RESULTS The institution averaged 31,000 medication preparation alerts per month. The "barcode not recognized" alert (13,000) was the highest volume over the study period. Eighty-five medication records were identified as contributing to a high volume of alerts (5,200/31,000), representing 49 unique drugs. Of the 85 medication records triggering alerts, 36 required staff education, 22 required informatics build changes, and 8 required workflow changes. Targeted interventions for 2 medications, resulted in reducing the rate of the "barcode not recognized" alert from 26.6% to 1.3% for polyethylene glycol and from 48.7% to 0% for cyproheptadine. CONCLUSION This quality improvement project highlighted opportunities to improve medication purchasing, storage, and preparation through development of a standard process to evaluate barcode-assisted medication preparation alert data. A data-driven approach can help identify and minimize inaccurate alerts ("noise") and promote medication safety.
Collapse
|
7
|
Colicchio TK, Cimino JJ. Beyond the override: Using evidence of previous drug tolerance to suppress drug allergy alerts; a retrospective study of opioid alerts. J Biomed Inform 2023; 147:104508. [PMID: 37748541 DOI: 10.1016/j.jbi.2023.104508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Revised: 08/29/2023] [Accepted: 09/22/2023] [Indexed: 09/27/2023]
Abstract
OBJECTIVE Despite the extensive literature exploring alert fatigue, most studies have focused on describing the phenomenon, but not on fixing it. The authors aimed to identify data useful to avert clinically irrelevant alerts to inform future research on clinical decision support (CDS) design. METHODS We conducted a retrospective observational study of opioid drug allergy alert (DAA) overrides for the calendar year of 2019 at a large academic medical center, to identify data elements useful to find irrelevant alerts to be averted. RESULTS Overall, 227,815 DAAs were fired in 2019, with an override rate of 91 % (n = 208196). Opioids represented nearly two-thirds of these overrides (n = 129063; 62 %) and were the drug class with the highest override rate (96 %). On average, 29 opioid DAAs were overridden per patient. While most opioid alerts (97.1 %) are fired for a possible match (the drug class of the allergen matches the drug class of the prescribed drug), they are overridden significantly less frequently for definite match (exact match between allergen and prescribed drug) (88 % vs. 95.9 %, p < 0.001). When comparing the triggering drug with previously administered drugs, override rates were equally high for both definite match (95.9 %), no match (95.5 %), and possible match (95.1 %). Likewise, when comparing to home medications, overrides were excessively high for possible match (96.3 %), no match (96 %), and definite match (94.4 %). CONCLUSION We estimate that 74.5% of opioid DAAs (46.4% of all DAAs) at our institution could be relatively safely averted, since they either have a definite match for previous inpatient administrations suggesting drug tolerance or are fired as possible match with low risk of cross-sensitivity. Future research should focus on identifying other relevant data elements ideally with automated methods and use of emerging standards to empower CDS systems to suppress false-positive alerts while avoiding safety hazards.
Collapse
Affiliation(s)
- Tiago K Colicchio
- Informatics Institute, University of Alabama at Birmingham, AL, USA.
| | - James J Cimino
- Informatics Institute, University of Alabama at Birmingham, AL, USA
| |
Collapse
|
8
|
Cho HJ, Poeran J, Alaiev D, Tsega S, Israilov S, Krouss M. A Nonintrusive, Normative Nudge Intervention to Reduce Four Low-Value Tests in a Large Safety Net System. J Gen Intern Med 2023; 38:3086-3089. [PMID: 37488370 PMCID: PMC10593705 DOI: 10.1007/s11606-023-08339-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/04/2023] [Accepted: 07/13/2023] [Indexed: 07/26/2023]
Affiliation(s)
- Hyung J Cho
- Department of Quality and Safety, Brigham and Women's Hospital, Boston, MA, USA.
| | - Jashvant Poeran
- Institute for Healthcare Delivery Science, Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai Health System, New York, NY, USA
| | - Daniel Alaiev
- Department of Quality and Safety, NYC Health + Hospitals, New York, NY, USA
| | - Surafel Tsega
- Department of Quality and Safety, NYC Health + Hospitals, New York, NY, USA
| | - Sigal Israilov
- Department of Anesthesiology, Perioperative and Pain Medicine, Mount Sinai Hospital, New York, NY, USA
| | - Mona Krouss
- Department of Quality and Safety, NYC Health + Hospitals, New York, NY, USA
- Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|
9
|
Klaiman T, Iannotte LG, Josephs M, Russell LB, Norton L, Mehta S, Troxel A, Zhu J, Volpp K, Asch DA. Qualitative analysis of a remote monitoring intervention for managing heart failure. BMC Cardiovasc Disord 2023; 23:440. [PMID: 37679712 PMCID: PMC10486103 DOI: 10.1186/s12872-023-03456-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Accepted: 08/17/2023] [Indexed: 09/09/2023] Open
Abstract
BACKGROUND Heart failure (HF) is one of the most common reasons for hospital admission and is a major cause of morbidity, mortality, and increasing health care costs. The EMPOWER study was a randomized trial that used remote monitoring technology to track patients' weight and diuretic adherence and a state-of-the-art approach derived from behavioral economics to motivate adherence to the reverse monitoring technology. OBJECTIVE The goal was to explore patient and clinician perceptions of the program and its impact on perceived health outcomes and better understand why some patients or clinicians did better or worse than others in response to the intervention. APPROACH This was a retrospective qualitative study utilizing semi-structured interviews with 43 patients and 16 clinicians to understand the trial's processes, reflecting on successes and areas for improvement for future iterations of behavioral economic interventions. KEY RESULTS Many patients felt supported, and they appreciated the intervention. Many also appreciated the lottery intervention, and while it was not an incentive for enrolling for many respondents, it may have increased adherence during the study. Clinicians felt that the intervention integrated well into their workflow, but the number of alerts was burdensome. Additionally, responses to alerts varied considerably by provider, perhaps because there are no professional guidelines for alerts unaccompanied by severe symptoms. CONCLUSION Our qualitative analysis indicates potential areas for additional exploration and consideration to design better behavioral economic interventions to improve cardiovascular health outcomes for patients with HF. Patients appreciated lottery incentives for adhering to program requirements; however, many were too far along in their disease progression to benefit from the intervention. Clinicians found the amount and frequency of electronic alerts burdensome and felt they did not improve patient outcomes. TRIAL REGISTRATION ClinicalTrials.gov Identifier: NCT02708654.
Collapse
Affiliation(s)
- Tamar Klaiman
- University of Pennsylvania, Perelman School of Medicine, Philadelphia, USA.
| | - L G Iannotte
- The Lake Erie School of Osteopathic Medicine, Erie, USA
| | - Michael Josephs
- University of Pennsylvania, Perelman School of Medicine, Philadelphia, USA
| | - Louise B Russell
- University of Pennsylvania, Perelman School of Medicine, Philadelphia, USA
- Rutgers University, New Jersey, USA
| | - Laurie Norton
- University of Pennsylvania, Perelman School of Medicine, Philadelphia, USA
| | - Shivan Mehta
- University of Pennsylvania, Perelman School of Medicine, Philadelphia, USA
| | - Andrea Troxel
- New York University, Grossman School of Medicine, New York, USA
| | - Jingsan Zhu
- University of Pennsylvania, Perelman School of Medicine, Philadelphia, USA
| | - Kevin Volpp
- University of Pennsylvania, Perelman School of Medicine, Philadelphia, USA
| | - David A Asch
- University of Pennsylvania, Perelman School of Medicine, Philadelphia, USA
| |
Collapse
|
10
|
Ager EE, Sturdavant W, Curry Z, Ahmed F, DeJonckheere M, Gutting AA, Merchant RC, Kocher KE, Solnick RE. Mixed-methods Evaluation of an Expedited Partner Therapy Take-home Medication Program: Pilot Emergency Department Intervention to Improve Sexual Health Equity. West J Emerg Med 2023; 24:993-1004. [PMID: 37788042 PMCID: PMC10527844 DOI: 10.5811/westjem.59506] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 05/31/2023] [Accepted: 06/01/2023] [Indexed: 10/04/2023] Open
Abstract
Background: Treatment for partners of patients diagnosed with sexually transmitted infections (STI), referred to as expedited partner therapy (EPT), is infrequently used in the emergency department (ED). This was a pilot program to initiate and evaluate EPT through medication-in-hand ("take-home") kits or paper prescriptions. In this study we aimed to assess the frequency of EPT prescribing, the efficacy of a randomized best practice advisory (BPA) on the uptake, perceptions of emergency clinicians regarding the EPT pilot, and factors associated with EPT prescribing. Methods: We conducted this pilot study at an academic ED in the midwestern US between August-October 2021. The primary outcome of EPT prescription uptake and the BPA impact was measured via chart abstraction and analyzed through summary statistics and the Fisher exact test. We analyzed the secondary outcome of barriers and facilitators to program implementation through ED staff interviews (physicians, physician assistants, and nurses). We used a rapid qualitative assessment method for the analysis of the interviews. Results: During the study period, 52 ED patients were treated for chlamydia/gonorrhea, and EPT was offered to 25% (95% CI 15%-39%) of them. Expedited partner therapy was prescribed significantly more often (42% vs 8%; P < 0.01) when the interruptive pop-up alert BPA was shown compared to not shown. Barriers identified in the interviews included workflow constraints and knowledge of EPT availability. The BPA was viewed positively by the majority of participants. Conclusion: In this pilot EPT program, expedited partner therapy was provided to 25% of ED patients who appeared eligible to receive it. The interruptive pop-up alert BPA significantly increased EPT prescribing. Barriers identified to EPT prescribing should be the subject of future interventions to improve provision of EPT from the emergency department.
Collapse
Affiliation(s)
- Emily E Ager
- University of Michigan, School of Medicine, Department of Emergency Medicine, Ann Arbor, Michigan
| | - William Sturdavant
- University of Michigan, School of Medicine, Department of Emergency Medicine, Ann Arbor, Michigan
| | - Zoe Curry
- Vanderbilt University Medical Center, School of Medicine, Department of Emergency Medicine, Nashville, Tennessee
| | - Fahmida Ahmed
- University of Michigan, School of Medicine, Department of Emergency Medicine, Ann Arbor, Michigan
| | - Melissa DeJonckheere
- University of Michigan, School of Medicine, Department of Emergency Medicine, Ann Arbor, Michigan
| | - Andrew A Gutting
- University of Michigan, Michigan Medicine, Department of Clinical Quality, Ann Arbor, Michigan
| | | | - Keith E Kocher
- University of Michigan, School of Medicine, Department of Emergency Medicine, Ann Arbor, Michigan
- University of Michigan, Institute for Healthcare Policy and Innovation, Ann Arbor, Michigan
| | | |
Collapse
|
11
|
Nguyen TH, Cunha PP, Rowland AF, Orenstein E, Lee T, Kandaswamy S. User-Centered Design and Evaluation of Clinical Decision Support to Improve Early Peanut Introduction: Formative Study. JMIR Form Res 2023; 7:e47574. [PMID: 37606983 PMCID: PMC10481213 DOI: 10.2196/47574] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Revised: 06/15/2023] [Accepted: 07/21/2023] [Indexed: 08/23/2023] Open
Abstract
BACKGROUND Peanut allergy has recently become more prevalent. Peanut introduction recommendations have evolved from suggesting peanut avoidance until the age of 3 years to more recent guidelines encouraging early peanut introduction after the Learning Early about Peanut Allergy (LEAP) study in 2015. Guideline adherence is poor, leading to missed care opportunities. OBJECTIVE In this study, we aimed to develop a user-centered clinical decision support (CDS) tool to improve implementation of the most recent early peanut introduction guidelines in the primary care clinic setting. METHODS We edited the note template of the well-child check (WCC) visits at ages 4 and 6 months with CDS prompts and point-of-care education. Formative and summative usability testing were completed with pediatric residents in a simulated electronic health record (EHR). We estimated task completion rates and perceived usefulness of the CDS in summative testing, comparing a test EHR with and without the CDS. RESULTS Formative usability testing with the residents provided qualitative data that led to improvements in the build for both the 4-month and 6-month WCC note templates. During summative usability testing, the CDS tool significantly improved discussion of early peanut introduction at the 4-month WCC visit compared to scenarios without the CDS tool (9/15, 60% with CDS and 0/15, 0% without CDS). All providers except one at the 4-month WCC scenario gave at least an adequate score for the ease of use of the CDS tool for the history of present illness and assessment and plan sections. During the summative usability testing with the 6-month WCC new build note template, providers more commonly provided comprehensive care once obtaining a patient history concerning for an immunoglobulin E-mediated peanut reaction by placing a referral to allergy/immunology (P=.48), prescribing an epinephrine auto-injector (P=.07), instructing on how to avoid peanut products (P<.001), and providing an emergency treatment plan (P=.003) with CDS guidance. All providers gave at least an adequate score for ease of use of the CDS tool in the after-visit summary. CONCLUSIONS User-centered CDS improved application of early peanut introduction recommendations and comprehensive care for patients who have symptoms concerning for peanut allergy in a simulation.
Collapse
Affiliation(s)
- Thinh Hoang Nguyen
- Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, United States
- Division of Immunology, Boston Children's Hospital, Boston, MA, United States
| | - Priscila Pereira Cunha
- Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, United States
| | | | - Evan Orenstein
- Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, United States
- Division of Hospital Medicine, Children's Healthcare of Atlanta, Atlanta, GA, United States
| | - Tricia Lee
- Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, United States
- Department of Allergy and Immunology, Children's Healthcare of Atlanta, Atlanta, GA, United States
| | - Swaminathan Kandaswamy
- Department of Pediatrics, Emory University School of Medicine, Atlanta, GA, United States
| |
Collapse
|
12
|
Rottman BM, Caddick ZA, Nokes-Malach TJ, Fraundorf SH. Cognitive perspectives on maintaining physicians' medical expertise: I. Reimagining Maintenance of Certification to promote lifelong learning. Cogn Res Princ Implic 2023; 8:46. [PMID: 37486508 PMCID: PMC10366070 DOI: 10.1186/s41235-023-00496-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/01/2022] [Accepted: 06/20/2023] [Indexed: 07/25/2023] Open
Abstract
Until recently, physicians in the USA who were board-certified in a specialty needed to take a summative test every 6-10 years. However, the 24 Member Boards of the American Board of Medical Specialties are in the process of switching toward much more frequent assessments, which we refer to as longitudinal assessment. The goal of longitudinal assessments is to provide formative feedback to physicians to help them learn content they do not know as well as serve an evaluation for board certification. We present five articles collectively covering the science behind this change, the likely outcomes, and some open questions. This initial article introduces the context behind this change. This article also discusses various forms of lifelong learning opportunities that can help physicians stay current, including longitudinal assessment, and the pros and cons of each.
Collapse
Affiliation(s)
- Benjamin M Rottman
- Learning Research and Development Center, University of Pittsburgh, 3420 Forbes Ave., Pittsburgh, PA, 15260, USA
- Department of Psychology, University of Pittsburgh, Pittsburgh, USA
| | - Zachary A Caddick
- Learning Research and Development Center, University of Pittsburgh, 3420 Forbes Ave., Pittsburgh, PA, 15260, USA
- Department of Psychology, University of Pittsburgh, Pittsburgh, USA
| | - Timothy J Nokes-Malach
- Learning Research and Development Center, University of Pittsburgh, 3420 Forbes Ave., Pittsburgh, PA, 15260, USA
- Department of Psychology, University of Pittsburgh, Pittsburgh, USA
| | - Scott H Fraundorf
- Learning Research and Development Center, University of Pittsburgh, 3420 Forbes Ave., Pittsburgh, PA, 15260, USA.
- Department of Psychology, University of Pittsburgh, Pittsburgh, USA.
| |
Collapse
|
13
|
Cull J, Brevetta R, Gerac J, Kothari S, Blackhurst D. Epic Sepsis Model Inpatient Predictive Analytic Tool: A Validation Study. Crit Care Explor 2023; 5:e0941. [PMID: 37405252 PMCID: PMC10317482 DOI: 10.1097/cce.0000000000000941] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/06/2023] Open
Abstract
Earlier treatment of sepsis leads to decreased mortality. Epic is an electronic medical record providing a predictive alert system for sepsis, the Epic Sepsis Model (ESM) Inpatient Predictive Analytic Tool. External validation of this system is lacking. This study aims to evaluate the ESM as a sepsis screening tool and determine whether an association exists between ESM alert system implementation and subsequent sepsis-related mortality. DESIGN Before-and-after study comparing baseline and intervention period. SETTING Urban 746-bed academic level 1 trauma center. PATIENTS Adult acute care inpatients discharged between January 12, 2018, and July 31, 2019. INTERVENTIONS During the before period, ESM was turned on in the background, but nurses and providers were not alerted of results. The system was then activated to alert providers of scores greater than or equal to 5, a set point determined using receiver operating characteristic curve analysis (area under the curve, 0.834; p < 0.001). MEASUREMENTS AND MAIN RESULTS Primary outcome was mortality during hospitalization; secondary outcomes were sepsis order set utilization, length of stay, and timing of administration of sepsis-appropriate antibiotics. Of the 11,512 inpatient encounters assessed by ESM, 10.2% (1,171) had sepsis based on diagnosis codes. As a screening test, the ESM had sensitivity, specificity, positive predictive value, and negative predictive value rates of 86.0%, 80.8%, 33.8%, and 98.11%, respectively. After ESM implementation, unadjusted mortality rates in patients with ESM score greater than or equal to 5 and who had not yet received sepsis-appropriate antibiotics declined from 24.3% to 15.9%; multivariable analysis yielded an odds ratio of sepsis-related mortality (95% CI) of 0.56 (0.39-0.80). CONCLUSIONS In this single-center before-and-after study, utilization of the ESM score as a screening test was associated with a 44% reduction in the odds of sepsis-related mortality. Due to wide utilization of Epic, this is a potentially promising tool to improve sepsis mortality in the United States. This study is hypothesis generating, and further work with more rigorous study design is needed.
Collapse
Affiliation(s)
- John Cull
- All authors: Prisma Health, Greenville, SC
| | | | - Jeff Gerac
- All authors: Prisma Health, Greenville, SC
| | | | | |
Collapse
|
14
|
Khan MS, Usman MS, Talha KM, Van Spall HGC, Greene SJ, Vaduganathan M, Khan SS, Mills NL, Ali ZA, Mentz RJ, Fonarow GC, Rao SV, Spertus JA, Roe MT, Anker SD, James SK, Butler J, McGuire DK. Leveraging electronic health records to streamline the conduct of cardiovascular clinical trials. Eur Heart J 2023; 44:1890-1909. [PMID: 37098746 DOI: 10.1093/eurheartj/ehad171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/17/2022] [Revised: 02/05/2023] [Accepted: 03/07/2023] [Indexed: 04/27/2023] Open
Abstract
Conventional randomized controlled trials (RCTs) can be expensive, time intensive, and complex to conduct. Trial recruitment, participation, and data collection can burden participants and research personnel. In the past two decades, there have been rapid technological advances and an exponential growth in digitized healthcare data. Embedding RCTs, including cardiovascular outcome trials, into electronic health record systems or registries may streamline screening, consent, randomization, follow-up visits, and outcome adjudication. Moreover, wearable sensors (i.e. health and fitness trackers) provide an opportunity to collect data on cardiovascular health and risk factors in unprecedented detail and scale, while growing internet connectivity supports the collection of patient-reported outcomes. There is a pressing need to develop robust mechanisms that facilitate data capture from diverse databases and guidance to standardize data definitions. Importantly, the data collection infrastructure should be reusable to support multiple cardiovascular RCTs over time. Systems, processes, and policies will need to have sufficient flexibility to allow interoperability between different sources of data acquisition. Clinical research guidelines, ethics oversight, and regulatory requirements also need to evolve. This review highlights recent progress towards the use of routinely generated data to conduct RCTs and discusses potential solutions for ongoing barriers. There is a particular focus on methods to utilize routinely generated data for trials while complying with regional data protection laws. The discussion is supported with examples of cardiovascular outcome trials that have successfully leveraged the electronic health record, web-enabled devices or administrative databases to conduct randomized trials.
Collapse
Affiliation(s)
- Muhammad Shahzeb Khan
- Division of Cardiology, Duke University School of Medicine, 2301 Erwin Rd., Durham, NC 27705, USA
| | - Muhammad Shariq Usman
- Department of Medicine, University of Mississippi Medical Center, 2500 N State St, Jackson, MS 39216, USA
| | - Khawaja M Talha
- Department of Medicine, University of Mississippi Medical Center, 2500 N State St, Jackson, MS 39216, USA
| | - Harriette G C Van Spall
- Department of Medicine, McMaster University, Hamilton, ON, Canada
- Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, ON, Canada
- Population Health Research Institute, Hamilton, ON, Canada
| | - Stephen J Greene
- Division of Cardiology, Duke University School of Medicine, 2301 Erwin Rd., Durham, NC 27705, USA
- Duke Clinical Research Institute, Durham, NC, USA
| | - Muthiah Vaduganathan
- Cardiovascular Division, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Sadiya S Khan
- Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Nicholas L Mills
- BHF Centre for Cardiovascular Science, University of Edinburgh, Chancellors Building, Royal Infirmary of Edinburgh, Edinburgh, Scotland, UK
- Usher Institute, University of Edinburgh, Edinburgh, Scotland, UK
| | - Ziad A Ali
- DeMatteis Cardiovascular Institute, St Francis Hospital and Heart Center, Roslyn, NY, USA
| | - Robert J Mentz
- Division of Cardiology, Duke University School of Medicine, 2301 Erwin Rd., Durham, NC 27705, USA
- Duke Clinical Research Institute, Durham, NC, USA
| | - Gregg C Fonarow
- Division of Cardiology, University of California Los Angeles, Los Angeles, CA, USA
| | - Sunil V Rao
- Division of Cardiology, New York University Langone Health System, New York, NY, USA
| | - John A Spertus
- Department of Cardiology, Saint Luke's Mid America Heart Institute, Kansas City, MO, USA
- Kansas City's Healthcare Institute for Innovations in Quality, University of Missouri, Kansas, MO, USA
| | - Matthew T Roe
- Division of Cardiology, Duke University School of Medicine, 2301 Erwin Rd., Durham, NC 27705, USA
- Duke Clinical Research Institute, Durham, NC, USA
| | - Stefan D Anker
- Department of Cardiology (CVK), Berlin Institute of Health Center for Regenerative Therapies (BCRT), and German Centre for Cardiovascular Research (DZHK) Partner Site Berlin, Charité Universitätsmedizin, Berlin, Germany
| | - Stefan K James
- Department of Medical Sciences, Scientific Director UCR, Uppsala University, Uppsala, Uppland, Sweden
| | - Javed Butler
- Department of Medicine, University of Mississippi Medical Center, 2500 N State St, Jackson, MS 39216, USA
- Baylor Scott & White Research Institute, Dallas, TX, USA
| | - Darren K McGuire
- Division of Cardiology, Department of Internal Medicine, UT Southwestern Medical Center and Parkland Health and Hospital System, Dallas, TX, USA
| |
Collapse
|
15
|
Ma JE, Lowe J, Berkowitz C, Kim A, Togo I, Musser RC, Fischer J, Shah K, Ibrahim S, Bosworth HB, Totten AM, Dolor R. Provider Interaction With an Electronic Health Record Notification to Identify Eligible Patients for a Cluster Randomized Trial of Advance Care Planning in Primary Care: Secondary Analysis. J Med Internet Res 2023; 25:e41884. [PMID: 37171856 DOI: 10.2196/41884] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2022] [Revised: 02/17/2023] [Accepted: 03/21/2023] [Indexed: 05/13/2023] Open
Abstract
BACKGROUND Advance care planning (ACP) improves patient-provider communication and aligns care to patient values, preferences, and goals. Within a multisite Meta-network Learning and Research Center ACP study, one health system deployed an electronic health record (EHR) notification and algorithm to alert providers about patients potentially appropriate for ACP and the clinical study. OBJECTIVE The aim of the study is to describe the implementation and usage of an EHR notification for referring patients to an ACP study, evaluate the association of notifications with study referrals and engagement in ACP, and assess provider interactions with and perspectives on the notifications. METHODS A secondary analysis assessed provider usage and their response to the notification (eg, acknowledge, dismiss, or engage patient in ACP conversation and refer patient to the clinical study). We evaluated all patients identified by the EHR algorithm during the Meta-network Learning and Research Center ACP study. Descriptive statistics compared patients referred to the study to those who were not referred to the study. Health care utilization, hospice referrals, and mortality as well as documentation and billing for ACP and related legal documents are reported. We evaluated associations between notifications with provider actions (ie, referral to study, ACP not documentation, and ACP billing). Provider free-text comments in the notifications were summarized qualitatively. Providers were surveyed on their satisfaction with the notification. RESULTS Among the 2877 patients identified by the EHR algorithm over 20 months, 17,047 unique notifications were presented to 45 providers in 6 clinics, who then referred 290 (10%) patients. Providers had a median of 269 (IQR 65-552) total notifications, and patients had a median of 4 (IQR 2-8). Patients with more (over 5) notifications were less likely to be referred to the study than those with fewer notifications (57/1092, 5.2% vs 233/1785, 13.1%; P<.001). The most common free-text comment on the notification was lack of time. Providers who referred patients to the study were more likely to document ACP and submit ACP billing codes (P<.001). In the survey, 11 providers would recommend the notification (n=7, 64%); however, the notification impacted clinical workflow (n=9, 82%) and was difficult to navigate (n=6, 55%). CONCLUSIONS An EHR notification can be implemented to remind providers to both perform ACP conversations and refer patients to a clinical study. There were diminishing returns after the fifth EHR notification where additional notifications did not lead to more trial referrals, ACP documentation, or ACP billing. Creation and optimization of EHR notifications for study referrals and ACP should consider the provider user, their workflow, and alert fatigue to improve implementation and adoption. TRIAL REGISTRATION ClinicalTrials.gov NCT03577002; https://clinicaltrials.gov/ct2/show/NCT03577002.
Collapse
Affiliation(s)
- Jessica E Ma
- Geriatric Research, Education, and Clinical Center, Durham Veterans Affairs Health Care System, Durham, NC, United States
- Division of General Internal Medicine, Department of Medicine, Duke University School of Medicine, Durham, NC, United States
| | - Jared Lowe
- Division of General Medicine & Clinical Epidemiology, Department of Medicine, University of North Carolina School of Medicine, Chapel Hill, NC, United States
| | - Callie Berkowitz
- Division of Hematology and Oncology, Department of Medicine, University of North Carolina School of Medicine, Chapel Hill, NC, United States
| | - Azalea Kim
- Division of General Internal Medicine, Department of Medicine, Duke University School of Medicine, Durham, NC, United States
| | - Ira Togo
- Duke Office of Clinical Research, Durham, NC, United States
| | - R Clayton Musser
- Division of General Internal Medicine, Department of Medicine, Duke University School of Medicine, Durham, NC, United States
| | - Jonathan Fischer
- Department of Community & Family Medicine, Duke University School of Medicine, Durham, NC, United States
- Duke Population Health Management Office, Durham, NC, United States
| | - Kevin Shah
- Department of Medicine, Duke University School of Medicine, Durham, NC, United States
| | - Salam Ibrahim
- Duke Health Performance Services, Duke University Health System, Durham, NC, United States
| | - Hayden B Bosworth
- Division of General Internal Medicine, Department of Medicine, Duke University School of Medicine, Durham, NC, United States
- Department of Community & Family Medicine, Duke University School of Medicine, Durham, NC, United States
- Department of Medicine, Duke University School of Medicine, Durham, NC, United States
- Department of Psychiatry and Behavioral Services, Duke University School of Medicine, Durham, NC, United States
- Department of Population Health Sciences, Duke University School of Medicine, Durham, NC, United States
- Center of Innovation to Accelerate Discovery and Practice Transformation, Durham Veterans Affairs Health Care System, Durham, NC, United States
| | - Annette M Totten
- Oregon Rural Practice Based Research Network, Oregon Health & Science University School of Medicine, Portland, OR, United States
| | - Rowena Dolor
- Division of General Medicine & Clinical Epidemiology, Department of Medicine, University of North Carolina School of Medicine, Chapel Hill, NC, United States
| |
Collapse
|
16
|
Samal L, Wu E, Aaron S, Kilgallon JL, Gannon M, McCoy A, Blecker S, Dykes PC, Bates DW, Lipsitz S, Wright A. Refining Clinical Phenotypes to Improve Clinical Decision Support and Reduce Alert Fatigue: A Feasibility Study. Appl Clin Inform 2023; 14:528-537. [PMID: 37437601 PMCID: PMC10338104 DOI: 10.1055/s-0043-1768994] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2022] [Accepted: 04/18/2023] [Indexed: 07/14/2023] Open
Abstract
BACKGROUND Chronic kidney disease (CKD) is common and associated with adverse clinical outcomes. Most care for early CKD is provided in primary care, including hypertension (HTN) management. Computerized clinical decision support (CDS) can improve the quality of care for CKD but can also cause alert fatigue for primary care physicians (PCPs). Computable phenotypes (CPs) are algorithms to identify disease populations using, for example, specific laboratory data criteria. OBJECTIVES Our objective was to determine the feasibility of implementation of CDS alerts by developing CPs and estimating potential alert burden. METHODS We utilized clinical guidelines to develop a set of five CPs for patients with stage 3 to 4 CKD, uncontrolled HTN, and indications for initiation or titration of guideline-recommended antihypertensive agents. We then conducted an iterative data analytic process consisting of database queries, data validation, and subject matter expert discussion, to make iterative changes to the CPs. We estimated the potential alert burden to make final decisions about the scope of the CDS alerts. Specifically, the number of times that each alert could fire was limited to once per patient. RESULTS In our primary care network, there were 239,339 encounters for 105,992 primary care patients between April 1, 2018 and April 1, 2019. Of these patients, 9,081 (8.6%) had stage 3 and 4 CKD. Almost half of the CKD patients, 4,191 patients, also had uncontrolled HTN. The majority of CKD patients were female, elderly, white, and English-speaking. We estimated that 5,369 alerts would fire if alerts were triggered multiple times per patient, with a mean number of alerts shown to each PCP ranging from 0.07-to 0.17 alerts per week. CONCLUSION Development of CPs and estimation of alert burden allows researchers to iteratively fine-tune CDS prior to implementation. This method of assessment can help organizations balance the tradeoff between standardization of care and alert fatigue.
Collapse
Affiliation(s)
- Lipika Samal
- Department of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Harvard Medical School, Boston, Massachusetts, United States
| | - Edward Wu
- Department of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Alabama College of Osteopathic Medicine, Dothan, Alabama, United States
| | - Skye Aaron
- Department of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
| | - John L. Kilgallon
- Department of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
| | - Michael Gannon
- Department of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Eastern Virginia Medical School, Norfolk, Virginia, United States
| | - Allison McCoy
- Vanderbilt University, Nashville, Tennessee, United States
| | - Saul Blecker
- NYU School of Medicine, New York, New York, United States
| | - Patricia C. Dykes
- Department of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Harvard Medical School, Boston, Massachusetts, United States
| | - David W. Bates
- Department of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Harvard Medical School, Boston, Massachusetts, United States
| | - Stuart Lipsitz
- Department of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Harvard Medical School, Boston, Massachusetts, United States
- Department of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, United States
| | - Adam Wright
- Vanderbilt University, Nashville, Tennessee, United States
| |
Collapse
|
17
|
Porter KM, Kraft SA, Speight CD, Duenas DM, Niyibizi NK, Mitchell A, O’Connor MR, Gregor C, Liljenquist K, Shah SK, Wilfond BS, Dickert NW. Research recruitment through the patient portal: perspectives of community focus groups in Seattle and Atlanta. JAMIA Open 2023; 6:ooad004. [PMID: 36751464 PMCID: PMC9897173 DOI: 10.1093/jamiaopen/ooad004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2022] [Revised: 06/16/2022] [Accepted: 01/13/2023] [Indexed: 02/05/2023] Open
Abstract
Objective Research recruitment through patient portals (ie, patient-facing, web-based clinical interfaces) has the potential to be effective, efficient, and inclusive, but best practices remain undefined. We sought to better understand how patients view this recruitment approach. Materials and Methods We conducted 6 focus groups in Atlanta, GA and Seattle, WA with members of patient advisory committees and the general public. Discussions addressed acceptability of patient portal recruitment and communication preferences. Focus groups were audio-recorded, transcribed, and analyzed using deductive and inductive codes. Iterative team discussions identified major themes. Results Of 49 total participants, 20 were patient advisory committee members. Participants' mean age was 49 (range 18-74); 59% identified as non-Hispanic White and 31% as Black/African American. Participants were supportive of patient portal recruitment and confident that messages were private and legitimate. Participants identified transparency and patient control over whether and how to participate as essential features. Concerns included the frequency of research messages and the ability to distinguish between research and clinical messages. Participants also discussed how patient portal recruitment might affect diversity and inclusion. Discussion Focus group participants generally found patient portal recruitment acceptable and perceived it as secure and trustworthy. Transparency, control, and attention to inclusiveness were identified as key considerations for developing best practices. Conclusion For institutions implementing patient portal recruitment programs, continued engagement with patient populations can help facilitate translation of these findings into best practices and ensure that implemented strategies accomplish intended goals.
Collapse
Affiliation(s)
- Kathryn M Porter
- Treuman Katz Center for Pediatric Bioethics, Seattle Children’s Research Institute, Seattle, Washington, USA
| | - Stephanie A Kraft
- Treuman Katz Center for Pediatric Bioethics, Seattle Children’s Research Institute, Seattle, Washington, USA
- Department of Pediatrics, University of Washington School of Medicine, Seattle, Washington, USA
| | - Candace D Speight
- Division of Cardiology, Department of Medicine, Emory University School of Medicine, Atlanta, Georgia, USA
- Georgia Clinical and Translational Science Alliance, Emory University, Atlanta, Georgia, USA
| | - Devan M Duenas
- Treuman Katz Center for Pediatric Bioethics, Seattle Children’s Research Institute, Seattle, Washington, USA
| | - Nyiramugisha K Niyibizi
- Georgia Clinical and Translational Science Alliance, Emory University, Atlanta, Georgia, USA
| | - Andrea Mitchell
- Division of Cardiology, Department of Medicine, Emory University School of Medicine, Atlanta, Georgia, USA
- Georgia Clinical and Translational Science Alliance, Emory University, Atlanta, Georgia, USA
| | - M Rebecca O’Connor
- Child, Family & Population Health Nursing, School of Nursing, University of Washington, Seattle, Washington, USA
| | - Charles Gregor
- Institute of Translational Health Sciences, University of Washington, Seattle, Washington, USA
| | - Kendra Liljenquist
- Department of Pediatrics, University of Washington School of Medicine, Seattle, Washington, USA
| | - Seema K Shah
- Bioethics Program, Lurie Children’s Hospital, Chicago, Illinois, USA
- Department of Pediatrics, Northwestern Feinberg School of Medicine, Chicago, Illinois, USA
| | - Benjamin S Wilfond
- Treuman Katz Center for Pediatric Bioethics, Seattle Children’s Research Institute, Seattle, Washington, USA
- Department of Pediatrics, University of Washington School of Medicine, Seattle, Washington, USA
| | - Neal W Dickert
- Division of Cardiology, Department of Medicine, Emory University School of Medicine, Atlanta, Georgia, USA
- Georgia Clinical and Translational Science Alliance, Emory University, Atlanta, Georgia, USA
| |
Collapse
|
18
|
Vinson DR, Rauchwerger AS, Karadi CA, Shan J, Warton EM, Zhang JY, Ballard DW, Mark DG, Hofmann ER, Cotton DM, Durant EJ, Lin JS, Sax DR, Poth LS, Gamboa SH, Ghiya MS, Kene MV, Ganapathy A, Whiteley PM, Bouvet SC, Babakhanian L, Kwok EW, Solomon MD, Go AS, Reed ME. Clinical decision support to Optimize Care of patients with Atrial Fibrillation or flutter in the Emergency department: protocol of a stepped-wedge cluster randomized pragmatic trial (O'CAFÉ trial). Trials 2023; 24:246. [PMID: 37004068 PMCID: PMC10064588 DOI: 10.1186/s13063-023-07230-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Accepted: 03/08/2023] [Indexed: 04/03/2023] Open
Abstract
BACKGROUND Management of adults with atrial fibrillation (AF) or atrial flutter in the emergency department (ED) includes rate reduction, cardioversion, and stroke prevention. Different approaches to these components of care may lead to variation in frequency of hospitalization and stroke prevention actions, with significant implications for patient experience, cost of care, and risk of complications. Standardization using evidence-based recommendations could reduce variation in management, preventable hospitalizations, and stroke risk. METHODS We describe the rationale for our ED-based AF treatment recommendations. We also describe the development of an electronic clinical decision support system (CDSS) to deliver these recommendations to emergency physicians at the point of care. We implemented the CDSS at three pilot sites to assess feasibility and solicit user feedback. We will evaluate the impact of the CDSS on hospitalization and stroke prevention actions using a stepped-wedge cluster randomized pragmatic clinical trial across 13 community EDs in Northern California. DISCUSSION We hypothesize that the CDSS intervention will reduce hospitalization of adults with isolated AF or atrial flutter presenting to the ED and increase anticoagulation prescription in eligible patients at the time of ED discharge and within 30 days. If our hypotheses are confirmed, the treatment protocol and CDSS could be recommended to other EDs to improve management of adults with AF or atrial flutter. TRIAL REGISTRATION ClinicalTrials.gov NCT05009225 . Registered on 17 August 2021.
Collapse
Affiliation(s)
- David R Vinson
- The Permanente Medical Group, Oakland, CA, USA.
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA.
- Department of Emergency Medicine, Kaiser Permanente Roseville Medical Center, Roseville, CA, USA.
| | - Adina S Rauchwerger
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
| | - Chandu A Karadi
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente San Jose Medical Center, San Jose, CA, USA
| | - Judy Shan
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
- School of Medicine, University of California, San Francisco, San Francisco, CA, USA
| | - E Margaret Warton
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
| | - Jennifer Y Zhang
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
| | - Dustin W Ballard
- The Permanente Medical Group, Oakland, CA, USA
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente San Rafael Medical Center, San Rafael, CA, USA
| | - Dustin G Mark
- The Permanente Medical Group, Oakland, CA, USA
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente Oakland Medical Center, Oakland, CA, USA
| | - Erik R Hofmann
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente South Sacramento Medical Center, Sacramento, CA, USA
| | - Dale M Cotton
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente South Sacramento Medical Center, Sacramento, CA, USA
| | - Edward J Durant
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente Modesto Medical Center, Modesto, CA, USA
| | - James S Lin
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente Santa Clara Medical Center, Santa Clara, CA, USA
| | - Dana R Sax
- The Permanente Medical Group, Oakland, CA, USA
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente Oakland Medical Center, Oakland, CA, USA
| | - Luke S Poth
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente Walnut Creek Medical Center, Walnut Creek, CA, USA
| | - Stephen H Gamboa
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente San Francisco Medical Center, San Francisco, CA, USA
| | - Meena S Ghiya
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente South San Francisco Medical Center, San Francisco, CA, USA
| | - Mamata V Kene
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente San Leandro Medical Center, San Leandro, CA, USA
| | - Anuradha Ganapathy
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente San Jose Medical Center, San Jose, CA, USA
| | - Patrick M Whiteley
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente San Jose Medical Center, San Jose, CA, USA
| | - Sean C Bouvet
- The Permanente Medical Group, Oakland, CA, USA
- Department of Emergency Medicine, Kaiser Permanente Walnut Creek Medical Center, Walnut Creek, CA, USA
| | | | | | - Matthew D Solomon
- The Permanente Medical Group, Oakland, CA, USA
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
- Department of Cardiology, Oakland Medical Center, Oakland, CA, USA
| | - Alan S Go
- The Permanente Medical Group, Oakland, CA, USA
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
- Departments of Epidemiology, Biostatistics, and Medicine, University of California, San Francisco, CA, USA
- Department of Medicine, Stanford University, Palo Alto, CA, USA
| | - Mary E Reed
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
| |
Collapse
|
19
|
Seki T, Aki M, Furukawa TA, Kawashima H, Miki T, Sawaki Y, Ando T, Katsuragi K, Kawashima T, Ueno S, Miyagi T, Noma S, Tanaka S, Kawakami K. Electronic Health Record-Nested Reminders for Serum Lithium Level Monitoring in Patients With Mood Disorder: Randomized Controlled Trial. J Med Internet Res 2023; 25:e40595. [PMID: 36947138 PMCID: PMC10139684 DOI: 10.2196/40595] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2022] [Revised: 01/12/2023] [Accepted: 02/21/2023] [Indexed: 03/23/2023] Open
Abstract
BACKGROUND Clinical guidelines recommend regular serum lithium monitoring every 3 to 6 months. However, in the real world, only a minority of patients receive adequate monitoring. OBJECTIVE This study aims to examine whether the use of the electronic health record (EHR)-nested reminder system for serum lithium monitoring can help achieve serum lithium concentrations within the therapeutic range for patients on lithium maintenance therapy. METHODS We conducted an unblinded, single-center, EHR-nested, parallel-group, superiority randomized controlled trial comparing EHR-nested reminders with usual care in adult patients receiving lithium maintenance therapy for mood disorders. The primary outcome was the achievement of therapeutically appropriate serum lithium levels between 0.4 and 1.0 mEq/L at 18 months after enrollment. The key secondary outcomes are included as follows: the number of serum lithium level monitoring except for the first and final monitoring; exacerbation of the mood disorder during the study period, defined by hospitalization, increase in lithium dose, addition of antipsychotic drugs or mood stabilizers, or addition or increase of antidepressants; adherence defined by the proportion of days covered by lithium carbonate prescription during the study period. RESULTS A total of 111 patients were enrolled in this study. A total of 56 patients were assigned to the reminder group, and 55 patients were assigned to the usual care group. At the follow-up, 38 (69.1%) patients in the reminder group and 33 (60.0%) patients in the usual care group achieved the primary outcome (odds ratio 2.14, 95% CI 0.82-5.58, P=.12). The median number of serum lithium monitoring was 2 in the reminder group and 0 in the usual care group (rate ratio 3.62; 95% CI 2.47-5.29, P<.001). The exacerbation of mood disorders occurred in 17 (31.5%) patients in the reminder group and in 16 (34.8%) patients in the usual care group (odds ratio 0.97, 95% CI 0.42-2.28, P=.95). CONCLUSIONS We found insufficient evidence for an EHR-nested reminder to increase the achievement of therapeutic serum lithium concentrations. However, the number of monitoring increased with relatively simple and inexpensive intervention. The EHR-based reminders may be useful to improve quality of care for patients on lithium maintenance therapy, and they have potentials to be applied to other problems. TRIAL REGISTRATION University Hospital Medical Information Network Clinical Trials Registry UMIN000033633; https://tinyurl.com/5n7wtyav.
Collapse
Affiliation(s)
- Tomotsugu Seki
- Department of Pharmacoepidemiology, School of Public Health, Graduate School of Medicine, Kyoto University, Kyoto, Japan
- Department of Cardiovascular Medicine, Graduate School of Medical Science, Kyoto Prefectural University of Medicine, Kyoto, Japan
| | - Morio Aki
- Department of Psychiatry, Toyooka Hospital, Toyooka, Japan
- Department of Psychiatry, Kyoto University Hospital, Kyoto, Japan
| | - Toshi A Furukawa
- Department of Health Promotion and Human Behavior, School of Public Health, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Hirotsugu Kawashima
- Department of Psychiatry, Toyooka Hospital, Toyooka, Japan
- Department of Psychiatry, Kyoto University Hospital, Kyoto, Japan
| | - Tomotaka Miki
- Department of Psychiatry, Toyooka Hospital, Toyooka, Japan
- Department of Psychiatry, Kyoto University Hospital, Kyoto, Japan
| | - Yujin Sawaki
- Department of Psychiatry, Toyooka Hospital, Toyooka, Japan
- Department of Psychiatry, Kyoto University Hospital, Kyoto, Japan
- National Epilepsy Center, National Hospital Organization Shizuoka Institute of Epilepsy and Neurological Disorders, Shizuoka, Japan
| | - Takaaki Ando
- Department of Psychiatry, Toyooka Hospital, Toyooka, Japan
- Department of Psychiatry, Kyoto University Hospital, Kyoto, Japan
| | - Kentaro Katsuragi
- Department of Psychiatry, Toyooka Hospital, Toyooka, Japan
- Department of Psychiatry, Kyoto University Hospital, Kyoto, Japan
| | - Takahiko Kawashima
- Department of Psychiatry, Toyooka Hospital, Toyooka, Japan
- Department of Psychiatry, Kyoto University Hospital, Kyoto, Japan
| | - Senkei Ueno
- Department of Psychiatry, Toyooka Hospital, Toyooka, Japan
- Department of Psychiatry, Kyoto University Hospital, Kyoto, Japan
| | - Takashi Miyagi
- Department of Psychiatry, Toyooka Hospital, Toyooka, Japan
- Department of Psychiatry, Kyoto University Hospital, Kyoto, Japan
- Department of Psychiatry, Kyoto-Katsura Hospital, Kyoto, Japan
| | - Shun'ichi Noma
- Department of Psychiatry, Toyooka Hospital, Toyooka, Japan
- Department of Psychiatry, Kyoto University Hospital, Kyoto, Japan
- Noma-Kokoro Clinic, Kyoto, Japan
| | - Shiro Tanaka
- Department of Clinical Biostatistics, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | - Koji Kawakami
- Department of Pharmacoepidemiology, School of Public Health, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| |
Collapse
|
20
|
Chen JS, Baxter SL, van den Brandt A, Lieu A, Camp AS, Do JL, Welsbie DS, Moghimi S, Christopher M, Weinreb RN, Zangwill LM. Usability and Clinician Acceptance of a Deep Learning-Based Clinical Decision Support Tool for Predicting Glaucomatous Visual Field Progression. J Glaucoma 2023; 32:151-158. [PMID: 36877820 PMCID: PMC9996451 DOI: 10.1097/ijg.0000000000002163] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Accepted: 11/19/2023] [Indexed: 03/08/2023]
Abstract
PRCIS We updated a clinical decision support tool integrating predicted visual field (VF) metrics from an artificial intelligence model and assessed clinician perceptions of the predicted VF metric in this usability study. PURPOSE To evaluate clinician perceptions of a prototyped clinical decision support (CDS) tool that integrates visual field (VF) metric predictions from artificial intelligence (AI) models. METHODS Ten ophthalmologists and optometrists from the University of California San Diego participated in 6 cases from 6 patients, consisting of 11 eyes, uploaded to a CDS tool ("GLANCE", designed to help clinicians "at a glance"). For each case, clinicians answered questions about management recommendations and attitudes towards GLANCE, particularly regarding the utility and trustworthiness of the AI-predicted VF metrics and willingness to decrease VF testing frequency. MAIN OUTCOMES AND MEASURES Mean counts of management recommendations and mean Likert scale scores were calculated to assess overall management trends and attitudes towards the CDS tool for each case. In addition, system usability scale scores were calculated. RESULTS The mean Likert scores for trust in and utility of the predicted VF metric and clinician willingness to decrease VF testing frequency were 3.27, 3.42, and 2.64, respectively (1=strongly disagree, 5=strongly agree). When stratified by glaucoma severity, all mean Likert scores decreased as severity increased. The system usability scale score across all responders was 66.1±16.0 (43rd percentile). CONCLUSIONS A CDS tool can be designed to present AI model outputs in a useful, trustworthy manner that clinicians are generally willing to integrate into their clinical decision-making. Future work is needed to understand how to best develop explainable and trustworthy CDS tools integrating AI before clinical deployment.
Collapse
Affiliation(s)
- Jimmy S Chen
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute
- UCSD Health Department of Biomedical Informatics, University of California San Diego, La Jolla, CA
| | - Sally L Baxter
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute
- UCSD Health Department of Biomedical Informatics, University of California San Diego, La Jolla, CA
| | | | - Alexander Lieu
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute
| | - Andrew S Camp
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute
| | - Jiun L Do
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute
| | - Derek S Welsbie
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute
| | - Sasan Moghimi
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute
| | - Mark Christopher
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute
| | - Robert N Weinreb
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute
| | - Linda M Zangwill
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute
| |
Collapse
|
21
|
Abstract
IMPORTANCE Peer relationships may motivate physicians to aspire to high professional standards but have not been a major focus of quality improvement efforts. OBJECTIVE To determine whether peer relationships between primary care physicians (PCPs) and specialists formed during training motivate improved specialist care for patients. DESIGN, SETTING, AND PARTICIPANTS In this quasi-experimental study, difference-in-differences analysis was used to estimate differences in experiences with specialist care reported by patients of the same PCP for specialists who did vs did not co-train with the PCP, controlling for any differences in patient ratings of the same specialists in the absence of co-training ties. Specialist visits resulting from PCP referrals from 2016 to 2019 in a large health system were analyzed, including a subset of undirected referrals in which PCPs did not specify a specialist. Data were collected from January 2016 to December 2019 and analyzed from March 2020 to October 2022. EXPOSURE The exposure was PCP-specialist overlap in training (medical school or postgraduate medical) at the same institution for at least 1 year (co-training). MAIN OUTCOMES AND MEASURES Composite patient experience rating of specialist care constructed from Press Ganey's Medical Practice Survey. RESULTS Of 9920 specialist visits for 8655 patients (62.9% female; mean age, 57.4 years) with 502 specialists in 13 specialties, 3.1% (306) involved PCP-specialist dyads with a co-training tie. Co-training ties between PCPs and specialists were associated with a 9.0 percentage point higher adjusted composite patient rating of specialist care (95% CI, 5.6-12.4 percentage points; P < .001), analogous to improvement from the median to the 91st percentile of specialist performance. This association was stronger for PCP-specialist dyads with full temporal overlap in training (same class or cohort) and consistently strong for 9 of 10 patient experience items, including clarity of communication and engagement in shared decision-making. In secondary analyses of objective markers of altered specialist practice in an expanded sample of visits not limited by the availability of patient experience data, co-training was associated with changes in medication prescribing, suggesting behavioral changes beyond interpersonal communication. Patient characteristics varied minimally by co-training status of PCP-specialist dyads. Results were similar in analyses restricted to undirected referrals (in which PCPs did not specify a specialist). Concordance between PCPs and specialists in physician age, sex, medical school graduation year, and training institution (without requiring temporal overlap) was not associated with better care experiences. CONCLUSIONS AND RELEVANCE In this quasi-experimental study, PCP-specialist co-training elicited changes in specialist care that substantially improved patient experiences, suggesting potential gains from strategies encouraging the formation of stronger physician-peer relationships.
Collapse
Affiliation(s)
- Maximilian J Pany
- Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts.,Harvard Business School, Boston, Massachusetts
| | - J Michael McWilliams
- Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts.,Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts.,Associate Editor, JAMA Internal Medicine
| |
Collapse
|
22
|
Erras A, Shahrvini B, Weinreb RN, Baxter SL. Review of glaucoma medication adherence monitoring in the digital health era. Br J Ophthalmol 2023; 107:153-159. [PMID: 33858837 PMCID: PMC8517037 DOI: 10.1136/bjophthalmol-2020-317918] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Revised: 02/19/2021] [Accepted: 04/05/2021] [Indexed: 02/03/2023]
Abstract
Current glaucoma treatments aim to lower intraocular pressure, often with topical ocular hypotensive medications. Unfortunately, the effectiveness of these medications depends on sustained patient adherence to regimens which may involve instilling multiple medications several times daily. Patient adherence to glaucoma medications is often low. Recent innovations in digital sensor technologies have been leveraged to confirm eyedrop medication usage in real-time and relay this information back to providers. Some sensors have also been designed to deliver medication reminders and notifications as well as assist with correct eyedrop administration technique. Here, we review recent innovations targeted at improving glaucoma medication adherence and discuss their limitations.
Collapse
Affiliation(s)
- Alaa Erras
- Hamilton Glaucoma Center, Shiley Eye Institute and Viterbi Family Department of Ophthalmology, University of California San Diego, La Jolla, California, USA
| | - Bita Shahrvini
- Hamilton Glaucoma Center, Shiley Eye Institute and Viterbi Family Department of Ophthalmology, University of California San Diego, La Jolla, California, USA
| | - Robert N Weinreb
- Hamilton Glaucoma Center, Shiley Eye Institute and Viterbi Family Department of Ophthalmology, University of California San Diego, La Jolla, California, USA
| | - Sally L Baxter
- Hamilton Glaucoma Center, Shiley Eye Institute and Viterbi Family Department of Ophthalmology, University of California San Diego, La Jolla, California, USA .,Health Department of Biomedical Informatics, University of California San Diego, La Jolla, California, USA
| |
Collapse
|
23
|
Simon AR, Ahmed KL, Limon DL, Duhon GF, Marzano G, Goin-Kochel RP. Utilization of a Best Practice Alert (BPA) at Point-of-Care for Recruitment into a US-Based Autism Research Study. J Autism Dev Disord 2023; 53:359-369. [PMID: 35089434 PMCID: PMC9329488 DOI: 10.1007/s10803-022-05444-y] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/10/2022] [Indexed: 02/04/2023]
Abstract
Provider referral is one of the most influential factors in research recruitment. To ease referral burden on providers, we adapted the Best Practice Alert (BPA) in the EPIC Electronic Health Record and assessed its utility in recruiting pediatric patients with autism spectrum disorder for the national SPARK study. During a year-long surveillance, 1203 (64.0%) patients were Interested in SPARK and 223 enrolled. Another 754 participants not recruited via the BPA also enrolled; 35.5% of these participants completed their participation compared to 58.3% of BPA-referred participants. Results suggest that (a) a BPA can successfully engage providers in the study-referral process and (b) families who learn about research through their providers may be more engaged and effectively retained.
Collapse
Affiliation(s)
- Andrea R Simon
- Department of Pediatrics, Baylor College of Medicine, Houston, TX, USA
- Autism Center, Texas Children's Hospital, Houston, TX, USA
- Department of Health Care Administration, Trinity University, San Antonio, TX, USA
| | - Kelli L Ahmed
- Autism Center, Texas Children's Hospital, Houston, TX, USA
- Department of Molecular and Human Genetics, Baylor College of Medicine, Houston, TX, USA
| | - Danica L Limon
- Autism Center, Texas Children's Hospital, Houston, TX, USA
- Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
- Department of Clinical Psychology, Brigham Young University, Provo, UT, USA
| | - Gabrielle F Duhon
- Department of Pediatrics, Baylor College of Medicine, Houston, TX, USA
- Autism Center, Texas Children's Hospital, Houston, TX, USA
- Department of Health Services Research, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Gabriela Marzano
- Department of Pediatrics, Baylor College of Medicine, Houston, TX, USA
- Autism Center, Texas Children's Hospital, Houston, TX, USA
| | - Robin P Goin-Kochel
- Department of Pediatrics, Baylor College of Medicine, Houston, TX, USA.
- Autism Center, Texas Children's Hospital, Houston, TX, USA.
- Meyer Center for Developmental Pediatrics and Autism, 8080 N. Stadium Drive, Suite 100, Houston, TX, 77054, USA.
| |
Collapse
|
24
|
Lyons PG, Chen V, Sekhar TC, McEvoy CA, Kollef MH, Govindan R, Westervelt P, Vranas KC, Maddox TM, Geng EH, Payne PRO, Politi MC. Clinician Perspectives on Barriers and Enablers to Implementing an Inpatient Oncology Early Warning System: A Mixed-Methods Study. JCO Clin Cancer Inform 2023; 7:e2200104. [PMID: 36706345 DOI: 10.1200/cci.22.00104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/28/2023] Open
Abstract
PURPOSE To elicit end-user and stakeholder perceptions regarding design and implementation of an inpatient clinical deterioration early warning system (EWS) for oncology patients to better fit routine clinical practices and enhance clinical impact. METHODS In an explanatory-sequential mixed-methods study, we evaluated a stakeholder-informed oncology early warning system (OncEWS) using surveys and semistructured interviews. Stakeholders were physicians, advanced practice providers (APPs), and nurses. For qualitative data, we used grounded theory and thematic content analysis via the constant comparative method to identify determinants of OncEWS implementation. RESULTS Survey respondents generally agreed that an oncology-focused EWS could add value beyond clinical judgment, with nurses endorsing this notion significantly more strongly than other clinicians (nurse: median 5 on a 6-point scale [6 = strongly agree], interquartile range 4-5; doctors/advanced practice providers: 4 [4-5]; P = .005). However, some respondents would not trust an EWS to identify risk accurately (n = 36 [42%] somewhat or very concerned), while others were concerned that institutional culture would not embrace such an EWS (n = 17 [28%]).Interviews highlighted important aspects of the EWS and the local context that might facilitate implementation, including (1) a model tailored to the subtleties of oncology patients, (2) transparent model information, and (3) nursing-centric workflows. Interviewees raised the importance of sepsis as a common and high-risk deterioration syndrome. CONCLUSION Stakeholders prioritized maximizing the degree to which the OncEWS is understandable, informative, actionable, and workflow-complementary, and perceived these factors to be key for translation into clinical benefit.
Collapse
Affiliation(s)
- Patrick G Lyons
- Division of Pulmonary and Critical Care Medicine, Department of Medicine, Washington University School of Medicine, St Louis, MO.,Healthcare Innovation Lab, BJC HealthCare, St Louis, MO.,Siteman Cancer Center, St Louis, MO
| | - Vanessa Chen
- Division of Pulmonary and Critical Care Medicine, Department of Medicine, Washington University School of Medicine, St Louis, MO
| | - Tejas C Sekhar
- Division of Pulmonary and Critical Care Medicine, Department of Medicine, Washington University School of Medicine, St Louis, MO
| | - Colleen A McEvoy
- Division of Pulmonary and Critical Care Medicine, Department of Medicine, Washington University School of Medicine, St Louis, MO
| | - Marin H Kollef
- Division of Pulmonary and Critical Care Medicine, Department of Medicine, Washington University School of Medicine, St Louis, MO
| | - Ramaswamy Govindan
- Siteman Cancer Center, St Louis, MO.,Division of Hematology and Oncology, Department of Medicine, Washington University School of Medicine, St Louis, MO
| | - Peter Westervelt
- Siteman Cancer Center, St Louis, MO.,Division of Hematology and Oncology, Department of Medicine, Washington University School of Medicine, St Louis, MO
| | - Kelly C Vranas
- Division of Pulmonary and Critical Care Medicine, Oregon Health and Science University, Portland, OR.,Center to Improve Veteran Involvement in Care, VA Portland Health Care System, Portland, OR
| | - Thomas M Maddox
- Healthcare Innovation Lab, BJC HealthCare, St Louis, MO.,Division of Cardiology, Department of Medicine, Washington University School of Medicine, St Louis, MO
| | - Elvin H Geng
- Division of Infectious Diseases, Department of Medicine, Washington University School of Medicine, St Louis, MO.,Center for Dissemination and Implementation in the Institute for Public Health, Washington University School of Medicine, St Louis, MO
| | - Philip R O Payne
- Institute for Informatics, Washington University School of Medicine, St Louis, MO
| | - Mary C Politi
- Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, St Louis, MO.,Center for Collaborative Care Decisions, Department of Surgery, Washington University School of Medicine, St Louis, MO
| |
Collapse
|
25
|
Millar MM, Taft T, Weir CR. Clinical trial recruitment in primary care: exploratory factor analysis of a questionnaire to measure barriers and facilitators to primary care providers' involvement. BMC PRIMARY CARE 2022; 23:311. [PMID: 36463123 PMCID: PMC9719201 DOI: 10.1186/s12875-022-01898-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/25/2022] [Accepted: 11/03/2022] [Indexed: 12/05/2022]
Abstract
BACKGROUND Recruitment of sufficient participants for clinical trials remains challenging. Primary care is an important avenue for patient recruitment but is underutilized. We developed and pilot tested a questionnaire to measure relevant barriers and facilitators to primary care providers' involvement in recruiting patients for clinical trials. METHODS Prior research informed the development of the questionnaire. The initial instrument was revised using feedback obtained from cognitive interviews. We invited all primary care providers practicing within the University of Utah Health system to complete the revised questionnaire. We used a mixed-mode design to collect paper responses via in-person recruitment and email contacts to collect responses online. Descriptive statistics, exploratory factor analysis, Cronbach's alpha, and multivariable regression analyses were conducted. RESULTS Sixty-seven primary care providers participated in the survey. Exploratory factor analysis suggested retaining five factors, representing the importance of clinical trial recruitment in providers' professional identity, clinic-level interventions to facilitate referral, patient-related barriers, concerns about patient health management, and knowledge gaps. The five factors exhibited good or high internal consistency reliability. Professional identity and clinic-level intervention factors were significant predictors of providers' intention to participate in clinical trial recruitment activities. CONCLUSIONS Results of this exploratory analysis provide preliminary evidence of the internal structure, internal consistency reliability, and predictive validity of the questionnaire to measure factors relevant to primary care providers' involvement in clinical trial recruitment.
Collapse
Affiliation(s)
- Morgan M Millar
- Department of Internal Medicine, University of Utah, 295 Chipeta Way, Salt Lake City, UT, USA.
| | - Teresa Taft
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA
| | - Charlene R Weir
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA
| |
Collapse
|
26
|
Hatherley J, Sparrow R. Diachronic and synchronic variation in the performance of adaptive machine learning systems: the ethical challenges. J Am Med Inform Assoc 2022; 30:361-366. [PMID: 36377970 PMCID: PMC9846684 DOI: 10.1093/jamia/ocac218] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2022] [Revised: 09/19/2022] [Accepted: 11/01/2022] [Indexed: 11/16/2022] Open
Abstract
OBJECTIVES Machine learning (ML) has the potential to facilitate "continual learning" in medicine, in which an ML system continues to evolve in response to exposure to new data over time, even after being deployed in a clinical setting. In this article, we provide a tutorial on the range of ethical issues raised by the use of such "adaptive" ML systems in medicine that have, thus far, been neglected in the literature. TARGET AUDIENCE The target audiences for this tutorial are the developers of ML AI systems, healthcare regulators, the broader medical informatics community, and practicing clinicians. SCOPE Discussions of adaptive ML systems to date have overlooked the distinction between 2 sorts of variance that such systems may exhibit-diachronic evolution (change over time) and synchronic variation (difference between cotemporaneous instantiations of the algorithm at different sites)-and underestimated the significance of the latter. We highlight the challenges that diachronic evolution and synchronic variation present for the quality of patient care, informed consent, and equity, and discuss the complex ethical trade-offs involved in the design of such systems.
Collapse
Affiliation(s)
- Joshua Hatherley
- Corresponding Author: Joshua Hatherley, MBioethics, Philosophy Department, School of Philosophical, Historical and International Studies, Monash University, Level 6, 20 Chancellor's Walk (Menzies Building), Wellington Road, Clayton, VIC 3800, Australia;
| | - Robert Sparrow
- Philosophy Department, School of Philosophical, Historical and International Studies, Monash University, Clayton, Victoria 3800, Australia
| |
Collapse
|
27
|
Apathy NC, Sanner L, Adams MCB, Mamlin BW, Grout RW, Fortin S, Hillstrom J, Saha A, Teal E, Vest JR, Menachemi N, Hurley RW, Harle CA, Mazurenko O. Assessing the use of a clinical decision support tool for pain management in primary care. JAMIA Open 2022; 5:ooac074. [PMID: 36128342 PMCID: PMC9476612 DOI: 10.1093/jamiaopen/ooac074] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2022] [Revised: 07/11/2022] [Accepted: 08/18/2022] [Indexed: 01/23/2023] Open
Abstract
Objective Given time constraints, poorly organized information, and complex patients, primary care providers (PCPs) can benefit from clinical decision support (CDS) tools that aggregate and synthesize problem-specific patient information. First, this article describes the design and functionality of a CDS tool for chronic noncancer pain in primary care. Second, we report on the retrospective analysis of real-world usage of the tool in the context of a pragmatic trial. Materials and methods The tool known as OneSheet was developed using user-centered principles and built in the Epic electronic health record (EHR) of 2 health systems. For each relevant patient, OneSheet presents pertinent information in a single EHR view to assist PCPs in completing guideline-recommended opioid risk mitigation tasks, review previous and current patient treatments, view patient-reported pain, physical function, and pain-related goals. Results Overall, 69 PCPs accessed OneSheet 2411 times (since November 2020). PCP use of OneSheet varied significantly by provider and was highly skewed (site 1: median accesses per provider: 17 [interquartile range (IQR) 9-32]; site 2: median: 8 [IQR 5-16]). Seven "power users" accounted for 70% of the overall access instances across both sites. OneSheet has been accessed an average of 20 times weekly between the 2 sites. Discussion Modest OneSheet use was observed relative to the number of eligible patients seen with chronic pain. Conclusions Organizations implementing CDS tools are likely to see considerable provider-level variation in usage, suggesting that CDS tools may vary in their utility across PCPs, even for the same condition, because of differences in provider and care team workflows.
Collapse
Affiliation(s)
- Nate C Apathy
- Leonard Davis Institute of Health Economics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
- Department of Health Policy and Management, Indiana University Richard M. Fairbanks School of Public Health, Indianapolis, Indiana, USA
- Clem McDonald Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, Indiana, USA
| | - Lindsey Sanner
- Department of Health Policy and Management, Indiana University Richard M. Fairbanks School of Public Health, Indianapolis, Indiana, USA
| | - Meredith C B Adams
- Anesthesiology, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA
| | - Burke W Mamlin
- Clem McDonald Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, Indiana, USA
- Internal Medicine, Eskenazi Health, Indianapolis, Indiana, USA
- Department of Clinical Medicine, Indiana University School of Medicine, Indianapolis, Indiana, USA
| | - Randall W Grout
- Clem McDonald Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, Indiana, USA
- Department of Pediatrics, Indiana University School of Medicine, Indianapolis, Indiana, USA
- Informatics, Eskenazi Health, Indianapolis, Indiana, USA
| | - Saura Fortin
- Primary Care, Eskenazi Health, Indianapolis, Indiana, USA
| | - Jennifer Hillstrom
- IS Ambulatory & Research Solutions, Eskenazi Health, Indianapolis, Indiana, USA
| | - Amit Saha
- Anesthesiology, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA
| | - Evgenia Teal
- Data Core, Regenstrief Institute, Indianapolis, Indiana, USA
| | - Joshua R Vest
- Department of Health Policy and Management, Indiana University Richard M. Fairbanks School of Public Health, Indianapolis, Indiana, USA
- Clem McDonald Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, Indiana, USA
| | - Nir Menachemi
- Department of Health Policy and Management, Indiana University Richard M. Fairbanks School of Public Health, Indianapolis, Indiana, USA
- Clem McDonald Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, Indiana, USA
| | - Robert W Hurley
- Anesthesiology, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA
| | - Christopher A Harle
- Clem McDonald Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, Indiana, USA
- Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, Florida, USA
| | - Olena Mazurenko
- Department of Health Policy and Management, Indiana University Richard M. Fairbanks School of Public Health, Indianapolis, Indiana, USA
| |
Collapse
|
28
|
Salmasian H, Rubins D, Bates DW. Using the Electronic Health Record User Context in Clinical Decision Support Criteria. Appl Clin Inform 2022; 13:910-915. [PMID: 36170882 PMCID: PMC9519266 DOI: 10.1055/s-0042-1756426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2021] [Accepted: 07/28/2022] [Indexed: 11/02/2022] Open
Abstract
BACKGROUND Computerized clinical decision support (CDS) used in electronic health record systems (EHRs) has led to positive outcomes as well as unintended consequences, such as alert fatigue. Characteristics of the EHR session can be used to restrict CDS tools and increase their relevance, but implications of this approach are not rigorously studied. OBJECTIVES To assess the utility of using "login location" of EHR users-that is, the location they chose on the login screen-as a variable in the CDS logic. METHODS We measured concordance between user's login location and the location of the patients they placed orders for and conducted stratified analyses by user groups. We also estimated how often login location data may be stale or inaccurate. RESULTS One in five CDS alerts incorporated the EHR users' login location into their logic. Analysis of nearly 2 million orders placed by nearly 8,000 users showed that concordance between login location and patient location was high for nurses, nurse practitioners, and physician assistance (all >95%), but lower for fellows (77%) and residents (55%). When providers switched between patients in the EHR, they usually did not update their login location accordingly. CONCLUSION CDS alerts commonly incorporate user's login location into their logic. User's login location is often the same as the location of the patient the user is providing care for, but substantial discordance can be observed for certain user groups. While this may provide additional information that could be useful to the CDS logic, a substantial amount of discordance happened in specific user groups or when users appeared not to change their login location across different sessions. Those who design CDS alerts should consider a data-driven approach to evaluate the appropriateness of login location for each use case.
Collapse
Affiliation(s)
- Hojjat Salmasian
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Mass General Brigham, Somerville, Massachusetts, United States
| | - David Rubins
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Mass General Brigham, Somerville, Massachusetts, United States
| | - David W. Bates
- Division of General Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Mass General Brigham, Somerville, Massachusetts, United States
| |
Collapse
|
29
|
Aiding the prescriber: developing a machine learning approach to personalized risk modeling for chronic opioid therapy amongst US Army soldiers. Health Care Manag Sci 2022; 25:649-665. [PMID: 35895214 DOI: 10.1007/s10729-022-09605-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2021] [Accepted: 06/13/2022] [Indexed: 11/04/2022]
Abstract
The opioid epidemic is a major policy concern. The widespread availability of opioids, which is fueled by physician prescribing patterns, medication diversion, and the interaction with potential illicit opioid use, has been implicated as proximal cause for subsequent opioid dependence and mortality. Risk indicators related to chronic opioid therapy (COT) at the point of care may influence physicians' prescribing decisions, potentially reducing rates of dependency and abuse. In this paper, we investigate the performance of machine learning algorithms for predicting the risk of COT. Using data on over 12 million observations of active duty US Army soldiers, we apply machine learning models to predict the risk of COT in the initial months of prescription. We use the area under the curve (AUC) as an overall measure of model performance, and we focus on the positive predictive value (PPV), which reflects the models' ability to accurately target military members for intervention. Of the many models tested, AUC ranges between 0.83 and 0.87. When we focus on the top 1% of members at highest risk, we observe a PPV value of 8.4% and 20.3% for months 1 and 3, respectively. We further investigate the performance of sparse models that can be implemented in sparse data environments. We find that when the goal is to identify patients at the highest risk of chronic use, these sparse linear models achieve a performance similar to models trained on hundreds of variables. Our predictive models exhibit high accuracy and can alert prescribers to the risk of COT for the highest risk patients. Optimized sparse models identify a parsimonious set of factors to predict COT: initial supply of opioids, the supply of opioids in the month being studied, and the number of prescriptions for psychotropic medications. Future research should investigate the possible effects of these tools on prescriber behavior (e.g., the benefit of clinician nudging at the point of care in outpatient settings).
Collapse
|
30
|
Verdonk F, Feyaerts D, Badenes R, Bastarache JA, Bouglé A, Ely W, Gaudilliere B, Howard C, Kotfis K, Lautrette A, Le Dorze M, Mankidy BJ, Matthay MA, Morgan CK, Mazeraud A, Patel BV, Pattnaik R, Reuter J, Schultz MJ, Sharshar T, Shrestha GS, Verdonk C, Ware LB, Pirracchio R, Jabaudon M. Upcoming and urgent challenges in critical care research based on COVID-19 pandemic experience. Anaesth Crit Care Pain Med 2022; 41:101121. [PMID: 35781076 PMCID: PMC9245393 DOI: 10.1016/j.accpm.2022.101121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2022] [Revised: 06/03/2022] [Accepted: 06/03/2022] [Indexed: 11/01/2022]
Abstract
While the coronavirus disease 2019 (COVID-19) pandemic placed a heavy burden on healthcare systems worldwide, it also induced urgent mobilisation of research teams to develop treatments preventing or curing the disease and its consequences. It has, therefore, challenged critical care research to rapidly focus on specific fields while forcing critical care physicians to make difficult ethical decisions. This narrative review aims to summarise critical care research -from organisation to research fields- in this pandemic setting and to highlight opportunities to improve research efficiency in the future, based on what is learned from COVID-19. This pressure on research revealed, i.e., i/ the need to harmonise regulatory processes between countries, allowing simplified organisation of international research networks to improve their efficiency in answering large-scale questions; ii/ the importance of developing translational research from which therapeutic innovations can emerge; iii/ the need for improved triage and predictive scores to rationalise admission to the intensive care unit. In this context, key areas for future critical care research and better pandemic preparedness are artificial intelligence applied to healthcare, characterisation of long-term symptoms, and ethical considerations. Such collaborative research efforts should involve groups from both high and low-to-middle income countries to propose worldwide solutions. As a conclusion, stress tests on healthcare organisations should be viewed as opportunities to design new research frameworks and strategies. Worldwide availability of research networks ready to operate is essential to be prepared for next pandemics. Importantly, researchers and physicians should prioritise realistic and ethical goals for both clinical care and research.
Collapse
Affiliation(s)
- Franck Verdonk
- Department of Anaesthesiology and Intensive Care, Hôpital Saint-Antoine Paris, Assistance Publique-Hôpitaux de Paris, France and GRC 29, DMU DREAM, Sorbonne University, Paris, France; Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford University, California, United States of America
| | - Dorien Feyaerts
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford University, California, United States of America
| | - Rafael Badenes
- Department of Anaesthesiology and Intensive Care, Hospital Clìnico Universitario de Valencia, University of Valencia, Valencia, Spain
| | - Julie A Bastarache
- Division of Allergy, Pulmonary and Critical Care Medicine, Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, United States of America
| | - Adrien Bouglé
- Sorbonne Université, GRC 29, AP-HP, DMU DREAM, Department of Anaesthesiology and Critical Care Medicine, Institute of Cardiology, Pitié-Salpêtrière Hospital, Paris, France
| | - Wesley Ely
- Critical Illness, Brain Dysfunction, and Survivorship (CIBS) Center, at the TN Valley VA Geriatric Research Education Clinical Center (GRECC) and Vanderbilt University Medical Center, Nashville, TN, United States of America
| | - Brice Gaudilliere
- Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford University, California, United States of America
| | - Christopher Howard
- Department of Medicine, Division of Pulmonary, Critical Care and Sleep Medicine, Baylor College of Medicine, Houston, Texas, United States of America
| | - Katarzyna Kotfis
- Department Anaesthesiology, Intensive Therapy and Acute Intoxications, Pomeranian Medical University, Szczecin, Poland
| | - Alexandre Lautrette
- Medical Intensive Care Unit, Gabriel-Montpied University Hospital, Clermont-Ferrand, France
| | - Matthieu Le Dorze
- Department of Anaesthesiology and Critical Care Medicine, AP-HP, Lariboisière University Hospital, Paris, France
| | - Babith Joseph Mankidy
- Department of Medicine, Division of Pulmonary, Critical Care and Sleep Medicine, Baylor College of Medicine, Houston, Texas, United States of America
| | - Michael A Matthay
- Departments of Medicine and Anaesthesia, University of California, and Cardiovascular Research Institute, San Francisco, California, United States of America
| | - Christopher K Morgan
- Department of Medicine, Division of Pulmonary, Critical Care and Sleep Medicine, Baylor College of Medicine, Houston, Texas, United States of America
| | - Aurélien Mazeraud
- Service d'Anesthésie-Réanimation, Groupe Hospitalier Université Paris Psychiatrie et Neurosciences, Pôle Neuro, Paris, France
| | - Brijesh V Patel
- Division of Anaesthetics, Pain Medicine, and Intensive Care, Department of Surgery and Cancer, Faculty of Medicine, Imperial College, and Department of Adult Intensive Care, Royal Brompton & Harefield Hospitals, Guys & St Thomas' NHS Foundation trust, London, UK
| | - Rajyabardhan Pattnaik
- Department of Intensive Care Medicine, Ispat General Hospital, Rourkela, Sundargarh, Odisha, India
| | - Jean Reuter
- Department of Intensive Care Medicine, Centre Hospitalier de Luxembourg, Luxembourg
| | - Marcus J Schultz
- Department of Intensive Care, Amsterdam UMC, location AMC, Amsterdam, The Netherlands
| | - Tarek Sharshar
- Service d'Anesthésie-Réanimation, Groupe Hospitalier Université Paris Psychiatrie et Neurosciences, Pôle Neuro, Paris, France
| | - Gentle S Shrestha
- Department of Anaesthesiology, Tribhuvan University Teaching Hospital, Maharajgunj, Kathmandu, Nepal
| | - Charles Verdonk
- Unit of Neurophysiology of Stress, Department of Neurosciences and Cognitive Sciences, French Armed Forces Biomedical Research Institute, Brétigny-sur-Orge, France
| | - Lorraine B Ware
- Division of Allergy, Pulmonary and Critical Care Medicine, Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, United States of America
| | - Romain Pirracchio
- Department of Anesthesia and Perioperative Medicine, Zuckerberg San Francisco General Hospital and Trauma Center, University of California San Francisco, California, United States of America
| | - Matthieu Jabaudon
- Department of Perioperative Medicine, CHU Clermont-Ferrand, Clermont-Ferrand, France; iGReD, Université Clermont Auvergne, CNRS, INSERM, Clermont-Ferrand, France.
| |
Collapse
|
31
|
Lauffenburger JC, DiFrancesco MF, Barlev RA, Robertson T, Kim E, Coll MD, Haff N, Fontanet CP, Hanken K, Oran R, Avorn J, Choudhry NK. Overcoming Decisional Gaps in High-Risk Prescribing by Junior Physicians Using Simulation-Based Training: Protocol for a Randomized Controlled Trial. JMIR Res Protoc 2022; 11:e31464. [PMID: 35475982 PMCID: PMC9096643 DOI: 10.2196/31464] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2021] [Revised: 02/03/2022] [Accepted: 02/05/2022] [Indexed: 11/16/2022] Open
Abstract
BACKGROUND Gaps between rational thought and actual decisions are increasingly recognized as a reason why people make suboptimal choices in states of heightened emotion, such as stress. These observations may help explain why high-risk medications continue to be prescribed to acutely ill hospitalized older adults despite widely accepted recommendations against these practices. Role playing and other efforts, such as simulation training, have demonstrated benefits to help people avoid decisional gaps but have not been tested to reduce overprescribing of high-risk medications. OBJECTIVE This study aims to evaluate the impact of a simulation-based training program designed to address decisional gaps on prescribing of high-risk medications compared with control. METHODS In this 2-arm pragmatic trial, we are randomizing at least 36 first-year medical resident physicians (ie, interns) who provide care on inpatient general medicine services at a large academic medical center to either intervention (simulation-based training) or control (online educational training). The intervention comprises a 40-minute immersive individual simulation training consisting of a reality-based patient care scenario in a simulated environment at the beginning of their inpatient service rotation. The simulation focuses on 3 types of high-risk medications, including benzodiazepines, antipsychotics, and sedative hypnotics (Z-drugs), in older adults, and is specifically designed to help the physicians identify their reactions and prescribing decisions in stressful situations that are common in the inpatient setting. The simulation scenario is followed by a semistructured debriefing with an expert facilitator. The trial's primary outcome is the number of medication doses for any of the high-risk medications prescribed by the interns to patients aged 65 years or older who were not taking one of the medications upon admission. Secondary outcomes include prescribing by all providers on the care team, being discharged on 1 of the medications, and prescribing of related medications (eg, melatonin, trazodone), or the medications of interest for the control intervention. These outcomes will be measured using electronic health record data. RESULTS Recruitment of interns began on March 29, 2021. Recruitment for the trial ended in Q42021, with follow-up completed by Q12022. CONCLUSIONS This trial will evaluate the impact of a simulation-based training program designed using behavioral science principles on prescribing of high-risk medications by junior physicians. If the intervention is shown to be effective, this approach could potentially be reproducible by others and for a broader set of behaviors. TRIAL REGISTRATION ClinicalTrials.gov NCT04668248; https://clinicaltrials.gov/ct2/show/NCT04668248. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) PRR1-10.2196/31464.
Collapse
Affiliation(s)
| | | | - Renee A Barlev
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States
| | | | - Erin Kim
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States
| | - Maxwell D Coll
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States
| | - Nancy Haff
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States
| | | | - Kaitlin Hanken
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States
| | | | - Jerry Avorn
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States
| | - Niteesh K Choudhry
- Brigham and Women's Hospital, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
32
|
Braun EJ, Singh S, Penlesky AC, Strong EA, Holt JM, Fletcher KE, Stadler ME, Nattinger AB, Crotty BH. Nursing implications of an early warning system implemented to reduce adverse events: a qualitative study. BMJ Qual Saf 2022; 31:716-724. [PMID: 35428684 DOI: 10.1136/bmjqs-2021-014498] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2021] [Accepted: 03/23/2022] [Indexed: 11/03/2022]
Abstract
BACKGROUND Unrecognised changes in a hospitalised patient's clinical course may lead to a preventable adverse event. Early warning systems (EWS) use patient data, such as vital signs, nursing assessments and laboratory values, to aid in the detection of early clinical deterioration. In 2018, an EWS programme was deployed at an academic hospital that consisted of a commercially available EWS algorithm and a centralised virtual nurse team to monitor alerts. Our objective was to understand the nursing perspective on the use of an EWS programme with centralised monitoring. METHODS We conducted and audio-recorded semistructured focus groups during nurse staff meetings on six inpatient units, stratified by alert frequency (high: >100 alerts/month; medium: 50-100 alerts/month; low: <50 alerts/month). Discussion topics included EWS programme experiences, perception of EWS programme utility and EWS programme implementation. Investigators analysed the focus group transcripts using a grounded theory approach. RESULTS We conducted 28 focus groups with 227 bedside nurses across all shifts. We identified six principal themes: (1) Alert timeliness, nurses reported being aware of the patient's deterioration before the EWS alert, (2) Lack of accuracy, nurses perceived most alerts as false positives, (3) Workflow interruptions caused by EWS alerts, (4) Questions of actionability of alerts, nurses were often uncertain about next steps, (5) Concerns around an underappreciation of core nursing skills via reliance on the EWS programme and (6) The opportunity cost of deploying the EWS programme. CONCLUSION This qualitative study of nurses demonstrates the importance of earning user trust, ensuring timeliness and outlining actionable next steps when implementing an EWS. Careful attention to user workflow is required to maximise EWS impact on improving hospital quality and patient safety.
Collapse
Affiliation(s)
- Emilie J Braun
- Collaborative for Healthcare Delivery Science, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Siddhartha Singh
- Collaborative for Healthcare Delivery Science, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Annie C Penlesky
- Collaborative for Healthcare Delivery Science, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Erin A Strong
- Collaborative for Healthcare Delivery Science, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Jeana M Holt
- Collaborative for Healthcare Delivery Science, Medical College of Wisconsin, Milwaukee, Wisconsin, USA.,School of Nursing, University of Wisconsin-Milwaukee, Milwaukee, Wisconsin, USA
| | - Kathlyn E Fletcher
- Collaborative for Healthcare Delivery Science, Medical College of Wisconsin, Milwaukee, Wisconsin, USA.,Internal Medicine, Clement J. Zablocki VAMC, Milwaukee, Wisconsin, USA
| | - Michael E Stadler
- Collaborative for Healthcare Delivery Science, Medical College of Wisconsin, Milwaukee, Wisconsin, USA.,Otolaryngology, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Ann B Nattinger
- Collaborative for Healthcare Delivery Science, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| | - Bradley H Crotty
- Collaborative for Healthcare Delivery Science, Medical College of Wisconsin, Milwaukee, Wisconsin, USA
| |
Collapse
|
33
|
Ivica J, Sanmugalingham G, Selvaratnam R. Alerting to Acute Kidney Injury - Challenges, benefits, and strategies. Pract Lab Med 2022; 30:e00270. [PMID: 35465620 PMCID: PMC9020093 DOI: 10.1016/j.plabm.2022.e00270] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2021] [Revised: 03/12/2022] [Accepted: 03/30/2022] [Indexed: 11/28/2022] Open
Affiliation(s)
- Josko Ivica
- Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario, Canada
- Hamilton Regional Laboratory Medicine Program, Hamilton Health Sciences and St. Joseph's Healthcare Hamilton, Hamilton, Ontario, Canada
| | - Geetha Sanmugalingham
- Division of Nephrology, Department of Pediatrics, Hospital for Sick Children, Toronto, ON, Canada
| | - Rajeevan Selvaratnam
- University Health Network, Laboratory Medicine Program, Division of Clinical Biochemistry, Toronto, Ontario, Canada
- University of Toronto, Department of Laboratory Medicine and Pathobiology, Toronto, Ontario, Canada
- Corresponding author. University Health Network, Laboratory Medicine Program, Division of Clinical Biochemistry, Toronto, Ontario, Canada.
| |
Collapse
|
34
|
Felton WL, Kornstein SG, Gondwe T, Huynh C, Wallenborn JT, Henry J. Evaluation of an Electronic Health Record Alert to Improve Screening and Management of Cardiovascular Disease and Stroke Factors in a High-Risk Population. South Med J 2022; 115:232-237. [PMID: 35237844 DOI: 10.14423/smj.0000000000001355] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
OBJECTIVES Cardiovascular disease and stroke risk factor screening and management by primary care providers (PCPs) have a significant impact on their patients' health. The objective of this study was to investigate the effectiveness of an electronic health record (EHR) cardiovascular disease and stroke risk alert in improving the ability of PCPs to manage risk factors among women and men aged 45 years and older. METHODS PCPs at a tertiary care hospital were randomized. The intervention group received an EHR alert, which calculated the individual patient risk and provided an order set incorporating the American Heart Association and American Stroke Association guidelines, whereas the control group used the EHR in the usual manner. Multilevel analysis compared the rate of prescriptions between the intervention and control groups. RESULTS A total of 23 PCPs were randomized: 12 in the intervention group and 11 in the control group, attending to 7190 patients between September 2016 and January 2017. None of the providers in the intervention group used the programmed order set. Intervention group providers were significantly more likely to prescribe smoking cessation medication to women than to the control group (adjusted odds ratio 2.37, 95% confidence interval 1.23-4.57). There were no statistically significant differences between the intervention and control groups in the rate of other medication prescriptions. CONCLUSIONS As measured by prescriptions for medications, other than those for smoking cessation, the EHR alert was not shown to be successful in increasing the management of high-risk patients. Physicians receiving numerous messages in the EHR may experience alert desensitization.
Collapse
Affiliation(s)
- Warren L Felton
- From the Departments of Neurology, Psychiatry, Family Medicine and Population Health, and Internal Medicine, School of Medicine, and Institute for Women's Health, Virginia Commonwealth University, Richmond
| | - Susan G Kornstein
- From the Departments of Neurology, Psychiatry, Family Medicine and Population Health, and Internal Medicine, School of Medicine, and Institute for Women's Health, Virginia Commonwealth University, Richmond
| | - Tamala Gondwe
- From the Departments of Neurology, Psychiatry, Family Medicine and Population Health, and Internal Medicine, School of Medicine, and Institute for Women's Health, Virginia Commonwealth University, Richmond
| | - Christine Huynh
- From the Departments of Neurology, Psychiatry, Family Medicine and Population Health, and Internal Medicine, School of Medicine, and Institute for Women's Health, Virginia Commonwealth University, Richmond
| | - Jordyn T Wallenborn
- From the Departments of Neurology, Psychiatry, Family Medicine and Population Health, and Internal Medicine, School of Medicine, and Institute for Women's Health, Virginia Commonwealth University, Richmond
| | - Jeneane Henry
- From the Departments of Neurology, Psychiatry, Family Medicine and Population Health, and Internal Medicine, School of Medicine, and Institute for Women's Health, Virginia Commonwealth University, Richmond
| |
Collapse
|
35
|
Alshekhabobakr HM, AlSaqatri SO, Rizk NM. Laboratory Test Utilization Practices in Hamad Medical Corporation; Role of Laboratory Supervisors and Clinicians in Improper Test Utilization; a Descriptive Pilot Study. J Multidiscip Healthc 2022; 15:413-429. [PMID: 35264855 PMCID: PMC8901233 DOI: 10.2147/jmdh.s320545] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2021] [Accepted: 01/07/2022] [Indexed: 11/23/2022] Open
Affiliation(s)
| | | | - Nasser Moustafa Rizk
- Biomedical Sciences Department, College of Health Sciences, QU Health, Qatar University, Doha, Qatar
- Biomedical Research Center (BRC), Qatar University, Doha, Qatar
- Correspondence: Nasser Moustafa Rizk, Biomedical Sciences Department, College of Health Sciences, QU Health, Qatar University, Doha, Qatar, Email
| |
Collapse
|
36
|
Wang L, Goh KH, Yeow A, Poh H, Li K, Yeow JJL, Tan G, Soh C. Habit and Automaticity in Medical Alert Override: Cohort Study. J Med Internet Res 2022; 24:e23355. [PMID: 35171102 PMCID: PMC8892274 DOI: 10.2196/23355] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2020] [Revised: 04/10/2021] [Accepted: 12/10/2021] [Indexed: 11/13/2022] Open
Abstract
Background Prior literature suggests that alert dismissal could be linked to physicians’ habits and automaticity. The evidence for this perspective has been mainly observational data. This study uses log data from an electronic medical records system to empirically validate this perspective. Objective We seek to quantify the association between habit and alert dismissal in physicians. Methods We conducted a retrospective analysis using the log data comprising 66,049 alerts generated from hospitalized patients in a hospital from March 2017 to December 2018. We analyzed 1152 physicians exposed to a specific clinical support alert triggered in a hospital’s electronic medical record system to estimate the extent to which the physicians’ habit strength, which had been developed from habitual learning, impacted their propensity toward alert dismissal. We further examined the association between a physician’s habit strength and their subsequent incidences of alert dismissal. Additionally, we recorded the time taken by the physician to respond to the alert and collected data on other clinical and environmental factors related to the alerts as covariates for the analysis. Results We found that a physician’s prior dismissal of alerts leads to their increased habit strength to dismiss alerts. Furthermore, a physician’s habit strength to dismiss alerts was found to be positively associated with incidences of subsequent alert dismissals after their initial alert dismissal. Alert dismissal due to habitual learning was also found to be pervasive across all physician ranks, from junior interns to senior attending specialists. Further, the dismissal of alerts had been observed to typically occur after a very short processing time. Our study found that 72.5% of alerts were dismissed in under 3 seconds after the alert appeared, and 13.2% of all alerts were dismissed in under 1 second after the alert appeared. We found empirical support that habitual dismissal is one of the key factors associated with alert dismissal. We also found that habitual dismissal of alerts is self-reinforcing, which suggests significant challenges in disrupting or changing alert dismissal habits once they are formed. Conclusions Habitual tendencies are associated with the dismissal of alerts. This relationship is pervasive across all levels of physician rank and experience, and the effect is self-reinforcing.
Collapse
Affiliation(s)
- Le Wang
- City University of Hong Kong, Hong Kong, China (Hong Kong)
| | - Kim Huat Goh
- Nanyang Technological University, Singapore, Singapore
| | - Adrian Yeow
- Singapore University of Social Sciences, Singapore, Singapore
| | - Hermione Poh
- Medical Informatics, National University Health System, Singapore, Singapore
| | - Ke Li
- Medical Informatics, National University Health System, Singapore, Singapore
| | | | - Gamaliel Tan
- Medical Informatics, National University Health System, Singapore, Singapore.,Ng Teng Fong General Hospital, Singapore, Singapore
| | - Christina Soh
- Nanyang Technological University, Singapore, Singapore
| |
Collapse
|
37
|
Rock C, Abosi O, Bleasdale S, Colligan E, Diekema DJ, Dullabh P, Gurses AP, Heaney-Huls K, Jacob JT, Kandiah S, Lama S, Leekha S, Mayer J, Mena Lora AJ, Morgan DJ, Osei P, Pau S, Salinas JL, Spivak E, Wenzler E, Cosgrove SE. Clinical Decision Support Systems to Reduce Unnecessary Clostridoides difficile Testing Across Multiple Hospitals. Clin Infect Dis 2022; 75:1187-1193. [PMID: 35100620 DOI: 10.1093/cid/ciac074] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2021] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Inappropriate C. difficile testing has adverse consequences for the patient, hospital, and public health. Computerized Clinical Decision Supports (CCDS) in the Electronic Health Record (EHR) may reduce C. difficile test ordering; however, effectiveness of different approaches, ease of use, and best fit into the healthcare providers' (HCP) workflow, are not well understood. METHODS Nine academic and 6 community U.S. hospitals participated in this 2-year cohort study. CCDS (hard- or soft-stop) triggered when duplicate C. difficile test order attempted, or if laxatives were recently received. The primary outcome was the difference in testing rates pre- and post-CCDS interventions, using incident rate ratios (IRR) and mixed effect Poisson regression models. We performed qualitative evaluation (contextual inquiry, interviews, focus groups) based on a human factors model. We identified themes using a codebook with primary- and sub-nodes. RESULTS In 9 hospitals implementing hard-stop CCDS and 4 hospitals implementing soft-stop CCDS, C. difficile testing IRR reduction was 33% (95% CI, 30-36%), and 23% (95% CI 21-25%), respectively. Two hospitals implemented a non-EMR based human intervention with IRR reduction of 21% (95% CI 15-28%). HCPs reported generally favorable experiences, and highlighted time efficiencies such as inclusion of the patients most recent laxative administration on the CCDS. Organizational factors including hierarchical cultures, and communication between HCPs caring for the same patient, impact CCDS acceptance and integration. CONCLUSIONS CCDS reduced unnecessary C. difficile testing and were perceived positively by HCPs when integrated into their workflow, and when displayed relevant patient specific information needed for decision-making.
Collapse
Affiliation(s)
- Clare Rock
- Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Oluchi Abosi
- University of Iowa Hospitals & Clinics, Iowa City, Iowa, United States
| | - Susan Bleasdale
- University of Illinois College of Medicine at Chicago, Chicago, United States
| | - Erin Colligan
- NORC at the University of Chicago, Chicago IL 60603, United States
| | - Daniel J Diekema
- University of Iowa Carver College of Medicine, Iowa City, Iowa, United States
| | - Prashila Dullabh
- NORC at the University of Chicago, Chicago IL 60603, United States
| | - Ayse P Gurses
- Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | | | - Jesse T Jacob
- Emory University School of Medicine, Atlanta, Georgia, United States
| | - Sheetal Kandiah
- Emory University School of Medicine, Atlanta, Georgia, United States
| | - Sonam Lama
- NORC at the University of Chicago, Chicago IL 60603, United States
| | - Surbhi Leekha
- University of Maryland School of Medicine, Baltimore, Maryland, United States
| | - Jeanmarie Mayer
- University of Utah School of Medicine, Salt Lake City, Utah, United States
| | - Alfredo J Mena Lora
- University of Illinois College of Medicine at Chicago, Chicago, United States
| | - Daniel J Morgan
- University of Maryland School of Medicine, Baltimore, Maryland, United States
| | - Patience Osei
- Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Sara Pau
- Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | - Jorge L Salinas
- University of Iowa Carver College of Medicine, Iowa City, Iowa, United States
| | - Emily Spivak
- University of Utah School of Medicine, Salt Lake City, Utah, United States
| | - Eric Wenzler
- University of Illinois College of Pharmacy at Chicago, Chicago, United States
| | - Sara E Cosgrove
- Johns Hopkins University School of Medicine, Baltimore, Maryland, United States
| | | |
Collapse
|
38
|
Kam S, Angaramo S, Antoun J, Bhatta MR, Bonds PD, Cadar AG, Chukwuma VU, Donegan PJ, Feldman Z, Grusky AZ, Gupta VK, Hatcher JB, Lee J, Morales NG, Vrana EN, Wessinger BC, Zhang MZ, Fowler MJ, Hendrickson CD. Improving annual albuminuria testing for individuals with diabetes. BMJ Open Qual 2022; 11:bmjoq-2021-001591. [PMID: 35101868 PMCID: PMC8804706 DOI: 10.1136/bmjoq-2021-001591] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2021] [Accepted: 01/20/2022] [Indexed: 11/17/2022] Open
Abstract
Background Annual albuminuria screening detects the early stages of nephropathy in individuals with diabetes. Because early detection of albuminuria allows for interventions that lower the risk of developing chronic kidney disease, guidelines recommend annual testing for all individuals with type 2 diabetes mellitus and for those with type 1 diabetes for at least 5 years. However, at the Eskind Diabetes Clinic at the Vanderbilt University Medical Center, testing occurred less frequently than desired. Methods A quality improvement team first analysed the clinic’s processes, identifying the lack of a systematic approach to testing as the likely cause for the low rate. The team then implemented two successive interventions in a pilot of patients seen by nurse practitioners in the clinic. In the first intervention, staff used a dashboard within the electronic health record while triaging each patient, pending an albuminuria order if testing had not been done within the past year. In the second intervention, clinic leadership sent daily reminders to the triage staff. A statistical process control chart tracked monthly testing rates. Results After 6 months, annual albuminuria testing increased from a baseline of 69% to 82%, with multiple special-cause signals in the control chart. Conclusions This project demonstrates that a series of simple interventions can significantly impact annual albuminuria testing. This project’s success likely hinged on using an existing workflow to systematically determine if a patient was due for testing and prompting the provider to sign a pended order for an albuminuria test. Other diabetes/endocrinology and primary care clinics can likely implement a similar process and so improve testing rates in other settings. When coupled with appropriate interventions to reduce the development of chronic kidney disease, such interventions would improve patient outcomes, in addition to better adhering to an established quality metric.
Collapse
Affiliation(s)
- Sharon Kam
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | | | | | - Manasa R Bhatta
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | | | - Adrian G Cadar
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | | | | | - Zachary Feldman
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Alan Z Grusky
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Veerain K Gupta
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Jeremy B Hatcher
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Jaclyn Lee
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | | | - Erin N Vrana
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | | | - Michael Z Zhang
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
| | - Michael J Fowler
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
- Department of Medicine, Division of Diabetes, Endocrinology, and Metabolism, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Chase D Hendrickson
- Vanderbilt University School of Medicine, Nashville, Tennessee, USA
- Department of Medicine, Division of Diabetes, Endocrinology, and Metabolism, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| |
Collapse
|
39
|
Selecting EHR-driven recruitment strategies: An evidence-based decision guide. J Clin Transl Sci 2022; 6:e108. [PMID: 36285016 PMCID: PMC9549481 DOI: 10.1017/cts.2022.439] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2022] [Revised: 08/02/2022] [Accepted: 08/02/2022] [Indexed: 11/09/2022] Open
Abstract
Participant recruitment for research is a persistent bottleneck that can be improved by leveraging electronic health records (EHRs). Despite emerging evidence for various EHR-driven approaches, guidance for those attempting to select and use such approaches is limited. The national Recruitment Innovation Center established the EHR Recruitment Consult Resource (ERCR) service line to support multisite studies through implementation of EHR-driven recruitment strategies. As the ERCR, we evolved a guide through 17 consultations over 3 years with multisite studies recruiting in diverse biomedical research domains. We assessed literature and engaged domain experts to identify five key EHR-driven recruitment strategies: direct to patient messages, candidate lists for mailings/calls, direct to research alerts, point of care alerts, and participant registries. Differentiating factors were grouped into factors of study population, study protocol and recruitment workflows, and recruitment site capabilities. The decision matrix indicates acceptable or preferred strategies based on the differentiating factors. Across the ERCR consultations, candidate lists for mailing or calls were most common, participant registries were least frequently recommended, and for some studies no EHR-driven recruitment was recommended. Comparative effectiveness research is needed to refine further evidence for these and potentially new strategies to come.
Collapse
|
40
|
Jones GF, Fabre V, Hinson J, Levin S, Toerper M, Townsend J, Cosgrove SE, Saheed M, Klein EY. Improving antimicrobial prescribing for upper respiratory infections in the emergency department: Implementation of peer comparison with behavioral feedback. ANTIMICROBIAL STEWARDSHIP & HEALTHCARE EPIDEMIOLOGY : ASHE 2021; 1:e70. [PMID: 36168488 PMCID: PMC9495637 DOI: 10.1017/ash.2021.240] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Revised: 11/08/2021] [Accepted: 11/09/2021] [Indexed: 06/16/2023]
Abstract
OBJECTIVE To reduce inappropriate antibiotic prescribing for acute respiratory infections (ARIs) by employing peer comparison with behavioral feedback in the emergency department (ED). DESIGN A controlled before-and-after study. SETTING The study was conducted in 5 adult EDs at teaching and community hospitals in a health system. PATIENTS Adults presenting to the ED with a respiratory condition diagnosis code. Hospitalized patients and those with a diagnosis code for a non-respiratory condition for which antibiotics are or may be warranted were excluded. INTERVENTIONS After a baseline period from January 2016 to March 2018, 3 EDs implemented a feedback intervention with peer comparison between April 2018 and December 2019 for attending physicians. Also, 2 EDs in the health system served as controls. Using interrupted time series analysis, the inappropriate ARI prescribing rate was calculated as the proportion of antibiotic-inappropriate ARI encounters with a prescription. Prescribing rates were also evaluated for all ARIs. Attending physicians at intervention sites received biannual e-mails with their inappropriate prescribing rate and had access to a dashboard that was updated daily showing their performance relative to their peers. RESULTS Among 28,544 ARI encounters, the inappropriate prescribing rate remained stable at the control EDs between the 2 periods (23.0% and 23.8%). At the intervention sites, the inappropriate prescribing rate decreased significantly from 22.0% to 15.2%. Between periods, the overall ARI prescribing rate was 38.1% and 40.6% in the control group and 35.9% and 30.6% in the intervention group. CONCLUSIONS Behavioral feedback with peer comparison can be implemented effectively in the ED to reduce inappropriate prescribing for ARIs.
Collapse
Affiliation(s)
- George F. Jones
- Division of Infectious Diseases, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
- Eastern Virginia Medical School, Norfolk, Virginia
| | - Valeria Fabre
- Division of Infectious Diseases, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Jeremiah Hinson
- Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Scott Levin
- Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Matthew Toerper
- Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Jennifer Townsend
- Division of Infectious Diseases, Greater Baltimore Medical Center, Towson, Maryland
| | - Sara E. Cosgrove
- Division of Infectious Diseases, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Mustapha Saheed
- Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
| | - Eili Y. Klein
- Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland
- Center for Disease Dynamics, Economics & Policy, Washington DC
| |
Collapse
|
41
|
Shliakhtsitsava K, Fisher ES, Trovillion EM, Bush K, Kuo DJ, Newfield RS, Thornburg CD, Roberts W, Aristizabal P. Improving vitamin D testing and supplementation in children with newly diagnosed cancer: A quality improvement initiative at Rady Children's Hospital San Diego. Pediatr Blood Cancer 2021; 68:e29217. [PMID: 34286891 PMCID: PMC8463415 DOI: 10.1002/pbc.29217] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/11/2021] [Revised: 06/14/2021] [Accepted: 06/17/2021] [Indexed: 11/10/2022]
Abstract
BACKGROUND Vitamin D deficiency and insufficiency have been associated with poorer health outcomes. Children with cancer are at high risk for vitamin D deficiency and insufficiency. At our institution, we identified high variability in vitamin D testing and supplementation in this population. Of those tested, 65% were vitamin D deficient/insufficient. We conducted a quality improvement (QI) initiative with aim to improve vitamin D testing and supplementation among children aged 2-18 years with newly diagnosed cancer to ≥80% over 6 months. METHODS An inter-professional team reviewed baseline data, then developed and implemented interventions using Plan-Do-Study-Act (PDSA) cycles. Barriers were identified using QI tools, including lack of automated triggers for testing and inconsistent supplementation criteria and follow-up testing post supplementation. Interventions included an institutional vitamin D guideline, clinical decision-making tree for vitamin D deficiency, insufficiency and sufficiency, electronic medical record triggers, and automated testing options. RESULTS Baseline: N = 26 patients, four (15%) had baseline vitamin D testing; two (8%) received appropriate supplementation. Postintervention: N = 33 patients; 32 (97%) had baseline vitamin D testing; 33 (100%) received appropriate supplementation and completed follow-up testing timely (6-8 weeks post supplementation). Change was sustained over 24 months. CONCLUSIONS We achieved and sustained our aim for vitamin D testing and supplementation in children with newly diagnosed cancer through inter-professional collaboration of hematology/oncology, endocrinology, hospital medicine, pharmacy, nursing, and information technology. Future PDSA cycles will address patient compliance with vitamin D supplementation and impact on patients' vitamin D levels.
Collapse
Affiliation(s)
- Ksenya Shliakhtsitsava
- Department of Pediatrics, Division of Pediatric Hematology/Oncology, University of California, San Diego/Rady Children’s Hospital San Diego, San Diego, CA,Now with Department of Pediatrics, Division of Pediatric Hematology/Oncology, University of Texas Southwestern, Dallas, TX
| | - Erin Stucky Fisher
- Department of Pediatrics, Division of Pediatric Hospital Medicine, University of California, San Diego/Rady Children’s Hospital San Diego, San Diego, CA
| | - Erin M. Trovillion
- Department of Pediatrics, Division of Pediatric Hematology/Oncology, University of California, San Diego/Rady Children’s Hospital San Diego, San Diego, CA,Now with Department of Pediatrics, Division of Pediatric Hematology/Oncology, Atrium Health, Levine Children’s Cancer and Blood Disorders, Charlotte, NC
| | - Kelly Bush
- Department of Pediatrics, Division of Pediatric Hematology/Oncology, University of California, San Diego/Rady Children’s Hospital San Diego, San Diego, CA,Quality Improvement Committee, Division of Pediatric Hematology/Oncology, Rady Children’s Hospital San Diego, San Diego, CA
| | - Dennis John Kuo
- Department of Pediatrics, Division of Pediatric Hematology/Oncology, University of California, San Diego/Rady Children’s Hospital San Diego, San Diego, CA
| | - Ron S. Newfield
- Department of Pediatrics, Division of Pediatric Endocrinology, University of California, San Diego/Rady Children’s Hospital San Diego, San Diego, CA
| | - Courtney D. Thornburg
- Department of Pediatrics, Division of Pediatric Hematology/Oncology, University of California, San Diego/Rady Children’s Hospital San Diego, San Diego, CA,Department of Pediatrics, Division of Pediatric Hospital Medicine, University of California, San Diego/Rady Children’s Hospital San Diego, San Diego, CA
| | - William Roberts
- Department of Pediatrics, Division of Pediatric Hematology/Oncology, University of California, San Diego/Rady Children’s Hospital San Diego, San Diego, CA,Quality Improvement Committee, Division of Pediatric Hematology/Oncology, Rady Children’s Hospital San Diego, San Diego, CA
| | - Paula Aristizabal
- Department of Pediatrics, Division of Pediatric Hematology/Oncology, University of California, San Diego/Rady Children’s Hospital San Diego, San Diego, CA,Quality Improvement Committee, Division of Pediatric Hematology/Oncology, Rady Children’s Hospital San Diego, San Diego, CA,Population Sciences, Disparities and Community Engagement, University of California, San Diego Moores Cancer Center, La Jolla, CA
| |
Collapse
|
42
|
Tolan NV, Ahmed S, Terebo T, Virk ZM, Petrides AK, Ransohoff JR, Demetriou CA, Kelly YP, Melanson SE, Mendu ML. The Impact of Outpatient Laboratory Alerting Mechanisms in Patients with AKI. KIDNEY360 2021; 2:1560-1568. [PMID: 35372977 PMCID: PMC8785781 DOI: 10.34067/kid.0003312021] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/17/2021] [Accepted: 07/14/2021] [Indexed: 02/04/2023]
Abstract
Background AKI is an abrupt decrease in kidney function associated with significant morbidity and mortality. Electronic notifications of AKI have been utilized in patients who are hospitalized, but their efficacy in the outpatient setting is unclear. Methods We evaluated the effect of two outpatient interventions: an automated comment on increasing creatinine results (intervention I; 6 months; n=159) along with an email to the provider (intervention II; 3 months; n=105), compared with a control (baseline; 6 months; n=176). A comment was generated if a patient's creatinine increased by >0.5 mg/dl (previous creatinine ≤2.0 mg/dl) or by 50% (previous creatinine >2.0 mg/dl) within 180 days. Process measures included documentation of AKI and clinical actions. Clinical outcomes were defined as recovery from AKI within 7 days, prolonged AKI from 8 to 89 days, and progression to CKD with in 120 days. Results Providers were more likely to document AKI in interventions I (P=0.004; OR, 2.80; 95% CI, 1.38 to 5.67) and II (P=0.01; OR, 2.66; 95% CI, 1.21 to 5.81). Providers were also more likely to discontinue nephrotoxins in intervention II (P<0.001; OR, 4.88; 95% CI, 2.27 to 10.50). The median time to follow-up creatinine trended shorter among patients with AKI documented (21 versus 42 days; P=0.11). There were no significant differences in clinical outcomes. Conclusions An automated comment was associated with improved documented recognition of AKI and the additive intervention of an email alert was associated with increased discontinuation of nephrotoxins, but neither improved clinical outcomes. Translation of these findings into improved outcomes may require corresponding standardization of clinical practice protocols for managing AKI.
Collapse
Affiliation(s)
- Nicole V. Tolan
- Department of Pathology, Brigham and Women’s Hospital, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| | - Salman Ahmed
- Department of Medicine, Baylor College of Medicine, Houston, Texas
| | - Tolumofe Terebo
- Department of Pathology, Brigham and Women’s Hospital, Boston, Massachusetts
- Department of Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
| | | | - Athena K. Petrides
- Department of Pathology, Brigham and Women’s Hospital, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| | - Jaime R. Ransohoff
- Department of Epidemiology, Bloomberg School of Public Health, Baltimore, Maryland
| | - Christiana A. Demetriou
- Department of Primary Care and Population Health, University of Nicosia Medical School, Nicosia, Cyprus
- The Cyprus School of Molecular Medicine, The Cyprus Institute of Neurology and Genetics, Nicosia, Cyprus
| | - Yvelynne P. Kelly
- Department of Critical Care Medicine, St. James Hospital, Dublin, Ireland
| | - Stacy E.F. Melanson
- Department of Pathology, Brigham and Women’s Hospital, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| | - Mallika L. Mendu
- Harvard Medical School, Boston, Massachusetts
- Department of Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
| |
Collapse
|
43
|
Olakotan OO, Mohd Yusof M. The appropriateness of clinical decision support systems alerts in supporting clinical workflows: A systematic review. Health Informatics J 2021; 27:14604582211007536. [PMID: 33853395 DOI: 10.1177/14604582211007536] [Citation(s) in RCA: 22] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
Abstract
A CDSS generates a high number of inappropriate alerts that interrupt the clinical workflow. As a result, clinicians silence, disable, or ignore alerts, thereby undermining patient safety. Therefore, the effectiveness and appropriateness of CDSS alerts need to be evaluated. A systematic review was carried out to identify the factors that affect CDSS alert appropriateness in supporting clinical workflow. Seven electronic databases (PubMed, Scopus, ACM, Science Direct, IEEE, Ovid Medline, and Ebscohost) were searched for English language articles published between 1997 and 2018. Seventy six papers met the inclusion criteria, of which 26, 24, 15, and 11 papers are retrospective cohort, qualitative, quantitative, and mixed-method studies, respectively. The review highlights various factors influencing the appropriateness and efficiencies of CDSS alerts. These factors are categorized into technology, human, organization, and process aspects using a combination of approaches, including socio-technical framework, five rights of CDSS, and Lean. Most CDSS alerts were not properly designed based on human factor methods and principles, explaining high alert overrides in clinical practices. The identified factors and recommendations from the review may offer valuable insights into how CDSS alerts can be designed appropriately to support clinical workflow.
Collapse
|
44
|
Yerrapragada G, Siadimas A, Babaeian A, Sharma V, O'Neill TJ. Machine Learning to Predict Tamoxifen Nonadherence Among US Commercially Insured Patients With Metastatic Breast Cancer. JCO Clin Cancer Inform 2021; 5:814-825. [PMID: 34383580 DOI: 10.1200/cci.20.00102] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023] Open
Abstract
PURPOSE Adherence to tamoxifen citrate among women diagnosed with metastatic breast cancer can improve survival and minimize recurrence. This study aimed to use real-world data and machine learning (ML) methods to classify tamoxifen nonadherence. METHODS A cohort of women diagnosed with metastatic breast cancer from 2012 to 2017 were identified from IBM MarketScan Commercial Claims and Encounters and Medicare claims databases. Patients with < 80% proportion of days coverage in the year following treatment initiation were classified as nonadherent. Training and internal validation cohorts were randomly generated (4:1 ratio). Clinical procedures, comorbidity, treatment, and health care encounter features in the year before tamoxifen initiation were used to train logistic regression, boosted logistic regression, random forest, and feedforward neural network models and were internally validated on the basis of area under receiver operating characteristic curve. The most predictive ML approach was evaluated to assess feature importance. RESULTS A total of 3,022 patients were included with 40% classified as nonadherent. All models had moderate predictive accuracy. Logistic regression (area under receiver operating characteristic 0.64) was interpreted with 94% sensitivity (95% CI, 89 to 92) and 0.31 specificity (95% CI, 29 to 33). The model accurately classified adherence (negative predictive value 89%) but was nondiscriminate for nonadherence (positive predictive value 48%). Variable importance identified top predictive factors, including age ≥ 55 years and pretreatment procedures (lymphatic nuclear medicine, radiation oncology, and arterial surgery). CONCLUSION ML using baseline administrative data predicts tamoxifen nonadherence. Screening at treatment initiation may support personalized care, improve health outcomes, and minimize cost. Baseline claims may not be sufficient to discriminate adherence. Further validation with enriched longitudinal data may improve model performance.
Collapse
Affiliation(s)
- Gayathri Yerrapragada
- School of Computing, Clemson University, Clemson, SC.,Data Science & Services, Diagnostics Information Solutions, Roche Diagnostics, Belmont, CA
| | - Athanasios Siadimas
- Data Science & Services, Diagnostics Information Solutions, Roche Diagnostics, Belmont, CA
| | - Amir Babaeian
- Data Science & Services, Diagnostics Information Solutions, Roche Diagnostics, Belmont, CA
| | - Vishakha Sharma
- Data Science & Services, Diagnostics Information Solutions, Roche Diagnostics, Belmont, CA
| | - Tyler J O'Neill
- Data Science & Services, Diagnostics Information Solutions, Roche Diagnostics, Belmont, CA
| |
Collapse
|
45
|
Physician Well-being and the Future of Health Information Technology. Mayo Clin Proc Innov Qual Outcomes 2021; 5:753-761. [PMID: 34377947 PMCID: PMC8332366 DOI: 10.1016/j.mayocpiqo.2021.06.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
Abstract
The issue of clinician burnout has become a growing concern in health care, with an increased emphasis on health information technology as a contributing factor. Technology-mediated stresses have arisen with the electronic health record, and we can anticipate new and different impacts from future information tools. This article discusses technology's pivotal role in physician well-being, not only in the quality of its design but also through its capacity to enable future models of care that are more manageable for physicians and more effective for patients. Three general aims along with specific efforts are proposed to benefit physician well-being in technology-mediated work.
Collapse
|
46
|
Rogers JR, Lee J, Zhou Z, Cheung YK, Hripcsak G, Weng C. Contemporary use of real-world data for clinical trial conduct in the United States: a scoping review. J Am Med Inform Assoc 2021; 28:144-154. [PMID: 33164065 DOI: 10.1093/jamia/ocaa224] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2020] [Revised: 08/11/2020] [Accepted: 09/02/2020] [Indexed: 12/28/2022] Open
Abstract
OBJECTIVE Real-world data (RWD), defined as routinely collected healthcare data, can be a potential catalyst for addressing challenges faced in clinical trials. We performed a scoping review of database-specific RWD applications within clinical trial contexts, synthesizing prominent uses and themes. MATERIALS AND METHODS Querying 3 biomedical literature databases, research articles using electronic health records, administrative claims databases, or clinical registries either within a clinical trial or in tandem with methodology related to clinical trials were included. Articles were required to use at least 1 US RWD source. All abstract screening, full-text screening, and data extraction was performed by 1 reviewer. Two reviewers independently verified all decisions. RESULTS Of 2020 screened articles, 89 qualified: 59 articles used electronic health records, 29 used administrative claims, and 26 used registries. Our synthesis was driven by the general life cycle of a clinical trial, culminating into 3 major themes: trial process tasks (51 articles); dissemination strategies (6); and generalizability assessments (34). Despite a diverse set of diseases studied, <10% of trials using RWD for trial process tasks evaluated medications or procedures (5/51). All articles highlighted data-related challenges, such as missing values. DISCUSSION Database-specific RWD have been occasionally leveraged for various clinical trial tasks. We observed underuse of RWD within conducted medication or procedure trials, though it is subject to the confounder of implicit report of RWD use. CONCLUSION Enhanced incorporation of RWD should be further explored for medication or procedure trials, including better understanding of how to handle related data quality issues to facilitate RWD use.
Collapse
Affiliation(s)
- James R Rogers
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| | - Junghwan Lee
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| | - Ziheng Zhou
- Institute of Human Nutrition, Columbia University, New York, New York, USA
| | - Ying Kuen Cheung
- Department of Biostatistics, Columbia University, New York, New York, USA, and
| | - George Hripcsak
- Department of Biomedical Informatics, Columbia University, New York, New York, USA.,Medical Informatics Services, New York-Presbyterian Hospital, New York, New York, USA
| | - Chunhua Weng
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
| |
Collapse
|
47
|
Hayashi M, Grover TR, Small S, Staples T, Roosevelt G. Improving timeliness of hepatitis B vaccine administration in an urban safety net level III NICU. BMJ Qual Saf 2021; 30:911-919. [PMID: 34001649 DOI: 10.1136/bmjqs-2020-012869] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2020] [Revised: 04/11/2021] [Accepted: 04/25/2021] [Indexed: 11/03/2022]
Abstract
OBJECTIVE To avoid preventable consequences of perinatal hepatitis B infection, all infants should be given hepatitis B vaccine (HBV) within 24 hours of birth if birth weight is ≥2 kg and at 30 days of life or at discharge if <2 kg, to provide highest seroprotection rates while ensuring universal vaccination prior to discharge. We aimed to achieve timely HBV administration in >80% of eligible infants in both birthweight groups and decrease infants discharged home without receiving HBV to <1% over an 18-month period and sustain results for an additional 15 months. METHODS Data were collected from June 2016 to May 2020 in a level III neonatal intensive care unit. A multidisciplinary team identified barriers and interventions through Plan-Do-Study-Act cycles from September 2017 to February 2019: using pharmacists as champions, overcoming legal barriers, staff education and best practice alerts (BPAs) embedded in electronic health records. Statistical process control (SPC) p charts were used to evaluate the primary outcome measure, monthly percentage of infants receiving timely HBV administration stratified by birthweight categories (≥2 and <2 kg). For infants receiving HBV outside the time frame, absolute difference of timeliness was calculated. RESULTS Mean timely HBV administration improved from 45% to 95% (≥2 kg) and from 45% to 85% (<2 kg) with special cause variation in SPC charts. Infants discharged without receiving HBV decreased from 4.6% to 0.22%. Of those given HBV outside the recommended time frame, median absolute time between recommended and actual administration time decreased significantly: from 3.5 days (IQR 1.6, 8.6) to 0.3 day (IQR 0.1, 0.8) (p<0.001) in ≥2 kg group and from 6 days (IQR 1, 15) to 1 day (IQR 1, 6.5) (p=0.009) in <2 kg group. CONCLUSIONS Using a multidisciplinary approach, we significantly improved and sustained timely HBV administration and nearly eliminated infants discharged home without receiving HBV. Pharmacists as champions and BPAs were critical to our success.
Collapse
Affiliation(s)
- Madoka Hayashi
- Department of Pediatrics, Denver Health and Hospital Authority, Denver, CO, USA .,Department of Pediatrics, Section of Neonatology, University of Colorado School of Medicine, Aurora, CO, USA.,Department of Pediatrics, Section of Neonatology, Children's Hospital Colorado, Aurora, CO, USA
| | - Theresa R Grover
- Department of Pediatrics, Section of Neonatology, University of Colorado School of Medicine, Aurora, CO, USA.,Department of Pediatrics, Section of Neonatology, Children's Hospital Colorado, Aurora, CO, USA
| | - Steve Small
- Department of Pediatrics, Denver Health and Hospital Authority, Denver, CO, USA
| | - Tessa Staples
- Department of Pediatrics, Denver Health and Hospital Authority, Denver, CO, USA
| | - Genie Roosevelt
- Department of Emergency Medicine, Denver Health and Hospital Authority, Denver, CO, USA.,Department of Emergency Medicine, University of Colorado School of Medicine, Aurora, CO, USA
| |
Collapse
|
48
|
Sloss EA, Jones TL. Nurse Cognition, Decision Support, and Barcode Medication Administration: A Conceptual Framework for Research, Practice, and Education. Comput Inform Nurs 2021; 39:851-857. [PMID: 33935198 DOI: 10.1097/cin.0000000000000724] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
This article synthesizes theoretical perspectives related to nurse cognition. We present a conceptual model that can be used by multiple stakeholders to study and contemplate how nurses use clinical decision support systems, and specifically, Barcode-Assisted Medication Administration, to make decisions during the delivery of care. Theoretical perspectives integrated into the model include dual process theory, the Cognitive Continuum Theory, human factors engineering, and the Recognition-Primed Decision model. The resulting framework illustrates the process of nurse cognition during Barcode-Assisted Medication Administration. Additionally, the model includes individual or human and environmental factors that may influence nurse cognition and decision making. It is important to consider the influence of individual, human, and environmental factors on the process of nurse cognition and decision making. Specifically, it is necessary to explore the impact of heuristics and biases on clinician decision making, particularly related to the development of alarm and alert fatigue. Aided by the proposed framework, stakeholders may begin to identify heuristics and cognitive biases that influence the decision of clinicians to accept or override a clinical decision support system alert and whether heuristics and biases are associated with inappropriate alert override.
Collapse
Affiliation(s)
- Elizabeth Ann Sloss
- Author Affiliations: Department of Professional Nursing Practice, Georgetown University (Ms Sloss), Washington, DC; and Department of Adult Health and Nursing Systems, Virginia Commonwealth University (Dr Jones), Richmond
| | | |
Collapse
|
49
|
Joglekar NN, Patel Y, Keller MS. Evaluation of Clinical Decision Support to Reduce Sedative-Hypnotic Prescribing in Older Adults. Appl Clin Inform 2021; 12:436-444. [PMID: 34107541 PMCID: PMC8189759 DOI: 10.1055/s-0041-1730030] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023] Open
Abstract
OBJECTIVE We sought to characterize the performance of inpatient and outpatient computerized clinical decision support (CDS) alerts aimed at reducing inappropriate benzodiazepine and nonbenzodiazepine sedative medication prescribing in older adults 18 months after implementation. METHODS We reviewed the performance of two CDS alerts in the outpatient and inpatient settings in 2019. To examine the alerts' effectiveness, we analyzed metrics including overall alert adherence, provider-level adherence, and reasons for alert trigger and override. RESULTS In 2019, we identified a total of 14,534 and 4,834 alerts triggered in the outpatient and inpatient settings, respectively. Providers followed only 1% of outpatient and 3% of inpatient alerts. Most alerts were ignored (68% outpatient and 60% inpatient), while providers selected to override the remaining alerts. In each setting, the top 2% of clinicians were responsible for approximately 25% of all ignored or overridden alerts. However, a small proportion of clinicians (2% outpatient and 4% inpatient) followed the alert at least half of the time and accounted for a disproportionally large fraction of the total followed alerts. Our analysis of the free-text comments revealed that many alerts were to continue outpatient prescriptions or for situational anxiety. CONCLUSION Our findings highlight the importance of evaluation of CDS performance after implementation. We found large variation in response to the inpatient and outpatient alerts, both with respect to follow and ignore rates. Reevaluating the alert design by providing decision support by indication may be more helpful and may reduce alert fatigue.
Collapse
Affiliation(s)
- Natasha N. Joglekar
- Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Boston, Massachusetts, United Sates
| | - Yatindra Patel
- Division of General Internal Medicine, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, California, United Sates
| | - Michelle S. Keller
- Division of General Internal Medicine, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, California, United Sates,Division of Informatics, Department of Biomedical Sciences, Cedars-Sinai Medical Center, Los Angeles, California, United Sates,Department of Health Policy and Management, Fielding School of Public Health, University of California-Los Angeles, Los Angeles, California, United Sates,Address for correspondence Michelle S. Keller, PhD, MPH Division of General Internal Medicine, Department of Medicine, Cedars-Sinai Medical CenterLos Angeles, CA 90048United Sates
| |
Collapse
|
50
|
Machine Learning and Clinical Informatics for Improving HIV Care Continuum Outcomes. Curr HIV/AIDS Rep 2021; 18:229-236. [PMID: 33661445 DOI: 10.1007/s11904-021-00552-3] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/23/2021] [Indexed: 10/22/2022]
Abstract
PURPOSE OF REVIEW This manuscript reviews the use of electronic medical record (EMR) data for HIV care and research along the HIV care continuum with a specific focus on machine learning methods and clinical informatics interventions. RECENT FINDINGS EMR-based clinical decision support tools and electronic alerts have been effectively utilized to improve HIV care continuum outcomes. Accurate EMR-based machine learning models have been developed to predict HIV diagnosis, retention in care, and viral suppression. Natural language processing (NLP) of clinical notes and data sharing between healthcare systems and public health agencies can enhance models for identifying people living with HIV who are undiagnosed or in need of relinkage to care. Challenges related to using these technologies include inconsistent EMR documentation, alert fatigue, and the potential for bias. Clinical informatics and machine learning models are promising tools for improving HIV care continuum outcomes. Future research should focus on methods for combining EMR data with additional data sources (e.g., social media, geospatial data) and studying how to effectively implement predictive models for HIV care into clinical practice.
Collapse
|