1
|
Cahill A, Martin M, Beachy B, Bauman D, Howard-Young J. The contextual interview: a cross-cutting patient-interviewing approach for social context. MEDICAL EDUCATION ONLINE 2024; 29:2295049. [PMID: 38320114 PMCID: PMC10848999 DOI: 10.1080/10872981.2023.2295049] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Accepted: 12/11/2023] [Indexed: 02/08/2024]
Abstract
Patient interviewing pedagogy in medical education has not evolved to comprehensively capture the biopsychosocial model of healthcare delivery. While gathering a patient's social history targets important aspects of social context it does not adequately capture and account for the real-time reassessment required to understand evolving factors that influence exposure to drivers of health inequities, social determinants of health, and access to supports that promote health. The authors offer a patient interviewing approach called the Contextual Interview (CI) that specifically targets dynamic and ever-changing social context information. To substantiate the use of the CI in medical education, the authors conducted a qualitative review of the Accreditation Council for Graduate Medical Education Milestones for primary care specialties (Family Medicine, Internal Medicine, and Pediatrics). Milestones were coded to the extent to which they reflected the learner's need to acknowledge, assess, synthesize and/or apply patient contextual data in real-time patient encounters. Approximately 1 in 5 milestones met the context-related and patient-facing criteria. This milestone review further highlights the need for more intentional training in eliciting meaningful social context data during patient interviewing. The CI as a cross-cutting, practical, time-conscious, and semi-structured patient interviewing approach that deliberately elicits information to improve the clinician's sense and understanding of a patient's social context. The authors reviewed future directions in researching adapted versions of the CI for undergraduate and graduate medical education.
Collapse
Affiliation(s)
- Amber Cahill
- Department of Family Medicine and Community Health, University of Massachusetts Chan Medical School, Worcester, MA, USA
| | - Matthew Martin
- College of Health Solutions, Arizona State University, Tempe, AZ, USA
| | - Bridget Beachy
- Central Washington Family Medicine Residency, Community Health of Central Washington, Yakima, WA, USA
| | - David Bauman
- Central Washington Family Medicine Residency, Community Health of Central Washington, Yakima, WA, USA
| | - Jordan Howard-Young
- Department of Family Medicine and Community Health, University of Massachusetts Chan Medical School, Worcester, MA, USA
| |
Collapse
|
2
|
Kim K, Faruque SC, Kulp D, Lam S, Sperling LS, Eapen DJ. Primary care clinician engagement in implementing a machine-learning algorithm for targeted screening of familial hypercholesterolemia. Am J Prev Cardiol 2024; 19:100710. [PMID: 39176132 PMCID: PMC11338952 DOI: 10.1016/j.ajpc.2024.100710] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2024] [Revised: 07/09/2024] [Accepted: 07/17/2024] [Indexed: 08/24/2024] Open
Abstract
Objective To assess the impact of a multi-pronged educational approach on the knowledge, attitudes, and behaviors regarding Familial Hypercholesterolemia (FH) management at a large academic medical center with the aim of empowering primary care clinicians (PCC) to diagnose and treat FH. Methods A comprehensive educational program for PCCs on FH management was developed and piloted from July 2022 to March 2024. Components of our intervention included: 1. Implementation of a novel clinical decision support tool in the electronic medical record for FH management, 2. Development and dissemination of an interactive educational website focused on FH and its management, 3. Delivery of virtual instructional sessions to increase awareness of the tool, provide education on its use, and obtain support from institutional leadership, and 4. Direct outreach to a pilot subset of PCCs whose patients had been detected using the validated FIND FH® machine learning algorithm. Participating clinicians were surveyed at baseline before the intervention and after the educational session. Results 70 PCC consented to participate in the study with a survey completion rate of 79 % (n = 55) and 42 % (n = 23) for the baseline and follow-up surveys, respectively. Objective PCC knowledge scores improved from 40 to 65 % of responders correctly responding to at least 2/3rds of survey questions. Despite the fact that 87 % identified PCC's as most effective for early detection of FH, 100 % of PCCs who received direct outreach chose to defer care to an outpatient cardiologist over pursuing workup in the primary care setting. Conclusion Empowering PCCs in management of FH serves as a key strategy in addressing this underdiagnosed and undertreated potentially life-threatening condition. A systems-based approach to addressing these aims may include leveraging EMR-based clinical decision support models and cross-disciplinary educational partnerships with medical specialists.
Collapse
Affiliation(s)
- Kain Kim
- Emory School of Medicine, Atlanta GA 30306, USA
| | - Samir C. Faruque
- Division of General Medicine, Washington University School of Medicine, St. Louis MO 63110, USA
| | - David Kulp
- Emory School of Medicine, Atlanta GA 30306, USA
| | - Shivani Lam
- Department of Biology, Emory University, Wayne Rollins Research Center, Atlanta GA 30306, USA
| | - Laurence S. Sperling
- Hubert Department of Global Health, Rollins School of Public Health at Emory University, Atlanta GA 30306, USA
- Emory Center for Heart Disease Prevention, Atlanta GA 30306, USA
- Emory Clinical Cardiovascular Research Institute, Atlanta GA 30306, USA
| | - Danny J. Eapen
- Department of Medicine, Emory School of Medicine, Atlanta GA 30306, USA
| |
Collapse
|
3
|
Esquerda M, Pifarré-Esquerda F. Artificial intelligence in medicine: Ethical, deontological aspects and the impact on the doctor-patient relationship. Med Clin (Barc) 2024; 163:e44-e48. [PMID: 38719685 DOI: 10.1016/j.medcli.2024.03.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 03/04/2024] [Accepted: 03/07/2024] [Indexed: 08/07/2024]
Affiliation(s)
- Montse Esquerda
- Institut Borja de Bioètica-URL, Comissió de Deontología Consell de Col·legis de Metges de Catalunya, Esplugues de Llobregat, Barcelona, España.
| | - Francesc Pifarré-Esquerda
- Estudiante de matemáticas, Facultat de Matemàtiques i Estadística (FME), Universitat Politècnica de Catalunya (UPC), Barcelona, España
| |
Collapse
|
4
|
Rhee CH, Brown JT, Lang A, Pentz RD, Nazha B. Billing for Electronic Patient-Physician Communications: An Ethical Analysis. JCO Oncol Pract 2024; 20:1040-1045. [PMID: 38593382 DOI: 10.1200/op.23.00569] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/06/2023] [Revised: 01/22/2024] [Accepted: 03/13/2024] [Indexed: 04/11/2024] Open
Abstract
This review paper analyzes the ethical implications of billing patients for electronic communication with physicians through electronic health records, a practice already adopted by medical institutions such as the Cleveland Clinic. The analysis assesses how billing aligns with pillars of medical ethics which include beneficence, respect for persons, and justice. Although billing may enhance communication, improve patient care, and alleviate physician burnout, concerns arise over potential consequences on patient autonomy, trust, and health care disparities. The review delves into the intricate balance of these ethical principles by first considering the potential benefits of incentivizing concise questions and improving physician workload management through billing. By reducing messages, this approach can potentially mitigate burnout and enhance care. It also acknowledges potential drawbacks such as deterring patients because of financial constraints and eroding trust in physicians and the medical team. It emphasizes the necessity of thoroughly examining all aspects of this intricate ethical dilemma to formulate a nuanced solution that protects patient well-being while respecting physicians. We propose a middle-ground approach involving nominal and transparent billing on the basis of the question's complexity, urgency, and level of expertise required in the response. Transparent billing policies, up-front communication of costs, and potential fee waivers on the basis of socioeconomic status can address equity concerns and maintain patient trust. Striking a balance between the potential benefits and drawbacks of billing for patient questions is crucial in maintaining ethical patient-physician interactions and equitable health care provision. The analysis underscores the importance of aligning online patient-physician communication with ethical principles within the evolving digital health care landscape.
Collapse
Affiliation(s)
- Christopher H Rhee
- Medical College of Georgia Augusta University/University of Georgia Medical Partnership, Athens, GA
| | - Jacqueline T Brown
- Department of Hematology and Medical Oncology, Emory University School of Medicine, Atlanta, GA
- Ambulatory Infusion Center, Winship Cancer Institute of Emory University, Atlanta, GA
| | - Ayannah Lang
- Hematology and Oncology in Research Ethics, Emory University School of Medicine, Atlanta, GA
| | - Rebecca D Pentz
- Hematology and Oncology in Research Ethics, Emory University School of Medicine, Atlanta, GA
| | - Bassel Nazha
- Department of Hematology and Medical Oncology, Emory University School of Medicine, Atlanta, GA
| |
Collapse
|
5
|
Hendrix N, Maisel N, Everson J, Patel V, Bazemore A, Rotenstein LS, Holmgren AJ, Krist AH, Adler-Milstein J, Phillips RL. Impact of response bias in three surveys on primary care providers' experiences with electronic health records. J Am Med Inform Assoc 2024; 31:1754-1762. [PMID: 38894620 PMCID: PMC11258403 DOI: 10.1093/jamia/ocae148] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 05/30/2024] [Accepted: 06/03/2024] [Indexed: 06/21/2024] Open
Abstract
OBJECTIVE To identify impacts of different survey methodologies assessing primary care physicians' (PCPs') experiences with electronic health records (EHRs), we compared three surveys: the 2022 Continuous Certification Questionnaire (CCQ) from the American Board of Family Medicine, the 2022 University of California San Francisco (UCSF) Physician Health IT Survey, and the 2021 National Electronic Health Records Survey (NEHRS). MATERIALS AND METHODS We evaluated differences between survey pairs using Rao-Scott corrected chi-square tests, which account for weighting. RESULTS CCQ received 3991 responses from PCPs (100% response rate), UCSF received 1375 (3.6% response rate), and NEHRS received 858 (18.2% response rate). Substantial, statistically significant differences in demographics were detected across the surveys. CCQ respondents were younger and more likely to work in a health system; NEHRS respondents were more likely to work in private practice; and UCSF respondents disproportionately practiced in larger academic settings. Many EHR experience indicators were similar between CCQ and NEHRS, but CCQ respondents reported higher documentation burden. DISCUSSION The UCSF approach is unlikely to supply reliable data. Significant demographic differences between CCQ and NEHRS raise response bias concerns, and while there were similarities in some reported EHR experiences, there were important, significant differences. CONCLUSION Federal EHR policy monitoring and maintenance require reliable data. This test of existing and alternative sources suggest that diversified data sources are necessary to understand physicians' experiences with EHRs and interoperability. Comprehensive surveys administered by specialty boards have the potential to contribute to these efforts, since they are likely to be free of response bias.
Collapse
Affiliation(s)
- Nathaniel Hendrix
- American Board of Family Medicine, Lexington, KY 40511, United States
- Center for Professionalism and Value in Health Care, Washington, DC 20036, United States
| | - Natalya Maisel
- Division of Clinical Informatics and Digital Transformation, University of California, San Francisco, San Francisco, CA 94117, United States
| | - Jordan Everson
- Department of Health and Human Services, Washington, DC 20201, United States
| | - Vaishali Patel
- Department of Health and Human Services, Washington, DC 20201, United States
| | - Andrew Bazemore
- American Board of Family Medicine, Lexington, KY 40511, United States
- Center for Professionalism and Value in Health Care, Washington, DC 20036, United States
| | - Lisa S Rotenstein
- Division of Clinical Informatics and Digital Transformation, University of California, San Francisco, San Francisco, CA 94117, United States
| | - A Jay Holmgren
- Division of Clinical Informatics and Digital Transformation, University of California, San Francisco, San Francisco, CA 94117, United States
| | - Alex H Krist
- Family Medicine & Population Health, Virginia Commonwealth University, Richmond, VA 23219, United States
| | - Julia Adler-Milstein
- Division of Clinical Informatics and Digital Transformation, University of California, San Francisco, San Francisco, CA 94117, United States
| | - Robert L Phillips
- American Board of Family Medicine, Lexington, KY 40511, United States
- Center for Professionalism and Value in Health Care, Washington, DC 20036, United States
| |
Collapse
|
6
|
Hoffer EP. Primary Care in the United States: Past, Present and Future. Am J Med 2024; 137:702-705. [PMID: 38499134 DOI: 10.1016/j.amjmed.2024.03.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/05/2024] [Revised: 03/11/2024] [Accepted: 03/12/2024] [Indexed: 03/20/2024]
Abstract
Even though a well-functioning primary care system is widely acknowledged as critical to population health, the number of primary care physicians (PCPs) practicing in the United States has steadily declined, and PCPs are in short supply. The reasons are multiple and include inadequate income relative to other specialties, excessive administrative demands on PCPs and the lack of respect given to primary care specialties during medical school and residency. Advanced practice nurses can augment the services of primary care physicians but cannot substitute for them. To change this situation, we need action on several fronts. Medical schools should give preference to students who are more likely to enter the primary care specialties. The income gap between primary care and other specialties should be narrowed. The administrative load placed on PCPs, including cumbersome electronic medical records, must be lessened. Insurers, including Medicare and Medicaid, must provide the resources to allow primary care physicians to act as leaders of multidisciplinary teams.
Collapse
|
7
|
Apathy NC, Biro J, Holmgren AJ. Consistency is key: documentation distribution and efficiency in primary care. J Am Med Inform Assoc 2024; 31:1657-1664. [PMID: 38905016 PMCID: PMC11258406 DOI: 10.1093/jamia/ocae156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/16/2024] [Revised: 05/22/2024] [Accepted: 06/11/2024] [Indexed: 06/23/2024] Open
Abstract
OBJECTIVES We analyzed the degree to which daily documentation patterns in primary care varied and whether specific patterns, consistency over time, and deviations from clinicians' usual patterns were associated with note-writing efficiency. MATERIALS AND METHODS We used electronic health record (EHR) active use data from the Oracle Cerner Advance platform capturing hourly active documentation time for 498 physicians and advance practice clinicians (eg, nurse practitioners) for 65 152 clinic days. We used k-means clustering to identify distinct daily patterns of active documentation time and analyzed the relationship between these patterns and active documentation time per note. We determined each primary care clinician's (PCC) modal documentation pattern and analyzed how consistency and deviations were related to documentation efficiency. RESULTS We identified 8 distinct daily documentation patterns; the 3 most common patterns accounted for 80.6% of PCC-days and differed primarily in average volume of documentation time (78.1 minutes per day; 35.4 minutes per day; 144.6 minutes per day); associations with note efficiency were mixed. PCCs with >80% of days attributable to a single pattern demonstrated significantly more efficient documentation than PCCs with lower consistency; for high-consistency PCCs, days that deviated from their usual patterns were associated with less efficient documentation. DISCUSSION We found substantial variation in efficiency across daily documentation patterns, suggesting that PCC-level factors like EHR facility and consistency may be more important than when documentation occurs. There were substantial efficiency returns to consistency, and deviations from consistent patterns were costly. CONCLUSION Organizational leaders aiming to reduce documentation burden should pay specific attention to the ability for PCCs to execute consistent documentation patterns day-to-day.
Collapse
Affiliation(s)
- Nate C Apathy
- Health Policy and Management, University of Maryland School of Public Health, College Park, MD 20742, United States
| | - Joshua Biro
- National Center for Human Factors in Healthcare, MedStar Health Research Institute, Hyattsville, MD 20782, United States
| | - A Jay Holmgren
- Division of Clinical Informatics and Digital Transformation, University of California—San Francisco School of Medicine, San Francisco, CA 94117, United States
| |
Collapse
|
8
|
Owens LM, Wilda JJ, Grifka R, Westendorp J, Fletcher JJ. Effect of Ambient Voice Technology, Natural Language Processing, and Artificial Intelligence on the Patient-Physician Relationship. Appl Clin Inform 2024; 15:660-667. [PMID: 38834180 PMCID: PMC11305826 DOI: 10.1055/a-2337-4739] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Accepted: 05/31/2024] [Indexed: 06/06/2024] Open
Abstract
BACKGROUND The method of documentation during a clinical encounter may affect the patient-physician relationship. OBJECTIVES Evaluate how the use of ambient voice recognition, coupled with natural language processing and artificial intelligence (DAX), affects the patient-physician relationship. METHODS This was a prospective observational study with a primary aim of evaluating any difference in patient satisfaction on the Patient-Doctor Relationship Questionnaire-9 (PDRQ-9) scale between primary care encounters in which DAX was utilized for documentation as compared to another method. A single-arm open-label phase was also performed to query direct feedback from patients. RESULTS A total of 288 patients were include in the open-label arm and 304 patients were included in the masked phase of the study comparing encounters with and without DAX use. In the open-label phase, patients strongly agreed that the provider was more focused on them, spent less time typing, and made the encounter feel more personable. In the masked phase of the study, no difference was seen in the total PDRQ-9 score between patients whose encounters used DAX (median: 45, interquartile range [IQR]: 8) and those who did not (median: 45 [IQR: 3.5]; p = 0.31). The adjusted odds ratio for DAX use was 0.8 (95% confidence interval: 0.48-1.34) for the patient reporting complete satisfaction on how well their clinician listened to them during their encounter. CONCLUSION Patients strongly agreed with the use of ambient voice recognition, coupled with natural language processing and artificial intelligence (DAX) for documentation in primary care. However, no difference was detected in the patient-physician relationship on the PDRQ-9 scale.
Collapse
Affiliation(s)
- Lance M. Owens
- Department of Family Medicine, University of Michigan Health-West, Wyoming, Michigan, United States
| | - J Joshua Wilda
- Health Information Technology, University of Michigan Health-West, Wyoming, Michigan, United States
| | - Ronald Grifka
- Department of Research, University of Michigan Health West, Wyoming, Michigan, United States
| | - Joan Westendorp
- Department of Research, University of Michigan Health West, Wyoming, Michigan, United States
| | - Jeffrey J. Fletcher
- Department of Research, University of Michigan Health West, Wyoming, Michigan, United States
| |
Collapse
|
9
|
Murad MH, Vaa Stelling BE, West CP, Hasan B, Simha S, Saadi S, Firwana M, Viola KE, Prokop LJ, Nayfeh T, Wang Z. Measuring Documentation Burden in Healthcare. J Gen Intern Med 2024:10.1007/s11606-024-08956-8. [PMID: 39073484 DOI: 10.1007/s11606-024-08956-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/23/2024] [Accepted: 07/17/2024] [Indexed: 07/30/2024]
Abstract
BACKGROUND The enactment of the Health Information Technology for Economic and Clinical Health Act and the wide adoption of electronic health record (EHR) systems have ushered in increasing documentation burden, frequently cited as a key factor affecting the work experience of healthcare professionals and a contributor to burnout. This systematic review aims to identify and characterize measures of documentation burden. METHODS We integrated discussions with Key Informants and a comprehensive search of the literature, including MEDLINE, Embase, Scopus, and gray literature published between 2010 and 2023. Data were narratively and thematically synthesized. RESULTS We identified 135 articles about measuring documentation burden. We classified measures into 11 categories: overall time spent in EHR, activities related to clinical documentation, inbox management, time spent in clinical review, time spent in orders, work outside work/after hours, administrative tasks (billing and insurance related), fragmentation of workflow, measures of efficiency, EHR activity rate, and usability. The most common source of data for most measures was EHR usage logs. Direct tracking such as through time-motion analysis was fairly uncommon. Measures were developed and applied across various settings and populations, with physicians and nurses in the USA being the most frequently represented healthcare professionals. Evidence of validity of these measures was limited and incomplete. Data on the appropriateness of measures in terms of scalability, feasibility, or equity across various contexts were limited. The physician perspective was the most robustly captured and prominently focused on increased stress and burnout. DISCUSSION Numerous measures for documentation burden are available and have been tested in a variety of settings and contexts. However, most are one-dimensional, do not capture various domains of this construct, and lack robust validity evidence. This report serves as a call to action highlighting an urgent need for measure development that represents diverse clinical contexts and support future interventions.
Collapse
Affiliation(s)
- M Hassan Murad
- Mayo Clinic Evidence-Based Practice Center, Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN, USA.
| | - Brianna E Vaa Stelling
- Division of Community Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA
| | - Colin P West
- Division of General Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, USA
| | - Bashar Hasan
- Mayo Clinic Evidence-Based Practice Center, Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN, USA
| | - Suvyaktha Simha
- Mayo Clinic Evidence-Based Practice Center, Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN, USA
| | - Samer Saadi
- Mayo Clinic Evidence-Based Practice Center, Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN, USA
| | - Mohammed Firwana
- Mayo Clinic Evidence-Based Practice Center, Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN, USA
| | - Kelly E Viola
- Mayo Clinic Evidence-Based Practice Center, Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN, USA
| | | | - Tarek Nayfeh
- Mayo Clinic Evidence-Based Practice Center, Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN, USA
| | - Zhen Wang
- Mayo Clinic Evidence-Based Practice Center, Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Healthcare Delivery, Mayo Clinic, Rochester, MN, USA
| |
Collapse
|
10
|
Ahn A, Morgan AU, Burke RE, Honig K, Long JA, McGlaughlin N, Jointer C, Asch DA, Bressman E. Postdischarge needs identified by an automated text messaging program: A mixed-methods study. J Hosp Med 2024. [PMID: 39051626 DOI: 10.1002/jhm.13466] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/10/2024] [Revised: 06/18/2024] [Accepted: 07/09/2024] [Indexed: 07/27/2024]
Abstract
BACKGROUND Text messaging has emerged as a popular strategy to engage patients after hospital discharge. Little is known about how patients use these programs and what types of needs are addressed through this approach. OBJECTIVE The goal of this study was to describe the types and timing of postdischarge needs identified during a 30-day automated texting program. METHODS The program ran from January to August 2021 at a primary care practice in Philadelphia. In this mixed-methods study, two reviewers conducted a directed content analysis of patient needs expressed during the program, categorizing them along a well-known transitional care framework. We describe the frequency of need categories and their timing relative to discharge. RESULTS A total of 405 individuals were enrolled; the mean (SD) age was 62.7 (16.2); 64.2% were female; 47.4% were Black; and 49.9% had Medicare insurance. Of this population, 178 (44.0%) expressed at least one need during the 30-day program. The most frequent needs addressed were related to symptoms (26.8%), coordinating follow-up care (20.4%), and medication issues (15.7%). The mean (SD) number of days from discharge to need was 10.8 (7.9); there were no significant differences in timing based on need category. CONCLUSIONS The needs identified via an automated texting program were concentrated in three areas relevant to primary care practice and within nursing scope of practice. This program can serve as a model for health systems looking to support transitions through an operationally efficient approach, and the findings of this analysis can inform future iterations of this type of program.
Collapse
Affiliation(s)
- Aiden Ahn
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Anna U Morgan
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Robert E Burke
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Corporal Michael J. Crescenz VA Medical Center, Philadelphia, Pennsylvania, USA
| | - Katherine Honig
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Judith A Long
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Corporal Michael J. Crescenz VA Medical Center, Philadelphia, Pennsylvania, USA
| | - Nancy McGlaughlin
- Penn Primary Care, University of Pennsylvania Health System, Philadelphia, Pennsylvania, USA
| | - Carlondra Jointer
- Penn Primary Care, University of Pennsylvania Health System, Philadelphia, Pennsylvania, USA
| | - David A Asch
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Eric Bressman
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Corporal Michael J. Crescenz VA Medical Center, Philadelphia, Pennsylvania, USA
| |
Collapse
|
11
|
Weiner M, Flanagan ME, Ernst K, Cottingham AH, Rattray NA, Franks Z, Savoy AW, Lee JL, Frankel RM. Accuracy, thoroughness, and quality of outpatient primary care documentation in the U.S. Department of Veterans Affairs. BMC PRIMARY CARE 2024; 25:262. [PMID: 39026167 PMCID: PMC11264844 DOI: 10.1186/s12875-024-02501-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/07/2024] [Accepted: 06/27/2024] [Indexed: 07/20/2024]
Abstract
BACKGROUND Electronic health records (EHRs) can accelerate documentation and may enhance details of notes, or complicate documentation and introduce errors. Comprehensive assessment of documentation quality requires comparing documentation to what transpires during the clinical encounter itself. We assessed outpatient primary care notes and corresponding recorded encounters to determine accuracy, thoroughness, and several additional key measures of documentation quality. METHODS Patients and primary care clinicians across five midwestern primary care clinics of the US Department of Veterans Affairs were recruited into a prospective observational study. Clinical encounters were video-recorded and transcribed verbatim. Using the Physician Documentation Quality Instrument (PDQI-9) added to other measures, reviewers scored quality of the documentation by comparing transcripts to corresponding encounter notes. PDQI-9 items were scored from 1 to 5, with higher scores indicating higher quality. RESULTS Encounters (N = 49) among 11 clinicians were analyzed. Most issues that patients initiated in discussion were omitted from notes, and nearly half of notes referred to information or observations that could not be verified. Four notes lacked concluding assessments and plans; nine lacked information about when patients should return. Except for thoroughness, PDQI-9 items that were assessed achieved quality scores exceeding 4 of 5 points. CONCLUSIONS Among outpatient primary care electronic records examined, most issues that patients initiated in discussion were absent from notes, and nearly half of notes referred to information or observations absent from transcripts. EHRs may contribute to certain kinds of errors. Approaches to improving documentation should consider the roles of the EHR, patient, and clinician together.
Collapse
Affiliation(s)
- Michael Weiner
- Center for Health Information and Communication, Department of Veterans Affairs, Health Services Research and Development Service, Veterans Health Administration, Richard L. Roudebush VA Medical Center, CIN 13 416, Indianapolis, IN, USA.
- Department of Medicine, Indiana University School of Medicine, Indianapolis, IN, USA.
- Indiana University Center for Health Services and Outcomes Research, Regenstrief Institute, Inc, 1101 West 10th Street, 46202-4800, Indianapolis, IN, USA.
| | - Mindy E Flanagan
- Center for Health Information and Communication, Department of Veterans Affairs, Health Services Research and Development Service, Veterans Health Administration, Richard L. Roudebush VA Medical Center, CIN 13 416, Indianapolis, IN, USA
| | - Katie Ernst
- Applied Decision Science LLC, Cincinnati, OH, USA
| | - Ann H Cottingham
- Indiana University Center for Health Services and Outcomes Research, Regenstrief Institute, Inc, 1101 West 10th Street, 46202-4800, Indianapolis, IN, USA
| | - Nicholas A Rattray
- Center for Health Information and Communication, Department of Veterans Affairs, Health Services Research and Development Service, Veterans Health Administration, Richard L. Roudebush VA Medical Center, CIN 13 416, Indianapolis, IN, USA
- Department of Medicine, Indiana University School of Medicine, Indianapolis, IN, USA
- Indiana University Center for Health Services and Outcomes Research, Regenstrief Institute, Inc, 1101 West 10th Street, 46202-4800, Indianapolis, IN, USA
| | - Zamal Franks
- Center for Health Information and Communication, Department of Veterans Affairs, Health Services Research and Development Service, Veterans Health Administration, Richard L. Roudebush VA Medical Center, CIN 13 416, Indianapolis, IN, USA
| | - April W Savoy
- Center for Health Information and Communication, Department of Veterans Affairs, Health Services Research and Development Service, Veterans Health Administration, Richard L. Roudebush VA Medical Center, CIN 13 416, Indianapolis, IN, USA
- Indiana University Center for Health Services and Outcomes Research, Regenstrief Institute, Inc, 1101 West 10th Street, 46202-4800, Indianapolis, IN, USA
- Purdue School of Engineering and Technology, Indiana University-Purdue University Indianapolis, Indianapolis, IN, USA
| | - Joy L Lee
- Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, MA, USA
| | - Richard M Frankel
- Center for Health Information and Communication, Department of Veterans Affairs, Health Services Research and Development Service, Veterans Health Administration, Richard L. Roudebush VA Medical Center, CIN 13 416, Indianapolis, IN, USA
- Department of Medicine, Indiana University School of Medicine, Indianapolis, IN, USA
- Indiana University Center for Health Services and Outcomes Research, Regenstrief Institute, Inc, 1101 West 10th Street, 46202-4800, Indianapolis, IN, USA
| |
Collapse
|
12
|
Streiffer C, Saini D, Whitehead G, Daniel J, Garzon-Mrad C, Kavanaugh L, Anyanwu E. The incremental design of a machine learning framework for medical records processing. J Am Med Inform Assoc 2024:ocae194. [PMID: 39018499 DOI: 10.1093/jamia/ocae194] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/22/2024] [Revised: 07/04/2024] [Accepted: 07/10/2024] [Indexed: 07/19/2024] Open
Abstract
OBJECTIVES This work presents the development and evaluation of coordn8, a web-based application that streamlines fax processing in outpatient clinics using a "human-in-the-loop" machine learning framework. We demonstrate the effectiveness of the platform at reducing fax processing time and producing accurate machine learning inferences across the tasks of patient identification, document classification, spam classification, and duplicate document detection. METHODS We deployed coordn8 in 11 outpatient clinics and conducted a time savings analysis by observing users and measuring fax processing event logs. We used statistical methods to evaluate the machine learning components across different datasets to show generalizability. We conducted a time series analysis to show variations in model performance as new clinics were onboarded and to demonstrate our approach to mitigating model drift. RESULTS Our observation analysis showed a mean reduction in individual fax processing time by 147.5 s, while our event log analysis of over 7000 faxes reinforced this finding. Document classification produced an accuracy of 81.6%, patient identification produced an accuracy of 83.7%, spam classification produced an accuracy of 98.4%, and duplicate document detection produced a precision of 81.0%. Retraining document classification increased accuracy by 10.2%. DISCUSSION coordn8 significantly decreased fax-processing time and produced accurate machine learning inferences. Our human-in-the-loop framework facilitated the collection of high-quality data necessary for model training. Expanding to new clinics correlated with performance decline, which was mitigated through model retraining. CONCLUSION Our framework for automating clinical tasks with machine learning offers a template for health systems looking to implement similar technologies.
Collapse
Affiliation(s)
- Christopher Streiffer
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Divya Saini
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Gideon Whitehead
- Center for Health Care Transformation and Innovation, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Jency Daniel
- Center for Health Care Transformation and Innovation, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Carolina Garzon-Mrad
- Center for Health Care Transformation and Innovation, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Laura Kavanaugh
- Center for Health Care Transformation and Innovation, University of Pennsylvania, Philadelphia, PA 19104, United States
| | - Emeka Anyanwu
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, United States
- Center for Health Care Transformation and Innovation, University of Pennsylvania, Philadelphia, PA 19104, United States
| |
Collapse
|
13
|
Kim S, Warner BC, Lew D, Lou SS, Kannampallil T. Measuring cognitive effort using tabular transformer-based language models of electronic health record-based audit log action sequences. J Am Med Inform Assoc 2024:ocae171. [PMID: 39001791 DOI: 10.1093/jamia/ocae171] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2024] [Accepted: 06/24/2024] [Indexed: 07/15/2024] Open
Abstract
OBJECTIVES To develop and validate a novel measure, action entropy, for assessing the cognitive effort associated with electronic health record (EHR)-based work activities. MATERIALS AND METHODS EHR-based audit logs of attending physicians and advanced practice providers (APPs) from four surgical intensive care units in 2019 were included. Neural language models (LMs) were trained and validated separately for attendings' and APPs' action sequences. Action entropy was calculated as the cross-entropy associated with the predicted probability of the next action, based on prior actions. To validate the measure, a matched pairs study was conducted to assess the difference in action entropy during known high cognitive effort scenarios, namely, attention switching between patients and to or from the EHR inbox. RESULTS Sixty-five clinicians performing 5 904 429 EHR-based audit log actions on 8956 unique patients were included. All attention switching scenarios were associated with a higher action entropy compared to non-switching scenarios (P < .001), except for the from-inbox switching scenario among APPs. The highest difference among attendings was for the from-inbox attention switching: Action entropy was 1.288 (95% CI, 1.256-1.320) standard deviations (SDs) higher for switching compared to non-switching scenarios. For APPs, the highest difference was for the to-inbox switching, where action entropy was 2.354 (95% CI, 2.311-2.397) SDs higher for switching compared to non-switching scenarios. DISCUSSION We developed a LM-based metric, action entropy, for assessing cognitive burden associated with EHR-based actions. The metric showed discriminant validity and statistical significance when evaluated against known situations of high cognitive effort (ie, attention switching). With additional validation, this metric can potentially be used as a screening tool for assessing behavioral action phenotypes that are associated with higher cognitive burden. CONCLUSION An LM-based action entropy metric-relying on sequences of EHR actions-offers opportunities for assessing cognitive effort in EHR-based workflows.
Collapse
Affiliation(s)
- Seunghwan Kim
- Roy and Diana Vagelos Division of Biology and Biomedical Sciences, Washington University in St. Louis, St. Louis, MO 63110, United States
- Institute for Informatics, Data Science and Biostatistics (I2DB), Washington University School of Medicine, St. Louis, MO 63110, United States
| | - Benjamin C Warner
- Department of Computer Science and Engineering, Washington University St. Louis, St. Louis, MO 63130-4899, United States
| | - Daphne Lew
- Institute for Informatics, Data Science and Biostatistics (I2DB), Washington University School of Medicine, St. Louis, MO 63110, United States
| | - Sunny S Lou
- Institute for Informatics, Data Science and Biostatistics (I2DB), Washington University School of Medicine, St. Louis, MO 63110, United States
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO 63110, United States
| | - Thomas Kannampallil
- Institute for Informatics, Data Science and Biostatistics (I2DB), Washington University School of Medicine, St. Louis, MO 63110, United States
- Department of Computer Science and Engineering, Washington University St. Louis, St. Louis, MO 63130-4899, United States
- Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO 63110, United States
| |
Collapse
|
14
|
Chan KKY, Yeung NCY, Mo PKH, Yang X. Common stressors, coping processes, and professional help-seeking of medical professionals in Hong Kong: A qualitative study. J Health Psychol 2024; 29:891-904. [PMID: 38160404 DOI: 10.1177/13591053231218658] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2024] Open
Abstract
Despite the high prevalence of perceived stress and mental health problems among medical professionals (MPs), their professional help-seeking is extremely low. This qualitative study explored MPs' stressors, stress-coping, barriers and facilitators of professional help-seeking. 10 MPs (30% male, Mage = 34.8 years) were recruited by purposive-sampling for views from different roles/settings. Thematic analyses revealed five central stressors: emerging novel diseases, challenges from technology-advancement, patient-communication difficulties, lack of workplace mental health care culture, excessive workload/manpower shortage. Participants predominantly used peer support/supervision and de-stress activities for stress-coping. Five factors affecting professional help-seeking were time constraint versus flexibility, mental health stigma versus de-stigmatization, concern over confidentiality/anonymity versus sense of privacy, worry about damage on professional role versus least work disruption, doubts of service providers versus perceived efficacy. All participants indicated a preference for online mental health service delivery. Results reflected unmet needs and service gaps from MPs' perspectives for the development of future interventions.
Collapse
Affiliation(s)
| | | | | | - Xue Yang
- The Chinese University of Hong Kong, Hong Kong
| |
Collapse
|
15
|
Holt G, Zabinski J. Amend Notes as the Next Frontier of the OpenNotes Initiative. J Gen Intern Med 2024:10.1007/s11606-024-08904-6. [PMID: 38943016 DOI: 10.1007/s11606-024-08904-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/07/2024] [Accepted: 06/14/2024] [Indexed: 06/30/2024]
|
16
|
Tu SP, Garcia B, Zhu X, Sewell D, Mishra V, Matin K, Dow A. Patient care in complex Sociotechnological ecosystems and learning health systems. Learn Health Syst 2024; 8:e10427. [PMID: 38883874 PMCID: PMC11176594 DOI: 10.1002/lrh2.10427] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 04/12/2024] [Accepted: 04/16/2024] [Indexed: 06/18/2024] Open
Abstract
The learning health system (LHS) model was proposed to provide real-time, bi-directional flow of learning using data captured in health information technology systems to deliver rapid learning in healthcare delivery. As highlighted by the landmark National Academy of Medicine report "Crossing the Quality Chasm," the U.S. healthcare delivery industry represents complex adaptive systems, and there is an urgent need to develop innovative methods to identify efficient team structures by harnessing real-world care delivery data found in the electronic health record (EHR). We offer a discussion surrounding the complexities of team communication and how solutions may be guided by theories such as the Multiteam System (MTS) framework and the Multitheoretical Multilevel Framework of Communication Networks. To advance healthcare delivery science and promote LHSs, our team has been building a new line of research using EHR data to study MTS in the complex real world of cancer care delivery. We are developing new network metrics to study MTSs and will be analyzing the impact of EHR communication network structures on patient outcomes. As this research leads to patient care delivery interventions/tools, healthcare leaders and healthcare professionals can effectively use health IT data to implement the most evidence-based collaboration approaches in order to achieve the optimal LHS and patient outcomes.
Collapse
Affiliation(s)
- Shin-Ping Tu
- Department of Internal Medicine University of California, Davis Sacramento California USA
| | - Brittany Garcia
- Department of Internal Medicine University of California, Davis Sacramento California USA
| | - Xi Zhu
- Department of Health Policy and Management University of California, Los Angeles Los Angeles California USA
| | - Daniel Sewell
- Department of Biostatistics University of Iowa Iowa City Iowa USA
| | - Vimal Mishra
- Department of Internal Medicine University of California, Davis Sacramento California USA
| | - Khalid Matin
- Department of Internal Medicine Virginia Commonwealth University Richmond Virginia USA
| | - Alan Dow
- Department of Internal Medicine Virginia Commonwealth University Richmond Virginia USA
| |
Collapse
|
17
|
Sulejmani P, Negris O, Aoki V, Chu CY, Eichenfield L, Misery L, Mosca A, Orfali RL, Aroman MS, Stalder JF, Trzeciak M, Wollenberg A, Lio P. A large language model artificial intelligence for patient queries in atopic dermatitis. J Eur Acad Dermatol Venereol 2024; 38:e531-e535. [PMID: 38168874 DOI: 10.1111/jdv.19737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Accepted: 11/27/2023] [Indexed: 01/05/2024]
Affiliation(s)
- Pranvera Sulejmani
- Rush Medical College, Rush University Medical Center, Chicago, Illinois, USA
| | - Olivia Negris
- Rush Medical College, Rush University Medical Center, Chicago, Illinois, USA
| | - Valeria Aoki
- Departamento of Dermatology, Faculdade de Medicina FMUSP, Universidade de São Paulo, São Paulo, Brazil
| | - Chia-Yu Chu
- Department of Dermatology, National Taiwan University Hospital and National Taiwan University College of Medicine, Taipei, Taiwan
| | - Lawrence Eichenfield
- Departments of Dermatology and Pediatrics, University of California, San Diego and Rady Children's Hospital-San Diego, San Diego, California, USA
| | - Laurent Misery
- Department of Dermatology, University Hospital of Brest, Brest, France
| | - Ana Mosca
- Hospital Municipal Jesus/Hospital Universitário, Pedro Ernesto/Universidade do Estado do Rio de Janeiro, Rio De Janeiro, Brazil
| | - Raquel Leão Orfali
- Departamento of Dermatology, Faculdade de Medicina FMUSP, Universidade de São Paulo, São Paulo, Brazil
| | - Markéta Saint Aroman
- Medical Direction Pharma, Dermocosmetics Care & Personal Care, Pierre Fabre, Toulouse, France
| | | | - Magdalena Trzeciak
- Department of Dermatology, Venereology and Allergology, Medical University of Gdansk, Gdansk, Poland
| | - Andreas Wollenberg
- Department of Dermatology and Allergy, University Hospital Augsburg, Augsburg, Germany
- Department of Dermatology and Allergy, Ludwig-Maximilian University of Munich, Munich, Germany
| | - Peter Lio
- Department of Dermatology, Feinberg School of Medicine, Northwestern University, Chicago, Illinois, USA
| |
Collapse
|
18
|
Liu S, McCoy AB, Wright AP, Carew B, Genkins JZ, Huang SS, Peterson JF, Steitz B, Wright A. Leveraging large language models for generating responses to patient messages-a subjective analysis. J Am Med Inform Assoc 2024; 31:1367-1379. [PMID: 38497958 PMCID: PMC11105129 DOI: 10.1093/jamia/ocae052] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Revised: 01/17/2024] [Accepted: 02/28/2024] [Indexed: 03/19/2024] Open
Abstract
OBJECTIVE This study aimed to develop and assess the performance of fine-tuned large language models for generating responses to patient messages sent via an electronic health record patient portal. MATERIALS AND METHODS Utilizing a dataset of messages and responses extracted from the patient portal at a large academic medical center, we developed a model (CLAIR-Short) based on a pre-trained large language model (LLaMA-65B). In addition, we used the OpenAI API to update physician responses from an open-source dataset into a format with informative paragraphs that offered patient education while emphasizing empathy and professionalism. By combining with this dataset, we further fine-tuned our model (CLAIR-Long). To evaluate fine-tuned models, we used 10 representative patient portal questions in primary care to generate responses. We asked primary care physicians to review generated responses from our models and ChatGPT and rated them for empathy, responsiveness, accuracy, and usefulness. RESULTS The dataset consisted of 499 794 pairs of patient messages and corresponding responses from the patient portal, with 5000 patient messages and ChatGPT-updated responses from an online platform. Four primary care physicians participated in the survey. CLAIR-Short exhibited the ability to generate concise responses similar to provider's responses. CLAIR-Long responses provided increased patient educational content compared to CLAIR-Short and were rated similarly to ChatGPT's responses, receiving positive evaluations for responsiveness, empathy, and accuracy, while receiving a neutral rating for usefulness. CONCLUSION This subjective analysis suggests that leveraging large language models to generate responses to patient messages demonstrates significant potential in facilitating communication between patients and healthcare providers.
Collapse
Affiliation(s)
- Siru Liu
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37212, United States
| | - Allison B McCoy
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37212, United States
| | - Aileen P Wright
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37212, United States
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN 37212, United States
| | - Babatunde Carew
- Department of General Internal Medicine and Public Health, Vanderbilt University Medical Center, Nashville, TN 37212, United States
| | - Julian Z Genkins
- Department of Medicine, Stanford University, Stanford, CA 94304, United States
| | - Sean S Huang
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37212, United States
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN 37212, United States
| | - Josh F Peterson
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37212, United States
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN 37212, United States
| | - Bryan Steitz
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37212, United States
| | - Adam Wright
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN 37212, United States
| |
Collapse
|
19
|
Couser GP, Hagen PT, Swift MD, Newcomb RD, Cowl CT. Physician Health Series Part One: Characteristics of Physicians as Patients. Mayo Clin Proc 2024; 99:836-843. [PMID: 38702130 DOI: 10.1016/j.mayocp.2023.08.027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/27/2022] [Revised: 04/25/2023] [Accepted: 08/30/2023] [Indexed: 05/06/2024]
Abstract
This is the first article of a 3-part series about physician health. In this installment, we outline the unique characteristics of physicians as patients, challenges and opportunities presented by physician-patients, and recommendations for treating physicians. Future articles will delve into role clarity, occupational considerations, mental health, and interactions with third parties such as the physician's employer or licensing board. Ultimately, this series will help treating clinicians provide the best care to their physician-patients and successfully navigate the unique challenges that may arise, especially when the diagnosis may have an impact on their ability to practice medicine.
Collapse
Affiliation(s)
- Greg P Couser
- Mayo Clinic Physician Health Center, Public Health, Infectious Disease, and Occupational Medicine, Mayo Clinic, Rochester, MN.
| | - Philip T Hagen
- Mayo Clinic Physician Health Center, Public Health, Infectious Disease, and Occupational Medicine, Mayo Clinic, Rochester, MN
| | - Melanie D Swift
- Mayo Clinic Physician Health Center, Public Health, Infectious Disease, and Occupational Medicine, Mayo Clinic, Rochester, MN
| | - Richard D Newcomb
- Mayo Clinic Physician Health Center, Public Health, Infectious Disease, and Occupational Medicine, Mayo Clinic, Rochester, MN
| | - Clayton T Cowl
- Mayo Clinic Physician Health Center, Public Health, Infectious Disease, and Occupational Medicine, Mayo Clinic, Rochester, MN
| |
Collapse
|
20
|
Shumer G, Bhandiwad A, Holkeboer J, Marshall L. Team-Based Management of High-Priority In-Basket Messages. Ann Fam Med 2024; 22:260. [PMID: 38806263 PMCID: PMC11237223 DOI: 10.1370/afm.3094] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Revised: 12/07/2023] [Accepted: 12/19/2023] [Indexed: 05/30/2024] Open
Affiliation(s)
- Gregory Shumer
- University of Michigan, Department of Family Medicine, Ann Arbor, Michigan
| | - Anup Bhandiwad
- University of Michigan, Department of Family Medicine, Ann Arbor, Michigan
| | - John Holkeboer
- University of Michigan, Department of Family Medicine, Ann Arbor, Michigan
| | - Lauren Marshall
- University of Michigan, Department of Family Medicine, Ann Arbor, Michigan
| |
Collapse
|
21
|
Lotter W, Hassett MJ, Schultz N, Kehl KL, Van Allen EM, Cerami E. Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions. Cancer Discov 2024; 14:711-726. [PMID: 38597966 PMCID: PMC11131133 DOI: 10.1158/2159-8290.cd-23-1199] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 01/29/2024] [Accepted: 02/28/2024] [Indexed: 04/11/2024]
Abstract
Artificial intelligence (AI) in oncology is advancing beyond algorithm development to integration into clinical practice. This review describes the current state of the field, with a specific focus on clinical integration. AI applications are structured according to cancer type and clinical domain, focusing on the four most common cancers and tasks of detection, diagnosis, and treatment. These applications encompass various data modalities, including imaging, genomics, and medical records. We conclude with a summary of existing challenges, evolving solutions, and potential future directions for the field. SIGNIFICANCE AI is increasingly being applied to all aspects of oncology, where several applications are maturing beyond research and development to direct clinical integration. This review summarizes the current state of the field through the lens of clinical translation along the clinical care continuum. Emerging areas are also highlighted, along with common challenges, evolving solutions, and potential future directions for the field.
Collapse
Affiliation(s)
- William Lotter
- Department of Data Science, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Pathology, Brigham and Women’s Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Michael J. Hassett
- Harvard Medical School, Boston, MA, USA
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Nikolaus Schultz
- Marie-Josée and Henry R. Kravis Center for Molecular Oncology, Memorial Sloan Kettering Cancer Center; New York, NY, USA
- Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Kenneth L. Kehl
- Harvard Medical School, Boston, MA, USA
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Eliezer M. Van Allen
- Harvard Medical School, Boston, MA, USA
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
- Cancer Program, Broad Institute of MIT and Harvard, Cambridge, MA, USA
| | - Ethan Cerami
- Department of Data Science, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| |
Collapse
|
22
|
Chiodo CP, Striano BM, Parker E, Smith JT, Bluman EM, Martin EA, Greco JM, Healey MJ. Primary Care Physician Preferences Regarding Communication from Orthopaedic Surgeons. J Bone Joint Surg Am 2024; 106:760-766. [PMID: 38386720 DOI: 10.2106/jbjs.23.00836] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/24/2024]
Abstract
BACKGROUND Musculoskeletal consultations constitute a growing portion of primary care physician (PCP) referrals. Optimizing communication between PCPs and orthopaedists can potentially reduce time spent in the electronic medical record (EMR) as well as physician burnout. Little is known about the preferences of PCPs regarding communication from orthopaedic surgeons. Hence, the present study investigated, across a large health network, the preferences of PCPs regarding communication from orthopaedists. METHODS A total of 175 PCPs across 15 practices within our health network were surveyed. These providers universally utilized Epic as their EMR platform. Five-point, labeled Likert scales were utilized to assess the PCP-perceived importance of communication from orthopaedists in specific clinical scenarios. PCPs were further asked to report their preferred method of communication in each scenario and their overall interest in communication from orthopaedists. Logistic regression analyses were performed to determine whether any PCP characteristics were associated with the preferred method of communication and the overall PCP interest in communication from orthopaedists. RESULTS A total of 107 PCPs (61.1%) responded to the survey. PCPs most commonly rated communication from orthopaedists as highly important in the scenario of an orthopaedist needing information from the PCP. In this scenario, PCPs preferred to receive an Epic Staff Message. Scenarios involving a recommendation for surgery, hospitalization, or a major clinical change were also rated as highly important. In these scenarios, an Epic CC'd Chart rather than a Staff Message was preferred. Increased after-hours EMR use was associated with diminished odds of having a high interest in communication from orthopaedists (odds ratio, 0.65; 95% confidence interval, 0.48 to 0.88; p = 0.005). Ninety-three PCPs (86.9%) reported spending 1 to 1.5 hours or more per day in Epic after normal clinical hours, and 27 (25.2%) spent >3 hours per day. Forty-six PCPs (43.0%) reported experiencing ≥1 symptom of burnout. CONCLUSIONS There were distinct preferences among PCPs regarding clinical communication from orthopaedic surgeons. There was also evidence of substantial burnout and after-hours work effort by PCPs. These results may help to optimize communication between PCPs and orthopaedists while reducing the amount of time that PCPs spend in the EMR.
Collapse
Affiliation(s)
- Christopher P Chiodo
- Foot and Ankle Division, Department of Orthopaedic Surgery, Brigham and Women's Hospital, Boston, Massachusetts
| | - Brendan M Striano
- Harvard Combined Orthopaedic Residency Program, Boston, Massachusetts
| | - Emily Parker
- Department of Orthopaedic Surgery, Brigham and Women's Hospital, Boston, Massachusetts
| | - Jeremy T Smith
- Department of Orthopaedic Surgery, Brigham and Women's Hospital, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| | - Eric M Bluman
- Department of Orthopaedic Surgery, Brigham and Women's Hospital, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| | - Elizabeth A Martin
- Department of Orthopaedic Surgery, Brigham and Women's Hospital, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| | - Julia M Greco
- Department of Orthopaedic Surgery, Brigham and Women's Hospital, Boston, Massachusetts
| | - Michael J Healey
- Harvard Medical School, Boston, Massachusetts
- Department of Internal Medicine, Brigham and Women's Hospital, Boston, Massachusetts
| |
Collapse
|
23
|
Shachak A, Buchanan F, Kuziemsky C. When rules turn into tools: An activity theory-based perspective on implementation processes and unintended consequences. Healthc Manage Forum 2024; 37:177-182. [PMID: 38377181 PMCID: PMC11044511 DOI: 10.1177/08404704241233169] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/22/2024]
Abstract
The idea that actions of people, organizations or governments may lead to Unintended Consequences (UICs) is not new. In health, UICs have been reported as a result of various interventions including quality improvement initiatives, health information technology implementation, and knowledge translation, especially those involving translation of broad policies (evidence-based medicine and patient-centred care) or system level improvement into actionable items or tools. While some unintended consequences cannot be anticipated, others may be predictable. In this article, we present a model based on cultural historical activity theory, which may help policy-makers, health leaders, and researchers better anticipate UICs resulting from implementation of new programs or technologies and take action to address them or mitigate their risk of occurrence. We support this model using examples of UICs of implementing family centred care principles, electronic health records, and computerized templates for quality improvement in chronic disease management.
Collapse
Affiliation(s)
- Aviv Shachak
- University of Toronto, Toronto, Ontario, Canada
- The Wilson Centre for Research in Education, Toronto, Ontario, Canada
| | - Francine Buchanan
- University of Toronto, Toronto, Ontario, Canada
- The Hospital for Sick Children (SickKids), Toronto, Ontario, Canada
| | | |
Collapse
|
24
|
Rotenstein L, Melnick ER, Iannaccone C, Zhang J, Mugal A, Lipsitz SR, Healey MJ, Holland C, Snyder R, Sinsky CA, Ting D, Bates DW. Virtual Scribes and Physician Time Spent on Electronic Health Records. JAMA Netw Open 2024; 7:e2413140. [PMID: 38787556 PMCID: PMC11127114 DOI: 10.1001/jamanetworkopen.2024.13140] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/16/2024] [Accepted: 03/18/2024] [Indexed: 05/25/2024] Open
Abstract
Importance Time on the electronic health record (EHR) is associated with burnout among physicians. Newer virtual scribe models, which enable support from either a real-time or asynchronous scribe, have the potential to reduce the burden of the EHR and EHR-related documentation. Objective To characterize the association of use of virtual scribes with changes in physicians' EHR time and note and order composition and to identify the physician, scribe, and scribe response factors associated with changes in EHR time upon virtual scribe use. Design, Setting, and Participants Retrospective, pre-post quality improvement study of 144 physicians across specialties who had used a scribe for at least 3 months from January 2020 to September 2022, were affiliated with Brigham and Women's Hospital and Massachusetts General Hospital, and cared for patients in the outpatient setting. Data were analyzed from November 2022 to January 2024. Exposure Use of either a real-time or asynchronous virtual scribe. Main Outcomes Total EHR time, time on notes, and pajama time (5:30 pm to 7:00 am on weekdays and nonscheduled weekends and holidays), all per appointment; proportion of the note written by the physician and team contribution to orders. Results The main study sample included 144 unique physicians who had used a virtual scribe for at least 3 months in 152 unique scribe participation episodes (134 [88.2%] had used an asynchronous scribe service). Nearly two-thirds of the physicians (91 physicians [63.2%]) were female and more than half (86 physicians [59.7%]) were in primary care specialties. Use of a virtual scribe was associated with significant decreases in total EHR time per appointment (mean [SD] of 5.6 [16.4] minutes; P < .001) in the 3 months after vs the 3 months prior to scribe use. Scribe use was also associated with significant decreases in note time per appointment and pajama time per appointment (mean [SD] of 1.3 [3.3] minutes; P < .001 and 1.1 [4.0] minutes; P = .004). In a multivariable linear regression model, the following factors were associated with significant decreases in total EHR time per appointment with a scribe use at 3 months: practicing in a medical specialty (-7.8; 95% CI, -13.4 to -2.2 minutes), greater baseline EHR time per appointment (-0.3; 95% CI, -0.4 to -0.2 minutes per additional minute of baseline EHR time), and decrease in the percentage of the note contributed by the physician (-9.1; 95% CI, -17.3 to -0.8 minutes for every percentage point decrease). Conclusions and Relevance In 2 academic medical centers, use of virtual scribes was associated with significant decreases in total EHR time, time spent on notes, and pajama time, all per appointment. Virtual scribes may be particularly effective among medical specialists and those physicians with greater baseline EHR time.
Collapse
Affiliation(s)
- Lisa Rotenstein
- Harvard Medical School, Boston, Massachusetts
- Brigham and Women’s Hospital, Boston, Massachusetts
- University of California at San Francisco
| | - Edward R. Melnick
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut
- Department of Biostatistics (Health Informatics), Yale School of Public Health, New Haven, Connecticut
| | | | - Jianyi Zhang
- Brigham and Women’s Hospital, Boston, Massachusetts
| | - Aqsa Mugal
- Brigham and Women’s Hospital, Boston, Massachusetts
| | - Stuart R. Lipsitz
- Harvard Medical School, Boston, Massachusetts
- Brigham and Women’s Hospital, Boston, Massachusetts
| | - Michael J. Healey
- Harvard Medical School, Boston, Massachusetts
- Brigham and Women’s Hospital, Boston, Massachusetts
| | | | | | | | - David Ting
- Harvard Medical School, Boston, Massachusetts
- Mass General Brigham, Boston, Massachusetts
- Massachusetts General Hospital, Boston
| | - David W. Bates
- Harvard Medical School, Boston, Massachusetts
- Brigham and Women’s Hospital, Boston, Massachusetts
- Harvard School of Public Health, Boston, Massachusetts
| |
Collapse
|
25
|
Chedid M, Chebib FT, Dahlen E, Mueller T, Schnell T, Gay M, Hommos M, Swaminathan S, Garg A, Mao M, Amberg B, Balderes K, Johnson KF, Bishop A, Vaughn JK, Hogan M, Torres V, Chaudhry R, Zoghby Z. An Electronic Health Record-Integrated Application for Standardizing Care and Monitoring Patients With Autosomal Dominant Polycystic Kidney Disease Enrolled in a Tolvaptan Clinic: Design and Implementation Study. JMIR Med Inform 2024; 12:e50164. [PMID: 38717378 PMCID: PMC11085039 DOI: 10.2196/50164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Revised: 03/06/2024] [Accepted: 03/25/2024] [Indexed: 05/12/2024] Open
Abstract
Background Tolvaptan is the only US Food and Drug Administration-approved drug to slow the progression of autosomal dominant polycystic kidney disease (ADPKD), but it requires strict clinical monitoring due to potential serious adverse events. Objective We aimed to share our experience in developing and implementing an electronic health record (EHR)-based application to monitor patients with ADPKD who were initiated on tolvaptan. Methods The application was developed in collaboration with clinical informatics professionals based on our clinical protocol with frequent laboratory test monitoring to detect early drug-related toxicity. The application streamlined the clinical workflow and enabled our nursing team to take appropriate actions in real time to prevent drug-related serious adverse events. We retrospectively analyzed the characteristics of the enrolled patients. Results As of September 2022, a total of 214 patients were enrolled in the tolvaptan program across all Mayo Clinic sites. Of these, 126 were enrolled in the Tolvaptan Monitoring Registry application and 88 in the Past Tolvaptan Patients application. The mean age at enrollment was 43.1 (SD 9.9) years. A total of 20 (9.3%) patients developed liver toxicity, but only 5 (2.3%) had to discontinue the drug. The 2 EHR-based applications allowed consolidation of all necessary patient information and real-time data management at the individual or population level. This approach facilitated efficient staff workflow, monitoring of drug-related adverse events, and timely prescription renewal. Conclusions Our study highlights the feasibility of integrating digital applications into the EHR workflow to facilitate efficient and safe care delivery for patients enrolled in a tolvaptan program. This workflow needs further validation but could be extended to other health care systems managing chronic diseases requiring drug monitoring.
Collapse
Affiliation(s)
| | - Fouad T Chebib
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Jacksonville, FL, United States
| | - Erin Dahlen
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, MN, United States
| | - Theodore Mueller
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, MN, United States
| | - Theresa Schnell
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, MN, United States
| | - Melissa Gay
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, MN, United States
| | - Musab Hommos
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Scottsdale, AZ, United States
| | - Sundararaman Swaminathan
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Scottsdale, AZ, United States
| | - Arvind Garg
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, LaCrosse, WI, United States
| | - Michael Mao
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Jacksonville, FL, United States
| | - Brigid Amberg
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, MN, United States
| | - Kirk Balderes
- Division of Information Technology, Mayo Clinic, Rochester, MN, United States
| | - Karen F Johnson
- Division of Information Technology, Mayo Clinic, Rochester, MN, United States
| | - Alyssa Bishop
- Division of Information Technology, Mayo Clinic, Rochester, MN, United States
| | | | - Marie Hogan
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, MN, United States
| | - Vicente Torres
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, MN, United States
| | - Rajeev Chaudhry
- Division of Community Internal Medicine, Department of Medicine, Mayo Clinic, Rochester, MN, United States
| | - Ziad Zoghby
- Division of Nephrology and Hypertension, Department of Medicine, Mayo Clinic, Rochester, MN, United States
| |
Collapse
|
26
|
Avdagovska M, Kuziemsky C, Koosha H, Hadizadeh M, Pauly RP, Graham T, Stafinski T, Bigam D, Kassam N, Menon D. Exploring the Impact of In Basket Metrics on the Adoption of a New Electronic Health Record System Among Specialists in a Tertiary Hospital in Alberta: Descriptive Study. J Med Internet Res 2024; 26:e53122. [PMID: 38684079 DOI: 10.2196/53122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2023] [Revised: 03/12/2024] [Accepted: 03/19/2024] [Indexed: 05/02/2024] Open
Abstract
BACKGROUND Health care organizations implement electronic health record (EHR) systems with the expectation of improved patient care and enhanced provider performance. However, while these technologies hold the potential to create improved care and system efficiencies, they can also lead to unintended negative consequences, such as patient safety issues, communication problems, and provider burnout. OBJECTIVE This study aims to document metrics related to the In Basket communication hub (time in In Basket per day, time in In Basket per appointment, In Basket messages received per day, and turnaround time) of the EHR system implemented by Alberta Health Services, the province-wide health delivery system called Connect Care (Epic Systems). The objective was to identify how a newly implemented EHR system was used, the timing of its use, and the duration of use specifically related to In Basket activities. METHODS A descriptive study was conducted. Due to the diversity of specialties, the providers were grouped into medical and surgical based on previous similar studies. The participants were further subgrouped based on their self-reported clinical full-time equivalent (FTE ) measure. This resulted in 3 subgroups for analysis: medical FTE <0.5, medical FTE >0.5, and surgical (all of whom reported FTE >0.5). The analysis was limited to outpatient clinical interactions and explicitly excluded inpatient activities. RESULTS A total of 72 participants from 19 different specialties enrolled in this study. The providers had, on average, 8.31 appointments per day during the reporting periods. The providers received, on average, 21.93 messages per day, and they spent 7.61 minutes on average in the time in In Basket per day metric and 1.84 minutes on average in the time in In Basket per appointment metric. The time for the providers to mark messages as done (turnaround time) was on average 11.45 days during the reporting period. Although the surgical group had, on average, approximately twice as many appointments per scheduled day, they spent considerably less connected time (based on almost all time metrics) than the medical group. However, the surgical group took much longer than the medical group to mark messages as done (turnaround time). CONCLUSIONS We observed a range of patterns with no consistent direction. There does not seem to be evidence of a "learning curve," which would have shown a consistent reduction in time spent on the system over time due to familiarity and experience. While this study does not show how the included metrics could be used as predictors of providers' satisfaction or feelings of burnout, the use trends could be used to start discussions about future Canadian studies needed in this area.
Collapse
Affiliation(s)
- Melita Avdagovska
- School of Public Health, University of Alberta, Edmonton, AB, Canada
| | - Craig Kuziemsky
- Office of Research Services and School of Business, MacEwan University, Edmonton, AB, Canada
| | - Helia Koosha
- School of Public Health, University of Alberta, Edmonton, AB, Canada
| | - Maliheh Hadizadeh
- School of Public Health, University of Alberta, Edmonton, AB, Canada
| | - Robert P Pauly
- Medicine Department, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, AB, Canada
| | - Timothy Graham
- Department of Emergency Medicine, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, AB, Canada
| | - Tania Stafinski
- School of Public Health, University of Alberta, Edmonton, AB, Canada
| | - David Bigam
- Surgery Department, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, AB, Canada
| | - Narmin Kassam
- Medicine Department, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, AB, Canada
| | - Devidas Menon
- School of Public Health, University of Alberta, Edmonton, AB, Canada
| |
Collapse
|
27
|
Tremoulet PD, Lobo AF, Simmons CA, Baliga G, Brady M. Assessing the Usability and Feasibility of Digital Assistant Tools for Direct Support Professionals: Participatory Design and Pilot-Testing. JMIR Hum Factors 2024; 11:e51612. [PMID: 38662420 PMCID: PMC11082739 DOI: 10.2196/51612] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2023] [Revised: 03/07/2024] [Accepted: 03/16/2024] [Indexed: 04/26/2024] Open
Abstract
BACKGROUND The United States is experiencing a direct support professional (DSP) crisis, with demand far exceeding supply. Although generating documentation is a critical responsibility, it is one of the most wearisome aspects of DSPs' jobs. Technology that enables DSPs to log informal time-stamped notes throughout their shift could help reduce the burden of end-of-shift documentation and increase job satisfaction, which in turn could improve the quality of life of the individuals with intellectual and developmental disabilities (IDDs) whom DSPs support. However, DSPs, with varied ages, levels of education, and comfort using technology, are not likely to adopt tools that detract from caregiving responsibilities or increase workload; therefore, technological tools for them must be relatively simple, extremely intuitive, and provide highly valued capabilities. OBJECTIVE This paper describes the development and pilot-testing of a digital assistant tool (DAT) that enables DSPs to create informal notes throughout their shifts and use these notes to facilitate end-of-shift documentation. The purpose of the pilot study was to assess the usability and feasibility of the DAT. METHODS The research team applied an established user-centered participatory design process to design, develop, and test the DAT prototypes between May 2020 and April 2023. Pilot-testing entailed having 14 DSPs who support adults with IDDs use the first full implementation of the DAT prototypes during 2 or 3 successive work shifts and fill out demographic and usability questionnaires. RESULTS Participants used the DAT prototypes to create notes and help generate end-of-shift reports. The System Usability Scale score of 81.79 indicates that they found the prototypes easy to use. Survey responses imply that using the DAT made it easier for participants to produce required documentation and suggest that they would adopt the DAT if this tool were available for daily use. CONCLUSIONS Simple technologies such as the DAT prototypes, which enable DSPs to use mobile devices to log time-stamped notes throughout their shift with minimal effort and use the notes to help write reports, have the potential to both reduce the burden associated with producing documentation and enhance the quality (level of detail and accuracy) of this documentation. This could help to increase job satisfaction and reduce turnover in DSPs, both of which would help improve the quality of life of the individuals with IDDs whom they support. The pilot test results indicate that DSPs found the DAT easy to use. Next steps include (1) producing more robust versions of the DAT with additional capabilities, such as storing data locally on mobile devices when Wi-Fi is not available; and (2) eliciting input from agency directors, families, and others who use data about adults with IDDs to help care for them to ensure that data produced by DSPs are relevant and useful.
Collapse
Affiliation(s)
| | - Andrea F Lobo
- Department of Computer Science, Rowan University, Glassboro, NJ, United States
| | | | - Ganesh Baliga
- Department of Computer Science, Rowan University, Glassboro, NJ, United States
| | - Matthew Brady
- Department of Computer Science, Rowan University, Glassboro, NJ, United States
| |
Collapse
|
28
|
Owens LM, Wilda JJ, Hahn PY, Koehler T, Fletcher JJ. The association between use of ambient voice technology documentation during primary care patient encounters, documentation burden, and provider burnout. Fam Pract 2024; 41:86-91. [PMID: 37672297 DOI: 10.1093/fampra/cmad092] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 09/07/2023] Open
Abstract
BACKGROUND The burden of documentation in the electronic medical record has been cited as a major factor in provider burnout. The aim of this study was to evaluate the association between ambient voice technology, coupled with natural language processing and artificial intelligence (DAX™), on primary care provider documentation burden and burnout. METHODS An observational study of 110 primary care providers within a community teaching health system. The primary objectives were to determine the association between DAX™ usage and provider burnout scores on the Oldenburg Burnout Inventory (OLBI) as well as the effect on documentation time per patient encounter (minutes). RESULTS The completion rate for the survey was 75% (83/110) and high DAX™ use (>60% of encounters) was seen in 28% of providers (23/83). High DAX™ use was associated with significantly less burnout on the OLBI disengagement sub-score (MD [Mean Difference] -2.1; 95% confidence interval [CI] -3.8 to -0.4) but not the OLBI disengagement sub-score (-1.0; 95% CI -2.9 to 1.0) or total score (MD -3.0; 95% CI -6.4 to 0.3). Nineteen providers with high implementation of DAX™ had pre and postimplementation data on documentation time per encounter. After DAX™ implementation average documentation time in notes per encounter was significantly reduced by 28.8% (1.8 min; 95% CI 1.4-2.2). CONCLUSIONS The use of ambient voice technology during patient encounters was associated with significantly reduced documentation burden and primary care provider disengagement but not total provider burnout scores.
Collapse
Affiliation(s)
- Lance M Owens
- Department of Primary Care, University of Michigan Health West, Grand Rapids, MI, United States
| | - Joshua J Wilda
- Information Technology, University of Michigan Health West, Grand Rapids, MI, United States
| | - Peter Y Hahn
- Department of Clinical Research, University of Michigan Health West, Grand Rapids, MI, United States
| | - Tracy Koehler
- Department of Clinical Research, University of Michigan Health West, Grand Rapids, MI, United States
| | - Jeffrey J Fletcher
- Department of Clinical Research, University of Michigan Health West, Grand Rapids, MI, United States
| |
Collapse
|
29
|
Asgari E, Kaur J, Nuredini G, Balloch J, Taylor AM, Sebire N, Robinson R, Peters C, Sridharan S, Pimenta D. Impact of Electronic Health Record Use on Cognitive Load and Burnout Among Clinicians: Narrative Review. JMIR Med Inform 2024; 12:e55499. [PMID: 38607672 PMCID: PMC11053390 DOI: 10.2196/55499] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Revised: 02/15/2024] [Accepted: 03/11/2024] [Indexed: 04/13/2024] Open
Abstract
The cognitive load theory suggests that completing a task relies on the interplay between sensory input, working memory, and long-term memory. Cognitive overload occurs when the working memory's limited capacity is exceeded due to excessive information processing. In health care, clinicians face increasing cognitive load as the complexity of patient care has risen, leading to potential burnout. Electronic health records (EHRs) have become a common feature in modern health care, offering improved access to data and the ability to provide better patient care. They have been added to the electronic ecosystem alongside emails and other resources, such as guidelines and literature searches. Concerns have arisen in recent years that despite many benefits, the use of EHRs may lead to cognitive overload, which can impact the performance and well-being of clinicians. We aimed to review the impact of EHR use on cognitive load and how it correlates with physician burnout. Additionally, we wanted to identify potential strategies recommended in the literature that could be implemented to decrease the cognitive burden associated with the use of EHRs, with the goal of reducing clinician burnout. Using a comprehensive literature review on the topic, we have explored the link between EHR use, cognitive load, and burnout among health care professionals. We have also noted key factors that can help reduce EHR-related cognitive load, which may help reduce clinician burnout. The research findings suggest that inadequate efforts to present large amounts of clinical data to users in a manner that allows the user to control the cognitive burden in the EHR and the complexity of the user interfaces, thus adding more "work" to tasks, can lead to cognitive overload and burnout; this calls for strategies to mitigate these effects. Several factors, such as the presentation of information in the EHR, the specialty, the health care setting, and the time spent completing documentation and navigating systems, can contribute to this excess cognitive load and result in burnout. Potential strategies to mitigate this might include improving user interfaces, streamlining information, and reducing documentation burden requirements for clinicians. New technologies may facilitate these strategies. The review highlights the importance of addressing cognitive overload as one of the unintended consequences of EHR adoption and potential strategies for mitigation, identifying gaps in the current literature that require further exploration.
Collapse
Affiliation(s)
- Elham Asgari
- Guy's and St Thomas' NHS Trust, London, United Kingdom
- Tortus AI, London, United Kingdom
| | - Japsimar Kaur
- Manchester University NHS Foundation Trust, Manchester, United Kingdom
| | | | | | | | - Neil Sebire
- Great Ormond Street Hospital, London, United Kingdom
| | | | | | | | | |
Collapse
|
30
|
Tawfik D, Bayati M, Liu J, Nguyen L, Sinha A, Kannampallil T, Shanafelt T, Profit J. Predicting Primary Care Physician Burnout From Electronic Health Record Use Measures. Mayo Clin Proc 2024:S0025-6196(24)00037-5. [PMID: 38573301 DOI: 10.1016/j.mayocp.2024.01.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Accepted: 01/08/2024] [Indexed: 04/05/2024]
Abstract
OBJECTIVE To evaluate the ability of routinely collected electronic health record (EHR) use measures to predict clinical work units at increased risk of burnout and potentially most in need of targeted interventions. METHODS In this observational study of primary care physicians, we compiled clinical workload and EHR efficiency measures, then linked these measures to 2 years of well-being surveys (using the Stanford Professional Fulfillment Index) conducted from April 1, 2019, through October 16, 2020. Physicians were grouped into training and confirmation data sets to develop predictive models for burnout. We used gradient boosting classifier and other prediction modeling algorithms to quantify the predictive performance by the area under the receiver operating characteristics curve (AUC). RESULTS Of 278 invited physicians from across 60 clinics, 233 (84%) completed 396 surveys. Physicians were 67% women with a median age category of 45 to 49 years. Aggregate burnout score was in the high range (≥3.325/10) on 111 of 396 (28%) surveys. Gradient boosting classifier of EHR use measures to predict burnout achieved an AUC of 0.59 (95% CI, 0.48 to 0.77) and an area under the precision-recall curve of 0.29 (95% CI, 0.20 to 0.66). Other models' confirmation set AUCs ranged from 0.56 (random forest) to 0.66 (penalized linear regression followed by dichotomization). Among the most predictive features were physician age, team member contributions to notes, and orders placed with user-defined preferences. Clinic-level aggregate measures identified the top quartile of clinics with 56% sensitivity and 85% specificity. CONCLUSION In a sample of primary care physicians, routinely collected EHR use measures demonstrated limited ability to predict individual burnout and moderate ability to identify high-risk clinics.
Collapse
Affiliation(s)
- Daniel Tawfik
- Stanford University School of Medicine, Stanford, CA.
| | | | - Jessica Liu
- Stanford University School of Medicine, Stanford, CA
| | - Liem Nguyen
- Stanford University School of Engineering, Stanford, CA
| | | | | | - Tait Shanafelt
- Stanford University School of Medicine, Stanford, CA; Stanford Medicine WellMD & WellPhD Center, Stanford, CA
| | - Jochen Profit
- Stanford University School of Medicine, Stanford, CA
| |
Collapse
|
31
|
Seng EC, Mehdipour S, Simpson S, Gabriel RA. Tracking persistent postoperative opioid use: a proof-of-concept study demonstrating a use case for natural language processing. Reg Anesth Pain Med 2024; 49:241-247. [PMID: 37419509 DOI: 10.1136/rapm-2023-104629] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Accepted: 06/24/2023] [Indexed: 07/09/2023]
Abstract
BACKGROUND Large language models have been gaining tremendous popularity since the introduction of ChatGPT in late 2022. Perioperative pain providers should leverage natural language processing (NLP) technology and explore pertinent use cases to improve patient care. One example is tracking persistent postoperative opioid use after surgery. Since much of the relevant data may be 'hidden' within unstructured clinical text, NLP models may prove to be advantageous. The primary objective of this proof-of-concept study was to demonstrate the ability of an NLP engine to review clinical notes and accurately identify patients who had persistent postoperative opioid use after major spine surgery. METHODS Clinical documents from all patients that underwent major spine surgery during July 2015-August 2021 were extracted from the electronic health record. The primary outcome was persistent postoperative opioid use, defined as continued use of opioids greater than or equal to 3 months after surgery. This outcome was ascertained via manual clinician review from outpatient spine surgery follow-up notes. An NLP engine was applied to these notes to ascertain the presence of persistent opioid use-this was then compared with results from clinician manual review. RESULTS The final study sample consisted of 965 patients, in which 705 (73.1%) were determined to have persistent opioid use following surgery. The NLP engine correctly determined the patients' opioid use status in 92.9% of cases, in which it correctly identified persistent opioid use in 95.6% of cases and no persistent opioid use in 86.1% of cases. DISCUSSION Access to unstructured data within the perioperative history can contextualize patients' opioid use and provide further insight into the opioid crisis, while at the same time improve care directly at the patient level. While these goals are in reach, future work is needed to evaluate how to best implement NLP within different healthcare systems for use in clinical decision support.
Collapse
Affiliation(s)
- Eri C Seng
- Division of Perioperative Informatics, Department of Anesthesiology, University of California San Diego, La Jolla, California, USA
| | - Soraya Mehdipour
- Division of Perioperative Informatics, Department of Anesthesiology, University of California San Diego, La Jolla, California, USA
| | - Sierra Simpson
- Division of Perioperative Informatics, Department of Anesthesiology, University of California San Diego, La Jolla, California, USA
| | - Rodney A Gabriel
- Division of Perioperative Informatics, Department of Anesthesiology, University of California San Diego, La Jolla, California, USA
- Division of Regional Anesthesia, Department of Anesthesiology, University of California, San Diego, La Jolla, California, USA
| |
Collapse
|
32
|
Tai-Seale M, Baxter SL, Vaida F, Walker A, Sitapati AM, Osborne C, Diaz J, Desai N, Webb S, Polston G, Helsten T, Gross E, Thackaberry J, Mandvi A, Lillie D, Li S, Gin G, Achar S, Hofflich H, Sharp C, Millen M, Longhurst CA. AI-Generated Draft Replies Integrated Into Health Records and Physicians' Electronic Communication. JAMA Netw Open 2024; 7:e246565. [PMID: 38619840 PMCID: PMC11019394 DOI: 10.1001/jamanetworkopen.2024.6565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Accepted: 02/16/2024] [Indexed: 04/16/2024] Open
Abstract
Importance Timely tests are warranted to assess the association between generative artificial intelligence (GenAI) use and physicians' work efforts. Objective To investigate the association between GenAI-drafted replies for patient messages and physician time spent on answering messages and the length of replies. Design, Setting, and Participants Randomized waiting list quality improvement (QI) study from June to August 2023 in an academic health system. Primary care physicians were randomized to an immediate activation group and a delayed activation group. Data were analyzed from August to November 2023. Exposure Access to GenAI-drafted replies for patient messages. Main Outcomes and Measures Time spent (1) reading messages, (2) replying to messages, (3) length of replies, and (4) physician likelihood to recommend GenAI drafts. The a priori hypothesis was that GenAI drafts would be associated with less physician time spent reading and replying to messages. A mixed-effects model was used. Results Fifty-two physicians participated in this QI study, with 25 randomized to the immediate activation group and 27 randomized to the delayed activation group. A contemporary control group included 70 physicians. There were 18 female participants (72.0%) in the immediate group and 17 female participants (63.0%) in the delayed group; the median age range was 35-44 years in the immediate group and 45-54 years in the delayed group. The median (IQR) time spent reading messages in the immediate group was 26 (11-69) seconds at baseline, 31 (15-70) seconds 3 weeks after entry to the intervention, and 31 (14-70) seconds 6 weeks after entry. The delayed group's median (IQR) read time was 25 (10-67) seconds at baseline, 29 (11-77) seconds during the 3-week waiting period, and 32 (15-72) seconds 3 weeks after entry to the intervention. The contemporary control group's median (IQR) read times were 21 (9-54), 22 (9-63), and 23 (9-60) seconds in corresponding periods. The estimated association of GenAI was a 21.8% increase in read time (95% CI, 5.2% to 41.0%; P = .008), a -5.9% change in reply time (95% CI, -16.6% to 6.2%; P = .33), and a 17.9% increase in reply length (95% CI, 10.1% to 26.2%; P < .001). Participants recognized GenAI's value and suggested areas for improvement. Conclusions and Relevance In this QI study, GenAI-drafted replies were associated with significantly increased read time, no change in reply time, significantly increased reply length, and some perceived benefits. Rigorous empirical tests are necessary to further examine GenAI's performance. Future studies should examine patient experience and compare multiple GenAIs, including those with medical training.
Collapse
Affiliation(s)
- Ming Tai-Seale
- Department of Family Medicine, University of California San Diego School of Medicine, La Jolla
- Department of Medicine, University of California San Diego School of Medicine, La Jolla
| | - Sally L. Baxter
- Department of Medicine, University of California San Diego School of Medicine, La Jolla
- Division of Ophthalmology Informatics and Data Science, Viterbi Family Department of Ophthalmology and Shiley Eye Institute, University of California San Diego School of Medicine, La Jolla
| | - Florin Vaida
- Division of Biostatistics, University of California San Diego Herbert Wertheim School of Public Health and Human Longevity Science, La Jolla
| | - Amanda Walker
- Department of Family Medicine, University of California San Diego School of Medicine, La Jolla
| | - Amy M. Sitapati
- Department of Medicine, University of California San Diego School of Medicine, La Jolla
| | - Chad Osborne
- Department of Family Medicine, University of California San Diego School of Medicine, La Jolla
| | - Joseph Diaz
- Department of Medicine, University of California San Diego School of Medicine, La Jolla
| | - Nimit Desai
- University of California San Diego School of Medicine, La Jolla
| | - Sophie Webb
- Department of Family Medicine, University of California San Diego School of Medicine, La Jolla
| | - Gregory Polston
- Department of Anesthesiology, University of California San Diego School of Medicine, La Jolla
| | - Teresa Helsten
- Department of Medicine, University of California San Diego School of Medicine, La Jolla
| | - Erin Gross
- Department of Obstetrics and Gynecology, University of California San Diego School of Medicine, La Jolla
| | - Jessica Thackaberry
- Department of Psychiatry, University of California San Diego School of Medicine, La Jolla
| | - Ammar Mandvi
- Department of Family Medicine, University of California San Diego School of Medicine, La Jolla
| | - Dustin Lillie
- Department of Family Medicine, University of California San Diego School of Medicine, La Jolla
| | - Steve Li
- Department of Medicine, University of California San Diego School of Medicine, La Jolla
| | - Geneen Gin
- Department of Family Medicine, University of California San Diego School of Medicine, La Jolla
| | - Suraj Achar
- Department of Family Medicine, University of California San Diego School of Medicine, La Jolla
| | - Heather Hofflich
- Department of Medicine, University of California San Diego School of Medicine, La Jolla
| | - Christopher Sharp
- Department of Medicine, Stanford School of Medicine, Stanford, California
| | - Marlene Millen
- Department of Medicine, University of California San Diego School of Medicine, La Jolla
| | - Christopher A. Longhurst
- Department of Medicine, University of California San Diego School of Medicine, La Jolla
- Department of Pediatrics, University of California San Diego School of Medicine, La Jolla
| |
Collapse
|
33
|
Fogleman BM, Goldman M, Holland AB, Dyess G, Patel A. Charting Tomorrow's Healthcare: A Traditional Literature Review for an Artificial Intelligence-Driven Future. Cureus 2024; 16:e58032. [PMID: 38738104 PMCID: PMC11088287 DOI: 10.7759/cureus.58032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/11/2024] [Indexed: 05/14/2024] Open
Abstract
Electronic health record (EHR) systems have developed over time in parallel with general advancements in mainstream technology. As artificially intelligent (AI) systems rapidly impact multiple societal sectors, it has become apparent that medicine is not immune from the influences of this powerful technology. Particularly appealing is how AI may aid in improving healthcare efficiency with note-writing automation. This literature review explores the current state of EHR technologies in healthcare, specifically focusing on possibilities for addressing EHR challenges through the automation of dictation and note-writing processes with AI integration. This review offers a broad understanding of existing capabilities and potential advancements, emphasizing innovations such as voice-to-text dictation, wearable devices, and AI-assisted procedure note dictation. The primary objective is to provide researchers with valuable insights, enabling them to generate new technologies and advancements within the healthcare landscape. By exploring the benefits, challenges, and future of AI integration, this review encourages the development of innovative solutions, with the goal of enhancing patient care and healthcare delivery efficiency.
Collapse
Affiliation(s)
- Brody M Fogleman
- Internal Medicine, Edward Via College of Osteopathic Medicine - Carolinas, Spartanburg, USA
| | - Matthew Goldman
- Neurological Surgery, Houston Methodist Hospital, Houston, USA
| | - Alexander B Holland
- General Surgery, Edward Via College of Osteopathic Medicine - Carolinas, Spartanburg, USA
| | - Garrett Dyess
- Medicine, University of South Alabama College of Medicine, Mobile, USA
| | - Aashay Patel
- Neurological Surgery, University of Florida College of Medicine, Gainesville, USA
| |
Collapse
|
34
|
Van Veen D, Van Uden C, Blankemeier L, Delbrouck JB, Aali A, Bluethgen C, Pareek A, Polacin M, Reis EP, Seehofnerová A, Rohatgi N, Hosamani P, Collins W, Ahuja N, Langlotz CP, Hom J, Gatidis S, Pauly J, Chaudhari AS. Adapted large language models can outperform medical experts in clinical text summarization. Nat Med 2024; 30:1134-1142. [PMID: 38413730 DOI: 10.1038/s41591-024-02855-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2023] [Accepted: 02/02/2024] [Indexed: 02/29/2024]
Abstract
Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor-patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.
Collapse
Affiliation(s)
- Dave Van Veen
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
- Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA.
| | - Cara Van Uden
- Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
- Department of Computer Science, Stanford University, Stanford, CA, USA
| | - Louis Blankemeier
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
- Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
| | - Jean-Benoit Delbrouck
- Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
| | - Asad Aali
- Department of Electrical and Computer Engineering, The University of Texas at Austin, Austin, TX, USA
| | - Christian Bluethgen
- Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
- Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Anuj Pareek
- Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
- Copenhagen University Hospital, Copenhagen, Denmark
| | - Malgorzata Polacin
- Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Zurich, Switzerland
| | - Eduardo Pontes Reis
- Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
- Albert Einstein Israelite Hospital, São Paulo, Brazil
| | - Anna Seehofnerová
- Department of Medicine, Stanford University, Stanford, CA, USA
- Department of Radiology, Stanford University, Stanford, CA, USA
| | - Nidhi Rohatgi
- Department of Medicine, Stanford University, Stanford, CA, USA
- Department of Neurosurgery, Stanford University, Stanford, CA, USA
| | - Poonam Hosamani
- Department of Medicine, Stanford University, Stanford, CA, USA
| | - William Collins
- Department of Medicine, Stanford University, Stanford, CA, USA
| | - Neera Ahuja
- Department of Medicine, Stanford University, Stanford, CA, USA
| | - Curtis P Langlotz
- Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
- Department of Medicine, Stanford University, Stanford, CA, USA
- Department of Radiology, Stanford University, Stanford, CA, USA
- Department of Biomedical Data Science, Stanford University, Stanford, CA, USA
| | - Jason Hom
- Department of Medicine, Stanford University, Stanford, CA, USA
| | - Sergios Gatidis
- Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
- Department of Radiology, Stanford University, Stanford, CA, USA
| | - John Pauly
- Department of Electrical Engineering, Stanford University, Stanford, CA, USA
| | - Akshay S Chaudhari
- Stanford Center for Artificial Intelligence in Medicine and Imaging, Palo Alto, CA, USA
- Department of Radiology, Stanford University, Stanford, CA, USA
- Department of Biomedical Data Science, Stanford University, Stanford, CA, USA
- Stanford Cardiovascular Institute, Stanford, CA, USA
| |
Collapse
|
35
|
Li H, Rotenstein L, Jeffery MM, Paek H, Nath B, Williams BL, McLean RM, Goldstein R, Nuckols TK, Hoq L, Melnick ER. Quantifying EHR and Policy Factors Associated with the Gender Productivity Gap in Ambulatory, General Internal Medicine. J Gen Intern Med 2024; 39:557-565. [PMID: 37843702 DOI: 10.1007/s11606-023-08428-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Accepted: 09/11/2023] [Indexed: 10/17/2023]
Abstract
BACKGROUND The gender gap in physician compensation has persisted for decades. Little is known about how differences in use of the electronic health record (EHR) may contribute. OBJECTIVE To characterize how time on clinical activities, time on the EHR, and clinical productivity vary by physician gender and to identify factors associated with physician productivity. DESIGN, SETTING, AND PARTICIPANTS This longitudinal study included general internal medicine physicians employed by a large ambulatory practice network in the Northeastern United States from August 2018 to June 2021. MAIN MEASURES Monthly data on physician work relative value units (wRVUs), physician and practice characteristics, metrics of EHR use and note content, and temporal trend variables. KEY RESULTS The analysis included 3227 physician-months of data for 108 physicians (44% women). Compared with men physicians, women physicians generated 23.8% fewer wRVUs per month, completed 22.1% fewer visits per month, spent 4.0 more minutes/visit and 8.72 more minutes on the EHR per hour worked (all p < 0.001), and typed or dictated 36.4% more note characters per note (p = 0.006). With multivariable adjustment for physician age, practice characteristics, EHR use, and temporal trends, physician gender was no longer associated with productivity (men 4.20 vs. women 3.88 wRVUs/hour, p = 0.31). Typing/dictating fewer characters per note, relying on greater teamwork to manage orders, and spending less time on documentation were associated with higher wRVUs/hour. The 2021 E/M code change was associated with higher wRVUs/hour for all physicians: 10% higher for men physicians and 18% higher for women physicians (p < 0.001 and p = 0.009, respectively). CONCLUSIONS Increased team support, briefer documentation, and the 2021 E/M code change were associated with higher physician productivity. The E/M code change may have preferentially benefited women physicians by incentivizing time-intensive activities such as medical decision-making, preventive care discussion, and patient counseling that women physicians have historically spent more time performing.
Collapse
Affiliation(s)
- Huan Li
- Department of Emergency Medicine, Yale University School of Medicine, New Haven, CT, USA
- Computational Biology and Bioinformatics, Yale School of Medicine, New Haven, CT, USA
| | - Lisa Rotenstein
- Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Molly M Jeffery
- Department of Emergency Medicine, Mayo Clinic, Rochester, MN, USA
- Division of Health Care Delivery Research, Mayo Clinic, Rochester, MN, USA
| | - Hyung Paek
- Information Technology Services, Yale New Haven Health System, New Haven, CT, USA
| | - Bidisha Nath
- Department of Emergency Medicine, Yale University School of Medicine, New Haven, CT, USA
| | | | - Robert M McLean
- Northeast Medical Group, Stratford, CT, USA
- Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA
| | | | - Teryl K Nuckols
- Division of General Internal Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
- Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Lalima Hoq
- Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Edward R Melnick
- Department of Emergency Medicine, Yale University School of Medicine, New Haven, CT, USA.
- Department of Biostatistics (Health Informatics), Yale School of Public Health, New Haven, CT, USA.
| |
Collapse
|
36
|
Higa-McMillan CK, Park AL, Daleiden EL, Becker KD, Bernstein A, Chorpita BF. Getting More Out of Clinical Documentation: Can Clinical Dashboards Yield Clinically Useful Information? ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2024; 51:268-285. [PMID: 38261119 DOI: 10.1007/s10488-023-01329-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/29/2023] [Indexed: 01/24/2024]
Abstract
This study investigated coded data retrieved from clinical dashboards, which are decision-support tools that include a graphical display of clinical progress and clinical activities. Data were extracted from clinical dashboards representing 256 youth (M age = 11.9) from 128 practitioners who were trained in the Managing and Adapting Practice (MAP) system (Chorpita & Daleiden in BF Chorpita EL Daleiden 2014 Structuring the collaboration of science and service in pursuit of a shared vision. 43(2):323 338. 2014, Chorpita & Daleiden in BF Chorpita EL Daleiden 2018 Coordinated strategic action: Aspiring to wisdom in mental health service systems. 25(4):e12264. 2018) in 55 agencies across 5 regional mental health systems. Practitioners labeled up to 35 fields (i.e., descriptions of clinical activities), with the options of drawing from a controlled vocabulary or writing in a client-specific activity. Practitioners then noted when certain activities occurred during the episode of care. Fields from the extracted data were coded and reliability was assessed for Field Type, Practice Element Type, Target Area, and Audience (e.g., Caregiver Psychoeducation: Anxiety would be coded as Field Type = Practice Element; Practice Element Type = Psychoeducation; Target Area = Anxiety; Audience = Caregiver). Coders demonstrated moderate to almost perfect interrater reliability. On average, practitioners recorded two activities per session, and clients had 10 unique activities across all their sessions. Results from multilevel models showed that clinical activity characteristics and sessions accounted for the most variance in the occurrence, recurrence, and co-occurrence of clinical activities, with relatively less variance accounted for by practitioners, clients, and regional systems. Findings are consistent with patterns of practice reported in other studies and suggest that clinical dashboards may be a useful source of clinical information. More generally, the use of a controlled vocabulary for clinical activities appears to increase the retrievability and actionability of healthcare information and thus sets the stage for advancing the utility of clinical documentation.
Collapse
|
37
|
Rushlow DR, Thacher TD, Barry BA. Building Capacity for Pragmatic Trials of Digital Technology in Primary Care. Mayo Clin Proc 2024; 99:491-501. [PMID: 38432751 DOI: 10.1016/j.mayocp.2023.07.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/18/2023] [Revised: 06/30/2023] [Accepted: 07/17/2023] [Indexed: 03/05/2024]
Abstract
Frontline primary care teams face important challenges in seeking to transform the quality of care delivered to patients and to reduce clerical burden for clinicians. Digital technologies using artificial intelligence hold substantial promise to aid in this transformation. Both pragmatic clinical trials and implementation science are key tools to successfully introduce, evaluate, and sustain innovations in real-world primary care practices. Previous articles in this thematic series have provided an in-depth overview of pragmatic trials and implementation science. This paper demonstrates and provides a framework for how these concepts, together with digital transformation, can be used to solve many of the challenges facing primary care. This framework is conceived as the collaboration of frontline primary care teams with innovators in academic institutions and industry through pragmatic trials and implementation science.
Collapse
Affiliation(s)
| | | | - Barbara A Barry
- Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, MN
| |
Collapse
|
38
|
Overhage JM, Qeadan F, Choi EHE, Vos D, Kroth PJ. Explaining Variability in Electronic Health Record Effort in Primary Care Ambulatory Encounters. Appl Clin Inform 2024; 15:212-219. [PMID: 38508654 PMCID: PMC10954376 DOI: 10.1055/s-0044-1782228] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2023] [Accepted: 01/30/2024] [Indexed: 03/22/2024] Open
Abstract
BACKGROUND Electronic health record (EHR) user interface event logs are fast providing another perspective on the value and efficiency EHR technology brings to health care. Analysis of these detailed usage data has demonstrated their potential to identify EHR and clinical process design factors related to user efficiency, satisfaction, and burnout. OBJECTIVE This study aimed to analyze the event log data across 26 different health systems to determine the variability of use of a single vendor's EHR based on four event log metrics, at the individual, practice group, and health system levels. METHODS We obtained de-identified event log data recorded from June 1, 2018, to May 31, 2019, from 26 health systems' primary care physicians. We estimated the variability in total Active EHR Time, Documentation Time, Chart Review Time, and Ordering Time across health systems, practice groups, and individual physicians. RESULTS In total, 5,444 physicians (Family Medicine: 3,042 and Internal Medicine: 2,422) provided care in a total of 2,285 different practices nested in 26 health systems. Health systems explain 1.29, 3.55, 3.45, and 3.30% of the total variability in Active Time, Documentation Time, Chart Review Time, and Ordering Time, respectively. Practice-level variability was estimated to be 7.96, 13.52, 8.39, and 5.57%, respectively, and individual physicians explained the largest proportion of the variability for those same outcomes 17.09, 27.49, 17.51, and 19.75%, respectively. CONCLUSION The most variable physician EHR usage patterns occurs at the individual physician level and decreases as you move up to the practice and health system levels. This suggests that interventions to improve individual users' EHR usage efficiency may have the most potential impact compared with those directed at health system or practice levels.
Collapse
Affiliation(s)
| | - Fares Qeadan
- Department of Public Health Sciences, Loyola University Chicago, Chicago, Illinois, United States
| | - Eun Ho Eunice Choi
- University of New Mexico School of Medicine, Albuquerque, New Mexico, United States
| | - Duncan Vos
- Division of Epidemiology and Biostatistics, Department of Biomedical Sciences, Western Michigan University Homer Stryker M.D. School of Medicine, Kalamazoo, Michigan, United States
| | - Philip J. Kroth
- Department of Biomedical Informatics, Western Michigan University Homer Stryker M.D. School of Medicine, Kalamazoo, Michigan, United States
| |
Collapse
|
39
|
Mandal S, Wiesenfeld BM, Mann DM, Szerencsy AC, Iturrate E, Nov O. Quantifying the impact of telemedicine and patient medical advice request messages on physicians' work-outside-work. NPJ Digit Med 2024; 7:35. [PMID: 38355913 PMCID: PMC10867011 DOI: 10.1038/s41746-024-01001-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Accepted: 01/03/2024] [Indexed: 02/16/2024] Open
Abstract
The COVID-19 pandemic has boosted digital health utilization, raising concerns about increased physicians' after-hours clinical work ("work-outside-work"). The surge in patients' digital messages and additional time spent on work-outside-work by telemedicine providers underscores the need to evaluate the connection between digital health utilization and physicians' after-hours commitments. We examined the impact on physicians' workload from two types of digital demands - patients' messages requesting medical advice (PMARs) sent to physicians' inbox (inbasket), and telemedicine. Our study included 1716 ambulatory-care physicians in New York City regularly practicing between November 2022 and March 2023. Regression analyses assessed primary and interaction effects of (PMARs) and telemedicine on work-outside-work. The study revealed a significant effect of PMARs on physicians' work-outside-work and that this relationship is moderated by physicians' specialties. Non-primary care physicians or specialists experienced a more pronounced effect than their primary care peers. Analysis of their telemedicine load revealed that primary care physicians received fewer PMARs and spent less time in work-outside-work with more telemedicine. Specialists faced increased PMARs and did more work-outside-work as telemedicine visits increased which could be due to the difference in patient panels. Reducing PMAR volumes and efficient inbasket management strategies needed to reduce physicians' work-outside-work. Policymakers need to be cognizant of potential disruptions in physicians carefully balanced workload caused by the digital health services.
Collapse
Affiliation(s)
- Soumik Mandal
- Dept of Population Health, New York University Grossman School of Medicine, New York, NY, USA.
- Technology Management & Innovation, New York University Tandon School of Engineering, New York, NY, USA.
| | - Batia M Wiesenfeld
- New York University Leonard N Stern School of Business, New York, NY, USA
| | - Devin M Mann
- Dept of Population Health, New York University Grossman School of Medicine, New York, NY, USA
- MCIT Department of Health Informatics, NYU Langone Health, New York, USA
| | - Adam C Szerencsy
- MCIT Department of Health Informatics, NYU Langone Health, New York, USA
| | - Eduardo Iturrate
- Dept of Population Health, New York University Grossman School of Medicine, New York, NY, USA
| | - Oded Nov
- Technology Management & Innovation, New York University Tandon School of Engineering, New York, NY, USA
| |
Collapse
|
40
|
Köngeter A, Schickhardt C, Jungkunz M, Mehlis K, Winkler EC. Physicians' attitudes towards secondary use of clinical data for biomedical research purposes in Germany. Results of a quantitative survey. PLoS One 2024; 19:e0274032. [PMID: 38349908 PMCID: PMC10863899 DOI: 10.1371/journal.pone.0274032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2022] [Accepted: 03/10/2023] [Indexed: 02/15/2024] Open
Abstract
BACKGROUND For biomedical data-driven research purposes, secondary use of clinical data carries great but largely untapped potential. Physicians' attitudes and their needs towards secondary data use are essential to inform its practical and ethically sound implementation but are currently understudied. OBJECTIVE Therefore, the objectives of the study are to assess physicians' (i) general attitudes and concerns, (ii) willingness to adapt workflows and to make data available for secondary use, (iii) group-specific conditions toward implementation of secondary use and associated concerns of physician-scientists and purely clinical physicians. METHODS We developed an online survey based on a literature review and an expert interview study. Physicians in private practice and at two large German university hospitals were surveyed from May 2021 until January 2022. RESULTS In total, 446 physicians participated in the survey. 96% [380/397] of all physicians reported a positive attitude towards secondary use; 87% [31/397] are in-principle willing to support secondary use of clinical data along with a small proportion of physicians with fundamental reservations. Secondly, the most important conditions for adapting workflows were funding of additional time and effort for research-adequate documentation (71% [286/390]) and the most important condition for providing patients' clinical data was reliable protection of patients' privacy (67% [254/382]). Thirdly, physician-scientists were more likely than purely clinical physicians to request additional funding for research-adequate documentation as a precondition for support (83% vs 69%, P = .002) and the privilege to conduct research with their own patients' clinical data before other researchers are allowed to (43% vs 11%, P < .001); while purely clinical physicians more frequently require reliable protection of patient privacy (76% vs 62%, P = .007) and monetary compensation (45% vs 25%, P < .001). CONCLUSION Since this study presents high in-principle willingness of physicians to support secondary use along with little general concerns, it seems essential to address physicians' group-specific conditions toward secondary use in order to gain their support.
Collapse
Affiliation(s)
- Anja Köngeter
- Section for Translational Medical Ethics, Department of Medical Oncology, National Center for Tumor Diseases, Heidelberg University Hospital, Heidelberg, Germany
| | - Christoph Schickhardt
- Section for Translational Medical Ethics, Department of Medical Oncology, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Martin Jungkunz
- Section for Translational Medical Ethics, Department of Medical Oncology, National Center for Tumor Diseases, German Cancer Research Center, Heidelberg, Germany
| | - Katja Mehlis
- Section for Translational Medical Ethics, Department of Medical Oncology, National Center for Tumor Diseases, Heidelberg University Hospital, Heidelberg, Germany
| | - Eva C. Winkler
- Section for Translational Medical Ethics, Department of Medical Oncology, National Center for Tumor Diseases, Heidelberg University Hospital, Heidelberg, Germany
| |
Collapse
|
41
|
Rivera CG. We Shape Our Tools and Then Our Tools Shape Us: OPAT and the EHR. Open Forum Infect Dis 2024; 11:ofae006. [PMID: 38356783 PMCID: PMC10866570 DOI: 10.1093/ofid/ofae006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Accepted: 01/08/2024] [Indexed: 02/16/2024] Open
Abstract
A commentary on Canterino et al. (2024) and Munsiff et al. (2024), articles where clinicians from two large OPAT programs.
Collapse
|
42
|
Cross DA, Holmgren AJ, Apathy NC. The role of organizations in shaping physician use of electronic health records. Health Serv Res 2024; 59:e14203. [PMID: 37438938 PMCID: PMC10771898 DOI: 10.1111/1475-6773.14203] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/14/2023] Open
Abstract
OBJECTIVE The aim of the study was to (1) characterize organizational differences in primary care physicians' electronic health record (EHR) behavior; (2) assess within-organization consistency in EHR behaviors; and (3) identify whether organizational consistency is associated with physician-level efficiency. DATA SOURCES EHR metadata capturing averaged weekly measures of EHR time and documentation composition from 75,124 US primary care physicians across 299 organizations between September 2020 and May 2021 were taken. EHR time measures include active time in orders, chart review, notes, messaging, time spent outside of scheduled hours, and total EHR time. Documentation composition measures include note length and percentage use of templated text or copy/paste. Efficiency is measured as the percent of visits with same-day note completion. STUDY DESIGN All analyses are cross-sectional. Across-organization differences in EHR use and documentation composition are presented via 90th-to-10th percentile ratios of means and SDs. Multilevel modeling with post-estimation variance partitioning assesses the extent of an organizational signature-the proportion of variation in our measures attributable to organizations (versus specialty and individual behaviors). We measured organizational internal consistency for each measure via organization-level SD, which we grouped into quartiles for regression. Association between internally consistent (i.e., low SD) organizational EHR use and physician-level efficiency was assessed with multi-variable OLS models. DATA COLLECTION Extraction from Epic's Signal platform used for measuring provider EHR efficiency. PRINCIPAL FINDINGS EHR time per visit for physicians at a 90th percentile organization is 1.94 times the average EHR time at a 10th percentile organization. There is little evidence, on average, of an organizational signature. However, physicians in organizations with high internal consistency in EHR use demonstrate increased efficiency. Physicians in organizations with the highest internal consistency (top quartile) have a 3.77 percentage point higher same-day visit closure rates compared with peers in bottom quartile organizations (95% confidence interval: 0.0142-0.0612). CONCLUSIONS Results suggest unrealized opportunities for organizations and policymakers to support consistency in how physicians engage in EHR-supported work.
Collapse
Affiliation(s)
- Dori A. Cross
- Division of Health Policy and ManagementUniversity of Minnesota School of Public HealthMinneapolisMinnesotaUSA
| | - A Jay Holmgren
- Department of MedicineUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| | - Nate C. Apathy
- Center for Human Factors in Healthcare, MedStar Health Research InstituteHyattsvilleMarylandUSA
| |
Collapse
|
43
|
Xu Z, Xu X, Zhu X, Niu K, Dong J, He Z. Attention-Based Deep Learning Model for Prediction of Major Adverse Cardiovascular Events in Peritoneal Dialysis Patients. IEEE J Biomed Health Inform 2024; 28:1101-1109. [PMID: 38048232 DOI: 10.1109/jbhi.2023.3338729] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/06/2023]
Abstract
Major adverse cardiovascular events (MACE) encompass pivotal cardiovascular outcomes such as myocardial infarction, unstable angina, and cardiovascular-related mortality. Patients undergoing peritoneal dialysis (PD) exhibit specific cardiovascular risk factors during the treatment, which can escalate the likelihood of cardiovascular events. Hence, the prediction and key factor analysis of MACE have assumed paramount significance for peritoneal dialysis patients. Current pathological methodologies for prognosis prediction are not only costly but also cumbersome in effectively processing electronic health records (EHRs) data with high dimensionality, heterogeneity, and time series. Therefore in this study, we propose the CVEformer, an attention-based neural network designed to predict MACE and analyze risk factors. CVEformer leverages the self-attention mechanism to capture temporal correlations among time series variables, allowing for weighted integration of variables and estimation of the probability of MACE. CVEformer first captures the correlations among heterogeneous variables through attention scores. Then, it analyzes the correlations within the time series data to identify key risk variables and predict the probability of MACE. When trained and evaluated on data from a large cohort of peritoneal dialysis patients across multiple centers, CVEformer outperforms existing models in terms of predictive performance.
Collapse
|
44
|
Allen MR, Webb S, Mandvi A, Frieden M, Tai-Seale M, Kallenberg G. Navigating the doctor-patient-AI relationship - a mixed-methods study of physician attitudes toward artificial intelligence in primary care. BMC PRIMARY CARE 2024; 25:42. [PMID: 38281026 PMCID: PMC10821550 DOI: 10.1186/s12875-024-02282-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 01/19/2024] [Indexed: 01/29/2024]
Abstract
BACKGROUND Artificial intelligence (AI) is a rapidly advancing field that is beginning to enter the practice of medicine. Primary care is a cornerstone of medicine and deals with challenges such as physician shortage and burnout which impact patient care. AI and its application via digital health is increasingly presented as a possible solution. However, there is a scarcity of research focusing on primary care physician (PCP) attitudes toward AI. This study examines PCP views on AI in primary care. We explore its potential impact on topics pertinent to primary care such as the doctor-patient relationship and clinical workflow. By doing so, we aim to inform primary care stakeholders to encourage successful, equitable uptake of future AI tools. Our study is the first to our knowledge to explore PCP attitudes using specific primary care AI use cases rather than discussing AI in medicine in general terms. METHODS From June to August 2023, we conducted a survey among 47 primary care physicians affiliated with a large academic health system in Southern California. The survey quantified attitudes toward AI in general as well as concerning two specific AI use cases. Additionally, we conducted interviews with 15 survey respondents. RESULTS Our findings suggest that PCPs have largely positive views of AI. However, attitudes often hinged on the context of adoption. While some concerns reported by PCPs regarding AI in primary care focused on technology (accuracy, safety, bias), many focused on people-and-process factors (workflow, equity, reimbursement, doctor-patient relationship). CONCLUSION Our study offers nuanced insights into PCP attitudes towards AI in primary care and highlights the need for primary care stakeholder alignment on key issues raised by PCPs. AI initiatives that fail to address both the technological and people-and-process concerns raised by PCPs may struggle to make an impact.
Collapse
Affiliation(s)
- Matthew R Allen
- Department of Family Medicine, University of California San Diego, La Jolla, CA, 92093, USA.
- Division of Biomedical Informatics, University of California San Diego, La Jolla, CA, 92093, USA.
| | - Sophie Webb
- Department of Family Medicine, University of California San Diego, La Jolla, CA, 92093, USA
| | - Ammar Mandvi
- Department of Family Medicine, University of California San Diego, La Jolla, CA, 92093, USA
| | - Marshall Frieden
- Department of Family Medicine, University of California San Diego, La Jolla, CA, 92093, USA
| | - Ming Tai-Seale
- Department of Family Medicine, University of California San Diego, La Jolla, CA, 92093, USA
| | - Gene Kallenberg
- Department of Family Medicine, University of California San Diego, La Jolla, CA, 92093, USA
| |
Collapse
|
45
|
Magon HS, Helkey D, Shanafelt T, Tawfik D. Creating Conversion Factors from EHR Event Log Data: A Comparison of Investigator-Derived and Vendor-Derived Metrics for Primary Care Physicians. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2024; 2023:1115-1124. [PMID: 38222350 PMCID: PMC10785859] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 01/16/2024]
Abstract
Physicians spend a large amount of time with the electronic health record (EHR), which the majority believe contributes to their burnout. However, there are limitedstandardized measures of physician EHR time. Vendor-derived metrics are standardized but may underestimate real-world EHR experience. Investigator-derived metrics may be more reliable but not standardized, particularly with regard to timeout thresholds defining inactivity. This study aimed to enable standardized investigator-derived metrics using conversion factors between raw event log-derived metrics and Signal (Epic System's standardized metric) for primary care physicians. This was an observational, retrospective longitudinal study of EHR raw event logs and Signal data from a quaternary academic medical center and its community affiliates in California, over a 6-month period. The study evaluated 242 physicians over 1370 physician-months, comparing 53.7 million event logs to 6850 Signal metrics, in five different time based metrics. Results show that inactivity thresholds for event log metric derivation that most closely approximate Signal metrics ranged from 90 seconds (Visit Navigator) to 360 seconds ("Pajama time") depending on the metric. Based on this data, conversion factors for investigator-derived metrics across a wide range of inactivity thresholds, via comparison with Signal metrics, are provided which may allow researchers to consistently quantify EHR experience.
Collapse
Affiliation(s)
- Honor S Magon
- Stanford University School of Medicine, Stanford, CA
| | - Daniel Helkey
- Stanford University School of Medicine, Stanford, CA
| | | | - Daniel Tawfik
- Stanford University School of Medicine, Stanford, CA
| |
Collapse
|
46
|
Ruan EL, Rossetti SC, Hsu H, Kim EY, Trepp RC. A Practical Approach to Optimize Computerized Provider Order Entry Systems and Reduce Clinician Burden: Pre-Post Evaluation of Vendor-Derived "Order Friction" Data. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2024; 2023:1246-1256. [PMID: 38222358 PMCID: PMC10785931] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 01/16/2024]
Abstract
Computerized provider order entry (CPOE) systems have been cited as a significant contributor to clinician burden. Vendor-derived measures and data sets have been developed to help with optimization of CPOE systems. We describe how we analyzed vendor-derived Order Friction (OF) EHR log data at our health system and propose a practical approach for optimizing CPOE systems by reducing OF. We also conducted a pre-post intervention study using OF data to evaluate the impact of defaulting the frequency of urine, stool and nasal swab tests and found that all modified orders had significantly fewer changes required per order (p<0.01). Our proposed approach is a six-step process: 1) understand the ordering process, 2) understand OF data elements contextually, 3) explore ordering user-level factors, 4) evaluate order volume and friction from different order sources, 5) optimize order-level design, 6) identify high volume alerts to evaluate for appropriateness.
Collapse
Affiliation(s)
- Elise L Ruan
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
- Department of Medicine, NewYork-Presbyterian/Columbia University Irving Medical Center, New York, New York, USA
| | - Sarah C Rossetti
- Department of Biomedical Informatics, Columbia University, New York, New York, USA
- School of Nursing, Columbia University, New York, New York, USA
| | - Hanson Hsu
- Department of Emergency Medicine, NewYork-Presbyterian Hospital/Weill Cornell Medical Center, New York, New York, USA
| | - Eugene Y Kim
- Department of Emergency Medicine, NewYork-Presbyterian Hospital/Columbia University Irving Medical Center, New York, New York, USA
| | - Richard C Trepp
- Department of Emergency Medicine, NewYork-Presbyterian Hospital/Columbia University Irving Medical Center, New York, New York, USA
| |
Collapse
|
47
|
Underdahl L, Ditri M, Duthely LM. Physician Burnout: Evidence-Based Roadmaps to Prioritizing and Supporting Personal Wellbeing. J Healthc Leadersh 2024; 16:15-27. [PMID: 38192639 PMCID: PMC10773242 DOI: 10.2147/jhl.s389245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2023] [Accepted: 12/19/2023] [Indexed: 01/10/2024] Open
Abstract
Current literature validates the magnitude of physician burnout as a complex challenge affecting physicians, patients, and healthcare delivery that mandates science-informed intervention. Physician burnout affects both individual practitioners and patient care delivery. Interventions, defined as roadmaps, to prioritizing and supporting personal wellbeing encompass organizational, individual, and moral injury, with virtually no consensus on optimal approaches. The purpose of this conceptual review is to present evidence-based innovative insights on contributing factors, mitigation, and designing adaptive systems to combat and prevent burnout. Science-informed policy initiatives that support long-term organizational changes endorsed by both leadership and institutional stakeholders are keys to sustaining personal wellbeing and ending burnout.
Collapse
Affiliation(s)
- Louise Underdahl
- College of Doctoral Studies, University of Phoenix, Phoenix, AZ, USA
| | - Mary Ditri
- Community Health, New Jersey Hospital Association, Princeton, NJ, USA
| | - Lunthita M Duthely
- Obstetrics, Gynecology and Reproductive Sciences and the Department of Public Health Sciences, University of Miami Health System, Miami, FL, USA
| |
Collapse
|
48
|
Thandi M, Wong ST, Price M, Baumbusch J. Perspectives on the representation of frailty in the electronic frailty index. BMC PRIMARY CARE 2024; 25:4. [PMID: 38166753 PMCID: PMC10759446 DOI: 10.1186/s12875-023-02225-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Accepted: 11/27/2023] [Indexed: 01/05/2024]
Abstract
BACKGROUND Frailty is a state of increased vulnerability from physical, social, and cognitive factors resulting in greater risk of negative health-related outcomes and increased healthcare expenditure. A 36-factor electronic frailty index (eFI) developed in the United Kingdom calculates frailty scores using electronic medical record data. There is currently no standardization of frailty screening in Canadian primary care. In order to implement the eFI in a Canadian context, adaptation of the tool is necessary because frailty is represented by different clinical terminologies in the UK and Canada. In considering the promise of implementing an eFI in British Columbia, Canada, we first looked at the content validation of the 36-factor eFI. Our research question was: Does the eFI represent frailty from the perspectives of primary care clinicians and older adults in British Columbia? METHODS A modified Delphi using three rounds of questionnaires with a panel of 23 experts (five family physicians, five nurse practitioners, five nurses, four allied health professionals, four older adults) reviewed and provided feedback on the 36-factor eFI. These professional groups were chosen because they closely work as interprofessional teams within primary care settings with older adults. Older adults provide real life context and experiences. Questionnaires involved rating the importance of each frailty factor on a 0-10 scale and providing rationale for ratings. Panelists were also given the opportunity to suggest additional factors that ought to be included in the screening tool. Suggested factors were similarly rated in two Delphi rounds. RESULTS Thirty-three of the 36 eFI factors achieved consensus (> 80% of panelists provided a rating of ≥ 8). Factors that did not achieve consensus were hypertension, thyroid disorder and peptic ulcer. These factors were perceived as easily treatable or manageable and/or not considered reflective of frailty on their own. Additional factors suggested by panelists that achieved consensus included: cancer, challenges to healthcare access, chronic pain, communication challenges, fecal incontinence, food insecurity, liver failure/cirrhosis, mental health challenges, medication noncompliance, poverty/financial difficulties, race/ethnic disparity, sedentary/low activity levels, and substance use/misuse. There was a 100% retention rate in each of the three Delphi rounds. CONCLUSIONS AND NEXT STEPS Three key findings emerged from this study: the conceptualization of frailty varied across participants, identification of frailty in community/primary care remains challenging, and social determinants of health affect clinicians' assessments and perceptions of frailty status. This study will inform the next phase of a broader mixed-method sequential study to build a frailty screening tool that could ultimately become a standard of practice for frailty screening in Canadian primary care. Early detection of frailty can help tailor decision making, frame discussions about goals of care, prevent advancement on the frailty trajectory, and ultimately decrease health expenditures, leading to improved patient and system level outcomes.
Collapse
Affiliation(s)
- Manpreet Thandi
- School of Nursing, University of British Columbia, T201 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5, Canada.
| | - Sabrina T Wong
- School of Nursing, University of British Columbia, T201 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5, Canada
- Centre for Health Services and Policy Research, University of British Columbia, 201-2206 East Mall, Vancouver, BC, V6T 1Z3, Canada
| | - Morgan Price
- Department of Family Practice, University of British Columbia, David Strangway Building, Suite 300, 5950 University Boulevard, Vancouver, BC, V6T 1Z3, Canada
| | - Jennifer Baumbusch
- School of Nursing, University of British Columbia, T201 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5, Canada
| |
Collapse
|
49
|
Wallace TD, Knollman-Porter K, Brown J, Schwartz A, Hodge A, Brown G, Beardslee J, Gore RK. mTBI evaluation, management, and referral to allied healthcare: practices of first-line healthcare professionals. Brain Inj 2024; 38:32-44. [PMID: 38333958 DOI: 10.1080/02699052.2024.2309245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Accepted: 01/19/2024] [Indexed: 02/10/2024]
Abstract
PRIMARY OBJECTIVE To gain an understanding of current evaluation practices, post-injury recommendations, and referrals to allied healthcare professions (AHP) by first-line healthcare professionals (FHPs) providing care for people with mild traumatic brain injury (mTBI). RESEARCH DESIGN Survey study. METHODS AND PROCEDURES Physicians, physician assistants, nurse practitioners, nurses, and athletic trainers (n = 126) completed an online survey, including Likert scale and free response question relating to mTBI evaluation, management, and referral practices. MAIN OUTCOMES AND RESULTS FHPs surveyed reported being confident in their ability to evaluate patients with suspected mTBI, relying most heavily on patient-reported symptoms and physical signs as methods of evaluation. Most FHPs reported making recommendations to compensate for the symptoms experienced following mTBI diagnosis. In contrast, FHPs expressed challenges in the evaluation and management of symptoms associated with mTBI along with limited knowledge of and referrals to AHPs. CONCLUSIONS Overall, FHPs feel confident in the diagnosis of mTBI but experience assessment and management challenges. AHPs are underutilized on mTBI management teams calling for a need for multidisciplinary collaboration on research, education, and rehabilitation efforts to optimally care for people experiencing mTBI symptoms.
Collapse
Affiliation(s)
- Tracey D Wallace
- Complex Concussion Clinic, Shepherd Center, Atlanta, Georgia, USA
- SHARE Military Initiative, Shepherd Center, Atlanta, Georgia, USA
| | | | | | - Amber Schwartz
- Complex Concussion Clinic, Shepherd Center, Atlanta, Georgia, USA
- SHARE Military Initiative, Shepherd Center, Atlanta, Georgia, USA
| | - April Hodge
- Complex Concussion Clinic, Shepherd Center, Atlanta, Georgia, USA
| | - Gregory Brown
- Complex Concussion Clinic, Shepherd Center, Atlanta, Georgia, USA
- SHARE Military Initiative, Shepherd Center, Atlanta, Georgia, USA
| | | | - Russell K Gore
- Complex Concussion Clinic, Shepherd Center, Atlanta, Georgia, USA
- SHARE Military Initiative, Shepherd Center, Atlanta, Georgia, USA
| |
Collapse
|
50
|
Arndt BG, Micek MA, Rule A, Shafer CM, Baltus JJ, Sinsky CA. More Tethered to the EHR: EHR Workload Trends Among Academic Primary Care Physicians, 2019-2023. Ann Fam Med 2024; 22:12-18. [PMID: 38253499 PMCID: PMC11233089 DOI: 10.1370/afm.3047] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 08/23/2023] [Accepted: 09/05/2023] [Indexed: 01/24/2024] Open
Abstract
PURPOSE The purpose of this study is to evaluate recent trends in primary care physician (PCP) electronic health record (EHR) workload. METHODS This longitudinal study observed the EHR use of 141 academic PCPs over 4 years (May 2019 to March 2023). Ambulatory full-time equivalency (aFTE), visit volume, and panel size were evaluated. Electronic health record time and inbox message volume were measured per 8 hours of scheduled clinic appointments. RESULTS From the pre-COVID-19 pandemic year (May 2019 to February 2020) to the most recent study year (April 2022 to March 2023), the average time PCPs spent in the EHR per 8 hours of scheduled clinic appointments increased (+28.4 minutes, 7.8%), as did time in orders (+23.1 minutes, 58.9%), inbox (+14.0 minutes, 24.4%), chart review (+7.2 minutes, 13.0%), notes (+2.9 minutes, 2.3%), outside scheduled hours on days with scheduled appointments (+6.4 minutes, 8.2%), and on unscheduled days (+13.6 minutes, 19.9%). Primary care physicians received more patient medical advice requests (+5.4 messages, 55.5%) and prescription messages (+2.3, 19.5%) per 8 hours of scheduled clinic appointments, but fewer patient calls (-2.8, -10.5%) and results messages (-0.3, -2.7%). While total time in the EHR continued to increase in the final study year (+7.7 minutes, 2.0%), inbox time decreased slightly from the year prior (-2.2 minutes, -3.0%). Primary care physicians' average aFTE decreased 5.2% from 0.66 to 0.63 over 4 years. CONCLUSIONS Primary care physicians' time in the EHR continues to grow. While PCPs' inbox time may be stabilizing, it is still substantially higher than pre-pandemic levels. It is imperative health systems develop strategies to change the EHR workload trajectory to minimize PCPs' occupational stress and mitigate unnecessary reductions in effective physician workforce resulting from the increased EHR burden.
Collapse
Affiliation(s)
- Brian G Arndt
- Department of Family Medicine and Community Health, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Mark A Micek
- Department of Medicine, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Adam Rule
- Information School, University of Wisconsin-Madison, Madison, Wisconsin
| | - Christina M Shafer
- Department of Medicine, University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
- Independent consultant, Madison, Wisconsin
| | - Jeffrey J Baltus
- University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin
| | - Christine A Sinsky
- Professional Satisfaction and Practice Sustainability, American Medical Association, Chicago, Illinois
| |
Collapse
|