1
|
Tomlin HR, Wissing M, Tanikella S, Kaur P, Tabas L. Challenges and Opportunities for Professional Medical Publications Writers to Contribute to Plain Language Summaries (PLS) in an AI/ML Environment - A Consumer Health Informatics Systematic Review. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2024; 2023:709-717. [PMID: 38222388 PMCID: PMC10785924] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Subscribe] [Scholar Register] [Indexed: 01/16/2024]
Abstract
Professional medical publications writers (PMWs) cover a wide range of biomedical writing activities that recently includes translation of biomedical publications to plain language summaries (PLS). The consumer health informatics literature (CHI) consistently describes the importance of incorporating health literacy principles in any natural language processing (NLP) app designed to communicate medical information to lay audiences, particularly patients. In this stepwise systematic review, we searched PubMed indexed literature for CHI NLP-based apps that have the potential to assist PMWs in developing text based PLS. Results showed that available apps are limited to patient portals and other technologies used to communicate medical text and reports from electronic health records. PMWs can apply the lessons learned from CHI NLP-based apps to supervise development of tools specific to text simplification and summarization for PLS from biomedical publications.
Collapse
Affiliation(s)
- Holly R Tomlin
- Certara Synchrogenix, Wilmington, DE, USA
- Consumer Health Informatics Lab (CHIL), Section of Biostatistics and Data Sciences, Yale School of Medicine, New Haven, CT
- Weill Cornell Medicine, Department of Population Health Sciences, Division of Health Analytics, New York, NY
| | | | | | | | | |
Collapse
|
2
|
Ayre J, Muscat DM, Mac O, Bonner C, Dunn AG, Dalmazzo J, Mouwad D, McCaffery K. Helping patient educators meet health literacy needs: End-user testing and iterative development of an innovative health literacy editing tool. PEC INNOVATION 2023; 2:100162. [PMID: 37384149 PMCID: PMC10294045 DOI: 10.1016/j.pecinn.2023.100162] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 03/15/2023] [Accepted: 05/06/2023] [Indexed: 06/30/2023]
Abstract
Objective The Sydney Health Literacy Lab (SHeLL) Editor is an online text-editing tool that provides real-time assessment and feedback on written health information (assesses grade reading score, complex language, passive voice). This study aimed to explore how the design could be further enhanced to help health information providers interpret and act on automated feedback. Methods The prototype was iteratively refined across four rounds of user-testing with health services staff (N = 20). Participants took part in online interviews and a brief follow-up survey using validated usability scales (System Usability Scale, Technology Acceptance Model). After each round, Yardley's (2021) optimisation criteria guided which changes would be implemented. Results Participants rated the Editor as having adequate usability (M = 82.8 out of 100, SD = 13.5). Most modifications sought to reduce information overload (e.g. simplifying instructions for new users) or make feedback motivating and actionable (e.g. using frequent incremental feedback to highlight changes to the text altered assessment scores). Conclusion terative user-testing was critical to balancing academic values and the practical needs of the Editor's target users. The final version emphasises actionable real-time feedback and not just assessment. Innovation The Editor is a new tool that will help health information providers apply health literacy principles to written text.
Collapse
Affiliation(s)
- Julie Ayre
- Sydney Health Literacy Lab, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, NSW, Australia
| | - Danielle M. Muscat
- Sydney Health Literacy Lab, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, NSW, Australia
| | - Olivia Mac
- Sydney Health Literacy Lab, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, NSW, Australia
| | - Carissa Bonner
- Sydney Health Literacy Lab, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, NSW, Australia
- Menzies Centre for Health Policy and Economics, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Adam G. Dunn
- Discipline of Biomedical Informatics and Digital Health, School of Medical Sciences, Faculty of Medicine and Health, The University of Sydney, NSW, Australia
| | - Jason Dalmazzo
- Discipline of Biomedical Informatics and Digital Health, School of Medical Sciences, Faculty of Medicine and Health, The University of Sydney, NSW, Australia
| | - Dana Mouwad
- Western Sydney Local Health District, Health Literacy Hub, Westmead, NSW, Australia
| | - Kirsten McCaffery
- Sydney Health Literacy Lab, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, NSW, Australia
| |
Collapse
|
3
|
Roscoe RD, Balyan R, McNamara DS, Banawan M, Schillinger D. Automated Strategy Feedback Can Improve the Readability of Physicians' Electronic Communications to Simulated Patients. INTERNATIONAL JOURNAL OF HUMAN-COMPUTER STUDIES 2023; 176:103059. [PMID: 37193118 PMCID: PMC10174593 DOI: 10.1016/j.ijhcs.2023.103059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/18/2023]
Abstract
Modern communication between health care professionals and patients increasingly relies upon secure messages (SMs) exchanged through an electronic patient portal. Despite the convenience of secure messaging, challenges include gaps between physician and patient expertise along with the asynchronous nature of such communication. Importantly, less readable SMs from physicians (e.g., too complicated) may result in patient confusion, non-adherence, and ultimately poorer health outcomes. The current simulation trial synthesizes work on patient-physician electronic communication, message readability assessments, and feedback to explore the potential for automated strategy feedback to improve the readability of physicians' SMs to patients. Within a simulated secure messaging portal featuring multiple simulated patient scenarios, computational algorithms assessed the complexity of SMs written by 67 participating physicians to patients. The messaging portal provided strategy feedback for how physician responses might be improved (e.g., adding details and information to reduce complexity). Analyses of changes in SM complexity revealed that automated strategy feedback indeed helped physicians compose and refine more readable messages. Although the effects for any individual SM were slight, the cumulative effects within and across patient scenarios showed trends of decreasing complexity. Physicians appeared to learn how to craft more readable SMs via interactions with the feedback system. Implications for secure messaging systems and physician training are discussed, along with considerations for further investigation of broader physician populations and effects on patient experience.
Collapse
Affiliation(s)
- Rod D Roscoe
- Arizona State University 7271 E. Sonoran Arroyo Mall Santa Catalina Hall 150 Mesa, AZ 85212 USA
| | - Renu Balyan
- State University of New York at Old Westbury PO Box 210, Old Westbury, NY 11568 USA
| | | | - Michelle Banawan
- Asian Institute of Management 123 Paseo de Roxas Avenue Makati, Metro Manila 1229, Philippines
| | - Dean Schillinger
- School of Medicine, Division of General Internal Medicine University of California, San Francisco 500 Parnassus Avenue San Francisco, CA 94143 USA
| |
Collapse
|
4
|
Mavragani A, Bonner C, Muscat DM, Dunn AG, Harrison E, Dalmazzo J, Mouwad D, Aslani P, Shepherd HL, McCaffery KJ. Multiple Automated Health Literacy Assessments of Written Health Information: Development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor v1. JMIR Form Res 2023; 7:e40645. [PMID: 36787164 PMCID: PMC9975914 DOI: 10.2196/40645] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Revised: 12/13/2022] [Accepted: 12/29/2022] [Indexed: 02/15/2023] Open
Abstract
Producing health information that people can easily understand is challenging and time-consuming. Existing guidance is often subjective and lacks specificity. With advances in software that reads and analyzes text, there is an opportunity to develop tools that provide objective, specific, and automated guidance on the complexity of health information. This paper outlines the development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor, an automated tool to facilitate the implementation of health literacy guidelines for the production of easy-to-read written health information. Target users were any person or organization that develops consumer-facing education materials, with or without prior experience with health literacy concepts. Anticipated users included health professionals, staff, and government and nongovernment agencies. To develop this tool, existing health literacy and relevant writing guidelines were collated. Items amenable to programmable automated assessment were incorporated into the Editor. A set of natural language processing methods were also adapted for use in the SHeLL Editor, though the approach was primarily procedural (rule-based). As a result of this process, the Editor comprises 6 assessments: readability (school grade reading score calculated using the Simple Measure of Gobbledygook (SMOG)), complex language (percentage of the text that contains public health thesaurus entries, words that are uncommon in English, or acronyms), passive voice, text structure (eg, use of long paragraphs), lexical density and diversity, and person-centered language. These are presented as global scores, with additional, more specific feedback flagged in the text itself. Feedback is provided in real-time so that users can iteratively revise and improve the text. The design also includes a "text preparation" mode, which allows users to quickly make adjustments to ensure accurate calculation of readability. A hierarchy of assessments also helps users prioritize the most important feedback. Lastly, the Editor has a function that exports the analysis and revised text. The SHeLL Health Literacy Editor is a new tool that can help improve the quality and safety of written health information. It provides objective, immediate feedback on a range of factors, complementing readability with other less widely used but important objective assessments such as complex and person-centered language. It can be used as a scalable intervention to support the uptake of health literacy guidelines by health services and providers of health information. This early prototype can be further refined by expanding the thesaurus and leveraging new machine learning methods for assessing the complexity of the written text. User-testing with health professionals is needed before evaluating the Editor's ability to improve the health literacy of written health information and evaluating its implementation into existing Australian health services.
Collapse
Affiliation(s)
| | - Carissa Bonner
- Sydney Health Literacy Lab, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Danielle M Muscat
- Sydney Health Literacy Lab, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Adam G Dunn
- Biomedical Informatics and Digital Health, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Eliza Harrison
- Biomedical Informatics and Digital Health, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Jason Dalmazzo
- Biomedical Informatics and Digital Health, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Dana Mouwad
- Western Sydney Local Health District, Health Literacy Hub, Sydney, Australia
| | - Parisa Aslani
- School of Pharmacy, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Heather L Shepherd
- Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| | - Kirsten J McCaffery
- Sydney Health Literacy Lab, Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia
| |
Collapse
|
5
|
Liu T, Xiao X. A Framework of AI-Based Approaches to Improving eHealth Literacy and Combating Infodemic. Front Public Health 2021; 9:755808. [PMID: 34917575 PMCID: PMC8669242 DOI: 10.3389/fpubh.2021.755808] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2021] [Accepted: 10/19/2021] [Indexed: 11/13/2022] Open
Abstract
The global COVID-19 pandemic has put everyone in an urgent need of accessing and comprehending health information online. Meanwhile, there has been vast amount of information/misinformation/disinformation generated over the Internet, particularly social media platforms, resulting in an infodemic. This public health crisis of COVID-19 pandemic has put each individual and the entire society in a test: what is the level of eHealth literacy is needed to seek accurate health information from online resources and to combat infodemic during a pandemic? This article aims to summarize the significances and challenges of improving eHealth literacy in both communicable (e.g., COVID-19) and non-communicable diseases [e.g., cancer, Alzheimer's disease, and cardiovascular diseases (CVDs)]. Also, this article will make our recommendations of a general framework of AI-based approaches to improving eHealth literacy and combating infodemic, including AI-augmented lifelong learning, AI-assisted translation, simplification, and summarization, and AI-based content filtering. This general framework of AI-based approaches to improving eHealth literacy and combating infodemic has the general advantage of matching the right online health information to the right people.
Collapse
Affiliation(s)
- Tianming Liu
- Department of Computer Science, University of Georgia, Athens, GA, United States
| | - Xiang Xiao
- Department of Network and New Media, College of Humanities and Arts, Hunan International Economics University, Changsha, China
| |
Collapse
|
6
|
Schillinger D, Duran ND, McNamara DS, Crossley SA, Balyan R, Karter AJ. Precision communication: Physicians' linguistic adaptation to patients' health literacy. SCIENCE ADVANCES 2021; 7:eabj2836. [PMID: 34919437 PMCID: PMC8682984 DOI: 10.1126/sciadv.abj2836] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/02/2021] [Accepted: 10/29/2021] [Indexed: 05/27/2023]
Abstract
Little quantitative research has explored which clinician skills and behaviors facilitate communication. Mutual understanding is especially challenging when patients have limited health literacy (HL). Two strategies hypothesized to improve communication include matching the complexity of language to patients’ HL (“universal tailoring”); or always using simple language (“universal precautions”). Through computational linguistic analysis of 237,126 email exchanges between dyads of 1094 physicians and 4331 English-speaking patients, we assessed matching (concordance/discordance) between physicians’ linguistic complexity and patients’ HL, and classified physicians’ communication strategies. Among low HL patients, discordance was associated with poor understanding (P = 0.046). Physicians’ “universal tailoring” strategy was associated with better understanding for all patients (P = 0.01), while “universal precautions” was not. There was an interaction between concordance and communication strategy (P = 0.021): The combination of dyadic concordance and “universal tailoring” eliminated HL-related disparities. Physicians’ ability to adapt communication to match their patients’ HL promotes shared understanding and equity. The ‘Precision Medicine’ construct should be expanded to include the domain of ‘Precision Communication.’
Collapse
Affiliation(s)
- Dean Schillinger
- UCSF Division of General Internal Medicine and Heath Communications Research Program at the Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital and Trauma Center, San Francisco, CA, USA
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
| | - Nicholas D. Duran
- School of Social and Behavioral Sciences, Arizona State University, Glendale, AZ, USA
| | | | - Scott A. Crossley
- Department of Applied Linguistics/ESL, College of Arts and Sciences, Georgia State University, Atlanta, GA, USA
| | - Renu Balyan
- Department of Mathematics, Computer and Information Science, State University of New York, Old Westbury, NY, USA
| | - Andrew J. Karter
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, USA
| |
Collapse
|
7
|
Bompelli A, Wang Y, Wan R, Singh E, Zhou Y, Xu L, Oniani D, Kshatriya BSA, Balls-Berry J(JE, Zhang R. Social and Behavioral Determinants of Health in the Era of Artificial Intelligence with Electronic Health Records: A Scoping Review. HEALTH DATA SCIENCE 2021; 2021:9759016. [PMID: 38487504 PMCID: PMC10880156 DOI: 10.34133/2021/9759016] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/22/2021] [Accepted: 06/28/2021] [Indexed: 03/17/2024]
Abstract
Background. There is growing evidence that social and behavioral determinants of health (SBDH) play a substantial effect in a wide range of health outcomes. Electronic health records (EHRs) have been widely employed to conduct observational studies in the age of artificial intelligence (AI). However, there has been limited review into how to make the most of SBDH information from EHRs using AI approaches.Methods. A systematic search was conducted in six databases to find relevant peer-reviewed publications that had recently been published. Relevance was determined by screening and evaluating the articles. Based on selected relevant studies, a methodological analysis of AI algorithms leveraging SBDH information in EHR data was provided.Results. Our synthesis was driven by an analysis of SBDH categories, the relationship between SBDH and healthcare-related statuses, natural language processing (NLP) approaches for extracting SBDH from clinical notes, and predictive models using SBDH for health outcomes.Discussion. The associations between SBDH and health outcomes are complicated and diverse; several pathways may be involved. Using NLP technology to support the extraction of SBDH and other clinical ideas simplifies the identification and extraction of essential concepts from clinical data, efficiently unlocks unstructured data, and aids in the resolution of unstructured data-related issues.Conclusion. Despite known associations between SBDH and diseases, SBDH factors are rarely investigated as interventions to improve patient outcomes. Gaining knowledge about SBDH and how SBDH data can be collected from EHRs using NLP approaches and predictive models improves the chances of influencing health policy change for patient wellness, ultimately promoting health and health equity.
Collapse
Affiliation(s)
- Anusha Bompelli
- Department of Pharmaceutical Care & Health Systems, University of Minnesota, USA
| | - Yanshan Wang
- Department of Health Information Management, University of Pittsburgh, USA
| | - Ruyuan Wan
- Department of Computer Science, University of Minnesota, USA
| | - Esha Singh
- Department of Computer Science, University of Minnesota, USA
| | - Yuqi Zhou
- Institute for Health Informatics and College of Pharmacy, University of Minnesota, USA
| | - Lin Xu
- Carlson School of Business, University of Minnesota, USA
| | - David Oniani
- Department of Computer Science and Mathematics, Luther College, USA
| | | | | | - Rui Zhang
- Institute for Health Informatics, Department of Pharmaceutical Care & Health Systems, University of Minnesota, USA
| |
Collapse
|
8
|
Brown W, Balyan R, Karter AJ, Crossley S, Semere W, Duran ND, Lyles C, Liu J, Moffet HH, Daniels R, McNamara DS, Schillinger D. Challenges and solutions to employing natural language processing and machine learning to measure patients' health literacy and physician writing complexity: The ECLIPPSE study. J Biomed Inform 2021; 113:103658. [PMID: 33316421 PMCID: PMC8186847 DOI: 10.1016/j.jbi.2020.103658] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2020] [Revised: 12/07/2020] [Accepted: 12/08/2020] [Indexed: 11/18/2022]
Abstract
OBJECTIVE In the National Library of Medicine funded ECLIPPSE Project (Employing Computational Linguistics to Improve Patient-Provider Secure Emails exchange), we attempted to create novel, valid, and scalable measures of both patients' health literacy (HL) and physicians' linguistic complexity by employing natural language processing (NLP) techniques and machine learning (ML). We applied these techniques to > 400,000 patients' and physicians' secure messages (SMs) exchanged via an electronic patient portal, developing and validating an automated patient literacy profile (LP) and physician complexity profile (CP). Herein, we describe the challenges faced and the solutions implemented during this innovative endeavor. MATERIALS AND METHODS To describe challenges and solutions, we used two data sources: study documents and interviews with study investigators. Over the five years of the project, the team tracked their research process using a combination of Google Docs tools and an online team organization, tracking, and management tool (Asana). In year 5, the team convened a number of times to discuss, categorize, and code primary challenges and solutions. RESULTS We identified 23 challenges and associated approaches that emerged from three overarching process domains: (1) Data Mining related to the SM corpus; (2) Analyses using NLP indices on the SM corpus; and (3) Interdisciplinary Collaboration. With respect to Data Mining, problems included cleaning SMs to enable analyses, removing hidden caregiver proxies (e.g., other family members) and Spanish language SMs, and culling SMs to ensure that only patients' primary care physicians were included. With respect to Analyses, critical decisions needed to be made as to which computational linguistic indices and ML approaches should be selected; how to enable the NLP-based linguistic indices tools to run smoothly and to extract meaningful data from a large corpus of medical text; and how to best assess content and predictive validities of both the LP and the CP. With respect to the Interdisciplinary Collaboration, because the research required engagement between clinicians, health services researchers, biomedical informaticians, linguists, and cognitive scientists, continual effort was needed to identify and reconcile differences in scientific terminologies and resolve confusion; arrive at common understanding of tasks that needed to be completed and priorities therein; reach compromises regarding what represents "meaningful findings" in health services vs. cognitive science research; and address constraints regarding potential transportability of the final LP and CP to different health care settings. DISCUSSION Our study represents a process evaluation of an innovative research initiative to harness "big linguistic data" to estimate patient HL and physician linguistic complexity. Any of the challenges we identified, if left unaddressed, would have either rendered impossible the effort to generate LPs and CPs, or invalidated analytic results related to the LPs and CPs. Investigators undertaking similar research in HL or using computational linguistic methods to assess patient-clinician exchange will face similar challenges and may find our solutions helpful when designing and executing their health communications research.
Collapse
Affiliation(s)
- William Brown
- Center for AIDS Prevention Studies, University of California, San Francisco, San Francisco, CA, United States; Bakar Computational Health Science Institute, University of California, San Francisco, San Francisco, CA, United States; University of California San Francisco Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, CA, United States; Department of Medicine, University of California, San Francisco, San Francisco, CA, United States.
| | - Renu Balyan
- State University of New York Old Westbury, NY, United States; Department of Psychology, Arizona State University, Tempe, AZ, United States
| | - Andrew J Karter
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, United States
| | - Scott Crossley
- Department of Applied Linguistics and English as a Second Language, Georgia State University, Atlanta, GA, United States
| | - Wagahta Semere
- Department of Medicine, University of California, San Francisco, San Francisco, CA, United States
| | - Nicholas D Duran
- School of Social and Behavioral Sciences, Arizona State University, Glendale, AZ, United States
| | - Courtney Lyles
- University of California San Francisco Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, CA, United States; Department of Medicine, University of California, San Francisco, San Francisco, CA, United States; Division of Research, Kaiser Permanente Northern California, Oakland, CA, United States
| | - Jennifer Liu
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, United States
| | - Howard H Moffet
- Division of Research, Kaiser Permanente Northern California, Oakland, CA, United States
| | - Ryane Daniels
- University of California San Francisco Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, CA, United States
| | - Danielle S McNamara
- Department of Psychology, Arizona State University, Tempe, AZ, United States
| | - Dean Schillinger
- University of California San Francisco Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, CA, United States; Department of Medicine, University of California, San Francisco, San Francisco, CA, United States; Division of Research, Kaiser Permanente Northern California, Oakland, CA, United States
| |
Collapse
|