1
|
Tepe M, Emekli E. Assessing the Responses of Large Language Models (ChatGPT-4, Gemini, and Microsoft Copilot) to Frequently Asked Questions in Breast Imaging: A Study on Readability and Accuracy. Cureus 2024; 16:e59960. [PMID: 38726360 PMCID: PMC11080394 DOI: 10.7759/cureus.59960] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/09/2024] [Indexed: 05/12/2024] Open
Abstract
Background Large language models (LLMs), such as ChatGPT-4, Gemini, and Microsoft Copilot, have been instrumental in various domains, including healthcare, where they enhance health literacy and aid in patient decision-making. Given the complexities involved in breast imaging procedures, accurate and comprehensible information is vital for patient engagement and compliance. This study aims to evaluate the readability and accuracy of the information provided by three prominent LLMs, ChatGPT-4, Gemini, and Microsoft Copilot, in response to frequently asked questions in breast imaging, assessing their potential to improve patient understanding and facilitate healthcare communication. Methodology We collected the most common questions on breast imaging from clinical practice and posed them to LLMs. We then evaluated the responses in terms of readability and accuracy. Responses from LLMs were analyzed for readability using the Flesch Reading Ease and Flesch-Kincaid Grade Level tests and for accuracy through a radiologist-developed Likert-type scale. Results The study found significant variations among LLMs. Gemini and Microsoft Copilot scored higher on readability scales (p < 0.001), indicating their responses were easier to understand. In contrast, ChatGPT-4 demonstrated greater accuracy in its responses (p < 0.001). Conclusions While LLMs such as ChatGPT-4 show promise in providing accurate responses, readability issues may limit their utility in patient education. Conversely, Gemini and Microsoft Copilot, despite being less accurate, are more accessible to a broader patient audience. Ongoing adjustments and evaluations of these models are essential to ensure they meet the diverse needs of patients, emphasizing the need for continuous improvement and oversight in the deployment of artificial intelligence technologies in healthcare.
Collapse
Affiliation(s)
- Murat Tepe
- Radiology, Mediclinic City Hospital, Dubai, ARE
| | - Emre Emekli
- Radiology, Eskişehir Osmangazi University Health Practice and Research Hospital, Eskişehir, TUR
| |
Collapse
|
2
|
Sobrinho ACDS, Gomes GADO, Bueno Júnior CR. Developing a Multiprofessional Mobile App to Enhance Health Habits in Older Adults: User-Centered Approach. JMIR Form Res 2024; 8:e54214. [PMID: 38619865 PMCID: PMC11058554 DOI: 10.2196/54214] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 02/14/2024] [Accepted: 02/19/2024] [Indexed: 04/16/2024] Open
Abstract
BACKGROUND Although comprehensive lifestyle habits are crucial for healthy aging, their adherence tends to decline as individuals grow older. Sustaining a healthy life over time poses a motivational challenge. Some digital tools, such as smartphone apps aimed at promoting healthy habits, have been used to counteract this decline. However, a more profound investigation is necessary into the diverse experiences of users, particularly when it concerns older adults or those who are unfamiliar with information and communications technologies. OBJECTIVE We aimed to develop a mobile app focused on promoting the health of older adults based on the principles of software engineering and a user-centered design. The project respected all ethical guidelines and involved the participation of older adults at various stages of the development of the app. METHODS This study used a mixed methods approach, combining both quantitative and qualitative methodologies for data collection. The study was conducted in Ribeirão Prêto, São Paulo, Brazil, and involved 20 older adults of both genders who were aged ≥60 years and enrolled in the Physical Education Program for the Elderly at the University of São Paulo. The research unfolded in multiple phases, encompassing the development and refinement of the app with active engagement from the participants. RESULTS A total of 20 participants used a mobile health app with an average age of 64.8 (SD 2.7) years. Most participants had a high school education, middle-class status, and varying health literacy (mean score 73.55, SD 26.70). Overall, 90% (18/20) of the participants owned smartphones. However, 20% (4/20) of the participants faced installation challenges and 30% (6/20) struggled with web-based searches. The focus groups assessed app usability and satisfaction. Adjustments increased satisfaction scores significantly (Suitability Assessment of Materials: 34.89% to 70.65%; System Usability Scale: 71.23 to 87.14). Participant feedback emphasized font size, navigation, visual feedback, and personalization, and suggestions included health device integration, social interaction, and in-app communication support. CONCLUSIONS This study contributes to the development of health care technologies tailored to the older adult population, considering their specific needs. It is anticipated that the resulting app will serve as a valuable tool for promoting healthy habits and enhancing the quality of life for older adults.
Collapse
|
3
|
Haver HL, Gupta AK, Ambinder EB, Bahl M, Oluyemi ET, Jeudy J, Yi PH. Evaluating the Use of ChatGPT to Accurately Simplify Patient-centered Information about Breast Cancer Prevention and Screening. Radiol Imaging Cancer 2024; 6:e230086. [PMID: 38305716 PMCID: PMC10988327 DOI: 10.1148/rycan.230086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2023] [Revised: 11/28/2023] [Accepted: 12/26/2023] [Indexed: 02/03/2024]
Abstract
Purpose To evaluate the use of ChatGPT as a tool to simplify answers to common questions about breast cancer prevention and screening. Materials and Methods In this retrospective, exploratory study, ChatGPT was requested to simplify responses to 25 questions about breast cancer to a sixth-grade reading level in March and August 2023. Simplified responses were evaluated for clinical appropriateness. All original and simplified responses were assessed for reading ease on the Flesch Reading Ease Index and for readability on five scales: Flesch-Kincaid Grade Level, Gunning Fog Index, Coleman-Liau Index, Automated Readability Index, and the Simple Measure of Gobbledygook (ie, SMOG) Index. Mean reading ease, readability, and word count were compared between original and simplified responses using paired t tests. McNemar test was used to compare the proportion of responses with adequate reading ease (score of 60 or greater) and readability (sixth-grade level). Results ChatGPT improved mean reading ease (original responses, 46 vs simplified responses, 70; P < .001) and readability (original, grade 13 vs simplified, grade 8.9; P < .001) and decreased word count (original, 193 vs simplified, 173; P < .001). Ninety-two percent (23 of 25) of simplified responses were considered clinically appropriate. All 25 (100%) simplified responses met criteria for adequate reading ease, compared with only two of 25 original responses (P < .001). Two of the 25 simplified responses (8%) met criteria for adequate readability. Conclusion ChatGPT simplified answers to common breast cancer screening and prevention questions by improving the readability by four grade levels, though the potential to produce incorrect information necessitates physician oversight when using this tool. Keywords: Mammography, Screening, Informatics, Breast, Education, Health Policy and Practice, Oncology, Technology Assessment Supplemental material is available for this article. © RSNA, 2023.
Collapse
Affiliation(s)
- Hana L. Haver
- From the University of Maryland Medical Intelligent Imaging (UM2ii)
Center, Department of Diagnostic Radiology and Nuclear Medicine, University of
Maryland School of Medicine, 670 W Baltimore St, First Floor, Rm 1172,
Baltimore, MD 21201 (H.L.H., A.K.G., J.J., P.H.Y.); The Russell H. Morgan
Department of Radiology and Radiological Science, Johns Hopkins University
School of Medicine, Baltimore, Md (E.B.A., E.T.O.); Department of Radiology,
Division of Breast Imaging, Massachusetts General Hospital, Boston, Mass (M.B.);
Malone Center for Engineering in Healthcare, Whiting School of Engineering,
Johns Hopkins University, Baltimore, Md (P.H.Y.); and Fischell Department of
Bioengineering, A. James Clark School of Engineering, University of
Maryland–College Park, College Park, Md (P.H.Y.)
| | - Anuj K. Gupta
- From the University of Maryland Medical Intelligent Imaging (UM2ii)
Center, Department of Diagnostic Radiology and Nuclear Medicine, University of
Maryland School of Medicine, 670 W Baltimore St, First Floor, Rm 1172,
Baltimore, MD 21201 (H.L.H., A.K.G., J.J., P.H.Y.); The Russell H. Morgan
Department of Radiology and Radiological Science, Johns Hopkins University
School of Medicine, Baltimore, Md (E.B.A., E.T.O.); Department of Radiology,
Division of Breast Imaging, Massachusetts General Hospital, Boston, Mass (M.B.);
Malone Center for Engineering in Healthcare, Whiting School of Engineering,
Johns Hopkins University, Baltimore, Md (P.H.Y.); and Fischell Department of
Bioengineering, A. James Clark School of Engineering, University of
Maryland–College Park, College Park, Md (P.H.Y.)
| | - Emily B. Ambinder
- From the University of Maryland Medical Intelligent Imaging (UM2ii)
Center, Department of Diagnostic Radiology and Nuclear Medicine, University of
Maryland School of Medicine, 670 W Baltimore St, First Floor, Rm 1172,
Baltimore, MD 21201 (H.L.H., A.K.G., J.J., P.H.Y.); The Russell H. Morgan
Department of Radiology and Radiological Science, Johns Hopkins University
School of Medicine, Baltimore, Md (E.B.A., E.T.O.); Department of Radiology,
Division of Breast Imaging, Massachusetts General Hospital, Boston, Mass (M.B.);
Malone Center for Engineering in Healthcare, Whiting School of Engineering,
Johns Hopkins University, Baltimore, Md (P.H.Y.); and Fischell Department of
Bioengineering, A. James Clark School of Engineering, University of
Maryland–College Park, College Park, Md (P.H.Y.)
| | - Manisha Bahl
- From the University of Maryland Medical Intelligent Imaging (UM2ii)
Center, Department of Diagnostic Radiology and Nuclear Medicine, University of
Maryland School of Medicine, 670 W Baltimore St, First Floor, Rm 1172,
Baltimore, MD 21201 (H.L.H., A.K.G., J.J., P.H.Y.); The Russell H. Morgan
Department of Radiology and Radiological Science, Johns Hopkins University
School of Medicine, Baltimore, Md (E.B.A., E.T.O.); Department of Radiology,
Division of Breast Imaging, Massachusetts General Hospital, Boston, Mass (M.B.);
Malone Center for Engineering in Healthcare, Whiting School of Engineering,
Johns Hopkins University, Baltimore, Md (P.H.Y.); and Fischell Department of
Bioengineering, A. James Clark School of Engineering, University of
Maryland–College Park, College Park, Md (P.H.Y.)
| | - Eniola T. Oluyemi
- From the University of Maryland Medical Intelligent Imaging (UM2ii)
Center, Department of Diagnostic Radiology and Nuclear Medicine, University of
Maryland School of Medicine, 670 W Baltimore St, First Floor, Rm 1172,
Baltimore, MD 21201 (H.L.H., A.K.G., J.J., P.H.Y.); The Russell H. Morgan
Department of Radiology and Radiological Science, Johns Hopkins University
School of Medicine, Baltimore, Md (E.B.A., E.T.O.); Department of Radiology,
Division of Breast Imaging, Massachusetts General Hospital, Boston, Mass (M.B.);
Malone Center for Engineering in Healthcare, Whiting School of Engineering,
Johns Hopkins University, Baltimore, Md (P.H.Y.); and Fischell Department of
Bioengineering, A. James Clark School of Engineering, University of
Maryland–College Park, College Park, Md (P.H.Y.)
| | - Jean Jeudy
- From the University of Maryland Medical Intelligent Imaging (UM2ii)
Center, Department of Diagnostic Radiology and Nuclear Medicine, University of
Maryland School of Medicine, 670 W Baltimore St, First Floor, Rm 1172,
Baltimore, MD 21201 (H.L.H., A.K.G., J.J., P.H.Y.); The Russell H. Morgan
Department of Radiology and Radiological Science, Johns Hopkins University
School of Medicine, Baltimore, Md (E.B.A., E.T.O.); Department of Radiology,
Division of Breast Imaging, Massachusetts General Hospital, Boston, Mass (M.B.);
Malone Center for Engineering in Healthcare, Whiting School of Engineering,
Johns Hopkins University, Baltimore, Md (P.H.Y.); and Fischell Department of
Bioengineering, A. James Clark School of Engineering, University of
Maryland–College Park, College Park, Md (P.H.Y.)
| | - Paul H. Yi
- From the University of Maryland Medical Intelligent Imaging (UM2ii)
Center, Department of Diagnostic Radiology and Nuclear Medicine, University of
Maryland School of Medicine, 670 W Baltimore St, First Floor, Rm 1172,
Baltimore, MD 21201 (H.L.H., A.K.G., J.J., P.H.Y.); The Russell H. Morgan
Department of Radiology and Radiological Science, Johns Hopkins University
School of Medicine, Baltimore, Md (E.B.A., E.T.O.); Department of Radiology,
Division of Breast Imaging, Massachusetts General Hospital, Boston, Mass (M.B.);
Malone Center for Engineering in Healthcare, Whiting School of Engineering,
Johns Hopkins University, Baltimore, Md (P.H.Y.); and Fischell Department of
Bioengineering, A. James Clark School of Engineering, University of
Maryland–College Park, College Park, Md (P.H.Y.)
| |
Collapse
|
4
|
Laza C, Niño de Guzmán E, Gea M, Plazas M, Posso M, Rué M, Castells X, Román M. "For and against" factors influencing participation in personalized breast cancer screening programs: a qualitative systematic review until March 2022. Arch Public Health 2024; 82:23. [PMID: 38389068 PMCID: PMC10882761 DOI: 10.1186/s13690-024-01248-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 02/05/2024] [Indexed: 02/24/2024] Open
Abstract
BACKGROUND Personalized breast cancer screening is a novel strategy that estimates individual risk based on age, breast density, family history of breast cancer, personal history of benign breast lesions, and polygenic risk. Its goal is to propose personalized early detection recommendations for women in the target population based on their individual risk. Our aim was to synthesize the factors that influence women's decision to participate in personalized breast cancer screening, from the perspective of women and health care professionals. METHODS Systematic review of qualitative evidence on factors influencing participation in personalized Breast Cancer Screening. We searched in Medline, Web of science, Scopus, EMBASE, CINAHL and PsycINFO for qualitative and mixed methods studies published up to March 2022. Two reviewers conducted study selection and extracted main findings. We applied the best-fit framework synthesis and adopted the Multilevel influences on the cancer care continuum model for analysis. After organizing initial codes into the seven levels of the selected model, we followed thematic analysis and developed descriptive and analytical themes. We assessed the methodological quality with the Critical Appraisal Skills Program tool. RESULTS We identified 18 studies published between 2017 and 2022, conducted in developed countries. Nine studies were focused on women (n = 478) and in four studies women had participated in a personalized screening program. Nine studies focused in health care professionals (n = 162) and were conducted in primary care and breast cancer screening program settings. Factors influencing women's decision to participate relate to the women themselves, the type of program (personalized breast cancer screening) and perspective of health care professionals. Factors that determined women participation included persistent beliefs and insufficient knowledge about breast cancer and personalized screening, variable psychological reactions, and negative attitudes towards breast cancer risk estimates. Other factors against participation were insufficient health care professionals knowledge on genetics related to breast cancer and personalized screening process. The factors that were favourable included the women's perceived benefits for themselves and the positive impact on health systems. CONCLUSION We identified the main factors influencing women's decisions to participate in personalized breast cancer screening. Factors related to women, were the most relevant negative factors. A future implementation requires improving health literacy for women and health care professionals, as well as raising awareness of the strategy in society.
Collapse
Affiliation(s)
- Celmira Laza
- Department of Nursing and Physiotherapy, University of Lleida, Lleida, Spain
- Biomedical Research Institute of Lleida Fundació Dr. Pifarré (IRBLleida), Lleida, Spain
| | - Ena Niño de Guzmán
- Cancer Prevention and Control Program, Institut Català d' Oncologia, Barcelona, Spain
| | - Montserrat Gea
- Department of Nursing and Physiotherapy, University of Lleida, Lleida, Spain
- Biomedical Research Institute of Lleida Fundació Dr. Pifarré (IRBLleida), Lleida, Spain
| | - Merideidy Plazas
- Cochrane Associated Center- University Foundation of Health Sciences, Bogotá, Colombia
| | - Margarita Posso
- Department of Epidemiology and Evaluation, Hospital del Mar Research Institute, Barcelona, Spain
| | - Montserrat Rué
- Biomedical Research Institute of Lleida Fundació Dr. Pifarré (IRBLleida), Lleida, Spain
- Basic Medical Sciences, University of Lleida, Lleida, Spain
| | - Xavier Castells
- Department of Epidemiology and Evaluation, Hospital del Mar Research Institute, Barcelona, Spain
| | - Marta Román
- Department of Epidemiology and Evaluation, Hospital del Mar Research Institute, Barcelona, Spain.
| |
Collapse
|
5
|
Pichardo CM, Dwyer LA, Ferrer RA, Oh AY. The Association of Context with Reported Self-Efficacy for Cancer-Preventive Behaviors and Perceived Cancer Risk in U.S. Adults from the Midlife in the United States (MIDUS) Study. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2024; 21:62. [PMID: 38248527 PMCID: PMC10815586 DOI: 10.3390/ijerph21010062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/24/2023] [Revised: 11/20/2023] [Accepted: 12/13/2023] [Indexed: 01/23/2024]
Abstract
Background: Cancer is one of the leading causes of death in the United States. It is critical to understand the associations among multilevel determinants of cancer prevention and control behaviors. This study examined associations of neighborhood factors with perceived risk of cancer and self-efficacy for reducing cancer risk. Methods: Cross-sectional analyses included 2324 U.S. adults from the Midlife in the U.S. Wave 3. Participants completed surveys of neighborhood environment (perceived neighborhood trust and safety, built environment conditions, social integration), perceived cancer risk and cancer prevention efficacy. Multivariate linear regressions examined associations of neighborhood context with risk perceptions and self-efficacy. Results: In the model that adjusted for sociodemographic characteristics, better perceived neighborhood trust and safety were associated with lower perceived cancer risk. In fully adjusted models for sociodemographic characteristics and contextual factors, higher perceptions of neighborhood trust and safety were associated with higher cancer prevention self-efficacy. Perceptions of better built neighborhood conditions and higher social integration were significantly associated with lower perceived cancer risk and higher perceived cancer prevention efficacy. Conclusions: Perceptions of neighborhood context may play a role in shaping psychosocial factors such as perceived cancer risk and self-efficacy, even after controlling for robust predictors of these perceptions.
Collapse
Affiliation(s)
- Catherine M. Pichardo
- Behavioral Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health, Rockville, MD 20892, USA;
| | | | - Rebecca A. Ferrer
- Behavioral Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health, Rockville, MD 20892, USA;
| | - April Y. Oh
- Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health, Rockville, MD 20892, USA;
| |
Collapse
|
6
|
Faryabi R, Daneshi S, Davarani ER, Yusefi AR, Arabpour M, Ezoji K, Movahed E, Daneshi-Maskooni M, Hussaini SM. The assessment of risk factors and risk perception status of breast cancer in Northern Iran. BMC Womens Health 2023; 23:268. [PMID: 37194061 PMCID: PMC10189944 DOI: 10.1186/s12905-023-02422-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2022] [Accepted: 05/09/2023] [Indexed: 05/18/2023] Open
Abstract
BACKGROUND Breast cancer (BC) is the most common malignancy in women. Identifying and avoiding the preventable risk factors of BC reduces its occurrence effectively. So, this study aimed to assess BC's risk factors and risk perception status in Babol, Northern Iran. METHODS This cross-sectional study was conducted on 400 women aged 18 to 70 in Babol, Northern Iran. According to the eligibility criteria, the selected participants completed the demographic characteristics and researcher-made valid and reliable questionnaires. The statistical software was SPSS20. RESULTS The significant risk factors related to BC were old age (60 years old and more) (30.2%), obesity (25.8%), history of radiation (10%), and familial history of BC (9.5%), respectively (P < 0.05). Suspected symptoms of BC were observed in 78 (19.5%) women, including indentations in 27 (6.75%), redness in 15 (3.75%), pain in 16 (4%), and enlargement of lymph nodes in 20 (5%). The BC risk perception score was 107.72±13.22. CONCLUSION Most participants had at least one risk factor for BC. It is essential to implement intervention programs to control obesity and BC screening programs in obese and overweight women to prevent BC and its complications. Further studies are needed.
Collapse
Affiliation(s)
- Reza Faryabi
- School of Public Health, Jiroft University of Medical Sciences, Jiroft, Kerman Iran
| | - Salman Daneshi
- School of Public Health, Jiroft University of Medical Sciences, Jiroft, Kerman Iran
| | - Esmat Rezabeigi Davarani
- Health in Disasters and Emergencies Research Centre, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | - Ali Reza Yusefi
- School of Public Health, Jiroft University of Medical Sciences, Jiroft, Kerman Iran
| | - Mahdieh Arabpour
- School of Medicine, Jiroft University of Medical Sciences, Jiroft, Kerman Iran
| | - Khadijeh Ezoji
- Social Determinants of Health Research Centre, Health Research Institute, Babol University of Medical Sciences, Babol, Iran
| | - Ehsan Movahed
- School of Public Health, Jiroft University of Medical Sciences, Jiroft, Kerman Iran
| | | | | |
Collapse
|