An artificial intelligence-based chatbot for prostate cancer education: Design and patient evaluation study

Physician and Artificial Intelligence Chatbot Responses to Cancer Questions From Social Media

Importance

Artificial intelligence (AI) chatbots pose the opportunity to draft template responses to patient questions. However, the ability of chatbots to generate responses based on domain-specific knowledge of cancer remains to be tested.

Objective

To evaluate the competency of AI chatbots (GPT-3.5 [chatbot 1], GPT-4 [chatbot 2], and Claude AI [chatbot 3]) to generate high-quality, empathetic, and readable responses to patient questions about cancer.

Design, Setting, and Participants

This equivalence study compared the AI chatbot responses and responses by 6 verified oncologists to 200 patient questions about cancer from a public online forum. Data were collected on May 31, 2023.

Exposures

Random sample of 200 patient questions related to cancer from a public online forum (Reddit r/AskDocs) spanning from January 1, 2018, to May 31, 2023, was posed to 3 AI chatbots.

Main Outcomes and Measures

The primary outcomes were pilot ratings of the quality, empathy, and readability on a Likert scale from 1 (very poor) to 5 (very good). Two teams of attending oncology specialists evaluated each response based on pilot measures of quality, empathy, and readability in triplicate. The secondary outcome was readability assessed using Flesch-Kincaid Grade Level.

Results

Responses to 200 questions generated by chatbot 3, the best-performing AI chatbot, were rated consistently higher in overall measures of quality (mean, 3.56 [95% CI, 3.48-3.63] vs 3.00 [95% CI, 2.91-3.09]; P < .001), empathy (mean, 3.62 [95% CI, 3.53-3.70] vs 2.43 [95% CI, 2.32-2.53]; P < .001), and readability (mean, 3.79 [95% CI, 3.72-3.87] vs 3.07 [95% CI, 3.00-3.15]; P < .001) compared with physician responses. The mean Flesch-Kincaid Grade Level of physician responses (mean, 10.11 [95% CI, 9.21-11.03]) was not significantly different from chatbot 3 responses (mean, 10.31 [95% CI, 9.89-10.72]; P > .99) but was lower than those from chatbot 1 (mean, 12.33 [95% CI, 11.84-12.83]; P < .001) and chatbot 2 (mean, 11.32 [95% CI, 11.05-11.79]; P = .01).

Conclusions and Relevance

The findings of this study suggest that chatbots can generate quality, empathetic, and readable responses to patient questions comparable to physician responses sourced from an online forum. Further research is required to assess the scope, process integration, and patient and physician outcomes of chatbot-facilitated interactions.

Testing and Validation of a Custom Retrained Large Language Model for the Supportive Care of HN Patients with External Knowledge Base

PURPOSE

This study aimed to develop a retrained large language model (LLM) tailored to the needs of HN cancer patients treated with radiotherapy, with emphasis on symptom management and survivorship care.

METHODS

A comprehensive external database was curated for training ChatGPT-4, integrating expert-identified consensus guidelines on supportive care for HN patients and correspondences from physicians and nurses within our institution's electronic medical records for 90 HN patients. The performance of our model was evaluated using 20 patient post-treatment inquiries that were then assessed by three Board certified radiation oncologists (RadOncs). The rating of the model was assessed on a scale of 1 (strongly disagree) to 5 (strongly agree) based on accuracy, clarity of response, completeness s, and relevance.

RESULTS

The average scores for the 20 tested questions were 4.25 for accuracy, 4.35 for clarity, 4.22 for completeness, and 4.32 for relevance, on a 5-point scale. Overall, 91.67% (220 out of 240) of assessments received scores of 3 or higher, and 83.33% (200 out of 240) received scores of 4 or higher.

CONCLUSION

The custom-trained model demonstrates high accuracy in providing support to HN patients offering evidence-based information and guidance on their symptom management and survivorship care.

Global trends and hotspots of ChatGPT in medical research: a bibliometric and visualized study

Objective

With the rapid advancement of Chat Generative Pre-Trained Transformer (ChatGPT) in medical research, our study aimed to identify global trends and focal points in this domain.

Method

All publications on ChatGPT in medical research were retrieved from the Web of Science Core Collection (WoSCC) by Clarivate Analytics from January 1, 2023, to January 31, 2024. The research trends and focal points were visualized and analyzed using VOSviewer and CiteSpace.

Results

A total of 1,239 publications were collected and analyzed. The USA contributed the largest number of publications (458, 37.145%) with the highest total citation frequencies (2,461) and the largest H-index. Harvard University contributed the highest number of publications (33) among all full-time institutions. The Cureus Journal of Medical Science published the most ChatGPT-related research (127, 10.30%). Additionally, Wiwanitkit V contributed the majority of publications in this field (20). "Artificial Intelligence (AI) and Machine Learning (ML)," "Education and Training," "Healthcare Applications," and "Data Analysis and Technology" emerged as the primary clusters of keywords. These areas are predicted to remain hotspots in future research in this field.

Conclusion

Overall, this study signifies the interdisciplinary nature of ChatGPT research in medicine, encompassing AI and ML technologies, education and training initiatives, diverse healthcare applications, and data analysis and technology advancements. These areas are expected to remain at the forefront of future research, driving continued innovation and progress in the field of ChatGPT in medical research.

Talking technology: exploring chatbots as a tool for cataract patient education

CLINICAL RELEVANCE

Worldwide, millions suffer from cataracts, which impair vision and quality of life. Cataract education improves outcomes, satisfaction, and treatment adherence. Lack of health literacy, language and cultural barriers, personal preferences, and limited resources may all impede effective communication.

BACKGROUND

AI can improve patient education by providing personalised, interactive, and accessible information tailored to patient understanding, interest, and motivation. AI chatbots can have human-like conversations and give advice on numerous topics.

METHODS

This study investigated the efficacy of chatbots in cataract patient education relative to traditional resources like the AAO website, focusing on information accuracy,understandability, actionability, and readability. A descriptive comparative design was used to analyse quantitative data from frequently asked questions about cataracts answered by ChatGPT, Bard, Bing AI, and the AAO website. SOLO taxonomy, PEMAT, and the Flesch-Kincaid ease score were used to collect and analyse the data.

RESULTS

Chatbots scored higher than AAO website on cataract-related questions in terms of accuracy (mean SOLO score ChatGPT: 3.1 ± 0.31, Bard: 2.9 ± 0.72, Bing AI: 2.65 ± 0.49, AAO website: 2.4 ± 0.6, (p < 0.001)). For understandability (mean PEMAT-U score AAO website: 0,89 ± 0,04, ChatGPT 0,84 ± 0,02, Bard: 0,84 ± 0,02, Bing AI: 0,81 ± 0,02, (p < 0.001)), and actionability (mean PEMAT-A score ChatGPT: 0.86 ± 0.03, Bard: 0.85 ± 0.06, Bing AI: 0.81 ± 0.05, AAO website: 0.81 ± 0.06, (p < 0.001)) AAO website scored better than chatbots. Flesch-Kincaid readability ease analysis showed that Bard (55,5 ± 8,48) had the highest mean score, followed by AAO website (51,96 ± 12,46), Bing AI (41,77 ± 9,53), and ChatGPT (34,38 ± 9,75, (p < 0.001)).

CONCLUSION

Chatbots have the potential to provide more detailed and accurate data than the AAO website. On the other hand, the AAO website has the advantage of providing information that is more understandable and practical. When patient preferences are not taken into account, generalised or biased information can decrease reliability.

Potential of AI-Driven Chatbots in Urology: Revolutionizing Patient Care Through Artificial Intelligence

PURPOSE OF REVIEW

Artificial intelligence (AI) chatbots have emerged as a potential tool to transform urology by improving patient care and physician efficiency. With an emphasis on their potential advantages and drawbacks, this literature review offers a thorough assessment of the state of AI-driven chatbots in urology today.

RECENT FINDINGS

The capacity of AI-driven chatbots in urology to give patients individualized and timely medical advice is one of its key advantages. Chatbots can help patients prioritize their symptoms and give advice on the best course of treatment. By automating administrative duties and offering clinical decision support, chatbots can also help healthcare providers. Before chatbots are widely used in urology, there are a few issues that need to be resolved. The precision of chatbot diagnoses and recommendations might be impacted by technical constraints like system errors and flaws. Additionally, issues regarding the security and privacy of patient data must be resolved, and chatbots must adhere to all applicable laws. Important issues that must be addressed include accuracy and dependability because any mistakes or inaccuracies could seriously harm patients. The final obstacle is resistance from patients and healthcare professionals who are hesitant to use new technology or who value in-person encounters. AI-driven chatbots have the potential to significantly improve urology care and efficiency. However, it is essential to thoroughly test and ensure the accuracy of chatbots, address privacy and security concerns, and design user-friendly chatbots that can integrate into existing workflows. By exploring various scenarios and examining the current literature, this review provides an analysis of the prospects and limitations of implementing chatbots in urology.

Multilayer framework for digital multicomponent platform design for colorectal survivors and carers: a qualitative study

Background

The advent of eHealth services offers the potential to support colorectal cancer (CRC) survivors and their informal caregivers (ICs), yet research into user needs and design requirements remains scant. This exploratory qualitative study addresses this knowledge gap by focusing on the development of a Digital Multicomponent Platform (DMP) designed to provide comprehensive support to these populations.

Aims

The objective of this research is to use qualitative methodologies to identify key user needs and design requirements for eHealth services. It seeks to propose and apply a multi-tiered framework for creating a DMP that encapsulates the needs of CRC survivors and their ICs.

Methods

Skype-based focus groups (FGs) were utilized to gather qualitative data from CRC survivors and ICs. This approach served to elicit crucial themes integral to the design of the DMP. A multi-tiered framework was subsequently developed to integrate user-centered design (UCD) principles and requirements with predetermined outcomes, eHealth services, and IT infrastructure.

Results

The first stage of the analysis identified five crucial themes: (1) the importance of healthcare system interaction via eHealth, (2) interaction between healthcare providers and peers, (3) lifestyle and wellness considerations, (4) platform content and user interface requirements, (5) caregiver support. The second stage analysis applied the multi-tiered framework, to determine the DMP that was conceptualized from these themes, underscores the significance of personalized content, caregiver involvement, and integration with electronic health records (EHRs).

Conclusion

The study offers novel insights into the design and development of digital supportive care interventions for CRC survivors and their caregivers. The results highlight the utility of user-centered design principles, the significance of personalized content and caregiver involvement, and the need for a unified health data platform that promotes communication among patients, healthcare providers, and peers. This multi-tiered framework could serve as a prototype for future eHealth service designs.

Public Awareness of Obesity as a Risk Factor for Cancer in Central Saudi Arabia: Feasibility of ChatGPT as an Educational Intervention

BACKGROUND

While the link between obesity and chronic diseases such as diabetes and cardiovascular disorders is well-documented, there is a growing body of evidence connecting obesity with an increased risk of cancer. However, public awareness of this connection remains limited.

STUDY PURPOSE

To analyze public awareness of overweight/obesity as a risk factor for cancer and analyze public perceptions on the feasibility of ChatGPT, an artificial intelligence-based conversational agent, as an educational intervention tool.

METHODS

A mixed-methods approach including deductive quantitative cross-sectional approach to draw precise conclusions based on empirical evidence on public awareness of the link between obesity and cancer; and inductive qualitative approach to interpret public perceptions on using ChatGPT for creating awareness of obesity, cancer and its risk factors was used in this study. Participants included adult residents in Saudi Arabia. A total of 486 individuals and 21 individuals were included in the survey and semi-structured interviews respectively.

RESULTS

About 65% of the participants are not completely aware of cancer and its risk factors. Significant differences in awareness were observed concerning age groups (p < .0001), socio-economic status (p = .041), and regional distribution (p = .0351). A total of 10 themes were analyzed from the interview data, which included four positive factors (accessibility, personalization, cost-effectiveness, anonymity and privacy, multi-language support) and five negative factors (information inaccuracy, lack of emotional intelligence, dependency and overreliance, data privacy and security, and inability to provide physical support or diagnosis).

CONCLUSION

This study has underscored the potential of leveraging ChatGPT as a valuable public awareness tool for cancer in Saudi Arabia.

Enhancing Prostate Cancer Diagnosis with a Novel Artificial Intelligence-Based Web Application: Synergizing Deep Learning Models, Multimodal Data, and Insights from Usability Study with Pathologists

Prostate cancer remains a significant cause of male cancer mortality in the United States, with an estimated 288,300 new cases in 2023. Accurate grading of prostate cancer is crucial for ascertaining disease severity and shaping treatment strategies. Modern deep learning techniques show promise in grading biopsies, but there is a gap in integrating these advances into clinical practice. Our web platform tackles this challenge by integrating human expertise with AI-driven grading, incorporating diverse data sources. We gathered feedback from four pathologists and one medical practitioner to assess usability and real-world alignment through a survey and the NASA TLX Usability Test. Notably, 60% of users found it easy to navigate, rating it 5.5 out of 7 for ease of understanding. Users appreciated self-explanatory information in popup tabs. For ease of use, all users favored the detailed summary tab, rating it 6.5 out of 7. While 80% felt patient demographics beyond age were unnecessary, high-resolution biopsy images were deemed vital. Acceptability was high, with all users willing to adopt the app, and some believed it could reduce workload. The NASA TLX Usability Test indicated a low-moderate perceived workload, suggesting room for improved explanations and data visualization.

Evaluating the Performance of Different Large Language Models on Health Consultation and Patient Education in Urolithiasis

OBJECTIVES

To evaluate the effectiveness of four large language models (LLMs) (Claude, Bard, ChatGPT4, and New Bing) that have large user bases and significant social attention, in the context of medical consultation and patient education in urolithiasis.

MATERIALS AND METHODS

In this study, we developed a questionnaire consisting of 21 questions and 2 clinical scenarios related to urolithiasis. Subsequently, clinical consultations were simulated for each of the four models to assess their responses to the questions. Urolithiasis experts then evaluated the model responses in terms of accuracy, comprehensiveness, ease of understanding, human care, and clinical case analysis ability based on a predesigned 5-point Likert scale. Visualization and statistical analyses were then employed to compare the four models and evaluate their performance.

RESULTS

All models yielded satisfying performance, except for Bard, who failed to provide a valid response to Question 13. Claude consistently scored the highest in all dimensions compared with the other three models. ChatGPT4 ranked second in accuracy, with a relatively stable output across multiple tests, but shortcomings were observed in empathy and human caring. Bard exhibited the lowest accuracy and overall performance. Claude and ChatGPT4 both had a high capacity to analyze clinical cases of urolithiasis. Overall, Claude emerged as the best performer in urolithiasis consultations and education.

CONCLUSION

Claude demonstrated superior performance compared with the other three in urolithiasis consultation and education. This study highlights the remarkable potential of LLMs in medical health consultations and patient education, although professional review, further evaluation, and modifications are still required.

The Intersection of ChatGPT, Clinical Medicine, and Medical Education

As we progress deeper into the digital age, the robust development and application of advanced artificial intelligence (AI) technology, specifically generative language models like ChatGPT (OpenAI), have potential implications in all sectors including medicine. This viewpoint article aims to present the authors' perspective on the integration of AI models such as ChatGPT in clinical medicine and medical education. The unprecedented capacity of ChatGPT to generate human-like responses, refined through Reinforcement Learning with Human Feedback, could significantly reshape the pedagogical methodologies within medical education. Through a comprehensive review and the authors' personal experiences, this viewpoint article elucidates the pros, cons, and ethical considerations of using ChatGPT within clinical medicine and notably, its implications for medical education. This exploration is crucial in a transformative era where AI could potentially augment human capability in the process of knowledge creation and dissemination, potentially revolutionizing medical education and clinical practice. The importance of maintaining academic integrity and professional standards is highlighted. The relevance of establishing clear guidelines for the responsible and ethical use of AI technologies in clinical medicine and medical education is also emphasized.

ChatGPTs' Journey in Medical Revolution: A Potential Panacea or a Hidden Pathogen?

At the fascinating intersection of artificial intelligence and medicine, ChatGPT morphs into a compact, personal digital physician. With a simple click, it furnishes an abundance of health-related information, initial medical consultations, and a plethora of disease management recommendations. Moreover, it stands at the ready to provide immediate mental health assistance in times of psychological distress. Yet, each innovation carries inherent challenges. As we embrace the conveniences proffered by ChatGPT, it is imperative that we grapple with associated issues such as data privacy, risk of misdiagnosis, complexities in human-machine interaction, and particular situations that elude its understanding. Let's probe further into this intriguing world, brimming with contention and prospects, and observe how ChatGPT traverses the landscape of digital health, uncovering the potential it holds for the future evolution of medical practice.

Revolutionizing healthcare: the role of artificial intelligence in clinical practice

INTRODUCTION

Healthcare systems are complex and challenging for all stakeholders, but artificial intelligence (AI) has transformed various fields, including healthcare, with the potential to improve patient care and quality of life. Rapid AI advancements can revolutionize healthcare by integrating it into clinical practice. Reporting AI's role in clinical practice is crucial for successful implementation by equipping healthcare providers with essential knowledge and tools.

RESEARCH SIGNIFICANCE

This review article provides a comprehensive and up-to-date overview of the current state of AI in clinical practice, including its potential applications in disease diagnosis, treatment recommendations, and patient engagement. It also discusses the associated challenges, covering ethical and legal considerations and the need for human expertise. By doing so, it enhances understanding of AI's significance in healthcare and supports healthcare organizations in effectively adopting AI technologies.

MATERIALS AND METHODS

The current investigation analyzed the use of AI in the healthcare system with a comprehensive review of relevant indexed literature, such as PubMed/Medline, Scopus, and EMBASE, with no time constraints but limited to articles published in English. The focused question explores the impact of applying AI in healthcare settings and the potential outcomes of this application.

RESULTS

Integrating AI into healthcare holds excellent potential for improving disease diagnosis, treatment selection, and clinical laboratory testing. AI tools can leverage large datasets and identify patterns to surpass human performance in several healthcare aspects. AI offers increased accuracy, reduced costs, and time savings while minimizing human errors. It can revolutionize personalized medicine, optimize medication dosages, enhance population health management, establish guidelines, provide virtual health assistants, support mental health care, improve patient education, and influence patient-physician trust.

CONCLUSION

AI can be used to diagnose diseases, develop personalized treatment plans, and assist clinicians with decision-making. Rather than simply automating tasks, AI is about developing technologies that can enhance patient care across healthcare settings. However, challenges related to data privacy, bias, and the need for human expertise must be addressed for the responsible and effective implementation of AI in healthcare.