Consumer Evaluation of the Quality of Online Health Information: Systematic Literature Review of Relevant Criteria and Indicators

Mitigating the influence of message features on health misinformation sharing intention in social media: Experimental evidence for accuracy-nudge intervention

RATIONALE

The escalating dissemination of health misinformation on social media platforms poses a significant threat to users' well-being. It is imperative to identify the types of health misinformation that are more susceptible to widespread dissemination and to explore strategies to curb its spread.

METHOD

This study designed a 2 (emotional appeal type: positive vs. negative) × 2 (fabricated source type: pseudo-common vs. pseudo-authoritative) × 2 (accuracy-nudge label: No vs. Yes) online between-subjects experiment controlling for factors such as e-health literacy, prior sharing experience, and personal involvement. A snowball sampling approach was used to recruit 1952 participants through social media, resulting in a final sample of 1393 valid responses.

RESULTS

Compared to positive emotional appeal and pseudo-common sources, negative emotional appeal and pseudo-authoritative sources resulted in higher levels of sharing intention. Under the condition of negative emotional appeal, the promotion effect of pseudo-authoritative sources on sharing intention was intensified. The accuracy-nudge intervention could significantly mitigate this tendency. The underlying mechanisms revealed more details: both negative emotional appeals and pseudo-authoritative sources increased the perceived credibility of health misinformation, thereby increasing users' sharing intention. However, in contrast to pseudo-authoritative sources, excessive negative emotional appeal induced vigilant verification behavior among users, which reduced sharing to some extent. Adding an accuracy-nudge label to health misinformation reduced users' misguided trust in health misinformation features and stimulated information verification, ultimately reducing health misinformation sharing intention.

CONCLUSIONS

Negative emotional appeal and pseudo-authoritative sources can enhance the perceived credibility of health misinformation, thereby strengthening the sharing intention of social media users. Therefore, health misinformation with negative emotional appeal and pseudo-authoritative sources is more likely to be widely shared. The accuracy nudge intervention can trigger users' information verification behavior, suppress the persuasive effects of the misinformation features mentioned above, and help prevent the spread of health misinformation on social media.

Bibliometric analysis confirms expected trends in consumer health information publications

BACKGROUND

This study provides an overview of consumer health information (CHI) scholarly communication trends.

OBJECTIVES

To explore CHI publications trends, journal citations, prolific authors, countries of origin, and distribution of CHI publications.

METHODS

A bibliometric analysis was used; 8953 records from the Web of Science (WoS) and peer-reviewed journal articles from databases, including LISA, MEDLINE, ERIC, PREMEDLINE and EMBASE were analysed.

RESULTS

Publications on CHI rapidly increased from 1980 to 2019, especially during the 1990s and 2000s. Most journals that have published CHI research are based in North America and Europe.

CONCLUSION

The increase in the CHI literature corresponds with that in Internet usage in the 2000s, and explains the availability of CHI content online. This trend is associated with the widespread adoption of personal computers (PCs) and other Internet-enabled gadgets.

The predation relationship between online medical search and online medical consultation-empirical research based on Baidu platform data

Introduction

This study investigates the mutual influence between online medical search and online medical consultation. It focuses on understanding the health information needs that drive these health information-seeking behaviors by utilizing insights from behavioral big data.

Methods

We used actual behavioral data from Chinese internet users on Baidu platform's "Epidemic Index" from November 26, 2022, to January 25, 2023. Data modeling was conducted to ensure the reliability of the model. Drawing on the logistic model, we constructed a foundational model to quantify the evolutionary patterns of online medical search and online medical consultation. An impact function was defined to measure their mutual influence. Additionally, a pattern detection experiment was conducted to determine the structure of the impact function with maximum commonality through data fitting.

Results

The analysis allowed us to build a mathematical model that quantifies the nonlinear correlation between online medical search and online medical consultation. Numerical analysis revealed a predation mechanism between online medical consultation and online medical search, highlighting the role of health information needs in this dynamic.

Discussion

This study offers a novel practical approach to better meet the public's health information needs by understanding the interplay between online medical search and consultation. Additionally, the modeling method used here is broadly applicable, providing a framework for quantifying nonlinear correlations among different behaviors when appropriate data is available.

Video quality of nonalcoholic fatty liver disease on TikTok: A cross-sectional study

The short-video application TikTok shows great potential for disseminating health information. We assessed the content, sources, and quality of information in videos related to nonalcoholic fatty liver disease (NAFLD) on TikTok. Our study aims to identify upload sources, content, and characteristic information for NAFLD videos on TikTok and further evaluate factors related to video quality. We investigated the top 100 videos related to NAFLD on TikTok and analyzed the upload sources, content, and characteristics of these videos. Evaluate video quality using the DISCERN tool and Global Quality Score (GQS). In addition, the correlation between video quality and video characteristics is further studied. In terms of video sources, the majority of NAFLD videos on TikTok (85/100, 85%) were posted by doctors, ensuring the professionalism of the content, and among the video content, disease knowledge was the most dominant video content, accounting for 57% (57/100) of all videos, and the average DISCERN and GQS scores of all 100 videos were 39.59 (SD 3.31) and 2.99 (SD 0.95), respectively. DISCERN and GQS data show that videos related to NAFLD do not have high-quality scores on TikTok, mainly fair (68/100, 68%) and moderate (49/100, 49%). In general, the quality of NAFLD video information from professional content and professional sources was higher than that of nonprofessional sources and nonprofessional content, the video quality of general surgeons was better than that of other department physicians, and the video quality of junior physicians was better than that of senior physicians. In terms of video correlation, durations, the number of fans, and the total number of works were negatively correlated with DISCERN scores (R < 0, P < .05), while likes, comments, collections, shares, and days since upload were not significantly correlated with DISCERN and GQS scores (P > .05). The medical information on TikTok is not rigorous enough to guide patients to make accurate judgments, platforms should monitor and guide publishers to help promote and disseminate quality content.

Do ChatGPT and Google differ in answers to commonly asked patient questions regarding total shoulder and total elbow arthroplasty?

BACKGROUND

Artificial intelligence (AI) and large language models (LLMs) offer a new potential resource for patient education. The answers by Chat Generative Pre-Trained Transformer (ChatGPT), a LLM AI text bot, to frequently asked questions (FAQs) were compared to answers provided by a contemporary Google search to determine the reliability of information provided by these sources for patient education in upper extremity arthroplasty.

METHODS

"Total shoulder arthroplasty" (TSA) and "total elbow arthroplasty" (TEA) were entered into Google Search and ChatGPT 3.0 to determine the ten most FAQs. On Google, the FAQs were obtained through the "people also ask" section, while ChatGPT was asked to provide the ten most FAQs. Each question, answer, and reference(s) cited were recorded. A modified version of the Rothwell system was used to categorize questions into 10 subtopics: special activities, timeline of recovery, restrictions, technical details, cost, indications/management, risks and complications, pain, longevity, and evaluation of surgery. Each reference was categorized into the following groups: commercial, academic, medical practice, single surgeon personal, or social media. Questions for TSA and TEA were combined for analysis and compared between Google and ChatGPT with a 2 sample Z-test for proportions.

RESULTS

Overall, most questions were related to procedural indications or management (17.5%). There were no significant differences between Google and ChatGPT between question categories. The majority of references were from academic websites (65%). ChatGPT produced a greater number of academic references compared to Google (80% vs. 50%; P = .047), while Google more commonly provided medical practice references (25% vs. 0%; P = .017).

CONCLUSION

In conjunction with patient-physician discussions, AI LLMs may provide a reliable resource for patients. By providing information based on academic references, these tools have the potential to improve health literacy and improved shared decision making for patients searching for information about TSA and TEA.

CLINICAL SIGNIFICANCE

With the rising prevalence of AI programs, it is essential to understand how these applications affect patient education in medicine.

Readability, Understandability, Usability, and Cultural Sensitivity of Online Patient Educational Materials (PEMs) for Lower Extremity Reconstruction: A Cross-Sectional Study

Background: Lower extremity reconstructive surgery is an evolving field wherein patients rely on accessible online materials to engage with their perioperative care. This study furthers existing research in this area by evaluating the readability, understandability, actionability, and cultural sensitivity of online health materials for lower extremity reconstruction. Methods: We identified the 10 first-appearing, educational sites found by searching the phrases "leg saving surgery", "limb salvage surgery," and "leg reconstruction surgery". Readability analysis was conducted with validated tools, including Simple Measure of Gobbledygook (SMOG). Understandability and actionability were assessed with Patient Education and Materials Assessment Tool (PEMAT), while cultural sensitivity was measured with Cultural Sensitivity Assessment Tool (CSAT). A Cohen's κ value was calculated (PEMAT and CSAT analyses) for inter-rater agreement. Results: The mean SMOG reading level for websites was 13.12 (college-freshman reading level). The mean PEMAT understandability score was 61.8% and actionability score was 26.0% (κ = 0.8022), both below the 70% acceptability threshold. The mean CSAT score was 2.6 (κ = 0.73), exceeding the 2.5 threshold for cultural appropriateness. Conclusion: Online PEM for lower extremity reconstruction continue to fall below standards of readability, understandability, and actionability; however, they meet standards of cultural appropriateness. As patients rely on these materials, creators can use validated tools and positive examples from existing PEM for greater patient accessibility.

Development of a community intervention combining social media-based health education plus exercise programme (SHEEP) to improve muscle function among young-old adults with possible sarcopenia: Co-design approach

OBJECTIVES

There is no precedent for the use of social media in preventing sarcopenia. The aim of this study is to develop a social media-based intervention programme for the young-old population in the community in China to improve their awareness and behaviours regarding sarcopenia prevention.

STUDY DESIGN

Using guidelines for developing complex interventions, this study was divided into two main phases: a co-development phase and a preliminary test phase. Both were carried out in Changsha, China. The development phase employed co-design methodology with relevant stakeholders, including two rounds of consultation with patient and public involvement (12 members) and two rounds of focus groups (30 participants); this was followed by the three-week preliminary test phase (22 participants).

MAIN OUTCOME MEASURES

This study evaluated the consultation with patient and public involvement, and mainly collected qualitative data from the two rounds of focus group interviews and a final semi-structured interview following the preliminary test, so as to explore the participants' experiences, comments, and suggestions for revising the social media-based intervention. Handgrip strength was also evaluated.

RESULTS

The health education included seven videos of 4-6 min each related to sarcopenia, including information on the concept, influencing factors, adverse effects, manifestations, screening methods, and preventions. The exercise video consisted of four types of training (warm-up, aerobic, resistance, and flexibility training) and lasted 30 min, with a suggested engagement of at least 3 days/week. The specific contents and "dosage" of the final intervention were unanimously favourable to the diverse stakeholders involved (older adults with possible sarcopenia, experts, researchers). After the preliminary test, an improvement in handgrip strength was observed, from M15.92±SD5.22 kg to M19.13±SD5.44 kg (T = -5.44, P < 0.001). Subgroup analysis revealed that this improvement was evident in both men and women.

CONCLUSIONS

The social media-based intervention was universally endorsed by the participants and showed indications of a positive influence on sarcopenia. A feasibility study is now needed.

An evaluation of physical medicine and rehabilitation accredited sports medicine fellowship websites

BACKGROUND

Due to the virtual format of fellowship applications, prospective physical medicine and rehabilitation (PM&R) sports medicine fellowship applicants commonly rely upon fellowship website content to make decisions about applications. Studies have shown that information available to fellowship applicants on program websites is inadequate, and there is no standard for important components of sports medicine fellowship websites.

OBJECTIVE

To determine what sports medicine fellowship program features PM&R residents, fellows, and attending physicians consider the most important to be listed on fellowship websites, as well as to assess the scope of information available on fellowship websites.

DESIGN

We developed a 30-item list of criteria that fell into categories of general information, fellowship education, recruitment, experience, and academic research. Survey participants were asked to rank the importance of those various items. We then assessed the 21 PM&R accredited sports medicine fellowship websites for the presence of these criteria.

PARTICIPANTS

Ninety-six survey respondents composed of PM&R residents, current sports medicine fellows, and attendings.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURE(S)

Not applicable.

RESULTS

Items ranked as most important were those in the categories of general information, fellowship experience, and fellowship recruitment. The most valued items were program location, program coordinator contact information, program length, specific requirements for applying, number of positions, types of procedures taught, and specific sites covered. Academic research items were ranked as least important.

CONCLUSIONS

Survey respondents identified many fellowship website items as important factors when applying to programs. Many of these items were not listed on fellowship websites, suggesting that adding these criteria to fellowship sites might benefit applicants. Our findings can be used to develop standardized criteria for important components of sports medicine fellowship websites to improve the application and recruitment process.

Assessing the Quality of ChatGPT's Dietary Advice for College Students from Dietitians' Perspectives

BACKGROUND

As ChatGPT becomes a primary information source for college students, its performance in providing dietary advice is under scrutiny. This study assessed ChatGPT's performance in providing nutritional guidance to college students.

METHODS

ChatGPT's performance on dietary advice was evaluated by 30 experienced dietitians and assessed using an objective nutrition literacy (NL) test. The dietitians were recruited to assess the quality of ChatGPT's dietary advice, including its NL achievement and response quality.

RESULTS

The results indicate that ChatGPT's performance varies across scenarios and is suboptimal for achieving NL with full achievement rates from 7.50% to 37.56%. While the responses excelled in readability, they lacked understandability, practicality, and completeness. In the NL test, ChatGPT showed an 84.38% accuracy rate, surpassing the NL level of Taiwanese college students. The top concern among the dietitians, cited 52 times in 242 feedback entries, was that the "response information lacks thoroughness or rigor, leading to misunderstandings or misuse". Despite the potential of ChatGPT as a supplementary educational tool, significant gaps must be addressed, especially in detailed dietary inquiries.

CONCLUSION

This study highlights the need for improved AI educational approaches and suggests the potential for developing ChatGPT teaching guides or usage instructions to train college students and support dietitians.

Evaluating ChatGPT's Utility in Medicine Guidelines Through Web Search Analysis

INTRODUCTION

With the rise of machine learning applications in health care, shifts in medical fields that rely on precise prognostic models and pattern detection tools are anticipated in the near future. Chat Generative Pretrained Transformer (ChatGPT) is a recent machine learning innovation known for producing text that mimics human conversation. To gauge ChatGPT's capability in addressing patient inquiries, the authors set out to juxtapose it with Google Search, America's predominant search engine. Their comparison focused on: 1) the top questions related to clinical practice guidelines from the American Academy of Family Physicians by category and subject; 2) responses to these prevalent questions; and 3) the top questions that elicited a numerical reply.

METHODS

Utilizing a freshly installed Google Chrome browser (version 109.0.5414.119), the authors conducted a Google web search (www.google.com) on March 4, 2023, ensuring minimal influence from personalized search algorithms. Search phrases were derived from the clinical guidelines of the American Academy of Family Physicians. The authors prompted ChatGPT with: "Search Google using the term '(refer to search terms)' and document the top four questions linked to the term." The same 25 search terms were employed. The authors cataloged the primary 4 questions and their answers for each term, resulting in 100 questions and answers.

RESULTS

Of the 100 questions, 42% (42 questions) were consistent across all search terms. ChatGPT predominantly sourced from academic (38% vs 15%, p = 0.0002) and government (50% vs 39%, p = 0.12) domains, whereas Google web searches leaned toward commercial sources (32% vs 11%, p = 0.0002). Thirty-nine percent (39 questions) of the questions yielded divergent answers between the 2 platforms. Notably, 16 of the 39 distinct answers from ChatGPT lacked a numerical reply, instead advising a consultation with a medical professional for health guidance.

CONCLUSION

Google Search and ChatGPT present varied questions and answers for both broad and specific queries. Both patients and doctors should exercise prudence when considering ChatGPT as a digital health adviser. It's essential for medical professionals to assist patients in accurately communicating their online discoveries and ensuing inquiries for a comprehensive discussion.

ChatGPT-4 Generates More Accurate and Complete Responses to Common Patient Questions About Anterior Cruciate Ligament Reconstruction Than Google's Search Engine

Purpose

To replicate a patient's internet search to evaluate ChatGPT's appropriateness in answering common patient questions about anterior cruciate ligament reconstruction compared with a Google web search.

Methods

A Google web search was performed by searching the term "anterior cruciate ligament reconstruction." The top 20 frequently asked questions and responses were recorded. The prompt "What are the 20 most popular patient questions related to 'anterior cruciate ligament reconstruction?'" was input into ChatGPT and questions and responses were recorded. Questions were classified based on the Rothwell system and responses assessed via Flesch-Kincaid Grade Level, correctness, and completeness were for both Google web search and ChatGPT.

Results

Three of 20 (15%) questions were similar between Google web search and ChatGPT. The most common question types among the Google web search were value (8/20, 40%), fact (7/20, 35%), and policy (5/20, 25%). The most common question types amongst the ChatGPT search were fact (12/20, 60%), policy (6/20, 30%), and value (2/20, 10%). Mean Flesch-Kincaid Grade Level for Google web search responses was significantly lower (11.8 ± 3.8 vs 14.3 ± 2.2; P = .003) than for ChatGPT responses. The mean correctness for Google web search question answers was 1.47 ± 0.5, and mean completeness was 1.36 ± 0.5. Mean correctness for ChatGPT answers was 1.8 ± 0.4 and mean completeness was 1.9 ± 0.3, which were both significantly greater than Google web search answers (P = .03 and P = .0003).

Conclusions

ChatGPT-4 generated more accurate and complete responses to common patient questions about anterior cruciate ligament reconstruction than Google's search engine.

Clinical Relevance

The use of artificial intelligence such as ChatGPT is expanding. It is important to understand the quality of information as well as how the results of ChatGPT queries compare with those from Google web searches.

Using Google web search to analyze and evaluate the application of ChatGPT in femoroacetabular impingement syndrome

Background

Chat Generative Pre-trained Transformer (ChatGPT) is a new machine learning tool that allows patients to access health information online, specifically compared to Google, the most commonly used search engine in the United States. Patients can use ChatGPT to better understand medical issues. This study compared the two search engines based on: (i) frequently asked questions (FAQs) about Femoroacetabular Impingement Syndrome (FAI), (ii) the corresponding answers to these FAQs, and (iii) the most FAQs yielding a numerical response.

Purpose

To assess the suitability of ChatGPT as an online health information resource for patients by replicating their internet searches.

Study design

Cross-sectional study.

Methods

The same keywords were used to search the 10 most common questions about FAI on both Google and ChatGPT. The responses from both search engines were recorded and analyzed.

Results

Of the 20 questions, 8 (40%) were similar. Among the 10 questions searched on Google, 7 were provided by a medical practice. For numerical questions, there was a notable difference in answers between Google and ChatGPT for 3 out of the top 5 most common questions (60%). Expert evaluation indicated that 67.5% of experts were satisfied or highly satisfied with the accuracy of ChatGPT's descriptions of both conservative and surgical treatment options for FAI. Additionally, 62.5% of experts were satisfied or highly satisfied with the safety of the information provided. Regarding the etiology of FAI, including cam and pincer impingements, 52.5% of experts expressed satisfaction or high satisfaction with ChatGPT's explanations. Overall, 62.5% of experts affirmed that ChatGPT could serve effectively as a reliable medical resource for initial information retrieval.

Conclusion

This study confirms that ChatGPT, despite being a new tool, shows significant potential as a supplementary resource for health information on FAI. Expert evaluations commend its capacity to provide accurate and comprehensive responses, valued by medical professionals for relevance and safety. Nonetheless, continuous improvements in its medical content's depth and precision are recommended for ongoing reliability. While ChatGPT offers a promising alternative to traditional search engines, meticulous validation is imperative before it can be fully embraced as a trusted medical resource.

Enabling Health Information Recommendation Using Crowdsourced Refinement in Web-Based Health Information Applications: User-Centered Design Approach and EndoZone Informatics Case Study

BACKGROUND

In the digital age, search engines and social media platforms are primary sources for health information, yet their commercial interests-focused algorithms often prioritize irrelevant content. Web-based health applications by reputable sources offer a solution to circumvent these biased algorithms. Despite this advantage, there remains a significant gap in research on the effective integration of content-ranking algorithms within these specialized health applications to ensure the delivery of personalized and relevant health information.

OBJECTIVE

This study introduces a generic methodology designed to facilitate the development and implementation of health information recommendation features within web-based health applications.

METHODS

We detail our proposed methodology, covering conceptual foundation and practical considerations through the stages of design, development, operation, review, and optimization in the software development life cycle. Using a case study, we demonstrate the practical application of the proposed methodology through the implementation of recommendation functionalities in the EndoZone platform, a platform dedicated to providing targeted health information on endometriosis.

RESULTS

Application of the proposed methodology in the EndoZone platform led to the creation of a tailored health information recommendation system known as EndoZone Informatics. Feedback from EndoZone stakeholders as well as insights from the implementation process validate the methodology's utility in enabling advanced recommendation features in health information applications. Preliminary assessments indicate that the system successfully delivers personalized content, adeptly incorporates user feedback, and exhibits considerable flexibility in adjusting its recommendation logic. While certain project-specific design flaws were not caught in the initial stages, these issues were subsequently identified and rectified in the review and optimization stages.

CONCLUSIONS

We propose a generic methodology to guide the design and implementation of health information recommendation functionality within web-based health information applications. By harnessing user characteristics and feedback for content ranking, this methodology enables the creation of personalized recommendations that align with individual user needs within trusted health applications. The successful application of our methodology in the development of EndoZone Informatics marks a significant progress toward personalized health information delivery at scale, tailored to the specific needs of users.

Investigating Patient Satisfaction Through Web-Based Reviews of Norwegian Dentists: Quantitative Study Using the Meaning Extraction Method

BACKGROUND

Challenging encounters in health care professions, including in dentistry, are relatively common. Challenging encounters can be defined as stressful or emotional situations involving patients that could impact both treatment outcomes and patients' experiences. Through written web-based reviews, patients can share their experiences with health care providers, and these posts can be a useful source for investigating patient satisfaction and their experiences of challenging encounters.

OBJECTIVE

This study aims to identify dominant themes from patient-written, web-based reviews of dentists and investigate how these themes are related to patient satisfaction with dental treatment.

METHODS

The study data consisted of 11,764 reviews written by dental patients, which included 1- to 5-star ratings on overall satisfaction and free-text comments. The free-text comments were analyzed using Linguistic Inquiry and Word Count software, and the meaning extraction method was used to group words into thematic categories. These themes were used as variables in a multilevel logistic regression analysis to predict patient satisfaction.

RESULTS

Eight themes emerged from the analyses, of which 6 (75%)-explanation (odds ratio [OR] 2.56, 95% CI 2.16-3.04; P<.001), assurance (OR 3.61, 95% CI 2.57-5.06; P<.001), performance assessment (OR 2.17, 95% CI 1.84-2.55; P<.001), professional advice (OR 1.81, 95% CI 1.55-2.13; P<.001), facilities (OR 1.78, 95% CI 1.08-2.91; P=.02), and recommendation (OR 1.31, 95% CI 1.12-1.53; P<.001)-increased the odds of high patient satisfaction. The remaining themes (2/8, 25%)-consequences of treatment need (OR 0.24, 95% CI 0.20-0.29; P<.001) and patient-centered care (OR 0.62, 95% CI 0.52-0.74; P<.001)-reduced the odds of high patient satisfaction.

CONCLUSIONS

The meaning extraction method is an interesting approach to explore patients' written accounts of encounters with dental health professionals. The experiences described by patients provide insight into key elements related to patient satisfaction that can be used in the education of dental health professionals and to improve the provision of dental health services.

Chat Generative Pretrained Transformer (ChatGPT) and Bard: Artificial Intelligence Does not yet Provide Clinically Supported Answers for Hip and Knee Osteoarthritis

BACKGROUND

Advancements in artificial intelligence (AI) have led to the creation of large language models (LLMs), such as Chat Generative Pretrained Transformer (ChatGPT) and Bard, that analyze online resources to synthesize responses to user queries. Despite their popularity, the accuracy of LLM responses to medical questions remains unknown. This study aimed to compare the responses of ChatGPT and Bard regarding treatments for hip and knee osteoarthritis with the American Academy of Orthopaedic Surgeons (AAOS) Evidence-Based Clinical Practice Guidelines (CPGs) recommendations.

METHODS

Both ChatGPT (Open AI) and Bard (Google) were queried regarding 20 treatments (10 for hip and 10 for knee osteoarthritis) from the AAOS CPGs. Responses were classified by 2 reviewers as being in "Concordance," "Discordance," or "No Concordance" with AAOS CPGs. A Cohen's Kappa coefficient was used to assess inter-rater reliability, and Chi-squared analyses were used to compare responses between LLMs.

RESULTS

Overall, ChatGPT and Bard provided responses that were concordant with the AAOS CPGs for 16 (80%) and 12 (60%) treatments, respectively. Notably, ChatGPT and Bard encouraged the use of non-recommended treatments in 30% and 60% of queries, respectively. There were no differences in performance when evaluating by joint or by recommended versus non-recommended treatments. Studies were referenced in 6 (30%) of the Bard responses and none (0%) of the ChatGPT responses. Of the 6 Bard responses, studies could only be identified for 1 (16.7%). Of the remaining, 2 (33.3%) responses cited studies in journals that did not exist, 2 (33.3%) cited studies that could not be found with the information given, and 1 (16.7%) provided links to unrelated studies.

CONCLUSIONS

Both ChatGPT and Bard do not consistently provide responses that align with the AAOS CPGs. Consequently, physicians and patients should temper expectations on the guidance AI platforms can currently provide.

Readability and Quality of Online Information on Sickle Cell Retinopathy for Patients

PURPOSE

This study aims to evaluate the readability and quality of Internet-based health information on sickle cell retinopathy.

DESIGN

Retrospective cross-sectional website analysis.

METHODS

To simulate a patient's online search, the terms "sickle cell retinopathy" and "sickle cell disease in the eye" were entered into the top 3 search engines (Google, Bing and Yahoo). The first 20 results of each search were retrieved and screened for analysis. The DISCERN questionnaire, the Journal of the American Medical Association (JAMA) standards, and the Health on the Net (HON) criteria were used to evaluate the quality of the information. The Flesch-Kincaid Grade Level (FKGL), the Flesch Reading Ease (FRES), and the Automated Readability Index (ARI) were used to assess the readability of each website.

RESULTS

Of 16 online sources, 12 (75%) scored moderately on the DISCERN tool. The mean DISCERN score was 40.91 (SD, 10.39; maximum possible, 80). None of the sites met all of the JAMA benchmarks, and only 3 (18.75%) of the websites had HONcode certification. All of the websites had scores above the target American Medical Association grade level of 6 on both the FKGL and ARI. The mean FRES was 57.76 (±4.61), below the recommended FRES of 80 to 90.

CONCLUSION

There is limited online information available on sickle cell retinopathy. Most included websites were fairly difficult to read and of substandard quality. The quality and readability of Internet-based, patient-focused information on sickle cell retinopathy needs to be improved.

Adapting and Evaluating an AI-Based Chatbot Through Patient and Stakeholder Engagement to Provide Information for Different Health Conditions: Master Protocol for an Adaptive Platform Trial (the MARVIN Chatbots Study)

BACKGROUND

Artificial intelligence (AI)-based chatbots could help address some of the challenges patients face in acquiring information essential to their self-health management, including unreliable sources and overburdened health care professionals. Research to ensure the proper design, implementation, and uptake of chatbots is imperative. Inclusive digital health research and responsible AI integration into health care require active and sustained patient and stakeholder engagement, yet corresponding activities and guidance are limited for this purpose.

OBJECTIVE

In response, this manuscript presents a master protocol for the development, testing, and implementation of a chatbot family in partnership with stakeholders. This protocol aims to help efficiently translate an initial chatbot intervention (MARVIN) to multiple health domains and populations.

METHODS

The MARVIN chatbots study has an adaptive platform trial design consisting of multiple parallel individual chatbot substudies with four common objectives: (1) co-construct a tailored AI chatbot for a specific health care setting, (2) assess its usability with a small sample of participants, (3) measure implementation outcomes (usability, acceptability, appropriateness, adoption, and fidelity) within a large sample, and (4) evaluate the impact of patient and stakeholder partnerships on chatbot development. For objective 1, a needs assessment will be conducted within the setting, involving four 2-hour focus groups with 5 participants each. Then, a co-construction design committee will be formed with patient partners, health care professionals, and researchers who will participate in 6 workshops for chatbot development, testing, and improvement. For objective 2, a total of 30 participants will interact with the prototype for 3 weeks and assess its usability through a survey and 3 focus groups. Positive usability outcomes will lead to the initiation of objective 3, whereby the public will be able to access the chatbot for a 12-month real-world implementation study using web-based questionnaires to measure usability, acceptability, and appropriateness for 150 participants and meta-use data to inform adoption and fidelity. After each objective, for objective 4, focus groups will be conducted with the design committee to better understand their perspectives on the engagement process.

RESULTS

From July 2022 to October 2023, this master protocol led to four substudies conducted at the McGill University Health Centre or the Centre hospitalier de l'Université de Montréal (both in Montreal, Quebec, Canada): (1) MARVIN for HIV (large-scale implementation expected in mid-2024), (2) MARVIN-Pharma for community pharmacists providing HIV care (usability study planned for mid-2024), (3) MARVINA for breast cancer, and (4) MARVIN-CHAMP for pediatric infectious conditions (both in preparation, with development to begin in early 2024).

CONCLUSIONS

This master protocol offers an approach to chatbot development in partnership with patients and health care professionals that includes a comprehensive assessment of implementation outcomes. It also contributes to best practice recommendations for patient and stakeholder engagement in digital health research.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05789901; https://classic.clinicaltrials.gov/ct2/show/NCT05789901.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/54668.

Exploring Online Health Information Seeking Behaviour (OHISB) among young adults: A scoping review protocol

INTRODUCTION

In recent years, information technology and social media have experienced unprecedented growth, particularly in the Nordic countries. However, there is a noticeable lack of comprehensive understanding regarding the latest research findings on online health information seeking behaviour (OHISB) among young adults (18 to >30). There is a need to conduct an updated review to identify knowledge gaps in where young adults find health information and their user interface preferences and to provide research-based guidance and recommendations to governments, health organisations and social media platforms on how to facilitate this prominent pattern. The scoping review protocol outlines a study that will systematically map the existing literature on young adults' preferences for digital platforms and platform characteristics in relation to OHISB, enabling the identification of promising areas for further research and the development of more effective interventions to promote healthy and informed choices. Conducting a scoping review is imperative to gain a comprehensive understanding of young adults' OHISB and support the next generation of dissemination that promotes accurate and reliable digital health information.

METHODS AND ANALYSIS

The scoping review will use Arksey and O'Malley's methodological framework (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR)) and employ the citation pearl method and the Sample, Phenomenon of Interest, Design, Evaluation, Research type model to design the search strategy. To identify relevant literature, three databases will undergo a search: Scopus, Web of Science and EMBASE. Additionally, a subsidiarily grey literature search will be conducted in Google Scholar. The data charting process will conform to the PRISMA-ScR standard and will be further structured with EndNote. Qualitative and quantitative analyses of the extracted data will be developed using EndNote and Excel.

ETHICS AND DISSEMINATION

Conducting a scoping review involves secondary data analysis of publicly available sources and does not require an ethical review. The protocol will be published to ensure transparency. The scoping review results will be disseminated through open-access peer-reviewed publications, national and international conferences, social media platforms, newspapers and YouTube to service users and stakeholders.

Dietary and herbal supplement consumer health information for pain: A cross-sectional survey and quality assessment of online content

Background

Patients are increasingly utilizing the internet to learn about dietary and herbal supplements (DHSs) for various diseases/conditions, including pain management. Online health information has been found to be inconsistent and of poor quality in prior studies, which may have detrimental effects on patient health. This study assessed the quality of online DHSs consumer health information for pain.

Methods

Six search items related to DHSs and pain were used to generate the first 20 websites on Google across four English-speaking countries. The identified 480 webpages produced 68 eligible websites, which were then evaluated using the DISCERN tool. The mean scores and standard deviations (SD) of the reviewers' ratings on each of the 15 DISCERN instrument items as well as the overall total score were calculated.

Results

The mean summed score for the 68 eligible websites was 46.6 (SD = 10.1), and the mean overall rating was 3.3 (SD = 0.8). Websites lacked information regarding areas of uncertainty, the effects of no treatment being used, and how treatments affect the overall quality of life. These shortcomings were especially apparent across commercial websites, which frequently displayed bias, failed to report the risks of DHS products, and lacked support for shared decision-making regarding the use of DHSs.

Conclusion

Variability exists in the quality of online consumer health information regarding DHS use for pain. Healthcare providers should be aware of and provide guidance to patients regarding the identification of reliable online resources so that they can make informed decisions about DHS use for pain management.

Health literacy and patient rights awareness in ethnic Hungarian mothers in Eastern Europe

OBJECTIVES

The paper proposes to identify the determinants of patients' rights awareness in mothers and to examine the relationship of health literacy with awareness of those rights.

METHODS

Our results are based on data from a convenience sample of 894 non-health professional ethnic Hungarian mothers from Hungary, Romania and Slovakia. Health literacy is measured with the HLS-EU-16 questionnaire.

RESULTS

Analysis of variance reveals a significant association of health literacy with patient rights awareness. Our results show that health literacy is the highest among patients who filed a complaint through formal channels and/or took legal measures to restore their rights upon violation. A logistic regression model is built to identify the likelihood of having high patient rights awareness, that is, acting formally for the restoration of rights upon infringement. The model controls for covariates. When controlled for covariates, the likelihood of having high patient rights awareness increases with age, and is higher for mothers with highest education, for inhabitants of larger towns, as well as for those with adequate health literacy.

CONCLUSIONS

The findings of our study have implications for health policy, as they reveal significant inequalities in patient rights culture.

What generative AI means for trust in health communications

ABSTRACTLarge language models are fundamental technologies used in interfaces like ChatGPT and are poised to change the way people access and make sense of health information. The speed of uptake and investment suggests that these will be transformative technologies, but it is not yet clear what the implications might be for health communications. In this viewpoint, we draw on research about the adoption of new information technologies to focus on the ways that generative artificial intelligence (AI) tools like large language models might change how health information is produced, what health information people see, how marketing and misinformation might be mixed with evidence, and what people trust. We conclude that transparency and explainability in this space must be carefully considered to avoid unanticipated consequences.

Patient centeredness, independent health-related Internet use, and online communication with healthcare providers in later life: A cross-sectional study

OBJECTIVE

To test the links between patient centeredness, independent health-related Internet use, and online communication with healthcare provider in later life.

METHODS

These links were tested on a multivariable level through the prism of the socioemotional selectivity theory and the channel complementarity theory. The data were obtained from the Health Information National Trends Survey (Wave 5, Cycle 4) and analyzed using logistic regression models. The sample included older Internet users (N = 1165).

RESULTS

Low patient centeredness corresponded to a decreased likelihood of communicating with healthcare providers online. No particular patient centeredness component was associated with the studied phenomenon. Online health information seeking corresponded to an increased likelihood of communicating with healthcare providers online.

CONCLUSION

Patient centeredness plays a relatively modest role in explanation of the online communication with healthcare providers in later life. In addition, older adults' online health information seeking behavior and online patient-provider communication seem to complement each other.

PRACTICE IMPLICATIONS

The findings can serve public health officials for developing programs aimed at increasing the rates of the online communication with healthcare providers in older population. The findings can also serve healthcare providers in their efforts to improve the quality of (online) communication with their older patients.

Evaluating a Public Health Information Service According to Users' Socioeconomic Position and Health Status: Protocol for a Cross-Sectional Study

BACKGROUND

The increasing use of information technology in the field of health is supposed to promote users' empowerment but can also reinforce social inequalities. Some health authorities in various countries have developed mechanisms to offer accurate and relevant information to health care system users, often through health websites. However, the evaluation of these sociotechnical tools is inadequate, particularly with respect to differences and inequalities in use by social groups.

OBJECTIVE

Our study aims to evaluate the access, understanding, appraisal, and use of the French website Santé.fr by users according to their socioeconomic position and perceived health status.

METHODS

This cross-sectional study involves the entire French population to which Santé.fr is offered. Data will be collected through mixed methods, including a web-based questionnaire for quantitative data and interviews and focus groups for qualitative data. Collected data will cover users' access, understanding, appraisal, and use of Santé.fr, as well as sociodemographic and socioeconomic characteristics, health status, and digital health literacy. A validation of the dimensions of access, understanding, appraisal, and use of Santé.fr will be conducted, followed by principal component analysis and ascendant hierarchical classification based on the 2 main components of principal component analysis to characterize homogeneous users' profiles. Regression models will be used to investigate the relationships between each dimension and socioeconomic position and health status variables. NVivo 11 software (Lumivero) will be used to categorize interviewees' comments into preidentified themes or themes emerging from the discourse and compare them with the comments of various types of interviewees to understand the factors influencing people's access, understanding, appraisal, and use of Santé.fr.

RESULTS

Recruitment is scheduled to begin in January 2024 and will conclude when the required number of participants is reached. Data collection is expected to be finalized approximately 7 months after recruitment, with the final data analysis programmed to be completed around December 2024.

CONCLUSIONS

This study would be the first in France and in Europe to evaluate a public health information service, in this case the Santé.fr website (the official website of the French Ministry of Health), according to users' socioeconomic position and health status. The study could discover issues related to inequalities in access to, and the use of, digital technologies for obtaining health information on the internet. Given that access to health information on the internet is crucial for health decision-making and empowerment, inequalities in access may have subsequent consequences on health inequalities among social categories. Therefore, it is important to ensure that all social categories have access to Santé.fr.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/51123.

How do patients with high cardiovascular risk evaluate online health information? A qualitative study

BACKGROUND

People are exposed to variable health information from the Internet, potentially influencing their health decision-making and behaviour. It remains a challenge for people to discern between good- and poor-quality online health information (OHI). This study explored how patients evaluate and determine trust in statin-related OHI in patients with high cardiovascular risk.

METHODS

This qualitative study used vignettes and think-aloud methods. We recruited patients from a primary care clinic who were at least 18 years old, had high cardiovascular risk and had previously sought OHI. Participants were given two statin-related vignettes: Vignette 1 (low-quality information) and Vignette 2 (high-quality information). Participants voiced their thoughts aloud when reading the vignettes and determined the trust level for each vignette using a 5-point Likert scale. This was followed by a semi-structured interview which was audio-recorded and transcribed verbatim. The transcripts were coded and analysed using thematic analysis.

RESULTS

A total of 20 participants were recruited, with age ranging from 38-74 years. Among all the high cardiovascular-risk participants, eight had pre-existing cardiovascular diseases. For Vignette 1 (low-quality information), five participants trusted it while nine participants were unsure of their trust. 17 participants (85%) trusted Vignette 2 (high-quality information). Five themes emerged from the analysis of how patients evaluated OHI: (1) logical content, (2) neutral stance and tone of OHI content, (3) credibility of the information source, (4) consistent with prior knowledge and experience, and (5) corroboration with information from other sources.

CONCLUSION

Patients with high cardiovascular risks focused on the content, source credibility and information consistency when evaluating and determining their trust in statin-related OHI. Doctors should adopt a more personalised approach when discussing statin-related online misinformation with patients by considering their prior knowledge, beliefs and experience of statin use.

Nutrition and Food Literacy: Framing the Challenges to Health Communication

Nutrition and food literacy are two important concepts that are often used interchangeably, but they are not synonymous. Nutrition refers to the study of how food affects the body, while food literacy refers to the knowledge, skills, and attitudes necessary to make informed decisions about food and its impact on health. Despite the growing awareness of the importance of food literacy, food illiteracy remains a global issue, affecting people of all ages, backgrounds, and socioeconomic status. Food illiteracy has serious health implications as it contributes to health inequities, particularly among vulnerable populations. In addition, food literacy is a complex and multidisciplinary field, and there are numerous challenges to health communication that must be addressed to effectively promote food literacy and improve health outcomes. Addressing food illiteracy and the challenges to health communication is essential to promote health equity and improve health outcomes for all populations.

A comprehensive evaluation of ChatGPT consultation quality for augmentation mammoplasty: A comparative analysis between plastic surgeons and laypersons

OBJECTIVES

ChatGPT has gained significant popularity as a source of healthcare information among the general population. Evaluating the quality of chatbot responses is crucial, requiring comprehensive and qualitative analysis. This study aims to assess the answers provided by ChatGPT during hypothetical breast augmentation consultations across various categories and depths. The evaluation involves the utilization of validated tools and a comparison of scores between plastic surgeons and laypersons.

METHODS

A panel consisting of five plastic surgeons and five laypersons evaluated ChatGPT's responses to 25 questions spanning consultation, procedure, recovery, and sentiment categories. The DISCERN and PEMAT tools were employed to assess the responses, while emotional context was examined through ten specific questions. Additionally, readability was measured using the Flesch Reading Ease score. Qualitative analysis was performed to identify the overall strengths and weaknesses.

RESULTS

Plastic surgeons generally scored lower than laypersons across most domains. Scores for each evaluation domain varied by category, with the consultation category demonstrating lower scores in terms of DISCERN reliability, information quality, and DISCERN score. Plastic surgeons assigned significantly lower overall quality ratings to the procedure category compared to other question categories. They also gave lower emotion scores in the procedure category compared to laypersons. The depth of the questions did not impact the scoring.

CONCLUSIONS

Existing health information evaluation tools may not be entirely suitable for comprehensively evaluating the quality of individual responses generated by ChatGPT. Consequently, the development and implementation of appropriate evaluation tools to assess the appropriateness and quality of AI consultations are necessary.

A cross-sectional quality assessment of TikTok content on benign prostatic hyperplasia

BACKGROUND

With an increasing reliance on online sources for medical information, we studied the quality and completeness of health literacy videos on TikTok regarding BPH.

METHODS

A cross-sectional systematic evaluation of TikTok videos using the search term "Benign Prostatic Hyperplasia" was performed on 14th April 2023, and included 49 patient information and educational videos. The videos were then analysed by two reviewers and scored using two instruments: the DISCERN instrument and a completeness analysis.

RESULTS

Of the 49 videos, 38 were created by healthcare professionals (HCPs). The average length of each video was 62.7 ± 59.3 s, with a large average number of total views (24,990.1 ± 109,534.9 views). The DISCERN score trended higher in every category in videos published by HCPs compared to non-HCPs, with HCPs providing a statistically significant increase in reliability (19.0,14.6, p < 0.05) and total score (29.4,23, p < 0.05). Majority of videos were deemed as poor or worse (91.8%) in quality. The completeness of the videos' content was also evaluated across five categories with an average score of 2.53 ± 2.1 out of the maximum 12. The DISCERN scores did not correlate with the degree of completeness of the videos (r = 0.226).

CONCLUSION

BPH videos on TikTok have a wide reach, but the videos are mostly of low quality and completeness. Future videos should be made with quality and completeness in mind given the large viewership and more can be done to evaluate the extent of BPH misinformation and its impact on patients.

Public Perceptions Towards Online Health Information: A Mixed-Method Study in Eastern Province of Saudi Arabia

Background

Online health disinformation is one of the key challenges faced globally. Although there are efforts to address this issue, most of the proposed solutions were ineffective or could not address it effectively. One of the key reasons for the ineffective solutions is lack of understanding on how public deal with disinformation and the different areas that could be influenced by health disinformation.

Purpose

This study aims to analyze the public perceptions towards online health information, investigate the impact of online health disinformation, and propose a framework for addressing the issue of online health disinformation.

Methods

A mixed-methods approach design is used in this study which includes a cross-sectional survey design to gather data from public; and a semi-structured interviews approach with healthcare experts to analyze the impact of online health disinformation on the society. A total of 409 Saudi residents were included for survey, and 22 health informatics experts were considered for interviews.

Results

The results revealed low digital literacy levels and poor critical analysis skills among the public resulting in poor management of health disinformation. Age and education can influence the users' perceptions about the use of various online health information evaluation techniques. Thematic analysis of the interviews identified impact in four areas including social, economic, technical, and legal fronts; and various strategies were identified for public, governments, and healthcare institutions for managing the online health disinformation.

Conclusion

The occurrence of online health-related disinformation in Saudi Arabia is high, and there is a need to implement strict regulatory framework for managing disinformation.

Chinese Patients' Virtual Consultation Use of Different Sponsorship Types of Telemedicine Platforms: A Cross-Sectional Study in Zhejiang Province, China

Background: Virtual consultation has been one of the most prevalent direct-to-consumer telemedicine services in China. However, little is known about patients' virtual consultation use of different sponsorship types of telemedicine platforms. This study aimed to examine Chinese patients' virtual consultation use and identify the factors that affect consultation use of different sponsorship types of platforms. Methods: We conducted a cross-sectional survey on 1,653 participants in tier 1, tier 2, and tier 3 hospitals in 3 cities with different income levels, in Zhejiang Province, in May and June 2019. Multinomial logistic regression analysis was used to identify the factors that affect patients' virtual consultation use of different sponsorship types of platforms. Results: The most frequently used consultation platform was digital health company-sponsored platforms (36.60%), hospital-sponsored platforms (34.57%), doctors' personal social media (11.09%), other company-sponsored platforms (9.24%), and medical e-commerce company-sponsored platforms (8.50%). The patients' virtual consultation use of different sponsorship types of platforms was determined by education levels, monthly income levels, perceived health status, internet use, and city income levels. Conclusions: Chinese patients' virtual consultation service use varied by platform sponsorship type. Digital health company-sponsored platforms owned advantages over other platform types in high-end consumers with higher education levels, higher income levels, living in high-income cities, and active in internet use. This study implied that different sponsorship types of direct-to-consumer telemedicine platforms in China differ from each other in online health care resource allocation, business model, and competitive advantage.

Physician weight influences responses to a public health message about the genetics of obesity

OBJECTIVE

Many patients prefer lean physicians to physicians with obesity and place higher credence in their weight management advice. Because genetic information about weight can be viewed as self-serving for individuals with overweight or obesity, physicians with obesity may be discounted when sharing such information. As provision of genetic information regarding weight becomes increasingly common in public health messaging, could a physician's own weight influence how these messages are received by the public?

METHODS

In an online survey, 967 participants were randomly assigned a physician profile (lean v. has obesity) with a media interview transcript discussing genetic factors of a common health condition (obesity v. osteoporosis).

RESULTS

Participants perceived the physician with obesity who discussed genetic factors in obesity as less trustworthy and less credible. Participants were also less likely to anticipate following her advice on weight-related issues. Participants with higher BMI had less negative perceptions of this physician.

CONCLUSION

Physicians with obesity, when providing public health messaging regarding genetic information about obesity, may be met with distrust and negative attitudes toward the physician.

PRACTICAL IMPLICATIONS

Future research should investigate health communication strategies that address this form of weight stigma while accurately conveying genetic factors that contribute to weight.

The promise and peril of using a large language model to obtain clinical information: ChatGPT performs strongly as a fertility counseling tool with limitations

OBJECTIVE

To compare the responses of the large language model-based "ChatGPT" to reputable sources when given fertility-related clinical prompts.

DESIGN

The "Feb 13" version of ChatGPT by OpenAI was tested against established sources relating to patient-oriented clinical information: 17 "frequently asked questions (FAQs)" about infertility on the Centers for Disease Control (CDC) Website, 2 validated fertility knowledge surveys, the Cardiff Fertility Knowledge Scale and the Fertility and Infertility Treatment Knowledge Score, as well as the American Society for Reproductive Medicine committee opinion "optimizing natural fertility."

SETTING

Academic medical center.

PATIENT(S)

Online AI Chatbot.

INTERVENTION(S)

Frequently asked questions, survey questions and rephrased summary statements were entered as prompts in the chatbot over a 1-week period in February 2023.

MAIN OUTCOME MEASURE(S)

For FAQs from CDC: words/response, sentiment analysis polarity and objectivity, total factual statements, rate of statements that were incorrect, referenced a source, or noted the value of consulting providers.

FOR FERTILITY KNOWLEDGE SURVEYS

Percentile according to published population data.

FOR COMMITTEE OPINION

Whether response to conclusions rephrased as questions identified missing facts.

RESULT(S)

When administered the CDC's 17 infertility FAQ's, ChatGPT produced responses of similar length (207.8 ChatGPT vs. 181.0 CDC words/response), factual content (8.65 factual statements/response vs. 10.41), sentiment polarity (mean 0.11 vs. 0.11 on a scale of -1 (negative) to 1 (positive)), and subjectivity (mean 0.42 vs. 0.35 on a scale of 0 (objective) to 1 (subjective)). In total, 9 (6.12%) of 147 ChatGPT factual statements were categorized as incorrect, and only 1 (0.68%) statement cited a reference. ChatGPT would have been at the 87th percentile of Bunting's 2013 international cohort for the Cardiff Fertility Knowledge Scale and at the 95th percentile on the basis of Kudesia's 2017 cohort for the Fertility and Infertility Treatment Knowledge Score. ChatGPT reproduced the missing facts for all 7 summary statements from "optimizing natural fertility."

CONCLUSION(S)

A February 2023 version of "ChatGPT" demonstrates the ability of generative artificial intelligence to produce relevant, meaningful responses to fertility-related clinical queries comparable to established sources. Although performance may improve with medical domain-specific training, limitations such as the inability to reliably cite sources and the unpredictable possibility of fabricated information may limit its clinical use.

A 10-year follow-up survey of US academic libraries highlights the COVID-19 experience and greater interest in health information outreach

BACKGROUND

As follow-up to their previous survey on health information outreach (HIO) in 2010, the authors became interested in the evolving nature of HIO activities at academic libraries within the past decade.

OBJECTIVES

The aim of this study was to understand how HIO activities at academic libraries have evolved since 2010, especially considering the current COVID-19 pandemic.

METHODS

An online survey, designed to collect quantitative data on general HIO activities and qualitative data specifically on COVID-19 information outreach, was distributed to over 1700 librarians at US general academic and academic health science libraries.

RESULTS

Two hundred and fifty-five respondents completed the survey. Quantitative findings generally paralleled the 2010 results, except in a few areas. Most notably, a greater percentage of librarians in 2020 were participating in HIO (44%) than in 2010 (37%). Qualitative findings revealed that librarians are leveraging traditional information services and implementing innovative partnerships to promote reputable health information resources on COVID-19.

DISCUSSION

Evidence suggests that further engagement and campus partnerships can enhance libraries' supportive roles as trustworthy purveyors of quality health information.

CONCLUSION

US academic librarians are increasingly engaging in HIO to support the health information needs of campus communities and should consider aligning outreach activities with national health goals.

Using a Google Web Search Analysis to Assess the Utility of ChatGPT in Total Joint Arthroplasty

BACKGROUND

Rapid technological advancements have laid the foundations for the use of artificial intelligence in medicine. The promise of machine learning (ML) lies in its potential ability to improve treatment decision making, predict adverse outcomes, and streamline the management of perioperative healthcare. In an increasing consumer-focused health care model, unprecedented access to information may extend to patients using ChatGPT to gain insight into medical questions. The main objective of our study was to replicate a patient's internet search in order to assess the appropriateness of ChatGPT, a novel machine learning tool released in 2022 that provides dialogue responses to queries, in comparison to Google Web Search, the most widely used search engine in the United States today, as a resource for patients for online health information. For the 2 different search engines, we compared i) the most frequently asked questions (FAQs) associated with total knee arthroplasty (TKA) and total hip arthroplasty (THA) by question type and topic; ii) the answers to the most frequently asked questions; as well as iii) the FAQs yielding a numerical response.

METHODS

A Google web search was performed with the following search terms: "total knee replacement" and "total hip replacement." These terms were individually entered and the first 10 FAQs were extracted along with the source of the associated website for each question. The following statements were inputted into ChatGPT: 1) "Perform a google search with the search term 'total knee replacement' and record the 10 most FAQs related to the search term" as well as 2) "Perform a google search with the search term 'total hip replacement' and record the 10 most FAQs related to the search term." A Google web search was repeated with the same search terms to identify the first 10 FAQs that included a numerical response for both "total knee replacement" and "total hip replacement." These questions were then inputted into ChatGPT and the questions and answers were recorded.

RESULTS

There were 5 of 20 (25%) questions that were similar when performing a Google web search and a search of ChatGPT for all search terms. Of the 20 questions asked for the Google Web Search, 13 of 20 were provided by commercial websites. For ChatGPT, 15 of 20 (75%) questions were answered by government websites, with the most frequent one being PubMed. In terms of numerical questions, 11 of 20 (55%) of the most FAQs provided different responses between a Google web search and ChatGPT.

CONCLUSION

A comparison of the FAQs by a Google web search with attempted replication by ChatGPT revealed heterogenous questions and responses for open and discrete questions. ChatGPT should remain a trending use as a potential resource to patients that needs further corroboration until its ability to provide credible information is verified and concordant with the goals of the physician and the patient alike.

Quality and accuracy of online nutrition-related information: a systematic review of content analysis studies

OBJECTIVE

This systematic review aimed to summarise the level of quality and accuracy of nutrition-related information on websites and social media and determine if quality and accuracy varied between websites and social media or publishers of information.

DESIGN

This systematic review was registered with PROSPERO (CRD42021224277). CINAHL, MEDLINE, Embase, Global Health and Academic Search Complete were systematically searched on 15 January 2021 to identify content analysis studies, published in English after 1989, that evaluated the quality and/or accuracy of nutrition-related information published on websites or social media. A coding framework was used to classify studies' findings about information quality and/or accuracy as poor, good, moderate or varied. The Academy of Nutrition and Dietetics Quality Criteria Checklist was used to assess the risk of bias.

SETTING

N/A.

PARTICIPANTS

N/A.

RESULTS

From 10 482 articles retrieved, sixty-four were included. Most studies evaluated information from websites (n 53, 82·8 %). Similar numbers of studies assessed quality (n 41, 64·1 %) and accuracy (n 47, 73·4 %). Almost half of the studies reported that quality (n 20, 48·8 %) or accuracy (n 23, 48·9 %) was low. Quality and accuracy of information were similar on social media and websites, however, varied between information publishers. High risk of bias in sample selection and quality or accuracy evaluations was a common limitation.

CONCLUSION

Online nutrition-related information is often inaccurate and of low quality. Consumers seeking information online are at risk of being misinformed. More action is needed to improve the public's eHealth and media literacy and the reliability of online nutrition-related information.

Evaluating the implementation of a referral system for virtual pharmacy counselling in a province-wide nurse phone line

Digital technology offers several opportunities to improve access to professional expertise in primary care, and the offer of various "virtual" services has exploded in the past few years. The aim of this study was to evaluate the implementation of a direct-to-consumer on-line pharmacy consultation service (Ask Your Pharmacist - AYP) to a universal phone consultation service led by the universal public health system in Quebec (811 Info-Santé), through a direct bridge. Semi-structured interviews were conducted with clinician users of the service, and stakeholders involved in this pilot project funded by the Ministry of Economy (n = 22); documents were also analyzed, and content of the question was asked through the AYP service. Adoption of the service was low, and it suggests a poor alignment between the need and the service as implemented. Further research should investigate the mechanisms for an appropriate integration of digital services for primary care universal consultation services.

Patterns of Parental Information Seeking Behavior on Early Childhood Caries: A Qualitative Exploration

Objectives

This study aims to explore the information-seeking behavior patterns of parents with children receiving treatment for early childhood caries (ECC).

Materials and Methods

Semistructured in-depth interviews were conducted with 20 parents of children with ECC. A topic guide was developed, focusing on questions relating to (i) the timing of their seeking information on ECC, (ii) the types of EEC information they seek, and (iii) the resources used to seek information. The interviews were audio-recorded and transcribed verbatim. Thematic analysis was performed, whereby the data were coded and categorized into themes and subthemes.

Results

Four main themes were identified: the immediacy of seeking information, perceived information need, use of resources, and barriers to seeking information. Parents either sought information immediately after detecting changes to the appearance of their child's teeth, with some being aware of the changes after signs and symptoms developed. The types of information parents usually sought covered the disease, its prevention, and management. Common sources of information were friends, family, the internet, and healthcare professionals. Barriers to seeking information discussed by parents were lack of time as well as insufficiency and inaccuracy of the information they received.

Conclusion

This study highlighted the need for comprehensive, tailored early education on ECC for parents using reliable information sources. There is also a need to empower other nondental healthcare professionals to provide oral healthcare education for parents.

Accuracy of prostate cancer screening recommendations for high-risk populations on YouTube and TikTok

Objectives

This study aimed to evaluate content quality and racial/ethnic representation, particularly of high-risk cohorts, of prostate cancer screening videos on YouTube (YT) and TikTok (TK).

Materials and Methods

The top 50 videos populated for the search term 'prostate cancer screening' on YT and TK that met inclusion criteria were retrieved in a cache-cleared browser. Three reviewers analysed all videos using validated criteria for the quality of consumer health information (DISCERN and Patient Education Materials Assessment Tool [PEMAT]). High quality was defined as follows: DISCERN ≥ 4, PEMAT understandability ≥75% and PEMAT actionability ≥75%. A 5-point Likert scale was used to demonstrate the level of misinformation compared to American Urological Association and National Comprehensive Cancer Network guidelines. Perceived race and ethnicity of people in the videos were assessed by consensus approach.

Results

TK videos were shorter (median 3.7 vs. 0.5 min, p < 0.001) and had more views per month (5437.5 vs. 19.3, p = 0.03) than YT videos. Perceived Black and Hispanic representation was present in 10% and 6% of YT videos and 20% and 12% of TK videos, respectively. High-risk racial/ethnic groups were explicitly discussed in 46% of YT videos and 8% of TK videos. A total of 98% of YT videos and 100% of TK videos had low- to moderate-quality consumer health information, and 88% of YT videos and 100% of TK videos had moderate to high levels of misinformation based on screening guidelines.

Conclusions

YT and TK videos about prostate cancer screening are widely viewed but do not provide quality consumer health information. Black and Hispanic men remain under-represented on both platforms, and high-risk racial groups were not discussed in most videos despite the importance for screening criteria. The low understandability and actionability, significant misinformation and lack of diversity in online videos support the need for higher quality videos with adequate attention to high-risk ethnic cohorts.

Predictors of online health information seeking behaviour of non-medical undergraduate students might vary

BACKGROUND

As more people across Generations X, Y, and Z continue to use the Internet meet their information needs, this study assesses the characteristics influencing the online health information seeking behaviour (OHISB) of non-medical university students. Previous studies on the subject in Nigeria have largely assessed the concept of OHISB among university students without examining the determinants of such behaviour.

OBJECTIVE

To determine the predictors of OHISB among non-medical undergraduate students of a Nigerian private university.

METHODS

Adopting a cross-sectional research design, the study utilized a semi-structured questionnaire administered via Google forms. Some 361 non-medical undergraduate students were recruited purposively with a response rate of 84.2%. The questionnaire obtained data on students' socio-demographics, Internet usage, and OHISB.

RESULTS

Presence of a chronic condition, gender, and college showed significant associations with students' OHISB. 'Mothers' occupation' and 'prior experience' became significant predictors of students' OHISB.

DISCUSSION

The results align with studies that stressed that chronically ill persons would search for health information on the internet and that presence of chronic illness is among the predictors of OHISB.

CONCLUSION

These findings add to evidence from other countries that had identified health literacy and presence of chronic illness as significant predictors of OHISB among students.

The quality of informational social support in online health communities: A content analysis of cancer-related discussions

Objective

Informational social support is one of the main reasons for patients to visit online health communities (OHCs). Calls have been made to investigate the objective quality of such support in the light of a worrying number of inaccurate online health-related information. The main aim of this study is to conceptualize the Quality of Informational Social Support (QISS) and develop and test a measure of QISS for content analysis. A further aim is to investigate the level of QISS in cancer-related messages in the largest OHC in Slovenia and examine the differences among various types of discussion forums, namely, online consultation forums, online support group forums, and socializing forums.

Methods

A multidimensional measurement instrument was developed, which included 20 items in a coding scheme for a content analysis of cancer-related messages. On a set of almost three million posts published between 2015 and 2019, a machine-learning algorithm was used to detect cancer-related discussions in the OHC. We then identified the messages providing informational social support, and through quantitative content analysis, three experts coded a random sample of 403 cancer-related messages for the QISS.

Results

The results demonstrate a good level of interrater reliability and agreement for a QISS scale with six dimensions, each demonstrating good internal consistency. The results reveal large differences among the social support, socializing, and consultation forums, with the latter recording significantly higher quality in terms of accuracy (M = 4.48, P < .001), trustworthiness (M = 4.65, P < .001), relevance (M = 3.59, P < .001), and justification (M = 3.81, P = .05) in messages providing informational social support regarding cancer-related issues.

Conclusions

This study provides the research field with a valid tool to further investigate the factors and consequences of varying quality of information exchanged in supportive communication. From a practical perspective, OHCs should dedicate more resources and develop mechanisms for the professional moderation of health-related topics in socializing forums and thereby suppress the publication and dissemination of low-quality information among OHC users and visitors.

The Reliability and Quality of Short Videos as a Source of Dietary Guidance for Inflammatory Bowel Disease: Cross-sectional Study

BACKGROUND

Dietary management is considered a potential adjunctive treatment for inflammatory bowel disease (IBD). Short-video sharing platforms have enabled patients to obtain dietary advice more conveniently. However, accessing useful resources while avoiding misinformation is not an easy task for most patients.

OBJECTIVE

This study aimed to evaluate the quality of the information in IBD diet-related videos on Chinese short-video sharing platforms.

METHODS

We collected and extracted information from a total of 125 video samples related to the IBD diet on the 3 Chinese short-video sharing platforms with the most users: TikTok, Bilibili, and Kwai. Two independent physicians evaluated each video in terms of content comprehensiveness, quality (rated by Global Quality Score), and reliability (rated by a modified DISCERN tool). Finally, comparative analyses of the videos from different sources were conducted.

RESULTS

The videos were classified into 6 groups based on the identity of the uploaders, which included 3 kinds of medical professionals (ie, gastroenterologists, nongastroenterologists, and clinical nutritionists) and 3 types of non-medical professionals (ie, nonprofit organizations, individual science communicators, and IBD patients). The overall quality of the videos was poor. Further group comparisons demonstrated that videos from medical professionals were more instructive in terms of content comprehensiveness, quality, and reliability than those from non-medical professionals. Moreover, IBD diet-related recommendations from clinical nutritionists and gastroenterologists were of better quality than those from nongastroenterologists, while recommendations from nonprofit organizations did not seem to be superior to other groups of uploaders.

CONCLUSIONS

The overall quality of the information in IBD diet-related videos is unsatisfactory and varies significantly depending on the source. Videos from medical professionals, especially clinical nutritionists and gastroenterologists, may provide dietary guidance with higher quality for IBD patients.

Current Situations and Challenges in the Development of Health Information Literacy

Health information literacy (HIL) is a significant concept that has gradually become known to the broader public in recent years. Although the definitions of HIL and health literacy seem to overlap, as an independent subconcept, HIL still shows a unique influence on improvements in people's health and health education. Remarkable evidence indicates that online health information (OHI) can effectively enrich people's knowledge and encourage patients to actively join the medical process, which is also accompanied by the emergence of various assessment tools. Although the current assessment tools, to a certain extent, can help people identify their shortcomings and improve their HIL, many studies have indicated that the deficiencies of the scales induce incomplete or unreal results of their HIL. In addition, continuing research has revealed an increasing number of influencing factors that have great effects on HIL and even regulate the different trends in doctor-patient relationships. Simultaneously, most of the uncensored OHI broadcasts have also affected the improvement in HIL in various ways. Thus, this review aims to summarize the assessment tools, influencing factors and current situations and challenges related to HIL. Further studies are required to provide more trusted and deeper references for the development of HIL.

The invisible burden of managing COVID-19 for Australian women: Cognitive labor and public health information

Providing accurate and timely public health information is an ongoing challenge for public health officials. The COVID-19 pandemic has exacerbated such challenges and presented unique difficulties in providing public health information, through the parallel rise of an "infodemic" of mis/dis-information. Understanding why individuals select, use and change their public health information seeking behaviors around COVID-19, and the relationship of these decisions relative to participant characteristics, is therefore an important step in understanding and responding to infodemics. This study used a qualitative survey (n = 255) and free-text qualitative questions to ask (1) Why participants use an information source, (2) How participants used an information source, and (3) How information seeking behavior has changed since the COVID-19 pandemic. Participants were primarily women, born in Australia, with de-facto/married relationships, without children at home, with university/college qualifications, and employed full-time or unemployed/retired. Most participants identified "easiness" and "immediacy" as reasons why they chose and used information, with sources primarily used for planning, communication, and decision making. A minority of participants changed their information seeking behavior since the COVID-19 pandemic. Those who did change, desired more immediate and accurate information. Emergent themes of care and anxiety were also noted, raising questions around the impact of mental load and cognitive labor in some female populations. Women may be suffering from increased cognitive labor and a gendering of public health information seeking behavior in the context of COVID-19. The impact of these attributes on women requires greater empirical research and consideration amongst front line practitioners and public health professionals.

Evaluating online health information quality using machine learning and deep learning: A systematic literature review

Background

Due to the large volume of online health information, while quality remains dubious, understanding the usage of artificial intelligence to evaluate health information and surpass human-level performance is crucial. However, the existing studies still need a comprehensive review highlighting the vital machine, and Deep learning techniques for the automatic health information evaluation process.

Objective

Therefore, this study outlines the most recent developments and the current state of the art regarding evaluating the quality of online health information on web pages and specifies the direction of future research.

Methods

In this article, a systematic literature is conducted according to the PRISMA statement in eight online databases PubMed, Science Direct, Scopus, ACM, Springer Link, Wiley Online Library, Emerald Insight, and Web of Science to identify all empirical studies that use machine and deep learning models for evaluating the online health information quality. Furthermore, the selected techniques are compared based on their characteristics, such as health quality criteria, quality measurement tools, algorithm type, and achieved performance.

Results

The included papers evaluate health information on web pages using over 100 quality criteria. The results show no universal quality dimensions used by health professionals and machine or deep learning practitioners while evaluating health information quality. In addition, the metrics used to assess the model performance are not the same as those used to evaluate human performance.

Conclusions

This systemic review offers a novel perspective in approaching the health information quality in web pages that can be used by machine and deep learning practitioners to tackle the problem more effectively.

The quality of health information provided on web sites selling cannabis to consumers in Canada is poor

BACKGROUND

Cannabis is used by millions of people for both medical and recreational purposes, and this use is even greater in jurisdictions where it is legalized, such as Canada. Online cannabis vendors have gained popularity for purchasing cannabis due to easy access and convenience to consumers. The objective of this study was to evaluate the quality of health information provided by web sites of cannabis vendors selling products to Canadian consumers and to further identify trends in the information provided.

METHODS

Six different searches were conducted on Google.ca, and the first 40 webpages of each search were screened for eligibility. A total of 33 unique web sites of cannabis vendors selling products to Canadian consumers were identified and included. The DISCERN instrument, which consists of 16 questions divided into three sections, was used to evaluate the quality of cannabis-related health information on these web sites.

RESULTS

Across the 33 web sites, the average of the summed DISCERN scores was 36.83 (SD = 9.73) out of 75, and the mean score for the overall quality of the publication (DISCERN question 16) was 2.41 (SD = 0.71) out of 5. Many of these web sites failed to discuss uncertainties in research evidence on cannabis, the impact of cannabis use on quality of life, alternatives to cannabis use, risks associated with cannabis use, and lacked references to support claims on effects and benefits of use.

CONCLUSION

Our findings indicate that the quality of cannabis-related health information provided by online vendors is poor. Healthcare providers should be aware that patients may use these web sites as primary sources of information and appropriately caution patients while directing them to high-quality sources. Future research should serve to replicate this study in other jurisdictions and assess the accuracy of information provided by online cannabis vendors, as this was outside the scope of the DISCERN instrument.

How users make judgements about the quality of online health information: a cross-sectional survey study

BACKGROUND

People increasingly use the Internet to seek health information. However, the overall quality of online health information remains low. This situation is exacerbated by the unprecedented "infodemic", which has had negative consequences for patients. Therefore, it is important to understand how users make judgements about health information by applying different judgement criteria.

OBJECTIVE

The objective of this study is to determine how patients apply different criteria in their judgement of the quality of online health information during the pandemic. In particular, we investigate whether there is consistency between the likelihood of using a particular judgement criterion and its perceived importance among different groups of users.

METHODS

A cross-sectional survey was conducted in one of the leading hospitals in a coastal province of China with a population of forty million. Combined-strategy sampling was used to balance the randomness and the practicality of the recruiting process. A total of 1063 patients were recruited for this study. Chi-square and Kruskal-Wallis analyses were used to analyse the survey data.

RESULTS

In general, patients make quality judgement of health information more frequently based on whether it is familiar, aesthetic, and with expertise. In comparison, they put more weights on whether health information is secure, trustworthy, and with expertise when determining its quality. Criteria that were considered more important were not always those with a higher likelihood of being used. Patients may not use particular criteria, such as familiarity, identification, and readability, more frequently than others even if they consider them to be more important than other do and vice versa. Surprisingly, patients with a primary school degree put more weight on whether health information is comprehensive than those with higher degrees do in determining its quality. However, they are less likely to use this guideline in practice.

CONCLUSIONS

To the best of our knowledge, this is the first study to investigate the consistency between the likelihood of using certain quality judgement criteria and their perceived importance among patients grouped by different demographic variables and eHealth literacy levels. The findings highlight how to improve online health information services and provide fine-grained customization of information for users.

Health Information Sourcing and Health Knowledge Quality: Repeated Cross-sectional Survey

Background

People’s health-related knowledge influences health outcomes, as this knowledge may influence whether individuals follow advice from their doctors or public health agencies. Yet, little attention has been paid to where people obtain health information and how these information sources relate to the quality of knowledge.

Objective

We aim to discover what information sources people use to learn about health conditions, how these sources relate to the quality of their health knowledge, and how both the number of information sources and health knowledge change over time.

Methods

We surveyed 200 different individuals at 12 time points from March through September 2020. At each time point, we elicited participants’ knowledge about causes, risk factors, and preventative interventions for 8 viral (Ebola, common cold, COVID-19, Zika) and nonviral (food allergies, amyotrophic lateral sclerosis [ALS], strep throat, stroke) illnesses. Participants were further asked how they learned about each illness and to rate how much they trust various sources of health information.

Results

We found that participants used different information sources to obtain health information about common illnesses (food allergies, strep throat, stroke) compared to emerging illnesses (Ebola, common cold, COVID-19, Zika). Participants relied mainly on news media, government agencies, and social media for information about emerging illnesses, while learning about common illnesses from family, friends, and medical professionals. Participants relied on social media for information about COVID-19, with their knowledge accuracy of COVID-19 declining over the course of the pandemic. The number of information sources participants used was positively correlated with health knowledge quality, though there was no relationship with the specific source types consulted.

Conclusions

Building on prior work on health information seeking and factors affecting health knowledge, we now find that people systematically consult different types of information sources by illness type and that the number of information sources people use affects the quality of individuals’ health knowledge. Interventions to disseminate health information may need to be targeted to where individuals are likely to seek out information, and these information sources differ systematically by illness type.

Comparing Transactional eHealth Literacy of Individuals With Cancer and Surrogate Information Seekers: Mixed Methods Study

Background

The number of adults entering higher-risk age groups for receiving a cancer diagnosis is rising, with predicted numbers of cancer cases expected to increase by nearly 50% by 2050. Living with cancer puts exceptional burdens on individuals and families during treatment and survivorship, including how they navigate their relationships with one another. One role that a member of a support network may enact is that of a surrogate seeker, who seeks information in an informal capacity on behalf of others. Individuals with cancer and surrogate seekers often use the internet to learn about cancer, but differences in their skills and strategies have received little empirical attention.

Objective

This study aimed to examine the eHealth literacy of individuals with cancer and surrogate information seekers, including an investigation of how each group evaluates the credibility of web-based cancer information. As a secondary aim, we sought to explore the differences that exist between individuals with cancer and surrogate seekers pertaining to eHealth literacies and sociodemographic contexts.

Methods

Between October 2019 and January 2020, we conducted a web-based survey of 282 individuals with cancer (n=185) and surrogate seekers (n=97). We used hierarchical linear regression analyses to explore differences in functional, communicative, critical, and translational eHealth literacy between individuals with cancer and surrogate seekers using the Transactional eHealth Literacy Instrument. Using a convergent, parallel mixed methods design, we also conducted a thematic content analysis of an open-ended survey response to qualitatively examine how each group evaluates web-based cancer information.

Results

eHealth literacy scores did not differ between individuals with cancer and surrogate seekers, even after adjusting for sociodemographic variables. Individuals with cancer and surrogate seekers consider the credibility of web-based cancer information based on its channel (eg, National Institutes of Health). However, in evaluating web-based information, surrogate seekers were more likely than individuals with cancer to consider the presence and quality of scientific references supporting the information. Individuals with cancer were more likely than surrogate seekers to cross-reference other websites and web-based sources to establish consensus.

Conclusions

Web-based cancer information accessibility and evaluation procedures differ among individuals with cancer and surrogate seekers and should be considered in future efforts to design web-based cancer education interventions. Future studies may also benefit from more stratified recruitment approaches and account for additional contextual factors to better understand the unique circumstances experienced within this population.