Assessment of readability, quality and popularity of online information on ureteral stents

Quality of Information About Kidney Stones from Artificial Intelligence Chatbots

Introduction: Kidney stones are common and morbid conditions in the general population with a rising incidence globally. Previous studies show substantial limitations of online sources of information regarding prevention and treatment. The objective of this study was to examine the quality of information on kidney stones from artificial intelligence (AI) chatbots. Methods: The most common online searches about kidney stones from Google Trends and headers from the National Institute of Diabetes and Digestive and Kidney Diseases website were used as inputs to four AI chatbots (ChatGPT version 3.5, Perplexity, Chat Sonic, and Bing AI). Validated instruments were used to assess the quality (DISCERN instrument from 1 low to 5 high), understandability, and actionability (PEMAT, from 0% to 100%) of the chatbot outputs. In addition, we examined the reading level of the information and whether there was misinformation compared with guidelines (5 point Likert scale). Results: AI chatbots generally provided high-quality consumer health information (median DISCERN 4 out of 5) and did not include misinformation (median 1 out of 5). The median understandability was moderate (median 69.6%), and actionability was moderate to poor (median 40%). Responses were presented at an advanced reading level (11th grade; median Flesch-Kincaid score 11.3). Conclusions: AI chatbots provide generally accurate information on kidney stones and lack misinformation; however, it is not easily actionable and is presented above the recommended reading level for consumer health information.

Evaluation of the Accuracy, Credibility, and Readability of Statin-Related Websites: Cross-Sectional Study

BACKGROUND

Cardiovascular disease (CVD) represents the greatest burden of mortality worldwide, and statins are the most commonly prescribed drug in its management. A wealth of information pertaining to statins and their side effects is on the internet; however, to date, no assessment of the accuracy, credibility, and readability of this information has been undertaken.

OBJECTIVE

This study aimed to evaluate the quality (accuracy, credibility, and readability) of websites likely to be visited by the general public undertaking a Google search of the side effects and use of statin medications.

METHODS

Following a Google web search, we reviewed the top 20 consumer-focused websites with statin information. Website accuracy, credibility, and readability were assessed based on website category (commercial, not-for-profit, and media), website rank, and the presence or absence of the Health on the Net Code of Conduct (HONcode) seal. Accuracy and credibility were assessed following the development of checklists (with 20 and 13 items, respectively). Readability was assessed using the Simple Measure of Gobbledegook scores.

RESULTS

Overall, the accuracy score was low (mean 14.35 out of 20). While side effects were comprehensively covered by 18 websites, there was little information about statin use in primary and secondary prevention. None of the websites met all criteria on the credibility checklist (mean 7.8 out of 13). The median Simple Measure of Gobbledegook score was 9.65 (IQR 8.825-10.85), with none of the websites meeting the recommended reading grade of 6, even the media websites. A website bearing the HONcode seal did not mean that the website was more comprehensive or readable.

CONCLUSIONS

The quality of statin-related websites tended to be poor. Although the information contained was accurate, it was not comprehensive and was presented at a reading level that was too difficult for an average reader to fully comprehend. As such, consumers risk being uninformed about this pharmacotherapy.

Readability and Quality of Online Information on Sickle Cell Retinopathy for Patients

PURPOSE

This study aims to evaluate the readability and quality of Internet-based health information on sickle cell retinopathy.

DESIGN

Retrospective cross-sectional website analysis.

METHODS

To simulate a patient's online search, the terms "sickle cell retinopathy" and "sickle cell disease in the eye" were entered into the top 3 search engines (Google, Bing and Yahoo). The first 20 results of each search were retrieved and screened for analysis. The DISCERN questionnaire, the Journal of the American Medical Association (JAMA) standards, and the Health on the Net (HON) criteria were used to evaluate the quality of the information. The Flesch-Kincaid Grade Level (FKGL), the Flesch Reading Ease (FRES), and the Automated Readability Index (ARI) were used to assess the readability of each website.

RESULTS

Of 16 online sources, 12 (75%) scored moderately on the DISCERN tool. The mean DISCERN score was 40.91 (SD, 10.39; maximum possible, 80). None of the sites met all of the JAMA benchmarks, and only 3 (18.75%) of the websites had HONcode certification. All of the websites had scores above the target American Medical Association grade level of 6 on both the FKGL and ARI. The mean FRES was 57.76 (±4.61), below the recommended FRES of 80 to 90.

CONCLUSION

There is limited online information available on sickle cell retinopathy. Most included websites were fairly difficult to read and of substandard quality. The quality and readability of Internet-based, patient-focused information on sickle cell retinopathy needs to be improved.

Any Questions? Understanding the Functional Health Literacy in Patients Who Underwent Ureteroscopy

OBJECTIVE

To correlate health literacy of patients undergoing ureteroscopy and identify gaps within current patient education practices in order to better tailor the preoperative experience.

METHODS

Eighteen patients were retrospectively recruited to complete an in-depth semistructured interview and the Test of Functional Health Literacy for Adults (TOFHLA). All interviews were recorded, transcribed, and then coded and analyzed using the grounded theory of analysis.

RESULTS

The average participant age was 56.2 ± 12.8years, and 10 (55.6%) identified as female. Education level ranged from some high school to a professional degree. The average TOFHLA score was 88.1 ± 11.7. Irrespective of score, all participants felt they understood the purpose and basic elements of a ureteroscopy. The use of nontechnical language, repetition, and previous healthcare experiences were identified as positive aspects of the education experience. However, 72.2% (n = 13) identified the primary gap in understanding revolved around the use, purpose, and pain associated with stents.

CONCLUSION

Functional health literacy is an essential element, but not the only factor informing patient education and comprehension. Current practices are effective in explaining the basics of a ureteroscopy, but even when identified health literacy is higher than expected, a gap remains in stent education. Efforts should be made to better understand how stents can be effectively explained to patients in addition to continuing to refine education practices to elicit true comprehension.

Readability and reliability of online patient education materials about statins

Objective

Statins are the cornerstone for the prevention and treatment of cardiovascular disease. Patients often consult online patient education materials (OPEMs) to inform medical decision-making. We therefore aimed to assess the readability and reliability of OPEMs related to statins.

Methods

A total of 17 statin-related terms were queried using an online search engine to identify the top 20 search results for each statin-related term. Each OPEM was then grouped into the following categories based on 2 independent reviewers: government OPEMs (national, state, or local government agencies); healthcare/nonprofit OPEMs (major health systems and nonprofit organizations with a specific cardiovascular health focus); industry/commercial OPEMs (pharmaceutical manufacturers and online pharmacies); lay press OPEMs (healthcare-oriented news organizations); and dictionary/encyclopedia OPEMs. Grade-level readability for each OPEM was calculated using 5 standard readability metrics and compared with AMA-recommended readability recommendations. Reliability of each OPEM was evaluated using the JAMA benchmark criteria for online health information and certification from Health on the Net (HONCode).

Results

A total of 340 websites were identified across the 17 statin search terms. There were 211 statin OPEMs after excluding non-OPEM results; 172 OPEMs had unique content. Statin OPEM readability exceeded the recommended 6th grade AMA reading level (average reading grade level of 10.9). The average JAMA benchmark criteria score was 2.13 (on a scale of 0-4, with higher scores indicating higher reliability), and only 60% of statin OPEMs were HONCode-certified. There was an inverse association between readability and reliability. The most readable results were from industry and commercial sources, while the most reliable sites were from lay press sources.

Conclusions

Statin OPEMs are written at an overall averaging reading grade level of 10.9. There was an inverse association between readability and reliability. Lack of accessible, high-quality online health information may contribute to statin nonadherence.

Assessment of online patient education material for eye cancers: A cross-sectional study

The objective of this study was to assess online American patient education material (PEM) related to eye cancers in order to determine the quality of the content and appropriateness of the contents' reading level as it relates to the American population. PEMs were extracted from fifteen American cancer and ophthalmology associations and evaluated for their reading level using ten validated readability scales. PEMs then had all words extracted and evaluated for their difficulty and familiarity. The quality of the PEMS were assessed according to DISCERN, Heath On the Net Foundation Code of Conduct (HONCode), and JAMA benchmarks. Overall, online PEMs from the associations were written at a 11th grade reading level, which is above the recommended 6th grade reading level. The difficult word analysis identified that 26% of words were unfamiliar. Only one of the fifteen association held a HONCode certification while no organization met the standards of all four JAMA benchmarks. The average score for DISCERN was 2.4 out of a total of 5 for the fifteen questions related to treatment option information quality. Consideration should be made to create PEMs at an appropriate grade reading level to encourage health literacy and ultimately promote health outcomes. Associations should also focus on incorporating easily identifiable quality indicators to allow patients to better identify reputable resources.

Patient education information material assessment criteria: A scoping review

BACKGROUND

Patient education information material (PEIM) is an essential component of patient education programs in increasing patients' ability to cope with their diseases. Therefore, it is essential to consider the criteria that will be used to prepare and evaluate these resources.

OBJECTIVE

This paper aims to identify these criteria and recognize the tools or methods used to evaluate them.

METHODS

National and international databases and indexing banks, including PubMed, Scopus, Web of Science, ProQuest, the Cochrane Library, Magiran, SID and ISC, were searched for this review. Original or review articles, theses, short surveys, and conference papers published between January 1990 and June 2022 were included.

RESULTS

Overall, 4688 documents were retrieved, of which 298 documents met the inclusion criteria. The criteria were grouped into 24 overarching criteria. The most frequently used criteria were readability, quality, suitability, comprehensibility and understandability.

CONCLUSION

This review has provided empirical evidence to identify criteria, tools, techniques or methods for developing or evaluating a PEIM. The authors suggest that developing a comprehensive tool based on these findings is critical for evaluating the overall efficiency of PEIM using effective criteria.

Evaluating breast cancer websites targeting Arabic speakers: empirical investigation of popularity, availability, accessibility, readability, and quality

Background

Nowadays, patients have access to all types of health information on the internet, influencing their decision-making process. The Middle East and North Africa (MENA) region consists of 22 countries with an estimated population of around 600 million. Breast cancer is the highest diagnosed cancer in this region. Websites are commonly the go-to cancer information sources. A large population of the MENA region is only fluent in the Arabic language, thus access to Arabic websites is in more demand. However, little is known about breast cancer websites that cater to an Arabic-speaking audience. This study aims at evaluating Arabic breast cancer websites and offering recommendations to improve engagement and access to health information.

Methods

This study employed a cross-sectional analysis approach. Google trends was used to reveal the top searched topics across the MENA region, which in turn were used as search terms to identify the websites. To be included, a website had to be active, available in Arabic, and contain breast cancer information. The evaluation was based on a combination of automated and expert-based evaluation methods through five dimensions: Availability, Accessibility, Readability, Quality, and Popularity.

Results

Overall most of the websites performed poorly in the five dimensions and require careful reassessment concerning design, content, and readability levels; Only one website performed well in all dimensions, except for readability. Generally, the readability scores indicated that the websites were above the recommended level of reading. None of the websites passed the automated accessibility tests. The expert evaluation using the “Health on the Net” checklist showed good results for most websites.

Conclusions

Breast cancer rates are rising in the MENA region, therefore having comprehensive, accurate, trustworthy, and easy-to-understand health information in their native language is a must. The results from this study show a need for improving the accessibility to breast cancer information websites available to Arabic speakers. The search was limited to three search engines yielding 10 websites and only one tool was used per dimension. Future work is needed to overcome these limitations. Collaboration between multiple stakeholders is necessary to develop websites that contain easy-to-read and understand high-quality information.

Quality and readability of online patient information on treatment for erectile dysfunction

Objectives

To investigate the quality and readability of online patient information on treatment for erectile dysfunction using a Google search.

Materials and methods

The results of a Google search for “erectile dysfunction treatment” were reviewed. Webpages that contained written information on erectile dysfunction except those containing scientific publications and paywall protected webpages were included in further analysis. Typographic and treatment information were recorded. Readability was assessed using the Fleisch‐Kincaid grade level, the Gunning‐Fog index, the Coleman‐Liau index, and Simple Measure of Gobbledygook. Website quality was assessed using the DISCERN instrument, Journal of the American Medical Association (JAMA) benchmark criteria, and presence of Health on the net (HON) code certification. Website typography, discussed treatment types, readability scores, and quality measures were reported. Parametric and nonparametric statistical tests were used to compare the data as appropriate dependent on the normality of data.

Results

Eighty‐one webpages were included. Urologists and hospitals were the most common producers with 15 (18%) each. Seventy‐four (91%) webpages contained specific information on treatment for erectile dysfunction and 15 (19%) contained advertisements. Seventeen (21%) webpages were HON code certified. The median DISCERN score was 35 (IQR 26.5‐44) out of 80. The mean combined readability score was 12.32 (SD 1.91). The median JAMA benchmark score was 1 (IQR 1‐2) out of 4. Google rank had a small negative correlation with DISCERN score (τ = −0.16, P = .036). HON code certified webpages had higher DISCERN scores (median of 44 [IQR 35‐58.5] vs 32.5 [IQR 25.25‐42.25], U = 832, Z = 6561, P < .001). A linear regression was used to predict DISCERN score based on meeting each JAMA benchmark criterion (F(2, 78) = 22.7, P < .001) R² = 0.368, P < .001. Within this model the effects of meeting attribution (β = 11.09) and currency (β = 8.79) criterion were significant.

Conclusions

The quality of online information on treatment for erectile dysfunction is generally poor. However, easy to identify markers of quality like HON code certification, or meeting JAMA benchmark criterion for attribution and currency may help patients to navigate to better quality online information on treatment for erectile dysfunction. Webpages are written at senior high school level, above any recommendations for patient medical information. Health professionals should use validated instruments to assess the quality of online information on treatment for erectile dysfunction prior to publication to improve their utility for patients.

[Empowerment for our patients : Innovation in everyday urology]

Patient empowerment includes measures that promote their autonomy and self-determination in the physician-patient relationship. In addition to successful medical treatment, goals include long-term treatment satisfaction and the best possible quality of life for the patients and their social environment. Various initiatives and projects from German urology are already used to empower our patients. Entscheidungshilfe Prostatakrebs (decision aid for prostate cancer) and the German language decision aid for advanced bladder cancer and for the choice of urinary diversion are implemented under the umbrella of the PatientenAkademie (patient academy) of German Urologists. With more than 12,000 users, Entscheidungshilfe Prostatakrebs is very well established in urological care in Germany. The randomized evaluation study with planned 1200 participants is close to the successful completion of recruitment. Another project from the German urology sector is the project "antiCoagulation Help App for SurgERy" (CHASER). The aim is to develop a smartphone-based decision aid for the perioperative management of patients undergoing antithrombotic therapy. Online support groups can also contribute to empowerment and have been analyzed systematically for prostate cancer patients. A large randomized study on this topic is currently in preparation. Continuing advances in digitalization can thus provide us with useful support in order to provide individual information to our patients. The German Urological Association and its PatientenAkademie have been committed to this for many years.

Quantitative readability analysis of websites providing information on traumatic brain injury and epilepsy: A need for clear communication

OBJECTIVE

The use of the Internet for health-related questions is increasing, but it is not clear whether individuals can understand the information available online. Most health organizations recommend that health educational materials (HEMs) be written below the sixth grade reading level. This study was designed to evaluate the readability level of available online HEMs pertaining to traumatic brain injury (TBI), epilepsy, and posttraumatic epilepsy (PTE).

METHODS

This cross-sectional readability assessment included HEMs from TBI and epilepsy stakeholder organizations and those obtained from four Internet searches. The search strategy was designed to replicate a nonmedical individual's keyword searches. Each HEM was assessed with an online automated readability tool using three indices (Flesch Reading Ease Score, Flesch-Kincaid Grade Level, and Simple Measure of Gobbledygook). Findings were compared as a function of organization type (journalistic news or health organization), targeted medical condition (TBI, epilepsy, or PTE), or content topic (patient health education, clinical research education, or both).

RESULTS

Readability analysis of 405 identified HEMs revealed scores above the sixth grade reading level recommendation. Only 6.2% of individual HEMs met the sixth grade recommendation. Journalistic news organizations' HEMs had similar readability levels to health organizations' HEMs. PTE-related HEMs required the highest readability level, >11th grade (P < .001). There were significant differences in the readability scores (P < .01 for all indices) among HEMs with information on health education, research education, or both topics. The highest required readability level (>12 grade level) was for HEMs that included both health and research education.

SIGNIFICANCE

The majority of TBI-, epilepsy-, and PTE-related online HEMs do not meet the sixth grade reading recommendation. Improving the readability of HEMs may advance health literacy around TBI, epilepsy, and PTE, leading to more effective participant recruitment/retention strategies for future antiepileptogenesis trials in persons with TBI and perhaps better patient-centered outcomes.

Differences in Perceptions of Health Information Between the Public and Health Care Professionals: Nonprobability Sampling Questionnaire Survey

Background

In the new media age, the public searches for information both online and offline. Many studies have examined how the public reads and understands this information but very few investigate how people assess the quality of journalistic articles as opposed to information generated by health professionals.

Objective

The aim of this study was to examine how public health care workers (HCWs) and the general public seek, read, and understand health information and to investigate the criteria by which they assess the quality of journalistic articles.

Methods

A Web-based nonprobability sampling questionnaire survey was distributed to Israeli HCWs and members of the public via 3 social media outlets: Facebook, WhatsApp, and Instagram. A total of 979 respondents participated in the online survey via the Qualtrics XM platform.

Results

The findings indicate that HCWs find academic articles more reliable than do members of the general public (44.4% and 28.4%, respectively, P<.001). Within each group, we found disparities between the places where people search for information and the sources they consider reliable. HCWs consider academic articles to be the most reliable, yet these are not their main information sources. In addition, HCWs often use social networks to search for information (18.2%, P<.001), despite considering them very unreliable (only 2.2% found them reliable, P<.001). The same paradoxes were found among the general public, where 37.5% (P<.001) seek information via social networks yet only 8.4% (P<.001) find them reliable. Out of 6 quality criteria, 4 were important both to HCWs and to the general public.

Conclusions

In the new media age where information is accessible to all, the quality of articles about health is of critical importance. It is important that the criteria examined in this research become the norm in health writing for all stakeholders who write about health, whether they are professional journalists or citizen journalists writing in the new media.

Men's health on the web: an analysis of current resources

INTRODUCTION

Men's health research covers a broad range of topics. Men and women face different barriers to health, with men almost universally having a lower life expectancy than women. Access to high-quality information on men's health topics is potentially an important part of engaging men with medical services. We aim to assess the quality of men's health resources available on the internet across 4 developed countries using a tier-based rating system as well as the World Health Organisation Health on the Net (HON) standards.

METHODS

The Google search engine imbedded with the Health on the Net toolbar was used to assess 357 websites across Australia, Canada, America and United Kingdom using the search term 'men's health'. The websites were further subdivided into 3 tiers by 2 independent investigators, with tier 1 websites defined as government or health organisation sponsored, tier 2 websites defined as being sponsored by health services such as private clinics and insurance providers, and tier 3 websites being websites that did not meet criteria for the first 2 tiers.

RESULTS

Overall, 28% of websites were rated as tier 1, 26% as tier 2 and 46% as tier 3. The HONcode accreditation was overall 39% of tier 1 websites. The majority of websites reviewed were in the tier 3 category, and 35% of overall websites being non-health or non-medically related.

DISCUSSION

The lack of 'relevant' and HONcode-accredited websites relating to men's health should be appreciated by health care professionals.

Analysis of the Patient Information Quality and Readability on Esophagogastroduodenoscopy (EGD) on the Internet

Objective

Patients are increasingly using the Internet to inform themselves of health-related topics and procedures, including EGD. We analyzed the quality of information and readability of websites after a search on 3 different search engines.

Methods

We used an assessment tool for website quality analysis that we developed in addition to using validated instruments for website quality, Global Quality Score (GQS) and Health on Net (HON) certification. The readability was assessed using Flesch-Kincaid Reading Ease (FRE) and Flesch-Kincaid Grade level (FKG). 30 results of each search terms 'EGD' and 'Upper Endoscopy' from Google and 15 each from Bing and Yahoo were analyzed. A total of 45 websites were included from 100 URLs after removing duplicates, video links, and journal articles.

Results

Only 3 websites were found to have good quality and comprehensive and authentic information. These websites were https://www.healthline.com, https://www.uptodate.com, and https://www.emedicine.medscape.com. There were additional 13 sites with moderate quality of information. The mean Flesch-Kincaid Reading Ease (FRE) score was 46.92 (range 81.6-6.5). The mean Flesch-Kincaid Grade level (FKG) was 11th grade, with a range of 6th grade to 12th grade and above making them difficult to read.

Conclusions

Our study shows that there are quite a few websites with moderate quality content. We recommend 3 comprehensive and authentic websites out of 45 URLs analyzed for information on Internet for EGD. In addition, the readability of the websites was consistently at a higher level than recommended by AMA at 11th grade level. In addition, we identified 3 websites with moderate quality content written at 8th grade and below readability level. We feel that gastroenterologists can help their patients better understand this procedure by directing them to these comprehensive websites.