The Use of eHealth Practices by United States Patients with Self-Reported Glaucoma

Performance of Artificial Intelligence Chatbots on Glaucoma Questions Adapted From Patient Brochures

Introduction With the potential for artificial intelligence (AI) chatbots to serve as the primary source of glaucoma information to patients, it is essential to characterize the information that chatbots provide such that providers can tailor discussions, anticipate patient concerns, and identify misleading information. Therefore, the purpose of this study was to evaluate glaucoma information from AI chatbots, including ChatGPT-4, Bard, and Bing, by analyzing response accuracy, comprehensiveness, readability, word count, and character count in comparison to each other and glaucoma-related American Academy of Ophthalmology (AAO) patient materials. Methods Section headers from AAO glaucoma-related patient education brochures were adapted into question form and asked five times to each AI chatbot (ChatGPT-4, Bard, and Bing). Two sets of responses from each chatbot were used to evaluate the accuracy of AI chatbot responses and AAO brochure information, and the comprehensiveness of AI chatbot responses compared to the AAO brochure information, scored 1-5 by three independent glaucoma-trained ophthalmologists. Readability (assessed with Flesch-Kincaid Grade Level (FKGL), corresponding to the United States school grade levels), word count, and character count were determined for all chatbot responses and AAO brochure sections. Results Accuracy scores for AAO, ChatGPT, Bing, and Bard were 4.84, 4.26, 4.53, and 3.53, respectively. On direct comparison, AAO was more accurate than ChatGPT (p=0.002), and Bard was the least accurate (Bard versus AAO, p<0.001; Bard versus ChatGPT, p<0.002; Bard versus Bing, p=0.001). ChatGPT had the most comprehensive responses (ChatGPT versus Bing, p<0.001; ChatGPT versus Bard p=0.008), with comprehensiveness scores for ChatGPT, Bing, and Bard at 3.32, 2.16, and 2.79, respectively. AAO information and Bard responses were at the most accessible readability levels (AAO versus ChatGPT, AAO versus Bing, Bard versus ChatGPT, Bard versus Bing, all p<0.0001), with readability levels for AAO, ChatGPT, Bing, and Bard at 8.11, 13.01, 11.73, and 7.90, respectively. Bing responses had the lowest word and character count. Conclusion AI chatbot responses varied in accuracy, comprehensiveness, and readability. With accuracy scores and comprehensiveness below that of AAO brochures and elevated readability levels, AI chatbots require improvements to be a more useful supplementary source of glaucoma information for patients. Physicians must be aware of these limitations such that patients are asked about existing knowledge and questions and are then provided with clarifying and comprehensive information.

Lifestyle Measures for Glaucoma Patients: An Objective Social Media Content Analysis

Aim

In this study, we analyze the content quality and characteristics of the most viewed search results on various internet platforms related to lifestyle measures for patients with glaucoma.

Materials and methods

In this internet-based cross-sectional study, we used search keywords "glaucoma," "lifestyle," "glaucoma," and "exercise" on the most popular internet platforms-Google, Facebook, YouTube, and Reddit. The top 30 Google searches about each of the keyword combinations were identified. We also assessed the first 30 videos on YouTube and Facebook Watch, the first 30 Reddit posts and the first 30 Google images about each of the keyword combinations. The quality of content from the platforms was evaluated by three independent reviewers using the well-established Sandvik score, Health on Net (HON) code, and risk score for different uploaders. The quality of content regarding lifestyle measures in glaucoma uploaded by healthcare professionals (HCPs) was further evaluated.

Results

The established criteria resulted in 48 websites from the Google search engine, 22 videos from YouTube, 37 posts from Reddit, and 28 videos from Facebook Watch, which were included in the final analysis. The mean Sandvik scores were 11.14 ± 1.8 (Google webpages), 10.4 ± 2.19 (YouTube videos), 10.54 ± 2.21 (Facebook Watch), and 4.24 ± 1.18 (Reddit). The mean risk scores were 0.22 ± 0.68 (YouTube videos), 0.18 ± 0.47 (Facebook Watch), and 0.11 ± 0.31 (Reddit). The mean HON code scores were 5.45 ± 1.62 (YouTube), 6.55 ± 1.44 (Google webpages), 5.29 ± 1.04 (Facebook Watch), and 8.27 ± 3.05 (Reddit). The content uploaded by HCPs was primarily from ophthalmologists and had significantly (p < 0.05) higher content quality scores. The majority of the content recommended aerobic exercise as a lifestyle measure in patients with glaucoma as an adjuvant to medical and surgical management.

Conclusion

The majority of the content regarding lifestyle measures in glaucoma was uploaded by HCPs and had medically accurate and well-referenced information, especially on Google and YouTube.

Clinical significance

Primary care physicians and ophthalmologists can reliably use social media content to guide recently diagnosed patients about the requisite lifestyle measures.

How to cite this article

Chahal R, Jindal A, Parmar UPS, et al. Lifestyle Measures for Glaucoma Patients: An Objective Social Media Content Analysis. J Curr Glaucoma Pract 2023;17(3):141-148.

What Glaucoma Patients Are Reading on the Internet: A Systematic Analysis of Online Glaucoma Content

PURPOSE

To systematically assess the content, quality, accuracy, navigability, and readability of the highest ranked online information regarding glaucoma.

DESIGN

Internet-based, cross-sectional study.

PARTICIPANTS

Not applicable.

METHODS

The top 15 Google websites that resulted from the search term "glaucoma" were independently evaluated by 3 glaucoma specialists for content, accuracy (1-4 scale), and navigability (1-3 scale) in March 2020. Content was evaluated by answers to 25 questions regarding information most relevant to glaucoma patients (0-2 scale). Readability (Flesch-Kincaid grade level) and quality (Quality Component Scoring System) were also analyzed.

MAIN OUTCOME MEASURES

Overall content, accuracy, navigability, quality component scores, and reading grade level as described. Additional analyses included type of websites, presence and type of graphic content.

RESULTS

The mean (standard deviation) content score for all websites was 28 (7.6) of 50 possible points (range, 15-45). Inter-rater reliability for content score was good (0.619, 0.746, 0.872 for 3 pairwise comparisons between graders). Mean accuracy score was 9.8 (2.2) of 12 possible points (range, 5-12). Mean navigability score was 7.5 (1.4) of 9 possible points (range, 5-9). Mean reading grade level was 9.3 (1.7) with a range of 7-13 reading grades. Mean quality component score was 7.7 (2.6) of 13 possible points (range, 3-11). Six websites (40%) attempted to visually simulate glaucoma symptoms as tunnel vision (n = 5) and black spots (n = 1). Google rank did not correlate with any of the measures.

CONCLUSIONS

The most readily accessible online information about glaucoma varies in content, quality, accuracy, navigability, and readability. This systematic analysis identifies potential areas of improvement.