An Evaluation of the Quality of COVID-19 Internet Resources for Cancer Patients

The quality of YouTube videos on radiotherapy and prostatectomy for prostate cancer

INTRODUCTION

Prostate cancer ranks as the third leading cause of death among Canadian men and is primarily treated with radiotherapy and prostatectomy. Given YouTube's significant global traffic, patients often turn to it for information on treatment and side effects. This study assessed YouTube videos for prostate cancer patients, focusing on radiotherapy, prostatectomy, and side effect management.

METHODS

The study analyzed 50 YouTube videos, comparing their accuracy and coverage against the National Comprehensive Cancer Network (NCCN), UpToDate, and cancer.ca. Two raters were involved in the review of the videos to ensure inter-rater reliability.

RESULTS

Video lengths ranged from 1-60 minutes (mean 11 minutes) and creation dates ranged from 2012-2021. Videos were presented by physicians, patients, or allied health professionals (75%, 16%, and 8%, respectively). Results showed physician presenters had a Video Power Index (video popularity) of 23.45, while patient presenters had an average of 61.36 (almost three times as popular as physician-led videos). The overall accuracy of videos showed that 82% demonstrated completely accurate and detailed information, 18% showed partially complete information, and 76% showed no biased information. No false information was found in any videos.

CONCLUSIONS

This study showed that while the YouTube informational videos included good coverage of treatment side effects, there were gaps in information regarding quality of life after treatment or management of side effects. Information from this study can benefit the provider-patient interaction, with the hope that healthcare providers create more videos on quality of life after treatment and management of side effects to satisfy patient needs.

A comprehensive evaluation of ChatGPT consultation quality for augmentation mammoplasty: A comparative analysis between plastic surgeons and laypersons

OBJECTIVES

ChatGPT has gained significant popularity as a source of healthcare information among the general population. Evaluating the quality of chatbot responses is crucial, requiring comprehensive and qualitative analysis. This study aims to assess the answers provided by ChatGPT during hypothetical breast augmentation consultations across various categories and depths. The evaluation involves the utilization of validated tools and a comparison of scores between plastic surgeons and laypersons.

METHODS

A panel consisting of five plastic surgeons and five laypersons evaluated ChatGPT's responses to 25 questions spanning consultation, procedure, recovery, and sentiment categories. The DISCERN and PEMAT tools were employed to assess the responses, while emotional context was examined through ten specific questions. Additionally, readability was measured using the Flesch Reading Ease score. Qualitative analysis was performed to identify the overall strengths and weaknesses.

RESULTS

Plastic surgeons generally scored lower than laypersons across most domains. Scores for each evaluation domain varied by category, with the consultation category demonstrating lower scores in terms of DISCERN reliability, information quality, and DISCERN score. Plastic surgeons assigned significantly lower overall quality ratings to the procedure category compared to other question categories. They also gave lower emotion scores in the procedure category compared to laypersons. The depth of the questions did not impact the scoring.

CONCLUSIONS

Existing health information evaluation tools may not be entirely suitable for comprehensively evaluating the quality of individual responses generated by ChatGPT. Consequently, the development and implementation of appropriate evaluation tools to assess the appropriateness and quality of AI consultations are necessary.

An evaluation of the suitability, readability, quality, and usefulness of online resources for patients with cancer receiving immunotherapy

OBJECTIVE

To evaluate the suitability, readability, quality, and usefulness of available online resources about the side effects of immunotherapy.

METHODS

Resources were identified through online searches (Google, Microsoft Bing, and Yahoo search engines) and evaluated using the Suitability Assessment Materials (SAM), three readability formulae, the DISCERN tool (quality), and a checklist of symptom management strategies and self-management skills (usefulness). Hierarchical cluster analysis identified resources scoring highly for suitability and quality. Factors associated with the clusters were identified using Fischer's exact tests.

RESULTS

The 51 resources retained for evaluation were of low suitability and quality and exceeded recommended reading levels. Resources were lacking key features to support learning (e.g., illustrations) as well as information about the risks of treatments and the effect of treatments on quality of life. Few resources addressed key self-management skills of problem-solving and taking action. The cluster analysis identified "low" and "moderate" clusters for suitability and quality, and pamphlets and booklets were more likely to be in the moderate cluster as compared to webpages.

CONCLUSION

There is a lack of high-quality resources to support patients' learning to self-manage side effects of immunotherapy.

PRACTICE IMPLICATIONS

Results identify the best available online resources for self-managing side effects of immunotherapy and gaps for future work. Based on findings, a template for patient education materials was developed.