Online Patient Education Materials for Common Sports Injuries Are Written at Too-High of a Reading Level: A Systematic Review

Content analysis of oral (mouth) cancer-related posts on Instagram

OBJECTIVE

To examine the content of Instagram posts about oral cancer and assess its usefulness in promoting oral cancer awareness and early detection practices.

METHODS

A systematic search of Instagram for posts about oral (mouth) cancer was conducted using the hashtags #oral cancer and #mouth cancer. Posts usefulness in promoting awareness and early detection was assessed using the early detection usefulness score, and caption readability was assessed using the Flesch Kincaid readability score.

RESULTS

A total of 81,000 posts were identified, and 200 posts were thoroughly evaluated. Included posts gathered a total of 48,118 (mean = 420.59 likes) and 27,898 views. Most posts (81.5%) were educational to the lay person, and India and the UK were the major contributors. The most discussed topics were prevention and early detection (55%). Representative clinical images were present in 35.5% of posts. Only 9.5% of posts mentioned the source of information, and the mean usefulness score was only 2.1 out of 10. The mean reading ease score was 56.7 ± 43.8 (range from 1 to 98 out of 100).

CONCLUSION

Instagram shows potential for promoting oral cancer awareness, particularly in prevention and early detection. However, concerns regarding content quality, scientific validity, and clarity persist.

Comparison of ChatGPT versions in informing patients with rotator cuff injuries

Background

The aim of this study is to evaluate whether Chat Generative Pretrained Transformer (ChatGPT) can be recommended as a resource for informing patients planning rotator cuff repairs, and to assess the differences between ChatGPT 3.5 and 4.0 versions in terms of information content and readability.

Methods

In August 2023, 13 commonly asked questions by patients with rotator cuff disease were posed to ChatGPT 3.5 and ChatGPT 4 programs using different internet protocol computers by 3 experienced surgeons in rotator cuff surgery. After converting the answers of both versions into text, the quality and readability of the answers were examined.

Results

The average Journal of the American Medical Association score for both versions was 0, and the average DISCERN score was 61.6. A statistically significant and strong correlation was found between ChatGPT 3.5 and 4.0 DISCERN scores. There was excellent agreement in DISCERN scores for both versions among the 3 evaluators. ChatGPT 3.5 was found to be less readable than ChatGPT 4.0.

Conclusion

The information provided by the ChatGPT conversational system was evaluated as of high quality, but there were significant shortcomings in terms of reliability due to the lack of citations. Despite the ChatGPT 4.0 version having higher readability scores, both versions were considered difficult to read.

How good is ChatGPT at answering patients' questions related to early detection of oral (mouth) cancer?

OBJECTIVES

To examine the quality, reliability, readability, and usefulness of ChatGPT in promoting oral cancer early detection.

STUDY DESIGN

About 108 patient-oriented questions about oral cancer early detection were compiled from expert panels, professional societies, and web-based tools. Questions were categorized into 4 topic domains and ChatGPT 3.5 was asked each question independently. ChatGPT answers were evaluated regarding quality, readability, actionability, and usefulness using. Two experienced reviewers independently assessed each response.

RESULTS

Questions related to clinical appearance constituted 36.1% (n = 39) of the total questions. ChatGPT provided "very useful" responses to the majority of questions (75%; n = 81). The mean Global Quality Score was 4.24 ± 1.3 of 5. The mean reliability score was 23.17 ± 9.87 of 25. The mean understandability score was 76.6% ± 25.9% of 100, while the mean actionability score was 47.3% ± 18.9% of 100. The mean FKS reading ease score was 38.4% ± 29.9%, while the mean SMOG index readability score was 11.65 ± 8.4. No misleading information was identified among ChatGPT responses.

CONCLUSION

ChatGPT is an attractive and potentially useful resource for informing patients about early detection of oral cancer. Nevertheless, concerns do exist about readability and actionability of the offered information.

Readability and Quality of Online Patient Education Materials Concerning Posterior Cruciate Ligament Reconstruction

Objective This study aimed to assess the quality of online patient educational materials regarding posterior cruciate ligament (PCL) reconstruction. Methods We performed a search of the top-50 results on Google® (terms: "posterior cruciate ligament reconstruction," "PCL reconstruction," "posterior cruciate ligament surgery," and "PCL surgery") and subsequently filtered to rule out duplicated/inaccessible websites or those containing only videos (67 websites included). Readability was assessed using six formulas: Flesch-Kincaid Reading Ease (FRE), Flesch-Kincaid Grade Level (FKG), Gunning Fog Score (GF), Simple Measure of Gobbledygook (SMOG) Index, Coleman-Liau Index (CLI), Automated Readability Index (ARI); quality was assessed using the JAMA benchmark criteria and recording the presence of the HONcode seal. Results The mean FRE was 49.3 (SD 11.2) and the mean FKG level was 8.09. These results were confirmed by the other readability formulae (average: GF 8.9; SMOG Index 7.3; CLI 14.7; ARI 6.5). A HONcode seal was available for 7.4 % of websites. The average JAMA score was 1.3. Conclusion The reading level of online patient materials concerning PCL reconstruction is too high for the average reader, requiring high comprehension skills. Practice implications Online medical information has been shown to influence patient healthcare decision processes. Patient-oriented educational materials should be clear and easy to understand.

What are patients asking and reading online? An analysis of online patient searches for rotator cuff repair

BACKGROUND

Patients undergoing rotator cuff surgery often search the internet for information regarding the procedure. One popular source, Google, compiles frequently asked questions and links to websites that may provide answers. This study provides an analysis of the most frequently searched questions associated with rotator cuff surgery. We hypothesize that there will be distinct search patterns associated with online queries about rotator cuff surgery that could provide unique insights into patient concerns.

METHODS

A set of search terms were entered into Google Web Search using a clean-install Google Chrome browser. Frequently associated questions and their webpages were extracted to a database via a data mining extension. Questions were categorized by topics relevant for rotator cuff arthroscopy. Websites were categorized by source and scored for quality using the JAMA Benchmark Criteria. Pearson's χ2 tests were used to analyze nominal data. Student t tests were performed to compare JAMA Benchmark Scores.

RESULTS

Of the 595 questions generated from the initial search, 372 unique questions associated with 293 websites were extracted and categorized. The most popular question topics were activities/restrictions (20.7%), pain (18.8%), and indications/management (13.2%). The 2 most common websites searched were academic (35.2%) and medical practice (27.4%). Commercial websites were significantly more likely to be associated with questions about cost (57.1% of all cost questions, P = .01), anatomy/function (62.5%, P = .001), and evaluation of surgery (47.6%, P < .001). Academic websites were more likely to be associated with questions about technical details of surgery (58.1%, P < .001). Medical practice and social media websites were more likely associated with activities/restrictions (48.1%, P < .001, and 15.6%, P < .001, respectively). Government websites were more likely associated with timeline of recovery (12.8%, P = .01). On a scale of 0-4, commercial and academic websites had the highest JAMA scores (3.06 and 2.39, respectively).

CONCLUSION

Patients seeking information regarding rotator cuff repair primarily use the Google search engine to ask questions regarding postoperative activity and restrictions, followed by pain, indications, and management. Academic websites, which were associated with technical details of surgery, and medical practice websites, which were associated with activities/restrictions, were the 2 most commonly searched resources. These results emphasize the need for orthopedic surgeons to provide detailed and informative instructions to patients undergoing rotator cuff repair, especially in the postoperative setting.

Differences in the Medical Advisability of Online Pitching Recommendations for Youth Softball Players Based on Website Source

Background

Guidelines regarding injury prevention in fast-pitch softball pitchers have yet to be widely adopted, risking the online dissemination of misleading advice.

Purpose/Hypothesis

The purpose of this study was to assess the source and medical advisability of online pitching recommendations for youth softball players and highlight the risk of misinformation. It was hypothesized that many popular websites would contain content discordant with current medical guidelines regarding windmill softball pitching recommendations.

Study Design

Cross-sectional study.

Methods

A Google search using the phrase "youth softball pitching recommendations" was performed. Up to 100 websites were extracted and analyzed for website source type (commercial, medical/educational, or athletic organization) and informational quality (advisable, neutral, or discordant). The latter was determined with respect to the STOP Sports Injuries guidelines for arm injury prevention in youth softball players. Descriptive and inferential statistics were used to assess potential associations between website source type and the informational quality of content therein.

Results

A total of 86 websites were included in the analysis. Website source type was significantly predictive of informational quality (P = .018). Among the 3 source types, medical/educational websites had the highest proportion classified as advisable (12/24 [50.0%]) and the lowest proportion classified as discordant (3/24 [12.5%]). Only 17.6% (6/34) of commercial websites were advisable, and advisable websites as a whole were more likely to be from medical/educational sources than athletic organization (P = .016) or commercial (P = .026) sources. The advisability rate among all websites was 25.6% (22/86). Although there was a significant association between position in the search results (first 10 vs remaining 76) and website source type (P = .006), there was no association between position and informational quality (P = .116). The first 10 websites, which trended toward greater advisability than the remaining 76 websites (P = .060), were more likely than the remaining websites to be medical/educational sources (P = .002).

Conclusion

Website source type was significantly predictive of medical advisability. Medical/educational websites were the most advisable, while commercial and athletic organization websites were especially poor in their advisability. The overall advisability rate was only 25.6%. When making recommendations to patients, sports medicine providers should highlight the prevalence of discordant online softball pitching guidelines and take the opportunity to share medically advisable resources.

Evaluation of quality, readability, suitability, and usefulness of online resources available to cancer survivors

PURPOSE

The aim of this study was to evaluate the quality, readability, suitability, and usefulness of resources publicly available to adult cancer survivors (aged 18 +) who have completed primary treatment.

METHODS

Resources were identified in July 2021 through Google. Search completeness was verified using Yahoo, Bing, and MedlinePlus. Retrieved resources were assessed for quality using the DISCERN, readability, suitability using the Suitability Assessment Measure (SAM), and usefulness based on a list of unmet needs and self-management skills derived from the literature. Descriptive analyses were conducted, and a cluster analysis identified the highest-scoring resources.

RESULTS

Forty-five resources were included. The mean DISCERN score was fair at 63.3% (SD 13.7%) with low-rated items being sources, publication date, and risks and mechanisms of treatment. The mean reading grade level was 11.19 (SD 1.61, range 8-16) with only one resource scoring an 8. The mean SAM score was in the adequate range at 48.2% (SD 10.6%), with graphics being the lowest-rated section. On average, included resources addressed 57.7% (SD 27.3%) of the unmet needs and 48.4% (SD 20.9%) of the self-management skills, the least addressed being problem-solving.

CONCLUSION

Quality and suitability were fair, whereas readability exceeded recommended levels. Only one resource had a superior score in both quality and suitability.

IMPLICATIONS FOR CANCER SURVIVORS

The most pressing need is to develop resources for cancer survivors that address their unmet needs and are accessible in terms of literacy. Study findings outline the highest-scoring resources currently available to survivors, families, and clinicians.