Health literacy among neurosurgery and other surgical subspecialties: Readability of online patient materials found with Google

Text-to-video generative artificial intelligence: sora in neurosurgery

Artificial intelligence (AI) has increased in popularity in neurosurgery, with recent interest in generative AI algorithms such as the Large Language Model (LLM) ChatGPT. Sora, an innovation in generative AI, leverages natural language processing, deep learning, and computer vision to generate impressive videos from text prompts. This new tool has many potential applications in neurosurgery. These include patient education, public health, surgical training and planning, and research dissemination. However, there are considerable limitations to the current model such as physically implausible motion generation, spontaneous generation of subjects, unnatural object morphing, inaccurate physical interactions, and abnormal behavior presentation when many subjects are generated. Other typical concerns are with respect to patient privacy, bias, and ethics. Further, appropriate investigation is required to determine how effective generative videos are compared to their non-generated counterparts, irrespective of any limitations. Despite these challenges, Sora and other iterations of its text-to-video generative application may have many benefits to the neurosurgical community.

Readability of Neurosurgical Patient Education Resources by the American Association of Neurological Surgeons

BACKGROUND

The importance of patient education has become increasingly apparent in recent years. A prominent patient education tool in neurosurgery is the Neurosurgical Conditions and Treatments page provided by the American Association of Neurological Surgeons (AANS). This study aimed to investigate the readability of this resource page as many new articles have been incorporated in the past decade.

METHODS

Flesch-Kincaid Grade Level and Flesch-Kincaid Reading Ease scores were calculated for each article. One-way analysis of variance and Scheffé, Tukey-Kramer, Bonferroni, Fisher least significant difference, and Dunnett test post hoc analyses were conducted to determine differences between each category with respect to their mean grade level and reading ease scores.

RESULTS

Overall mean (SD) Flesch-Kincaid Reading Ease score was 40.2 (12.24), and overall mean (SD) Flesch-Kincaid Grade Level score was 7.48 (1.26). Significant differences were found between mean reading ease scores between each categorization by the AANS (P = 0.014). No significant differences were found between mean grade level score for each categorization (analysis of variance, P = 0.154).

CONCLUSIONS

As compared to a single previous investigation conducted 10 years ago, the readability of articles has changed modestly, and the reading grade level remains well above the recommendations by the American Medical Association and National Institutes of Health. The 6 new articles introduced in the past decade have demonstrated similar readability, presenting a persistent challenge in the realm of patient education in neurosurgery.

BPPV Information on Google Versus AI (ChatGPT)

OBJECTIVE

To quantitatively compare online patient education materials found using traditional search engines (Google) versus conversational Artificial Intelligence (AI) models (ChatGPT) for benign paroxysmal positional vertigo (BPPV).

STUDY DESIGN

The top 30 Google search results for "benign paroxysmal positional vertigo" were compared to the OpenAI conversational AI language model, ChatGPT, responses for 5 common patient questions posed about BPPV in February 2023. Metrics included readability, quality, understandability, and actionability.

SETTING

Online information.

METHODS

Validated online information metrics including Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease (FRE), DISCERN instrument score, and Patient Education Materials Assessment Tool for Printed Materials were analyzed and scored by reviewers.

RESULTS

Mean readability scores, FKGL and FRE, for the Google webpages were 10.7 ± 2.6 and 46.5 ± 14.3, respectively. ChatGPT responses had a higher FKGL score of 13.9 ± 2.5 (P < .001) and a lower FRE score of 34.9 ± 11.2 (P = .005), both corresponding to lower readability. The Google webpages had a DISCERN part 2 score of 25.4 ± 7.5 compared to the individual ChatGPT responses with a score of 17.5 ± 3.9 (P = .001), and the combined ChatGPT responses with a score of 25.0 ± 0.9 (P = .928). The average scores of the reviewers for all ChatGPT responses for accuracy were 4.19 ± 0.82 and 4.31 ± 0.67 for currency.

CONCLUSION

The results of this study suggest that the information on ChatGPT is more difficult to read, of lower quality, and more difficult to comprehend compared to information on Google searches.

Impact of Educational Background on Preoperative Disease Severity and Postoperative Outcomes Among Patients With Cervical Spondylotic Myelopathy

STUDY DESIGN

Retrospective review of a prospectively maintained database.

OBJECTIVE

Assess differences in preoperative status and postoperative outcomes among patients of different educational backgrounds undergoing surgical management of cervical spondylotic myelopathy (CSM).

SUMMARY OF BACKGROUND DATA

Patient education level (EL) has been suggested to correlate with health literacy, disease perception, socioeconomic status (SES), and access to health care.

METHODS

The CSM data set of the Quality Outcomes Database (QOD) was queried for patients undergoing surgical management of CSM. EL was grouped as high school or below, graduate-level, and postgraduate level. The association of EL with baseline disease severity (per patient-reported outcome measures), symptoms >3 or ≤3 months, and 24-month patient-reported outcome measures were evaluated.

RESULTS

Among 1141 patients with CSM, 509 (44.6%) had an EL of high school or below, 471 (41.3%) had a graduate degree, and 161 (14.1%) had obtained postgraduate education. Lower EL was statistically significantly associated with symptom duration of >3 months (odds ratio=1.68), higher arm pain numeric rating scale (NRS) (coefficient=0.5), and higher neck pain NRS (coefficient=0.79). Patients with postgraduate education had statistically significantly lower Neck Disability Index (NDI) scores (coefficient=-7.17), lower arm pain scores (coefficient=-1), and higher quality-adjusted life-years (QALY) scores (coefficient=0.06). Twenty-four months after surgery, patients of lower EL had higher NDI scores, higher pain NRS scores, and lower QALY scores ( P <0.05 in all analyses).

CONCLUSIONS

Among patients undergoing surgical management for CSM, those reporting a lower educational level tended to present with longer symptom duration, more disease-inflicted disability and pain, and lower QALY scores. As such, patients of a lower EL are a potentially vulnerable subpopulation, and their health literacy and access to care should be prioritized.

Readability of cerebrovascular diseases online educational material from major cerebrovascular organizations

INTRODUCTION

The internet is an essential resource for patients and their loved ones to understand their medical conditions, and professional medical organizations have taken great strides to develop educational material targeting patients. The average American reads at a seventh to eighth grade reading level, hence it is important to understand the readability of this medical information to ensure patients comprehend what is being presented.

METHODS

In January 2023, online patient education material was downloaded from major cerebrovascular healthcare organizations and assessed using eight assessments, including Bormuth Cloze Mean, Bormuth Grade Placement, Coleman-Liau (grade levels), Coleman-Liau (predictive cloze scores), Flesch Reading Ease (FRE), and Fry.

RESULTS

A total of 32 files were extracted from six organizations and analyzed across 15 readability measures. None of the organizations met the federal government guidelines for grade-level readability. This held constant across all measured tests. Two organizations had above a postgraduate level. The FRE graphs do not identify any organizations with material below a ninth grade reading level, while the American Association of Neurological Surgeons (AANS) and the Society of Interventional Radiology (SIR) have a postgraduate readability level. The Fry graphs show similar results, with AANS/CNS Cerebrovascular Section, Society of NeuroInterventional Surgery (SNIS), SIR, and AANS having college-level readability. The lowest readability across all measures is only at an early seventh grade reading level.

CONCLUSIONS

Current health literacy content for cerebrovascular patients is far above the recommended readability level. We provide straightforward suggestions for how major professional organizations should improve their informational material on cerebrovascular diseases to improve patient understanding.

Readability and Quality of Online Health Information Regarding Parathyroidectomy

Objective

Assessment of the readability and quality of online health information regarding parathyroidectomy.

Study Design

Cross-sectional analysis.

Setting

Websites providing patient-oriented health information regarding parathyroidectomy obtained via the Google search engine.

Methods

The top 75 Google search results for “parathyroidectomy,”“parathyroid surgery,” and “parathyroid gland removal” were reviewed. Websites were categorized by website type and country of origin. Readability was assessed by Flesch-Kincaid Grade Level and Simple Measure of Gobbledygook. Website quality was assessed per JAMA benchmark criteria and the DISCERN instrument.

Results

A total of 74 unique websites were evaluated. The mean readability of the assessed websites exceeded the recommended sixth-grade reading level on the Flesch-Kincaid Grade Level and Simple Measure of Gobbledygook (P < .001). Readability did not vary significantly by website type. Websites originating from the United Kingdom were significantly more readable than those from the United States. The majority of assessed websites were of poor quality (n = 42, 56.8%) on assessment based on the DISCERN instrument. Quality varied significantly by website category on the JAMA benchmark criteria (P < .001) and DISCERN score (P = .049) with commercial websites receiving the highest scores. DISCERN score also varied significantly by country of origin (P = .036) with UK sites receiving highest mean DISCERN scores.

Conclusion

Online health information regarding parathyroidectomy is largely of poor quality and is poorly readable for many patients. Institutions utilizing well-defined guidelines for development of patient educational resources may provide online health information of greater quality and readability.

Health Literacy in Neurosurgery: A Scoping Review

OBJECTIVE

Low health literacy is prevalent and associated with suboptimal health outcomes. In neurosurgery, social determinants of health are increasingly recognized as factors underpinning outcomes, as well as access to and use of care. We conducted a scoping review to delineate the scope of existing literature regarding health literacy in the field and facilitate future research.

METHODS

A scoping review was conducted using the PubMed, Embase, and Scopus databases. Titles and abstracts were screened for relevance. Studies meeting prespecified inclusion criteria underwent full text review. Relevant data were extracted.

RESULTS

Of 5056 resultant articles, 57 manuscripts were included. Thirty-seven studies (64.9%) investigated personal health literacy, while the remaining 20 (35.1%) investigated organizational health literacy. Domains of health literacy investigated were science (36, 63.2%), fundamental (20, 35.1%), and civic (1, 1.7%). No studies investigated numeracy. Recall among patients after discussions with neurosurgeons is low. Patient perspectives are often erroneous. Patient informational needs are often unmet. Written patient educational materials are written at a level too complex for the average patients. Videos are mostly of poor quality. Multimodal audiovisual interventions, eBooks, models, and virtual reality are shown to be effective methods for promoting recall.

CONCLUSIONS

Studies examining health literacy in neurosurgery primarily focus on the topic indirectly, most often via written educational materials and recall after educational interventions. Increasing awareness of health literacy among neurosurgeons, assessing health literacy, and incorporating health literacy-informed counseling approaches are warranted to improve patient care.

Readability and quality assessment of internet-based patient education materials related to nasal septoplasty

BACKGROUND

Given that nasal septoplasty is a common procedure in otolaryngology - head and neck surgery, the objective of this study was to evaluate the quality and readability of online patient education materials on septoplasty.

METHODS

A Google search was performed using eight different search terms related to septoplasty. Six different tools were used to assess the readability of included patient education materials. These included the Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning-Fog Index, Simple Measure of Gobbledygook Index, Coleman-Liau Index, and Automated Readability Index. The DISCERN tool was used to assess quality and reliability.

RESULTS

Eighty-five online patient education materials were included. The average Flesch-Reading Ease score for all patient education materials was 54.9 ± 11.5, indicating they were fairly difficult to read. The average reading grade level was 10.5 ± 2.0, which is higher than the recommended reading level for patient education materials. The mean DISCERN score was 42.9 ± 10.5 and 42% (36/85) of articles had DISCERN scores less than 39, corresponding to poor or very poor quality.

CONCLUSION

The majority of online patient education materials on septoplasty are written above the recommended reading levels and have significant deficiencies in terms of their quality and reliability. Clinicians and patients should be aware of the shortcomings of these resources and consider the impact they may have on patients' decision making.