Analysis of the readability of patient education materials from surgical subspecialties

Improving readability and comprehension levels of otolaryngology patient education materials using ChatGPT

OBJECTIVE

A publicly available large language learning model platform may help determine current readability levels of otolaryngology patient education materials, as well as translate these materials to the recommended 6th-grade and 8th-grade reading levels.

STUDY DESIGN

Cross-sectional analysis.

SETTING

Online using large language learning model, ChatGPT.

METHODS

The Patient Education pages of the American Laryngological Association (ALA) and American Academy of Otolaryngology-Head and Neck Surgery (AAO-HNS) websites were accessed. Materials were input into ChatGPT (OpenAI, San Francisco, CA; version 3.5) and Microsoft Word (Microsoft, Redmond, WA; version 16.74). Programs calculated Flesch Reading Ease (FRE) scores, with higher scores indicating easier readability, and Flesch-Kincaid (FK) grade levels, estimating U.S. grade level required to understand text. ChatGPT was prompted to "translate to a 5th-grade reading level" and provide new scores. Scores were compared for statistical differences, as well as differences between ChatGPT and Word gradings.

RESULTS

Patient education materials were reviewed and 37 ALA and 72 AAO-HNS topics were translated. Overall FRE scores and FK grades demonstrated significant improvements following translation of materials, as scored by ChatGPT (p < 0.001). Word also scored significant improvements in FRE and FK following translation by ChatGPT for AAO-HNS materials overall (p < 0.001) but not for individual topics or for subspecialty-specific categories. Compared with Word, ChatGPT significantly exaggerated the change in FRE grades and FK scores (p < 0.001).

CONCLUSION

Otolaryngology patient education materials were found to be written at higher reading levels than recommended. Artificial intelligence may prove to be a useful resource to simplify content to make it more accessible to patients.

Readability of Neurosurgical Patient Education Resources by the American Association of Neurological Surgeons

BACKGROUND

The importance of patient education has become increasingly apparent in recent years. A prominent patient education tool in neurosurgery is the Neurosurgical Conditions and Treatments page provided by the American Association of Neurological Surgeons (AANS). This study aimed to investigate the readability of this resource page as many new articles have been incorporated in the past decade.

METHODS

Flesch-Kincaid Grade Level and Flesch-Kincaid Reading Ease scores were calculated for each article. One-way analysis of variance and Scheffé, Tukey-Kramer, Bonferroni, Fisher least significant difference, and Dunnett test post hoc analyses were conducted to determine differences between each category with respect to their mean grade level and reading ease scores.

RESULTS

Overall mean (SD) Flesch-Kincaid Reading Ease score was 40.2 (12.24), and overall mean (SD) Flesch-Kincaid Grade Level score was 7.48 (1.26). Significant differences were found between mean reading ease scores between each categorization by the AANS (P = 0.014). No significant differences were found between mean grade level score for each categorization (analysis of variance, P = 0.154).

CONCLUSIONS

As compared to a single previous investigation conducted 10 years ago, the readability of articles has changed modestly, and the reading grade level remains well above the recommendations by the American Medical Association and National Institutes of Health. The 6 new articles introduced in the past decade have demonstrated similar readability, presenting a persistent challenge in the realm of patient education in neurosurgery.

A Mobile Application for Child-Focused Tonsillectomy Education: Development and User-Testing

OBJECTIVE

Patient education is central to Enhanced Recovery After Surgery protocols, but child-focused materials are lacking. We developed and piloted a mobile application to support accessible, interactive patient and caregiver education about pediatric tonsillectomy.

METHODS

Thirty children ages 5-12 who were preparing for tonsillectomy, their caregivers, and six attending otolaryngologists participated in a user-testing trial of a web-based prototype. The trial measured feasibility, fidelity, and patient-centered outcomes. Patients and caregivers rated usability/likeability on the mHealth App Usability Questionnaire. Otolaryngologists rated quality on the Mobile App Rating Scale. The full mobile application, "Ready for Tonsillectomy," was then developed for iOS and Android.

RESULTS

Enrollment was 88.2%, retention was 90.0%, and use was 96.3%. Mean (SD) patient ratings for usability/likeability were 6.3 (1.1) out of 7; caregiver ratings were 6.5 (1.1). In common themes from open-ended feedback, patients described the application as helpful and appealing, and caregivers described it as informative, easy to understand, calming, and easy to use. Among caregivers who used the application during recovery, 92.3% reported that it helped them manage their child's pain. Providers would recommend the application to many or all of their patients (mean [SD]: 4.7 [0.5] out of 5). Mean provider ratings for domains of engagement, functionality, aesthetics, information quality, subjective quality, and app-specific value ranged from 4.1 to 4.8 out of 5.

CONCLUSION

Feasibility and fidelity were high. Families and otolaryngologists endorsed the resource as an engaging, informative tool that supports positive coping. Our mobile application offers a patient-centered solution readily scalable to other surgeries.

LEVEL OF EVIDENCE

NA Laryngoscope, 134:2455-2463, 2024.

Large language models and bariatric surgery patient education: a comparative readability analysis of GPT-3.5, GPT-4, Bard, and online institutional resources

BACKGROUND

The readability of online bariatric surgery patient education materials (PEMs) often surpasses the recommended 6th grade level. Large language models (LLMs), like ChatGPT and Bard, have the potential to revolutionize PEM delivery. We aimed to evaluate the readability of PEMs produced by U.S. medical institutions compared to LLMs, as well as the ability of LLMs to simplify their responses.

METHODS

Responses to frequently asked questions (FAQs) related to bariatric surgery were gathered from top-ranked health institutions. FAQ responses were also generated from GPT-3.5, GPT-4, and Bard. LLMs were then prompted to improve the readability of their initial responses. The readability of institutional responses, initial LLM responses, and simplified LLM responses were graded using validated readability formulas. Accuracy and comprehensiveness of initial and simplified LLM responses were also compared.

RESULTS

Responses to 66 FAQs were included. All institutional and initial LLM responses had poor readability, with average reading levels ranging from 9th grade to college graduate. Simplified responses from LLMs had significantly improved readability, with reading levels ranging from 6th grade to college freshman. When comparing simplified LLM responses, GPT-4 responses demonstrated the highest readability, with reading levels ranging from 6th to 9th grade. Accuracy was similar between initial and simplified responses from all LLMs. Comprehensiveness was similar between initial and simplified responses from GPT-3.5 and GPT-4. However, 34.8% of Bard's simplified responses were graded as less comprehensive compared to initial.

CONCLUSION

Our study highlights the efficacy of LLMs in enhancing the readability of bariatric surgery PEMs. GPT-4 outperformed other models, generating simplified PEMs from 6th to 9th grade reading levels. Unlike GPT-3.5 and GPT-4, Bard's simplified responses were graded as less comprehensive. We advocate for future studies examining the potential role of LLMs as dynamic and personalized sources of PEMs for diverse patient populations of all literacy levels.

Generative artificial intelligence chatbots may provide appropriate informational responses to common vascular surgery questions by patients

OBJECTIVES

Generative artificial intelligence (AI) has emerged as a promising tool to engage with patients. The objective of this study was to assess the quality of AI responses to common patient questions regarding vascular surgery disease processes.

METHODS

OpenAI's ChatGPT-3.5 and Google Bard were queried with 24 mock patient questions spanning seven vascular surgery disease domains. Six experienced vascular surgery faculty at a tertiary academic center independently graded AI responses on their accuracy (rated 1-4 from completely inaccurate to completely accurate), completeness (rated 1-4 from totally incomplete to totally complete), and appropriateness (binary). Responses were also evaluated with three readability scales.

RESULTS

ChatGPT responses were rated, on average, more accurate than Bard responses (3.08 ± 0.33 vs 2.82 ± 0.40, p < .01). ChatGPT responses were scored, on average, more complete than Bard responses (2.98 ± 0.34 vs 2.62 ± 0.36, p < .01). Most ChatGPT responses (75.0%, n = 18) and almost half of Bard responses (45.8%, n = 11) were unanimously deemed appropriate. Almost one-third of Bard responses (29.2%, n = 7) were deemed inappropriate by at least two reviewers (29.2%), and two Bard responses (8.4%) were considered inappropriate by the majority. The mean Flesch Reading Ease, Flesch-Kincaid Grade Level, and Gunning Fog Index of ChatGPT responses were 29.4 ± 10.8, 14.5 ± 2.2, and 17.7 ± 3.1, respectively, indicating that responses were readable with a post-secondary education. Bard's mean readability scores were 58.9 ± 10.5, 8.2 ± 1.7, and 11.0 ± 2.0, respectively, indicating that responses were readable with a high-school education (p < .0001 for three metrics). ChatGPT's mean response length (332 ± 79 words) was higher than Bard's mean response length (183 ± 53 words, p < .001). There was no difference in the accuracy, completeness, readability, or response length of ChatGPT or Bard between disease domains (p > .05 for all analyses).

CONCLUSIONS

AI offers a novel means of educating patients that avoids the inundation of information from "Dr Google" and the time barriers of physician-patient encounters. ChatGPT provides largely valid, though imperfect, responses to myriad patient questions at the expense of readability. While Bard responses are more readable and concise, their quality is poorer. Further research is warranted to better understand failure points for large language models in vascular surgery patient education.

Bridging the literacy gap for surgical consents: an AI-human expert collaborative approach

Despite the importance of informed consent in healthcare, the readability and specificity of consent forms often impede patients' comprehension. This study investigates the use of GPT-4 to simplify surgical consent forms and introduces an AI-human expert collaborative approach to validate content appropriateness. Consent forms from multiple institutions were assessed for readability and simplified using GPT-4, with pre- and post-simplification readability metrics compared using nonparametric tests. Independent reviews by medical authors and a malpractice defense attorney were conducted. Finally, GPT-4's potential for generating de novo procedure-specific consent forms was assessed, with forms evaluated using a validated 8-item rubric and expert subspecialty surgeon review. Analysis of 15 academic medical centers' consent forms revealed significant reductions in average reading time, word rarity, and passive sentence frequency (all P < 0.05) following GPT-4-faciliated simplification. Readability improved from an average college freshman to an 8th-grade level (P = 0.004), matching the average American's reading level. Medical and legal sufficiency consistency was confirmed. GPT-4 generated procedure-specific consent forms for five varied surgical procedures at an average 6th-grade reading level. These forms received perfect scores on a standardized consent form rubric and withstood scrutiny upon expert subspeciality surgeon review. This study demonstrates the first AI-human expert collaboration to enhance surgical consent forms, significantly improving readability without sacrificing clinical detail. Our framework could be extended to other patient communication materials, emphasizing clear communication and mitigating disparities related to health literacy barriers.

Readability of Online Information for Parents Concerning Paediatric In-Toeing: An Analysis of the Most Popular Online Public Sources

Background Parents often access online resources to educate themselves on their child's condition. In-toeing, also referred to as pigeon toeing, is a common paediatric condition that has a variety of causes and is often a cause of concern for parents. With the increasing usage of the internet, parents of children with this condition may look to the web for answers. However, to be understood by the average adult, online health information must be written at an elementary school reading level. We hypothesised that currently available online resources regarding in-toeing would score poorly on objective measures of readability and understandability. Methods Patient education materials were identified via three commonly used online search engines (Google.com, Yahoo.com, and Bing.com). The terms "intoeing" and "pigeon toeing" were used for the search. From the top 50 search results, websites were included if directed at educating patients and their families regarding in-toeing. News articles, non-text material (video), industry websites, and articles not related to in-toeing were excluded. The readability was analysed using a specialised website www.readable.com to produce the following three scores: Gunning Fog Index (GFI), Flesch Reading Ease (FRE), and Flesch-Kincaid Grade (FKG). Understandability was calculated using the 19-point Patient Education Materials Assessment Tool (PEMAT). Results After removing duplicates, 84 unique websites were assessed for inclusion. A total of 48 websites and articles (57.14%) met the inclusion criteria. Of note, 23 articles out of 84 (27.38%) were excluded as they were intended for healthcare professionals. The means for the FRE, FKG, and GFI were 57.92 (±12.26), 7.92 (±1.91), and 9.35 (±2.36), respectively. Less than half of online resources had an FRE score at or higher than the recommended reading level for the general population. Mean understandability scores were 69.63% (±11.55%), with only 45.83% of articles being greater than the 70% requirement of adequate understandability. Conclusion Overall, online in-toeing educational materials scored poorly with respect to readability and understandability. Given the popularity of online resources in patient education, we should seek to improve this situation. Articles that are easier to read are thus more accessible to the general public and will aid in the shared decision-making process. Improved patient and parent satisfaction and overall standard of care can be expected.

Readability of patient education materials for bariatric surgery

INTRODUCTION

Bariatric surgery is a successful treatment for obesity, but barriers to surgery exist, including low health literacy. National organizations recommend patient education materials (PEM) not exceed a sixth-grade reading level. Difficult to comprehend PEM can exacerbate barriers to bariatric surgery, especially in the Deep South where high obesity and low literacy rates exist. This study aimed to assess and compare the readability of webpages and electronic medical record (EMR) bariatric surgery PEM from one institution.

METHODS

Readability of online bariatric surgery and standardized perioperative EMR PEM were analyzed and compared. Text readability was assessed by validated instruments: Flesch Reading Ease Formula (FRE), Flesch Kincaid Grade Level (FKGL), Gunning Fog (GF), Coleman-Liau Index (CL), Simple Measure of Gobbledygook (SMOG), Automated Readability Index (ARI), and Linsear Write Formula (LWF). Mean readability scores were calculated with standard deviations and compared using unpaired t-tests.

RESULTS

32 webpages and seven EMR education documents were analyzed. Webpages were overall "difficult to read" compared to "standard/average" readability EMR materials (mean FRE 50.5 ± 18.3 vs. 67.4 ± 4.2, p = 0.023). All webpages were at or above high school reading level: mean FKGL 11.8 ± 4.4, GF 14.0 ± 3.9, CL 9.5 ± 3.2, SMOG 11.0 ± 3.2, ARI 11.7 ± 5.1, and LWF 14.9 ± 6.6. Webpages with highest reading levels were nutrition information and lowest were patient testimonials. EMR materials were sixth to ninth grade reading level: FKGL 6.2 ± 0.8, GF 9.3 ± 1.4, CL 9.7 ± 0.9, SMOG 7.1 ± 0.8, ARI 6.1 ± 1.0, and LWF 5.9 ± 0.8.

CONCLUSION

Surgeon curated bariatric surgery webpages have advanced reading levels above recommended thresholds compared to standardized PEM from an EMR. This readability gap may unintentionally contribute to barriers to surgery and affect postoperative outcomes. Streamlined efforts are needed to create materials that are easier to read and comply with recommendations.

Readability of patient and family education materials on pediatric surgical association websites

PURPOSE

Globally, pediatric surgical association websites present patient/family education materials on an extensive list of conditions, including descriptions of the condition, signs and symptoms, diagnostic modalities, and treatment options. The purpose of this project was to assess the readability of pediatric surgical association websites' patient/family education materials.

METHODS

With IRB approval, we accessed all patient/family education materials on pediatric surgical association websites from around the globe and used multiple grade-level assessments and readability assessments to determine the reading level at which the information is presented.

RESULTS

The American Pediatric Surgical Association (APSA) website and the British Association of Paediatric Surgeons (BAPS) present publicly accessible patient/family education materials. Seventy-four (74) conditions on APSA's website were analyzed. Three grade-level assessments and the Flesch Reading Ease assessment indicated that the articles are written at high school reading levels. No articles were available in languages other than English. BAPS presented 6 conditions, most of which were more readable than their APSA counterparts.

CONCLUSIONS

Our analysis indicates that the patient/family education materials available on pediatric surgical association websites may not be written at a level that is comprehensible by the general population. Potential solutions include re-writing the materials with an emphasis on readability and presenting materials in languages other than English.

LEVEL OF EVIDENCE

V.

Readability Analysis of Otolaryngology Consent Documents on the iMed Consent Platform

INTRODUCTION

The American Medical Association (AMA) and National Institutes of Health (NIH) recommend all patient information and consent materials be provided at the fourth- to sixth-grade level. The iMed Consent platform is used nationally by the Veterans Health Administration and private hospitals. We aimed to assess the readability of otolaryngology consents at the West Los Angeles Veterans Affairs (WLA-VA) hospital to determine whether they conform with AMA/NIH guidelines.

MATERIALS AND METHODS

A readability analysis of 27 otolaryngology iMed consent documents was performed. The main outcome measure was the Flesch-Kincaid Grade Level (FKGL). The setting of the study was an otolaryngology clinic at a major VA hospital. All consents used in the WLA-VA otolaryngology clinic for the month of October 2018 were analyzed using readability metrics. These included the Flesch Reading Ease (FRE) score, the FKGL, the Gunning Fog Index (GFI), Simple Measure of Gobbledygook (SMOG), and Coleman-Liau Index (CLI).

RESULTS

The following means of all consents were calculated for each of the readability metrics: FRE 56.3, FKGL 8.3, GFI 14.5, SMOG 11.3, and CLI 11.2. The standardized anesthesia and blood consent were analyzed separately with the following scores: FRE 45.1, FKGL 11.7, GFI 15.5, SMOG 14.6, and CLI 12.6. The average FKGL of the consents was found to be significantly above the sixth-grade level (P: .0013).

CONCLUSION

The average grade level of the otolaryngology iMed consents reviewed was at a reading level above the AMA/NIH recommendations. This objective measure should be taken into consideration when revising the iMed system and in the creation of future standardized consents. Readability analysis does not take into consideration the significant variance that exists as part of the verbal consent process that takes place between patient and provider.

Readability of Online Patient Education Materials for Total Joint Arthroplasty: A Systematic Review

BACKGROUND

As the demand for total joint arthroplasty continues to grow, patients increasingly turn to internet sources for accessible orthopaedic health information. We sought to evaluate the readability of online hip and knee arthroplasty patient education materials (PEMs).

METHODS

Our systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-P guidelines (PROSPERO registration of the study protocol: CRD42022358872, September 19th, 2022). PubMed, EBSCOhost, Medline, and Google Scholar electronic databases were utilized to identify all studies evaluating online PEMs related to total joint arthroplasty between January 1, 2000, and August 1, 2022. The quality of studies was assessed with the Joanna Briggs Institute scale.

RESULTS

Our initial search yielded 360 publications which were then screened for appropriate studies aligned with our review's purpose. A total of eleven cross-sectional studies were included. The total sample size across the studies consisted of 662 PEMs. Five studies evaluated professional or academic orthopaedic websites, six evaluated PEMs through web engines, and three evaluated apps or commercial websites. Most included studies found PEMs to be well above the recommended 6th-grade reading level.

CONCLUSION

Arthroplasty PEMs produced by orthopaaedic websites, web searches, and apps have readability scores well above the recommended levels. Given the importance of health literacy on patient outcomes and satisfaction, work needs to be done to improve the readability of these materials.

Patient education in pediatric ophthalmology: a systematic review

BACKGROUND

Patient and caregiver education may have the potential to improve understanding, adherence, and disease outcomes in pediatric ophthalmology. Research suggests that common clinical practices may result in suboptimal patient understanding. We summarize the current literature on patient education interventions in pediatric ophthalmology.

METHODS

A predefined search strategy was used to systematically review the PubMed database. Peer-reviewed published studies that utilized a specific educational intervention regarding any condition in pediatric ophthalmology and measured its impact were included.

RESULTS

Our search method yielded 453 studies; 30 passed title and abstract screening, and 14 were included in the final analysis. Of the 14 studies, 9 were randomized controlled trials. Eight studies relied solely on printed information, 2 were computer-based, 1 was an animated video, and 3 were multifactorial. Outcome measures included adherence (6/14), caregiver knowledge (6/14), psychological impact (4/14), visual outcome (2/14), and clinic attendance (1/14).

CONCLUSIONS

The educational interventions varied widely in methodology, content, and focus of intervention; nonetheless, they were widely successful across outcome measures. A number of studies featured highly time- and cost-effective interventions that resulted in increased knowledge, decreased anxiety, enhanced adherence, and improved visual outcomes. Educational efforts may be especially beneficial among non-native language speakers, particularly via image-based means of communication.

Improving surgical patient education materials using informational design with visual elements

Patient education materials (PEMs) serve as a foundation for educating patients and families across all surgical fields but are often not understandable. The National Institute of Health (NIH) recommends that PEMs be written at a grade 6-7 reading level; however, most current materials exceed that measure.3 Lack of understandable and appropriate surgical PEMs compounds the difficulties that low health literacy patients face with resultant poor surgical outcomes.2,3 The challenge for surgeons is to adequately educate patients pre-operatively and post-operatively on the complexities of surgery. Another challenge is to compact decades of education and training into an easy-to-understand medium for patients. To address this challenge, many physicians have utilized visual aids to improve PEM efficacy. While visual aids are a critical piece of education materials, they must be designed intentionally to be effective. The most important consideration is that the PEM communicates the information clearly to users. With this in mind, we created a framework for productive utilization of visual aids by integrating the C.A.R.P. graphic design technique into an existing surgical PEM to enhance communication and understandability.

Readability and Quality of Online Health Information Regarding Parathyroidectomy

Objective

Assessment of the readability and quality of online health information regarding parathyroidectomy.

Study Design

Cross-sectional analysis.

Setting

Websites providing patient-oriented health information regarding parathyroidectomy obtained via the Google search engine.

Methods

The top 75 Google search results for “parathyroidectomy,”“parathyroid surgery,” and “parathyroid gland removal” were reviewed. Websites were categorized by website type and country of origin. Readability was assessed by Flesch-Kincaid Grade Level and Simple Measure of Gobbledygook. Website quality was assessed per JAMA benchmark criteria and the DISCERN instrument.

Results

A total of 74 unique websites were evaluated. The mean readability of the assessed websites exceeded the recommended sixth-grade reading level on the Flesch-Kincaid Grade Level and Simple Measure of Gobbledygook (P < .001). Readability did not vary significantly by website type. Websites originating from the United Kingdom were significantly more readable than those from the United States. The majority of assessed websites were of poor quality (n = 42, 56.8%) on assessment based on the DISCERN instrument. Quality varied significantly by website category on the JAMA benchmark criteria (P < .001) and DISCERN score (P = .049) with commercial websites receiving the highest scores. DISCERN score also varied significantly by country of origin (P = .036) with UK sites receiving highest mean DISCERN scores.

Conclusion

Online health information regarding parathyroidectomy is largely of poor quality and is poorly readable for many patients. Institutions utilizing well-defined guidelines for development of patient educational resources may provide online health information of greater quality and readability.

Health Literacy in Neurosurgery: A Scoping Review

OBJECTIVE

Low health literacy is prevalent and associated with suboptimal health outcomes. In neurosurgery, social determinants of health are increasingly recognized as factors underpinning outcomes, as well as access to and use of care. We conducted a scoping review to delineate the scope of existing literature regarding health literacy in the field and facilitate future research.

METHODS

A scoping review was conducted using the PubMed, Embase, and Scopus databases. Titles and abstracts were screened for relevance. Studies meeting prespecified inclusion criteria underwent full text review. Relevant data were extracted.

RESULTS

Of 5056 resultant articles, 57 manuscripts were included. Thirty-seven studies (64.9%) investigated personal health literacy, while the remaining 20 (35.1%) investigated organizational health literacy. Domains of health literacy investigated were science (36, 63.2%), fundamental (20, 35.1%), and civic (1, 1.7%). No studies investigated numeracy. Recall among patients after discussions with neurosurgeons is low. Patient perspectives are often erroneous. Patient informational needs are often unmet. Written patient educational materials are written at a level too complex for the average patients. Videos are mostly of poor quality. Multimodal audiovisual interventions, eBooks, models, and virtual reality are shown to be effective methods for promoting recall.

CONCLUSIONS

Studies examining health literacy in neurosurgery primarily focus on the topic indirectly, most often via written educational materials and recall after educational interventions. Increasing awareness of health literacy among neurosurgeons, assessing health literacy, and incorporating health literacy-informed counseling approaches are warranted to improve patient care.

Readability of Patient Educational Materials in Sports Medicine

Background

The internet has become an increasingly popular resource among sports medicine patients seeking injury-related information. Numerous organizations recommend that patient educational materials (PEMs) should not exceed sixth-grade reading level. Despite this, studies have consistently shown the reading grade level (RGL) of PEMs to be too demanding across a range of surgical specialties.

Purpose

To determine the readability of online sports medicine PEMs.

Study Design

Cross-sectional study.

Methods

The readability of 363 articles pertaining to sports medicine from 5 leading North American websites was assessed using 8 readability formulas: Flesch-Kincaid Reading Grade Level, Flesch Reading Ease Score, Raygor Estimate, Fry Readability Formula, Simple Measure of Gobbledygook, Coleman-Liau Index, FORCAST Readability Formula, and Gunning Fog Index. The mean RGL of each article was compared with the sixth- and eighth-grade reading level in the United States. The cumulative mean website RGL was also compared among individual websites.

Results

The overall cumulative mean RGL was 12.2 (range, 7.0-17.7). No article (0%) was written at a sixth-grade reading level, and only 3 articles (0.8%) were written at or below the eighth-grade reading level. The overall cumulative mean RGL was significantly higher than the sixth-grade [95% CI for the difference, 6.0-6.5; P < .001] and eighth-grade (95% CI, 4.0-4.5; P < .001) reading levels. There was a significant difference among the cumulative mean RGLs of the 5 websites assessed.

Conclusion

Sports medicine PEMs produced by leading North American specialty websites have readability scores that are above the recommended levels. Given the increasing preference of patients for online health care materials, the imperative role of health literacy in patient outcomes, and the growing body of online resources, significant work needs to be undertaken to improve the readability of these materials.

Readability of the American, Canadian, and British Otolaryngology-Head and Neck Surgery Societies' Patient Materials

OBJECTIVE

Patient education materials across 3 national English otolaryngology-head and neck surgery (OHNS) societies: the American Academy of Otolaryngology-Head and Neck Surgery (AAO-HNS), the Canadian Society of Otolaryngology-Head and Neck Surgery (CSOHNS), and Ear, Nose, and Throat United Kingdom (ENT UK) were examined to determine whether they are written at a level suitable for patient comprehension.

STUDY DESIGN

Cross-sectional study.

SETTING

Online patient materials presented through OHNS national societies.

METHODS

Readability was calculated using the Flesch-Kincaid Grade Level, Flesch-Kincaid Reading Ease Score, and Simple Measure of Gobbledygook Index. All public patient education materials available through the CSOHNS, AAO-HNS, and ENT UK websites were assessed. Patient education materials were grouped into categories by subspecialty.

RESULTS

In total, 128 patient materials from the 3 societies were included in the study. All 3 societies required a minimum grade 9 reading comprehension level to understand their online materials. According to Flesch-Kincaid Grade Level, the CSOHNS required a significantly higher reading grade level to comprehend the materials presented when compared to AAO-HNS (11.3 vs 9.9; 95% CI, 0.5-2.4; P < .01) and ENT UK (11.3 vs 9.4; 95% CI, 0.9-2.9; P < .01). Patient education materials related to rhinology were the least readable among all 3 societies.

CONCLUSION

This study suggests that the reading level of the current patient materials presented through 3 national OHNS societies are written at a level that exceeds current recommendations. Promisingly, it highlights an improvement for the readability of patient materials presented through the AAO-HNS.

Accuracy of Online Pictorial Resources for Common Hand Surgery Procedures

Background: Online medical platforms can provide patients with easily accessible information and greater opportunities to self-advocate. However, the lack of quality control and presence of inaccurate information can lead to miscommunications between the physician and the patient. The objective of this study was to examine the quality and accuracy of online pictorial information regarding common hand conditions. Methods: Medical image information was searched on the search engine Google (http://www.google.com) using the terms "de Quervain's tenosynovitis," "carpal tunnel syndrome," and "trigger finger." The first 20 illustrations to appear on the search were recorded along with the type of source. The images were then examined for veracity of information conveyed. Results: Sixty images were collected from 48 different Web sources. Nonacademic/private medical institutions were the most common image source (35%). The rate of erroneous images was 40%. Web sites of academic hospitals were most frequently the source of incorrect images. Of the conditions, trigger finger had the highest rate of errors (55%), most commonly occurring in positioning of the annular pulleys. The search results did not contain any Web sites from professional hand societies. Conclusion: The quality of online medical information is a significant but often overlooked aspect of health professional and patient education. This study demonstrates the prevalence of incorrect information online and the misunderstandings that patients can have about common conditions. Higher quality online resources are needed to improve patient education and patient-physician interactions. Avenues for improvement are to provide greater accessibility of educational resources offered by professional hand surgery organizations.

Readability and quality of online information for patients pertaining to revision knee arthroplasty: An objective analysis

BACKGROUND

Modern patients use the internet more frequently for their health information. Our aim was to assess the quality and readability of online patient education materials relating to revision knee arthroplasty. We hypothesised that this information would be too difficult to read for the average patient and of a lesser quality than desired.

METHOD

A search of the top 50 results on 3 search engines (Google, Bing, Yahoo) was completed (MeSH "revision knee arthroplasty", "revision knee replacement"). Readability of these websites was calculated using a specialised website, www.readable.com to produce 3 scores (Gunning-Fog (GF), Flesch Reading Ease (FRE), Flesch-Kincaid Grade (FKG). Quality was assessed using the Journal of American Medical Association (JAMA) benchmark criteria and the Health On the Net (HON) code toolbar extension.

RESULTS

89 unique websites were identified. Mean FRE was 44.01 ± 11.05. The mean GF was 12.01 ± 2.94. The mean FKG level was 10.06 ± 1.95. As per the FRE score, no webpage was pitched at or below a sixth grade reading level. The GF index identified only 3 websites (3.4%) at 6th grade reading level. As per FKG score, most webpages (n = 58, 65.2%) found themselves in the 10th grade reading level and above. Four websites (4.5%) displayed a HONcode certificate. However, all had expired. Most websites (n = 34, 38.2%) didn't meet any of the JAMA criteria.

CONCLUSION

The reading level of these materials is too high for the average patient and of a low quality. Given what we know about levels of health literacy and their relationship with patient outcomes, it is vital that we address these deficiencies promptly.

Reducing Children's Preoperative Fear with an Educational Pop-up Book: A Randomized Controlled Trial

OBJECTIVE

Preoperative education empowers children to approach surgery with positive expectations, and providers need efficient, child-focused resources. This study aimed to evaluate an interactive pop-up book as a tool for explaining surgery, managing preoperative anxiety, and strengthening coping strategies.

STUDY DESIGN

Prospective randomized controlled trial.

SETTING

Pediatric outpatient surgery center.

METHODS

Patients ages 5 to 12 undergoing outpatient surgery read a pop-up book about anesthesia (intervention) or received standard care (control). Patients self-reported their preoperative fear, pain expectations, views of the procedure and preoperative explanations, and coping strategies. Outcomes also included observer-rated behavioral anxiety and caregiver satisfaction.

RESULTS

In total, 148 patients completed the study. The pop-up book had a significant, large effect in reducing patients' fear of anesthesia induction (Cohen's d effect size = 0.94; P < .001). Intervention patients also expected less pain than control patients from the anesthesia mask and during surgery (d = 0.60-0.80; P < .001). The book encouraged more positive views of the procedure and preoperative explanations (P < .005). Furthermore, the book prepared patients to cope adaptively: intervention patients were significantly more likely to generate positive active coping strategies, distraction strategies, and support-seeking strategies (P < .001). Observer-rated behavioral anxiety at anesthesia induction did not differ between groups (P = .75). Caregivers in the intervention group were significantly more satisfied with each aspect of the surgical experience (P≤ .02).

CONCLUSION

The educational pop-up book offers a child-focused resource that helps alleviate children's preoperative fears, encourages positive coping, and improves caregivers' perceptions of the experience. This study was registered at ClinicalTrials.gov (NCT04796077).

Quality of Carpal Tunnel Syndrome Patient Education Handouts Available on the Internet: A Systematic Analysis of Content and Design

OBJECTIVE

To evaluate the quality of carpal tunnel syndrome (CTS) patient education handouts and identify the best resources for patients and clinicians.

DESIGN

A document content analysis of handouts identified through a systematic internet search using 8 search terms on Google and Bing and a hand search of professional association websites.

SETTING

Not applicable.

PARTICIPANTS

Documents (N=56) were identified from the top 50 search results across 16 individual searches. Included documents provided general patient education for CTS; descriptive websites, videos, and research studies were excluded.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Content analysis was conducted using the Information Score (IS) tool to evaluate completeness of information (0%-100%) and misleading treatment recommendations were identified. Design analysis was conducted using the Patient Material Assessment Tool for Printable Materials (PEMAT-P) (0%-100%) and 2 widely used readability formulas, Flesch Reading Ease and Flesch Kincaid Grade Level. Using these results, all handouts were rated with a summative 12-point scale.

RESULTS

Of 805 unique search results, we included 56 CTS handouts. The average IS was 74.6%±17.9%, and 78.6% of the handouts mentioned non-evidence-based treatment recommendations. The average PEMAT-P score was 70.2%±10.9%, and the average readability grade level was 7.7±1.7. Only 3 handouts were identified as high quality based on the 12-point summative scores, 22 handouts had mixed quality, and 17 handouts had low quality on both content and design.

CONCLUSIONS

Findings of this study suggest a lack of high-quality and easily understandable CTS patient education handouts. Most handouts contained unreliable treatment information. Improvements are needed to ensure patients' ability to understand and manage this condition.

Ankle Fractures: The Current State of Online Patient Information

Background. We assessed the quality, accuracy, and readability of websites for ankle fractures. Methods. Ankle Fracture, Broken Ankle, and Fibular Fracture were entered into 3 search engines. The first 25 results from each search were collected. Quality, accuracy, and readability were assessed by a custom rubric, 3 surgeons, and Fleisch-Kincaid grade level (FKGL), respectively. Results. A total of 57 websites were included: 24 were assigned to Ankle Fracture, 26 to Broken Ankle, and 7 to Fibular Fracture. The average quality score out of 36, accuracy score out of 12, and FKGL for all websites were 13.1 ± 6.8, 10.8 ± 1.2, and 9.6 ± 1.7, respectively. Websites assigned to the term Broken Ankle had a significantly lower New Dale-Chall score. Websites of lower FKGL and appearing earlier in results had significantly higher quality scores. Physician specialty societies (PSSs) had a significantly lower FKGL than websites of other types. Conclusion. The readability of patient materials is above the recommended level for ankle fractures. Encouragingly, a trade-off between readability and quality was not found. Patients should use search terms they are familiar with and prioritize websites that appear earlier in search results, are easier to read, and produced by PSSs.Level of Evidence: Level 3.

Readability of Online Patient Information Relating to Cleft Palate Surgery

OBJECTIVE

It is important for health care education materials to be easily understood by caretakers of children requiring craniofacial surgery. This study aimed to analyze the readability of Google search results as they pertain to "Cleft Palate Surgery" and "Palatoplasty." Additionally, the study included a search from several locations globally to identify possible geographic differences.

DESIGN

Google searches of the terms "Cleft Palate Surgery" and "Palatoplasty" were performed. Additionally, searches of only "Cleft Palate Surgery" were run from several internet protocol addresses globally.

MAIN OUTCOME MEASURES

Flesch-Kincaid Grade Level and Readability Ease, Gunning Fog Index, Simple Measure of Gobbledygook (SMOG) index, and Coleman-Liau Index.

RESULTS

Search results for "Cleft Palate Surgery" were easier to read and comprehend compared to search results for "Palatoplasty." Mean Flesch-Kincaid Grade Level scores were 7.0 and 10.11, respectively (P = .0018). Mean Flesch-Kincaid Reading Ease scores were 61.29 and 40.71, respectively (P = .0003). Mean Gunning Fog Index scores were 8.370 and 10.34, respectively (P = .0458). Mean SMOG Index scores were 6.84 and 8.47, respectively (P = .0260). Mean Coleman-Liau Index scores were 12.95 and 15.33, respectively (P = .0281). No significant differences were found in any of the readability measures based on global location.

CONCLUSIONS

Although some improvement can be made, craniofacial surgeons can be confident in the online information pertaining to cleft palate repair, regardless of where the search is performed from. The average readability of the top search results for "Cleft Palate Surgery" is around the seventh-grade reading level (US educational system) and compares favorably to other health care readability analyses.

Osteotomy around the knee: Assessment of quality, content and readability of online information

BACKGROUND

As knee osteotomy surgery becomes increasingly accessible, more patients may turn to the Internet for information. This study examined the source, quality, content and readability of online information regarding osteotomy around the knee.

METHODS

The first 70 websites returned by the top four search engines were identified using the key words: "knee osteotomy" and "high tibial osteotomy." The websites were categorised by type and assessed using the DISCERN score, Journal of the American Medical Association (JAMA) benchmark criteria and a novel Knee Osteotomy-Specific Score (KOSS). The presence of the Health On the Net (HON) code accreditation seal was noted. Readability of each website was assessed using eight readability formulae. The mean reading grade level (RGL) was compared to the 6th and 8th grade reading levels. The mean RGL of each category was also compared.

RESULTS

Of the 45 unique websites analysed, the majority were Physician (33%) and Journal websites (31%). The mean DISCERN score was 36.7 (±8.9) which is classified as 'poor.' The mean JAMA benchmark criteria score was 2.04 (±1.5) and Physician websites were most likely to be scored zero. The mean KOSS was 15.4 (±5.7). The highest scoring website was a Commercial site but, overall, Journal category sites provided the best quality information. Websites that bore the HONcode seal obtained higher DISCERN, JAMA benchmark criteria and Knee Osteotomy - Specific Scores. The cumulative mean RGL was 13.2 (±2.2) which exceeded the 6th grade level by an average of 7.2 grade levels and the 8th grade level by an average of 5.2 grade levels. No website (0%) was written at or below either the 6th or the 8th grade reading levels. The mean Flesch Reading Ease Score of all websites was 41.13 (±14.7) which is classified as 'difficult.' Journal websites had the highest RGL.

CONCLUSION

The information available online regarding osteotomy around the knee varies tremendously in quality and completeness. Physician sites predominate, but these were among the lowest scoring of all websites. Even where high quality information is available, it is set at too high a level to be easily understood.

LEVEL OF EVIDENCE

Survey of materials - Internet.

Readability of Patient Educational Materials in Hip and Knee Arthroplasty: Has a Decade Made a Difference?

BACKGROUND

The Internet has become an increasingly popular resource among orthopedic patients for health education. Numerous organisations recommend that patient educational materials (PEMs) should not exceed the 6^th grade reading level. Despite this, studies have repeatedly shown the reading grade level (RGL) of PEMs to be too advanced across a range of surgical specialties. We aimed to determine the readability of online hip and knee arthroplasty PEMs.

METHODS

The readability of 134 articles pertaining to hip and knee arthroplasty from 5leading worldwide healthcare websites were assessed, using 8 readability formulae; the Flesch-Kincaid Reading Grade Level, the Flesch Reading Ease Score, Raygor Estimate, SMOG, Coleman-Liau, Fry, FORCAST and Gunning Fog. The mean RGL was compared to the 6^th and 8^thgrade reading level. The mean RGL of each website was also compared.

RESULTS

The mean cumulative RGL was 12 (range = 7-16.1). No articles (0%) were written at a 6^th grade reading level and only 4 articles (3%) were written at or below the 8^th grade reading level. The mean RGL was significantly higher than the 6^th (95% CI, 5.62-6.30; P < .0001) and 8^th grade reading level (95% CI, 3.63-4.30; P < .0001). There was a significant difference between the RGLs of the 5websites (P = .001).

CONCLUSION

Arthroplasty PEMs produced by leading worldwide healthcare organisations have readability scores that are above the recommended levels. Given the imperative role of health literacy in patient outcomes and satisfaction, and the increasing prevalence of Internet use among orthopedic patients, a substantial amount of work needs to be done to improve the readability of these materials.

Exploring Knowledge and Experience of Health Literacy for Chinese-Speaking Nurses in Taiwan: A Cross-Sectional Study

Background: Health literacy has become the best predictor of healthcare status. However, two-thirds of health providers are unaware of patients’ health literacy. Thus, the aim of the study is to investigate factors related to Chinese-speaking nurses’ knowledge and experience of health literacy. Methods: This cross-sectional study used a web-based survey. A total of 430 nurses were recruited by stratified sampling from different levels of hospitals and community health centers in Taiwan. Primary outcome measure by Health Literacy Knowledge and Experience. Results: The participants’ overall health literacy knowledge was limited; the correct responses were 51%. The education level of the participants, job category, working years, and having attended in-service patient education programs were the predictors of knowledge of health literacy (p < 0.05); Institute, job category, and having attended in-service patient education programs were the predictors of experience of health literacy (p < 0.01). Conclusions: Participants’ education levels and In-service patient education programs are beneficial factors to improve nurses’ knowledge of health literacy. Furthermore, nursing education should emphasize on how to identify individuals’ health literacy and using readable healthcare materials to improve health education.

Assessment of online patient education materials designed for people with age-related macular degeneration

Background

Age-related macular degeneration (AMD) is a chronic eye condition that leads to permanent vision loss in the central visual field. AMD makes reading challenging and inefficient. People with AMD often find it difficult to access, process and understand written patient education materials (PEMs). To promote health literacy, the demands of written PEMs must match the literacy capacities of the target audience. This study aims to evaluate the readability (grade level) and suitability (appropriateness) of online PEMs designed for people with AMD.

Methods

Online PEMs were sourced from websites of national organizations providing patient education materials designed for people with AMD. The Flesch-Kincaid Grade Level formula and the Suitability Assessment of Materials instrument were used to assess the readability and suitability of PEMs. Descriptive statistics were used to compare online PEMs by organization based on national guidelines for readability level (≤ sixth grade) and the recommended suitability score (≥ 70%) for “superior” material.

Results

One hundred online PEMs were evaluated from websites of 16 professional organizations. The mean readability level was 9.3 (range 5.0–16.6). The mean suitability score was 53% (range 18–78%). Only six (6%) of PEMs achieved the recommended guidelines for readability level and suitability score.

Conclusion

The majority of online PEMs designed for people with AMD were written above the recommended readability level, and below the suggested suitability score. To promote health literacy, the demands of written health information must match the reading capacities of the target audience. Heeding to evidence-based guidelines for providing written information to patients with low health literacy and low vision is beneficial for both patients and health care providers. Future research is warranted.

Health literacy among neurosurgery and other surgical subspecialties: Readability of online patient materials found with Google

OBJECTIVE

To both determine whether the most high-yield online patient materials for surgical specialties meet the 6th grade readability level recommended by the National Institutes of Health (NIH) and American Medical Association (AMA), and to discover differences in readability across specialties. We hypothesize average readability scores will exceed an 11th grade level.

METHODS

The top five most common procedures for each of seven surgical specialties (neurological, orthopedic, plastic, general, thoracic, pediatric, and vascular) were searched using an incognito Google query to minimize location bias. The text from the top five patient-relevant links per procedure, excluding Wikipedia, journal articles, and videos, was extracted and inserted into Readability Studio Software for analysis.

RESULTS

The combined average grade level of materials (± standard deviation) was: 10.47 ± 2.51 Flesh-Kincaid Grade Level (FKGL), 11-12 New Dale-Chall (NDC), 10.09 ± 1.97 Simple Measure of Gobbledygook (SMOG), 12 Fry Graph (FG). Thoracic, neurologic, vascular, plastic, and orthopedic were least readable (grade level 10+ by all metrics).

CONCLUSIONS

High readability of procedure materials for patients is not unique to neurosurgery: all specialties exceeded the recommended 6th grade level by three or more grades. Online patient education materials related to surgical subspecialties must be written more comprehensibly.

Evaluation of Hip Preservation-related Patient Education Materials From Leading Orthopaedic Academic Centers in the United States and Description of a Novel Video Assessment Tool

The readability, reliability, and quality of online hip preservation–related patient education materials from the top 20 orthopaedic academic centers in the United States were evaluated.

Simple contents and good readability: Improving health literacy for LEP populations

Accessing and receiving quality healthcare in an unfamiliar health system is a significant challenge for many new immigrants and refugees. This study aims to provide a three-phase model to develop a web-based health information website that helps populations with limited English proficiency (LEP) increase health literacy and improve healthcare service access.

METHOD

First, we conducted a needs assessment from community leaders and service providers. Second, we developed contents from credible sources and tested each item using multiple readability tests. Last, we revised each item to lower the readability and retest its readability.

RESULTS

The average reading level for the original 99 topics was assessed at 10.84 (SD= 3.26). After revisions, we were able to lower the readability to 8.56 (SD= 2.96), which was around two grade levels lower, on average.

CONCLUSION

the main purpose for building an English based health information website was to assist the population with LEP. By using simple English with lower readability, it will ease the translation process. This study demonstrates a process to develop suitable contents for populations in need. In the future, incorporating visual aid and other multimedia will be beneficial in user engagement and knowledge retention.

Reading Level of Online Patient Education Materials From Major Obstetrics and Gynecology Societies

OBJECTIVE

To assess whether the readability of online patient education materials provided by eight nationally recognized obstetrics and gynecology societies is in accordance with the recommended 6th-grade reading level outlined by the American Medical Association, National Institute of Health, and United States Department of Health and Human Services.

METHODS

An analysis of 410 online patient education materials from the American Association of Gynecologic Laparoscopists, the American College of Obstetricians and Gynecologists, the American Society for Reproductive Medicine, the American Urogynecologic Society, the Association of Reproductive Health Professionals, the Society of Gynecologic Oncology, the Society for Maternal-Fetal Medicine, and Voices for Pelvic Floor Disorders was completed, and the readability scores using the Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning Fog Scale, and Simple Measure of Gobbledygook readability scales were calculated. These scales are used to assess print and web-based written material across a wide range of medical specialties and are endorsed by the National Institutes of Health. All four scales are computer-based assessments of readability that consider word count, number of syllables, and length of sentences when calculating a score that corresponds to grade level.

RESULTS

Majority of the patient education materials across all eight obstetrics and gynecology societies had readability scores above the recommended 6th-grade reading level. The average reading level for the 69 obstetrics-related articles ranged from 9th to 12th grade. The mean grade level for the 341 gynecology articles had a similar range across the four readability scales.

CONCLUSION

Online patient education materials provided by major obstetrics and gynecology societies do not currently adhere to recommended readability guidelines. Continued efforts to provide accessible and informative patient education materials is recommended to help improve health literacy for women.

Systematic Analysis of Readability of Patient Information on Internet Pages from Departments for Trauma Surgery of German University Hospitals

INTRODUCTION

The need-based information transfer in education as well as for the recruitment of patients becomes more and more relevant. Here, the internet has emerged as an increasingly important factor in recent years and therefore information pages on hospital homepages can be very helpful. However, it is known that basic text comprehension skills are lacking among large populations.

METHOD

The aim of the present study was to evaluate the readability of the patient information available on the websites of trauma departments of German university hospitals. For this purpose, a search for information material on 10 different diagnoses was carried out. Out of 360 texts possible, 185 were found and assigned to 2 superordinate thematic areas (emergency vs. elective operations), subjected to a systematic text analysis via software "Text-Lab" and rated using 5 known readability indices (Amstad, G-SMOG, LIX, HIX, WSTF).

RESULTS

The indices used for both thematic complexes consistently showed poor readability, so that the texts only seem sufficiently comprehensible to readers with higher education. (Amstad: 21.4 ± 20.8; G-SMOG: 11.6 ± 2.0; WSTF 13.3 ± 2.1; LIX: 60.9 ± 7.6; HIX: 4.1 ± 3.2).

CONCLUSION

For the medical information provided by university hospitals, there is a clear need for improvement in order to make the adequate acquisition of knowledge accessible to a broader spectrum of patients.

Too Dense and Too Detailed: Evaluation of the Health Literacy Attributes of an Informed Consent Document

The US government recently updated the Common Rule, a set of federal regulations to ensure the ethical conduct of human subjects research. The new regulations require that consent documents provide information that is clear and concise enough to enable truly informed consent. This study explores potential American Indian research participants' understanding and perceptions of an example consent document, focusing on possible improvements to better serve the requirements of the revised Common Rule. Participants completed a survey that collected demographic data and measured health literacy, numeracy, and comprehension of the example document. Next, they participated in focus groups to answer open-ended questions regarding their views on the example document. We calculated mean scores and frequencies of response to analyze quantitative survey data and performed a qualitative thematic analysis of focus group transcripts. Results demonstrated that American Indian participants with relatively strong health literacy skills clearly understood key elements of the consent document, including the purpose of signing it, confidentiality, compensation, and whom to contact for questions. However, they were overwhelmed by details on research procedures and were concerned about the document's layout. To make consent documents more readily comprehensible, participants recommended the addition of headings, bullets, graphics, and relevant pictures. They also recommended a two-step consent process, comprising a short introduction to the research project followed by a longer explanation of procedures. These results illustrate the potential advantages of community engagement in drafting consent materials. Health researchers would likely benefit from community recommendations like the ones we elicited as they design consent documents adherent to the revised Common Rule.

Assessment of the Readability, Adequacy, and Suitability of Online Patient Education Resources for Benign Vascular Tumours Using the DISCERN Instrument

Objective

This study aims to assess the quality and readability of Internet-based patient resources for vascular tumours in order to understand which areas require improvement.

Methods

A World Wide Web search was performed, in addition to a literature review using PubMed, Ovid MEDLINE, and EMBASE. Any material that contained information on vascular tumours pertaining to patient education was included. We evaluated resources with DISCERN and Flesch Reading Ease scores when applicable. The language of publication was restricted to English and French. This review was registered with PROSPERO (CRD42018087885).

Results

A total of 117 online resources were screened, with 73 resources included in the final analysis. The overall DISCERN rating for the patient resources was 1.8 (0.8). The majority of online resources failed to depict the entire spectrum of benign vascular tumours. The mean Flesch score was 36 (19), which translates to a college-level readability.

Conclusion

The majority of resources were not adequate or comprehensive and were written at a much higher level than the average reader would be expected to comprehend.

Readability of online materials for Dupuytren's contracture

STUDY DESIGN

Descriptive.

BACKGROUND

Dupuytren's contracture is a common disorder involving fibrosis of the palmar fascia. As patients are increasingly using online materials to gather health care information, it is imperative to assess the readability and appropriateness of this content. The recommended grade level for patient educational materials is seventh to eighth grade according to the National Institutes of Health. This study aims to assess the readability and content of online patient resources for Dupuytren's contracture.

PURPOSE OF THE STUDY

Evaluate readability of online patient education materials for Dupuytren's contracture.

METHODS

The largest public search engine, Google, was queried using the term "Dupuytren's contracture surgery" on February 26, 2016. Location filters were disabled, and sponsored results were excluded to avoid any inadvertent search bias. The 10 most popular Web sites were identified, and all relevant patient-directed information within 1 click from the original site was downloaded and saved as plain text. Readability was analyzed using 6 established analyses (Readable.io, Added Bytes, Ltd, UK).

RESULTS

Analysis of 10 Web sites demonstrates an average grade level of at least 11th grade (Flesch-Kincaid grade level, 10.2; Gunning-Fog grade level, 13.1; Coleman-Liau grade level, 14.4; Simple Measure of Gobbledygook grade level, 10.0; automated readability grade level, 9.7; and average grade level, 11.5). Overall Flesch-Kincaid reading ease index was 46.4, which is difficult. No single article was at the recommended reading level.

CONCLUSIONS

Online materials available for treatment of Dupuytren's contracture are above recommended reading levels and do not include a comprehensive explanation of treatment options, which may negatively impact decision making in patients seeking treatment for this condition. Surgeons and hand therapists alike should be cognizant of available online patient materials and make efforts to develop and provide more appropriate materials.

LEVEL OF EVIDENCE

V.

Parental health literacy in pediatric otolaryngology: A pilot study

OBJECTIVES

Pediatric otolaryngology bears the highest frequency of elective surgical cases in children, but little is known regarding the health literacy of these children's parents. In a questionnaire-based pilot study, we assessed parental health literacy in our pediatric otolaryngology clinic and evaluated its relation to personal demographics. We also evaluated postoperative and diagnosis-based leaflets for readability using validated measures.

METHODS

Parents completed the Short Assessment of Health Literacy-English (SAHL-E) and were defined as low (0-14) or high (>14) literacy based on mean scores.

RESULTS

Fifty parents participated (mean SAHL-E score, 17.6), and all scored >14, indicating proficient health literacy. No differences existed in mean or median scores based on demographics. Continued education after high school was associated with higher scores (P = 0.003) and was the only significant variable in multivariable linear regression. All leaflets were at or below the recommended reading level for public health information.

CONCLUSIONS

Parents generally had high health literacy.

Readability of Patient Education Materials From RadiologyInfo.org: Has There Been Progress Over the Past 5 Years?

OBJECTIVE. As patients increasingly turn to the Internet for healthcare information, it is imperative that patient educational materials be written at an appropriate readability level. Although RadiologyInfo.org, a patient education library sponsored by the American College of Radiology (ACR) and Radiological Society of North America, was shown in 2012 to be written at levels too high for the average patient to adequately comprehend, it is unclear if there has been progress made in the past 5 years. The purpose of this study was to provide a 5-year update on the readability of patient education materials from RadiologyInfo.org. MATERIALS AND METHODS. All patient education articles available in 2017 from the ACR and RSNA-sponsored RadiologyInfo.org patient education library were reviewed. We assessed each article for readability using 6 quantitative readability scales: the Flesch-Kincaid (FK) grade level, Flesch Reading Ease, Gunnin-Fog Index, Coleman-Liau Index, Automated Readability Index, and the Simple Measure of Gobbledygook (SMOG). The number of articles with readability ≤ the 8^th grade level (average reading ability of US adults) and the 6^th-grade level (NIH-recommended level for patient materials) were determined. RESULTS. 131 patient education articles were reviewed. The mean readability grade level was greater than the 11^th grade reading level for all readability scales. None of the articles were written at less than the 8^th-grade or the 6^th-grade levels. CONCLUSION. Although there has been an increasing awareness of the issue of readability of patient educational materials within the radiological community, the patient educational materials within the ACR and RSNA-sponsored RadiologyInfo.org website are still written at levels too high for the average patient. Future efforts should be made to improve the readability of those patient education materials.

Readability assessment of online gynecologic oncology patient education materials from major governmental, non-profit and pharmaceutical organizations

OBJECTIVE

Patients are increasingly using online materials to learn about gynecologic cancer. Providers can refer patients to online educational materials produced by a number of different major medical organizations and pharmacology companies. The National Institutes of Health (NIH) and the American Medical Association (AMA) recommend that patient educational materials (PEMs) are written between a sixth and eighth grade reading level. In this study, we assess the readability of online PEMs published by major medical organizations and industry partners.

METHODS

Websites from twelve websites providing educational materials for gynecologic oncology patients were surveyed. Online PEMs were identified and analyzed using seven validated readability indices. One-way ANOVA and Tukey's Honestly Significant Difference (HSD) post-hoc analysis were performed to detect differences in readability between publishers.

RESULTS

Two-hundred and sixty PEMs were included in this analysis. Overall, PEMs were written at a mean 11th±0.6 grade reading level. Only 6.5% of articles were written at the AMA/NIH recommended reading grade level of 6th to 8th grade or below. ANOVA demonstrated a significant difference in readability between publishing associations (p<0.01). PEMs from the Centers for Disease Control had a mean 9th±1.2 grade reading level and were significantly lower than all other organizations. PEMs from The Foundation for Women's Cancer had a mean 13th±1.8 grade reading level and were significantly higher than most other organizations. PEMs from pharmaceutical companies (mean readability=10.1±1.1, N=30) required the lowest reading grade level and were significantly more readable than those from governmental organizations (11.1±1.7, p<0.05) and nonprofit medical associations (12.4±1.7, p<0.01) in ANOVA and Tukey-Kramer post hoc analysis.

CONCLUSIONS

Gynecologic oncology PEMs available from twelve major organization websites are written well above the recommended sixth to eighth grade reading difficulty level.

eHealth Literacy in Otolaryngology Patients

Objectives:

The aim of this study was to compare eHealth literacy—one’s perception of one’s ability to use the Internet for health care—among otolaryngology patients in 3 geographic settings of the same department.

Setting:

An academic otolaryngology department.

Method:

Patients’ opinions and perceptions of their eHealth literacy were assessed with a validated paper survey administered in the summer of 2017.

Results:

Of 381 asked, 351 people completed the survey, 149 at a university town teaching hospital clinic (group A), 101 at a nearby rural clinic (group B), and 101 at a remote rural clinic (group C). Mean scores were 30.80, 28.97, and 29.03 for groups A, B, and C, respectively. The overall mean was 29.76 ± 5.97. Three surveys reported the minimum score of 8, and 26 reported the maximum score of 40. Results were statistically significantly different among all sites ( P = .001), between groups A and B ( P = .027), and between groups A and C ( P = .0175). Women reported higher eHealth literacy (30.13 ± 6.27) than men (28.87 ± 5.11) ( P = .045). Participant age and role (patient or parent of a patient) were statistically insignificant. Mean scores were similar to those previously reported in other patient populations.

Conclusions:

Otolaryngology patients in a university town had better eHealth literacy than patients in more rural settings, suggesting that online medical resources and access points are less likely to be useful in rural populations.

Readability of online patient-based information on bariatric surgery

Background: Web-based patient education literature has been shown to be written at reading levels far above what is recommended. Little is known about the overall readability of current internet-based bariatric surgery information. The purpose of this study was to assess the readability of current bariatric material on the internet.

Methods: The term "weight loss surgery" was searched using the Chrome browser on the first 15pages of URLs that appeared with content written in English. Using five readability measures, scores were generated using Readable.io for written content on a sample of 96 websites. Scores were sorted into the readability categories of "easy," "average," and "difficult."

Results: Almost 93% of websites, both .com and .org, sampled received an unacceptable readability score on each assessment.

Conclusion: Accurate and appropriate information about bariatric procedures is critical for patient comprehension and adherence to recommended protocols.

A Comparative Multimetric Assessment of English and Spanish Carpal Tunnel Syndrome Materials

BACKGROUND

Spanish-speaking Hispanics living in the United States utilize the internet as a primary means to obtain health information. Accurate, accessible information is important for English speakers; however, it could have even greater utility for Spanish speakers who have lower health literacy levels. The aim of this study was to evaluate and compare online English and Spanish carpal tunnel surgery materials provided by using a multimetric approach.

MATERIALS AND METHODS

A web search using the English term "carpal tunnel surgery" was performed. The first 10 institutional/organizational websites that provided carpal tunnel surgery information in English and Spanish were included. All relevant online materials were evaluated using the Patient Education and Materials Assessment Tool (PEMAT), Cultural Sensitivity Assessment Tool (CSAT), and Simplified Measure of Gobbledygook, Spanish (SOL) to assess understandability and actionability, cultural sensitivity, and readability, respectively.

RESULTS

There were no statistically significant differences in understandability or actionability scores between Spanish and English materials. Average cultural sensitivity scores for Spanish materials were significantly lower than English materials (P = 0.015). The average reading grade level of online English materials was greater than that for Spanish materials (P = 0.011). Both mean values were above the recommended sixth-grade reading level.

CONCLUSIONS

Online patient-directed information regarding carpal tunnel surgery exceeded the recommended reading grade level for both English and Spanish-speaking populations. Most Spanish materials were often direct translations and were not contoured to the elevated literacy needs of this demographic. Institutions must caution their authors to tailor their web material in a way that is sensitive to their target population to optimize understanding.

Assessing the readability and patient comprehension of rheumatology medicine information sheets: a cross-sectional Health Literacy Study

OBJECTIVES

Patients are often provided with medicine information sheets (MIS). However, up to 60% of patients have low health literacy. The recommended readability level for health-related information is ≤grade 8. We sought to assess the readability of MIS given to patients by rheumatologists in Australia, the UK and Canada and to examine Australian patient comprehension of these documents.

DESIGN

Cross-sectional study.

SETTING

Community-based regional rheumatology practice.

PARTICIPANTS

Random sample of patients attending the rheumatology practice.

OUTCOME MEASURES

Readability of MIS was assessed using readability formulae (Flesch Reading Ease formula, Simple Measure of Gobbledygook scale, FORCAST (named after the authors FORd, CAylor, STicht) and the Gunning Fog scale). Literal comprehension was assessed by asking patients to read various Australian MIS and immediately answer five simple multiple choice questions about the MIS.

RESULTS

The mean (±SD) grade level for the MIS from Australia, the UK and Canada was 11.6±0.1, 11.8±0.1 and 9.7±0.1 respectively. The Flesch Reading Ease score for the Australian (50.8±0.6) and UK (48.5±1.5) MIS classified the documents as 'fairly difficult' to 'difficult'. The Canadian MIS (66.1±1.0) were classified as 'standard'. The five questions assessing comprehension were correctly answered by 9/21 patients for the adalimumab MIS, 7/11 for the methotrexate MIS, 6/28 for the non-steroidal anti-inflammatory MIS, 10/11 for the prednisone MIS and 13/24 for the abatacept MIS.

CONCLUSIONS

The readability of MIS used by rheumatologists in Australia, the UK and Canada exceeds grade 8 level. This may explain why patient literal comprehension of these documents may be poor. Simpler, shorter MIS with pictures and infographics may improve patient comprehension. This may lead to improved medication adherence and better health outcomes.

Poor Readability of Online Patient Resources Regarding Sentinel Lymph Node Biopsy for Melanoma

Background Established guidelines recommend that patient educational materials should be set at no higher than a sixth-grade reading level to be considered adequately comprehensible to the general public. Our study objective was to assess the readability of online patient resources related to sentinel lymph node biopsy (SLNB) as part of treatment for melanoma. Materials and methods The top 50 results from a Google search (search terms: "sentinel lymph node biopsy melanoma") were analyzed using seven established readability formulae in order to determine their level of adherence to current guidelines. Results We found that the readability of available online patient resources is currently very poor, with only 12% of the websites meeting the sixth-grade reading level criteria according to at least one measure, and 0% meeting the criteria according to all seven assessment tools. Furthermore, half of search results were peer-reviewed academic journal articles not intended for the general public. Discussion and conclusions Online patient resources related to SLNB carried out as part of melanoma treatment have poor readability. Several simple measures may be taken in order to make these resources more accessible and comprehensible to a broader audience. These resources should undergo ongoing evaluation, with the ultimate goal being improved readability and patient education.

Caught in a Web of Confusion: Assessing the Readability of University Webpages for Victims of Sexual Assault

Assuring the effectiveness of Web sites in communicating critical information to a college student who has experienced sexual assault is important and complex. Recognizing that the average American reads at the eighth grade level, the National Institutes of Health and the American Medical Association recommend that information be written at a sixth-grade reading level. A sample of 10 U.S. institutions of higher education (IHE) Web sites made up the sample of the current study. The IHEs included were all participants in a project called "Cultivating Safe College Campuses," funded by the Department of Health and Human Services. This study aimed to gain more insight into the readability of college Web sites about sexual assault resources for the intended consumer-college students. The reading level of IHE Web sites with information about sexual assault should align with the reading level of their intended audience. The average readability of sexual assault Webpages for this study's sample of IHEs was over 13, well above the reading level of the average "first time in college" student. All IHEs should review the readability of their Web sites and revise them to use consistent and defined terms and present the material in a way that is clear and accessible for the student who has experienced trauma.

Patient self-reported utility of hand surgery online patient education materials

OBJECTIVE

Online patient education materials in orthopaedic surgery are consistently written above the recommended grade level. However, no algorithmic measure of readability has been validated in a medical context. The primary objective of the present study was to determine whether Flesch-Kincaid readability scores correlate with patient self-reported utility of online education materials from the American Society for Surgery of the Hand (ASSH).

METHODS

This was a prospective survey study of 35 patients with one of five common upper extremity diagnoses. Study outcomes included self-reported utility, understandability, clarity, novelty and scope of the information of the ASSH online patient education material. Flesch-Kincaid Grade Level and Flesch Reading Ease were calculated for each article and correlated with self-reported utility.

RESULTS

The majority of patients found the articles useful, understandable and clear. Self-reported utility was not correlated with Flesch-Kincaid Grade Level (ρ = 0.017) or Flesch Reading Ease (ρ = -0.020). Patients with high school education or below found the articles less useful and more difficult to understand than those with post-secondary education.

CONCLUSIONS

Flesch-Kincaid readability scores do not correlate with self-reported utility of ASSH online patient education materials. In the evaluation of these materials, metrics other than algorithmic readability scores should be considered.

Health-Related Internet Use Among Otolaryngology Patients

Objective:

The association between online health resources use in specific otolaryngology patients is poorly understood. To better understand health-related Internet use by otolaryngology patients, we surveyed first-visit patients at academic and private practice clinics in Iowa.

Methods:

Data on socioeconomic status, access, and utilization of online resources were collected. Age distributions were compared by t test, and categorical variables were compared by chi-square analysis. Multivariate logistic regression was used to estimate odds ratios for association between independent variables (age, sex, educational attainment, otolaryngology subspecialty, etc).

Results:

Data showed that 8.7% lacked Internet access; an additional 5.4% reported access only in a public place or at work. Younger, more educated, and more urban patients reported higher rates of Internet access. Among university patients, patients seeing head and neck oncologists were most likely to report no Internet access (10.9%). Just over one-third of patients used the Internet to research their health condition prior to their appointment.

Conclusions:

Internet access was far from universal among this large cohort of otolaryngology outpatients. Head and neck cancer (HNC) patients report the least online access among all otolaryngology subspecialties. Providers should consider nonelectronic patient resources for older, more rural, less educated, and HNC patient populations as online/electronic methods of communication may not be accessible to these groups.

Assessing the Readability of Online Information About Hip Arthroscopy

PURPOSE

To investigate the current readability of online information pertaining to hip arthroscopy.

METHODS

The terms "hip arthroscopy" and "hip scope" were entered into the advanced search functions of Google, Yahoo!, and Bing on March 25, 2017, and results from the first 3 pages were analyzed. Results were required to be unique, accessible websites with information about hip arthroscopy conveyed primarily via analyzable text. Two reviewers applied inclusion criteria to the initial 97 results, discussing to reach consensus in cases of disagreement. Overall, 60 unique results were reviewed with 48 meeting inclusion criteria. Websites were categorized as physician-sponsored, academic, commercial, governmental and nonprofit organization (NPO), or unspecified. Readability was measured via 6 different indices: the Flesch-Kincaid grade level (FKGL), Flesch Reading Ease (FRE), Gunning Fog Score, SMOG Index, Coleman-Liau Index (CLI), and Automated Readability Index (ARI) along with an average grade level and readability classification score.

RESULTS

Forty-eight unique websites were assessed for readability, with physician-sponsored webpages composing the majority (47.92%) followed by academic sources (35.42%). The webpages' average grade level, incorporating information from all 6 metrics, was 12.79 ± 1.98.

CONCLUSIONS

The current readability of online information pertaining to hip arthroscopy is at an inappropriately high reading level compared with the sixth-grade level recommended by the American Medical Association and National Institutes of Health, thus introducing significant barriers to understanding for many patients. Online materials should be edited to reduce word and sentence length and complexity, use simpler terms, and minimize use of passive voice to facilitate patient knowledge acquisition and understanding of online information about hip arthroscopy.

CLINICAL RELEVANCE

This study shows that the current readability of online information on hip arthroscopy exceeds the suggested sixth-grade reading level. It also emphasizes the need for simplifying written materials and offers specific suggestions on doing so to increase accessibility of information for patients.

Technology for performing ocular self-examination: comparison between printed and virtual booklets

OBJECTIVE

Comparing the results of the ocular self-examination performed with the aid of printed and virtual versions of an educational booklet.

METHOD

A quasi-experimental study carried out in a state (public) school of a capital in northeast Brazil, with 100 students equally divided into control and intervention groups according to age, gender, schooling and economic status. Pearson's Chi-square test and Fisher's exact test were applied with a significance level of 5%.

RESULTS

The results of the self-examination obtained by the virtual and printed booklets were statistically similar, except for the item 'Alterations of the pupillary reflex', in which the virtual booklet was more effective for its identification (p=0.049).

CONCLUSION

The printed and virtual versions of the ocular educational booklet have similar efficacy for performing ocular self-examination.