Readability analysis of healthcare-oriented education resources from the American Academy of Facial Plastic and Reconstructive Surgery

Assessing the Readability, Reliability, and Quality of AI-Modified and Generated Patient Education Materials for Endoscopic Skull Base Surgery

BACKGROUND

Despite National Institutes of Health and American Medical Association recommendations to publish online patient education materials at or below sixth-grade literacy, those pertaining to endoscopic skull base surgery (ESBS) have lacked readability and quality. ChatGPT is an artificial intelligence (AI) system capable of synthesizing vast internet data to generate responses to user queries but its utility in improving patient education materials has not been explored.

OBJECTIVE

To examine the current state of readability and quality of online patient education materials and determined the utility of ChatGPT for improving articles and generating patient education materials.

METHODS

An article search was performed utilizing 10 different search terms related to ESBS. The ten least readable existing patient-facing articles were modified with ChatGPT and iterative queries were used to generate an article de novo. The Flesch Reading Ease (FRE) and related metrics measured overall readability and content literacy level, while DISCERN assessed article reliability and quality.

RESULTS

Sixty-six articles were located. ChatGPT improved FRE readability of the 10 least readable online articles (19.7 ± 4.4 vs. 56.9 ± 5.9, p < 0.001), from university to 10th grade level. The generated article was more readable than 48.5% of articles (38.9 vs. 39.4 ± 12.4) and higher quality than 94% (51.0 vs. 37.6 ± 6.1). 56.7% of the online articles had "poor" quality.

CONCLUSIONS

ChatGPT improves the readability of articles, though most still remain above the recommended literacy level for patient education materials. With iterative queries, ChatGPT can generate more reliable and higher quality patient education materials compared to most existing online articles and can be tailored to match readability of average online articles.

Health Disparities Research in Facial Plastic and Reconstructive Surgery: A Scoping Review

OBJECTIVE

Health disparities contribute significantly to disease, health outcomes, and access to care. Little is known about the state of health disparities in facial plastic and reconstructive surgery (FPRS). This scoping review aims to synthesize the existing disparities research in FPRS and guide future disparities-related efforts.

DATA SOURCES

PubMed, Embase, Web of Science.

REVIEW METHODS

We conducted a scoping review in adherence with the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews checklist. Our search included all years through March 03, 2023. All peer-reviewed primary literature of any design related to disparities in FPRS was eligible for inclusion.

RESULTS

Of the 12283 unique abstracts identified, 215 studies underwent full-text review, and 108 remained for final review. The most frequently examined topics were cleft lip and palate (40.7%), facial trauma (29.6%), and gender affirmation (9.3%). There was limited coverage of other areas. Consideration of race/ethnicity (68.5%), socioeconomic status (65.7%), and gender/sex (40.7%) were most common. Social capital (0%), religion, occupation, and features of relationships were least discussed (0.01% each). The majority of studies were published after 2018 (59.2%) and were of nonprospective designs (95.4%). Most studies focused on disparity detection (80.6%) and few focused on understanding (13.9%) or reducing disparities (0.06%).

CONCLUSION

This study captures the existing literature on health disparities in FPRS. Studies are concentrated in a few areas of FPRS and are primarily in the detecting phase of public health research. Our review highlights several gaps and opportunities for future disparities-related focus.

What Is Health Literacy Among Orthognathic Surgery Patients?

BACKGROUND

Health literacy of orthognathic surgery patients has not been thoroughly evaluated.

PURPOSE

The purpose of this study was to estimate health literacy and identify risk factors associated with inadequate health literacy in orthognathic surgery patients.

STUDY DESIGN, SETTING, SAMPLE

A cross-sectional study was implemented utilizing patients ages 14-80 years who presented for orthognathic surgery evaluation between September 2021 and December 2022. Subjects were excluded from the study if they did not complete the orthognathic surgery evaluation, were not between the ages of 14-80 years old, or did not complete the Brief Health Literacy Screening Tool (BRIEF) questionnaire during intake. Subjects who have not undergone orthognathic surgery but completed the initial evaluation for orthognathic surgery were included in the study.

PREDICTOR VARIABLES

The predictor variables were a set of risk factors for inadequate health literacy: age, sex, primary language, race, estimated household income, and diagnosis.

MAIN OUTCOME VARIABLE

The main outcome variable was health literacy assessed using the BRIEF questionnaire. During intake, subjects completed the BRIEF questionnaire consisting of four questions scored on an ordinal scale of 1-5. Inadequate health literacy was defined as a BRIEF score ≤16.

COVARIATES

Not applicable.

ANALYSES

Bivariate and multivariate analyses were performed. P < .05 was considered statistically significant.

RESULTS

Of 150 patients presenting for orthognathic surgery, fifteen percent of patients had inadequate health literacy via the BRIEF test. The mean age of those with adequate health literacy was 27.9 years (standard deviation, ±12.5) compared to 18.5 years (standard deviation, ±5.7) for those with inadequate health literacy (P = <.001). After adjusting for sex, language, race, estimated household income, and diagnosis via multivariate analysis, increasing age was associated with decreased odds of inadequate health literacy (adjusted odds ratio = 0.81; confidence interval, 0.72-0.92; P = <.001).

CONCLUSION AND RELEVANCE

In the complex process of orthognathic surgery, it is essential to identify patients with inadequate health literacy that may require additional health literacy interventions. Ultimately, 15% of orthognathic surgery subjects had inadequate health literacy, and younger patients were the most susceptible as the odds of inadequate health literacy decreased with increasing age.

Readability of cerebrovascular diseases online educational material from major cerebrovascular organizations

INTRODUCTION

The internet is an essential resource for patients and their loved ones to understand their medical conditions, and professional medical organizations have taken great strides to develop educational material targeting patients. The average American reads at a seventh to eighth grade reading level, hence it is important to understand the readability of this medical information to ensure patients comprehend what is being presented.

METHODS

In January 2023, online patient education material was downloaded from major cerebrovascular healthcare organizations and assessed using eight assessments, including Bormuth Cloze Mean, Bormuth Grade Placement, Coleman-Liau (grade levels), Coleman-Liau (predictive cloze scores), Flesch Reading Ease (FRE), and Fry.

RESULTS

A total of 32 files were extracted from six organizations and analyzed across 15 readability measures. None of the organizations met the federal government guidelines for grade-level readability. This held constant across all measured tests. Two organizations had above a postgraduate level. The FRE graphs do not identify any organizations with material below a ninth grade reading level, while the American Association of Neurological Surgeons (AANS) and the Society of Interventional Radiology (SIR) have a postgraduate readability level. The Fry graphs show similar results, with AANS/CNS Cerebrovascular Section, Society of NeuroInterventional Surgery (SNIS), SIR, and AANS having college-level readability. The lowest readability across all measures is only at an early seventh grade reading level.

CONCLUSIONS

Current health literacy content for cerebrovascular patients is far above the recommended readability level. We provide straightforward suggestions for how major professional organizations should improve their informational material on cerebrovascular diseases to improve patient understanding.

Perspectives of Healthcare Providers and the General Population on a Cancer Awareness Portal: a Qualitative Study

This study was aimed at a qualitative assessment of a cancer informational website, "India Against Cancer," developed by the ICMR-National Institute of Cancer Prevention and Research with the dual purpose of promoting cancer awareness among the general population and providing comprehensive information to healthcare providers. We used the narrative qualitative approach (data collection using in-depth interviews (IDIs) and focused group discussions (FGDs)) and content analysis. Purposive sampling was undertaken for both IDIs and FGDs. Thematic analysis was done on four themes: general cancer awareness, ease of comprehension of the Web portal, the relevance of the website content, and general comments and suggestions. Readability assessment of content of the website was conducted using Flesch-Kincaid Readability methodology. The general cancer awareness was enhanced among the participants after visiting the website. Ease of comprehension of the Web portal content was reported to fare well. The sections on "risk factors," "warning signs," and "diet and cancer" were mentioned as most relevant by the general population, whereas "myths and facts" and "screening guidelines" were most relevant for the healthcare workers. Similarly, references and epidemiology sections were most useful for healthcare researchers/professionals. Readability scores of content in English were appropriate for high school pass audience. The content on "India Against Cancer" website was found to serve its intended purpose of promoting cancer awareness among the general population and providing authentic cancer-related information to the healthcare providers. The suggestions received would enable us to enhance the utility of our website by tailoring it to the needs of the target audience.

Health Literacy in Plastic Surgery: A Scoping Review

Health literacy of plastic surgery patients may affect surgical decision-making and perioperative outcomes. In addition to consulting a plastic surgeon, patients often refer to online-based resources to learn about surgical options. The aim of this scoping review was to identify evidence detailing the state of health literacy of plastic surgery patients and available resources to highlight areas of improvement for clinical practice and future research.

A Multimetric Readability Analysis of Online Patient Educational Materials for Submental Fat Reduction

Background

Patients often utilize the Internet to seek information related to their care. This study assesses the readability of online patient educational materials for submental fat reduction.

Methods

Patient educational materials from the 12 most popular websites related to submental fat reduction were downloaded and assessed for readability grade level using 10 unique scales.

Results

Analysis of the 12 most popular websites (and corresponding 47 articles) revealed that patient educational materials were written, on average, at an 11th grade reading level. The Flesch Reading Ease score was 48.9 (range 39.8–59.2), representing a “difficult” level of reading. Mean readability grade levels (range 9–13th grade for individual websites) were as follows: Coleman-Liau, 11.1; Flesch-Kincaid, 10.8; FORCAST, 10.8; Fry Graph, 10.1; Gunning Fog, 12.7; New Dale-Chall, 10.1; New Fog Count, 11.8; Simple Measure of Gobbledygook, 11.7; Raygor, 6.7. No website was at the 6th grade reading level for patient educational materials recommended by the American Medical Association and National Institutes of Health.

Conclusions

Online patient educational materials for submental fat reduction are written well above the recommended reading level. Recognition of disparities in health literacy is necessary to enable patients to make informed decisions and become active participants in their own care.

Level of Evidence V

This journal requires that authors assign a level of evidence to each article. For a full description of these Evidence-Based Medicine ratings, please refer to the Table of Contents or the online Instructions to Authors  www.springer.com/00266

Internet-Based Patient Education Materials Regarding Diabetic Foot Ulcers: Readability and Quality Assessment

BACKGROUND

While diabetic foot ulcers (DFU) are a common complication of diabetes, little is known about the content and readability of online patient education materials (PEM) for DFU. The recommended reading grade level for these materials is grades 6-8.

OBJECTIVE

The aim of this paper was to evaluate the quality and readability of online PEM on DFU.

METHODS

A Google search was performed using 4 different search terms related to DFU. Two readability formulas were used to assess the readability of the included PEM. These included the Flesch-Kincaid grade level and the Flesch-Reading ease score. The DISCERN tool was used to determine quality and reliability.

RESULTS

A total of 41 online PEM were included. The average Flesch-Reading ease score for all PEM was 63.43 (SD 14.21), indicating a standard difficulty level of reading. The average reading grade level was 7.85 (SD 2.38), which is higher than the recommended reading level for PEM. The mean DISCERN score was 45.66 (SD 3.34), and 27% (11/41) of the articles had DISCERN scores of less than 39, corresponding to poor or very poor quality.

CONCLUSIONS

The majority of online PEM on DFU are written above the recommended reading levels and have significant deficiencies in quality and reliability. Clinicians and patients should be aware of the shortcomings of these resources and consider the impact they may have on patients' self-management.

A Simple Brochure Improves Disposal of Unused Opioids: An Observational Cross-Sectional Study

BACKGROUND

We evaluated the effectiveness of a simple, low-cost educational brochure in improving disposal rates of unused opioids after outpatient upper extremity surgery.

METHODS

This cross-sectional study enrolled eligible adult patients from a peripheral nerve clinic between November 2017 and September 2018. Patients either received or did not receive the educational brochure, which outlined a simple method to dispose of unused opioids and completed a survey at 2 weeks after surgery. We compared the proportion of patients who disposed of unused opioids after surgery between the group that had received the brochure and the group that had not. Categorical data were analyzed with χ² test, proportions data with binomial tests, and numerical data with Mann-Whitney U test, all with a significance level of P < .05.

RESULTS

There were 339 survey respondents. Nineteen patients who did not meet inclusion criteria were excluded. Of the 320 remaining patients, 139 received the brochure and 181 did not. An additional 55 patients were excluded due to preoperative opioid use. Overall, 35.3% of recipients and 38.3% of those who did not receive the brochure used all of their prescribed opioid medication (P = .625; confidence interval = -14.6%-8.8%). Among patients with unused opioid medication, a significantly higher proportion of brochure recipients disposed of the medicine compared with those who did not receive the brochure (46.7% vs 19.6%, P < .001).

CONCLUSIONS

Distribution of an educational brochure significantly improved disposal of unused opioids after surgery. This easily implemented intervention can improve disposal of unused opioids and ultimately decrease excess opioids available for diverted use in the community.

Readability assessment of the British Association of Dermatologists' patient information leaflets

BACKGROUND

The British Association of Dermatologists (BAD) develops and produces patient information leaflets (PILs) for British clinicians and the general public, and its website provides access to all the PILs. Health literacy across the UK remains variable. Readability instruments assess the comprehensibility of text, predominately using a composite of sentence length and/or word-syllable number. Instruments usually report text readability categorized by United States (US) grades; ideally, health literature should be rated at US grade ≤ 6 (UK Year 7; age 11-12 years).

METHODS

In collaboration with the BAD, PILs on the BAD website (n = 203) were downloaded for readability assessment. PILs were processed prior to analysis using Readability Studio software (Oleander Software, Vandalia, OH, USA). Established readability metrics were used: Flesch-Kincaid (FK), Simple Measure of Gobbledygook (SMOG), Gunning fog index (GFI), Fry, FORCAST and Flesch Reading Ease (FRE).

RESULTS

The mean (95% CI) US grade levels for all BAD PILs were: 9.8 (9.7-10.0) for FK, 12.1 (12.0-12.3) for SMOG, 11.8 (11.6-11.9) for GFI, 11.5 (11.1-11.8) for Fry and 10.7 (10.6-10.8) for FORCAST. For FRE, the level is reported from a normal spectrum of 0-100, and was found to be 52.2 (95% CI 34.0-78.0) in this study. In the UK context, the mean readability levels of the BAD PILs were rated as Year 10 (age 14-15 years) for FK and Year 13 (aged 17-18 years) for SMOG. For FK, outputs, only 1.0% of PILs (2 of 203) were the recommended US grade ≤ 6 according to FK, and for SMOG rating, none was rated at this level.

DISCUSSION

The majority of BAD PILs have been written at a level that will be challenging for some patients to read. Reducing sentence length and aiming for shorter words will improve accessibility.

A critical readability and quality analysis of internet-based patient information on neck dissections

Objective

Patients are increasingly turning to the Internet as a source of healthcare information. Given that neck dissection is a common procedure within the field of Otolaryngology - Head and Neck Surgery, the aim of this study was to evaluate the quality and readability of online patient education materials on neck dissection.

Methods

A Google search was performed using the term "neck dissection." The first 10 pages of a Google search using the term "neck dissection" were analyzed. The DISCERN instrument was used to assess quality of information. Readability was calculated using the Flesch-Reading Ease, Flesch-Kincaid Grade Level, Gunning-Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook Index.

Results

Thirty-one online patient education materials were included. Fifty-five percent (n = 17) of results originated from academic institutions or hospitals. The mean Flesch-Reading Ease score was 61.2 ± 11.9. Fifty-two percent (n = 16) of patient education materials had Flesch-Reading Ease scores above the recommended score of 65. The average reading grade level was 10.5 ± 2.1. The average total DISCERN score was 43.6 ± 10.1. Only 26% of patient education materials (PEMs) had DISCERN scores corresponding to a "good quality" rating. There was a significant positive correlation between DISCERN scores and both Flesch-Reading Ease scores and average reading grade level.

Conclusions

The majority of patient education materials were written above the recommended sixth-grade reading level and the quality of online information pertaining to neck dissections was found to be suboptimal. This research highlights the need for patient education materials regarding neck dissection that are high quality and easily understandable by patients.

Forecasting Erroneous Neural Machine Translation of Disease Symptoms: Development of Bayesian Probabilistic Classifiers for Cross-Lingual Health Translation

BACKGROUND

Machine translation (MT) technologies have increasing applications in healthcare. Despite their convenience, cost-effectiveness, and constantly improved accuracy, research shows that the use of MT tools in medical or healthcare settings poses risks to vulnerable populations.

OBJECTIVES

We aimed to develop machine learning classifiers (MNB and RVM) to forecast nuanced yet significant MT errors of clinical symptoms in Chinese neural MT outputs.

METHODS

We screened human translations of MSD Manuals for information on self-diagnosis of infectious diseases and produced their matching neural MT outputs for subsequent pairwise quality assessment by trained bilingual health researchers. Different feature optimisation and normalisation techniques were used to identify the best feature set.

RESULTS

The RVM classifier using optimised, normalised (L₂ normalisation) semantic features achieved the highest sensitivity, specificity, AUC, and accuracy. MNB achieved similar high performance using the same optimised semantic feature set. The best probability threshold of the best performing RVM classifier was found at 0.6, with a very high positive likelihood ratio (LR+) of 27.82 (95% CI: 3.99, 193.76), and a low negative likelihood ratio (LR-) of 0.19 (95% CI: 0.08, 046), suggesting the high diagnostic utility of our model to predict the probabilities of erroneous MT of disease symptoms to help reverse potential inaccurate self-diagnosis of diseases among vulnerable people without adequate medical knowledge or an ability to ascertain the reliability of MT outputs.

CONCLUSION

Our study demonstrated the viability, flexibility, and efficiency of introducing machine learning models to help promote risk-aware use of MT technologies to achieve optimal, safer digital health outcomes for vulnerable people.

Readability of Online Patient Information Relating to Cleft Palate Surgery

OBJECTIVE

It is important for health care education materials to be easily understood by caretakers of children requiring craniofacial surgery. This study aimed to analyze the readability of Google search results as they pertain to "Cleft Palate Surgery" and "Palatoplasty." Additionally, the study included a search from several locations globally to identify possible geographic differences.

DESIGN

Google searches of the terms "Cleft Palate Surgery" and "Palatoplasty" were performed. Additionally, searches of only "Cleft Palate Surgery" were run from several internet protocol addresses globally.

MAIN OUTCOME MEASURES

Flesch-Kincaid Grade Level and Readability Ease, Gunning Fog Index, Simple Measure of Gobbledygook (SMOG) index, and Coleman-Liau Index.

RESULTS

Search results for "Cleft Palate Surgery" were easier to read and comprehend compared to search results for "Palatoplasty." Mean Flesch-Kincaid Grade Level scores were 7.0 and 10.11, respectively (P = .0018). Mean Flesch-Kincaid Reading Ease scores were 61.29 and 40.71, respectively (P = .0003). Mean Gunning Fog Index scores were 8.370 and 10.34, respectively (P = .0458). Mean SMOG Index scores were 6.84 and 8.47, respectively (P = .0260). Mean Coleman-Liau Index scores were 12.95 and 15.33, respectively (P = .0281). No significant differences were found in any of the readability measures based on global location.

CONCLUSIONS

Although some improvement can be made, craniofacial surgeons can be confident in the online information pertaining to cleft palate repair, regardless of where the search is performed from. The average readability of the top search results for "Cleft Palate Surgery" is around the seventh-grade reading level (US educational system) and compares favorably to other health care readability analyses.

Readability and quality assessment of internet-based patient education materials related to nasal septoplasty

BACKGROUND

Given that nasal septoplasty is a common procedure in otolaryngology - head and neck surgery, the objective of this study was to evaluate the quality and readability of online patient education materials on septoplasty.

METHODS

A Google search was performed using eight different search terms related to septoplasty. Six different tools were used to assess the readability of included patient education materials. These included the Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning-Fog Index, Simple Measure of Gobbledygook Index, Coleman-Liau Index, and Automated Readability Index. The DISCERN tool was used to assess quality and reliability.

RESULTS

Eighty-five online patient education materials were included. The average Flesch-Reading Ease score for all patient education materials was 54.9 ± 11.5, indicating they were fairly difficult to read. The average reading grade level was 10.5 ± 2.0, which is higher than the recommended reading level for patient education materials. The mean DISCERN score was 42.9 ± 10.5 and 42% (36/85) of articles had DISCERN scores less than 39, corresponding to poor or very poor quality.

CONCLUSION

The majority of online patient education materials on septoplasty are written above the recommended reading levels and have significant deficiencies in terms of their quality and reliability. Clinicians and patients should be aware of the shortcomings of these resources and consider the impact they may have on patients' decision making.

Detecting Symptom Errors in Neural Machine Translation of Patient Health Information on Depressive Disorders: Developing Interpretable Bayesian Machine Learning Classifiers

Background: Due to its convenience, wide availability, low usage cost, neural machine translation (NMT) has increasing applications in diverse clinical settings and web-based self-diagnosis of diseases. Given the developing nature of NMT tools, this can pose safety risks to multicultural communities with limited bilingual skills, low education, and low health literacy. Research is needed to scrutinise the reliability, credibility, usability of automatically translated patient health information. Objective: We aimed to develop high-performing Bayesian machine learning classifiers to assist clinical professionals and healthcare workers in assessing the quality and usability of NMT on depressive disorders. The tool did not require any prior knowledge from frontline health and medical professionals of the target language used by patients. Methods: We used Relevance Vector Machine (RVM) to increase generalisability and clinical interpretability of classifiers. It is a typical sparse Bayesian classifier less prone to overfitting with small training datasets. We optimised RVM by leveraging automatic recursive feature elimination and expert feature refinement from the perspective of health linguistics. We evaluated the diagnostic utility of the Bayesian classifier under different probability cut-offs in terms of sensitivity, specificity, positive and negative likelihood ratios against clinical thresholds for diagnostic tests. Finally, we illustrated interpretation of RVM tool in clinic using Bayes' nomogram. Results: After automatic and expert-based feature optimisation, the best-performing RVM classifier (RVM_DUFS12) gained the highest AUC (0.8872) among 52 competing models with distinct optimised, normalised features sets. It also had statistically higher sensitivity and specificity compared to other models. We evaluated the diagnostic utility of the best-performing model using Bayes' nomogram: it had a positive likelihood ratio (LR+) of 4.62 (95% C.I.: 2.53, 8.43), and the associated posterior probability (odds) was 83% (5.0) (95% C.I.: 73%, 90%), meaning that approximately 10 in 12 English texts with positive test are likely to contain information that would cause clinically significant conceptual errors if translated by Google; it had a negative likelihood ratio (LR-) of 0.18 (95% C.I.: 0.10,0.35) and associated posterior probability (odds) was 16% (0.2) (95% C.I: 10%, 27%), meaning that about 10 in 12 English texts with negative test can be safely translated using Google.

Cleft Care Readability: Can Patients Access Helpful Online Resources?

OBJECTIVE

The present study identifies and analyzes online patient resources for cleft lip with or without cleft palate to survey the online educational landscape relative to the recommended difficulty set by the National Institutes of Health (NIH) and American Medical Association (AMA).

METHODS

An internet search of "cleft palate," "cleft lip," and 12 similar inputs were entered into a search engine. The first 50 links for each search term were identified, collected, and reviewed individually for relevance and accessibility. The content of the websites was analyzed with Readability Studio Version 2019.1. The following readability metrics were utilized in this study: (1) Coleman-Liau (grade levels), (2) New Dale-Chall, (3) Flesch-Kincaid, (4) Flesch Reading Ease, (5) FORCAST, (6) Fry, (7) Gunning Fog, (8) New Fog Count, (9) Raygor Readability Estimate, and (10) Simple Measure of Gobbledygook.

RESULTS

In no combination of search terms did any collection of links provide information within the mid-seventh grade levels recommended by the NIH. The analysis of 143 unique websites in the "Cleft Palate" group showed a readability level appropriate to high school students. The analysis of 144 unique websites in the "Cleft Lip" group showed a readability level appropriate for eighth grade students with 6 months of class complete.

CONCLUSIONS

The information presented to patients on cleft care is too complex and well above the recommended 7th-grade reading level target set forth by the NIH and AMA, which hinders functional health literacy.

Factors affecting the quality and reliability of online health information

Background

Internet represents a relevant source of information, but reliability of data that can be obtained by the web is still an unsolved issue. Non-reliable online information may have a relevance, especially in taking decisions related to health problems. Uncertainties on the quality of online health data may have a negative impact on health-related choices of citizens.

Objective

This work consisted in a cross-sectional literature review of published papers on online health information. The two main research objectives consisted in the analysis of trends in the use of health web sites and in the quality assessment and reliability levels of web medical sites.

Methods

Literature research was made using four digital reference databases, namely PubMed, British Medical Journal, Biomed, and CINAHL. Entries used were “trustworthy of medical information online,” “survey to evaluate medical information online,” “medical information online,” and “habits of web-based health information users”. Analysis included only papers published in English. The Newcastle Ottawa Scale was used to conduct quality checks of selected works.

Results

Literature analysis using the above entries resulted in 212 studies. Twenty-four articles in line with study objectives, and user characteristics were selected. People more prone to use the internet for obtaining health information were females, younger people, scholars, and employees. Reliability of different online health sites is an issue taken into account by the majority of people using the internet for obtaining health information and physician assistance could help people to surf more safe health web sites.

Conclusions

Limited health information and/or web literacy can cause misunderstandings in evaluating medical data found in the web. An appropriate education plan and evaluation tools could enhance user skills and bring to a more cautious analysis of health information found in the web.

Health literacy among neurosurgery and other surgical subspecialties: Readability of online patient materials found with Google

OBJECTIVE

To both determine whether the most high-yield online patient materials for surgical specialties meet the 6th grade readability level recommended by the National Institutes of Health (NIH) and American Medical Association (AMA), and to discover differences in readability across specialties. We hypothesize average readability scores will exceed an 11th grade level.

METHODS

The top five most common procedures for each of seven surgical specialties (neurological, orthopedic, plastic, general, thoracic, pediatric, and vascular) were searched using an incognito Google query to minimize location bias. The text from the top five patient-relevant links per procedure, excluding Wikipedia, journal articles, and videos, was extracted and inserted into Readability Studio Software for analysis.

RESULTS

The combined average grade level of materials (± standard deviation) was: 10.47 ± 2.51 Flesh-Kincaid Grade Level (FKGL), 11-12 New Dale-Chall (NDC), 10.09 ± 1.97 Simple Measure of Gobbledygook (SMOG), 12 Fry Graph (FG). Thoracic, neurologic, vascular, plastic, and orthopedic were least readable (grade level 10+ by all metrics).

CONCLUSIONS

High readability of procedure materials for patients is not unique to neurosurgery: all specialties exceeded the recommended 6th grade level by three or more grades. Online patient education materials related to surgical subspecialties must be written more comprehensibly.

A Critical Analysis of the Information Available Online for Ménière’s Disease

Objective

Patients increasingly rely on online resources for medical information; however, the Internet is unregulated and prone to misinformation. This study analyzes the reliability, quality, and readability of websites for Ménière’s disease.

Study Design

A Google search was performed using keywords Ménière’s disease. The first 5 pages (50 results) were reviewed. Websites were sorted into 5 categories: academic institutions, government agencies, professional organizations, medical information websites, and miscellaneous. The reliability, quality, and readability of each website were evaluated using the DISCERN instrument and Flesch-Kincaid Grade Level (FKGL). DISCERN assesses reliability and quality by scoring 15 questions on a scale from 1 (low) to 3 (high). The reliability score emphasizes clear objectives and sources, as well as lack of bias, whereas the quality score emphasizes information on treatment options. The FKGL of each website was calculated using a formula to determine the equivalent US grade reading level.

Setting

Ambulatory.

Subjects

None.

Results

Forty-two websites were analyzed. Academic institutions were the most common (n = 13, 31%) but scored the lowest using DISCERN at 1.75 ± 0.13. Medical information websites scored highest at 2.24 ± 0.09 ( P = .024 compared to academic institutions). The average FKGL of all websites was 10.12 ± 0.57 with medical information websites being the easiest to read at 8.84 ± 0.83. Only 5 (13%) of websites scored below the eighth-grade reading level.

Conclusions

Most top online search results for Ménière’s disease are deficient in quality and readability. Medical information websites are generally the most reliable and easy to read.

Readability of online materials for Dupuytren's contracture

STUDY DESIGN

Descriptive.

BACKGROUND

Dupuytren's contracture is a common disorder involving fibrosis of the palmar fascia. As patients are increasingly using online materials to gather health care information, it is imperative to assess the readability and appropriateness of this content. The recommended grade level for patient educational materials is seventh to eighth grade according to the National Institutes of Health. This study aims to assess the readability and content of online patient resources for Dupuytren's contracture.

PURPOSE OF THE STUDY

Evaluate readability of online patient education materials for Dupuytren's contracture.

METHODS

The largest public search engine, Google, was queried using the term "Dupuytren's contracture surgery" on February 26, 2016. Location filters were disabled, and sponsored results were excluded to avoid any inadvertent search bias. The 10 most popular Web sites were identified, and all relevant patient-directed information within 1 click from the original site was downloaded and saved as plain text. Readability was analyzed using 6 established analyses (Readable.io, Added Bytes, Ltd, UK).

RESULTS

Analysis of 10 Web sites demonstrates an average grade level of at least 11th grade (Flesch-Kincaid grade level, 10.2; Gunning-Fog grade level, 13.1; Coleman-Liau grade level, 14.4; Simple Measure of Gobbledygook grade level, 10.0; automated readability grade level, 9.7; and average grade level, 11.5). Overall Flesch-Kincaid reading ease index was 46.4, which is difficult. No single article was at the recommended reading level.

CONCLUSIONS

Online materials available for treatment of Dupuytren's contracture are above recommended reading levels and do not include a comprehensive explanation of treatment options, which may negatively impact decision making in patients seeking treatment for this condition. Surgeons and hand therapists alike should be cognizant of available online patient materials and make efforts to develop and provide more appropriate materials.

LEVEL OF EVIDENCE

V.

Readability of Individuals With Disabilities Education Act Part B Procedural Safeguards: An Update

Purpose

The purpose of this study was to analyze the readability of Individuals with Disabilities Education Act (IDEA) Part B procedural safeguard documents, as distributed by each of the 50 U.S. states. Results were compared to the 5th- to 6th-grade readability guideline for documents recommended by experts in health literacy and health communication.

Method

A commercially available readability software, Readability Studio ( Oleander Software, 2009 ), was used to assess document readability. Text-based files of each IDEA Part B procedural safeguard document were analyzed using four readability formulas: Flesch-Kincaid ( Flesch, 1965 ), Gunning fog index ( Gunning, 1952 ), Flesch Reading Ease ( Flesch, 1948 ), and Simple Measure of Gobbledygook ( McLaughlin, 1969 ).

Results

No procedural safeguard document scored below an 11th-grade reading level. Seventy-four percent of these documents were found to be written at a graduate reading level—meaning these documents are written for a reader who is currently enrolled in a master's degree or higher education program.

Conclusion

In an effort to decrease barriers to parent participation in the Individualized Education Planning process, those who administer IDEA Part B procedural safeguards should be sensitive to the potential mismatch between the literacy skills of the parent/guardian and the literacy skills needed to comprehend these documents. Developers of IDEA Part B procedural safeguards should account for estimated literacy skills of the general public as ongoing revisions to these safeguards are made.

Comprehension of Online Educational Material Regarding Orofacial Clefts

BACKGROUND

Cleft lip and palate are among the common birth defects in the United States. Management is complex and may be difficult for parents and patients to understand. The goal of this study was to evaluate the readability of online educational material provided by the American Society of Plastic Surgeons (ASPS) and the American Cleft Palate-Craniofacial Association (ACPA) websites regarding cleft lip and palate and to assess parents' comprehension.

METHODS

Parents of children younger than 10 years of age presenting to the plastic and reconstructive surgery clinic at Johns Hopkins All Children's Hospital for conditions other than cleft lip and palate were recruited for participation. Parents were given the literature published on the ASPS and ACPA web sites to review and were asked to complete a 5-question test specifically targeted to the content of the passage they reviewed. We tested the readability of the 2 online passages using Readability Studio software (Oleander Software Solutions Ltd).

RESULTS

Sixty parents completed the study. Across all groups, the mean score was 4.35 of a possible 5. The mean score was 4.4 for the ASPS web site and 4.29 for the ACPA web site. Fifty-three percent of participants answered all questions correctly. No significant difference in scores was noted between web sites (P = .48).

CONCLUSIONS

Despite containing content of varying reading-level difficulty, the literature posted on the ACPA and ASPS web sites concerning cleft lip and palate conveyed information equally well to the target parent demographic with high levels of comprehension.

Quantitative Readability Assessment of the Internal Medicine Online Patient Information on Annals.org

Background

Approximately 90% of Americans have access to the internet with the majority of people searching online for medical information pertaining to their health, or the health of loved ones. The public relies immensely on online health information to make decisions related to their healthcare. The American Medical Association (AMA) and the National Institute of Health (NIH) recommend that publicly available health-related information be written at the level of the sixth-seventh grade.

Materials and methods

Patient education materials available to the public on the Annals.org, a website sponsored by the American College of Physicians, were collected. All 89 patient education articles were downloaded from the website and analyzed for their ease of readability. The articles were analyzed utilizing a readability software generating five quantitative readability scores: Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG). All scores, with the exception of FRE, generate a grade level that correlates with the required school-grade level to ensure adequate readability of the information.

Results

Eighty-nine articles were analyzed generating an average score as follows: FRE 62.8, FKGL 7.0, GFI 8.6, CLI 9.6 and SMOG 9.8. Overall, 87.6% of the articles were written at a level higher than the 7th-grade level, which is recommended by the AMA and NIH.

Conclusion

In an era of increased reliance on the internet for medical information pertaining to patients’ health, materials written at a higher grade than recommended has the potential to negatively impact patients’ well-being, in addition to tremendous ramifications on the healthcare system. Potentially redrafting, these articles can prove beneficial to patients who rely on these resources for making healthcare-related decisions.

A Comparative Multimetric Assessment of English and Spanish Carpal Tunnel Syndrome Materials

BACKGROUND

Spanish-speaking Hispanics living in the United States utilize the internet as a primary means to obtain health information. Accurate, accessible information is important for English speakers; however, it could have even greater utility for Spanish speakers who have lower health literacy levels. The aim of this study was to evaluate and compare online English and Spanish carpal tunnel surgery materials provided by using a multimetric approach.

MATERIALS AND METHODS

A web search using the English term "carpal tunnel surgery" was performed. The first 10 institutional/organizational websites that provided carpal tunnel surgery information in English and Spanish were included. All relevant online materials were evaluated using the Patient Education and Materials Assessment Tool (PEMAT), Cultural Sensitivity Assessment Tool (CSAT), and Simplified Measure of Gobbledygook, Spanish (SOL) to assess understandability and actionability, cultural sensitivity, and readability, respectively.

RESULTS

There were no statistically significant differences in understandability or actionability scores between Spanish and English materials. Average cultural sensitivity scores for Spanish materials were significantly lower than English materials (P = 0.015). The average reading grade level of online English materials was greater than that for Spanish materials (P = 0.011). Both mean values were above the recommended sixth-grade reading level.

CONCLUSIONS

Online patient-directed information regarding carpal tunnel surgery exceeded the recommended reading grade level for both English and Spanish-speaking populations. Most Spanish materials were often direct translations and were not contoured to the elevated literacy needs of this demographic. Institutions must caution their authors to tailor their web material in a way that is sensitive to their target population to optimize understanding.

A readability analysis of online mental health resources

Analyzing the reading grade level of online mental health information is an important first step in ensuring that information is largely accessible by the general public, so as not to perpetuate existing health disparities across socioeconomic groups. The present study systematically examined grade-level readability of mental health information related to various psychiatric diagnoses, obtained from 6 highly utilized mental health websites, using a generalized estimating equations approach. Results suggest that, in general, the readability of mental health information is largely well above the 6th-to-8th grade level recommended by several national health organizations, including the CDC and NIH (Kutner, Greenberg, Jin, & Paulsen, 2006; National Institutes of Health, 2001, 2017), with reading-grade-level estimates from the model ranging from 5.62 to 17.9. Further efforts are required to ensure that writers of online health information do not exacerbate existing health disparities by ignoring these guidelines. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Evaluating the complexity of online patient education materials about brain aneurysms published by major academic institutions

OBJECTIVE

Health care education resources are increasingly available on the Internet. A majority of people reference these resources at one point or another. A threshold literacy level is needed to comprehend the information presented within these materials. A key component of health literacy is the readability of educational resources. The National Institutes of Health (NIH) and the American Medical Association have recommended that patient education materials be written between a 4th- and a 6th-grade education level. The authors assessed the readability of online patient education materials about brain aneurysms that have been published by several academic institutions across the US.

METHODS

Online patient education materials about brain aneurysms were downloaded from the websites of 20 academic institutions. The materials were assessed via 8 readability scales using Readability Studio software (Oleander Software Solutions), and then were statistically analyzed.

RESULTS

None of the patient education materials were written at or below the NIH's recommended 6th-grade reading level. The average educational level required to comprehend the texts across all institutions, as assessed by 7 of the readability scales, was 12.4 ± 2.5 (mean ± SD). The Flesch Reading Ease Scale classified the materials as “difficult” to understand, correlating with a college-level education or higher. An ANOVA test found that there were no significant differences in readability among the materials from the institutions (p = 0.215).

CONCLUSIONS

Brain aneurysms affect 3.2% of adults 50 years or older across the world and can cause significant patient anxiety and uncertainty. Current patient education materials are not written at or below the NIH's recommended 4th- to 6th-grade education level.

Evaluation of internet-based patient education materials from internal medicine subspecialty organizations: will patients understand them?

The majority of Americans use the Internet daily, if not more often, and many search online for health information to better understand a diagnosis they have been given or to research treatment options. The average American reads at an eighth-grade level. The purpose of this study is to evaluate the readability of online patient education materials on the websites of 14 professional organizations representing the major internal medicine subspecialties. We used ten well-established quantitative readability scales to assess written text from patient education materials published on the websites of the major professional organizations representing the following subspecialty groups: allergy and immunology, cardiology, endocrinology, gastroenterology, geriatrics, hematology, hospice and palliative care, infectious disease, nephrology, oncology, pulmonology and critical care, rheumatology, sleep medicine, and sports medicine. Collectively the 540 articles analyzed were written at an 11th-grade level (SD 1.4 grade levels). The sleep medicine and nephrology websites had the most readable materials, written at an academic grade level of 8.5 ± 1.5 and 9.0 ± 0.2, respectively. Material at the infectious disease site was written at the most difficult level, with average readability corresponding to grades 13.9 ± 0.3. None of the patient education materials we reviewed conformed to the American Medical Association (AMA) and the National Institutes of Health (NIH) guidelines requiring that patient education articles be written at a third- to seventh-grade reading level. If these online resources were rewritten, it is likely that more patients would derive benefit from reading them.

Abdominal imaging and patient education resources: enhancing the radiologist-patient relationship through improved communication

INTRODUCTION

The relative ease of Internet access and its seemingly endless amount of information creates opportunities for Americans to research medical diseases, diagnoses, and treatment plans. Our objective is quantitative evaluation of the readability level of patient education websites, written for the lay public, pertaining to common radiologic diagnostic test, and radiologic diagnoses specific to abdominal imaging.

METHODS

In October 2015, 10 search terms were entered in the Google search engine, and the top 10 links for each term were collected and independently examined for their readability level using 10 well-validated quantitative readability scales. Search terms included CT abdomen, MRI abdomen, MRI enterography, ultrasound abdomen, X-ray abdomen, cholecystitis, diverticulitis, hepatitis, inflammatory bowel disease, and pancreatitis. Websites not written exclusively for patients were excluded from the analysis.

RESULTS

As a group, the 100 articles were assessed at an 11.7 grade level. Only 2% (2/100) were written at the National Institutes of Health (NIH), and American Medical Association (AMA) suggested 3rd to 7th grade level to meet the 8th grade average reading level in the United States. In fact, 49% were written at a level that required a high school education or higher (greater than 12th grade).

CONCLUSIONS

With websites like radiologyinfo.org, generating over a million visitors a month, it is that clear there is a public interest in learning about radiology. However, given the discordance between the level of readability of the majority of the Internet articles and the NIH and AMA guidelines noted in this study on abdominal imaging readability, it is likely that many readers do not fully benefit from these resources on abdominal imaging.

Pediatric rhinoplasty: A discussion of perioperative considerations and systematic review

OBJECTIVES

Pediatric rhinoplasty has traditionally raised numerous concerns, including its impact on growth as well as the psychological sequelae of undergoing a potentially appearance-altering procedure. Our objective was to critically evaluate available individual patient data relevant to pediatric rhinoplasty, and further discuss perioperative considerations.

METHODS

A systematic review was conducted using PubMed/MEDLINE databases. Data extracted and analyzed from included studies included patient demographics, surgical indications, operative approaches, outcomes, complications, revision rates, and other clinical considerations.

RESULTS

Seven studies encompassing 253 patients were included, with age ranging from 7 months to 19 years. Two-thirds of patients were male. 41.7% reported antecedent trauma, and common overall surgical indications included "functional aesthetic" (24.5%) followed by cleft lip nasal deformity (15.8%). The majority (79.1%) underwent open approaches, and 71.1% of patients underwent concomitant septal intervention. The most frequently used grafting materials were septal cartilage (52.8%) and conchal cartilage (16.5%). Surgical outcomes were heterogeneous among these studies. Complication rates were only specified in 5 of the 7 studies and totaled 57 patients (39.6%). Aesthetic dissatisfaction (11.8%) and postoperative nasal obstruction (5.6%) were the most commonly reported complications. Revisions were performed in 13.5%.

CONCLUSION

Rhinoplasty is safe in the pediatric population, although revisions rates appear greater than those reported in adults. This study of 253 represents the largest pooled sample size to date; nonetheless, non-standardized outcome measures, minimal long-term followup data, and lack of discussion regarding psychological sequelae all contribute to the need for further high-quality studies evaluating this topic.

Colorectal cancer screening patient education materials-how effective is online health information?

PURPOSE

Patients screened for colorectal cancer (CRC) frequently turn to the Internet to improve their understanding of tests used for detection, including colonoscopy, flexible sigmoidoscopy, fecal occult blood test (FOBT), and CT colonography. It was of interest to determine the quality and readability levels of online health information.

METHODS

The screening tools were googled, and the top 20 results of each test were analyzed for readability, accessibility, usability, and reliability. The 80 articles excluded scientific literature and blogs. We used ten validated readability scales to measure grade levels, and one-way ANOVA and Tukey's honestly statistical different (HSD) post hoc analyses to determine any statistically significant differences among the four diagnostic tests. The LIDA tool assessed overall quality by measuring accessibility, usability, and reliability.

RESULTS

The 80 articles were written at an 11.7 grade level, with CT colonography articles written at significantly higher levels than FOBT articles, F(3, 75) = 3.07, p = 0.033. LIDA showed moderate percentages in accessibility (83.9 %), usability (73.0 %), and reliability (75.9 %).

CONCLUSIONS

Online health information about CRC screening tools are written at higher levels than the National Institute of Health (NIH) and American Medical Association (AMA) recommended third to seventh grade levels. More patients could benefit from this modality of information if it were written at a level and quality that would better facilitate understanding.

Are Online Zenker’s Diverticulum Materials Readable and Understandable?

Objective

Patients use a multitude of resources to learn about Zenker’s diverticulum (ZD). The objectives of this study were to assess the readability and understandability of online materials on ZD, evaluate them against the existing criteria, and investigate the relationship between readability and understandability.

Methods

The first 50 webpages from an online search for ZD were analyzed. Twenty-one webpages had materials intended for patients and were included in the study. The patient education materials (PEMs) were analyzed using 6 readability tools. Four individuals used the Patient Education Materials Evaluation Tool (PEMAT) to assess the understandability. Fleiss κ interrater reliability analysis determined consistency among the raters. Finally, Pearson correlation coefficient analyzed the relationship between readability and understandability.

Results

The reading grade level of the materials reviewed ranged from 10th to 16th grade while the understandability ranged from 31% to 74%. Correlation analysis demonstrated a strong negative correlation between readability and understandability ( r = −0.62, P < .05). Fleiss’ κ interrater reliability for the raters demonstrated substantial agreement between the 4 raters (κ = 0.64).

Conclusion

Online PEMs pertaining to ZD are written well above the recommended reading level. Materials written at a lower reading level are more understandable. A wide range of understandability exists among materials with identical reading grade levels.

Implications for Practice

Health care providers need to create new PEMs for ZD that are available online that are both readable and understandable. The PEMAT and readability formulas can provide a framework for authors to create these materials.

Radiation Oncology and Online Patient Education Materials: Deviating From NIH and AMA Recommendations

PURPOSE

Physicians encourage patients to be informed about their health care options, but much of the online health care-related resources can be beneficial only if patients are capable of comprehending it. This study's aim was to assess the readability level of online patient education resources for radiation oncology to conclude whether they meet the general public's health literacy needs as determined by the guidelines of the United States National Institutes of Health (NIH) and the American Medical Association (AMA).

METHODS

Radiation oncology-related internet-based patient education materials were downloaded from 5 major professional websites (American Society for Radiation Oncology, American Association of Physicists in Medicine, American Brachytherapy Society, RadiologyInfo.org, and Radiation Therapy Oncology Group). Additional patient education documents were downloaded by searching for key radiation oncology phrases using Google. A total of 135 articles were downloaded and assessed for their readability level using 10 quantitative readability scales that are widely accepted in the medical literature.

RESULTS

When all 10 assessment tools for readability were taken into account, the 135 online patient education articles were written at an average grade level of 13.7 ± 2.0. One hundred nine of the 135 articles (80.7%) required a high school graduate's comprehension level (12th-grade level or higher). Only 1 of the 135 articles (0.74%) met the AMA and NIH recommendations for patient education resources to be written between the third-grade and seventh-grade levels.

CONCLUSION

Radiation oncology websites have patient education material written at an educational level above the NIH and AMA recommendations; as a result, average American patients may not be able to fully understand them. Rewriting radiation oncology patient education resources would likely contribute to the patients' understanding of their health and treatment options, making each physician-patient interaction more productive and efficient.

Readability Assessment of Patient Information about Lymphedema and Its Treatment

BACKGROUND

Patient use of online resources for health information is increasing, and access to appropriately written information has been associated with improved patient satisfaction and overall outcomes. The American Medical Association and the National Institutes of Health recommend that patient materials be written at a sixth-grade reading level. In this study, the authors simulated a patient search of online educational content for lymphedema and evaluated readability.

METHODS

An online search for the term "lymphedema" was performed, and the first 12 hits were identified. User and location filters were disabled and sponsored results were excluded. Patient information from each site was downloaded and formatted into plain text. Readability was assessed using established tests: Coleman-Liau, Flesch-Kincaid, Flesch Reading Ease Index, FORCAST Readability Formula, Fry Graph, Gunning Fog Index, New Dale-Chall Formula, New Fog Count, Raygor Readability Estimate, and Simple Measure of Gobbledygook Readability Formula.

RESULTS

There were 152 patient articles downloaded; the overall mean reading level was 12.6. Individual website reading levels ranged from 9.4 (cancer.org) to 16.7 (wikipedia.org). There were 36 articles dedicated to conservative treatments for lymphedema; surgical treatment was mentioned in nine articles across four sites. The average reading level for conservative management was 12.7, compared with 15.6 for surgery (p < 0.001).

CONCLUSIONS

Patient information found through an Internet search for lymphedema is too difficult for many American adults to read. Websites queried had a range of readability, and surgeons should direct patients to sites appropriate for their level. There is limited information about surgical treatment available on the most popular sites; this information is significantly harder to read than sections on conservative measures.

Readability Analysis of the Package Leaflets for Biological Medicines Available on the Internet Between 2007 and 2013: An Analytical Longitudinal Study

Background

The package leaflet included in the packaging of all medicinal products plays an important role in the transmission of medicine-related information to patients. Therefore, in 2009, the European Commission published readability guidelines to try to ensure that the information contained in the package leaflet is understood by patients.

Objective

The main objective of this study was to calculate and compare the readability levels and length (number of words) of the package leaflets for biological medicines in 2007, 2010, and 2013.

Methods

The sample of this study included 36 biological medicine package leaflets that were downloaded from the European Medicines Agency website in three different years: 2007, 2010, and 2013. The readability of the selected package leaflets was obtained using the following readability formulas: SMOG grade, Flesch-Kincaid grade level, and Szigriszt’s perspicuity index. The length (number of words) of the package leaflets was also measured. Afterwards, the relationship between these quantitative variables (three readability indexes and length) and categorical (or qualitative) variables were analyzed. The categorical variables were the year when the package leaflet was downloaded, the package leaflet section, type of medicine, year of authorization of biological medicine, and marketing authorization holder.

Results

The readability values of all the package leaflets exceeded the sixth-grade reading level, which is the recommended value for health-related written materials. No statistically significant differences were found between the three years of study in the readability indexes, although differences were observed in the case of the length (P=.002), which increased over the study period. When the relationship between readability indexes and length and the other variables was analyzed, statistically significant differences were found between package leaflet sections (P<.001) and between the groups of medicine only with regard to the length over the three studied years (P=.002 in 2007, P=.007 in 2010, P=.009 in 2013). Linear correlation was observed between the readability indexes (SMOG grade and Flesch-Kincaid grade level: r²=.92; SMOG grade and Szigriszt’s perspicuity index: r²=.81; Flesch-Kincaid grade level and Szigriszt’s perspicuity index: r²=.95), but not between the readability indexes and the length (length and SMOG grade: r²=.05; length and Flesch-Kincaid grade level: r²=.03; length and Szigriszt’s perspicuity index: r²=.02).

Conclusions

There was no improvement in the readability of the package leaflets studied between 2007 and 2013 despite the European Commission’s 2009 guideline on the readability of package leaflets. The results obtained from the different readability formulas coincided from a qualitative point of view. Efforts to improve the readability of package leaflets for biological medicines are required to promote the understandability and accessibility of this online health information by patients and thereby contribute to the appropriate use of medicines and medicine safety.

Evaluation of the Readability of Dermatological Postoperative Patient Information Leaflets Across England

BACKGROUND

Postoperative patient information leaflets (PILs) provide important guidance to patients after skin surgery. Readability is a method of evaluating information for text comprehension. The recommended level for PIL readability is US grade ≤6.

OBJECTIVE

To evaluate the readability of public English dermatological postoperative PILs.

MATERIALS AND METHODS

All dermatology departments in England were requested to provide their postoperative PILs. Patient information leaflets were evaluated using Readability Studio (Oleander Software, Vandalia, OH). Two preselected parameters were also noted: whether the PIL was doctor or nurse-written, and whether the PIL was Information Standard hallmarked.

RESULTS

Eighty-five of one hundred thirty (65.4%) of PILs were evaluated. Only 29.4% of the PILs were grade level ≤6 with Flesch-Kincaid. The mean readability levels were 7.8 for Flesch-Kincaid, 67 for Flesch reading ease, 10.5 for Simple Measure of Gobbledygook (SMOG), 9.4 for Gunning-Fog, 8 for Fry, and 9.8 for FORCAST. No instruments demonstrated a significant difference between doctor (6) and nurse-written (7) PILs. Two instruments found that the 3 Information Standard hallmarked PILs had a higher (harder) readability than ordinary PILs (n = 82) (Gunning-Fog, p = .029*; SMOG p = .049*).

CONCLUSION

Most English postoperative dermatological PILs' readability levels exceed recommendations (US grade ≤6). Departmental PILs should be reviewed to ensure that they are comprehensible to their patients.

Readability of Spine-Related Patient Education Materials From Leading Orthopedic Academic Centers

STUDY DESIGN

Cross-sectional analysis of online spine-related patient education materials from leading academic centers.

OBJECTIVE

To assess the readability levels of spine surgery-related patient education materials available on the websites of academic orthopedic surgery departments.

SUMMARY OF BACKGROUND DATA

The Internet is becoming an increasingly popular resource for patient education. Yet many previous studies have found that Internet-based orthopedic-related patient education materials from subspecialty societies are written at a level too difficult for the average American; however, no prior study has assessed the readability of spine surgery-related patient educational materials from leading academic centers.

METHODS

All spine surgery-related articles from the online patient education libraries of the top five US News & World Report-ranked orthopedic institutions were assessed for readability using the Flesch-Kincaid (FK) readability test. Mean readability levels of articles amongst the five academic institutions and articles were compared. We also determined the number of articles with readability levels at or below the recommended sixth- or eight-grade levels. Intraobserver and interobserver reliability of readability assessment were assessed.

RESULTS

A total of 122 articles were reviewed. The mean overall FK grade level was 11.4; the difference in mean FK grade level between each department varied significantly (range, 9.3-13.4; P < 0.0001). Twenty-three articles (18.9%) had a readability level at or below the eighth grade level, and only one (0.8%) was at or below the sixth grade level. Intraobserver and interobserver reliability were both excellent (intraclass correlation coefficient of 1 for both).

CONCLUSION

Online patient education materials related to spine from academic orthopedic centers are written at a level too high for the average patient, consistent with spine surgery-related patient education materials provided by the American Academy of Orthopaedic Surgeons and spine subspecialty societies. This study highlights the potential difficulties patients might have in reading and comprehending the information in publicly available education materials related to spine.

LEVEL OF EVIDENCE

N/A.

Readability and Understandability of Online Vocal Cord Paralysis Materials

Objective

Patients use several online resources to learn about vocal cord paralysis (VCP). The objective of this study was to assess the readability and understandability of online VCP patient education materials (PEMs), with readability assessments and the Patient Education Materials Evaluation Tool (PEMAT), respectively. The relationship between readability and understandability was then analyzed.

Study Design and Setting

Descriptive and correlational design.

Method

Online PEMs were identified by performing a Google search with the term “vocal cord paralysis.” After scientific webpages, news articles, and information for medical professionals were excluded, 29 articles from the first 50 search results were considered. Readability analysis was performed with 6 formulas. Four individuals with different educational backgrounds conducted understandability analysis with the PEMAT. Fleiss’s Kappa interrater reliability analysis determined consistency among raters. Correlation between readability and understandability was determined with Pearson’s correlation test.

Results

The reading level of the reviewed articles ranged from grades 9 to 17. Understandability ranged from 29% to 82%. Correlation analysis demonstrated a strong negative correlation between materials’ readability and understandability ( r = −0.462, P < .05).

Conclusion

Online PEMs pertaining to VCP are written above the recommended reading levels. Overall, materials written at lower grade levels are more understandable. However, articles of identical grade levels had varying levels of understandability. The PEMAT may provide a more critical evaluation of the quality of a PEM when compared with readability formulas. Both readability and understandability should be used to evaluate PEMs.

Readability of Orthopaedic Oncology-related Patient Education Materials Available on the Internet

INTRODUCTION

A person's health literacy is one of the most important indicators of a patient's health status. According to national recommendations, patient education materials should be written at no higher than the sixth- to eighth-grade reading level. The purpose of our study was to assess the readability of online patient education materials related to orthopaedic oncology on the websites of the American Academy of Orthopaedic Surgeons (AAOS), American Cancer Society (ACS), Bone and Cancer Foundation (BCF), and National Cancer Institute (NCI).

METHODS

We searched the online patient education libraries of the AAOS, ACS, BCF, and NCI for all articles related to orthopaedic oncology. The Flesch-Kincaid (FK) readability score was calculated for each article and compared between sources.

RESULTS

A total of 227 articles were identified with an overall mean FK grade level of 9.8. Stratified by source, the mean FK grade levels were 10.1, 9.6, 11.1, and 9.5 for the AAOS, ACS, BCF, and NCI, respectively (P < 0.003). Only 31 articles (14%) and 1 article (0.4%) were at or below the recommended eighth- and sixth-grade levels, respectively.

DISCUSSION

Online patient education materials related to orthopaedic oncology appear to be written at a level above the comprehension ability of the average patient.

Third Molars on the Internet: A Guide for Assessing Information Quality and Readability

BACKGROUND

Directing patients suffering from third molars (TMs) problems to high-quality online information is not only medically important, but also could enable better engagement in shared decision making.

OBJECTIVES

This study aimed to develop a scale that measures the scientific information quality (SIQ) for online information concerning wisdom tooth problems and to conduct a quality evaluation for online TMs resources. In addition, the study evaluated whether a specific piece of readability software (Readability Studio Professional 2012) might be reliable in measuring information comprehension, and explored predictors for the SIQ Scale.

METHODS

A cross-sectional sample of websites was retrieved using certain keywords and phrases such as "impacted wisdom tooth problems" using 3 popular search engines. The retrieved websites (n=150) were filtered. The retained 50 websites were evaluated to assess their characteristics, usability, accessibility, trust, readability, SIQ, and their credibility using DISCERN and Health on the Net Code (HoNCode).

RESULTS

Websites' mean scale scores varied significantly across website affiliation groups such as governmental, commercial, and treatment provider bodies. The SIQ Scale had a good internal consistency (alpha=.85) and was significantly correlated with DISCERN (r=.82, P<.01) and HoNCode (r=.38, P<.01). Less than 25% of websites had SIQ scores above 75%. The mean readability grade (10.3, SD 1.9) was above the recommended level, and was significantly correlated with the Scientific Information Comprehension Scale (r=.45. P<.01), which provides evidence for convergent validity. Website affiliation and DISCERN were significantly associated with SIQ (P<.01) and explained 76% of the SIQ variance.

CONCLUSION

The developed SIQ Scale was found to demonstrate reliability and initial validity. Website affiliation, DISCERN, and HoNCode were significant predictors for the quality of scientific information. The Readability Studio software estimates were associated with scientific information comprehensiveness measures.

Patient education for carpal tunnel syndrome: analysis of readability

BACKGROUND

The National Institutes of Health and American Medical Association recommend a sixth grade reading level for patient-directed content. This study aims to quantitatively evaluate the readability of the most commonly used resources for surgical treatment of carpal tunnel syndrome.

METHODS

A web search for "carpal tunnel surgery" was performed using an Internet search engine, and the 13 most popular sites were identified. Relevant, patient-directed articles immediately accessible from the main site were downloaded and formatted into plain text. A total of 102 articles were assessed for readability using ten established analyses: first overall, then by website for comparison.

RESULTS

Patient information about carpal tunnel surgery had an overall average reading level of 13.1. Secondary analysis by website revealed a range of mean readability from 10.8 (high school sophomore level) to 15.3 (university junior level). All sites exceeded the recommended sixth grade reading level.

CONCLUSIONS

Online patient resources for carpal tunnel surgery uniformly exceed the recommended reading level. These are too difficult to be understood by a large portion of American adults. A better understanding of readability may be useful in tailoring more appropriate resources for average patient literacy.

The impact of health literacy in the care of surgical patients: a qualitative systematic review

BACKGROUND

Inadequate health literacy affects more than 90 million Americans and it has been associated with adverse outcomes in the medicine field including increased hospitalization rates and greater mortality. Since surgical patients are often required to make complex decisions and adhere to complex instructions, health literacy may have a profound impact in the surgical practice. The main objective of the current study was to systematically evaluate the role of health literacy in surgical patients.

METHODS

A systematic search was performed to identify studies that evaluated the role of health literacy in the perioperative setting following the PRISMA guidelines. Only studies that examined health literacy using a validated instrument in the perioperative setting were included.

RESULTS

Ten studies including data on 1147 patients were included. The median (IQR) number of patients in the included studies was 101 (30 to 152). The majority of studies used the Short Test of Functional Literacy in adults (STOFHLA) to evaluate patients' health literacy. Five studies evaluated the patients preoperatively, four studies evaluated patients in the postoperative period and in one study the time of evaluation in relation to the surgical procedure was not defined. The lowest prevalence of inadequate health literacy was detected in kidney transplant patients, 6 out of 124 (5 %), while the highest prevalence of inadequate health literacy was detected in orthopedic patients having total joint replacement, 86 out of 126 (60 %). Inadequate health literacy in the preoperative period was associated with poor medical information comprehension and it may adversely affect adherence to preoperative medications and even modulate surgical disparities. Inadequate health literacy in the postoperative period was associated with poor comprehension of discharge instructions and worse kidney function in transplant recipients.

CONCLUSIONS

Health literacy seems to have a very significant impact in the care of surgical patients. More studies to establish the impact of poor health literacy on perioperative outcomes are needed.

Health literacy and online educational resources: an opportunity to educate patients

OBJECTIVE

Given the increasing accessibility of material on the Internet and the use of these materials by patients as a source of health care information, the purpose of this study was to quantitatively evaluate the level of readability of resources made available on the European Society of Radiology website to determine whether these materials meet the health literacy needs of the general public as set forth by guidelines of the U.S. National Institutes of Health (NIH) and the American Medical Association (AMA).

MATERIALS AND METHODS

All 41 patient education articles created by the European Society of Radiology (ESR) were downloaded and analyzed with the following 10 quantitative readability scales: the Coleman-Liau Index, Flesch-Kincaid Grade Level, Flesch Reading Ease, FORCAST Formula, Fry Graph, Gunning Fog Index, New Dale-Chall, New Fog Count, Raygor Reading Estimate, and the Simple Measure of Gobbledygook.

RESULTS

The 41 articles were written collectively at a mean grade level of 13.0 ± 1.6 with a range from 10.8 to 17.2. For full understanding of the material, 73.2% of the articles required the reading comprehension level of, at minimum, a high school graduate (12th grade).

CONCLUSION

The patient education resources on the ESR website are written at a comprehension level well above that of the average Internet viewer. The resources fail to meet the NIH and AMA guidelines that patient education material be written between the third and seventh grade levels. Recasting these resources in a simpler format would probably lead to greater comprehension by ESR website viewers.

Reading grade level and completeness of freely available materials on thyroid nodules: there is work to be done

BACKGROUND

Patient education is one of the key components required for patients to participate in decision making for their care. Sources of information should be complete and understandable. The objective of this study was to assess the quality and reading grade level of freely available materials pertaining to the evaluation and treatment of thyroid nodules.

METHODS

All available materials were gathered through a comprehensive Internet search or direct contact with each identified organization associated with thyroid care. Materials meeting inclusion criteria were assessed for completeness of content and reading grade level using the SMOG readability formula. Then, the DISCERN instrument was utilized to assess the quality of the five materials which had the best (lowest) and the five materials that had the worst (highest) SMOG reading grade levels.

RESULTS

A total of 63 materials on thyroid nodule evaluation and management were identified. A minority of materials presented readers with a complete picture of the topic, describing both benefits and cautions of thyroid nodule evaluation, benefits and cautions of fine-needle thyroid nodule biopsy, or risks, benefits, and long-term effects of thyroidectomy. The SMOG reading grade level assessment was well above recommended levels. The range was eighth grade to college level; the mean and median reading grade level were 13. Lastly, the DISCERN instrument revealed that most of the materials with both the best and the worst reading grade levels assessed had "extensive or serious shortcomings" that would impede their usefulness as high-quality sources of information regarding management options.

CONCLUSIONS

Existing thyroid nodule evaluation and management materials have serious shortcomings. Future work should focus on the development of patient materials that are both grade level appropriate and meet the DISCERN criteria for topic completeness and accuracy.