A systematic review of readability and comprehension instruments used for print and web-based cancer information

A Bilingual Readability Assessment of Online Breast Cancer Screening and Treatment Information

INTRODUCTION

Presenting health information at a sixth-grade reading level is advised to accommodate the general public's abilities. Breast cancer (BC) is the second-most common malignancy in women, but the readability of online BC information in English and Spanish, the two most commonly spoken languages in the United States, is uncertain.

METHODS

Three search engines were queried using: "how to do a breast examination," "when do I need a mammogram," and "what are the treatment options for breast cancer" in English and Spanish. Sixty websites in each language were studied and classified by source type and origin. Three readability frameworks in each language were applied: Flesch Kincaid Reading Ease, Flesch Kincaid Grade Level, and Simple Measure of Gobbledygook (SMOG) for English, and Fernández-Huerta, Spaulding, and Spanish adaptation of SMOG for Spanish. Median readability scores were calculated, and corresponding grade level determined. The percentage of websites requiring reading abilities >sixth grade level was calculated.

RESULTS

English-language websites were predominantly hospital-affiliated (43.3%), while Spanish websites predominantly originated from foundation/advocacy sources (43.3%). Reading difficulty varied across languages: English websites ranged from 5th-12th grade (Flesch Kincaid Grade Level/Flesch Kincaid Reading Ease: 78.3%/98.3% above sixth grade), while Spanish websites spanned 4th-10th grade (Spaulding/Fernández-Huerta: 95%/100% above sixth grade). SMOG/Spanish adaptation of SMOG scores showed lower reading difficulty for Spanish, with few websites exceeding sixth grade (1.7% and 0% for English and Spanish, respectively).

CONCLUSIONS

Online BC resources have reading difficulty levels that exceed the recommended sixth grade, although these results vary depending on readability framework. Efforts should be made to establish readability standards that can be translated into Spanish to enhance accessibility for this patient population.

TrachGPT: Appraisal of tracheostomy care recommendations from an artificial intelligent Chatbot

Objective

Safe home tracheostomy care requires engagement and troubleshooting by patients, who may turn to online, AI-generated information sources. This study assessed the quality of ChatGPT responses to such queries.

Methods

In this cross-sectional study, ChatGPT was prompted with 10 hypothetical tracheostomy care questions in three domains (complication management, self-care advice, and lifestyle adjustment). Responses were graded by four otolaryngologists for appropriateness, accuracy, and overall score. The readability of responses was evaluated using the Flesch Reading Ease (FRE) and Flesch-Kincaid Reading Grade Level (FKRGL). Descriptive statistics and ANOVA testing were performed with statistical significance set to p < .05.

Results

On a scale of 1-5, with 5 representing the greatest appropriateness or overall score and a 4-point scale with 4 representing the highest accuracy, the responses exhibited moderately high appropriateness (mean = 4.10, SD = 0.90), high accuracy (mean = 3.55, SD = 0.50), and moderately high overall scores (mean = 4.02, SD = 0.86). Scoring between response categories (self-care recommendations, complication recommendations, lifestyle adjustments, and special device considerations) revealed no significant scoring differences. Suboptimal responses lacked nuance and contained incorrect information and recommendations. Readability indicated college and advanced levels for FRE (Mean = 39.5, SD = 7.17) and FKRGL (Mean = 13.1, SD = 1.47), higher than the sixth-grade level recommended for patient-targeted resources by the NIH.

Conclusion

While ChatGPT-generated tracheostomy care responses may exhibit acceptable appropriateness, incomplete or misleading information may have dire clinical consequences. Further, inappropriately high reading levels may limit patient comprehension and accessibility. At this point in its technological infancy, AI-generated information should not be solely relied upon as a direct patient care resource.

Readability and Accessibility of Patient Education Materials for Heart Failure in the United States

Online education materials are widely used by patients and caregivers to understand the management of complex chronic diseases such as heart failure (HF). Organizations such as the American Medical Association and National Institutes of Health recommend that materials be written at a 6th grade reading level. The current study examined the readability and accessibility of online education materials for patients with HF. Whole page text from each included website was entered into an online readability calculator. Five validated readability indices (Flesch-Kincaid Grade Level, Flesch Reading Ease Scale, Gunning Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook (SMOG Index)) were used to evaluate each source. Websites were categorized by source (government, public, and private). The availability of audiovisual accessibility features and content in non-English languages were assessed for each website. Of the 36 online resources analyzed, the median readability level was 9-10th grade by the Flesch-Kincaid Grade Level and college level using the Flesch Reading Ease Scale. The Gunning Fog Index and Coleman-Liau Index both showed median readability scores corresponding to a 12th grade reading level, while the SMOG Index showed a median score corresponding to that of the 9th grade. Only 10 websites (28%) offered information in languages other than English, and none provided comprehensive accessibility features for users with disabilities. Common online educational materials for patients with HF are characterized by a higher readability level than that recommended by the National Institutes of Health and American Medical Association with limited multilingual and accessibility options, potentially limiting the accessibility of resources to patients and caregivers.

Evaluating Health Literacy Resources for Secondary Hyperparathyroidism in End-Stage Kidney Disease

INTRODUCTION

Parathyroidectomy is recommended for severe secondary hyperparathyroidism (SHPT) due to end-stage kidney disease (ESKD), but surgery is underutilized. High quality and accessible online health information, recommended to be at a 6th-grade reading level, is vital to improve patient health literacy. This study evaluated available online resources for SHPT from ESKD based on information quality and readability.

METHODS

Three search engines were queried using the terms "parathyroidectomy for secondary hyperparathyroidism," "parathyroidectomy kidney/renal failure," "parathyroidectomy dialysis patients," "should I have surgery for hyperparathyroidism due to kidney failure?," and "do I need surgery for hyperparathyroidism due to kidney failure if I do not have symptoms?" Websites were categorized by source and origin. Two independent reviewers determined information quality using JAMA (0-4) and DISCERN (1-5) frameworks, and scores were averaged. Cohen's kappa evaluated inter-rater reliability. Readability was determined using the Flesch Kincaid Reading Ease, Flesch Kincaid Grade Level, and Simple Measure of Gobbledygook tools. Median readability scores were calculated, and corresponding grade level determined. Websites with reading difficulties >6th grade level were calculated.

RESULTS

Thirty one (86.1%) websites originated from the U.S., with most from hospital-associated (63.9%) and foundation/advocacy sources (30.6%). The mean JAMA and DISCERN scores for all websites were 1.3 ± 1.4 and 2.6 ± 0.7, respectively. Readability scores ranged from grade level 5-college level, and most websites scored above the recommended 6th grade level.

CONCLUSIONS

Patient-oriented websites tailoring SHPT from ESKD are at a reading level higher than recommended, and the quality of information is low. Efforts must be made to improve the accessibility and quality of information for all patients.

Readability of Trauma-related Patient Education Materials From the American Academy of Orthopaedic Surgeons and Orthopaedic Trauma Association Websites

INTRODUCTION

Web-based resources serve as a fundamental educational platform for orthopaedic trauma patients; however, they are frequently written above the recommended sixth-grade reading level, and previous studies have demonstrated this for the American Academy of Orthopaedic Surgeons (AAOS) web-based articles. In this study, we perform an updated assessment of the readability of AAOS trauma-related educational articles as compared with injury-matched education materials developed by the Orthopaedic Trauma Association (OTA).

METHODS

All 46 AAOS trauma-related web-based ( https://www.orthoinfo.org/ ) patient education articles were analyzed for readability. Two independent reviewers used (1) the Flesch-Kincaid Grade Level (FKGL) and (2) the Flesch Reading Ease (FRE) algorithms to calculate the readability level. Mean readability scores were compared across body part categories. A one-sample t -test was done to compare mean FKGL with the recommended sixth-grade readability level and the average American adult reading level. A two-sample t -test was used to compare the readability scores of the AAOS trauma-related articles with those of the OTA.

RESULTS

The average (SD) FKGL and FRE for the AAOS articles were 8.9 (0.74) and 57.2 (5.8), respectively. All articles were written above the sixth-grade reading level. The average readability of the AAOS articles was significantly greater than the recommended sixth-grade reading level ( P < 0.001). The average FKGL and FRE for all AAOS articles were significantly higher compared with those of the OTA articles (8.9 ± 0.74 versus 8.1 ± 1.14, P < 0.001 and 57.2 ± 5.8 versus 65.6 ± 6.6, P < 0.001, respectively). Excellent agreement was observed between raters for the FKGL 0.956 (95% confidence interval, 0.922 to 0.975) and FRE 0.993 (95% confidence interval, 0.987 to 0.996).

DISCUSSION

Our findings suggest that after almost a decade, the readability of the AAOS trauma-related articles remains unchanged. The AAOS and OTA trauma patient education materials have high readability levels and may be too difficult for patient comprehension. A need remains to improve the readability of these commonly used trauma education materials.

The Quality and Readability of Online Patient Information on Positional Head Shape Conditions

OBJECTIVE

Families increasingly use online resources to acquire medical information about their child's condition with little understanding of the legitimacy of the source of information or of the information itself. We evaluate the quality and readability of online information related to positional head shape conditions and identify unmet needs for healthcare providers to improve online patient education.

DESIGN

The search terms "flat head baby," "brachycephaly," and "plagiocephaly" were queried on the Google search engine and the first 20 websites for each were reviewed. Included websites were evaluated for quality using the DISCERN Instrument and readability using the Flesch-Kincaid Reading Grade Level (FKGL) and Flesch Reading Ease Score (FRES). Websites were categorized by upload source and results were compared using one-way ANOVA.

RESULTS

38 websites met inclusion criteria. There was no significant correlation between DISCERN score and Google search rank between the three search terms. Professional organizations provided websites with the highest mean DISCERN score (56.3) and commercial websites with the lowest score (36.6, P = .003), indicating "good" and "poor" quality content, respectively. Readability assessments showed an overall average FKGL of 9.9 and FRES of 54.4, suggesting "fairly difficult". Hospitals provided the most website results and tended to publish lower quality information, yet are the most readable.

CONCLUSIONS

High quality websites written at an appropriate reading level for the general public are lacking. A review of online resources for positional head shape conditions can be used to derive recommendations to improve the content of online patient education for pediatric healthcare.

Accuracy, readability, and quality of websites about metastatic renal cell carcinoma treatment

OBJECTIVES

Patients with metastatic renal cell carcinoma (mRCC) face complex treatment decisions and frequently turn to the Internet for treatment information. The content of patient educational websites about mRCC treatment has not been evaluated. This study evaluated the accuracy, readability, and quality of websites about the treatment of mRCC.

METHODS

A total of 2,700 Internet queries were performed. Across 3 Internet search engines, 25 links of 36 permutations of mRCC keywords and their synonyms were screened for eligibility. Eligible websites were English-language websites containing information about mRCC treatments. Sponsored, social media, provider-facing, and news websites were excluded. Accuracy of eligible websites was evaluated in 2 domains: (1) Completeness by calculating the percentage of mRCC facts included in websites using an investigator-created checklist based on the NCI's RCC Treatment (PDQ®)-patient version, and (2) Correctness by identifying incorrect statements that were inconsistent with guidelines. Websites containing ≥60% of checklist items had a "passing" completeness score. Incorrect statements were tallied and qualitatively categorized. Readability was evaluated using the Fry and SMOG formulae, which calculate reading grade levels. Quality was evaluated using validated instruments that appraise health information quality: QUEST (scored 0-28), which focuses on online information, and DISCERN (scored 16-80), which focuses on treatment choices.

RESULTS

Thirty-nine websites were analyzed. Mean completeness score was 30% (range 0%-69%); only 2 (5%) websites had a passing score. Twelve (31%) websites had ≥1 incorrect statement, such as listing homeopathy or hormone therapy as mRCC treatment options, or including outdated statements. Mean readability levels were 11th and 12th grades for the Fry and SMOG methods, respectively. No website had a reading level lower than 9th grade. Mean QUEST score was 19 (range 9-28); authorship, complementarity, and currency items had the lowest scores. Mean DISCERN score was 56 (range 42-76), with 7 (18%) websites rated "excellent", 22 (56%) rated "good", and 10 (26%) rated fair.

CONCLUSIONS

Many websites about mRCC treatment have incomplete, inaccurate, and unreadable information. Quality is highly variable. Efforts to improve accuracy, readability, and quality are needed to ensure that patients with mRCC can make well-informed treatment decisions and avoid harm from misinformation.

Quality and readability of online information and materials on post-surgery breast seroma

Aims/Background Seroma formation is the most common complication following breast surgery. However, there is little evidence on the readability of online patient education materials on this issue. This study aimed to assess the accessibility and readability of the relevant online information. Methods This systematic review of the literature identified 37 relevant websites for further analysis. The readability of each online article was assessed through using a range of readability formulae. Results The average Flesch-Reading Ease score for all patient education materials was 53.9 (± 21.9) and the average Flesch-Kincaid reading grade level was 7.32 (± 3.1), suggesting they were 'fairly difficult' to read and is higher than the recommended reading level. Conclusion Online patient education materials regarding post-surgery breast seroma are at a higher-than-recommended reading grade level for the public. Improvement would allow all patients, regardless of literacy level, to access such resources to aid decision-making around undergoing breast surgery.

An artificial intelligence language model improves readability of burns first aid information

AIMS

This study aimed to assess the potential of using an artificial intelligence (AI) large language model to improve the readability of burns first aid information.

METHODS

An AI language model (ChatGPT-3) was used to rewrite content from the top 50 English-language webpages containing burns first aid information to be understandable by an individual with the literacy level of an 11-year-old, as recommended by the American Medical Association and Health Education England. The assessment of readability was conducted using five validated tools.

RESULTS

In their original form, only 4% of the patient education materials (PEMs) met the target readability level across all tools. The median grade was 6.9 (SD=1.1). One-sample one-tailed t-test revealed that this was not significantly below the target (p = .31). After AI-modification, 18% of PEMs reached the target level using all tools, with a median grade of 6 (SD=0.9), which was significantly below the target level (p < .001). Once rewritten using AI, paired t-test demonstrated that all readability scores improved significantly (p < .001).

CONCLUSION

Utilising an AI language model proved an effective and viable method for enhancing readability of burns first aid information.

Quality and Accessibility of Liquid Biopsy Information

This cross-sectional study evaluates the information on a circulating tumor DNA test available to the public on popular internet resources.

Assessment of the Readability and Quality of Online Information for Patients and Their Families Regarding Schizophrenia

ABSTRACT

To our knowledge, this is the first study that specifically aims to assess the readability and quality of online information about schizophrenia. The analysis is performed on 93 of 195 websites that appeared in an advanced Google search of the term "schizophrenia" performed on a single day. The websites were categorized as commercial, nonprofit, professional, and government. The websites were evaluated using the Health on the Net Foundation (HON) code certificate, DISCERN tool, and Journal of the American Medical Association ( JAMA ) benchmark criteria for quality and the Flesch Reading Ease Formula, Flesch-Kincaid Grade Level (FKGL) Formula, Simple Measure of Gobbledygook, and Gunning Fog indices for readability. A total of 21.5% of all websites had a HON code certificate, 50.5% were accepted as high quality ( JAMA score ≥3), and 25.8% reached the recommended readability level (FKGL ≤8). Only three websites scored at a fifth- to sixth-grade reading level. Commercial and government websites had significantly lower DISCERN scores. Commercial websites had significantly lower Flesch Reading Ease Score and FKGL score than nonprofit websites. In conclusion, the current findings indicate that the quality of online information on schizophrenia is generally acceptable, but the readability is insufficient. Website creators, physicians, and health authorities should be more sensitive to the readability of online information about schizophrenia, considering the poor cognitive capacity of the patients and the unique nature of the disease.

An effort to improve the collection of patient-generated data: readability and understandability of patient-reported outcomes measures in a survivorship cohort

PURPOSE

In this study, we evaluated readability and understandability of nine French-language Patient-Reported Outcome Measures (PROMs) that are currently used in a contemporary longitudinal cohort of breast cancer survivors as part of an effort to improve equity in cancer care and research.

METHODS

Readability of PROMs was assessed using the Flesh Reading Ease Score (FRES), the Gunning's Fog Index (FOG), and the FRY graphics. Readability was considered ideal if mean score ≤ 6th-grade level and acceptable if between 6th and 8th grade. Understandability was evaluated using the Patient Education Materials Assessment Tool and defined as ideal if PEMAT ≥ 80%. The Evaluative Linguistic Framework for Questionnaires (ELF-Q) provided additional qualitative elements to assess understandability. Plain-language best practice was met if both readability and understandability were ideal.

RESULTS

None of the 9 PROMs evaluated had ideal readability scores and only 1 had an acceptable score. Understandability ranged from 55% to 91%, and only 3 PROMs had ideal scores. ELF-Q identified points for improvement in several understandability dimensions of the PROMs. None of the instruments met the definition of plain-language best practice.

CONCLUSION

None of the studied PROMs met the standards of readability and understandability. Future development and translation of PROMs should follow comprehensive linguistic and cultural frameworks to ensure plain-language standards and enhance equitable patient-centered care and research.

A standardised method for improving patient education material readability for orthopaedic trauma patients

PURPOSE

While the National Institutes of Health and American Medical Association recommend patient education materials (PEMs) should be written at the sixth-grade reading level or below, many patient education materials related to traumatic orthopaedic injuries do not meet these recommendations. The purpose of this study is to create a standardised method for enhancing the readability of trauma-related orthopaedic PEMs by reducing the use of ≥ three syllable words and reducing the use of sentences >15 words in length. We hypothesise that applying this standardized method will significantly improve the objective readability of orthopaedic trauma PEMs.

METHODS

A patient education website was queried for PEMs relevant to traumatic orthopaedic injuries. Orthopaedic trauma PEMs included (N = 40) were unique, written in a prose format, and <3500 words. PEM statistics, including scores for seven independent readability formulae, were determined for each PEM before and after applying this standard method.

RESULTS

All PEMs had significantly different readability scores when comparing original and edited PEMs (p < 0.01). The mean Flesch Kincaid Grade Level of the original PEMs (10.0 ± 1.0) was significantly higher than that of edited PEMs (5.8 ± 1.1) (p < 0.01). None of the original PEMs met recommendations of a sixth-grade reading level compared with 31 (77.5%) of edited PEMs.

CONCLUSIONS

This standard method that reduces the use of ≥ three syllable words and <15 word sentences has been shown to significantly reduce the reading-grade level of PEMs for traumatic orthopaedic injuries. Improving the readability of PEMs may lead to enhanced health literacy and improved health outcomes.

Study to evaluate the readability and visual appearance of online resources for blunt chest trauma: an evaluation of online resources using mixed methods

OBJECTIVES

Blunt chest trauma (BCT) is characterised by forceful and non-penetrative impact to the chest region. Increased access to the internet has led to online healthcare resources becoming used by the public to educate themselves about medical conditions. This study aimed to determine whether online resources for BCT are at an appropriate readability level and visual appearance for the public.

DESIGN

We undertook a (1) a narrative overview assessment of the website; (2) a visual assessment of the identified website material content using an adapted framework of predetermined key criteria based on the Centers for Medicare and Medicaid Services toolkit and (3) a readability assessment using five readability scores and the Flesch reading ease score using Readable software.

DATA SOURCES

Using a range of key search terms, we searched Google, Bing and Yahoo websites on 9 October 2023 for online resources about BCT.

RESULTS

We identified and assessed 85 websites. The median visual assessment score for the identified websites was 22, with a range of -14 to 37. The median readability score generated was 9 (14-15 years), with a range of 4.9-15.8. There was a significant association between the visual assessment and readability scores with a tendency for websites with lower readability scores having higher scores for the visual assessment (Spearman's r=-0.485; p<0.01). The median score for Flesch reading ease was 63.9 (plain English) with a range of 21.1-85.3.

CONCLUSIONS

Although the readability levels and visual appearance were acceptable for the public for many websites, many of the resources had much higher readability scores than the recommended level (8-10) and visually were poor.Better use of images would improve the appearance of websites further. Less medical terminology and shorter word and sentence length would also allow the public to comprehend the contained information more easily.

What Does ChatGPT Know About Dementia? A Comparative Analysis of Information Quality

The quality of information about dementia retrieved using ChatGPT is unknown. Content was evaluated for length, readability, and quality using the QUEST, a validated tool, and compared against online material from three North American organizations. Both sources of information avoided conflicts of interest, supported the patient-physician relationship, and used a balanced tone. Official bodies but not ChatGPT referenced identifiable research and pointed to local resources. Users of ChatGPT are likely to encounter accurate but shallow information about dementia. Recommendations are made for information creators and providers who counsel patients around digital health practices.

The Evaluation of the Suitability, Quality, and Readability of Publicly Available Online Resources for the Self-Management of Fear of Cancer Recurrence

Cancer survivors often rely on the internet for health information, which has varying levels of readability, suitability, and quality. There is a need for high-quality online self-management resources for cancer survivors with fear of cancer recurrence (FCR). This study evaluated the readability, suitability, and quality of publicly available online FCR self-management resources. A Google search using FCR-related keywords identified freely available FCR self-management resources for cancer survivors in English. Resource readability (reading grade level), suitability, and quality were evaluated using relevant assessment tools. Descriptive statistics and cluster analysis identified resources with higher suitability and quality scores. Mean resource (n = 23) readability score was grade 11 (SD = 1.6, Range = 9-14). The mean suitability score was 56.0% (SD = 11.4%, Range = 31.0-76.3%), indicating average suitability and the mean quality score was 53% (SD = 11.7%, Range = 27-80%), indicating fair quality. A cluster of 15 (65%) resources with higher suitability and quality scores was identified. There were no significant associations between suitability or quality scores and the type of organisation that published the resources. Online FCR self-management resources varied in readability, suitability and quality. Resources with higher quality and suitability scores relative to other resources are identified for use by healthcare professionals and cancer survivors. Resources that are more culturally appropriate, with lower reading grade levels and detailed self-management strategies are needed.

Readability, accountability, and quality of burns first aid information available online

AIM

To assess the readability, accountability, and quality of burns first aid information available online.

METHODS

The top 50 English language webpages containing burns first aid information were compiled and categorised. Readability was measured using five validated tools. Accountability was assessed using the Journal of the American Medical Association (JAMA) benchmarks. Quality was evaluated using a scale based on previous literature.

RESULTS

Two (4%) webpages were judged to be at the target reading level using all tools. Median grade ranged from 4.6 to 9.6 (M = 6.9, SD = 1.1). One-sample one-tailed t-test determined that median grade was not significantly below the target grade of ≤ 6.9 (p = 0.314). Only seven (14%) webpages satisfied all the JAMA accountability benchmarks. No webpages fulfilled all 15 quality criteria. Mean quality score was 9.8 (SD = 2.4). Only 27 (54%) advised 20 min of cooling. One-way analysis of variance demonstrated that accountability was influenced by source (p = 0.01). Pearson's correlation coefficient revealed that accountability and quality had a positive correlation (r = 0.32, p = 0.02).

CONCLUSION

Much of the burns first aid information available online is written above the recommended reading level and fails to meet standards of accountability or quality.

What Our Patients and Their Families Are Reading: Quality and Readability of Internet Information About Electroconvulsive Therapy

OBJECTIVES

The Internet is now more widely used than before by psychiatric patients and their families to search for medical conditions and treatments. To our knowledge, no study has addressed the quality and readability of online information about electroconvulsive therapy (ECT). We aimed to examine the quality and readability of English-language Internet information related to ECT.

METHODS

An advanced search of Internet Web sites containing information about ECT was conducted by using the search term "ECT or electroconvulsive therapy." The resulting Web sites were categorized into 1 of 3 categories (commercial, nonprofit, or professional organizations). Their quality was evaluated using Health on the Net code certification, the Journal of the American Medical Association (JAMA) benchmark criteria, and the DISCERN tool. The readability of the Web sites was assessed using the Flesch Reading Ease, Flesch-Kincaid Grade Level Formula, Simple Measure of Gobbledygook, and Gunning Fog indexes.

RESULTS

A total of 86 Web sites were included in the analysis. Of all these Web sites, 18 (20.9%) had a Health on the Net code certificate, and 16 (18.6%) were accepted as high quality (JAMA total score ≥3). The commercial Web sites had significantly lower DISCERN and JAMA benchmark scores compared with the other Web sites. A total of 30.23% of all the Web sites reached the recommended readability level (Flesch-Kincaid Grade Level Formula ≤ 8). Moreover, only 4 scored at the grade 5 to 6 reading level, which is considered ideal for patient educational materials.

CONCLUSIONS

Our study indicates that both the quality and readability of online information about ECT are not at the desired level. Physicians, patients, and their families should consider this failure in relation to online information about ECT. In addition, Web site creators and health authorities should be aware of their responsibilities for providing quality and readable health information to the public.

Communication strategies and effectiveness of early childhood obesity-related prevention programs for linguistically diverse communities: A rapid review

Children from culturally and linguistically diverse backgrounds experience higher rates of obesity and have poorer outcomes in obesity prevention studies. Interventions tailored to specific cultural groups may be limited within linguistically diverse, multicultural communities, and thus, alternative approaches to childhood obesity prevention in these communities are needed. This study aims to describe communication strategies used in interventions targeting prevention of obesity/obesity-related behaviors, among children 0-5 years, from linguistically diverse communities, and assess their effectiveness. A rapid review was conducted by systematically searching Medline, Embase, and CINAHL. The inclusion criteria are as follows: Studies reported an intervention tailored to linguistically diverse communities targeting at least one obesity-related behavior among children 0-5 years. The exclusion criteria are as follows: Interventions used simple language translations, targeted one language group, or treated obesity. A total of 4677 articles were identified with 14 studies meeting inclusion criteria. Key communication strategies included materials in multiple languages, English text written at a set readability level, and multimodal delivery. Six studies reported effectiveness data, of which five had effective primary or secondary outcomes. This is the first rapid review to identify communication strategies used in childhood obesity prevention interventions for linguistically diverse communities, highlighting a need for future research to incorporate and evaluate the communication strategies identified.

Health literacy in rotator cuff repair: a quantitative assessment of the understandability of online patient education material

Background

The American Medical Association and National Institutes of Health recommend online health information be written at a 6th grade or lower reading level for clear understanding. While syntax reading grade level has previously been utilized, those analyses do not determine whether readers are processing key information (understandability) or identifying available actions to take (actionability). The Patient Education Materials Assessment Tool (PEMAT-P) is a method to measure the understandability and actionability of online patient education materials. The purpose of this study was to evaluate online resources regarding rotator cuff repair utilizing measures of readability, understandability, and actionability.

Methods

The search term "rotator cuff surgery" was used in two independent online searches to obtain the top 50 search results. The readability of included resources was quantified using valid objective algorithms: Flesch-Kincaid Grade-Level, Simple Measure of Gobbledygook grade, Coleman-Liau Index, and Gunning Fog Index. The PEMAT-P form was used to assess actionability and understandability.

Results

A total of 49 unique websites were identified to meet our inclusion criteria and were included in our analysis. The mean Flesch-Kincaid Grade Level graded materials at a 10.6 (approximately a 10th grade reading level), with only two websites offering materials at a 6th grade reading level or below. The remaining readability studies graded the mean reading level at high school or greater, with the Gunning Fog Index scoring at a collegiate reading level. Mean understandability and actionability scores were 64.6% and 29.5%, respectively, falling below the 70% PEMAT score threshold for both scales. Fourteen (28.6%) websites were above the threshold for understandability, while no website (0%) scored above the 70% threshold for actionability. When comparing source categories, commercial health publishers provided websites that scored higher in understandability (P < .05), while private practice materials scored higher in actionability (P < .05). Resources published by academic institutions or organizations scored lower in both understandability and actionability than private practice and commercial health publishers (P < .05). No readability, understandability, or actionability score was significantly associated with search result rank.

Conclusion

Overall, online patient education materials related to rotator cuff surgery scored poorly with respect to readability, understandability, and actionability. Only two (4.1%) of the patient education websites scored at the American Medical Association and National Institutes of Health recommended reading level. Fourteen (28.6%) scored above the 70% PEMAT score for understandability; however, no website met the threshold for actionability.

Readability of spine-related patient education materials: a standard method for improvement

PURPOSE

Orthopaedic patient education materials (PEMs) have repeatedly been shown to be well above the recommended reading level by the National Institute of Health and American Medical Association. The purpose of this study is to create a standardized method to improve the readability of PEMs describing spine-related conditions and injuries. It is hypothesized that reducing the usage of complex words (≥ 3 syllables) and reducing sentence length to < 15 words per sentence improves readability of PEMs as measured by all seven readability formulas used.

METHODS

OrthoInfo.org was queried for spine-related PEMs. The objective readability of PEMs was evaluated using seven unique readability formulas before and after applying a standardized method to improve readability while preserving critical content. This method involved reducing the use of > 3 syllable words and ensuring sentence length is < 15 words. Paired samples t-tests were conducted to assess relationships with the cut-off for statistical significance set at p < 0.05.

RESULTS

A total of 20 spine-related PEM articles were used in this study. When comparing original PEMs to edited PEMs, significant differences were seen among all seven readability scores and all six numerical descriptive statistics used. Per the Flesch Kincaid Grade level readability formula, one original PEM (5%) versus 15 edited PEMs (75%) met recommendations of a sixth-grade reading level.

CONCLUSION

The current study shows that using this standardized method significantly improves the readability of spine-related PEMs and significantly increased the likelihood that PEMs will meet recommendations for being at or below the sixth-grade reading level.

Assessing readability and comprehension of web-based patient education materials by American Heart Association (AHA) and CardioSmart online platform by American College of Cardiology (ACC): How useful are these websites for patient understanding?

Cardiovascular diseases (CVD) are a leading cause of morbidity & mortality worldwide. Patient education materials help patients understand the disease and its management. Health literacy is an important challenge that may contribute to health inequities and disparities. The National Institute of Health and American Medical Association recommend patient education materials to be ≤6th-grade reading level.

Objective

To evaluate readability and comprehension of patient education materials related to CVD, available at the American Heart Association (AHA) & CardioSmart web platform by the American College of Cardiology (ACC) websites.

Method

We examined the readability and comprehension of 63 patient education materials (accessed June 2022) using: (a) Flesch Kincaid Readability Ease (FKRE): measures readability (0-100, goal > 70), (b) Flesch Kincaid Grade Level (FKGL) (goal = grade 7). We compared the AHA and ACC scores using descriptive and t-tests. P-value ≤ 0.05 was significant.

Results

Sixty-three web pages of patient education materials (AHA 24, ACC 39) were reviewed in June 2022. Mean ± standard deviation (SD) FKRE was 54.9 ± 6.8 for all the web pages. FKRE 50-60 equates to "fairly difficult to read." Mean ± SD FKGL was 10.0 ± 1.3. AHA patient education materials content was significantly more difficult to read and comprehend, were longer, and had more complex words than ACC patient education materials.

Conclusions

CVD-related patient education materials available online through leading national organizations are not congruent with the recommendations from national healthcare organizations. They are not as user-friendly as they can be. Urgent recognition of the gaps and unmet needs are indicated to optimize patient health literacy.

Readability assessment of patient educational materials for pediatric spinal conditions from top academic orthopedic institutions

Background

The Internet has become a popular source of health information for patients and their families. Healthcare experts recommend that the readability of online education materials be at or below a sixth grade reading level. This translates to a standardized Flesch Reading Ease Score between 81 and 90, which is equivalent to conversational English. However, previous studies have demonstrated that the readability of online education materials of various orthopedic topics is too advanced for the average patient. To date, the readability of online education materials for pediatric spinal conditions has not been analyzed. The objective of this study was to assess the readability of online educational materials of top pediatric orthopedic hospital websites for pediatric spinal conditions.

Methods

Online patient education materials from the top 25 pediatric orthopedic institutions, as ranked by the U.S. News and World Report hospitals for pediatric orthopedics, were assessed utilizing multiple readability assessment metrics including Flesch-Kincaid, Flesch Reading Ease, Gunning Fog Index, and others. Correlations between academic institutional ranking, geographic location, and the use of concomitant multimedia modalities with Flesch-Kincaid scores were evaluated using a Spearman regression.

Results

Only 32% (8 of 25) of top pediatric orthopedic hospitals provided online health information at or below a sixth grade reading level. The mean Flesch-Kincaid score was 9.3 ± 2.5, Flesch Reading Ease 48.3 ± 16.2, Gunning Fog Score 10.7 ± 3.0, Coleman-Liau Index 12.1 ± 2.8, Simple Measure of the Gobbledygook Index 11.7 ± 2.1, Automated Readability Index 9.0 ± 2.7, FORCAST 11.3 ± 1.2, and Dale-Chall Readability Index 6.7 ± 1.4. There was no significant correlation between institutional ranking, geographic location, or use of video material with Flesch-Kincaid scores (p = 0.1042, p = 0.7776, p = 0.3275, respectively).

Conclusion

Online educational material for pediatric spinal conditions from top pediatric orthopedic institutional websites is associated with excessively complex language which may limit comprehension for the majority of the US population.

Type of study/Level of evidence

Economic and Decision Analysis/level III.

Can pharmacy students use Wikipedia as a learning resource? Critical assessment of articles on chemotherapeutic drugs

Pharmacy students tend to use Wikipedia as a quick resource of knowledge. This study aimed to evaluate the accuracy of content and readability level of Wikipedia articles on chemotherapeutics, using quality and readability tools. Using the British National Formulary (BNF-2018) and ClinicalTrials.gov, we identified 188 chemotherapeutic drugs. We randomly selected 100 drugs with an Excel randomization program. The English Wikipedia was searched for the selected 100 drugs, and prints of the identified articles were obtained. Readability was calculated with an online instrument (http://www.readabilityformulas.com/). Articles were independently scored by two researchers using the modified DISCERN tool for content assessment. The modified DISCERN scores had a median value of 24 [interquartile range (IQR) = 7.5]. Two articles (2%) had good quality (DISCERN score 36-40), thirty-eight (38%) were moderate (DISCERN 26-35), and sixty (60%) were poor in score (DISCERN ≤25). The articles covered drug indications and most side effects. However, the majority lacked information on the routes of administration, contraindications, pharmacokinetics, and mechanisms of action. We found a correlation between DISCERN scores and number of edits (P value = 0.00033, R² = 0.1238). The number of references varied from 2 to 150 (median= 17, IQR = 17). Several problems were identified in the lists of references and citations. Most articles lacked tables and figures. The readability of the articles was 14.35 ± 3.13, consistent with the readability level of university students. In conclusion, the Wikipedia articles on chemotherapeutic drugs were not written for professional pharmacy students. Although they matched the expected readability level of university students, most were incomplete and lacked essential information.NEW & NOTEWORTHY Pharmacy students use Wikipedia as a quick resource of knowledge. However, Wikipedia articles are not written for professional pharmacy students. The study shows that although Wikipedia articles on chemotherapeutic drugs matched the expected readability level of university students, most needed to be completed and lacked essential information.

An evaluation of the suitability, readability, quality, and usefulness of online resources for patients with cancer receiving immunotherapy

OBJECTIVE

To evaluate the suitability, readability, quality, and usefulness of available online resources about the side effects of immunotherapy.

METHODS

Resources were identified through online searches (Google, Microsoft Bing, and Yahoo search engines) and evaluated using the Suitability Assessment Materials (SAM), three readability formulae, the DISCERN tool (quality), and a checklist of symptom management strategies and self-management skills (usefulness). Hierarchical cluster analysis identified resources scoring highly for suitability and quality. Factors associated with the clusters were identified using Fischer's exact tests.

RESULTS

The 51 resources retained for evaluation were of low suitability and quality and exceeded recommended reading levels. Resources were lacking key features to support learning (e.g., illustrations) as well as information about the risks of treatments and the effect of treatments on quality of life. Few resources addressed key self-management skills of problem-solving and taking action. The cluster analysis identified "low" and "moderate" clusters for suitability and quality, and pamphlets and booklets were more likely to be in the moderate cluster as compared to webpages.

CONCLUSION

There is a lack of high-quality resources to support patients' learning to self-manage side effects of immunotherapy.

PRACTICE IMPLICATIONS

Results identify the best available online resources for self-managing side effects of immunotherapy and gaps for future work. Based on findings, a template for patient education materials was developed.

Readability of Online Patient Education Materials for Graft-Versus-Host Disease

Graft-versus-host disease (GVHD) is a complex systemic diagnosis which is associated with significant symptom distress in patients. Patient education has shown to mitigate uncertainty and distress, but to our knowledge, no studies have evaluated patient education materials on GVHD. We characterized the readability and understandability of patient education materials on GVHD available online. We conducted a Google search of the top 100 non-sponsored search results, selecting for full-text patient education that is not peer-reviewed or a news article. We evaluated the text of the eligible search results against the Flesch-Kincaid Reading Ease, Flesch Kincaid Grade Level, Gunning Fog, Automated Readability, Linsear Write Formula, Coleman-Liau Index, Smog Index, and Patient Education Materials Assessment Tool (PEMAT) for understandability. Among 52 included Web results, 17 (32.7%) were provider-authored and 15 (28.8%) were hosted on university Web sites. The total average scores on validated readability tools were Flesch-Kincaid Reading Ease (46.4), Flesch Kincaid Grade Level (11.6), Gunning Fog (13.6), Automated Readability (12.3), Linsear Write Formula (12.6), Coleman-Liau Index (12.3), Smog Index (10.0), and PEMAT Understandability (65.5). Provider-authored links scored poorer than non-provider-authored links on all metrics, with significant differences for the Gunning Fog index (p < 0.05). University-hosted links scored better than non-university-hosted links on all metrics. Evaluation of online patient education materials for GVHD demonstrates the need for more readable and understandable resources to mitigate the distress and uncertainty that patients may feel upon being diagnosed with GVHD.

Readability, reliability and credibility of online patient information on skin grafts

BACKGROUND

Skin grafting reflects a common dermatological procedure for closing skin defects. Patient education is important for managing expectation and optimising skin graft take. While health information is increasingly accessed on the internet, there are no existing studies assessing their quality.

METHODS

The first 25 results from Google, Microsoft Bing and Yahoo! search engines using the term 'skin graft' were analysed using a variety of standard instruments. Readability was assessed using the Flesch-Kincaid Grade score (FKG), Gunning Fog Index (GFI), Simple Measure of Gobbledygook (SMOG) and the New Dale-Chall Readability Index (NDC). Reliability was assessed using the DISCERN instrument and credibility with the Journal of the American Medical Association Benchmark Criteria (JAMA). Transparency was identified by presence of the Health On the Net Foundation Code certification (HON-code).

RESULTS

Seventy-five websites were identified. After exclusion, forty-three remaining websites were analysed with average FKG, GFI and SMOG scores of 7.8, 10.1 and 10.7, respectively. The average NDC was 5.9. The average reliability was fair with a DISCERN score based on the first 15 questions of the instrument of 42.6. The mean JAMA score was 2, and 9 websites displayed the HON-code certificate.

CONCLUSIONS

Readability, reliability and credibility of online health information regarding skin grafting can be improved. Health care providers should critically assess existing online patient information or develop alternative material to educate patients undergoing skin graft surgery.

Content Analysis of Patient-Facing Information Related to Preeclampsia

BACKGROUND

Previous research has shown pregnant people are not knowledgeable about preeclampsia, a significant cause of maternal morbidity and mortality. This lack of knowledge may impact their ability to report symptoms, comply with recommendations, and receive appropriate follow-up care. Pregnant people commonly seek information from sources outside their treating clinician, including pregnancy-specific books and online sources. We examined commonly used preeclampsia information sources to evaluate whether pregnant people are receiving up-to-date, guideline-based information.

METHODS

We conducted a content analysis of preeclampsia-related information in top-ranking websites and bestselling pregnancy books. We used American College of Obstetricians and Gynecologists preeclampsia guidelines to construct a codebook, which we used to examine source content completeness and accuracy. For each source, we analyzed information about preeclampsia diagnosis, patient-reported symptoms, risk factors, prevention, treatment, and complications.

RESULTS

Across 19 included sources (13 websites and 6 books), we found substantial variation in completeness and accuracy of preeclampsia information. We found high rates of mentions for preeclampsia symptoms. Risk factors were more commonly included in online sources than book sources. Most sources mentioned treatment options, including blood pressure medication and giving birth; however, one-third of online sources positively mentioned the nonrecommended treatment of bed rest. Prevention methods, including prenatal aspirin for high-risk pregnancies, and long-term complications of preeclampsia were infrequently mentioned.

CONCLUSIONS

Varying rates of accuracy in patient-facing preeclampsia information mean there is substantial room for improvement in these sources. Ensuring pregnant people receive current guideline-based information is critical for improving outcomes and implementing shared decision-making.

Reducing the Use of Complex Words and Reducing Sentence Length to <15 Words Improves Readability of Patient Education Materials Regarding Sports Medicine Knee Injuries

Purpose

To develop a standardized method to improve readability of orthopaedic patient education materials (PEMs) without diluting their critical content by reducing the use of complex words (≥3 syllables) and shortening sentence length to ≤15 words.

Methods

OrthoInfo, a patient education website developed by the Academy of American Orthopedic Surgeons, was queried for PEMs relevant to the care of athletic injuries of the knee. Inclusion criteria were PEMs that were unique, pertained to topics of knee pathology in sports medicine, and written in a prose format. Exclusion criteria were information presented in video or slideshow format, or topics not pertaining to knee pathology in sports medicine. Readability of PEMs was evaluated using 7 unique readability formulas before and after applying a standardized method to improve readability while preserving critical content (reducing the use of ≥3 syllable words and ensuring sentence length is ≤15 words). Paired samples t-tests were conducted to assess the relationship between reading levels of the original PEMs and reading level of edited PEMs.

Results

Reading levels differed significantly between the 22 original PEMs and edited PEMs across all 7 readability formulas (P < .01). Mean Flesch Kincaid Grade Level of original PEMs (9.8 ± 1.4) was significantly increased compared to that of edited PEMs (6.4 ± 1.1) (P = 1.9 × 10^-13). 4.0% of original PEMs met National Institutes of Health recommendations of a sixth-grade reading level compared with 48.0% of modified PEMs.

Conclusions

A standardized method that reduces the use of ≥3 syllable words and ensures sentence length is ≤15 words significantly reduces the reading-grade level of PEMs for sports-related knee injuries. Orthopaedic organizations and institutions should apply this simple standardized method when creating PEMs to enhance health literacy.

Clinical Relevance

The readability of PEMs is important when communicating technical material to patients. While many studies have suggested strategies to improve the readability of PEMs, literature describing the benefit of these proposed changes is scarce. The information from this study details a simple standardized method to use when creating PEMs that may enhance health literacy and improve patient outcomes.

Readability of online monkeypox patient education materials: Improved recognition of health literacy is needed for dissemination of infectious disease information

BACKGROUND

Health literacy is key to navigating the current global epidemic of misinformation and inaccuracy relating to healthcare. The American Medical Association (AMA) suggests health information should be written at the level of American sixth grade. With the monkeypox outbreak being declared a Public Health Emergency of International Concern (PHEIC) in July 2022, we sought to assess the readability of online patient education materials (PEMs) relating to monkeypox to see if they are at the target level of readability.

METHODS

A search was conducted on Google.com using the search term 'Monkeypox'. The top 50 English language webpages with patient education materials (PEMs) relating to monkeypox were compiled and categorised by country of publication and URL domain. Readability was assessed using five readability tools: Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), and, Simple Measure of Gobbledygook Index (SMOG). Unpaired t-test for URL domain, and one-way ANOVA for country were performed to determine influence on readability.

RESULTS

Three of the five tools (FRES, GFI, CLI) identified no webpages that met the target readability score. The FKGL and SMOG tools identified one (2%) and two (4%) webpages respectively that met the target level. County and URL domain demonstrated no influence on readability.

CONCLUSION

Online PEMs relating to monkeypox are written above the recommended reading level. Based on the previously established effect of health literacy, this is likely exacerbating health inequalities. This study highlights the need for readability to be considered when publishing online PEMs.

Readability assessment of patient educational materials for pediatric spinal deformity from top academic orthopedic institutions

STUDY DESIGN

Cross-sectional analysis of patient educational materials from top pediatric orthopedic hospital websites.

OBJECTIVE

To assess the readability of online educational materials of top pediatric orthopedic hospital websites for pediatric spinal deformity. The internet has become an increasingly popular source of health information for patients and their families. Healthcare experts recommend that the readability of online education materials be at or below a 6th-grade reading level. However, previous studies have demonstrated that the readability of online education materials on various orthopedic topics is too advanced for the average patient. To date, the readability of online education materials for pediatric spinal deformity has not been analyzed.

METHODS

Online patient education materials from the top 25 pediatric orthopedic institutions, as ranked by the U.S. News and World Report hospitals for pediatric orthopedics, were accessed utilizing the following readability assessments: Flesch-Kincaid (FK), Flesch Reading Ease, Gunning Fog Index, Coleman-Liau Index, Simple Measure of the Gobbledygook Index (SMOG), Automated Readability Index, FORCAST, and the New Dale and Chall Readability. Correlations between academic institutional ranking, geographic location, and the use of concomitant multi-media modalities with FK scores were evaluated using a Spearman regression.

RESULTS

Only 48% (12 of 25) of top pediatric orthopedic hospitals provided online information regarding pediatric spinal deformity at or below a 6th-grade reading level. The mean FK score was 9.0 ± 2.7, Flesch Reading Ease 50.8 ± 15.6, Gunning Fog Score 10.6 ± 3.1, Coleman-Liau Index 11.6 ± 2.6, SMOG index 11.7 ± 2.0, Automated Readability Index 8.6 ± 2.8, and Dale-Chall Readability Score 6.4 ± 1.4. There was no significant correlation between institutional ranking, geographic location, or use of multimedia with FK scores.

CONCLUSION

Online educational material for pediatric spinal deformity from top pediatric orthopedic institutional websites are associated with poor readability.

Readability and Quality of Online Health Information Regarding Parathyroidectomy

Objective

Assessment of the readability and quality of online health information regarding parathyroidectomy.

Study Design

Cross-sectional analysis.

Setting

Websites providing patient-oriented health information regarding parathyroidectomy obtained via the Google search engine.

Methods

The top 75 Google search results for “parathyroidectomy,”“parathyroid surgery,” and “parathyroid gland removal” were reviewed. Websites were categorized by website type and country of origin. Readability was assessed by Flesch-Kincaid Grade Level and Simple Measure of Gobbledygook. Website quality was assessed per JAMA benchmark criteria and the DISCERN instrument.

Results

A total of 74 unique websites were evaluated. The mean readability of the assessed websites exceeded the recommended sixth-grade reading level on the Flesch-Kincaid Grade Level and Simple Measure of Gobbledygook (P < .001). Readability did not vary significantly by website type. Websites originating from the United Kingdom were significantly more readable than those from the United States. The majority of assessed websites were of poor quality (n = 42, 56.8%) on assessment based on the DISCERN instrument. Quality varied significantly by website category on the JAMA benchmark criteria (P < .001) and DISCERN score (P = .049) with commercial websites receiving the highest scores. DISCERN score also varied significantly by country of origin (P = .036) with UK sites receiving highest mean DISCERN scores.

Conclusion

Online health information regarding parathyroidectomy is largely of poor quality and is poorly readable for many patients. Institutions utilizing well-defined guidelines for development of patient educational resources may provide online health information of greater quality and readability.

Quality of web-based information about the coronavirus disease 2019: a rapid systematic review of infodemiology studies published during the first year of the pandemic

BACKGROUND

Following the outbreak of the coronavirus disease 2019, adequate public information was of outmost importance. The public used the Web extensively to read information about the pandemic, which placed significant responsibility in, for many, an unfamiliar situation as the disease spread across the globe. The aim of this review was to synthesize the quality of web-based information concerning the coronavirus disease 2019 published during the first year of the pandemic.

MATERIALS AND METHODS

A rapid systematic review was undertaken by searching five electronic databases (CINAHL, Communication & Mass Media Complete, PsycINFO, PubMed, Scopus). Empirical infodemiology reports assessing quality of information were included (n = 22). Methodological quality and risk of bias was appraised with tools modified from previous research, while quality assessment scores were synthesized with descriptive statistics. Topics illustrating comprehensiveness were categorized with content analysis.

RESULTS

The included reports assessed text-based content (n = 13) and videos (n = 9). Most were rated good overall methodological quality (n = 17). In total, the reports evaluated 2,654 websites or videos and utilized 46 assessors. The majority of the reports concluded that websites and videos had poor quality (n = 20). Collectively, readability levels exceeded the recommended sixth grade level. There were large variations in ranges of the reported mean or median quality scores, with 13 of 15 total sample scores being classified as poor or moderate quality. Four studies reported that ≥ 28% of websites contained inaccurate statements. There were large variations in prevalence for the six categories illustrating comprehensiveness.

CONCLUSION

The results highlight quality deficits of web-based information about COVID-19 published during the first year of the pandemic, suggesting a high probability that this hindered the general population from being adequately informed when faced with the new and unfamiliar situation. Future research should address the highlighted quality deficits, identify methods that aid citizens in their information retrieval, and identify interventions that aim to improve the quality of information in the online landscape.

Improving health literacy with mumps, measles and rubella (MMR) vaccination: comparison of the readability of MMR patient-facing literature and MMR scientific abstracts

Background:

Historically, there have been many factors that have influenced mumps, measles and rubella (MMR) vaccine uptake, including media bias, social/economic determinants, parental education level, deprivation and concerns over vaccine safety. Readability metrics through online tools are now emerging as a means for healthcare professionals to determine the readability of patient-facing vaccine information. The aim of this study was to examine the readability of patient-facing materials describing MMR vaccination, through employment of nine readability and text parameter metrics, and to compare these with MMR vaccination literature for healthcare professionals and scientific abstracts relating to MMR vaccination.

Materials and methods:

The subscription-based online Readable program (readable.com) was used to determine nine readability indices using various readability formulae: Established readability metrics (n = 5) (Flesch–Kinkaid Grade Level, Gunning Fog Index, SMOG Index, Flesch Reading Ease and New Dale-Chall Score), as well as Text parameters (n = 4) (sentence count, word count, number of words per sentence, number of syllables per word) with 47 MMR vaccination texts [patient-facing literature (n = 22); healthcare professional–focused literature (n = 8); scientific abstracts (n = 17)].

Results:

Patient-facing vaccination literature had a Flesch Reading Ease score of 58.4 and a Flesch–Kincaid Grade Level of 8.1, in comparison with poorer readability scores for healthcare professional literature of 30.7 and 12.6, respectively. MMR scientific abstracts had the poorest readability (24.0 and 14.8, respectively). Sentence structure was also considered, where better readability metrics were correlated with significantly lower number of words per sentence and less syllables per word.

Conclusion:

Use of these readability tools enables the author to ensure their research is more readable to the lay audience. Patient co-production initiatives would help to ensure that not only can the target audience read the literature, but that they understand the content. Increased patient-centric focus groups would give better insights into reasons for MMR-associated vaccine hesitation and vaccine refusal.

Quality and readability of online information on plantar fasciitis and calcaneal spur

Plantar fasciitis and calcaneal spur are common causes of heel pain in the community. People use the Internet to obtain medical information about diseases. We reviewed Internet information sources on plantar fasciitis and calcaneal spur for quality and readability. The first 50 websites for each search term ("calcaneal spur", "heel spur", and "plantar fasciitis") were scanned on www.google.com . Six different valid tools were used for information quality and readability assessment. We searched for HONCode (Health On the Net Foundation Code) stamps on included websites. The total mean points for DISCERN were 50.52 ± 14.62, and the total mean points for JAMA (Journal of the American Medical Association) were 2.42 ± 1.26. In total, 25.72% of 97 websites had HONCode stamps. The average scores for the readability indicators were calculated to be Flesch-Kincaid Grade Level (FKGL): 7.27 ± 1.71, Gunning Fog: 8.46 ± 2.17, Simple Measure of Gobbledygook (SMOG): 6.89 ± 1.24, and Coleman Liau Index: 15.56 ± 1.85. In our study, when the website resources were examined, there were profit websites the most and website quality and readability were moderate level. A significant proportion of the websites have a financial bias and provide low-quality information. A mechanism for monitoring the quality and readability of online information must be established and managed systematically.

Academic and community hernia center websites in the United States fail to meet healthcare literacy standards of readability

BACKGROUND

Health literacy is considered the single best predictor of health status. Organizations including the American Medical Association (AMA) and the National Institutes of Health (NIH) have recommended that the readability of patient education materials not exceed the sixth-grade level. Our study focuses on the readability of self-designated hernia centers websites at both academic and community organizations across the United States to determine their ability to dispense patient information at an appropriate reading level.

METHODS

A search was conducted utilizing the Google search engine. The key words "Hernia Center" and "University Hernia Center" were used to identify links to surgical programs within the United States. The following readability tests were conducted via the program: Flesch-Kincaid Grade Level (FKGL), Gunning Fox Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG), and Flesch Reading Ease (FRE) score.

RESULTS

Of 96 websites, zero (0%) had fulfilled the recommended reading level in all four tests. The mean test scores for all non-academic centers (n = 50) were as follows: FKGL (11.14 ± 2.68), GFI (14.39 ± 3.07), CLI (9.29 ± 2.48) and SMOG (13.38 ± 2.03). The mean test scores [SK1] for all academic programs (n = 46) were as follows: FKGL (11.7 ± 2.66), GFI (15.01 ± 2.99), CLI (9.34 ± 1.91) and SMOG (13.71 ± 2.02). A one-sample t test was performed to compare the FKGL, GFI, CLI, and SMOG scores for each hernia center to a value of 6.9 (6.9 or less is considered an acceptable reading level) and a p value of 0.001 for all four tests were noted demonstrating statistical significance. The Academic and Community readability scores for both groups were compared to each other with a two-sample t test with a p value of > 0.05 for all four tests and there were no statistically significant differences.

CONCLUSION

Neither Academic nor Community hernia centers met the appropriate reading level of sixth-grade or less. Steps moving forward to improve patient comprehension and/or involving with their care should include appropriate reading level material, identification of a patient with a low literacy level with intervention or additional counseling when appropriate, and the addition of adjunct learning materials such as videos.

Assessment of the Readability of Web-Based Patient Education Material From Major Canadian Pediatric Associations: Cross-sectional Study

BACKGROUND

Web-based patient education materials (PEMs) are frequently written above the recommended reading level in North America. Poor PEM readability limits the accessibility of medical information for individuals with average literacy levels or lower. Pediatric hospital and association websites have not only been shown to be a preferred source of information among caregivers but have also become a necessity during the COVID-19 pandemic. The readability of Canadian pediatric association websites has not yet been assessed.

OBJECTIVE

The aim of this study is to determine if the content of PEMs from Canadian pediatric associations is written at a reading level that the majority of Canadians can understand.

METHODS

A total of 258 PEMs were extracted from 10 Canadian pediatric associations and evaluated for their reading level using 10 validated readability scales. The PEMs underwent a difficult word analysis and comparisons between PEMs from different associations were conducted.

RESULTS

Web-based PEMs were identified from 3 pediatric association websites, where the reading level (calculated as a grade level) was found to be an average of 8.8 (SD 1.8) for the Caring for Kids website, 9.5 (SD 2.2) for the Pediatric Endocrine Group website, and 13.1 (SD 2.1) for the Atlantic Pediatric Society website. The difficult word analysis identified that 19.9% (SD 6.6%) of words were unfamiliar, with 13.3% (SD 5.3%) and 31.9% (SD 6.1%) of words being considered complex (≥3 syllables) and long (≥6 letters), respectively.

CONCLUSIONS

The web-based PEMs were found to be written above the recommended seventh-grade reading level for Canadians. Consideration should be made to create PEMs at an appropriate reading level for both patients and their caregivers to encourage health literacy and ultimately promote preventative health behaviors and improve child health outcomes.

The readability of online patient education materials for slipped capital femoral epiphysis

Given the long-term complications of undiagnosed slipped capital femoral epiphysis (SCFE) and the importance of readable health information materials on positive, equitable health outcomes, the objective of this study was to determine if the online patient education materials regarding SCFE are written at or below accepted recommendations. The secondary objective was to determine whether the readability of these materials varied when stratified by the type of website. 'Slipped capital femoral epiphysis', 'SCFE', and 'slipped femoral head' were used as search queries in three common search engines. The readability of each website was evaluated using five established metrics, and the scores were compared by website type and by the complexity of the search query. In this study of 53 unique websites about SCFE, we demonstrated that only one of the web pages was written at the recommended sixth-grade level, and the mean reading level of the online material was above the 10th-grade level. Post hoc testing showed that only websites associated with pediatric academic institutions were written at a significantly lower grade level than general health websites [P < 0.05 for all, range (0.003, 0.04)]. The materials about SCFE that are available to patients and their families continue to be written at an inappropriate level. To increase accessibility and allow for equitable long-term health outcomes, physicians, universities, hospitals and medical societies must ensure that they produce readable education to increase patients' understanding of SCFE, its symptoms and available treatment options. Future studies evaluating progress regarding these metrics are warranted.

Readability assessment of vaccine information: A systematic review for addressing vaccine hesitancy

OBJECTIVE

Vaccine hesitancy is a problem attracting growing attention and concern. Communication can be an effective tool to counteract vaccine hesitancy and promote optimal vaccine uptake. Readability has been recognized as one of the more important aspects of health communication for achieving good health literacy. We reviewed studies of readability assessment in the area of vaccine communication.

METHODS

We conducted a systematic literature search in September 2020, using four online databases (Medline, CINAHL, PsycArticles, and PsycINFO). We included studies that assessed the readability level of online and offline vaccine information materials.

RESULTS

We found 12 articles that were appropriate for inclusion. Ten of the studies were published after 2016. The readability levels of the majority of the materials assessed were found to be difficult and higher than 8th-grade level.

CONCLUSION

Readability assessments of vaccine information are scarce. The limited evidence shows that the readability level of vaccine information supplied by health care providers is more difficult to read than recommended. More studies on the readability of vaccine information are recommended.

PRACTICE IMPLICATIONS

Difficulty reading vaccine information may influence attitudes toward acceptance of or hesitancy to take vaccines. It is recommended that health care professionals use guidelines and tools to create easy-to-read vaccine information.

Optimizing Patient Information Material for a New Psycho-Oncological Care Program Using a Participatory Health Research Approach in Germany

High-quality patient information material (PIM) is essential for patients´ informed decision-making, and its quality may influence a care program's acceptance. In the new psycho-oncological care program, isPO, the initial PIM was developed top-down and required optimization. In this paper, we report on the process and experiences of optimizing PIM's quality bottom-up by applying a Participatory Health Research (PHR) approach. Cancer-patient representatives of the national peer-support group contributed as co-researchers as part of the optimization team. A mixed-methods design was chosen. First, the quality of the initially utilized PIM was assessed with the newly designed user-friendly instrument UPIM-Check. Next, three Participatory Action Research loops were conducted, including cancers survivors and isPO service providers. The initial isPO PIM's were assed to be of low quality, limited usability and incomplete. Bottom-up generated optimization suggestions led to the improvement of two initially used PIMs (leaflet, patient information folder) and the design of two new PIMs (poster, study information overview). The optimized PIM facilitates isPO service providers' care provision and helps newly diagnosed cancer patients in understanding and accepting the new program. PIM optimization benefited from applying PHR. The patient representatives' contribution and active patient engagement were central for quality assessment and designing needs-driven, mature and complete PIM.

Critical Quality and Readability Analysis of Online Patient Education Materials on Parotidectomy: A Cross-Sectional Study

PURPOSE

Complications related to parotidectomy can cause significant morbidity, and thus, the decision to pursue this surgery needs to be well-informed. Given that information available online plays a critical role in patient education, this study aimed to evaluate the readability and quality of online patient education materials (PEMs) regarding parotidectomy.

METHODS

A Google search was performed using the term "parotidectomy" and the first 10 pages of the search were analyzed. Quality and reliability of the online information was assessed using the DISCERN instrument. Flesch-Kincaid Grade Level (FKGL) and Flesch-Reading Ease Score (FRE) were used to evaluate readability.

RESULTS

Thirty-five PEMs met the inclusion criteria. The average FRE score was 59.3 and 16 (46%) of the online PEMs had FRE scores below 60 indicating that they were fairly difficult to very difficult to read. The average grade level of the PEMs was above the eighth grade when evaluated with the FKGL. The average DISCERN score was 41.7, which is indicative of fair quality. There were no significant differences between PEMs originating from medical institutions and PEMs originating from other sources in terms of quality or readability.

CONCLUSION

Online PEMs on parotidectomy may not be comprehensible to the average individual. This study highlights the need for the development of more appropriate PEMs to inform patients about parotidectomy.

Notebook of experiences: a therapeutic resource in educational audiology

PURPOSE

To translate the My Experience Book toll into Portuguese, evaluate the translation content, readability, quality, and visual identity of the material, and make it available online.

METHODS

Descriptive, cross-sectional, quantitative, and qualitative study. The procedures consisted of five stages: translation of the material; evaluation and response to questionnaire one regarding translation; determining the validity of the content; readability assessment; availability of material online and evaluation of content aimed at quality and visual identity (questionnaire two). Twenty-five professionals (audiologists and physicians) participated in the study.

RESULTS

The translation of the notebook showed validity agreement rates greater than 90%. Readability rated the material easy to read. Among the 184 invited professionals, only 25 agreed to participate, demonstrating low adherence to the study. Most respondents agreed positively about the content and consistency of the translated material, videos, illustrative images, and captions. The material was also considered necessary by the majority of the evaluators. All audiologists reported using the instrument in clinical practice.

CONCLUSION

The translated toll is of great relevance. It gathers practical information to create a notebook of experiences and guidance on using the material as an additional resource to stimulate the auditory skills of children with hearing impairment.

Readability and quality assessment of internet-based patient education materials related to deep vein thrombosis

Access to health-related internet information has increased dramatically during the past decade. A significant proportion of this information has been demonstrated to be set at too high a level for general comprehension. The goal of this study was to evaluate the readability and quality of information available on the internet relating to deep vein thrombosis (DVT). A search for 'DVT' and 'Deep Vein Thrombosis' was performed on five most commonly accessed search engines. Top hundred websites were reviewed. Website authorship was classified. Each site was assessed using the recognized quality and readability scoring systems. The presence of the health on the net foundation code of conduct (HON-code), a reported quality-assurance marker, was noted. The majority of web sites (45%) were academic or physician compiled. Only 36% of the websites were HON-code certified. HON-code certified sites achieved significantly greater DISCERN and Journal of the American Medical Association (JAMA) scores. Seventy-three percent of the websites were above the recommended sixth-grade readability level. Flesch--Kincaid readability test (FRES) of more than half of the websites (56%) was below 60, which makes them fairly difficult to read by general public. Internet information relating to DVT is of variable quality and largely set at an inappropriate readability level. Given this variability in quality, healthcare providers should direct patients to known sources of reliable, readable online information. Identification of reliable sources may be aided by known markers of quality such as HONcode certification and to educate them to use academic and physician-provided sites which have been shown to contain better-quality information. Moreover, academic and physician-compiled websites should be written in a fashion appropriate for general public consumption.

Accessibility of web-based health information for women in midlife from culturally and linguistically diverse backgrounds or with low health literacy

OBJECTIVE

To measure the accessibility of Australian web-based health information for midlife women including those from culturally and linguistically diverse (CALD) backgrounds or with low health literacy.

METHODS

Search terms relating to midlife health were entered into Google Australia to identify health information websites. The content of the first two results pages was assessed using the European Commission's quality criteria for health websites. Readability was assessed using the Flesch Readability Ease Score with Grade 8 accepted as the average Australian reading level.

RESULTS

Sixteen websites were evaluated. Accessibility scores ranged between 0 and 8. The Victorian Government's health website Better Health Channel and the Jean Hailes for Women's Health website contained the most accessible information, each scoring 8, but were both 'difficult to read' on the readability test. Four websites included written resources in languages other than English and two had information in audio-visual format in languages other than English.

CONCLUSIONS

There is a gap in accessible online health information for Australian women from CALD backgrounds or those with low health literacy. Implications for public health: Healthy behaviour changes in midlife may lead to better health in older age. More accessible health information resources are needed for women in midlife from CALD backgrounds and those with low health literacy.

Readability assessment of the British Association of Dermatologists' patient information leaflets

BACKGROUND

The British Association of Dermatologists (BAD) develops and produces patient information leaflets (PILs) for British clinicians and the general public, and its website provides access to all the PILs. Health literacy across the UK remains variable. Readability instruments assess the comprehensibility of text, predominately using a composite of sentence length and/or word-syllable number. Instruments usually report text readability categorized by United States (US) grades; ideally, health literature should be rated at US grade ≤ 6 (UK Year 7; age 11-12 years).

METHODS

In collaboration with the BAD, PILs on the BAD website (n = 203) were downloaded for readability assessment. PILs were processed prior to analysis using Readability Studio software (Oleander Software, Vandalia, OH, USA). Established readability metrics were used: Flesch-Kincaid (FK), Simple Measure of Gobbledygook (SMOG), Gunning fog index (GFI), Fry, FORCAST and Flesch Reading Ease (FRE).

RESULTS

The mean (95% CI) US grade levels for all BAD PILs were: 9.8 (9.7-10.0) for FK, 12.1 (12.0-12.3) for SMOG, 11.8 (11.6-11.9) for GFI, 11.5 (11.1-11.8) for Fry and 10.7 (10.6-10.8) for FORCAST. For FRE, the level is reported from a normal spectrum of 0-100, and was found to be 52.2 (95% CI 34.0-78.0) in this study. In the UK context, the mean readability levels of the BAD PILs were rated as Year 10 (age 14-15 years) for FK and Year 13 (aged 17-18 years) for SMOG. For FK, outputs, only 1.0% of PILs (2 of 203) were the recommended US grade ≤ 6 according to FK, and for SMOG rating, none was rated at this level.

DISCUSSION

The majority of BAD PILs have been written at a level that will be challenging for some patients to read. Reducing sentence length and aiming for shorter words will improve accessibility.

How Do African-American Community Members' Perceptions About Environmental Risks of Breast Cancer Compare with the Current State of the Science?

African-American (AA) women experience higher mortality from breast cancer than any other racial group. Understanding community-held perceptions of environmental contaminants as risk factors for breast cancer can inform the development of tailored prevention and education efforts for improve health outcomes. Six focus groups were conducted with AA participants in two counties in South Carolina, and themes were identified using open and axial coding. Perceived environmental risks for breast cancer most frequently discussed by participants were compared to findings from published systematic reviews. Frequently discussed environmental risk factors by participants were deodorants containing aluminum, plastics, pesticides, and air and water pollution. While perceptions of aluminum and air pollution as risk factors did not align with the state of the science, perceived risk factors of chemicals in plastics and pesticides were found to be in alignment. There is some congruence between perceived environmental risks for breast cancer within the AA community and the current state of the science; however, there is a need to communicate information that reflects current science regarding commonly held misconceptions. Development of evidence-based, clear, and culturally appropriate messaging that reflects the current state of the science is warranted.

Assessing the Accuracy, Quality, and Readability of Information Related to the Surgical Management of Benign Prostatic Hyperplasia

Objectives: To assess the accuracy, quality, and readability of online educational health information in English related to the most common benign prostatic hyperplasia (BPH) guideline-approved surgical treatments. Methods: The terms "benign prostatic hyperplasia," "BPH," and all eight guideline-approved treatment modalities studied, were searched to retrieve the first five relevant websites and first two paid advertised websites related to the surgical treatment options for BPH. These modalities included transurethral resection of the prostate (TURP), GreenLight photovaporization, endoscopic enucleation of the prostate, Rezum, Urolift, Aquablation, open simple prostatectomy, and robotic simple prostatectomy (RSP). All relevant websites were assessed for their accuracy, quality, and readability using standardized scoring systems. Results: The mean accuracy score for each of the treatment modalities were all indicative of good accuracy, with 76%-99% of the information presented as being accurate. The median quality score was statistically different across the eight treatment modalities (p = 0.015). The median readability grade level was statistically different across the eight treatment modalities (p = 0.009). Websites that described TURP (median readability grade level, 9.00 [interquartile range (IQR) 8.00-10.80]) were significantly easier to read than those related to RSP (median readability grade level, 14.35 [IQR, 11.08-16.50]) (p = 0.011). No other statistically significant differences were found within the other treatment modality websites. Conclusions: The majority of websites retrieved were found to be of high accuracy, good quality, and poor readability. Additionally, it was found that none of the retrieved websites included descriptions for all the other included treatment modalities. Given these findings, the authors recommend the development of centralized resources with all guideline-approved treatment modalities and accurate, readable, and high-quality information related to the surgical treatment of BPH.

Web-Based Information Seeking Behaviors of Low-Literacy Hispanic Survivors of Breast Cancer: Observational Pilot Study

BACKGROUND

Internet searching is a useful tool for seeking health information and one that can benefit low-literacy populations. However, low-literacy Hispanic survivors of breast cancer do not normally search for health information on the web. For them, the process of searching can be frustrating, as frequent mistakes while typing can result in misleading search results lists. Searches using voice (dictation) are preferred by this population; however, even if an appropriate result list is displayed, low-literacy Hispanic women may be challenged in their ability to fully understand any individual article from that list because of the complexity of the writing.

OBJECTIVE

This observational study aims to explore and describe web-based search behaviors of Hispanic survivors of breast cancer by themselves and with their caregivers, as well as to describe the challenges they face when processing health information on the web.

METHODS

We recruited 7 Hispanic female survivors of breast cancer. They had the option to bring a caregiver. Of the 7 women, 3 (43%) did, totaling 10 women. We administered the Health LiTT health literacy test, a demographic survey, and a breast cancer knowledge assessment. Next, we trained the participants to search on the web with either a keyboard or via voice. Then, they had to find information about 3 guided queries and 1 free-form query related to breast cancer. Participants were allowed to search in English or in Spanish. We video and audio recorded the computer activity of all participants and analyzed it.

RESULTS

We found web articles to be written for a grade level of 11.33 in English and 7.15 in Spanish. We also found that most participants preferred searching using voice but struggled with this modality. Pausing while searching via voice resulted in incomplete search queries, as it confused the search engine. At other times, background noises were detected and included in the search. We also found that participants formulated overly general queries to broaden the results list hoping to find more specific information. In addition, several participants considered their queries satisfied based on information from the snippets on the result lists alone. Finally, participants who spent more time reviewing articles scored higher on the health literacy test.

CONCLUSIONS

Despite the problems of searching using speech, we found a preference for this modality, which suggests a need to avoid potential errors that could appear in written queries. We also found the use of general questions to increase the chances of answers to more specific concerns. Understanding search behaviors and information evaluation strategies for low-literacy Hispanic women survivors of breast cancer is fundamental to designing useful search interfaces that yield relevant and reliable information on the web.

Readability of patient education materials related to radiation safety: What are the implications for patient-centred radiology care?

Background

Increasing numbers of patients and carers rely on online resources for healthcare information. Radiation safety can be misunderstood by patients and clinicians and lead to patient anxiety. We aimed to assess the readability of online patient educational materials (PEMs) related to radiation safety.

Methods

A total of 84 articles pertaining to radiation safety from 14 well-known online resources were identified. PEMs were then analysed using Readability Studio Professional Edition Version 2019. Readability was assessed using eight different instruments: the Flesch-Kincaid Reading Grade Level, Raygor Estimate, SMOG, Coleman–Liau, Fry, FORCAST, Gunning Fog, and Flesch Reading Ease Score formula. The mean reading grade level (RGL) of each article was compared to the 6th and 8th grade reading level using 1-sample t-tests.

Results

The cumulative mean RGL for all 84 articles was 13.3 (range = 8.6–17.4), and none were written at or below the 6th or 8th grade level. The cumulative mean RGL exceeded the 6th grade reading level by an average of 7.3 levels (95% CI, 6.8–7.8; p < 0.001) and the 8th grade level by an average of 5.3 grade levels (95% CI, 4.8–5.8; p < 0.001). The mean Flesch Reading Ease Score was 39/100 (‘difficult’).

Conclusion

Currently available online PEMs related to radiation safety are still written at higher than recommended reading levels. Radiation safety is a topic in which the specialist training of radiologists is crucial in providing guidance to patients. Addressing the readability of online PEMs can improve radiology-patient communication and support the shift to a patient-centred model of practice.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13244-021-01094-3.

Readability Assessment of HPV Vaccination and Cervical Cancer Information: A Systematic Scoping Review

It is important to provide information on HPV vaccination and on early detection and early treatment for cervical cancer. Readability is a key aspect in the success of cancer communication using written health information. We reviewed studies assessing the readability of information on HPV vaccination and cervical cancer. We conducted a systematic literature search in June 2021 using four online databases (Medline, CINAHL, PsycArticles, and PsycINFO). Studies that assessed the readability level of online and offline information regarding HPV vaccination and cervical cancer were included in the review. Twelve articles were deemed fit for study inclusion. Our results showed that most of the materials assessed were difficult to read and higher than eighth-grade reading level. Few of the materials assessed were at the recommended grade 5-6 level or below. Readability assessments of HPV vaccination and cervical cancer information are scarce. Additional studies on the readability of information regarding HPV vaccination and cervical cancer are recommended. Health professionals should develop health information on HPV vaccination and cervical cancer that is easy to read.