Readability, complexity, and suitability analysis of online lymphedema resources

A systematic review and meta-analysis of English language online patient education materials in breast cancer: Is readability the only story?

BACKGROUND

Online patient education materials (OPEMs) are an increasingly popular resource for women seeking information about breast cancer. The AMA recommends written patient material to be at or below a 6th grade level to meet the general public's health literacy. Metrics such as quality, understandability, and actionability also heavily influence the usability of health information, and thus should be evaluated alongside readability.

PURPOSE

A systematic review and meta-analysis was conducted to determine: 1) Average readability scores and reporting methodologies of breast cancer readability studies; and 2) Inclusion frequency of additional health literacy-associated metrics.

MATERIALS AND METHODS

A registered systematic review and meta-analysis was conducted in Ovid MEDLINE, Web of Science, Embase.com, CENTRAL via Ovid, and ClinicalTrials.gov in June 2022 in adherence with the PRISMA 2020 statement. Eligible studies performed readability analyses on English-language breast cancer-related OPEMs. Study characteristics, readability data, and reporting of non-readability health literacy metrics were extracted. Meta-analysis estimates were derived from generalized linear mixed modeling.

RESULTS

The meta-analysis included 30 studies yielding 4462 OPEMs. Overall, average readability was 11.81 (95% CI [11.14, 12.49]), with a significant difference (p < 0.001) when grouped by OPEM categories. Commercial organizations had the highest average readability at 12.2 [11.3,13.0]; non-profit organizations had one of the lowest at 11.3 [10.6,12.0]. Readability also varied by index, with New Fog, Lexile, and FORCAST having the lowest average scores (9.4 [8.6, 10.3], 10.4 [10.0, 10.8], and 10.7 [10.2, 11.1], respectively). Only 57% of studies calculated average readability with more than two indices. Only 60% of studies assessed other OPEM metrics associated with health literacy.

CONCLUSION

Average readability of breast cancer OPEMs is nearly double the AMA's recommended 6th grade level. Readability and other health literacy-associated metrics are inconsistently reported in the current literature. Standardization of future readability studies, with a focus on holistic evaluation of patient materials, may aid shared decision-making and be critical to increased screening rates and breast cancer awareness.

Health Literacy in Plastic Surgery: A Scoping Review

Health literacy of plastic surgery patients may affect surgical decision-making and perioperative outcomes. In addition to consulting a plastic surgeon, patients often refer to online-based resources to learn about surgical options. The aim of this scoping review was to identify evidence detailing the state of health literacy of plastic surgery patients and available resources to highlight areas of improvement for clinical practice and future research.

A Mixed-Methods Evaluation of Patient Education Materials for Colorectal Cancer

BACKGROUND

Patient education materials are created by professional organizations to inform patients about their disease and its treatment. However, it remains unclear if these materials are appropriate for patients.

OBJECTIVE

This study aims to broadly evaluate the education materials for patients with colorectal cancer.

DESIGN

Patient education materials from the National Comprehensive Cancer Network, the National Cancer Institute, and the American Society of Colon and Rectal Surgeons were assessed quantitatively by using 1) the Flesch-Kincaid readability formula and 2) the Patient Education Material Assessment Tool. The Patient Education Material Assessment Tool scores materials in 2 domains: understandability and actionability. These materials were further evaluated qualitatively via an exploratory focus group with patients and their caregivers (n = 5) and semi-structured interviews with board-certified/eligible colorectal surgeons (n = 10).

SETTING

This study was conducted at academic centers and a regional professional society meeting.

PARTICIPANTS

The mean patient age was 63. Most surgeons (8/10) practiced in an academic setting, and 4/10 were female.

MAIN OUTCOME MEASURES

The primary outcomes measured were reading grade level and domain scores for the Patient Education Material Assessment Tool. Qualitative data were recorded, transcribed, and coded. Themes were generated through data interpretation and data reduction.

RESULTS

Materials ranged from 7th to 11th grade reading level. National Comprehensive Cancer Network materials scored highest for understandability (92.2% ± 6.1%, mean ± SD), followed by National Cancer Institute (84.0% ± 6.6%) and American Society of Colon and Rectal Surgeons (82.2% ± 6.3%) materials. Actionability scores varied; the National Comprehensive Cancer Network materials scored 82.5% ± 1.7%, whereas the National Cancer Institute and American Society of Colon and Rectal Surgeons materials scored 23.3% ± 6.7% and 50.0% ± 8.2%. Critical gaps were identified in the content of these materials. Patients wanted more information about self-care, both emotional and physical. Specifically, patients sought details about postoperative bowel function. Whereas surgeons wanted information about the typical hospital course and recovery, all wanted materials to be customizable.

LIMITATIONS

A limited number of materials were reviewed, and patient focus groups were exploratory.

CONCLUSIONS

Commonly available printed education materials for colorectal cancer are written at a high reading grade level, vary in their usability, and neglect important details about postoperative recovery. See Video Abstract at http://links.lww.com/DCR/B535.

EVALUACIN DE MTODOS MIXTOS DE MATERIALES EDUCATIVOS PARA PACIENTES SOBRE CNCER COLORECTAL

ANTECEDENTES:Los materiales educativos para pacientes son creados por organizaciones profesionales para informar a los pacientes sobre su enfermedad y su tratamiento. Sin embargo, no está claro si estos materiales son apropiados para los pacientes.OBJETIVO:Evaluar ampliamente los materiales para el cáncer colorrectal.DISEÑO:Los materiales educativos para pacientes de la Red Nacional Integral del Cáncer (NCCN), el Instituto Nacional del Cáncer (NCI) y la Sociedad Americana de Cirujanos de Colon y Recto (ASCRS) se evaluaron cuantitativamente utilizando (1) la fórmula de legibilidad de Flesch-Kincaid y (2) la herramienta de evaluación de material educativo para pacientes. La Herramienta de evaluación de materiales educativos para pacientes califica los materiales en dos dominios: comprensibilidad y viabilidad. Estos materiales fueron evaluados cualitativamente a través de un grupo de enfoque exploratorio con pacientes y sus cuidadores (n = 5) y entrevistas semiestructuradas con cirujanos colorrectales certificados o elegibles para certificación por el consejo (n = 10).ESCENARIO:Centros académicos y un encuentro regional de una sociedad profesional.PACIENTES:La edad media de los pacientes fue de 63 años. La mayoría de los cirujanos (8/10) practicaban en un entorno académico, y 4/10 eran mujeres.PRINCIPALES MEDIDAS DE RESULTADO:Nivel de grado de lectura y puntajes de dominios para la Herramienta de evaluación de materiales educativos para pacientes. Los datos cualitativos se registraron, transcribieron y codificaron. Los temas se generaron mediante la interpretación y la reducción de datos.RESULTADOS:Los materiales variaron desde el nivel de lectura del 7° al 11° grado. Los materiales de la NCCN obtuvieron la puntuación más alta en comprensibilidad (92.2 ± 6.1%, media ± DE), seguidos por los materiales de NCI (84.0 ± 6.6%) y ASCRS (82.2 ± 6.3%). Los puntajes de viabilidad variaron; Los materiales de NCCN obtuvieron una puntuación de 82.5 ± 1.7%, mientras que los materiales de NCI y ASCRS obtuvieron una puntuación de 23.3 ± 6.7% y 50.0 ± 8.2%, respectivamente. Se identificaron lagunas críticas en el contenido de estos materiales. Los pacientes querían más información sobre el autocuidado, tanto emocional como físico. Específicamente, los pacientes buscaron detalles sobre la función intestinal posoperatoria. Mientras que los cirujanos querían información sobre el curso hospitalario típico y la recuperación, y todos querían que los materiales fueran personalizables.LIMITACIONES:Se revisó una cantidad limitada de materiales y los grupos de enfoque de pacientes fueron exploratorios.CONCLUSIONES:Los materiales educativos impresos comúnmente disponibles para el cáncer colorrectal están escritos a un alto nivel de grado de lectura, varían en su usabilidad y omiten detalles importantes sobre la recuperación postoperatoria. Consulte Video Resumen en http://links.lww.com/DCR/B535.

A Multimetric Health Literacy Analysis of Online Information for Gluteal Augmentation With Fat Grafting

BACKGROUND AND OBJECTIVES

According to the American Society of Plastic Surgeons the number of gluteal augmentations with fat grafting, colloquially known as the "Brazilian buttock lift" (BBL), doubled from 2014 to 2018. Recent studies have demonstrated that gluteal augmentation with fat grafting has a higher mortality rate when compared with other esthetic surgeries. We hypothesize that online patient-directed information for gluteal augmentation are overly complex, written at an elevated reading grade level, with incomplete or no information regarding associated surgical risks.

OBJECTIVE

To perform an objective appraisal of the most popular online patient education materials for gluteal augmentation surgery and explore potential interventions for improving material quality.

METHODS

An online Google search using the term "Brazilian butt lift" was performed and the top 10 websites providing relevant information were identified. Readability for each website was determined using Readability Studio. Understandability, actionability and cultural sensitivity was assessed with validated tools by 2 independent raters.

RESULTS

The top 10 websites were written at a mean reading grade level of 13.0, appropriate for a high school graduate. No website was written at the American Medical Association and the National Institutes of Health recommended 6th to 8th grade reading level. Mean understandability and actionability scores of 51% and 18.9%, respectively, suggest significant room for improvement. The average cultural sensitivity assessment tool score was 2.30, below the recommended threshold for acceptability of 2.5. Ninety percent of all websites did not meet the threshold for cultural acceptability.

CONCLUSIONS

Online patient education materials for gluteal augmentation with fat grafting were written at an inappropriately elevated reading grade level and were overly complex, did not promote reader action, and were not sensitive to the needs of the target demographic.

An assessment of the readability of online material related to fluoride

Community water fluoridation has been deemed one of the greatest public health achievements, yet it remains a controversial topic. As those who are concerned about the benefits and safety of community water fluoridation are likely to turn to the Internet to find information, it is important to know the ease with which a person can read popular materials online. The aim of this study was to assess the readability levels of pro- and anti-fluoride articles on the Internet. Using a highly-recommended readability site, the first 100 articles garnered from websites in a Google were analyzed for a multitude of scores on popular readability tests. Of the 100 articles analyzed, 71 were deemed to be pro-fluoride, and 29 were anti-fluoride. Anti-fluoride materials were more readable across all scores when compared to the pro-fluoride materials. It is critical for consumers to be aware of the credibility of the health information they are reading and to be educated on how to determine if a website or source is reliable, especially when information is conflicting. It is also critical that health associations and other websites devoted to public health promotion develop and provide more accessible health information in terms of readability.

Evaluating the Quality of Health-Related WeChat Public Accounts: Cross-Sectional Study

Background

As representatives of health information communication platforms accessed through mobile phones and mobile terminals, health-related WeChat public accounts (HWPAs) have a large consumer base in the Chinese-speaking world. However, there is still a lack of general understanding of the status quo of HWPAs and the quality of the articles they release.

Objective

The aims of this study were to assess the conformity of HWPAs to the Health on the Net Foundation Code of Conduct (HONcode) and to evaluate the suitability of articles disseminated by HWPAs.

Methods

The survey was conducted from April 23 to May 5, 2019. Based on the monthly (March 1-31, 2019) WeChat Index provided by Qingbo Big Data, the top 100 HWPAs were examined to evaluate their HONcode compliance. The first four articles published by each HWPA on the survey dates were selected as samples to evaluate their suitability. All materials were assessed by three raters. The materials were assessed using the HONcode checklist and the Suitability Assessment of Materials (SAM) score sheet. Data analysis was performed with SPSS version 17.0 (SPSS Inc, Chicago, IL, USA) and Excel version 2013 (Microsoft Inc, Washington DC, USA).

Results

A total of 93 HWPAs and 210 of their released articles were included in this study. For six of the eight principles, the 93 HWPAs nearly consistently did not meet the requirements of the HONcode. The HWPAs certified by Tencent Corporation (66/93, 71%) were generally slightly superior to those without such certification (27/93, 29%) in terms of compliance with HONcode principles. The mean SAM score for the 210 articles was 67.72 (SD 10.930), which indicated “adequate” suitability. There was no significant difference between the SAM scores of the articles published by certified and uncertified HWPAs (P=.07), except in the literacy requirements dimension (t_df=97=–2.418, P=.02).

Conclusions

The HWPAs had low HONcode conformity. Although the suitability of health information released by HWPAs was at a moderate level, there were still problems identified, such as difficulty in tracing information sources, excessive implicit advertisements, and irregular usage of charts. In addition, the low approval requirements of HWPAs were not conducive to improvement of their service quality.

A Multimetric Health Literacy Analysis of Autologous Versus Implant-Based Breast Reconstruction

BACKGROUND

Over the past decade, the demand for breast reconstruction has mirrored the rising incidence of breast cancer. Common postoncologic surgical options include autologous and implant-based reconstruction. Patient-directed health information for breast reconstruction can play a critical role in the decision-making process. This study comparatively evaluates the top online resources for autologous versus implant-based reconstruction using a multimetric health literacy analysis.

METHODS

The top 10 websites for autologous and implant-based reconstruction were identified using a Google search. A total of 20 unique links were appraised by 2 independent raters for understandability and actionability using the Patient Education Materials Assessment Tool and cultural sensitivity using the Cultural Sensitivity Assessment Tool. A Cohen κ for interrater reliability was calculated. Mean reading grade level and word complexity were also determined.

RESULTS

Websites for both autologous and implant-based modalities exceeded the recommended sixth- to eighth-grade reading level (12.4 and 12.1, respectively; P = 0.65). Mean understandability scores for each modality were low (60.5 and 62.5, P = 0.65). Autologous-based resources had a lower mean actionability score compared with implant-based materials (19.5 and 24, respectively; P = 0.04). Both reconstructive modalities met the threshold for acceptability for cultural sensitivity (2.79 and 2.58, P = 0.09).

CONCLUSIONS

Our study revealed a chasm between the health literacy needs of the average adult and the quality of both implant-based and autologous breast reconstruction resources. Materials for both modalities were often too complex and failed to include tools to facilitate active decision making, particularly for autologous-based reconstruction. Strategies to improve materials should be patient centered and include simplification of reading grade level, incorporation of clear visual aids, and inclusion of procedural risks to promote patient comprehension, participation, and ultimately health outcomes.

Trustworthiness, Readability, and Suitability of Web-Based Information for Stroke Prevention and Self-Management for Korean Americans: Critical Evaluation

Background

Websites are common sources of health information to stroke survivors and caregivers for continual management of stroke and its long-term sequelae. The presence of risk factors and mortality rates related to stroke are high in Korean Americans. A vast majority of this group are active Web users and rely on the Web-based information due to lack of insurance and, thus, limited access to long-term stroke care. Thus, it is critical to evaluate existing stroke websites for their trustworthiness, readability, and suitability.

Objective

The objective of our study was to provide a systematic evaluation of stroke-related websites regarding (1) trustworthiness, (2) readability, and (3) suitability for stroke prevention and self-management for Korean Americans.

Methods

We selected a total of 156 websites using search terms “stroke,” “CVA,” “중풍 (jungpung),” and “뇌졸증 (noejoljung)” on Google and Yahoo. After eliminating duplicates and irrelevant websites (n=116), we evaluated a total of 42 websites (15 in English and 27 in Korean) using the National Library of Medicine’s health website’s evaluation tool for trustworthiness; Simple Measure of Gobbledygook for readability; and Suitability Assessment of Materials for suitability. All three instruments used the 3-point Likert scale: superior (=2), adequate (=1), or not suitable (=0).

Results

Of the 42 websites evaluated, we rated 62% (26/42) websites as “adequate” or above for trustworthiness. The information on 48% (20/42) websites had not been updated for more than a year, which indicates poor currency; 33% (14/42) websites failed to provide the publisher and contact information, which yields poor authority; 50% (21/42) websites did not cite sources of health information, which indicates lack of accuracy. Only 2 websites met the recommended readability (5th grade or lower reading level). The suitability was also suboptimal; only 1 website was rated as “superior”; 60% (25/42) websites were “adequate,” and 38% (16/42) were “not suitable.” Most websites were limited in graphical directions, interactive motivations for desired healthy behaviors, and multiple language translations.

Conclusions

The existing stroke-related websites in either English or Korean are trustworthy and suitable, yet precise citation of evidence-based information will improve trustworthiness. The contents requiring high reading level may set a barrier to the utilization of Web-based health information for Korean Americans with a lower level of education. In addition, supplementing graphical examples, interaction features, and culturally relevant information in multiple languages are the areas for improvement in suitability. The improved features can reduce the reading burden of stroke patients or caregivers and build their confidence when applying the information for stroke management in daily living. These strategies are especially crucial to Korean Americans, who inevitably seek Web-based information to fill the gap between their demand and access to health care for a long-term self-management after a stroke.