Readability of online patient resources for the operative treatment of breast cancer

A systematic review and meta-analysis of English language online patient education materials in breast cancer: Is readability the only story?

BACKGROUND

Online patient education materials (OPEMs) are an increasingly popular resource for women seeking information about breast cancer. The AMA recommends written patient material to be at or below a 6th grade level to meet the general public's health literacy. Metrics such as quality, understandability, and actionability also heavily influence the usability of health information, and thus should be evaluated alongside readability.

PURPOSE

A systematic review and meta-analysis was conducted to determine: 1) Average readability scores and reporting methodologies of breast cancer readability studies; and 2) Inclusion frequency of additional health literacy-associated metrics.

MATERIALS AND METHODS

A registered systematic review and meta-analysis was conducted in Ovid MEDLINE, Web of Science, Embase.com, CENTRAL via Ovid, and ClinicalTrials.gov in June 2022 in adherence with the PRISMA 2020 statement. Eligible studies performed readability analyses on English-language breast cancer-related OPEMs. Study characteristics, readability data, and reporting of non-readability health literacy metrics were extracted. Meta-analysis estimates were derived from generalized linear mixed modeling.

RESULTS

The meta-analysis included 30 studies yielding 4462 OPEMs. Overall, average readability was 11.81 (95% CI [11.14, 12.49]), with a significant difference (p < 0.001) when grouped by OPEM categories. Commercial organizations had the highest average readability at 12.2 [11.3,13.0]; non-profit organizations had one of the lowest at 11.3 [10.6,12.0]. Readability also varied by index, with New Fog, Lexile, and FORCAST having the lowest average scores (9.4 [8.6, 10.3], 10.4 [10.0, 10.8], and 10.7 [10.2, 11.1], respectively). Only 57% of studies calculated average readability with more than two indices. Only 60% of studies assessed other OPEM metrics associated with health literacy.

CONCLUSION

Average readability of breast cancer OPEMs is nearly double the AMA's recommended 6th grade level. Readability and other health literacy-associated metrics are inconsistently reported in the current literature. Standardization of future readability studies, with a focus on holistic evaluation of patient materials, may aid shared decision-making and be critical to increased screening rates and breast cancer awareness.

Digital Trends, Digital Literacy, and E-Health Engagement Predictors of Breast and Colorectal Cancer Survivors: A Population-Based Cross-Sectional Survey

Introduction: Advances in information and communication technology (ICT) and post-COVID-19 tectonic changes in healthcare delivery have made it possible for cancer survivors to obtain disease-related information for remote management online rather than through healthcare providers. To comprehend and evaluate health information, digital literacy is crucial. Objectives: This study examined cancer survivors’ information-seeking behaviour, information sources, digital health literacy, and digital trends, as well as potential determinants of e-health information receptivity and online resource use. Methods: A national 30-item cross-sectional survey using a representative random sample of cancer survivors from Jordan’s cancer registry was conducted. Chi-square tests established categorical variable relationships. Using the mean and standard deviation, we calculated the Likert scale’s ordinal data average. A p-value < 0.05 was statistically significant. Logistic regression identified predictors of interest in late-trajectory information acquisition and use of e-health platforms (apps, portals) for cancer self-management. Results: Lower digital literacy and electronic searching were associated with older age and lower income, education, and employment status (p ≤ 0.001). Digital literacy independently predicted m-health app use for remote management and interest in cancer supportive care information. Digitally literate survivors preferred the use of digital platforms (p ≤ 0.001). Information acquisition barriers included “reliability” (26%, n = 25) and “health information trustworthiness” (16.2%, n = 25). Following treatment completion, Internet-seeking behaviour decreased significantly when compared to the early cancer trajectory. Conclusion: Our findings imply that Jordanian cancer survivors’ low digital literacy may hinder information acquisition and technology-enabled cancer care. Digital interventions for cancer survivors should be adaptable to varying levels of digital health literacy. Healthcare policymakers should recognise digital inequities and devise focused initiatives to bridge the digital divide while responding to the urgent need to digitalise cancer care delivery.

One Size Does Not Fit All; Patient Preference for Breast Reconstruction

BACKGROUND

A myriad of patient education modalities for breast reconstruction exist, although the optimal tools for patient education remain undetermined. The aim of this study is to determine patient preferences for breast reconstruction education modalities based on demographic variables.

METHODS

A prospective observational study at a tertiary care university health system was conducted between November 2020 and May 2021. A questionnaire was administered to breast reconstruction patients to collect information on demographics, research sources used before the initial appointment, and preferred education modalities. Differences based on age were analyzed using an independent samples t test, whereas a Fisher exact test was used to analyze differences based on ethnicity and education level. Statistical significance was defined as P < .05.

RESULTS

The most preferred patient education tools overall were books/written materials and videos. Younger patients were significantly more likely than older patients to have referenced additional physician sources (P = .0174) and to seek out information on the institution's website (P = .0465). Those with a college degree were significantly more likely to have performed research prior to the initial appointment (P = .0206). White patients were significantly more likely than nonwhite patients to talk to friends/family as a research source (P = .0150).

CONCLUSIONS

Regardless of age, education, or ethnicity, most patients prefer books/written materials and video presentations for education on breast reconstruction. Providers should strive to include written and video options to meet the needs of this diverse patient population.

Accuracy of Online Pictorial Resources for Common Hand Surgery Procedures

Background: Online medical platforms can provide patients with easily accessible information and greater opportunities to self-advocate. However, the lack of quality control and presence of inaccurate information can lead to miscommunications between the physician and the patient. The objective of this study was to examine the quality and accuracy of online pictorial information regarding common hand conditions. Methods: Medical image information was searched on the search engine Google (http://www.google.com) using the terms "de Quervain's tenosynovitis," "carpal tunnel syndrome," and "trigger finger." The first 20 illustrations to appear on the search were recorded along with the type of source. The images were then examined for veracity of information conveyed. Results: Sixty images were collected from 48 different Web sources. Nonacademic/private medical institutions were the most common image source (35%). The rate of erroneous images was 40%. Web sites of academic hospitals were most frequently the source of incorrect images. Of the conditions, trigger finger had the highest rate of errors (55%), most commonly occurring in positioning of the annular pulleys. The search results did not contain any Web sites from professional hand societies. Conclusion: The quality of online medical information is a significant but often overlooked aspect of health professional and patient education. This study demonstrates the prevalence of incorrect information online and the misunderstandings that patients can have about common conditions. Higher quality online resources are needed to improve patient education and patient-physician interactions. Avenues for improvement are to provide greater accessibility of educational resources offered by professional hand surgery organizations.

Are English-language online patient education materials related to breast cancer risk assessment understandable, readable, and actionable?

PURPOSE

To evaluate the readability, understandability, and actionability of online patient education materials (OPEM) related to breast cancer risk assessment.

MATERIAL AND METHODS

We queried seven English-language search terms related to breast cancer risk assessment: breast cancer high-risk, breast cancer risk factors, breast cancer family history, BRCA, breast cancer risk assessment, Tyrer-Cuzick, and Gail model. Websites were categorized as: academic/hospital-based, commercial, government, non-profit or academic based on the organization hosting the site. Grade-level readability of qualifying websites and categories was determined using readability metrics and generalized estimating equations based on written content only. Readability scores were compared to the recommended parameters set by the American Medical Association (AMA). Understandability and actionability of OPEM related to breast cancer high-risk were evaluated using the Patient Education Materials Assessment Tool (PEMAT) and compared to criteria set at ≥70%. Descriptive statistics and inter-rater reliability analysis were utilized.

RESULTS

343 websites were identified, of which 162 met study inclusion criteria. The average grade readability score was 12.1 across all websites (range 10.8-13.4). No website met the AMA recommendation. Commercial websites demonstrated the highest overall average readability of 13.1. Of the 26 websites related to the search term breast cancer high-risk, the average understandability and actionability scores were 62% and 34% respectively, both below criteria.

CONCLUSIONS

OPEM on breast cancer risk assessment available to the general public do not meet criteria for readability, understandability, or actionability. To ensure patient comprehension of medical information online, future information should be published in simpler, more appropriate terms.

The quality, suitability, content and readability of online health-related information regarding sexual dysfunction after rectal cancer surgery

AIM

Patients are not well informed about sexual dysfunction after rectal cancer surgery and often turn to the Internet for information. The purpose of this study was to assess online information for patients on sexual dysfunction after rectal cancer surgery.

METHODS

An online search of Google, Yahoo and Bing was performed using specific (e.g., rectal cancer surgery and vaginal pain) and general (e.g., rectal cancer surgery and sex) search terms. Inclusion criteria were websites in English, designed for patients, and including content regarding sexual dysfunction after rectal cancer surgery. Websites were assessed for readability (nine standardized tests), quality (DISCERN tool), suitability (Suitability Assessment of Materials tool), and content.

RESULTS

Of 5040 websites identified, 99 unique websites met inclusion criteria. Three (3%) websites fulfilled the American Medical Association recommendation of a 6th-grade reading level. Using the DISCERN instrument, two (2%) websites were assigned good/excellent quality, nineteen (19%) referenced their sources of information, and thirty-one (31%) fully discussed the impact of sexual dysfunction on quality of life. Using the SAM instrument, three (3%) websites were classified as highly suitable for rectal cancer patients, sixty-five (66%) were adequate, and thirty-one (31%) were inadequate. With regards to content, nine (9%) websites fully discussed the impact of sexual dysfunction on patients partners and fifty-one (52%) websites did not cover prognosis.

CONCLUSION

Online health information available to patients on sexual dysfunction after rectal cancer surgery is suboptimal. Websites are not suitable, lack important content, and are written at too complex a reading level for patients.

An Evaluation of the Influence of Web-Based Patient Education on the Anxiety and Life Quality of Patients Who Have Undergone Mammaplasty: a Randomized Controlled Study

The aim of this study is to evaluate the influence of web-based education on the anxiety and quality of life of patients who were hospitalized and underwent breast surgery (modified radical mastectomy or breast-conserving surgery) with axilla lymph node dissection. The patients were divided into three groups by the block randomization method as follows: web-based education group, brochure group, and control group (total N = 75). To obtain the study data, the Risk Factors for Breast Cancer and Data Collection Form for the Disease, SF 36 Quality of Life Scale, State-Trait Anxiety Inventory, and Website Usability Scale were used. The learning content was patient education associated with the pre-operative and post-operative periods. The differences in the state of anxiety scores 1 day before surgery, the 2nd day after surgery, and 1 month after surgery were statistically lower in the web-based education group than in the other two groups. The mean difference in the trait anxiety scores after 1 month was higher in the control group than in the other two groups. Web-based patient education was identified as a more effective method than the brochure and control groups in terms of patients' physical and emotional well-being, vitality/fatigue, and role limitations emotional and general health perception. Web-based patient education is effective in decreasing the anxiety of patients and improving their quality of life.

The readability of online health resources for phenylketonuria

Phenylketonuria (PKU) is a condition that results in the build-up of phenylalanine in the blood. This can cause severe brain damage and neurological issues if left untreated. Management can be complex and many individuals may turn to the internet to access further information. It is important that resources are understood as misinterpretation could result in harm to health. The aim of this study was to assess the readability of online resources for PKU and to assess their visual appearance using a communication sciences assessment framework. We searched the top five websites through Google using the search term “phenylketonuria/PKU”. We then analysed the text content of the identified websites using five readability formulae to determine the USA and UK reading grade. The median readability level across the five websites was US grade/UK grade 10.6/11.6, with individual grades ranging from 10/11 to 13.3/14.3. We found wide differences in the focus, layout and general appearance of the websites. The readability of resources was much higher than the recommended US 6th grade level. Online resources for PKU need to be simplified to ensure they can be easily understood.

Critical Assessment of Surveys in Plastic and Reconstructive Surgery: A Systematic Review

BACKGROUND

Surveys are a fundamental research tool used in the field of plastic surgery. Although inherent biases within surveys have been recognized, the extent to which this problem exists in the plastic surgery literature remains unknown. The authors aim to (1) determine the extent to which common survey biases occur in plastic surgery surveys and (2) identify areas of improvement for survey researchers.

METHODS

The authors conducted a systematic review of surveys in the plastic surgery literature from 1997 to 2017. After applying their inclusion and exclusion criteria, two reviewers examined a total of 199 studies containing a survey. A modified checklist based on the published literature was used to determine the presence, type, and severity of biases in plastic surgery surveys.

RESULTS

The mean percentage of biased questions in each survey was 24 percent. The most common biases were moderate- (12 percent) and low-impact biases (10 percent). Specifically, the most common biases were forced choice questions (6 percent), horizontal responses (6 percent), and double-barreled questions (1 percent). Only 13 percent of the articles that fit the authors' inclusion criteria contained a full-length survey.

CONCLUSIONS

Although surveys in plastic surgery do not contain high-impact biases, the authors have identified areas of improvement for surveys used in this specialty. The authors found that the surveys contain biases in question structure and the survey design, which can be improved through minor revisions to the question stem or responses. Moreover, journals should require researchers to submit a full-text survey for research transparency.

The impact of cancer information online on cancer fatalism: education and eHealth literacy as moderators

One critical yet understudied concept associated with cancer information is cancer fatalism, i.e. deterministic thoughts about the cause of cancer, the inability to prevent it and the unavoidability of death upon diagnosis. The aim of this study is to understand how information seeking about cancer online influences cancer fatalism and whether and to what extent education and eHealth literacy moderate the relationship between them. Findings from an online survey of a nationally representative sample in the United States (N = 578) showed differential impacts of using the internet to search for information about cancer among the more and the less educated. For the less educated, more exposure to information about cancer via medical and health websites led to an increased level of cancer fatalism, whereas among the more educated, greater exposure lowered cancer fatalism. These differences were explained by the fact that the more educated were equipped with a higher level of eHealth literacy skills than the less educated. Findings show that only when one has necessary skills to apply digital resources can those resources help mitigate cancer fatalism. We suggest the need to enhance eHealth literacy skills among the less educated to reduce cancer fatalism.

Readability of online materials for Dupuytren's contracture

STUDY DESIGN

Descriptive.

BACKGROUND

Dupuytren's contracture is a common disorder involving fibrosis of the palmar fascia. As patients are increasingly using online materials to gather health care information, it is imperative to assess the readability and appropriateness of this content. The recommended grade level for patient educational materials is seventh to eighth grade according to the National Institutes of Health. This study aims to assess the readability and content of online patient resources for Dupuytren's contracture.

PURPOSE OF THE STUDY

Evaluate readability of online patient education materials for Dupuytren's contracture.

METHODS

The largest public search engine, Google, was queried using the term "Dupuytren's contracture surgery" on February 26, 2016. Location filters were disabled, and sponsored results were excluded to avoid any inadvertent search bias. The 10 most popular Web sites were identified, and all relevant patient-directed information within 1 click from the original site was downloaded and saved as plain text. Readability was analyzed using 6 established analyses (Readable.io, Added Bytes, Ltd, UK).

RESULTS

Analysis of 10 Web sites demonstrates an average grade level of at least 11th grade (Flesch-Kincaid grade level, 10.2; Gunning-Fog grade level, 13.1; Coleman-Liau grade level, 14.4; Simple Measure of Gobbledygook grade level, 10.0; automated readability grade level, 9.7; and average grade level, 11.5). Overall Flesch-Kincaid reading ease index was 46.4, which is difficult. No single article was at the recommended reading level.

CONCLUSIONS

Online materials available for treatment of Dupuytren's contracture are above recommended reading levels and do not include a comprehensive explanation of treatment options, which may negatively impact decision making in patients seeking treatment for this condition. Surgeons and hand therapists alike should be cognizant of available online patient materials and make efforts to develop and provide more appropriate materials.

LEVEL OF EVIDENCE

V.

Readability and Understandability Analysis of Online Materials Related to Abdominal Aortic Aneurysm Repair

Background:

Patients commonly use online materials as a source of health information. Since poor health literacy has been shown to correlate with negative outcomes, it is recommended that patient-directed materials be written at a sixth-grade reading level. This study evaluates the readability and understandability of commonly accessed online materials pertaining to both endovascular and open repair of abdominal aortic aneurysm.

Methods:

Searches for “endovascular repair abdominal aortic aneurysm” and “open repair abdominal aortic aneurysm” were performed on both Google and Bing, and the top 10 websites from each search engine were identified. Relevant websites (total N = 28, endovascular n = 15, open n = 15, and 2 redundant sites) with patient-directed content were analyzed. Readability was assessed using 9 established methods, and understandability was assessed using the Patient Education Materials Assessment Tool scoring system.

Results:

The average reading grade level for all sites was 12.8. Endovascular sites averaged a reading grade level of 13.6 with a range from 11.5 to 15.6. Open-repair websites had a grade-level average of 12.1 with a range from 9.9 to 14.1. Readability was found to be inversely related to understandability, with a Pearson correlation coefficient of −0.551 ( P = .003). No website was written at or below the recommended sixth-grade reading level.

Conclusions:

Patient-directed online health information pertaining to open and endovascular repair of abdominal aortic aneurysm exceeds the recommended sixth-grade reading level. Increasing complexity of health literature correlates with poor understandability. Modifications such as shorter sentences, fewer words with more than 6 letters, and increasing usage of clear visual aids can increase readability and understandability.

Evaluation of Content on Commonly Used Web sites About Induction of Labor and Pain Management During Labor

PURPOSE

The Internet is a common source of information for women during pregnancy. A multitude of data about birth can be found on the Internet, yet there are few guidelines to direct women to credible Web sites. Providers are often unaware of the health sites their patients access. The purpose of this study was to evaluate the quality of the written content on Web sites about induction of labor and pain management during labor.

STUDY DESIGN AND METHODS

Descriptive study of 11 Web sites on induction of labor and pain management during labor. Searches on Google, Yahoo, and Bing were used to identify Web sites women use to seek information about labor and birth. The Web sites were evaluated using the Health Information Technology Institute (HITI) criteria, readability, and content criteria from UpToDate, an evidence-based, healthcare professional-authored clinical resource.

RESULTS

No Web sites met all the HITI criteria for both topics. One Web site was written at or below a seventh-grade reading level and six were written at or above a ninth-grade level. Although there was no perfect Web site, we noted government Web sites provided the highest quality information.

CLINICAL IMPLICATIONS

Evaluation and discussion of Web sites used by pregnant women is needed. Nurses and other healthcare providers should be aware of what information exists and what Web sites women are using so they can guide women to credible, evidence-based data. Directing women to government-sponsored Web sites on childbirth topics may help them access accurate information.

A Comprehensive Analysis of How Environmental Risks of Breast Cancer are Portrayed on the Internet

BACKGROUND

Effective online communication about the environmental risk factors of breast cancer is essential because of the multitude of environmental exposures and debate regarding the conclusiveness of scientific evidence.

PURPOSE

The aim of this study was to assess the content, readability, and cultural sensitivity of online resources focused on the environmental risks factors of breast cancer.

METHODS

A purposive sample of webpages focused on environmental risk factors of breast cancer was obtained through a Google search using 17 search terms. Using nonparametric statistics, we assessed the content, readability, and cultural appropriateness of 235 webpages.

RESULTS

Eighty-two percent of webpages referred to research studies in their content. For the majority of sites, readability was at a high-school reading grade level. Webpages were not explicitly intended for specific racial/ethnic groups.

DISCUSSION

Technical language and non-culturally specific messages may hinder users' attention to and comprehension of online breast cancer information. Additional research is needed to examine in-depth the accuracy of this online content.

TRANSLATION TO HEALTH EDUCATION PRACTICE

Findings suggest that collaborations between scientists, health educators, website designers/media professionals, and the community will be critical to the delivery of accurate, up-to-date, plain-language, and culturally sensitive information about breast cancer and the environment.

Evaluation of Information Sources in Plastic Surgery Decision-making

Background

Today, patients can access a myriad of information sources regarding plastic surgery procedures prior to meeting with a surgeon. Despite their widespread use, the role of these sources in a patient’s decision-making remains undefined. We hypothesized that the physician remains the key information source for patients making surgical decisions in plastic surgery, but that other sources may deliver important insights and prove helpful to varying degrees. We also explored motivations for this outside information search and any differences in perceived value among patients.

Methods

We administered a survey regarding various information sources to our breast reconstruction, reduction, and abdominoplasty patients. Responses were compared between surgery groups and demographic groups. Ordinal logistic regression analysis was used to determine the impact of patient characteristics on helpfulness rank of different sources.

Results

Survey results were obtained from 58 patients, of whom 10 (17.2%) had abdominoplasty, 35 (60.3%) breast reconstruction, and 13 (22.4%) breast reduction. The most popular information sources prior to the first surgical appointment were Internet searches (56.9%) and family/friends/other patients (39.7%). After the initial appointment, the most useful sources were plastic surgeons (84.5%), and the Internet (36.2%). Most patients (73.5%) still sought outside information after their appointment.

On a Likert-type scale of helpfulness, plastic surgeons ranked 4.28/5, followed by the web-based patient education platform, 3.73 and the Internet, 3.6. A total of 63% of participants listed plastic surgeons as their single most important source of information.

In ordinal logistic regression analysis, non-white race was significantly associated with higher rank of surgeon helpfulness (p < 0.05). Relative to low-income patients, income $50-100k (p < 0.05) and $100k+ (p < 0.05) were associated with lower rank of surgeon helpfulness.

Conclusions

Most patients seek outside information prior to visiting with a surgeon from the Internet, social media, or family and friends. Patients consider plastic surgeons their most valuable information source overall, though still in need of supplementation for varying reasons. Additionally, certain demographic differences affect patient perception of information sources, and this is an important factor for surgeons to consider as they approach educating patients.

The accessibility, readability, and quality of online resources for gender affirming surgery

BACKGROUND

The transgender population is disproportionally affected by health disparities related to access to care. In many communities, transgender specialists are geographically distant and locally available medical professionals may be unfamiliar with unique needs of transgender patients. As a result, use of Internet resources for information about gender affirming surgery is particularly important. This study simulates a patient search for online educational material about gender affirming surgery and evaluates the accessibility, readability, and quality of the information.

METHODS

An Internet search for the term "transgender surgery" was performed, and the first 10 relevant hits were identified. Readability was assessed using 10 established tests: Coleman-Liau, Flesch-Kincaid, FORCAST, Fry, Gunning Fog, New Dale-Chall, New Fog Count, Raygor Estimate, Simple Measure of Gobbledygook, and Flesch Reading Ease. Quality was assessed using Journal of the American Medical Association criteria and the DISCERN instrument.

RESULTS

Review of 69 results was required to identify 10 sites with relevant patient information. There were 97 articles collected; overall mean reading level was 14.7. Individual Web site reading levels ranged from 12.0 to 17.5. All articles and Web sites exceeded the recommended sixth grade level. Quality ranged from 0 to 4 (Journal of the American Medical Association) and 35 to 79 (DISCERN) across Web sites.

CONCLUSIONS

Web sites with relevant patient information about gender affirming surgery were difficult to identify from search results. The content of these sites universally exceeded the recommended reading level. A wide range of Web site quality was noted, and this may further complicate successful navigation. Barriers in access to appropriately written patient information on the Internet may contribute to disparities in referral, involvement, satisfaction, and outcomes.

Online nutrition information for pregnant women: a content analysis

Pregnant women actively seek health information online, including nutrition and food-related topics. However, the accuracy and readability of this information have not been evaluated. The aim of this study was to describe and evaluate pregnancy-related food and nutrition information available online. Four search engines were used to search for pregnancy-related nutrition web pages. Content analysis of web pages was performed. Web pages were assessed against the 2013 Australian Dietary Guidelines to assess accuracy. Flesch-Kincaid (F-K), Simple Measure of Gobbledygook (SMOG), Gunning Fog Index (FOG) and Flesch reading ease (FRE) formulas were used to assess readability. Data was analysed descriptively. Spearman's correlation was used to assess the relationship between web page characteristics. Kruskal-Wallis test was used to check for differences among readability and other web page characteristics. A total of 693 web pages were included. Web page types included commercial (n = 340), not-for-profit (n = 113), blogs (n = 112), government (n = 89), personal (n = 36) and educational (n = 3). The accuracy of online nutrition information varied with 39.7% of web pages containing accurate information, 22.8% containing mixed information and 37.5% containing inaccurate information. The average reading grade of all pages analysed measured by F-K, SMOG and FOG was 11.8. The mean FRE was 51.6, a 'fairly difficult to read' score. Only 0.5% of web pages were written at or below grade 6 according to F-K, SMOG and FOG. The findings suggest that accuracy of pregnancy-related nutrition information is a problem on the internet. Web page readability is generally difficult and means that the information may not be accessible to those who cannot read at a sophisticated level. © 2016 John Wiley & Sons Ltd.

Readability, complexity, and suitability analysis of online lymphedema resources

BACKGROUND

Over 72% of Americans use online health information to assist in health care decision-making. Previous studies of lymphedema literature have focused only on reading level of patient-oriented materials online. Findings indicate they are too advanced for most patients to comprehend. This, more comprehensive study, expands the previous analysis to include critical elements of health materials beyond readability using assessment tools to report on the complexity and density of data as well as text design, vocabulary, and organization.

METHODS

The top 10 highest ranked websites on lymphedema were identified using the most popular search engine (Google). Website content was analyzed for readability, complexity, and suitability using Simple Measure of Gobbledygook, PMOSE/iKIRSCH, and Suitability Assessment of Materials (SAM), respectively. PMOSE/iKIRSCH and SAM were performed by two independent raters. Fleiss' kappa score was calculated to ensure inter-rater reliability.

RESULTS

Online lymphedema literature had a reading grade level of 14.0 (SMOG). Overall complexity score was 6.7 (PMOSE/iKIRSCH) corresponding to "low" complexity and requiring a 8th-12th grade education. Fleiss' kappa score was 80% (P = 0.04, "substantial" agreement). Overall suitability score was 45% (SAM) correlating to the lowest level of "adequate" suitability. Fleiss' kappa score was 76% (P = 0.06, "substantial" agreement).

CONCLUSIONS

Online resources for lymphedema are above the recommended levels for readability and complexity. The suitability level is barely adequate for the intended audience. Overall, these materials are too sophisticated for the average American adult, whose literacy skills are well documented. Further efforts to revise these materials are needed to improve patient comprehension and understanding.

Readability Formulas and User Perceptions of Electronic Health Records Difficulty: A Corpus Study

Background

Electronic health records (EHRs) are a rich resource for developing applications to engage patients and foster patient activation, thus holding a strong potential to enhance patient-centered care. Studies have shown that providing patients with access to their own EHR notes may improve the understanding of their own clinical conditions and treatments, leading to improved health care outcomes. However, the highly technical language in EHR notes impedes patients’ comprehension. Numerous studies have evaluated the difficulty of health-related text using readability formulas such as Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), and Gunning-Fog Index (GFI). They conclude that the materials are often written at a grade level higher than common recommendations.

Objective

The objective of our study was to explore the relationship between the aforementioned readability formulas and the laypeople’s perceived difficulty on 2 genres of text: general health information and EHR notes. We also validated the formulas’ appropriateness and generalizability on predicting difficulty levels of highly complex technical documents.

Methods

We collected 140 Wikipedia articles on diabetes and 242 EHR notes with diabetes International Classification of Diseases, Ninth Revision code. We recruited 15 Amazon Mechanical Turk (AMT) users to rate difficulty levels of the documents. Correlations between laypeople’s perceived difficulty levels and readability formula scores were measured, and their difference was tested. We also compared word usage and the impact of medical concepts of the 2 genres of text.

Results

The distributions of both readability formulas’ scores (P<.001) and laypeople’s perceptions (P=.002) on the 2 genres were different. Correlations of readability predictions and laypeople’s perceptions were weak. Furthermore, despite being graded at similar levels, documents of different genres were still perceived with different difficulty (P<.001). Word usage in the 2 related genres still differed significantly (P<.001).

Conclusions

Our findings suggested that the readability formulas’ predictions did not align with perceived difficulty in either text genre. The widely used readability formulas were highly correlated with each other but did not show adequate correlation with readers’ perceived difficulty. Therefore, they were not appropriate to assess the readability of EHR notes.

Online Patient Resources for Liposuction: A Comparative Analysis of Readability

BACKGROUND

As patients strive to become informed about health care, inadequate functional health literacy is a significant barrier. Nearly half of American adults have poor or marginal health literacy skills and the National Institutes of Health and American Medical Association have recommended that patient information should be written at a sixth grade level. The aim of this study is to identify the most commonly used online patient information about liposuction and to evaluate its readability relative to average American literacy.

METHODS

An internet search of "liposuction" was performed and the 10 most popular websites identified. User and location data were disabled and sponsored results excluded. All relevant, patient-directed articles were downloaded and formatted into plain text. Articles were then analyzed using 10 established readability tests. A comparison group was constructed to identify the most popular online consumer information about tattooing. Mean readability scores and specific article characteristics were compared.

RESULTS

A total of 80 articles were collected from websites about liposuction. Readability analysis revealed an overall 13.6 grade reading level (range, 10-16 grade); all articles exceeded the target sixth grade level. Consumer websites about tattooing were significantly easier to read, with a mean 7.8 grade level. These sites contained significantly fewer characters per word and words per sentence, as well as a smaller proportion of complex, long, and unfamiliar words.

CONCLUSIONS

Online patient resources about liposuction are potentially too difficult for a large number of Americans to understand. Liposuction websites are significantly harder to read than consumer websites about tattooing. Aesthetic surgeons are advised to discuss with patients resources they use and guide patients to appropriate information for their skill level.

Readability analysis of online resources related to lung cancer

BACKGROUND

Patients seeking health information commonly use the Internet as the first source for material. Studies show that well-informed patients have increased involvement, satisfaction, and healthcare outcomes. As one-third of Americans have only basic or below basic health literacy, the National Institutes of Health and American Medical Association recommend patient-directed health resources be written at a sixth-grade reading level. This study evaluates the readability of commonly accessed online resources on lung cancer.

METHODS

A search for "lung cancer" was performed using Google and Bing, and the top 10 websites were identified. Location services were disabled, and sponsored sites were excluded. Relevant articles (n = 109) with patient-directed content available directly from the main sites were downloaded. Readability was assessed using 10 established methods and analyzed with articles grouped by parent website.

RESULTS

The average reading grade level across all sites was 11.2, with a range from 8.8 (New Fog Count) to 12.2 (Simple Measure of Gobbledygook). The average Flesch Reading Ease score was 52, corresponding with fairly difficult to read text. The readability varied when compared by individual website, ranging in grade level from 9.2 to 15.2. Only 10 articles (9%) were written below a sixth-grade level and these tended to discuss simpler topics.

CONCLUSIONS

Patient-directed online information about lung cancer exceeds the recommended sixth-grade reading level. Readability varies between individual websites, allowing physicians to direct patients according to level of health literacy. Modifications to existing materials can significantly improve readability while maintaining content for patients with low health literacy.

Guiding Oncology Patients Through the Maze of Precision Medicine

As the role of genomics in health care grows, patients increasingly require adequate genetic literacy to fully engage in their care. This study investigated a model for delivering consumer-friendly genetic information to improve understanding of precision medicine using health literacy and learning style principles. My Cancer Genome (MCG), a freely available cancer decision support tool, was used as a testbed. MCG content on a melanoma tumor mutation, BRAF V600E, was translated to a 6th-grade reading level, incorporating multiple learning modalities. A total of 90 patients and caregivers were recruited from a melanoma clinic at an academic medical center and randomized to 3 groups. Group A (control) received an exact copy of text from MCG. Group B was given the same content with hyperlinks to videos explaining key genetic concepts, identified and labeled by the team as knowledge pearls. Group C received the translated content with the knowledge pearls embedded. Changes in knowledge were measured through pre and post questionnaires. Group C showed the greatest improvement in knowledge. The study results demonstrate that providing information based on health literacy and learning style principles can improve patients' understanding of genetic concepts, thus increasing their likelihood of taking an active role in any decision making concerning their health.

Readability assessment of patient-provider electronic messages in a primary care setting

BACKGROUND

The high prevalence of limited health literacy among patients threatens the success of secure electronic messaging between patients from diverse populations and their providers.

OBJECTIVE

The purpose of this study is to generate hypotheses about the readability of patient and provider electronic messages.

METHODS

We collected 31 patient-provider e-mail exchanges (n = 119 total messages) from a safety-net primary care clinic. We compared the messages' mean word count and Flesch-Kincaid Grade Levels (FKGLs), calculated the frequency of provider messages below an FKGL = 8, and assessed readability concordance between patients' and providers' messages.

RESULTS

Patients used more words in their initial e-mails compared to providers, but the FKGLs were similar, and 68% of provider messages were written below an FKGL = 8. Of 31 exchanges, 9 (29%) contained at least one patient message with an FKGL > 3 grade levels lower than the corresponding provider message(s).

CONCLUSION

Our study demonstrates that most providers are able to respond to patient electronic messages with a matching reading level.

Accuracy and readability of cardiovascular entries on Wikipedia: are they reliable learning resources for medical students?

OBJECTIVE

To evaluate accuracy of content and readability level of English Wikipedia articles on cardiovascular diseases, using quality and readability tools.

METHODS

Wikipedia was searched on the 6 October 2013 for articles on cardiovascular diseases. Using a modified DISCERN (DISCERN is an instrument widely used in assessing online resources), articles were independently scored by three assessors. The readability was calculated using Flesch-Kincaid Grade Level. The inter-rater agreement between evaluators was calculated using the Fleiss κ scale.

RESULTS

This study was based on 47 English Wikipedia entries on cardiovascular diseases. The DISCERN scores had a median=33 (IQR=6). Four articles (8.5%) were of good quality (DISCERN score 40-50), 39 (83%) moderate (DISCERN 30-39) and 4 (8.5%) were poor (DISCERN 10-29). Although the entries covered the aetiology and the clinical picture, there were deficiencies in the pathophysiology of diseases, signs and symptoms, diagnostic approaches and treatment. The number of references varied from 1 to 127 references; 25.9±29.4 (mean±SD). Several problems were identified in the list of references and citations made in the articles. The readability of articles was 14.3±1.7 (mean±SD); consistent with the readability level for college students. In comparison, Harrison's Principles of Internal Medicine 18th edition had more tables, less references and no significant difference in number of graphs, images, illustrations or readability level. The overall agreement between the evaluators was good (Fleiss κ 0.718 (95% CI 0.57 to 0.83).

CONCLUSIONS

The Wikipedia entries are not aimed at a medical audience and should not be used as a substitute to recommended medical resources. Course designers and students should be aware that Wikipedia entries on cardiovascular diseases lack accuracy, predominantly due to errors of omission. Further improvement of the Wikipedia content of cardiovascular entries would be needed before they could be considered a supplementary resource.

Patient education for carpal tunnel syndrome: analysis of readability

BACKGROUND

The National Institutes of Health and American Medical Association recommend a sixth grade reading level for patient-directed content. This study aims to quantitatively evaluate the readability of the most commonly used resources for surgical treatment of carpal tunnel syndrome.

METHODS

A web search for "carpal tunnel surgery" was performed using an Internet search engine, and the 13 most popular sites were identified. Relevant, patient-directed articles immediately accessible from the main site were downloaded and formatted into plain text. A total of 102 articles were assessed for readability using ten established analyses: first overall, then by website for comparison.

RESULTS

Patient information about carpal tunnel surgery had an overall average reading level of 13.1. Secondary analysis by website revealed a range of mean readability from 10.8 (high school sophomore level) to 15.3 (university junior level). All sites exceeded the recommended sixth grade reading level.

CONCLUSIONS

Online patient resources for carpal tunnel surgery uniformly exceed the recommended reading level. These are too difficult to be understood by a large portion of American adults. A better understanding of readability may be useful in tailoring more appropriate resources for average patient literacy.

Readability Assessment of Online Patient Abdominoplasty Resources

BACKGROUND

Limited functional health literacy is recognized as an important contributor to health disparities in the United States. As internet access becomes more universal, there is increasing concern about whether patients with poor or marginal literacy can access understandable healthcare information. As such, the National Institutes of Health and American Medical Association recommend that patient information be written at a sixth grade level. This study identifies the most popular online resources for patient information about abdominoplasty and evaluates their readability in the context of average American literacy.

METHODS

The two largest internet search engines were queried for "tummy tuck surgery" to simulate a patient search in lay terms. The ten most popular sites common to both search engines were identified, and all relevant articles from the main sites were downloaded. Sponsored results were excluded. Readability analysis of the articles was performed using ten established tests.

RESULTS

Online information about abdominoplasty from the ten most popular publically available websites had an overall average readability of 12th grade. Mean reading grade level scores among tests were: Coleman-Liau 11.9, Flesch-Kincaid 11.4, FORCAST 11.1, Fry 13, Gunning Fog 13.5, New Dale-Chall 11.8, New Fog Count 9.9, Raygor Estimate 12, and SMOG 13.4; Flesch Reading Ease index score was 46.

CONCLUSIONS

Online patient resources about abdominoplasty are uniformly above the recommended target readability level and are likely too difficult for many patients to understand. A range of readability identified among websites could allow surgeons to guide patients to more appropriate resources for their literacy skills.