A systematic review of readability and comprehension instruments used for print and web-based cancer information

Preparation, validation, and evaluation of an information leaflet for patients undergoing day-care surgeries under general anesthesia at a busy tertiary care hospital

Background and Aims:

It is essential that patients posted for day-care surgeries are adequately prepared preoperatively. Verbal information alone may not be always effective. This study aimed to prepare, validate, and evaluate the efficacy of a patient information leaflet (PIL) for patients undergoing day-care surgeries under general anesthesia (GA).

Material and Methods:

After approval from the hospital ethics committee a PIL was prepared in English. Readability and design of the leaflet were checked using standard tests: Flesch readability ease test (FRE), Flesch Kincaid grade level (FKGL), and Baker Able leaflet design (BALD). It was translated into three regional languages. The PIL was tested among patients using a questionnaire. Seventy-nine adult patients posted for elective day-care procedures were included while emergency surgeries were excluded. Patient knowledge pre and post-PIL was compared using paired 't' test. The influence of age, gender, and education level on the usefulness of PIL were analyzed using the Chi-square test and knowledge was compared using ANOVA.

Results:

The English leaflet had an FRE Score of 63.9 and FKGL of 6.4, which is “standard“. The BALD score for all leaflets was 25 (“above standard“). The overall knowledge scores significantly improved from 52.6% (preintervention) to 70.7% (postintervention), P < 0.001. Knowledge improvement was seen with the use of PIL in all four languages. Sixty eight percent of patients strongly recommended the PIL while 31% were willing to recommend it to others.

Conclusion:

The PILs developed in this study have standard readability, good design and validated for efficacy.

Assessing the readability of patient-targeted online information on musculoskeletal radiology procedures

OBJECTIVE

To assess the readability of patient-targeted online information on musculoskeletal radiology procedures.

METHODS

Eleven common musculoskeletal radiology procedures were queried in three online search engines (Google, Yahoo!, Bing). All unique patient-targeted websites were identified (n = 384) from the first three pages of search results. The reading grade level of each website was calculated using 6 separate validated metrics for readability assessment. Analysis of word and sentence complexity was also performed. Results were compared between academic vs. non-academic websites and between websites found on different pages of the search results. Statistics were performed using a t test.

RESULTS

The mean reading grade level across all procedures was 10th-14th grade. Webpages for nerve block were written at a higher reading grade level on non-academic websites (p = 0.025). There was no difference in reading grade levels between academic and non-academic sources for all other procedures. There was no difference in reading grade levels between websites found on the first page of search results compared with the second and third pages. Across all websites, 16-22% of the words used had 3+ syllables and 31-43% of the words used had 6+ characters (complex words); 13-24% of the sentences used had 22+ words (complex sentences).

CONCLUSION

Patient-targeted online information on musculoskeletal radiology procedures are written at the 10th-14th grade reading level, which is well beyond the AMA and NIH recommendation. Readability can be lowered by decreasing text complexity through limitation of high-syllable words and reduction in word and sentence length.

Patient insights on features of an effective pharmacogenomics patient portal

OBJECTIVES

We built a novel mock pharmacogenomics web portal to deliver pharmacogenomic information and results to patients. Utilizing a patient focus group, we then sought to understand patient insights on desired features of an effective pharmacogenomics patient portal.

METHODS

The mock YourPGx Portal delivered four sample pharmacogenomic results (omeprazole, simvastatin, clopidogrel, and codeine). Patients from our existing institutional, prospective pharmacogenomics implementation study were recruited to pilot the mock portal and then asked to participate in a focus group discussion led by two facilitators. All patients had been previously genotyped, but none had been directly provided access to their own genotyping results and none had previously used the YourPGx portal. The focus group discussion explored nine domains: (1) factors influencing drug response, (2) concerns about drug effects, (3) understanding of genomics and pharmacogenomics, (4) reasons to undergo pharmacogenomic testing, (5) sources of pharmacogenomic information for patient education, (6) attributes of pharmacogenomic sources of information, (7) considerations about privacy and personal pharmacogenomic information, (8) sharing of pharmacogenomic information, and (9) features of an effective patient portal.

RESULTS

The median age of patients (n = 10) was 65.5 years old (range 38-72), 70% female, 50% Caucasian/30% Black, and 60% held a bachelor/advanced degree. When asked about resources for seeking pharmacogenomic information, patients preferred consulting their providers first, followed by self-education, then using information provided by university research organizations. A theme emerged regarding attributes of these sources, namely a desire for understandability and trust. Patients said that the effectiveness of a pharmacogenomics patient portal is improved with use of symbolisms/graphics and clear and concise content. Effective use of colors, quantifying information, consistency, and use of layperson's language were additional important facets. Patients communicated the appeal of secured phone/app-enabled access and said that they would desire linking to their electronic medical records to allow sharing of information with different members of their healthcare team.

CONCLUSIONS

Patients named providers as their primary source of pharmacogenomic information, but a pharmacogenomics patient portal that is carefully constructed to incorporate desired features may be a favorable tool to effectively deliver pharmacogenomic information and results to patients.

Readability of the Most Commonly Accessed Online Patient Education Materials Pertaining to Surgical Treatments of the Spine

OBJECTIVE

The American Medical Association and National Institutes of Health have recommended that patient education materials should be written at the sixth-grade reading level to maximize patient comprehension. The objective of the present study was to evaluate the readability of Internet information for the 9 most common spinal surgeries.

METHODS

We reviewed 90 online patient educational materials regarding the 9 most common spinal surgeries as reported by the North American Spine Society. A Google search was performed on March 23, 2019 for each surgery, and the top 10 most visited websites for each surgery were assessed for reading level using the Flesch-Kincaid formula.

RESULTS

Using the Flesch-Kincaid formula, the average grade reading level of the 90 websites included was 12.82, with a reading ease of 37.04 ("difficult college"). Only 6 websites relayed information to patients at or below the national average of an eighth-grade reading level. The websites for bone morphogenic protein had the highest average grade reading level at 15.88 ± 2.6. Lumbar microscopic discectomy had the lowest average grade reading level at 10.37 ± 2.89. All surgical options discussed had an average readability above the recommended sixth-grade reading level.

CONCLUSIONS

The most accessed online materials for common spinal surgeries, not only exceeded the readability limits recommended by both the American Medical Association and the National Institutes of Health, but they also exceeded the average reading ability of most adults in the United States. Patients, therefore, might not fully comprehend the information from commonly accessed websites regarding surgical spine treatment options.

Readability Analysis of the American Society of Ophthalmic Plastic & Reconstructive Surgery Patient Educational Brochures

INTRODUCTION

Previous studies have shown patient education material (PEM) in ophthalmology has been written at levels exceeding appropriate reading levels. However, information for readability in the field of oculoplastics remains limited. The aim of this study was to evaluate the readability of patient educational brochures from the American Society of Ophthalmic Plastic & Reconstructive Surgery (ASOPRS).

METHODS

Patient educational brochures from ASOPRS were analyzed for readability. The body of text from all 18 ASOPRS patient brochures was analyzed by ten validated tests for English readability assessment: Flesch Reading Ease Test (FRE), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), Coleman-Liau Index (CLI), Gunning Fog Index (GFI), New Dale-Chall Readability (NDC), FORCAST, Fry Graph Readability (FG), Raygor Readability Estimate (RRE), and New Fog Count (NFC).

RESULTS

The mean (± SD) readability scores from the 18 ASOPRS patient brochures were 48 (4.3), 11.0 (0.8), 13.0 (0.7), 11.7 (0.8), 13.6 (0.9), 11.3 (0.8), 11.1 (0.5), 12.1 (1.5), 12.2 (1.0), and 10.6 (1.3) for FRE, FKGL, SMOG, CLI, GFI, NDC, FORCAST, FG, RRE, and NFC, respectively. All ten of the mean readability scores were above the recommended reading levels.

CONCLUSIONS

These findings show that the average patient may have difficulty understanding educational information provided by ASOPRS patient brochures, thereby hindering their ability to make informed decisions on their healthcare. Revision with readability as a primary goal, with input from patients and caregivers, may be necessary to improve health literacy among patients who seek oculoplastic care.

Use of Machine Learning Algorithms to Predict the Understandability of Health Education Materials: Development and Evaluation Study

BACKGROUND

Improving the understandability of health information can significantly increase the cost-effectiveness and efficiency of health education programs for vulnerable populations. There is a pressing need to develop clinically informed computerized tools to enable rapid, reliable assessment of the linguistic understandability of specialized health and medical education resources. This paper fills a critical gap in current patient-oriented health resource development, which requires reliable and accurate evaluation instruments to increase the efficiency and cost-effectiveness of health education resource evaluation.

OBJECTIVE

We aimed to translate internationally endorsed clinical guidelines to machine learning algorithms to facilitate the evaluation of the understandability of health resources for international students at Australian universities.

METHODS

Based on international patient health resource assessment guidelines, we developed machine learning algorithms to predict the linguistic understandability of health texts for Australian college students (aged 25-30 years) from non-English speaking backgrounds. We compared extreme gradient boosting, random forest, neural networks, and C5.0 decision tree for automated health information understandability evaluation. The 5 machine learning models achieved statistically better results compared to the baseline logistic regression model. We also evaluated the impact of each linguistic feature on the performance of each of the 5 models.

RESULTS

We found that information evidentness, relevance to educational purposes, and logical sequence were consistently more important than numeracy skills and medical knowledge when assessing the linguistic understandability of health education resources for international tertiary students with adequate English skills (International English Language Testing System mean score 6.5) and high health literacy (mean 16.5 in the Short Assessment of Health Literacy-English test). Our results challenge the traditional views that lack of medical knowledge and numerical skills constituted the barriers to the understanding of health educational materials.

CONCLUSIONS

Machine learning algorithms were developed to predict health information understandability for international college students aged 25-30 years. Thirteen natural language features and 5 evaluation dimensions were identified and compared in terms of their impact on the performance of the models. Health information understandability varies according to the demographic profiles of the target readers, and for international tertiary students, improving health information evidentness, relevance, and logic is critical.

A mixed-methods study to evaluate the acceptability of information leaflets for breast cancer screening

AIMS AND OBJECTIVES

To obtain feedback regarding the comprehension and acceptability of an information leaflet on breast cancer screening among women.

BACKGROUND

Women usually receive a leaflet inviting them to participate in breast cancer screening programmes for early detection of breast cancer. These leaflets include relevant information about mammograms to aid with decision-making. It is important to know how the target population perceives these leaflets, as this may influence the attitude towards breast cancer screening.

DESIGN

A mixed-methods study (concurrent triangulation design).

METHODS

Participants were 41 women aged between 40-60 years in the north of Spain. The leaflet included information about breast cancer, mammograms, benefits/risks of screening, and the breast cancer mortality and survival rates in relation to screening. Three written methods were used to assess the leaflet: a readability assessment (Flesch Index) and two comprehension assessments (Cloze and multiple-choice questions). In addition, 26 semi-structured interviews were conducted to explore the women's opinions regarding comprehension, acceptability and the contribution of the leaflet as a decision aid. The COREQ checklist was used to guarantee the quality and rigour of the qualitative study.

RESULTS

Overall, women found the leaflet fairly easy to read and most understood the content. Some women found the information on mortality alarming, difficult to understand and a cause for rejection. The leaflet was viewed as a decision aid, although further information was requested on mammograms, the need for complementary tests and the risks and symptoms of breast cancer.

CONCLUSION

The leaflet was accepted by women and, overall, easy to understand. The participation of the target population enabled the content and format of the leaflet to be adjusted to their needs.

RELEVANCE FOR CLINICAL PRACTICE

Nurses can contribute towards improving the breast cancer screening leaflets, considering women's perceptions surrounding screening.

Patient-Reported Outcome Measures in Voice: An Updated Readability Analysis

PURPOSE

The purpose of this study was to investigate whether voice-related patient-reported outcome measures (PROMs) developed and validated since 2011 meet the recommendation by health literacy experts that such materials be written at a fifth-to-sixth grade reading level.

METHOD

A readability analysis of eight voice-related PROMs was conducted. Readability formulas utilized were the Coleman-Liau index, Flesch-Kincaid reading ease, FORCAST, simple measure of Gobbledygook index, and Gunning-Fog score.

RESULT

Three-fourths of the PROMs exceeded the recommended fifth- to sixth-grade reading level.

CONCLUSION

Although awareness of health literacy has grown, voice-related PROMs continue to be developed without full consideration of their reading grade level. Researchers should consider revising or developing PROMs with consideration to reading grade level as well as other features to enhance readability.

Readability of Patient Education Materials From High-Impact Medical Journals: A 20-Year Analysis

Comprehensive patient education is necessary for shared decision-making. While patient–provider conversations primarily drive patient education, patients also use published materials to enhance their understanding. In this investigation, we evaluated the readability of 2585 patient education materials published in high-impact medical journals from 1998 to 2018 and compared our findings to readability recommendations from national groups. For all materials, mean readability grade levels ranged from 11.2 to 13.8 by various metrics. Fifty-four (2.1%) materials met the American Medical Association recommendation of sixth grade reading level, and 215 (8.2%) met the National Institutes of Health recommendation of eighth grade level. When stratified by journal and material type, general medical education materials from Annals of Internal Medicine were the most readable (P < .001), with 79.8% meeting the eighth grade level. Readability did not differ significantly over time. Efforts to standardize publication practice with the incorporation of readability evaluation during the review process may improve patients’ understanding of their disease processes and treatment options.

Quality and readability of internet information about stuttering

PURPOSE

We examined the quality and readability of English-language Internet information about stuttering and evaluated the results considering recommendations by experts in health literacy.

METHOD

A search of Internet websites containing information about stuttering was conducted. Three key words (i.e., stuttering, stammering, speech disfluency) were entered into five country-specific versions of the most commonly used Internet search engine. A total of 79 websites were assessed. Their origin (commercial, non-profit, government, personal or university), quality [Health On the Net (HON) certification and DISCERN scores], and readability [Flesch Reading Ease (FRE) score, Flesch-Kincaid Grade Level Formula (F-KGL), and Simple Measure of Gobbledygook (SMOG)] were assessed.

RESULTS

Of the 79 websites, 38 % were of commercial, 42 % were of nonprofit organization, 15 % were of government and 5% were of university origins, respectively. Only 13 % had obtained HON certification and the mean DISCERN scores was 3.10 in a 5-point scale. The mean reading grade levels were at 13th and 14th grade and 100 % of the websites exceeded the recommended 5th to 6th reading grade level for health information.

CONCLUSIONS

The quality of Internet-based health information about the treatment of stuttering is generally adequate, but actual usability of the sites examined in this study may be limited due to poor readability levels. This is problematic in persons with poor literacy skills. Since the Internet can be readily accessed as a valuable consumer information resource, speech-language pathologists and other healthcare professionals have an opportunity to direct consumers to websites that provide readable information of good quality.

Exploring fear of childbirth in Kenya through evaluation of the readability of Wijma Delivery Expectancy/Experience Questionnaire Version A (W-DEQ-A)

BACKGROUND

Several tools measuring fear of childbirth (FOC) have been developed in the last three decades, however concerns about their readability have been raised.

AIM

To explore the fear of childbirth in a sample of women of reproductive age by evaluating the readability of Wijma Delivery Expectancy/Experience Questionnaire version A (W-DEQ-A).

METHODS

The Flesch Reading Ease Formula, the Flesch-Kincaid Grade Level, the FOG Scale, the SMOG Index, the Coleman-Liau Index, the Automated Readability Index, and the Linsear Write Formula were used to evaluate the readability of the W-DEQ-A. Also, focus group discussions were held to validate the findings of the readability scales mentioned above.

FINDINGS

The SMOG Index (score = 7.6), Coleman-Liau Index (score = 7.6), and the Linsear Write Formula (score = 9.4) were easily readable by women of reproductive age who had at least secondary school education (grade 12). Concerns were raised over some terms used such as desolate and deserted, which were rarely used in day to day English language conversations.

CONCLUSIONS

In this study, participants observed that W-DEQ- A was readable if administered to expectant women with a basic secondary school certificate; but there is a need to simplify some words. It was emphasized that societal dynamics play an important role in the fear of childbirth and therefore the questionnaire should address all aspects contributing to fear of childbirth and not merely the feelings and thoughts women may have at the prospect of labor and delivery.

Evaluating Readability Scores of Treatment Summaries and Cancer Survivorship Care Plans

PURPOSE

Treatment Summaries and Survivorship Care Plans (TS/SCPs) may be difficult for patients to comprehend because of readability, magnitude of information, and complex medical verbiage.

METHODS

Readability scores were calculated for TS/SCP templates including ASCO, Oncolink, Journey Forward, and the authors' institution. The Simple Measure of Gobbledygook (SMOG) index, Flesch-Kincaid reading grade level, Coleman-Liau Index, and Gunning Fog index were used to assess readability.

RESULTS

The Flesch-Kincaid reading ease scores for the blank ASCO templates ranged from 47.4 to 53.3, requiring a reading grade level of 10-12. Coleman-Liau and Gunning Fog scores showed that an 11th grade reading level is essential, and SMOG required a college education to comprehend the ASCO templates. For the colorectal case exemplar, Oncolink's template resulted in the lowest SMOG score (11.3; 11th grade), Flesch-Kincaid reading grade level (11; 11th grade), and Coleman-Liau score (12; 12th grade). Journey Forward's TS/SCP template scored the highest on the SMOG (21.2; college graduate), Flesch-Kincaid reading grade level (18.3; college graduate), and Gunning-Fog index (25.8; college graduate) compared with other TS/SCPs.

CONCLUSION

The existing TS/SCP templates used by US cancer centers are written at a grade level beyond the comprehension of most adults. Cancer care teams should assess TS/SCP content for readability and use of plain language and reduce medical jargon.

The Experience of Chemotherapy Teaching and Readability of Chemotherapy Educational Materials for Women with Breast Cancer

Chemotherapy is one of the most common forms of treatment for women with breast cancer. While chemotherapy is often effective, managing side effects can be challenging. Chemotherapy education is critical in assisting patients to manage side effects and to improve the treatment experience. However, materials are often not thoroughly assessed for readability and format which could be problematic for patients learning self-care while in treatment. We used a mixed-method design to illuminate chemotherapy teaching and focused on readability and format of education materials. We scored the materials using three readability assessments: (1) Flesch Reading Ease (FRE), (2) Flesch-Kincaid (F-K), and (3) a Simple Measure of Gobbledygook (SMOG). We evaluated the format of the materials using Suitability Assessment of Materials (SAM) guidelines. Lastly, we used thematic analysis to describe the experience of 37 women with breast cancer undergoing chemotherapy education. The mean readability of the materials ranged from "difficult" to "fairly difficult" based on the FRE scoring, and the material was written on a 9th- to 13th-grade reading level. Most of the materials scored as "adequate" using SAM guidelines but lacked incorporation of graphics or illustrations. The thematic analysis revealed three major findings: (1) finding control in learning, (2) receiving unexpected support, and (3) learning in unforeseen ways. Nurses need to supplement chemotherapy education materials with individualized teaching to ensure comprehension. Additionally, nurses and website developers may want to consider implementing culturally appropriate information and use videos to combat challenging readability.

Comparing quality and readability of online English language information to patient use and perspectives for common rheumatologic conditions

Evaluate quality and readability of online information for common rheumatologic diseases. Compare rheumatology patients' internet use and preferences to an objective evaluation of internet quality and readability. Five common rheumatologic diseases were searched on the web browser Google using English language. The first twenty websites from each of the five searches were evaluated for internet quality (e.g. content that is current, balanced, has specific aims, and is appropriately cited) using the DISCERN criteria and readability using the Flesch-Kincaid Grade Level (FKGL). The results were contrasted with a survey sent to patients with rheumatic disease. The survey measured patient likeliness to use and trust identified websites. Internet quality was similar (good) for all five diseases searched while readability was poor. There was an inverse relationship between internet quality and readability. Internet quality significantly differed across website sponsor, and readability significantly differed across disease and website sponsor. Common medical website sponsors with the highest combined quality and readability scores were Mayo Clinic and Web MD. Eight hundred and fifty-eight patients were sent a survey, of which 147 (17%) completed. Patients indicated they were most likely to use and trust a Mayo Clinic-sponsored website when compared to other common sponsored websites from our evaluation, followed by the American College of Rheumatology. Although we found good-quality information, all websites evaluated had readability levels above the recommended sixth-grade reading level. The website sponsor with the highest combined readability and quality score was also the most used and trusted by patients. Patients would like more information about credible and trusted websites from their medical providers.

Quality assessment of cancer patient education materials: the current state of systemic therapy patient education in fourteen cancer centres across Ontario, Canada

PURPOSE

Most patients diagnosed with cancer are administered systemic therapy and these patients are counselled and given printed education (PE) materials. High rates of low health literacy highlight the need to evaluate the quality of these PE materials.

METHODS

A current state assessment of the quality of PE materials was conducted in Ontario, Canada. Patient education leaders from 14 cancer centres submitted print materials on the topic of systemic cancer therapy to the assessment team. To report adherence to PE quality and health literacy best practices, the following validated measures were used: readability (FRY, SMOG and Flesch Reading Ease), understandability and actionability (Patient Education Materials Assessment Tool (PEMAT)). Materials at grade level 6 or lower and with PEMAT scores greater than 80% were considered to meet health literacy best practices.

RESULTS

A total of 1146 materials were submitted; 366 met inclusion criteria and 83 were selected for assessment. Most materials scored below the 80% target for understandability (x̄ = 73%, 31-100%) and actionability (x̄ = 68%, 20-100%), and above the recommended grade 6 readability level (x̄ = grade 9) meaning that the majority did not meet quality standards or best practices.

CONCLUSION

Results suggest that there is significant opportunity to improve the quality of PE materials distributed by cancer centres. The quality of PE materials is a critical safety and equity consideration when these materials convey important safety and self-care directives.

Assessing clinical education tools for expanded carrier screening

Expanded carrier screening (ECS) is increasingly offered to a broader population and raises challenges of how to best educate and counsel the volume of screened individuals. For this study, we compared three educational tools (brochure, video and comic) about ECS on knowledge and decision making. A convenience online sample of 151 pregnant women was randomized to one of three groups (Video, n = 42; Comic n = 54; Brochure n = 55). Knowledge scores were significantly higher for the comic group compared to the video or the brochure groups (p < .001). No significant differences in preparation for decision making, decisional conflict, or perceptions of shared decision making were identified between the study groups. This study suggests that a comic about ECS may improve patient attention and retention of information. The use of graphic narratives may enable individuals to better understand medical information in general.

The Quality, Readability, Completeness, and Accuracy of PTSD Websites for Firefighters

Firefighters appear at an increased risk for post-traumatic stress disorder (PTSD). Because of PTSD-related stigma, firefighters may search for information online. The current study evaluated the quality, readability, and completeness of PTSD online resources, and to determine how the online treatment recommendations align with current evidence. Google.ca (Canada) searches were performed using four phrases: ‘firefighter PTSD’, ‘firefighter operational stress’, ‘PTSD symptoms’, and ‘PTSD treatment’. The 75 websites identified were assessed using quality criteria for consumer health information (DISCERN), readability and health literacy statistics, content analysis, and a comparison of treatments mentioned to the current best evidence. The average DISCERN score was 43.8 out of 75 (indicating ‘fair’ quality), with 9 ‘poor’ websites (16–30), 31 ‘fair’ websites (31–45), 26 “good” websites (46–60), and nine excellent websites (61–75). The average grade level required to understand the health-related content was 10.6. The most mentioned content was PTSD symptoms (48/75 websites) and PTSD treatments (60/75 websites). The most frequently mentioned treatments were medications (41/75 websites) and cognitive behavioural therapy (40/75 websites). Cognitive behavioural therapy is supported by strong evidence, but evidence for medications appears inconsistent in current systematic reviews. Online PTSD resources exist for firefighters, but the information is challenging to read and lacks evidence-based treatment recommendations.

Predicting the readability of physicians' secure messages to improve health communication using novel linguistic features: Findings from the ECLIPPSE study

Background

Low literacy skills impact important aspects of communication, including health-related information exchanges. Unsuccessful communication on the part of physician or patient contributes to lower quality of care, is associated with poorer chronic disease control, jeopardizes patient safety and can lead to unfavorable healthcare utilization patterns. To date, very little research has focused on digital communication between physicians and patients, such as secure messages sent via electronic patient portals.

Method

The purpose of the current study is to develop an automated readability formula to better understand what elements of physicians' digital messages make them more or less difficult to understand. The formula is developed using advanced natural language processing (NLP) to predict human ratings of physician text difficulty.

Results

The results indicate that NLP indices that capture a diverse set of linguistic features predict the difficulty of physician messages better than classic readability tools such as Flesch Kincaid Grade Level. Our results also provide information about the textual features that best explain text readability.

Conclusion

Implications for how the readability formula could provide feedback to physicians to improve digital health communication by promoting linguistic concordance between physician and patient are discussed.

Gist Inference Scores predict gist memory for authentic patient education cancer texts

OBJECTIVES

Develop a tool to evaluate and improve written medical communication to patients. Determine how effectively Gist Inference Scores (GIS) predict comprehension of patient education texts independently of health literacy. Explicate the text characteristics and psychological mechanism underlying GIS.

METHODS

For study 1, a nationally representative sample of older women (N = 61) completed a fill-in-the-blank comprehension task on authentic National Cancer Institute (NCI) texts of varying GIS levels. In study 2, participants (N = 198) read NCI texts (high or low GIS) then recalled what they read.

RESULTS

Study 1 showed that a higher percentage of different words yielding semantically similar sentence meaning were used to correctly fill-the-blanks on high GIS texts and there was no significant interaction with health literacy. In study 2, a greater proportion of decision-relevant information was recalled for high GIS texts.

CONCLUSIONS

GIS predicts the likelihood that readers will form gist representations of medical texts on free recall and fill-in-the-blank tasks. High GIS texts allow for more semantic flexibility to mentally represent the same meaning, and more strongly emphasizes gist rather than verbatim representations.

PRACTICAL IMPLICATIONS

GIS provides medical communicators with an automated and user-friendly method to evaluate medical texts for their ability to convey the bottom-line meaning.

How good is online information on fibromyalgia? An analysis of quality and readability of websites on fibromyalgia in Spanish

OBJECTIVE

To assess the content, quality and readability of websites with information on fibromyalgia in Spanish.

METHODS

Websites were retrieved entering the keyword 'fibromyalgia' in Google, Yahoo! and Bing, and by searching records of patients associations in Spain and Latin America. The Bermúdez-Tamayo and DISCERN questionnaires were employed for evaluating quality and content, and INFLESZ for readability. Statistical analysis was conducted using IBM SPSSV.24 (Chicago, USA).

RESULTS

Three hundred and five websites were found. After applying the exclusion criteria, 73 websites were analysed. Websites retrieved by search engines obtained median scores of 27.0 (interquartile interval (IQI): 24.5-32.0) with DISCERN, 35.0 (IQI: 31.0-40.5) with Bermúdez-Tamayo and 53.7 (IQI: 47.4-56.2) with INFLESZ, whereas those from patients associations scored 21.0 (IQI: 19.2-23.8), 26.0 (IQI: 25.0-31.0) and 51.7 (IQI: 47.9-55.1), respectively. In general, content was not up-to-date.

CONCLUSIONS

Overall quality was medium-low, content quality was very low and readability was poor. Further effort is needed to guarantee meeting quality criteria and accessing updated, relevant, and legible information.This study exposes the quality and readability of websites on fibromyalgia in Spanish, which can help healthcare workers to better appraise this resource and its potential influence on the development of the pathology.

Assessment of Hypertension Self-Care Behaviors and Self-Efficacy Among Men in Saudi Arabia

BACKGROUND AND PURPOSE

Hypertension is rapidly increasing in the Kingdom of Saudi Arabia (KSA), particularly among men. Assessment of hypertension self-care behaviors is a critical step to promoting blood pressure control. This study aimed to evaluate the Hypertension Self-Care Profile (HBP-SCP) among Saudi men in KSA.

METHODS

The HBP-SCP self-efficacy and behavior scales were translated into Arabic and tested on a convenience sample of 160 Saudi men with hypertension. Cronbach's alpha and factor analysis were conducted using SPSS.

RESULTS

Internal consistency of the Arabic HBP self-care and self-efficacy scales was 0.84 and 0.90, respectively. Factor loading ranged from 0.25 to 0.70 for HBP self-care behaviors and from 0.28 to 0.77 for HBP self-efficacy.

CONCLUSIONS

The Arabic HBP-SCP is acceptable in its reliability and validity for measuring HBP self-care behaviors and self-efficacy among Saudi men with hypertension.

Evaluation of patient education materials for stereotactic radiosurgery from high-performing neurosurgery hospitals and professional societies

Background

Little is known about the readability and utility of patient education materials for stereotactic radiosurgery (SRS). Therefore, the goal of this investigation was to evaluate such materials from high-performing neurosurgery hospitals and professional societies through an analysis of readability and educational content.

Methods

In this cross-cross sectional study, 61 websites associated with the top 50 neurosurgery and neurology hospitals according to U.S. News & World Report (USNWR) and 11 predetermined professional medical societies were queried. Identified SRS education materials were analyzed by 6 readability indices. Educational content was assessed by 10 criteria based on surveys of patients' perspectives about SRS.

Results

Fifty-four materials were identified from the target population (45 from USNWR hospital websites and 9 from professional society websites). Mean readability of materials ranged from 11.7 to 15.3 grade level, far more difficult than national recommendations of sixth and eighth grade. Materials were found to have deficiencies in educational content. Compared with high-performing hospitals, materials from websites of professional societies were longer (P = .002), and more likely to discuss risks and benefits specific to SRS (P = .008), alternative treatment options (P = .05) and expected outcomes or postprocedure descriptions (P = .004). Hospital materials were also more likely to favor brand-specific terminology (eg, GammaKnife) over generic terminology (eg, radiosurgery; P = .019).

Conclusion

Publicly available online patient educational materials for SRS are written at reading levels above national recommendations. Furthermore, many lack information identified as important by patients. Reevaluation and improvement of online SRS educational materials on a national scale are warranted.

Scientific abstracts and plain language summaries in psychology: A comparison based on readability indices

Findings from psychological research are usually difficult to interpret for non-experts. Yet, non-experts resort to psychological findings to inform their decisions (e.g., whether to seek a psychotherapeutic treatment or not). Thus, the communication of psychological research to non-expert audiences has received increasing attention over the last years. Plain language summaries (PLS) are abstracts of peer-reviewed journal articles that aim to explain the rationale, methods, findings, and interpretation of a scientific study to non-expert audiences using non-technical language. Unlike media articles or other forms of accessible research summaries, PLS are usually written by the authors of the respective journal article, ensuring that research content is accurately reproduced. In this study, we compared the readability of PLS and corresponding scientific abstracts in a sample of 103 journal articles from two psychological peer-reviewed journals. To assess readability, we calculated four readability indices that quantify text characteristics related to reading comprehension (e.g., word difficulty, sentence length). Analyses of variance revealed that PLS were easier to read than scientific abstracts. This effect emerged in both included journals and across all readability indices. There was only little evidence that this effect differed in magnitude between the included journals. In sum, this study shows that PLS may be an effective instrument for communicating psychological research to non-expert audiences. We discuss future research avenues to increase the quality of PLS and strengthen their role in science communication.

Assessment of quality and readability of internet-based health information related to commonly prescribed angiotensin receptor blockers

Introduction

Hypertension is a global public health burden. Angiotensin receptor blockers (ARBs) have proven efficacy in the management of hypertension and related complications. The Internet has become a major source of health information for patients and healthcare professionals. The study aimed to assess the quality and readability of internet-based information related to selected Angiotensin Receptor Blockers (ARBs).

Methods

The three most widely used ARBs were identified from published literature, after which internet-based patient information was identified from the first five pages of three search engines (Google, Yahoo and Bing). Quality of identified websites were assessed using the DISCERN instrument, while readability was evaluated using the SMOG instrument and the Flesch-Kincaid readability algorithm. Final ratings were then calculated as described by the instruments developers. Further, inter-class correlation coefficients (ICC) were calculated using the Statistical Package for Social Sciences.

Results

The average overall DISCERN score in this study was 2.99 (SD±1.05). No website received an excellent rating, 15% were rated good, 66% as moderate and 19% as poor. The inter-class reliability was 0.804 for losartan and 0.695 for valsartan. The mean Flesch Reading Ease score for the websites was 48.87 (SD±16.12), mean Flesch-Kincaid Reading Grade Level was 9.29 (SD±1.98) while mean SMOG value was 11.29 (SD±1.70).

Conclusion

Overall, patient information on the reviewed ARBs websites was found to be of moderate quality and suboptimal readability. Content providers on websites should ensure that health information is of favorable quality and easy to read by patients with varying degree of health literacy.

The development and comparative effectiveness of a patient-centered prostate biopsy report: a prospective, randomized study

PURPOSE

The prostate biopsy pathology report represents a critical document used for decision-making in patients diagnosed with prostate cancer, yet the content exceeds the health literacy of most patients. We sought to create and compare the effectiveness of a patient-centered prostate biopsy report compared with standard reports.

MATERIALS AND METHODS

Using a modified Delphi approach, prostate cancer experts identified critical components of a prostate biopsy report. Patient focus groups provided input for syntax and formatting of patient-centered pathology reports. Ninety-four patients with recent prostate biopsies were block randomized to the standard report with or without the patient-centered report. We evaluated patient activation, self-efficacy, provider communication skills, and prostate cancer knowledge.

RESULTS

Experts selected primary and secondary Gleason score and the number of positive scores as the most important elements of the report. Patients prioritized a narrative design, non-threatening language and information on risk classification. Initial assessments were completed by 87% (40/46) in the standard report group and 81% (39/48) in the patient-centered report group. There were no differences in patient activation, self-efficacy, or provider communication skills between groups. Patients who received the patient-centered report had significantly improved ability to recall their Gleason score (100% vs. 85%, p = 0.026) and number of positive cores (90% vs. 65%, p = 0.014). In total, 86% of patients who received the patient-centered report felt that it helped them better understand their results and should always be provided.

CONCLUSIONS

Patient-centered pathology reports are associated with significantly higher knowledge about a prostate cancer diagnosis. These important health information documents may improve patient-provider communication and help facilitate shared decision-making among patients diagnosed with prostate cancer.

Influence of high versus low readability level of written health information on self-efficacy: A randomized controlled study of the processing fluency effect

We investigated the relationship of processing fluency of written information about exercise to participants’ perceived interest, safety, self-efficacy, outcome expectation, and behavioral intention regarding the exercise. We randomly assigned 400 men and women aged 40–69 years to control or intervention conditions. Perceived self-efficacy of performing the exercise in the intervention group (i.e. easy to read) was significantly higher than that in the control group (i.e. difficult to read) (p = 0.04). Easy-to-read written health information may be important not only for making written health information comprehensible but also for increasing readers’ self-efficacy for adopting health-related behaviors.

The Difficulty of German Information Booklets on Psoriasis and Psoriatic Arthritis: Automated Readability and Vocabulary Analysis

Background

Information-seeking Psoriasis or Psoriatic Arthritis patients are confronted with numerous educational materials when looking through the internet. Literature suggests that only 17.0-21.4% (Psoriasis, Psoriatic Arthritis) of patients have a good level of knowledge about psoriasis treatment and self-management. A study from 1994 found that English Psoriasis/Psoriatic Arthritis brochures required a reading level between grades 8-12 to be understandable, which was confirmed in a follow-up study 20 years later. As readability of written health-related text material should not exceed the sixth-grade level, Psoriasis/Psoriatic Arthritis material seems to be ill-suited to its target audience. However, no data is available on the readability levels of Psoriasis/Psoriatic Arthritis brochures for German-speaking patients, and both the volume and their scope are unclear.

Objective

This study aimed to analyze freely available educational materials for Psoriasis/Psoriatic Arthritis patients written in German, quantifying their difficulty by assessing both the readability and the vocabulary used in the collected brochures.

Methods

Data collection was conducted manually via an internet search engine for Psoriasis/Psoriatic Arthritis–specific material, published as PDF documents. Next, raw text was extracted, and a computer-based readability and vocabulary analysis was performed on each brochure. For the readability analysis, we applied the Flesch Reading Ease (FRE) metric adapted for the German language, and the fourth Vienna formula (WSTF). To assess the laymen-friendliness of the vocabulary, the computation of an expert level was conducted using a specifically trained Support Vector Machine classifier. A two-sided, two-sample Wilcoxon test was applied to test whether the difficulty of brochures of pair-wise topic groups was different from each other.

Results

In total, n=68 brochures were included for readability assessment, of which 71% (48/68) were published by pharmaceutical companies, 22% (15/68) by nonprofit organizations, and 7% (5/68) by public institutions. The collection was separated into four topic groups: basic information on Psoriasis/Psoriatic Arthritis (G1/G2), lifestyle, and behavior with Psoriasis/Psoriatic Arthritis (G3/G4), medication and therapy guidance (G5), and other topics (G6). On average, readability levels were comparatively low, with FRE=31.58 and WSTF=11.84. However, two-thirds of the educational materials (69%; 47/68) achieved a vocabulary score ≦4 (ie, easy, very easy) and were, therefore, suitable for a lay audience. Statistically significant differences between brochure groups G1 and G3 for FRE (P=.001), WSTF (P=.003), and vocabulary measure (L) (P=.01) exist, as do statistically significant differences for G2 and G4 in terms of FRE (P=.03), WSTF (P=.03) and L (P=.03).

Conclusions

Online Psoriasis/Psoriatic Arthritis patient education materials in German require, on average, a college or university education level. As a result, patients face barriers to understanding the available material, even though the vocabulary used seems appropriate. For this reason, publishers of Psoriasis/Psoriatic Arthritis brochures should carefully revise their educational materials to provide easier and more comprehensible information for patients with lower health literacy levels.

Reading Level of Online Patient Education Materials From Major Obstetrics and Gynecology Societies

OBJECTIVE

To assess whether the readability of online patient education materials provided by eight nationally recognized obstetrics and gynecology societies is in accordance with the recommended 6th-grade reading level outlined by the American Medical Association, National Institute of Health, and United States Department of Health and Human Services.

METHODS

An analysis of 410 online patient education materials from the American Association of Gynecologic Laparoscopists, the American College of Obstetricians and Gynecologists, the American Society for Reproductive Medicine, the American Urogynecologic Society, the Association of Reproductive Health Professionals, the Society of Gynecologic Oncology, the Society for Maternal-Fetal Medicine, and Voices for Pelvic Floor Disorders was completed, and the readability scores using the Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning Fog Scale, and Simple Measure of Gobbledygook readability scales were calculated. These scales are used to assess print and web-based written material across a wide range of medical specialties and are endorsed by the National Institutes of Health. All four scales are computer-based assessments of readability that consider word count, number of syllables, and length of sentences when calculating a score that corresponds to grade level.

RESULTS

Majority of the patient education materials across all eight obstetrics and gynecology societies had readability scores above the recommended 6th-grade reading level. The average reading level for the 69 obstetrics-related articles ranged from 9th to 12th grade. The mean grade level for the 341 gynecology articles had a similar range across the four readability scales.

CONCLUSION

Online patient education materials provided by major obstetrics and gynecology societies do not currently adhere to recommended readability guidelines. Continued efforts to provide accessible and informative patient education materials is recommended to help improve health literacy for women.

Using Natural Language Processing to Examine the Uptake, Content, and Readability of Media Coverage of a Pan-Canadian Drug Safety Research Project: Cross-Sectional Observational Study

Background

Isotretinoin, for treating cystic acne, increases the risk of miscarriage and fetal abnormalities when taken during pregnancy. The Health Canada–approved product monograph for isotretinoin includes pregnancy prevention guidelines. A recent study by the Canadian Network for Observational Drug Effect Studies (CNODES) on the occurrence of pregnancy and pregnancy outcomes during isotretinoin therapy estimated poor adherence to these guidelines. Media uptake of this study was unknown; awareness of this uptake could help improve drug safety communication.

Objective

The aim of this study was to understand how the media present pharmacoepidemiological research using the CNODES isotretinoin study as a case study.

Methods

Google News was searched (April 25-May 6, 2016), using a predefined set of terms, for mention of the CNODES study. In total, 26 articles and 3 CNODES publications (original article, press release, and podcast) were identified. The article texts were cleaned (eg, advertisements and links removed), and the podcast was transcribed. A dictionary of 1295 unique words was created using natural language processing (NLP) techniques (term frequency-inverse document frequency, Porter stemming, and stop-word filtering) to identify common words and phrases. Similarity between the articles and reference publications was calculated using Euclidian distance; articles were grouped using hierarchical agglomerative clustering. Nine readability scales were applied to measure text readability based on factors such as number of words, difficult words, syllables, sentence counts, and other textual metrics.

Results

The top 5 dictionary words were pregnancy (250 appearances), isotretinoin (220), study (209), drug (201), and women (185). Three distinct clusters were identified: Clusters 2 (5 articles) and 3 (4 articles) were from health-related websites and media, respectively; Cluster 1 (18 articles) contained largely media sources; 2 articles fell outside these clusters. Use of the term isotretinoin versus Accutane (a brand name of isotretinoin), discussion of pregnancy complications, and assignment of responsibility for guideline adherence varied between clusters. For example, the term pregnanc appeared most often in Clusters 1 (14.6 average times per article) and 2 (11.4) and relatively infrequently in Cluster 3 (1.8). Average readability for all articles was high (eg, Flesch-Kincaid, 13; Gunning Fog, 15; SMOG Index, 10; Coleman Liau Index, 15; Linsear Write Index, 13; and Text Standard, 13). Readability increased from Cluster 2 (Gunning Fog of 16.9) to 3 (12.2). It varied between clusters (average 13th-15th grade) but exceeded the recommended health information reading level (grade 6th to 8th), overall.

Conclusions

Media interpretation of the CNODES study varied, with differences in synonym usage and areas of focus. All articles were written above the recommended health information reading level. Analyzing media using NLP techniques can help determine drug safety communication effectiveness. This project is important for understanding how drug safety studies are taken up and redistributed in the media.

Quality of information on the Internet for Korean patients with inflammatory bowel disease

BACKGROUND/AIMS

The Internet is the main resource for health-related information. The incidence of inflammatory bowel disease (IBD) is rapidly increasing in Asian countries. However, the quality of websites for IBD available in this region has not been evaluated. We aimed to evaluate the quality of the information on IBD obtained from Korean websites.

METHODS

Using the terms "Crohn's disease" or "ulcerative colitis," websites were selected from those obtained with the three most renowned search engines in Korea; 60 websites from the results of each engine were chosen. The websites were classified into institutional, commercial, charitable, supportive, or alternative medicine types according to the characteristics of each site. The websites were evaluated regarding content quality using the validated DISCERN instrument and the Journal of the American Medical Association benchmarks.

RESULTS

The median score of all the websites according to the DISCERN instrument was 32 (interquartile range, 25 to 47) out of 80, indicating an insufficient overall quality of information. The alternative medicine sites scored the lowest, whereas the institutional sites scored the highest (p < 0.05). The quality of information was significantly different among the search engines (p = 0.028). The rank of appearance in the Google search result did not correlate with the quality level of the information.

CONCLUSION

The quality of information on the Internet regarding IBD varied according to the website type and search engine. Accreditation and quality assurance systems should be implemented for websites to ensure that the public and patients obtain accurate information on IBD.

Readability of online patient education materials for parents after a failed newborn hearing screen

OBJECTIVE

A cross-sectional study design was utilized to evaluate the readability of patient education materials on the newborn hearing screen from Google and major institutions.

METHODS

The top 55 websites from the Google search "failed newborn hearing screen" and websites from major institutions (the U.S. News & World Report ranked top 10 children's hospitals, the top 5 pediatric otolaryngology fellowships as ranked by Doximity Residency Navigator, the Centers for Disease and Control, the American Academy of Pediatrics and the American Academy of Otolaryngology-Head and Neck Surgery) were compiled. Text from each website was edited to remove extraneous text. Readability grade was calculated using the Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease Score, Gunning-Fog Index (GFI), Simple Measure of Gobbledygook (SMOG), Coleman-Liau Index, and Automated Readability Index. Intra- and inter-observer reliability were assessed.

RESULTS

26 websites from Google and 29 websites from major institutions were evaluated. From Google, provider-oriented websites (n = 2) were more difficult to read than patient-oriented websites (n = 24) with statistical significance for FKGL (p < 0.001), GFI (p < 0.013) and SMOG (p < 0.001). From the major institutions, more than half were at a reading level that exceeded the average American adult with an average FKGL 9.71 ± 2.69. Intra- and inter-observer reliability were both excellent with an intra-class correlation coefficient for each readability tool ≥0.950 (p < 0.001).

CONCLUSION

Online patient education materials about the newborn hearing screen may be too difficult for the average reader. Revisions to these materials and redirection to more readable online resources may be necessary to benefit a more inclusive patient population.

Evaluation of the scope, quality, and health literacy demand of Internet-based anal cancer information

OBJECTIVES

As there is a dearth of information about anal cancer available at cancer centers, patients often use the Internet to search for information. This is problematic, however, because the quality of information on the Internet is variable, and the health literacy demanded is higher than the average patrons' capacity. The purposes of this study were to (1) determine the most common websites with anal cancer consumer health information, (2) identify the supportive care needs that each website addresses, and (3) evaluate the websites' quality and health literacy demand.

METHODS

Medical Subject Headings (MeSH) entry terms for "Anus Neoplasms" were used in Google Canada to identify websites. Seven domains of supportive care needs were defined using Fitch's Supportive Care Framework for Cancer Care. Website quality was evaluated using the DISCERN tool. Health literacy demand was assessed using readability calculators, where best practice dictates a grade 6 or lower, and the Patient Education Material Assessment Tool (PEMAT) that computes a percentage score in 2 domains, understandability and actionability, with 80% being an acceptable score.

RESULTS

Eighteen unique websites were evaluated. One website met health literacy best practices and had a "good" quality rating. Most websites addressed only 1 supportive care domain (61%), were of "fair" quality (67%), had readability scores higher than grade 6 (89%), and had PEMAT scores ranging from 41%-92% for understandability and 0-70% for actionability.

CONCLUSION

The information gaps on anal cancer websites warrant a need for more health literate anal cancer health information on the Internet.

Assessment of Patient Knowledge and Perceptions of Pharmacogenomics Before and After Using a Mock Results Patient Web Portal

Our objective was to build a mock pharmacogenomic (PGx) patient portal and assess its ability to disseminate test results and information to patients. The YourPGx Portal delivered four sample PGx results (omeprazole, simvastatin, clopidogrel, and codeine). We hosted two study groups to assess patient knowledge and perceptions of PGx before and after accessing the portal. Ten PGx‐tested and 10 traditional care participants were included (average 61 years, 60% women, 50% African American, and 55% had a bachelor's/advanced degree). Participants scored significantly higher on the post‐test compared with the pre‐test, with no significant differences between baseline scores or score change between the groups. Patient perceptions also improved after accessing the portal—more patients wanted their providers to have access to test results, and more patients would encourage family/friends to get PGx testing. Patients would share their test results with their healthcare providers, spouse/partner, and family; none would share results with their friends or social media. Almost all patients (95%) said the portal was easy to use and 65% said it was easy to understand. In this pilot study, patients’ knowledge and perceptions of PGx improved after accessing the YourPGx Portal.

Automatic Evaluation of Cancer Treatment Texts for Gist Inferences and Comprehension

Background. It is difficult to write about cancer for laypeople such that everyone understands. One common approach to readability is the Flesch-Kincaid Grade Level (FKGL). However, FKGL has been shown to be less effective than emerging discourse technologies in predicting readability. Objective. Guided by fuzzy-trace theory, we used the discourse technology Coh-Metrix to create a Gist Inference Score (GIS) and applied it to texts from the National Cancer Institute website written for patients and health care providers. We tested the prediction that patient cancer texts with higher GIS scores are likely to be better understood than others. Design. In study 1, all 244 cancer texts were systematically subjected to an automated Coh-Metrix analysis. In study 2, 9 of those patient texts (3 each at high, medium, and low GIS) were systematically converted to fill-the-blanks (Cloze) tests in which readers had to supply the missing words. Participants (162) received 3 texts, 1 at each GIS level. Measures. GIS was measured as the mean of 7 Coh-Metrix variables, and comprehension was measured through a Cloze procedure. Results. Although texts for patients scored lower on FKGL than those for providers, they also scored lower on GIS, suggesting difficulties for readers. In study 2, participants scored higher on the Cloze task for high GIS texts than for low- or medium-GIS texts. High-GIS texts seemed to better lend themselves to correct responses using different words. Limitations. GIS is limited to text and cannot assess inferences made from images. The systematic Cloze procedure worked well in aggregate but does not make fine-grained distinctions. Conclusions. GIS appears to be a useful, theoretically motivated supplement to FKGL for use in research and clinical practice.

Readability of Patient Education Materials in Physical Medicine and Rehabilitation (PM&R): A Comparative Cross-Sectional Study

BACKGROUND

Patients are increasingly using the Internet to access health information. Patient awareness and education are crucial to advancing the field of PM&R, but many U.S. adults have insufficient health literacy skills to read and understand patient education materials (PEM), frequently written at the 10th-15th-grade level. Reading ability is key for health literacy, but no previous research has assessed the readability of PEM provided by professional PM&R societies.

OBJECTIVES

Evaluate whether the readability of PM&R PEM meets the NIH-recommended eighth-grade reading level; compare readability of PM&R PEM to two commonly accessed patient resources for sports and rehabilitation medicine topics, handouts from the American Academy of Orthopaedic Surgeons (AAOS) and American Academy of Family Physicians (AAFP), whose readability has been previously analyzed.

DESIGN

Cross-sectional study.

METHODS

Publicly accessible entries within the patient education section of websites sanctioned by professional PM&R societies, as well as the AAOS and AAFP, were analyzed for readability using two validated and widely used tools, the Flesch-Kincaid Grade Level (FKGL) and Simplified Measure of Gobbledygook (SMOG) formulas. Comparative statistics were performed between the three surveyed specialties.

MAIN OUTCOME MEASURES

FKGL and SMOG readability scores, which estimate U.S. grade level, or years of education, needed to comprehend text.

RESULTS

A total of 167 online PM&R resources were identified and compared to 94 articles from AAOS and 65 from AAFP. Mean SMOG and FKGL levels exceeded the eighth-grade level for both PM&R (SMOG-9.71, 95% CI 9.42-10.0; FKGL-10.35, 95% CI 9.99-10.7) and AAOS (SMOG-9.15, 95% CI 8.96-9.35; FKGL-9.51, 95% CI 9.29-9.74), whereas AAFP met readability guidelines for both measures (SMOG-7.00, 95% CI 6.74-7.27; FKGL-6.76, 95% CI 6.45-7.07). SMOG and FKGL scores suggested significantly higher reading difficulty for PM&R compared to AAOS (SMOG P = .017; FKGL P = .0001) and AAFP (SMOG P < .0001; FKGL P < .0001). Results indicated that 17% of PM&R resources complied with NIH guidelines, vs 8% for AAOS and 83% for AAFP.

CONCLUSIONS

The average readability of PM&R PEM exceeds the NIH-recommended and average U.S. adult eighth-grade reading level. The physiatry community can make its patient materials more comprehensible and accessible for patients by providing resources at a more appropriate reading level.

Assessment of Use, Specificity, and Readability of Written Clinical Informed Consent Forms for Patients With Cancer Undergoing Radiotherapy

Importance

Appropriate informed consent processes are crucial to preservation of patient autonomy and shared decision making. Although half of patients with cancer receive radiotherapy, it is unknown whether current consent practices are comprehensible for patients.

Objective

To characterize use, specificity, and readability of clinical informed consent forms for radiotherapy, hypothesizing that forms would be higher than the recommended sixth- to eighth-grade readability level.

Design, Setting, and Participants

This nationwide cross-sectional survey study and readability analysis was conducted from 2016 to 2018 and included 89 academic radiation oncology departments that were part of the 2016 Electronic Residency Application Service. Department leaders (clinical directors, chairs, and personal contacts of study authors) at academic radiation oncology departments were contacted via email.

Main Outcomes and Measures

Readability levels were measured by 7 validated readability indices, including the Ford, Caylor, Sticht (FORCAST) index for nonnarrative texts. Difficult words were identified using The Living Word Vocabulary, which describes the readability grade levels of 40 000 common words.

Results

Of 89 departments, 67 (75%) responded to questions and 57 (64%) provided 113 forms for analysis. Departments providing forms did not differ substantially from others in terms of region, residency size, research output, rural vs urban location, or public vs private institution status. All departments obtained patient written informed consent before radiotherapy; 38 (57%) used body site-specific forms. Using the most conservative (low-score) estimate, mean form readability ranged from grade level 10.6 to 14.2. By 7 distinct indices, only 9 (8%) of 113 forms met the recommended eighth-grade readability level, and 4 (4%) forms met a sixth-grade level. Not a single form met either recommendation based on the FORCAST index. Forms used an average of 7.2 difficult words. Body site-specific forms had considerably better readability than general consent forms.

Conclusions and Relevance

This nationwide study of informed consent practices for cancer treatment with radiotherapy demonstrates that while all US academic radiotherapy departments use written consent forms, it is rare for templates to meet the recommended readability levels for patient materials. These data suggest the need for reevaluation and modification of the approach to radiotherapy consent, ideally with guidance and templates designed by national professional organizations.

Readability of Patient Education Materials From RadiologyInfo.org: Has There Been Progress Over the Past 5 Years?

OBJECTIVE. As patients increasingly turn to the Internet for healthcare information, it is imperative that patient educational materials be written at an appropriate readability level. Although RadiologyInfo.org, a patient education library sponsored by the American College of Radiology (ACR) and Radiological Society of North America, was shown in 2012 to be written at levels too high for the average patient to adequately comprehend, it is unclear if there has been progress made in the past 5 years. The purpose of this study was to provide a 5-year update on the readability of patient education materials from RadiologyInfo.org. MATERIALS AND METHODS. All patient education articles available in 2017 from the ACR and RSNA-sponsored RadiologyInfo.org patient education library were reviewed. We assessed each article for readability using 6 quantitative readability scales: the Flesch-Kincaid (FK) grade level, Flesch Reading Ease, Gunnin-Fog Index, Coleman-Liau Index, Automated Readability Index, and the Simple Measure of Gobbledygook (SMOG). The number of articles with readability ≤ the 8^th grade level (average reading ability of US adults) and the 6^th-grade level (NIH-recommended level for patient materials) were determined. RESULTS. 131 patient education articles were reviewed. The mean readability grade level was greater than the 11^th grade reading level for all readability scales. None of the articles were written at less than the 8^th-grade or the 6^th-grade levels. CONCLUSION. Although there has been an increasing awareness of the issue of readability of patient educational materials within the radiological community, the patient educational materials within the ACR and RSNA-sponsored RadiologyInfo.org website are still written at levels too high for the average patient. Future efforts should be made to improve the readability of those patient education materials.

Impact of an electronic medium delivery of warfarin education in a low income, minority outpatient population: a pilot intervention study

Background

Warfarin is classified as a high-alert medication for ambulatory healthcare and safe guards for high-alert medications are necessary, including the practice of mandatory patient education. The high cost of hospitalizations related to adverse events combined with the average bleeding event rate of 7–8% in spite of routine patient education, suggests the importance of new approaches to standardized health education on warfarin. We sought to evaluate the impact of a warfarin educational video using an electronic tablet on patient knowledge and to determine patients’ satisfaction with the use of an electronic tablet for educational purposes in outpatient clinics serving a low income, minority population.

Methods

A warfarin educational video delivered on an electronic tablet (iPad) was delivered at two pharmacist-managed anticoagulation clinics to uninsured patients whose annual income is equal or less than two hundred percent below the poverty level were offered. Patients (n = 18) completed a pre-video and post-video knowledge test on warfarin before and after viewing the warfarin educational video on an electronic tablet and a follow-up test to measure the retention of knowledge and a patient satisfaction survey at 60 days. The primary outcome was change in knowledge test scores. Other outcome measures included adherence rates, adverse events, time in therapeutic INR range, and patient-reported satisfaction scores.

Results

The majority of patients were uninsured men taking warfarin for atrial fibrillation (n = 5). The median scores at post-video knowledge test and follow-up knowledge test were significantly higher than that for the pre-knowledge test (12 (11–12) vs. 10(8–11), p < 0.001). The study group had a ‘time in therapeutic INR’ range of 56.3%, a rate of adverse events of 24.5%, and a self-reported adherence rate to warfarin of 94.1%. The majority of patients also had positive responses to the patient satisfaction survey.

Conclusions

Patient education delivered via iPad to facilitate knowledge of medication can serve as a useful tool for educating patients about warfarin and warfarin therapy. Use of an electronic medium may be a unique way to provide standard medication education to patients.

Trial registration

The study was retrospectively registered with: NCT03650777; 9/18/18.

Computer-Based Readability Testing of Information Booklets for German Cancer Patients

Understandable health information is essential for treatment adherence and improved health outcomes. For readability testing, several instruments analyze the complexity of sentence structures, e.g., Flesch-Reading Ease (FRE) or Vienna-Formula (WSTF). Moreover, the vocabulary is of high relevance for readers. The aim of this study is to investigate the agreement of sentence structure and vocabulary-based (SVM) instruments. A total of 52 freely available German patient information booklets on cancer were collected from the Internet. The mean understandability level L was computed for 51 booklets. The resulting values of FRE, WSTF, and SVM were assessed pairwise for agreement with Bland-Altman plots and two-sided, paired t tests. For the pairwise comparison, the mean L values are L_FRE = 6.81, L_WSTF = 7.39, L_SVM = 5.09. The sentence structure-based metrics gave significantly different scores (P < 0.001) for all assessed booklets, confirmed by the Bland-Altman analysis. The study findings suggest that vocabulary-based instruments cannot be interchanged with FRE/WSTF. However, both analytical aspects should be considered and checked by authors to linguistically refine texts with respect to the individual target group. Authors of health information can be supported by automated readability analysis. Health professionals can benefit by direct booklet comparisons allowing for time-effective selection of suitable booklets for patients.

Readability assessment of online gynecologic oncology patient education materials from major governmental, non-profit and pharmaceutical organizations

OBJECTIVE

Patients are increasingly using online materials to learn about gynecologic cancer. Providers can refer patients to online educational materials produced by a number of different major medical organizations and pharmacology companies. The National Institutes of Health (NIH) and the American Medical Association (AMA) recommend that patient educational materials (PEMs) are written between a sixth and eighth grade reading level. In this study, we assess the readability of online PEMs published by major medical organizations and industry partners.

METHODS

Websites from twelve websites providing educational materials for gynecologic oncology patients were surveyed. Online PEMs were identified and analyzed using seven validated readability indices. One-way ANOVA and Tukey's Honestly Significant Difference (HSD) post-hoc analysis were performed to detect differences in readability between publishers.

RESULTS

Two-hundred and sixty PEMs were included in this analysis. Overall, PEMs were written at a mean 11th±0.6 grade reading level. Only 6.5% of articles were written at the AMA/NIH recommended reading grade level of 6th to 8th grade or below. ANOVA demonstrated a significant difference in readability between publishing associations (p<0.01). PEMs from the Centers for Disease Control had a mean 9th±1.2 grade reading level and were significantly lower than all other organizations. PEMs from The Foundation for Women's Cancer had a mean 13th±1.8 grade reading level and were significantly higher than most other organizations. PEMs from pharmaceutical companies (mean readability=10.1±1.1, N=30) required the lowest reading grade level and were significantly more readable than those from governmental organizations (11.1±1.7, p<0.05) and nonprofit medical associations (12.4±1.7, p<0.01) in ANOVA and Tukey-Kramer post hoc analysis.

CONCLUSIONS

Gynecologic oncology PEMs available from twelve major organization websites are written well above the recommended sixth to eighth grade reading difficulty level.

Readability of Individuals With Disabilities Education Act Part B Procedural Safeguards: An Update

Purpose

The purpose of this study was to analyze the readability of Individuals with Disabilities Education Act (IDEA) Part B procedural safeguard documents, as distributed by each of the 50 U.S. states. Results were compared to the 5th- to 6th-grade readability guideline for documents recommended by experts in health literacy and health communication.

Method

A commercially available readability software, Readability Studio ( Oleander Software, 2009 ), was used to assess document readability. Text-based files of each IDEA Part B procedural safeguard document were analyzed using four readability formulas: Flesch-Kincaid ( Flesch, 1965 ), Gunning fog index ( Gunning, 1952 ), Flesch Reading Ease ( Flesch, 1948 ), and Simple Measure of Gobbledygook ( McLaughlin, 1969 ).

Results

No procedural safeguard document scored below an 11th-grade reading level. Seventy-four percent of these documents were found to be written at a graduate reading level—meaning these documents are written for a reader who is currently enrolled in a master's degree or higher education program.

Conclusion

In an effort to decrease barriers to parent participation in the Individualized Education Planning process, those who administer IDEA Part B procedural safeguards should be sensitive to the potential mismatch between the literacy skills of the parent/guardian and the literacy skills needed to comprehend these documents. Developers of IDEA Part B procedural safeguards should account for estimated literacy skills of the general public as ongoing revisions to these safeguards are made.

Implementing a health literacy module fostering patient-centered written communication in a cardiovascular prevention elective course

BACKGROUND

A universal approach to health literacy employs clear communication and emphasizes patient action in support of understanding cardiovascular risks and making healthy lifestyle changes. The aim of this project was to evaluate the impact of a health literacy module on enhancing students' written patient education material using standardized readability assessment tools.

EDUCATIONAL ACTIVITY

A professional elective course employed team-based learning and a community health fair activity. The course was enhanced with four hours of health literacy content. Pharmacy student learners were assigned a cardiovascular condition and designed an informational pamphlet. Two faculty members evaluated the student pamphlets in the pre-health literacy module (n = 15) and post-module (n = 23) groups using the Flesch grade level, Flesch reading ease score, and 15 health literacy criteria identified from previous literature.

CRITICAL ANALYSIS OF THE EDUCATIONAL ACTIVITY

There was a modest integration of health literacy concepts based on the Flesch grade level and the mean total health literacy criteria achieved. Student learners improved in areas of readability, message content, numeracy/statistics, and patient actionability concepts. Simplifying technical jargon remains to be a barrier. The results will be used to improve our health literacy pamphlet rubric and classroom instruction. The health literacy module was valuable in fostering understanding and application of health literacy concepts, and preparing student learners for providing patient-centered communication.

The Easier the Better? Comparing the Readability and Engagement of Online Pro- and Anti-Vaccination Articles

Online anti-vaccine articles contribute to the anti-vaccine movement, which leads to recent outbreaks of vaccine-preventable diseases. Previous studies indicate that anti-vaccine articles are easy to read and understand, which may increase their abilities to engage viewers. The present study aims to examine if readability levels are related to engagement. Using combination of terms to search for vaccine articles in Google in May 2017, this study examined 541 pro-vaccine online articles with a total of 508,571 words and 382 anti-vaccine articles with a total of 843,805 words. Almost all vaccine articles exceeded the American average reading comprehension level. No significant difference in readability was found between pro- and anti-vaccine articles. Pro-vaccine articles that could only be understood by college graduates were less engaging than those with lower readability levels. No significant relationship between anti-vaccine articles’ readability and engagement was discovered. Different vaccine topics had different readability and engagement levels, which implied that certain combinations of themes and readability levels could enhance the health messages’ persuasion effect. Recommendations for designing effective health messages are provided.

Can Patients and Families Read the Questionnaires for Patient-related Outcome Measures?

BACKGROUND

Patient-reported Outcome Measures (PROMs) quantifying health-related quality of life are widely used to evaluate illness and the impact of health care interventions. The questionnaires must be comprehensible by the persons completing them. Literacy Surveys in the United States reveal nearly half of the US adult population have a literacy level below the 8th grade. Recommendations are that health-related written material directed at adults should be at a reading level of grade 6 or below. Reading level for materials aimed at minor patients should be no higher than their grade level. Our aim was to determine the readability of pediatric orthopaedic PROMs as determined by a validated literacy tool.

METHODS

A literature search was conducted to identify PROMs cited in pediatric orthopaedics. In total, 79 PROMs were identified. Only text-based PROMs that were developed in English for pediatrics and have published evidence of validation were included, leaving 35 outcome scores for analysis. Text was extracted and analyzed using the Flesch Reading Ease Score (FRES).

RESULTS

Of 35 PROMs: 15 (43%) were general, 12 (34%) spine, 5 (14%) lower, and 3 (9%) upper extremity. Ten (29%) were designed to be completed by the caregiver, whereas 25 (71%) by the patient. Ten PROMs intended for completion by a parent had an FRES readability of 8th to 10th grade level with only 2 of 10 corresponding to that would be understood by the average adult. Of the 25 PROMs intended for completion by pediatric patients, only 4 (16%) had a FRES corresponding with the age group intended to complete them.

CONCLUSIONS

The majority of PROMs are written at a level of complexity higher than likely to be understood by the persons intended to complete them. This is of significance, given that researchers and practitioners alike rely on data from PROMs in drawing conclusions about the impact of orthopaedic conditions and their treatments on health-related quality of life. Analysis of easily comprehended PROMs will aid in the design of future scores to maximize their utility in research and clinical practice.

LEVEL OF EVIDENCE

Level II-decision analysis study.