Systematic Assessment Reveals Lack of Understandability for Prostate Biopsy Online Patient Education Materials

Can Patients Read, Understand, and Act on Online Resources for Anterior Cruciate Ligament Surgery?

Background:

Patients undergoing elective procedures often utilize online educational materials to familiarize themselves with the surgical procedure and expected postoperative recovery. While the Internet is easily accessible and ubiquitous today, the ability of patients to read, understand, and act on these materials is unknown.

Purpose:

To evaluate online resources about anterior cruciate ligament (ACL) surgery utilizing measures of readability, understandability, and actionability.

Study Design:

Cross-sectional study; Level of evidence, 4.

Methods:

Using the term “ACL surgery,” 2 independent searches were performed utilizing a public search engine (Google.com). Patient education materials were identified from the top 50 results. Audiovisual materials, news articles, materials intended for advertising or medical professionals, and materials unrelated to ACL surgery were excluded. Readability was quantified using the Flesch Reading Ease, Flesch-Kincaid Grade Level, Simple Measure of Gobbledygook, Coleman-Liau Index, Automated Readability Index, and Gunning Fog Index. The Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P) was utilized to assess the actionability and understandability of materials. For each online source, the relationship between its Google search rank (from first to last) and its readability, understandability, and actionability was calculated utilizing the Spearman rank correlation coefficient (ρ_S).

Results:

Overall, we identified 68 unique websites, of which 39 met inclusion criteria. The mean Flesch-Kincaid Grade Level was 10.08 ± 2.34, with no website scoring at or below the 6th-grade level. Mean understandability and actionability scores were 59.18 ± 10.86 (range, 33.64-79.17) and 34.41 ± 22.31 (range, 0.00-81.67), respectively. Only 5 (12.82%) and 1 (2.56%) resource scored above the 70% adequate PEMAT-P threshold mark for understandability and actionability, respectively. Readability (lowest P value = .103), understandability (ρ_S = –0.13; P = .441), and actionability (ρ_S = 0.28; P = .096) scores were not associated with Google rank.

Conclusion:

Patient education materials on ACL surgery scored poorly with respect to readability, understandability, and actionability. No online resource scored at the recommended reading level of the American Medical Association or National Institutes of Health. Only 5 resources scored above the proven threshold for understandability, and only 1 resource scored above it for actionability.

Internet-Based Patient Education Materials Regarding Diabetic Foot Ulcers: Readability and Quality Assessment

BACKGROUND

While diabetic foot ulcers (DFU) are a common complication of diabetes, little is known about the content and readability of online patient education materials (PEM) for DFU. The recommended reading grade level for these materials is grades 6-8.

OBJECTIVE

The aim of this paper was to evaluate the quality and readability of online PEM on DFU.

METHODS

A Google search was performed using 4 different search terms related to DFU. Two readability formulas were used to assess the readability of the included PEM. These included the Flesch-Kincaid grade level and the Flesch-Reading ease score. The DISCERN tool was used to determine quality and reliability.

RESULTS

A total of 41 online PEM were included. The average Flesch-Reading ease score for all PEM was 63.43 (SD 14.21), indicating a standard difficulty level of reading. The average reading grade level was 7.85 (SD 2.38), which is higher than the recommended reading level for PEM. The mean DISCERN score was 45.66 (SD 3.34), and 27% (11/41) of the articles had DISCERN scores of less than 39, corresponding to poor or very poor quality.

CONCLUSIONS

The majority of online PEM on DFU are written above the recommended reading levels and have significant deficiencies in quality and reliability. Clinicians and patients should be aware of the shortcomings of these resources and consider the impact they may have on patients' self-management.

Factors Affecting Engagement in Web-Based Health Care Patient Information: Narrative Review of the Literature

BACKGROUND

Web-based content is rapidly becoming the primary source of health care information. There is a pressing need for web-based health care content to not only be accurate but also be engaging. Improved engagement of people with web-based health care content has the potential to inform as well as influence behavioral change to enable people to make better health care choices. The factors associated with better engagement with web-based health care content have previously not been considered.

OBJECTIVE

The aims of this study are to identify the factors that affect engagement with web-based health care content and develop a framework to be considered when creating such content.

METHODS

A comprehensive search of the PubMed and MEDLINE database was performed from January 1, 1946, to January 5, 2020. The reference lists of all included studies were also searched. The Medical Subject Headings database was used to derive the following keywords: "patient information," "online," "internet," "web," and "content." All studies in English pertaining to the factors affecting engagement in web-based health care patient information were included. No restrictions were set on the study type. Analysis of the themes arising from the results was performed using inductive content analysis.

RESULTS

The search yielded 814 articles, of which 56 (6.9%) met our inclusion criteria. The studies ranged from observational and noncontrolled studies to quasi-experimental studies. Overall, there was significant heterogeneity in the types of interventions and outcome assessments, which made quantitative assessment difficult. Consensus among all authors of this study resulted in six categories that formed the basis of a framework to assess the factors affecting engagement in web-based health care content: easy to understand, support, adaptability, accessibility, visuals and content, and credibility and completeness.

CONCLUSIONS

There is a paucity of high-quality data relating to the factors that improve the quality of engagement with web-based health care content. Our framework summarizes the reported studies, which may be useful to health care content creators. An evaluation of the utility of web-based content to engage users is of significant importance and may be accessible through tools such as the Net Promoter score. Web 3.0 technology and development of the field of psychographics for health care offer further potential for development. Future work may also involve improvement of the framework through a co-design process.

Quality of Carpal Tunnel Syndrome Patient Education Handouts Available on the Internet: A Systematic Analysis of Content and Design

OBJECTIVE

To evaluate the quality of carpal tunnel syndrome (CTS) patient education handouts and identify the best resources for patients and clinicians.

DESIGN

A document content analysis of handouts identified through a systematic internet search using 8 search terms on Google and Bing and a hand search of professional association websites.

SETTING

Not applicable.

PARTICIPANTS

Documents (N=56) were identified from the top 50 search results across 16 individual searches. Included documents provided general patient education for CTS; descriptive websites, videos, and research studies were excluded.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

Content analysis was conducted using the Information Score (IS) tool to evaluate completeness of information (0%-100%) and misleading treatment recommendations were identified. Design analysis was conducted using the Patient Material Assessment Tool for Printable Materials (PEMAT-P) (0%-100%) and 2 widely used readability formulas, Flesch Reading Ease and Flesch Kincaid Grade Level. Using these results, all handouts were rated with a summative 12-point scale.

RESULTS

Of 805 unique search results, we included 56 CTS handouts. The average IS was 74.6%±17.9%, and 78.6% of the handouts mentioned non-evidence-based treatment recommendations. The average PEMAT-P score was 70.2%±10.9%, and the average readability grade level was 7.7±1.7. Only 3 handouts were identified as high quality based on the 12-point summative scores, 22 handouts had mixed quality, and 17 handouts had low quality on both content and design.

CONCLUSIONS

Findings of this study suggest a lack of high-quality and easily understandable CTS patient education handouts. Most handouts contained unreliable treatment information. Improvements are needed to ensure patients' ability to understand and manage this condition.

Developing Strategies for Targeted Improvement of Perioperative Education for Postbariatric Surgery Body-Contouring Patients

BACKGROUND

The quality of perioperative patient education impacts surgical outcomes, patient experiences, and resources needed to address patient concerns and unplanned visits. We examined patient inquiries and education materials to assess the quality of perioperative education and identify areas of targeted improvement for postbariatric surgery body-contouring procedures.

METHODS

We examined 100 consecutive postbariatric procedures at an academic center. Themes of patient-generated calls, e-mails, and electronic medical record portal messages during the perioperative period were identified via qualitative analysis. Understandability and actionability of perioperative educational resources were assessed using the Patient Education Materials Assessment Tool (PEMAT).

RESULTS

Among 212 communications identified, 167 (79%) were postoperative. Common themes were concerns regarding the surgical site (38%), medications (10%), and activity restrictions (10%). One hundred thirty inquiries were resolved through patient re-education (57%), but 36 (16%) required in-person evaluation including 4 unplanned emergency department visits and 3 readmissions for surgical-site concerns. The PEMAT scores for institutional materials were fair for understandability (69%) and actionability (60%). American Society of Plastic Surgeons materials were more understandable (84%) but less actionable (40%).

CONCLUSIONS

Patient queries can be leveraged as a source of qualitative data to identify gaps in perioperative education. High-yield topics, such as education regarding the surgical site and medications, can be targeted for quality improvement through better communication and potentially reduce the number of unnecessary visits. Using the PEMAT, we also identified how directly the education materials can be revised. Improving perioperative education can promote mutual understanding between patients and surgeons, better outcomes, and efficient resource utilization.

Availability and Quality of Web Resources for Parents of Children With Disability: Content Analysis and Usability Study

BACKGROUND

The internet is a valuable resource for parents of typical children, who are looking for information about their children's growth and development and how to boost them. However, for parents of children with special needs, especially for non-English-speaking parents, there are anecdotal reports stating that specific and accurate information is not available on the internet.

OBJECTIVE

This study aims to describe the type of information available on the internet for French-speaking parents of children with disability as well as assess the quality of the information collected.

METHODS

We carried out a search of the existing relevant websites targeted at parents of children with disability. We used a validated instrument to extract structural, textual, and visual characteristics of these websites and evaluate their usability.

RESULTS

In all, 42 websites were analyzed; of these, the information had been validated by a trustworthy source in only 18 (43%) websites. Networking opportunities for parents were available in only 7 (17%) websites. Most websites provided information related to autism spectrum disorder (20/42, 42%) and learning disabilities (19/42, 45%), and only a few websites discussed other disability types such as behavorial disorders and developmental language disorders (4/42, 10% each). Community, social, and civic life (9/42, 22%); domestic life (12/42, 29%); and mobility (15/42, 36%) were the less frequently covered topics. With regard to the usability evaluation, 22 of the 42 (52%) websites received a global score <70%, whereas 20 (48%) scored ≥70.

CONCLUSIONS

Although the internet is an infinite source of information, it is not necessarily actionable for parents of children with disability. Some information remains difficult to find online, and networking opportunities with other parents dealing with similar challenges are scarce.

Osteosarcoma Online Resources: A Quantitative Assessment of the Understandability and Readability of Web-based Patient Education Material (Preprint)

Background

Patients often turn to web-based resources following the diagnosis of osteosarcoma. To be fully understood by average American adults, the American Medical Association (AMA) and National Institutes of Health (NIH) recommend web-based health information to be written at a 6th grade level or lower. Previous analyses of osteosarcoma resources have not measured whether text is written such that readers can process key information (understandability) or identify available actions to take (actionability). The Patient Education Materials Assessment Tool (PEMAT) is a validated measurement of understandability and actionability.

Objective

The purpose of this study was to evaluate web-based osteosarcoma resources using measures of readability, understandability, and actionability.

Methods

Using the search term “osteosarcoma,” two independent Google searches were performed on March 7, 2020 (by AGS), and March 11, 2020 (by TRG). The top 50 results were collected. Websites were included if they were directed at providing patient education on osteosarcoma. Readability was quantified using validated algorithms: Flesh-Kincaid Grade Ease (FKGE), Flesch-Kincaid Grade-Level (FKGL). A higher FKGE score indicates that the material is easier to read. All other readability scores represent the US school grade level. Two independent PEMAT assessments were performed with independent scores assigned for both understandability and actionability. A PEMAT score of 70% or below is considered poorly understandable or poorly actionable. Statistical significance was defined as P≤.05.

Results

Two searches yielded 53 unique websites, of which 37 (70%) met the inclusion criteria. The mean FKGE and FKGL scores were 40.8 (SD 13.6) and 12.0 (SD 2.4), respectively. No website scored within the acceptable NIH or AHA recommended reading level. Only 4 (11%) and 1 (3%) website met the acceptable understandability and actionability threshold. Both understandability and actionability were positively correlated with FKGE (ρ=0.55, P<.001; ρ=0.60, P<.001), but were otherwise not significantly associated with other readability scores. There were no associations between readability (P=.15), understandability (P=.20), or actionability (P=.31) scores and Google rank.

Conclusions

Overall, web-based osteosarcoma patient educational materials scored poorly with respect to readability, understandability, and actionability. None of the web-based resources scored at the recommended reading level. Only 4 achieved the appropriate score to be considered understandable by the general public. Authors of patient resources should incorporate PEMAT and readability criteria to improve web-based resources to support patient understanding.

Implementation of an organisation-wide health literacy approach to improve the understandability and actionability of patient information and education materials: A pre-post effectiveness study

OBJECTIVE

Limited examples exist globally of coordinated, organisation-wide health literacy approaches to systematically improve the understandability and actionability of patient health information. Even fewer have been formally evaluated. The aim of this study was to use the Patient Education Materials Assessment Tool (PEMAT) to evaluate the effectiveness of an organisation-wide, evidence-based approach to improve the understandability and actionability of patient information materials in regional health service in New South Wales, Australia.

METHODS

Two independent raters (blinded to the document version) evaluated pre- and post-implementation versions of 50 randomly-selected patient information materials using the PEMAT, with differences in understandability and actionability analysed using paired samples tests.

RESULTS

Mean (±SD) overall scores for understandability increased significantly by 5% (95% CI 2-8; p = 0.002) up to 77%±10%, and mean actionability (±SD) increased significantly by 4% (95% CI 0-8; p = 0.046) up to 56%±22%.

CONCLUSION

These results demonstrate that organisation-wide approaches with standardised processes for staff to prepare, review and store written patient information and education materials can be successfully implemented to address the impacts and risks of low health literacy.

PRACTICE IMPLICATIONS

The success of this approach provides a framework for other health organisations to work in partnership with patients to make health information more understandable and actionable.