Readability of Patient Education Materials in Hand Surgery and Health Literacy Best Practices for Improvement

Optimizing Readability of Patient-Facing Hand Surgery Education Materials Using Chat Generative Pretrained Transformer 3.5

PURPOSE

To address patient health literacy, the American Medical Association and the National Institutes of Health recommend that readability of patient education materials should not exceed an eighth grade reading level. However, patient-facing materials often remain above the recommended average reading level. Current online calculators provide readability scores; however, they lack the ability to provide text-specific feedback, which may streamline the process of simplifying patient materials. The purpose of this study was to evaluate Chat Generative Pretrained Transformer (ChatGPT) 3.5 as a tool for optimizing patient-facing hand surgery education materials through reading level analysis and simplification.

METHODS

The readability of 18 patient-facing hand surgery education materials was compared by a traditional online calculator for reading level and ChatGPT 3.5. The original excerpts were then entered into ChatGPT 3.5 and simplified by the artificial intelligence tool. The simplified excerpts were scored by the same calculators.

RESULTS

The readability scores for the original excerpts from the online calculator and ChatGPT 3.5 were similar. The simplified excerpts' scores were lower than the originals, with a mean of 7.28, less than the maximum recommended 8.

CONCLUSIONS

The use of ChatGPT 3.5 for the purpose of simplification and readability analysis of patient-facing hand surgery materials is efficient and may help facilitate the conveyance of important health information. ChatGPT 3.5 rendered readability scores comparable with traditional readability calculators, in addition to excerpt-specific feedback. It was also able to simplify materials to the recommended grade levels.

CLINICAL RELEVANCE

By confirming ChatGPT3.5's ability to assess and simplify patient education materials, this study offers a practical solution for potentially improving patient comprehension, engagement, and health outcomes in clinical settings.

Educational quality of YouTube content on brachial plexus injury and treatment

INTRODUCTION

While surgical literature exists on birth-related brachial plexus injury (BPI), there are not validated sources of information on BPI for patients, which can impact patient autonomy and decision-making. With YouTube as a popular source for patients to research diagnoses, this study aims to evaluate the quality of information regarding BPI and its treatment available on the platform.

MATERIALS AND METHODS

BPI YouTube videos were screened independently by two reviewers. Videos were categorized by source: (1) academic, government, and non-profit organizations; (2) private practices, companies, and for-profit organizations; (3) independent users. Each video was evaluated for reliability, credibility, and quality using the modified DISCERN criteria (scale, 0-5), Journal of the American Medical Association (JAMA) criteria (scale, 0-4), and Global Quality Scale (GQS; scale, 1-5). Surgical treatment videos were analyzed by the senior author using a modified "treatment" DISCERN criteria (scale, 8-40). Non-English videos were excluded from this study. Analysis of variance tests were used to compare means.

RESULTS

One hundred and fifteen videos were selected for final analysis. The mean modified DISCERN score was 3.26; JAMA was 2.31; GQS was 3.48. Videos were subdivided according to source. Group 1, 2, and 3 had 45, 24, and 46 videos, respectively. Modified DISCERN score was greater for Group 1 than Group 2 (3.58 vs. 3.04, p < .001) and Group 3 (3.58 vs. 3.07, p < .001). JAMA score was greater for Group 1 than Group 2 (2.63 vs 2.15, p = .041) and Group 3 (2.63 vs. 2.08, p = .002). GQS score was greater for Group 1 than Group 2 (3.93 vs. 3.31, p = .031) and Group 3 (3.93 vs. 3.13, p < .001). Of the 34 videos (44.7%) that mentioned treatment, the DISCERN score was 14.32.

CONCLUSION

The videos analyzed were found to have moderate reliability, credibility, and quality. The reliability of information regarding treatments for BPI was poor. Healthcare providers should supply additional information on treatment of BPI.

Internet Search Trends for Common Hand Surgery Diagnoses

PURPOSE

The internet is a common resource for patients seeking health information. Trends in internet search interests for common hand surgery diagnoses and their seasonal variations have not been previously studied. The objectives of this study were (1) to describe the temporal trends in internet search interest for common hand surgery diagnoses in the recent five-year time period and (2) to assess seasonal variations in search term interest.

METHODS

An internet-based study of internet search term interest of 10 common hand surgery diagnoses was performed using Google Trends (Google, Inc., Mountain View, CA) from January 2017 to December 2021. The 10 diagnoses were "carpal tunnel syndrome," "trigger finger," "thumb arthritis," "ganglion cyst," "de Quervain's tenosynovitis," "lateral epicondylitis," "Dupuytren disease," "distal radius fracture," "finger fracture," and "scaphoid fracture." Analysis of variance (ANOVA) was used to assess for seasonal differences in search interest, and temporal trends were assessed using the two-tailed Mann-Kendall trend test.

RESULTS

During the study period, there was an increasing trend for search interest for "carpal tunnel syndrome," "trigger finger," "thumb arthritis," "Dupuytren disease," and "finger fracture," both in the United States and worldwide. There was no significant temporal trend for "ganglion cyst," "de Quervain's tenosynovitis," "lateral epicondylitis," and "distal radius fracture." There were no significant temporal trend for "scaphoid fracture" in the United States and a decreasing trend worldwide. There was significant seasonal variation in search term interest for "finger fracture" in the United States, "finger fracture" worldwide, and "scaphoid fracture" in the United States, with popularity peaking in the fall.

CONCLUSIONS

Despite growth in global internet usage, internet search interest has remained stagnant for many common hand surgery conditions, which may represent a shifting preference for patients to obtain health information from other resources. Internet search interest for traumatic hand conditions corresponds to seasonal variations in fracture epidemiology and peaks in the fall season.

Evaluation of Online Artificial Intelligence-Generated Information on Common Hand Procedures

PURPOSE

The purpose of this study was to analyze the quality and readability of the information generated by an online artificial intelligence (AI) platform regarding 4 common hand surgeries and to compare AI-generated responses to those provided in the informational articles published by the American Society for Surgery of the Hand (ASSH) HandCare website.

METHODS

An open AI model (ChatGPT) was used to answer questions commonly asked by patients on 4 common hand surgeries (carpal tunnel release, cubital tunnel release, trigger finger release, and distal radius fracture fixation). These answers were evaluated for medical accuracy, quality and readability and compared to answers derived from the ASSH HandCare materials.

RESULTS

For the AI model, the Journal of the American Medical Association benchmark criteria score was 0/4, and the DISCERN score was 58 (considered good). The areas in which the AI model lost points were primarily related to the lack of attribution, reliability and currency of the source material. For AI responses, the mean Flesch Kinkaid Reading Ease score was 15, and the Flesch Kinkaid Grade Level was 34, which is considered to be college level. For comparison, ASSH HandCare materials scored 3/4 on the Journal of the American Medical Association Benchmark, 71 on DISCERN (excellent), 9 on Flesch Kinkaid Grade Level, and 60 on Flesch Kinkaid Reading Ease score (eighth/ninth grade level).

CONCLUSION

An AI language model (ChatGPT) provided generally high-quality answers to frequently asked questions relating to the common hand procedures queried, but it is unclear when or where these answers came from without citations to source material. Furthermore, a high reading level was required to comprehend the information presented. The AI software repeatedly referenced the need to discuss these questions with a surgeon, the importance of shared decision-making and individualized care, and compliance with surgeon treatment recommendations.

CLINICAL RELEVANCE

As novel AI applications become increasingly mainstream, hand surgeons must understand the limitations and ramifications these technologies have for patient care.

Health literacy in rotator cuff repair: a quantitative assessment of the understandability of online patient education material

Background

The American Medical Association and National Institutes of Health recommend online health information be written at a 6th grade or lower reading level for clear understanding. While syntax reading grade level has previously been utilized, those analyses do not determine whether readers are processing key information (understandability) or identifying available actions to take (actionability). The Patient Education Materials Assessment Tool (PEMAT-P) is a method to measure the understandability and actionability of online patient education materials. The purpose of this study was to evaluate online resources regarding rotator cuff repair utilizing measures of readability, understandability, and actionability.

Methods

The search term "rotator cuff surgery" was used in two independent online searches to obtain the top 50 search results. The readability of included resources was quantified using valid objective algorithms: Flesch-Kincaid Grade-Level, Simple Measure of Gobbledygook grade, Coleman-Liau Index, and Gunning Fog Index. The PEMAT-P form was used to assess actionability and understandability.

Results

A total of 49 unique websites were identified to meet our inclusion criteria and were included in our analysis. The mean Flesch-Kincaid Grade Level graded materials at a 10.6 (approximately a 10th grade reading level), with only two websites offering materials at a 6th grade reading level or below. The remaining readability studies graded the mean reading level at high school or greater, with the Gunning Fog Index scoring at a collegiate reading level. Mean understandability and actionability scores were 64.6% and 29.5%, respectively, falling below the 70% PEMAT score threshold for both scales. Fourteen (28.6%) websites were above the threshold for understandability, while no website (0%) scored above the 70% threshold for actionability. When comparing source categories, commercial health publishers provided websites that scored higher in understandability (P < .05), while private practice materials scored higher in actionability (P < .05). Resources published by academic institutions or organizations scored lower in both understandability and actionability than private practice and commercial health publishers (P < .05). No readability, understandability, or actionability score was significantly associated with search result rank.

Conclusion

Overall, online patient education materials related to rotator cuff surgery scored poorly with respect to readability, understandability, and actionability. Only two (4.1%) of the patient education websites scored at the American Medical Association and National Institutes of Health recommended reading level. Fourteen (28.6%) scored above the 70% PEMAT score for understandability; however, no website met the threshold for actionability.

Statistical Literacy in Hand and Upper-Extremity Patients

Purpose

Statistical literacy is the ability of a patient to apply basic statistical concepts to their health care. Understanding statistics is a critical component of shared decision making. The purpose of this investigation was to define levels of statistical literacy in an upper-extremity (UE) patient population. We aimed to determine if patient demographics would be associated with statistical literacy.

Methods

An electronic survey was administered to a consecutive series of UE patients at a single institution. We recorded baseline demographics, Single Assessment Numeric Evaluation scores, the Berlin Numeracy Test (BNT), and General Health Numeracy Test. We also included a surgical risk question, which asked: "Approximately 3% of patients who get carpal tunnel surgery develop an infection. If 100 patients get this surgery, how many would you expect to develop an infection?" A covariate-controlled adjusted odds ratio reflecting the association between each statistical literacy outcome measure and patient characteristics was reported.

Results

A total 254 surveys were administered, 148 of which were completed and included. Fifty percent of respondents had a high-school education or less. For the BNT, 78% scored in the bottom quartile, and 52% incorrectly answered all questions. For the General Health Numeracy Test, 34% answered 0 or 1/6 questions correctly. For the surgical risk question, 24% of respondents answered incorrectly. Respondents who had a college or graduate degree had 2.62 times greater odds (95% confidence interval, 1.09-6.32) of achieving a BNT score in a higher quartile than patients who did not have a college or graduate degree.

Conclusions

Overall levels of statistical literacy are low for UE patients.

Clinical relevance

When engaging in management discussions and shared decision making, UE surgeons should assume low levels of statistical literacy. Consideration of alternative formats, such as frequencies, video-based materials, and pictographs, may be warranted when discussing outcomes and risks of surgical procedures.

Health Literacy in Oculofacial Plastic Surgery: A Literature Review

Patient satisfaction following oculofacial cosmetic procedures depends on preoperative expectations, which may be influenced by online material. Patients with poor health literacy are particularly vulnerable to misinformation and low-quality resources. However, few studies have evaluated the quality of online information on common oculofacial plastic surgeries and procedures. This study aimed to review the literature on the readability and quality of online material related to oculofacial plastic surgery. We conducted a systematic search of the PubMed/MEDLINE database and included 10 studies in our review. Among the readability scores reported in these studies, the lowest was 10, representing a tenth-grade reading level. Furthermore, the online materials were often rated as "poor" quality based on multiple grading scales. Our systematic review of the literature demonstrates that online materials covering common oculofacial plastic surgery procedures are consistently of poor quality and exceed the recommended readability level. Therefore, considering these online materials that influence patient expectations could enable oculofacial plastic surgeons to better tailor their preoperative counseling.

Readability of Online Hand and Upper Extremity Patient Resources

Background Online patient resources regarding hand and upper extremity topics published by professional societies are written at a level that exceeds that of the average reader. Methodology Online patient resources focused on hand and upper extremity topics published by the American Society for Surgery of the Hand (ASSH), the American Association for Hand Surgery (AAHS), and the American Academy of Orthopaedic Surgeons (AAOS) were reviewed. The reading material from each topic page was analyzed using the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) formulas. The reading level (FKGL) of each topic page was compared against an eighth-grade reading level, which corresponds to the average US reading level. Results A total of 170 online patient resources were reviewed, including 84 from the ASSH, 74 from the AAOS, and 12 from the AAHS. Overall, the mean FKGL was 9.1, and the mean FRE was 57.3. Overall, 50% of all hand and upper extremity online resources were written at or below an eighth-grade reading level. Pairwise testing revealed topic pages written by the ASSH had lower FKGL compared to those written by the AAHS (p = 0.046). Conclusions Online patient resources focused on hand and upper extremity topics are, on average, written at a level that exceeds the ability of the average reader. Comparisons between organizations showed a statistical, but not clinical, difference in readability measures. An emphasis on improving readability should be maintained as professional organizations continue to develop their online patient resources.

The Impact of Social Deprivation and Hand Therapy Attendance on Range of Motion After Flexor Tendon Repair

PURPOSE

To examine the influence of social deprivation and hand therapy attendance on active range of motion (AROM) outcomes following flexor tendon repair.

METHODS

We performed a retrospective analysis of patients who underwent primary zone I-III flexor tendon repair between November 2016 and November 2020. Area deprivation index (ADI) was used to quantify social deprivation. Medical record review determined each patient's demographic characteristics, injury details, total hand therapy visits, and final AROM outcome. Active range of motion was converted to Strickland's percentage for analysis. Spearman correlation and simple and multivariable linear regression models were used to assess relationships between explanatory variables and outcomes.

RESULTS

There were a total of 109 patients, with a mean ADI of 53 and mean therapy attendance of 13 visits. Higher ADI and lower therapy attendance were correlated, and each was associated with significantly decreased Strickland's percentage. In the multivariable model, therapy attendance, ADI, zone 2 injury, and age maintained significant associations with Strickland's percentage.

CONCLUSIONS

Socially deprived patients attend fewer therapy sessions and obtain poorer AROM after flexor tendon repair. Social deprivation is likely to contribute to poor outcomes both by its association with decreased therapy attendance and by other potential pathways that make it difficult for deprived patients to achieve good surgical outcomes.

TYPE OF STUDY/LEVEL OF EVIDENCE

Prognostic IV.

Readability and quality of online patient materials in the websites of the Japanese Orthopaedic Association and related orthopaedic societies

BACKGROUND

The purpose of this study was to quantify the readability and quality of patient materials on the websites of the Japanese Orthopaedic Association and related orthopaedic societies.

METHODS

We analyzed 143 online patient materials of seven societies, including the Japanese Orthopaedic Association and related six societies. The readability of the text in the material was quantified using two web-based programs ("Obi-3" and "Readability Research Lab") and lexical density. The materials with a difficulty level ≤ second grade of junior high school were classified as "average difficulty," and those ≤ sixth grade of elementary school were classified as "easy to read." The quality of the materials was quantified using the validated Clear Communication Index, DISCERN, and Journal of American Medical Association benchmark criteria scores. The proportion of materials classified as sufficient quality was determined. The measurement values were compared among societies using Kruskal-Wallis tests.

RESULTS

The median difficulty level of the overall materials was the third grade of junior high school using the Obi-3 program. Only 44 (31%) were classified as average difficulty, and none were classified as easy to read. Based on lexical density, 31 (22%) materials were classified as average difficulty. The median Clear Communication Index score was 18 points out of 100. The median DISCERN and Journal of American Medical Association benchmark criteria scores were 27 (out of 75) and 0 (out of 4), respectively. Only one material met the criteria as having sufficient quality. There were significant differences among societies in the lexical density, Clear Communication Index, and DISCERN scores.

CONCLUSIONS

The patient materials on the Japanese Orthopaedic Association and related society websites were too difficult to understand. Furthermore, the quality of the materials was not sufficient. Improvement in readability and quality may be necessary to enhance patient-physician communication.

Readability of Patient Educational Materials in Sports Medicine

Background:

The internet has become an increasingly popular resource among sports medicine patients seeking injury-related information. Numerous organizations recommend that patient educational materials (PEMs) should not exceed sixth-grade reading level. Despite this, studies have consistently shown the reading grade level (RGL) of PEMs to be too demanding across a range of surgical specialties.

Purpose:

To determine the readability of online sports medicine PEMs.

Study Design:

Cross-sectional study.

Methods:

The readability of 363 articles pertaining to sports medicine from 5 leading North American websites was assessed using 8 readability formulas: Flesch-Kincaid Reading Grade Level, Flesch Reading Ease Score, Raygor Estimate, Fry Readability Formula, Simple Measure of Gobbledygook, Coleman-Liau Index, FORCAST Readability Formula, and Gunning Fog Index. The mean RGL of each article was compared with the sixth- and eighth-grade reading level in the United States. The cumulative mean website RGL was also compared among individual websites.

Results:

The overall cumulative mean RGL was 12.2 (range, 7.0-17.7). No article (0%) was written at a sixth-grade reading level, and only 3 articles (0.8%) were written at or below the eighth-grade reading level. The overall cumulative mean RGL was significantly higher than the sixth-grade [95% CI for the difference, 6.0-6.5; P < .001] and eighth-grade (95% CI, 4.0-4.5; P < .001) reading levels. There was a significant difference among the cumulative mean RGLs of the 5 websites assessed.

Conclusion:

Sports medicine PEMs produced by leading North American specialty websites have readability scores that are above the recommended levels. Given the increasing preference of patients for online health care materials, the imperative role of health literacy in patient outcomes, and the growing body of online resources, significant work needs to be undertaken to improve the readability of these materials.

Health Literacy in Plastic Surgery: A Scoping Review

Health literacy of plastic surgery patients may affect surgical decision-making and perioperative outcomes. In addition to consulting a plastic surgeon, patients often refer to online-based resources to learn about surgical options. The aim of this scoping review was to identify evidence detailing the state of health literacy of plastic surgery patients and available resources to highlight areas of improvement for clinical practice and future research.

Addressing the Drivers of Medical Test Overuse and Cascades: User-Centered Design to Improve Patient-Doctor Communication

BACKGROUND

Low-value medical testing is a major component of health care overuse, both directly and through the potential for borderline and/or incidental results to trigger cascades (downstream services of uncertain value). The costs and harms from marginal test results and their cascades can add up. It is thus important to both prevent low-value tests at the outset and mitigate cascades when they arise.

METHODS

Informed by a framework for understanding and reducing overuse of care, this study employed user-centered design methods (focus groups and 1:1 design meetings) with patients and primary care physicians (PCPs) to understand the problem and iteratively develop an intervention.

RESULTS

Design meetings with 15 PCPs, 12 patients, and 3 patient focus groups revealed myriad drivers for medical test overuse and cascades. Patients commonly believed that all medical tests yield definitive results and lack downsides. PCPs cited expert recommendations, limited time during visits, fear of lawsuits, and desire to be responsive to patients as reasons for ordering potentially low-value medical tests. To address these issues, an intervention was designed using patient pre-visit educational materials, clinician reference materials on test interpretation and incidental findings, and clinician peer comparison on test overuse.

CONCLUSION

Overuse of medical testing is driven by a range of factors related to PCPs, patients, and their interactions. Multipronged interventions may have the potential to address these drivers after they are rigorously tested.

Readability of Online Hand Surgery Patient Educational Materials: Evaluating the Trend Since 2008

PURPOSE

Online patient educational materials have historically been written at a higher-than-recommended sixth grade reading level. The objectives of this study were to assess the readability of online hand surgery patient educational materials from the official online patient resource website of the American Society for Surgery of the Hand (ASSH) and to compare changes in the readability of the current ASSH online patient educational materials with those in 2008 and 2015.

METHODS

An internet-based study of all 88 English language patient educational materials on HandCare.org, the official online patient resource website of the ASSH, was performed. The readability of each article was assessed using the Flesch reading ease formula, Flesch-Kincaid grade level, Coleman-Liau index, Gunning-Fog index, and Simple Measure of Gobbledygook grade level. To evaluate the trend in the readability of ASSH online hand surgery patient educational materials, the Flesch-Kincaid grade levels of articles published in 2020 were compared with those of data published in 2008 and 2015.

RESULTS

The average Flesch reading ease score of the patient educational materials was 57.6, which is at the high-school reading level. The average reading grade level of patient educational materials ranged from 9.0 to 12.3 depending on the readability metric used. The average Flesch-Kincaid grade level of all the ASSH patient educational materials was 9.8 in 2020, which is significantly better than 10.4 in 2008 but significantly worse than 8.5 in 2015.

CONCLUSIONS

Online hand surgery patient educational materials continue to be written for the general public at a higher-than-recommended reading grade level. There has been no substantial improvement in the readability of online hand surgery patient educational materials since 2008.

CLINICAL RELEVANCE

Improvements are needed in the readability of online patient educational materials to ensure that patients with all health literacy levels are able to comprehend and benefit from health information.

Hand It to Dr Google: The Quality of Online Information on Ganglion Cysts

Background: The internet is becoming a common source of health information for hand surgery patients. This study evaluates the quality of web-based resources on ganglion cysts of the hand. Methods: We completed a search for "ganglion cyst" on 3 search engines (Google, Dogpile, and Yippy). The quality of the top-100 patient education websites was assessed using a validated internet rating tool. Websites were evaluated based on affiliation, accountability, currency, interactivity, website organization, readability, coverage, and accuracy. Results: Of the 100 websites, the majority (74%) had commercial affiliations. Only 34% of websites identified an author, and even fewer identified the authors' credentials (27%) or affiliations (26%). A third of the websites cited references, and less than half provided an update date. The average readability based on Flesch-Kincaid grade level was 9.2, and only 3% could be read at or below 6th grade reading level. Prevention was the most poorly covered topic at 13% due to omission. In all, 66% of the websites were completely accurate in terms of global accuracy. Websites were most likely to present inaccurate information on treatment, often failing to mention conservative treatment (watch-and-wait approach) or promoting the use of natural health products. We also found 5% of websites presented closed rupture of the ganglion cyst as a legitimate home remedy. Conclusions: The overall quality of online information on ganglion cysts is highly variable and may occasionally be harmful for patients. It is increasingly important for physicians to prompt patients about their internet use.

Distal Biceps Tendon Rupture Videos on YouTube: An Analysis of Video Content and Quality

Purpose

Our purpose was to analyze the content and quality of YouTube videos related to distal biceps tendon (DBT) ruptures and repair. We aimed to compare differences between academic and nonacademic video sources.

Methods

The most popular YouTube videos related to DBT injuries were compiled and analyzed according to source. Viewing characteristics were determined for each video. Video content and quality were assessed by 2 reviewers and analyzed according to the Journal of the American Medical Association benchmark criteria, DISCERN criteria, and a Distal Biceps Content Score. Cohen’s kappa was used to measure interrater reliability.

Results

A total of 59 DBT YouTube videos were included. The intraclass correlation coefficients ranged from moderate to excellent for the content scores. The mean DISCERN score was 29, and no videos were rated as either “good” or “excellent” for content quality. With the exception of the mean Journal of the American Medical Association criteria score (1.5 vs 0.5), videos from academic sources did not demonstrate significantly higher levels of content quality. Only 4/59 videos (7%) discussed the natural history of nonsurgically treated DBT ruptures. Of the 32 videos that discussed surgical techniques, only 3/32 (9%) had a preference for 2-incision techniques. No videos discussed the association between spontaneous DBT ruptures and cardiac amyloidosis.

Conclusions

The overall content, quality, and reliability of DBT videos on YouTube are poor. Videos from academic sources do not provide higher-quality information than videos from nonacademic sources. Videos related to operative treatment of DBT ruptures more frequently discuss single-incision techniques.

Clinical relevance

Social media videos can function as direct-to-consumer marketing materials, and surgeons should be prepared to address misconceptions regarding the management of DBT tears. Patients are increasingly seeking health information online, and surgeons should direct patients toward more reliable and vetted sources of information.

Readability and Comprehension of Printed Patient Education Materials

Background: Health literacy, a recently determined construct plays an important role in how individuals are able to manage their health. A useful approach for the assessment of health literacy is to measure the comprehension of available patient education materials (PEMs).

Objective: We aimed at assessing the usefulness of PEMS available in Hungarian by testing comprehension of selected PEMs in different groups of users.

Methods: Comprehension of patient education materials in the domain of healthcare was tested by selecting PEMs and creating questions based on their text in 3 dimensions of health literacy: understand, process/appraise, apply/use. Twenty questions were created that could be answered without pre-existing knowledge by reading the appropriate text taken from PEMs. Comprehension was examined in four groups: laypersons, non-professional healthcare workers, 1st year healthcare students, and 5th year medical students. Readability indices were calculated for the same texts to which questions were created.

Results: Laypersons answered <50% of the PEMs-based questions correctly. Non-professional healthcare workers performed better with 57% of right answers but significantly worse than healthcare students or medical students. Those with at least high school qualification (maturity exam) showed significantly higher comprehension compared to those with lower educational attainment. Persons in good or very good health also had significantly better comprehension than those in less favorable health. All readability indices showed that comprehension of the tested PEMs required at least 10 years of schooling or more. Therefore, these PEMS are difficult to understand for persons with less than high school level of education.

Conclusion: Rephrasing of the investigated patient educational materials would be recommended so that they better fit the educational attainment of the Hungarian population. Evaluation of the readability and comprehensibility of other PEMs also seems warranted.

Dupuytren's Contracture: The Readability of Online Information

Dupuytren's contracture is a common hand pathology for which consultation and treatment are largely at the patient's discretion. The objective of this study was to evaluate the readability of current online patient information regarding Dupuytren's contracture. The largest public search engines (Google, Yahoo, and Bing) were queried using the search terms “Dupuytren's contracture,” “Dupuytren's disease,” “Viking's disease,” and “bent finger.” The first 30 unique websites by each search were analyzed and readability assessed using five established algorithms: Flesch Reading Ease, Gunning-Fog Index, Flesch–Kincaid Grade level, Coleman–Liau index, and Simple Measure of Gobbledygook grade level. Analysis of 73 websites demonstrated an average Flesch Reading Ease score of 48.6 ± 8.0, which corresponds to college reading level. The readability of websites ranged from 10.5 to 13.3 reading grade level. No article was written at or below the recommended sixth grade reading level. Information on the internet on Dupuytren's contracture is written at higher than recommended reading grade level. There is a need for high-quality patient information on Dupuytren's contracture at appropriate reading grade levels for patients of various health literacy backgrounds. Hospitals, universities, and academic organizations focused on the development of readable online information should consider patients’ input and preferences.

The Readability of Online Educational Materials for Femoroacetabular Impingement Syndrome

Level III.

Health websites on COVID-19: are they readable and credible enough to help public self-care?

Objective

There are concerns about nonscientific and/or unclear information on the coronavirus disease 2019 (COVID-19) that is available on the Internet. Furthermore, people's ability to understand health information varies and depends on their skills in reading and interpreting information. This study aims to evaluate the readability and creditability of websites with COVID-19-related information.

Methods

The search terms "coronavirus," "COVID," and "COVID-19" were input into Google. The websites of the first thirty results for each search term were evaluated in terms of their credibility and readability using the Health On the Net Foundation code of conduct (HONcode) and Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG), Gunning Fog, and Flesch Reading Ease Score (FRE) scales, respectively.

Results

The readability of COVID-19-related health information on websites was suitable for high school graduates or college students and, thus, was far above the recommended readability level. Most websites that were examined (87.2%) had not been officially certified by HONcode. There was no significant difference in the readability scores of websites with and without HONcode certification.

Conclusion

These results suggest that organizations should improve the readability of their websites and provide information that more people can understand. This could lead to greater health literacy, less health anxiety, and the provision of better preventive information about the disease.

Assessing the Readability of Online Information About Achilles Tendon Ruptures

The American Medical Association (AMA) and National Institutes of Health (NIH) currently suggest that health care materials be written at a sixth-grade reading level. Our study investigates the readability of online information on Achilles rupture and reconstruction. Achilles tendon rupture, Achilles tendon repair, and Achilles tendon reconstruction were queried using advanced search functions of Google, Bing, and Yahoo!. Individual websites and text from the first 3 pages of results for each search engine were recorded and categorized as physician based, academic, commercial, government and nongovernmental organization, or unspecified. Individual readability scores were calculated via 6 different indices: Flesch-Kincaid grade level, Flesch Reading Ease, Gunning Fog, SMOG, Coleman-Liau index, and Automated Readability Index along with a readability classification score and average grade level. A total of 56 websites were assessed. Academic webpages composed the majority (51.8%), followed by physician-based sources (32.1%). The average overall grade level was 10.7 ± 2.54. Academic websites were written at the highest-grade level (11.5 ± 2.77), significantly higher than physician-based websites (P = .040), and only 2 were written at, or below, a sixth-grade reading level. Currently, online information on Achilles tendon rupture and reconstruction is written at an inappropriately high reading level compared with recommendations from the AMA and NIH.Level of Evidence: Level IV.

The Usefulness of Instagram Posts Tagging Hand Surgery Conditions

Introduction The objective of this study was to assess the usefulness of Instagram posts for users seeking information about common hand conditions.

Materials and Methods An internet-based study was performed of the top 200 trending posts on Instagram for each of the tags #carpaltunnel and #triggerfinger. The trending position (1 through 200), number of likes (for all posts), and number of views (for video posts only) were recorded. Each post was assessed to determine whether it would be useful for a user seeking medical information. Useful and not useful posts were further classified into thematic categories.

Results Useful posts comprised only 21% of #carpaltunnel posts and 12% of #triggerfinger posts. #carpaltunnel had a significantly higher proportion of useful posts than #triggerfinger. For both #carpaltunnel and #trigger finger, useful posts were not associated with trending position or number of likes. Useful posts commonly provided a clinical image or an anatomical diagram. Posts that were not useful were generally irrelevant to the hand condition, but a minority provided information not supported by evidence or were solely advertising material.

Conclusion Hand surgery patients can be educated that, in its current form, Instagram is not a source of useful information about common hand conditions.

Readability of foot and ankle consent forms in Queensland

BACKGROUND

The aim of this study was to conduct a readability analysis on both patient take-home information and consent forms for common foot and ankle procedures. Our hypothesis was that the objective reading skills required to read and comprehend the documentation currently in use would exceed the recommendations in place by both national and international bodies.

METHODS

The current Queensland Health consent forms are divided into specific subsections. The readability of consent form subsections C and G (sections containing detailed information on risks of the procedure and pertaining to informed patient consent specifically) and patient take-home information (provided as take-home leaflet from the consent form which is procedure specific) was assessed by an online readability software program using five validated methods calculated by application of the algorithms for (i) Flesch-Kincaid grade level, (ii) the SMOG (Simple Measure of Gobbledygook), (iii) Coleman-Liau index, (iv) automated readability index and the (v) Linsear Wriste formula.

RESULTS

The mean ± standard deviation reading grade level of risk (section C), grade level of patient consent (section G) and grade level for procedure-specific take-home patient information were 8.7 ± 0.9, 11.6 ± 1.2 and 7.5 ± 0.2, respectively.

CONCLUSION

The readability of sections C and G of the Queensland Health consent form exceeds the recommendations by national and international bodies, but the patient take-home information appears suitable. Consideration should be given to lower the reading grade level of patient consent forms to better reflect the reading grade of the Australian population.

Patient-oriented online resources in pediatric surgery: Are we failing the readability test?

BACKGROUND

The study aim was to evaluate the readability of patient-oriented resources in pediatric surgery from children's hospitals in the US.

METHODS

The websites of 30 children's hospitals were evaluated for information on 10 common pediatric surgical procedures. Hospitals of varying characteristics including bed number, geographic location and ACS Children's Surgery Verification (CSV) were selected for the study. Readability scores were calculated using validated algorithms, and text was assigned an overall grade level.

RESULTS

Of 195 patient-oriented resources identified, only three (2%) were written at or below the recommended sixth grade level. Larger hospitals provided patient information at a higher grade level than medium and smaller sized centers (10.7 vs 9.3 vs 9.0 respectively, p < 0.001). Hospital size also correlated with availability of information, with large and medium sized hospitals having information more often. Hospitals with ACS CSV had information available more often, and written at a lower grade level, compared to nonverified centers (78% vs 62%, p = 0.023; 9.0 vs 10.0, p = 0.013).

CONCLUSION

Most hospital provided patient-oriented resources in pediatric surgery are written at a grade level well above the national guidelines. Centers with ACS CSV status have improved availability and readability of this material, while larger hospitals have improved availability, but decreased readability.

TYPE OF STUDY

Modeling study.

LEVEL OF EVIDENCE

III.

Will the Effect of New Federal Breast Density Legislation Be Diminished by Currently Available Online Patient Educational Materials?

RATIONALE AND OBJECTIVES

To evaluate readability of commonly accessed online patient educational materials (OPEM) on breast density in setting of recently passed federal breast density legislation.

MATERIALS AND METHODS

The term "breast density" was queried using an online search engine to identify the top 50 commonly accessed websites based on order of search results on December 15, 2018. Location, cookies, and user account information were disabled prior to our query. Only websites with OPEM, defined as any educational material on breast density targeted towards the general public, were evaluated in our study. Sponsored hits and research journal articles were excluded. Available patient-directed information from websites meeting inclusion criteria was then downloaded. Grade-level readability was then determined from formatted content using generalized estimating equations, with observations nested within readability metrics from each website. Results were compared to American Medical Association recommended readability parameters (sixth-grade reading level). All interval estimates were calculated for 95% confidence.

RESULTS

Fouty-one websites met inclusion criteria representing patient-directed OPEM on breast density. Average grade-level readability of health information on breast density in our study ranged from 8.5-16.5 years with an average grade reading level of 11.1 years across all websites. Of websites fitting into a specific category, academic websites had the highest average grade reading level (12.0), while nonprofit websites had the lowest average grade reading level (10.4). Nearly half (19/41) of all websites in our study had diagrams to aid in patient comprehension, while few websites (2/41; 4.8%) displayed videos in addition to written content. The website with the lowest average grade reading level was WebMD, which had an average reading level of 8.5. No individual website in our study met American Medical Association recommended parameters of a sixth-grade reading level CONCLUSION: Readability of currently available OPEM on breast density may be written at a level too difficult for the general public to comprehend, which may represent a barrier to educational goals of newly passed federal breast density legislation.

Kienböck Disease: Quality, Accuracy, and Readability of Online Information

Background: Patients with limited health literacy require online educational materials to be written below a sixth grade level for optimal understanding. We assessed the quality, accuracy, and readability of online materials for Kienböck disease (KD). Methods: "Kienbock's Disease" and "Lunate Avacular Necrosis" were entered into 3 search engines. The first 25 Web sites from each search were collected. Quality was assessed via a custom grading rubric, accuracy by 2 residents and a fellow, and readability by Flesch-Kincaid grade level (FKGL) and New Dale-Chall test. Web sites were stratified according to the search term, FKGL, order of appearance, and authorship type. Results: A total of 38 unique Web sites were included, of which 22 were assigned to "KD" and 16 to "Lunate Avascular Necrosis." The average quality score out of 30, accuracy score out of 12, and FKGL for all Web sites were 13.3 ± 7.3, 10.4 ± 1.9, and 10.5 ± 1.4, respectively. Web sites assigned to the term "Kienbock's Disease" had a significantly higher FKGL. Web sites of higher FKGL had significantly worse accuracy scores. Order of appearance had no influence. Physician specialty societies (PSS) had a significantly lower FKGL than Web sites of other authorship types. Conclusions: Despite concerted efforts by national organizations, the readability of online patient materials is above the recommended level for KD. Patients with limited health literacy will be most affected by this reality. Until readability improves, patients should continue to consult their physicians when uncertain and prioritize Web sites that are easier to read and produced by PSS.

Reading Level of Online Patient Education Materials From Major Obstetrics and Gynecology Societies

OBJECTIVE

To assess whether the readability of online patient education materials provided by eight nationally recognized obstetrics and gynecology societies is in accordance with the recommended 6th-grade reading level outlined by the American Medical Association, National Institute of Health, and United States Department of Health and Human Services.

METHODS

An analysis of 410 online patient education materials from the American Association of Gynecologic Laparoscopists, the American College of Obstetricians and Gynecologists, the American Society for Reproductive Medicine, the American Urogynecologic Society, the Association of Reproductive Health Professionals, the Society of Gynecologic Oncology, the Society for Maternal-Fetal Medicine, and Voices for Pelvic Floor Disorders was completed, and the readability scores using the Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning Fog Scale, and Simple Measure of Gobbledygook readability scales were calculated. These scales are used to assess print and web-based written material across a wide range of medical specialties and are endorsed by the National Institutes of Health. All four scales are computer-based assessments of readability that consider word count, number of syllables, and length of sentences when calculating a score that corresponds to grade level.

RESULTS

Majority of the patient education materials across all eight obstetrics and gynecology societies had readability scores above the recommended 6th-grade reading level. The average reading level for the 69 obstetrics-related articles ranged from 9th to 12th grade. The mean grade level for the 341 gynecology articles had a similar range across the four readability scales.

CONCLUSION

Online patient education materials provided by major obstetrics and gynecology societies do not currently adhere to recommended readability guidelines. Continued efforts to provide accessible and informative patient education materials is recommended to help improve health literacy for women.

Universal Measures of Support Are Needed: A Cross-Sectional Study of Health Literacy in Patients with Dupuytren's Disease

BACKGROUND

Health literacy represents the degree to which patients can understand and act on health information. The relevance of health literacy to health care delivery, outcomes, and overall surgical care is unambiguous. This study aimed (1) to determine the prevalence of limited health literacy in patients diagnosed with Dupuytren's contracture and (2) to identify independent predictors of limited health literacy.

METHODS

This cross-sectional study included patients with Dupuytren's disease and with self-reported English fluency. The Newest Vital Sign, a rapid, validated, and reliable screening tool, was selected to measure health literacy. An exploratory multivariable logistic regression model was used to identify possible predictors of limited health literacy.

RESULTS

A total of 185 patients met eligibility criteria and were included. From those, 82 (44 percent) were found to have limited health literacy, defined as a score of 3 or less on the Newest Vital Sign. The domain of prose literacy was most highly scored compared to numeracy and document literacy. Lower household income was associated with a 4.7-fold increase in the odds of having limited health literacy. Being an immigrant also increased the odds of having limited health literacy by a factor of 3.6. Sensitivity analyses and subgroup analyses (based on education, maternal language, and immigration status) corroborated these independent predictor findings.

CONCLUSIONS

Limited health literacy is common among patients with Dupuytren's contracture. System level changes are necessary such as the access and integration to clinical care of universal measures of support to promote productive patient-surgeon interactions.

Readability of Online Patient Educational Materials Related to Breast Lesions Requiring Surgery

Purpose To evaluate readability of websites that are commonly accessed for information on breast lesions requiring surgery. Materials and Methods An internet search using three malignant and eight nonmalignant breast lesions that traditionally require lumpectomy or excisional biopsy as search terms was conducted to identify websites commonly accessed for patient information on breast lesions requiring surgery. Nine websites with information on breast diagnoses were identified based on search engine results for each breast lesion queried. Available patient-directed information was downloaded for each lesion from each website on May 15, 2018. Grade-level readability of downloaded content for each lesion was then determined by using generalized estimating equations, with observations nested within readability metrics from each website. Readability of associated terms breast biopsy, breast cancer, and breast surgery was also evaluated with the same method. Results were compared with American Medical Association (AMA) recommended readability parameters (sixth-grade reading level). All interval estimates were calculated for 95% confidence. Results Average grade level readability score of health information on breast lesions requiring surgery was 11.7, which exceeded the AMA parameters. Information on Wikipedia was written at the highest reading level (grade level readability score, 14.2), while information on the National Institutes of Health website ( http://cancer.gov ) was written at the lowest reading level (grade level readability score, 9.7). Educational materials on malignant breast lesions (grade level readability score, 12.3) were written at a higher reading level than were those on nonmalignant breast lesions (grade level readability score,11.4). Information on the terms breast biopsy (grade level readability score, 10.9), breast cancer (grade level readability score, 10.6), and breast surgery (grade level readability score, 11.1) were all written above a sixth-grade reading level. Conclusion Readability of current online resources on breast biopsy lesions traditionally requiring surgery may be too complex for the general public to comprehend, leading to misinformation and confusion. © RSNA, 2019 See also the editorial by Haygood in this issue.

Assessing the Readability of Online Information About Hip Arthroscopy

PURPOSE

To investigate the current readability of online information pertaining to hip arthroscopy.

METHODS

The terms "hip arthroscopy" and "hip scope" were entered into the advanced search functions of Google, Yahoo!, and Bing on March 25, 2017, and results from the first 3 pages were analyzed. Results were required to be unique, accessible websites with information about hip arthroscopy conveyed primarily via analyzable text. Two reviewers applied inclusion criteria to the initial 97 results, discussing to reach consensus in cases of disagreement. Overall, 60 unique results were reviewed with 48 meeting inclusion criteria. Websites were categorized as physician-sponsored, academic, commercial, governmental and nonprofit organization (NPO), or unspecified. Readability was measured via 6 different indices: the Flesch-Kincaid grade level (FKGL), Flesch Reading Ease (FRE), Gunning Fog Score, SMOG Index, Coleman-Liau Index (CLI), and Automated Readability Index (ARI) along with an average grade level and readability classification score.

RESULTS

Forty-eight unique websites were assessed for readability, with physician-sponsored webpages composing the majority (47.92%) followed by academic sources (35.42%). The webpages' average grade level, incorporating information from all 6 metrics, was 12.79 ± 1.98.

CONCLUSIONS

The current readability of online information pertaining to hip arthroscopy is at an inappropriately high reading level compared with the sixth-grade level recommended by the American Medical Association and National Institutes of Health, thus introducing significant barriers to understanding for many patients. Online materials should be edited to reduce word and sentence length and complexity, use simpler terms, and minimize use of passive voice to facilitate patient knowledge acquisition and understanding of online information about hip arthroscopy.

CLINICAL RELEVANCE

This study shows that the current readability of online information on hip arthroscopy exceeds the suggested sixth-grade reading level. It also emphasizes the need for simplifying written materials and offers specific suggestions on doing so to increase accessibility of information for patients.

Preoperative Education for Hip and Knee Replacement: Never Stop Learning

PURPOSE OF REVIEW

Participation in alternative payment models has focused efforts to improve outcomes and patient satisfaction while also lowering cost for elective hip and knee replacement. The purpose of this review is to determine if preoperative education classes for elective hip and knee replacement achieve these goals.

RECENT FINDINGS

Recent literature demonstrates that patients who attend education classes prior to surgery have decreased anxiety, better post-operative pain control, more realistic expectations of surgery, and a better understanding of their surgery. As a result, comprehensive clinical pathways incorporating a preoperative education program for elective hip and knee replacement lead to lower hospital length of stay, higher home discharge, lower readmission, and improved cost. In summary, we report convincing evidence that preoperative education classes are an essential element to successful participation in alternative payment models such as the Bundle Payment Care Initiative.