Readability of arthroscopy-related patient education materials from the American Academy of Orthopaedic Surgeons and Arthroscopy Association of North America Web sites

Is Information About Musculoskeletal Malignancies From Large Language Models or Web Resources at a Suitable Reading Level for Patients?

BACKGROUND

Patients and caregivers may experience immense distress when receiving the diagnosis of a primary musculoskeletal malignancy and subsequently turn to internet resources for more information. It is not clear whether these resources, including Google and ChatGPT, offer patients information that is readable, a measure of how easy text is to understand. Since many patients turn to Google and artificial intelligence resources for healthcare information, we thought it was important to ascertain whether the information they find is readable and easy to understand. The objective of this study was to compare readability of Google search results and ChatGPT answers to frequently asked questions and assess whether these sources meet NIH recommendations for readability.

QUESTIONS/PURPOSES

(1) What is the readability of ChatGPT-3.5 as a source of patient information for the three most common primary bone malignancies compared with top online resources from Google search? (2) Do ChatGPT-3.5 responses and online resources meet NIH readability guidelines for patient education materials?

METHODS

This was a cross-sectional analysis of the 12 most common online questions about osteosarcoma, chondrosarcoma, and Ewing sarcoma. To be consistent with other studies of similar design that utilized national society frequently asked questions lists, questions were selected from the American Cancer Society and categorized based on content, including diagnosis, treatment, and recovery and prognosis. Google was queried using all 36 questions, and top responses were recorded. Author types, such as hospital systems, national health organizations, or independent researchers, were recorded. ChatGPT-3.5 was provided each question in independent queries without further prompting. Responses were assessed with validated reading indices to determine readability by grade level. An independent t-test was performed with significance set at p < 0.05.

RESULTS

Google (n = 36) and ChatGPT-3.5 (n = 36) answers were recorded, 12 for each of the three cancer types. Reading grade levels based on mean readability scores were 11.0 ± 2.9 and 16.1 ± 3.6, respectively. This corresponds to the eleventh grade reading level for Google and a fourth-year undergraduate student level for ChatGPT-3.5. Google answers were more readable across all individual indices, without differences in word count. No difference in readability was present across author type, question category, or cancer type. Of 72 total responses across both search modalities, none met NIH readability criteria at the sixth-grade level.

CONCLUSION

Google material was presented at a high school reading level, whereas ChatGPT-3.5 was at an undergraduate reading level. The readability of both resources was inadequate based on NIH recommendations. Improving readability is crucial for better patient understanding during cancer treatment. Physicians should assess patients' needs, offer them tailored materials, and guide them to reliable resources to prevent reliance on online information that is hard to understand.

LEVEL OF EVIDENCE

Level III, prognostic study.

Readability of Trauma-related Patient Education Materials From the American Academy of Orthopaedic Surgeons and Orthopaedic Trauma Association Websites

INTRODUCTION

Web-based resources serve as a fundamental educational platform for orthopaedic trauma patients; however, they are frequently written above the recommended sixth-grade reading level, and previous studies have demonstrated this for the American Academy of Orthopaedic Surgeons (AAOS) web-based articles. In this study, we perform an updated assessment of the readability of AAOS trauma-related educational articles as compared with injury-matched education materials developed by the Orthopaedic Trauma Association (OTA).

METHODS

All 46 AAOS trauma-related web-based ( https://www.orthoinfo.org/ ) patient education articles were analyzed for readability. Two independent reviewers used (1) the Flesch-Kincaid Grade Level (FKGL) and (2) the Flesch Reading Ease (FRE) algorithms to calculate the readability level. Mean readability scores were compared across body part categories. A one-sample t -test was done to compare mean FKGL with the recommended sixth-grade readability level and the average American adult reading level. A two-sample t -test was used to compare the readability scores of the AAOS trauma-related articles with those of the OTA.

RESULTS

The average (SD) FKGL and FRE for the AAOS articles were 8.9 (0.74) and 57.2 (5.8), respectively. All articles were written above the sixth-grade reading level. The average readability of the AAOS articles was significantly greater than the recommended sixth-grade reading level ( P < 0.001). The average FKGL and FRE for all AAOS articles were significantly higher compared with those of the OTA articles (8.9 ± 0.74 versus 8.1 ± 1.14, P < 0.001 and 57.2 ± 5.8 versus 65.6 ± 6.6, P < 0.001, respectively). Excellent agreement was observed between raters for the FKGL 0.956 (95% confidence interval, 0.922 to 0.975) and FRE 0.993 (95% confidence interval, 0.987 to 0.996).

DISCUSSION

Our findings suggest that after almost a decade, the readability of the AAOS trauma-related articles remains unchanged. The AAOS and OTA trauma patient education materials have high readability levels and may be too difficult for patient comprehension. A need remains to improve the readability of these commonly used trauma education materials.

Readability and Quality of Online Patient Education Materials Concerning Posterior Cruciate Ligament Reconstruction

Objective This study aimed to assess the quality of online patient educational materials regarding posterior cruciate ligament (PCL) reconstruction. Methods We performed a search of the top-50 results on Google® (terms: "posterior cruciate ligament reconstruction," "PCL reconstruction," "posterior cruciate ligament surgery," and "PCL surgery") and subsequently filtered to rule out duplicated/inaccessible websites or those containing only videos (67 websites included). Readability was assessed using six formulas: Flesch-Kincaid Reading Ease (FRE), Flesch-Kincaid Grade Level (FKG), Gunning Fog Score (GF), Simple Measure of Gobbledygook (SMOG) Index, Coleman-Liau Index (CLI), Automated Readability Index (ARI); quality was assessed using the JAMA benchmark criteria and recording the presence of the HONcode seal. Results The mean FRE was 49.3 (SD 11.2) and the mean FKG level was 8.09. These results were confirmed by the other readability formulae (average: GF 8.9; SMOG Index 7.3; CLI 14.7; ARI 6.5). A HONcode seal was available for 7.4 % of websites. The average JAMA score was 1.3. Conclusion The reading level of online patient materials concerning PCL reconstruction is too high for the average reader, requiring high comprehension skills. Practice implications Online medical information has been shown to influence patient healthcare decision processes. Patient-oriented educational materials should be clear and easy to understand.

Don't Throw Away the Handout: Efficacy of Patient Education in Pediatric Orthopaedic Trauma

BACKGROUND

Printed educational materials (PEMs) have been used for patient education in various settings. The purpose of this study was to determine the readability, understandability, and actionability of trauma-related educational material from the Pediatric Orthopaedic Society of North America (POSNA, Orthokids), as well as determine its efficacy in educating pediatric orthopaedic trauma patients and caregivers.

METHODS

The readability, understandability and actionability of PEMs was assessed using the Patient Education materials Assessment Tool (PEMAT). Five reviewers ranging in experience independently evaluated the educational materials. The efficacy of PEMs was assessed prospectively by randomizing patients into 2 groups. The first group (Education) received the OrthoKids educational material related to the patient's fracture. The second group (No Education) did not receive the educational material. At the first follow-up visit, parents/guardians in both groups completed surveys. Statistical analyses included descriptive and univariate statistics.

RESULTS

The understandability of PEMs was similar (68% to 74%); however, the educational materials had varying actionability scores ranging from 20% for femoral shaft fractures to 60% for elbow fractures. In total, 101 patients were randomized to assess the efficacy of educational materials (Education=51, No Education=50). There were no significant differences in sex, age, race/ethnicity, and level of education between caregivers in both groups ( P > 0.05). Only 61% (31/51) participants in the Education group reported using the educational material; however, 67% to 68% of participants in either group reported wanting PEMs. Participants in the group that did not receive PEMs were significantly more likely to use the internet to find more information (74% vs. 51%, P < 0.05).

CONCLUSIONS

This study suggests that participants that did not receive PEMs were significantly more likely to search the internet for more information. Improving the quality and actionability of educational resources on electronic platforms is needed to improve patient education. A multi-modal approach using PEMs that includes a list of high-quality online sources would likely be most effective in educating pediatric trauma patients and caregivers.

LEVEL OF EVIDENCE

I.

Readability assessment of patient educational materials for pediatric spinal conditions from top academic orthopedic institutions

Background

The Internet has become a popular source of health information for patients and their families. Healthcare experts recommend that the readability of online education materials be at or below a sixth grade reading level. This translates to a standardized Flesch Reading Ease Score between 81 and 90, which is equivalent to conversational English. However, previous studies have demonstrated that the readability of online education materials of various orthopedic topics is too advanced for the average patient. To date, the readability of online education materials for pediatric spinal conditions has not been analyzed. The objective of this study was to assess the readability of online educational materials of top pediatric orthopedic hospital websites for pediatric spinal conditions.

Methods

Online patient education materials from the top 25 pediatric orthopedic institutions, as ranked by the U.S. News and World Report hospitals for pediatric orthopedics, were assessed utilizing multiple readability assessment metrics including Flesch-Kincaid, Flesch Reading Ease, Gunning Fog Index, and others. Correlations between academic institutional ranking, geographic location, and the use of concomitant multimedia modalities with Flesch-Kincaid scores were evaluated using a Spearman regression.

Results

Only 32% (8 of 25) of top pediatric orthopedic hospitals provided online health information at or below a sixth grade reading level. The mean Flesch-Kincaid score was 9.3 ± 2.5, Flesch Reading Ease 48.3 ± 16.2, Gunning Fog Score 10.7 ± 3.0, Coleman-Liau Index 12.1 ± 2.8, Simple Measure of the Gobbledygook Index 11.7 ± 2.1, Automated Readability Index 9.0 ± 2.7, FORCAST 11.3 ± 1.2, and Dale-Chall Readability Index 6.7 ± 1.4. There was no significant correlation between institutional ranking, geographic location, or use of video material with Flesch-Kincaid scores (p = 0.1042, p = 0.7776, p = 0.3275, respectively).

Conclusion

Online educational material for pediatric spinal conditions from top pediatric orthopedic institutional websites is associated with excessively complex language which may limit comprehension for the majority of the US population.

Type of study/Level of evidence

Economic and Decision Analysis/level III.

Systematic Assessment of the Quality and Comprehensibility of YouTube Content on Ulnar Collateral Ligament Injury and Management

Background

Ulnar collateral ligament (UCL) reconstruction has received a unique level of attention in the press and social media. There has also been an increasing use of the internet by patients to seek medical information. Concern exists regarding the quality and comprehensibility of online information when used for patient education.

Purpose

To evaluate the quality and comprehensibility of the most-viewed YouTube videos related to the diagnosis and management of UCL injuries. Based on our new evidence-based scoring rubrics, we hypothesized that the quality and comprehensibility of these videos would be poor.

Study Design

Cross-sectional study.

Methods

The YouTube platform was searched on September 7, 2021, with the terms "UCL injury," "ulnar collateral ligament injury," "UCL surgery," "ulnar collateral ligament surgery," and "Tommy John surgery," and the 50 most-viewed videos from each search were compiled, yielding 250 videos. After removal of duplicates and application of exclusion criteria, the 100 most-viewed videos remained. Basic attributes, including duration of video and number of views, were recorded. Each video was then analyzed by 2 independent reviewers and evaluated for 4 key parameters (quality of diagnostic content [QAR-D], quality of treatment content [QAR-T], presence of inaccurate information, and comprehensibility) and graded on a novel scale from 1 to 4 (4 being the most appropriate for patient education).

Results

The mean QAR-D was 4.83 ± 3.41 (fair quality), and the mean QAR-T was 2.76 ± 3.26 (poor quality). Physician-led educational videos had both the highest mean QAR-D (6.37) and the highest mean QAR-T (4.34). No correlation was observed between video quality and views/likes. A total of 12 videos included ≥1 inaccuracy. The mean comprehensibility score was 2.66 ± 1.12, with 39 videos falling below the acceptable comprehensibility threshold (score <3).

Conclusion

The overall quality of UCL injury-related YouTube content was low. In addition, the absence of correlation between video quality and views/likes suggests that patients are not preferentially utilizing the limited high-quality content that does exist on the YouTube platform. In addition, inaccurate videos were prevalent (12%), and almost half of all videos were deemed inappropriate for patient education in terms of comprehensibility, as defined by our comprehensibility parameter.

Content Analysis of Patient-Facing Information Related to Preeclampsia

BACKGROUND

Previous research has shown pregnant people are not knowledgeable about preeclampsia, a significant cause of maternal morbidity and mortality. This lack of knowledge may impact their ability to report symptoms, comply with recommendations, and receive appropriate follow-up care. Pregnant people commonly seek information from sources outside their treating clinician, including pregnancy-specific books and online sources. We examined commonly used preeclampsia information sources to evaluate whether pregnant people are receiving up-to-date, guideline-based information.

METHODS

We conducted a content analysis of preeclampsia-related information in top-ranking websites and bestselling pregnancy books. We used American College of Obstetricians and Gynecologists preeclampsia guidelines to construct a codebook, which we used to examine source content completeness and accuracy. For each source, we analyzed information about preeclampsia diagnosis, patient-reported symptoms, risk factors, prevention, treatment, and complications.

RESULTS

Across 19 included sources (13 websites and 6 books), we found substantial variation in completeness and accuracy of preeclampsia information. We found high rates of mentions for preeclampsia symptoms. Risk factors were more commonly included in online sources than book sources. Most sources mentioned treatment options, including blood pressure medication and giving birth; however, one-third of online sources positively mentioned the nonrecommended treatment of bed rest. Prevention methods, including prenatal aspirin for high-risk pregnancies, and long-term complications of preeclampsia were infrequently mentioned.

CONCLUSIONS

Varying rates of accuracy in patient-facing preeclampsia information mean there is substantial room for improvement in these sources. Ensuring pregnant people receive current guideline-based information is critical for improving outcomes and implementing shared decision-making.

Readability assessment of patient educational materials for pediatric spinal deformity from top academic orthopedic institutions

STUDY DESIGN

Cross-sectional analysis of patient educational materials from top pediatric orthopedic hospital websites.

OBJECTIVE

To assess the readability of online educational materials of top pediatric orthopedic hospital websites for pediatric spinal deformity. The internet has become an increasingly popular source of health information for patients and their families. Healthcare experts recommend that the readability of online education materials be at or below a 6th-grade reading level. However, previous studies have demonstrated that the readability of online education materials on various orthopedic topics is too advanced for the average patient. To date, the readability of online education materials for pediatric spinal deformity has not been analyzed.

METHODS

Online patient education materials from the top 25 pediatric orthopedic institutions, as ranked by the U.S. News and World Report hospitals for pediatric orthopedics, were accessed utilizing the following readability assessments: Flesch-Kincaid (FK), Flesch Reading Ease, Gunning Fog Index, Coleman-Liau Index, Simple Measure of the Gobbledygook Index (SMOG), Automated Readability Index, FORCAST, and the New Dale and Chall Readability. Correlations between academic institutional ranking, geographic location, and the use of concomitant multi-media modalities with FK scores were evaluated using a Spearman regression.

RESULTS

Only 48% (12 of 25) of top pediatric orthopedic hospitals provided online information regarding pediatric spinal deformity at or below a 6th-grade reading level. The mean FK score was 9.0 ± 2.7, Flesch Reading Ease 50.8 ± 15.6, Gunning Fog Score 10.6 ± 3.1, Coleman-Liau Index 11.6 ± 2.6, SMOG index 11.7 ± 2.0, Automated Readability Index 8.6 ± 2.8, and Dale-Chall Readability Score 6.4 ± 1.4. There was no significant correlation between institutional ranking, geographic location, or use of multimedia with FK scores.

CONCLUSION

Online educational material for pediatric spinal deformity from top pediatric orthopedic institutional websites are associated with poor readability.

Impingement on the internet: evaluating the quality and readability of online subacromial impingement information

Patients increasingly access the internet to learn about their orthopaedic conditions. Despite this, online information may be unregulated, of questionable quality and difficulty to read.

OBJECTIVES

Therefore, this study aimed to evaluate the readability and quality of the online information concerning subacromial impingement syndrome.

METHODS

A search using Australia's three most popular online search engines was undertaken using the search terms 'subacromial impingement syndrome' and 'shoulder impingement'. The first 15 websites for each term were evaluated. Duplicates, advertisements and sponsored links were removed.The quality and readability of each website were calculated using the DISCERN and Flesch-Kincaid Reading Ease (FKRE) tools, respectively. The differences in quality and readability between each website type (healthcare/academic, commercial, news outlet, charitable/not-for-profit, layperson, government) was assessed using analysis of variance. The correlation between quality and readability was assessed using the Pearson correlation coefficient.

RESULTS

The majority of 35 unique websites analysed were of 'poor'/'fair' quality (determined via the DISCERN instrument) and 'difficult' readability (per the FKRE tool), with no correlation established between the scores. There was no statistically significant difference in quality across website types, however layperson, news outlet and government websites were found to be significantly more readable than alternate website categories (p<0.05).

CONCLUSIONS

We determined that much of the online information concerning subacromial impingement syndrome may be difficult to read and/or of poor quality. By recognising the shortcomings of information accessed by patients online, it is hoped clinicians may be prompted to better educate their patients.

Readability of the Most Commonly Accessed Online Patient Education Materials Pertaining to Surgical Treatments of the Spine

OBJECTIVE

The American Medical Association and National Institutes of Health have recommended that patient education materials should be written at the sixth-grade reading level to maximize patient comprehension. The objective of the present study was to evaluate the readability of Internet information for the 9 most common spinal surgeries.

METHODS

We reviewed 90 online patient educational materials regarding the 9 most common spinal surgeries as reported by the North American Spine Society. A Google search was performed on March 23, 2019 for each surgery, and the top 10 most visited websites for each surgery were assessed for reading level using the Flesch-Kincaid formula.

RESULTS

Using the Flesch-Kincaid formula, the average grade reading level of the 90 websites included was 12.82, with a reading ease of 37.04 ("difficult college"). Only 6 websites relayed information to patients at or below the national average of an eighth-grade reading level. The websites for bone morphogenic protein had the highest average grade reading level at 15.88 ± 2.6. Lumbar microscopic discectomy had the lowest average grade reading level at 10.37 ± 2.89. All surgical options discussed had an average readability above the recommended sixth-grade reading level.

CONCLUSIONS

The most accessed online materials for common spinal surgeries, not only exceeded the readability limits recommended by both the American Medical Association and the National Institutes of Health, but they also exceeded the average reading ability of most adults in the United States. Patients, therefore, might not fully comprehend the information from commonly accessed websites regarding surgical spine treatment options.

Evaluation of Hip Preservation-related Patient Education Materials From Leading Orthopaedic Academic Centers in the United States and Description of a Novel Video Assessment Tool

The readability, reliability, and quality of online hip preservation–related patient education materials from the top 20 orthopaedic academic centers in the United States were evaluated.

Readability of the Most Commonly Accessed Online Patient Education Materials Pertaining to Pathology of the Hand

BACKGROUND

The American Medical Association (AMA) and National Institutes of Health (NIH) recommend that patient education materials be written at no higher than a sixth-grade reading level.

METHODS

We examined 100 online educational materials for the 10 hand conditions most commonly treated by hand surgeons, as reported by the American Society for Surgery of the Hand. The listed conditions were carpal tunnel syndrome, basal joint arthritis of the thumb, de Quervain syndrome, Dupuytren's contracture, ganglion cysts, hand fractures, trigger finger, extensor tendon injuries, flexor tendon injuries, and mallet finger. Following a Google search for each condition, we analyzed the 10 most visited websites for each disorder utilizing the Flesch-Kincaid formula.

RESULTS

The average grade reading level of the 100 websites studied was 9.49 with a reading ease of 53.03 ("fairly difficult high school"). Only 29% of the websites were at or below the national average of an eighth-grade reading level. Carpal tunnel syndrome had the highest average grade reading level at 10.32 (standard deviation: 1.52), whereas hand fractures had the lowest at 8.14 (2.03). Every hand condition in this study had an average readability at or above the ninth-grade reading level.

CONCLUSIONS

The most frequently accessed materials for common maladies of the hand exceed both the readability limits recommended by the AMA and NIH, and the average reading ability of most US adults. Therefore, the most commonly accessed websites pertaining to hand pathology may not be comprehended by the audience for which it is intended.

Readability of Sports Injury and Prevention Patient Education Materials From the American Academy of Orthopaedic Surgeons Website

Introduction:

The purpose of this study is to evaluate the readability of 114 Sports Injury and Prevention patient education materials provided by the American Academy of Orthopaedic Surgeons (AAOS).

Methods:

We evaluated all articles written in English posted under the Sports Injury and Prevention section of the AAOS website using readability software to compute six readability scores, which we compared with the eighth-grade level using a two-tailed one-sample Student t-test.

Results:

The mean reading grade level calculated by each readability test was markedly higher than the eighth-grade level. We reported mean ± SD for each test: Flesch-Kincaid grade level (8.95 ± 1.51; P < 0.001), Simple Measure of Gobbledygook (11.53 ± 1.18; P < 0.001), Coleman-Liau index (11.16 ± 1.33; P < 0.001), Gunning Fog index (11.06 ± 1.63; P < 0.001), New Dale-Chall (9.49 ± 1.66; P < 0.001), and FORCAST formulas (10.96 ± 0.60; P < 0.001).

Discussion:

This study shows that patient education materials provided by the AAOS concerning sports injury and prevention are written at a readability level too high for patients to understand. On average, patient materials are written at least 2.5 grade levels higher than national recommendations. Only 7% of the 114 articles had readability scores in line with national recommendations. These findings indicate a need for revised patient education materials geared toward bringing the readability level down to the recommended eighth-grade level.

Readability of the Most Commonly Accessed Arthroscopy-Related Online Patient Education Materials

PURPOSE

To assess the readability and comprehension of written text by the most commonly visited websites containing patient education materials on common conditions that can be treated arthroscopically.

METHODS

We examined 50 websites, assessed independently by 2 orthopaedic surgery residents (S.A. and G.G.), with educational materials on 5 common conditions treated by arthroscopic surgeons: anterior cruciate ligament (ACL) tear, meniscus tear, hip labral tear, shoulder labral tear, and rotator cuff tear. Following a Google search for each condition, we analyzed the 10 most visited websites for each disorder using a widely used and validated tool for assessing the reading levels of written materials (Flesch-Kincaid formula).

RESULTS

The average grade reading level of the 50 websites studied was 9.90 with a reading ease of 52.14 ("fairly difficult, high school"). Only 26% of the websites were at or below the national average of an eighth-grade reading level. Of the 5 conditions treated by arthroscopic surgery, ACL tear had the highest average grade reading level at 10.73 ± 1.54, whereas meniscus tear had the lowest at 9.31 ± 1.81. Every condition in this study had an average readability at or above the ninth-grade reading level.

CONCLUSIONS

The most frequently accessed materials for patients with injuries requiring arthroscopic surgery exceeds the readability recommendations of the American Medical Association and National Institutes of Health, as well as the average reading ability of US adults. Given the fact that these are the most commonly visited websites by the lay public, there needs to be a greater emphasis on tailoring written information to the literacy levels of the patient population.

CLINICAL RELEVANCE

This study emphasizes the discrepancy between the recommended versus the measured reading levels of online patient education materials related to conditions treated by arthroscopic surgeons. The subject matter of these conditions is inherently complex; thus, relying solely on text to inform patients increases the likelihood that the reading level of the material exceeds that of the majority of the lay public.

Assessing Online Patient Education Readability for Spine Surgery Procedures

BACKGROUND CONTEXT

Increased patient reliance on Internet-based health information has amplified the need for comprehensible online patient education articles. As suggested by the American Medical Association and National Institute of Health, spine fusion articles should be written for a 4th-6th-grade reading level to increase patient comprehension, which may improve postoperative outcomes.

PURPOSE

The purpose of this study is to determine the readability of online health care education information relating to anterior cervical discectomy and fusion (ACDF) and lumbar fusion procedures.

STUDY DESIGN

Online health-education resource qualitative analysis.

METHODS

Three search engines were utilized to access patient education articles for common cervical and lumbar spine procedures. Relevant articles were analyzed for readability using Readability Studio Professional Edition software (Oleander Software Ltd). Articles were stratified by organization type as follows: General Medical Websites (GMW), Healthcare Network/Academic Institutions (HNAI), and Private Practices (PP). Thirteen common readability tests were performed with the mean readability of each compared between subgroups using analysis of variance.

RESULTS

ACDF and lumbar fusion articles were determined to have a mean readability of 10.7±1.5 and 11.3±1.6, respectively. GMW, HNAI, and PP subgroups had a mean readability of 10.9±2.9, 10.7±2.8, and 10.7±2.5 for ACDF and 10.9±3.0, 10.8±2.9, and 11.6±2.7 for lumbar fusion articles. Of 310 total articles, only 6 (3 ACDF and 3 lumbar fusion) were written for comprehension below a 7th-grade reading level.

CONCLUSIONS

Current online literature from medical websites containing information regarding ACDF and lumbar fusion procedures are written at a grade level higher than the suggested guidelines. Therefore, current patient education articles should be revised to accommodate the average reading level in the United States and may result in improved patient comprehension and postoperative outcomes.

Readability of Patient Education Materials From the Web Sites of Orthopedic Implant Manufacturers

BACKGROUND

Prior studies indicate that orthopedic patient education materials are written at a level that is too high for the average patient. The purpose of this study was to assess the readability of online patient education materials provided by orthopedic implant manufacturers.

METHODS

All patient education articles available in 2013 from the web sites of the 5 largest orthopedic implant manufacturers were identified. Each article was evaluated with the Flesch-Kincaid (FK) readability test. The number of articles with readability ≤ the eighth-grade level (average reading ability of US adults) and the sixth-grade level (recommended level for patient education materials) was determined. Mean readability levels of each company's articles were compared using analysis of variance (significance set at P < .05).

RESULTS

A total of 581 articles were reviewed from the 5 largest implant manufacturers. The mean overall FK grade level was 10.9 (range, 3.8-16.1). Only 58 articles (10%) were written ≤ the eighth-grade level, and only 13 (2.2%) were ≤ the sixth-grade level. The mean FK grade level was significantly different among groups (Smith & Nephew = 12.0, Stryker = 11.6, Biomet = 11.3, DePuy = 10.6, Zimmer = 10.1; P < .0001).

CONCLUSION

The majority of patient education materials from implant manufacturers are written at a level too high to be comprehended by the average patient. Future efforts should be made to improve the readability of orthopedic patient education materials.

Patient Perceptions and Current Trends in Internet Use by Orthopedic Outpatients

BACKGROUND

Many studies have highlighted concerns about the completeness and quality of information found online and how this may affect patients' education about their medical problems. One aspect of internet usage that has received less attention in the literature, however, is patient perception of the information that is gathered online, and how patients use it related to their musculoskeletal care.

QUESTIONS/PURPOSES

The objective of the study is to utilize a cross-sectional study design to describe internet usage and patient perceptions of orthopedic online information and to identify differences in usage patterns.

METHODS

One thousand two hundred ninety-six questionnaires were distributed to consecutive patients at orthopedic outpatient clinics which consisted of questions pertaining to patients' internet use. Basic demographic data were collected, and subgroup analyses were performed to examine the effect of three variables (age, gender, and clinic type) on various outcomes.

RESULTS

84.9% of patients reported access to the internet. Of patients with internet access, 64.7% reported using the internet for obtaining orthopedic information. 43.1% of the respondents who searched for orthopedic information rated it as "very useful," 56.3% found it "somewhat useful," and 0.6% found it "not at all useful". Younger patients were more likely to have used the internet for health and orthopedic information and to have found this information either very or somewhat useful. Males were more likely to have found the internet information very useful. Overall, only 33.7% of patients who researched their current orthopedic complaint accessed the institutional website for information.

CONCLUSION

A large proportion of patients use the internet to research orthopedic information and most patients, especially younger males, find the information useful.

Health literacy in the eHealth era: A systematic review of the literature

OBJECTIVES

This study aimed to identify studies on online health service use by people with limited health literacy, as the findings could provide insights into how health literacy has been, and should be, addressed in the eHealth era.

METHODS

To identify the relevant literature published since 2010, we performed four rounds of selection-database selection, keyword search, screening of the titles and abstracts, and screening of full texts. This process produced a final of 74 publications.

RESULTS

The themes addressed in the 74 publications fell into five categories: evaluation of health-related content, development and evaluation of eHealth services, development and evaluation of health literacy measurement tools, interventions to improve health literacy, and online health information seeking behavior.

CONCLUSION

Barriers to access to and use of online health information can result from the readability of content and poor usability of eHealth services. We need new health literacy screening tools to identify skills for adequate use of eHealth services. Mobile apps hold great potential for eHealth and mHealth services tailored to people with low health literacy.

PRACTICE IMPLICATIONS

Efforts should be made to make eHealth services easily accessible to low-literacy individuals and to enhance individual health literacy through educational programs.

Evaluation of the Quality, Accuracy, and Readability of Online Patient Resources for the Management of Articular Cartilage Defects

Objective Patients commonly use the Internet to obtain their health-related information. The purpose of this study was to investigate the quality, accuracy, and readability of online patient resources for the management of articular cartilage defects. Design Three search terms ("cartilage defect," "cartilage damage," "cartilage injury") were entered into 3 Internet search engines (Google, Bing, Yahoo). The first 25 websites from each search were collected and reviewed. The quality and accuracy of online information were independently evaluated by 3 reviewers using predetermined scoring criteria. The readability was evaluated using the Flesch-Kincaid (FK) grade score. Results Fifty-three unique websites were evaluated. Quality ratings were significantly higher in websites with a FK score >11 compared to those with a score of ≤11 ( P = 0.021). Only 10 websites (19%) differentiated between focal cartilage defects and diffuse osteoarthritis. Of these, 7 (70%) were elicited using the search term "cartilage defect" ( P = 0.038). The average accuracy of the websites was high (11.7 out of maximum 12), and the average FK grade level (13.4) was several grades higher than the recommended level for readable patient education material (eighth grade level). Conclusions The quality and readability of online patient resources for articular cartilage defects favor those with a higher level of education. Additionally, the majority of these websites do not distinguish between focal chondral defects and diffuse osteoarthritis, which can fail to provide appropriate patient education and guidance for available treatment. Clinicians should help guide patients toward high-quality, accurate, and readable online patient education material.

Is the Readability of Spine-Related Patient Education Material Improving?: An Assessment of Subspecialty Websites

STUDY DESIGN

Analysis of spine-related patient education materials (PEMs) from subspecialty websites.

OBJECTIVE

The aim of this study was to assess the readability of spine-related PEMs and compare to readability data from 2008.

SUMMARY OF BACKGROUND DATA

Many spine patients use the Internet for health information. Several agencies recommend that the readability of online PEMs should be no greater than a sixth-grade reading level, as health literacy predicts health-related quality of life outcomes. This study evaluated whether the North American Spine Society (NASS), American Association of Neurological Surgeons (AANS), and American Academy of Orthopaedic Surgeons (AAOS) online PEMs meet recommended readability guidelines for medical information.

METHODS

All publicly accessible spine-related entries within the patient education section of the NASS, AANS, and AAOS websites were analyzed for grade level readability using the Flesch-Kincaid formula. Readability scores were also compared with a similar 2008 analysis. Comparative statistics were performed.

RESULTS

A total of 125 entries from the subspecialty websites were analyzed. The average (SD) readability of the online articles was grade level 10.7 (2.3). Of the articles, 117 (93.6%) had a readability score above the sixth-grade level. The readability of the articles exceeded the maximum recommended level by an average of 4.7 grade levels (95% CI, 4.292-5.103; P < 0.001). Compared with 2008, the three societies published more spine-related patient education articles (61 vs. 125, P = 0.045) and the average readability level improved from 11.5 to 10.7 (P = 0.018). Of three examined societies, only one showed significant improvement over time.

CONCLUSION

Our findings suggest that the spine-related PEMs on the NASS, AAOS, and AANS websites have readability levels that may make comprehension difficult for a substantial portion of the patient population. Although some progress has been made in the readability of PEMs over the past 7 years, additional improvement is necessary.

LEVEL OF EVIDENCE

2.

Readability of Spine-Related Patient Education Materials From Leading Orthopedic Academic Centers

STUDY DESIGN

Cross-sectional analysis of online spine-related patient education materials from leading academic centers.

OBJECTIVE

To assess the readability levels of spine surgery-related patient education materials available on the websites of academic orthopedic surgery departments.

SUMMARY OF BACKGROUND DATA

The Internet is becoming an increasingly popular resource for patient education. Yet many previous studies have found that Internet-based orthopedic-related patient education materials from subspecialty societies are written at a level too difficult for the average American; however, no prior study has assessed the readability of spine surgery-related patient educational materials from leading academic centers.

METHODS

All spine surgery-related articles from the online patient education libraries of the top five US News & World Report-ranked orthopedic institutions were assessed for readability using the Flesch-Kincaid (FK) readability test. Mean readability levels of articles amongst the five academic institutions and articles were compared. We also determined the number of articles with readability levels at or below the recommended sixth- or eight-grade levels. Intraobserver and interobserver reliability of readability assessment were assessed.

RESULTS

A total of 122 articles were reviewed. The mean overall FK grade level was 11.4; the difference in mean FK grade level between each department varied significantly (range, 9.3-13.4; P < 0.0001). Twenty-three articles (18.9%) had a readability level at or below the eighth grade level, and only one (0.8%) was at or below the sixth grade level. Intraobserver and interobserver reliability were both excellent (intraclass correlation coefficient of 1 for both).

CONCLUSION

Online patient education materials related to spine from academic orthopedic centers are written at a level too high for the average patient, consistent with spine surgery-related patient education materials provided by the American Academy of Orthopaedic Surgeons and spine subspecialty societies. This study highlights the potential difficulties patients might have in reading and comprehending the information in publicly available education materials related to spine.

LEVEL OF EVIDENCE

N/A.

Thomas N, Yang H, Daniels AH, Born CT. Readability of Trauma-Related Patient Education Materials From the American Academy of Orthopaedic Surgeons

Context:

According to the american medical association (AMA) and the national institutes of health (NIH), the recommended readability of patient education materials should be no greater than a sixth-grade reading level. The online patient education information produced by the american academy of orthopaedic surgeons (AAOS) may be too complicated for some patients to understand. This study evaluated whether the AAOS’s online trauma-related patient education materials meet recommended readability guidelines for medical information.

Evidence Acquisition:

Ninety-nine articles from the “Broken Bones and Injuries” section of the AAOS-produced patient education website, orthoinfo.org, were analyzed for grade level readability using the Flesch-Kincaid formula, a widely-used and validated tool to evaluate the text reading level. Results for each webpage were compared to the AMA/NIH recommended sixth-grade reading level and the average reading level of U.S. adults (eighth-grade).

Results:

The mean (SD) grade level readability for all patient education articles was 8.8 (1.1). All but three of the articles had a readability score above the sixth-grade level. The readability of the articles exceeded this level by an average of 2.8 grade levels (95% confidence interval, 2.6 - 3.0; P < 0.0001). Furthermore, the average readability of the articles exceeded the average reading skill level of U.S. adults (eighth grade) by nearly an entire grade level (95% confidence interval, 0.6-1.0; P < 0.0001).

Conclusions:

The majority of the trauma-related articles from the AAOS patient education website have readability levels that may make comprehension difficult for a substantial portion of the patient population.

Readability of Invasive Procedure Consent Forms

BACKGROUND

Informed consent is a pillar of ethical medicine which requires patients to fully comprehend relevant issues including the risks, benefits, and alternatives of an intervention. Given the average reading skill of US adults is at the 8th grade level, the American Medical Association (AMA) and the National Institutes of Health (NIH) recommend patient information materials should not exceed a 6th grade reading level. We hypothesized that text provided in invasive procedure consent forms would exceed recommended readability guidelines for medical information.

MATERIALS AND METHODS

To test this hypothesis, we gathered procedure consent forms from all surgical inpatient hospitals in the state of Rhode Island. For each consent form, readability analysis was measured with the following measures: Flesch Reading Ease Formula, Flesch-Kincaid Grade Level, Fog Scale, SMOG Index, Coleman-Liau Index, Automated Readability Index, and Linsear Write Formula. These readability scores were used to calculate a composite Text Readability Consensus Grade Level.

RESULTS

Invasive procedure consent forms were found to be written at an average of 15th grade level (i.e., third year of college), which is significantly higher than the average US adult reading level of 8th grade (p < 0.0001) and the AMA/NIH recommended readability guidelines for patient materials of 6th grade (p < 0.0001).

CONCLUSION

Invasive procedure consent forms have readability levels which makes comprehension difficult or impossible for many patients. Efforts to improve the readability of procedural consent forms should improve patient understanding regarding their healthcare decisions.

Readability of Orthopaedic Oncology-related Patient Education Materials Available on the Internet

INTRODUCTION

A person's health literacy is one of the most important indicators of a patient's health status. According to national recommendations, patient education materials should be written at no higher than the sixth- to eighth-grade reading level. The purpose of our study was to assess the readability of online patient education materials related to orthopaedic oncology on the websites of the American Academy of Orthopaedic Surgeons (AAOS), American Cancer Society (ACS), Bone and Cancer Foundation (BCF), and National Cancer Institute (NCI).

METHODS

We searched the online patient education libraries of the AAOS, ACS, BCF, and NCI for all articles related to orthopaedic oncology. The Flesch-Kincaid (FK) readability score was calculated for each article and compared between sources.

RESULTS

A total of 227 articles were identified with an overall mean FK grade level of 9.8. Stratified by source, the mean FK grade levels were 10.1, 9.6, 11.1, and 9.5 for the AAOS, ACS, BCF, and NCI, respectively (P < 0.003). Only 31 articles (14%) and 1 article (0.4%) were at or below the recommended eighth- and sixth-grade levels, respectively.

DISCUSSION

Online patient education materials related to orthopaedic oncology appear to be written at a level above the comprehension ability of the average patient.

Caring for the surgically anxious patient: a review of the interventions and a guide to optimizing surgical outcomes

BACKGROUND

Preoperative surgical anxiety is an unpleasant and common reaction exhibited by patients who are scheduled for surgical procedures. Beyond emotional effects on the patient, it can also have negative repercussions on the surgery including longer hospital stays and poorer outcomes. Given the widespread impacts of preoperative anxiety, it is critical for surgeons to gain a better understanding of how to identify and reduce surgical anxiety in their patients.

DATA SOURCES

This study used the PubMed database to review the current literature to evaluate screening tools and interventions for surgically anxious patients.

CONCLUSIONS

Psychiatric anxiety surveys are currently the most appropriate form of assessment for surgical anxiety. Patient education is important for preventing and reducing anxiety levels in patients. Both nonpharmacological and pharmacological interventions have been shown to be effective in reducing patient anxiety and treatment should be based on patient preference, resources available, and the surgeon's experience.

Evaluating online information regarding the direct anterior approach for total hip arthroplasty

We evaluated the quality of information available on the Internet regarding the direct anterior approach (DAA). The top 50 Web sites from three major search engines (Google, Yahoo!, and Bing) were tabulated utilizing the search term direct anterior hip replacement. Of these, only 22% were authored by a hospital/university, while 60% were by a private physician/clinic. Most Web sites presented the DAA as "better" than other surgical approaches describing benefits, such as accelerated recovery though only 35% described risks of the approach. While only 39% of sites presented patient eligibility criteria, greater than 75% had the ability to make an appointment. Web sites regarding the DAA provide patients with a limited perspective and may be focused on attracting new patients, as opposed to accurately educating them.

Availability of accessible and high-quality information on the Internet for patients regarding the diagnosis and management of rotator cuff tears

BACKGROUND

In the era of informed patient decision-making, educated patients are vital contributors. The Internet provides a vast information source that patients will access. It is imperative that this information be relevant and understandable. Various treatments, each with advantages and disadvantages, are available. We sought to examine the standard of information about the diagnosis and management of rotator cuff tears available to patients on the Internet.

METHODS

We identified 125 websites from searching "rotator cuff tear" in the 5 most popular Internet search engines. The websites were examined for readability by measuring the Flesch Reading Ease Score, the Flesch-Kincaid Grade Level, and the Gunning Fog Index. The quality of the websites was measured by the DISCERN instrument, the Journal of the American Medical Association benchmark criteria, and Health on the Net Foundation certification.

RESULTS

There were 59 individual URLs analyzed. Overall, the quality was poor, with the average DISCERN score being only 39.47. Furthermore, the mean reading grade level was above 9 (recommended level, 6). Health on the Net Foundation certification did correspond to significantly worse readability scores (P = .004) but did not correlate with improved DISCERN scores. Those that satisfied more of the Journal of the American Medical Association benchmark criteria had significantly better DISCERN scores (P < .001).

CONCLUSION

Information about rotator cuff tears is of a low standard and is in many cases written at too high a level for the general population. There are instruments of which we, as surgeons, must be aware to evaluate the resources available and to recommend them to patients to ensure that they understand their condition and treatment options.

Most American Academy of Orthopaedic Surgeons' online patient education material exceeds average patient reading level

BACKGROUND

Advancing health literacy has the potential to improve patient outcomes. The American Academy of Orthopaedic Surgeons' (AAOS) online patient education materials serve as a tool to improve health literacy for orthopaedic patients; however, it is unknown whether the materials currently meet the National Institutes of Health/American Medical Association's recommended sixth grade readability guidelines for health information or the mean US adult reading level of eighth grade.

QUESTIONS/PURPOSES

The purposes of this study were (1) to evaluate the mean grade level readability of online AAOS patient education materials; and (2) to determine what proportion of the online materials exceeded recommended (sixth grade) and mean US (eighth grade) reading level.

METHODS

Reading grade levels for 99.6% (260 of 261) of the online patient education entries from the AAOS were analyzed using the Flesch-Kincaid formula built into Microsoft Word software.

RESULTS

Mean grade level readability of the AAOS patient education materials was 9.2 (SD ± 1.6). Two hundred fifty-one of the 260 articles (97%) had a readability score above the sixth grade level. The readability of the AAOS articles exceeded the sixth grade level by an average of 3.2 grade levels. Of the 260 articles, 210 (81%) had a readability score above the eighth grade level, which is the average reading level of US adults.

CONCLUSIONS

Most of the online patient education materials from the AAOS had readability levels that are far too advanced for many patients to comprehend. Efforts to adjust the readability of online education materials to the needs of the audience may improve the health literacy of orthopaedic patients. Patient education materials can be made more comprehensible through use of simpler terms, shorter sentences, and the addition of pictures. More broadly, all health websites, not just those of the AAOS, should aspire to be comprehensible to the typical reader.

Readability of patient education materials on the American Orthopaedic Society for Sports Medicine website

BACKGROUND

The recommended readability of patient education materials by the American Medical Association (AMA) and National Institutes of Health (NIH) should be no greater than a sixth-grade reading level. However, online resources may be too complex for some patients to understand, and poor health literacy predicts inferior health-related quality of life outcomes.

AIM

This study evaluated whether the American Orthopaedic Society for Sports Medicine (AOSSM) website's patient education materials meet recommended readability guidelines for medical information. We hypothesized that the readability of these online materials would have a Flesch-Kincaid formula grade above the sixth grade.

METHODS

All 65 patient education entries of the AOSSM website were analyzed for grade level readability using the Flesch-Kincaid formula, a widely used and validated tool to evaluate the text reading level.

RESULTS

The average (standard deviation) readability of all 65 articles was grade level 10.03 (1.44); 64 articles had a readability score above the sixth-grade level, which is the maximum level recommended by the AMA and NIH. Mean readability of the articles exceeded this level by 4.03 grade levels (95% CI, 3.7-4.4; P < 0.0001). We found post-hoc that only 7 articles had a readability score ≤ an eighth-grade level, the average reading level of US adults. Mean readability of the articles exceeded this level by 2.03 grade levels (95% CI, 1.7-2.4; P < 0.0001).

CONCLUSION

The readability of online AOSSM patient education materials exceeds the readability level recommended by the AMA and NIH, and is above the average reading level of the majority of US adults. This online information may be of limited utility to most patients due to a lack of comprehension. Our study provides a clear example of the need to improve the readability of specific education material in order to maximize the efficacy of multimedia sources.

Educational resources for patients considering a left ventricular assist device: a cross-sectional review of internet, print, and multimedia materials

BACKGROUND

Left ventricular assist devices (LVADs) are being used with increasing frequency to treat severe heart failure. Patients seek out informational resources when considering implantation. The primary study objective was to characterize the scope and quality of available LVAD educational materials.

METHODS AND RESULTS

In July 2013, we performed a cross-sectional search of Internet, print, and multimedia resources available to patients considering LVAD. Written materials <10 sentences, videos <2 minutes, and materials clearly directed to healthcare professionals were excluded. Seventy-seven materials met inclusion criteria. Potential benefits of LVAD therapy were discussed in all (n=77), whereas less often mentioned were risks (n=43), lifestyle considerations (n=29), surgical details (n=26), caregiver information (n=9), and hospice or palliative care (n=2). Of the 14 materials that recognized a decision or alternate treatment option, 7 used outdated statistics, 12 scored above an eighth grade reading comprehension level, and 12 met <50% of International Patient Decision Aid Standards criteria. In the survey participants rated all but one as biased toward accepting LVAD therapy.

CONCLUSIONS

Although many resources exist for patients considering an LVAD, the content is suboptimal. Benefits of LVADs are often presented in the absence of risks, alternative options, and caregiver considerations. Most materials use outdated statistics, are above the reading level of average Americans, and are biased toward accepting LVAD therapy. There is no tool that would qualify as a formal decision aid.

Evaluating the quality of Internet information for femoroacetabular impingement

PURPOSE

The Internet has become a ubiquitous source of medical information for both the patient and the physician. However, the quality of this information is highly variable. We evaluated the quality of Internet information available for femoroacetabular impingement (FAI).

METHODS

Four popular search engines were used to collect 100 Web sites containing information on FAI. Web sites were evaluated based on authorship, various content criteria, and the presence of Health On the Net Code of Conduct (HONcode) certification. By use of a novel evaluation system for quality, Web sites were also classified as excellent, high, moderate, poor, or inadequate and were subsequently analyzed. Web sites were evaluated as a group, followed by authorship type, by HONcode certification, and by quality level.

RESULTS

Of the Web sites, 73 offered the ability to contact the author, 91 offered a considerable explanation of FAI, 54 provided surgical treatment options, 58 offered nonsurgical treatment options, 27 discussed possible complications, 11 discussed eligibility criteria, 31 discussed rehabilitation, 67 discussed a differential diagnosis, and 48 included peer-reviewed citations. We categorized 40 Web sites as academic, 33 as private, 9 as industry, 9 as public education, and 9 as blogs. Our novel quality evaluation system classified 16 Web sites as excellent, 18 as high, 17 as moderate, 18 as poor, and 31 as inadequate. Only 8% of all evaluated Web sites contained HONcode certification.

CONCLUSIONS

We found that the quality of information available on the Internet about FAI was dramatically variable. A significantly large proportion of Web sites were from academic sources, but this did not necessarily indicate higher quality. Sites with HONcode certification showed as much variability in quality as noncertified sites.

CLINICAL RELEVANCE

This study increases clinician competence in the available Internet information about FAI and helps them to confidently guide patients to formulate appropriate medical decisions based on high-quality information.

Readability of patient education materials on the american association for surgery of trauma website

Background:

Because the quality of information on the Internet is of dubious worth, many patients seek out reliable expert sources. As per the American Medical Association (AMA) and the National Institutes of Health (NIH) recommendations, readability of patient education materials should not exceed a sixth-grade reading level. The average reading skill of U.S. adults is at the eighth-grade level.

Objectives:

This study evaluates whether a recognized source of expert content, the American Association for Surgery of Trauma (AAST) website’s patient education materials, recommended readability guidelines for medical information.

Materials and Methods:

Using the well-validated Flesch-Kincaid formula to analyze grade level readability, we evaluated the readability of all 16 of the publicly-accessible entries within the patient education section of the AAST website.

Results:

Mean ± SD grade level readability was 10.9 ± 1.8 for all the articles. All but one of the articles had a readability score above the sixth-grade level. Readability of the articles exceeded the maximum recommended level by an average of 4.9 grade levels (95% confidence interval, 4.0-5.8; P < 0.0001). Readability of the articles exceeded the eighth-grade level by an average of 2.9 grade levels (95% confidence interval, 2.0-3.8; P < 0.0001). Only one of the articles had a readability score below the eighth-grade level.

Conclusions:

The AAST’s online patient education materials may be of limited utility to many patients, as the readability of the information exceeds the average reading skill level of adults in the U.S. Lack of patient comprehension represents a discrepancy that is not in accordance with the goals of the AAST’s objectives for its patient education efforts.