Readability and quality assessment of internet-based patient education materials related to nasal septoplasty

Quality and Readability Analysis of Turbinoplasty on the Web

This study aimed to assess the quality, credibility, and readability of online health information concerning turbinoplasty, given the increasing reliance on internet resources for health education. Using four search terms related to turbinoplasty, we analyzed 71 text-based webpages from Google.com, Bing.com, and Yahoo.com. Readability was evaluated using the Flesch-Kincaid grade level, Gunning-Fog Index, SMOG Index, and Coleman-Liau Index were utilized. Web page quality was computed through the DISCERN Instrument (DISCERN), the Journal of the American Medical Association benchmark criteria (JAMA), Novel Turbinoplasty Index (NTI), and presence of code certification by Health on The Net (HON). Seventy-one text-based web pages were assessed. Information quality was measured by an average DISCERN score of 47.4±7.40, indicating "fair" quality. The average readability was a grade level of 9.7±1.57, notably higher than AMA and NIH recommendations. Of all web pages, only 11 (15.49%) proficiently met all 4 listed JAMA criteria. Significant correlations between web page classification and average DISCERN (P=0.0042), as well as JAMA score (P<0.001) were discovered. The web pages that had HON code certification revealed significantly higher quality metrics such as DISCERN scores (P<0.001), JAMA scores (P<0.001), and NTI scores (P=0.038). Online health information for turbinoplasty is of "fair" quality, and the average readability is several grade levels above current AMA and NIH recommendations. Health care providers should aim to guide their patients on finding appropriate educational resources and should improve the readability of their patient education materials.

Improving readability and comprehension levels of otolaryngology patient education materials using ChatGPT

OBJECTIVE

A publicly available large language learning model platform may help determine current readability levels of otolaryngology patient education materials, as well as translate these materials to the recommended 6th-grade and 8th-grade reading levels.

STUDY DESIGN

Cross-sectional analysis.

SETTING

Online using large language learning model, ChatGPT.

METHODS

The Patient Education pages of the American Laryngological Association (ALA) and American Academy of Otolaryngology-Head and Neck Surgery (AAO-HNS) websites were accessed. Materials were input into ChatGPT (OpenAI, San Francisco, CA; version 3.5) and Microsoft Word (Microsoft, Redmond, WA; version 16.74). Programs calculated Flesch Reading Ease (FRE) scores, with higher scores indicating easier readability, and Flesch-Kincaid (FK) grade levels, estimating U.S. grade level required to understand text. ChatGPT was prompted to "translate to a 5th-grade reading level" and provide new scores. Scores were compared for statistical differences, as well as differences between ChatGPT and Word gradings.

RESULTS

Patient education materials were reviewed and 37 ALA and 72 AAO-HNS topics were translated. Overall FRE scores and FK grades demonstrated significant improvements following translation of materials, as scored by ChatGPT (p < 0.001). Word also scored significant improvements in FRE and FK following translation by ChatGPT for AAO-HNS materials overall (p < 0.001) but not for individual topics or for subspecialty-specific categories. Compared with Word, ChatGPT significantly exaggerated the change in FRE grades and FK scores (p < 0.001).

CONCLUSION

Otolaryngology patient education materials were found to be written at higher reading levels than recommended. Artificial intelligence may prove to be a useful resource to simplify content to make it more accessible to patients.

Assessing the Readability, Reliability, and Quality of AI-Modified and Generated Patient Education Materials for Endoscopic Skull Base Surgery

BACKGROUND

Despite National Institutes of Health and American Medical Association recommendations to publish online patient education materials at or below sixth-grade literacy, those pertaining to endoscopic skull base surgery (ESBS) have lacked readability and quality. ChatGPT is an artificial intelligence (AI) system capable of synthesizing vast internet data to generate responses to user queries but its utility in improving patient education materials has not been explored.

OBJECTIVE

To examine the current state of readability and quality of online patient education materials and determined the utility of ChatGPT for improving articles and generating patient education materials.

METHODS

An article search was performed utilizing 10 different search terms related to ESBS. The ten least readable existing patient-facing articles were modified with ChatGPT and iterative queries were used to generate an article de novo. The Flesch Reading Ease (FRE) and related metrics measured overall readability and content literacy level, while DISCERN assessed article reliability and quality.

RESULTS

Sixty-six articles were located. ChatGPT improved FRE readability of the 10 least readable online articles (19.7 ± 4.4 vs. 56.9 ± 5.9, p < 0.001), from university to 10th grade level. The generated article was more readable than 48.5% of articles (38.9 vs. 39.4 ± 12.4) and higher quality than 94% (51.0 vs. 37.6 ± 6.1). 56.7% of the online articles had "poor" quality.

CONCLUSIONS

ChatGPT improves the readability of articles, though most still remain above the recommended literacy level for patient education materials. With iterative queries, ChatGPT can generate more reliable and higher quality patient education materials compared to most existing online articles and can be tailored to match readability of average online articles.

Patient education in the digital age: An analysis of quality and readability of online information on rhinoplasty

This study aimed to investigate quality and readability of online rhinoplasty information provided on Turkish websites. We searched for the terms "rhinoplasty" (rinoplasti) and "nose job" (burun estetiği) in Turkish using the Google search engine in May 2023. The first 30 sites for each term were included in the evaluation. We used the DISCERN tool to evaluate quality and the Atesman and Cetinkaya-Uzun formulas to assess readability. According to the Atesman formula, the readability scores of all the websites were moderately difficult. According to the Cetinkaya-Uzun formula, the readability scores of websites were at the instructional reading level. The mean total DISCERN score was 2.33 ± 0.60, indicating poor quality. No statistically significant correlations were found between the Atesman or Cetinkaya-Uzun readability scores and the DISCERN scores across all websites (P > .05). Our analysis revealed key areas in which Turkish websites can improve the quality and readability of rhinoplasty information to support decision-making.

Quality and Readability of Noise-Induced Hearing Loss Prevention-Related Websites

OBJECTIVE

This cross-sectional website analysis aimed to determine the readability and quality of English and Spanish websites pertaining to the prevention of noise-induced hearing loss.

STUDY DESIGN

Cross-sectional website analysis.

SETTING

Various online search engines.

METHODS

We queried four popular search engines using the term "noise-induced hearing loss prevention" to reveal the top 50 English and top 50 Spanish websites for data collection. Websites meeting inclusion criteria were stratified based on the presence of a Health on the Net Code certificate (independent assessment of honesty, reliability, and quality). Websites were then independently reviewed by experts using the DISCERN criteria in order to assess information quality. Readability was calculated using the Flesch reading ease score for English and the Fernandez-Huerta formula for Spanish websites.

RESULTS

Thirty-six English websites and 32 Spanish websites met the inclusion criteria. English websites had significantly lower readability (average = 56.34, SD = 11.17) compared to Spanish websites (average = 61.88, SD = 5.33) (P < .05). Spanish websites (average = 37, SD = 8.47) were also of significantly higher quality than English websites (average = 25.13, SD = 10.11).

CONCLUSION

This study emphasizes the importance of providing quality and readable materials to patients seeking information about noise-induced hearing loss prevention. All of the English and Spanish websites reviewed were written at a level higher than the American Medical Association-recommended sixth-grade reading level. The study also highlights the need for evidence-based information online provided by experts in our field.

Lessons to be learned when designing comprehensible patient-oriented online information about temporomandibular disorders

BACKGROUND

Temporomandibular disorders (TMD) are a prevalent ailment with a global impact, affecting a substantial number of individuals. While some individuals are receiving treatment from orthodontists for TMD, a significant proportion of individuals obtain knowledge through websites.

OBJECTIVES

Our purpose had been to evaluate, from a patient-oriented perspective, the readability of home pages of websites scored in the 10 most prominent devoted to TMD. We also determined what level of education would have been needed to get an overview of the information on the websites under scrutiny. This approach ensures that our findings are centred on the patient experience, providing insights into how accessible and understandable websites about TMD.

METHODS

We determined the top 10 patient-focused English language websites by searching for 'temporomandibular disorders' in the 'no country redirect' plugin of the Google Chrome browser (www.google.com/ncr). The readability of the texts was assessed using the Gunning fog index (GFI), Coleman Liau index (CLI), Automated readability index (ARI) Simple Measure of Gobbledygook (SMOG), Flesch Kincald grade level (FKGL), Flesh reasing ease (FRE) (https://readabilityformulas.com).

RESULTS

The mean Flesch reading ease index score was determined to be 48.67, accompanied by a standard deviation of 15.04 and these websites require an average of 13.49 years of formal education (GFI), with a standard deviation of 2.62, for ease of understanding.

CONCLUSION

Our research indicates that a significant proportion of websites related to TMD can be defined as a level of complexity that exceeds the ability to read comprehension of the general population.

A quality analysis of internet patient information for immediate, sequential, bilateral cataract surgery (ISBCS)

BACKGROUND

Due to surgical capacity pressures, induced by the COVID-19 pandemic, immediate bilateral simultaneous cataract surgery (ISBCS) has been utilised increasingly throughout the U.K. This surgical method comes with both novel risks and benefits, so the consent process must be modified. Prior randomised trials have demonstrated that appropriate online information may act as an adjunct to the surgical consent process. This study aims to assess the quality of available internet information for ISBCS.

METHODS

Terms searched were 'bilateral cataract surgery patient information', 'double cataract surgery patient information', and 'immediate sequential bilateral cataract surgery patient information'. The Google search engine was used. The DISCERN instrument and JAMA benchmarks were used to assess healthcare information for quality. The Flesch Reading Ease Score (FRE), Flesch Kincaid Grade (FKG) Level and Gunning Fog Score (GFS) were used to assess for readability. HONcode certification was used to assess transparency and quality.

RESULTS

Forty-six websites were found. The average DISCERN score was 41.3, meaning a "fair" quality which is below what many patients would anticipate discovering when trying to find information. National Healthcare Service websites had higher DISCERN scores than private healthcare-provided websites (p < 0.01; 95% CI: 1.13-1.88.).

CONCLUSIONS

Fair patient information for ISBCS has been demonstrated. Specific internet information sources with appropriate information should be further developed, with cited sources, and patients signposted to them if felt appropriate.

Study to evaluate the readability and visual appearance of online resources for blunt chest trauma: an evaluation of online resources using mixed methods

OBJECTIVES

Blunt chest trauma (BCT) is characterised by forceful and non-penetrative impact to the chest region. Increased access to the internet has led to online healthcare resources becoming used by the public to educate themselves about medical conditions. This study aimed to determine whether online resources for BCT are at an appropriate readability level and visual appearance for the public.

DESIGN

We undertook a (1) a narrative overview assessment of the website; (2) a visual assessment of the identified website material content using an adapted framework of predetermined key criteria based on the Centers for Medicare and Medicaid Services toolkit and (3) a readability assessment using five readability scores and the Flesch reading ease score using Readable software.

DATA SOURCES

Using a range of key search terms, we searched Google, Bing and Yahoo websites on 9 October 2023 for online resources about BCT.

RESULTS

We identified and assessed 85 websites. The median visual assessment score for the identified websites was 22, with a range of -14 to 37. The median readability score generated was 9 (14-15 years), with a range of 4.9-15.8. There was a significant association between the visual assessment and readability scores with a tendency for websites with lower readability scores having higher scores for the visual assessment (Spearman's r=-0.485; p<0.01). The median score for Flesch reading ease was 63.9 (plain English) with a range of 21.1-85.3.

CONCLUSIONS

Although the readability levels and visual appearance were acceptable for the public for many websites, many of the resources had much higher readability scores than the recommended level (8-10) and visually were poor.Better use of images would improve the appearance of websites further. Less medical terminology and shorter word and sentence length would also allow the public to comprehend the contained information more easily.

Assessing the readability and quality of online information on anosmia

INTRODUCTION

Anosmia can have a significant impact on well-being and quality of life. Due to an ageing population and the coronavirus disease 2019, increasing numbers of patients are seeking online information on anosmia. This report systematically assesses the readability and quality of online information on anosmia.

METHODS

The terms 'anosmia' and 'loss of smell' were entered into Google. The first 50 websites generated for each search term were screened. Readability was assessed using the Flesch-Kincaid Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), Simple Measure of Gobbledygook (SMOG) Index and Gunning Fog Index (GFI). Quality was assessed using the DISCERN instrument. Spearman's correlation between quality and readability was calculated.

RESULTS

A total of 79 websites met the inclusion criteria. The mean and 95% confidence interval for the FRES, FKGL, SMOG, GFI and DISCERN scores were 46.31 (42.94-49.68), 12.00 (11.27-12.73), 10.70 (10.16-11.23), 14.62 (13.85-15.39) and 2.90 (2.69-3.11), respectively. Significant negative correlation was noted between the DISCERN and FRES (rs=-0.500; p<0.05).

DISCUSSION

Online information on anosmia is written above the recommended reading age guidance in the UK, and has moderate deficiencies in quality. As a result, the information may be used inappropriately and could result in worse health outcomes. We recommend that patients are directed to websites produced by health providers or nonprofit organisations that develop material for patient health education.

CONCLUSIONS

Online information on anosmia is of low readability and moderate quality. Healthcare professionals should direct patients towards high-quality resources written for the layperson.

Readability and quality of online patient health information on parotidectomy

OBJECTIVE

Complications of parotidectomy can have a massive impact on patients' quality of life. This study aimed to evaluate the readability and quality of online health information on parotidectomy.

METHOD

The search terms 'parotidectomy', 'parotid surgery', 'parotidectomy patient information' and 'parotid surgery patient information' were parsed through three popular search engines.

RESULTS

The websites were analysed using readability scores of the Flesch Reading Ease test and the Gunning Fog Index. The DISCERN instrument was used to assess quality and reliability. The average Flesch Reading Ease score was 50.2 ± 9.0, indicating that the materials were fairly difficult to read, the Gunning Fog Index score showed that the patient health information was suitable for an individual above 12th grade level, and the DISCERN score indicated that the online patient health information had fair quality. The Kruskal-Wallis test showed a significant difference in Flesch Reading Ease and DISCERN tool scores according to website category (p < 0.05).

CONCLUSION

Current online patient health information on parotidectomy is too difficult for the public to understand, and it exceeds the reading levels recommended by Health Education England and the American Medical Association.

The Complexity of Online Patient Education Materials About Interventional Neuroradiology Procedures Published by Major Academic Institutions

Introduction Health literacy is an independent predictor of population health status and is directly related to the readability of available patient education material. The National Institutes of Health (NIH) and the American Medical Association have recommended that patient education materials (PEMs) be written between a fourth- and a sixth-grade education level. The authors assessed the readability of online PEMs about neurointerventional procedures that have been published by several academic institutions across the US. Methods Online PEMs regarding five common neurointerventional procedures, including mechanical thrombectomy for large vessel occlusion, cerebral diagnostic angiography, carotid artery stenting, endovascular aneurysm embolization, and epidural steroid injection collected from the websites of 20 top institutions in Neurology and Neurosurgery. The materials were assessed via five readability scales and then were statistically analyzed and compared to non-institutional education websites (Wikipedia.com and WebMD.com). Results None of the PEMs were written at or below the NIH's recommended 6th-grade reading level. The average educational level required to comprehend the texts across all institutions, as assessed by the readability scales, was 10-11^th grade level. Some materials required a college-level education or higher. Material from non-institutional websites had significantly lower readability scores compared to the 20 institutions. Conclusions Current PEMs related to neurointerventional procedures are not written at or below the NIH's recommended fourth- to sixth-grade education level. Given the complexity of those procedures, significant attention should be pointed toward an improvement in the available online materials.

Readability Assessment of Online Peripheral Artery Disease Education Materials

OBJECTIVE

Online resources can be a valuable source of information for patients and have been shown to result in more inquiry during medical office visits, following physician medical recommendation more closely, and making self-directed lifestyle changes. The accessibility to these resources is limited by the readability level of the article and the literacy level of the population. Peripheral artery disease (PAD) is estimated to affect between 8 and 12 million people in the United States with greater disease severity among under insured or un-insured populations. As PAD continues to increase in prevalence, it is imperative that patients have access to comprehensible patient-centered health information. This study aims to evaluate the readability of online PAD patient education materials.

METHODS

The search engine Google was utilized to collect the first 25 patient-accessible online articles pertaining to the search term "peripheral artery disease." Articles were then categorized by source type: hospital, professional society, or other. Readability was measured using the following tests: Automated Readability Index (ARI), Coleman-Liau Index, (CLI) Flesch-Kincaid Grade Level (FKGL), Gunning Fog, Linsear Write Formula, and the SMOG Index. Statistical analyses were performed using Statistical Analysis Software (SAS), with p-values <0.05 being statistically significant.

RESULTS

Twenty-five articles were categorized by source and statistically analyzed. The average readability of PAD patient education materials was 10.8 and significantly above the AMA, NIH, and USDHHS recommended reading level of 6th grade. Readability scores among source categories were not significantly different.

CONCLUSIONS

Commonly available online PAD resources are written at a grade level above that currently recommended by medical societies. Hospitals, professional societies, and other stakeholders in PAD patient education should take into consideration the readability of their materials to make medicine more accessible. Readable articles may combat historic, and structural racism often found in our health care system that marginalizes those with lower health literacy. It is imperative to develop patient education at an appropriate level to enrich patient autonomy.

Quality and readability of online information on plantar fasciitis and calcaneal spur

Plantar fasciitis and calcaneal spur are common causes of heel pain in the community. People use the Internet to obtain medical information about diseases. We reviewed Internet information sources on plantar fasciitis and calcaneal spur for quality and readability. The first 50 websites for each search term ("calcaneal spur", "heel spur", and "plantar fasciitis") were scanned on www.google.com . Six different valid tools were used for information quality and readability assessment. We searched for HONCode (Health On the Net Foundation Code) stamps on included websites. The total mean points for DISCERN were 50.52 ± 14.62, and the total mean points for JAMA (Journal of the American Medical Association) were 2.42 ± 1.26. In total, 25.72% of 97 websites had HONCode stamps. The average scores for the readability indicators were calculated to be Flesch-Kincaid Grade Level (FKGL): 7.27 ± 1.71, Gunning Fog: 8.46 ± 2.17, Simple Measure of Gobbledygook (SMOG): 6.89 ± 1.24, and Coleman Liau Index: 15.56 ± 1.85. In our study, when the website resources were examined, there were profit websites the most and website quality and readability were moderate level. A significant proportion of the websites have a financial bias and provide low-quality information. A mechanism for monitoring the quality and readability of online information must be established and managed systematically.

Quality and Readability of Online Patient Education Materials Related to Branchial Cleft Cysts

Introduction

Branchial cleft cysts are the second most common congenital neck mass and can cause significant anxiety for patients and families despite their benign nature. Education through online patient education materials (PEMs) is critical for informing patients and reducing stress. We aimed to determine the content, quality, and readability of online PEMs related to branchial cleft cysts.

Methods

The search engine Google was used to collect the first 100 website results for the query “branchial cleft cyst.” PEMs were included and assessed for content, quality via the DISCERN tool, and readability via Flesch Reading Ease Score (FRES), Flesch-Kincaid Reading Grade Level (FKGL), Gunning Frequency of Gobbledygook (GFOG), and Simple Measure of Gobbledygook (SMOG).

Results

Twenty-six websites containing PEMs related to branchial cleft cysts were assessed. Most websites were from universities or medical centers and did not contain any media. The mean DISCERN score was 49.3 (SD: 11.1, Median: 52.5), the mean FRES score was 51.9 (SD: 12.1, Median: 54.0), the mean FKGL score was 10.35 (SD: 2.52, Median: 9.95), the mean GFOG score was 13.32 (SD: 2.52, Median: 13.00), and the mean SMOG score was 10.25 (SD: 1.83, Median: 9.95). DISCERN was not significantly correlated with FRES, FKGL, GFOG, or SMOG.

Conclusion

Online PEMs related to branchial cleft cysts are consistently written above the National Institutes of Health (NIH) recommended sixth-grade reading level and are often of unsatisfactory overall quality. Writers of online PEMs for branchial cleft cysts should consider the readability and quality of their materials to improve patient education and reduce anxiety.

Readability of Online Materials Related to Vocal Cord Leukoplakia

Objectives

To assess readability and understandability of online materials for vocal cord leukoplakia.

Study Design

Review of online materials.

Setting

Academic medical center.

Methods

A Google search of "vocal cord leukoplakia" was performed, and the first 50 websites were considered for analysis. Readability was measured by the Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level (FKGL), and Simple Measure of Gobbledygook (SMOG). Understandability and actionability were assessed by 2 independent reviewers with the PEMAT-P (Patient Education Materials Assessment Tool for Printable Materials). Unpaired t tests compared scores between sites aimed at physicians and those at patients, and a Cohen's kappa was calculated to measure interrater reliability.

Results

Twenty-two websites (17 patient oriented, 5 physician oriented) met inclusion criteria. For the entire cohort, FRES, FKGL, and SMOG scores (mean ± SD) were 36.90 ± 20.65, 12.96 ± 3.28, and 15.65 ± 3.57, respectively, indicating that materials were difficult to read at a >12th-grade level. PEMAT-P understandability and actionability scores were 73.65% ± 7.05% and 13.63% ± 22.47%. Statistically, patient-oriented sites were more easily read than physician-oriented sites (P < .02 for each of the FRES, FKGL, and SMOG comparisons); there were no differences in understandability or actionability scores between these categories of sites.

Conclusion

Online materials for vocal cord leukoplakia are written at a level more advanced than what is recommended for patient education materials. Awareness of the current ways that these online materials are failing our patients may lead to improved education materials in the future.

Effectiveness of modified cutting and suture technique for endonasal caudal septoplasty in correcting nasal obstruction and preventing nasal tip projection loss

PURPOSE

Caudal septoplasty is a difficult procedure. The cutting and suture technique is suitable for caudal septoplasty, but a batten graft is always necessary and bears the risk of nasal tip projection loss. We established a modified cutting and suture technique (MCAST), without using a batten graft, and investigated its effectiveness in correcting nasal obstruction and preventing nasal tip projection loss.

METHODS

We retrospectively reviewed the medical records of 22 patients who underwent caudal septoplasty using MCAST. Subjective assessment by Nasal Obstruction Symptom Evaluation (NOSE) score and objective assessment by computed tomography (CT) were performed before and after the surgery. For evaluating nasal tip projection, we asked patients about their awareness of external nasal deformity. Additionally, the nasal tip projection was measured by CT and compared before and after surgery.

RESULTS

The median preoperative NOSE score reduced significantly after MCAST (P < 0.001). On CT, the ratio of the area of the convex side to that of the concave side in the anterior portion of the nasal cavity increased significantly after MCAST (P < 0.001). All patients were unaware of external nasal deformity. There were no significant differences in the mean preoperative and postoperative nasal tip height and nasolabial angle. The mean supra tip height was significantly greater postoperatively than preoperatively (P = 0.02).

CONCLUSIONS

The MCAST was useful for correcting nasal obstruction with caudal septal deviation. There was no postoperative loss of nasal tip projection. The MCAST can be suitable for correcting C-shaped caudal deviations without dislocating the caudal septum from the anterior nasal septum.