The Readability of Online Patient Resources for Skin Cancer Treatment

Study to evaluate the readability and visual appearance of online resources for blunt chest trauma: an evaluation of online resources using mixed methods

OBJECTIVES

Blunt chest trauma (BCT) is characterised by forceful and non-penetrative impact to the chest region. Increased access to the internet has led to online healthcare resources becoming used by the public to educate themselves about medical conditions. This study aimed to determine whether online resources for BCT are at an appropriate readability level and visual appearance for the public.

DESIGN

We undertook a (1) a narrative overview assessment of the website; (2) a visual assessment of the identified website material content using an adapted framework of predetermined key criteria based on the Centers for Medicare and Medicaid Services toolkit and (3) a readability assessment using five readability scores and the Flesch reading ease score using Readable software.

DATA SOURCES

Using a range of key search terms, we searched Google, Bing and Yahoo websites on 9 October 2023 for online resources about BCT.

RESULTS

We identified and assessed 85 websites. The median visual assessment score for the identified websites was 22, with a range of -14 to 37. The median readability score generated was 9 (14-15 years), with a range of 4.9-15.8. There was a significant association between the visual assessment and readability scores with a tendency for websites with lower readability scores having higher scores for the visual assessment (Spearman's r=-0.485; p<0.01). The median score for Flesch reading ease was 63.9 (plain English) with a range of 21.1-85.3.

CONCLUSIONS

Although the readability levels and visual appearance were acceptable for the public for many websites, many of the resources had much higher readability scores than the recommended level (8-10) and visually were poor.Better use of images would improve the appearance of websites further. Less medical terminology and shorter word and sentence length would also allow the public to comprehend the contained information more easily.

Can Large Language Models Generate Outpatient Clinic Letters at First Consultation That Incorporate Complication Profiles From UK and USA Aesthetic Plastic Surgery Associations?

The importance of written communication between clinicians and patients, especially in the wake of the Supreme Court case of Montgomery vs Lanarkshire, has led to a shift toward patient-centric care in the United Kingdom. This study investigates the use of large language models (LLMs) like ChatGPT and Google Bard in enhancing clinic letters with gold-standard complication profiles, aiming to improve patients' understanding and save clinicians' time in aesthetic plastic surgery. The aim of this study is to assess the effectiveness of LLMs in integrating complication profiles from authoritative sources into clinic letters, thus enhancing patient comprehension and clinician efficiency in aesthetic plastic surgery. Seven widely performed aesthetic procedures were chosen, and complication profiles were sourced from the British Association of Aesthetic Plastic Surgeons (BAAPS) and the American Society of Plastic Surgeons (ASPS). We evaluated the proficiency of the ChatGPT4, ChatGPT3.5, and Google Bard in generating clinic letters which incorporated complication profiles from online resources. These letters were assessed for readability using an online tool, targeting a recommended sixth-grade reading level. ChatGPT4 achieved the highest compliance in integrating complication profiles from BAAPS and ASPS websites, with average readability grades between eighth and ninth. ChatGPT3.5 and Google Bard showed lower compliance, particularly when accessing paywalled content like the ASPS Informed Consent Bundle. In conclusion, LLMs, particularly ChatGPT4, show promise in enhancing patient communications in aesthetic plastic surgery by effectively incorporating standard complication profiles into clinic letters. This aids in informed decision making and time saving for clinicians. However, the study underscores the need for improvements in data accessibility, search capabilities, and ethical considerations for optimal LLM integration into healthcare communications. Future enhancements should focus on better interpretation of inaccessible formats and a Human in the Loop approach to combine Artifical Intelligence capabilities with clinician expertise.

Level of Evidence 3

Assessment of online patient education material for eye cancers: A cross-sectional study

The objective of this study was to assess online American patient education material (PEM) related to eye cancers in order to determine the quality of the content and appropriateness of the contents' reading level as it relates to the American population. PEMs were extracted from fifteen American cancer and ophthalmology associations and evaluated for their reading level using ten validated readability scales. PEMs then had all words extracted and evaluated for their difficulty and familiarity. The quality of the PEMS were assessed according to DISCERN, Heath On the Net Foundation Code of Conduct (HONCode), and JAMA benchmarks. Overall, online PEMs from the associations were written at a 11th grade reading level, which is above the recommended 6th grade reading level. The difficult word analysis identified that 26% of words were unfamiliar. Only one of the fifteen association held a HONCode certification while no organization met the standards of all four JAMA benchmarks. The average score for DISCERN was 2.4 out of a total of 5 for the fifteen questions related to treatment option information quality. Consideration should be made to create PEMs at an appropriate grade reading level to encourage health literacy and ultimately promote health outcomes. Associations should also focus on incorporating easily identifiable quality indicators to allow patients to better identify reputable resources.

Accessing publicly funded donor insemination treatment in the UK: is funding information available on fertility clinic websites?

We sought to find out if information about public funding for regulated donor insemination (DI) was available on UK fertility clinic websites, and if so, what information was provided for same-sex couples and single women; and if the available information was easily readable. The 'Choose a fertility clinic' pages of the Human Fertilisation and Embryology Authority (HFEA) website were used to identify all licensed fertility clinics in the UK, and any available text on public funding for DI treatment was extracted. The Flesch reading ease scores were calculated to determine the readability of the extracted text. Of the 52 clinics included in the synthesis, 23 mentioned public funding, and for 16 of these, it was unclear whether public funding was available. Six of the 23 clinics mentioned public funding for same-sex couples, and two mentioned public funding for single women. The Flesch reading ease scores indicated that none of the text about funding for DI treatment on any of the clinic websites met the NHS-advised level of readability for health information. Fertility clinic websites should specify whether they offer publicly funded treatment, and to whom, as well as clearly stating the eligibility criteria, using suitably readable language to communicate this.

An evaluation of the readability and visual appearance of online patient resources for fibroadenoma

INTRODUCTION

Fibroadenomas are benign lesions found in the breast tissue. Widespread access to and use of the internet has resulted in more individuals using online resources to better understand health conditions, their prognosis and treatment. The aim of this study was to investigate the readability and visual appearance of online patient resources for fibroadenoma.

METHODS

We searched GoogleTM, BingTM and YahooTM on 6 July 2022 using the search terms "fibroadenoma", "breast lumps", "non-cancerous breast lumps", "benign breast lumps" and "benign breast lesions" to identify the top ten websites that appeared on each of the search engines. We excluded advertised websites, links to individual pdf documents and links to blogs/chats. We compiled a complete list of websites identified using the three search engines and the search terms and analysed the content. We only selected pages that were relevant to fibroadenoma. We excluded pages which only contained contact details and no narrative information relating to the condition. We did not assess information where links were directed to alternative websites. We undertook a qualitative visual assessment of each of the websites using a framework of pre-determined key criteria based on the Centers for Medicare and Medicaid Services toolkit. This involved assessing characteristics such as overall design, page layout, font size and colour. Each criterion was scored as: +1- criterion achieved; -1- criterion not achieved; and 0- no evidence, unclear or not applicable (maximum total score 43). We then assessed the readability of each website to determine the UK and US reading age using five different readability tests: Flesch Kincaid, Gunning Fog, Coleman Liau, SMOG, and the Automated Readability Index. We compared the readability scores to determine if there were any significant differences across the websites identified. We also generated scores for the Flesh Reading Ease as well as information about sentence structure (number of syllables per sentence and proportion of words with a high number of syllables) and proportion of people the text was readable to.

RESULTS

We identified 39 websites for readability and visual assessment. The visual assessment scores for the 39 websites identified ranged from -19 to 31 points out of a possible score of 43. The median readability score for the identified websites was 8.58 (age 14-15), with a range of 6.69-12.22 (age 12-13 to university level). There was a statistically significant difference between the readability scores obtained across websites (p<0.001). Almost half of the websites (18/39; 46.2%) were classified as very difficult by the Flesch Reading Ease score, with only 13/39 (33.33%) classified as being fairly easy or plain English.

CONCLUSION

We found wide differences in the general appearance, layout and focus of the fibroadenoma websites identified. The readability of most of the websites was also much higher than the recommended level for the public to understand. Fibroadenoma website information needs to be simplified to reduce the use of jargon and specificity to the condition for individuals to better comprehend it. In addition, their visual appearance could be improved by changing the layout and including images and diagrams.

Assessing the quality and readability of NHS urology webpages in England

Objective:

To examine the readability of urology webpages to assess their suitability for the general population and their inclusivity for varied levels of health literacy.

Methods:

We accessed all urology department websites from NHS trusts in England. A standard framework was used to assess these for ease of navigation, availability and quality of urology-specific information. Text was analysed using Flesch–Kincaid Grade Level and Ease Score to measure readability.

Results:

From the 132 NHS trust websites investigated, 117 (88.6%) had a urology-specific webpage, a considerable increase from 98 trusts (68%) in 2010. However, a smaller proportion of websites explained their services and procedures, and only 23.9% explained common urological conditions. Websites still appeared outdated, 23.9% had spelling errors, and 75.2% made none or poor use of images. The mean reading grade level was 13.40, equivalent to a UK reading age above 18 years, and the mean reading ease score was 30.35, in the range of ‘difficult’ text.

Conclusion:

Our data suggest that urology-specific webpages are not providing useful content for patients and are too complex for the general population to understand. Improvement would allow patients to be able to utilise such resources to aid decision making and to navigate complex NHS pathways.

Level of evidence:

Not applicable

Orthodontic treatment consent forms: A readability analysis

OBJECTIVE

To evaluate the readability of orthodontic treatment informed consent forms (ICFs).

DESIGN

A cross-sectional observational investigation.

METHODS

An online search strategy was adopted to identify ICFs for orthodontic treatment. The text of each form that satisfied inclusion criteria was modified to a standardised protocol. The readability was evaluated using three validated instruments. GraphPad Software (GraphPad Software Inc., La Jolla, CA, USA) was used for statistical analyses.

RESULTS

A total of 59 ICFs were evaluated. The majority were available on the websites of orthodontists (66.1%) and general dentists (27.1%). The scores recorded from the three instruments indicated that the content of 93.2%-98.3% of ICFs was difficult to read. A strong correlation was observed between the Simple Measure of Gobbledegook (SMOG) and the Flesch Kincaid Grade level (FKGL) instruments (r = 0.9782; P < 0.0001). The mean SMOG score for all assessed forms was 11.19 (95% confidence interval [CI] 10.85-11.54). The ICFs authored by the manufacturers of orthodontic appliances provided by clinicians recorded a mean Flesch Reading (FRE) score of 40.14 (95% CI 33.91-46.37) out of a maximum 100. An analysis of the SMOG scores indicated that the content of ICFs authored by national orthodontic societies was more difficult to read than those written by all other authors (P=0.01; 95% CI -1.6 to -0.2).

CONCLUSION

The readability of the ICFs was too difficult for a significant number of the population. Many patients will be unable to validly consent to treatment based solely on reading of the content of the evaluated ICFs. A greater effort is required to improve the readability of ICFs to help ensure patient autonomy regarding orthodontic treatment decision-making and management.

The Readability of Outpatient Plastic Surgery Clinic Letters: Are We Adhering to Plain English Writing Standards?

The readability of letters sent to patients plays a pivotal role in facilitating joint decision making and positive health outcomes. Guidance suggests that all correspondence should be directed to patients. Covid-19 led to an increase in direct-to-patient communication. This study aims to determine the quantity and quality of plastic surgery clinic letters sent to patients before and during the Covid-19 pandemic. A total of 127 pre-Covid clinic letters from November to December 2019 and 103 peri-Covid clinic letters from April to May 2020 were identified and assessed for readability. Text was analyzed using a standardized set of commonly used readability formulae, including SMOG, Flesch Reading Ease, and the Coleman-Liao Index. A total of 100 pre-Covid and 58 peri-Covid letters were suitable for inclusion. Median results for formulae that output a U.S. grade score ranged between 9 and 12.9 for letters written prior to Covid-19 and 9 and 13.2 for those written during Covid-19. Eight percent of letters were sent to patients pre-Covid, increasing to 28% during the pandemic. Letters sent to patients had a median grade score of between 8 and 12 prior to Covid-19 and 8 and 10.6 during Covid-19. Letters sent to clinicians ranged between 9 and 13 and 9 and 13.3, respectively. Outpatient plastic surgery letters are written at a readability level too high to facilitate understanding among the general population. The increase in direct-to-patient contact during the Covid-19 outbreak has not led to a significant improvement in the readability of clinic letters. The authors suggest an increase in letter standardization and raising awareness of readability when writing clinic letters.

Written Informed Consent-Translating into Plain Language. A Pilot Study

Background: Informed consent is important in clinical practice, as a person’s written consent is required prior to many medical interventions. Many informed consent forms fail to communicate simply and clearly. The aim of our study was to create an easy-to-understand form. Methods: Our assessment of a Polish-language plastic surgery informed consent form used the Polish-language comprehension analysis program (jasnopis.pl, SWPS University) to assess the readability of texts written for people of various education levels; and this enabled us to modify the form by shortening sentences and simplifying words. The form was re-assessed with the same software and subsequently given to 160 adult volunteers to assess the revised form’s degree of difficulty or readability. Results: The first software analysis found the language was suitable for people with a university degree or higher education, and after revision and re-assessment became suitable for persons with 4–6 years of primary school education and above. Most study participants also assessed the form as completely comprehensible. Conclusions: There are significant benefits possible for patients and practitioners by improving the comprehensibility of written informed consent forms.

Microtia: A Data Linkage Study of Epidemiology and Implications for Service Delivery

Introduction: Previous studies of microtia epidemiology globally have demonstrated significant geographical and ethnic variation, cited broadly as affecting 3-5 in 10,000 live births. The aim of this study was to determine the incidence of microtia in a largely homogeneous ethnic population in the United Kingdom (Wales) and to identify factors, such as distance and socioeconomic status, which may influence the access to surgical intervention. Materials and Methods: A retrospective cohort study was conducted using data linkage to identify patients born between 2000 and 2018 with a diagnosis of microtia. Microtia incidence was calculated using annual and geographic birth rates. Surgical operation codes were used to classify patients into those that had no surgery, autologous reconstruction or prosthetic reconstruction. Sociodemographic attributes were compared using descriptive statistics to determine differences in access to each type of surgical intervention. Results: A total of 101 patients were identified, 64.4% were male and the median age was 12 (8-16). The mean annual incidence was 2.13 microtia cases per 10,000 births over the 19-year study period. Both temporal and geographic variation was noted. The majority of patients undergoing surgery opted for autologous reconstruction (72.9%) at a median age of 9 (7-10) compared to 7 (5-8) for prosthetic reconstruction. Autologous reconstruction had a higher median number of surgeries (2, 1-3) than prosthetic (1.5, 1-2) and a higher median socioeconomic status of 3 (2-4) compared to 2 (1-4) for the prosthetic cohort. There were no statistically significant differences in the distance traveled for surgery. Discussion: This study highlights a role for data linkage in epidemiological analyses and provides a revised incidence of microtia in Wales. Although the majority of patients opted for autologous reconstruction, demographic disparities in socioeconomic status warrant further investigation, emphasizing the importance of striving for equity in accessibility to surgical intervention.

Evaluating the Readability of Online Patient Education Materials for Trigeminal Neuralgia

BACKGROUND

Patients, including patients with chronic pain conditions, increasingly turn to the Internet for health information. To facilitate comprehension, this information ought to be written at or below the 8th grade reading level, which is the average American adult's reading level. This study measured the reading level of popular online sources for trigeminal neuralgia.

METHODS

The top 10 search results for the search term "trigeminal neuralgia" were selected for inclusion. Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, Simple Measure of Gobbledygook, Coleman-Liau Index, Automated Readability Index, and Linsear Write Formula were used to assess readability. One-way analysis of variance was used to test for statistical differences in average readability scores among the different web pages.

RESULTS

Across the web pages, the average readability scores were as follows: Flesch Reading Ease, 42.1 ± 7.7; Flesch-Kincaid Grade Level, 10.9 ± 0.9; Gunning Fog Index, 15 ± 1.5; Simple Measure of Gobbledygook, 10.9 ± 1.2; Coleman-Liau Index, 12.1 ± 1.3; Automated Readability Index, 11.9 ± 1.4; Linsear Write Formula, 12.4 ± 1.7. Results from one-way analysis of variance demonstrated no statistically significant difference in overall readability scores (F_12,78 = 0.008; P > 0.05).

CONCLUSIONS

The writing of popular online education materials for trigeminal neuralgia is likely too complex for the average American adult to comprehend. This material should be revised to be readable at or below the 8th grade reading level. A variety of easily readable online education materials for trigeminal neuralgia can assist patients in understanding their illness and potentially improve patient decision making and outcomes.

Readability of Cancer Clinical Trials Websites

Clinical trials are critically important for the development of new cancer treatments. According to recent estimates, however, clinical trial enrollment is only about 8%. Lack of patient understanding or awareness of clinical trials is one reason for the low rate of participation. The purpose of this observational study was to evaluate the readability of cancer clinical trial websites designed to educate the general public and patients about clinical trials. Nearly 90% of Americans use Google to search for health-related information. We conducted a Google Chrome Incognito search in 2018 using the keywords “cancer clinical trial” and “cancer clinical trials.” Content of the 100 cancer clinical trial websites was analyzed using an online readability panel consisting of Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning-Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook scales. Reading level difficulty was assessed and compared between commercial versus non-commercial URL extensions. Content readability was found to be “difficult” (10.7 grade level). No significant difference in readability, overall, and between commercial and non-commercial URL extensions was found using 4/5 measures of readability; 90.9% of commercial versus 49.4% of non-commercial websites were written at a >10th grade (P = .013) using Gunning-Fog Index. Written cancer clinical trials content on the Internet is written at a reading level beyond the literacy capabilities of the average American reader. Improving readability to accommodate readers with basic literacy skills will provide an opportunity for greater comprehension that could potentially result in higher rates of clinical trial enrollment.

Readability of online material related to skin cancer

OBJECTIVES

The key to reducing mortality from skin cancer depends on early detection and treatment, which, in many cases, means a patient should be informed and possess the self-efficacy necessary to seek expert opinion. The purpose of this study was to determine the readability of skin cancer materials on the Internet using the commonly used readability tests.

STUDY DESIGN

This is a cross-sectional study.

METHODS

A search was conducted using the key words 'skin cancer.' The first 15 pages of websites that contained English language articles on skin cancer comprised the sample for this study. All English language articles appearing on these websites were analyzed using the Readable.io service to automate popular readability scores. For each article, it was determined if it was generated from a .org, .com, .gov, .net, .edu or other source. Five readability tests were conducted on the materials to determine the ease with which one can read each article.

RESULTS

Of the 102 websites sampled, none received an acceptable score on all five assessments. In fact, more than 90% of the websites sampled received an unacceptable score on any one of the five assessments, and more than 78% of the websites sampled received an unacceptable level on all five assessments. All five readability assessments demonstrated statistically significant results; each P-value obtained from the t-tests was substantially below 0.01 and, hence, below the α = 0.05 threshold. These results indicate that it is unlikely that skin cancer websites are being written at the recommended level. Of the websites sampled, roughly 42% were .com, and slightly more than 36% were .org. Statistical evidence indicated that neither .com nor .org websites are likely to be written at the acceptable level.

CONCLUSIONS

Both for-profit and non-profit agencies which aim to provide health information to consumers should be mindful of the readability levels of the materials they disperse.