Consulting Dr. Google: Quality of Online Resources About Tympanostomy Tube Placement

Quality of Popular Online Resources About Vestibular Migraine

Objective

To evaluate the readability, understandability, actionability, and accuracy of online resources covering vestibular migraine (VM).

Study Design

Cross-sectional descriptive study design.

Setting

Digital collection of websites appearing on Google search.

Methods

Google searches were conducted to identify common online resources for VM. We examined readability using the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level scores, understandability and actionability using the Patient Education Materials Assessment Tool (PEMAT), and accuracy by comparing the website contents to the consensus definition of "probable vestibular migraine."

Results

Eleven of the most popular websites were analyzed. Flesch-Kincaid Grade Level averaged at a 13th-grade level (range: 9th-18th). FRE scores averaged 35.5 (range: 9.1-54.4). No website had a readability grade level at the US Agency for Healthcare Research and Quality recommended 5th-grade level or an equivalent FRE score of 90 or greater. Understandability scores varied ranging from 49% to 88% (mean 70%). Actionability scores varied more, ranging from 12% to 87% (mean 44%). There was substantial inter-rater agreement for both PEMAT understandability scoring (mean κ = 0.76, SD = 0.08) and actionability scoring (mean κ = 0.65, SD = 0.08). Three sites included all 3 "probable vestibular migraine" diagnostic criteria as worded in the consensus statement.

Conclusion

The quality of online resources for VM is poor overall in terms of readability, actionability, and agreement with diagnostic criteria.

Health Literacy in Shoulder Arthroscopy: A Quantitative Assessment of the Understandability and Readability of Online Patient Education Material

Background

The National Institutes of Health (NIH) and American Medical Association (AMA) recommend that online health information be written at a maximum 6th grade reading level. The aim was to evaluate online resources regarding shoulder arthroscopy utilizing measures of readability, understandability, and actionability, using syntax reading grade level and the Patient Education Materials Assessment Tool (PEMAT-P).

Methods

An online Google™ search utilizing "shoulder arthroscopy" was performed. From the top 50 results, websites directed at educating patients were included. News and scientific articles, audiovisual materials, industry websites, and unrelated materials were excluded. Readability was calculated using objective algorithms: Flesch-Kincaid Grade-Level (FKGL), Simple Measure of Gobbledygook (SMOG) grade, Coleman-Liau Index (CLI), and Gunning-Fog Index (GFI). The PEMAT-P was used to assess understandability and actionability, with a 70% score threshold. Scores were compared across academic institutions, private practices, and commercial health publishers. The correlation between search rank and readability, understandability, and actionability was calculated.

Results

Two independent searches yielded 53 websites, with 44 (83.02%) meeting inclusion criteria. No mean readability score performed below a 10th grade reading level. Only one website scored at or below 6th grade reading level. Mean understandability and actionability scores were 63.02%±12.09 and 29.77%±20.63, neither of which met the PEMAT threshold. Twelve (27.27%) websites met the understandability threshold, while none met the actionability threshold. Institution categories scored similarly in understandability (61.71%, 62.68%, 63.67%) among academic, private practice, and commercial health publishers respectively (p=0.9536). No readability or PEMAT score correlated with search rank.

Conclusion

Online shoulder arthroscopy patient education materials score poorly in readability, understandability, and actionability. One website scored at the NIH and AMA recommended reading level, and 27.27% of websites scored above the 70% PEMAT score for understandability. None met the actionability threshold. Future efforts should improve online resources to optimize patient education and facilitate informed decision-making. Level of Evidence: IV.

Carpal Tunnel Surgery: Can Patients Read, Understand, and Act on Online Educational Resources?

Background

Patients often access online resources to educate themselves prior to undergoing elective surgery such as carpal tunnel release (CTR). The purpose of this study was to evaluate available online resources regarding CTR on objective measures of readability (syntax reading grade-level), understandability (ability to convey key messages in a comprehensible manner), and actionability (providing actions the reader may take).

Methods

The study conducted two independent Google searches for "Carpal Tunnel Surgery" and among the top 50 results, analyzed articles aimed at educating patients about CTR. Readability was assessed using six different indices: Flesch-Kincaid Grade Level Index, Flesch Reading Ease, Gunning Fog Index, Simple Measure of Gobbledygook (SMOG) Index, Coleman Liau Index, Automated Readability Index. The Patient Education Materials Assessment Tool evaluated understandability and actionability on a 0-100% scale. Spearman's correlation assessed relationships between these metrics and Google search ranks, with p<0.05 indicating statistical significance.

Results

Of the 39 websites meeting the inclusion criteria, the mean readability grade level exceeded 9, with the lowest being 9.4 ± 1.5 (SMOG index). Readability did not correlate with Google search ranking (lowest p=0.25). Mean understandability and actionability were 59% ± 15 and 26% ± 24, respectively. Only 28% of the articles used visual aids, and few provided concise summaries or clear, actionable steps. Notably, lower grade reading levels were linked to higher actionability scores (p ≤ 0.02 in several indices), but no readability metrics significantly correlated with understandability. Google search rankings showed no significant association with either understandability or actionability scores.

Conclusion

Online educational materials for CTR score poorly in readability, understandability, and actionability. Quality metrics do not appear to affect Google search rankings. The poor quality metric scores found in our study highlight a need for hand specialists to improve online patient resources, especially in an era emphasizing shared decision-making in healthcare. Level of Evidence: IV.

A critical readability and quality analysis of internet-based patient information on neck dissections

Objective

Patients are increasingly turning to the Internet as a source of healthcare information. Given that neck dissection is a common procedure within the field of Otolaryngology - Head and Neck Surgery, the aim of this study was to evaluate the quality and readability of online patient education materials on neck dissection.

Methods

A Google search was performed using the term "neck dissection." The first 10 pages of a Google search using the term "neck dissection" were analyzed. The DISCERN instrument was used to assess quality of information. Readability was calculated using the Flesch-Reading Ease, Flesch-Kincaid Grade Level, Gunning-Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook Index.

Results

Thirty-one online patient education materials were included. Fifty-five percent (n = 17) of results originated from academic institutions or hospitals. The mean Flesch-Reading Ease score was 61.2 ± 11.9. Fifty-two percent (n = 16) of patient education materials had Flesch-Reading Ease scores above the recommended score of 65. The average reading grade level was 10.5 ± 2.1. The average total DISCERN score was 43.6 ± 10.1. Only 26% of patient education materials (PEMs) had DISCERN scores corresponding to a "good quality" rating. There was a significant positive correlation between DISCERN scores and both Flesch-Reading Ease scores and average reading grade level.

Conclusions

The majority of patient education materials were written above the recommended sixth-grade reading level and the quality of online information pertaining to neck dissections was found to be suboptimal. This research highlights the need for patient education materials regarding neck dissection that are high quality and easily understandable by patients.

Patient education information material assessment criteria: A scoping review

BACKGROUND

Patient education information material (PEIM) is an essential component of patient education programs in increasing patients' ability to cope with their diseases. Therefore, it is essential to consider the criteria that will be used to prepare and evaluate these resources.

OBJECTIVE

This paper aims to identify these criteria and recognize the tools or methods used to evaluate them.

METHODS

National and international databases and indexing banks, including PubMed, Scopus, Web of Science, ProQuest, the Cochrane Library, Magiran, SID and ISC, were searched for this review. Original or review articles, theses, short surveys, and conference papers published between January 1990 and June 2022 were included.

RESULTS

Overall, 4688 documents were retrieved, of which 298 documents met the inclusion criteria. The criteria were grouped into 24 overarching criteria. The most frequently used criteria were readability, quality, suitability, comprehensibility and understandability.

CONCLUSION

This review has provided empirical evidence to identify criteria, tools, techniques or methods for developing or evaluating a PEIM. The authors suggest that developing a comprehensive tool based on these findings is critical for evaluating the overall efficiency of PEIM using effective criteria.

Patients' perspectives on the quality of online patient education materials: A qualitative study

BACKGROUND

Access to online patient education information can lead to more effective self-care and disease management. However, the large amount of online information provided through unknown or unreliable sources can challenge patients to trust and use this information. This study was designed to examine the opinions of Iranian gastrointestinal patients about the quality of online information used.

MATERIALS AND METHODS

A qualitative study was conducted using thematic analysis. Data were gathered via a semi-structured interview with 29 gastrointestinal patients, and data analysis was performed by qualitative content analysis using open coding with MAXQDA 2018 software.

RESULTS

Based on the study, 22 codes were extracted in nine subcategories named as: "Emphasis on the identity of providers," "Nature of online information," "Distrust on online information," "Poor quality of information," "Giving misinformation," "False impact," "Improve communication," "Positive effect on the patient," "Better Diagnosis."

CONCLUSIONS

In the current situation, Iranian patients are not confident enough about the quality of available online information. They believe that the use of current poor-quality information has negative consequences. However, they tend to use online patient education materials are produced in Persian by reputable scientific authorities. Using online information can increase patients' knowledge and lead to better communication with medical staff and other similar patients. They can use this information for self-care with more confidence, and such an approach can also have significant benefits for the national health system.

Can Patients Read, Understand, and Act on Online Resources for Anterior Cruciate Ligament Surgery?

Background:

Patients undergoing elective procedures often utilize online educational materials to familiarize themselves with the surgical procedure and expected postoperative recovery. While the Internet is easily accessible and ubiquitous today, the ability of patients to read, understand, and act on these materials is unknown.

Purpose:

To evaluate online resources about anterior cruciate ligament (ACL) surgery utilizing measures of readability, understandability, and actionability.

Study Design:

Cross-sectional study; Level of evidence, 4.

Methods:

Using the term “ACL surgery,” 2 independent searches were performed utilizing a public search engine (Google.com). Patient education materials were identified from the top 50 results. Audiovisual materials, news articles, materials intended for advertising or medical professionals, and materials unrelated to ACL surgery were excluded. Readability was quantified using the Flesch Reading Ease, Flesch-Kincaid Grade Level, Simple Measure of Gobbledygook, Coleman-Liau Index, Automated Readability Index, and Gunning Fog Index. The Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P) was utilized to assess the actionability and understandability of materials. For each online source, the relationship between its Google search rank (from first to last) and its readability, understandability, and actionability was calculated utilizing the Spearman rank correlation coefficient (ρ_S).

Results:

Overall, we identified 68 unique websites, of which 39 met inclusion criteria. The mean Flesch-Kincaid Grade Level was 10.08 ± 2.34, with no website scoring at or below the 6th-grade level. Mean understandability and actionability scores were 59.18 ± 10.86 (range, 33.64-79.17) and 34.41 ± 22.31 (range, 0.00-81.67), respectively. Only 5 (12.82%) and 1 (2.56%) resource scored above the 70% adequate PEMAT-P threshold mark for understandability and actionability, respectively. Readability (lowest P value = .103), understandability (ρ_S = –0.13; P = .441), and actionability (ρ_S = 0.28; P = .096) scores were not associated with Google rank.

Conclusion:

Patient education materials on ACL surgery scored poorly with respect to readability, understandability, and actionability. No online resource scored at the recommended reading level of the American Medical Association or National Institutes of Health. Only 5 resources scored above the proven threshold for understandability, and only 1 resource scored above it for actionability.

Clinical Practice Guideline: Tympanostomy Tubes in Children (Update)

OBJECTIVE

Insertion of tympanostomy tubes is the most common ambulatory surgery performed on children in the United States. Tympanostomy tubes are most often inserted because of persistent middle ear fluid, frequent ear infections, or ear infections that persist after antibiotic therapy. All these conditions are encompassed by the term otitis media (middle ear inflammation). This guideline update provides evidence-based recommendations for patient selection and surgical indications for managing tympanostomy tubes in children. The guideline is intended for any clinician involved in managing children aged 6 months to 12 years with tympanostomy tubes or children being considered for tympanostomy tubes in any care setting as an intervention for otitis media of any type. The target audience includes specialists, primary care clinicians, and allied health professionals.

PURPOSE

The purpose of this clinical practice guideline update is to reassess and update recommendations in the prior guideline from 2013 and to provide clinicians with trustworthy, evidence-based recommendations on patient selection and surgical indications for managing tympanostomy tubes in children. In planning the content of the updated guideline, the guideline update group (GUG) affirmed and included all the original key action statements (KASs), based on external review and GUG assessment of the original recommendations. The guideline update was supplemented with new research evidence and expanded profiles that addressed quality improvement and implementation issues. The group also discussed and prioritized the need for new recommendations based on gaps in the initial guideline or new evidence that would warrant and support KASs. The GUG further sought to bring greater coherence to the guideline recommendations by displaying relationships in a new flowchart to facilitate clinical decision making. Last, knowledge gaps were identified to guide future research.

METHODS

In developing this update, the methods outlined in the American Academy of Otolaryngology-Head and Neck Surgery Foundation's "Clinical Practice Guideline Development Manual, Third Edition: A Quality-Driven Approach for Translating Evidence Into Action" were followed explicitly. The GUG was convened with representation from the disciplines of otolaryngology-head and neck surgery, otology, pediatrics, audiology, anesthesiology, family medicine, advanced practice nursing, speech-language pathology, and consumer advocacy.

ACTION STATEMENTS

The GUG made strong recommendations for the following KASs: (14) clinicians should prescribe topical antibiotic ear drops only, without oral antibiotics, for children with uncomplicated acute tympanostomy tube otorrhea; (16) the surgeon or designee should examine the ears of a child within 3 months of tympanostomy tube insertion AND should educate families regarding the need for routine, periodic follow-up to examine the ears until the tubes extrude.The GUG made recommendations for the following KASs: (1) clinicians should not perform tympanostomy tube insertion in children with a single episode of otitis media with effusion (OME) of less than 3 months' duration, from the date of onset (if known) or from the date of diagnosis (if onset is unknown); (2) clinicians should obtain a hearing evaluation if OME persists for 3 months or longer OR prior to surgery when a child becomes a candidate for tympanostomy tube insertion; (3) clinicians should offer bilateral tympanostomy tube insertion to children with bilateral OME for 3 months or longer AND documented hearing difficulties; (5) clinicians should reevaluate, at 3- to 6-month intervals, children with chronic OME who do not receive tympanostomy tubes, until the effusion is no longer present, significant hearing loss is detected, or structural abnormalities of the tympanic membrane or middle ear are suspected; (6) clinicians should not perform tympanostomy tube insertion in children with recurrent acute otitis media who do not have middle ear effusion in either ear at the time of assessment for tube candidacy; (7) clinicians should offer bilateral tympanostomy tube insertion in children with recurrent acute otitis media who have unilateral or bilateral middle ear effusion at the time of assessment for tube candidacy; (8) clinicians should determine if a child with recurrent acute otitis media or with OME of any duration is at increased risk for speech, language, or learning problems from otitis media because of baseline sensory, physical, cognitive, or behavioral factors; (10) the clinician should not place long-term tubes as initial surgery for children who meet criteria for tube insertion unless there is a specific reason based on an anticipated need for prolonged middle ear ventilation beyond that of a short-term tube; (12) in the perioperative period, clinicians should educate caregivers of children with tympanostomy tubes regarding the expected duration of tube function, recommended follow-up schedule, and detection of complications; (13) clinicians should not routinely prescribe postoperative antibiotic ear drops after tympanostomy tube placement; (15) clinicians should not encourage routine, prophylactic water precautions (use of earplugs or headbands, avoidance of swimming or water sports) for children with tympanostomy tubes.The GUG offered the following KASs as options: (4) clinicians may perform tympanostomy tube insertion in children with unilateral or bilateral OME for 3 months or longer (chronic OME) AND symptoms that are likely attributable, all or in part, to OME that include, but are not limited to, balance (vestibular) problems, poor school performance, behavioral problems, ear discomfort, or reduced quality of life; (9) clinicians may perform tympanostomy tube insertion in at-risk children with unilateral or bilateral OME that is likely to persist as reflected by a type B (flat) tympanogram or a documented effusion for 3 months or longer; (11) clinicians may perform adenoidectomy as an adjunct to tympanostomy tube insertion for children with symptoms directly related to the adenoids (adenoid infection or nasal obstruction) OR in children aged 4 years or older to potentially reduce future incidence of recurrent otitis media or the need for repeat tube insertion.

Quality-Based Evaluation of Patient-Facing Online Education Materials Regarding Facial Filler Procedures

Injectable facial fillers have become tremendously more popular in recent years, and the Internet offers a proportional amount of consumer-facing educational material. This study sought to explore the quality of these online materials. The top 20 Web sites offering educational materials about facial filler were identified via Google search and sorted by source: Medical Professional Boards, Hospitals and Providers, Medical News and Reference, and Fashion. The materials were assessed for overall quality with the validated DISCERN instrument. The authors also assessed understandability and actionability (Patient Education Material Assessment Tool - PEMAT), accuracy, comprehensiveness, and readability (Flesch-Kincaid Grade Level and Flesch Reading Ease). The mean DISCERN score was 46.9 ± 7.6, which is considered "fair" quality educational material; above "poor," but below "good" and "excellent." Understandability and actionability scores were low, particularly with respect to visual aids. The materials were generally accurate (76-99%), but scored poorly in comprehensiveness, as 15% failed to mention any risks/adverse effects and only 35% mentioned cost. On average, readability was at an 11th grade level, far more complex than ideal (< 6th grade level). Information disseminated from seemingly reputable sources such as professional boards and hospitals/providers were not of higher quality or superior in any of the above studied domains. In conclusion, online educational materials related to injectable facial fillers are of subpar quality, including those from academic and professional organizations. Visual aids were particularly weak. The facial rejuvenation community should make a concerted effort to set a higher standard for disseminating such information.

Critical Quality and Readability Analysis of Online Patient Education Materials on Parotidectomy: A Cross-Sectional Study

PURPOSE

Complications related to parotidectomy can cause significant morbidity, and thus, the decision to pursue this surgery needs to be well-informed. Given that information available online plays a critical role in patient education, this study aimed to evaluate the readability and quality of online patient education materials (PEMs) regarding parotidectomy.

METHODS

A Google search was performed using the term "parotidectomy" and the first 10 pages of the search were analyzed. Quality and reliability of the online information was assessed using the DISCERN instrument. Flesch-Kincaid Grade Level (FKGL) and Flesch-Reading Ease Score (FRE) were used to evaluate readability.

RESULTS

Thirty-five PEMs met the inclusion criteria. The average FRE score was 59.3 and 16 (46%) of the online PEMs had FRE scores below 60 indicating that they were fairly difficult to very difficult to read. The average grade level of the PEMs was above the eighth grade when evaluated with the FKGL. The average DISCERN score was 41.7, which is indicative of fair quality. There were no significant differences between PEMs originating from medical institutions and PEMs originating from other sources in terms of quality or readability.

CONCLUSION

Online PEMs on parotidectomy may not be comprehensible to the average individual. This study highlights the need for the development of more appropriate PEMs to inform patients about parotidectomy.

Total Knee Arthroplasty: A Quantitative Assessment of Online Patient Education Resources

Background

Patients often turn to the online resources to learn about orthopedic procedures. As the rate of joint arthroplasty is projected to increase, the corresponding interest in relevant online education material will increase as well. The American Medical Association (AMA) and National Institutes of Health (NIH) recommend that publicly available online health information be written at the 6th grade or lower reading level to be fully understood by the average adult in the United States. Additionally, educational resources should be written such that readers can process key information (understandability) or identify available actions to take (actionability). The purpose of this study was to quantify the readability, understandability, and actionability of online patient educational materials regarding total knee arthroplasty (TKA).

Methods

The most common Google™ search term utilized by the American public was determined to be "knee replacement". Subsequently two independent online searches (Google.com) were performed. From the top 50 search results, websites were included if directed at educating patients regarding TKA. Non-text websites (audiovisual), articles (news/research/industry), and unrelated resources were excluded. Readability was quantified using the following valid objective algorithms: Flesch-Kincaid Grade-Level (FKGL), Simple Measure of Gobbledygook (SMOG) grade, Coleman-Liau Index (CLI), and Gunning-Fog Index (GFI). PEMAT was utilized to assess understandability and actionability (0-100%; score ≥70% indicates acceptable scoring). The relationship between search rank with FKGL and PEMAT scores was quantified.

Results

A total of 34 (68%) unique websites met inclusion criteria. The mean FKGL, SMOG, CLI, and GFI was 11.8±1.6, 11.1±1.2, 11.9±1.4, and 14.7±1.6, respectively. None of the websites scored within the acceptable NIH/AMA recommended reading levels. Mean understandability and actionability scores were 54.9±12.1 and 30.3±22.0. Only 5.9% (n=2) and 9.2% (n=1) of websites met the ≥70% threshold for understandability and actionability. Only 29.4% (n=10) sources used common language and only 26.9% (n=9) properly defined complicated medical terms. Based on website type, the mean understandability scores for academic institution, private practice, and health information publisher websites were 57.2±8.8%, 52.6±11.1%, and 54.3±15.3% (p=0.67). Readability (rho: -0.07; p=0.69), understandability (rho: -0.02; p=0.93), and actionability (rho: -0.22; p=0.23) scores were not associated with Google™ search rank.

Conclusion

TKA materials scored poorly with respect to readability, understandability, and actionability. None of the resources scored within the recommended AMA/NIH reading levels. Only 5.9% scored adequately on understandability measures. Substantial efforts are needed to improve online resources to optimize patient comprehension and facilitate informed decision-making. Level of Evidence: III.

Factors Affecting Engagement in Web-Based Health Care Patient Information: Narrative Review of the Literature

BACKGROUND

Web-based content is rapidly becoming the primary source of health care information. There is a pressing need for web-based health care content to not only be accurate but also be engaging. Improved engagement of people with web-based health care content has the potential to inform as well as influence behavioral change to enable people to make better health care choices. The factors associated with better engagement with web-based health care content have previously not been considered.

OBJECTIVE

The aims of this study are to identify the factors that affect engagement with web-based health care content and develop a framework to be considered when creating such content.

METHODS

A comprehensive search of the PubMed and MEDLINE database was performed from January 1, 1946, to January 5, 2020. The reference lists of all included studies were also searched. The Medical Subject Headings database was used to derive the following keywords: "patient information," "online," "internet," "web," and "content." All studies in English pertaining to the factors affecting engagement in web-based health care patient information were included. No restrictions were set on the study type. Analysis of the themes arising from the results was performed using inductive content analysis.

RESULTS

The search yielded 814 articles, of which 56 (6.9%) met our inclusion criteria. The studies ranged from observational and noncontrolled studies to quasi-experimental studies. Overall, there was significant heterogeneity in the types of interventions and outcome assessments, which made quantitative assessment difficult. Consensus among all authors of this study resulted in six categories that formed the basis of a framework to assess the factors affecting engagement in web-based health care content: easy to understand, support, adaptability, accessibility, visuals and content, and credibility and completeness.

CONCLUSIONS

There is a paucity of high-quality data relating to the factors that improve the quality of engagement with web-based health care content. Our framework summarizes the reported studies, which may be useful to health care content creators. An evaluation of the utility of web-based content to engage users is of significant importance and may be accessible through tools such as the Net Promoter score. Web 3.0 technology and development of the field of psychographics for health care offer further potential for development. Future work may also involve improvement of the framework through a co-design process.

The Patient Perspective: Evaluating the Accessibility of Transoral Robotic Surgery Online Resources

PURPOSE

The varied treatment options available to patients with HPV-positive oropharyngeal squamous cell carcinoma (OPSCC) can cause significant patient confusion. In particular, transoral robotic surgery (TORS) has become widely used for treatment of HPV-positive OPSCC. As patients commonly refer to the internet for additional information, we aim to evaluate the quality of online patient educational materials for TORS in comparison to other otolaryngology surgical procedures.

METHODS

The terms "transoral robotic surgery," "glossectomy," "thyroidectomy," and "neck dissection" were searched on Google. Flesch reading ease, Flesh-Kincaid Grade Level, MD review rates, and PEMAT understandability and actionability were assessed for each search term. Google trends was used to determine search interest for each term between May 2015 and May 2020.

RESULTS

Of the 30 TORS websites that met inclusion criteria, the average FRE and FKGL scores were 40.74 and 11.60 (that of an average high school senior). The FRE and FKGL scores for TORS were all statistically significantly lower than those of all comparator search terms (P < .05). Only 1 out of 30 TORS sites were MD reviewed, representing the lowest MD review rate across all search terms. Understandability and actionability scores for TORS were also the lowest across all search terms at 77.71% and 2.66%, respectively. TORS had the lowest search interest.

CONCLUSIONS

In comparison to other common otolaryngology procedures, TORS websites are the least patient friendly. As TORS becomes a more widely used procedure, it is critical that TORS websites become higher quality resources that patients can reliably reference.

Readability and quality assessment of internet-based patient education materials related to nasal septoplasty

BACKGROUND

Given that nasal septoplasty is a common procedure in otolaryngology - head and neck surgery, the objective of this study was to evaluate the quality and readability of online patient education materials on septoplasty.

METHODS

A Google search was performed using eight different search terms related to septoplasty. Six different tools were used to assess the readability of included patient education materials. These included the Flesch-Kincaid Grade Level, Flesch Reading Ease, Gunning-Fog Index, Simple Measure of Gobbledygook Index, Coleman-Liau Index, and Automated Readability Index. The DISCERN tool was used to assess quality and reliability.

RESULTS

Eighty-five online patient education materials were included. The average Flesch-Reading Ease score for all patient education materials was 54.9 ± 11.5, indicating they were fairly difficult to read. The average reading grade level was 10.5 ± 2.0, which is higher than the recommended reading level for patient education materials. The mean DISCERN score was 42.9 ± 10.5 and 42% (36/85) of articles had DISCERN scores less than 39, corresponding to poor or very poor quality.

CONCLUSION

The majority of online patient education materials on septoplasty are written above the recommended reading levels and have significant deficiencies in terms of their quality and reliability. Clinicians and patients should be aware of the shortcomings of these resources and consider the impact they may have on patients' decision making.

Quality and Readability of Web-Based Information for Patients With Pancreatic Cysts: DISCERN and Readability Test Analysis

Background

Pancreatic cysts are a complex medical problem with several treatment options. Patients use web-based health information to understand their conditions and to guide treatment choices.

Objective

The goal of this study was to describe the quality and readability of publicly available web-based information on pancreatic cysts and to compare this information across website affiliations.

Methods

A Google search for “pancreatic cysts” was performed and the first 30 websites were evaluated. Website affiliations were classified as academic, media, nonprofit, government, or not disclosed. Information describing cancer risk was recorded. The DISCERN instrument measured the quality of content regarding treatment choices. Four standardized tests were used to measure readability.

Results

Twenty-one websites were included. The majority of the websites (20/21, 95%) described the cancer risk associated with pancreatic cysts. Nearly half of the websites were written by an academic hospital or organization. The average DISCERN score for all websites was 40.4 (range 26-65.5, maximum 80). Websites received low scores due to lack of references, failure to describe the risks of treatment, or lack of details on how treatment choices affect quality of life. The average readability score was 14.74 (range 5.76-23.85, maximum 19+), indicating a college reading level. There were no significant differences across website affiliation groups.

Conclusions

Web-based information for patients with pancreatic cysts is of moderate quality and is written above the reading level of most Americans. Gastroenterological, cancer treatment organizations, and physicians should advocate for improving the available information by providing cancer risk stratification, treatment impact on quality of life, references, and better readability.

Search Trends and Quality of Online Resources Regarding Thyroidectomy

OBJECTIVE

Thyroidectomy is one of the most common procedures performed in head and neck surgery. The quality of online resources for thyroidectomy is unknown. We aim to evaluate search trends and online resource quality regarding thyroidectomy.

STUDY DESIGN

Cross-sectional analysis.

SETTING

Websites appearing on Google search.

METHODS

The first 30 Google websites for thyroidectomy were reviewed, excluding research, video, and restricted sites. Search patterns were obtained with Google Trends. Quality was measured by readability (Flesch Reading Ease and Flesch-Kinkaid Grade Level), understandability and actionability (Patient Education Materials Assessment Tool), and clinical practice guideline (CPG) compatibility. Fleiss kappa interrater reliability analysis was performed for 2 raters.

RESULTS

Twenty-one sites were evaluated. Search popularity for thyroidectomy has increased since 2004. Median reading ease was 42.2 (range, 15.4-62.7) on a scale from 1 to 100, with 100 indicating maximum readability. Median reading grade level was 12 (range, 7-16). Thyroidectomy resources were poorly understandable (median, 66%; range, 21%-88%) and actionable (median, 10%; range, 0%-60%). Median CPG compatibility was 4 out of 5 (range, 0-5). Interrater reliability ranged from substantial to moderate for understandability (0.78), actionability (0.57), and CPG compatibility (0.58), with P < .05 for all results.

CONCLUSION

Online resources about thyroidectomy vary in quality and reliability and are written at grade levels above the average reading level of the public. Providers should be aware of existing resources and work to create education resources that meet universal health literacy guidelines. The framework provided in this article may also serve as a guide and provide tangible steps that providers can take to help patients access care.

Osteosarcoma Online Resources: A Quantitative Assessment of the Understandability and Readability of Web-based Patient Education Material (Preprint)

Background

Patients often turn to web-based resources following the diagnosis of osteosarcoma. To be fully understood by average American adults, the American Medical Association (AMA) and National Institutes of Health (NIH) recommend web-based health information to be written at a 6th grade level or lower. Previous analyses of osteosarcoma resources have not measured whether text is written such that readers can process key information (understandability) or identify available actions to take (actionability). The Patient Education Materials Assessment Tool (PEMAT) is a validated measurement of understandability and actionability.

Objective

The purpose of this study was to evaluate web-based osteosarcoma resources using measures of readability, understandability, and actionability.

Methods

Using the search term “osteosarcoma,” two independent Google searches were performed on March 7, 2020 (by AGS), and March 11, 2020 (by TRG). The top 50 results were collected. Websites were included if they were directed at providing patient education on osteosarcoma. Readability was quantified using validated algorithms: Flesh-Kincaid Grade Ease (FKGE), Flesch-Kincaid Grade-Level (FKGL). A higher FKGE score indicates that the material is easier to read. All other readability scores represent the US school grade level. Two independent PEMAT assessments were performed with independent scores assigned for both understandability and actionability. A PEMAT score of 70% or below is considered poorly understandable or poorly actionable. Statistical significance was defined as P≤.05.

Results

Two searches yielded 53 unique websites, of which 37 (70%) met the inclusion criteria. The mean FKGE and FKGL scores were 40.8 (SD 13.6) and 12.0 (SD 2.4), respectively. No website scored within the acceptable NIH or AHA recommended reading level. Only 4 (11%) and 1 (3%) website met the acceptable understandability and actionability threshold. Both understandability and actionability were positively correlated with FKGE (ρ=0.55, P<.001; ρ=0.60, P<.001), but were otherwise not significantly associated with other readability scores. There were no associations between readability (P=.15), understandability (P=.20), or actionability (P=.31) scores and Google rank.

Conclusions

Overall, web-based osteosarcoma patient educational materials scored poorly with respect to readability, understandability, and actionability. None of the web-based resources scored at the recommended reading level. Only 4 achieved the appropriate score to be considered understandable by the general public. Authors of patient resources should incorporate PEMAT and readability criteria to improve web-based resources to support patient understanding.

Quality, Readability, and Suitability of Hearing Health-Related Materials: A Descriptive Review

Objectives The objective of this descriptive review was to determine the quality, readability, and suitability of ear and hearing health information and materials for patients and their significant others. Method A literature search was conducted between August 2018 and April 2019 in the databases CINAHL Complete, MEDLINE, and PsychInfo. Inclusion and exclusion criteria were used to shortlist studies. Data regarding quality, suitability, and readability were extracted from the included studies. Data were assessed qualitatively. Results There were 34 studies included in this review. Of those, eight examined quality, 33 assessed readability, and four investigated the suitability of materials. The range of materials assessed included diagnostic reports, patient education materials (PEMs), patient-reported outcome measures, and websites. Quality elements were examined in studies focusing on website information. Findings indicated that most websites were of poor quality. Suitability was examined in studies focusing on PEMs such as hearing aid user guides. Findings indicated that most of the existing materials were not suitable for the intended populations. The reading grade level of information across all four categories was found to be higher than the recommended fifth or sixth reading grade level for health-related materials. Revisions of some diagnostic reports and PEMs showed that improvements are possible. Conclusions This review suggests that ear- and hearing-related materials generally have lower quality and suitability with higher readability (more difficult to read). Development of materials that are suitable, of high quality, and at the appropriate readability levels is required to improve accessibility of ear- and hearing-related materials.

Consulting "Dr. YouTube": an objective evaluation of hypospadias videos on a popular video-sharing website

INTRODUCTION

Parents who make decisions about hypospadias repair for their child may seek information from online platforms such as YouTube.

OBJECTIVE

The purpose of this study is to evaluate the health literacy demand of hypospadias videos on YouTube using the Patient Education Materials Assessment Tool for Audiovisual Materials (PEMAT-A/V).

STUDY DESIGN

We performed a YouTube search using the term "hypospadias," limiting results to the first 100 videos. We excluded videos that were <1 min or >20 min and videos that were not in English or did not include subtitles. Two evaluators independently examined videos and determined PEMAT-A/V scores for understandability and actionability (i.e., ability to identify actions the viewer can take). Videos with scores >70% are understandable or actionable. The inter-rater reliability (kappa) and intraclass correlation coefficient (ICC) of PEMAT scores were calculated. Bivariate and multivariable linear regression models assessed the association of video characteristics with respective scores.

RESULTS

Of the 100 videos that were identified on YouTube, 47 (47%) were excluded leaving 53 for analysis: 14 were >20 min, 14 were <1 min, 9 had no audio or subtitles, 7 were not in English, 1 was a duplicate, 1 was unrelated to hypospadias, and 1 was deleted at the time of data analysis. Three (5.6%) were understandable (mean score 54.5%, standard deviation (SD) 14.9) and eight (15.1%) were actionable (mean score 21.8%, SD 16.6) (Extended Summary Figure). Kappa values ranged from 0.4 to 1. The ICC's were 0.55 and 0.33 for understandability and actionability, respectively. In the bivariate analysis, mean understandability scores were significantly higher for English language videos (p = 0.04), videos with animation (p = 0.002), and those produced by industry (p = 0.02). In the multivariable analysis, mean understandability scores were significantly higher for "expert testimonial" or "other" video types after adjusting for graphics type and overall tone (p = 0.04). Mean understandability scores were also significantly higher for videos with animation after adjusting for video type and overall tone (p = 0.01). Mean actionability scores were significantly higher for videos with a negative tone (p = 0.01).

DISCUSSION

The vast majority of hypospadias-related YouTube content is not appropriate for users with low health literacy although certain types of videos, such those with animation and expert testimonials, scored higher on understandability than other types.

CONCLUSION

Due to the lack of sufficient online informational content regarding hypospadias, we plan to engage parents of sons with hypospadias in the development of high-quality patient educational materials about hypospadias.

Understandability and actionability of audiovisual patient education materials on sinusitis

BACKGROUND

Acute and chronic rhinosinusitis are debilitating diseases that affect from 5% to 16% of the population. YouTube is the second most commonly used search engine and is often utilized by patients to garner health information regarding various disease processes and their respective management options. An evaluation of these information resources for quality and reliability is warranted, especially in an era in which patients are increasingly turning to audiovisual (A/V) media to educate themselves regarding their ailments.

METHODS

The YouTube video database was searched using the term "sinusitis" from its inception through to November 2018. The first 50 videos populated under the relevance-based ranking option were collected and parsed by time and language. Of the 50 videos, 10 were eliminated either for length (≤1 minute or ≥20 minutes), language (any language other than English), and/or for later being removed by YouTube for copyright violations. The videos were then assessed using the Patient Education Materials Assessment Tool-Audio/Visual (PEMAT-A/V) by 2 independent reviewers for understandability and actionability.

RESULTS

A total of 40 videos were examined using the PEMAT-A/V tool. The average understandability score was 57.7%, whereas the average actionability score was 46.3%. Eleven videos (28%) had actionability scores of 0%. Videos most commonly addressed disease management options (38%). The second largest category was case presentations regarding surgical techniques (30%). There were only 6 videos focused primarily on education about the definition and common traits of sinusitis (15%).

CONCLUSION

Our results show a paucity of high-quality online A/V educational material pertaining to sinusitis, with a majority of videos being neither understandable nor actionable. As patients increasingly turn to internet video databases like YouTube for medical information, it is critical that physicians and institutions create A/V material that is accurate, understandable, and actionable.

Readability of online patient education materials for parents after a failed newborn hearing screen

OBJECTIVE

A cross-sectional study design was utilized to evaluate the readability of patient education materials on the newborn hearing screen from Google and major institutions.

METHODS

The top 55 websites from the Google search "failed newborn hearing screen" and websites from major institutions (the U.S. News & World Report ranked top 10 children's hospitals, the top 5 pediatric otolaryngology fellowships as ranked by Doximity Residency Navigator, the Centers for Disease and Control, the American Academy of Pediatrics and the American Academy of Otolaryngology-Head and Neck Surgery) were compiled. Text from each website was edited to remove extraneous text. Readability grade was calculated using the Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease Score, Gunning-Fog Index (GFI), Simple Measure of Gobbledygook (SMOG), Coleman-Liau Index, and Automated Readability Index. Intra- and inter-observer reliability were assessed.

RESULTS

26 websites from Google and 29 websites from major institutions were evaluated. From Google, provider-oriented websites (n = 2) were more difficult to read than patient-oriented websites (n = 24) with statistical significance for FKGL (p < 0.001), GFI (p < 0.013) and SMOG (p < 0.001). From the major institutions, more than half were at a reading level that exceeded the average American adult with an average FKGL 9.71 ± 2.69. Intra- and inter-observer reliability were both excellent with an intra-class correlation coefficient for each readability tool ≥0.950 (p < 0.001).

CONCLUSION

Online patient education materials about the newborn hearing screen may be too difficult for the average reader. Revisions to these materials and redirection to more readable online resources may be necessary to benefit a more inclusive patient population.

YouTube Videos as a Source of Information About Immunology for Medical Students: Cross-Sectional Study

BACKGROUND

The use of the internet as a source of information has grown exponentially in the last decade. YouTube is currently the second most visited website and a major Web-based educational resource for medical students.

OBJECTIVE

The aim of this study was to evaluate the quality, accuracy, and attractiveness of the information acquired from YouTube videos about 2 central concepts in immunology.

METHODS

YouTube videos posted before August 27, 2018 were searched using selected keywords related to either antigen presentation or immunoglobulin gene rearrangement. Video characteristics were recorded, and the Video Power Index (VPI) was calculated. Videos were assessed using 5 validated scoring systems: understandability and attractiveness, reliability, content and comprehensiveness, global quality score (GQS), and a subjective score. Videos were categorized by educational usefulness and by source.

RESULTS

A total of 82 videos about antigen presentation and 70 about immunoglobulin gene rearrangement were analyzed. Videos had a mean understandability and attractiveness score of 6.57/8 and 5.84/8, content and comprehensiveness score of 9.84/20 and 5.84/20, reliability score of 1.65/4 and 1.53/4, GQS of 3.38/5 and 2.76/5, and subjective score of 2.00/3 and 2.00/3, respectively. The organized channels group tended to have the highest VPI and GQS.

CONCLUSIONS

YouTube can provide medical students with some useful information about immunology, although content wise it cannot substitute textbooks and academic courses. Students and teachers should be aware of the educational quality of available videos if they intend to use them in the context of blended learning.

A Comparative Multimetric Assessment of English and Spanish Carpal Tunnel Syndrome Materials

BACKGROUND

Spanish-speaking Hispanics living in the United States utilize the internet as a primary means to obtain health information. Accurate, accessible information is important for English speakers; however, it could have even greater utility for Spanish speakers who have lower health literacy levels. The aim of this study was to evaluate and compare online English and Spanish carpal tunnel surgery materials provided by using a multimetric approach.

MATERIALS AND METHODS

A web search using the English term "carpal tunnel surgery" was performed. The first 10 institutional/organizational websites that provided carpal tunnel surgery information in English and Spanish were included. All relevant online materials were evaluated using the Patient Education and Materials Assessment Tool (PEMAT), Cultural Sensitivity Assessment Tool (CSAT), and Simplified Measure of Gobbledygook, Spanish (SOL) to assess understandability and actionability, cultural sensitivity, and readability, respectively.

RESULTS

There were no statistically significant differences in understandability or actionability scores between Spanish and English materials. Average cultural sensitivity scores for Spanish materials were significantly lower than English materials (P = 0.015). The average reading grade level of online English materials was greater than that for Spanish materials (P = 0.011). Both mean values were above the recommended sixth-grade reading level.

CONCLUSIONS

Online patient-directed information regarding carpal tunnel surgery exceeded the recommended reading grade level for both English and Spanish-speaking populations. Most Spanish materials were often direct translations and were not contoured to the elevated literacy needs of this demographic. Institutions must caution their authors to tailor their web material in a way that is sensitive to their target population to optimize understanding.

Evaluation of Children Quality of Life after Serous Otitis Media Surgery

Introduction: Otitis media with effusion is a common childhood disease diagnosed with the accumulation of serous fluid or mucous in the middle ear. If not treated, the sustainable effusion leads to change and destruction of middle ear structures. One of the most successful treatment methods is myringotomy and ventilation tube insertion, which leads to improvement of patients' quality of life. The aim of this study was to evaluate children's quality of life after inserting ventilation tubes. Methods: This prospective cross-sectional study examined the quality of life of the children suffering from otitis media with sustainable effusion, who were candidates for ventilation tube insertion. Otitis Media outcome-22 (OMO -22) questionnaire, was used to record the patients' problems in a 12-week period before and after surgery. Results: In this study, 70 patients, including 43 (61.4%) boys and 27 (38.6%) girls (1y-13y), who completed the questionnaire, were studied. Treatment with ventilation tubes improved many physical symptoms of the hearing loss along with speech and behavioral symptoms. A significant reduction was observed in the number of physician visits and the use of antibiotics for ear problems. Conclusion: Ventilation tube insertion not only reduced the symptoms of otitis media with effusion, but also improved the children's quality of life.

Health literacy in pediatric otolaryngology: A scoping review

OBJECTIVE

To review research on status and outcomes of health literacy in pediatric otolaryngology and identify opportunities for quality improvement.

METHODS

We performed a scoping review, adhering to methodologic standards. A combination of MeSH terms and keywords related to health literacy in otolaryngology was used to conduct a search. Relevant studies were identified using PubMed, Ovid MEDLINE, and Google Scholar databases. Studies were selected for inclusion by two authors if they addressed the domains of pediatric otolaryngology as well as health literacy. Data were abstracted from each study on the number of participants, the setting, the study design, the outcome measure, the intervention used, and the overall theme. Authors identified prominent overarching themes and grouped studies accordingly. Results were then tabulated for further review and to discern implications for future practice and research.

RESULTS

Of 1046 articles identified, 20 articles were included. Studies fell into three major themes: readability of patient materials, patient recall after informed consent, and optimal patient education. Prominent findings included the following: 1. Much of the printed and electronic educational material in otolaryngology is above the recommended reading level for public health information; 2. Parents do not easily recall information provided verbally or in written form; and 3. Adding visual and multimodal components improves the success of parental education.

CONCLUSION

Health literacy in pediatric otolaryngology may influence comprehension of educational materials and adequacy of informed consent. Future research may address whether patient health literacy affects clinical outcomes.

Quality of online otolaryngology health information

OBJECTIVE

Up to 70 per cent of the population search online for medical or health-related information. This study aimed to assess the quality of online health resources available to educate patients on a variety of otolaryngological conditions.

METHODS

Two clinicians independently analysed the quality and content of educational websites (patient.co.uk and wikipedia.org) for common and uncommon diagnoses, with cancerresearchuk.org also used to assess two head and neck cancers.

RESULTS

Cancerresearchuk.org, followed by patient.co.uk, scored most highly in their ability to inform readers on a selection of otolaryngological conditions. Although wikipedia.org was less likely to include all relevant information and was more difficult to read, it still provided mostly accurate information.

CONCLUSION

Where possible, patients should be advised to access professionally maintained health information websites (patient.co.uk and cancerresearchuk.org). However, wikipedia.org can provide adequate information, although it lacks depth and can be difficult to understand.