The QUEST for quality online health information: validation of a short quantitative tool

Accuracy, readability, and quality of websites about metastatic renal cell carcinoma treatment

OBJECTIVES

Patients with metastatic renal cell carcinoma (mRCC) face complex treatment decisions and frequently turn to the Internet for treatment information. The content of patient educational websites about mRCC treatment has not been evaluated. This study evaluated the accuracy, readability, and quality of websites about the treatment of mRCC.

METHODS

A total of 2,700 Internet queries were performed. Across 3 Internet search engines, 25 links of 36 permutations of mRCC keywords and their synonyms were screened for eligibility. Eligible websites were English-language websites containing information about mRCC treatments. Sponsored, social media, provider-facing, and news websites were excluded. Accuracy of eligible websites was evaluated in 2 domains: (1) Completeness by calculating the percentage of mRCC facts included in websites using an investigator-created checklist based on the NCI's RCC Treatment (PDQ®)-patient version, and (2) Correctness by identifying incorrect statements that were inconsistent with guidelines. Websites containing ≥60% of checklist items had a "passing" completeness score. Incorrect statements were tallied and qualitatively categorized. Readability was evaluated using the Fry and SMOG formulae, which calculate reading grade levels. Quality was evaluated using validated instruments that appraise health information quality: QUEST (scored 0-28), which focuses on online information, and DISCERN (scored 16-80), which focuses on treatment choices.

RESULTS

Thirty-nine websites were analyzed. Mean completeness score was 30% (range 0%-69%); only 2 (5%) websites had a passing score. Twelve (31%) websites had ≥1 incorrect statement, such as listing homeopathy or hormone therapy as mRCC treatment options, or including outdated statements. Mean readability levels were 11th and 12th grades for the Fry and SMOG methods, respectively. No website had a reading level lower than 9th grade. Mean QUEST score was 19 (range 9-28); authorship, complementarity, and currency items had the lowest scores. Mean DISCERN score was 56 (range 42-76), with 7 (18%) websites rated "excellent", 22 (56%) rated "good", and 10 (26%) rated fair.

CONCLUSIONS

Many websites about mRCC treatment have incomplete, inaccurate, and unreadable information. Quality is highly variable. Efforts to improve accuracy, readability, and quality are needed to ensure that patients with mRCC can make well-informed treatment decisions and avoid harm from misinformation.

What Do People Want to Know About Cochlear Implants: A Google Analytic Study

OBJECTIVE

Identify the questions most frequently asked online about cochlear implants (CI) and assess the readability and quality of the content.

METHODS

A Google search engine observational study was conducted via a search response optimization (SEO) tool. The SEO tool listed the questions generated by Google's "People Also Ask" (PAA) feature for the search queries "cochlear implant" and "cochlear implant surgery." The top 50 PAA questions for each query were conceptually classified. Sourced websites were evaluated for readability, transparency and information quality, and ability to answer the question. Readability and accuracy in answering questions were also compared to the responses from ChatGPT 3.5.

RESULTS

The PAA questions were commonly related to technical details (21%), surgical factors (18%), and postoperative experiences (12%). Sourced websites mainly were from academic institutions, followed by commercial companies. Among all types of websites, readability, on average, did not meet the recommended standard for health-related patient education materials. Only two websites were at or below the 8th-grade level. Responses by ChatGPT had significantly poorer readability compared to the websites (p < 0.001). These online resources were not significantly different in the percentage of accurately answering the questions (websites: 78%, ChatGPT: 85%, p = 0.136).

CONCLUSIONS

The most searched topics were technical details about devices, surgical factors, and the postoperative experience. Unfortunately, most websites did not meet the ideal criteria of readability, quality, and credibility for patient education. These results highlight potential knowledge gaps for patients, deficits in current online education materials, and possible tools to better support CI candidate decision-making.

LEVEL OF EVIDENCE

NA Laryngoscope, 2024.

Assessment of the Quality of Patient-Oriented Internet Information on Fluorescence Imaging in Surgery

BACKGROUND

In the digital age, patients are increasingly turning to the Internet to seek medical information to aid in their decision-making process before undergoing medical treatments. Fluorescence imaging is an emerging technological tool that holds promise in enhancing intra-operative decision-making during surgical procedures. This study aims to evaluate the quality of patient information available online regarding fluorescence imaging in surgery and assesses whether it adequately supports informed decision-making.

METHOD

The term "patient information on fluorescence imaging in surgery" was searched on Google. The websites that fulfilled the inclusion criteria were assessed using 2 scoring instruments. DISCERN was used to evaluate the reliability of consumer health information. QUEST was used to assess authorship, tone, conflict of interest and complementarity.

RESULTS

Out of the 50 websites identified from the initial search, 10 fulfilled the inclusion criteria. Only two of these websites were updated in the last two years. The definition of fluorescence imaging was stated in only 50% of the websites. Although all websites mentioned the benefits of fluorescence imaging, none mentioned potential risks. Assessment by DISCERN showed that 30% of the websites were rated low and 70% were rated moderate. With QUEST, the websites demonstrated an average score of 62.5%.

CONCLUSION

This study highlights the importance of providing patients with accurate and balanced information about medical technologies and procedures they may undergo. Fluorescence imaging in surgery is a promising technology that can potentially improve surgical outcomes. However, patients need to be well-informed about its benefits and limitations in order to make informed decisions about their healthcare.

Evaluating Health Literacy Resources for Secondary Hyperparathyroidism in End-Stage Kidney Disease

INTRODUCTION

Parathyroidectomy is recommended for severe secondary hyperparathyroidism (SHPT) due to end-stage kidney disease (ESKD), but surgery is underutilized. High quality and accessible online health information, recommended to be at a 6th-grade reading level, is vital to improve patient health literacy. This study evaluated available online resources for SHPT from ESKD based on information quality and readability.

METHODS

Three search engines were queried using the terms "parathyroidectomy for secondary hyperparathyroidism," "parathyroidectomy kidney/renal failure," "parathyroidectomy dialysis patients," "should I have surgery for hyperparathyroidism due to kidney failure?," and "do I need surgery for hyperparathyroidism due to kidney failure if I do not have symptoms?" Websites were categorized by source and origin. Two independent reviewers determined information quality using JAMA (0-4) and DISCERN (1-5) frameworks, and scores were averaged. Cohen's kappa evaluated inter-rater reliability. Readability was determined using the Flesch Kincaid Reading Ease, Flesch Kincaid Grade Level, and Simple Measure of Gobbledygook tools. Median readability scores were calculated, and corresponding grade level determined. Websites with reading difficulties >6th grade level were calculated.

RESULTS

Thirty one (86.1%) websites originated from the U.S., with most from hospital-associated (63.9%) and foundation/advocacy sources (30.6%). The mean JAMA and DISCERN scores for all websites were 1.3 ± 1.4 and 2.6 ± 0.7, respectively. Readability scores ranged from grade level 5-college level, and most websites scored above the recommended 6th grade level.

CONCLUSIONS

Patient-oriented websites tailoring SHPT from ESKD are at a reading level higher than recommended, and the quality of information is low. Efforts must be made to improve the accessibility and quality of information for all patients.

eHealth Literacy and Web-Based Health Information-Seeking Behaviors on COVID-19 in Japan: Internet-Based Mixed Methods Study

BACKGROUND

During the COVID-19 pandemic, much misinformation and disinformation emerged and spread rapidly via the internet, posing a severe public health challenge. While the need for eHealth literacy (eHL) has been emphasized, few studies have compared the difficulties involved in seeking and using COVID-19 information between adult internet users with low or high eHL.

OBJECTIVE

This study examines the association between eHL and web-based health information-seeking behaviors among adult Japanese internet users. Moreover, this study qualitatively shed light on the difficulties encountered in seeking and using this information and examined its relationship with eHL.

METHODS

This cross-sectional internet-based survey (October 2021) collected data from 6000 adult internet users who were equally divided into sample groups by gender, age, and income. We used the Japanese version of the eHL Scale (eHEALS). We also used a Digital Health Literacy Instrument (DHLI) adapted to the COVID-19 pandemic to assess eHL after we translated it to Japanese. Web-based health information-seeking behaviors were assessed by using a 10-item list of web sources and evaluating 10 topics participants searched for regarding COVID-19. Sociodemographic and other factors (eg, health-related behavior) were selected as covariates. Furthermore, we qualitatively explored the difficulties in information seeking and using. The descriptive contents of the responses regarding difficulties in seeking and using COVID-19 information were analyzed using an inductive qualitative content analysis approach.

RESULTS

Participants with high eHEALS and DHLI scores on information searching, adding self-generated information, evaluating reliability, determining relevance, and operational skills were more likely to use all web sources of information about COVID-19 than those with low scores. However, there were negative associations between navigation skills and privacy protection scores when using several information sources, such as YouTube (Google LLC), to search for COVID-19 information. While half of the participants reported no difficulty seeking and using COVID-19 information, participants who reported any difficulties, including information discernment, incomprehensible information, information overload, and disinformation, had lower DHLI score. Participants expressed significant concerns regarding "information quality and credibility," "abundance and shortage of relevant information," "public trust and skepticism," and "credibility of COVID-19-related information." Additionally, they disclosed more specific concerns, including "privacy and security concerns," "information retrieval challenges," "anxieties and panic," and "movement restriction."

CONCLUSIONS

Although Japanese internet users with higher eHEALS and total DHLI scores were more actively using various web sources for COVID-19 information, those with high navigation skills and privacy protection used web-based information about COVID-19 cautiously compared with those with lower proficiency. The study also highlighted an increased need for information discernment when using social networking sites in the "Health 2.0" era. The identified categories and themes from the qualitative content analysis, such as "information quality and credibility," suggest a framework for addressing the myriad challenges anticipated in future infodemics.

ChatGPT can help guide and empower patients after prostate cancer diagnosis

BACKGROUND/OBJECTIVES

Patients often face uncertainty about what they should know after prostate cancer diagnosis. Web-based information is common but is at risk of being of poor quality or readability.

SUBJECTS/METHODS

We used ChatGPT, a freely available Artificial intelligence (AI) platform, to generate enquiries about prostate cancer that a newly diagnosed patient might ask and compared to Google search trends. Then, we evaluated ChatGPT responses to these questions for clinical appropriateness and quality using standardised tools.

RESULTS

ChatGPT generates broad and representative questions, and provides understandable, clinically sound advice.

CONCLUSIONS

AI can guide and empower patients after prostate cancer diagnosis through education. However, the limitations of the ChatGPT language-model must not be ignored and require further evaluation and optimisation in the healthcare field.

Navigating Challenges and Seizing Opportunities in China's Era of Online Health Education

The rapid growth of internet users in China presents opportunities for advancing the "Healthy China 2030" initiative through online health education. Platforms like "Shanghai Health Cloud" and "National Health Information Platform" improve health literacy and management, enhancing overall public health. However, challenges such as the digital divide and the spread of unverified health information hinder progress. Addressing these issues requires enhancing digital infrastructure, employing advanced technologies for information validation, and setting high standards for online health services. Integrated efforts from various sectors are essential to maximize the benefits of online health education in China.

Ostomy Information on the Internet-Is It Good Enough?

PURPOSE

The aim of this study was to determine which internet search engines and keywords patients with ostomies utilize, to identify the common websites using these terms, to determine what aspects of information they wanted, and to perform a quality and readability assessment for these websites.

DESIGN

A cross-sectional survey of persons with ostomies to identify search engines and terms, followed by a structured assessment of the quality and readability of the identified web pages.

SUBJECT AND SETTINGS

The sample comprised 20 hospitalized patients with ostomies cared for on a colorectal surgical ward of a tertiary care hospital located in Melbourne, Australia. There were 15 (75%) adult males and 5 (25%) adult females; their mean age was 52.2 years. Participants were surveyed between August and December 2020.

METHODS

Patients with newly formed ostomies were surveyed about which search engines and keywords they would use to look for information and for which questions regarding ostomies they wanted answers. In addition, 2 researchers then performed independent searches using the search terms identified by patient participants. These searches were conducted in August 2021, with the geographical location set to Australia. The quality of the websites was graded using the DISCERN, Ensuring Quality Information for Patients, and Quality Evaluation Scoring Tool scoring assessments, and their readability was graded using the Flesch Reading Ease Score tool.

RESULTS

Participants used Google as their primary search engine. Four keywords/phrases were identified: stoma for bowel surgery, ileostomy, colostomy, and caring for stoma. Multiple web pages were identified, 8 (21%) originated from Australia, 7 (18%) were from the United Kingdom, and 23 (61%) were from the United States. Most web pages lacked recent updates; only 18% had been undated within the last 12 months. The overall quality of the online information on ostomies was moderate with an average level of readability, deemed suitable for patient educational purposes.

CONCLUSIONS

Information for persons living with an ostomy can be obtained from multiple web pages, and many sites have reasonable quality and are written at a suitable level. Unfortunately, these websites are rarely up-to-date and may contain advice that may not be applicable to individual patients.

Standardized reporting for systematic global evaluation of axial spondyloarthritis: An evidence-based and consensus-driven initiative

INTRODUCTION

National and international scientific societies advocate for a regular, systematic, and standardized global evaluation of axial spondyloarthritis (axSpA) patients. However, there are no recommendations specifying the content of this global evaluation. This initiative aimed to propose a standardized reporting framework, using evidence-based and consensus approaches, to collect data on all domains of axSpA.

METHODS

A literature review and consensus process involved a steering committee and an expert panel of 37 rheumatologists and health professionals. The first steering committee took place in March 2022 and identified the main domains for inclusion in the standardized report. A hierarchical literature review was conducted to identify items within these domains and tools for assessment. The items and tools for assessment were discussed and consensus was reached through a vote session during an expert meeting that took place in March 2023.

RESULTS

The steering committee identified four main domains to include in the standardized reporting framework: disease assessment, comorbidities, lifestyle, and quality of life. Items and tools for assessment were adopted after the expert meeting. Additionally, recommendations regarding digital tools (websites, apps, social media) were provided.

CONCLUSION

This initiative led to a consensus, based on evidence and expertise, on a reporting framework for use during periodic systematic global evaluations of axSpa in daily practice.

ChatGPT Is Moderately Accurate in Providing a General Overview of Orthopaedic Conditions

Background

ChatGPT is an artificial intelligence chatbot capable of providing human-like responses for virtually every possible inquiry. This advancement has provoked public interest regarding the use of ChatGPT, including in health care. The purpose of the present study was to investigate the quantity and accuracy of ChatGPT outputs for general patient-focused inquiries regarding 40 orthopaedic conditions.

Methods

For each of the 40 conditions, ChatGPT (GPT-3.5) was prompted with the text "I have been diagnosed with [condition]. Can you tell me more about it?" The numbers of treatment options, risk factors, and symptoms given for each condition were compared with the number in the corresponding American Academy of Orthopaedic Surgeons (AAOS) OrthoInfo website article for information quantity assessment. For accuracy assessment, an attending orthopaedic surgeon ranked the outputs in the categories of <50%, 50% to 74%, 75% to 99%, and 100% accurate. An orthopaedics sports medicine fellow also independently ranked output accuracy.

Results

Compared with the AAOS OrthoInfo website, ChatGPT provided significantly fewer treatment options (mean difference, -2.5; p < 0.001) and risk factors (mean difference, -1.1; p = 0.02) but did not differ in the number of symptoms given (mean difference, -0.5; p = 0.31). The surgical treatment options given by ChatGPT were often nondescript (n = 20 outputs), such as "surgery" as the only operative treatment option. Regarding accuracy, most conditions (26 of 40; 65%) were ranked as mostly (75% to 99%) accurate, with the others (14 of 40; 35%) ranked as moderately (50% to 74%) accurate, by an attending surgeon. Neither surgeon ranked any condition as mostly inaccurate (<50% accurate). Interobserver agreement between accuracy ratings was poor (κ = 0.03; p = 0.30).

Conclusions

ChatGPT provides at least moderately accurate outputs for general inquiries of orthopaedic conditions but is lacking in the quantity of information it provides for risk factors and treatment options. Professional organizations, such as the AAOS, are the preferred source of musculoskeletal information when compared with ChatGPT.

Clinical Relevance

ChatGPT is an emerging technology with potential roles and limitations in patient education that are still being explored.

Quality and Readability Assessment of Web-based Arabic Health Information on Early Childhood Caries

Background

Patients are increasingly using the internet for searching health-related information. However, the quality and readability of the information available on the internet need to be assessed. To date, no study has assessed the quality and readability of web-based Arabic health information on early childhood caries.

Objectives

To evaluate the quality and readability of patient-oriented online Arabic health information regarding early childhood caries.

Materials and Methods

For this infodemiological study, the Google and Yahoo search engines were searched using specific Arabic terms for early childhood caries, and the top 100 searches from both search engines were considered. Eligible websites were categorized in terms of affiliation as commercial, health portal, dental practice, professional, and journalism. The quality of the websites was assessed using the QUality Evaluation Scoring Tool (QUEST), and readability using the Gunning Fog index (GFI).

Results

A total of 140 websites were included after applying the exclusion criteria, of which 50.7% websites were of journalism. The majority of the websites (70%) had an overall low-quality level, with a QUEST score <10. The quality of websites retrieved from Google searches was of significantly higher quality than those from Yahoo (P < 0.0001). More than half (51.4%) of the websites had good readability, with a GFI score ≤8. Journalism websites had a significantly higher proportion of websites with poor readability level (62%) compared with other affiliations (P = 0.0072).

Conclusion

The web-based Arabic information regarding early childhood caries is currently of low quality and moderate readability level, thereby indicating a need for improving such patient-facing content.

Quality Assessment of Degenerative Cervical Myelopathy Information on the Internet

BACKGROUND

Patient education is a key element of spinal surgery informed consent. Patients frequently access health information online, yet this information is unregulated and of variable quality. We aimed to assess the quality of information available on degenerative cervical myelopathy (DCM) websites with a focus on identifying high-quality information websites.

METHODS

We performed a Google search using keywords pertaining to DCM. The top 50 websites returned were classified based on their publication source, intended audience, and country of origin. The quality of these websites was assessed using both the DISCERN instrument and Journal of the American Medical Association (JAMA) benchmark criteria. We also utilized a novel Myelopathy Information Scoring Tool (MIST) to assess the comprehensiveness, accuracy, and detail of online DCM information.

RESULTS

The mean DISCERN score was 39.9 out of 80. Only one-quarter of these websites were rated "good" or "excellent" using DISCERN, and the remaining were rated "very poor," "poor," and "fair." The mean JAMA benchmark score was 1.6 out of 4, with 23 out of 50 websites scoring 0. Evaluation using MIST found a mean score of 25.6 out of 50. Using 30 points as a satisfactory MIST cutoff, 72% of DCM websites were deemed critically deficient and unsatisfactory for comprehensive patient education. Both DISCERN and MIST indicated poorest information pertaining to surgical risks and complications as well as treatment outcomes. Websites such as Orthoinfo.aaos.org and Myelopathy.org provided reliable, trustworthy, and comprehensive patient education.

CONCLUSIONS

Information available on almost three-quarters of DCM websites was of poor quality, with information regarding complications and treatment outcomes most deficient. Clinicians should be aware of quality sites where patients may be directed to augment patient education and surgical counseling.

Utility of ChatGPT for Automated Creation of Patient Education Handouts: An Application in Neuro-Ophthalmology

BACKGROUND

Patient education in ophthalmology poses a challenge for physicians because of time and resource limitations. ChatGPT (OpenAI, San Francisco) may assist with automating production of patient handouts on common neuro-ophthalmic diseases.

METHODS

We queried ChatGPT-3.5 to generate 51 patient education handouts across 17 conditions. We devised the "Quality of Generated Language Outputs for Patients" (QGLOP) tool to assess handouts on the domains of accuracy/comprehensiveness, bias, currency, and tone, each scored out of 4 for a total of 16. A fellowship-trained neuro-ophthalmologist scored each passage. Handout readability was assessed using the Simple Measure of Gobbledygook (SMOG), which estimates years of education required to understand a text.

RESULTS

The QGLOP scores for accuracy, bias, currency, and tone were found to be 2.43, 3, 3.43, and 3.02 respectively. The mean QGLOP score was 11.9 [95% CI 8.98, 14.8] out of 16 points, indicating a performance of 74.4% [95% CI 56.1%, 92.5%]. The mean SMOG across responses as 10.9 [95% CI 9.36, 12.4] years of education.

CONCLUSIONS

The mean QGLOP score suggests that a fellowship-trained ophthalmologist may have at-least a moderate level of satisfaction with the write-up quality conferred by ChatGPT. This still requires a final review and editing before dissemination. Comparatively, the rarer 5% of responses collectively on either extreme would require very mild or extensive revision. Also, the mean SMOG score exceeded the accepted upper limits of grade 8 reading level for health-related patient handouts. In its current iteration, ChatGPT should be used as an efficiency tool to generate an initial draft for the neuro-ophthalmologist, who may then refine the accuracy and readability for a lay readership.

Can ChatGPT Provide Quality Information on Integrative Oncology? A Brief Report

This short report evaluated the accuracy and quality of information provided by ChatGPT regarding the use of complementary and integrative medicine for cancer. Using the QUality Evaluation Scoring Tool, a panel of 12 reviewers assessed ChatGPT's responses to 8 questions. The study found that ChatGPT provided moderate-quality responses that were relatively unbiased and not misleading. However, the chatbot's inability to reference specific scientific studies was a significant limitation. Patients with cancer should not rely on ChatGPT for clinical advice until further systematic validation. Future studies should examine how patients perceive ChatGPT's information and its impact on communication with health care professionals.

What Does ChatGPT Know About Dementia? A Comparative Analysis of Information Quality

The quality of information about dementia retrieved using ChatGPT is unknown. Content was evaluated for length, readability, and quality using the QUEST, a validated tool, and compared against online material from three North American organizations. Both sources of information avoided conflicts of interest, supported the patient-physician relationship, and used a balanced tone. Official bodies but not ChatGPT referenced identifiable research and pointed to local resources. Users of ChatGPT are likely to encounter accurate but shallow information about dementia. Recommendations are made for information creators and providers who counsel patients around digital health practices.

Social robotics for children: an investigation of manufacturers' claims

As the market for commercial children's social robots grows, manufacturers' claims around the functionality and outcomes of their products have the potential to impact consumer purchasing decisions. In this work, we qualitatively and quantitatively assess the content and scientific support for claims about social robots for children made on manufacturers' websites. A sample of 21 robot websites was obtained using location-independent keyword searches on Google, Yahoo, and Bing from April to July 2021. All claims made on manufacturers' websites about robot functionality and outcomes (n = 653 statements) were subjected to content analysis, and the quality of evidence for these claims was evaluated using a validated quality evaluation tool. Social robot manufacturers made clear claims about the impact of their products in the areas of interaction, education, emotion, and adaptivity. Claims tended to focus on the child rather than the parent or other users. Robots were primarily described in the context of interactive, educational, and emotional uses, rather than being for health, safety, or security. The quality of the information used to support these claims was highly variable and at times potentially misleading. Many websites used language implying that robots had interior thoughts and experiences; for example, that they would love the child. This study provides insight into the content and quality of parent-facing manufacturer claims regarding commercial social robots for children.

Are popular books about diet and health written based on scientific evidence? A comparison of citations between the USA and Japan

OBJECTIVES

To describe and compare the references cited in popular books about diet and health between the USA and Japan.

DESIGN

Books were selected based on their best-seller rankings in the diet and health category of online bookstores. We identified references throughout all pages of the books and examined the number of references, reference format (identifiable or not) and presence of specific types of references, such as systematic reviews of human research. We compared the characteristics of references between the two countries and examined related factors to citation.

SETTING

Cross-sectional study.

PARTICIPANTS

Books (n 100 in each country).

RESULTS

Among 100 books from each country, sixty-five US and sixty-six Japanese books had references. Forty-five US books cited more than 100 references, against only five Japanese books. The number of books that cited systematic reviews of human research differed between the USA (n 49) and Japan (n 9). Additionally, the number of books that provided identifiable information for all references was significantly higher in the USA (n 63) than in Japan (n 42). Books whose first authors have licences of medical doctors were more likely to cite references than those without in both countries.

CONCLUSIONS

Two-thirds of books about diet and health cited references in both the USA and Japan, but Japanese books cited fewer references and were less likely to cite systematic reviews and provide identifiable references than US books. Further research into the scientific reliability of information in books about diet and health is warranted.

Web-Based Content on Diet and Nutrition Written in Japanese: Infodemiology Study Based on Google Trends and Google Search

BACKGROUND

The increased availability of content of uncertain integrity obtained through the internet is a major concern. To date, however, there has been no comprehensive scrutiny of the fitness-for-purpose of web-based content on diet and nutrition.

OBJECTIVE

This cross-sectional study aims to describe diet- and nutrition-related web-based content written in Japanese, identified via a systematic extraction strategy using Google Trends and Google Search.

METHODS

We first identified keywords relevant for extracting web-based content (eg, blogs) on diet and nutrition written in Japanese using Google Trends. This process included identification of 638 seed terms, identification of approximately 1500 pairs of related queries (top) and search terms, the top 10% of which were extracted to identify 160 relevant pairs of related queries (top) and search terms, and identification of 107 keywords for search. We then extracted relevant web-based content using Google Search.

RESULTS

The content (N=1703) examined here was extracted following a search based on 107 keywords. The most common themes included food and beverages (390/1703, 22.9%), weight management (366/1703, 21.49%), health benefits (261/1703, 15.33%), and healthy eating (235/1703, 13.8%). The main disseminators were information technology companies and mass media (474/1703, 27.83%), food manufacturers (246/1703, 14.45%), other (236/1703, 13.86%), and medical institutions (214/1703, 12.57%). Less than half of the content (790/1703, 46.39%) clearly indicated the involvement of editors or writers. More than half of the content (983/1703, 57.72%) was accompanied by one or more types of advertisement. The proportion of content with any type of citation reference was 40.05% (682/1703). The themes and disseminators of content were significantly associated with the involvement of editors or writers, accompaniment with advertisement, and citation of reference. In particular, content focusing on weight management was more likely to clearly indicate the involvement of editors or writers (212/366, 57.9%) and to be accompanied by advertisement (273/366, 74.6%), but less likely to have references cited (128/366, 35%). Content from medical institutions was less likely to have citation references (62/214, 29%).

CONCLUSIONS

This study highlights concerns regarding the authorship, conflicts of interest (advertising), and the scientific credibility of web-based diet- and nutrition-related information written in Japanese. Nutrition professionals and experts should take these findings seriously because exposure to nutritional information that lacks context or seems contradictory can lead to confusion and backlash among consumers. However, more research is needed to draw firm conclusions about the accuracy and quality of web-based diet- and nutrition-related content and whether similar results can be obtained in other major mass media or social media outlets and even other languages.

Designing a doctor evaluation index system for an online medical platform based on the information system success model in China

Objective

In the context of "internet + medical health" and emphasis on evaluation mechanism for medical and health talents in China, we design an evaluation index system for doctors on online medical platforms by synthesizing two patterns of existing online medical platforms, which is the first step to enhance the capabilities of doctors on online medical platforms.

Methods

Based on the doctor evaluation model integrating information systems success model (ISS-DE model) and grounded theory, the evaluation indicators were obtained through expert interviews, offline medical institutions investigation, online platforms investigation, and literature research, and were assigned weights using the analytic hierarchy process (AHP) method. A working group composed of 23 experts was set up to review and determine the competency standards of doctors on the online medical platforms.

Results

A new indicator framework covering 3 dimensions of system quality, service quality and information quality was constructed in this study. The index system included 3 first-level indicators, 8 s-level indicators and 60 third-level indicators, and each indicator was given different weightage.

Conclusion

The complete index system constructed by the Delphi method in this study is suitable for China's online medical platforms, which will help to improve the quality of platforms and the ability of doctors, thus promoting the process of internet medical integration.

Availability and use of web-based interventions for patients with head and neck cancer: a scoping review

PURPOSE

To identify and review the nature, scope and use of web-based interventions for patients with head and neck cancer (HNC).

METHOD

A scoping review guided by the methodological framework described by the Joanna Briggs Institute was performed to review empirical studies and websites. Seven electronic databases (CINAHL, Medline, Scopus, Embase, Cochrane, PubMed and PsycInfo) were searched from 2010 to 2020, data extracted and synthesised using thematic analysis. The Google search engine was employed, identifying the first 100 websites, using the search term head and neck cancer. Websites meeting eligibility criteria were assessed using the QUEST analysis tool, and descriptively summarised.

RESULTS

Thirteen empirical studies and 32 websites were included. As identified by empirical studies, web-based interventions were developed to provide (1) patient information on HNC and related treatments, (2) advice and support during treatment and (3) management strategies promoting adjustment to life with and beyond HNC. The reviewed websites provided minimal information to aid shared decision-making and facilitate preparedness for treatment, with few utilising patient narratives. Web-based interventions for HNC patients were mainly text based and focused on survivorship.

CONCLUSIONS

There is a paucity of theory-based, co-designed web-based interventions using patient narratives.

IMPLICATIONS FOR CANCER SURVIVORS

As patients increasingly look to the internet for advice and support, healthcare professionals are in a position to provide high-quality web-based interventions. There is an opportunity to rigorously develop a web-based intervention, containing narratives of peoples' lives before and after HNC treatment, aiding decision-making, preparedness for treatment and self-management.

WeChat official accounts' posts on medication use of 251 community healthcare centers in Shanghai, China: content analysis and quality assessment

Background

The dissemination of online health information (OHI) on medication use via WeChat Official Accounts (WOAs) is an effective way to help primary care practitioners (PCPs) address drug-related problems (DRPs) in the community. Although an increasing number of primary care institutions in China have published WOA posts on medication use, their content and quality have not yet been assessed.

Objective

This study aimed to explore the general features and content of WOA posts on medication use published by community healthcare centers (CHCs) in Shanghai, China and to assess their quality of content. It also aimed to explore the factors associated with the number of post views.

Methods

From June 1 to October 31, 2022, two coauthors independently screened WOA posts on medication use published throughout 2021 by the CHCs in Shanghai. Content analysis was performed to analyze their general features (format, length, and source, etc.) and content (types of medicines and diseases). The QUEST tool was used to assess the quality of the posts. We compared the differences among posts published by CHCs in central urban areas and suburban areas, and used multiple linear regression to explore the factors associated with the number of post views.

Results

A total of 236 WOAs of interest published 37,147 posts in 2021, and 275 (0.74%) of them were included in the study. The median number of post views was 152. Thirty percent of the posts were reviewed by the CHCs' staff before publication and only 6% provided information on PCPs' consultations. The most commonly mentioned medicines and diseases in the posts were Chinese patent medicines (37.1%) and respiratory diseases (29.5%). The posts frequently provided information on indications (77%) and usage (56%) but rarely on follow-up (13%) and storage (11%). Of the posts, 94.9% had a total QUEST score < 17 (full score = 28). The median number of post views and total post quality scores did not significantly differ among the CHCs in central urban and suburban areas. In the multiple linear regression model, the number of post views was associated with scores of complementarity (B = 56.47, 95% CI 3.05, 109.89) and conflict of interest (B = -46.40, 95% CI -56.21, -36.60).

Conclusion

The quantity and quality of WOA posts on medication use published by CHCs in China need improvement. The quality of posts may partially impact the dissemination effect, but intrinsic causal associations merit further exploration.

Development and testing of the health information website evaluation tool on neck pain websites - An analysis of reliability, validity, and utility

OBJECTIVE

Online health information contributes to patient education and knowledge on disease management. The aims of this study were to design the Health Information Website Evaluation Tool (HIWET) to evaluate the quality of online information, and to investigate the reliability, validity, and utility of HIWET.

METHODS

HIWET was developed by a literature search and small-scale pilot testing. Upon development, psychometric properties of HIWET were evaluated on 20 neck pain websites. Reliability was analysed using Intra class correlation coefficient (ICC). Validity was analysed using Pearson and Spearman correlation coefficients. Utility was analysed using an independent samples t-test.

RESULTS

HIWET demonstrated excellent intra-rater reliability (0.94 (0.98-0.99), p < .001) and fair inter-rater reliability (0.55 (0.88-0.10), p = .04). HIWET demonstrated validity with strong correlation against DISCERN (r = 0.656, n = 20, p = .002) and LIDA (r = 0.564, n = 20, p = 0.010). HIWET was time-efficient when compared to three comparison tools combined.

CONCLUSION

HIWET is a reliable and valid tool for evaluating the qualities of online health information.

PRACTICAL IMPLICATIONS

HIWET has the advantages of being a simple, quick to use and freely accessible tool. It can be implemented into clinical practice, education, and research to evaluate quality of online health information.

Bridging the gap: Evidence-based practice guidelines for sports nutritionists

Evidence-based practice is a systematic approach to decision-making developed in the 1990s to help healthcare professionals identify and use the best available evidence to guide clinical practice and patient outcomes amid a plethora of information in often challenging, time-constrained circumstances. Today’s sports nutrition practitioners face similar challenges, as they must assess and judge the quality of evidence and its appropriateness to their athlete, in the often chaotic, time-pressed environment of professional sport. To this end, we present an adapted version of the evidence-based framework to support practitioners in navigating their way through the deluge of available information and guide their recommendations to athletes whilst also reflecting on their practice experience and skills as evidence-based practitioners, thus, helping to bridge the gap between science and practice in sport and exercise nutrition.

Educational videos about dyspareunia and the assessment of their quality and reliability: An observational study

Objective:

The aim of this study was to analyze the characteristics, availability, quality and reliability of the information found in educational videos about dyspareunia.

Methods:

A systematic search was carried out on YouTube searching for educational videos about dyspareunia. Video quality scores were assessed with the DISCERN instrument, Global Quality Scale (GQS) and QUEST tool. Video popularity was evaluated with the Video Power Index (VPI). Interobserver agreement and statistical correlations of the assessment scores were also carried out.

Results:

The first 150 videos that appeared were selected. The mean number of views was 35,513.1 views (SD 71,443.9), the mean video length was 05:39 (SD 04:38), the mean days online was 1,396.3 days (SD 706), the mean view ratio was 29.8 (SD 68.1), the mean Like count was 333 likes (SD 1000.6), the mean Dislike count was 13 dislikes (SD 25.3) and the mean VPI Index was 92.1% (SD 16.4%). The mean values for the DISCERN instrument were for Observer-1 40.9 points (SD 9.9) and for Observer-2 39.3 points (SD 8.5). Mean values for the GQS score were for Observer-1 3.1 points (SD 0.9) and for Observer-2 3.0 points (SD 0.8). Finally, mean values for the QUEST tool were for Observer-1 13.3 points (SD 4.1) and for Observer-2 13.1 points (SD 4.2).

Conclusions:

The quality of the videos found on YouTube about dyspareunia, according to DISCERN, GQS, and QUEST scores, are average. Videos with high quality scores were produced by Academic Institutions, despite having lower frequency visit rates.

An Evaluation of Online Cirrhosis-related Information Intended for Patients and Caregivers

With nearly 70% of American adults seeking online health information,¹ the internet represents a powerful space to provide patients with evidence-based, usable education about their chronic conditions. Although effective patient-oriented communication recommendations are available, the accessibility, reliability, and accuracy of online health content is largely unmonitored or evaluated. In a complex chronic disease, such as cirrhosis, understanding the online information landscape specific to patients can provide insight into resources that are available to support their self-management behaviors and to fill knowledge gaps.

Quality and accuracy of gastric cancer related videos in social media videos platforms

Background

Gastric cancer is a major public health problem worldwide. Social media has affected public’s daily lives in ways no one ever thought possible. Both TikoTok and its Chinese version Douyin are the most popular short video posting platform. This study aimed to evaluate the quality, accuracy, and completeness of videos for gastric cancer on TikTok and Douyin.

Methods

The terms “gastric cancer” was searched on TikTok in both English and Japanese, and on Douyin in Chinese. The first 100 videos in three languages (website’s default setting) were checked. QUality Evaluation Scoring Tool (QUEST) and DISCERN as the instrument for assessing the quality of the information in each video. Content was analysed under six categories (aetiology, anatomy, symptoms, preventions, treatments, and prognosis). The educational value and completeness were evaluated with a checklist developed by the researchers.

Results

A total of 78 videos in English, 63 in Japanese, and 99 in Chinese were analyzed. The types of sources were as follows: 6.4% in English, 4.8% in Japanese, and 57.6% in Chinese for health professionals; 93.6% in English, 95.2% in Japanese, and 3.0% in Chinese for private users; none in English and Japanese, but 39.4% in Chinese for other sources. In all, 20.5% in English, 17.5% in Japanese, and 93.9% in Chinese of videos had useful information about gastric cancer. Among the useful videos, the videos published in Chinese had the highest QUEST(p < 0.05) and DISCERN scores(p < 0.05), followed by those published in Japanese. Among the educational videos, prognosis in English (37.5%), symptoms in Japanese (54.5%), and prevention in Chinese (47.3%) were the most frequently covered topic.

Conclusions

TikTok in English and Japanese might not fully meet the gastric cancer information needs of public, but Douyin in Chinese was the opposite.

Development of Principles for Health-Related Information on Social Media: Delphi Study

Background

Health-related misinformation can be propagated via social media and is a threat to public health. Several quality assessment tools and principles to evaluate health-related information in the public domain exist; however, these were not designed specifically for social media.

Objective

This study aims to develop Principles for Health-related Information on Social Media (PRHISM), which can be used to evaluate the quality of health-related social media content.

Methods

A modified Delphi approach was used to obtain expert consensus on the principles and functions of PRHISM. Health and social media experts were recruited via Twitter, email, and snowballing. A total of 3 surveys were administered between February 2021 and May 2021. The first survey was informed by a literature review and included open-ended questions and items from existing quality assessment tools. Subsequent surveys were informed by the results of the proceeding survey. Consensus was deemed if ≥80% agreement was reached, and items with consensus were considered relevant to include in PRHISM. After the third survey, principles were finalized, and an instruction manual and scoring tool for PRHISM were developed and circulated to expert participants for final feedback.

Results

A total of 34 experts consented to participate, of whom 18 (53%) responded to all 3 Delphi surveys. In total, 13 principles were considered relevant and were included in PRHISM. When the instructions and PRHISM scoring tool were circulated, no objections to the wording of the final principles were received.

Conclusions

A total of 13 quality principles were included in the PRHISM tool, along with a scoring system and implementation tool. The principles promote accessibility, transparency, provision of authoritative and evidence-based information and support for consumers’ relationships with health care providers. PRHISM can be used to evaluate the quality of health-related information provided on social media. These principles may also be useful to content creators for developing high-quality health-related social media content and assist consumers in discerning high- and low-quality information.

A comparative study of strategies for identifying credible sources of mental health information online: Can clinical services deliver a youth-specific internet prescription?

BACKGROUND

Adolescents and young adults undertake internet searches before and after attending mental health consultations, but they express concerns about how they can identify reliable online mental health information (OMHI). This action research project examines how clinicians might direct young people towards credible OMHI by analysing which search strategies are most helpful for developing an 'internet prescription' targeted at a youth audience.

METHODS

Recently qualified doctors undertook searches for OMHI about the diagnosis and treatment of seven mental health conditions. Rankings of websites in search results were compared with reliability and quality ratings derived from established evaluation instruments [the Brief DISCERN and the 12-item Health on the Net (HoN) tool].

RESULTS

Of 140 websites identified through seven Google searches, only 15% of those evaluated were categorized as high quality. About 17% of websites were selling a treatment or service and about 25% were regarded as untrustworthy. Also, higher rankings in the search results were not indicative of better quality OMHI (Wilcoxon signed rank test: Z = -5.28; p < .001). Although the HoN and DISCERN tools provided useful insights into the credibility of OMHI, investigators did not think either instrument could be recommended for independent use by youth.

CONCLUSIONS

Our findings suggest that clinical services should take responsibility for identifying a range of credible sources of OMHI. The next step in translating the research strategy into clinical practice involves collaboration with young people to select those platforms most orientated to their needs and to develop rating instruments that enable youth to evaluate OMHI.

Digital Neurology Platform: Developing and implementing a rigorous content quality guideline

Background

Digital communication has emerged as a major source of scientific and medical information for health care professionals. There is a need to set up an effective and reliable methodology to assess and monitor the quality of content that is published on the internet.

Objective

The aim of this project was to develop content quality guidelines for Neurodiem, an independent scientific information platform dedicated to neurology for health care professionals and neuroscientists. These content quality guidelines are intended to be used by (1) content providers as a framework to meet content quality standards and (2) reviewers as a tool for analyzing and scoring quality of content.

Methods

Specific scientific criteria were designed using a 5-point scale to measure the quality of curated and original content published on the website: for Summaries, (1) source reliability and topic relevance for neurologists, (2) structure, and (3) scientific and didactic value; for Congress highlights, (1) relevance of congress selection, (2) congress coverage based on the original program, and (3) scientific and didactic value of individual abstracts; for Expert points of view and talks, (1) credibility (authorship) and topic relevance for neurologists, (2) scientific and didactic value, and (3) reliability (references) and format. The criteria were utilized on a monthly basis and endorsed by an independent scientific committee of widely recognized medical experts in neurology.

Results

Summary content quality for the 3 domains (reliability and relevance, structure, and scientific and didactic value) increased in the second month after the implementation of the guidelines. The domain scientific and didactic value had a mean score of 8.20/10. Scores for the domains reliability and relevance (8-9/10) and structure (45-55/60) showed that the maintenance of these 2 quality items over time was more challenging. Talks (either in the format of interviews or slide deck–supported scientific presentations) and expert point of view demonstrated high quality after the implementation of the content quality guidelines that was maintained over time (15-25/25).

Conclusions

Our findings support that content quality guidelines provide both (1) a reliable framework for generating independent high-quality content that addresses the educational needs of neurologists and (2) are an objective evaluation tool for improving and maintaining scientific quality level. The use of these criteria and this scoring system could serve as a standard and reference to build an editorial strategy and review process for any medical news or platforms.

Publicly Available, Interactive Web-Based Tools to Support Advance Care Planning: Systematic Review

Background

There is an increasing number of interactive web-based advance care planning (ACP) support tools, which are web-based aids in any format encouraging reflection, communication, and processing of publicly available information, most of which cannot be found in the peer-reviewed literature.

Objective

This study aims to conduct a systematic review of web-based ACP support tools to describe the characteristics, readability, and quality of content and investigate whether and how they are evaluated.

Methods

We systematically searched the web-based gray literature databases OpenGrey, ClinicalTrials.gov, ProQuest, British Library, Grey Literature in the Netherlands, and Health Services Research Projects in Progress, as well as Google and app stores, and consulted experts using the following eligibility criteria: web-based, designed for the general population, accessible to everyone, interactive (encouraging reflection, communication, and processing of information), and in English or Dutch. The quality of content was evaluated using the Quality Evaluation Scoring Tool (score 0-28—a higher score indicates better quality). To synthesize the characteristics of the ACP tools, readability and quality of content, and whether and how they were evaluated, we used 4 data extraction tables.

Results

A total of 30 tools met the eligibility criteria, including 15 (50%) websites, 10 (33%) web-based portals, 3 (10%) apps, and 2 (7%) with a combination of formats. Of the 30 tools, 24 (80%) mentioned a clear aim, including 7 (23%) that supported reflection or communication, 8 (27%) that supported people in making decisions, 7 (23%) that provided support to document decisions, and 2 (7%) that aimed to achieve all these aims. Of the 30 tools, 7 (23%) provided information on the development, all of which were developed in collaboration with health care professionals, and 3 (10%) with end users. Quality scores ranged between 11 and 28, with most of the lower-scoring tools not referring to information sources.

Conclusions

A variety of ACP support tools are available on the web, varying in the quality of content. In the future, users should be involved in the development process of ACP support tools, and the content should be substantiated by scientific evidence.

Trial Registration

PROSPERO CRD42020184112; https://tinyurl.com/mruf8b43

Research to Clinical Practice-Youth seeking mental health information online and its impact on the first steps in the patient journey

BACKGROUND

Online searches about anxiety and depression are recorded every 3-5 s. As such, information and communication technologies (ICT) have enormous potential to enable or impair help-seeking and patient-professional interactions. Youth studies indicate that ICT searches are undertaken before initial mental health consultations, but no publications have considered how this online activity affects the first steps of the patient journey in youth mental health settings.

METHODS

State-of-the-art review using an iterative, evidence mapping approach to identify key literature and expert consensus to synthesize and prioritise clinical and research issues.

RESULTS

Adolescents and young adults are more likely to seek health advice via online search engines or social media platforms than from a health professional. Young people not only search user-generated content and social media to obtain advice and support from online communities but increasingly contribute personal information online.

CONCLUSIONS

A major clinical challenge is to raise professional awareness of the likely impact of this activity on mental health consultations. Potential strategies range from modifying the structure of clinical consultations to ensure young people are able to disclose ICT activities related to mental health, through to the development and implementation of 'internet prescriptions' and a youth-focused 'toolkit'.

Quality and accuracy of patient-oriented Web-based information regarding tooth avulsion

BACKGROUND/AIM

Tooth avulsion (TA) is a severe form of traumatic dental injury. The tooth's prognosis depends on the immediate measures taken. First responders are often laypeople, and the Internet is a favored platform to access health-related information. The aim of this descriptive study was to evaluate the accuracy, quality, readability, and popularity of patient-oriented web information regarding the emergency management of TA.

METHODS

Three search engines (Google, Bing, and Yahoo.com) were browsed with six VPNs using "knocked out tooth," "fallen out tooth," and "tooth avulsion" as keywords. The top 20 results for each were evaluated. Webpages were included if they had information oriented to caregivers. They were analyzed for accuracy (cutoff ≥95%). Quality of web pages was assessed using DISCERN and the Journal of the American Medical Association (JAMA) benchmark. Readability was evaluated using Flesch-Kincaid Grade Level (FKGL), Gunning Fog (FOG), Coleman-Liau Index (CLI), and Simple Measure of Gobbledygook (SMOG). Popularity was assessed by Alexa Popularity Rank (APR). Accurate and inaccurate webpages were compared using chi-square analysis and the Mann-Whitney U test. Spearman's correlation between the studied metrics was established. A p < .05 was considered significant.

RESULTS

Seventy-two webpages were included, of which 23 (32%) were accurate. The median DISCERN ratings for accurate and inaccurate webpages were 3.0 [2.5-3.0] and 3.0 [2.0-3.0], respectively (p = .331). Ten (44%) accurate and 12 (24.5%) inaccurate webpages had high-quality JAMA (p = .089). The median readability scores for accurate webpages were FKGL (8 [6.75-9.2]), FOG (10.6 [9.55-12.3]), CLI (7 [6-7]), SMOG (6.7 [6.15-7.5]), while scores for the inaccurate webpages were FKGL (7.85 [6.275-8.525]), FOG (10.55 [8.875-11.425]), CLI (7 [7-8.25]), and SMOG (7 [6.2-7.925]), with a higher CLI score for inaccurate webpages (p = .0035). The median APRs for accurate and inaccurate webpages were 1,02,538 [26,852-14,43,755] and 7,63,190 [2,19,799-29,92,067], respectively (p = .163). Accuracy had a positive correlation with quality (DISCERN: rho = .293, p = .013; JAMA: rho = .249, p = .036), while popularity (APR) was negatively correlated with quality (DISCERN: rho = -.330, p = .013; JAMA: rho = -.287, p = .032).

CONCLUSION

Relevant webpages were limited and had low accuracy. This can impact the emergency management by laypeople and adversely affect the prognosis following avulsion.

Assessing COVID-19 Health Information on Google Using the Quality Evaluation Scoring Tool (QUEST): Cross-sectional and Readability Analysis

BACKGROUND

The COVID-19 pandemic spurred an increase in online information regarding disease spread and symptomatology.

OBJECTIVE

Our purpose is to systematically assess the quality and readability of articles resulting from frequently Google-searched COVID-19 terms in the United States.

METHODS

We used Google Trends to determine the 25 most commonly searched health-related phrases between February 29 and April 30, 2020. The first 30 search results for each term were collected, and articles were analyzed using the Quality Evaluation Scoring Tool (QUEST). Three raters scored each article in authorship, attribution, conflict of interest, currency, complementarity, and tone. A readability analysis was conducted.

RESULTS

Exactly 709 articles were screened, and 195 fulfilled inclusion criteria. The mean article score was 18.4 (SD 2.6) of 28, with 7% (14/189) scoring in the top quartile. National news outlets published the largest share (70/189, 36%) of articles. Peer-reviewed journals attained the highest average QUEST score compared to national/regional news outlets, national/state government sites, and global health organizations (all P<.05). The average reading level was 11.7 (SD 1.9, range 5.4-16.9). Only 3 (1.6%) articles were written at the recommended sixth grade level.

CONCLUSIONS

COVID-19-related articles are vastly varied in their attributes and levels of bias, and would benefit from revisions for increased readability.

The quality and readability of online molar incisor hypomineralisation patient education materials: a systematic analysis

Background

Molar incisor hypomineralization (MIH) is estimated to affect 14% of children worldwide. It is crucial that patients and their families have access to easily comprehensible and reliable MIH‐relevant information. This study aims to determine the quality, reliability and readability of online patient education materials about MIH.

Methods

A systematic search strategy was adopted. Five validated tools were used to assess the content of the 21 websites that satisfied inclusion/exclusion criteria. Data analyses were applied via GraphPad Prism software version 9 (GraphPad Software, San Diego, CA, USA).

Results

Five (23.8%) websites only satisfied the criteria for understandability and two (9.5%) websites satisfied the criteria for actionability using the Patient Education Materials Assessment Tool (PEMAT). No website contained the Health on the Net (HON)Code Seal and the mean (SD) Journal of theAmerican Medical Association number of benchmarks per website was 1.33/4 (1.02). All websites failed to reach recommended minimum readability levels. Higher PEMAT scores were associated with ‘easier’ readability.

Conclusions

Online patient education materials related to MIH are lacking in quality and reliability, and are too difficult for most to read easily. The authors of MIH‐related online content should consider reference to quality of information tools when developing patient education materials.

Online health information-seeking behaviour of patients attending a primary care clinic in Malaysia: a cross-sectional study

BACKGROUND

The internet has become a common source of health information; however, little is known about online health information-seeking behaviour (HISB) among patients in low- and middle-income countries (LMICs).

OBJECTIVES

This study aimed to determine the prevalence of online health information-seeking and its associated factors among patients in primary care in Malaysia. We also examined the reasons for, and the sources of, online health information-seeking, patients' level of trust in the information found and what the information was used for.

METHODS

A cross-sectional study using a self-administered questionnaire was conducted on patients who attended a primary care clinic. The questionnaire included the use of the internet to seek health information, sources and types of health information, eHealth literacy, patients' trust in online information, and how patients appraise and use online health information.

RESULTS

Out of 381 patients in this study, 54.7% (n = 208) used the internet to search for health information. Patients mainly sought information via Google (96.2%) and the most common websites that they visited were Wikipedia (45.2%) and MyHEALTH (37.5%). Higher levels of education, longer duration of internet use, and higher eHealth literacy were significantly associated with online HISB. Patients' trust in websites (45.6%) and social media (20.7%) was low when compared to trust in healthcare professionals (87.9%). Only 12.9% (n = 22) of patients had discussed online health information with their doctors.

CONCLUSION

Online HISB was common among primary care patients; however, their eHealth literacy was low, with suboptimal appraisal skills to evaluate the accuracy of online health information.

Association between online health information-seeking and medication adherence: A systematic review and meta-analysis

Background

The evidence of the impact of online health information-seeking (OHIS) on health outcomes has been conflicting. OHIS is increasingly recognised as a factor influencing health behaviour but the impact of OHIS on medication adherence remains unclear.

Objectives

We conducted a systematic review and meta-analysis to examine the associations between OHIS and medication adherence.

Methods

We searched Medline, Embase, Web of Science, Scopus, CINAHL and Psychology and Behavioural Science Collection for studies published up to December 2020. The inclusion criteria were studies that reported the associations of OHIS and medication adherence, quantitative design, reported primary data only, related to any health condition where medications are used and conducted on patients either in clinical or community settings. A meta-analysis was used to examine the association between OHIS and medication adherence.

Results

A total of 17 studies involving 24,890 patients were included in this review. The study designs and results were mixed. In the meta-analysis, there was no significant association (n = 7, OR 1.356, 95% CI 0.793-2.322, p = 0.265), or correlation (n = 4, r = -0.085, 95% CI −0.572-0.446, p = 0.768) between OHIS and medication adherence. In the sub-group analysis of people living with HIV/AIDS, OHIS was associated with better medication adherence (OR 1.612, 95% CI 1.266-2.054, p < 0.001).

Conclusions

The current evidence of an association between OHIS and medication adherence is inconclusive. This review highlights methodological issues on how to measure OHIS objectively and calls for in-depth exploration of how OHIS affects health decisions and behaviour.

Quality Assessment of Online Complementary and Alternative Medicine Information Resources Relevant to Cancer

Background:

Complementary and alternative medicine (CAM) is often used by cancer patients and survivors in the US. Many people turn to the internet as their first source of information. Health information seeking through the internet can be useful for patients to gain a better understanding of specific CAM treatments to discuss with their healthcare team, but only if the information is comprehensive, high quality, and reliable. The aim of this article is to examine the content, writing/vetting processes, and visibility of cancer CAM online informational resources.

Methods:

Online CAM resources were identified by Google and PubMed searches, literature reviews, and through sources listed on various websites. The websites were analyzed through a modified online health information evaluation tool, DISCERN (score range = 1-5). The website’s features relevant to the quality assessment were described.

Results:

Eleven CAM websites were chosen for analysis. The DISCERN analysis showed a range of quality scores from 3.6 to 4.9. Lower DISCERN scores were generally due to deficiencies in describing the writing, editing, and updating processes. A lack of transparency with authorship and references was commonly present.

Conclusion:

Cancer patients interested in CAM need unbiased, evidence-based, reliable, high-quality, easily accessible educational materials. Individuals should use the guidelines followed in this analysis (including DISCERN and Medline Plus) to find reliable sources. Website developers can use CAM Cancer (NAFKAM), Beyond Conventional Cancer Therapies, Memorial Sloan Kettering Cancer Center, breastcancer.org, Office of Dietary Supplements, National Center for Complementary and Integrative Health, and Cancer.gov as models for trustworthy content.

The use of social media platforms to discuss and educate the public on allergic contact dermatitis

BACKGROUND

Social media platforms are increasingly used by patients to research and discuss medical problems.

OBJECTIVE

The aim of this study was to identify by whom, how frequently, and how allergic contact dermatitis (ACD) is discussed on social media sites.

METHODS

Search terms allergic contact dermatitis and contact dermatitis were queried across Twitter, Instagram, Reddit, Facebook, YouTube, and Google search metrics. The frequency, content, and creators of content were assessed.

RESULTS

ACD content was identified on all platforms and was made by patients, physicians, professional organizations, and companies. When comparing the volume of posts, more content was on Instagram than Twitter, particularly among patients. Patients support groups were identified on Facebook but not on Reddit. A formal analysis of YouTube videos found that the medical information presented in these videos was often of poor quality.

CONCLUSIONS

Patch testing physicians should be aware information on ACD exist across social media sites. While some content is generated by physicians, patients and industry groups also post and share material. Patch testing physicians should be aware that there is an opportunity to share ACD information, but they should also be aware our patients are posting and creating online support communities independent of us. This article is protected by copyright. All rights reserved.

Medication-related osteonecrosis of the jaw: A cross-sectional survey assessing the quality of information on the Internet

OBJECTIVE

The objective of this study was to evaluate the quality of information and readability of online content regarding medication-related osteonecrosis of the jaw (MRONJ).

STUDY DESIGN

Three search engines were used to identify websites containing information regarding MRONJ. The content was assessed with 5 quality of information and readability tools. Statistical analyses were performed via GraphPad Software (GraphPad Software Inc., La Jolla, Calif, United States).

RESULTS

The Health on the Net (HONcode) seal was present in 2 of the 21 websites that satisfied inclusion/exclusion criteria. The mean JAMA (Journal of the American Medical Association) benchmark satisfied per website was 1.619 out of 4 (standard deviation [SD] = 1.2; 95% confidence interval [CI], 1.07-2.17). The mean overall Quality Evaluation Scoring Tool score was 15.29 out of a maximum 28 (SD = 6.174; 95% CI, 12.48-18.1). The Quality Evaluation Scoring Tool scores of those websites that satisfied ≥3 JAMA criteria were significantly higher than those that did not (P = .01; 95% CI, -11.41 to -1.79). The readability levels of all content ranged from fairly difficult to very difficult.

CONCLUSIONS

The information related to MRONJ on the Internet is unreliable and too difficult for the general public to read. Authors of online MRONJ information should consider use of quality of information and readability tools to ensure that reliable information is understood by those with poor health literacy skills.

A Health Information Quality Assessment Tool for Korean Online Newspaper Articles: Development Study

BACKGROUND

Concern regarding the reliability and accuracy of the health-related information provided by online newspaper articles has increased. Numerous criteria and items have been proposed and published regarding the quality assessment of online information, but there is no standard quality assessment tool available for online newspapers.

OBJECTIVE

This study aimed to develop the Health Information Quality Assessment Tool (HIQUAL) for online newspaper articles.

METHODS

We reviewed previous health information quality assessment tools and related studies and accordingly developed and customized new criteria. The interrater agreement for the new assessment tool was assessed for 3 newspaper articles on different subjects (colorectal cancer, obesity genetic testing, and hypertension diagnostic criteria) using the Fleiss κ and Gwet agreement coefficient. To compare the quality scores generated by each pair of tools, convergent validity was measured using the Kendall τ ranked correlation.

RESULTS

Overall, the HIQUAL for newspaper articles comprised 10 items across 5 domains: reliability, usefulness, understandability, sufficiency, and transparency. The interrater agreement for the article on colorectal cancer was in the moderate to substantial range (Fleiss κ=0.48, SE 0.11; Gwet agreement coefficient=0.74, SE 0.13), while for the article introducing obesity genetic testing it was in the substantial range, with values of 0.63 (SE 0.28) and 0.86 (SE 0.10) for the two measures, respectively. There was relatively low agreement for the article on hypertension diagnostic criteria at 0.20 (SE 0.10) and 0.75 (SE 0.13), respectively. Validity of the correlation assessed with the Kendall τ showed good correlation between tools (HIQUAL vs DISCERN=0.72, HIQUAL vs QUEST [Quality Evaluation Scoring Tool]=0.69).

CONCLUSIONS

We developed a new assessment tool to evaluate the quality of health information in online newspaper articles, to help consumers discern accurate sources of health information. The HIQUAL can help increase the accuracy and quality of online health information in Korea.

Evaluating Local Multilingual Health Care Information Environments on the Internet: A Pilot Study

For foreign-born populations, difficulty in finding health care information in their primary language is a structural barrier to accessing timely health care. While such information may be available at a national level, it may not always be relevant or appropriate to the living situations of these people. Our objective was to explore the quality of online multilingual health information environments by pilot-testing a framework for assessing such information at the prefectural level in Japan. The framework consisted of five health care domains (health system, hospitals, emergency services, medical interpreters, and health insurance). Framework scores varied considerably among prefectures; many resources were machine-translated. These scores were significantly associated with foreign population proportion and the number of hospitals in each prefecture. Our multilingual health care information environment (MHCIE) framework provides a measure of health access inclusivity, which has not been quantified before. It is adaptable to other international contexts, but further validation is required.

Quality of web-based information at the beginning of a global pandemic: a cross-sectional infodemiology study investigating preventive measures and self care methods of the coronavirus disease 2019

BACKGROUND

reducing the spread and impact epidemics and pandemics requires that members of the general population change their behaviors according to the recommendations, restrictions and laws provided by leading authorities. When a new epidemic or pandemic emerges, people are faced with the challenge of sorting through a great volume of varied information. Therefore, the dissemination of high-quality web-based information is essential during this time period. The overarching aim was to investigate the quality of web-based information about preventive measures and self care methods at the beginning of the COVID-19 pandemic.

METHODS

in May 2020, consumer-oriented websites written in Swedish were identified via systematic searches in Google (n = 76). Websites were assessed with inductive content analysis, the JAMA benchmarks, the QUEST tool and the DISCERN instrument.

RESULTS

seven categories and 33 subcategories were identified concerning preventive measures (md = 6.0 subcategories), with few specifying a method for washing hands (n = 4), when to sanitize the hands (n = 4), and a method for sanitizing the hands (n = 1). Eight categories and 30 subcategories were identified concerning self care methods (md = 3.0 subcategories), with few referring to the national number for telephone-based counseling (n = 20) and an online symptom assessment tool (n = 16). Overall, the median total quality scores were low (JAMA = 0/4, QUEST =13/28, DISCERN = 29/80).

CONCLUSIONS

at the beginning of the pandemic, substantial quality deficits of websites about COVID-19 may have counteracted the public recommendations for preventive measures. This illustrates a critical need for standardized and systematic routines on how to achieve dissemination of high-quality web-based information when new epidemics and pandemics emerge.

Awareness of diagnosis, treatment and risk of late effects in Chinese survivors of childhood cancer in Hong Kong

BACKGROUND

For survivors of childhood cancer, awareness of personal health risks is a critical component of long-term health management.

OBJECTIVE

To evaluate the awareness of the diagnosis, treatment and risk of late effects among survivors of childhood cancer in Hong Kong.

METHODS

Between June 2019 and March 2020, this cross-sectional study recruited 155 adult survivors (mean age = 26.9, standard deviation [SD] = 6.4 years) and 45 parents of paediatric survivors (mean age = 11.1, SD = 3.6 years) from a long-term follow-up clinic. At >10 years post-treatment (mean = 13.4, SD = 7.6 years), they completed a structured questionnaire to report their cancer-specific knowledge. Multiple linear regression analysis was conducted to identify clinical, socioeconomic and behavioural factors associated with poor awareness.

RESULTS

The majority of participants accurately recalled their diagnoses (73.5%) and major treatment modalities (chemotherapy 92.4%, radiation 82.9% and surgery 88.2%). However, less than half (45%) of the participants recognized more than 25% of the total late effects for which they were at risk. The highest levels of awareness were reported for endocrine problems (49%), neurocognitive impairment (44%) and secondary cancers (43%), and the lowest for peripheral neuropathy (21%) and vision problems (23%). Compared with survivors of haematological malignancies, those of central nervous system (CNS) tumours (standardized estimate [B] = -9.33, 95% confidence interval [95% CI]: -13.41 to -5.26) and non-CNS solid tumours (B = -8.47, 95% CI: -12.39 to -4.94) had less knowledge about their diagnosis. Retaining medical records (P < .0001) and better medical information-seeking habits (P = .048) were associated with better awareness.

CONCLUSIONS

Survivors of childhood cancer in Hong Kong have deficient awareness of their personal health risks. They may benefit from the provision of a survivorship care plan and personalized education regarding treatment-related late effects.

PATIENT CONTRIBUTION

Patients contributed in designing the study tools. Results were presented at a non-governmental organization.

Benefits of Structured Engagement with Target Audiences of a Health Website: Study Design for a Multi-Case Study

Access to evidence and practice knowledge precedes use, but availability does not guarantee reach and uptake by intended audiences. The CareSearch project provides online palliative care evidence and information to support health and aged care professionals as well as patients, carers and families to make informed decisions about care at the end of life. Already established in the palliative care sector, CareSearch commenced planning to extend its reach, and ensure website use is maximised for different audiences. This paper reports on the development of the Engagement Framework which will be used to guide and deliver an Engagement Project which will actively seek feedback and insights from intended users in a structured process. The process for developing the Engagement Framework commenced with a literature review of approaches used in knowledge translation, implementation science, and social marketing. The Engagement Framework comprising eight steps was then developed. The Engagement Framework outlines the series of tasks to be undertaken by team members when working with three target groups (Aged Care; Allied Health; and Patients, Carers and Families). A process/formative evaluation collecting data using qualitative methods is also described for use in the subsequent Engagement Project. The evaluation will explore the experiences of project participants as well as staff implementing the engagement activities. The three target groups will enable a cross-case comparison of the strengths and weaknesses of the approach. Planning, implementing and evaluating engagement with intended audiences, offers one mechanism to identify ways to increase interaction and integration with knowledge users.

Robotic-Assisted Total Knee Arthroplasty: An Assessment of Content, Quality, and Readability of Available Internet Resources

BACKGROUND

The use of robotic-assisted total knee arthroplasty (TKA) has significantly increased over the past decade. Internet content is largely unregulated and may contain inaccurate and/or misleading information about robotic TKA. Our goal was to assess the content, quality, and readability of online material regarding robotic-assisted TKA.

METHODS

We conducted an internet search for the top 50 web sites from each of the 3 most popular search engines (Google, Yahoo, and Bing) using the search term robotic total knee replacement. Each web site was assessed for content, quality, and readability. Web site quality was assessed utilizing the QUality Evaluation Scoring Tool (QUEST). Readability was assessed utilizing the Simple Measure of Gobbledygook, Flesch-Kincaid Grade Level, and Flesch Reading Ease Formula scores.

RESULTS

General risks of TKA were discussed in 47.2%, while benefits were discussed in 98.6% of all web sites. Inaccurate claims occurred at a significantly higher rate in physician/community hospital sources compared to university/academic web sites (59% vs 28%, P = .045). Web sites from university/academic web sites had the highest QUEST scores, while physician/community hospital sources scored the lowest (16.1 vs 10.6, P = .01). Most web sites were written at a college reading level or higher.

CONCLUSION

Patients should be counseled on the largely unregulated nature of online information regarding robotic-assisted TKA. Physicians and hospitals should consider revising the readability of their online information to a more appropriate level in order to provide accurate, evidence-based information to allow the patient to make an informed consent decision.

Automatically Assessing Quality of Online Health Articles

Today Information in the world wide web is overwhelmed by unprecedented quantity of data on versatile topics with varied quality. However, the quality of information disseminated in the field of medicine has been questioned as the negative health consequences of health misinformation can be life-threatening. There is currently no generic automated tool for evaluating the quality of online health information spanned over broad range. To address this gap, in this paper, we applied data mining approach to automatically assess the quality of online health articles based on 10 quality criteria. We have prepared a labelled dataset with 53012 features and applied different feature selection methods to identify the best feature subset with which our trained classifier achieved an accuracy of [Formula: see text] varied over 10 criteria. Our semantic analysis of features shows the underpinning associations between the selected features & assessment criteria and further rationalize our assessment approach. Our findings will help in identifying high quality health articles and thus aiding users in shaping their opinion to make right choice while picking health related help from online.