The role of quality tools in assessing reliability of the internet for health information

Assessing parental comprehension of online resources on childhood pain

We aimed to examine the patient education materials (PEMs) on the internet about "Child Pain" in terms of readability, reliability, quality and content. For our observational study, a search was made on February 28, 2024, using the keywords "Child Pain," "Pediatric Pain," and "Children Pain" in the Google search engine. The readability of PEMs was assessed using computer-based readability formulas (Flesch Reading Ease Score [FRES], Flesch-Kincaid Grade Level [FKGL], Automated readability index (ARI), Gunning Fog [GFOG], Coleman-Liau score [CL], Linsear Write [LW], Simple Measure of Gobbledygook [SMOG]). The reliability and quality of websites were determined using the Journal of American Medical Association (JAMA) score, Global Quality Score (GQS), and DISCERN score. 96 PEM websites included in our study. We determined that the FRES was 64 (32-84), the FKGL was 8.24 (4.01-15.19), ARI was 8.95 (4.67-17.38), GFOG was 11 (7.1-19.2), CL was 10.1 (6.95-15.64), LW was 8.08 (3.94-19.0) and SMOG was 8.1 (4.98-13.93). The scores of readability formulas showed that, the readability level of PEMs was statistically higher than sixth-grade level with all formulas (P = .011 for FRES, P < .001 for GFOG, P < .001 for ARI, P < .001 for FKGL, P < .001 for CL and P < .001 for SMOG), except LW formula (P = .112). The websites had moderate-to-low reliability and quality. Health-related websites had the highest quality with JAMA score. We found a weak negative correlation between Blexb score and JAMA score (P = .013). Compared to the sixth-grade level recommended by the American Medical Association and the National Institutes of Health, the readability grade level of child pain-related internet-based PEMs is quite high. On the other hand, the reliability and quality of PEMs were determined as moderate-to-low. The low readability and quality of PEMs could cause an anxious parent and unnecessary hospital admissions. PEMs on issues threatening public health should be prepared with attention to the recommendations on readability.

Assessing Credibility: Quality Criteria for Patients, Caregivers, and the Public in Online Health Information-A Qualitative Study

The increasing reliance on the Internet for health information has raised concerns about patients using unreliable and potentially harmful content. This study aimed to establish quality criteria to assist patients, caregivers, and the public in evaluating the reliability of online health information. We conducted focus group workshops with 25 participants recruited across Canada, proficient in either English or French. The participants included 13 females and 12 males, with the majority having a college or higher level of education. Through an in-depth analysis comparing various aspects, the participants determined 6 quality criteria: authorship, reliability, usefulness, accessibility, readability, and privacy & confidentiality. The findings from this study present a comprehensive list of quality criteria that will contribute to developing evidence-based quality benchmarks and policy frameworks in multiple languages. These criteria are not only valid but also well-suited to the diverse needs and preferences of patients and the public, providing a reliable framework for evaluating online health information through an evidence-based approach.

Knowledge and Awareness of Stroke among the Elderly Population: Analysis of Data from a Sample of Older Adults in a Developing Country

Background and Objectives: Stroke prevention has traditionally concentrated on research to improve knowledge and awareness of the disease in the general population. Since stroke incidents increase with age, there is a need to focus on the elderly, a high-risk group for developing the disease. This study aimed to examine the level of stroke awareness and knowledge, their predictors, and their source of information. Materials and Methods: A prospective cross-sectional study targeted Lebanese senior citizens aged 65 years and above. A total of 513 participants enrolled in the study through a self-administered survey distributed using a snowball sampling technique. Results: Most participants had appropriate baseline knowledge (more than 75% correct answers) of stroke, including risk factors, alarming signs, and preventive measures. Better knowledge of disease risks was significantly associated with having a university degree (ORa = 1.609; p = 0.029). Participants who had previous ischemic attacks showed significantly lower knowledge of the alarming signs (ORa = 0.467; p = 0.036) and prevention measures (ORa = 0.427; p = 0.029). Those suffering from depression had better knowledge of stroke alarming signs (ORa = 2.060.; p = 0.050). Seeking information from pharmacists, physicians, or the internet was not significantly associated with better knowledge of stroke risks, alarming signs, and preventive measures. Conclusions: The present study showed that seniors had fair knowledge of stroke, despite gaps in stroke prevention measures. Healthcare providers could play a leading role in improving public health by educating seniors to enhance awareness about prevention measures, detecting alarming signs, and acting fast to save a life.

Google searches for stress urinary incontinence treatment options: a worldwide trend analysis in the synthetic mesh controversy era

PURPOSE

To analyze trends in worldwide public interest in the treatment options for stress urinary incontinence around the latest FDA 2019 ban on vaginal mesh for prolapse.

METHODS

We used the web-based tool Google Trends to analyse online searches related to the following terms: 'pelvic floor muscle exercises', 'continence pessary', 'pubovaginal slings', 'Burch colposuspension', 'midurethral slings', 'injectable bulking agents'. Data were expressed as relative search volume on a scale of 0-100. Comparisons of annual relative search volume and average annual percentage change, were analyzed to assess loss or gain of interest. Finally, we assessed the impact of the last FDA alert.

RESULTS

The mean annual relative search volume for midurethral slings was 20% in 2006 and significantly decreased until 8% in 2022 (p < 0.01). A regular decrease interest was recorded for autologous surgeries but a regain of interest for pubovaginal slings was registered since 2020 (+ 2.8%; p < 0.01). Conversely, a steep interest was noted for injectable bulking agents (average annual percentage change: + 4.4%; p < 0.01) and conservative therapies (p < 0.01). When trends were compared before and after the FDA 2019 alert, a lower research volume was found for midurethral slings, while a higher research volume was observed all others treatments (all p < 0.05).

CONCLUSION

The online public researches about midurethral slings have considerably decreased following warnings on the use of transvaginal mesh. There seems to be a growing interest in conservative measures, bulking agents, and recently pubovaginal slings.

Evaluating the readability, quality and reliability of online patient education materials on transcutaneuous electrical nerve stimulation (TENS)

Increasing digitization also raises concerns regarding the reliability and comprehensibility of online health information. In this study, we aimed to examine the readability, reliability, and quality of internet-based patient education materials on "transcutaneous electrical nerve stimulation." On September 15, 2022, we used Google search engine to search the keyword "Transcutaneous Electrical Nerve Stimulation" and obtained information from 200 websites. The readability of the websites was evaluated using the Flesch Reading Ease Score (FRES), Flesch-Kincaid Grade Level, Simple Measure of Gobbledygook, and Gunning Fog. The Journal of American Medical Association score and Health on the Net Foundation code of conduct were used to determine the reliability of the websites, whereas the DISCERN score and Global Quality Score were used to evaluate the quality of the websites. In the readability analysis of 102 websites that met the inclusion criteria of this study, we found that the Flesch Reading Ease Score was 47.91 ± 13.79 (difficult), average Flesch-Kincaid Grade Level and Simple Measure of Gobbledygook were 11.20 ± 2.85 and 10.53 ± 2.11 years, respectively, and average Gunning Fog score was 14.04 ± 2.74 (very difficult). Commercial websites constituted the highest proportion of websites (n = 36, 35.5%). Overall, 16.7% of the websites were found to be of high quality according to the Global Quality Score, 16 (15.7%) websites had Health on the Net Foundation code of conduct certification, and 8.8% of the websites were found to be highly reliable according to the Journal of American Medical Association scores. There was a statistically significant difference between website typologies and quality and reliability scores (P < .001). Compared with the sixth-grade level recommended by the American Medical Association and the National Institute of Health, the readability of transcutaneous electrical nerve stimulation-related internet-based patient education materials was considerably high, but they showed low reliability and moderate-to-poor quality. Thus, the quality, reliability, and readability of websites developed by health professionals play a major role in conveying accurate and easily understandable information.

Web-Based Health Information–Seeking Methods and Time Since Provider Engagement: Cross-sectional Study

Background

The use of web-based methods to seek health information is increasing in popularity. As web-based health information (WHI)–seeking affects health-related decision support and chronic symptom self-management, WHI-seeking from online sources may impact health care decisions and outcomes, including care-seeking decisions. Patients who are routinely connected to physicians are more likely to receive better and more consistent care. Little is known about whether WHI-seeking impacts the frequency at which patients engage with health care providers.

Objective

Our primary objective was to describe the associations between the use of web-based methods to seek information about one’s own health and the time since last engaging with a health care provider about one’s own health. Additionally, we aimed to assess participants’ trust in health care organizations to contextualize our findings.

Methods

We analyzed data from US adults participating in the nationally representative Tufts Equity in Health, Wealth, and Civic Engagement Survey (N=1034). Bivariate associations between demographic characteristics and health information–seeking methods were assessed with Pearson chi-squared tests. Bivariate associations of Medical Mistrust Index (MMI) scores with each health information–seeking method and time since provider engagement were assessed with F tests and adjusted Wald tests. We fit a multivariable logistic regression model to assess the association between WHI-seeking within the 12 months prior to survey (alone or in combination with provider-based methods versus provider only) and engagement with a provider more than 1 year prior to the time of survey, adjusting for age, race and ethnicity, sex, education, insurance coverage, and MMI.

Results

Age, race and ethnicity, educational attainment, health insurance source, MMI, and time since provider engagement were each significantly associated with the health information–seeking method in bivariate analyses. Compared to using only provider-based health information seeking methods, WHI-based methods alone or in combination with provider-based methods were associated with a 51% lower likelihood (odds ratio 0.49, 95% CI 0.27-0.87) of engaging with a provider within the previous year. Participants who used WHI-seeking methods alone and those who had not engaged with a health care provider within the previous year demonstrated a higher mean MMI score; however, MMI was not a significant predictor of time since engagement with a provider in the multivariable analysis.

Conclusions

Our findings from a nationally representative survey suggest that for those who use WHI-seeking methods (alone or in combination with provider-based information-seeking methods), there is a statistically significant lower likelihood of engaging with a provider in a year compared to those who only use provider-based methods. Future research should consider the intent of a person’s visit with a provider, trust in health care systems, methods of provider engagement, and specific web-based platforms for health information.

Age and neurocognition are associated with credibility evaluations of health websites

The internet has become a common means by which many older adults seek out health information. The prevalence of misinformation on the internet makes the search for accurate online health information a more complex and evaluative process. This study examined the role of age and neurocognition in credibility evaluations of credible and non-credible health websites. Forty-one older adults and fifty younger adults completed a structured credibility rating task in which they evaluated a series of webpages displaying health information about migraine treatments. Participants also completed measures of neurocognition, internet use, and health literacy. Results suggested that older adults rated non-credible health websites as more credible than younger adults, but the age groups did not differ in their ratings of credible sites. Within the full sample, neurocognition was associated with credibility ratings for non-credible health websites, whereas health literacy was related to the ratings of credible sites. Findings indicate that older adults may be more likely to trust non-credible health websites than younger adults, which may be related to differences in higher-order neurocognitive functions. Future work might examine whether cognitive-based supports for credibility training in older adults can be used to improve the accuracy with which they evaluate online health information.

Quantifying the Quality of Web-Based Health Information on Student Health Center Websites Using a Software Tool: Design and Development Study

Background

The internet has become a major source of health information, especially for adolescents and young adults. Unfortunately, inaccurate, incomplete, or outdated health information is widespread on the web. Often adolescents and young adults turn to authoritative websites such as the student health center (SHC) website of the university they attend to obtain reliable health information. Although most on-campus SHC clinics comply with the American College Health Association standards, their websites are not subject to any standards or code of conduct. In the absence of quality standards or guidelines, monitoring and compliance processes do not exist for SHC websites. Thus, there is no oversight of the health information published on SHC websites by any central governing body.

Objective

The aim of this study is to develop, describe, and validate an open-source software that can effectively and efficiently assess the quality of health information on SHC websites in the United States.

Methods

Our cross-functional team designed and developed an open-source software, QMOHI (Quantitative Measures of Online Health Information), that assesses information quality for a specified health topic from all SHC websites belonging to a predetermined list of universities. The tool was designed to compute 8 different quality metrics that quantify various aspects of information quality based on the retrieved text. We conducted and reported results from 3 experiments that assessed the QMOHI tool in terms of its scalability, generalizability in health topics, and robustness to changes in universities’ website structure.

Results

Empirical evaluation has shown the QMOHI tool to be highly scalable and substantially more efficient than manually assessing web-based information quality. The tool’s runtime was dominated by network-related tasks (98%), whereas the metric computations take <2 seconds. QMOHI demonstrated topical versatility, evaluating SHC website information quality for four disparate and broad health topics (COVID, cancer, long-acting reversible contraceptives, and condoms) and two narrowly focused topics (hormonal intrauterine device and copper intrauterine device). The tool exhibited robustness, correctly measuring information quality despite changes in SHC website structure. QMOHI can support longitudinal studies by being robust to such website changes.

Conclusions

QMOHI allows public health researchers and practitioners to conduct large-scale studies of SHC websites that were previously too time- and cost-intensive. The capability to generalize broadly or focus narrowly allows a wide range of applications of QMOHI, allowing researchers to study both mainstream and underexplored health topics. QMOHI’s ability to robustly analyze SHC websites periodically promotes longitudinal investigations and allows QMOHI to be used as a monitoring tool. QMOHI serves as a launching pad for our future work that aims to develop a broadly applicable public health tool for web-based health information studies with potential applications far beyond SHC websites.

Developing a Quality Benchmark for Determining the Credibility of Web Health Information- a Protocol of a Gold Standard Approach

Background: The current pandemic of COVID-19 has changed the way health information is distributed through online platforms. These platforms have played a significant role in informing patients and the public with knowledge that has changed the virtual world forever. Simultaneously, there are growing concerns that much of the information is not credible, impacting patient health outcomes, causing human lives, and tremendous resource waste. With the increasing use of online platforms, patients/the public require new learning models and sharing medical knowledge. They need to be empowered with strategies to navigate disinformation on online platforms.

Methods and Design: To meet the urgent need to combat health “misinformation,” the research team proposes a structured approach to develop a quality benchmark, an evidence-based tool that identifies and addresses the determinants of online health information reliability. The specific methods to develop the intervention are the following: (1) systematic reviews: two comprehensive systematic reviews to understand the current state of the quality of online health information and to identify research gaps, (2) content analysis: develop a conceptual framework based on established and complementary knowledge translation approaches for analyzing the existing quality assessment tools and draft a unique set of quality of domains, (3) focus groups: multiple focus groups with diverse patients/the public and health information providers to test the acceptability and usability of the quality domains, (4) development and evaluation: a unique set of determinants of reliability will be finalized along with a preferred scoring classification. These items will be used to develop and validate a quality benchmark to assess the quality of online health information.

Expected Outcomes: This multi-phase project informed by theory will lead to new knowledge that is intended to inform the development of a patient-friendly quality benchmark. This benchmark will inform best practices and policies in disseminating reliable web health information, thus reducing disparities in access to health knowledge and combat misinformation online. In addition, we envision the final product can be used as a gold standard for developing similar interventions for specific groups of patients or populations.

Organ Donation on the Internet: Quality and Readability in English

BACKGROUND

Many websites do not appear to be sources from which the public will learn reliable information about organ donation (OD). We visited websites related to OD and analyzed the quality of their content and the readability of the texts in English.

METHODS

OD websites were identified using the search terms "organ donor" or "organ donation" on Google. A relevant websites list was compiled consisting of the 100 top-ranking ".gov" websites, the top 100 ".edu" websites, and the 100 top-ranking ".org" websites concerning their domain suffixes. We generated a scoring system to identify the quality of information about OD. Flesch-Kincaid grade formula, FOG index, Flesch Reading ease test, and a Fry graph test were used to assess the readability grade.

RESULTS

Of 300 websites, 50 were eligible for evaluation. Only 3 (27.3%) of the relevant 11 ".gov" websites were of high quality. Seven (43.8%) of 16 ".edu" websites and only 9 (39.1%) of 23 ".org" websites were deemed as being high quality. None of these websites had fairly easy, easy, or very easy readability levels. The median readability score was 11.5 (interquartile range, 10.25-13.50) grade level. Quality scores and readability grades were not different among the website sources containing ".edu", ".gov", and ".org" (P = .795, P = .218, respectively).

CONCLUSION

In the present study, the most important finding was that the content of websites related to OD far exceeds current readability grade recommendations, and they do not have a satisfactory quality.

Do Brief Educational Sessions Increase Electronic Health Literacy of Low-Income Persons Living With HIV/AIDS?

This research explored whether participating in a brief educational intervention using the National Library of Medicine video, Evaluating Health Information: A Tutorial From the National Library of Medicine, would increase electronic health literacy. A quasi-experimental longitudinal design was used in two randomly selected settings of a treatment program for low-income persons living with HIV/acquired immune deficiency syndrome (N = 100). Individuals in both intervention groups watched the video and completed an at-home assignment brought to the second session 1 week later; one group received an additional 15 minutes with an HIV nurse clinician who reinforced video content. Generalized linear models were used in order to account for the longitudinal nature of the data; a full model was fitted first that included age, gender, race, ethnicity, education, acquired immune deficiency syndrome diagnosis, time, group, and the interaction of time by intervention group with electronic health literacy as the dependent variable. Group means were not significantly different, and the overall group pattern were the same; the only significant variable was older age, which is consistent with the findings of other literature. Electronic health literacy can be increased by viewing a free video; making this video available in a variety of settings and encouraging clients to use the Internet as a source of health information may improve self-management strategies of persons living with chronic illnesses.

Moving Beyond the Gym: A Content Analysis of YouTube as an Information Resource for Physical Literacy

The Internet, and particularly YouTube, has been found to be and continues to develop as a resourceful educational space for health-related information. Understanding physical literacy as a lifelong health-related outcome and facilitator of an active lifestyle, we sought to assess the content, exposure, engagement, and information quality of uploaded physical literacy videos on YouTube. Two researchers collected 300 YouTube videos on physical literacy and independently coded each video's: title, media source of upload, content topics related to physical literacy, content delivery style, and adherence to adapted Health on the Net Foundation Code of Conduct (HONcode) principles of information quality. Physical literacy videos that focused on physical activity and behaviors were the strongest predictor of high quality ratings, followed closely by videos covering affective domains (motivation, confidence, and self-esteem) of physical literacy. The content delivery method was also important, with videos utilizing presentations and testimonials containing high quality information about physical activity. Thus, providers of physical literacy and health-related online video content should be aware of and adhere to the expected quality standards. As health information expectations and ethical standards increase, the Internet, and specifically YouTube, has the potential to enhance video resources, virtual networking opportunities, as well as the sharing, dissemination, accumulation, and enrichment of physical literacy information for all.

The Reliability and Validity of the Telephone-Based and Online Polish eHealth Literacy Scale Based on Two Nationally Representative Samples

Adequate ehealth literacy is one of the key instruments safeguarding people against unreliable health-related information obtained from the Internet. This paper presents an assessment of the reliability and the validity of a Polish version of the ehealth literacy scale (Pl-eHEALS). The assessment was carried out on the basis of two nationally representative samples of the Polish population. In the first survey of adults at least 50 years old, the technique of computer-assisted telephone interviewing (CATI) was applied. In the second survey of young adult women (18-35 years old), the technique of computer-assisted web interviewing (CAWI) was used. The reliability and the validity of the Pl-eHEALS was analyzed. There were no floor or ceiling effects revealed in either sample. The Cronbach's alpha coefficients were 0.90 and 0.88, and Guttman split-half coefficients were 0.89 and 0.81, respectively. Exploratory factors analysis revealed single factor models in both cases. The sum of squared loadings in the first survey was 6.090 and accounted for 58.72% of the variance. In the second survey, the sum was 5.927 and was responsible for 55.06% of the variance. Hypothesis testing showed that, for older adults, higher ehealth literacy was prevalent in the respondents who used the Internet more frequently. Among young adult women, higher readiness to use the Internet as a primary source of health-related information and to undertake specific internet health-related activities was associated with higher ehealth literacy. The analysis reported in this paper confirmed the reliability and the validity of the instrument. It should be stressed that, prior to this study, there was no validated Polish version of the eHEALS that could be used with Polish-speaking respondents.

Readability of Online Health Information: A Meta-Narrative Systematic Review

Online health information should meet the reading level for the general public (set at sixth-grade level). Readability is a key requirement for information to be helpful and improve quality of care. The authors conducted a systematic review to evaluate the readability of online health information in the United States and Canada. Out of 3743 references, the authors included 157 cross-sectional studies evaluating 7891 websites using 13 readability scales. The mean readability grade level across websites ranged from grade 10 to 15 based on the different scales. Stratification by specialty, health condition, and type of organization producing information revealed the same findings. In conclusion, online health information in the United States and Canada has a readability level that is inappropriate for general public use. Poor readability can lead to misinformation and may have a detrimental effect on health. Efforts are needed to improve readability and the content of online health information.

Health literacy: a study of internet-based information on advance directives

AIM

The aim of this study was to evaluate the quality and value of web-based information on advance directives. Internet-based information on advance directives was selected because, if it is inaccurate or difficult to understand, patients risk making decisions about their care that may not be followed in practice.

METHOD

Two validated health information evaluation tools, the Suitability Assessment of Materials and DISCERN, and a focus group were used to assess credibility, user orientation and effectiveness.

FINDINGS

Only one of the 34 internet-based information items on advance directives reviewed fulfilled the study criteria and 30% of the sites were classed as unreadable. In terms of learning and informing, 79% of the sites were considered unsuitable.

CONCLUSION

Using health literacy tools to evaluate internet-based health information highlights that often it is not at a functional literacy level and neither informs nor empowers users to make independent and valid healthcare decisions.

Lessons learnt from a MOOC about social media for digital health literacy

Nowadays, the Internet and social media represent prime channels for health information seeking and peer support. However, benefits of health social media can be reduced by low digital health literacy. We designed a massive open online course (MOOC) course about health social media to increase the students' digital health literacy. In this course, we wanted to explore the difficulties confronted by the MOOC users in relation to accessing quality online health information and to propose methods to overcome the issues. An online survey was carried out to assess the students' digital health literacy. This survey was one of the activities for the enrolled learners in an online course entitled "Social Media in Health Care" on "FutureLearn", one of the popular MOOC platforms. The course was hosted by Taipei Medical University, Taiwan. Data from a total of 300 respondents were collected through the online survey from 14 December 2015 to 10 January 2016. Most participants (61%) considered finding online health information is easy or very easy, while 39% were unsure or found it difficult to retrieve online health information. Most (63%) were not sure about judging whether available information can be used for making health decisions. This study indicates a demand for more training to increase skills to improve the capability of health consumers to identify trustworthy, useful health information. More research to understand the health information seeking process will be crucial in identifying the skillsets that need to be further developed. MOOCs about digital health can be a great source of knowledge when it comes to studying patients' needs.

Internet and social network users' profiles in Renal Transplant Recipients in France

BACKGROUND

The use of the Internet for searching and sharing health information and for health care interactions may have a great potential for Renal Transplant Recipients (RTR). This study aims to determine the characteristics associated with Internet and social network use in a representative sample of RTR at the time of their inclusion in the study.

METHODS

Data of this cross-sectional design is retrieved from a longitudinal study conducted in five French kidney transplant centers in 2011, and included Renal Transplant Recipients aged 18 years with a functioning graft for at least 1 year. Measures include demographic characteristics (age, gender, level of education, employment status, living arrangement, having children, invalidity and monthly incomes in the household), psycho-social characteristics measured by the perceived social support questionnaire, and medical characteristics (previous dialysis treatment, duration since transplantation, graft rejection episodes, chronic graft dysfunction, health status and comorbidities: neoplasia for the current transplant, hypertension, diabetes mellitus, smoking status, BMI > 30 kg/m2 and Charlson Comorbidity Index (CCI)). Polytomous linear regression analysis was performed to describe the Internet and social network users' profiles, using lack of Internet access as the comparison category.

RESULTS

Among the 1416 RTR participating in the study, 20.1% had no Internet access in the household, 29.4% connected to social networks and 50.5% were not connected to social networks. Patients who connected the most to the Internet and social networks were younger, male, without children, employed, with high monthly incomes in the household, without hypertension and having felt a need for an informative or an esteem support.

CONCLUSION

In our study, the majority of RTR were actively using Internet and social networks. Renal transplant units should develop flexible and Web-based sources related to transplant information, which will allow a rapid adaptation to changes in prevalent practice, improve the health of the patients and reflect their preferences.

Content and quality of websites supporting self-management of chronic breathlessness in advanced illness: a systematic review

Chronic breathlessness is a common, burdensome and distressing symptom in many advanced chronic illnesses. Self-management strategies are essential to optimise treatment, daily functioning and emotional coping. People with chronic illness commonly search the internet for advice on self-management. A review was undertaken in June 2015 to describe the content and quality of online advice on breathlessness self-management, to highlight under-served areas and to identify any unsafe content. Google was searched from Sydney, Australia, using the five most common search terms for breathlessness identified by Google Trends. We also hand-searched the websites of national associations. Websites were included if they were freely available in English and provided practical advice on self-management. Website quality was assessed using the American Medical Association Benchmarks. Readability was assessed using the Flesch-Kincaid grades, with grade 8 considered the maximum acceptable for enabling access. Ninety-one web pages from 44 websites met the inclusion criteria, including 14 national association websites not returned by Google searches. Most websites were generated in the USA (n=28, 64%) and focused on breathing techniques (n=38, 86%) and chronic obstructive pulmonary disease (n=27, 61%). No websites were found to offer unsafe advice. Adherence to quality benchmarks ranged from 9% for disclosure to 77% for currency. Fifteen (54%) of 28 written websites required grade ⩾9 reading level. Future development should focus on advice and tools to support goal setting, problem solving and monitoring of breathlessness. National associations are encouraged to improve website visibility and comply with standards for quality and readability.

Use of the DISCERN tool for evaluating web searches in childhood epilepsy

Epilepsy is an important cause of neurological disability in children. Nowadays, an increasing number of parents or caregivers use the Internet as a source of health information concerning symptoms, therapy, and prognosis of epilepsy occurring during childhood. Therefore, high-quality websites are necessary to satisfy this request. Using the DISCERN tool, we evaluated online information on childhood epilepsy provided by the first 50 links displayed on the Google search engine. The same links were evaluated by a team of pediatric neurologists (PNs) and by a lay subject (LS). The evaluation performed by the PNs found out that only 9.6% of the websites showed good reliability, that only 7.2% of the websites had a good quality of information on treatment choices, and that only 21.5% of the websites showed good overall quality of the content. With regard to the evaluation performed by the neutral subject, it was found that 21.4% of the websites showed good reliability, that 59.5% of the websites showed poor quality of information on treatment choices, and that only 2% of the websites showed good overall quality of the content. Our conclusion is that online information about childhood epilepsy still lacks reliability, accuracy, and relevance as well as fails to provide a thorough review of treatment choices.

Wikis to facilitate patient participation in developing information leaflets: first experiences

OBJECTIVE

Although patients have gained a wealth of experienced based knowledge they are usually not involved in the development of patient information. We sought to determine the technical feasibility of wikis in generating dynamic patient information leaflets with participation from patients and healthcare professionals and identified barriers and facilitators for wiki use.

METHODS

An open wiki for patients receiving intrauterine inseminations and a closed wiki for patients receiving in vitro fertilization at one Dutch university clinic were used. Feasibility was assessed by analyzing logging data, content and users' experiences and expectations. The latter were addressed by means of semi-structured interviews, which were also used to identify barriers and facilitators to wiki use.

RESULTS

Both patients and professionals worked on the information simultaneously, not hindered by time or location restrictions. The open IUI wiki was visited by 2957 and the closed IVF wiki by 424 users. About 28 barriers and 14 facilitators for using the wiki were identified.

CONCLUSION

Wikis are promising tools to improve patient participation in the creation of patient information. Future projects should aim to increase active use of wikis by testing interventions based on the wide spectrum of barriers and should evaluate the quality of the content produced.

Health care development: integrating transaction cost theory with social support theory

The emergence of Web 2.0 technologies has already been influential in many industries, and Web 2.0 applications are now beginning to have an impact on health care. These new technologies offer a promising approach for shaping the future of modern health care, with the potential for opening up new opportunities for the health care industry as it struggles to deal with challenges including the need to cut costs, the increasing demand for health services and the increasing cost of medical technology. Social media such as social networking sites are attracting more individuals to online health communities, contributing to an increase in the productivity of modern health care and reducing transaction costs. This study therefore examines the potential effect of social technologies, particularly social media, on health care development by adopting a social support/transaction cost perspective. Viewed through the lens of Information Systems, social support and transaction cost theories indicate that social media, particularly online health communities, positively support health care development. The results show that individuals join online health communities to share and receive social support, and these social interactions provide both informational and emotional support.

What is consumer health informatics? A systematic review of published definitions

BACKGROUND

Consumer health informatics (CHI) is an emerging field that utilizes technology to provide health information to enhance health-care decision making by the public. There is, however, no widely accepted or uniform definition of CHI. A consensus definition would be important for pedagogical reasons, to build capacity and to reduce confusion about what the discipline consists of.

AIM

We undertook a systematic review of published definitions of CHI and evaluated them using five quality assessment criteria and measures of similarity.

METHODS

Five databases were searched (Embase, Web of Science, MEDLINE, CINAHL and Business Source Complete) resulting in 1101 citations. Twenty-three studies met the inclusion criteria. Definitions were appraised using five criteria (with each scoring out of one): use of published citation, multi-disciplinarity, journal impact, definition comprehensibility, text readability.

RESULTS

Most definitions scored low on citation (Mean ± SD: 0.22 ± 0.42), multi-disciplinarity (0.15 ± 0.28) and readability (0.04 ± 0.21) and somewhat higher on IF (0.35 ± 0.45) and definition comprehensibility (idea density) (0.87 ± 0.34) criteria. Overall, the quality of the published definitions was low 1.63 ± 0.80 (out of five).

CONCLUSIONS

The definitions of CHI were variable in terms of the quality assessment criteria. This suggests the need for continued discussion amongst consumer health informaticians to develop a clear consensus definition about CHI.

[Evaluation of the Andalusia Public Health System hospital websites in the period 2010-2012]

OBJECTIVES

Evaluate the quality, accessibility and presence of Web 2.0 tools in the Andalusia Public Health System hospitals websites

METHODS

Observational, descriptive study carried out between 2010 and 2012. The variables analyzed were: quality, accessibility and innovation. The quality was evaluated using a Bermudez-Tamayo questionnaire. Accessibility was measured using the Web Accessibility Test (TAW) tool. Web 2.0 tools were identified by direct observation.

RESULTS

A total of 31 of the 45 hospitals (68.9%) had a website in the year 2010, increasing to 34 (75.5%) in 2012. The average score+standard deviation (SD) of the Bermudez-Tamayo quality questionnaire was 11.1+3.8 points in 2010, and 12.3+3.9 points in 2012, observing a statistically significant difference of 0.25 being observed between the means (P=.007), 95% CI; 0.00 to 0.50) In the accessibility evaluation only 7 websites (n=31) in 2010, and 10 (n=34) in 2012, fulfilled the legal criteria for accessibility. The use of Web 2.0 tools has increased throughout the study. In 2010, 19.4% (n=6) of the hospital websites had this type of tool, in comparison to 58.8% (n=20) in 2012.

CONCLUSIONS

In general, the quality of the websites studied is good. However, current legislation regarding accessibility is not fulfilled and must be revised and adapted to the current legal rules. There is an incipient use of Web 2.0 resources as education and communication strategies with regard to health.

Waiting for a heart or lung transplant: Relatives' experience of information and support

OBJECTIVES

To describe the relatives' experiences of information and support while heart or lung transplant candidates were waiting for a transplantation.

METHODOLOGY/DESIGN

The critical incident technique was used. Incidents were collected via interviews with 18 relatives (28-73 years old) of heart or lung transplant patients within four weeks of the patients being accepted onto the transplant waiting list.

FINDINGS

A total of 286 important events, both positive and negative, were identified and divided into two main areas: "Experiencing information and support" and "Reflecting upon information and support." "Experiencing information and support" was associated with "dissatisfaction with the health-care system", "being relatively satisfied", "supporting patients" and "the role of social networks." "Reflecting upon information and support" was associated with "finding strength" and "uncomfortable with emotions". Relatives experiences suggest that needs can be met through specific targeted information and support for them as well as mediating contact to previously transplanted persons.

CONCLUSION

The findings indicate a gap between the information and support that relatives need and receive, and that more attention should be paid to information and support for this group. By increasing awareness among healthcare professionals and the community, interventions can be developed that benefit relatives.

Development and content validation of the information assessment method for patients and consumers

BACKGROUND

Online consumer health information addresses health problems, self-care, disease prevention, and health care services and is intended for the general public. Using this information, people can improve their knowledge, participation in health decision-making, and health. However, there are no comprehensive instruments to evaluate the value of health information from a consumer perspective.

OBJECTIVE

We collaborated with information providers to develop and validate the Information Assessment Method for all (IAM4all) that can be used to collect feedback from information consumers (including patients), and to enable a two-way knowledge translation between information providers and consumers.

METHODS

Content validation steps were followed to develop the IAM4all questionnaire. The first version was based on a theoretical framework from information science, a critical literature review and prior work. Then, 16 laypersons were interviewed on their experience with online health information and specifically their impression of the IAM4all questionnaire. Based on the summaries and interpretations of interviews, questionnaire items were revised, added, and excluded, thus creating the second version of the questionnaire. Subsequently, a panel of 12 information specialists and 8 health researchers participated in an online survey to rate each questionnaire item for relevance, clarity, representativeness, and specificity. The result of this expert panel contributed to the third, current, version of the questionnaire.

RESULTS

The current version of the IAM4all questionnaire is structured by four levels of outcomes of information seeking/receiving: situational relevance, cognitive impact, information use, and health benefits. Following the interviews and the expert panel survey, 9 questionnaire items were confirmed as relevant, clear, representative, and specific. To improve readability and accessibility for users with a lower level of literacy, 19 items were reworded and all inconsistencies in using a passive or active voice have been solved. One item was removed due to redundancy. The current version of the IAM4all questionnaire contains 28 items.

CONCLUSIONS

We developed and content validated the IAM4all in partnership with information providers, information specialists, researchers and representatives of information consumers. This questionnaire can be integrated within electronic knowledge resources to stimulate users' reflection (eg, their intention to use information). We claim that any organization (eg, publishers, community organizations, or patient associations), can evaluate and improve their online consumer health information from a consumers' perspective using this method.

Development and validation of a tool to evaluate the quality of medical education websites in pathology

BACKGROUND

The exponential use of the internet as a learning resource coupled with varied quality of many websites, lead to a need to identify suitable websites for teaching purposes.

AIM

The aim of this study is to develop and to validate a tool, which evaluates the quality of undergraduate medical educational websites; and apply it to the field of pathology.

METHODS

A tool was devised through several steps of item generation, reduction, weightage, pilot testing, post-pilot modification of the tool and validating the tool. Tool validation included measurement of inter-observer reliability; and generation of criterion related, construct related and content related validity. The validated tool was subsequently tested by applying it to a population of pathology websites.

RESULTS AND DISCUSSION

Reliability testing showed a high internal consistency reliability (Cronbach's alpha = 0.92), high inter-observer reliability (Pearson's correlation r = 0.88), intraclass correlation coefficient = 0.85 and κ =0.75. It showed high criterion related, construct related and content related validity. The tool showed moderately high concordance with the gold standard (κ =0.61); 92.2% sensitivity, 67.8% specificity, 75.6% positive predictive value and 88.9% negative predictive value. The validated tool was applied to 278 websites; 29.9% were rated as recommended, 41.0% as recommended with caution and 29.1% as not recommended.

CONCLUSION

A systematic tool was devised to evaluate the quality of websites for medical educational purposes. The tool was shown to yield reliable and valid inferences through its application to pathology websites.

Seeing is believing? A mixed-methods study exploring the quality and perceived trustworthiness of online information about chronic conditions aimed at children and young people

The numbers of children and young people with chronic conditions are increasing. While their information needs may vary, providing health care information can have considerable benefits, including better emotional health, less distress during treatments, and greater satisfaction with medical care. The Internet is increasingly being used to communicate health-related information to children about a range of chronic conditions. However, the quality of such websites is underexplored. Thus, the objectives of this study were to evaluate the reliability and quality of online information for children about chronic conditions using a standardized evaluation tool, and to explore children's and young people's perceptions of quality and trustworthiness regarding online health information. The study consisted of two phases. In Phase 1, websites about common pediatric chronic conditions aimed at children and containing treatment or management options were identified and the quality assessment tool DISCERN was completed. Test-retest and interrater reliability were calculated. In Phase 2, two focus groups with laptop computers were conducted with children and young people with a chronic condition to explore their perceptions of trustworthiness of online health information. In Phase 1, 165 websites were identified and 100 met the criteria and were assessed. The mean DISCERN score of all sites was 48.16 out of 75 (SD = 7.97, range 28-71, min 15 to max 75). Quality scores varied widely across the sample. The internal consistency and interrater reliability scores were both lower than previously reported in studies using the DISCERN to assess information for adults. In Phase 2, two focus groups with a total of six participants aged 11-23 years revealed a relative lack of concern about the quality and trustworthiness of online health information. Older participants reported judging the source and authorship of websites, but other participants did not question the source of the information they found online. Although personal websites were perceived to be less reliable than those from well-known medical institutions, they were still valued by many of the participants. Overall, there were relatively few websites about pediatric chronic conditions aimed at children, with variable quality and reliability. However, DISCERN's use with pediatric websites across a broad spectrum of developmental stages is limited. Children and young people demonstrated a need for more guidance around assessing trustworthiness of online information. A more appropriate quality assessment tool is warranted, which could usefully be employed by health care professionals, children, and parents.

[Internet resources for radiologists : a positive selection]

In radiology the use of internet resources is part of the daily routine. The critical selection of but not the access to radiologically relevant information is the key topic in internet use. We offer a selection of reviewed internet sources for radiologists. We present sources that offer guidance in decision-making in daily routine as well as internet sources on technical topics in radiology. Case selection and radiological search engines are helpful sites to begin an internet research. Online anatomy sites can be of assistance for interpretation of findings in radiology. Writing and publishing scientific works in radiology requires access to numerous internet services, a positive selection of which are reviewed in the present article. A variety of freeware is available on the internet, some of which may be a substantial benefit to projects in radiology.

YouTube: a promotional vehicle for little cigars and cigarillos?

OBJECTIVE

As the third most popular website in the world, messages embedded in the video-sharing site, YouTube, have the potential to influence tobacco-related attitudes, beliefs and behaviours. Despite the growing number of videos depicting little cigars/cigarillos (LCCs), there has been no examination of the portrayal of these products on YouTube.

METHODS

Researchers identified up to the top 20 search results on YouTube by relevance and view count for the following search terms: 'little cigars', 'cigarillos', 'Black and Mild', 'Swisher Sweets', 'White owl', 'Garcia y Vega', and 'Winchester'. Reviewers rated whether videos were 'pro', 'anti' or 'neutral' to the use of LCCs, and documented statistics on the reach and viewer demographics. Several main themes around LCCs were noted, as was video quality (amateur vs professional) and demographics of video participants.

RESULTS

Of the 196 videos retrieved, only 56 were unique, eligible videos. The majority of these (n=43) were 'pro' LCCs, 11 were 'neutral', and only two were 'anti' LCCs. Videos were primarily viewed by males in the USA and Canada and most were amateur. Common themes included where to purchase LCCs, their candy flavours, and that they are cheap or cheaper than cigarettes, and 'smooth'.

CONCLUSIONS

The vast majority of information on YouTube about LCCs promotes their use. It is critical to monitor content on LCCs posted on YouTube, and develop appropriate health messages to counter pro-LCC content, and appropriately inform potential consumers of the harms associated with their use.

Quality of online health information about oral contraceptives from Hebrew-language websites

Background

The Internet is a frequently used source of health information. Adolescents in particular seem to be receptive to online health information (OHI) and often incorporate such information in their decision-making processes. Yet, OHI is often incomplete, inaccurate, or unreliable. This study assessed the quality of Hebrew online (non-user-generated) content on oral contraceptives (OC), with regard to accuracy/completeness, credibility, and usability.

Methods

Twenty-nine websites in Hebrew, including those of the four Israeli HMOs, were identified and evaluated. The websites were categorized as: HMO, health portal, contraception-specific, promotional-commercial, and life style and women’s health. A set of established content parameters was selected by a family planning expert to assess accuracy/completeness. The Health on the Net Foundation Code of Conduct (HONcode) principles were used to assess the websites’ reliability. Usability was assessed by applying items selected from the Minervation Validation and the University of Michigan’s ′Website Evaluation checklist′ scale. Mean scores, standard deviations (SD), and ranges were calculated for all websites and for category-specific websites. Correlation between dimensions and Inter-rater reliability were also examined.

Results

The mean score for accuracy/completeness was 50.9% for all websites (SD=30.1%, range 8–100%). Many websites failed to provide complete information, or provided inaccurate information regarding what to do when a pill is missed and when to use back–up methods. The average credibility score for all websites was 70.6% (SD=15.1, range=38=98%). The credibility parameters that were most commonly absent were funding source, authoring, date of content creation and last modification, explicit reference to evidence-based information, and references and citations. The average usability score for all websites was 94.5% (SD=6.9%, range 79–100%). A weak correlation was found between the three quality parameters assessed.

Conclusions

Wide variation was noted in the quality of Hebrew-language OC websites. HMOs’ websites scored highest on credibility and usability, and contraceptive-specific websites exhibited the greatest accuracy/completeness. The findings highlight the need to establish quality guidelines for health website content, train health care providers in assisting their patients to seek high quality OHI, and strengthen e-health literacy skills among online-information seekers, including perhaps health professionals.

A survey of public health and consumer health informatics programmes and courses in Canadian universities and colleges

INTRODUCTION

As information technology becomes more widely used by people for health-care decisions, training in consumer and public health informatics will be important for health practitioners working directly with the public.

METHODS

Using information from 74 universities and colleges across Canada, we searched websites and online calendars for programmes (undergraduate, graduate) regarding availability and scope of education in programmes, courses and topics geared to public health and/or consumer health informatics.

RESULTS

Of the 74 institutions searched, 31 provided some content relevant to health informatics (HI) and 8 institutions offered full HI-related programmes. Of these 8 HI programmes, only 1 course was identified with content relevant to public health informatics and 1 with content about consumer health informatics. Some institutions (n  =  22) - which do not offer HI-degree programmes - provide health informatics-related courses, including one on consumer health informatics. We found few programmes, courses or topic areas within courses in Canadian universities and colleges that focus on consumer or public health informatics education.

DISCUSSION

Given the increasing emphasis on personal responsibility for health and health-care decision-making, skills training for health professionals who help consumers navigate the Internet should be considered in health informatics education.

Can consumers trust web-based information about celiac disease? Accuracy, comprehensiveness, transparency, and readability of information on the internet

Background

Celiac disease is an autoimmune disease that affects approximately 1% of the US population. Disease is characterized by damage to the small intestinal lining and malabsorption of nutrients. Celiac disease is activated in genetically susceptible individuals by dietary exposure to gluten in wheat and gluten-like proteins in rye and barley. Symptoms are diverse and include gastrointestinal and extraintestinal manifestations. Treatment requires strict adherence to a gluten-free diet. The Internet is a major source of health information about celiac disease. Nonetheless, information about celiac disease that is available on various websites often is questioned by patients and other health care professionals regarding its reliability and content.

Objectives

To determine the accuracy, comprehensiveness, transparency, and readability of information on 100 of the most widely accessed websites that provide information on celiac disease.

Methods

Using the search term celiac disease, we analyzed 100 of the top English-language websites published by academic, commercial, nonprofit, and other professional (nonacademic) sources for accuracy, comprehensiveness, transparency, and reading grade level. Each site was assessed independently by 3 reviewers. Website accuracy and comprehensiveness were probed independently using a set of objective core information about celiac disease. We used 19 general criteria to assess website transparency. Website readability was determined by the Flesch-Kincaid reading grade level. Results for each parameter were analyzed independently. In addition, we weighted and combined parameters to generate an overall score, termed website quality.

Results

We included 98 websites in the final analysis. Of these, 47 (48%) provided specific information about celiac disease that was less than 95% accurate (ie, the predetermined cut-off considered a minimum acceptable level of accuracy). Independent of whether the information posted was accurate, 51 of 98 (52%) websites contained less than 50% of the core celiac disease information that was considered important for inclusion on websites that provide general information about celiac disease. Academic websites were significantly less transparent (P = .005) than commercial websites in attributing authorship, timeliness of information, sources of information, and other important disclosures. The type of website publisher did not predict website accuracy, comprehensiveness, or overall website quality. Only 4 of 98 (4%) websites achieved an overall quality score of 80 or above, which a priori was set as the minimum score for a website to be judged trustworthy and reliable.

Conclusions

The information on many websites addressing celiac disease was not sufficiently accurate, comprehensive, and transparent, or presented at an appropriate reading grade level, to be considered sufficiently trustworthy and reliable for patients, health care providers, celiac disease support groups, and the general public. This has the potential to adversely affect decision making about important aspects of celiac disease, including its appropriate and proper diagnosis, treatment, and management.

Social networks lack useful content for incontinence

OBJECTIVE

To assess the incontinence resources readily available for patients among social networks. Social networks allow users to connect with each other and share content and are a widely popular resource on the Internet. These sites attract millions of users; however, social media are underused in the healthcare industry.

METHODS

A search for "incontinence" was performed on Facebook, Twitter, and YouTube in September 2010. The first 30 results were reviewed for each. The results were evaluated as useful or not useful and additionally catalogued as healthcare professionals, commercial products, or complementary and alternative medicine resources.

RESULTS

On Facebook, 4 results (13%) were informative, 12 (40%) advertised commercial incontinence products, and 14 (47%) had no usable information. Of the live "tweets" reviewed on Twitter, 18 (60%) linked to incontinence-related healthcare information (none were from, or referred one to, healthcare professionals), 9 (30%) advertised for commercial incontinence products, 1 (3%) advertised complementary and alternative medicine resources, and 2 (7%) were humorous. Of 4 Twitter user results, 1 was comic, 1 provided incontinence-related health information, and 2 were incontinence medical supply companies. There were 14 (47%) informational YouTube videos, of which 9 came from healthcare professionals or professional organizations. Of the remainder, 12 (40%) were commercial, 1 (3%) advertised complementary and alternative medicine resources, and the remaining 3 (10%) were not useful.

CONCLUSION

The results of our study show that social networks have insufficient useful incontinence content, especially from healthcare professionals and incontinence organizations. Medical professionals and societies should target these avenues to reach and educate patients.

Trust in the internet as a health resource among older adults: analysis of data from a nationally representative survey

Background

Distrust in the Internet as a source of health information remains common among older adults. The influence of this distrust on Internet use for health-related purposes, however, is unclear.

Objective

The objective of our study was to explore how older adults’ trust in the Internet influences their online health-related activities, and to identify potential targets for improving health-related Internet resources for older adults.

Methods

Data were obtained from a nationally representative, random digit-dial telephone survey of 1450 adults 50 years of age and older in the United States. A model was developed to conceptualize the hypothesized relationships among individual characteristics, distrust, and avoidance of the Internet as a health resource. Multivariate logistic regression analyses were conducted to examine the association between trust in online health information and use of the Internet for health-related purposes. Additional multivariate logistic regression analyses were conducted to identify the key characteristics associated with trust in online health information, adding sequentially the variables hypothesized to account for distrust among older adults: sociodemographic and health characteristics, inexperience and technical difficulties with the Internet, negative feelings toward the Internet, and lack of awareness about the sources providing the health information found online.

Results

The mean (SD) age of the study population was 63.7 (10.6) years. Of the 823 (56.8%) Internet users, 628 (76.3%) reported using the Internet as a health resource. Trust in the Internet as a source of health information was associated with using the Internet for a number of health activities, including searching for information about a specific health condition (adjusted OR 4.43, P < .001), purchasing prescription drugs (adjusted OR 2.61, P = .03), and talking with a health care provider about information found online (adjusted OR 2.54, P = .002). Older adults (age ≥65 years) were less likely to trust the Internet as a source of health information (OR 0.63, P = .04), even after adjusting for other sociodemographic characteristics and health and function. This age effect was only slightly attenuated (adjusted OR 0.69, P = .13) after adjusting for inexperience and technical difficulties with the Internet, but it disappeared entirely (adjusted OR 0.96, P = .91) after adjusting for other hypothesized contributors to distrust (including finding the Internet confusing because it provides “too much information,” and lacking awareness about the source providing health information found online).

Conclusions

Website design features that clearly identify the source and credibility of information and minimize confusion may build trust among older adults and offer an opportunity to increase the utility of the Internet as a health resource for this population.