American Public Health Association. Criteria for assessing the quality of health information on the Internet

Fetal growth restriction: How reliable is information available to patients on Google? A systematic review

OBJECTIVES

To review systematically the quality, readability and credibility of English language webpages offering patient information on fetal growth restriction.

STUDY DESIGN

A systematic review of patient information was undertaken on Google with location services and browser history disabled. Websites from the first page were included providing they gave at least 300 words of health information on fetal growth restriction aimed at patients. Validated assessment of readability, credibility and quality were undertaken. An accuracy assessment was performed based on international guidance. Characteristics were tabulated.

RESULTS

Thirty-one websites including 30 different texts were included. No pages had a reading age of 11 years or less, none were credible, and only one was of high quality. Median accuracy rating was 9/24.

CONCLUSION

Patients cannot rely on Google as a source of information on fetal growth restriction. As well as being difficult to read, information tends to be low quality, low accuracy and not credible. Healthcare professionals must consider how to enable access to high-quality patient information and give time for discussion of information patients have found: failure to do so may disenfranchise patients.

Reliability of online pregnancy-related information and associated feelings of worry among expectant women in Qatar

Background

Although the internet can be a source of reassurance and clarification for expectant women, it could cause concerns or feelings of worry when reading about pregnancy-related information. This research study sought to assess feelings of worry and perceived reliability of online pregnancy-related information and the associated factors among expectant women attending antenatal clinics at primary healthcare centers in Qatar.

Methods

A cross-sectional study design was used. The participants were recruited through a systematic random sampling technique. A self-administered questionnaire was used to collect data from the participants. Descriptive and analytic statistics were used as appropriate.

Results

A total of 327 expectant women completed the questionnaire. Most were aged between 26–34 years (74.1%), held a college/university degree (76.4%), and were multigravidas (73.1%). About one-third of the women (31.2%) reported feeling worried due to information they read online. They coped with these feelings by consulting their antenatal care provider at their next appointment (51.0%) or by talking with relatives and friends (47.0%). Most participants (79.2%) considered online pregnancy-related information to be reliable or highly reliable. Holding a college/university degree and being primigravidae were factors significantly associated with a high perception of reliability of online pregnancy information.

Conclusion

Although online pregnancy information caused feelings of worry for some expectant women, most perceived such information to be reliable. Thus, antenatal care providers should guide expectant women on how to access high-quality web-based information.

Awareness, Information-Seeking Behavior, and Information Preferences About Early Childhood Allergy Prevention Among Different Parent Groups: Protocol for a Mixed Methods Study

Background

In early childhood allergy prevention (ECAP), parents act on behalf of their children. Parental health literacy and the availability of high-quality information, both online and offline, are crucial for effective ECAP. Recent research highlights three main points. First, parents need sufficient health literacy to discriminate between high-quality and low-quality information. Second, ECAP information behaviors may vary between phases of childhood development and according to individual circumstances. Third, to strengthen user-centeredness of available services, a better overview of parents’ information practices and needs and how they handle uncertainties is required.

Objective

This study aims to explore why, how, and when parents search for and apply ECAP-specific health information and which individual (eg, understanding of advice) and organizational challenges (eg, information services, information complexity, and changing recommendations) they perceive and how they handle them. This study also aims to assess the needs and preferences that parents express for future information formats and contents. The findings should inform the practical design of ECAP information as well as formats and channels specific to different parent groups.

Methods

The above-named issues will be explored with parents in four German cities as one element in our efforts to cover the spectrum of perspectives. Based on a mixed methods design, including qualitative and quantitative assessments, the first year serves to prepare focus groups, a piloted focus group guide, a short standardized survey adapted from the European Health Literacy Project, recruitment channels, and the recruitment of participants. After conducting 20 focus groups in the second year, data will be analyzed via a constant comparison method in the third year. Based on this, practice implications on channels (ie, Where?), formats (ie, How?), and contents (ie, What?) of ECAP-specific information will be derived and discussed with parents and associated project partners before its dissemination to relevant ECAP actors (eg, childcare institutions and pediatricians).

Results

The study began with preselection of recruitment channels, drafting of recruitment and study information for potential participants, and agreement on a first full version of the guideline. Then, a detailed contact list was compiled of health professionals, administrative and social institutions, and relevant social media channels (N=386) to be approached for assistance in contacting parents. The recruitment was postponed due to COVID-19 and will start in January 2021.

Conclusions

ECAP is a relevant example for assessing how users (ie, parents) handle not only health information but the various and continuous changes, uncertainties, and controversies attached to it. So far, it is unclear how parents implement the respective scientific recommendations and expert advice, which is why this study aims to inform those who communicate with parents about ECAP information.

International Registered Report Identifier (IRRID)

PRR1-10.2196/25474

What makes a good health 'app'? Identifying the strengths and limitations of existing mobile application evaluation tools

Research using mHealth apps has the potential to positively impact health care management and outcomes. However, choosing an appropriate mHealth app may be challenging for the health researcher. The author team used existing evaluation tools, checklists, and guidelines to assess selected mHealth apps to identify strengths, challenges, and potential gaps within existing evaluation tools. They identified specific evaluation tool components, questions, and items most effective in examining app content, usability, and features, including literacy demand and cultural appropriateness; technical information; practical aspects of app functionality; and evolving capabilities of mobile medical apps. Challenges included the subjective nature of the results, time required to complete the evaluation, lack of emphasis on evidence-based content, and inadequate tool flexibility. Health researchers considering the integration of mobile apps into research will benefit from evaluation tools that assess both evidence-based content and the ability of the mobile app to securely integrate with other digital technologies involved in patient care. Next steps will include the involvement of health care providers and professionals, including nurses a wide range of expertise, to develop an mHealth evaluation tool that focuses on identifying quality, evidence-based mobile apps into patient outcomes research.

The quality of information about sickle cell disease on the Internet for youth

BACKGROUND

Adolescence is a vulnerable time for teens with sickle cell disease (SCD). Although there is evidence to support the use of web-based education to promote self-management skills in patients with chronic illnesses, the quality of SCD-related information on the Internet has not been assessed.

PROCEDURE

A website review was conducted to appraise the quality, content, accuracy, readability, and desirability of online information for the adolescents with SCD. Relevant keywords were searched on the most popular search engines. Websites meeting predetermined criteria were reviewed. The quality of information was appraised using the validated DISCERN tool. Two physicians independently rated website completeness and accuracy. Readability of the sites was documented using the simple measure of gobbledygook (SMOG) scores and the Flesch Reading Ease (FRE). The website features considered desirable by youth were tracked.

RESULTS

Search results yielded >600 websites with 25 unique hits meeting criteria. The overall quality of the information was "fair" and the average DISCERN rating score was 50.1 (±9.3, range 31.0-67.5). Only 12 of 25 (48%) websites had scores >50. The average completeness score was 20 of 29 (±5, range 12-27). No errors were identified. The mean SMOG score was 13.04 (±2.80, range 10.21-22.85) and the mean FRE score was 46.05 (±11.47; range 17.50-66.10), suggesting that the material was written well beyond the acceptable reading level for patient education. The websites were text-heavy and lacked the features that appeal to youth (chat, games, videos, etc.).

CONCLUSION

Given the paucity of high-quality health information available for the teens with SCD, it is essential that additional online resources be developed.

Clinical calculators in hospital medicine: Availability, classification, and needs

OBJECTIVE

Clinical calculators are widely used in modern clinical practice, but are not generally applied to electronic health record (EHR) systems. Important barriers to the application of these clinical calculators into existing EHR systems include the need for real-time calculation, human-calculator interaction, and data source requirements. The objective of this study was to identify, classify, and evaluate the use of available clinical calculators for clinicians in the hospital setting.

METHODS

Dedicated online resources with medical calculators and providers of aggregated medical information were queried for readily available clinical calculators. Calculators were mapped by clinical categories, mechanism of calculation, and the goal of calculation. Online statistics from selected Internet resources and clinician opinion were used to assess the use of clinical calculators.

RESULTS

One hundred seventy-six readily available calculators in 4 categories, 6 primary specialties, and 40 subspecialties were identified. The goals of calculation included prediction, severity, risk estimation, diagnostic, and decision-making aid. A combination of summation logic with cutoffs or rules was the most frequent mechanism of computation. Combined results, online resources, statistics, and clinician opinion identified 13 most utilized calculators.

CONCLUSION

Although not an exhaustive list, a total of 176 validated calculators were identified, classified, and evaluated for usefulness. Most of these calculators are used for adult patients in the critical care or internal medicine settings. Thirteen of 176 clinical calculators were determined to be useful in our institution. All of these calculators have an interface for manual input.

‘How to stop a nosebleed’: an assessment of the quality of epistaxis treatment advice on YouTube

Objective:

Video hosting websites are increasingly being used to disseminate health education messages. This study aimed to assess the quality of advice contained within YouTube videos on the conservative management of epistaxis.

Method:

YouTube.com was searched using the phrase ‘how to stop a nosebleed’. The first 50 videos were screened. Objective advice scores and subjective production quality scores were attributed by independent raters.

Results:

Forty-five videos were analysed. The mean advice score was 2.0 out of 8 and the mean production quality score was 1.6 out of 3. There were no correlations between a video's advice score and its search results rank (ρ = −0.28, p = 0.068), its view count (ρ = 0.20, p = 0.19) or its number of ‘likes’ (ρ = 0.21, p = 0.18).

Conclusion:

The quality of information on conservative epistaxis management within YouTube videos is extremely variable. A high search rank is no indication of video quality. Many videos proffer inappropriate and dangerous ‘alternative’ advice. We do not recommend YouTube as a source for patient information.

An overview of current and potential use of information and communication technologies for immunization promotion among adolescents

Information and communication technologies (ICT), such as the Internet or mobile telephony, have become an important part of the life of today's adolescents and their main means of procuring information. The new generation of the Internet based on social-networking technologies, Web 2.0, is increasingly used for health purposes by both laypeople and health professionals. A broad spectrum of Web 2.0 applications provides several opportunities for healthcare workers, in that they can reach large numbers of teenagers in an individualized way and promote vaccine-related knowledge in an interactive and entertaining manner. These applications, namely social-networking and video-sharing websites, wikis and microblogs, should be monitored in order to identify current attitudes toward vaccination, to reply to vaccination critics and to establish a real-time dialog with users. Moreover, the ubiquity of mobile telephony makes it a valuable means of involving teenagers in immunization promotion, especially in developing countries.

Identifying Measures Used for Assessing Quality of YouTube Videos with Patient Health Information: A Review of Current Literature

BACKGROUND

Recent publications on YouTube have advocated its potential for patient education. However, a reliable description of what could be considered quality information for patient education on YouTube is missing.

OBJECTIVE

To identify topics associated with the concept of quality information for patient education on YouTube in the scientific literature.

METHODS

A literature review was performed in MEDLINE, ISI Web of Knowledge, Scopus, and PsychINFO. Abstract selection was first conducted by two independent reviewers; discrepancies were discussed in a second abstract review with two additional independent reviewers. Full text of selected papers were analyzed looking for concepts, definitions, and topics used by its authors that focused on the quality of information on YouTube for patient education.

RESULTS

In total, 456 abstracts were extracted and 13 papers meeting eligibility criteria were analyzed. Concepts identified related to quality of information for patient education are categorized as expert-driven, popularity-driven, or heuristic-driven measures. These include (in descending order): (1) quality of content in 10/13 (77%), (2) view count in 9/13 (69%), (3) health professional opinion in 8/13 (62%), (4) adequate length or duration in 6/13 (46%), (5) public ratings in 5/13 (39%), (6) adequate title, tags, and description in 5/13 (39%), (7) good description or a comprehensive narrative in 4/13 (31%), (8) evidence-based practices included in video in 4/13 (31%), (9) suitability as a teaching tool in 4/13 (31%), (10) technical quality in 4/13 (31%), (11) credentials provided in video in 4/13 (31%), (12) enough amount of content to identify its objective in 3/13 (23%), and (13) viewership share in 2/13 (15%).

CONCLUSIONS

Our review confirms that the current topics linked to quality of information for patient education on YouTube are unclear and not standardized. Although expert-driven, popularity-driven, or heuristic-driven measures are used as proxies to estimate the quality of video information, caution should be applied when using YouTube for health promotion and patient educational material.

Identifying Measures Used for Assessing Quality of YouTube Videos with Patient Health Information: A Review of Current Literature

BACKGROUND

Recent publications on YouTube have advocated its potential for patient education. However, a reliable description of what could be considered quality information for patient education on YouTube is missing.

OBJECTIVE

To identify topics associated with the concept of quality information for patient education on YouTube in the scientific literature.

METHODS

A literature review was performed in MEDLINE, ISI Web of Knowledge, Scopus, and PsychINFO. Abstract selection was first conducted by two independent reviewers; discrepancies were discussed in a second abstract review with two additional independent reviewers. Full text of selected papers were analyzed looking for concepts, definitions, and topics used by its authors that focused on the quality of information on YouTube for patient education.

RESULTS

In total, 456 abstracts were extracted and 13 papers meeting eligibility criteria were analyzed. Concepts identified related to quality of information for patient education are categorized as expert-driven, popularity-driven, or heuristic-driven measures. These include (in descending order): (1) quality of content in 10/13 (77%), (2) view count in 9/13 (69%), (3) health professional opinion in 8/13 (62%), (4) adequate length or duration in 6/13 (46%), (5) public ratings in 5/13 (39%), (6) adequate title, tags, and description in 5/13 (39%), (7) good description or a comprehensive narrative in 4/13 (31%), (8) evidence-based practices included in video in 4/13 (31%), (9) suitability as a teaching tool in 4/13 (31%), (10) technical quality in 4/13 (31%), (11) credentials provided in video in 4/13 (31%), (12) enough amount of content to identify its objective in 3/13 (23%), and (13) viewership share in 2/13 (15%).

CONCLUSIONS

Our review confirms that the current topics linked to quality of information for patient education on YouTube are unclear and not standardized. Although expert-driven, popularity-driven, or heuristic-driven measures are used as proxies to estimate the quality of video information, caution should be applied when using YouTube for health promotion and patient educational material.

Information on the internet for asplenic patients: a systematic review

BACKGROUND

Asplenic patients in general have poor knowledge about their condition. Patients are increasingly turning to the Internet for their health care information, therefore this is a resource that many asplenic patients will use. The aim of our study was to determine the quality of information on the Internet for asplenic patients.

METHODS

We identified websites by entering "splenectomy OR spleen removal" into 3 Internet search engines on July 28, 2008. The top 50 English-language websites from each search engine were included in our analysis. We evaluated the websites with our own 21-point content scale as well as 4 commonly used quality-assessment tools. All websites were analyzed independently by 2 reviewers. Correlations were made between the quality assessment instruments, content, readability and target audience.

RESULTS

We included 89 websites in the study. The mean content score percentage for all websites was 49% (95% confidence interval 44%-54%). The long-term risk of infection was mentioned in 84% of websites, and the need for vaccination was mentioned in 79%. The mean quality assessment tool score was 61%, and the mean reading grade level was 11.

CONCLUSION

Whereas websites on average did not cover most of the information that asplenic patients should receive, the long-term risk of serious infection and the need for vaccination was consistently mentioned. Websites were inconsistent with respect to adhering to standards advocated by the quality assessment instruments we used, and the mean reading grade level was far above what is recommended for patient literature.

[Study of the health food information for cancer patients on Japanese websites]

The aim of this paper is to evaluate the reliability of websites providing health food information for cancer patients and, to assess the status to get this information online. We used four common Japanese search engines (Yahoo!, Google, goo, and MSN) to look up websites on Dec. 2, 2008. The search keywords were "health food" and "cancer". The websites for the first 100 hits generated by each search engine were screened and extracted by three conditions. We extracted 64 unique websites by the result of retrieval, of which 54 websites had information about health food factors. The two scales were used to evaluate the quality of the content on 54 websites. On the scale of reliability of information on the Web, the average score was 2.69+/-1.70 (maximum 6) and the median was 2.5. The other scale was matter need to check whether listed to use safely this information. On this scale, the average score was 0.72+/-1.22 (maximum 5) and the median was 0. Three engines showed poor correlation between the ranking and the latter score. But several websites on the top indicated 0 score. Fifty-four websites were extracted with one to four engines and the average number of search engines was 1.9. The two scales were positively correlated with the number of search engines, but these correlations were very poor. Ranking high and extraction by multiple search engines were of minor benefit to pick out more reliable information.

Diagnosis blog: checking up on health blogs in the blogosphere

OBJECTIVES

We analyzed the content and characteristics of influential health blogs and bloggers to provide a more thorough understanding of the health blogosphere than was previously available.

METHODS

We identified, through a purposive-snowball approach, 951 health blogs in 2007 and 2008. All blogs were US focused and updated regularly. We described their features, topics, perspectives, and blogger demographics.

RESULTS

Approximately half of the bloggers in our sample were employed in the health field. A majority were female, aged in their 30s, and highly educated. Two thirds posted at least weekly; one quarter accepted advertisements. Most blogs were established after 2004. They typically focused on bloggers' experiences with 1 disease or condition or on the personal experiences of health professionals. Half were written from a professional perspective, one third from a patient-consumer perspective, and a few from the perspective of an unpaid caregiver.

CONCLUSIONS

Data collected from health blogs could be aggregated for large-scale empirical investigations. Future research should assess the quality of the information posted and identify what blog features and elements best reflect adherence to prevailing norms of conduct.

Myelomeningocele Information on the Internet is Accessible and of Variable Quality, and Requires a High Reading Level

PURPOSE

We assessed the accessibility, readability and quality of myelomeningocele information on the Internet.

MATERIALS AND METHODS

We entered the term "spina bifida" into the Google, Yahoo! and Microsoft Network search engines, and stored the first 100 links from each Web site. A total of 164 unique Web sites remained for analysis, of which 159 were classified as relevant. Relevant Web sites were considered to have relevant content if more than 50% of the text was directly relevant to the disease, and to have relevant educational content if more than 50% of the text was aimed at relaying educational information. Readability was assessed using the Flesch-Kincaid Grade Level scale. The quality of the 159 Web sites with relevant content was assessed using American Public Health Association Criteria for Assessing Health Information on the Internet. Six criteria were assessed, namely credibility, content, links, design, interactivity and caveats. Web sites were analyzed by 2 separate medical doctors, with each reviewer blinded to the findings of the other. A weighted kappa statistic was used to calculate interrater reliability.

RESULTS

Of 159 relevant sites 146 (91.8%) had relevant content and 122 (76.7%) had relevant educational content. The average Flesch-Kincaid reading level was 10.9 (range 6 to 12). Quality was assessed on a 3-point scale, with 1 denoting poor quality, 2 fair and 3 good. Average scores were 1.92 for credibility, 1.88 for content, 2.29 for links, 2.53 for design, 1.99 for interactivity and 2.90 for caveats. The weighted kappa statistic for interrater reliability was 0.83.

CONCLUSIONS

Myelomeningocele information on the Internet is relatively accessible, requires a high reading level for comprehension and is of variable quality.

Evaluation of breastfeeding Web sites for patient education

PURPOSE

To evaluate the quality of Web sites on breastfeeding for patient education.

STUDY DESIGN AND METHOD

Descriptive study of 30 Web sites on breastfeeding for patient education, evaluated based on the Health Information Technology Institute (HITI) criteria, readability, and eight content criteria from the American Academy of Pediatrics (AAP) policy statement on breastfeeding.

RESULTS

The mean Flesch-Kincaid Grade Level for readability of the 30 sites was 9.2. Seven of the sites included all eight of the content criteria from the AAP, and three sites did not include any of the information recommended by the AAP content criteria.

CLINICAL IMPLICATIONS

Nurses should be able to recommend best patient education materials for their patients. The five best Web sites for breastfeeding education are identified for patient teaching, and the HITI criteria are explained for nurses to learn how to evaluate Web sites for themselves and their patients.

Genetics and type 1 diabetes: online resources for diabetes educators

PURPOSE

Genetic education Internet sites and peer-reviewed medical literature were reviewed and critiqued to develop tables summarizing online resources for diabetes health professionals.

METHODS

Using Internet search engines, each Web site identified for this project met the following criteria: (1) accurate and valid site content based on widely accepted genetic texts, (2) credibility of the organization that maintained the Web site, (3) ease of navigation, and (4) provision of continuing education credits. PubMed was used to find journal articles using similar criteria.

RESULTS

There were 33 Web sites on genetic education for diabetes health professionals that met the inclusion criteria. The literature search identified 36 articles regarding the importance of genetic education for nurses and other health professionals, as well as information regarding genetics and diabetes.

CONCLUSIONS

Valid and credible information on genetics and type 1 diabetes is available for diabetes health professionals on the Internet and in the medical literature.

Developing a peer review process for web-based curricula: minting a new coin of the realm

The World Wide Web creates new challenges and opportunities for medical educators. Prominent among these are the lack of consistent standards by which to evaluate web-based educational tools. We present the instrument that was used to review web-based innovations in medical education submissions to the 2003 Society of General Internal Medicine (SGIM) national meeting, and discuss the process used by the SGIM web-based clinical curriculum interest group to develop the instrument. The 5 highest-ranked submissions are summarized with commentary from the reviewers.

Evaluation of asthma websites for patient and parent education

The purpose of this study was to evaluate the quality of websites on asthma for patient and parent education. Seventy websites were evaluated based on the Health Information Technology Institute (HITI) criteria; eight core content criteria from the National Heart, Lung, and Blood Institute guidelines; and readability. The ten best websites for asthma education were identified for patient teaching and educating parents about asthma.

Minimization and management of wastes from biomedical research

Several committees were established by the National Association of Physicians for the Environment to investigate and report on various topics at the National Leadership Conference on Biomedical Research and the Environment held at the 1--2 November 1999 at the National Institutes of Health in Bethesda, Maryland. This is the report of the Committee on Minimization and Management of Wastes from Biomedical Research. Biomedical research facilities contribute a small fraction of the total amount of wastes generated in the United States, and the rate of generation appears to be decreasing. Significant reductions in generation of hazardous, radioactive, and mixed wastes have recently been reported, even at facilities with rapidly expanding research programs. Changes in the focus of research, improvements in laboratory techniques, and greater emphasis on waste minimization (volume and toxicity reduction) explain the declining trend in generation. The potential for uncontrolled releases of wastes from biomedical research facilities and adverse impacts on the general environment from these wastes appears to be low. Wastes are subject to numerous regulatory requirements and are contained and managed in a manner protective of the environment. Most biohazardous agents, chemicals, and radionuclides that find significant use in research are not likely to be persistent, bioaccumulative, or toxic if they are released. Today, the primary motivations for the ongoing efforts by facilities to improve minimization and management of wastes are regulatory compliance and avoidance of the high disposal costs and liabilities associated with generation of regulated wastes. The committee concluded that there was no evidence suggesting that the anticipated increases in biomedical research will significantly increase generation of hazardous wastes or have adverse impacts on the general environment. This conclusion assumes the positive, countervailing trends of enhanced pollution prevention efforts by facilities and reductions in waste generation resulting from improvements in research methods will continue.

Quality of breast cancer sites on the World Wide Web

The Internet is a powerful tool for accessing information about complex health topics. The purpose of this study was to evaluate breast cancer Internet sites using published criteria about website structure. Two searches were undertaken (November 1998 and June 1999) using the Yahoo search engine, providing a sample of 136 unique addresses. The results showed 1) owner's credentials were identified in 31.6% of sites, 2) financial charges were stated in 10.3% of sites, 3) less than 14.0% identified site creation date, 4) 33.1% identified content posting update, 5) 30.1% identified information sources, and 6) just under 88% of sites provided e-mail interactivity. The results indicate variability in breast cancer Internet sites with respect to framework criteria of accountability. We suggest that websites that lack fundamental indicators (such as dating and sources) do not provide the user with fundamental information that could enable informed decision making about site quality.

Quality of breast cancer sites on the World Wide Web

The Internet is a powerful tool for accessing information about complex health topics. The purpose of this study was to evaluate breast cancer Internet sites using published criteria about website structure. Two searches were undertaken (November 1998 and June 1999) using the Yahoo search engine, providing a sample of 136 unique addresses. The results showed 1) owner's credentials were identified in 31.6% of sites, 2) financial charges were stated in 10.3% of sites, 3) less than 14.0% identified site creation date, 4) 33.1% identified content posting update, 5) 30.1% identified information sources, and 6) just under 88% of sites provided e-mail interactivity. The results indicate variability in breast cancer Internet sites with respect to framework criteria of accountability. We suggest that websites that lack fundamental indicators (such as dating and sources) do not provide the user with fundamental information that could enable informed decision making about site quality.