Health literacy and the Internet: a study on the readability of Australian online health information

Lessons to be learned when designing comprehensible patient-oriented online information about temporomandibular disorders

BACKGROUND

Temporomandibular disorders (TMD) are a prevalent ailment with a global impact, affecting a substantial number of individuals. While some individuals are receiving treatment from orthodontists for TMD, a significant proportion of individuals obtain knowledge through websites.

OBJECTIVES

Our purpose had been to evaluate, from a patient-oriented perspective, the readability of home pages of websites scored in the 10 most prominent devoted to TMD. We also determined what level of education would have been needed to get an overview of the information on the websites under scrutiny. This approach ensures that our findings are centred on the patient experience, providing insights into how accessible and understandable websites about TMD.

METHODS

We determined the top 10 patient-focused English language websites by searching for 'temporomandibular disorders' in the 'no country redirect' plugin of the Google Chrome browser (www.google.com/ncr). The readability of the texts was assessed using the Gunning fog index (GFI), Coleman Liau index (CLI), Automated readability index (ARI) Simple Measure of Gobbledygook (SMOG), Flesch Kincald grade level (FKGL), Flesh reasing ease (FRE) (https://readabilityformulas.com).

RESULTS

The mean Flesch reading ease index score was determined to be 48.67, accompanied by a standard deviation of 15.04 and these websites require an average of 13.49 years of formal education (GFI), with a standard deviation of 2.62, for ease of understanding.

CONCLUSION

Our research indicates that a significant proportion of websites related to TMD can be defined as a level of complexity that exceeds the ability to read comprehension of the general population.

Evaluation of online text-based information resources of gynaecological cancer symptoms

BACKGROUND

Gynaecological cancer symptoms are often vague and non-specific. Quality health information is central to timely cancer diagnosis and treatment. The aim of this study was to identify and evaluate the quality of online text-based patient information resources regarding gynaecological cancer symptoms.

METHODS

A targeted website search and Google search were conducted to identify health information resources published by the Australian government and non-government health organisations. Resources were classified by topic (gynaecological health, gynaecological cancers, cancer, general health); assessed for reading level (Simple Measure of Gobbledygook, SMOG) and difficulty (Flesch Reading Ease, FRE); understandability and actionability (Patient Education Materials Assessment Tool, PEMAT, 0-100), whereby higher scores indicate better understandability/actionability. Seven criteria were used to assess cultural inclusivity specific for Aboriginal and Torres Strait Islander people; resources which met 3-5 items were deemed to be moderately inclusive and 6+ items as inclusive.

RESULTS

A total of 109 resources were identified and 76% provided information on symptoms in the context of gynaecological cancers. The average readability was equivalent to a grade 10 reading level on the SMOG and classified as 'difficult to read' on the FRE. The mean PEMAT scores were 95% (range 58-100) for understandability and 13% (range 0-80) for actionability. Five resources were evaluated as being moderately culturally inclusive. No resource met all the benchmarks.

CONCLUSIONS

This study highlights the inadequate quality of online resources available on pre-diagnosis gynaecological cancer symptom information. Resources should be revised in line with the recommended standards for readability, understandability and actionability and to meet the needs of a culturally diverse population.

Investigation of the quality and health literacy demand of online information on pelvic floor exercises to reduce stress urinary incontinence

BACKGROUND

Many women are embarrassed to discuss pelvic floor exercises with health professionals during pregnancy; instead they search the internet for information on pelvic floor exercises.

AIMS

This project investigated the quality and health literacy demand in terms of readability, understandability and actionability of information available on the internet relating to pelvic floor exercises.

MATERIALS AND METHODS

An analysis was conducted on 150 websites and 43 videos obtained from three consecutive Google searches relating to pelvic floor exercises. Websites were assessed for Health on the Net certification. Readability was assessed using a readability calculator, and understandability/actionability of website and video information was assessed using the Patient Education Materials Assessment Tool (PEMAT) for Print or Audiovisual Materials.

RESULTS

Less than one-third (24%) of the websites were certified for quality. The median readability score for the websites was grade 9 and the websites and videos both rated highly for understandability and actionability. The median understandability score for the website information was 87%, and the median actionability score was 71%. The median understandability score of the video information was 95.5% and the median actionability score was 100%. The understandability and actionability of video information was also significantly greater than that for website information (P < 0.01).

CONCLUSION

Clinicians could consider directing patients to video-based resources when advising online pelvic floor exercise resources for women during pregnancy, especially those with lower health literacy.

Evaluating the Diagnostic Performance of Symptom Checkers: Clinical Vignette Study

BACKGROUND

Medical self-diagnostic tools (or symptom checkers) are becoming an integral part of digital health and our daily lives, whereby patients are increasingly using them to identify the underlying causes of their symptoms. As such, it is essential to rigorously investigate and comprehensively report the diagnostic performance of symptom checkers using standard clinical and scientific approaches.

OBJECTIVE

This study aims to evaluate and report the accuracies of a few known and new symptom checkers using a standard and transparent methodology, which allows the scientific community to cross-validate and reproduce the reported results, a step much needed in health informatics.

METHODS

We propose a 4-stage experimentation methodology that capitalizes on the standard clinical vignette approach to evaluate 6 symptom checkers. To this end, we developed and peer-reviewed 400 vignettes, each approved by at least 5 out of 7 independent and experienced primary care physicians. To establish a frame of reference and interpret the results of symptom checkers accordingly, we further compared the best-performing symptom checker against 3 primary care physicians with an average experience of 16.6 (SD 9.42) years. To measure accuracy, we used 7 standard metrics, including M1 as a measure of a symptom checker's or a physician's ability to return a vignette's main diagnosis at the top of their differential list, F1-score as a trade-off measure between recall and precision, and Normalized Discounted Cumulative Gain (NDCG) as a measure of a differential list's ranking quality, among others.

RESULTS

The diagnostic accuracies of the 6 tested symptom checkers vary significantly. For instance, the differences in the M1, F1-score, and NDCG results between the best-performing and worst-performing symptom checkers or ranges were 65.3%, 39.2%, and 74.2%, respectively. The same was observed among the participating human physicians, whereby the M1, F1-score, and NDCG ranges were 22.8%, 15.3%, and 21.3%, respectively. When compared against each other, physicians outperformed the best-performing symptom checker by an average of 1.2% using F1-score, whereas the best-performing symptom checker outperformed physicians by averages of 10.2% and 25.1% using M1 and NDCG, respectively.

CONCLUSIONS

The performance variation between symptom checkers is substantial, suggesting that symptom checkers cannot be treated as a single entity. On a different note, the best-performing symptom checker was an artificial intelligence (AI)-based one, shedding light on the promise of AI in improving the diagnostic capabilities of symptom checkers, especially as AI keeps advancing exponentially.

'You are making it sound like you are talking to a child': exploring community sentiment on developing and disseminating tailored sexual health education resources for migrants

Health communication is a critical component of public health, which includes health education resources. Resource effectiveness is improved where health literacy demands, including the cultural appropriateness of resources, match the intended audience. International guidelines support the tailoring of resources for migrants from culturally and linguistically diverse backgrounds (CaLD). Five focus groups (n = 18) and interviews (n = 9) with people from CaLD migrant backgrounds explored community perspectives on sexual health resources developed by a state department of health, specifically clarity, comprehensiveness, cultural appropriateness and strategies for dissemination. We identified three major thematic areas relating to simplicity, cultural norms and beliefs and dissemination. Participants recommended resource delivery in different formats as part of a broader intervention. Generally, sexual health information was deemed appropriate and easily understood. However, the resources used simplified language that participants reported was vague and inaccurate at times, potentially contributing to misinformation and reinforcing stigma relating both to the status of being a migrant and sexual health. Findings suggest the need for more nuanced health resource development beyond translation and language simplification. Resources developed in different formats, including different health literacy demands, using approaches that engage the target group in design and dissemination and contextualized within a comprehensive health promotion project, are likely to be more effective.

Development and evaluation of health education resources for culturally and linguistically diverse populations: a systematic review

People from culturally and linguistically diverse (CaLD) backgrounds in Australia generally experience poorer health outcomes, explained in part by low levels of health literacy. We conducted a systematic review to examine the development and evaluation of health education resources designed for CaLD populations. Five electronic databases were searched for English language, peer-reviewed studies published between 1980 and 2020. Thirty-four studies met the inclusion criteria. Twenty-four different health education resources were described and broadly categorized into four types: media campaigns (n = 10), text-based materials (n = 5), films (n = 8) and radio (n = 1). Studies were assessed against domains adapted from a health literacy guideline incorporating: need, collaboration, audience, health literacy, theory, test and process and impact evaluation. All but one study met the majority of the domains. All studies reported positive evaluation outcomes; this may be due to studies involving community early in resource design and including health literacy considerations in their design. Reporting resource design and evaluation against standard practice controls is recommended to build a more robust evidence base for developing effective health education resources for use by audiences from CaLD backgrounds.

Evaluation of the Accuracy, Credibility, and Readability of Statin-Related Websites: Cross-Sectional Study

BACKGROUND

Cardiovascular disease (CVD) represents the greatest burden of mortality worldwide, and statins are the most commonly prescribed drug in its management. A wealth of information pertaining to statins and their side effects is on the internet; however, to date, no assessment of the accuracy, credibility, and readability of this information has been undertaken.

OBJECTIVE

This study aimed to evaluate the quality (accuracy, credibility, and readability) of websites likely to be visited by the general public undertaking a Google search of the side effects and use of statin medications.

METHODS

Following a Google web search, we reviewed the top 20 consumer-focused websites with statin information. Website accuracy, credibility, and readability were assessed based on website category (commercial, not-for-profit, and media), website rank, and the presence or absence of the Health on the Net Code of Conduct (HONcode) seal. Accuracy and credibility were assessed following the development of checklists (with 20 and 13 items, respectively). Readability was assessed using the Simple Measure of Gobbledegook scores.

RESULTS

Overall, the accuracy score was low (mean 14.35 out of 20). While side effects were comprehensively covered by 18 websites, there was little information about statin use in primary and secondary prevention. None of the websites met all criteria on the credibility checklist (mean 7.8 out of 13). The median Simple Measure of Gobbledegook score was 9.65 (IQR 8.825-10.85), with none of the websites meeting the recommended reading grade of 6, even the media websites. A website bearing the HONcode seal did not mean that the website was more comprehensive or readable.

CONCLUSIONS

The quality of statin-related websites tended to be poor. Although the information contained was accurate, it was not comprehensive and was presented at a reading level that was too difficult for an average reader to fully comprehend. As such, consumers risk being uninformed about this pharmacotherapy.

New Frontiers in Health Literacy: Using ChatGPT to Simplify Health Information for People in the Community

BACKGROUND

Most health information does not meet the health literacy needs of our communities. Writing health information in plain language is time-consuming but the release of tools like ChatGPT may make it easier to produce reliable plain language health information.

OBJECTIVE

To investigate the capacity for ChatGPT to produce plain language versions of health texts.

DESIGN

Observational study of 26 health texts from reputable websites.

METHODS

ChatGPT was prompted to 'rewrite the text for people with low literacy'. Researchers captured three revised versions of each original text.

MAIN MEASURES

Objective health literacy assessment, including Simple Measure of Gobbledygook (SMOG), proportion of the text that contains complex language (%), number of instances of passive voice and subjective ratings of key messages retained (%).

KEY RESULTS

On average, original texts were written at grade 12.8 (SD = 2.2) and revised to grade 11.0 (SD = 1.2), p < 0.001. Original texts were on average 22.8% complex (SD = 7.5%) compared to 14.4% (SD = 5.6%) in revised texts, p < 0.001. Original texts had on average 4.7 instances (SD = 3.2) of passive text compared to 1.7 (SD = 1.2) in revised texts, p < 0.001. On average 80% of key messages were retained (SD = 15.0). The more complex original texts showed more improvements than less complex original texts. For example, when original texts were ≥ grade 13, revised versions improved by an average 3.3 grades (SD = 2.2), p < 0.001. Simpler original texts (< grade 11) improved by an average 0.5 grades (SD = 1.4), p < 0.001.

CONCLUSIONS

This study used multiple objective assessments of health literacy to demonstrate that ChatGPT can simplify health information while retaining most key messages. However, the revised texts typically did not meet health literacy targets for grade reading score, and improvements were marginal for texts that were already relatively simple.

My Wellbeing Journal: Development of a communication and goal-setting tool to improve care for older adults with chronic conditions and multimorbidity

BACKGROUND

Chronic conditions and multimorbidity, the presence of two or more chronic conditions, are increasingly common in older adults. Effective management of chronic conditions and multimorbidity in older adults requires a collaborative and person-centred approach that considers the individual's goals, preferences and priorities. However, ensuring high-quality personalised care for older adults with multimorbidity can be challenging due to the complexity of their care needs, limited time and a lack of patient preparation to discuss their personal goals and preferences with their healthcare team.

OBJECTIVE

To codesign a communication and goal-setting tool, My Wellbeing Journal, to support personalised care planning for older adults with chronic conditions and multimorbidity.

DESIGN

We drew on an experience-based codesign approach to develop My Wellbeing Journal. This article reports on the final end-user feedback, which was collected via an online survey with older adults and their carers.

SETTING AND PARTICIPANTS

Older adults with chronic conditions, multimorbidity and informal carers living in Australia. Personalised care planning was considered in the context of primary care.

RESULTS

A total of 88 participants completed the online survey. The survey focused on participants' feedback on the tool in terms of effectiveness, efficiency, satisfaction and errors encountered. This feedback resulted in modifications to My Wellbeing Journal, which can be used during clinical encounters to facilitate communication, goal setting and progress tracking.

DISCUSSION AND CONCLUSIONS

Clinicians and carers can use the tool to guide discussions with older adults about their care planning and help them set realistic goals that are meaningful to them. The findings of this study could be used to inform the development of recommendations for healthcare providers to implement person-centred, goal-oriented care for older adults with chronic conditions and multimorbidity.

PATIENT OR PUBLIC CONTRIBUTION

Older adults living with chronic conditions and multimorbidity and their carers have contributed to the development of a tool that has the potential to significantly enhance the experience of personalised care planning. Their direct involvement as collaborators has ensured that the tool is optimised to meet the standards of effectiveness and usability.

Scientific clickbait: Examining media coverage and readability in genome-wide association research

In the present study, we analyzed a large corpus of English-language online media articles covering genome-wide association studies (GWAS), exemplifying the use of computational methods to study science communication in biological sciences. We analyzed trends in media coverage, readability, themes, and mentions of ethical and social issues, in over 5,000 websites published from 2005 to 2018 from 3,555 GWAS publications on 1,943 different traits, identified via GWAS Catalog using a text-mining approach to inform the discussion about genetic literacy and media coverage. We found that 22.9% of GWAS papers received media attention but most were described in language too complex to be understood by the public. Ethical issues are rarely mentioned and mentions of translation are increasing over time. We predicted media attention based on year of publication, number of genetic associations identified, study sample size, and journal impact factor, using a regression model (r2 = 38.7%). We found that chronotype, educational attainment, alcohol and coffee consumption, sexual orientation, tanning, and hair color received substantially more attention than predicted by the regression model. We also evaluated the prevalence of the clickbait "one gene, one disease" headlines (e.g., "Scientists Say They've Found Gene That Causes Breast Cancer") and found that it is declining. In sum, online media coverage of GWAS should be more accessible, introduce more modern genetics terms, and when appropriate, ELSI should be mentioned. Science communication research can benefit from big data and text-mining techniques which allow us to study trends and changes in coverage trends across thousands of media outlets. Results can be explored interactively in a website we have built for this manuscript: https://jjmorosoli.shinyapps.io/newas/.

Barriers and enablers to accessing child health resources and services: Findings from qualitative interviews with Arabic and Mongolian immigrant mothers in Australia

BACKGROUND

Over the past two decades, there has been an increase of immigrants in Australia. Despite this, the availability of culturally responsive resources and services that cater to their needs remains insufficient.

OBJECTIVE

The aim of this study was to explore the resources used and trusted by Mongolian- and Arabic-speaking migrant mothers in Australia for child health information and examine how they navigate and overcome challenges they encounter accessing this information.

DESIGN

Semi-structured telephone interview.

METHODS

A theory informed semi-structured 60-min telephone interview was conducted in Arabic and Mongolian with 20 Arabic- and 20 Mongolian-speaking migrant mothers of children younger than 2 years or currently pregnant and living in Australia. Data were analysed thematically using the framework method.

RESULTS

The reliance on digital platforms such as google emerged as a common trend among both groups of mothers when seeking child health information. Notably, there were differences in resources selection, with Mongolian mothers showing a preference for Australian-based websites, while Arabic-speaking mothers tended to opt for culturally familiar resources. There were various barriers that hindered their access to health services and resources, including language barriers, cost, and limited knowledge or familiarity with their existence. Negative encounters with healthcare professionals contributed to a perception among many mothers that they were unhelpful. Both groups of mothers employed a cross-checking approach across multiple websites to verify trustworthiness of information. Acculturation was shown only among the Mongolian-speaking mothers who adapted their cultural practices in line with their country of residence.

CONCLUSION

The findings of this study highlight the importance of addressing the needs of migrant mothers in accessing child health information. Health professionals, government agencies, and researchers have an opportunity to provide culturally responsive support by fostering a culturally inclusive approach to developing and promoting equitable access to services and resources, ultimately enhancing the wellbeing of migrant families.

The Evaluation of the Suitability, Quality, and Readability of Publicly Available Online Resources for the Self-Management of Fear of Cancer Recurrence

Cancer survivors often rely on the internet for health information, which has varying levels of readability, suitability, and quality. There is a need for high-quality online self-management resources for cancer survivors with fear of cancer recurrence (FCR). This study evaluated the readability, suitability, and quality of publicly available online FCR self-management resources. A Google search using FCR-related keywords identified freely available FCR self-management resources for cancer survivors in English. Resource readability (reading grade level), suitability, and quality were evaluated using relevant assessment tools. Descriptive statistics and cluster analysis identified resources with higher suitability and quality scores. Mean resource (n = 23) readability score was grade 11 (SD = 1.6, Range = 9-14). The mean suitability score was 56.0% (SD = 11.4%, Range = 31.0-76.3%), indicating average suitability and the mean quality score was 53% (SD = 11.7%, Range = 27-80%), indicating fair quality. A cluster of 15 (65%) resources with higher suitability and quality scores was identified. There were no significant associations between suitability or quality scores and the type of organisation that published the resources. Online FCR self-management resources varied in readability, suitability and quality. Resources with higher quality and suitability scores relative to other resources are identified for use by healthcare professionals and cancer survivors. Resources that are more culturally appropriate, with lower reading grade levels and detailed self-management strategies are needed.

A self-directed digital exercise program for hip osteoarthritis ("My Hip Exercise"): protocol for a randomised controlled trial

BACKGROUND

Hip osteoarthritis (OA) is a leading global cause of chronic pain and disability. Given there is no cure for OA, patient self management is vital with education and exercise being core recommended treatments. However, there is under-utilisation of these treatments due to a range of clinician and patient factors. Innovative service models that increase patient accessibility to such treatments and provide support to engage are needed. This study primarily aims to determine the effects of a self-directed digital exercise intervention comprising online education and exercise supported by a mobile app to facilitate adherence on the primary outcomes of changes in hip pain during walking and patient-reported physical function at 24-weeks when compared to online education control for people with hip OA.

METHODS

We will conduct a two-arm, superiority parallel-design, randomised controlled trial involving 182 community volunteers aged 45 years and over, with painful hip OA. After completing the baseline assessment, participants will be randomly assigned to either: i) digital exercise intervention; or ii) digital education (control). Participants randomised to the intervention group will have access to a website that provides information about hip OA and its management, advice about increasing their physical activity levels, a 24-week lower limb strength exercise program to be undertaken at home three times per week, and a mobile app to reinforce home exercise program adherence. Participants in the control group will have access to a website containing only information about hip OA and its management. All participants will be reassessed at 24 weeks after randomisation. Primary outcomes are severity of hip pain while walking using an 11-point numeric rating scale and physical function using the Western Ontario and McMaster Universities Osteoarthritis Index subscale. Secondary outcomes are the Hip dysfunction and Osteoarthritis Outcome Score subscales of pain, hip-related quality of life, and function, sports and recreational activities; global change in hip condition; health-related quality of life; measures of physical activity levels; fear of movement; self efficacy for pain and for exercise; and use of oral pain medications.

DISCUSSION

Innovative and scalable approaches to OA education, physical activity, and exercise are required in order to improve exercise participation/engagement and mitigate physical inactivity in the hip OA population. This will help minimise the burden of this major public health issue on individuals and society.

TRIAL REGISTRATION

Australia New Zealand Clinical Trials Registry (ACTRN12622001533785).

"We are in the forgotten corner!" a qualitative study of experiences and challenges among Chinese older women at the onset of acute myocardial infarction

Background

Acute myocardial infarction (AMI) is a common and serious cardiovascular disease (CVD) that is one of the leading causes of death among women globally and in China. However, there are sex-associated differences and inequalities in the detection and management of AMI, especially in older people. There is little research demonstrating how challenges and barriers affect older women's help-seeking behavior and health-related procedures in China.

Purpose

The objective of this study was to explore the experiences of older women with AMI, focusing on their perception, challenges, and coping strategies at the onset of AMI in Wuhan, China.

Methods

This study utilized a qualitative research design approach and conducted semi-structured, in-depth, and audio-recorded interviews with 18 women aged 65-84 years, purposively selected from two tertiary hospitals in Wuhan City from November 2021 to April 2022.

Results

Interpretative Phenomenological Analysis (IPA) was used in this study to analyze the data on 18 participants and three major themes were generated: disease perception disorder, negative coping strategies, and barriers due to social-environmental contexts.

Conclusion

To reduce older women's delay in seeking help, healthcare professionals should provide public health education that emphasizes sex-related disparities, and age-specific knowledge-attitude aspects to high-risk groups. Policy-based and health administration recommendations, including e-health information support, access to care, and social-environmental factors, should be highlighted to promote women's health behavior.

Healthy eating blog readership: A cross-sectional survey in Australian adults

AIMS

To investigate whether members of the public read blogs for the purpose of accessing healthy eating information; examine demographic predictors of healthy eating blog readership, specifically education, gender, age, body mass index, and residential location; and explore the reasons for reading, and not reading, healthy eating blogs.

METHODS

This study used a cross-sectional online self-reported survey design collected over three time points (round 1: December 2017-March 2018, round 2: August 2018-December 2018, round 3: December 2021-March 2022). The total sample of participants comprised of 238 respondents with a mean age of 46 years old, who mostly reported gender as female (82%), being educated with a university degree (69%), and predominantly resided in urban and city areas (84%).

RESULTS

Fifty-one percent of respondents reported reading healthy eating blogs, suggesting that consumers were proactively seeking healthy eating information through this avenue. Participants who identified as female were 3.2 times more likely to read healthy eating blogs. Commonly, healthy eating blogs were read to receive practical information that aligned with current food choices. The main reason participants reported not reading healthy eating blogs was not thinking about using them (29%).

CONCLUSIONS

Understanding who is seeking healthy eating information through blogs, and their reasons doing so, is important to continue research into the potential effectiveness of blogs as a platform to communicate healthy eating and nutrition messages. This study provides direction for further investigation into how dietetics professionals could effectively use blogs to disseminate healthy eating information and positively influence consumer food choices and dietary intake.

How socioeconomically disadvantaged people access, understand, appraise, and apply health information: A qualitative study exploring health literacy skills

OBJECTIVES

Health literacy, or a person's competence to access, understand, appraise and apply health information, can be considered a mediating factor between socioeconomic characteristics and health disparities. Socioeconomically disadvantaged people in particular present with less health literacy skills. To develop targeted interventions tailored to their real needs, it is important to understand how they function and what difficulties they encounter when dealing with health information. The purpose of this study was to explore their experiences when accessing, understanding, appraising, and applying health information in their everyday lives.

METHODS

Semi-structured face-to-face interviews were conducted with 12 socioeconomically disadvantaged adults living in the community in Switzerland (age range: 44-60 years old).

RESULTS

Thematic analysis of the interviews yielded four themes, describing the health literacy processes of participants, related barriers, and compensatory strategies used: Financial insecurity triggers the need for health information; Pathway 1: Physicians as ideal (but expensive) interlocutors; Pathway 2: The internet as a suboptimal alternative; and Pathway 3: Relatives as a default resource. The progression of socioeconomically disadvantaged people in the health literacy process is like an 'obstacle course', with numerous steps taken backwards before they can develop compensatory strategies to overcome the barriers to obtaining health information.

CONCLUSIONS

Financial deprivation seems to be the most important factor contributing to health literacy barriers. Appraising health information is the health literacy skill with which socioeconomically disadvantaged people struggle the most. Physician-based, individual skills-based, organizational, and policy-based interventions are needed to help them overcome their health literacy challenges.

Accessing publicly funded donor insemination treatment in the UK: is funding information available on fertility clinic websites?

We sought to find out if information about public funding for regulated donor insemination (DI) was available on UK fertility clinic websites, and if so, what information was provided for same-sex couples and single women; and if the available information was easily readable. The 'Choose a fertility clinic' pages of the Human Fertilisation and Embryology Authority (HFEA) website were used to identify all licensed fertility clinics in the UK, and any available text on public funding for DI treatment was extracted. The Flesch reading ease scores were calculated to determine the readability of the extracted text. Of the 52 clinics included in the synthesis, 23 mentioned public funding, and for 16 of these, it was unclear whether public funding was available. Six of the 23 clinics mentioned public funding for same-sex couples, and two mentioned public funding for single women. The Flesch reading ease scores indicated that none of the text about funding for DI treatment on any of the clinic websites met the NHS-advised level of readability for health information. Fertility clinic websites should specify whether they offer publicly funded treatment, and to whom, as well as clearly stating the eligibility criteria, using suitably readable language to communicate this.

Integrating consumer perspectives into a large-scale health literacy audit of health information materials: learnings and next steps

BACKGROUND

Health information is less effective when it does not meet the health literacy needs of its consumers. For health organisations, assessing the appropriateness of their existing health information resources is a key step to addressing this issue. This study describes novel methods for a consumer-centred large-scale health literacy audit of existing resources and reflects on opportunities to further refine the method.

METHODS

This audit focused on resources developed by NPS MedicineWise, an Australian not-for-profit that promotes safe and informed use of medicines. The audit comprised 4 stages, with consumers engaged at each stage: 1) Select a sample of resources for assessment; 2) Assess the sample using subjective (Patient Education Materials Assessment Tool) and objective (Sydney Health Literacy Lab Health Literacy Editor) assessment tools; 3) Review audit findings through workshops and identify priority areas for future work; 4) Reflect and gather feedback on the audit process via interviews.

RESULTS

Of 147 resources, consumers selected 49 for detailed assessment that covered a range of health topics, health literacy skills, and formats, and which had varied web usage. Overall, 42 resources (85.7%) were assessed as easy to understand, but only 26 (53.1%) as easy to act on. A typical text was written at a grade 12 reading level and used the passive voice 6 times. About one in five words in a typical text were considered complex (19%). Workshops identified three key areas for action: make resources easier to understand and act on; consider the readers' context, needs, and skills; and improve inclusiveness and representation. Interviews with workshop attendees highlighted that audit methods could be further improved by setting clear expectations about the project rationale, objectives, and consumer roles; providing consumers with a simpler subjective health literacy assessment tool, and addressing issues related to diverse representation.

CONCLUSIONS

This audit yielded valuable consumer-centred priorities for improving organisational health literacy with regards to updating a large existing database of health information resources. We also identified important opportunities to further refine the process. Study findings provide valuable practical insights that can inform organisational health actions for the upcoming Australian National Health Literacy Strategy.

Assessment of Reliability and Quality of Videos on Medial Epicondylitis Shared on YouTube

Introduction Video-sharing websites, especially YouTube search engine, have become popular sources for obtaining health information over the years. The reliability and quality of contents on YouTube are unpredictable and questionable. The purpose of this study was to evaluate the reliability and quality of videos on medial epicondylitis shared on the YouTube platform. Methods A YouTube search with the keywords "medial epicondylitis" and "golfer's elbow" was conducted. After typing each keyword, the first 50 videos that appeared in the first three pages were evaluated. The titles and duration of the videos, the uploading sources, the time passed since the upload, number of total views, likes, dislikes and comments were recorded. All videos were analyzed and scored using the DISCERN scoring system, modified DISCERN (mod-DISCERN) scale, and the Global Quality Score (GQS). Results Eighty-eight videos met the inclusion criteria and were included in the study. The mean DISCERN score, mod-DISCERN score and GQS were 40.68±10.5, 1.81±0.76, and 2.72±0.9, respectively. Most of the videos had poor and very poor quality content according to the DISCERN instrument, the mod-DISCERN scale and GQS tool. The main upload sources were doctors (36.4%), physiotherapists (25%), patients (2.2%), and other (36.4%). The videos uploaded by doctors had higher quality scores than other uploading sources. Of the 88 videos, 10 were about diagnosis alone, 39 were on treatment alone, and 39 contained information about both of diagnosis and treatment. The mean DISCERN score, mod-DISCERN score and GQS of the videos on "both" were significantly higher than those "diagnosis alone" and "treatment alone" groups. Conclusion Nearly 80% of YouTube videos on medial epicondylitis according to the DISCERN and GQS tool, and also 97.7% of videos according to the mod-DISCERN scale had medium or poor quality. YouTube videos on medial epicondylitis could not be considered as accurate and reliable sources. Because the videos uploaded by doctors had higher quality scores, the physicians should prepare and upload more reliable and quality contents with detailed information on YouTube.

Patient Information Resources on Cardiovascular Health After Cancer Treatment: An Audit of Australian Resources

PURPOSE

Up to one third of patients with cancer are thought to experience adverse cardiovascular events after their cancer diagnosis and treatment. High-quality information about cancer treatment-related cardiovascular disease can prepare patients and reduce anxiety. The aim of this project was to systematically identify Australian online information resources about cardiovascular health after cancer and assess the readability, understandability, actionability, and cultural relevance for Aboriginal and Torres Strait Islander patients.

METHODS

We conducted systematic Google and website searches to identify potentially relevant resources. Eligibility was assessed using predefined criteria. For each eligible resource, we summarized the content and assessed readability, understandability, actionability, and cultural relevance for Aboriginal and Torres Strait Islander people.

RESULTS

Seventeen online resources addressing cardiovascular health after cancer were identified: three focused solely on cardiovascular health and the remaining 14 dedicated between <1% and 48% of the word count to this topic. On average, three of 12 predefined content areas were covered by the resources. Only one resource was considered comprehensive, covering eight of 12 content areas. Overall, 18% of the resources were deemed readable for the average Australian adult, 41% deemed understandable, and only 24% had moderate actionability. None of the resources were considered culturally relevant for Aboriginal and Torres Strait Islander people, with 41% addressing only one of the seven possible criteria and the remainder addressing none of the criteria.

CONCLUSION

This audit confirms a gap in online information resources about cardiovascular health after cancer. New resources, especially for Aboriginal and Torres Strait Islander people, are needed. The development of such resources must be done through involvement and collaboration with Aboriginal and Torres Strait Islander patients, families, and carers, through a codesign process.

Dr Google - assessing the reliability and readability of information on general surgical procedures found via search engines

BACKGROUND

The most common general surgical emergency operations are laparoscopic appendicectomy, laparoscopic cholecystectomy, hernia repair, hemorrhoidectomy and colectomy. Patients commonly perform an internet search for more information prior to undergoing surgery, which can lead to an inappropriate understanding of their procedure. The aim is to assess the quality of information available on three of the most used search engines.

METHODS

A search was conducted on Google.com, Bing.com and Yahoo.com using the terms related to laparoscopic appendicectomy, laparoscopic cholecystectomy, hemorrhoidectomy, hernia repair and colectomy. First 20 results from each search engine were collected for evaluation. Results were excluded if they were sponsored, duplicates, academic publications, advertisements, forums, audiovisual tools, social media or any non-English information. Included results were assessed for reliability using DISCERN and JAMA benchmark score. Readability was assessed using Flesch Reading Ease (FRE) Score and Simple Measure of Gobbledygook (SMOG).

RESULTS

Hundred and ninety-seven websites were analysed, 44.7% were published by institutions, 34.5% by health websites and 20.8% by independent surgeons. Mean DISCERN scores for Institutions was 54.6 ± 11.3, independent surgeons 45.9 ± 11.4 and health websites 58.7 ± 10.3. Mean JAMA score for Institutions was 1.0 ± 1.0, independent surgeons 0.1 ± 0.4 and health websites 1.7 ± 1.1. FRE scores for institutions was 51.6 ± 10.3, independent surgeons 40.9 ± 10.2, and health websites 45.7 ± 12.3. SMOG scores were 9.8 ± 1.5 for institutions, 11.4 ± 1.6 for independent surgeons and 10.6 ± 1.7 for health websites.

CONCLUSION

Health information on common general surgical procedures found on search engines are generally fair to good quality but still above the suggested reading level of the population. Information on surgical procedures should be written at recommended reading level of 13-14 years old.

Multiple Automated Health Literacy Assessments of Written Health Information: Development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor v1

Producing health information that people can easily understand is challenging and time-consuming. Existing guidance is often subjective and lacks specificity. With advances in software that reads and analyzes text, there is an opportunity to develop tools that provide objective, specific, and automated guidance on the complexity of health information. This paper outlines the development of the SHeLL (Sydney Health Literacy Lab) Health Literacy Editor, an automated tool to facilitate the implementation of health literacy guidelines for the production of easy-to-read written health information. Target users were any person or organization that develops consumer-facing education materials, with or without prior experience with health literacy concepts. Anticipated users included health professionals, staff, and government and nongovernment agencies. To develop this tool, existing health literacy and relevant writing guidelines were collated. Items amenable to programmable automated assessment were incorporated into the Editor. A set of natural language processing methods were also adapted for use in the SHeLL Editor, though the approach was primarily procedural (rule-based). As a result of this process, the Editor comprises 6 assessments: readability (school grade reading score calculated using the Simple Measure of Gobbledygook (SMOG)), complex language (percentage of the text that contains public health thesaurus entries, words that are uncommon in English, or acronyms), passive voice, text structure (eg, use of long paragraphs), lexical density and diversity, and person-centered language. These are presented as global scores, with additional, more specific feedback flagged in the text itself. Feedback is provided in real-time so that users can iteratively revise and improve the text. The design also includes a "text preparation" mode, which allows users to quickly make adjustments to ensure accurate calculation of readability. A hierarchy of assessments also helps users prioritize the most important feedback. Lastly, the Editor has a function that exports the analysis and revised text. The SHeLL Health Literacy Editor is a new tool that can help improve the quality and safety of written health information. It provides objective, immediate feedback on a range of factors, complementing readability with other less widely used but important objective assessments such as complex and person-centered language. It can be used as a scalable intervention to support the uptake of health literacy guidelines by health services and providers of health information. This early prototype can be further refined by expanding the thesaurus and leveraging new machine learning methods for assessing the complexity of the written text. User-testing with health professionals is needed before evaluating the Editor's ability to improve the health literacy of written health information and evaluating its implementation into existing Australian health services.

A scoping review on the use and usefulness of online symptom checkers and triage systems: How to proceed?

Background

Patients are increasingly turning to the Internet for health information. Numerous online symptom checkers and digital triage tools are currently available to the general public in an effort to meet this need, simultaneously acting as a demand management strategy to aid the overburdened health care system. The implementation of these services requires an evidence-based approach, warranting a review of the available literature on this rapidly evolving topic.

Objective

This scoping review aims to provide an overview of the current state of the art and identify research gaps through an analysis of the strengths and weaknesses of the presently available literature.

Methods

A systematic search strategy was formed and applied to six databases: Cochrane library, NICE, DARE, NIHR, Pubmed, and Web of Science. Data extraction was performed by two researchers according to a pre-established data charting methodology allowing for a thematic analysis of the results.

Results

A total of 10,250 articles were identified, and 28 publications were found eligible for inclusion. Users of these tools are often younger, female, more highly educated and technologically literate, potentially impacting digital divide and health equity. Triage algorithms remain risk-averse, which causes challenges for their accuracy. Recent evolutions in algorithms have varying degrees of success. Results on impact are highly variable, with potential effects on demand, accessibility of care, health literacy and syndromic surveillance. Both patients and healthcare providers are generally positive about the technology and seem amenable to the advice given, but there are still improvements to be made toward a more patient-centered approach. The significant heterogeneity across studies and triage systems remains the primary challenge for the field, limiting transferability of findings.

Conclusion

Current evidence included in this review is characterized by significant variability in study design and outcomes, highlighting the significant challenges for future research.An evolution toward more homogeneous methodologies, studies tailored to the intended setting, regulation and standardization of evaluations, and a patient-centered approach could benefit the field.

Online public information about advance care planning: An evaluation of UK and international websites

Introduction

Healthcare information is increasingly internet-based. Standards require websites to be 'perceivable, operable, understandable and robust' with relevant content for citizens in appropriate language. This study examined UK and international websites offering public healthcare information on advance care planning (ACP) using current recommendations for website accessibility and content and informed by a public engagement exercise.

Methods

Google searches identified websites in English from health service providers, governmental or third sector organisations based in the UK and internationally. Target keywords that would be used by a member of the public informed the search terms. Data extraction used criterion-based assessment and web content analysis of the first two pages of each search result. Public patient representatives as key members of the multidisciplinary research team guided the development of the evaluation criteria.

Results

A total of 1158 online searches identified 89 websites, reduced to 29 by inclusion/exclusion criteria. Most sites met international recommendations for knowledge/understanding about ACP. Differences in terminology, lack of information about ACP limitations and non-adherence to recommended reading levels, accessibility standards and translation options were apparent. Sites targeting members of the public used more positive, non-technical language than those for both professional and lay users.

Conclusions

Some websites met accepted standards required to facilitate understanding and public engagement in ACP. Others could be improved significantly. Website providers have important roles and responsibilities in increasing people's understanding of their health conditions, future care options and ability to take an active role in planning for their health and care.

Assessment of the quality of health information on the Internet: evidence-based accuracy indicators for tuberculosis

ABSTRACT Not long ago, someone had to buy a newspaper, a book, or a magazine or go to a library to obtain information. Today, the Internet quickly facilitates a myriad of information. However, the information provided may be obsolete, incomplete, incorrect, or deliberately false: fake news. In the health field, this information can affect well-being or harm individuals and society. Thus, professionals, researchers, and institutions have assessed the quality of information on health websites to address this issue. Evaluations often verify the accuracy of the information provided. However, the information accuracy indicators have yet to be constructed from Evidence-Based Medicine (EBM). This article aims to build indicators from EBM practices, analyzing the case of tuberculosis. This manuscript proposes 43 information accuracy indicators that evaluated the tuberculosis information available on the Brazilian Ministry of Health. The results indicate that much information needs to be included, and some data must be corrected. This evaluation reiterates the importance of building EBM accuracy indicators. This work intends to encourage new studies about assessing the quality of health information on the Internet.

Avaliação da qualidade da informação de saúde na internet: indicadores de acurácia baseados em evidência para tuberculose

RESUMO Pouco tempo atrás, para alguém obter informação, era preciso comprar um jornal, um livro, uma revista ou ir até uma biblioteca. Hoje, a internet disponibiliza uma miríade de informação rapidamente. Entretanto, as informações veiculadas podem estar desatualizadas, incompletas, incorretas ou deliberadamente mentirosas: as fakenews. Na saúde, essas informações podem afetar o bem-estar ou causar dano ao indivíduo e à sociedade. Para enfrentar esse problema, avaliações da qualidade da informação de sites de saúde têm sido realizadas por profissionais, pesquisadores e instituições. As avaliações verificam frequentemente a exatidão da informação oferecida. Contudo, os indicadores de acurácia da informação não têm sido construídos a partir da Medicina Baseada em Evidências (MBE). O objetivo desse artigo é construir indicadores a partir das práticas da MBE, analisando o caso da tuberculose. O artigo propõe 43 indicadores de acurácia da informação. Com eles, foi avaliada a informação disponível sobre tuberculose no site do Ministério da Saúde do Brasil. Os resultados indicam que falta muita informação e há informação incorreta. Essa avaliação reitera a importância da construção de indicadores de acurácia da informação a partir da MBE. Este trabalho pretende incentivar a realização de novos estudos sobre avaliação da qualidade da informação de saúde na internet.

eHealth Literacy of Australian Undergraduate Health Profession Students: A Descriptive Study

Rapid growth in digital health technologies has increased demand for eHealth literacy of all stakeholders within health and social care environments. The digital future of health care services requires the next generation of health professionals to be well-prepared to confidently provide high-quality and safe health care. The aim of this study was to explore the eHealth literacy of undergraduate health profession students to inform undergraduate curriculum development to promote work-readiness. A cross-sectional survey was undertaken at an Australian university using the seven-domain eHealth Literacy Questionnaire (eHLQ), with 610 students participating. A one-way Multivariate Analysis of Variance (MANOVA) with follow-up univariate analysis (ANOVA) was used to determine if there were differences in eHLQ scores across 11 sociodemographic variables. Students generally had good knowledge of health (Scale 2); however, they had concerns over the security of online health data (Scale 4). There were also significant differences in age and ownership of digital devices. Students who were younger reported higher scores across all seven eHLQ scales than older students. This research provided an understanding of eHealth literacy of health profession students and revealed sub-groups that have lower eHealth literacy, suggesting that digital health skills should be integrated into university curriculums, especially related to practice-based digital applications with special focus to address privacy and security concerns. Preparation of health profession students so they can efficiently address their own needs, and the needs of others, is recommended to minimise the digital divide within health and social care environments.

Comprehensiveness of infant formula and bottle feeding resources: A review of information from Australian healthcare organisations

The use of infant formula is widespread internationally. In Australia, 55% of infants receive formula before 6 months of age, with higher rates among disadvantaged communities. Infant formula use can contribute to childhood overweight and obesity, through formula composition and feeding behaviours, such as adding cereal to bottles and parental feeding style. While information abounds to promote and support breastfeeding, formula‐feeding parents report a paucity of advice and support; many rely on formula packaging for information. This study systematically searched and reviewed online resources for infant formula and bottle feeding from Australian governments, health services, hospitals, and not‐for‐profit parenting organisations. A comprehensive search strategy located 74 current resources, mostly for parents. Researchers evaluated the resources against best practice criteria derived from Australian government and UNICEF guidelines on six topics. They assessed how comprehensively the resources addressed each topic and whether the resources provided all the information necessary for parents to understand each topic. The mean ‘comprehensiveness’ rating for topics across all resources was 54.36%. However, some topics were addressed more fully than others. Information on ‘discussing infant formula with health workers’ and on ‘preparing infant formula’ was more frequently accurate and comprehensive. However, there was much less comprehensive information on ‘using infant formula’, including amounts of formula to feed, use of bottle teats, appropriate bottle‐feeding practice and responsiveness to infant satiety cues. Over half the resources were written at an acceptable reading level.

Insufficient formula‐feeding support from health services can lead to parents relying on formula packaging or other commercial information.

Most formula‐feeding resources from Australian healthcare organisations focus on preparing infant formula and using infant formula. However, comprehensiveness of information varies—on average, information on preparing infant formula was more comprehensive than that on using infant formula.

Many resources do not address responsive bottle‐feeding practices, which impacts risk of infant overweight and obesity.

The Australian government infant feeding guidelines should be revised to include more comprehensive best‐practice formula‐feeding recommendations; subsequently, this should be reflected in formula‐feeding information from health organisations.

'They should come forward with the information': menopause-related health literacy and health care experiences among Vietnamese-born women in Melbourne, Australia

Objectives: Health literacy refers to an individual's capacity to access, understand, evaluate and use health information to make well informed health-related decision to maintain and promote optimal health. Low health literacy is linked with worse health outcomes and is more common in people from socio-economically disadvantaged backgrounds and from culturally and linguistically diverse backgrounds and among people with limited education. Peri-menopausal and postmenopausal health behaviour predicts health in later life. This qualitative study was conducted in Melbourne, Australia. The aim of this study was to explore menopause-related health literacy and experiences with menopause-related health care among Vietnamese-born women who had immigrated to Australia as adults.Design: A qualitative study using semi-structured interviews was conducted with women aged between 45 and 60 years and who were either in the peri or postmenopausal phase. Transcripts were analysed thematically.Results: A total of 12 women were interviewed. Participants viewed menopause as a natural event and obtained most of their menopause-related information from family and friends. Limited English language proficiency affected their capacity to access, understand, evaluate and use menopause-related health information. They identified their Vietnamese speaking General Practitioners (GPs) as a reliable source of health information, but 'shyness' prevented them from asking questions about menopause and they suggested that GPs need to initiate menopause-related health conversations.Conclusion: Low menopause-related health literacy among Vietnamese-born immigrant women may limit their opportunities to access information about and benefit from menopause-related health-promoting behaviours. Access to menopause-related health information in relevant community languages is essential to support immigrant women to make well informed menopause-related health decisions.

Evaluation of the Accuracy and Quality of Information in Videos About Lateral Epicondylitis Shared on Internet Video Sharing Services

Purpose

In this study, it was aimed to determine the quality and accuracy of the videos on YouTube about lateral epicondylitis.

Methods

The first 100 videos were included in the study by typing the keyword "lateral epicondylitis" in the YouTube search tab without using any filters. The video power index (VPI) was used to evaluate the popularity of the videos, and the global quality score (GQS), Journal of the American Medical Association (JAMA), and DISCERN scoring systems were used to evaluate the quality. The obtained data were statistically analyzed according to these scoring systems.

Results

The mean DISCERN, JAMA, and GQS of the analyzed videos were 46.66, 3.13, and 3.85, respectively. According to these results, it was determined that the videos were of medium quality. A statistically insignificant and weak correlation was found between the VPI and DISCERN, GQS, and JAMA scores (p>0.05, intraclass correlation coefficient, ICC: −0.05, 0.09, and −0.05, respectively). While there was no significant relationship between the video source and the DISCERN, JAMA, and GQS scores (p>0.05), it was determined that the DISCERN, JAMA, and GQS scores in the exercise videos were significantly higher than in the other content types in terms of the video content (p=0.041).

Conclusions

According to the results obtained, it was determined that YouTube videos about lateral epicondylitis were not of sufficient quality. In order to ensure standardization for quality videos, internationally acceptable guidelines should be determined and studies should be carried out to provide an adequate infrastructure for the preparation of quality medical videos that can meet the increasing needs of patients by health institutions.

Accessibility of web-based health information for women in midlife from culturally and linguistically diverse backgrounds or with low health literacy

OBJECTIVE

To measure the accessibility of Australian web-based health information for midlife women including those from culturally and linguistically diverse (CALD) backgrounds or with low health literacy.

METHODS

Search terms relating to midlife health were entered into Google Australia to identify health information websites. The content of the first two results pages was assessed using the European Commission's quality criteria for health websites. Readability was assessed using the Flesch Readability Ease Score with Grade 8 accepted as the average Australian reading level.

RESULTS

Sixteen websites were evaluated. Accessibility scores ranged between 0 and 8. The Victorian Government's health website Better Health Channel and the Jean Hailes for Women's Health website contained the most accessible information, each scoring 8, but were both 'difficult to read' on the readability test. Four websites included written resources in languages other than English and two had information in audio-visual format in languages other than English.

CONCLUSIONS

There is a gap in accessible online health information for Australian women from CALD backgrounds or those with low health literacy. Implications for public health: Healthy behaviour changes in midlife may lead to better health in older age. More accessible health information resources are needed for women in midlife from CALD backgrounds and those with low health literacy.

Readability and Comprehension of Printed Patient Education Materials

Background: Health literacy, a recently determined construct plays an important role in how individuals are able to manage their health. A useful approach for the assessment of health literacy is to measure the comprehension of available patient education materials (PEMs).

Objective: We aimed at assessing the usefulness of PEMS available in Hungarian by testing comprehension of selected PEMs in different groups of users.

Methods: Comprehension of patient education materials in the domain of healthcare was tested by selecting PEMs and creating questions based on their text in 3 dimensions of health literacy: understand, process/appraise, apply/use. Twenty questions were created that could be answered without pre-existing knowledge by reading the appropriate text taken from PEMs. Comprehension was examined in four groups: laypersons, non-professional healthcare workers, 1st year healthcare students, and 5th year medical students. Readability indices were calculated for the same texts to which questions were created.

Results: Laypersons answered <50% of the PEMs-based questions correctly. Non-professional healthcare workers performed better with 57% of right answers but significantly worse than healthcare students or medical students. Those with at least high school qualification (maturity exam) showed significantly higher comprehension compared to those with lower educational attainment. Persons in good or very good health also had significantly better comprehension than those in less favorable health. All readability indices showed that comprehension of the tested PEMs required at least 10 years of schooling or more. Therefore, these PEMS are difficult to understand for persons with less than high school level of education.

Conclusion: Rephrasing of the investigated patient educational materials would be recommended so that they better fit the educational attainment of the Hungarian population. Evaluation of the readability and comprehensibility of other PEMs also seems warranted.

Active Annotation in Evaluating the Credibility of Web-Based Medical Information: Guidelines for Creating Training Data Sets for Machine Learning

Background

The spread of false medical information on the web is rapidly accelerating. Establishing the credibility of web-based medical information has become a pressing necessity. Machine learning offers a solution that, when properly deployed, can be an effective tool in fighting medical misinformation on the web.

Objective

The aim of this study is to present a comprehensive framework for designing and curating machine learning training data sets for web-based medical information credibility assessment. We show how to construct the annotation process. Our main objective is to support researchers from the medical and computer science communities. We offer guidelines on the preparation of data sets for machine learning models that can fight medical misinformation.

Methods

We begin by providing the annotation protocol for medical experts involved in medical sentence credibility evaluation. The protocol is based on a qualitative study of our experimental data. To address the problem of insufficient initial labels, we propose a preprocessing pipeline for the batch of sentences to be assessed. It consists of representation learning, clustering, and reranking. We call this process active annotation.

Results

We collected more than 10,000 annotations of statements related to selected medical subjects (psychiatry, cholesterol, autism, antibiotics, vaccines, steroids, birth methods, and food allergy testing) for less than US $7000 by employing 9 highly qualified annotators (certified medical professionals), and we release this data set to the general public. We developed an active annotation framework for more efficient annotation of noncredible medical statements. The application of qualitative analysis resulted in a better annotation protocol for our future efforts in data set creation.

Conclusions

The results of the qualitative analysis support our claims of the efficacy of the presented method.

Quality of condition suggestions and urgency advice provided by the Ada symptom assessment app evaluated with vignettes optimised for Australia

When people face a health problem, they often first ask, 'Is there an app for that?'. We investigated the quality of advice provided by the Ada symptom assessment application to address the question, 'How do I know the app on my phone is safe and provides good advice?'. The app was tested with 48 independently created vignettes developed for a previous study, including 18 specifically developed for the Australian setting, using an independently developed methodology to evaluate the accuracy of condition suggestions and urgency advice. The correct condition was listed first in 65% of vignettes, and in the Top 3 results in 83% of vignettes. The urgency advice in the app exactly matched the gold standard 63% of vignettes. The app's accuracy of condition suggestion and urgency advice is higher than that of the best-performing symptom assessment app reported in a previous study (61%, 77% and 52% for conditions suggested in the Top 1, Top 3 and exactly matching urgency advice respectively). These results are relevant to the application of symptom assessment in primary and community health, where medical quality and safety should determine app choice.

Validity Evidence Based on Relations to Other Variables of the eHealth Literacy Questionnaire (eHLQ): Bayesian Approach to Test for Known-Groups Validity

Background

As health resources and services are increasingly delivered through digital platforms, eHealth literacy is becoming a set of essential capabilities to improve consumer health in the digital era. To understand eHealth literacy needs, a meaningful measure is required. Strong initial evidence for the reliability and construct validity of inferences drawn from the eHealth Literacy Questionnaire (eHLQ) was obtained during its development in Denmark, but validity testing for varying purposes is an ongoing and cumulative process.

Objective

This study aims to examine validity evidence based on relations to other variables—using data collected with the known-groups approach—to further explore if the eHLQ is a robust tool to understand eHealth literacy needs in different contexts. A priori hypotheses are set for the expected score differences among age, sex, education, and information and communication technology (ICT) use for each of the 7 eHealth literacy constructs represented by the 7 eHLQ scales.

Methods

A Bayesian mediated multiple indicators multiple causes model approach was used to simultaneously identify group differences and test measurement invariance through differential item functioning across the groups, with ICT use as a mediator. A sample size of 500 participants was estimated. Data were collected at 3 diverse health sites in Australia.

Results

Responses from 525 participants were included for analysis. Being older was significantly related to lower scores in 4 eHLQ scales, with 3. Ability to actively engage with digital services having the strongest effect (total effect –0.37; P<.001), followed by 1. Using technology to process health information (total effect –0.32; P<.001), 5. Motivated to engage with digital services (total effect –0.21; P=.01), and 7. Digital services that suit individual needs (total effect –0.21; P=.02). However, the effects were only partially mediated by ICT use. Higher education was associated with higher scores in 1. Using technology to process health information (total effect 0.22; P=.01) and 3. Ability to actively engage with digital services (total effect 0.25; P<.001), with the effects mostly mediated by ICT use. Higher ICT use was related to higher scores in all scales except 2. Understanding health concepts and language and 4. Feel safe and in control. Either no or ignorable cases of differential item functioning were found across the 4 groups.

Conclusions

By using a Bayesian mediated multiple indicators multiple causes model, this study provides supportive validity evidence for the eHLQ based on relations to other variables as well as established evidence regarding internal structure related to measurement invariance across the groups for the 7 scales in the Australian community health context. This study also demonstrates that the eHLQ can be used to gain valuable insights into people’s eHealth literacy needs to help optimize access and use of digital health and promote health equity.

Online Patient Information on Domiciliary Oxygen Therapy: An Evaluation of Quality, Suitability, Reliability, Readability and Content

BACKGROUND

Domiciliary oxygen therapy (DOT) is a complex intervention and has significant impact on patients' daily activities, quality of life, and mental wellbeing. Suitable education is pertinent in improving patients' understanding and usage of DOT, as those receiving appropriate education have a better knowledge of their prescription, clearer expectations, and improved adherence to DOT.

RESEARCH QUESTION

Do currently available online patient resources on DOT provide high-quality information for patients?

STUDY DESIGN AND METHODS

We evaluated the first 100 results of three major search engines [Google, Yahoo and Bing] using the terms, "home oxygen therapy" and "information or education". Website content was assessed based on Thoracic Society of Australia and New Zealand and British Thoracic Society domiciliary oxygen guidelines. Validated tools were used to evaluate resource quality [DISCERN instrument], suitability [Suitability Assessment of Materials (SAM)], reliability [Journal of the American Medical Association (JAMA) benchmarks and the Health on the Net (HON) code], and readability [Flesch Reading Ease and Flesch-Kincaid Grade Level].

RESULTS

Thirty-six websites met study inclusion criteria. Websites from foundation/advocacy organisations scored the highest in quality and suitability, with a median DISCERN total score of 48.0 (interquartile range: 43.5-60.0) or "fair" and a median SAM suitability score of 70% (53.0-71.0) or "superior". Industry/for-profit websites had the best content score of 7.8 (5.0-8.6). The HON accreditation seal was present on 14% of the websites and only five websites met the four JAMA benchmarks. The median readability scores exceeded the recommended reading grades of 6th to 8th level for consumer health-related educational resources.

INTERPRETATION

The overall quality, suitability, reliability, and content of online health resources for DOT are of a low-to-moderate standard, with the reading grade at an unsuitable level for the general population. Health professionals should be aware of the limitations of currently available online DOT patient resources.

Orthodontic treatment consent forms: A readability analysis

OBJECTIVE

To evaluate the readability of orthodontic treatment informed consent forms (ICFs).

DESIGN

A cross-sectional observational investigation.

METHODS

An online search strategy was adopted to identify ICFs for orthodontic treatment. The text of each form that satisfied inclusion criteria was modified to a standardised protocol. The readability was evaluated using three validated instruments. GraphPad Software (GraphPad Software Inc., La Jolla, CA, USA) was used for statistical analyses.

RESULTS

A total of 59 ICFs were evaluated. The majority were available on the websites of orthodontists (66.1%) and general dentists (27.1%). The scores recorded from the three instruments indicated that the content of 93.2%-98.3% of ICFs was difficult to read. A strong correlation was observed between the Simple Measure of Gobbledegook (SMOG) and the Flesch Kincaid Grade level (FKGL) instruments (r = 0.9782; P < 0.0001). The mean SMOG score for all assessed forms was 11.19 (95% confidence interval [CI] 10.85-11.54). The ICFs authored by the manufacturers of orthodontic appliances provided by clinicians recorded a mean Flesch Reading (FRE) score of 40.14 (95% CI 33.91-46.37) out of a maximum 100. An analysis of the SMOG scores indicated that the content of ICFs authored by national orthodontic societies was more difficult to read than those written by all other authors (P=0.01; 95% CI -1.6 to -0.2).

CONCLUSION

The readability of the ICFs was too difficult for a significant number of the population. Many patients will be unable to validly consent to treatment based solely on reading of the content of the evaluated ICFs. A greater effort is required to improve the readability of ICFs to help ensure patient autonomy regarding orthodontic treatment decision-making and management.

The Lancet women and cardiovascular disease Commission: reducing the global burden by 2030

Cardiovascular disease is the leading cause of death in women. Decades of grassroots campaigns have helped to raise awareness about the impact of cardiovascular disease in women, and positive changes affecting women and their health have gained momentum. Despite these efforts, there has been stagnation in the overall reduction of cardiovascular disease burden for women in the past decade. Cardiovascular disease in women remains understudied, under-recognised, underdiagnosed, and undertreated. This Commission summarises existing evidence and identifies knowledge gaps in research, prevention, treatment, and access to care for women. Recommendations from an international team of experts and leaders in the field have been generated with a clear focus to reduce the global burden of cardiovascular disease in women by 2030. This Commission represents the first effort of its kind to connect stakeholders, to ignite global awareness of sex-related and gender-related disparities in cardiovascular disease, and to provide a springboard for future research.

Communications in the time of a pandemic: the readability of documents for public consumption

OBJECTIVE

Government communications in a crisis can influence public health outcomes. This research aimed to investigate if written communications of the most commonly sought sources of COVID-19 information available on the internet have readability levels commensurate with those of the general public.

METHODS

Online documents from the World Health Organization (WHO), and the governments of Australia, the UK and the US were assessed for readability using an online instrument that calculated scores for the Flesch Reading Ease Score, the SMOG Index and the Readability Consensus Grade Level.

RESULTS

Similar to the previous research, most documents assessed had a readability standard that was at or above the recommended grade level, and as such inaccessible to substantial portions of the general public. A one-way ANOVA with post hoc tests revealed significant differences among the data, with Australian documents significantly more difficult to read than those from the UK and US.

CONCLUSIONS

Government departments need to consider their audience and monitor readability of the documents they produce to ensure that readers can understand them. Implications for public health: Health communications need to be written at a level appropriate for the targeted population in order to be fit for purpose.

Explaining the concept of maternal health information verification and assessment during pregnancy: a qualitative study

Background

Pregnant women use information sources for their own health and health of their children. However, despite the importance of trusting the information sources, pregnant women may not have the ability to verify the maternal health information, which could have negative consequences for their health. The purpose of this study was to explain the concept of maternal health information verification and assessment in pregnant women according to their experiences and perception.

Methods

This is a qualitative study that was conducted in 2017 in Tehran, Iran. The participants in this study consisted of 19 pregnant women who were selected by purposeful sampling. To collect data, semi-structured, in-depth and face to face interviews were conducted with participants and continued until saturation of data. Conventional content analysis method was used to analyze the data and to identify concepts and synthesize them into general classes. MAXQDA software version 10 was used to manage the data.

Results

In the process of data analysis, the concept of verification and assessment of maternal health information in pregnancy was explained in two main categories, including “Validity of information resources” and “Reliance on information resources.” The category of Validity of information resources had two subcategories of valid and invalid sources, and the main category of Reliance on information resources had two subcategories of indicators of assurance, and confusion and trying to obtain assurance.

Conclusion

The results indicated that pregnant women used various sources and indicators, as well as different evaluation methods to obtain information and verify it, especially when they are confused. Thus, health authorities and healthcare professionals should provide appropriate programs to familiarize mothers with credible sources, train pregnant women on standards and practices for judging the accuracy of information, and create a safe margin of information.

Getting the right message: a content analysis and application of the health literacy INDEX tool to online HIV resources in Australia

Health literacy is considered by some as a determinant of health. Research suggests that different levels of health literacy may be associated with human immunodeficiency virus (HIV) knowledge and behaviour, including willingness to test. This study assessed the health literacy demands of existing Australia HIV prevention resources available online using a health literacy assessment and content analysis. Two reviewers assessed 66 resources using the Health Literacy INDEX. Sixty-one (92%) scored below 50%, with an average score of 35.5%. A coding frame was developed to conduct a content analysis of the resources excluding videos, multiple webpages and booklets (n = 52). We coded for six categories of knowledge: HIV knowledge (100% of resources), transmission (96%), acquired immune deficiency syndrome (AIDS) knowledge (88%), testing (87%), consequence of infection (85%) and prevention (77%). We found that resources required a reading grade above grade 8, and very few resources considered audience appropriateness. There were missed opportunities to encourage HIV prevention or testing. Some resources used incorrect language to refer towards people living with HIV, and transmission and prevention messages were often inconsistent. Guidelines for developing HIV prevention resources are warranted to improve health literacy, accessibility and appropriateness of resources and ensure consistent messages and framing of HIV risk.

Estimating the Economic Burden of Low Health Literacy in the Blacktown Community in Sydney, Australia: A Population-Based Study

Evidence shows that inadequate or low health literacy (LHL) levels are significantly associated with economic ramifications at the individual, employer, and health care system levels. Therefore, this study aims to estimate the economic burden of LHL among a culturally and linguistically diverse (CALD) community in Blacktown: a local government area (LGA) in Sydney, Australia. This study is a secondary analysis of cross-sectional data from publicly available datasets, including 2011 and 2016 census data and National Health Survey (NHS) data (2017–2018) from the Australian Bureau of Statistics (ABS), and figures on Disease Expenditure in Australia for 2015–2016 provided by the Australian Institute of Health and Welfare (AIHW). This study found that 20% of Blacktown residents reported low levels of active engagement with health care providers (Domain 6 of the Health Literacy Questionnaire (HLQ)), with 14% reporting a limited understanding of the health information required to take action towards improving health or making health care decisions (Domain 9 of the HLQ). The overall extra/delta cost (direct and indirect health care costs) associated with LHL in the Blacktown LGA was estimated to be between $11,785,528 and $15,432,239 in 2020. This is projected to increase to between $18,922,844 and $24,191,911 in 2030. Additionally, the extra disability-adjusted life year (DALY) value in 2020, for all chronic diseases and age-groups—comprising the extra costs incurred due to years of life lost (YLL) and years lived with disability (YLD)—was estimated at $414,231,335. The findings of our study may enable policymakers to have a deeper understanding of the economic burden of LHL in terms of its impact on the health care system and the production economy.

Depression in cancer: quality assessment of online patient education resources

OBJECTIVE

Psychiatric comorbidities are common among cancer patients. However, little is known about the quality of online information regarding these conditions. This study uses a validated tool to systematically determine the strengths and limitations of websites addressing depression in cancer patients.

METHODS

The term "depression in cancer patients" was searched online using the search engines Google, Yippy, and Dogpile. A set of predetermined inclusion/exclusion criteria was applied to all search results, which yielded 48 websites for inclusion. A validated rating tool was used to score the websites based on the six domains of Affiliation, Accountability, Interactivity, Structure & Organization, Readability, and Content Quality. The results were analyzed using descriptive statistics.

RESULTS

Of the 48 websites evaluated, 50% were commercial. 63% of websites identified authorship, 54% cited reliable sources, 37% provided links, and 38% were updated within the last 2 years. 94% of websites featured a search engine and 60% had at least four structural tools. Average readability was at a grade 12.3 level using the Flesch-Kincaid scale and 11.3 using the Simple Measure of Gobbledygook Index. The most completely and accurately covered topics of depression were symptoms and treatment-83% and 73% respectively. Its prevention and prognosis were not covered by any of the websites.

CONCLUSIONS

A validated rating tool was applied to evaluate the quality of online information for depression in cancer patients. Website accountability was poor, readability was often at a level that is too difficult for the lay audience, and the topics of prevention and prognosis were seldom covered.

Reaching out to stakeholders: The use of knowledge terminology on the websites of Australian public hospitals

Background

The objective of the study described in this article was to examine whether, and to what extent, Australian public hospitals use knowledge terminology, i.e. a body of knowledge-related terms, on their websites. The paper also discusses the difference in the level of such communication between large and small hospitals, the factors affecting the use of the knowledge-related terms in the communication and the similarities/differences between the use of knowledge terms in Australian public hospitals and large/small companies in Australia.

Methods

151 Australian public hospitals were included in the research sample: 51 large and 100 small hospitals. Using the method of content analysis, websites mentioning knowledge creation, knowledge sharing, knowledge implementation, and knowledge retention were identified, along with the number of these mentions. Descriptive statistics and chi square test of independence were used to provide answers to four research questions.

Results

Of the 151 hospitals included in the sample, 30 had no website and 62 (50 small and 12 large) had a single page website. The study found that there are differences between Australian public hospitals regarding the level of their knowledge communication on their websites, both between small and large hospitals and between the individual hospitals within the large and small hospital groups.

Conclusions

A well-known saying goes “For the mouth speaks what the heart is full of”. Effective communication of knowledge-related terminologies to both internal and external stakeholders, i.e. the parties who access the websites, is therefore an indication of a knowledge focus in the public hospitals. Large hospitals are generally more active in communicating knowledge terms, although there are some exceptions. Some of the small hospitals can lead by example, but most of them do not include knowledge terminology in their communication on websites.