Comprehensibility of Health-Related Documents for Older Adults with Different Levels of Health Literacy: A Systematic Review

Pilot Testing of a Tool to Standardize the Assessment of the Quality of Health Information Generated by Artificial Intelligence-Based Models

Background Artificial intelligence (AI)-based conversational models, such as Chat Generative Pre-trained Transformer (ChatGPT), Microsoft Bing, and Google Bard, have emerged as valuable sources of health information for lay individuals. However, the accuracy of the information provided by these AI models remains a significant concern. This pilot study aimed to test a new tool with key themes for inclusion as follows: Completeness of content, Lack of false information in the content, Evidence supporting the content, Appropriateness of the content, and Relevance, referred to as "CLEAR", designed to assess the quality of health information delivered by AI-based models. Methods Tool development involved a literature review on health information quality, followed by the initial establishment of the CLEAR tool, which comprised five items that aimed to assess the following: completeness, lack of false information, evidence support, appropriateness, and relevance. Each item was scored on a five-point Likert scale from excellent to poor. Content validity was checked by expert review. Pilot testing involved 32 healthcare professionals using the CLEAR tool to assess content on eight different health topics deliberately designed with varying qualities. The internal consistency was checked with Cronbach's alpha (α). Feedback from the pilot test resulted in language modifications to improve the clarity of the items. The final CLEAR tool was used to assess the quality of health information generated by four distinct AI models on five health topics. The AI models were ChatGPT 3.5, ChatGPT 4, Microsoft Bing, and Google Bard, and the content generated was scored by two independent raters with Cohen's kappa (κ) for inter-rater agreement. Results The final five CLEAR items were: (1) Is the content sufficient?; (2) Is the content accurate?; (3) Is the content evidence-based?; (4) Is the content clear, concise, and easy to understand?; and (5) Is the content free from irrelevant information? Pilot testing on the eight health topics revealed acceptable internal consistency with a Cronbach's α range of 0.669-0.981. The use of the final CLEAR tool yielded the following average scores: Microsoft Bing (mean=24.4±0.42), ChatGPT-4 (mean=23.6±0.96), Google Bard (mean=21.2±1.79), and ChatGPT-3.5 (mean=20.6±5.20). The inter-rater agreement revealed the following Cohen κ values: for ChatGPT-3.5 (κ=0.875, P<.001), ChatGPT-4 (κ=0.780, P<.001), Microsoft Bing (κ=0.348, P=.037), and Google Bard (κ=.749, P<.001). Conclusions The CLEAR tool is a brief yet helpful tool that can aid in standardizing testing of the quality of health information generated by AI-based models. Future studies are recommended to validate the utility of the CLEAR tool in the quality assessment of AI-generated health-related content using a larger sample across various complex health topics.

Health Literacy Level and Comprehension of Prescription and Nonprescription Drug Information

The aim of this study was to investigate the level of misunderstanding of medication information in Korean adults after stratifying by level of health literacy and to identify the factors influencing the misunderstanding of medication information and reading amounts of information on OTC drug labels. A cross-sectional survey was performed with 375 adult participants using the survey instrument. Multiple linear regression analyses were performed to identify factors which influence misunderstanding of medication information. Participants misunderstood 20% of words on OTC drug labels, 9% of prescription drug instructions, and 9% of pictograms. Participants on average read 59% of the overall contents of the OTC drug labels. As prescription drugs’ dosing regimens became more complicated, the level of misunderstanding instructions increased. The level of misunderstanding words on OTC drug labels significantly decreased as participants had adequate health literacy (β = −18.11, p < 0.001) and higher education levels (β = −6.83, p < 0.001), after adjusting for the study variables. The level of misunderstanding instructions for prescription drugs increased as participants became older (β = 8.81, p < 0.001) and had lower education levels (β = −5.05, p < 0.001), after adjusting for the study variables. The level of misunderstanding pictograms was similar to that of misunderstanding instructions for prescription drug labels. The amount of reading information on OTC drug labels significantly increased as respondents had adequate health literacy (β = 9.27, p < 0.001), were older (β = 12.49, p < 0.001), or had chronic diseases (β = 7.49, p = 0.007). Individuals’ health literacy level, reading behaviors, and complexity of medication instructions are associated with misunderstanding of medication information. Appropriate word choices in drug labels and an improved format of medication instructions could increase understanding of medication information and prevent adverse drug reactions.

Multimorbidity prevalence and patterns and their associations with health literacy among chronic kidney disease patients

Background

Health literacy is the ability to deal with information related to one’s health. Patients with low health literacy have poor disease-management skills for chronic diseases, such as chronic kidney disease (CKD). This could influence the number and combination of their diseases.

Methods

We included adult patients with CKD stages 1–5 from the Lifelines Study (n = 2,742). We assessed the association between low health literacy and the number and patterns of comorbidities, considering them globally and stratified by age and sex, using multinomial logistic regression and latent class analysis, respectively.

Results

Low health literacy was associated with a higher number of comorbidities in the crude models, and after adjustment for age, sex, eGFR, smoking, and BMI. In the crude model, the OR for low health literacy increased from 1.71 (1.25–2.33) for two comorbidities to 2.71 (2.00–3.68) for four comorbidities. In the fully-adjusted model, the associations remained significant with a maximum OR of 1.70 (1.16–2.49) for four comorbidities. The patterns of multimorbidity were similar for low and adequate health literacy, overall and by sex, bur tended to be different for patients older than 65. Older patients with low health literacy had higher comorbidity prevalence and a relatively greater share of cardiovascular, psychiatric, and central nervous system diseases.

Conclusions

Among CKD patients, low health literacy is associated with more multimorbidity. Health literacy is not associated with patterns of multimorbidity in younger patients, but a difference was observed in older ones. Improving low health literacy could be an intervention efficient also in decreasing multimorbidity in CKD patients.

Graphical abstract

Supplementary Information

The online version contains supplementary material available at 10.1007/s40620-021-01229-1.

Evaluation of a mammography decision aid for women 75 and older at risk for lower health literacy in a pretest-posttest trial

OBJECTIVE

The evaluation of the effect of a mammography decision aid (DA) designed for older women at risk for lower health literacy (LHL) on their knowledge of mammography's benefits and harms and decisional conflict.

METHODS

Using a pretest-posttest design, women > 75 years at risk for LHL reviewing a mammography DA before and after their [B] primary care provider visit. Women were recruited from an academic medical center and community health centers and clinics.

RESULTS

Of 147 eligible women approached, 43 participated. Receipt of the DA significantly affected knowledge of mammography's benefits and harms [B] (pre-test (M = 3.75, SD = 1.05) to post-test (M = 4.42, SD = 1.19), p = .03). Receipt of the DA did not significantly affect decisional conflict (pre-test (M = 3.10, SD = .97) to post-test (M = 3.23, SD = 1.02), p = .71, higher scores = lower decisional conflict). The majority of the women (97%) indicated that the DA was helpful.

CONCLUSIONS

Women found a mammography screening DA helpful and its use was associated with these women having increased knowledge of mammography's benefits and harms.

PRACTICE IMPLICATIONS

With the shift toward shared decision-making for women > 75 years, there is a need to engage women of all literacy levels to participate in these decisions and have tools such as the one tested in this study.

Modifying a Mammography Decision Aid for Older Adult Women with Risk Factors for Low Health Literacy

Background:

Guidelines recommend that before being offered mammography screening, women age 75 years and older be informed of the uncertainty of benefit and potential for harm (e.g., being diagnosed with a breast cancer that would otherwise never have shown up in one's lifetime); however, few older women are informed of the risks of mammography screening and most overestimate its benefits.

Objective:

The aim of this study was to learn from women older than age 75 years who have predisposing risk factors for low health literacy (LHL) how they make decisions about mammography screening, whether an existing decision aid (DA) on mammography screening for them was acceptable and helpful, and suggestions for improving the DA.

Methods:

We conducted semi-structured interviews with 18 women who were between ages 75 and 89 years and had predisposing risk factors for LHL (i.e., answered somewhat to not at all confident to the health literacy screening question “How confident are you filling out medical forms by yourself?” and/or had an education level of some college or less).

Key Results:

Findings indicate that women in this study lacked knowledge and understanding that one can decide on mammography screening based on their personal values. Women were enthusiastic about screening based on an interest in taking care of themselves but rely on their providers for health care decisions. Overall, most women found the DA helpful and would recommend the use of the DA.

Conclusions:

Findings from this study provide formative data to test the efficacy of the modified DA in practice. Failing to consider the informational needs of adults with LHL in design of DAs could inadvertently exacerbate existing inequalities in health. It is essential that DAs consider older women's diverse backgrounds and educational levels to support their decision-making. [HLRP: Health Literacy Research and Practice. 2021;5(2):e78–e90.]

Plain Language Summary:

The goal of this research was to understand how women older than age 75 years with risk factors for low health literacy made decisions about getting mammograms, whether an educational pamphlet was helpful, and suggestions for improving it. This research helps in understanding how to involve this population in the process of designing patient-related materials for mammogram decision-making.

How to tackle health literacy problems in chronic kidney disease patients? A systematic review to identify promising intervention targets and strategies

BACKGROUND

Limited health literacy (LHL) is associated with multiple adverse health outcomes in chronic kidney disease (CKD). Interventions are needed to improve this situation, but evidence on intervention targets and strategies is lacking. This systematic review aims to identify potential targets and strategies by summarizing the evidence on: (i) patient- and system-level factors potentially mediating the relation between LHL and health outcomes; and (ii) the effectiveness of health literacy interventions customized to CKD patients.

METHODS

We performed a systematic review of peer-reviewed research articles in Medline, Embase and Web of Science, 2009-19. We assessed the quality of the studies and conducted a best-evidence synthesis.

RESULTS

We identified 860 publications and included 48 studies. Most studies were of low quality (n = 26) and focused on dialysis and transplantation (n = 38). We found strong evidence for an association of LHL with smoking and having a suboptimal transplantation process. Evidence was weak for associations between LHL and a variety of factors related to self-care management (n = 25), utilization of care (n = 23), patient-provider interaction (n = 8) and social context (n = 5). Six interventions were aimed at improving knowledge, decision-making and health behaviours, but evidence for their effectiveness was weak.

CONCLUSIONS

Study heterogeneity, low quality and focus on kidney failure largely impede the identification of intervention targets and strategies for LHL. More and higher quality studies in earlier CKD stages are needed to unravel how LHL leads to worse health outcomes, and to identify targets and strategies to prevent disease deterioration. Healthcare organizations need to develop and evaluate efforts to support LHL patients.

The Dunedin dementia risk awareness project: pilot study in older adults

AIMS

The USA and UK governmental and academic agencies suggest that up to 35% of dementia cases are preventable. We canvassed dementia risk and protective factor awareness among New Zealand older adults to inform the design of a larger survey.

METHOD

The modified Lifestyle for Brain Health scale quantifying dementia risk was introduced to a sample of 304 eligible self-selected participants.

RESULTS

Two hundred and sixteen older adults (≥50 years), with mean ± standard deviation age 65.5 ± 11.4 years (50-93 years), completed the survey (71% response rate). Respondents were mostly women (n = 172, 80%), European (n = 207, 96%), and well educated (n = 100, 46%, with a tertiary qualification; including n = 17, 8%, with a postgraduate qualification). Around half of the participants felt that they were at a future risk of living with dementia (n = 101, 47%), and the majority felt that this would change their lives significantly (n = 205, 95%), that lifestyle changes would reduce their risk (n = 197, 91%), and that they could make the necessary changes (n = 189, 88%) and wished to start changes soon (n = 160, 74%). Only 4 of 14 modifiable risk or protective factors for dementia were adequately identified by the participants: physical exercise (81%), depression (76%), brain exercises (75%), and social isolation (83%). Social isolation was the commonly cited risk factor for dementia, while physical exercise was the commonly cited protective factor. Three clusters of brain health literacy were identified: psychosocial, medical, and modifiable.

CONCLUSION

The older adults in our study are not adequately knowledgeable about dementia risk and protective factors. However, they report optimism about modifying risks through lifestyle interventions.

Systematic Development of Materials for Inviting Low Health-Literate Individuals to Participate in Preconception Counseling

In this study we aimed to systematically analyze problems in the recruitment of women with low health literacy for preconception counseling and to adapt and evaluate written invitations for this group. In a problem analysis (stage 1) we used structured interviews (n = 72) to assess comprehension of the initial invitations, perception of perinatal risks, attitude and intention to participate in preconception counseling. These outcomes were used to adapt the invitation. The adapted flyer was pretested in interviews (n = 16) (stage 2) and evaluated in structured interviews among a new group of women (n = 67) (stage 3). Differences between women in stages 1 and 3 regarding comprehension, risk perception, attitude and intention to participate in counseling were analyzed by linear regression analysis and chi-square tests. Women in stage 3 (who read the adapted flyer) had a more positive attitude towards participation in preconception counselling and a better understanding of how to apply for a consultation than women in stage 1 (who read the initial invitations). No differences were found in intention to participate in preconception counseling and risk perception. Systematic adaptation of written invitations can improve the recruitment of low health-literate women for preconception counselling. Further research should gain insight into additional strategies to reach and inform this group.

Using Photo Stories to Support Doctor-Patient Communication: Evaluating a Communicative Health Literacy Intervention for Older Adults

Older adults often have limited health literacy and experience difficulties in communicating about their health. In view of the need for efficacious interventions, we compared a narrative photo story booklet regarding doctor-patient communication with a non-narrative but otherwise highly similar brochure. The photo story booklet included seven short picture-based stories about themes related to doctor-patient communication. The non-narrative brochure had comparable pictures and layout and dealt with the same themes, but it did not include any stories. We conducted two Randomized Controlled Trials (RCTs) among older adults with varying levels of health literacy: one RCT in Germany (N = 66) and one RCT in the Netherlands (N = 54); the latter one was followed by an in-depth interview study among a subset of the participants (81.5%; n = 44). In the RCTs, we did not find significant differences between the photo story booklet and the non-narrative brochure. In the interview study, a majority of the participants expressed a preference for the photo story booklet, which was perceived as recognizable, relevant, entertaining and engaging. We conclude that photo story booklets are a promising format but that there is room for improving their effectiveness.

Testing a Photo Story Intervention in Paper Versus Electronic Tablet Format Compared to a Traditional Brochure Among Older Adults in Germany: Randomized Controlled Trial

BACKGROUND

To increase effective communication in primary care consultations among older adults in Germany, the photo story is considered to be a useful tool based on Bandura's social cognitive theory. With information technology helping to increase effective communication, the use of tablets is gaining attention in health care settings, especially with older adults. However, the effectiveness of tablet technology and photo stories has rarely been tested.

OBJECTIVE

The aim is to compare the effectiveness of a photo story intervention to a traditional brochure. Both were delivered either in paper or tablet format.

METHODS

A trial was conducted with 126 older adults, aged 50 years and older, who were approached and recruited by researchers and administrative staff from senior day care, doctors in rehabilitation centers, and trainers in sports clubs in Germany. Open and face-to-face assessment methodologies were used. Participants were randomly assigned to one of four intervention conditions: traditional brochure in paper format (condition 1) and tablet format (condition 2), and photo story in paper format (condition 3) and tablet format (condition 4). Each participant received a questionnaire and either the traditional brochure or photo story in a paper or tablet version. To evaluate the effectiveness of each intervention, participants completed evaluation questionnaires before and after each intervention. The second part of the questionnaire measured different indicators of health literacy, communication skills, health measurements, and possible underlying mechanisms.

RESULTS

Compared to the traditional brochure, participants considered the photo story easier to understand (t₁₂₄=2.62, P=.01) and more informative (t₁₂₄=-2.17, P=.03). Participants preferred the paper format because they found it less monotonous (t₁₂₄=-3.05, P=.003), less boring (t₁₂₄=-2.65, P=.009), and not too long (t₁₂₄=-2.26, P=.03) compared to the tablet format. Among all conditions, the traditional brochure with a tablet (condition 2) was also perceived as more monotonous (mean 3.07, SD 1.08), boring (mean 2.77, SD 1.19), and too long to read (mean 2.50, SD 1.33) in comparison to the traditional brochure in paper format (condition 1). Moreover, the participants scored significantly higher on self-referencing on the traditional brochure in paper format (condition 1) than tablet format for both types of the brochure (conditions 2 and 4).

CONCLUSIONS

Traditional brochures on a tablet seem to be the least effective communication option in primary care consultations among all conditions for older adults. The findings might be specific for the current generation of older adults in Germany and need to be replicated in other countries with larger sample sizes. Although information technology brings advantages, such as effective interventions in different fields and settings, it may also come with several disadvantages, such as technical requirements of the users and devices. These should be considered when integrating information technology into wider situations and populations.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02502292; https://clinicaltrials.gov/ct2/show/NCT02502292 (Archived by Webcite at http://www.webcitation.org/747jdJ8pU).

Sweet Temptations: How Does Reading a Fotonovela About Diabetes Affect Dutch Adults with Different Levels of Literacy?

Recent studies suggest that health-related fotonovelas-booklets that portray a dramatic story using photographs and captions-may be effective health communication tools, especially for readers with a low level of literacy. In this experiment, effects on knowledge and behavioral intentions were assessed of a fotonovela originally developed for a Latin-American audience. Dutch readers from a low literacy group (N = 89) and a high literacy group (N = 113) were randomly assigned to one of three conditions: a fotonovela condition (all captions translated into Dutch), a traditional brochure condition (also in Dutch), and a control condition. On knowledge about diabetes, participants in the fotonovela condition outperformed participants in both other conditions. This finding was consistent across literacy levels. On behavioral intentions, however, readers of the fotonovela did not score significantly higher than participants in the other conditions. We also evaluated hypotheses proposed in the Entertainment Overcoming Resistance Model (EORM; Moyer-Gusé, 2008) on the possible mechanisms underlying persuasion through narratives. No support was found for the mechanisms proposed in the EORM. The outcomes of this study suggest that a fotonovela may be a valuable health education format for adults with varying levels of literacy, even if it was developed for a target group with a different cultural background.

Association of Healthy Habits Beliefs and Mortality in Older Adults: A Longitudinal Analysis of the Mexican Health and Aging Study

OBJECTIVE

The aim of this article is to establish the association between beliefs about healthy habits and mortality in a group of Mexican older adults.

METHOD

This is an 11-year follow-up secondary analysis of the Mexican Health and Aging Study.

RESULTS

There was a significant difference ( p < .001) in survival rate between those participants who believed that healthy habits have the potential to improve health compared with those who did not. After adjustment for confounders, Cox regression models showed a hazard ratio (HR) of 0.17 (95% confidence interval [CI] [0.07, 0.38], p < .001) for the group that believed in healthy habits.

DISCUSSION

Although the mechanism is not completely clear, according to our results, believing that healthy habits can improve health was associated with lower rates of mortality. Further research should elucidate potential strategies for changing beliefs in older adults with the goal of improving their overall health.

Which Type of Risk Information to Use for Whom? Moderating Role of Outcome-Relevant Involvement in the Effects of Statistical and Exemplified Risk Information on Risk Perceptions

The extant empirical research examining the effectiveness of statistical and exemplar-based health information is largely inconsistent. Under the premise that the inconsistency may be due to an unacknowledged moderator (O'Keefe, 2002), this study examined a moderating role of outcome-relevant involvement (Johnson & Eagly, 1989) in the effects of statistical and exemplified risk information on risk perception. Consistent with predictions based on elaboration likelihood model (Petty & Cacioppo, 1984), findings from an experiment (N = 237) concerning alcohol consumption risks showed that statistical risk information predicted risk perceptions of individuals with high, rather than low, involvement, while exemplified risk information predicted risk perceptions of those with low, rather than high, involvement. Moreover, statistical risk information contributed to negative attitude toward drinking via increased risk perception only for highly involved individuals, while exemplified risk information influenced the attitude through the same mechanism only for individuals with low involvement. Theoretical and practical implications for health risk communication are discussed.

Development of a Communication Intervention for Older Adults With Limited Health Literacy: Photo Stories to Support Doctor-Patient Communication

Successful doctor-patient communication relies on appropriate levels of communicative health literacy, the ability to deal with and communicate about health information. This article aims to describe the development of a narrative- and picture-based health literacy intervention intended to support older patients with limited health literacy when communicating during their primary care consultations. We performed a formative evaluation that included a review of the literature and interviews with stakeholders on relevant health literacy issues, qualitative studies with the target group, intervention planning, and a small-scale evaluation. Cocreation with the target group was a major component. Seven photo stories were developed incorporating principles from narrative and social learning theory and covering communication themes and strategies identified during focus group discussions and role-play exercises. The intervention was developed in 3 different formats: 1-page photo stories, narrated video clips using the original photo story pictures, and interactive video clips covering participation and communication during primary care consultations. In our small-scale evaluation, older adults considered the cocreated intervention appealing and comprehensible. The intervention shows promise for improving the health of older adults but needs further evaluation. This study provides a rigorous template for the participatory development of health literacy interventions.