User Experiences of the McMaster Optimal Aging Portal's Evidence Summaries and Blog Posts: Usability Study

Plain language summaries: A systematic review of theory, guidelines and empirical research

Plain language summaries (PLSs) have been introduced to communicate research in an understandable way to a nonexpert audience. Guidelines for writing PLSs have been developed and empirical research on PLSs has been conducted, but terminology and research approaches in this comparatively young field vary considerably. This prompted us to review the current state of the art of the theoretical and empirical literature on PLSs. The two main objectives of this review were to develop a conceptual framework for PLS theory, and to synthesize empirical evidence on PLS criteria. We began by searching Web of Science, PubMed, PsycInfo and PSYNDEX (last search 07/2021). In our review, we included empirical investigations of PLSs, reports on PLS development, PLS guidelines, and theoretical articles referring to PLSs. A conceptual framework was developed through content analysis. Empirical studies investigating effects of PLS criteria on defined outcomes were narratively synthesized. We identified 7,714 records, of which 90 articles met the inclusion criteria. All articles were used to develop a conceptual framework for PLSs which comprises 12 categories: six of PLS aims and six of PLS characteristics. Thirty-three articles empirically investigated effects of PLSs on several outcomes, but study designs were too heterogeneous to identify definite criteria for high-quality PLSs. Few studies identified effects of various criteria on accessibility, understanding, knowledge, communication of research, and empowerment. We did not find empirical evidence to support most of the criteria we identified in the PLS writing guidelines. We conclude that although considerable work on establishing and investigating PLSs is available, empirical evidence on criteria for high-quality PLSs remains scarce. The conceptual framework developed in this review may provide a valuable starting point for future guideline developers and PLS researchers.

Impact of a Knowledge Translation Intervention on Physical Activity and Mobility in Older Adults (the Move4Age Study): Randomized Controlled Trial

Background

The McMaster Optimal Aging Portal (the Portal) was launched in 2014 as a knowledge translation (KT) tool to increase access to evidence-based health information.

Objective

The purpose of this study was to understand if and how dissemination of mobility information through the Portal impacts physical activity (PA) in older adults.

Methods

In this randomized controlled trial, participants (n=510) were assigned to a 12-week mobility-focused KT intervention or self-serve control group. The intervention included weekly email alerts and a study-specific social media hashtag linking to mobility-focused Portal materials. The control group was able to access the Portal on their own but did not receive targeted KT strategies. Participants completed questionnaires (including the Rapid Assessment of Physical Activity to quantify PA) at baseline, end of the study, and 3-month follow-up.

Results

Participants were predominantly female (430/510, 84.3%), mean age 64.7 years, with no baseline differences between groups. Over half (277/510, 54.3%) of the participants were classified as “active” at baseline. There was no significant between-group difference in the PA category. Overall, both groups increased their PA with improvements maintained at 3-month follow-up (P<.001). In planned subgroup analyses, the KT intervention had a significant effect for those with poor or fair baseline self-rated health (P=.03).

Conclusions

No differences were found between those who received the targeted intervention and a control group with self-serve access to the Portal, except in subgroups with low self-rated health. Both groups did report increases in PA that were sustained beyond participation in a research study. Findings suggest that different KT strategies may be needed for different types of users, with more intense interventions being most impactful for certain groups (ie, those with lower self-rated health).

Trial Registration

ClinicalTrials.gov NCT02947230; https://clinicaltrials.gov/ct2/show/NCT02947230

Evaluation of an online knowledge translation intervention to promote cancer risk reduction behaviours: findings from a randomized controlled trial

Background

Many cancers are preventable through lifestyle modification; however, few adults engage in behaviors that are in line with cancer prevention guidelines. This may be partly due to the mixed messages on effective cancer prevention strategies in popular media. The goal of the McMaster Optimal Aging Portal (the Portal) is to increase access to trustworthy health information. The purpose of this study was to explore if and how knowledge translation strategies to disseminate cancer prevention evidence using the Portal influence participants’ knowledge, intentions and health behaviors related to cancer risk.

Methods

Adults ≥40 years old, with no cancer history were randomized to a 12-week intervention (weekly emails and social media posts) or control group. Quantitative data on knowledge, intentions and behaviors (physical activity, diet, alcohol consumption and use of tobacco products) were collected at baseline, end of study and 3 months later. Participant engagement was assessed using Google Analytics, and participant satisfaction through open-ended survey questions and semi-structured interviews.

Results

Participants (n = 557, mean age 64.9) were predominantly retired (72%) females (81%). Knowledge of cancer prevention guidelines was higher in the intervention group at end of study only (+ 0.3, p = 0.01). Intentions to follow cancer prevention guidelines increased in both groups, with no between-group differences. Intervention participants reported greater light-intensity physical activity at end of study (+ 0.7 vs. 0.1, p = 0.03), and reduced alcohol intake at follow u (− 0.2 vs. + 0.3, p < 0.05), but no other between-group differences were found. Overall satisfaction with the Portal and intervention materials was high.

Conclusions

Dissemination of evidence-based cancer prevention information through the Portal results in small increases in knowledge of risk-reduction strategies and with little to no impact on self-reported health behaviours, except in particular groups. Further tailoring of knowledge translation strategies may be needed to see more meaningful change in knowledge and health behaviours.

Trial registration

ClinicalTrials.gov NCT03186703, June 14, 2017.

A Mobility-Focused Knowledge Translation Randomized Controlled Trial to Improve Physical Activity: Process Evaluation of the Move4Age Study

Background

Maintaining physical activity and physical function is important for healthy aging. We recently completed a randomized controlled trial of a targeted knowledge translation (KT) intervention delivered through the McMaster Optimal Aging Portal with the goal to increase physical activity and physical mobility in middle-aged and older adults, with results reported elsewhere.

Objective

The purpose of this process evaluation study is to explore which KT strategies were used by both intervention and control group participants, as well as the intervention groups’ engagement, satisfaction, and perceived usefulness of the targeted KT intervention.

Methods

Data on engagement with the intervention materials were gathered quantitatively through Google Analytics and Hootsuite throughout the intervention. Qualitative data were collected through a combination of open-ended surveys and qualitative interviews with a subset of participants at the end of the study to further understand engagement, satisfaction, and usefulness of the KT strategies.

Results

Throughout the intervention period, engagement with content delivered through weekly emails was highest, and participants rated email content most favorably in both surveys and interviews. Participants were generally satisfied with the intervention, noting the ease of participating and the distillation of information in an easy-to-access format being beneficial features. Participants who did not find the intervention useful were those with already high levels of baseline physical activity or physical function and those who were looking for more specific or individualized content.

Conclusions

This process evaluation provides insight into our randomized controlled trial findings and provides information that can be used to improve future online KT interventions.

Trial Registration

ClinicalTrials.gov NCT02947230; https://clinicaltrials.gov/ct2/show/nct02947230 (Archived by WebCite at http://www.webcitation.org/78t4tR8tM)

A Tool That Assesses the Evidence, Transparency, and Usability of Online Health Information: Development and Reliability Assessment

Background

The internet is commonly used by older adults to obtain health information and this trend has markedly increased in the past decade. However, studies illustrate that much of the available online health information is not informed by good quality evidence, developed in a transparent way, or easy to use. Furthermore, studies highlight that the general public lacks the skills necessary to distinguish between online products that are credible and trustworthy and those that are not. A number of tools have been developed to assess the evidence, transparency, and usability of online health information; however, many have not been assessed for reliability or ease of use.

Objective

The first objective of this study was to determine if a tool assessing the evidence, transparency, and usability of online health information exists that is easy and quick to use and has good reliability. No such tool was identified, so the second objective was to develop such a tool and assess it for reliability when used to assess online health information on topics of relevant to optimal aging.

Methods

An electronic database search was conducted between 2002 and 2012 to identify published papers describing tools that assessed the evidence, transparency, and usability of online health information. Papers were retained if the tool described was assessed for reliability, assessed the quality of evidence used to create online health information, and was quick and easy to use. When no one tool met expectations, a new instrument was developed and tested for reliability. Reliability between two raters was assessed using the intraclass correlation coefficient (ICC) for each item at two time points. SPSS Statistics 22 software was used for statistical analyses and a one-way random effects model was used to report the results. The overall ICC was assessed for the instrument as a whole in July 2015. The threshold for retaining items was ICC>0.60 (ie, “good” reliability).

Results

All tools identified that evaluated online health information were either too complex, took a long time to complete, had poor reliability, or had not undergone reliability assessment. A new instrument was developed and assessed for reliability in April 2014. Three items had an ICC<0.60 (ie, “good” reliability). One of these items was removed (“minimal scrolling”) and two were retained but reworded for clarity. Four new items were added that assessed the level of research evidence that informed the online health information and the tool was retested in July 2015. The total ICC score showed excellent agreement with both single measures (ICC=0.988; CI 0.982–0.992) and average measures (ICC=0.994; CI 0.991–0.996).

Conclusions

The results of this study suggest that this new tool is reliable for assessing the evidence, transparency, and usability of online health information that is relevant to optimal aging.

If You Build It, Who Will Come? A Description of User Characteristics and Experiences With the McMaster Optimal Aging Portal

Objectives: The McMaster Optimal Aging Portal (the Portal) aims to increase access to evidence-based health information. We would now like to understand who uses the Portal, why, and for what, and elicit feedback and suggestions for future initiatives. Methods: An online survey of users collected data on demographics, eHealth literacy, Internet use, information-seeking behavior, site acceptability and perceived impact on health behaviors, participant satisfaction, and suggestions for improvements using mixed methods. Results: Participants (n = 163, age 69.8 ± 8.6 years) were predominantly female (76%), married (67%), retired (80%), and well-educated with very good/excellent health (55%). The Portal was easy to use (83%) and relevant (80%), with 68% intending to, and 48% having changed behavior after using the Portal. A number of suggestions for improvement were obtained. Discussion: A better understanding of users' characteristics, needs, and preferences will allow us to improve content, target groups who are not engaging with the Portal, and plan future directions.