Framing the numerical findings of Cochrane plain language summaries: two randomized controlled trials

Practices and Barriers in Developing and Disseminating Plain-Language Resources Reporting Medical Research Information: A Scoping Review

BACKGROUND

The intent of plain-language resources (PLRs) reporting medical research information is to advance health literacy among the general public and enable them to participate in shared decision-making (SDM). Regulatory mandates coupled with academic and industry initiatives have given rise to an increasing volume of PLRs summarizing medical research information. However, there is significant variability in the quality, format, readability, and dissemination channels for PLRs. In this scoping review, we identify current practices, guidance, and barriers in developing and disseminating PLRs reporting medical research information to the general public including patients and caregivers. We also report on the PLR preferences of these intended audiences.

METHODS

A literature search of three bibliographic databases (PubMed, EMBASE, Web of Science) and three clinical trial registries (NIH, EMA, ISRCTN registry) was performed. Snowball searches within reference lists of primary articles were added. Articles with PLRs or reporting topics related to PLRs use and development available between January 2017 and June 2023 were identified. Evidence mapping and synthesis were used to make qualitative observations. Identified PLRs were quantitatively assessed, including temporal annual trends, availability by field of medicine, language, and publisher types.

RESULTS

A total of 9116 PLRs were identified, 9041 from the databases and 75 from clinical trial registries. The final analysis included 6590 PLRs from databases and 72 from registries. Reported barriers to PLR development included ambiguity in guidance, lack of incentives, and concerns of researchers writing for the general public. Available guidance recommendations called for greater dissemination, increased readability, and varied content formats. Patients preferred visual PLRs formats (e.g., videos, comics), which were easy to access on the internet and used short jargon-free text. In some instances, older audiences and more educated readers preferred text-only PLRs. Preferences among the general public were mostly similar to those of patients. Psychology, followed by oncology, showed the highest number of PLRs, predominantly from academia-sponsored research. Text-only PLRs were most commonly available, while graphical, digital, or online formats were less available. Preferred dissemination channels included paywall-free journal websites, indexing on PubMed, third-party websites, via email to research participants, and social media.

CONCLUSIONS

This scoping review maps current practices, recommendations, and patients' and the general public's preferences for PLR development and dissemination. The results suggest that making PLRs available to a wider audience by improving nomenclature, accessibility, and providing translations may contribute to empowerment and SDM. Minimizing variability among available guidance for PLR development may play an important role in amplifying the value and impact of these resources.

Does the format of result presentation and type of conclusion in Cochrane plain language summaries matter? A randomised controlled trial

OBJECTIVES

This study aimed to investigate whether the format and type of conclusion in Cochrane plain language summaries (PLSs) influence readers' perception of treatment benefit and decision-making.

DESIGN

An online parallel group, three-arm randomised controlled trial was conducted.

SETTING

The study was conducted online.

PARTICIPANTS

The participants were physiotherapy students.

INTERVENTIONS

The participants read two Cochrane PLSs, one with a positive conclusion (strong evidence of benefit) and another with a negative conclusion (strong evidence of non-benefit). Each participant read the results of both reviews presented in one of three formats: (1) numerical, (2) textual or (3) numerical and textual.

MAIN OUTCOME MEASURES

The primary outcome measure was the participants' perception of treatment benefit.

RESULTS

All three groups of participants perceived the treatment to have positive effects when the Cochrane PLS had a positive conclusion, regardless of the format of presentation (mean perception of treatment benefit score: textual 7.7 (SD 2.3), numerical 7.9 (SD 1.8), numerical and textual 7.7 (SD 1.7), p=0.362). However, when the Cochrane PLS had a negative conclusion, all three groups of participants failed to perceive a negative effect (mean perception of treatment benefit score: textual 5.5 (SD 3.3), numerical 5.6 (SD 2.7), numerical and textual 5.9 (SD 2.8), p=0.019).

CONCLUSIONS

The format of Cochrane PLSs does not appear to significantly impact physiotherapy students' perception of treatment benefit, understanding of evidence, persuasiveness or confidence in their decision. However, participants' perception of treatment benefit does not align with the conclusion when the Cochrane PLS indicates strong evidence of non-benefit from the intervention.

TRIAL REGISTRATION NUMBER

CTRI/2022/10/046476.

Linguistic analysis of plain language summaries and corresponding scientific summaries of Cochrane systematic reviews about oncology interventions

BACKGROUND

Cochrane plain language summaries (PLSs) are an important format to present high-quality healthcare evidence to patients with cancer and their families. They should be written in a way everyone can understand, since they serve as a tool in decision-making and present a bridge to overcome the gap between the healthcare users and professionals.

OBJECTIVE

The aim of the study was to assess the language characteristics of PLSs of Cochrane systematic reviews of oncology interventions in comparison with corresponding Cochrane scientific abstracts (SAs).

METHODS

In this cross-sectional study, we included all Cochrane PLSs and SAs of systematic reviews of oncology interventions available in the Cochrane Database of Systematic Reviews. We assessed text readability, measured using the Simple Measure of Gobbledygook (SMOG) index, and the prevalence of words related to different language tones (clout, authenticity, emotions and analytical tones). Two independent assessors categorized the conclusiveness of the efficacy of interventions into nine categories.

RESULTS

The overall median SMOG index for 275 PLSs was 13.0 (95% confidence interval [CI] 12.8-13.3). Readability scores did not differ across Cochrane Review Groups. SAs had a higher readability index than the corresponding PLSs (median = 16.6, 95% CI = 16.4-16.8). Regarding linguistic characteristics, PLSs were shorter than SAs, with less use of analytical tone, but more use of a positive emotional tone and authenticity. Overall, the 'Unclear' category of conclusiveness was the most common among all PLSs. Also, PLSs with 'No evidence' conclusions were the shortest and had the lowest SMOG index.

CONCLUSION

PLSs of Cochrane systematic reviews of oncological interventions have low readability and most give unclear conclusions about the efficacy of interventions. PLSs should be simplified so that patients and their families can benefit from appropriate health information on evidence synthesis. Further research is needed into reasons for unclear language to describe evidence from oncology trials.

Knowledge mobilization activities to support decision-making by youth, parents, and adults using a systematic and living map of evidence and recommendations on COVID-19: protocol for three randomized controlled trials and qualitative user-experience studies

INTRODUCTION

The COVID-19 pandemic underlined that guidelines and recommendations must be made more accessible and more understandable to the general public to improve health outcomes. The objective of this study is to evaluate, quantify, and compare the public's understanding, usability, satisfaction, intention to implement, and preference for different ways of presenting COVID-19 health recommendations derived from the COVID-19 Living Map of Recommendations and Gateway to Contextualization (RecMap).

METHODS AND ANALYSIS

This is a protocol for a multi-method study. Through an online survey, we will conduct pragmatic allocation-concealed, blinded superiority randomized controlled trials (RCTs) in three populations to test alternative formats of presenting health recommendations: adults, parents, and youth, with at least 240 participants in each population. Prior to initiating the RCT, our interventions will have been refined with relevant stakeholder input. The intervention arm will receive a plain language recommendation (PLR) format while the control arm will receive the corresponding original recommendation format as originally published by the guideline organizations (standard language version). Our primary outcome is understanding, and our secondary outcomes are accessibility and usability, satisfaction, intended behavior, and preference for the recommendation formats. Each population's results will be analyzed separately. However, we are planning a meta-analysis of the results across populations. At the end of each survey, participants will be invited to participate in an optional one-on-one, virtual semi-structured interview to explore their user experience. All interviews will be transcribed and analyzed using the principles of thematic analysis and a hybrid inductive and deductive approach.

ETHICS AND DISSEMINATION

Through Clinical Trials Ontario, the Hamilton Integrated Research Ethics Board has reviewed and approved this protocol (Project ID: 3856). The University of Alberta has approved the parent portion of the trial (Project ID:00114894). Findings from this study will be disseminated through open-access publications in peer-reviewed journals and using social media.

TRIAL REGISTRATION

Clinicaltrials.gov NCT05358990 . Registered on May 3, 2022.

The effectiveness and acceptability of evidence synthesis summary formats for clinical guideline development groups: a mixed-methods systematic review

INTRODUCTION

Clinical guideline development often involves a rigorous synthesis of evidence involving multidisciplinary stakeholders with different priorities and knowledge of evidence synthesis; this makes communicating findings complex. Summary formats are typically used to communicate the results of evidence syntheses; however, there is little consensus on which formats are most effective and acceptable for different stakeholders.

METHODS

This mixed-methods systematic review (MMSR) aimed to evaluate the effectiveness and acceptability (e.g. preferences and attitudes and preferences towards) of evidence synthesis summary formats for GDG members. We followed the PRISMA 2020 guideline and Joanna Briggs Institute Manual for Evidence Synthesis for MMSRs. We searched six databases (inception to April 20, 2021) for randomised controlled trials (RCTs), RCTs with a qualitative component, and qualitative studies. Screening, data extraction, and quality appraisal were performed in duplicate. Qualitative findings were synthesised using meta-aggregation, and quantitative findings are described narratively.

RESULTS

We identified 17,240 citations and screened 54 full-text articles, resulting in 22 eligible articles (20 unique studies): 4 articles reported the results of 5 RCTs, one of which also had a qualitative component. The other 18 articles discussed the results of 16 qualitative studies. Therefore, we had 5 trials and 17 qualitative studies to extract data from. Studies were geographically heterogeneous and included a variety of stakeholders and summary formats. All 5 RCTs assessed knowledge or understanding with 3 reporting improvement with newer formats. The qualitative analysis identified 6 categories of recommendations: 'presenting information', 'tailoring information' for end users, 'trust in producers and summary', 'knowledge required' to understand findings, 'quality of evidence', and properly 'contextualising information'. Across these categories, the synthesis resulted in 126 recommendations for practice. Nine recommendations were supported by both quantitative and qualitative evidence and 116 by only qualitative. A majority focused on how to present information (n = 64) and tailor content for different end users (n = 24).

CONCLUSIONS

This MMSR provides guidance on how to improve evidence summary structure and layout. This can be used by synthesis producers to better communicate to GDGs. Study findings will inform the co-creation of evidence summary format prototypes based on GDG member's needs. Trial registration The protocol for this project was previously published, and the project was preregistered on Open Science Framework (Clyne and Sharp, Evidence synthesis and translation of findings for national clinical guideline development: addressing the needs and preferences of guideline development groups, 2021; Sharp and Clyne, Evidence synthesis summary formats for decision-makers and Clinical Guideline Development Groups: A mixed-methods systematic review protocol, 2021).

Conclusiveness, linguistic characteristics and readability of Cochrane plain language summaries of intervention reviews: a cross-sectional study

Background

One of the most important formats to disseminate the evidence in health to different populations are Cochrane Plain Language Summaries (PLSs). PLSs should be written in a simplified language, easily understandable and providing clear message for the consumer. The aim of this study was to examine the extent to which PLSs are customized for lay persons, specifically by providing conclusive, comprehensible, and readable messages.

Methods

The study analyzed Cochrane PLSs of interventional studies (N = 4360) in the English language published from 1995 to 2019. We categorized the conclusiveness into one of the following categories: “positive”, “positive inconclusive”, “no evidence”, “no opinion”, “negative”, “negative inconclusive”, “unclear”, “equal”, “equal inconclusive”. Language characteristics were analyzed using Linguistic Inquiry and Word Count (LIWC) software. The level of readability was measured by SMOG (Simple Measure of Gobbledygook) index, indicating the number of years of education required to read the text. For each PLS, we also collected the following data: Cochrane Review Network, year of publication and number of authors.

Results

Most of the PLSs (80%) did not have a conclusive message. In 53% PLSs there was no concluding opinion about the studied intervention or the conclusion was unclear. The most frequent conclusiveness category was “no opinion” (30%), and its frequency increased over time. The conclusiveness categories were similarly dispersed across Cochrane Networks. PLSs were written in an objective style, with high levels of analytical tone and clout above neutral, but a lower relation to authenticity and tone. The median number of years of non-specific education needed to read the PLSs was 14.9 (IQR 13.8 to 16.1), indicating that the person needs almost 15 years of general education to read the content with ease.

Conclusion

Most of the Cochrane PLSs provided no concluding opinion or unclear conclusion regarding the effects of analyzed intervention. Analysis of readability indicated that they may be difficult to read for the lay population without medical education. Our results indicate that PLSs may not be so plain, and that the writing of Cochrane PLSs requires more effort. Tools used in this study could improve PLSs and make them better suited for lay audiences.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01721-7.

Plain language summaries: A systematic review of theory, guidelines and empirical research

Plain language summaries (PLSs) have been introduced to communicate research in an understandable way to a nonexpert audience. Guidelines for writing PLSs have been developed and empirical research on PLSs has been conducted, but terminology and research approaches in this comparatively young field vary considerably. This prompted us to review the current state of the art of the theoretical and empirical literature on PLSs. The two main objectives of this review were to develop a conceptual framework for PLS theory, and to synthesize empirical evidence on PLS criteria. We began by searching Web of Science, PubMed, PsycInfo and PSYNDEX (last search 07/2021). In our review, we included empirical investigations of PLSs, reports on PLS development, PLS guidelines, and theoretical articles referring to PLSs. A conceptual framework was developed through content analysis. Empirical studies investigating effects of PLS criteria on defined outcomes were narratively synthesized. We identified 7,714 records, of which 90 articles met the inclusion criteria. All articles were used to develop a conceptual framework for PLSs which comprises 12 categories: six of PLS aims and six of PLS characteristics. Thirty-three articles empirically investigated effects of PLSs on several outcomes, but study designs were too heterogeneous to identify definite criteria for high-quality PLSs. Few studies identified effects of various criteria on accessibility, understanding, knowledge, communication of research, and empowerment. We did not find empirical evidence to support most of the criteria we identified in the PLS writing guidelines. We conclude that although considerable work on establishing and investigating PLSs is available, empirical evidence on criteria for high-quality PLSs remains scarce. The conceptual framework developed in this review may provide a valuable starting point for future guideline developers and PLS researchers.

How to Put It Plainly? Findings From Two Randomized Controlled Studies on Writing Plain Language Summaries for Psychological Meta-Analyses

Plain language summaries (PLS) aim to communicate research findings to laypersons in an easily understandable manner. Despite the societal relevance of making psychological research findings available to the public, our empirical knowledge on how to write PLS of psychology studies is still scarce. In this article, we present two experimental studies investigating six characteristics of PLS for psychological meta-analyses. We specifically focused on approaches for (1) handling technical terms, (2) communicating the quality of evidence by explaining the methodological approach of meta-analyses, (3) explaining how synthesized studies operationalized their research questions, (4) handling statistical terms, (5) structuring PLS, and (6) explaining complex meta-analytic designs. To develop empirically validated guidelines on writing PLS, two randomized controlled studies including large samples stratified for education status, age, and gender (N Study1=2,288 and N Study2=2,211) were conducted. Eight PLS of meta-analyses from different areas of psychology were investigated as study materials. Main outcome variables were user experience (i.e., perceived accessibility, perceived understanding, and perceived empowerment) and knowledge acquisition, as well as understanding and knowledge of the quality of evidence. Overall, our hypotheses were partially confirmed, with our results underlining, among other things, the importance of explaining or replacing content-related technical terms (i.e., theoretical concepts) and indicating the detrimental effects of providing too many details on statistical concepts on user experience. Drawing on these and further findings, we derive five empirically well-founded rules on the lay-friendly communication of meta-analytic research findings in psychology. Implications for PLS authors and future research on PLS are discussed.