Presenting the results of Cochrane Systematic Reviews to a consumer audience: a qualitative study

Research priorities in health communication and participation: international survey of consumers and other stakeholders

OBJECTIVE

To identify research priorities of consumers and other stakeholders to inform Cochrane Reviews in 'health communication and participation' (including such concepts as patient experience, shared decision-making and health literacy).

SETTING

International.

PARTICIPANTS

We included anyone with an interest in health communication and participation. Up to 151 participants (18-80 years; 117 female) across 12 countries took part, including 48 consumers (patients, carers, consumer representatives) and 75 professionals (health professionals, policymakers, researchers) (plus 25 people who identified as both).

DESIGN

Survey.

METHODS

We invited people to submit their research ideas via an online survey open for 4 weeks. Using inductive thematic analysis, we generated priority research topics, then classified these into broader themes.

RESULTS

Participants submitted 200 research ideas, which we grouped into 21 priority topics. Key research priorities included: insufficient consumer involvement in research (19 responses), 'official' health information is contradictory and hard to understand (18 responses), communication/coordination breakdowns in health services (15 responses), health information provision a low priority for health professionals (15 responses), insufficient eliciting of patient preferences (14 responses), health services poorly understand/implement patient-centred care (14 responses), lack of holistic care impacting healthcare quality and safety (13 responses) and inadequate consumer involvement in service design (11 responses). These priorities encompassed acute and community health settings, with implications for policy and research. Priority populations of interest included people from diverse cultural and linguistic backgrounds, carers, and people with low educational attainment, or mental illness. Most frequently suggested interventions focused on training and cultural change activities for health services and health professionals.

CONCLUSIONS

Consumers and other stakeholders want research addressing structural and cultural challenges in health services (eg, lack of holistic, patient-centred, culturally safe care) and building health professionals' communication skills. Solutions should be devised in partnership with consumers, and focus on the needs of vulnerable groups.

Producing an evidence-based treatment information website in partnership with people affected by multiple sclerosis

BACKGROUND AND AIMS

In earlier work, we identified that people affected by multiple sclerosis (MS) can have difficulty finding online treatment information that is up to date, trustworthy, understandable, and applicable to personal circumstances, but does not provoke confusion or negative emotional consequences. The objective was to develop online consumer summaries of MS treatment evidence (derived from Cochrane Reviews) that respond to identified treatment information needs of people affected by MS.

METHODS

A 2-phase mixed-methods project, conducted in partnership with consumers and an MS organisation. Phase 1 included review panels with consumers (Australians affected by MS) and health professionals to test paper-based treatment summaries before development, and pilot testing of the website. Phase 2 involved an online survey after website launch.

RESULTS

Eighty-three participants (85% affected by MS) took part. Phase 1 participants strongly endorsed key review summary components, including layering information, and additional sections to aid personal applicability. Participants additionally suggested questions for health professionals. Participants across both phases were receptive to the idea of being provided with Cochrane Review summaries online but were seeking other types of evidence and information, such as personal experiences and the latest experimental treatments, which could not be provided. While the small survey sample size (n = 58) limits application of the results to a broader population, the website was viewed favourably, as a useful, understandable, and trustworthy information source.

CONCLUSION

We describe a partnership approach to developing online evidence-based treatment information, underpinned by an in-depth understanding of consumers' information needs.

Non-pharmacological interventions for treating chronic prostatitis/chronic pelvic pain syndrome

BACKGROUND

Chronic prostatitis/chronic pelvic pain syndrome (CP/CPPS) is a common disorder in which the two main clinical features are pelvic pain and lower urinary tract symptoms. There are currently many approaches for its management, using both pharmacological and non-pharmacological interventions. The National Institute of Health - Chronic Prostatitis Symptom Index (NIH-CPSI) score is a validated measure commonly used to measure CP/CPPS symptoms.

OBJECTIVES

To assess the effects of non-pharmacological therapies for chronic prostatitis/chronic pelvic pain syndrome (CP/CPPS).

SEARCH METHODS

We performed a comprehensive search using multiple databases, trial registries, grey literature and conference proceedings with no restrictions on the language of publication or publication status. The date of the latest search of all databases was August 2017.

SELECTION CRITERIA

We included randomised controlled trials. Inclusion criteria were men with a diagnosis of CP/CPPS. We included all available non-pharmacological interventions.

DATA COLLECTION AND ANALYSIS

Two review authors independently classified studies and abstracted data from the included studies, performed statistical analyses and rated quality of evidence (QoE) according to the GRADE methods.

MAIN RESULTS

We included 38 unique studies with 3290 men with CP/CPPS across 23 comparisons.1. Acupuncture: (three studies, 204 participants) based on short-term follow-up, acupuncture reduces prostatitis symptoms in an appreciable number of participants compared with sham procedure (mean difference (MD) in total NIH-CPSI score -5.79, 95% confidence interval (CI) -7.32 to -4.26, high QoE). Acupuncture likely results in little to no difference in adverse events (moderate QoE). It probably also decreases prostatitis symptoms compared with standard medical therapy in an appreciable number of participants (MD -6.05, 95% CI -7.87 to -4.24, two studies, 78 participants, moderate QoE).2. Circumcision: (one study, 713 participants) based on short-term follow-up, early circumcision probably decreases prostatitis symptoms slightly (NIH-CPSI score MD -3.00, 95% CI -3.82 to -2.18, moderate QoE) and may not be associated with a greater incidence of adverse events compared with control (a waiting list to be circumcised, low QoE).3. Electromagnetic chair: (two studies, 57 participants) based on short-term follow-up, we are uncertain of the effects of the use of an electromagnetic chair on prostatitis symptoms. It may be associated with a greater incidence of adverse events compared with sham procedure (low to very low QoE).4. Lifestyle modifications: (one study, 100 participants) based on short-term follow-up, lifestyle modifications may be associated with a greater improvement in prostatitis symptoms in an appreciable number of participants compared with control (risk ratio (RR) for improvement in NIH-CPSI scores 3.90, 95% CI 2.20 to 6.92, very low QoE). We found no information regarding adverse events.5. Physical activity: (one study, 85 participants) based on short-term follow-up, a physical activity programme may cause a small reduction in prostatitis symptoms compared with control (NIH-CPSI score MD -2.50, 95% CI -4.69 to -0.31, low QoE). We found no information regarding adverse events.6. Prostatic massage: (two studies, 115 participants) based on short-term follow-up, we are uncertain whether the prostatic massage reduces or increases prostatitis symptoms compared with control (very low QoE). We found no information regarding adverse events.7. Extracorporeal shockwave therapy: (three studies, 157 participants) based on short-term follow-up, extracorporeal shockwave therapy reduces prostatitis symptoms compared with control (NIH-CPSI score MD -6.18, 95% CI -7.46 to -4.89, high QoE). These results may not be sustained at medium-term follow-up (low QoE). This treatment may not be associated with a greater incidence of adverse events (low QoE).8. Transrectal thermotherapy compared to medical therapy: (two studies, 237 participants) based on short-term follow-up, transrectal thermotherapy alone or in combination with medical therapy may decrease prostatitis symptoms slightly when compared with medical therapy alone (NIH-CPSI score MD -2.50, 95% CI -3.82 to -1.18, low QoE). One included study reported that participants may experience transient adverse events.9. Other interventions: there is uncertainty about the effects of other interventions included in this review. We found no information regarding psychological support or prostatic surgery.

AUTHORS' CONCLUSIONS

Some of the interventions can decrease prostatitis symptoms in an appreciable number without a greater incidence of adverse events. The QoE was mostly low. Future clinical trials should include a full report of their methods including adequate masking, consistent assessment of all patient-important outcomes including potential treatment-related adverse events and appropriate sample sizes.

Selecting and implementing overview methods: implications from five exemplar overviews

BACKGROUND

Overviews of systematic reviews are an increasingly popular method of evidence synthesis; there is a lack of clear guidance for completing overviews and a number of methodological challenges. At the UK Cochrane Symposium 2016, methodological challenges of five overviews were explored. Using data from these five overviews, practical implications to support methodological decision making of authors writing protocols for future overviews are proposed.

METHODS

Methods, and their justification, from the five exemplar overviews were tabulated and compared with areas of debate identified within current literature. Key methodological challenges and implications for development of overview protocols were generated and synthesised into a list, discussed and refined until there was consensus.

RESULTS

Methodological features of three Cochrane overviews, one overview of diagnostic test accuracy and one mixed methods overview have been summarised. Methods of selection of reviews and data extraction were similar. Either the AMSTAR or ROBIS tool was used to assess quality of included reviews. The GRADE approach was most commonly used to assess quality of evidence within the reviews. Eight key methodological challenges were identified from the exemplar overviews. There was good agreement between our findings and emerging areas of debate within a recent published synthesis. Implications for development of protocols for future overviews were identified.

CONCLUSIONS

Overviews are a relatively new methodological innovation, and there are currently substantial variations in the methodological approaches used within different overviews. There are considerable methodological challenges for which optimal solutions are not necessarily yet known. Lessons learnt from five exemplar overviews highlight a number of methodological decisions which may be beneficial to consider during the development of an overview protocol.

Stakeholders apply the GRADE evidence-to-decision framework to facilitate coverage decisions

OBJECTIVES

Coverage decisions are complex and require the consideration of many factors. A well-defined, transparent process could improve decision-making and facilitate decision-maker accountability.

STUDY DESIGN AND SETTING

We surveyed key US-based stakeholders regarding their current approaches for coverage decisions. Then, we held a workshop to test an evidence-to-decision (EtD) framework for coverage based on the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) criteria.

RESULTS

A total of 42 individuals (including 19 US stakeholders as well as international health policymakers and GRADE working group members) attended the workshop. Of the 19 stakeholders, 14 (74%) completed the survey before the workshop. Almost all of their organizations (13 of 14; 93%) used systematic reviews for coverage decision-making; few (2 of 14; 14%) developed their own evidence synthesis; a majority (9 of 14; 64%) rated the certainty of evidence (using various systems); almost all (13 of 14; 93%) denied formal consideration of resource use; and half (7 of 14; 50%) reported explicit criteria for decision-making. At the workshop, stakeholders successfully applied the EtD framework to four case studies and provided narrative feedback, which centered on contextual factors affecting coverage decisions in the United States, the need for reliable data on subgroups of patients, and the challenge of decision-making without formal consideration of resource use.

CONCLUSION

Stakeholders successfully applied the EtD framework to four case studies and highlighted contextual factors affecting coverage decisions and affirmed its value. Their input informed the further development of a revised EtD framework, now publicly available (http://gradepro.org/).

Automated telephone communication systems for preventive healthcare and management of long-term conditions

BACKGROUND

Automated telephone communication systems (ATCS) can deliver voice messages and collect health-related information from patients using either their telephone's touch-tone keypad or voice recognition software. ATCS can supplement or replace telephone contact between health professionals and patients. There are four different types of ATCS: unidirectional (one-way, non-interactive voice communication), interactive voice response (IVR) systems, ATCS with additional functions such as access to an expert to request advice (ATCS Plus) and multimodal ATCS, where the calls are delivered as part of a multicomponent intervention.

OBJECTIVES

To assess the effects of ATCS for preventing disease and managing long-term conditions on behavioural change, clinical, process, cognitive, patient-centred and adverse outcomes.

SEARCH METHODS

We searched 10 electronic databases (the Cochrane Central Register of Controlled Trials; MEDLINE; Embase; PsycINFO; CINAHL; Global Health; WHOLIS; LILACS; Web of Science; and ASSIA); three grey literature sources (Dissertation Abstracts, Index to Theses, Australasian Digital Theses); and two trial registries (www.controlled-trials.com; www.clinicaltrials.gov) for papers published between 1980 and June 2015.

SELECTION CRITERIA

Randomised, cluster- and quasi-randomised trials, interrupted time series and controlled before-and-after studies comparing ATCS interventions, with any control or another ATCS type were eligible for inclusion. Studies in all settings, for all consumers/carers, in any preventive healthcare or long term condition management role were eligible.

DATA COLLECTION AND ANALYSIS

We used standard Cochrane methods to select and extract data and to appraise eligible studies.

MAIN RESULTS

We included 132 trials (N = 4,669,689). Studies spanned across several clinical areas, assessing many comparisons based on evaluation of different ATCS types and variable comparison groups. Forty-one studies evaluated ATCS for delivering preventive healthcare, 84 for managing long-term conditions, and seven studies for appointment reminders. We downgraded our certainty in the evidence primarily because of the risk of bias for many outcomes. We judged the risk of bias arising from allocation processes to be low for just over half the studies and unclear for the remainder. We considered most studies to be at unclear risk of performance or detection bias due to blinding, while only 16% of studies were at low risk. We generally judged the risk of bias due to missing data and selective outcome reporting to be unclear.For preventive healthcare, ATCS (ATCS Plus, IVR, unidirectional) probably increase immunisation uptake in children (risk ratio (RR) 1.25, 95% confidence interval (CI) 1.18 to 1.32; 5 studies, N = 10,454; moderate certainty) and to a lesser extent in adolescents (RR 1.06, 95% CI 1.02 to 1.11; 2 studies, N = 5725; moderate certainty). The effects of ATCS in adults are unclear (RR 2.18, 95% CI 0.53 to 9.02; 2 studies, N = 1743; very low certainty).For screening, multimodal ATCS increase uptake of screening for breast cancer (RR 2.17, 95% CI 1.55 to 3.04; 2 studies, N = 462; high certainty) and colorectal cancer (CRC) (RR 2.19, 95% CI 1.88 to 2.55; 3 studies, N = 1013; high certainty) versus usual care. It may also increase osteoporosis screening. ATCS Plus interventions probably slightly increase cervical cancer screening (moderate certainty), but effects on osteoporosis screening are uncertain. IVR systems probably increase CRC screening at 6 months (RR 1.36, 95% CI 1.25 to 1.48; 2 studies, N = 16,915; moderate certainty) but not at 9 to 12 months, with probably little or no effect of IVR (RR 1.05, 95% CI 0.99, 1.11; 2 studies, 2599 participants; moderate certainty) or unidirectional ATCS on breast cancer screening.Appointment reminders delivered through IVR or unidirectional ATCS may improve attendance rates compared with no calls (low certainty). For long-term management, medication or laboratory test adherence provided the most general evidence across conditions (25 studies, data not combined). Multimodal ATCS versus usual care showed conflicting effects (positive and uncertain) on medication adherence. ATCS Plus probably slightly (versus control; moderate certainty) or probably (versus usual care; moderate certainty) improves medication adherence but may have little effect on adherence to tests (versus control). IVR probably slightly improves medication adherence versus control (moderate certainty). Compared with usual care, IVR probably improves test adherence and slightly increases medication adherence up to six months but has little or no effect at longer time points (moderate certainty). Unidirectional ATCS, compared with control, may have little effect or slightly improve medication adherence (low certainty). The evidence suggested little or no consistent effect of any ATCS type on clinical outcomes (blood pressure control, blood lipids, asthma control, therapeutic coverage) related to adherence, but only a small number of studies contributed clinical outcome data.The above results focus on areas with the most general findings across conditions. In condition-specific areas, the effects of ATCS varied, including by the type of ATCS intervention in use.Multimodal ATCS probably decrease both cancer pain and chronic pain as well as depression (moderate certainty), but other ATCS types were less effective. Depending on the type of intervention, ATCS may have small effects on outcomes for physical activity, weight management, alcohol consumption, and diabetes mellitus. ATCS have little or no effect on outcomes related to heart failure, hypertension, mental health or smoking cessation, and there is insufficient evidence to determine their effects for preventing alcohol/substance misuse or managing illicit drug addiction, asthma, chronic obstructive pulmonary disease, HIV/AIDS, hypercholesterolaemia, obstructive sleep apnoea, spinal cord dysfunction or psychological stress in carers.Only four trials (3%) reported adverse events, and it was unclear whether these were related to the interventions.

AUTHORS' CONCLUSIONS

ATCS interventions can change patients' health behaviours, improve clinical outcomes and increase healthcare uptake with positive effects in several important areas including immunisation, screening, appointment attendance, and adherence to medications or tests. The decision to integrate ATCS interventions in routine healthcare delivery should reflect variations in the certainty of the evidence available and the size of effects across different conditions, together with the varied nature of ATCS interventions assessed. Future research should investigate both the content of ATCS interventions and the mode of delivery; users' experiences, particularly with regard to acceptability; and clarify which ATCS types are most effective and cost-effective.

User Experiences of the McMaster Optimal Aging Portal's Evidence Summaries and Blog Posts: Usability Study

BACKGROUND

Evidence summaries and blogs can support evidence-informed healthy aging, by presenting high-quality health research evidence in plain language for a nonprofessional (citizen) audience.

OBJECTIVE

Our objective was to explore citizens' perceptions about the usability of evidence summaries and blog posts on the Web-based McMaster Optimal Aging Portal.

METHODS

Twenty-two citizens (aged 50 years and older) and informal caregivers participated in a qualitative study using a think-aloud method and semistructured interviews. Eleven interviews were conducted in person, 7 over the telephone, and 4 by Skype.

RESULTS

We identified themes that fell under 4 user-experience categories: (1) desirability: personal relevance, (2) understandability: language comprehension, grasping the message, dealing with uncertainty, (3) usability: volume of information, use of numbers, and (4) usefulness: intention to use, facility for sharing.

CONCLUSIONS

Participants recognized that high-quality evidence on aging was valuable. Their intended use of the information was influenced by how much it applied to their own health circumstances or those of a loved one. Some specific formatting features that were preferred included consistent layout, content organized by subheadings, catchy titles, numerical information summarized in a table, and inclusion of a glossary.

Pros and cons of circumcision: an evidence-based overview

Based on three large randomized controlled trials (RCTs) conducted in Africa, it can clearly be stated that circumcision lowers the risk of infection with the human immunodeficiency virus (HIV) and some sexually transmitted infections (STIs) among males in settings of high HIV and STI endemicity. Similar effects on STI risk may exist for females, although this may result from an indirect effect of decreasing risk of infection among male partners. It is unknown whether circumcision prevents HIV acquisition in men who have sex with men (MSM), although there might be a protective effect for men who engage mainly in insertive anal intercourse. When the effects of adult circumcision on sexual function and satisfaction of men are examined, high-quality evidence strongly supports lack of harm. Whether circumcision alters sexual satisfaction of female partners is not known as fewer and smaller studies reported conflicting results. Circumcision rarely causes serious complications if practiced by trained practitioners, in a sterile setting, and with a proper follow-up. These conclusions are limited by the lack of high-quality data from areas outside of Africa. RCTs have not been conducted to assess the effects of circumcising infants or MSM. Circumcision has well-proven benefits for people residing in areas with high prevalence of STIs, including HIV, and is not unethical for those who choose to be circumcised or have their children circumcised on religious, social, or cultural grounds. For many others, a definite pro or con recommendation, based on a risk-benefit ratio, cannot be made.

A Simple Tool for Communicating the Benefits and Harms of Health Interventions: A Guide for Creating a Fact Box

One of the major hurdles to promoting informed decision making in health is the continued use of poor risk presentation formats. This article offers a guide to develop a Fact Box, a simple decision tool to present data about the benefits and harms of treatments that has been demonstrated to improve understanding of health risks, an important part of risk literacy. The article offers guidance about how to determine the evidence basis for a health topic, select outcomes to report, extract and present numbers or outcomes, and design the layout. The guide also addresses potential challenges for summarizing evidence and provides alternatives for addressing issues related to missing, insufficient, imprecise, or conflicting evidence and for dealing with issues related to statistical and clinical significance. The guide concludes with details on how to document the development of the Fact Box for the purpose of transparency and reproducibility. Fact Boxes are an efficient tool to promote risk literacy and should be available in every physician's office.

A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity

OBJECTIVES

We aimed to (1) identify and classify spin (i.e., a description that overstates efficacy and/or understates harm) in systematic reviews and (2) rank spin in abstracts of systematic reviews according to their severity (i.e., the likelihood of distorting readers' interpretation of the results).

STUDY DESIGN

First, we used a four-phase consensus process to develop a classification of different types of spin. Second, we ranked the types of spin in abstracts according to their severity using a Q-sort survey with members of the Cochrane Collaboration.

RESULTS

We identified 39 types of spin, 28 from the main text and 21 from the abstract; 13 were specific to the systematic review design. Spin was classified into three categories: (1) misleading reporting, (2) misleading interpretation, and (3) inappropriate extrapolation. Spin ranked as the most severe by the 122 people who participated in the survey were (1) recommendations for clinical practice not supported by findings in the conclusion, (2) misleading title, and (3) selective reporting.

CONCLUSION

This study allowed for identifying spin that is likely to distort interpretation. Our classification could help authors, editors, and reviewers avoid spin in reports of systematic reviews.

Online health information seeking: how people with multiple sclerosis find, assess and integrate treatment information to manage their health

BACKGROUND AND OBJECTIVE

The Internet is increasingly prominent as a source of health information for people with multiple sclerosis (MS). But there has been little exploration of the needs, experiences and preferences of people with MS for integrating treatment information into decision making, in the context of searching on the Internet. This was the aim of our study.

DESIGN

Sixty participants (51 people with MS; nine family members) took part in a focus group or online forum. They were asked to describe how they find and assess reliable treatment information (particularly online) and how this changes over time. Thematic analysis was underpinned by a coding frame.

RESULTS

Participants described that there was both too much information online and too little that applied to them. They spoke of wariness and scepticism but also empowerment. The availability of up-to-date and unbiased treatment information, including practical and lifestyle-related information, was important to many. Many participants were keen to engage in a 'research partnership' with health professionals and developed a range of strategies to enhance the trustworthiness of online information. We use the term 'self-regulation' to capture the variations in information seeking behaviour that participants described over time, as they responded to their changing information needs, their emotional state and growing expertise about MS.

CONCLUSIONS

People with MS have developed a number of strategies to both find and integrate treatment information from a range of sources. Their reflections informed the development of an evidence-based consumer web site based on summaries of MS Cochrane reviews.

Cochrane plain language summaries are highly heterogeneous with low adherence to the standards

BACKGROUND

The aim of this study was to analyze whether Cochrane plain language summaries (PLSs) adhere to the Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS).

METHODS

A systematic analysis of adherence to the measurable PLEACS items was performed for Cochrane PLSs published from March 2013 to the end of January 2015. Duplicate independent data extraction was performed. An adherence score was calculated for each PLS and for the Cochrane Review Groups (CRGs) that published them.

RESULTS

Of the 1738 analyzed PLSs, not a single one adhered fully to the measured PLEACS items. The highest adherence was found for absence of details of the search strategy (99 % adherence), and the lowest adherence for an item mandating to address quality according to the GRADE system (0.7 % adherence). Overall adherence percentage of PLSs reporting reviews with included studies was 57 %. Different CRGs had a wide range of adherence scores.

CONCLUSIONS

Cochrane plain language summaries are highly heterogeneous with a low adherence to the PLEACS standards. Therefore, there is much room for improving the content and consistency of the PLS. A standardization of PLSs is necessary to ensure delivery of proper and consistent information for consumers.

Do Cochrane summaries help student midwives understand the findings of Cochrane systematic reviews: the BRIEF randomised trial

BACKGROUND

Abstracts and plain language summaries (PLS) are often the first, and sometimes the only, point of contact between readers and systematic reviews. It is important to identify how these summaries are used and to know the impact of different elements, including the authors' conclusions. The trial aims to assess whether (a) the abstract or the PLS of a Cochrane Review is a better aid for midwifery students in assessing the evidence, (b) inclusion of authors' conclusions helps them and (c) there is an interaction between the type of summary and the presence or absence of the conclusions.

METHODS

Eight hundred thirteen midwifery students from nine universities in the UK and Ireland were recruited to this 2 × 2 factorial trial (abstract versus PLS, conclusions versus no conclusions). They were randomly allocated to one of four groups and asked to recall knowledge after reading one of four summary formats of two Cochrane Reviews, one with clear findings and one with uncertain findings. The primary outcome was the proportion of students who identified the appropriate statement to describe the main findings of the two reviews as assessed by an expert panel.

RESULTS

There was no statistically significant difference in correct response between the abstract and PLS groups in the clear finding example (abstract, 59.6 %; PLS, 64.2 %; risk difference 4.6 %; CI -0.2 to 11.3) or the uncertain finding example (42.7 %, 39.3 %, -3.4 %, -10.1 to 3.4). There was no significant difference between the conclusion and no conclusion groups in the example with clear findings (conclusions, 63.3 %; no conclusions, 60.5 %; 2.8 %; -3.9 to 9.5), but there was a significant difference in the example with uncertain findings (44.7 %; 37.3 %; 7.3 %; 0.6 to 14.1, p = 0.03). PLS without conclusions in the uncertain finding review had the lowest proportion of correct responses (32.5 %). Prior knowledge and belief predicted student response to the clear finding review, while years of midwifery education predicted response to the uncertain finding review.

CONCLUSIONS

Abstracts with and without conclusions generated similar student responses. PLS with conclusions gave similar results to abstracts with and without conclusions. Removing the conclusions from a PLS with uncertain findings led to more problems with interpretation.

Improving the user experience of patient versions of clinical guidelines: user testing of a Scottish Intercollegiate Guideline Network (SIGN) patient version

Background

Guideline producers are increasingly producing versions of guidelines for the public, and knowledge of what the public want from them is also increasing. The aim of this study was to user test a patient version of a SIGN clinical guideline that was designed based on preliminary work for the DECIDE project. SIGN is the leading national guideline producer in Scotland.

Methods

People with a diagnosis of glaucoma and non-professional carers were recruited from across Scotland. User testing was conducted using a think-aloud protocol method. Each session was conducted by one interviewer and one observer. All sessions were recorded and transcribed. The data was analysed, problems with the guideline were identified and resolved and key findings were themed using a user experience model.

Results

Thirteen user testing sessions with people with glaucoma and one with a carer were conducted. Key facilitators of desirability and usability identified include clear branding as a patient version and a clearly described purpose, audience and contents page. Other facilitators include a “friendly” tone which is achieved by the use of colour, quotes, icons, simple language and charts, and brief chunked text.

The value and usefulness of the patient guideline was influenced by its ability to: inform the public, link information to actions, and empower people in their interaction with healthcare professionals. Participants were disappointed by the lack of information on treatment in the patient version, which was outside its scope.

Information on the evidence based guideline production process and the involvement of appropriately skilled professionals was key to the credibility of the guideline. Lack of awareness of guidelines and guideline producing bodies, is a potentially serious threat to findability/accessibility.

Conclusions

It is important for guideline producers to maximise the user experience of the public when they access patient versions of guidelines, particularly given the current low level of access and awareness. One size does not fit all and guideline producers need to strike a balance between keeping the patient version simple and providing sufficient information to facilitate shared decision making and empower the public. Guideline producers may find the results of this study useful in designing their own patient versions.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-016-1287-8) contains supplementary material, which is available to authorized users.

Persistent musculoskeletal pain and productive employment; a systematic review of interventions

A systematic analysis of the literature was undertaken to determine which characteristics of workplace interventions are most effective in assisting people with persistent musculoskeletal pain (PMP) to remain productively employed. Databases of Medline, PsychINFO, CINAHL and Embase were searched using MeSH and other relevant terms. Studies that reported on interventions at, or involving, the workplace were included. Interventions were considered as either focused on the individual or multilevel. Outcome measures assessed included: job loss, productivity, sick leave, pain and cost benefit. A quality assessment was undertaken using GRADE criteria with development of impact statements to synthesise the results. Eighteen relevant articles (14 studies) were identified for inclusion in the review. No high-level evidence for workplace interventions to assist people with PMP were identified. Low numbers of participants and limited studies resulted in downgrading of evidence. However, individually focused interventions will probably reduce job loss and sick leave, but are unlikely to reduce pain. Multilevel focused interventions will probably result in decreased sick leave and provide some cost benefit. The evidence on productivity was limited and of poor quality. Further research is required because sustainable employment for individuals with PMP is important and understanding what works is necessary to ensure effective workplace interventions are developed.

Decision-Making about Healthcare Related Tests and Diagnostic Strategies: User Testing of GRADE Evidence Tables

OBJECTIVE

To develop guidance on what information to include and how to present it in tables summarizing the evidence from systematic reviews of test accuracy following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.

METHODS

To design and refine the evidence tables, we used an iterative process based on the analysis of data from four rounds of discussions, feedback and user testing. During the final round, we conducted one-on-one user testing with target end users. We presented a number of alternative formats of evidence tables to participants and obtained information about users' understanding and preferences.

RESULTS

More than 150 users participated in initial discussions and provided their formal and informal feedback. 20 users completed one-on-one user testing interviews. Almost all participants preferred summarizing the results of systematic reviews of test accuracy in tabular format rather than plain text. Users generally preferred less complex tables but found presenting sensitivity and specificity estimates only as too simplistic. Users found the presentation of test accuracy for several values of prevalence initially confusing but modifying table layout and adding sample clinical scenarios for each prevalence reduced this confusion. Providing information about clinical consequences of testing result was viewed as not feasible for authors of systematic reviews.

CONCLUSION

We present the current formats for tables presenting test accuracy following the GRADE approach. These tables can be developed using GRADEpro guidelines development tool (www.guidelinedevelopment.org or www.gradepro.org) and are being further developed into electronic interactive tables that will suit the needs of different end users. The formatting of these tables, and how they influence result interpretation and decision-making will be further evaluated in a randomized trial.

Making randomised trials more efficient: report of the first meeting to discuss the Trial Forge platform

Randomised trials are at the heart of evidence-based healthcare, but the methods and infrastructure for conducting these sometimes complex studies are largely evidence free. Trial Forge ( www.trialforge.org ) is an initiative that aims to increase the evidence base for trial decision making and, in doing so, to improve trial efficiency.This paper summarises a one-day workshop held in Edinburgh on 10 July 2014 to discuss Trial Forge and how to advance this initiative. We first outline the problem of inefficiency in randomised trials and go on to describe Trial Forge. We present participants' views on the processes in the life of a randomised trial that should be covered by Trial Forge.General support existed at the workshop for the Trial Forge approach to increase the evidence base for making randomised trial decisions and for improving trial efficiency. Agreed upon key processes included choosing the right research question; logistical planning for delivery, training of staff, recruitment, and retention; data management and dissemination; and close down. The process of linking to existing initiatives where possible was considered crucial. Trial Forge will not be a guideline or a checklist but a 'go to' website for research on randomised trials methods, with a linked programme of applied methodology research, coupled to an effective evidence-dissemination process. Moreover, it will support an informal network of interested trialists who meet virtually (online) and occasionally in person to build capacity and knowledge in the design and conduct of efficient randomised trials.Some of the resources invested in randomised trials are wasted because of limited evidence upon which to base many aspects of design, conduct, analysis, and reporting of clinical trials. Trial Forge will help to address this lack of evidence.

Web search behavior and information needs of people with multiple sclerosis: focus group study and analysis of online postings

BACKGROUND

Multiple sclerosis (MS) patients and their family members increasingly seek health information on the Internet. There has been little exploration of how MS patients integrate health information with their needs, preferences, and values for decision making. The INtegrating and Deriving Evidence, Experiences, and Preferences (IN-DEEP) project is a collaboration between Italian and Australian researchers and MS patients, aimed to make high-quality evidence accessible and meaningful to MS patients and families, developing a Web-based resource of evidence-based information starting from their information needs.

OBJECTIVE

The objective of this study was to analyze MS patients and their family members' experience about the Web-based health information, to evaluate how they asses this information, and how they integrate health information with personal values.

METHODS

We organized 6 focus groups, 3 with MS patients and 3 with family members, in the Northern, Central, and Southern parts of Italy (April-June 2011). They included 40 MS patients aged between 18 and 60, diagnosed as having MS at least 3 months earlier, and 20 family members aged 18 and over, being relatives of a person with at least a 3-months MS diagnosis. The focus groups were audio-recorded and transcribed verbatim (Atlas software, V 6.0). Data were analyzed from a conceptual point of view through a coding system. An online forum was hosted by the Italian MS society on its Web platform to widen the collection of information. Nine questions were posted covering searching behavior, use of Web-based information, truthfulness of Web information. At the end, posts were downloaded and transcribed.

RESULTS

Information needs covered a comprehensive communication of diagnosis, prognosis, and adverse events of treatments, MS causes or risk factors, new drugs, practical, and lifestyle-related information. The Internet is considered useful by MS patients, however, at the beginning or in a later stage of the disease a refusal to actively search for information could occur. Participants used to search on the Web before or after their neurologist's visit or when a new therapy was proposed. Social networks are widely used to read others' stories and retrieve information about daily management. A critical issue was the difficulty of recognizing reliable information on the Web. Many sources were used but the neurologist was mostly the final source of treatment decisions.

CONCLUSIONS

MS patients used the Internet as a tool to integrate information about the illness. Information needs covered a wide spectrum, the searched topics changed with progression of the disease. Criteria for evaluating Internet accuracy and credibility of information were often lacking or generic. This may limit the empowerment of patients in health care choices.

A summary to communicate evidence from systematic reviews to the public improved understanding and accessibility of information: a randomized controlled trial

OBJECTIVES

To evaluate a new format of a summary, which presents research from synthesized evidence to patients and the public.

STUDY DESIGN AND SETTING

We conducted a randomized controlled trial in 143 members of the public from five countries (Canada, Norway, Spain, Argentina, and Italy). Participants received either a new summary format (a plain language summary [PLS]) or the current format used in Cochrane systematic reviews. The new PLS presents information about the condition and intervention, a narrative summary of results, and a table of results with absolute numbers for effects of the intervention and quality of the evidence using Grading of Recommendations Assessment, Development, and Evaluation.

RESULTS

With the new PLS, more participants understood the benefits and harms and quality of evidence (53% vs. 18%, P < 0.001); more answered each of the five questions correctly (P ≤ 0.001 for four questions); and they answered more questions correctly, median 3 (interquartile range [IQR]: 1-4) vs. 1 (IQR: 0-1), P < 0.001). Better understanding was independent of education level. More participants found information in the new PLS reliable, easy to find, easy to understand, and presented in a way that helped make decisions. Overall, participants preferred the new PLS.

CONCLUSION

This new PLS format for patients and the public is a promising tool to translate evidence from synthesized research.

Low carbohydrate versus isoenergetic balanced diets for reducing weight and cardiovascular risk: a systematic review and meta-analysis

BACKGROUND

Some popular weight loss diets restricting carbohydrates (CHO) claim to be more effective, and have additional health benefits in preventing cardiovascular disease compared to balanced weight loss diets.

METHODS AND FINDINGS

We compared the effects of low CHO and isoenergetic balanced weight loss diets in overweight and obese adults assessed in randomised controlled trials (minimum follow-up of 12 weeks), and summarised the effects on weight, as well as cardiovascular and diabetes risk. Dietary criteria were derived from existing macronutrient recommendations. We searched Medline, EMBASE and CENTRAL (19 March 2014). Analysis was stratified by outcomes at 3-6 months and 1-2 years, and participants with diabetes were analysed separately. We evaluated dietary adherence and used GRADE to assess the quality of evidence. We calculated mean differences (MD) and performed random-effects meta-analysis. Nineteen trials were included (n = 3209); 3 had adequate allocation concealment. In non-diabetic participants, our analysis showed little or no difference in mean weight loss in the two groups at 3-6 months (MD 0.74 kg, 95%CI -1.49 to 0.01 kg; I2 = 53%; n = 1745, 14 trials; moderate quality evidence) and 1-2 years (MD 0.48 kg, 95%CI -1.44 kg to 0.49 kg; I2 = 12%; n = 1025; 7 trials, moderate quality evidence). Furthermore, little or no difference was detected at 3-6 months and 1-2 years for blood pressure, LDL, HDL and total cholesterol, triglycerides and fasting blood glucose (>914 participants). In diabetic participants, findings showed a similar pattern.

CONCLUSIONS

Trials show weight loss in the short-term irrespective of whether the diet is low CHO or balanced. There is probably little or no difference in weight loss and changes in cardiovascular risk factors up to two years of follow-up when overweight and obese adults, with or without type 2 diabetes, are randomised to low CHO diets and isoenergetic balanced weight loss diets.

Update of strategies to translate evidence from cochrane musculoskeletal group systematic reviews for use by various audiences

For rheumatology research to have a real influence on health and well-being, evidence must be tailored to inform the decisions of various audiences. The Cochrane Musculoskeletal Group (CMSG), one of 53 groups of the not-for-profit international Cochrane Collaboration, prepares, maintains, and disseminates systematic reviews of treatments for musculoskeletal diseases. While systematic reviews provided by the CMSG fill a major gap in meeting the need for high-quality evidence syntheses, our work does not end at the completion of a review. The term "knowledge translation" (KT) refers to the activities involved in bringing research evidence to various audiences in a useful form so it can be used to support decision making and improve practices. Systematic reviews give careful consideration to research methods and analysis. Because the review is often long and detailed, the clinically relevant results may not be apparent or in the optimal form for use by patients and their healthcare practitioners. This paper describes 10 formats, many of them new, for ways that evidence from Cochrane Reviews can be translated with the intention of meeting the needs of various audiences, including patients and their families, practitioners, policy makers, the press, and members of the public (the "5 Ps"). Current and future knowledge tools include summary of findings tables, patient decision aids, plain language summaries, press releases, clinical scenarios in general medical journals, frequently asked questions (Cochrane Clinical Answers), podcasts, Twitter messages, Journal Club materials, and the use of storytelling and narratives to support continuing medical education. Future plans are outlined to explore ways of improving the influence and usefulness of systematic reviews by providing results in formats suitable to our varied audiences.

Supporting non-experts in judging the credibility of risk assessments (CORA)

One of the crucial communication issues that have to be tackled by risk assessors is how to provide a comprehensible and informative characterization of their findings. The CORA framework (CORA stands for credibility of risk assessment) is designed for helping non-experts in judging the credibility of risk assessments. The CORA framework can be used by (1) stakeholders and policy makers, to make an educated judgment about the credibility of an assessment, and (2) the authors of a risk assessment, to improve the evaluability of their reports. The CORA framework consists of 18 criteria, leading to six main recommendations. The framework's application is not limited to (EMF) risk assessment, for which it was originally developed, but can be used in any area of risk or hazard assessment.

Assessing and presenting summaries of evidence in Cochrane Reviews

Cochrane Reviews are intended to help providers, practitioners and patients make informed decisions about health care. The goal of the Cochrane Applicability and Recommendation Methods Group (ARMG) is to develop approaches, strategies and guidance that facilitate the uptake of information from Cochrane Reviews and their use by a wide audience with specific focus on developers of recommendations and on healthcare decision makers. This paper is part of a series highlighting developments in systematic review methodology in the 20 years since the establishment of The Cochrane Collaboration, and its aim is to present current work and highlight future developments in assessing and presenting summaries of evidence, with special focus on Summary of Findings (SoF) tables and Plain Language Summaries.A SoF table provides a concise and transparent summary of the key findings of a review in a tabular format. Several studies have shown that SoF tables improve accessibility and understanding of Cochrane Reviews.The ARMG and GRADE Working Group are working on further development of the SoF tables, for example by evaluating the degree of acceptable flexibility beyond standard presentation of SoF tables, developing SoF tables for diagnostic test accuracy reviews and interactive SoF tables (iSoF).The plain language summary (PLS) is the other main building block for dissemination of review results to end-users. The PLS aims to summarize the results of a review in such a way that health care consumers can readily understand them. Current efforts include the development of a standardized language to describe statistical results, based on effect size and quality of supporting evidence.Producing high quality PLS and SoF tables and making them compatible and linked would make it easier to produce dissemination products targeting different audiences (for example, providers, health policy makers, guideline developers).Current issues of debate include optimal presentation formats of SoF tables, the training required to produce SoF tables, and the extent to which the authors of Cochrane Reviews should provide explicit guidance to target audiences of patients, clinicians and policy-makers.

Helping people make well-informed decisions about health care: old and new challenges to achieving the aim of the Cochrane Collaboration

The aim of the Cochrane Collaboration is to help people make well-informed decisions about health care by preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health care interventions. This aim is as relevant now as it was 20 years ago, when the Cochrane Collaboration was established. Substantial progress has been made toward addressing challenges to achieving the Collaboration's aim. At the same time, a huge amount of work remains to be done. Current challenges include improving the quality of reviews, methodological challenges, meeting the needs of contributors and users and taking on new challenges while staying focused on the Collaboration's aim. Radical thinking and substantial change may be needed to identify and implement pragmatic strategies to ensure that reviews are up-to-date and informative. Methodological challenges include the development and application of better methods for addressing explanatory factors, incorporating non-randomized evidence and making comparisons across multiple interventions. Innovations in editorial processes and strategies to meet the needs of low- and middle-income countries and diverse users of Cochrane reviews are needed. Finally, although it is important to consider broadening the aims of the Collaboration to include types of questions other than the effects of interventions and types of products other than the Cochrane Library, we should not lose sight of the aim of the Cochrane Collaboration. Addressing that aim is still a major challenge that requires the collaboration of thousands of people around the world and continuing improvements in the methods used to achieve that aim.

Policymakers' and other stakeholders' perceptions of key considerations for health system decisions and the presentation of evidence to inform those considerations: an international survey

BACKGROUND

The DECIDE framework was developed to support evidence-informed health system decisions through evidence summaries tailored to health policymakers. The objective of this study was to determine policymakers' perceptions regarding the criteria in the DECIDE framework and how best to summarise and present evidence to support health system decisions.

METHODS

We conducted an online survey of a diverse group of stakeholders with health system decision experience from 15 countries and the World Health Organization. We asked about perceptions of criteria relevant to making health system decisions, use of evidence, grading systems, and evidence summaries.

RESULTS

We received 112 responses (70% response rate). Most respondents had healthcare (85%) and research (79%) experience. They (99%) indicated that systematic consideration of the available evidence would help to improve health system decision-making processes and supported the use of evidence from other countries (94%) and grading systems (81%). All ten criteria in the DECIDE framework were rated as important in the decision-making process. Respondents had divergent views regarding whether the same (38%) or different (45%) grading systems should be used across different types of health decisions. All components of our evidence summary were rated as important by over 90% of respondents.

CONCLUSIONS

Survey respondents were supportive of the DECIDE framework for health system decisions and the use of succinct summaries of the estimated size of effects and the quality of evidence. It is uncertain whether the findings of this survey represent the views of policymakers with little or no healthcare and research experience.

Development of a decision aid to inform patients' and families' renal replacement therapy selection decisions

BACKGROUND

Few educational resources have been developed to inform patients' renal replacement therapy (RRT) selection decisions. Patients progressing toward end stage renal disease (ESRD) must decide among multiple treatment options with varying characteristics. Complex information about treatments must be adequately conveyed to patients with different educational backgrounds and informational needs. Decisions about treatment options also require family input, as families often participate in patients' treatment and support patients' decisions. We describe the development, design, and preliminary evaluation of an informational, evidence-based, and patient-and family-centered decision aid for patients with ESRD and varying levels of health literacy, health numeracy, and cognitive function.

METHODS

We designed a decision aid comprising a complementary video and informational handbook. We based our development process on data previously obtained from qualitative focus groups and systematic literature reviews. We simultaneously developed the video and handbook in "stages." For the video, stages included (1) directed interviews with culturally appropriate patients and families and preliminary script development, (2) video production, and (3) screening the video with patients and their families. For the handbook, stages comprised (1) preliminary content design, (2) a mixed-methods pilot study among diverse patients to assess comprehension of handbook material, and (3) screening the handbook with patients and their families.

RESULTS

The video and handbook both addressed potential benefits and trade-offs of treatment selections. The 50-minute video consisted of demographically diverse patients and their families describing their positive and negative experiences with selecting a treatment option. The video also incorporated health professionals' testimonials regarding various considerations that might influence patients' and families' treatment selections. The handbook was comprised of written words, pictures of patients and health care providers, and diagrams describing the findings and quality of scientific studies comparing treatments. The handbook text was written at a 4th to 6th grade reading level. Pilot study results demonstrated that a majority of patients could understand information presented in the handbook. Patient and families screening the nearly completed video and handbook reviewed the materials favorably.

CONCLUSIONS

This rigorously designed decision aid may help patients and families make informed decisions about their treatment options for RRT that are well aligned with their values.

Evidence maps: communicating risk assessments in societal controversies: the case of engineered nanoparticles

The transparent and fair characterization of scientific evidence for reporting the results of a hazard assessment is a demanding task. In this article, we present an approach for characterizing evidence--the evidence map approach. The theoretical starting point is to view evidence characterization as a form of argumentation. Thus, evidence maps are designed to depict the evidence base, the pro and con arguments, and the remaining uncertainties, which together lead experts to their conclusions when summarizing and evaluating the scientific evidence about a potential hazard. To illustrate its use, the evidence maps approach is applied to characterizing the health-relevant effects of engineered nanoparticles. Empirical data from an online survey suggests that the use of evidence maps improves the reporting of hazard assessments. Nonexperts prefer to receive the information included in an evidence map in order to come to an informed judgment. Furthermore, the benefits and limitations of evidence maps are discussed in the light of recent literature on risk communication. Finally, the article underlines the need for further research in order to increase quality of evidence reporting.

The Cochrane Library review titles that are important to users of health care, a Cochrane Consumer Network project

BACKGROUND

The Cochrane Consumer Network is an internet-based community of international users of health care contributing to the work of The Cochrane Collaboration, whose mission is to inform healthcare decision making through development of systematic reviews of best evidence on healthcare interventions.

OBJECTIVE

To prioritize existing review titles listed on The Cochrane Library from a healthcare user perspective, with particular emphasis on patients, carers and health consumers.

DESIGN

An online survey was developed and after piloting was made available internationally. The broad dissemination strategy targeted Consumer Network members and Cochrane Review Group editorial staff to identify champions who notified patient support groups and participated in snowballing. The first part of the survey defined criteria that could be applied to review titles and asked survey respondents to rank them. The second part asked respondents to select a health area and prioritize review titles that were of importance to them. Each health area corresponded to a Cochrane Review Group.

RESULTS AND DISCUSSION

Sufficient responses were obtained from 522 valid responses to prioritize review topics in 19 health areas. A total of 321 respondents completed the titles assessment. The types of prioritized interventions were determined by the health area. An important observation was the emphasis on lifestyle and non-medication therapies in many of the included health areas. The clearest exception to this broad observation was where acute care is required such as antibiotics for acute respiratory tract and HIV-associated infections and for cardiac conditions. For some cancers, advanced cancer interventions were prioritized. The most important criteria were for the title to convey a clear meaning and the title conveyed that the review would have an impact on health and well-being. The least important criteria were that the topic was newsworthy or prioritized in the healthcare system.

CONCLUSION

This project was able to identify priority Cochrane review topics for users of health care in 19 of the 50 areas of health care covered by The Cochrane Collaboration. Reviews addressing lifestyle and non-medical interventions were strongly represented in the prioritized review titles. These findings highlight the importance of developing readable, informative lay summaries to support evidence-based decision making by healthcare users.

Evidence summaries tailored to health policy-makers in low- and middle-income countries

OBJECTIVE

To describe how the SUPPORT collaboration developed a short summary format for presenting the results of systematic reviews to policy-makers in low- and middle-income countries (LMICs).

METHODS

We carried out 21 user tests in six countries to explore users' experiences with the summary format. We modified the summaries based on the results and checked our conclusions through 13 follow-up interviews. To solve the problems uncovered by the user testing, we also obtained advisory group feedback and conducted working group workshops.

FINDINGS

Policy-makers liked a graded entry format (i.e. short summary with key messages up front). They particularly valued the section on the relevance of the summaries for LMICs, which compensated for the lack of locally-relevant detail in the original review. Some struggled to understand the text and numbers. Three issues made redesigning the summaries particularly challenging: (i) participants had a poor understanding of what a systematic review was; (ii) they expected information not found in the systematic reviews and (iii) they wanted shorter, clearer summaries. Solutions included adding information to help understand the nature of a systematic review, adding more references and making the content clearer and the document quicker to scan.

CONCLUSION

Presenting evidence from systematic reviews to policy-makers in LMICs in the form of short summaries can render the information easier to assimilate and more useful, but summaries must be clear and easy to read or scan quickly. They should also explain the nature of the information provided by systematic reviews and its relevance for policy decisions.