Listen to the data when results are not significant

Do infographics 'spin' the findings of health and medical research?

OBJECTIVE

To compare the prevalence of 'spin', and specific reporting strategies for spin, between infographics, abstracts and full texts of randomised controlled trials (RCTs) reporting non-significant findings in the field of health and medicine and to assess factors associated with the presence of spin.

DESIGN

Cross-sectional observational study.

DATA SOURCE

Publications in top quintile health and medical journals from August 2018 to October 2020 (Journal Citation Reports database).

ELIGIBILITY CRITERIA

Infographics, abstracts and full texts of RCTs with non-significant results for a primary outcome.

MAIN OUTCOMES AND MEASURES

Presence of spin (any spin and spin in the results and conclusions of infographics, abstracts and full texts).

EXPOSURES

Conflicts of interest, industry sponsorship, trial registration, journal impact factor, spin in the abstract, spin in the full text.

RESULTS

119 studies from 40 journals were included. One-third (33%) of infographics contained spin. Infographics were not more likely to contain any spin than abstracts (33% vs 26%, OR 1.4; 95% CI 0.8 to 2.4) or full texts (33% vs 26%, OR 1.4; 95% CI 0.8 to 2.4). Higher journal impact factor was associated with slightly lower odds of spin in infographics and full texts, but not abstracts. Infographics, but not abstracts or full texts, were less likely to contain spin if the trial was prospectively registered. No other significant associations were found.

CONCLUSIONS

Nearly one-third of infographics spin the findings of RCTs with non-significant results for a primary outcome, but the prevalence of spin is not higher than in abstracts and full texts. Given the increasing popularity of infographics to disseminate research findings, there is an urgent need to improve the reporting of research in infographics.

Abstracts of randomized controlled trials in splint therapy for temporomandibular disorders: Reporting quality and spin

BACKGROUND

Poor reporting quality and spin in randomized controlled trial (RCT) abstracts can lead to misinterpretation and distorted interpretation of results.

OBJECTIVES

This methodological study aimed to assess the reporting quality and spin among RCT abstracts on splint therapy for temporomandibular disorders (TMD) and explore the association between spin and potentially related factors.

METHODS

The authors searched PubMed for RCTs on splint therapy for TMD. The reporting quality of each abstract was assessed using the original 16-item CONSORT for abstracts checklist. The authors evaluated the presence and characteristics of spin only in abstracts with nonsignificant primary outcomes according to pre-determined spin strategies. Logistic regression analyses were performed to identify factors associated with the presence of spin.

RESULTS

A total of 148 abstracts were included in the reporting quality evaluation. The mean overall CONSORT score (OCS) was 5.86 (score range: 0-16). Only interventions, objectives and conclusions were adequately reported. Of the 61 RCT abstracts included for spin analysis, spin was identified in 38 abstracts (62.3%), among which 32 abstracts (52.3%) had spin in the Results section and 21 (34.4%) had spin in the Conclusions section. A significantly lower presence of spin was found in studies with exact p-value reporting (OR: 0.170; 95% CI: 0.032-0.887; p = .036) and a two-arm comparison design (OR: 11.777; 95% CI: 2.171-63.877; p = .004).

CONCLUSIONS

The reporting quality of RCT abstracts on splint therapy for TMD is suboptimal and the prevalence of spin is high. More awareness and joint efforts are needed to improve reporting quality and minimize spin.

Food justice in Vermont's environmentally vulnerable communities

In this study, we examine cases of food insecurity and food justice issues in Vermont's environmentally vulnerable communities. Using a structured door-to-door survey (n = 569), semi-structured interviews (n = 32), and focus groups (n = 5), we demonstrate that: (1) food insecurity in Vermont's environmentally vulnerable communities is prominent and intersects with socioeconomic factors such as race and income, (2) food and social assistance programs need to be more accessible and address vicious cycles of multiple injustices, (3) an intersectional approach beyond distribution is required to address food justice issues in environmentally vulnerable communities, and (4) paying attention to broader contextual and environmental factors may provide a more nuanced approach to understanding food justice.

The Impact of Information Presentation and Cognitive Dissonance on Processing Systematic Review Summaries: A Randomized Controlled Trial on Bicycle Helmet Legislation

Background: Summaries of systematic reviews are a reference method for the dissemination of research evidence on the effectiveness of public health interventions beyond the scientific community. Motivated reasoning and cognitive dissonance may interfere with readers' ability to process the information included in such summaries. Methods: We conducted a web experiment on a panel of university-educated North Americans (N = 259) using a systematic review of the effectiveness of bicycle helmet legislation as a test case. The outcome variables were the perceived tentativeness of review findings and attitude toward bicycle helmet legislation. We manipulated two types of uncertainty: (i) deficient uncertainty (inclusion vs. non-inclusion of information on limitations of the studies included in the review) and (ii) consensus uncertainty (consensual findings showing legislation effectiveness vs. no evidence of effectiveness). We also examined whether reported expertise in helmet legislation and the frequency of wearing a helmet while cycling interact with the experimental factors. Results: None of the experimental manipulations had a main effect on the perceived tentativeness. The presentation of consensual efficacy findings had a positive main effect on the attitude toward the legislation. Self-reported expertise had a significant main effect on the perceived tentativeness, and exposing participants with reported expertise to results showing a lack of evidence of efficacy increased their favorable attitude toward the legislation. Participants' helmet use was positively associated with their attitude toward the legislation (but not with perceived tentativeness). Helmet use did not interact with the experimental manipulations. Conclusions: Motivated reasoning and cognitive dissonance influence a reader's ability to process information contained in a systematic review summary.

ANALYSIS OF SPIN IN VASCULAR SURGERY RANDOMIZED CONTROLLED TRIALS WITH NONSIGNIFICANT OUTCOMES

OBJECTIVES

Spin is the manipulation of language that distorts the interpretation of objective findings. The purpose of this study is to describe the characteristics of spin found in statistically nonsignificant randomized controlled trials (RCT) comparing carotid endarterectomy (CEA) to carotid artery stenting (CAS) for carotid stenosis (CS), and endovascular repair (EVAR) to open repair (OR) for abdominal aortic aneurysms (AAA).

METHODS

A search of MEDLINE, EMBASE, and the Cochrane Controlled Register of Trials was performed in June 2020 for studies published describing AAA or CS. All phase three RCTs with nonsignificant primary outcomes comparing OR to EVAR or CEA to CAS were included. Studies were appraised for the characteristics and severity of spin using a validated tool. Binary logistic regression was performed to assess the association of spin grade to (1) funding source (commercial vs non-commercial) and (2) the publishing journal's impact factor.

RESULTS

Thirty-one of 355 articles captured were included for analysis. Spin was identified in nine abstracts (9/18) and 13 main texts (13/18) of AAA articles and seven abstracts (7/13) and ten main texts (10/13) of CS articles. For both AAA and CS articles, spin was most commonly found in the manuscript discussion section, with the most commonly employed strategy being the interpretation of statistically nonsignificant primary results to show treatment equivalence or rule out adverse treatment effects. Increasing journal impact factor was associated with a statistically significant lower likelihood of spin in the study title or abstract conclusion (βOR = 0.96, 95% CI: 0.94 - 0.98, p < 0.01) while no significant association could be found with funding source (βOR = 1.33, 95% CI: 0.30-5.92, p = 0.71).

CONCLUSIONS

A large proportion of statistically nonsignificant RCTs contain interpretations that are inconsistent with their results. These findings should prompt authors and readers to appraise study findings independently and to limit the use of spin in study interpretations.

An acute change? Does treatment within a therapeutic community for personality disorders affect local acute service use

Purpose

The intensive 18-month treatment in the personality disorder (PD) therapeutic community (TC) is felt to offer improvement in many aspects of patients’ lives. This study aims to understand if the use of acute services was also affected via a service evaluation project.

Design/methodology/approach

The authors collected data from electronic records on the use of local services in the two years before, during and the two years after treatment in the TC. Specifically, the authors counted inpatient bed days, Emergency department (ED) presentations and days under home treatment team and liaison psychiatry; the authors used ANOVA to analyse the data.

Findings

The study included 25 adult service users, 17 female and 8 male, with an average age of 40. Whilst there were reductions in the use of inpatient beds and ED presentations, on analysis, these were not found statistically significant. The small size of the study is a limitation and may limit the generalisability of the findings. The study concludes there may be reductions in acute psychiatric service use during and after treatment in the TC. The findings were not statistically significant; the authors suggest larger multi-centre studies may be able to demonstrate statistical significance.

Originality/value

PD patients have a relatively high use of acute psychiatric services compared to other patient groups. The authors are not aware of any similar studies in the published literature.

A group randomized control trial to test the efficacy of the Road to Mental Readiness (R2MR) program among Canadian military recruits

BACKGROUND

Despite increased interest in workplace mental health interventions, the evidence for beneficial effects is mixed. Furthermore, many existing studies lack methodological rigor. We report results from a group randomized control trial to test the efficacy of a vastly popular intervention in Canada, the Road to Mental Readiness (R2MR) program, which has been widely disseminated in military, first responder, and civilian settings.

METHODS

The trial took place among Canadian Armed Forces military recruits completing their basic military qualification (BMQ) training, and randomized 65 platoons (N = 2831) into either (a) an Intervention (R2MR at week 2 of BMQ), or (b) a delayed Intervention Control (R2MR at week 9 of BMQ) condition. The principal investigator, participants, and data collection staff were blinded to platoon condition. Individual-level psychological functioning, resilience, mental health service use attitudes, intentions, and behaviours, and additional covariates were assessed with questionnaires around week 2 (a day or two before Intervention platoons received R2MR), at week 5, and at week 9 (a day or two before the Control platoons received R2MR). Military performance outcomes were obtained from administrative databases.

RESULTS

The full trial results were mixed; for some outcomes (psychological functioning, resilience, and military performance), we saw no evidence of beneficial effects; where we did see benefits (mental health service use attitudes, intentions, behaviours), the effects were very small, or disappeared over time. Analyses among two subsamples (Group 1: Intervention platoons with a Fidelity Check and their Controls, and Group 2: Intervention platoons without Fidelity Check and their Controls) indicated that for some outcomes (attitudes and help-seeking), under high fidelity conditions, the beneficial effects of R2MR were increased and better sustained; Conversely, under poor fidelity conditions, decreased beneficial effects or even iatrogenic effects were observed. Analyses across three training divisions indicated the larger organizational climate further influences efficacy.

CONCLUSIONS

Our findings paint a very complex picture in which it is made evident that sensible, evidence-informed workplace mental health interventions such as R2MR may work under high fidelity conditions, but may yield no discernable benefit or even inadvertent iatrogenic effects if implemented poorly or without sufficient consideration to the larger organizational context.

TRIAL REGISTRATION

ISRCTN 52557050 Registered 13 October 2016.

Level and Prevalence of Spin in Published Cardiovascular Randomized Clinical Trial Reports With Statistically Nonsignificant Primary Outcomes: A Systematic Review

Importance

Clinical researchers are obligated to present results objectively and accurately to ensure readers are not misled. In studies in which primary end points are not statistically significant, placing a spin, defined as the manipulation of language to potentially mislead readers from the likely truth of the results, can distract the reader and lead to misinterpretation and misapplication of the findings.

Objective

To determine the level and prevalence of spin in published reports of cardiovascular randomized clinical trial (RCT) reports.

Data Source

MEDLINE was searched from January 1, 2015, to December 31, 2017, using the Cochrane highly sensitive search strategy.

Study Selection

Inclusion criteria were parallel-group RCTs published from January 1, 2015, to December 31, 2017 in 1 of 6 high-impact journals (New England Journal of Medicine, The Lancet, JAMA, European Heart Journal, Circulation, and Journal of the American College of Cardiology) with primary outcomes that were not statistically significant were included in the analysis.

Data Extraction and Synthesis

Analysis began in August 2018. Data were extracted and verified by 2 independent investigators using a standard collection form. In cases of disagreement between the 2 investigators, a third investigators served as arbitrator.

Main Outcomes and Measures

The classifications of spin type, severity, and extent were determined according to predefined criteria. Primary clinical outcomes were divided into safety of treatment, efficacy of treatment, and both.

Results

Of 587 studies identified, 93 RCT reports (15.8%) met inclusion criteria. Spin was identified in 53 abstracts (57%; 95% CI, 47%-67%) and 62 main texts of published articles (67%; 95% CI, 57%-75%). Ten reports (11%; 95% CI, 6%-19%) had spin in the title, 35 reports (38%; 95% CI, 28%-48%) had spin in the results section, and 50 reports (54%; 95% CI, 44%-64%) had spin in the conclusions. Among the abstracts, spin was observed in 38 results sections (41%; 95% CI, 31%-51%) and 45 conclusions sections (48%; 95% CI, 38%-58%).

Conclusions and Relevance

This study suggests that in reports of cardiovascular RCTs with statistically nonsignificant primary outcomes, investigators often manipulate the language of the report to detract from the neutral primary outcomes. To best apply evidence to patient care, consumers of cardiovascular research should be aware that peer review does not always preclude the use of misleading language in scientific articles.

Reporting of Randomized Controlled Trials With Statistically Nonsignificant Primary Outcomes Published in High-impact Surgical Journals

OBJECTIVE

To determine the nature and frequency of distorted presentation or "spin" (ie, specific reporting strategies which highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or distract the reader from statistically nonsignificant results) in published reports of randomized controlled trials (RCTs) with statistically nonsignificant results for primary outcomes in surgical journals.

BACKGROUND

Multiple reports have suggested that interpretation of RCT results in medical journals can be distorted by authors of published reports.

METHODS

Using a defined search strategy, RCTs with clearly nonsignificant results for the primary outcome (P > 0.05) form 10 high-impact factor surgical journals (Annals of Surgery, Journal of Neurology, Neurosurgery and Psychiatry, Journal of Heart and Lung Transplantation, American Journal of Transplantation, British Journal of Surgery, Journal of Bone and Joint Surgery, Journal of the American College of Surgeons, Endoscopy, Archives of Surgery, and Liver transplantation), published between July 2013 to July 2015, were identified. Two reviewers independently appraised each selected article using a validated, standardized data abstraction form.

RESULTS

In all, 110 eligible RCTs with nonsignificant primary outcomes were appraised. The title was reported with spin in 8 (7%) articles. Forty-four (40%) included abstracts and 39 (35%) main texts were classified as having spin in at least 1 section. The level of spin was high in 16 (14%) abstract and 19 (19%) main-text "Conclusions" sections. Twenty-five articles (23%) recommended the intervention of interest despite a nonsignificant primary outcome. There was no relationship between trial funding source, use of statistician and article section, and the presence of spin.

CONCLUSIONS

In RCTs with statistically nonsignificant primary outcomes published in surgical journals, the reporting and interpretation of findings was frequently inconsistent with the results.

Primary endpoint discrepancies were found in one in ten clinical drug trials. Results of an inception cohort study

OBJECTIVE

To identify the occurrence and determinants of protocol-publication discrepancies in clinical drug trials.

STUDY DESIGN AND SETTING

All published clinical drug trials reviewed by the Dutch institutional review boards in 2007 were analyzed. Discrepancies between trial protocols and publications were measured among key reporting aspects. We evaluated the association of trial characteristics with discrepancies in primary endpoints by calculating the risk ratio (RR) and 95% confidence interval (CI).

RESULTS

Of the 334 published trials, 32 (9.6%) had a protocol/publication discrepancy in the primary endpoints. Among the subgroup of randomized controlled trials (RCTs; N = 204), 12 (5.9%) had a discrepancy in the primary endpoint. Investigator-initiated trials with and without industry (co-) funding were associated with having discrepancies in the primary endpoints compared with industry-sponsored trials (RR 3.7; 95% CI 1.4-9.9 and RR 4.4; 95% CI 2.0-9.5, respectively). Furthermore, other than phase 1-4 trials (vs. phase 1; RR 4.6; 95% CI 1.1-19.3), multicenter trials were also conducted outside the European Union (vs. single center; RR 0.2; 95% CI 0.1-0.6), not prospectively registered trials (RR 3.3; 95% CI 1.5-7.5), non-RCTs (vs. superiority RCT; RR 2.4; 95% CI 1.2-4.8) and, among the RCTs, crossover compared with a parallel group design (RR 3.7; 95% CI 1.1-12.3) were significantly associated with having discrepancies in the primary endpoints.

CONCLUSIONS

Improvement in completeness of reporting is still needed, especially among investigator-initiated trials and non-RCTs. To eliminate undisclosed discrepancies, trial protocols should be available in the public domain at the same time when the trial is published.

Non-Publication Is Common among Phase 1, Single-Center, Not Prospectively Registered, or Early Terminated Clinical Drug Trials

The objective of this study was to investigate the occurrence and determinants of non-publication of clinical drug trials in the Netherlands.All clinical drug trials reviewed by the 28 Institutional Review Boards (IRBs) in the Netherlands in 2007 were followed-up from approval to publication. Candidate determinants were the sponsor, phase, applicant, centers, therapeutic effect expected, type of trial, approval status of the drug(s), drug type, participant category, oncology or other disease area, prospective registration, and early termination. The main outcome was publication as peer reviewed article. The percentage of trials that were published, crude and adjusted odds ratio (OR), and 95% confidence interval (CI) were used to quantify the associations between determinants and publication. In 2007, 622 clinical drug trials were reviewed by IRBs in the Netherlands. By the end of follow-up, 19 of these were rejected by the IRB, another 19 never started inclusion, and 10 were still running. Of the 574 trials remaining in the analysis, 334 (58%) were published as peer-reviewed article. The multivariable logistic regression model identified the following determinants with a robust, statistically significant association with publication: phase 2 (60% published; adjusted OR 2.6, 95% CI 1.1-5.9), phase 3 (73% published; adjusted OR 4.1, 95% CI 1.7-10.0), and trials not belonging to phase 1-4 (60% published; adjusted OR 3.2, 95% CI 1.5 to 6.5) compared to phase 1 trials (35% published); trials with a company or investigator as applicant (63% published) compared to trials with a Contract Research Organization (CRO) as applicant (50% published; adjusted OR 1.7; 95% CI 1.1-2.8); and multicenter trials also conducted in other EU countries (68% published; adjusted OR 2.2, 95% CI 1.1-4.4) or also outside the European Union (72% published; adjusted OR 2.0, 95% CI 1.0-4.0) compared to single-center trials (45% published). Trials that were not prospectively registered (48% published) had a lower likelihood of publication compared to prospectively registered trials (75% published; adjusted OR 0.5, 95% CI 0.3-0.8), as well as trials that were terminated early (33% published) compared to trials that were completed as planned (64% published; adjusted OR 0.2, 95% CI 0.1-0.3). The non-publication rate of clinical trials seems to have improved compared to previous inception cohorts, but is still far from optimal, in particular among phase 1, single-center, not prospectively registered, and early terminated trials.

Outcome Reporting Bias in Government-Sponsored Policy Evaluations: A Qualitative Content Analysis of 13 Studies

The reporting of evaluation outcomes can be a point of contention between evaluators and policy-makers when a given reform fails to fulfil its promises. Whereas evaluators are required to report outcomes in full, policy-makers have a vested interest in framing these outcomes in a positive light–especially when they previously expressed a commitment to the reform. The current evidence base is limited to a survey of policy evaluators, a study on reporting bias in education research and several studies investigating the influence of industry sponsorship on the reporting of clinical trials. The objective of this study was twofold. Firstly, it aimed to assess the risk of outcome reporting bias (ORB or ‘spin’) in pilot evaluation reports, using seven indicators developed by clinicians. Secondly, it sought to examine how the government’s commitment to a given reform may affect the level of ORB found in the corresponding evaluation report. To answer these questions, 13 evaluation reports were content-analysed, all of which found a non-significant effect of the intervention on its stated primary outcome. These reports were systematically selected from a dataset of 233 pilot and experimental evaluations spanning three policy areas and 13 years of government-commissioned research in the UK. The results show that the risk of ORB is real. Indeed, all studies reviewed here resorted to at least one of the presentational strategies associated with a risk of spin. This study also found a small, negative association between the seniority of the reform’s champion and the risk of ORB in the evaluation of that reform. The publication of protocols and the use of reporting guidelines are recommended.

Peer reviewers identified spin in manuscripts of nonrandomized studies assessing therapeutic interventions, but their impact on spin in abstract conclusions was limited

OBJECTIVES

To describe the impact of peer reviewers on spin in reports of nonrandomized studies assessing a therapeutic intervention.

STUDY DESIGN AND SETTING

This is a systematic review and retrospective before-after study. The sample consists of primary reports (n = 128) published in BioMed Central Medical Series journals between January 1, 2011, and December 31, 2013. The main outcome measures are the following: number and type of spin examples identified, deleted, or added by peer reviewers in the whole manuscript; number of reports with spin in abstract conclusions not detected by peer reviewers; the level of spin (i.e., no, low, moderate, and high level of spin) in the abstract conclusions before and after the peer review.

RESULTS

For 70 (55%) submitted manuscripts, peer reviewers identified at least one example of spin. Of 123 unique examples of spin identified by peer reviewers, 82 (67%) were completely deleted by the authors. For 19 articles (15%), peer reviewers requested adding some spin, and for 11 (9%), the spin was added by the authors. Peer reviewers failed to identify spin in abstract conclusions of 97 (76%) reports.

CONCLUSION

Peer reviewers identified many examples of spin in submitted manuscripts. However, their influence on changing spin in the abstract conclusions was low.

Differences in antibiotic use between patients with and without a regular doctor in Hong Kong

BACKGROUND

Literature shows that continuity of care from a primary care physician is associated with better patient satisfaction and preventive care. This may also have an effect on patients' use of antibiotics. This study investigated the differences in antibiotic use between patients with and without a regular doctor in a pluralistic health care system.

METHODS

A cross-sectional telephone questionnaire survey using randomly selected household phone numbers was conducted in Hong Kong. Several key areas about antibiotic use were compared between the respondents with a regular doctor and those without.

RESULTS

The response rate was 68.3 %. Of the 2,471 respondents, 1,450 (58.7 %) had a regular doctor, 942 (38.1 %) without, and 79 (3.2 %) did not give a clear answer. The respondents with a regular doctor were more likely to report that they always finished the full course of antibiotics (74.2 % vs 62.4 %), as well as using antibiotics for their last upper respiratory tract infections (17.4 % vs 10.1 %). The association with antibiotic use remained significant in the multivariable logistic regression analysis after adjusting for other confounding factors (P < 0.001, OR = 1.76, 95 % CI:(1.27, 2.48)).

CONCLUSIONS

While patients with a regular doctor, compared to those without, were more likely to report finishing the full course of antibiotics, they also had nearly twice the chance of reporting antibiotic use for upper respiratory tract infections. This challenges the common belief of the benefits in having a regular doctor.

Interpretation of Results of Studies Evaluating an Intervention Highlighted in Google Health News: A Cross-Sectional Study of News

BACKGROUND

Mass media through the Internet is a powerful means of disseminating medical research. We aimed to determine whether and how the interpretation of research results is misrepresented by the use of "spin" in the health section of Google News. Spin was defined as specific way of reporting, from whatever motive (intentional or unintentional), to emphasize that the beneficial effect of the intervention is greater than that shown by the results.

METHODS

We conducted a cross-sectional study of news highlighted in the health section of US, UK and Canada editions of Google News between July 2013 and January 2014. We searched for news items for 3 days a week (i.e., Monday, Wednesday, and Friday) during 6 months and selected a sample of 130 news items reporting a scientific article evaluating the effect of an intervention on human health.

RESULTS

In total, 78% of the news did not provide a full reference or electronic link to the scientific article. We found at least one spin in 114 (88%) news items and 18 different types of spin in news. These spin were mainly related to misleading reporting (59%) such as not reporting adverse events that were reported in the scientific article (25%), misleading interpretation (69%) such as claiming a causal effect despite non-randomized study design (49%) and overgeneralization/misleading extrapolation (41%) of the results such as extrapolating a beneficial effect from an animal study to humans (21%). We also identified some new types of spin such as highlighting a single patient experience for the success of a new treatment instead of focusing on the group results.

CONCLUSIONS

Interpretation of research results was frequently misrepresented in the health section of Google News. However, we do not know whether these spin were from the scientific articles themselves or added in the news.

Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention

Background

Spin represents specific reporting strategies, either intentional or unintentional, to convince the reader that the beneficial effect of the experimental intervention in terms of efficacy and safety is greater than that shown by the results. The objectives of this study were to 1) develop a classification of spin specific to non-randomized studies assessing an intervention and 2) estimate the prevalence of spin in abstracts of reports of such studies.

Methods

In a first step, we developed a specific classification of spin for non-randomized studies by a literature review and pilot study. In a second step, 2 researchers trained in the field of methodology evaluated the prevalence of spin in the abstract of all non-randomized studies assessing an intervention published in the BioMed Central Medical Series journals between January 1, 2011 and December 31, 2013. All disagreements were resolved by consensus. We also determined whether the level of spin in abstract conclusions was high (spin reported without uncertainty or recommendations for further trials), moderate (spin reported with some uncertainty or recommendations for further trials) or low (spin reported with uncertainty and recommendations for further trials).

Results

Among the 128 assessed articles assessed, 107 (84 %) had at least one example of spin in their abstract. The most prevalent strategy of spin was the use of causal language, identified in 68 (53 %) abstracts. Other frequent strategies were linguistic spin, inadequate implications for clinical practice, and lack of focus on harm, identified in 33 (26 %), 25 (20 %), and 34 (27 %) abstracts respectively. Abstract conclusions of 61 (48 %) articles featured a high level of spin.

Conclusion

Abstract of reports of non-randomized studies assessing an intervention frequently includes spin. Efforts to reduce the prevalence of spin in abstract for such studies are needed.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-015-0079-x) contains supplementary material, which is available to authorized users.

Evaluation of holistic sexuality education: A European expert group consensus agreement

OBJECTIVES

Holistic sexuality education (HSE) is a new concept in sexuality education (SE). Since it differs from other types of SE in a number of important respects, strategies developed for the evaluation of the latter are not necessarily applicable to HSE. In this paper the authors provide a basis for discussion on how to evaluate HSE.

METHODS

First, the international literature on evaluation of SE in general was reviewed in terms of its applicability to HSE. Second, the European Expert Group on Sexuality Education extensively discussed the requirements of its evaluation and suggested appropriate indicators and methods for evaluating HSE.

RESULTS

The European experience in SE is scarcely represented in the general evaluation literature. The majority of the literature focuses on impact and neglects programme and implementation evaluations. Furthermore, the current literature demonstrates that evaluation criteria predominantly focus on the public health impact, while there is not yet a consensus on sexual well-being criteria and aspects of positive sexuality, which are crucial parts of HSE. Finally, experimental designs are still considered the gold standard, yet several of the conditions for their use are not fulfilled in HSE. Realising that a new evaluation framework for HSE is needed, the European expert group initiated its development and agreed upon a number of indicators that provide a starting point for further discussion.

CONCLUSIONS

Aside from the health impact, the quality of SE programmes and their implementation also deserve attention and should be evaluated. To be applicable to HSE, the evaluation criteria need to cover more than the typical public health aspects. Since they do not register long-term and multi-component characteristics, evaluation methods such as randomised controlled trials are not sufficiently suitable for HSE. The evaluation design should rely on a number of different information sources from mixed methods that are complemented and triangulated to build a plausible case for the effectiveness of SE in general and HSE in particular.

"Spin" in wound care research: the reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes

BACKGROUND

Spin in the reporting of randomized controlled trials, where authors report research in a way that potentially misrepresents results and mislead readers, has been demonstrated in the broader medical literature. We investigated spin in wound care trials with (a) no statistically significant result for the primary outcome and (b) no clearly specified primary outcome.

METHODS

We searched the Cochrane Wounds Group Specialised Register of Trials for randomized controlled trials (RCTs). Eligible studies were: Parallel-group RCTs of interventions for foot, leg or pressure ulcers published in 2004 to 2009 (inclusive) with either a clearly identified primary outcome for which there was a statistically non-significant result (Cohort A) or studies that had no clear primary outcome (Cohort B).We extracted general study details. For both Cohorts A and B we then assessed for the presence of spin. For Cohort A we used a pre-defined process to assess reports for spin. For Cohort B we aimed to assess spin by recording the number of positive treatment effect claims made. We also compared the number of statistically significant and non-significant results reported in the main text and the abstract looking specifically for spin in the form of selective outcome reporting.

RESULTS

Of the 71 eligible studies, 28 were eligible for Cohort A; of these, 71% (20/28) contained spin. Cohort B contained 43 studies; of these, 86% (37/43) had abstracts that claimed a favorable treatment claim. Whilst 74% (32/43) of main text results in Cohort B included at least one statistically non-significant result, this was not reflected in the abstract where only 28% contained (12/43) at least one statistically non-significant result.

CONCLUSIONS

Spin is a frequent phenomenon in reports of RCTs of wound treatments. Studies without statistically significant results for the primary outcome used spin in 71% of cases. Furthermore, 33% (43/132) of reports of wound RCTs did not specify a primary outcome and there was evidence of spin and selective outcome reporting in the abstracts of these. Readers should be wary of only reading the abstracts of reports of RCTs of wound treatments since they are frequently misleading regarding treatment effects.

Modification of the home environment for the reduction of injuries

BACKGROUND

Injury in the home is common, accounting for approximately a third of all injuries. The majority of injuries to children under five and people aged 75 and older occur at home. Multifactorial injury prevention interventions have been shown to reduce injuries in the home. However, few studies have focused specifically on the impact of physical adaptations to the home environment and the effectiveness of such interventions needs to be ascertained.

OBJECTIVES

To determine the effect of modifications to the home environment on the reduction of injuries due to environmental hazards.

SEARCH STRATEGY

We searched The Cochrane Library, MEDLINE, EMBASE and other specialised databases. We also scanned conference proceedings and reference lists. We contacted the first author of all included randomised controlled trials. The searches were last updated to the end of December 2009, and were not restricted by language or publication status.

SELECTION CRITERIA

Randomised controlled trials.

DATA COLLECTION AND ANALYSIS

Two authors screened all abstracts for relevance, outcome and design. Two authors independently assessed methodological quality and extracted data from each eligible study. We performed meta-analysis to combine effect measures, using a random-effects model. We assessed heterogeneity using an I(2) statistic and a Chi(2) test.

MAIN RESULTS

We found 28 published studies and one unpublished study. Only two studies were sufficiently similar to allow pooling of data for statistical analyses. Studies were divided into three groups; children, older people and the general population/mixed age group. None of the studies focusing on children or older people demonstrated a reduction in injuries that were a direct result of environmental modification in the home. One study in older people demonstrated a reduction in falls and one a reduction in falls and injurious falls that may have been due to hazard reduction. One meta-analysis was performed which examined the effects on falls of multifactorial interventions consisting of home hazard assessment and modification, medication review, health and bone assessment and exercise (RR 1.09, 95% CI 0.97 to 1.23).

AUTHORS' CONCLUSIONS

There is insufficient evidence to determine whether interventions focused on modifying environmental home hazards reduce injuries. Further interventions to reduce hazards in the home should be evaluated by adequately designed randomised controlled trials measuring injury outcomes. Recruitment of large study samples to measure effect must be a major consideration for future trials. Researchers should also consider using factorial designs to allow the evaluation of individual components of multifactorial interventions.

The effect on caregiver burden of a problem-based home visiting programme for frail older people

OBJECTIVE

caregiver effects of geriatric care models focusing primarily at the patient have not been consistently studied. We studied caregiver effects of a nurse-led comprehensive geriatric evaluation and management (GEM) programme for community-dwelling frail older people that showed-in a randomised comparison with usual care--health-related quality of life benefits for the care receivers.

METHODS

this randomised trial included 110 caregiver/patient dyads who were followed up for 6 months. Primary analyses were intention-to-treat analyses of caregiver burden assessed with Zarit Burden Interview (ZBI; 0-88; higher means more burden). Preplanned subgroup analyses were conducted for cognition, living arrangement and patient/caregiver co-residence.

RESULTS

overall, perceived caregiver burden showed no significant differences between study groups in changes over time. However, perceived burden was at baseline more than eight points higher in caregivers sharing a household with patients (n = 23) compared to caregivers living separately (n = 87). The intervention performed better in caregivers living together with the patient than in caregivers living separately (P for interaction = 0.04). Co-resident caregivers experienced six-Zarit point improvement compared with four-point deterioration in the non-co-resident caregivers.

CONCLUSIONS

GEM at home benefited patients, but maybe not caregivers. Caregiver effects are related to whether caregivers live with the patient or not.

Modification of the home environment for the reduction of injuries

BACKGROUND

Injury in the home is extremely common, accounting for around a third of all injuries. The majority of injuries of children under five and people aged 75 and over, occur at home. Multifactorial injury prevention interventions have been shown to reduce injuries in the home. However, few studies have focused specifically on the impact of physical adaptations to the home environment and the effectiveness of such interventions needs to be ascertained.

OBJECTIVES

To review the evidence for the effect on injuries of modification of the home environment with a primary focus on interventions to reduce physical hazards.

SEARCH STRATEGY

We searched The Cochrane Library, MEDLINE, EMBASE, National Research Register and other specialised databases. We also scanned conference proceedings and reference lists. In addition, we contacted experts and trialists in the field. The searches were not restricted by language or publication status. The searches were last updated in December 2004.

SELECTION CRITERIA

Randomised controlled trials.

DATA COLLECTION AND ANALYSIS

All abstracts were screened by two authors for relevance, outcome and design. Two authors independently assessed methodological quality and extracted data from each eligible study.

MAIN RESULTS

We found 18 published and one unpublished trials. Trials were not sufficiently similar to allow pooling of data by statistical analyses, so this review takes a narrative form. Studies were divided into three groups based on the primary population sample; children (five studies), older people (14 studies) and the general population/mixed age group (no studies). None of the studies focusing on children demonstrated a reduction in injuries that might have been due to environmental adaptation in the home; one study reported a reduction in injuries and in hazards but the two could not be linked. Of the 14 included studies in older people, none demonstrated a reduction in injuries due to hazard reduction, although two demonstrated a reduction in falls that could be due to hazard reduction.

AUTHORS' CONCLUSIONS

There is insufficient evidence to determine the effects of interventions to modify environmental home hazards. Further interventions to reduce hazards in the home should be evaluated by adequately designed randomised controlled trials measuring injury outcomes. Recruitment of large study samples to measure effect must be a major consideration for future trials.