Evolution of treatment effects over time: empirical insight from recursive cumulative metaanalyses

Effect of different non-pharmacologic placebo treatments on migraine prevention: a network meta-analysis of randomized controlled trials

BACKGROUND

Placebo control plays an important role in evaluating the effectiveness of interventions. Specifying differential effects of various placebo controls on migraine prevention would be essential in the explanation of preventive treatment for migraine and the indirect comparison between different prophylactic therapeutics.

OBJECTIVES

To access the impact of different non-pharmacologic placebo types on different outcomes in migraine patients.

METHODS

We searched PubMed, Cochrane Controlled Register of Trials, Embase, and Web of Science databases from the date of creation to June 19, 2023. Randomized controlled trials of migraine that included sham intervention of acupuncture or cognitive behavioural therapy (CBT) or non-invasive Vagus Nerve Stimulation (nVNS) or repetitive Transcranial Magnetic Stimulation (rTMS) or transcranial Direct Current Stimulation (tDCS) were conducted. The primary outcome was the migraine days, and the secondary outcomes were the number of migraine attacks, headache days, headache frequency, and responder's rate. Placebo effects were assessed using five individual placebos for network meta-analysis, using mean differences to measure the relative effect of pair-wise comparisons between interventions.

RESULT

A total of 50 trials with 4880 subjects were included. Twenty-seven trials were evaluated for low risk of bias. The results of indirect comparisons show that sham rTMS and sham tDCS had optimal and similar effects in reducing migraine days; sham acupuncture has the greatest effect on reducing the number of migraine attacks and relieving headache frequency; sham rTMS had a highly significant advantage in reducing headache days compared with the other placebo controls.

CONCLUSION

Based on the network meta-analysis results, we found that sham acupuncture had the greatest effect on migraine prophylaxis. The strong placebo effect of sham acupuncture should be considered when assessing the therapeutic effect.

Deprescribing fall-risk increasing drugs (FRIDs) for the prevention of falls and fall-related complications: a systematic review and meta-analysis

OBJECTIVES

Prevention of falls and fall-related injuries is a priority due to the substantial health and financial burden of falls on patients and healthcare systems. Deprescribing medications known as 'fall-risk increasing drugs' (FRIDs) is a common strategy to prevent falls. We conducted a systematic review to determine its efficacy for the prevention of falls and fall-related complications.

DESIGN

Systematic review and meta-analysis.

DATA SOURCES

MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, CINAHL and grey literature from inception to 1 August 2020.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Randomised controlled trials of FRID withdrawal compared with usual care evaluating the rate of falls, incidence of falls, fall-related injuries, fall-related fractures, fall-related hospitalisations or adverse effects related to the intervention in adults aged ≥65 years.

DATA EXTRACTION AND SYNTHESIS

Two reviewers independently performed citation screening, data abstraction, risk of bias assessment and certainty of evidence grading. Random-effects models were used for meta-analyses.

RESULTS

Five trials involving 1305 participants met eligibility criteria. Deprescribing FRIDs did not change the rate of falls (rate ratio (RaR) 0.98, 95% CI 0.63 to 1.51), the incidence of falls (risk difference 0.01, 95% CI -0.06 to 0.09; relative risk 1.04, 95% CI 0.86 to 1.26) or rate of fall-related injuries (RaR 0.89, 95% CI 0.57 to 1.39) over a follow-up period of 6-12 months. No trials evaluated the impact of deprescribing FRIDs on fall-related fractures or hospitalisations.

CONCLUSION

There is a paucity of robust high-quality evidence to support or refute that a FRID deprescribing strategy alone is effective at preventing falls or fall-related injury in older adults. Although there may be other reasons to deprescribe FRIDs, our systematic review found that it may result in little to no difference in the rate or risk of falls as a sole falls reduction strategy.

PROSPERO REGISTRATION NUMBER

CRD42016040203.

Intuitive Honesty Versus Dishonesty: Meta-Analytic Evidence

Is self-serving lying intuitive? Or does honesty come naturally? Many experiments have manipulated reliance on intuition in behavioral-dishonesty tasks, with mixed results. We present two meta-analyses (with evidential value) testing whether an intuitive mind-set affects the proportion of liars (k = 73; n = 12,711) and the magnitude of lying (k = 50; n = 6,473). The results indicate that when dishonesty harms abstract others, promoting intuition causes more people to lie, log odds ratio = 0.38, p = .0004, and people to lie more, Hedges's g = 0.26, p < .0001. However, when dishonesty inflicts harm on concrete others, promoting intuition has no significant effect on dishonesty (p > .63). We propose one potential explanation: The intuitive appeal of prosociality may cancel out the intuitive selfish appeal of dishonesty, suggesting that the social consequences of lying could be a promising key to the riddle of intuition's role in honesty. We discuss limitations such as the relatively unbalanced distribution of studies using concrete versus abstract victims and the overall large interstudy heterogeneity.

Pragmatic clinical trials in the context of regulation of medicines

The pragmatic clinical trial addresses scientific questions in a setting close to routine clinical practice and sometimes using routinely collected data. From a regulatory perspective, when evaluating a new medicine before approving marketing authorization, there will never be enough patients studied in all subgroups that may potentially be at higher risk for adverse outcomes, or sufficient patients to detect rare adverse events, or sufficient follow-up time to detect late adverse events that require long exposure times to develop. It may therefore be relevant that post-marketing trials sometimes have more pragmatic characteristics, if there is a need for further efficacy and safety information. A pragmatic study design may reflect a situation close to clinical practice, but may also have greater potential methodological concerns, e.g. regarding the validity and completeness of data when using routinely collected information from registries and health records, the handling of intercurrent events, and misclassification of outcomes. In a regulatory evaluation it is important to be able to isolate the effect of a specific product or substance, and to have a defined population that the results can be referred to. A study feature such as having a wide and permissive inclusion of patients might therefore actually hamper the utility of the results for regulatory purposes. Randomization in a registry-based setting addresses confounding that could otherwise complicate a corresponding non-interventional design, but not any other methodological issues. Attention to methodological basics can help generate reliable study results, and is more important than labelling studies as 'pragmatic'.

The intriguing evolution of effect sizes in biomedical research over time: smaller but more often statistically significant

Background

In medicine, effect sizes (ESs) allow the effects of independent variables (including risk/protective factors or treatment interventions) on dependent variables (e.g., health outcomes) to be quantified. Given that many public health decisions and health care policies are based on ES estimates, it is important to assess how ESs are used in the biomedical literature and to investigate potential trends in their reporting over time.

Results

Through a big data approach, the text mining process automatically extracted 814 120 ESs from 13 322 754 PubMed abstracts. Eligible ESs were risk ratio, odds ratio, and hazard ratio, along with their confidence intervals. Here we show a remarkable decrease of ES values in PubMed abstracts between 1990 and 2015 while, concomitantly, results become more often statistically significant. Medians of ES values have decreased over time for both “risk” and “protective” values. This trend was found in nearly all fields of biomedical research, with the most marked downward tendency in genetics. Over the same period, the proportion of statistically significant ESs increased regularly: among the abstracts with at least 1 ES, 74% were statistically significant in 1990–1995, vs 85% in 2010–2015.

Conclusions

whereas decreasing ESs could be an intrinsic evolution in biomedical research, the concomitant increase of statistically significant results is more intriguing. Although it is likely that growing sample sizes in biomedical research could explain these results, another explanation may lie in the “publish or perish” context of scientific research, with the probability of a growing orientation toward sensationalism in research reports. Important provisions must be made to improve the credibility of biomedical research and limit waste of resources.

A cumulative meta-analysis of the effects of individual physical activity interventions targeting healthy adults

Despite a large and increasing evidence base on physical activity interventions, the high rates of physical inactivity and associated chronic diseases are continuing to increase globally. The purpose of this cumulative meta-analysis was to investigate the evolution of randomized controlled trial evidence of individual-level physical activity interventions to asses if new trials are contributing novel evidence to the field. Through a two-staged search process, primary studies examining the effects of interventions targeted at increasing physical activity within healthy adult populations were pooled and selected from eligible systematic reviews. Cumulative meta-analyses were performed on effect sizes immediately post-intervention (n = 62), and for long-term behaviour change (≥12-month post-baseline; n = 27). Sufficiency and stability of the evidence was assessed through application of pre-published indicators. Meta-analyses suggest overall positive intervention effects on physical activity. The evidence base for effectiveness immediately post-intervention reached levels of sufficiency and stability in 2007; and for long-term follow-up in 2011. In the time since, intervention effectiveness has not substantially changed, and further trials are unlikely to change the direction and magnitude of effect. Substantial evidence exists demonstrating that physical activity interventions can modify individual behaviour in controlled settings. Researchers are urged to shift focus towards investigating the optimization, implementation, sustainability and cost-effectiveness of interventions.

Risk of mortality among children, adolescents, and adults with autism spectrum disorder or attention deficit hyperactivity disorder and their first-degree relatives: a protocol for a systematic review and meta-analysis of observational studies

BACKGROUND

Autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD) are childhood onset neurodevelopmental disorders that may persist into adulthood. ASD and ADHD tend to run in families and may have a significant negative impact on the health and longevity of those with the disorder and their relatives. The aim of this study will be to analyze the risk of mortality among children, adolescents, and adults with ASD or ADHD and their first-degree relatives.

METHODS/DESIGN

We will conduct a systematic review and meta-analysis of observational studies. Searches of PubMed/MEDLINE, EMBASE, PsycINFO, SCOPUS, and ISI Web of Science will be used to identify epidemiological studies. Eligible studies will be observational studies reporting study-specific data for all-cause mortality or cause-specific mortality in children, adolescents, or adults with ASD or ADHD and/or their first-degree relatives. Cohort studies and case-control studies will be included. The primary outcome will be all-cause mortality. The secondary outcome will be cause-specific mortality. Two reviewers will independently screen references identified by the literature search, as well as potentially relevant full-text articles. Data will be abstracted, and study risk of bias/methodological quality will be appraised by two reviewers independently. The methodological quality of epidemiological studies will be appraised using the Newcastle-Ottawa Scale (NOS). Conflicts at all levels of screening and abstraction will be resolved through discussion. Random-effects meta-analyses of primary studies will be conducted where appropriate. Subgroup analyses for exploring statistical heterogeneity, if feasible, will include gender, age group, ethnicity, comorbidities, classification of cause of death, and relevant study characteristics.

DISCUSSION

Our study will establish the extent of the epidemiological evidence underlying the risk of mortality among children, adolescents, and adults with ASD or ADHD and their first-degree relatives. We anticipate that our findings will be of interest to patients, their families, caregivers, healthcare professionals, scientists, and policy makers. Implications for future epidemiological research will be discussed.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42017059955 .

Anorexia nervosa and cancer: a protocol for a systematic review and meta-analysis of observational studies

BACKGROUND

Anorexia nervosa is characterized by a severe restriction of caloric intake, low body weight, fear of gaining weight or of becoming fat, and disturbance of body image. Pathogenesis of the disorder may include genetic predisposition, hormonal changes and a combination of environmental, psychosocial, and cultural factors. Cancer is the second leading cause of death worldwide. At present, no systematic reviews and meta-analyses have evaluated the risk of cancer in people with anorexia nervosa. The objective of this study will be to evaluate the association between anorexia nervosa and the risk of developing or dying from cancer.

METHODS/DESIGN

This study protocol is part of a systematic collection and assessment of multiple systematic reviews and meta-analyses (umbrella review) evaluating the association of cancer and multiple central nervous system disorders. We designed a specific protocol for a new systematic review and meta-analysis of observational studies of anorexia nervosa with risk of developing or dying from any cancer. Data sources will be PubMed, Embase, Scopus, Web of Science, and manual screening of references. Observational studies (case-control and cohort) in humans that examined the association between anorexia nervosa and risk of developing or dying from cancer will be sought. The primary outcomes will be cancer incidence and cancer mortality in association with anorexia nervosa. Secondary outcomes will be site-specific cancer incidence and mortality, respectively. Screening of abstracts and full texts, and data abstraction will be performed by two team members independently. Conflicts at all levels of screening and abstraction will be resolved through discussion. The quality of studies will be assessed by using the Ottawa-Newcastle scale by two team members independently. Random effects models will be conducted where appropriate. Subgroup and additional analyses will be conducted to explore the potential sources of heterogeneity. The World Cancer Research Fund (WCRF)/American Institute for Cancer Research (AICR) criteria and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach will be used for determining the quality of evidence for cancer outcomes.

DISCUSSION

Findings from this systematic review will inform an ongoing umbrella review on cancer and central nervous system disorders. Our systematic review and meta-analysis of observational studies will establish the extent of the epidemiological evidence underlying the association between anorexia nervosa and cancer.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42017067462.

Neuroimaging studies of GABA in schizophrenia: a systematic review with meta-analysis

Data from animal models and from postmortem studies suggest that schizophrenia is associated with brain GABAergic dysfunction. The extent to which this is reflected in data from in vivo studies of GABA function in schizophrenia is unclear. The Medline database was searched to identify articles published until 21 October 2016. The search terms included GABA, proton magnetic resonance spectroscopy (¹H-MRS), positron emission tomography (PET), single photon emission computed tomography (SPECT), schizophrenia and psychosis. Sixteen GABA ¹H-MRS studies (538 controls, 526 patients) and seven PET/SPECT studies of GABA_A/benzodiazepine receptor (GABA_A/BZR) availability (118 controls, 113 patients) were identified. Meta-analyses of ¹H-MRS GABA in the medial prefrontal cortex (mPFC), parietal/occipital cortex (POC) and striatum did not show significant group differences (mFC: g=-0.3, 409 patients, 495 controls, 95% confidence interval (CI): -0.6 to 0.1; POC: g=-0.3, 139 patients, 111 controls, 95% CI: -0.9 to 0.3; striatum: g=-0.004, 123 patients, 95 controls, 95% CI: -0.7 to 0.7). Heterogeneity across studies was high (I²>50%), and this was not explained by subsequent moderator or meta-regression analyses. There were insufficient PET/SPECT receptor availability studies for meta-analyses, but a systematic review did not suggest replicable group differences in regional GABA_A/BZR availability. The current literature does not reveal consistent alterations in in vivo GABA neuroimaging measures in schizophrenia, as might be hypothesized from animal models and postmortem data. The analysis highlights the need for further GABA neuroimaging studies with improved methodology and addressing potential sources of heterogeneity.

Cancer and central nervous system disorders: protocol for an umbrella review of systematic reviews and updated meta-analyses of observational studies

BACKGROUND

The objective of this study will be to synthesize the epidemiological evidence and evaluate the validity of the associations between central nervous system disorders and the risk of developing or dying from cancer.

METHODS/DESIGN

We will perform an umbrella review of systematic reviews and conduct updated meta-analyses of observational studies (cohort and case-control) investigating the association between central nervous system disorders and the risk of developing or dying from any cancer or specific types of cancer. Searches involving PubMed/MEDLINE, EMBASE, SCOPUS and Web of Science will be used to identify systematic reviews and meta-analyses of observational studies. In addition, online databases will be checked for observational studies published outside the time frames of previous reviews. Eligible central nervous system disorders will be Alzheimer's disease, anorexia nervosa, amyotrophic lateral sclerosis, autism spectrum disorders, bipolar disorder, depression, Down's syndrome, epilepsy, Huntington's disease, multiple sclerosis, Parkinson's disease and schizophrenia. The primary outcomes will be cancer incidence and cancer mortality in association with a central nervous system disorder. Secondary outcome measures will be site-specific cancer incidence and mortality, respectively. Two reviewers will independently screen references identified by the literature search, as well as potentially relevant full-text articles. Data will be abstracted, and study quality/risk of bias will be appraised by two reviewers independently. Conflicts at all levels of screening and abstraction will be resolved through discussion. Random-effects meta-analyses of primary observational studies will be conducted where appropriate. Parameters for exploring statistical heterogeneity are pre-specified. The World Cancer Research Fund (WCRF)/American Institute for Cancer Research (AICR) criteria and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach will be used for determining the quality of evidence for cancer outcomes.

DISCUSSION

Our study will establish the extent of the epidemiological evidence underlying the associations between central nervous system disorders and cancer and will provide a rigorous and updated synthesis of a range of important site-specific cancer outcomes.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016052762.

Poor replication validity of biomedical association studies reported by newspapers

Objective

To investigate the replication validity of biomedical association studies covered by newspapers.

Methods

We used a database of 4723 primary studies included in 306 meta-analysis articles. These studies associated a risk factor with a disease in three biomedical domains, psychiatry, neurology and four somatic diseases. They were classified into a lifestyle category (e.g. smoking) and a non-lifestyle category (e.g. genetic risk). Using the database Dow Jones Factiva, we investigated the newspaper coverage of each study. Their replication validity was assessed using a comparison with their corresponding meta-analyses.

Results

Among the 5029 articles of our database, 156 primary studies (of which 63 were lifestyle studies) and 5 meta-analysis articles were reported in 1561 newspaper articles. The percentage of covered studies and the number of newspaper articles per study strongly increased with the impact factor of the journal that published each scientific study. Newspapers almost equally covered initial (5/39 12.8%) and subsequent (58/600 9.7%) lifestyle studies. In contrast, initial non-lifestyle studies were covered more often (48/366 13.1%) than subsequent ones (45/3718 1.2%). Newspapers never covered initial studies reporting null findings and rarely reported subsequent null observations. Only 48.7% of the 156 studies reported by newspapers were confirmed by the corresponding meta-analyses. Initial non-lifestyle studies were less often confirmed (16/48) than subsequent ones (29/45) and than lifestyle studies (31/63). Psychiatric studies covered by newspapers were less often confirmed (10/38) than the neurological (26/41) or somatic (40/77) ones. This is correlated to an even larger coverage of initial studies in psychiatry. Whereas 234 newspaper articles covered the 35 initial studies that were later disconfirmed, only four press articles covered a subsequent null finding and mentioned the refutation of an initial claim.

Conclusion

Journalists preferentially cover initial findings although they are often contradicted by meta-analyses and rarely inform the public when they are disconfirmed.

Replication Validity of Initial Association Studies: A Comparison between Psychiatry, Neurology and Four Somatic Diseases

Context

There are growing concerns about effect size inflation and replication validity of association studies, but few observational investigations have explored the extent of these problems.

Objective

Using meta-analyses to measure the reliability of initial studies and explore whether this varies across biomedical domains and study types (cognitive/behavioral, brain imaging, genetic and “others”).

Methods

We analyzed 663 meta-analyses describing associations between markers or risk factors and 12 pathologies within three biomedical domains (psychiatry, neurology and four somatic diseases). We collected the effect size, sample size, publication year and Impact Factor of initial studies, largest studies (i.e., with the largest sample size) and the corresponding meta-analyses. Initial studies were considered as replicated if they were in nominal agreement with meta-analyses and if their effect size inflation was below 100%.

Results

Nominal agreement between initial studies and meta-analyses regarding the presence of a significant effect was not better than chance in psychiatry, whereas it was somewhat better in neurology and somatic diseases. Whereas effect sizes reported by largest studies and meta-analyses were similar, most of those reported by initial studies were inflated. Among the 256 initial studies reporting a significant effect (p<0.05) and paired with significant meta-analyses, 97 effect sizes were inflated by more than 100%. Nominal agreement and effect size inflation varied with the biomedical domain and study type. Indeed, the replication rate of initial studies reporting a significant effect ranged from 6.3% for genetic studies in psychiatry to 86.4% for cognitive/behavioral studies. Comparison between eight subgroups shows that replication rate decreases with sample size and “true” effect size. We observed no evidence of association between replication rate and publication year or Impact Factor.

Conclusion

The differences in reliability between biological psychiatry, neurology and somatic diseases suggest that there is room for improvement, at least in some subdomains.

Detecting neuroimaging biomarkers for schizophrenia: a meta-analysis of multivariate pattern recognition studies

Multivariate pattern recognition approaches have recently facilitated the search for reliable neuroimaging-based biomarkers in psychiatric disorders such as schizophrenia. By taking into account the multivariate nature of brain functional and structural changes as well as their distributed localization across the whole brain, they overcome drawbacks of traditional univariate approaches. To evaluate the overall reliability of neuroimaging-based biomarkers, we conducted a comprehensive literature search to identify all studies that used multivariate pattern recognition to identify patterns of brain alterations that differentiate patients with schizophrenia from healthy controls. A bivariate random-effects meta-analytic model was implemented to investigate the sensitivity and specificity across studies as well as to assess the robustness to potentially confounding variables. In the total sample of n=38 studies (1602 patients and 1637 healthy controls), patients were differentiated from controls with a sensitivity of 80.3% (95% CI: 76.7-83.5%) and a specificity of 80.3% (95% CI: 76.9-83.3%). Analysis of neuroimaging modality indicated higher sensitivity (84.46%, 95% CI: 79.9-88.2%) and similar specificity (76.9%, 95% CI: 71.3-81.6%) of rsfMRI studies as compared with structural MRI studies (sensitivity: 76.4%, 95% CI: 71.9-80.4%, specificity of 79.0%, 95% CI: 74.6-82.8%). Moderator analysis identified significant effects of age (p=0.029), imaging modality (p=0.019), and disease stage (p=0.025) on sensitivity as well as of positive-to-negative symptom ratio (p=0.022) and antipsychotic medication (p=0.016) on specificity. Our results underline the utility of multivariate pattern recognition approaches for the identification of reliable neuroimaging-based biomarkers. Despite the clinical heterogeneity of the schizophrenia phenotype, brain functional and structural alterations differentiate schizophrenic patients from healthy controls with 80% sensitivity and specificity.

Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap

The current difficulties in keeping systematic reviews up to date leads to considerable inaccuracy, hampering the translation of knowledge into action. Incremental advances in conventional review updating are unlikely to lead to substantial improvements in review currency. A new approach is needed. We propose living systematic review as a contribution to evidence synthesis that combines currency with rigour to enhance the accuracy and utility of health evidence. Living systematic reviews are high quality, up-to-date online summaries of health research, updated as new research becomes available, and enabled by improved production efficiency and adherence to the norms of scholarly communication. Together with innovations in primary research reporting and the creation and use of evidence in health systems, living systematic review contributes to an emerging evidence ecosystem.

Genetic polymorphism of β-fibrinogen gene-455G/A can contribute to the risk of ischemic stroke

Many studies have investigated the association between the β-fibrinogen gene-455G/A (FGβ-455G/A) polymorphism and the risk of ischemic stroke. However, these evidences were inadequate to provide stronger conclusions because most studies were generally small. To shed light on these inconclusive findings, we conducted a large sample size meta-analysis of studies relating to the FGβ-455G/A polymorphism and the risk of ischemic stroke. Odds ratios with a 95 % confidence interval were used to investigate the association between FGβ-455G/A polymorphism and ischemic stroke. Publication bias was tested by Egger's test and funnel plot. Inconsistency index and Cochran's Q statistic were used to check heterogeneity. Cumulative and recursive cumulative meta-analyses were performed to provide a framework for updating a genetic effect from all of the included studies. Twenty-six independent publications with 4,070 cases and 4,649 controls were included in this meta-analysis. Results showed that the β-fibrinogen-455G/A polymorphism was significantly associated with the risk of ischemic stroke. The FGβ-455G/A polymorphism was found to be a risk factor for ischemic stroke in Asians and adults, while association was not observed for Caucasians and juveniles based on the small size and it may be necessary to conduct larger studies on them to investigate the association in the future. The cumulative meta-analysis indicated a decline from 1998 to 2003, and the results remained stable during the period 2004-2012. The results indicate that FGβ-455G/A polymorphism may be a susceptible predictor of ischemic stroke. More studies are needed to elucidate the relationship further.

The decline effect in cardiovascular medicine: is the effect of cardiovascular medicine and stent on cardiovascular events decline over the years?

The term decline effect is referred to a diminution of scientifically discovered effects over time. Reasons for the decline effect are multifaceted and include publication bias, selective reporting, outcomes reporting bias, regression to the mean, scientific paradigm shift, overshadowing and habituation, among others. Such effects can be found in cardiovascular medicines through medications (e.g., aspirin, antithrombotics, proton pump inhibitor, beta-blockers, statins, estrogen/progestin, angiotensin converting enzyme inhibitor etc.), as well as with interventional devices (e.g., angioplasty, percutaneous coronary intervention, stents). The scientific community should understand the various dimensions of the decline effects, and effective steps should be undertaken to prevent or recognize such decline effects in cardiovascular medicines.

Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review

BACKGROUND

The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias and outcome reporting bias have been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making.

METHODOLOGY/PRINCIPAL FINDINGS

In this update, we review and summarise the evidence from cohort studies that have assessed study publication bias or outcome reporting bias in randomised controlled trials. Twenty studies were eligible of which four were newly identified in this update. Only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Fifteen of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.

CONCLUSIONS

This update does not change the conclusions of the review in which 16 studies were included. Direct empirical evidence for the existence of study publication bias and outcome reporting bias is shown. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Enteroaggregative Escherichia coli and acute diarrhea in children: a meta-analysis of South Asian populations

The purpose of this investigation was to evaluate the association of enteroaggregative Escherichia coli (EAEC) with acute diarrhea in children of South Asian populations. Our meta-analysis included 18 studies published between 1989 and 2011. The odds ratio (OR) was used to evaluate all available observational epidemiology studies. Modifying effects on the overall OR were approached with outlier, subgroup, cumulative, and cumulative recursive analyses. Synthesis of the 18 observational studies revealed an association between EAEC carriage and acute diarrhea, with an overall OR of 1.51, which was significant (p = 0.008), heterogeneous (Pheterogeneity < 0.0001), and unaffected by outlier analysis. This analysis, however, affected the subgroups by eliminating the following: (i) heterogeneity (from Pheterogeneity < 0.0001 to 0.30-0.72) of pooled ORs in the underpowered (OR 1.37, p = 0.15), Indian (OR 1.92, p = 0.09), and hospital-based (OR 1.66, p = 0.06) studies; (ii) non-significance of these three subgroups (OR 1.56-2.01, p < 0.0001-0.003); (iii) significance of the high-powered studies (from OR 1.70, p = 0.02 to OR 1.15, p = 0.28); (iv) heterogeneity (from Pheterogeneity < 0.0001-0.0002 to 0.11-0.15) of pooled ORs in period three (OR 1.85, p = 0.14), population-based (OR 1.36, p = 0.09), and pCVD432 (OR 1.53, p = 0.07) studies. In general, outlier treatment increased precision with the narrowing of confidence intervals, overall, and in the subgroups. Cumulative meta-analysis generally resulted in increases in the frequencies of significant effects and of heterogeneity. This meta-analysis on observational studies suggests that the association between EAEC and acute diarrhea in children is that of increased risk. This effect generally comes from heterogeneous studies of South Asian populations, but is modified with outlier and subgroup treatments.

Angiotensin-converting enzyme insertion/deletion polymorphism contributes to ischemic stroke risk: a meta-analysis of 50 case-control studies

Background

Many studies have investigated the association between the angiotensin-converting enzyme (ACE) gene insertion/deletion (I/D) polymorphism and risk of ischemic stroke. However, the evidence is inadequate to draw robust conclusions because most studies were generally small and conducted in heterogeneous populations. To shed light on these inconclusive findings, we conducted a large meta-analysis of studies relating the ACE I/D polymorphism to the risk of ischemic stroke.

Methods

Relevant studies were identified by searching PubMed and Embase through February 2012 and by reviewing the references of retrieved articles. We included studies that reported odds ratio (OR) with 95% confidence interval (CI) for the association between this polymorphism and ischemic stroke risk.

Results

Fifty independent publications, with 10 070 stroke cases and 22 103 controls, were included. The results indicated that the DD homozygote carriers had a 37% higher risk of ischemic stroke when compared with the homozygotes II and heterozygote ID [odds ratio (OR) = 1.37, 95% confidence interval (CI): 1.22–1.53]. Subgroup analyses indicated that this higher risk was more pronounced among Asians, hospital-based studies, and small vessel disease (SVD). Potential publication bias may exist, but correction for this bias using a formal statistical method did not materially alter the combined risk estimate.

Conclusion

The results of our meta-analysis indicate that the D allele of ACE I/D polymorphism is a low-penetrance susceptibility marker of ischemic stroke.

Sample size and power considerations in network meta-analysis

BACKGROUND

Network meta-analysis is becoming increasingly popular for establishing comparative effectiveness among multiple interventions for the same disease. Network meta-analysis inherits all methodological challenges of standard pairwise meta-analysis, but with increased complexity due to the multitude of intervention comparisons. One issue that is now widely recognized in pairwise meta-analysis is the issue of sample size and statistical power. This issue, however, has so far only received little attention in network meta-analysis. To date, no approaches have been proposed for evaluating the adequacy of the sample size, and thus power, in a treatment network.

FINDINGS

In this article, we develop easy-to-use flexible methods for estimating the 'effective sample size' in indirect comparison meta-analysis and network meta-analysis. The effective sample size for a particular treatment comparison can be interpreted as the number of patients in a pairwise meta-analysis that would provide the same degree and strength of evidence as that which is provided in the indirect comparison or network meta-analysis. We further develop methods for retrospectively estimating the statistical power for each comparison in a network meta-analysis. We illustrate the performance of the proposed methods for estimating effective sample size and statistical power using data from a network meta-analysis on interventions for smoking cessation including over 100 trials.

CONCLUSION

The proposed methods are easy to use and will be of high value to regulatory agencies and decision makers who must assess the strength of the evidence supporting comparative effectiveness estimates.

The number of patients and events required to limit the risk of overestimation of intervention effects in meta-analysis--a simulation study

BACKGROUND

Meta-analyses including a limited number of patients and events are prone to yield overestimated intervention effect estimates. While many assume bias is the cause of overestimation, theoretical considerations suggest that random error may be an equal or more frequent cause. The independent impact of random error on meta-analyzed intervention effects has not previously been explored. It has been suggested that surpassing the optimal information size (i.e., the required meta-analysis sample size) provides sufficient protection against overestimation due to random error, but this claim has not yet been validated.

METHODS

We simulated a comprehensive array of meta-analysis scenarios where no intervention effect existed (i.e., relative risk reduction (RRR) = 0%) or where a small but possibly unimportant effect existed (RRR = 10%). We constructed different scenarios by varying the control group risk, the degree of heterogeneity, and the distribution of trial sample sizes. For each scenario, we calculated the probability of observing overestimates of RRR>20% and RRR>30% for each cumulative 500 patients and 50 events. We calculated the cumulative number of patients and events required to reduce the probability of overestimation of intervention effect to 10%, 5%, and 1%. We calculated the optimal information size for each of the simulated scenarios and explored whether meta-analyses that surpassed their optimal information size had sufficient protection against overestimation of intervention effects due to random error.

RESULTS

The risk of overestimation of intervention effects was usually high when the number of patients and events was small and this risk decreased exponentially over time as the number of patients and events increased. The number of patients and events required to limit the risk of overestimation depended considerably on the underlying simulation settings. Surpassing the optimal information size generally provided sufficient protection against overestimation.

CONCLUSIONS

Random errors are a frequent cause of overestimation of intervention effects in meta-analyses. Surpassing the optimal information size will provide sufficient protection against overestimation.

Association between KCNJ11 gene polymorphisms and risk of type 2 diabetes mellitus in East Asian populations: a meta-analysis in 42,573 individuals

A number of studies have been performed to identify the association between potassium inwardly-rectifying channel, subfamily J, member 11 (KCNJ11) gene and type 2 diabetes mellitus (T2DM) in East Asian populations, with inconsistent results. The main aim of this work was to evaluate more precisely the genetic influence of KCNJ11 on T2DM in East Asian populations by means of a meta-analysis. We identified 20 articles for qualitative analysis and 16 were eligible for quantitative analysis (meta-analysis) by database searching up to May 2010. The association was assessed under different genetic models, and the pooled odds ratios (ORs) with 95% confidence intervals (95% CIs) were calculated. The allelic and genotypic contrast demonstrated that the association between KCNJ11 and T2DM was significant for rs5210. However, not all results for rs5215 and rs5218 showed significant associations. For rs5219, the combined ORs (95% CIs) for allelic contrast, dominant and recessive models contrast (with allelic frequency and genotypic distribution data) were 1.139 (1.093-1.188), 1.177 (1.099-1.259) and 1.207 (1.094-1.332), respectively (random effect model). The analysis on the most completely adjusted ORs (95% CIs) by the covariates of rs5219 all presented significant associations under different genetic models. Population-stratified analysis (Korean, Japanese and Chinese) and sensitivity analysis verified the significant results. Cumulative meta-analysis including publication time and sample size illustrated the exaggerated genetic effect in the earliest studies. Heterogeneity and publication bias were assessed. Our study verified that single nucleotide polymorphisms (SNPs) of KCNJ11 gene were significantly associated with the risk of T2DM in East Asian populations.

Absence of carious lesions at margins of glass-ionomer cement and amalgam restorations: An update of systematic review evidence

Background

This article aims to update the existing systematic review evidence elicited by Mickenautsch et al. up to 18 January 2008 (published in the European Journal of Paediatric Dentistry in 2009) and addressing the review question of whether, in the same dentition and same cavity class, glass-ionomer cement (GIC) restored cavities show less recurrent carious lesions on cavity margins than cavities restored with amalgam.

Methods

The systematic literature search was extended beyond the original search date and a further hand-search and reference check was done. The quality of accepted trials was assessed, using updated quality criteria, and the risk of bias was investigated in more depth than previously reported. In addition, the focus of quantitative synthesis was shifted to single datasets extracted from the accepted trials.

Results

The database search (up to 10 August 2010) identified 1 new trial, in addition to the 9 included in the original systematic review, and 11 further trials were included after a hand-search and reference check. Of these 21 trials, 11 were excluded and 10 were accepted for data extraction and quality assessment. Thirteen dichotomous datasets of primary outcomes and 4 datasets with secondary outcomes were extracted. Meta-analysis and cumulative meta-analysis were used in combining clinically homogenous datasets. The overall results of the computed datasets suggest that GIC has a higher caries-preventive effect than amalgam for restorations in permanent teeth. No difference was found for restorations in the primary dentition.

Conclusion

This outcome is in agreement with the conclusions of the original systematic review. Although the findings of the trials identified in this update may be considered to be less affected by attrition- and publication bias, their risk of selection- and detection/performance bias is high. Thus, verification of the currently available results requires further high-quality randomised control trials.

Caries-preventive effect of glass ionomer and resin-based fissure sealants on permanent teeth: An update of systematic review evidence

Background

This article constitutes a partial update of the original systematic review evidence by Yengopal et al. from 15 January 2008 (published in the Journal of Oral Science in 2009) with primary focus on research quality in regard to bias risk in trials. Its aim is to update the existing systematic review evidence from the English literature as to whether caries occurrence on pits and fissures of teeth sealed with either GIC or resin is the same.

Methods

In addition to the 12 trials included during the original systematic review, 5 new trials were identified during the database search (up to 26 August 2010) and 2 further trials were included from a hand search and reference check. Of these, 3 trials were excluded and 16 were accepted for data extraction and quality assessment. The quality of accepted trials was assessed, using updated quality criteria, and the risk of bias was investigated in more depth than previously reported. In addition, the focus of quantitative synthesis was shifted to single datasets that were extracted from the accepted trials.

Results

Twenty-six dichotomous and 4 continuous datasets were extracted. Meta-analysis and cumulative meta-analysis were used in combining clinically homogenous datasets. The overall outcome of the computed datasets suggest no difference between the caries-preventive effects of GIC- and resin-based fissure sealants.

Conclusions

This overall outcome is in agreement with the conclusions of the original systematic review. Although the findings of the trials identified in this update may be considered to be less affected by attrition- and publication bias, their risk of selection- and detection-/performance bias is high. Thus, verification of the currently available results requires further high quality randomised control trials.

Us versus Them in Context: Meta-Analysis as a Tool for Geotemporal Trends in Intergroup Relations

The increasing availability of studies from many nations offers important potential insights into group-based psychology and behavior, conflict, and violence. Nonetheless, to date, few cross-national or cultural comparisons of study findings have been made, representing a gap in our understanding of the historical causes and courses of intergroup conflict in current comparative approaches. Meta-analytic methods offer researchers the ability to combine data from studies with groups as well as across time. Our review of statistical methods available for comparative analyses in intergroup research found strengths and limitations for understanding group differences, conflict, and violence, and meta-analytic methods address these limitations by exploring potential structural-level moderators and by identifying how temporal and geographical variations may relate directly to group-based variables. Such methods can contribute to our understanding of broad structural effects on group-based variables by elucidating the mechanisms underlying them.

OM-85 BV, an immunostimulant in pediatric recurrent respiratory tract infections: a systematic review

BACKGROUND

This study was conducted to assess the efficacy of OM-85 BV (Broncho-Vaxom) in the prevention of pediatric recurrent respiratory tract infections (RTIs). Available evidence suggests that defining recurrent RTIs as >or=3 infections per fall-winter semester is both medically and epidemiologically justified. Therefore, this criterion was chosen as a primary endpoint.

METHODS

Trials were identified through consultation of bibliographic databases and other channels. Eleven non-blinded studies plus one dealing with primary prevention were excluded and eight randomized controlled trials were included in the meta-analysis. The data were compared at 6 months, which represented the end of most studies. The complete database was examined according to the guidelines of the Cochrane collaboration.

RESULTS

The mean age of children and the number of RTIs in the preceding year were comparable at admission. Of the patients in the OM-85 BV treated population (n=435), 32% had recurrent RTIs (that is, >or=3 RTIs/6 months) vs. 58.2% in the placebo treated population (n=416; P<0.001). Sensitivity analysis showed that this was not driven by any particular trial. The results of this review were also positive for the active treatment regarding the secondary variables, which were represented by the number of patients with at least one RTI and the mean number of RTIs.

CONCLUSIONS

This meta-analysis shows, as observed in several individual trials, that the population treated with OM-85 BV had significantly and consistently fewer cases of recurrent RTIs. The data suggest that the effect is greater in patients at increased risk of recurrent RTIs.

Angiotensin-converting enzyme gene polymorphism is associated with type 2 diabetes: a meta-analysis

The association of angiotensin-converting enzyme gene polymorphism with type 2 diabetes was investigated in many studies with conflicting results. To clarify this conflict, we performed a meta-analysis on recent previous reports on ACE gene polymorphism and its correlation to type 2 diabetes. A total of 15,166 subjects from 24 studies were included in this meta-analysis. Summary odds ratios (ORs) were estimated. Potential sources of heterogeneity and bias were explored. The D variant was associated with a 14% increased risk of T2D relative to the I variant (OR 1.14; 95% CI: 1.04-1.24). In subgroup analysis, Caucasian and East Asians showed significant association. No association was found in the Turkish groups. No publication bias was observed in this meta-analysis by using the Egger method (tau = 1.63, P = 0.12), as well as the Begg's test (z = 1.66, P = 0.10). Cumulative meta-analysis for the allelic contrast showed a trend of association as information accumulated. These data suggested that the variant of ACE I/D had a moderate positive association with type 2 diabetes.

The use of older studies in meta-analyses of medical interventions: a survey

BACKGROUND

Evidence for medical interventions sometimes derives from data that are no longer up to date. These data can influence the outcomes of meta-analyses, yet do not always reflect current clinical practice. We examined the age of the data used in meta-analyses contained within systematic reviews of medical interventions, and investigated whether authors consider the age of these data in their interpretations.

METHODS

From Issue 4, 2005, of the Cochrane Database of Systematic Reviews we randomly selected 10% of systematic reviews containing at least 1 meta-analysis. From this sample we extracted 1 meta-analysis per primary outcome. We calculated the number of years between the study's publication and 2005 (the year that the systematic review was published), as well as the number of years between the study's publication and the year of the literature search conducted in the study. We assessed whether authors discussed the implications of including less recent data, and, for systematic reviews containing meta-analyses of studies published before 1996, we calculated whether excluding the findings of those studies changed the significance of the outcomes. We repeated these calculations and assessments for 22 systematic reviews containing meta-analyses published in 6 high-impact general medical journals in 2005.

RESULTS

For 157 meta-analyses (n = 1149 trials) published in 2005, the median year of the most recent literature search was 2003 (interquartile range [IQR] 2002-04). Two-thirds of these meta-analyses (103/157, 66%) involved no trials published in the preceding 5 years (2001-05). Forty-seven meta-analyses (30%) included no trials published in the preceding 10 years (1996-2005). In another 16 (10%), the statistical significance of the outcomes would have been different had the studies been limited to those published between 1996 and 2005, although in some cases this change in significance would have been due to loss of power. Only 12 (8%) of the meta-analyses discussed the potential implications of including older studies. Among the 22 meta-analyses considered in high-impact general medical journals, 2 included no studies published in the 5 years prior to the reference year (2005), and 18 included at least 1 study published before 1996. Only 4 meta-analyses discussed the implications of including older studies.

INTERPRETATION

In most systematic reviews containing meta-analyses of evidence for health care interventions, very recent studies are rare. Researchers who conduct systematic reviews with meta-analyses, and clinicians who read the outcomes of these studies, should be made aware of the potential implications of including less recent data.

Systematic review of the empirical evidence of study publication bias and outcome reporting bias

BACKGROUND

The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention.

METHODOLOGY/PRINCIPAL FINDINGS

We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.

CONCLUSIONS

Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Effect of formal statistical significance on the credibility of observational associations

The author evaluated the implications of nominal statistical significance for changing the credibility of null versus alternative hypotheses across a large number of observational associations for which formal statistical significance (p < 0.05) was claimed. Calculation of the Bayes factor (B) under different assumptions was performed on 272 observational associations published in 2004-2005 and a data set of 50 meta-analyses on gene-disease associations (752 studies) for which statistically significant associations had been claimed (p < 0.05). Depending on the formulation of the prior, statistically significant results offered less than strong support to the credibility (B > 0.10) for 54-77% of the 272 epidemiologic associations for diverse risk factors and 44-70% of the 50 associations from genetic meta-analyses. Sometimes nominally statistically significant results even decreased the credibility of the probed association in comparison with what was thought before the study was conducted. Five of six meta-analyses with less than substantial support (B > 0.032) lost their nominal statistical significance in a subsequent (more recent) meta-analysis, while this did not occur in any of seven meta-analyses with decisive support (B < 0.01). In these large data sets of observational associations, formal statistical significance alone failed to increase much the credibility of many postulated associations. Bayes factors may be used routinely to interpret "significant" associations.

Cognitive-behavioral therapy for adolescent depression: a meta-analytic investigation of changes in effect-size estimates

OBJECTIVE

To explicate differences between early and recent meta-analytic estimates of the effects of cognitive-behavioral therapy (CBT) for adolescent depression.

METHOD

Meta-analytic procedures were used to investigate whether methodological characteristics moderated mean effect sizes among 11 randomized, controlled trials of CBT focusing on adolescents meeting diagnostic criteria for unipolar depression.

RESULTS

Cumulative meta-analyses indicated that effects of CBT have decreased from large effects in early trials, and confidence intervals have become narrower. Effect sizes were significantly smaller among studies that used intent-to-treat analytic strategies, compared CBT to active treatments, were conducted in clinical settings, and featured greater methodological rigor based on CONSORT (Consolidated Standards of Reporting Trials) criteria. The mean posttreatment effect size of 0.53 was statistically significant.

CONCLUSIONS

Differences in estimates of the efficacy of CBT for depressed adolescents may stem from methodological differences between early and more recent investigations. Overall, results support the effectiveness of CBT for the treatment of adolescent depression.

A systematic review identified few methods and strategies describing when and how to update systematic reviews

OBJECTIVE

Systematic reviews (SRs) are convenient summaries of evidence for health care practitioners. They form a basis for clinical practice guidelines and suggest directions for new research. SRs are most helpful if they are current; however, most of them are not being updated. This SR summarizes strategies and methods describing when and how to update SRs.

STUDY DESIGN AND SETTING

We searched MEDLINE (1966 to December 2005), PsycINFO, the Cochrane Methodology Register, and the 2005 Cochrane Colloquium proceedings to identify records describing when and how to update SRs in health care.

RESULTS

Four updating strategies, one technique, and two statistical methods were identified. Three strategies addressed steps for updating, and one strategy presented a model for assessing the need to update. One technique discussed the use of the "entry date" field in bibliographic searching. The statistical methods were cumulative meta-analysis and a test for detecting outdated meta-analyses with statistically nonsignificant results.

CONCLUSION

Little research has been conducted on when and how to update SRs in contrast to other methodological areas of conducting SRs (e.g., publication bias, variance imputation). The feasibility and efficiency of the identified approaches is uncertain. More research is needed to develop pragmatic and efficient methodologies for updating SRs.

Ursodeoxycholic acid for patients with primary biliary cirrhosis: an updated systematic review and meta-analysis of randomized clinical trials using Bayesian approach as sensitivity analyses

OBJECTIVES

Ursodeoxycholic acid (UDCA) is used for primary biliary cirrhosis (PBC), but the beneficial effects remain controversial.

METHODS

We performed an updated systematic review to evaluate the benefits and harms of UDCA in patients with PBC. We included randomized clinical trials evaluating UDCA versus placebo or no intervention in patients with PBC. The primary outcomes, mortality and mortality or liver transplantation, were reported as relative risk (RR) with 95% confidence interval (CI). Meta-regression was used to investigate the associations between UDCA effects and the trial's risk of bias, UDCA dose, duration, and PBC severity at trial entry. We used Bayesian meta-analytic approaches as sensitivity analyses.

RESULTS

Sixteen randomized clinical trials (1,447 patients) evaluating UDCA versus placebo or no intervention were identified. Over half of the trials had high risk of bias. Comparing with placebo or no intervention, UDCA did not significantly affect mortality (RR 0.97, 95% CI 0.67-1.42) and mortality or liver transplantation (RR 0.92, 95% CI 0.71-1.21). The findings were supported by the Bayesian meta-analyses. Meta-regression analyses suggested that UDCA effects seem to be associated with patient's disease severity and trial duration. UDCA did not improve pruritus, fatigue, autoimmune conditions, liver histology, or portal pressure. UDCA seemed to improve biochemical variables, such as serum bilirubin, and ascites and jaundice, but the findings were based on few trials with sparse data. The use of UDCA was significantly associated with adverse events, mainly weight gain.

CONCLUSIONS

This updated systematic review did not demonstrate any benefit of UDCA on mortality and mortality or liver transplantation in patients with PBC.

Systematic review and meta-analysis: D-Penicillamine vs. placebo/no intervention in patients with primary biliary cirrhosis--Cochrane Hepato-Biliary Group

BACKGROUND

D-Penicillamine is used for patients with primary biliary cirrhosis due to its ability to decrease hepatic copper and modulate the immune response. The results on effects of D--penicillamine in randomized-clinical trials of primary biliary cirrhosis patients are inconsistent.

AIM

To systematically evaluate the benefits and harms of D-penicillamine for patients with primary biliary cirrhosis.

METHODS

We have performed a systematic review with meta-analyses of randomized-clinical trials to evaluate the effects of D-penicillamine for primary biliary cirrhosis. The primary outcomes are mortality and mortality or liver transplantation. We analysed the data by fixed-effect and random-effect models.

RESULTS

Seven randomized trials including 706 patients were analysed. d-Penicillamine was without significant effects on mortality (RR 1.08, 95% CI: 0.82-1.43, P = 0.56), mortality or liver transplantation (RR 1.11, 95% CI: 0.74-1.68, P = 0.62), pruritus, liver complications, progression of liver histological stage and liver biochemical variables. D--Penicillamine significantly decreased serum alanine aminotransferase activity (weighted mean difference -45 IU/L, 95% CI: -75 to -15, P < 0.05) and led to significantly more adverse events (RR 4.18, 95% CI: 1.38-12.69, P = 0.01).

CONCLUSION

D-Penicillamine did not appear to reduce the risk of mortality or morbidity, and led to more adverse events in patients with primary biliary cirrhosis.

Evolution and translation of research findings: from bench to where?

The credibility and replication of research findings evolve over time, as data accumulate. However, translation of postulated research promises to real-life biomedical applications is uncommon. In some fields of research, we may observe diminishing effects for the strength of research findings and rapid alternations of exaggerated claims and extreme contradictions--the "Proteus Phenomenon." While these phenomena are probably more prominent in the basic sciences, similar manifestations have been documented even in clinical trials and they may undermine the credibility of clinical research. Significance-chasing bias may be in part responsible, but the greatest threat may come from the poor relevance and scientific rationale and thus low pre-study odds of success of research efforts. Given that we currently have too many research findings, often with low credibility, replication and rigorous evaluation become as important as or even more important than discovery. Credibility, replication, and translation are all desirable properties of research findings, but are only modestly correlated. In this essay, I discuss some of the evidence (or lack thereof) for the process of evolution and translation of research findings, with emphasis on the biomedical sciences.

Vitamin D receptor gene BsmI and TaqI polymorphisms and fracture risk: a meta-analysis

INTRODUCTION

Fracture is the major clinical outcome of osteoporosis. The vitamin D receptor (VDR) gene is thought to be a candidate gene for osteoporosis. Many genetic studies have suggested an association of VDR polymorphisms and osteoporosis, but evidence remains conflicting.

MATERIALS AND METHODS

We searched published studies from 1996 to September 2005 through PubMed and evaluated the genetic effect of the BsmI and TaqI polymorphism of VDR on fracture risk in a meta-analysis. Thirteen studies with a total of 20 eligible comparisons (1632 fracture cases and 5203 controls) were analyzed with fixed and random effects models.

RESULT

No evidence of relationship between the VDR BsmI or TaqI polymorphism and fracture risk was observed with any genetic model. The odds ratio (95% confidence interval) of b-allele versus B-allele was 0.98 (0.86-1.12) with random effects calculations. There was significant between-study heterogeneity. Small studies did not differ significantly from larger ones.

CONCLUSION

No relationship of the VDR BsmI or TaqI polymorphism and fracture risk was found in the meta-analysis of published data.

The case of the misleading funnel plot

Evidence based medicine insists on rigorous standards to appraise clinical interventions. Failure to apply the same rules to its own tools could be equally damaging

Conceptualizing changes in behavior in intervention research: the range of possible changes model

An international movement has focused on identifying evidence-based interventions that were developed to change psychological constructs and that are supported by controlled studies. However, inconsistent findings within individual intervention studies and among multiple studies raise critical problems in interpreting the evidence, and deciding when and whether an intervention is evidence-based. A theoretical and methodological framework (Range of Possible Changes [RPC] Model) is proposed to guide the study of change in intervention research. The authors recommend that future quantitative reviews of the research literature use the RPC Model to conceptualize, examine, and classify the available evidence for interventions. Future research should adopt the RPC Model to both develop theory-driven hypotheses and conduct examinations of the instances in which interventions may or may not change psychological constructs.

Effect of perioperative beta blockade in patients with diabetes undergoing major non-cardiac surgery: randomised placebo controlled, blinded multicentre trial

OBJECTIVES

To evaluate the long term effects of perioperative beta blockade on mortality and cardiac morbidity in patients with diabetes undergoing major non-cardiac surgery.

DESIGN

Randomised placebo controlled and blinded multicentre trial. Analyses were by intention to treat.

SETTING

University anaesthesia and surgical centres and one coordinating centre.

PARTICIPANTS

921 patients aged > 39 scheduled for major non-cardiac surgery.

INTERVENTIONS

100 mg metoprolol controlled and extended release or placebo administered from the day before surgery to a maximum of eight perioperative days.

MAIN OUTCOME MEASURES

The composite primary outcome measure was time to all cause mortality, acute myocardial infarction, unstable angina, or congestive heart failure. Secondary outcome measures were time to all cause mortality, cardiac mortality, and non-fatal cardiac morbidity.

RESULTS

Mean duration of intervention was 4.6 days in the metoprolol group and 4.9 days in the placebo group. Metoprolol significantly reduced the mean heart rate by 11% (95% confidence interval 9% to 13%) and mean blood pressure by 3% (1% to 5%). The primary outcome occurred in 99 of 462 patients in the metoprolol group (21%) and 93 of 459 patients in the placebo group (20%) (hazard ratio 1.06, 0.80 to 1.41) during a median follow-up of 18 months (range 6-30). All cause mortality was 16% (74/462) in the metoprolol group and 16% (72/459) in the placebo group (1.03, 0.74 to 1.42). The difference in risk for the proportion of patients with serious adverse events was 2.4% (- 0.8% to 5.6%).

CONCLUSIONS

Perioperative metoprolol did not significantly affect mortality and cardiac morbidity in these patients with diabetes. Confidence intervals, however, were wide, and the issue needs reassessment.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN58485613.

Ventilation strategies and outcome in randomised trials of high frequency ventilation

OBJECTIVE

Randomised controlled trials comparing elective use of high frequency ventilation (HFV) with conventional mechanical ventilation (CMV) in preterm infants have yielded conflicting results. We hypothesised that the variability of results may be explained by differences in study design, ventilation strategies, delay in initiation of HFV, and use of permissive hypercapnia.

METHODS

Randomised controlled trials comparing the elective use of HFV with any form of CMV were identified. Trials were classified according to the ventilation strategies used for HFV and CMV and oscillator device employed. For cumulative meta-analyses, trials were arranged by the following covariables: mean duration until randomisation, Paco(2) limits, publication date, and sample size. Odds ratios (OR) and 95% confidence intervals were calculated using fixed and random effects models.

RESULTS

Seventeen randomised trials enrolling 3776 patients were included. Unlike previous meta-analyses, there was no significant difference in the incidence of bronchopulmonary dysplasia or death (OR 0.87, 0.75-1.00) and severe intraventricular haemorrhage grade 3-4 (1.14, 0.96-1.37). The incidence of air leaks (OR 1.23, 1.06-1.44) was significantly increased with HFV. Subgroup analyses and cumulative meta-analyses demonstrated that trial results were related to the ventilation strategies used for HFV and CMV. No influence was found for mean time to randomisation, degree of permissive hypercapnia, or sample size.

CONCLUSIONS

Heterogeneity among trials of elective HFV compared to CMV in preterm infants is mainly due to differences in ventilatory strategies. Optimising CMV strategy appeared to be as effective as using HFV in improving pulmonary outcome in preterm infants.

Colchicine for primary biliary cirrhosis: a Cochrane Hepato-Biliary Group systematic review of randomized clinical trials

OBJECTIVES

Colchicine is used for patients with primary biliary cirrhosis due to its immunomodulatory and antifibrotic potential. The results from randomized clinical trials have, however, been inconsistent. We conducted a systematical review to evaluate the effect of colchicine for primary biliary cirrhosis.

METHODS

We identified randomized clinical trials comparing colchicine with placebo/no intervention. We analyzed effects by fixed and random effects model. We investigated heterogeneity by subgroup and sensitivity analyses.

RESULTS

We included 10 trials involving 631 patients, four of which were high-quality trials. No significant differences were detected between colchicine and placebo/no intervention regarding mortality (relative risk (RR), 1.21; 95% confidence interval (CI), 0.71-2.06), mortality or liver transplantation (RR = 1.00; 95% CI, 0.67-1.49), liver complications, liver biochemical variables, liver histology, or adverse events. Regarding mortality, an extreme case analysis favoring colchicine did not demonstrate beneficial effects of colchicine, whereas an extreme case analysis favoring placebo/no intervention demonstrated a detrimental effect of colchicine (RR = 2.28; 95% CI, 1.17-4.44). The number of patients without improvement of pruritus significantly decreased in the colchicine group (RR = 0.75; 95% CI, 0.65-0.87). However, this estimate was based on only 156 patients from three trials.

CONCLUSIONS

There is insufficient evidence to support the use of colchicine for patients with primary biliary cirrhosis. As we are unable to exclude a risk of increased mortality, we recommend to use colchicine only in randomized clinical trials.

Early extreme contradictory estimates may appear in published research: The Proteus phenomenon in molecular genetics research and randomized trials

BACKGROUND AND OBJECTIVE

Divergent results on the same scientific question generate controversy. We hypothesized that controversial data are attractive to investigators and editors, and thus the most extreme, opposite results would appear very early rather than late, as data accumulate, provided data can be generated rapidly.

METHODS

We used data from MEDLINE-indexed meta-analyses of case-control studies on genetic associations (retrospective, hypothesis-generating research with usually rapid turnaround) and meta-analyses of randomized trials of health care interventions (prospective, targeted research that usually takes longer) sampled from the Cochrane Library. Using cumulative meta-analysis, we evaluated how the between-study variance for studies on the same question changed over time and at what point the studies with the most extreme results ever observed had been published.

RESULTS

The maximal between-study variance was more likely to be recorded early in the 44 eligible meta-analyses of genetic associations than in the 37 meta-analyses of health care interventions (P = .013). At the time of the first heterogeneity assessment, the most favorable-ever result in support of a specific association was more likely to appear than the least favorable-ever result (22 vs. 10, P = .017); the opposite was seen at the second heterogeneity assessment (15 vs. 5, P = .031). Such a sequence of extreme opposite results was not seen in the clinical trials meta-analyses. The estimated between-study variance decreased over time in genetic association studies (P = .010), but not in clinical trials (P = .30).

CONCLUSION

In contrast to prospective trials, a rapid early sequence of extreme, opposite results is frequent in retrospective hypothesis-generating molecular research.

Neoadjuvant versus adjuvant systemic treatment in breast cancer: a meta-analysis

BACKGROUND

Interest in the use of preoperative systemic treatment in the management of breast cancer has increased because such neoadjuvant therapy appears to reduce the extent of local surgery required. We compared the clinical end points of patients with breast cancer treated preoperatively with systemic therapy (neoadjuvant therapy) and of those treated postoperatively with the same regimen (adjuvant therapy) in a meta-analysis of randomized trials.

METHODS

We evaluated nine randomized studies, including a total of 3946 patients with breast cancer, that compared neoadjuvant therapy with adjuvant therapy regardless of what additional surgery and/or radiation treatment was used. Fixed and random effects methods were used to combine data. Primary outcomes were death, disease progression, distant disease recurrence, and loco-regional disease recurrence. Secondary outcomes were local response and conservative local treatment. All statistical tests were two-sided.

RESULTS

We found no statistically or clinically significant difference between neoadjuvant therapy and adjuvant therapy arms associated with death (summary risk ratio [RR] = 1.00, 95% confidence interval [CI] = 0.90 to 1.12), disease progression (summary RR = 0.99, 95% CI = 0.91 to 1.07), or distant disease recurrence (summary RR = 0.94, 95% CI = 0.83 to 1.06). However, neoadjuvant therapy was statistically significantly associated with an increased risk of loco-regional disease recurrences (RR = 1.22, 95% CI = 1.04 to 1.43), compared with adjuvant therapy, especially in trials where more patients in the neoadjuvant, than the adjuvant, arm received radiation therapy without surgery (RR = 1.53, 95% CI = 1.11 to 2.10). Across trials, we observed heterogeneity in the rates of complete clinical response (range = 7%-65%; P for heterogeneity of <.001), pathologic response (range = 4%-29%; P for heterogeneity of <.001), and adoption of conservative local treatment (range = 28%-89% in neoadjuvant arms, P for heterogeneity of <.001).

CONCLUSIONS

Neoadjuvant therapy was apparently equivalent to adjuvant therapy in terms of survival and overall disease progression. Neoadjuvant therapy, compared with adjuvant therapy, was associated with a statistically significant increased risk of loco-regional recurrence when radiotherapy without surgery was adopted.

Ala45Thr polymorphism of the NEUROD1 gene and diabetes susceptibility: a meta-analysis

A meta-analysis assessed whether the Ala45Thr polymorphism of the neurogenic differentiation 1 (NEUROD1) gene is associated with increased risk of diabetes mellitus type 1 (T1D) or type 2 (T2D). Fourteen case-control studies were analyzed, including genotype data on 3,057 patients with diabetes (T1D n=1,213, T2D n=1,844) and 2,446 controls. Overall and race-specific summary odds ratios (ORs) were obtained with fixed and random effects models. The Thr allele did not significantly increase the overall risk for T1D (OR 1.27 [0.94-1.71], P=0.12) or T2D (OR 1.07 [0.90-1.28], P=0.46). The Thr allele conferred increased susceptibility in subjects of Asian racial descent to T1D (OR 1.88 [1.10-3.21], P=0.020), but not to T2D (OR 1.08 [0.74-1.56], P=0.70). There was no association in subjects of European descent (OR 0.97 [0.76-1.23], P=0.80 for T1D; OR 1.03 [0.88-1.21], P=0.68 for T2D). Larger studies seemed to show more conservative estimates for the association with T1D (P=0.083). The Ala45Thr polymorphism of the NEUROD1 gene has no effect on susceptibility to T2D. It may however be a risk factor for susceptibility to T1D, in particular for subjects of Asian descent, although bias cannot be totally excluded.

Radiotherapy vs. nonsteroidal anti-inflammatory drugs for the prevention of heterotopic ossification after major hip procedures: a meta-analysis of randomized trials

PURPOSE

To evaluate the efficacy of radiotherapy (RT) vs. nonsteroidal anti-inflammatory drugs (NSAIDs) in the prevention of heterotopic ossification (HO) after major hip procedures.

METHODS AND MATERIALS

We conducted a meta-analysis of 7 randomized studies (n = 1143) comparing RT with NSAIDs. Data were combined across studies using fixed and random effects models. We conducted separate analyses for clinically significant HO (Brooker Grade 3 and 4) and for any HO (any Brooker grade).

RESULTS

Overall RT tended to be more effective than NSAIDs in preventing Brooker 3 or 4 HO (risk ratio, 0.42; 95% confidence interval [CI], 0.18-0.97) or any HO (risk ratio, 0.75; 95% CI, 0.37-1.71), but with significant between-study heterogeneity for the second analysis. The overall absolute risk difference for Brooker 3 or 4 HO was small (-1.18%; 95% CI, -2.45% to 0.09%). Subgroup analyses showed that early preoperative RT (16-20 hours before surgery) and acetylsalicylic acid were less effective. For postoperative RT, there was a significant dose-response relationship (p = 0.008): 6 Gy of RT was equally effective as NSAIDs, whereas increasing radiation doses were more effective.

CONCLUSIONS

Although absolute differences may be small, postoperative RT is on average more effective than NSAIDs in preventing HO after major hip procedures, and its efficacy is dose dependent.

Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time

BACKGROUND AND OBJECTIVE

Meta-analyses of randomized trials may incorporate new evidence, and estimated treatment effects changeover time. We evaluated whether the certainty and estimates of efficacy and tolerability of mental health interventions change over time, as more trials appear on the same topics.

METHODS

One hundred meta-analyses (1,024 trial entries; 99,303 participants) with an outcome of death, relapse, failure or dropout and with five or more trials published in three or more different years were examined with cumulative meta-analysis and recursive cumulative meta-analysis.

RESULTS

Eight meta-analyses reached formal statistical significance (P < .05) at some point, but lost this significance eventually when more trials were published; typically large effect sizes in early trials were dissipated with further evidence. With 500 randomized subjects,95% of the time, subsequent changes in odds ratio might be up to 1.5-fold. For death, relapse, and failure outcomes, a decrease in effect size was somewhat more common than an increase, when more data became available (157 vs. 125, P = .06). This was most clear for comparisons of pharmacotherapies versus placebo (79 vs. 51, P = .009).

CONCLUSIONS

Evidence based on a small number of randomized subjects should be interpreted cautiously. Early treatment efficacy of pharmacotherapies is occasionally overestimated.