Meta-research: The art of getting it wrong

Seropositivity of toxoplasmosis in pregnant women living with HIV/AIDS worldwide: A systematic review and meta-analysis

BACKGROUND

Toxoplasmosis is caused by the protozoan parasite Toxoplasma gondii, a food- and water-borne zoonotic protozoan parasite that is able to infect almost all warm-blooded vertebrates. It has a major effect on public health, particularly in underdeveloped nations. Immune-competent individuals typically exhibit no symptoms or experience a mild influenza-like sickness, while there is a possibility of severe manifestation and fatal or high-risk for life-threatening diseases in immunocompromised people like pregnant women and HIV/AIDS patients and lead to severe pathological effects on the fetus.

METHOD

We conducted a systematic search of databases (PubMed, Google Scholar, Science Direct, EMBASE, and Scopus) using the PRISMA criteria. We used specific keywords such as Toxoplasma gondii, Toxoplasmosis, pregnant women, prevalence, HIV/AIDS, and worldwide studies published from 2018 to 2022. We use Stata (version 14) software to estimate the pooled prevalence and heterogeneity of toxoplasmosis in pregnant women and HIV-infected people using a random-effects model and the Cochran's Q-test, respectively. The Joanna Briggs Institute Critical Appraisal Instrument and Egger's regression asymmetry test were used to assess study quality and publication bias, respectively, while the single study omission analysis was used to test the robustness of a pooled estimate.

RESULTS

We included and analyzed a total of 12,887 individuals in this review. The pooled prevalence of T. gondii in this review was 40% (95% CI = 0.31-0.50). The sub-group analysis revealed that the evaluation included 11,967 pregnant women. In pregnant women, the pooled sero-prevalence was 40% (95% CI = 0.31-0.50). In pregnant women and HIV/AIDS patients, 920 individuals were evaluated, and the pooled sero-prevalence was 41% (95% CI = 0.20-0.61).

CONCLUSION

This review identified an overall sero-prevalence of Toxoplasma infection of 40% among pregnant women and HIV/AIDS. The expansion of prevention and control strategies, with a primary focus on enhancing educational initiatives, is necessary to avoid reactivation and stop the spread of infection, so investigative sero-prevalence is important work among pregnant women and HIV patients. In order to achieve a comprehensive explanation of the disease condition and reach this goal, we conducted a systematic review and meta-analysis in Worldwide for future use.

Finding the right power balance: Better study design and collaboration can reduce dependence on statistical power

Power analysis currently dominates sample size determination for experiments, particularly in grant and ethics applications. Yet, this focus could paradoxically result in suboptimal study design because publication biases towards studies with the largest effects can lead to the overestimation of effect sizes. In this Essay, we propose a paradigm shift towards better study designs that focus less on statistical power. We also advocate for (pre)registration and obligatory reporting of all results (regardless of statistical significance), better facilitation of team science and multi-institutional collaboration that incorporates heterogenization, and the use of prospective and living meta-analyses to generate generalizable results. Such changes could make science more effective and, potentially, more equitable, helping to cultivate better collaborations.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Metadata integrity in bioinformatics: Bridging the gap between data and knowledge

In the fast-evolving landscape of biomedical research, the emergence of big data has presented researchers with extraordinary opportunities to explore biological complexities. In biomedical research, big data imply also a big responsibility. This is not only due to genomics data being sensitive information but also due to genomics data being shared and re-analysed among the scientific community. This saves valuable resources and can even help to find new insights in silico. To fully use these opportunities, detailed and correct metadata are imperative. This includes not only the availability of metadata but also their correctness. Metadata integrity serves as a fundamental determinant of research credibility, supporting the reliability and reproducibility of data-driven findings. Ensuring metadata availability, curation, and accuracy are therefore essential for bioinformatic research. Not only must metadata be readily available, but they must also be meticulously curated and ideally error-free. Motivated by an accidental discovery of a critical metadata error in patient data published in two high-impact journals, we aim to raise awareness for the need of correct, complete, and curated metadata. We describe how the metadata error was found, addressed, and present examples for metadata-related challenges in omics research, along with supporting measures, including tools for checking metadata and software to facilitate various steps from data analysis to published research.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy. A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work. Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to Best Tools and Practices for Systematic Reviews

» Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.» A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.» Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Guidance to best tools and practices for systematic reviews1

Data continue to accumulate indicating that many systematic reviews are methodologically flawed, biased, redundant, or uninformative. Some improvements have occurred in recent years based on empirical methods research and standardization of appraisal tools; however, many authors do not routinely or consistently apply these updated methods. In addition, guideline developers, peer reviewers, and journal editors often disregard current methodological standards. Although extensively acknowledged and explored in the methodological literature, most clinicians seem unaware of these issues and may automatically accept evidence syntheses (and clinical practice guidelines based on their conclusions) as trustworthy.A plethora of methods and tools are recommended for the development and evaluation of evidence syntheses. It is important to understand what these are intended to do (and cannot do) and how they can be utilized. Our objective is to distill this sprawling information into a format that is understandable and readily accessible to authors, peer reviewers, and editors. In doing so, we aim to promote appreciation and understanding of the demanding science of evidence synthesis among stakeholders. We focus on well-documented deficiencies in key components of evidence syntheses to elucidate the rationale for current standards. The constructs underlying the tools developed to assess reporting, risk of bias, and methodological quality of evidence syntheses are distinguished from those involved in determining overall certainty of a body of evidence. Another important distinction is made between those tools used by authors to develop their syntheses as opposed to those used to ultimately judge their work.Exemplar methods and research practices are described, complemented by novel pragmatic strategies to improve evidence syntheses. The latter include preferred terminology and a scheme to characterize types of research evidence. We organize best practice resources in a Concise Guide that can be widely adopted and adapted for routine implementation by authors and journals. Appropriate, informed use of these is encouraged, but we caution against their superficial application and emphasize their endorsement does not substitute for in-depth methodological training. By highlighting best practices with their rationale, we hope this guidance will inspire further evolution of methods and tools that can advance the field.

Vaccines for the common cold

BACKGROUND

The common cold is a spontaneously remitting infection of the upper respiratory tract, characterised by a runny nose, nasal congestion, sneezing, cough, malaise, sore throat, and fever (usually < 37.8 ºC). Whilst the common cold is generally not harmful, it is a cause of economic burden due to school and work absenteeism. In the United States, economic loss due to the common cold is estimated at more than USD 40 billion per year, including an estimate of 70 million workdays missed by employees, 189 million school days missed by children, and 126 million workdays missed by parents caring for children with a cold. Additionally, data from Europe show that the total cost per episode may be up to EUR 1102. There is also a large expenditure due to inappropriate antimicrobial prescription. Vaccine development for the common cold has been difficult due to antigenic variability of the common cold viruses; even bacteria can act as infective agents. Uncertainty remains regarding the efficacy and safety of interventions for preventing the common cold in healthy people, thus we performed an update of this Cochrane Review, which was first published in 2011 and updated in 2013 and 2017.

OBJECTIVES

To assess the clinical effectiveness and safety of vaccines for preventing the common cold in healthy people.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) (April 2022), MEDLINE (1948 to April 2022), Embase (1974 to April 2022), CINAHL (1981 to April 2022), and LILACS (1982 to April 2022). We also searched three trials registers for ongoing studies, and four websites for additional trials (April 2022). We did not impose any language or date restrictions.

SELECTION CRITERIA

Randomised controlled trials (RCTs) of any virus vaccine compared with placebo to prevent the common cold in healthy people.

DATA COLLECTION AND ANALYSIS

We used Cochrane's Screen4Me workflow to assess the initial search results. Four review authors independently performed title and abstract screening to identify potentially relevant studies. We retrieved the full-text articles for those studies deemed potentially relevant, and the review authors independently screened the full-text reports for inclusion in the review, recording reasons for exclusion of the excluded studies. Any disagreements were resolved by discussion or by consulting a third review author when needed. Two review authors independently collected data on a data extraction form, resolving any disagreements by consensus or by involving a third review author. We double-checked data transferred into Review Manager 5 software. Three review authors independently assessed risk of bias using RoB 1 tool as outlined in the Cochrane Handbook for Systematic Reviews of Interventions. We carried out statistical analysis using Review Manager 5. We did not conduct a meta-analysis, and we did not assess publication bias. We used GRADEpro GDT software to assess the certainty of the evidence and to create a summary of findings table.  MAIN RESULTS: We did not identify any new RCTs for inclusion in this update. This review includes one RCT conducted in 1965 with an overall high risk of bias. The RCT included 2307 healthy young men in a military facility, all of whom were included in the analyses, and compared the effect of three adenovirus vaccines (live, inactivated type 4, and inactivated type 4 and 7) against a placebo (injection of physiological saline or gelatin capsule). There were 13 (1.14%) events in 1139 participants in the vaccine group, and 14 (1.19%) events in 1168 participants in the placebo group. Overall, we do not know if there is a difference between the adenovirus vaccine and placebo in reducing the incidence of the common cold (risk ratio 0.95, 95% confidence interval 0.45 to 2.02; very low-certainty evidence). Furthermore, no difference in adverse events when comparing live vaccine preparation with placebo was reported. We downgraded the certainty of the evidence to very low due to unclear risk of bias, indirectness because the population of this study was only young men, and imprecision because confidence intervals were wide and the number of events was low. The included study did not assess vaccine-related or all-cause mortality.  AUTHORS' CONCLUSIONS: This Cochrane Review was based on one study with very low-certainty evidence, which showed that there may be no difference between the adenovirus vaccine and placebo in reducing the incidence of the common cold. We identified a need for well-designed, adequately powered RCTs to investigate vaccines for the common cold in healthy people. Future trials on interventions for preventing the common cold should assess a variety of virus vaccines for this condition, and should measure such outcomes as common cold incidence, vaccine safety, and mortality (all-cause and related to the vaccine).

Project rebuild the evidence base (REB): A method to interpret randomised clinical trials and their meta-analysis to present solid benefit-risk assessments to patients

Evidence-based medicine is the cornerstone of shared-decision making in healthcare today. The public deserves clear, transparent and trust-worthy information on drug efficacy. Yet today, many drugs are prescribed and used without solid evidence of efficacy. Clinical trials and randomised clinical trials (RCTs) are the best method to evaluate drug efficacy and side effects. In a shared medical decision-making approach, general practitioners need drug assessment based on patient-important outcomes. The aim of project rebuild the evidence base (REB) is to bridge the gap between the data needed in clinical practice and the data available from clinical research. The drugs will be assessed on clinical patient important outcomes and for a population. Using the Cochrane tools, we propose to analyse for each population and outcome: 1) a meta-analysis based on RCTs with a low risk of bias overall; 2) an evaluation of results of confirmatory RCTs; 3) a statistical analysis of heterrogeneity between RCTs and 4) an analysis of publication bias. Depending on the results of these analyses, the evidence will be categorized in 4 different levels: firm evidence, evidence (to be confirmed), signal or absence of evidence. Project REB proposes a method for reading and interpreting RCTs and their meta-analysis to produce quality data for general practitioners to focus on risk-benefit assessment in the interest of patients. If this data does not exist, it could enable clinical research to better its aim.

Stepping up to meet the challenge of freezing of gait in Parkinson's disease

There has been a growing appreciation for freezing of gait as a disabling symptom that causes a significant burden in Parkinson’s disease. Previous research has highlighted some of the key components that underlie the phenomenon, but these reductionist approaches have yet to lead to a paradigm shift resulting in the development of novel treatment strategies. Addressing this issue will require greater integration of multi-modal data with complex computational modeling, but there are a number of critical aspects that need to be considered before embarking on such an approach. This paper highlights where the field needs to address current gaps and shortcomings including the standardization of definitions and measurement, phenomenology and pathophysiology, as well as considering what available data exist and how future studies should be constructed to achieve the greatest potential to better understand and treat this devastating symptom.

Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation

Background: An irreproducibility crisis currently afflicts a wide range of scientific disciplines, including public health and biomedical science. A study was undertaken to assess the reliability of a meta-analysis examining whether air quality components (carbon monoxide, particulate matter 10 µm and 2.5 µm (PM10 and PM2.5), sulfur dioxide, nitrogen dioxide and ozone) are risk factors for asthma exacerbation. Methods: The number of statistical tests and models were counted in 17 randomly selected base papers from 87 used in the meta-analysis. Confidence intervals from all 87 base papers were converted to p-values. p-value plots for each air component were constructed to evaluate the effect heterogeneity of the p-values. Results: The number of statistical tests possible in the 17 selected base papers was large, median = 15,360 (interquartile range = 1536–40,960), in comparison to results presented. Each p-value plot showed a two-component mixture with small p-values < 0.001 while other p-values appeared random (p-values > 0.05). Given potentially large numbers of statistical tests conducted in the 17 selected base papers, p-hacking cannot be ruled out as explanations for small p-values. Conclusions: Our interpretation of the meta-analysis is that random p-values indicating null associations are more plausible and the meta-analysis is unlikely to replicate in the absence of bias.

Reconciling estimates of global spread and infection fatality rates of COVID-19: An overview of systematic evaluations

BACKGROUND

Estimates of community spread and infection fatality rate (IFR) of COVID-19 have varied across studies. Efforts to synthesize the evidence reach seemingly discrepant conclusions.

METHODS

Systematic evaluations of seroprevalence studies that had no restrictions based on country and which estimated either total number of people infected and/or aggregate IFRs were identified. Information was extracted and compared on eligibility criteria, searches, amount of evidence included, corrections/adjustments of seroprevalence and death counts, quantitative syntheses and handling of heterogeneity, main estimates and global representativeness.

RESULTS

Six systematic evaluations were eligible. Each combined data from 10 to 338 studies (9-50 countries), because of different eligibility criteria. Two evaluations had some overt flaws in data, violations of stated eligibility criteria and biased eligibility criteria (eg excluding studies with few deaths) that consistently inflated IFR estimates. Perusal of quantitative synthesis methods also exhibited several challenges and biases. Global representativeness was low with 78%-100% of the evidence coming from Europe or the Americas; the two most problematic evaluations considered only one study from other continents. Allowing for these caveats, four evaluations largely agreed in their main final estimates for global spread of the pandemic and the other two evaluations would also agree after correcting overt flaws and biases.

CONCLUSIONS

All systematic evaluations of seroprevalence data converge that SARS-CoV-2 infection is widely spread globally. Acknowledging residual uncertainties, the available evidence suggests average global IFR of ~0.15% and ~1.5-2.0 billion infections by February 2021 with substantial differences in IFR and in infection spread across continents, countries and locations.

Validation of a search strategy for randomized clinical trials related to periodontitis

Background

Systematic reviews, considered the gold standard for the assessment of scientific evidence, may present conflicting findings for the same clinical issue, and such dissent may be justified by the forms of elaboration of the electronic search strategy. This paper aims to validate a search strategy to identify randomized clinical trials related to periodontitis. A gold standard reference set was developed to validate the identified clinical trials using the relative recall method. The choice of periodontitis is due to the fact that this disease has a high prevalence among chronic non-communicable diseases, is considered the second most common oral disease in the world, is associated with several health problems, such as cardiovascular diseases and diabetes, and, principally, has not been investigated sufficiently to prevent possible damages resulting from it.

Methods

A validation study was developed in MEDLINE/PubMed. In Stage 1, a methodological filter recommended by the Cochrane Collaboration to identify randomized clinical trials was applied. Stage 2 identified articles related only to periodontitis (gold standard reference set) from among the articles retrieved using the eligibility criteria. In Stage 3, a search statement for the retrieval of periodontitis-related articles was elaborated by experts. Stage 4 defined the proposed search strategy comprising of the combination of the search statement developed with the aforementioned methodological filter and subsequent application in MEDLINE/PubMed. The obtained data were analyzed using the set of articles identified in Stage 2, as the gold standard reference set. The following performance values were calculated - sensitivity, specificity, accuracy, and number needed to read - with their respective 95% confidence interval (95%CI).

Results

The search strategy under evaluation compared to the gold-standard showed a sensitivity of 93.2% (_95%CI, 83.8–97.3), specificity of 99.9% (_95%CI 99.8–99.9), and a precision of 77.5% (_95%CI, 66.48–85.63). In addition, the number needed to read was 1.3.

Conclusion

According to the proposed methodological approach, the search strategy under evaluation performed well in the identification of randomized clinical trials related to periodontitis.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13690-021-00560-0.

Interventions for treating leg ulcers in people with sickle cell disease

BACKGROUND

The frequency of skin ulceration makes an important contributor to the morbidity burden in people with sickle cell disease. Many treatment options are available to the healthcare professional, although it is uncertain which treatments have been assessed for effectiveness in people with sickle cell disease. This is an update of a previously published Cochrane Review.

OBJECTIVES

To assess the clinical effectiveness and harms of interventions for treating leg ulcers in people with sickle cell disease.

SEARCH METHODS

We searched the Cochrane Cystic Fibrosis and Genetic Disorders Group's Haemoglobinopathies Trials Register. We searched LILACS (1982 to January 2020), ISI Web of Knowledge (1985 to January 2020), and the Clinical Trials Search Portal of the World Health Organization (January 2020). We checked the reference lists of all the trials identified. We also contacted those groups or individuals who may have completed relevant randomised trials in this area. Date of the last search of the Cochrane Cystic Fibrosis and Genetic Disorders Group's Haemoglobinopathies Trials Register: 13 January 2020; date of the last search of the Cochrane Wounds Group Trials Register: 17 February 2017.

SELECTION CRITERIA

Randomised controlled trials of interventions for treating leg ulcers in people with sickle cell disease compared to placebo or an alternative treatment.

DATA COLLECTION AND ANALYSIS

Two authors independently selected studies for inclusion. All three authors independently assessed the risk of bias of the included studies and extracted data. We used GRADE to assess the quality of the evidence.

MAIN RESULTS

Six studies met the inclusion criteria (198 participants with 250 ulcers). Each trial investigated a different intervention and within this review we have grouped these as systemic pharmaceutical interventions (L-cartinine, arginine butyrate, isoxsuprine) and topical pharmaceutical interventions (Solcoseryl® cream, arginine-glycine-aspartic acid (RGD) peptide dressing and topical antibiotics). No trials on non-pharmaceutical interventions were included in the review. All trials had an overall unclear or high risk of bias, and drug companies sponsored four of them. We were unable to pool findings due to the heterogeneity in outcome definitions, and inconsistency between the units of randomisation and analysis. Three interventions reported on the change in ulcer size (arginine butyrate, RGD peptide, L-cartinine). Of these, only arginine butyrate showed a reduction of ulcer size compared with a control group, mean reduction -5.10 cm² (95% CI -9.65 to -0.55), but we are uncertain whether this reduces ulcer size compared to standard care alone as the certainty of the evidence has been assessed as very low. Three trials reported on complete leg ulcer closure (isoxsuprine, arginine butyrate, RGD peptide matrix; very low quality of evidence). None reported a clinical benefit. No trial reported on: the time to complete ulcer healing; ulcer-free survival following treatment for sickle cell leg ulcers; quality of life measures; incidence of amputation or harms.

AUTHORS' CONCLUSIONS

Given the very low quality of the evidence identified in this updated Cochrane Review we are uncertain whether any of the assessed pharmaceutical interventions reduce ulcer size or result in leg ulcer closure in treated participants compared to controls. However, this intervention was assessed as having a high risk of bias due to inadequacies in the single trial report. Other included studies were also assessed as having an unclear or high risk of bias. The harm profile of the all interventions remains inconclusive.

Citicoline for treating people with acute ischemic stroke

BACKGROUND

Stroke is one of the leading causes of long-lasting disability and mortality and its global burden has increased in the past two decades. Several therapies have been proposed for the recovery from, and treatment of, ischemic stroke. One of them is citicoline. This review assessed the benefits and harms of citicoline for treating patients with acute ischemic stroke.

OBJECTIVES

To assess the clinical benefits and harms of citicoline compared with placebo or any other control for treating people with acute ischemic stroke.

SEARCH METHODS

We searched in the Cochrane Stroke Group Trials Register, CENTRAL, MEDLINE Ovid, Embase Ovid, LILACS until 29 January 2020. We searched the World Health Organization Clinical Trials Search Portal and ClinicalTrials.gov. Additionally, we also reviewed reference lists of the retrieved publications and review articles, and searched the websites of the US Food and Drug Administration (FDA) and European Medicines Agency (EMA).

SELECTION CRITERIA

We included randomized controlled trials (RCTs) in any setting including participants with acute ischemic stroke. Trials were eligible for inclusion if they compared citicoline versus placebo or no intervention.

DATA COLLECTION AND ANALYSIS

We selected RCTs, assessed the risk of bias in seven domains, and extracted data by duplicate. Our primary outcomes of interest were all-cause mortality and the degree of disability or dependence in daily activities at 90 days. We estimated risk ratios (RRs) for dichotomous outcomes. We measured statistical heterogeneity using the I² statistic. We conducted our analyses using the fixed-effect and random-effects model meta-analyses. We assessed the overall quality of evidence for six pre-specified outcomes using the GRADE approach.

MAIN RESULTS

We identified 10 RCTs including 4281 participants. In all these trials, citicoline was given either orally, intravenously, or a combination of both compared with placebo or standard care therapy. Citicoline doses ranged between 500 mg and 2000 mg per day. We assessed all the included trials as having high risk of bias. Drug companies sponsored six trials. A pooled analysis of eight trials indicates there may be little or no difference in all-cause mortality comparing citicoline with placebo (17.3% versus 18.5%; RR 0.94, 95% CI 0.83 to 1.07; I² = 0%; low-quality evidence due to risk of bias). Four trials found no difference in the proportion of patients with disability or dependence in daily activities according to the Rankin scale comparing citicoline with placebo (21.72% versus 19.23%; RR 1.11, 95% CI 0.97 to 1.26; I² = 1%; low-quality evidence due to risk of bias). Meta-analysis of three trials indicates there may be little or no difference in serious cardiovascular adverse events comparing citicoline with placebo (8.83% versus 7.77%; RR 1.04, 95% CI 0.84 to 1.29; I² = 0%; low-quality evidence due to risk of bias). Overall, either serious or non-serious adverse events - central nervous system, gastrointestinal, musculoskeletal, etc. - were poorly reported and harms may have been underestimated. Four trials assessing functional recovery with the Barthel Index at a cut-off point of 95 points or more did not find differences comparing citicoline with placebo (32.78% versus 30.70%; RR 1.03, 95% CI 0.94 to 1.13; I² = 24%; low-quality evidence due to risk of bias). There were no differences in neurological function (National Institutes of Health Stroke Scale at a cut-off point of ≤ 1 points) comparing citicoline with placebo according to five trials (24.31% versus 22.44%; RR 1.08, 95% CI 0.96 to 1.21; I² = 27%, low-quality evidence due to risk of bias). A pre-planned Trial Sequential Analysis suggested that no more trials may be needed for the primary outcomes but no trial provided information on quality of life.

AUTHORS' CONCLUSIONS

This review assessed the clinical benefits and harms of citicoline compared with placebo or any other standard treatment for people with acute ischemic stroke. The findings of the review suggest there may be little to no difference between citicoline and its controls regarding all-cause mortality, disability or dependence in daily activities, severe adverse events, functional recovery and the assessment of the neurological function, based on low-certainty evidence. None of the included trials assessed quality of life and the safety profile of citicoline remains unknown. The available evidence is of low quality due to either limitations in the design or execution of the trials.

Sample size evolution in neuroimaging research: An evaluation of highly-cited studies (1990-2012) and of latest practices (2017-2018) in high-impact journals

We evaluated 1038 of the most cited structural and functional (fMRI) magnetic resonance brain imaging papers (1161 studies) published during 1990-2012 and 270 papers (300 studies) published in top neuroimaging journals in 2017 and 2018. 96% of highly cited experimental fMRI studies had a single group of participants and these studies had median sample size of 12, highly cited clinical fMRI studies (with patient participants) had median sample size of 14.5, and clinical structural MRI studies had median sample size of 50. The sample size of highly cited experimental fMRI studies increased at a rate of 0.74 participant/year and this rate of increase was commensurate with the median sample sizes of neuroimaging studies published in top neuroimaging journals in 2017 (23 participants) and 2018 (24 participants). Only 4 of 131 papers in 2017 and 5 of 142 papers in 2018 had pre-study power calculations, most for single t-tests and correlations. Only 14% of highly cited papers reported the number of excluded participants whereas 49% of papers with their own data in 2017 and 2018 reported excluded participants. Publishers and funders should require pre-study power calculations necessitating the specification of effect sizes. The field should agree on universally required reporting standards. Reporting formats should be standardized so that crucial study parameters could be identified unequivocally.

Interventions for treating intrahepatic cholestasis in people with sickle cell disease

BACKGROUND

Sickle cell disease is the most common hemoglobinopathy occurring worldwide and sickle cell intrahepatic cholestasis is a complication long recognized in this population. Cholestatic liver diseases are characterized by impaired formation or excretion (or both) of bile from the liver. There is a need to assess the clinical benefits and harms of the interventions used to treat intrahepatic cholestasis in people with sickle cell disease. This is an update of a previously published Cochrane Review.

OBJECTIVES

To assess the benefits and harms of the interventions for treating intrahepatic cholestasis in people with sickle cell disease.

SEARCH METHODS

We searched the Cystic Fibrosis and Genetic Disorders Group's Haemoglobinopathies Trials Register, which comprises references identified from comprehensive electronic database searches and handsearching of relevant journals and abstract books of conference proceedings. We also searched the LILACS database (1982 to 21 January 2020), the WHO International Clinical Trials Registry Platform Search Portal and ClinicalTrials.gov (21 January 2020). Date of last search of the Cochrane Cystic Fibrosis and Genetic Disorders Group's Haemoglobinopathies Trials Register: 25 November 2019.

SELECTION CRITERIA

We searched for published or unpublished randomised controlled trials.

DATA COLLECTION AND ANALYSIS

Each author intended to independently extract data, assess the risk of bias of the trials by standard Cochrane methodologies and assess the quality of the evidence using the GRADE criteria; however, no trials were included in the review.

MAIN RESULTS

We did not identify any randomised controlled trials.

AUTHORS' CONCLUSIONS

This updated Cochrane Review did not identify any randomised controlled trials assessing interventions for treating intrahepatic cholestasis in people with sickle cell disease. Randomised controlled trials are needed to establish the optimum treatment for this condition.

Reproducible research into human chemical communication by cues and pheromones: learning from psychology's renaissance

Despite the lack of evidence that the 'putative human pheromones' androstadienone and estratetraenol ever were pheromones, almost 60 studies have claimed 'significant' results. These are quite possibly false positives and can be best seen as potential examples of the 'reproducibility crisis', sadly common in the rest of the life and biomedical sciences, which has many instances of whole fields based on false positives. Experiments on the effects of olfactory cues on human behaviour are also at risk of false positives because they look for subtle effects but use small sample sizes. Research on human chemical communication, much of it falling within psychology, would benefit from vigorously adopting the proposals made by psychologists to enable better, more reliable science, with an emphasis on enhancing reproducibility. A key change is the adoption of study pre-registration and/or Registered Reports which will also reduce publication bias. As we are mammals, and chemical communication is important to other mammals, it is likely that chemical cues are important in our behaviour and that humans may have pheromones, but new approaches will be needed to reliably demonstrate them. This article is part of the Theo Murphy meeting issue 'Olfactory communication in humans'.

A comparison of different antibiotic regimens for the treatment of infective endocarditis

BACKGROUND

Infective endocarditis is a microbial infection of the endocardial surface of the heart. Antibiotics are the cornerstone of treatment, but due to the differences in presentation, populations affected, and the wide variety of micro-organisms that can be responsible, their use is not standardised. This is an update of a review previously published in 2016.

OBJECTIVES

To assess the existing evidence about the clinical benefits and harms of different antibiotics regimens used to treat people with infective endocarditis.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase Classic and Embase, LILACS, CINAHL, and the Conference Proceedings Citation Index - Science on 6 January 2020. We also searched three trials registers and handsearched the reference lists of included papers. We applied no language restrictions.

SELECTION CRITERIA

We included randomised controlled trials (RCTs) assessing the effects of antibiotic regimens for treating definitive infective endocarditis diagnosed according to modified Duke's criteria. We considered all-cause mortality, cure rates, and adverse events as the primary outcomes. We excluded people with possible infective endocarditis and pregnant women.

DATA COLLECTION AND ANALYSIS

Two review authors independently performed study selection, 'Risk of bias' assessment, and data extraction in duplicate. We constructed 'Summary of findings' tables and used GRADE methodology to assess the quality of the evidence. We described the included studies narratively.

MAIN RESULTS

Six small RCTs involving 1143 allocated/632 analysed participants met the inclusion criteria of this first update. The included trials had a high risk of bias. Three trials were sponsored by drug companies. Due to heterogeneity in outcome definitions and different antibiotics used data could not be pooled. The included trials compared miscellaneous antibiotic schedules having uncertain effects for all of the prespecified outcomes in this review. Evidence was either low or very low quality due to high risk of bias and very low number of events and small sample size. The results for all-cause mortality were as follows: one trial compared quinolone (levofloxacin) plus standard treatment (antistaphylococcal penicillin (cloxacillin or dicloxacillin), aminoglycoside (tobramycin or netilmicin), and rifampicin) versus standard treatment alone and reported 8/31 (26%) with levofloxacin plus standard treatment versus 9/39 (23%) with standard treatment alone; risk ratio (RR) 1.12, 95% confidence interval (CI) 0.49 to 2.56. One trial compared fosfomycin plus imipenem 3/4 (75%) versus vancomycin 0/4 (0%) (RR 7.00, 95% CI 0.47 to 103.27), and one trial compared partial oral treatment 7/201 (3.5%) versus conventional intravenous treatment 13/199 (6.53%) (RR 0.53, 95% CI 0.22 to 1.31). The results for rates of cure with or without surgery were as follows: one trial compared daptomycin versus low-dose gentamicin plus an antistaphylococcal penicillin (nafcillin, oxacillin, or flucloxacillin) or vancomycin and reported 9/28 (32.1%) with daptomycin versus 9/25 (36%) with low-dose gentamicin plus antistaphylococcal penicillin or vancomycin; RR 0.89, 95% CI 0.42 to 1.89. One trial compared glycopeptide (vancomycin or teicoplanin) plus gentamicin with cloxacillin plus gentamicin (13/23 (56%) versus 11/11 (100%); RR 0.59, 95% CI 0.40 to 0.85). One trial compared ceftriaxone plus gentamicin versus ceftriaxone alone (15/34 (44%) versus 21/33 (64%); RR 0.69, 95% CI 0.44 to 1.10), and one trial compared fosfomycin plus imipenem versus vancomycin (1/4 (25%) versus 2/4 (50%); RR 0.50, 95% CI 0.07 to 3.55). The included trials reported adverse events, the need for cardiac surgical interventions, and rates of uncontrolled infection, congestive heart failure, relapse of endocarditis, and septic emboli, and found no conclusive differences between groups (very low-quality evidence). No trials assessed quality of life.

AUTHORS' CONCLUSIONS

This first update confirms the findings of the original version of the review. Limited and low to very low-quality evidence suggests that the comparative effects of different antibiotic regimens in terms of cure rates or other relevant clinical outcomes are uncertain. The conclusions of this updated Cochrane Review were based on few RCTs with a high risk of bias. Accordingly, current evidence does not support or reject any regimen of antibiotic therapy for the treatment of infective endocarditis.

Null hypothesis significance testing and effect sizes: can we 'effect' everything … or … anything?

The Null Hypothesis Significance Testing (NHST) paradigm is increasingly criticized. Estimation approaches such as point estimates and confidence intervals, while having limitations, provide better descriptions of results than P-values and statements about significance levels. Their use is supported by many statisticians. The effect size approach is an important part of power and sample size calculations at the experimental design stage and in meta-analysis and in the interpretation of the biological importance of study results. Care is needed, however, to ensure that such effect sizes are relevant for the endpoint. Effect sizes should not be used to interpret results without accompanying limits, such as confidence intervals. New methods, especially Bayesian approaches, are being developed; however, no single method provides a simple answer. Rather there is a need to improve researchers understanding of the complex issues underlying experimental design, statistical analysis and interpretation of results.

Accumulation Bias in meta-analysis: the need to consider time in error control

Studies accumulate over time and meta-analyses are mainly retrospective. These two characteristics introduce dependencies between the analysis time, at which a series of studies is up for meta-analysis, and results within the series. Dependencies introduce bias —Accumulation Bias— and invalidate the sampling distribution assumed for p-value tests, thus inflating type-I errors. But dependencies are also inevitable, since for science to accumulate efficiently, new research needs to be informed by past results. Here, we investigate various ways in which time influences error control in meta-analysis testing. We introduce an Accumulation Bias Framework that allows us to model a wide variety of practically occurring dependencies including study series accumulation, meta-analysis timing, and approaches to multiple testing in living systematic reviews. The strength of this framework is that it shows how all dependencies affect p-value-based tests in a similar manner. This leads to two main conclusions. First, Accumulation Bias is inevitable, and even if it can be approximated and accounted for, no valid p-value tests can be constructed. Second, tests based on likelihood ratios withstand Accumulation Bias: they provide bounds on error probabilities that remain valid despite the bias. We leave the reader with a choice between two proposals to consider time in error control: either treat individual (primary) studies and meta-analyses as two separate worlds — each with their own timing — or integrate individual studies in the meta-analysis world. Taking up likelihood ratios in either approach allows for valid tests that relate well to the accumulating nature of scientific knowledge. Likelihood ratios can be interpreted as betting profits, earned in previous studies and invested in new ones, while the meta-analyst is allowed to cash out at any time and advice against future studies.

Vibration of effects from diverse inclusion/exclusion criteria and analytical choices: 9216 different ways to perform an indirect comparison meta-analysis

BACKGROUND

Different methodological choices such as inclusion/exclusion criteria and analytical models can yield different results and inferences when meta-analyses are performed. We explored the range of such differences, using several methodological choices for indirect comparison meta-analyses to compare nalmefene and naltrexone in the reduction of alcohol consumption as a case study.

METHODS

All double-blind randomized controlled trials (RCTs) comparing nalmefene to naltrexone or one of these compounds to a placebo in the treatment of alcohol dependence or alcohol use disorders were considered. Two reviewers searched for published and unpublished studies in MEDLINE (August 2017), the Cochrane Library, Embase, and ClinicalTrials.gov and contacted pharmaceutical companies, the European Medicines Agency, and the Food and Drug Administration. The indirect comparison meta-analyses were performed according to different inclusion/exclusion criteria (based on medical condition, abstinence of patients before inclusion, gender, somatic and psychiatric comorbidity, psychological support, treatment administered and dose, treatment duration, outcome reported, publication status, and risk of bias) and different analytical models (fixed and random effects). The primary outcome was the vibration of effects (VoE), i.e. the range of different results of the indirect comparison between nalmefene and naltrexone. The presence of a "Janus effect" was investigated, i.e. whether the 1st and 99th percentiles in the distribution of effect sizes were in opposite directions.

RESULTS

Nine nalmefene and 51 naltrexone RCTs were included. No study provided a direct comparison between the drugs. We performed 9216 meta-analyses for the indirect comparison with a median of 16 RCTs (interquartile range = 12-21) included in each meta-analysis. The standardized effect size was negative at the 1st percentile (- 0.29, favouring nalmefene) and positive at the 99th percentile (0.29, favouring naltrexone). A total of 7.1% (425/5961) of the meta-analyses with a negative effect size and 18.9% (616/3255) of those with a positive effect size were statistically significant (p < 0.05).

CONCLUSIONS

The choice of inclusion/exclusion criteria and analytical models for meta-analysis can result in entirely opposite results. VoE evaluations could be performed when overlapping meta-analyses on the same topic yield contradictory result.

TRIAL REGISTRATION

This study was registered on October 19, 2016, in the Open Science Framework (OSF, protocol available at https://osf.io/7bq4y/ ).

Visual Perceptual Load Does Not Affect the Frequency Mismatch Negativity

The mismatch negativity (MMN) has been of particular interest in auditory perception because of its sensitivity to auditory change. It is typically measured in an oddball task and is computed as the difference of deviant minus standard tones. Previous studies suggest that the oddball MMN can be reduced by crossmodal attention to a concurrent, difficult visual task. However, more recent studies did not replicate this effect. Because previous findings seem to be biased, we preregistered the present study and used Bayesian hypothesis testing to measure the strength of evidence for or against an effect of visual task difficulty. We manipulated visual perceptual load (high and low load). In the task, the visual stimuli were identical for both loads to avoid confounding effects from physical differences of the visual stimuli. We also measured the corrected MMN because the oddball MMN may be confounded by physical differences between deviant and standard tones. The corrected MMN is obtained with a separate control condition in which the same tone as the deviant (critical tone) is equiprobable with other tones. The corrected MMN is computed as deviant minus critical tones. Furthermore, we assessed working memory capacity to examine its moderating role. In our large sample (N = 49), the evidential strength in support of no effect of visual load was moderate for the oddball MMN (9.09 > BF01 > 3.57) and anecdotal to moderate for the corrected MMN (4.55 > BF01 > 2.17). Also, working memory capacity did not correlate with the visual load effect on the oddball MMN and the corrected MMN. The present findings support the robustness of the auditory frequency MMN to manipulations of crossmodal, visual attention and suggest that this relationship is not moderated by working memory capacity.

Non-inferiority versus superiority trial design for new antibiotics in an era of high antimicrobial resistance: the case for post-marketing, adaptive randomised controlled trials

Antimicrobial resistance is one of the most important threats to global health security. A range of Gram-negative bacteria associated with high morbidity and mortality are now resistant to almost all available antibiotics. In this context of urgency to develop novel drugs, new antibiotics for multidrug-resistant Gram-negative bacteria (namely, ceftazidime-avibactam, plazomicin, and meropenem-vaborbactam) have been approved by regulatory authorities based on non-inferiority trials that provided no direct evidence of their efficacy against multidrug-resistant bacteria such as Enterobacteriaceae spp, Pseudomonas aeruginosa, Stenotrophomonas maltophilia, Burkholderia cepacia, and Acinetobacter baumannii. The use of non-inferiority and superiority trials, and selection of appropriate and optimal study designs, remains a major challenge in the development, registration, and post-marketing implementation of new antibiotics. Using an example of the development process of ceftazidime-avibactam, we propose a strategy for a new research framework based on adaptive randomised clinical trials. The operational research strategy has the aim of assessing the efficacy of new antibiotics in special groups of patients, such as those infected with multidrug-resistant bacteria, who were not included in earlier phase studies, and for whom it is important to establish an appropriate standard of care.

Systematic reviews and meta-analyses in the health sciences: Best practice methods for research syntheses

RATIONALE

The journal Social Science & Medicine recently adopted the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Moher et al., 2009) as guidelines for authors to use when disseminating their systematic reviews (SRs).

APPROACH

After providing a brief history of evidence synthesis, this article describes why reporting standards are important, summarizes the sequential steps involved in conducting SRs and meta-analyses, and outlines additional methodological issues that researchers should address when conducting and reporting results from their SRs.

RESULTS AND CONCLUSIONS

Successful SRs result when teams of reviewers with appropriate expertise use the highest scientific rigor in all steps of the SR process. Thus, SRs that lack foresight are unlikely to prove successful. We advocate that SR teams consider potential moderators (M) when defining their research problem, along with Time, Outcomes, Population, Intervention, Context, and Study design (i.e., TOPICS + M). We also show that, because the PRISMA reporting standards only partially overlap dimensions of methodological quality, it is possible for SRs to satisfy PRISMA standards yet still have poor methodological quality. As well, we discuss limitations of such standards and instruments in the face of the assumptions of the SR process, including meta-analysis spanning the other SR steps, which are highly synergistic: Study search and selection, coding of study characteristics and effects, analysis, interpretation, reporting, and finally, re-analysis and criticism. When a SR targets an important question with the best possible SR methods, its results can become a definitive statement that guides future research and policy decisions for years to come.

Acetyl-L-carnitine for patients with hepatic encephalopathy

BACKGROUND

Hepatic encephalopathy is a common and devastating neuropsychiatric complication of acute liver failure or chronic liver disease. Ammonia content in the blood seems to play a role in the development of hepatic encephalopathy. Treatment for hepatic encephalopathy is complex. Acetyl-L-carnitine is a substance that may reduce ammonia toxicity. This review assessed the benefits and harms of acetyl-L-carnitine for patients with hepatic encephalopathy.

OBJECTIVES

To assess the benefits and harms of acetyl-L-carnitine for patients with hepatic encephalopathy.

SEARCH METHODS

We searched the Cochrane Hepato-Biliary Group Controlled Trials Register, CENTRAL, MEDLINE Ovid, Embase Ovid, LILACS, and Science Citation Index Expanded for randomised clinical trials. We sought additional randomised clinical trials from the World Health Organization Clinical Trials Search Portal and ClinicalTrials.gov. We performed all electronic searches until 10 September 2018. We looked through the reference lists of retrieved publications and review articles, and we searched the FDA and EMA websites.

SELECTION CRITERIA

We searched for randomised clinical trials in any setting, recruiting people with hepatic encephalopathy. Trials were eligible for inclusion if they compared acetyl-L-carnitine plus standard care (e.g. antibiotics, lactulose) versus placebo or no acetyl-L-carnitine plus standard care. We are well aware that by selecting randomised clinical trials, we placed greater focus on potential benefits than on potential harms.

DATA COLLECTION AND ANALYSIS

We selected randomised clinical trials, assessed risk of bias in eight domains, and extracted data in a duplicate and independent fashion. We estimated risk ratios (RRs) for dichotomous outcomes and mean differences (MDs) for continuous outcomes. We measured statistical heterogeneity using I² and D² statistics. We subjected our analyses to fixed-effect and random-effects model meta-analyses. We assessed bias risk domains to control systematic errors. We assessed overall quality of the data for each individual outcome by using the GRADE approach.

MAIN RESULTS

We identified five randomised clinical trials involving 398 participants. All trials included only participants with cirrhosis as the underlying cause of hepatic encephalopathy. Trials included participants with covert or overt hepatic encephalopathy. All trials were conducted in Italy by a single team and assessed acetyl-L-carnitine compared with placebo. Oral intervention was the most frequent route of administration. All trials were at high risk of bias and were underpowered. None of the trials were sponsored by the pharmaceutical industry.None of the identified trials reported information on all-cause mortality, serious adverse events, or days of hospitalisation. Only one trial assessed quality of life using the Short Form (SF)-36 scale (67 participants; very low-quality evidence). The effects of acetyl-L-carnitine compared with placebo on general health at 90 days are uncertain (MD -6.20 points, 95% confidence interval (CI) -9.51 to -2.89). Results for additional domains of the SF-36 are also uncertain. One trial assessed fatigue using the Wessely and Powell test (121 participants; very low-quality evidence). The effects are uncertain in people with moderate-grade hepatic encephalopathy (mental fatigue: MD 0.40 points, 95% CI -0.21 to 1.01; physical fatigue: MD -0.20 points, 95% CI -0.92 to 0.52) and mild-grade hepatic encephalopathy (mental fatigue: -0.80 points, 95% CI -1.48 to -0.12; physical fatigue: 0.20 points, 95% CI -0.72 to 1.12). Meta-analysis showed a reduction in blood ammonium levels favouring acetyl-L-carnitine versus placebo (MD -13.06 mg/dL, 95% CI -17.24 to -8.99; 387 participants; 5 trials; very low-quality evidence). It is unclear whether acetyl-L-carnitine versus placebo increases the risk of non-serious adverse events (8/126 (6.34%) vs 3/120 (2.50%); RR 2.51, 95% CI 0.68 to 9.22; 2 trials; very low-quality evidence). Overall, adverse events data were poorly reported and harms may have been underestimated.

AUTHORS' CONCLUSIONS

This Cochrane systematic review analysed a heterogeneous group of five trials at high risk of bias and with high risk of random errors conducted by only one research team. We assessed acetyl-L-carnitine versus placebo in participants with cirrhosis with covert or overt hepatic encephalopathy. Hence, we have no data on the drug for hepatic encephalopathy in acute liver failure. We found no information about all-cause mortality, serious adverse events, or days of hospitalisation. We found no clear differences in effect between acetyl-L-carnitine and placebo regarding quality of life, fatigue, and non-serious adverse events. Acetyl-L-carnitine reduces blood ammonium levels compared with placebo. We rated all evidence as of very low quality due to pitfalls in design and execution, inconsistency, small sample sizes, and very few events. The harms profile for acetyl-L-carnitine is presently unclear. Accordingly, we need further randomised clinical trials to assess acetyl-L-carnitine versus placebo conducted according to the SPIRIT statements and reported according to the CONSORT statements.

A Bayesian nonparametric test of significance chasing biases

There is a growing concern that much of the published research literature is distorted by the pursuit of statistically significant results. In a seminal article, Ioannidis and Trikalinos (2007, Clinical Trials) proposed an omnibus (I&T) test for significance chasing (SC) biases. This test compares the observed number of studies that report statistically significant results, against their expected number based on study power, assuming a common effect size across studies. The current article extends this approach by developing a Bayesian nonparametric (BNP) meta-regression model and test of SC bias, which can diagnose bias at the individual study level. This new BNP test is based on a flexible model of the predictive distribution of study power, conditionally on study-level covariates which account for study diversity, including diversity due to heterogeneous effect sizes across studies. A test of SC bias proceeds by comparing each study's significant outcome report indicator against its estimated posterior predictive distribution of study power, conditionally on the study's covariates. The BNP model and SC bias test are illustrated through the analyses of 3 meta-analytic data sets and through a simulation study. Software code for the BNP model and test, and the data sets, are provided as Supporting Information.

Variability of Coefficient Alpha: An Empirical Investigation of the Scales of Psychological Wellbeing

Using reliability generalization analysis, the purpose of this study was to characterize the average score reliability, the variability of the score reliability estimates, and explore possible characteristics (e.g., sample size) that influence the reliability of scores across studies using the Scales of Psychological Wellbeing (PWB; Ryff, 1989 , 2014 ). Published studies were included in this investigation if they appeared in a peer-reviewed journal, used 1 or more PWB subscales, estimated coefficient alpha value(s) for the PWB subscale(s), and were written in English. Of the 924 articles generated by the search strategy, a total of 264 were included in the final sample for meta-analysis. The average value reported for coefficient alpha referencing the composite PWB Scale was 0.858, with mean coefficient alphas ranging from 0.722 for the autonomy subscale to 0.801 for the self-acceptance subscale. The 95% prediction intervals ranged from [.653, .996] for the composite PWB. The lower bound of the prediction intervals for specific subscales were >.350. Moderator analyses revealed significant differences in score reliability estimates across select sample and test characteristics. Most notably, R2 values linked with test length ranged from 40% to 71%. Concerns were identified with the use of the 3-item per PWB subscale which reinforces claims advanced by Ryff (2014) . Suggestions for researchers using the PWB are advanced which span measurement considerations and standards of reporting. Psychological researchers who calculate score reliability estimates within their own work should recognize the implications of alpha coefficient values on validity, null hypothesis significant testing, and effect sizes.

Homocysteine-lowering interventions for preventing cardiovascular events

BACKGROUND

Cardiovascular disease, which includes coronary artery disease, stroke and peripheral vascular disease, is a leading cause of death worldwide. Homocysteine is an amino acid with biological functions in methionine metabolism. A postulated risk factor for cardiovascular disease is an elevated circulating total homocysteine level. The impact of homocysteine-lowering interventions, given to patients in the form of vitamins B6, B9 or B12 supplements, on cardiovascular events has been investigated. This is an update of a review previously published in 2009, 2013, and 2015.

OBJECTIVES

To determine whether homocysteine-lowering interventions, provided to patients with and without pre-existing cardiovascular disease are effective in preventing cardiovascular events, as well as reducing all-cause mortality, and to evaluate their safety.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL 2017, Issue 5), MEDLINE (1946 to 1 June 2017), Embase (1980 to 2017 week 22) and LILACS (1986 to 1 June 2017). We also searched Web of Science (1970 to 1 June 2017). We handsearched the reference lists of included papers. We also contacted researchers in the field. There was no language restriction in the search.

SELECTION CRITERIA

We included randomised controlled trials assessing the effects of homocysteine-lowering interventions for preventing cardiovascular events with a follow-up period of one year or longer. We considered myocardial infarction and stroke as the primary outcomes. We excluded studies in patients with end-stage renal disease.

DATA COLLECTION AND ANALYSIS

We performed study selection, 'Risk of bias' assessment and data extraction in duplicate. We estimated risk ratios (RR) for dichotomous outcomes. We calculated the number needed to treat for an additional beneficial outcome (NNTB). We measured statistical heterogeneity using the I2 statistic. We used a random-effects model. We conducted trial sequential analyses, Bayes factor, and fragility indices where appropriate.

MAIN RESULTS

In this third update, we identified three new randomised controlled trials, for a total of 15 randomised controlled trials involving 71,422 participants. Nine trials (60%) had low risk of bias, length of follow-up ranged from one to 7.3 years. Compared with placebo, there were no differences in effects of homocysteine-lowering interventions on myocardial infarction (homocysteine-lowering = 7.1% versus placebo = 6.0%; RR 1.02, 95% confidence interval (CI) 0.95 to 1.10, I2 = 0%, 12 trials; N = 46,699; Bayes factor 1.04, high-quality evidence), death from any cause (homocysteine-lowering = 11.7% versus placebo = 12.3%, RR 1.01, 95% CI 0.96 to 1.06, I2 = 0%, 11 trials, N = 44,817; Bayes factor = 1.05, high-quality evidence), or serious adverse events (homocysteine-lowering = 8.3% versus comparator = 8.5%, RR 1.07, 95% CI 1.00 to 1.14, I2 = 0%, eight trials, N = 35,788; high-quality evidence). Compared with placebo, homocysteine-lowering interventions were associated with reduced stroke outcome (homocysteine-lowering = 4.3% versus comparator = 5.1%, RR 0.90, 95% CI 0.82 to 0.99, I2 = 8%, 10 trials, N = 44,224; high-quality evidence). Compared with low doses, there were uncertain effects of high doses of homocysteine-lowering interventions on stroke (high = 10.8% versus low = 11.2%, RR 0.90, 95% CI 0.66 to 1.22, I2 = 72%, two trials, N = 3929; very low-quality evidence).We found no evidence of publication bias.

AUTHORS' CONCLUSIONS

In this third update of the Cochrane review, there were no differences in effects of homocysteine-lowering interventions in the form of supplements of vitamins B6, B9 or B12 given alone or in combination comparing with placebo on myocardial infarction, death from any cause or adverse events. In terms of stroke, this review found a small difference in effect favouring to homocysteine-lowering interventions in the form of supplements of vitamins B6, B9 or B12 given alone or in combination comparing with placebo.There were uncertain effects of enalapril plus folic acid compared with enalapril on stroke; approximately 143 (95% CI 85 to 428) people would need to be treated for 5.4 years to prevent 1 stroke, this evidence emerged from one mega-trial.Trial sequential analyses showed that additional trials are unlikely to increase the certainty about the findings of this issue regarding homocysteine-lowering interventions versus placebo. There is a need for additional trials comparing homocysteine-lowering interventions combined with antihypertensive medication versus antihypertensive medication, and homocysteine-lowering interventions at high doses versus homocysteine-lowering interventions at low doses. Potential trials should be large and co-operative.

Interventions for treating intrahepatic cholestasis in people with sickle cell disease

BACKGROUND

Sickle cell disease is the most common hemoglobinopathy occurring worldwide and sickle cell intrahepatic cholestasis is a complication long recognized in this population. Cholestatic liver diseases are characterized by impaired formation or excretion (or both) of bile from the liver. There is a need to assess the clinical benefits and harms of the interventions used to treat intrahepatic cholestasis in people with sickle cell disease. This is an update of a previously published Cochrane Review.

OBJECTIVES

To assess the benefits and harms of the interventions for treating intrahepatic cholestasis in people with sickle cell disease.

SEARCH METHODS

We searched the Cystic Fibrosis and Genetic Disorders Group's Haemoglobinopathies Trials Register, which comprises references identified from comprehensive electronic database searches and handsearching of relevant journals and abstract books of conference proceedings. We also searched the LILACS database (1982 to 23 May 2017), the WHO International Clinical Trials Registry Platform Search Portal (23 May 2017) and ClinicalTrials.gov.Date of last search of the Cochrane Cystic Fibrosis and Genetic Disorders Group's Haemoglobinopathies Trials Register: 12 April 2017.

SELECTION CRITERIA

We searched for published or unpublished randomised controlled trials.

DATA COLLECTION AND ANALYSIS

Each author intended to independently extract data and assess the risk of bias of the trials by standard Cochrane methodologies; however, no trials were included in the review.

MAIN RESULTS

There were no randomised controlled trials identified.

AUTHORS' CONCLUSIONS

This updated Cochrane Review did not identify any randomised controlled trials assessing interventions for treating intrahepatic cholestasis in people with sickle cell disease. Randomised controlled trials are needed to establish the optimum treatment for this condition.

The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research

The widespread use of 'statistical significance' as a license for making a claim of a scientific finding leads to considerable distortion of the scientific process (according to the American Statistical Association). We review why degrading p-values into 'significant' and 'nonsignificant' contributes to making studies irreproducible, or to making them seem irreproducible. A major problem is that we tend to take small p-values at face value, but mistrust results with larger p-values. In either case, p-values tell little about reliability of research, because they are hardly replicable even if an alternative hypothesis is true. Also significance (p ≤ 0.05) is hardly replicable: at a good statistical power of 80%, two studies will be 'conflicting', meaning that one is significant and the other is not, in one third of the cases if there is a true effect. A replication can therefore not be interpreted as having failed only because it is nonsignificant. Many apparent replication failures may thus reflect faulty judgment based on significance thresholds rather than a crisis of unreplicable research. Reliable conclusions on replicability and practical importance of a finding can only be drawn using cumulative evidence from multiple independent studies. However, applying significance thresholds makes cumulative knowledge unreliable. One reason is that with anything but ideal statistical power, significant effect sizes will be biased upwards. Interpreting inflated significant results while ignoring nonsignificant results will thus lead to wrong conclusions. But current incentives to hunt for significance lead to selective reporting and to publication bias against nonsignificant findings. Data dredging, p-hacking, and publication bias should be addressed by removing fixed significance thresholds. Consistent with the recommendations of the late Ronald Fisher, p-values should be interpreted as graded measures of the strength of evidence against the null hypothesis. Also larger p-values offer some evidence against the null hypothesis, and they cannot be interpreted as supporting the null hypothesis, falsely concluding that 'there is no effect'. Information on possible true effect sizes that are compatible with the data must be obtained from the point estimate, e.g., from a sample average, and from the interval estimate, such as a confidence interval. We review how confusion about interpretation of larger p-values can be traced back to historical disputes among the founders of modern statistics. We further discuss potential arguments against removing significance thresholds, for example that decision rules should rather be more stringent, that sample sizes could decrease, or that p-values should better be completely abandoned. We conclude that whatever method of statistical inference we use, dichotomous threshold thinking must give way to non-automated informed judgment.

Interventions for treating patients with chikungunya virus infection-related rheumatic and musculoskeletal disorders: A systematic review

Background

Chikungunya virus infection (CHIKV) is caused by a mosquito-borne alphavirus. CHIKV causes high fever and painful rheumatic disorders that may persist for years. Because little is known about interventions for treating CHIKV-related illness, we conducted a systematic review.

Methods

We used Cochrane methods. We searched PubMed, EMBASE, Cochrane Library, LILACS and other sources from the earliest records to March 2016. We had no language restrictions. We included randomized controlled trials assessing any intervention for treating acute or chronic CHIKV-related illness. Our primary outcomes were pain relief, global health status (GHS) or health related quality of life (HRQL), and serious adverse events (SAEs). We assessed bias risk with the Cochrane tool and used GRADE to assess evidence quality.

Results

We screened 2,229 records and found five small trials with a total of 402 participants. Patients receiving chloroquine (CHQ) had better chronic pain relief than those receiving placebo (relative risk [RR] 2.67, 95% confidence interval [CI] 1.23 to 5.77, N = 54), but acute pain relief was marginally not different between groups (mean difference [MD] 1.46, 95% CI 0.00 to 2.92, N = 54). SAEs were similar (RR = 15.00, 95% CI 0.90 to 250.24, N = 54). Comparing CHQ with paracetamol (PCM), CHQ patients had better pain relief (RR = 1.52, 95% CI 1.20 to 1.93, N = 86). Compared with hydroxychloroquine (HCHQ), disease-modifying anti-rheumatic drugs (DMARDs) reduced pain (MD = -14.80, 95% CI -19.12 to -10.48, N = 72). DMARDs patients had less disability (MD = -0.74, 95% CI -0.92 to -0.56, N = 72) and less disease activity (MD = -1.35; 95% CI -1.70 to -1.00; N = 72). SAEs were similar between DMARDs and HCHQ groups (RR = 2.84, 95% CI 0.12 to 67.53, N = 72). Comparing meloxicam (MXM) with CHQ, there was no difference in pain relief (MD = 0.24, 95% CI = -0.81 to 1.29; p = 0.65, N = 70), GHS or HRQL (MD = -0.31, 95% CI -2.06 to 1.44, N = 70) or SAEs (RR = 0.85, 95% CI 0.30 to 2.42, N = 70). Finally, a four-arm trial (N = 120) compared aceclofenac (ACF) monotherapy to ACF+HCHQ, ACF+ prednisolone (PRD), or ACF+HCHQ+PRD. Investigators found reduced pain (p<0.001) and better HRQL (p<0.001) in the two patient groups receiving PRD, compared to those receiving ACF monotherapy or ACF+HCHQ. Trials were at high risk of bias. GRADE evidence quality for all outcomes was very low.

Conclusion

Results from these small trials provide insufficient evidence to draw conclusions about the efficacy or safety of CHIKV interventions. Physicians should be cautious in prescribing and policy-makers should be cautious in recommending any intervention reviewed here. Rigorous trials with sufficient statistical power are urgently needed, with results stratified by disease stage and symptomology.

A post-publication analysis of the idealized upper reference value of 2.5 mIU/L for TSH: Time to support the thyroid axis with magnesium and iron especially in the setting of reproduction medicine

Laboratory medicine approaches the evaluation of thyroid function mostly through the single determination of the blood level of thyroid stimulating hormone (TSH). Some authors have suggested an upper reference value for TSH of 2.5 mIU/L. This suggestion has not been confirmed by recent clinical studies. These studies have delivered a clinically valid reference range going from 0.3 to 3.5 mIU/L. These values are valid for both for the general population as well as in the setting of fertility and pregnancy. Current biochemical evidence about the elements required to maintain thyroid function shows that these not only include dietary iodine but also magnesium, iron, selenium and coenzyme Q10. Iron is important for the synthesis of thyroid peroxidase; magnesium-ATP contributes to the active process of iodine uptake; iodine has to be sufficiently present in the diet; selenium acts through selenoproteins to protect the thyroid cell during hormone synthesis and in deiodination of thyroxine; coenzyme Q10 influences thyroid vascularity. As a consequence, good clinical practice requires additional biochemical information on the blood levels of magnesium, selenium, coenzyme Q10 as well as iron status. Since these elements are also important for the maintenance of reproductive function, we postulate that they constitute the connecting link between both endocrine systems.

Vaccines for the common cold

BACKGROUND

The common cold is a spontaneously remitting infection of the upper respiratory tract, characterised by a runny nose, nasal congestion, sneezing, cough, malaise, sore throat, and fever (usually < 37.8º C). The widespread morbidity caused by the common cold worldwide is related to its ubiquitousness rather than its severity. The development of vaccines for the common cold has been difficult because of antigenic variability of the common cold virus and the indistinguishable multiple other viruses and even bacteria acting as infective agents. There is uncertainty regarding the efficacy and safety of interventions for preventing the common cold in healthy people. This is an update of a Cochrane review first published in 2011 and previously updated in 2013.

OBJECTIVES

To assess the clinical effectiveness and safety of vaccines for preventing the common cold in healthy people.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) (September 2016), MEDLINE (1948 to September 2016), Embase (1974 to September 2016), CINAHL (1981 to September 2016), and LILACS (1982 to September 2016). We also searched three trials registers for ongoing studies and four websites for additional trials (February 2017). We included no language or date restrictions.

SELECTION CRITERIA

Randomised controlled trials (RCTs) of any virus vaccines compared with placebo to prevent the common cold in healthy people.

DATA COLLECTION AND ANALYSIS

Two review authors independently evaluated methodological quality and extracted trial data. We resolved disagreements by discussion or by consulting a third review author.

MAIN RESULTS

We found no additional RCTs for inclusion in this update. This review includes one RCT dating from the 1960s with an overall high risk of bias. The RCT included 2307 healthy participants, all of whom were included in analyses. This trial compared the effect of an adenovirus vaccine against placebo. No statistically significant difference in common cold incidence was found: there were 13 (1.14%) events in 1139 participants in the vaccines group and 14 (1.19%) events in 1168 participants in the placebo group (risk ratio 0.95, 95% confidence interval 0.45 to 2.02; P = 0.90). No adverse events related to the live vaccine were reported. The quality of the evidence was low due to limitations in methodological quality and a wide 95% confidence interval.

AUTHORS' CONCLUSIONS

This Cochrane Review was based on one study with low-quality evidence. We found no conclusive results to support the use of vaccines for preventing the common cold in healthy people compared with placebo. We identified a need for well-designed, adequately powered RCTs to investigate vaccines for the common cold in healthy people. Any future trials on medical treatments for preventing the common cold should assess a variety of virus vaccines for this condition. Outcome measures should include common cold incidence, vaccine safety, and mortality related to the vaccine.

Meta-evaluation of meta-analysis: ten appraisal questions for biologists

Meta-analysis is a statistical procedure for analyzing the combined data from different studies, and can be a major source of concise up-to-date information. The overall conclusions of a meta-analysis, however, depend heavily on the quality of the meta-analytic process, and an appropriate evaluation of the quality of meta-analysis (meta-evaluation) can be challenging. We outline ten questions biologists can ask to critically appraise a meta-analysis. These questions could also act as simple and accessible guidelines for the authors of meta-analyses. We focus on meta-analyses using non-human species, which we term 'biological' meta-analysis. Our ten questions are aimed at enabling a biologist to evaluate whether a biological meta-analysis embodies 'mega-enlightenment', a 'mega-mistake', or something in between.