Prognostic effect size of cardiovascular biomarkers in datasets from observational studies versus randomised trials: meta-epidemiology study

Rapid review: A review of methods and recommendations based on current evidence

Rapid review (RR) could accelerate the traditional systematic review (SR) process by simplifying or omitting steps using various shortcuts. With the increasing popularity of RR, numerous shortcuts had emerged, but there was no consensus on how to choose the most appropriate ones. This study conducted a literature search in PubMed from inception to December 21, 2023, using terms such as "rapid review" "rapid assessment" "rapid systematic review" and "rapid evaluation". We also scanned the reference lists and performed citation tracking of included impact studies to obtain more included studies. We conducted a narrative synthesis of all RR approaches, shortcuts and studies assessing their effectiveness at each stage of RRs. Based on the current evidence, we provided recommendations on utilizing certain shortcuts in RRs. Ultimately, we identified 185 studies focusing on summarizing RR approaches and shortcuts, or evaluating their impact. There was relatively sufficient evidence to support the use of the following shortcuts in RRs: limiting studies to those published in English-language; conducting abbreviated database searches (e.g., only searching PubMed/MEDLINE, Embase, and CENTRAL); omitting retrieval of grey literature; restricting the search timeframe to the recent 20 years for medical intervention and the recent 15 years for reviewing diagnostic test accuracy; conducting a single screening by an experienced screener. To some extent, the above shortcuts were also applicable to SRs. This study provided a reference for future RR researchers in selecting shortcuts, and it also presented a potential research topic for methodologists.

Tutorial: dos and don'ts in clinical prediction research for venous thromboembolism

Clinical prediction modeling has become an increasingly popular domain of venous thromboembolism research in recent years. Prediction models can help healthcare providers make decisions regarding starting or withholding therapeutic interventions, or referrals for further diagnostic workup, and can form a basis for risk stratification in clinical trials. The aim of the current guide is to assist in the practical application of complicated methodological requirements for well-performed prediction research by presenting key dos and don'ts while expanding the understanding of predictive research in general for (clinical) researchers who are not specifically trained in the topic; throughout we will use prognostic venous thromboembolism scores as an exemplar.

Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials: a meta-epidemiological study

BACKGROUND

Researchers and decision-makers often use evidence from randomised controlled trials (RCTs) to determine the efficacy or effectiveness of a treatment or intervention. Studies with observational designs are often used to measure the effectiveness of an intervention in 'real world' scenarios. Numerous study designs and their modifications (including both randomised and observational designs) are used for comparative effectiveness research in an attempt to give an unbiased estimate of whether one treatment is more effective or safer than another for a particular population. An up-to-date systematic analysis is needed to identify differences in effect estimates from RCTs and observational studies. This updated review summarises the results of methodological reviews that compared the effect estimates of observational studies with RCTs from evidence syntheses that addressed the same health research question.

OBJECTIVES

To assess and compare synthesised effect estimates by study type, contrasting RCTs with observational studies. To explore factors that might explain differences in synthesised effect estimates from RCTs versus observational studies (e.g. heterogeneity, type of observational study design, type of intervention, and use of propensity score adjustment). To identify gaps in the existing research comparing effect estimates across different study types.

SEARCH METHODS

We searched MEDLINE, the Cochrane Database of Systematic Reviews, Web of Science databases, and Epistemonikos to May 2022. We checked references, conducted citation searches, and contacted review authors to identify additional reviews.

SELECTION CRITERIA

We included systematic methodological reviews that compared quantitative effect estimates measuring the efficacy or effectiveness of interventions tested in RCTs versus in observational studies. The included reviews compared RCTs to observational studies (including retrospective and prospective cohort, case-control and cross-sectional designs). Reviews were not eligible if they compared RCTs with studies that had used some form of concurrent allocation.

DATA COLLECTION AND ANALYSIS

Using results from observational studies as the reference group, we examined the relative summary effect estimates (risk ratios (RRs), odds ratios (ORs), hazard ratios (HRs), mean differences (MDs), and standardised mean differences (SMDs)) to evaluate whether there was a relatively larger or smaller effect in the ratio of odds ratios (ROR) or ratio of risk ratios (RRR), ratio of hazard ratios (RHR), and difference in (standardised) mean differences (D(S)MD). If an included review did not provide an estimate comparing results from RCTs with observational studies, we generated one by pooling the estimates for observational studies and RCTs, respectively. Across all reviews, we synthesised these ratios to produce a pooled ratio of ratios comparing effect estimates from RCTs with those from observational studies. In overviews of reviews, we estimated the ROR or RRR for each overview using observational studies as the reference category. We appraised the risk of bias in the included reviews (using nine criteria in total). To receive an overall low risk of bias rating, an included review needed: explicit criteria for study selection, a complete sample of studies, and to have controlled for study methodological differences and study heterogeneity. We assessed reviews/overviews not meeting these four criteria as having an overall high risk of bias. We assessed the certainty of the evidence, consisting of multiple evidence syntheses, with the GRADE approach.

MAIN RESULTS

We included 39 systematic reviews and eight overviews of reviews, for a total of 47. Thirty-four of these contributed data to our primary analysis. Based on the available data, we found that the reviews/overviews included 2869 RCTs involving 3,882,115 participants, and 3924 observational studies with 19,499,970 participants. We rated 11 reviews/overviews as having an overall low risk of bias, and 36 as having an unclear or high risk of bias. Our main concerns with the included reviews/overviews were that some did not assess the quality of their included studies, and some failed to account appropriately for differences between study designs - for example, they conducted aggregate analyses of all observational studies rather than separate analyses of cohort and case-control studies. When pooling RORs and RRRs, the ratio of ratios indicated no difference or a very small difference between the effect estimates from RCTs versus from observational studies (ratio of ratios 1.08, 95% confidence interval (CI) 1.01 to 1.15). We rated the certainty of the evidence as low. Twenty-three of 34 reviews reported effect estimates of RCTs and observational studies that were on average in agreement. In a number of subgroup analyses, small differences in the effect estimates were detected: - pharmaceutical interventions only (ratio of ratios 1.12, 95% CI 1.04 to 1.21); - RCTs and observational studies with substantial or high heterogeneity; that is, I2 ≥ 50% (ratio of ratios 1.11, 95% CI 1.04 to 1.18); - no use (ratio of ratios 1.07, 95% CI 1.03 to 1.11) or unclear use (ratio of ratios 1.13, 95% CI 1.03 to 1.25) of propensity score adjustment in observational studies; and - observational studies without further specification of the study design (ratio of ratios 1.06, 95% CI 0.96 to 1.18). We detected no clear difference in other subgroup analyses.

AUTHORS' CONCLUSIONS

We found no difference or a very small difference between effect estimates from RCTs and observational studies. These findings are largely consistent with findings from recently published research. Factors other than study design need to be considered when exploring reasons for a lack of agreement between results of RCTs and observational studies, such as differences in the population, intervention, comparator, and outcomes investigated in the respective studies. Our results underscore that it is important for review authors to consider not only study design, but the level of heterogeneity in meta-analyses of RCTs or observational studies. A better understanding is needed of how these factors might yield estimates reflective of true effectiveness.

Interval estimation for operating characteristic of continuous biomarkers with controlled sensitivity or specificity

The receiver operating characteristic (ROC) curve provides a comprehensive performance assessment of a continuous biomarker over the full threshold spectrum. Nevertheless, a medical test often dictates to operate at a certain high level of sensitivity or specificity. A diagnostic accuracy metric directly targeting the clinical utility is specificity at the controlled sensitivity level, or vice versa. While the empirical point estimation is readily adopted in practice, the nonparametric interval estimation is challenged by the fact that the variance involves density functions due to estimated threshold. In addition, even with a fixed threshold, many standard confidence intervals including the Wald interval for binomial proportion could have erratic behaviors. In this article, we are motivated by the superior performance of the score interval for binomial proportion and propose a novel extension for the biomarker problem. Meanwhile, we develop exact bootstrap and establish consistency of the bootstrap variance estimator. Both single-biomarker evaluation and two-biomarker comparison are investigated. Extensive simulation studies were conducted, demonstrating competitive performance of our proposals. An illustration with aggressive prostate cancer diagnosis is provided.

Legume Intake Is Associated with Potential Savings in Coronary Heart Disease-Related Health Care Costs in Australia

Legume intake has been associated with lower risk for a number of chronic disorders of high financial burden, and is advocated by dietary guidelines as an important part of healthy dietary patterns. Still, the intake of legumes generally falls short of the recommended levels in most countries around the world despite their role as an alternative protein source. The aim of this study was to assess the potential savings in costs of health care services that would follow the reduction in incidences of coronary heart disease (CHD) when adult consumers achieve a targeted level of 50 g/day of legumes intake in Australia. A cost-of-illness analysis was developed using estimates of current and targeted legumes intake in adults (age 25+ y), the estimated percent reduction in relative risk (95% CI) of CHD following legumes intake, and recent data on health care costs related to CHD in Australia. A sensitivity analysis of ‘very pessimistic’ through to ‘universal’ scenarios suggested savings in CHD-related health care costs equal to AUD 4.3 (95% CI 1.2–7.4) to AUD 85.5 (95% CI 23.3–147.7) million annually. Findings of the study suggest an economic value of incorporating attainable levels of legumes within the dietary behaviors of Australians. Greater prominence of legumes in dietary guidelines could assist with achieving broader sustainability measures in relation to diet, helping to bring together the environment and health as an important pillar in relation to sustainability.

Trial-level characteristics associate with treatment effect estimates: a systematic review of meta-epidemiological studies

BACKGROUND

To summarize the up-to-date empirical evidence on trial-level characteristics of randomized controlled trials associated with treatment effect estimates.

METHODS

A systematic review searched three databases up to August 2020. Meta-epidemiological (ME) studies of randomized controlled trials on intervention effect were eligible. We assessed the methodological quality of ME studies using a self-developed criterion. Associations between treatment effect estimates and trial-level characteristics were presented using forest plots.

RESULTS

Eighty ME studies were included, with 25/80 (31%) being published after 2015. Less than one-third ME studies critically appraised the included studies (26/80, 33%), published a protocol (23/80, 29%), and provided a list of excluded studies with justifications (12/80, 15%). Trials with high or unclear (versus low) risk of bias on sequence generation (3/14 for binary outcome and 1/6 for continuous outcome), allocation concealment (11/18 and 1/6), double blinding (5/15 and 2/4) and smaller sample size (4/5 and 2/2) significantly associated with larger treatment effect estimates. Associations between high or unclear risk of bias on allocation concealment (5/6 for binary outcome and 1/3 for continuous outcome), double blinding (4/5 and 1/3) and larger treatment effect estimates were more frequently observed for subjective outcomes. The associations between treatment effect estimates and non-blinding of outcome assessors were removed in trials using multiple observers to reach consensus for both binary and continuous outcomes. Some trial characteristics in the Cochrane risk-of-bias (RoB2) tool have not been covered by the included ME studies, including using validated method for outcome measures and selection of the reported results from multiple outcome measures or multiple analysis based on results (e.g., significance of the results).

CONCLUSIONS

Consistently significant associations between larger treatment effect estimates and high or unclear risk of bias on sequence generation, allocation concealment, double blinding and smaller sample size were found. The impact of allocation concealment and double blinding were more consistent for subjective outcomes. The methodological and reporting quality of included ME studies were dissatisfactory. Future ME studies should follow the corresponding reporting guideline. Specific guidelines for conducting and critically appraising ME studies are needed.

Discovery of primary prostate cancer biomarkers using cross cancer learning

Prostate cancer (PCa), the second leading cause of cancer death in American men, is a relatively slow-growing malignancy with multiple early treatment options. Yet, a significant number of low-risk PCa patients are over-diagnosed and over-treated with significant and long-term quality of life effects. Further, there is ever increasing evidence of metastasis and higher mortality when hormone-sensitive or castration-resistant PCa tumors are treated indistinctively. Hence, the critical need is to discover clinically-relevant and actionable PCa biomarkers by better understanding the biology of PCa. In this paper, we have discovered novel biomarkers of PCa tumors through cross-cancer learning by leveraging the pathological and molecular similarities in the DNA repair pathways of ovarian, prostate, and breast cancer tumors. Cross-cancer disease learning enriches the study population and identifies genetic/phenotypic commonalities that are important across diseases with pathological and molecular similarities. Our results show that ADIRF, SLC2A5, C3orf86, HSPA1B are among the most significant PCa biomarkers, while MTRNR2L1, EEPD1, TEPP and VN1R2 are jointly important biomarkers across prostate, breast and ovarian cancers. Our validation results have further shown that the discovered biomarkers can predict the disease state better than any randomly selected subset of differentially expressed prostate cancer genes.

Systematic differences in effect estimates between observational studies and randomized control trials in meta-analyses in nephrology

The limited availability of randomized controlled trials (RCTs) in nephrology undermines causal inferences in meta-analyses. Systematic reviews of observational studies have grown more common under such circumstances. We conducted systematic reviews of all comparative observational studies in nephrology from 2006 to 2016 to assess the trends in the past decade. We then focused on the meta-analyses combining observational studies and RCTs to evaluate the systematic differences in effect estimates between study designs using two statistical methods: by estimating the ratio of odds ratios (ROR) of the pooled OR obtained from observational studies versus those from RCTs and by examining the discrepancies in their statistical significance. The number of systematic reviews of observational studies in nephrology had grown by 11.7-fold in the past decade. Among 56 records combining observational studies and RCTs, ROR suggested that the estimates between study designs agreed well (ROR 1.05, 95% confidence interval 0.90-1.23). However, almost half of the reviews led to discrepant interpretations in terms of statistical significance. In conclusion, the findings based on ROR might encourage researchers to justify the inclusion of observational studies in meta-analyses. However, caution is needed, as the interpretations based on statistical significance were less concordant than those based on ROR.

Global assessment of C-reactive protein and health-related outcomes: an umbrella review of evidence from observational studies and Mendelian randomization studies

C-reactive protein (CRP) has been studied extensively for association with a large number of non-infectious diseases and outcomes. We aimed to evaluate the breadth and validity of associations between CRP and non-infectious, chronic health outcomes and biomarkers. We conducted an umbrella review of systematic reviews and meta-analyses and a systematic review of Mendelian randomization (MR) studies. PubMed, Scopus, and Cochrane Database of Systematic Reviews were systematically searched from inception up to March 2019. Meta-analyses of observational studies and MR studies examining associations between CRP and health outcomes were identified, excluding studies on the diagnostic value of CRP for infections. We found 113 meta-analytic comparisons of observational studies and 196 MR analyses, covering a wide range of outcomes. The overwhelming majority of the meta-analyses of observational studies reported a nominally statistically significant result (95/113, 84.1%); however, the majority of the meta-analyses displayed substantial heterogeneity (47.8%), small study effects (39.8%) or excess significance (41.6%). Only two outcomes, cardiovascular mortality and venous thromboembolism, showed convincing evidence of association with CRP levels. When examining the MR literature, we found MR studies for 53/113 outcomes examined in the observational study meta-analyses but substantial support for a causal association with CRP was not observed for any phenotype. Despite the striking amount of research on CRP, convincing evidence for associations and causal effects is remarkably limited.

A Critical Evaluation of Wet Biomarkers for Huntington's Disease: Current Status and Ways Forward

There is an unmet clinical need for objective biomarkers to monitor disease progression and treatment response in Huntington's disease (HD). The aim of this review is, therefore, to provide practical advice for biomarker discovery and to summarise studies on biofluid markers for HD. A PubMed search was performed to review literature with regard to candidate saliva, urine, blood and cerebrospinal fluid biomarkers for HD. Information has been organised into tables to allow a pragmatic approach to the discussion of the evidence and generation of practical recommendations for future studies. Many of the markers published converge on metabolic and inflammatory pathways, although changes in other analytes representing antioxidant and growth factor pathways have also been found. The most promising markers reflect neuronal and glial degeneration, particularly neurofilament light chain. International collaboration to standardise assays and study protocols, as well as to recruit sufficiently large cohorts, will facilitate future biomarker discovery and development.

D-Dimer Predicts Long-Term Cause-Specific Mortality, Cardiovascular Events, and Cancer in Patients With Stable Coronary Heart Disease: LIPID Study

BACKGROUND

D-dimer, a degradation product of cross-linked fibrin, is a marker for hypercoagulability and thrombotic events. Moderately elevated levels of D-dimer are associated with the risk of venous and arterial events in patients with vascular disease. We assessed the role of D-dimer levels in predicting long-term vascular outcomes, cause-specific mortality, and new cancers in the LIPID trial (Long-Term Intervention with Pravastatin in Ischaemic Disease) in the context of other risk factors.

METHODS

LIPID randomized patients to placebo or pravastatin 40 mg/d 5 to 38 months after myocardial infarction or unstable angina. D-dimer levels were measured at baseline and at 1 year. Median follow-up was 6.0 years during the trial and 16 years in total.

RESULTS

Baseline D-dimer levels for 7863 patients were grouped by quartile (≤112, 112-173, 173-273, >273 ng/mL). Higher levels were associated with older age, female sex, history of hypertension, poor renal function, and elevated levels of B-natriuretic peptide, high-sensitivity C-reactive protein, and sensitive troponin I (each P<0.001). During the first 6 years, after adjustment for up to 30 additional risk factors, higher D-dimer was associated with a significantly increased risk of a major coronary event (quartile 4 versus 1: hazard ratio [HR], 1.45; 95% confidence interval, 1.21-1.74), major cardiovascular disease (CVD) event (HR, 1.45; 95% confidence interval, 1.23-1.71) and venous thromboembolism (HR, 4.03; 95% confidence interval, 2.31-7.03; each P<0.001). During the 16 years overall, higher D-dimer was an independent predictor of all-cause mortality (HR, 1.59), CVD mortality (HR, 1.61), cancer mortality (HR, 1.54), and non-CVD noncancer mortality (HR, 1.57; each P<0.001), remaining significant for deaths resulting from each cause occurring beyond 10 years of follow-up (each P≤0.01). Higher D-dimer also independently predicted an increase in cancer incidence (HR, 1.16; P=0.02).The D-dimer level increased the net reclassification index for all-cause mortality by 4.0 and venous thromboembolism by 13.6.

CONCLUSIONS

D-dimer levels predict long-term risk of arterial and venous events, CVD mortality, and non-CVD noncancer mortality independent of other risk factors. D-dimer is also a significant predictor of cancer incidence and mortality. These results support an association of D-dimer with fatal events across multiple diseases and demonstrate that this link extends beyond 10 years' follow-up.

Environmental Risk Factors for Inflammatory Bowel Diseases: An Umbrella Review of Meta-analyses

BACKGROUND & AIMS

Multiple environmental factors have been associated with the development of inflammatory bowel diseases (IBDs). We performed an umbrella review of meta-analyses to summarize available epidemiologic evidence and assess its credibility.

METHODS

We systematically identified and appraised meta-analyses of observational studies examining environmental factors and risk of IBD (Crohn's disease [CD] or ulcerative colitis [UC]). For each meta-analysis, we considered the random effects estimate, its 95% confidence interval, the estimates of heterogeneity, and small-study effects, and we graded the evidence according to prespecified criteria. Methodologic quality was assessed with AMSTAR (ie, A Measurement Tool to Assess Systematic Reviews) 2.

RESULTS

We examined 183 estimates in 53 meta-analyses of 71 environmental factors related to lifestyles and hygiene, surgeries, drug exposures, diet, microorganisms, and vaccinations. We identified 9 factors that increase risk of IBD: smoking (CD), urban living (CD and IBD), appendectomy (CD), tonsillectomy (CD), antibiotic exposure (IBD), oral contraceptive use (IBD), consumption of soft drinks (UC), vitamin D deficiency (IBD), and non-Helicobacter pylori-like enterohepatic Helicobacter species (IBD). We identified 7 factors that reduce risk of IBD: physical activity (CD), breastfeeding (IBD), bed sharing (CD), tea consumption (UC), high levels of folate (IBD), high levels of vitamin D (CD), and H pylori infection (CD, UC, and IBD). Epidemiologic evidence for all of these associations was of high to moderate strength; we identified another 11 factors associated with increased risk and 16 factors associated with reduced risk with weak credibility. Methodologic quality varied considerably among meta-analyses. Several associations were based on findings from retrospective studies, so it is not possible to determine if these are effects of IBD or the results of recall bias.

CONCLUSIONS

In an umbrella review of meta-analyses, we found varying levels of evidence for associations of different environmental factors with risk of IBD. High-quality prospective studies with analyses of samples from patients with recent diagnoses of IBD are needed to determine whether these factors cause or are results of IBD and their pathogenic mechanisms.

Unbiased data analytic strategies to improve biomarker discovery in precision medicine

Omics technologies promised improved biomarker discovery for precision medicine. The foremost problem of discovered biomarkers is irreproducibility between patient cohorts. From a data analytics perspective, the main reason for these failures is bias in statistical approaches and overfitting resulting from batch effects and confounding factors. The keys to reproducible biomarker discovery are: proper study design, unbiased data preprocessing and quality control analyses, and a knowledgeable application of statistics and machine learning algorithms. In this review, we discuss study design and analysis considerations and suggest standards from an expert point-of-view to promote unbiased decision-making in biomarker discovery in precision medicine.

Genetic and non-genetic risk factors for pre-eclampsia: umbrella review of systematic reviews and meta-analyses of observational studies

OBJECTIVE

To summarize evidence from the literature on genetic and non-genetic risk factors associated with pre-eclampsia (PE), assess the presence of statistical bias in the studies and identify risk factors for which there is robust evidence supporting their association with PE.

METHODS

PubMed and ISI Web of Science were searched from inception to October 2016, to identify systematic reviews and meta-analyses of observational studies examining associations between genetic or non-genetic risk factors and PE. For each meta-analysis, the summary-effect size was estimated using random-effects and fixed-effects models, along with 95% CIs and the 95% prediction interval. Between-study heterogeneity was expressed using the I² statistic, and evidence of small-study effects (large studies had significantly more conservative results than smaller studies) and evidence of excess significance bias (too many studies with statistically significant results) were estimated.

RESULTS

Fifty-eight eligible meta-analyses were identified, which included 1466 primary studies and provided data on 130 comparisons of risk factors associated with PE, covering a wide range of comorbid diseases, genetic factors, exposure to environmental agents and biomarkers. Sixty-five (50%) associations had nominally statistically significant findings at P < 0.05, while 16 (12%) were significant at P < 10^-6 . Sixty-five (50%) associations had large or very large heterogeneity. Evidence for small-study effects and excess significance bias was found in 10 (8%) and 26 (20%) associations, respectively. The only non-genetic risk factor with convincing evidence for an association with PE was oocyte donation vs spontaneous conception, which had a summary odds ratio of 4.33 (95% CI, 3.11-6.03), was supported by 2712 cases with small heterogeneity (I²  = 26%) and 95% prediction intervals excluding the null value, and without hints of small-study effects (P for Egger's test > 0.10) or excess of significance (P > 0.05). Of the statistically significant (P < 0.05) genetic risk factors for PE, only PAI-1 4G/5G (recessive model) polymorphism was supported by strong evidence for a contribution to the pathogenesis of PE. Eleven factors (serum iron level, pregnancy-associated plasma protein-A, chronic kidney disease, polycystic ovary syndrome, mental stress, bacterial and viral infections, cigarette smoking, oocyte donation vs assisted reproductive technology, obesity vs normal weight, severe obesity vs normal weight and primiparity) presented highly suggestive evidence for an association with PE.

CONCLUSIONS

A large proportion of meta-analyses of genetic and non-genetic risk factors for PE have caveats that threaten their validity. Oocyte donation vs spontaneous conception and PAI-1 4G/5G polymorphism (recessive model) showed the strongest consistent evidence for an association with risk for PE. Copyright © 2017 ISUOG. Published by John Wiley & Sons Ltd.

Methodological Rigor in Preclinical Cardiovascular Studies: Targets to Enhance Reproducibility and Promote Research Translation

RATIONALE

Methodological sources of bias and suboptimal reporting contribute to irreproducibility in preclinical science and may negatively affect research translation. Randomization, blinding, sample size estimation, and considering sex as a biological variable are deemed crucial study design elements to maximize the quality and predictive value of preclinical experiments.

OBJECTIVE

To examine the prevalence and temporal patterns of recommended study design element implementation in preclinical cardiovascular research.

METHODS AND RESULTS

All articles published over a 10-year period in 5 leading cardiovascular journals were reviewed. Reports of in vivo experiments in nonhuman mammals describing pathophysiology, genetics, or therapeutic interventions relevant to specific cardiovascular disorders were identified. Data on study design and animal model use were collected. Citations at 60 months were additionally examined as a surrogate measure of research impact in a prespecified subset of studies, stratified by individual and cumulative study design elements. Of 28 636 articles screened, 3396 met inclusion criteria. Randomization was reported in 21.8%, blinding in 32.7%, and sample size estimation in 2.3%. Temporal and disease-specific analyses show that the implementation of these study design elements has overall not appreciably increased over the past decade, except in preclinical stroke research, which has uniquely demonstrated significant improvements in methodological rigor. In a subset of 1681 preclinical studies, randomization, blinding, sample size estimation, and inclusion of both sexes were not associated with increased citations at 60 months.

CONCLUSIONS

Methodological shortcomings are prevalent in preclinical cardiovascular research, have not substantially improved over the past 10 years, and may be overlooked when basing subsequent studies. Resultant risks of bias and threats to study validity have the potential to hinder progress in cardiovascular medicine as preclinical research often precedes and informs clinical trials. Stroke research quality has uniquely improved in recent years, warranting a closer examination for interventions to model in other cardiovascular fields.

Decision support systems for personalized and participative radiation oncology

A paradigm shift from current population based medicine to personalized and participative medicine is underway. This transition is being supported by the development of clinical decision support systems based on prediction models of treatment outcome. In radiation oncology, these models 'learn' using advanced and innovative information technologies (ideally in a distributed fashion - please watch the animation: http://youtu.be/ZDJFOxpwqEA) from all available/appropriate medical data (clinical, treatment, imaging, biological/genetic, etc.) to achieve the highest possible accuracy with respect to prediction of tumor response and normal tissue toxicity. In this position paper, we deliver an overview of the factors that are associated with outcome in radiation oncology and discuss the methodology behind the development of accurate prediction models, which is a multi-faceted process. Subsequent to initial development/validation and clinical introduction, decision support systems should be constantly re-evaluated (through quality assurance procedures) in different patient datasets in order to refine and re-optimize the models, ensuring the continuous utility of the models. In the reasonably near future, decision support systems will be fully integrated within the clinic, with data and knowledge being shared in a standardized, dynamic, and potentially global manner enabling truly personalized and participative medicine.

Neutrophil-to-lymphocyte ratio as a bladder cancer biomarker: Assessing prognostic and predictive value in SWOG 8710

BACKGROUND

Risk stratification is a major challenge in bladder cancer (BC), and a biomarker is needed. Multiple studies have reported the neutrophil-to-lymphocyte ratio (NLR) as a promising candidate; however, these analyses have methodological limitations. Therefore, the authors performed a category B biomarker study to test whether NLR is prognostic for overall survival (OS) after curative treatment or is predictive for the survival benefit from neoadjuvant chemotherapy (NAC).

METHODS

This study is an unplanned secondary analysis of SWOG 8710, a randomized phase 3 trial that assessed cystectomy with or without NAC in 317 patients with muscle-invasive BC. NLR was calculated from prospectively collected complete blood counts. For the prognostic analysis, 230 patients were identified; for the predictive analysis, 263 were identified. NLR was evaluated with proportional hazards models including prespecified factors (age, sex, T-stage, lymphovascular invasion, and treatment arm).

RESULTS

With a median follow-up of 18.6 years, there were 172 and 205 deaths in the prognostic and predictive cohorts, respectively. In a multivariable analysis, NLR was not prognostic for OS (hazard ratio [HR], 1.04; 95% confidence interval [CI], 0.98-1.11; P = .24). Furthermore, NLR did not predict for the OS benefit from NAC (HR, 1.01; 95% CI, 0.90-1.14; P = .86). Factors associated with worse OS were older age (HR, 1.05; 95% CI, 1.04-1.07; P < .001) and surgery without NAC (HR, 1.39; 95% CI, 1.03-1.88; P = .03).

CONCLUSIONS

This is the first analysis of NLR in BC to use prospectively collected clinical trial data. In contrast to previous studies, it suggests that NLR is neither a prognostic nor predictive biomarker for OS in muscle-invasive BC. Cancer 2017;123:794-801. © 2016 American Cancer Society.

Systematic characterization of gastrointestinal clinical trials

BACKGROUND

Clinical guidelines are commonly based on inadequate evidence, suggesting deficiencies in the present portfolio of clinical research.

AIMS

To investigate characteristics of clinical trials examining gastrointestinal (GI) diseases registered in ClinicalTrials.gov.

METHODS

A cross-sectional analysis of 13,647 GI trials and 111,535 non-GI trials initiated between January 1997 and September 2013 was performed. Entries were sorted by operational status, purpose, interventions, trial design, and epochs to identify trends and interactions in trial properties.

RESULTS

The global production of GI trials has remained static in recent years and a majority of research efforts are focused on a few diseases. While GI trials are generally produced by highly populated US states and countries, they are also seldom larger than 500 patients. The likelihood of using data monitoring committees, randomization, and double blinding in GI trials has increased over time, though a substantial fraction of GI trials still do not employ rigorous trial designs. While levels of GI trials correlate with disease burden, the explained variance of GI trials by disease burden worldwide is poor.

CONCLUSION

GI trials are chiefly concentrated in few diseases and highly populated regions, exhibit heterogeneous trends and methodologies, and are sensitive to disease burdens, though more so within North America than worldwide.

Diverse criteria and methods are used to compare treatment effect estimates: a scoping review

OBJECTIVES

To determine what criteria researchers use to assess whether the estimates of effect of an intervention on a dichotomous outcome are different when obtained using different study designs.

STUDY DESIGN AND SETTING

Scoping review of the literature. We included studies of dichotomous outcomes in which authors compared the estimates of effects from different study designs. We performed searches in electronic databases and in the list of references of relevant studies. Two reviewers independently selected studies and abstracted data. We created a list of the criteria used to compare estimates of effects between study designs, described their main features, and classified them using a clinical perspective.

RESULTS

We included 26 studies, from which we identified 24 criteria. Most of the studies focused on comparing estimates from observational studies and randomized controlled trials (n = 19). The most common criteria aimed to determine whether there was a difference or not (n = 18), provided guidance for such a judgment (n = 16), and were based on the point estimates (n = 11). We judged 14 criteria to be appropriate and classified them as either statistically related or clinically related.

CONCLUSION

We found that diverse criteria are used to compare effect estimates between study designs. Familiarity with these would aid in the interpretation of results from different studies regarding the same question.

Cancer biomarker discovery and validation

With the emergence of genomic profiling technologies and selective molecular targeted therapies, biomarkers play an increasingly important role in the clinical management of cancer patients. Single gene/protein or multi-gene "signature"-based assays have been introduced to measure specific molecular pathway deregulations that guide therapeutic decision-making as predictive biomarkers. Genome-based prognostic biomarkers are also available for several cancer types for potential incorporation into clinical prognostic staging systems or practice guidelines. However, there is still a large gap between initial biomarker discovery studies and their clinical translation due to the challenges in the process of cancer biomarker development. In this review we summarize the steps of biomarker development, highlight key issues in successful validation and implementation, and overview representative examples in the oncology field. We also discuss regulatory issues and future perspectives in the era of big data analysis and precision medicine.

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration

The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) Statement includes a 22-item checklist, which aims to improve the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. This explanation and elaboration document describes the rationale; clarifies the meaning of each item; and discusses why transparent reporting is important, with a view to assessing risk of bias and clinical usefulness of the prediction model. Each checklist item of the TRIPOD Statement is explained in detail and accompanied by published examples of good reporting. The document also provides a valuable reference of issues to consider when designing, conducting, and analyzing prediction model studies. To aid the editorial process and help peer reviewers and, ultimately, readers and systematic reviewers of prediction model studies, it is recommended that authors include a completed checklist in their submission. The TRIPOD checklist can also be downloaded from www.tripod-statement.org.

Influence of food patterns on endothelial biomarkers: a systematic review

The purpose of this study was to conduct a systematic review on the association of food patterns (FPs) and endothelial biomarkers. An electronic literature search from 1990 to 2012 was conducted and reference lists and experts were consulted. Studies without dietary intervention and without language restrictions were considered. Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were employed. Methodological quality was assessed by Strengthening the Reporting of Observational Studies in Epidemiology guidelines. A total of 546 references were identified, of which 8 were finally included. Several FPs were identified. Healthy FPs (abundant in fruits and vegetables) had a beneficial impact on endothelial function as estimated by circulating levels of biomarkers such as C-reactive protein, soluble intercellular adhesion molecule 1, soluble vascular adhesion molecule 1, and E-selectin molecules. Westernized patterns (higher intakes of processed meats, sweets, fried foods, and refined grains) were positively associated with inflammation molecules and atherogenic promoters. The study of FPs in relation to endothelial function contributes to the development of dietary recommendations for improved cardiovascular health and therefore a better lifestyle.

Meta-epidemiology

The concept of meta-epidemiology has been introduced with considering the methodological limitations of systematic review for intervention trials. The paradigm of meta-epidemiology has shifted from a statistical method into a new methodology to close gaps between evidence and practice. Main interest of meta-epidemiology is to control potential biases in previous quantitative systematic reviews and draw appropriate evidences for establishing evidence-base guidelines. Nowadays, the network meta-epidemiology was suggested in order to overcome some limitations of meta-epidemiology. To activate meta-epidemiologic studies, implementation of tools for risk of bias and reporting guidelines such as the Consolidated Standards for Reporting Trials (CONSORT) should be done.

Glycated hemoglobin as a marker of subclinical atherosclerosis and cardiac remodeling among non-diabetic adults from the general population

BACKGROUND

Elevated glycated hemoglobin (HbA1c) is associated with increased risk of cardiovascular disease (CVD) and mortality but little is known about potential mechanisms underlying the reported associations.

METHODS

We used data from 1798 non-diabetic participants from the population-based cohort Study of Health in Pomerania (SHIP) to investigate cross-sectional and longitudinal associations of HbA1c with subclinical atherosclerosis (common carotid artery intima-media thickness [CCA-IMT]), cardiac structure (left ventricular mass [LVM]), and cardiac function (fractional shortening).

RESULTS

Cross-sectional analyses revealed a positive association between HbA1c and mean CCA-IMT with a 0.02 mm (95% confidence interval: 0.01-0.04) increase in CCA-IMT per 1% increase in HbA1c, and a similar positive trend across HbA1c quartiles (overall p-value <0.01). We also observed a graded association between HbA1c and high CCA-IMT (>75th percentile) with an odds ratio of 1.42 (95% CI: 1.11-1.81) per 1% increase in HbA1c. Longitudinal analyses showed no consistent associations of baseline HbA1c with mean follow-up CCA-IMT. There were no consistent associations of HbA1c with cardiac remodeling in cross-sectional and longitudinal analyses, respectively.

CONCLUSIONS

The association between HbA1c and CCA-IMT in non-diabetic adults may be a crucial link between high-normal HbA1c levels and an increased risk of CVD and mortality.

The clinical course of low back pain: a meta-analysis comparing outcomes in randomised clinical trials (RCTs) and observational studies

Background

Evidence suggests that the course of low back pain (LBP) symptoms in randomised clinical trials (RCTs) follows a pattern of large improvement regardless of the type of treatment. A similar pattern was independently observed in observational studies. However, there is an assumption that the clinical course of symptoms is particularly influenced in RCTs by mere participation in the trials. To test this assumption, the aim of our study was to compare the course of LBP in RCTs and observational studies.

Methods

Source of studies CENTRAL database for RCTs and MEDLINE, CINAHL, EMBASE and hand search of systematic reviews for cohort studies. Studies include individuals aged 18 or over, and concern non-specific LBP. Trials had to concern primary care treatments. Data were extracted on pain intensity. Meta-regression analysis was used to compare the pooled within-group change in pain in RCTs with that in cohort studies calculated as the standardised mean change (SMC).

Results

70 RCTs and 19 cohort studies were included, out of 1134 and 653 identified respectively. LBP symptoms followed a similar course in RCTs and cohort studies: a rapid improvement in the first 6 weeks followed by a smaller further improvement until 52 weeks. There was no statistically significant difference in pooled SMC between RCTs and cohort studies at any time point:- 6 weeks: RCTs: SMC 1.0 (95% CI 0.9 to 1.0) and cohorts 1.2 (0.7to 1.7); 13 weeks: RCTs 1.2 (1.1 to 1.3) and cohorts 1.0 (0.8 to 1.3); 27 weeks: RCTs 1.1 (1.0 to 1.2) and cohorts 1.2 (0.8 to 1.7); 52 weeks: RCTs 0.9 (0.8 to 1.0) and cohorts 1.1 (0.8 to 1.6).

Conclusions

The clinical course of LBP symptoms followed a pattern that was similar in RCTs and cohort observational studies. In addition to a shared ‘natural history’, enrolment of LBP patients in clinical studies is likely to provoke responses that reflect the nonspecific effects of seeking and receiving care, independent of the study design.

Increasing value and reducing waste in research design, conduct, and analysis

Correctable weaknesses in the design, conduct, and analysis of biomedical and public health research studies can produce misleading results and waste valuable resources. Small effects can be difficult to distinguish from bias introduced by study design and analyses. An absence of detailed written protocols and poor documentation of research is common. Information obtained might not be useful or important, and statistical precision or power is often too low or used in a misleading way. Insufficient consideration might be given to both previous and continuing studies. Arbitrary choice of analyses and an overemphasis on random extremes might affect the reported findings. Several problems relate to the research workforce, including failure to involve experienced statisticians and methodologists, failure to train clinical researchers and laboratory scientists in research methods and design, and the involvement of stakeholders with conflicts of interest. Inadequate emphasis is placed on recording of research decisions and on reproducibility of research. Finally, reward systems incentivise quantity more than quality, and novelty more than reliability. We propose potential solutions for these problems, including improvements in protocols and documentation, consideration of evidence from studies in progress, standardisation of research efforts, optimisation and training of an experienced and non-conflicted scientific workforce, and reconsideration of scientific reward systems.

Using multiple types of studies in systematic reviews of health care interventions--a systematic review

BACKGROUND

A systematic review may evaluate different aspects of a health care intervention. To accommodate the evaluation of various research questions, the inclusion of more than one study design may be necessary. One aim of this study is to find and describe articles on methodological issues concerning the incorporation of multiple types of study designs in systematic reviews on health care interventions. Another aim is to evaluate methods studies that have assessed whether reported effects differ by study types.

METHODS AND FINDINGS

We searched PubMed, the Cochrane Database of Systematic Reviews, and the Cochrane Methodology Register on 31 March 2012 and identified 42 articles that reported on the integration of single or multiple study designs in systematic reviews. We summarized the contents of the articles qualitatively and assessed theoretical and empirical evidence. We found that many examples of reviews incorporating multiple types of studies exist and that every study design can serve a specific purpose. The clinical questions of a systematic review determine the types of design that are necessary or sufficient to provide the best possible answers. In a second independent search, we identified 49 studies, 31 systematic reviews and 18 trials that compared the effect sizes between randomized and nonrandomized controlled trials, which were statistically different in 35%, and not different in 53%. Twelve percent of studies reported both, different and non-different effect sizes.

CONCLUSIONS

Different study designs addressing the same question yielded varying results, with differences in about half of all examples. The risk of presenting uncertain results without knowing for sure the direction and magnitude of the effect holds true for both nonrandomized and randomized controlled trials. The integration of multiple study designs in systematic reviews is required if patients should be informed on the many facets of patient relevant issues of health care interventions.

Aortic pulse wave velocity improves cardiovascular event prediction: an individual participant meta-analysis of prospective observational data from 17,635 subjects

OBJECTIVES

The goal of this study was to determine whether aortic pulse wave velocity (aPWV) improves prediction of cardiovascular disease (CVD) events beyond conventional risk factors.

BACKGROUND

Several studies have shown that aPWV may be a useful risk factor for predicting CVD, but they have been underpowered to examine whether this is true for different subgroups.

METHODS

We undertook a systematic review and obtained individual participant data from 16 studies. Study-specific associations of aPWV with CVD outcomes were determined using Cox proportional hazard models and random effect models to estimate pooled effects.

RESULTS

Of 17,635 participants, a total of 1,785 (10%) had a CVD event. The pooled age- and sex-adjusted hazard ratios (HRs) per 1-SD change in loge aPWV were 1.35 (95% confidence interval [CI]: 1.22 to 1.50; p < 0.001) for coronary heart disease, 1.54 (95% CI: 1.34 to 1.78; p < 0.001) for stroke, and 1.45 (95% CI: 1.30 to 1.61; p < 0.001) for CVD. Associations stratified according to sex, diabetes, and hypertension were similar but decreased with age (1.89, 1.77, 1.36, and 1.23 for age ≤50, 51 to 60, 61 to 70, and >70 years, respectively; pinteraction <0.001). After adjusting for conventional risk factors, aPWV remained a predictor of coronary heart disease (HR: 1.23 [95% CI: 1.11 to 1.35]; p < 0.001), stroke (HR: 1.28 [95% CI: 1.16 to 1.42]; p < 0.001), and CVD events (HR: 1.30 [95% CI: 1.18 to 1.43]; p < 0.001). Reclassification indices showed that the addition of aPWV improved risk prediction (13% for 10-year CVD risk for intermediate risk) for some subgroups.

CONCLUSIONS

Consideration of aPWV improves model fit and reclassifies risk for future CVD events in models that include standard risk factors. aPWV may enable better identification of high-risk populations that might benefit from more aggressive CVD risk factor management.

The expanding role of biomarkers in the assessment of smoking-related parenchymal lung diseases

Recent advances in the field of clinical biomarkers suggest that quantification of serum proteins could play an important role in the diagnosis, classification, prognosis, and treatment response of smoking-related parenchymal lung diseases. COPD and idiopathic pulmonary fibrosis (IPF), two common chronic progressive parenchymal lung diseases, share cigarette smoke exposure as a common dominant risk factor for their development. We have recently shown that COPD and interstitial lung disease may represent distinct outcomes of chronic tobacco use, whereas others have demonstrated that both diseases coexist in some individuals. In this perspective, we examine the potential role of peripheral blood biomarkers in predicting which individuals will develop COPD or IPF, as well as their usefulness in tracking disease progression and exacerbations. Additionally, given the current lack of sensitive and effective metrics to determine an individual's response to treatment, we evaluate the potential role of biomarkers as surrogate markers of clinical outcomes. Finally, we examine the possibility that changes in levels of select protein biomarkers can provide mechanistic insight into the common origins and unique individual susceptibilities that lead to the development of smoking-related parenchymal lung diseases. This discussion is framed by a consideration of the properties of ideal biomarkers for different clinical and research purposes and the best uses for those biomarkers that have already been proposed and investigated.

Why even more clinical research studies may be false: effect of asymmetrical handling of clinically unexpected values

BACKGROUND

In medical practice, clinically unexpected measurements might be quite properly handled by the remeasurement, removal, or reclassification of patients. If these habits are not prevented during clinical research, how much of each is needed to sway an entire study?

METHODS AND RESULTS

Believing there is a difference between groups, a well-intentioned clinician researcher addresses unexpected values. We tested how much removal, remeasurement, or reclassification of patients would be needed in most cases to turn an otherwise-neutral study positive. Remeasurement of 19 patients out of 200 per group was required to make most studies positive. Removal was more powerful: just 9 out of 200 was enough. Reclassification was most powerful, with 5 out of 200 enough. The larger the study, the smaller the proportion of patients needing to be manipulated to make the study positive: the percentages needed to be remeasured, removed, or reclassified fell from 45%, 20%, and 10% respectively for a 20 patient-per-group study, to 4%, 2%, and 1% for an 800 patient-per-group study. Dot-plots, but not bar-charts, make the perhaps-inadvertent manipulations visible. Detection is possible using statistical methods such as the Tadpole test.

CONCLUSIONS

Behaviours necessary for clinical practice are destructive to clinical research. Even small amounts of selective remeasurement, removal, or reclassification can produce false positive results. Size matters: larger studies are proportionately more vulnerable. If observational studies permit selective unblinded enrolment, malleable classification, or selective remeasurement, then results are not credible. Clinical research is very vulnerable to "remeasurement, removal, and reclassification", the 3 evil R's.

Predicting outcomes in radiation oncology--multifactorial decision support systems

With the emergence of individualized medicine and the increasing amount and complexity of available medical data, a growing need exists for the development of clinical decision-support systems based on prediction models of treatment outcome. In radiation oncology, these models combine both predictive and prognostic data factors from clinical, imaging, molecular and other sources to achieve the highest accuracy to predict tumour response and follow-up event rates. In this Review, we provide an overview of the factors that are correlated with outcome-including survival, recurrence patterns and toxicity-in radiation oncology and discuss the methodology behind the development of prediction models, which is a multistage process. Even after initial development and clinical introduction, a truly useful predictive model will be continuously re-evaluated on different patient datasets from different regions to ensure its population-specific strength. In the future, validated decision-support systems will be fully integrated in the clinic, with data and knowledge being shared in a standardized, instant and global manner.