Subgroup analysis as a source of spurious findings: an illustration using new data on alcohol intake and coronary heart disease

Statistical Analysis in the German National Cohort (NAKO) - Specific Aspects and General Recommendations

The German National Cohort (NAKO) is an ongoing, prospective multicenter cohort study, which started recruitment in 2014 and includes more than 205,000 women and men aged 19-74 years. The study data will be available to the global research community for analyses. Although the ultimate decision about the analytic methods will be made by the respective investigator, in this paper we provide the basis for a harmonized approach to the statistical analyses in the NAKO. We discuss specific aspects of the study (e.g., data collection, weighting to account for the sampling design), but also give general recommendations which may apply to other large cohort studies as well.

Standardized Protocol Items Recommendations for Observational Studies (SPIROS) for Observational Study Protocol Reporting Guidelines: Protocol for a Delphi Study

BACKGROUND

Approximately 90% of currently published clinical and public health research is in the form of observational studies. Having a detailed and registered study protocol prior to data collection is important in any empirical study. Without this, there is no reliable way to assess the occurrence of publication bias, outcome reporting bias, and other protocol deviations. However, there is currently no solid guidance available on the information that a protocol for an observational study should contain.

OBJECTIVE

The aim of this study is to formulate the Standardized Protocol Items Recommendations for Observational Studies (SPIROS) reporting guidelines, which focus on 3 main study designs of analytical epidemiology: cohort, case-control, and cross-sectional studies.

METHODS

A scoping review of published protocol papers of observational studies in epidemiology will identify candidate items for the SPIROS reporting guidelines. The list of items will be extended with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist items and recommendations from the SPIROS steering committee. This long list serves as the basis for a 2-round Delphi survey among experts to obtain consensus on which items to include. Each candidate item from the long list will be rated on a 5-point Likert scale to assess relevance for inclusion in the SPIROS reporting guidelines. Following the Delphi survey, an expert-driven consensus workshop will be convened to finalize the reporting guidelines.

RESULTS

A scoping review of published observational study protocols has been completed, with 59 candidate items identified for inclusion into the Delphi survey, itself launched in early 2020.

CONCLUSIONS

This project aims to improve the timeliness, completeness, and clarity of study protocols of observational studies in analytical epidemiology by producing expert-based recommendations of items to be addressed. These reporting guidelines will facilitate and encourage researchers to prepare and register study protocols of sufficient quality prior to data collection in order to improve the transparency, reproducibility, and quality of observational studies.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/17864.

How subgroup analyses can miss the trees for the forest plots: A simulation study

OBJECTIVES

Subgroup analyses of clinical trial data can be an important tool for understanding when treatment effects differ across populations. That said, even effect estimates from prespecified subgroups in well-conducted trials may not apply to corresponding subgroups in the source population. While this divergence may simply reflect statistical imprecision, there has been less discussion of systematic or structural sources of misleading subgroup estimates.

STUDY DESIGN AND SETTING

We use directed acyclic graphs to show how selection bias caused by associations between effect measure modifiers and trial selection, whether explicit (e.g., eligibility criteria) or implicit (e.g., self-selection based on race), can result in subgroup estimates that do not correspond to subgroup effects in the source population. To demonstrate this point, we provide a hypothetical example illustrating the sorts of erroneous conclusions that can result, as well as their potential consequences. We also provide a tool for readers to explore additional cases.

CONCLUSION

Treating subgroups within a trial essentially as random samples of the corresponding subgroups in the wider population can be misleading, even when analyses are conducted rigorously and all findings are internally valid. Researchers should carefully examine associations between (and consider adjusting for) variables when attempting to identify heterogeneous treatment effects.

Duplicate and salami publication: a prevalence study of journal policies

BACKGROUND

Duplicate and salami publication are unethical, but are common practices with substantial consequences for science and society at large. Scientific journals are the 'gatekeepers' of the publication process. We investigated journal policies on duplicate and salami publication.

METHODS

In 2018, we performed a content analysis of policies of journals in the disciplines of 'epidemiology and public health' and 'general and internal medicine'. Journal policies were searched, extracted, coded and cross-checked. The associations of disciplinary categories and journal impact factors with journal policies were examined using Poisson regression models with a robust error variance.

RESULTS

A total of 209 journals, including 122 in epidemiology and public health and 87 in general and internal medicine, were sampled and their policies investigated. Overall, 18% of journals did not have any policies on either practice, 33% only referred to a generic guideline or checklist without explicit mention about either practice, 36% included policies on duplicate publication and only 13% included policies on both duplicate and salami publication. Having explicit journal policies did not differ by journal disciplinary categories (epidemiology and public health vs general and internal medicine) or impact factors. Further analysis of journals with explicit policies found that although duplicate publication is universally discouraged, policies on salami publication are inconsistent and lack specific definitions of inappropriate divisions of papers.

CONCLUSIONS

Gaps exist in journal policies on duplicate and salami publication, characterized by an overall lack of explicit policies, inconsistency and confusion in definitions of bad practices, and lack of clearly defined consequences for non-compliance. Scientific publication and the academic reward systems must evolve to credit good research practice.

Association between clinically recorded alcohol consumption and initial presentation of 12 cardiovascular diseases: population based cohort study using linked health records

Objectives To investigate the association between alcohol consumption and cardiovascular disease at higher resolution by examining the initial lifetime presentation of 12 cardiac, cerebrovascular, abdominal, or peripheral vascular diseases among five categories of consumption.Design Population based cohort study of linked electronic health records covering primary care, hospital admissions, and mortality in 1997-2010 (median follow-up six years).Setting CALIBER (ClinicAl research using LInked Bespoke studies and Electronic health Records).Participants 1 937 360 adults (51% women), aged ≥30 who were free from cardiovascular disease at baseline.Main outcome measures 12 common symptomatic manifestations of cardiovascular disease, including chronic stable angina, unstable angina, acute myocardial infarction, unheralded coronary heart disease death, heart failure, sudden coronary death/cardiac arrest, transient ischaemic attack, ischaemic stroke, intracerebral and subarachnoid haemorrhage, peripheral arterial disease, and abdominal aortic aneurysm.Results 114 859 individuals received an incident cardiovascular diagnosis during follow-up. Non-drinking was associated with an increased risk of unstable angina (hazard ratio 1.33, 95% confidence interval 1.21 to 1.45), myocardial infarction (1.32, 1.24 to1.41), unheralded coronary death (1.56, 1.38 to 1.76), heart failure (1.24, 1.11 to 1.38), ischaemic stroke (1.12, 1.01 to 1.24), peripheral arterial disease (1.22, 1.13 to 1.32), and abdominal aortic aneurysm (1.32, 1.17 to 1.49) compared with moderate drinking (consumption within contemporaneous UK weekly/daily guidelines of 21/3 and 14/2 units for men and women, respectively). Heavy drinking (exceeding guidelines) conferred an increased risk of presenting with unheralded coronary death (1.21, 1.08 to 1.35), heart failure (1.22, 1.08 to 1.37), cardiac arrest (1.50, 1.26 to 1.77), transient ischaemic attack (1.11, 1.02 to 1.37), ischaemic stroke (1.33, 1.09 to 1.63), intracerebral haemorrhage (1.37, 1.16 to 1.62), and peripheral arterial disease (1.35; 1.23 to 1.48), but a lower risk of myocardial infarction (0.88, 0.79 to 1.00) or stable angina (0.93, 0.86 to 1.00).Conclusions Heterogeneous associations exist between level of alcohol consumption and the initial presentation of cardiovascular diseases. This has implications for counselling patients, public health communication, and clinical research, suggesting a more nuanced approach to the role of alcohol in prevention of cardiovascular disease is necessary.Registration clinicaltrails.gov (NCT01864031).