Quality of COVID-19 research: An overview of large-scale assessments and a case study on excess mortality estimates

COVID-19 is an ongoing public health emergency, which affected individual and population health, whereas it disrupted social and economic activities for more than 2 years. High-quality research evidence, which is published without delays in the peer-review and publication pipeline, is the most powerful tool for evidence-based decision-making by physicians, public health and health policy specialists, and politicians. However, from the early stages of the pandemic till now, there is a debate on the quality of COVID-19 research, whereas it has been observed that the decrease in the time from submission to publication was accompanied by a decrease in research quality. This presentation discusses the findings and main conclusions from two research projects. The first part focuses on a systematic review of published large-scale qualitative evaluations of COVID-19 research. By taking advantage of these large-scale assessments, we will provide a bird's eye view on the quality of COVID-19 research done so far. The second part focuses on a case study describing the methodological quality of studies calculating excess mortality estimates during COVID-19 pandemic. Excess mortality estimates depend on important choices about the pre-pandemic reference period, the pandemic period of interest, and the modelling of the comparison between these two periods. Such an assessment has not been done yet although there is an abundance of published approaches to estimate excess mortality during the COVID-19 pandemic. Overall, these two projects highlight the importance of meta-research in the time of pandemic by scrutinizing and assessing scientific evidence while deriving recommendations for improvement of future research. This presentation concludes with a summary of the implications of low-quality COVID-19 research in decision-making and some general considerations to improve research quality and integrity in the case of future pandemics.

Elucidating the risk factors for chronic obstructive pulmonary disease: an umbrella review of meta-analyses

Chronic obstructive pulmonary disease (COPD) is commonly attributed to smoking, and other potential risk factors are ignored. We aimed to critically appraise the epidemiological credibility of the risk factors for COPD that have been examined in published meta-analyses. We performed a systematic search to capture systematic reviews and meta-analyses of observational studies on environmental factors and biomarkers for risk of COPD. We applied a set of standardised methodological criteria based on the level of statistical significance, sample size, between-study heterogeneity and statistical biases. Our search yielded 11 eligible papers, including 18 meta-analyses on environmental factors or biomarkers for COPD risk, and eight eligible papers with systematic reviews only. Eleven associations achieved statistical significance at P < 0.001 and six associations at P < 1 × 10^-6. Thirteen associations presented an I²  50%, while six associations had evidence of small-study effects and/or excess significance bias. History of tuberculosis or rheumatoid arthritis, exposure to biomass fuels, tobacco smoking and second hand smoking were supported by high epidemiological credibility for an increased risk of COPD. Furthermore, highly suggestive evidence was found for increased levels of serum C-reactive protein, and serum fibrinogen in COPD patients compared with healthy controls. To summarise, our approach suggests that, while a proportion of COPD patients are non-smokers, only a narrow range of risk factors not related to smoking have been studied for an association with COPD. There is also a need to decipher possible protective factors in COPD pathogenesis given that more than a half of ever-smokers do not develop COPD.

Risk factors and peripheral biomarkers for schizophrenia spectrum disorders: an umbrella review of meta-analyses

OBJECTIVE

This study aimed to systematically appraise the meta-analyses of observational studies on risk factors and peripheral biomarkers for schizophrenia spectrum disorders.

METHODS

We conducted an umbrella review to capture all meta-analyses and Mendelian randomization studies that examined associations between non-genetic risk factors and schizophrenia spectrum disorders. For each eligible meta-analysis, we estimated the summary effect size estimate, its 95% confidence and prediction intervals and the I² metric. Additionally, evidence for small-study effects and excess significance bias was assessed.

RESULTS

Overall, we found 41 eligible papers including 98 associations. Sixty-two associations had a nominally significant (P-value <0.05) effect. Seventy-two of the associations exhibited large or very large between-study heterogeneity, while 13 associations had evidence for small-study effects. Excess significance bias was found in 18 associations. Only five factors (childhood adversities, cannabis use, history of obstetric complications, stressful events during adulthood, and serum folate level) showed robust evidence.

CONCLUSION

Despite identifying 98 associations, there is only robust evidence to suggest that cannabis use, exposure to stressful events during childhood and adulthood, history of obstetric complications, and low serum folate level confer a higher risk for developing schizophrenia spectrum disorders. The evidence on peripheral biomarkers for schizophrenia spectrum disorders remains limited.