Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

Depressed during the depression: has the economic crisis affected mental health inequalities in Europe? Findings from the European Social Survey (2014) special module on the determinants of health

Background

Economic crises constitute a shock to societies with potentially harmful effects to the mental health status of the population, including depressive symptoms, and existing health inequalities.

Methods

With recent data from the European Social Survey (2006–14), this study investigates how the economic recession in Europe starting in 2007 has affected health inequalities in 21 European nations. Depressive feelings were measured with the CES-D eight-item depression scale. We tested for measurement invariance across different socio-economic groups.

Results

Overall, depressive feelings have decreased between 2006 and 2014 except for Cyprus and Spain. Inequalities between persons whose household income depends mainly on public benefits and those who do not have decreased, while the development of depressive feelings was less favorable among the precariously employed and the inactive than among the persons employed with an unlimited work contract. There are no robust effects of the crisis measure on health inequalities.

Conclusion

Negative implications for mental health (in terms of depressive feelings) have been limited to some of the most strongly affected countries, while in the majority of Europe persons have felt less depressed over the course of the recession. Health inequalities have persisted in most countries during this time with little influence of the recession. Particular attention should be paid to the mental health of the inactive and the precariously employed.