Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

A crisis like no other? Unmet needs in healthcare during the first wave of the COVID-19 crisis in Austria

Background

Many patients experienced restricted access to healthcare during the Coronavirus Disease 2019 (COVID-19) pandemic. This study is among the first to provide systematic evidence on the existence of subjective unmet needs (SUN) in different population groups during the pandemic.

Methods

Using data on individuals aged 20–64 and living in Austria from the AKCOVID survey (June 2020) and the ‘European Social Survey’ (2015), SUN were compared between 2015 and 2020, either related to the pandemic (fear of infection, provider closed or treatment postponed) or not (barriers related to knowledge, affordability, time and reachability). Multinomial logistic regression models identified determinants of SUN during the pandemic, adjusting for socio-demographics, socio-economic status and self-reported health.

Results

Shares of the population with SUN in 2020 substantially exceeded SUN in 2015. Excess unmet needs were mostly attributable to the pandemic. Postponed treatments and closed providers were the most important reasons for SUN in June 2020. Older age groups (50–64 years), inactive and retired people were most likely to report pandemic-related SUN. We did not find socio-economic differences in pandemic-related SUN.

Conclusions

The pandemic resulted in a supply-side shock to healthcare, with vulnerabilities emerging especially among older people, people with poor health and/or people no longer active on the labour market. Further research could focus on health system resilience and the possibilities to improve management of healthcare services during pandemics without widening inequalities while maintaining population health.

Thresholds for clinical practice that directly link handgrip strength to remaining years of life: estimates based on longitudinal observational data

OBJECTIVE

Muscle strength is a powerful predictor of mortality that can quickly and inexpensively be assessed by measuring handgrip strength (HGS). What is missing for clinical practice, however, are empirically meaningful cut-off points that apply to the general population and that consider the correlation of HGS with gender and body height as well as the decline in HGS during processes of normal ageing. This study provides standardised thresholds that directly link HGS to remaining life expectancy (RLE), thus enabling practitioners to detect patients with an increased mortality risk early on.

DESIGN

Relying on representative observational data from the Health and Retirement Study, the HGS of survey participants was z-standardised by gender, age and body height. We defined six HGS groups based on cut-off points in SD; we use these as predictors in survival analyses with a 9-year follow-up and provide RLE by gender based on a Gompertz model for each HGS group.

PARTICIPANTS

8156 US American women and men aged 50-80 years.

MAIN OUTCOME MEASURES

Z-standardised HGS and all-cause mortality.

RESULTS

Even slight negative deviations in HGS from the reference group with [0.0 SD, 0.5 SD) have substantial effects on survival. RLE among individuals aged 60 years with standardised HGS of [-0.5 SD, 0.0 SD) is 3.0/1.4 years lower for men/women than for the reference group, increasing to a difference of 4.1/2.6 years in the group with HGS of [-1.0 SD, -0.5 SD). By contrast, we find no benefit of strong HGS related to survival.

CONCLUSIONS

HGS varies substantially with gender, age and body height. This confirms the importance of considering these heterogeneities when defining reference groups and risk thresholds. Moreover, survival appears to decrease at much higher levels of muscle strength than is assumed in previous literature, suggesting that medical practitioners should start to become concerned when HGS is slightly below that of the reference group.

[The impact of the first phase of the COVID-19 pandemic on the employment and financial situation of families in Austria]

In dieser Forschungsnotiz werden erste Ergebnisse der AKCOVID-Studie vorgestellt. Diese untersucht auf Basis von repräsentativen Befragungsdaten die Auswirkungen der Pandemie auf die finanzielle Lage von Familien in Österreich. Dabei wird die Situation von Haushalten mit Kindern vor Beginn der Corona-Krise (Februar 2020) mit der Situation im Frühsommer 2020 verglichen. Die deskriptiven Ergebnisse zeigen, dass ein großer Teil der Familien bereits drei Monate nach Beginn der Krise die ökonomischen Folgen der Pandemie spürte und aufgrund krisenbedingter Veränderungen der elterlichen Erwerbssituation auf Teile des Haushaltseinkommens verzichten musste. Die Zahl der Familien mit finanziellen Problemen (subjektive Armutsgefährdung) stieg deutlich an, v. a. unter Alleinerziehenden und Paaren mit mehr als zwei Kindern. Damit wird deutlich, dass vor allem jene Familien, die sich bereits vor der Krise in einer vulnerablen finanziellen Situation befanden, schon sehr früh direkt von den ökonomischen Folgen der Pandemie betroffen waren. Viele Familien sorgten sich, dass sie aufgrund der Krise Einkommenseinbußen erleiden und finanzielle Probleme bekommen werden.

Care in times of COVID-19: the impact of the pandemic on informal caregiving in Austria

Much attention has been paid to the effects of the COVID-19 pandemic on long-term care but the impact on informal caregivers has remained speculative. In Austria, like in other European countries, informal care is carried out overwhelmingly by (non-cohabiting) relatives. Limited care services available during the pandemic, social-distancing, increased unemployment and competing care needs within households (e.g. due to school closures) may have changed the prevalence and intensity of informal caregiving. Moreover, these changes may have increased the psychological strain experienced by caregivers. Focusing on Austria, this study aims to empirically analyse the following research questions: how have the prevalence and intensity of informal care changed due to the pandemic? How has the psychological well-being of informal caregivers been affected? We use a pre- and post-onset of the pandemic research design based on a representative survey carried out in Austria in June 2020 (N = 2000) in combination with comparable 2015 data from the European Social Survey. Findings suggest that neither prevalence nor intensity of informal care changed significantly due to the pandemic. However, the psychological well-being gap between carers and non-carers increased with the start of the pandemic, especially among men. Findings are discussed in relation to the policy measures implemented and possible policy implications for the future.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s10433-021-00611-z.

The impact of occupational structures on ethnic and gendered employment gaps: An event history analysis using social security register data

Ethnic and gendered employment gaps are mainly explained by individual characteristics, while less attention is paid to occupational structures. Drawing on administrative data, this article analyses the impact of occupational characteristics on top of individual attributes in the urban labour market of Vienna. Both set of variables can explain observed employment gaps to a large extent, but persistent gaps remain, in particular among females. The article's main finding is that the occupational structure appears to have gendered effects. While men tend to benefit from ethnic segregation, women face difficulties when looking for jobs with high shares of immigrant workers. Looking for jobs in occupations that recruit from relatively few educational backgrounds (credentials) is beneficial for both sexes at the outset unemployment, but among females this competitive advantage diminishes over time. The article concludes by discussing potential strategies to avoid the traps of occupational segregation.

Does Intergenerational Educational Mobility Shape the Well-Being of Young Europeans? Evidence from the European Social Survey

Using pooled European Social Survey data (Rounds 4-7, 2008-2014), we investigate the relationship between intergenerational educational mobility and subjective well-being (SWB) for young Europeans (N = 16,050 individuals aged 25-34 from 18 countries). Previous research has been struggling with inconclusive results due to the methodological challenge of disentangling the independent (i.e., 'net') effect of social mobility over and above the effects of social origin and destination. We contribute to this line of research by contrasting mobility effects estimated in a conventional linear regression framework with net mobility effects estimated by (non-linear) diagonal mobility models (DMM). We show how model selection influences estimates of mobility effects and how different specifications lead to radically different findings. Using DMM, we estimate how intergenerational educational mobility affects the SWB of young Europeans, differentiating between downward and upward mobility and different country groups. Our results suggest that status loss/gain across generations affects young adults' SWB in addition to the level-effect of ending up in a lower/higher status position only in Continental Europe.

Strong or Weak Handgrip? Normative Reference Values for the German Population across the Life Course Stratified by Sex, Age, and Body Height

Handgrip strength is an important biomarker of healthy ageing and a powerful predictor of future morbidity and mortality both in younger and older populations. Therefore, the measurement of handgrip strength is increasingly used as a simple but efficient screening tool for health vulnerability. This study presents normative reference values for handgrip strength in Germany for use in research and clinical practice. It is the first study to provide normative data across the life course that is stratified by sex, age, and body height. The study used a nationally representative sample of test participants ages 17–90. It was based on pooled data from five waves of the German Socio-Economic Panel (2006–2014) and involved a total of 11,790 persons living in Germany (providing 25,285 observations). Handgrip strength was measured with a Smedley dynamometer. Results showed that peak mean values of handgrip strength are reached in men’s and women’s 30s and 40s after which handgrip strength declines in linear fashion with age. Following published recommendations, the study used a cut-off at 2 SD below the sex-specific peak mean value across the life course to define a ‘weak grip’. Less than 10% of women and men aged 65–69 were classified as weak according to this definition, shares increasing to about half of the population aged 80–90. Based on survival analysis that linked handgrip strength to a relevant outcome, however, a ‘critically weak grip’ that warrants further examination was estimated to commence already at 1 SD below the group-specific mean value.