Establishment survey participation during the COVID-19 pandemic

Establishment surveys around the globe have measured the impact of the COVID-19 pandemic on establishments' conditions and business practices. At the same time, the consequences of the pandemic, such as closures, hygiene standards, or remote work arrangements, may have also altered patterns of survey participation and introduced nonresponse bias, threatening the quality of establishment survey data. To investigate these issues, this article examines fieldwork outcomes, nonresponse bias, and predictors of survey participation in the IAB-Job Vacancy Survey. As comparisons with previous survey years show, it became more difficult to successfully interview establishments during the COVID-19 pandemic. Using linked administrative data, we show that nonresponse bias was higher in 2020 compared to previous years, even after applying the standard weighting adjustment. However, general patterns of survey participation in 2020 were similar to previous years and COVID-19 related measures were not strong predictors of survey participation in 2020. Further, we provide evidence that nonresponse bias during the pandemic can be reduced by incorporating additional administrative variables into the weighting procedure relative to the standard weighting variables. We conclude this article with a discussion of the findings and implications for survey practitioners.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1186/s12651-022-00321-8.

Observing many researchers using the same data and hypothesis reveals a hidden universe of uncertainty

This study explores how researchers' analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers' expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team's workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers' results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

Development of a new COVID-19 panel survey: the IAB high-frequency online personal panel (HOPP)

Since January 2020, the COVID-19 crisis has affected everyday life around the world, and rigorous government lockdown restrictions have been implemented to prevent the further spread of the pandemic. The consequences of the corona crisis and the associated lockdown policies for public health, social life, and the economy are vast. In view of the rapidly changing situation during this crisis, policymakers require timely data and research results that allow for informed decisions. Addressing the requirement for adequate databases to assess people's  life and work situations during the pandemic, the Institute for Employment Research (IAB) developed the High-frequency Online Personal Panel (HOPP). The HOPP study started in May 2020 and is based on a random sample of individuals drawn from the administrative data of the Federal Employment Agency in Germany, containing information on all labour market participants except civil servants and self-employed. The main goal of the HOPP study is to assess the short-term as well as long-term changes in people's social life and working situation in Germany due to the corona pandemic. To assess individual dynamics the HOPP collected data on a monthly (wave  one to four) and bi-monthly (wave five to seven) basis. Furthermore, respondents were divided into four groups. The different groups of a new wave were invited to the survey at weekly intervals (wave two to four) or bi-weekly intervals (wave five to seven). This gives us the advantage of being able to provide weekly data while each participant only had to participate on a monthly or bi-monthly basis. In this article, we delineate the HOPP study in terms of its main goals and features, topics, and survey design. Furthermore, we provide a summary of results derived from HOPP and the future prospects of the study.

Variance Estimation by Linearisation for the At Risk of Poverty or Social Exclusion (AROPE) Rate

The At Risk of Poverty or Social Exclusion (AROPE) Rate is the key indicator for monitoring the European Commissions 2020 Strategy poverty target. But the variance of the AROPE Rate is not straightforward to estimate. Re-sampling methods can be used, but they are difficult to adapt to complex sampling design, that are often used for the surveys that provide the data source for estimating the AROPER. The presented work fills a methodological gap by providing a linearisation of the AROPE Rate estimator that can be used with well known variance estimators and therefore facilitate the reporting of appropriate inference for this important indicator. The precision of the developed variance estimators based on linearisation is assessed via simulation studies and compared with a bootstrap variance estimator, as an alternative.