Impact of mobility reduction on COVID-19 mortality: absence of evidence might be due to methodological issues

Computational reproducibility of Jupyter notebooks from biomedical publications

BACKGROUND

Jupyter notebooks facilitate the bundling of executable code with its documentation and output in one interactive environment, and they represent a popular mechanism to document and share computational workflows, including for research publications. The reproducibility of computational aspects of research is a key component of scientific reproducibility but has not yet been assessed at scale for Jupyter notebooks associated with biomedical publications.

APPROACH

We address computational reproducibility at 2 levels: (i) using fully automated workflows, we analyzed the computational reproducibility of Jupyter notebooks associated with publications indexed in the biomedical literature repository PubMed Central. We identified such notebooks by mining the article's full text, trying to locate them on GitHub, and attempting to rerun them in an environment as close to the original as possible. We documented reproduction success and exceptions and explored relationships between notebook reproducibility and variables related to the notebooks or publications. (ii) This study represents a reproducibility attempt in and of itself, using essentially the same methodology twice on PubMed Central over the course of 2 years, during which the corpus of Jupyter notebooks from articles indexed in PubMed Central has grown in a highly dynamic fashion.

RESULTS

Out of 27,271 Jupyter notebooks from 2,660 GitHub repositories associated with 3,467 publications, 22,578 notebooks were written in Python, including 15,817 that had their dependencies declared in standard requirement files and that we attempted to rerun automatically. For 10,388 of these, all declared dependencies could be installed successfully, and we reran them to assess reproducibility. Of these, 1,203 notebooks ran through without any errors, including 879 that produced results identical to those reported in the original notebook and 324 for which our results differed from the originally reported ones. Running the other notebooks resulted in exceptions.

CONCLUSIONS

We zoom in on common problems and practices, highlight trends, and discuss potential improvements to Jupyter-related workflows associated with biomedical publications.

Mobility during the pandemic: how did our movements shape the course of COVID-19?

In this manuscript, we critically assess the evidence around various methods of reducing mobility, and how these have impacted the course of the coronavirus disease 2019 (COVID-19) pandemic. We further highlight the difficulty in assessing the effectiveness of such measures before giving directions for future research.

Association of stay-at-home orders and COVID-19 incidence and mortality in rural and urban United States: a population-based study

OBJECTIVE

We examined the association between stay-at-home order implementation and the incidence of COVID-19 infections and deaths in rural versus urban counties of the United States.

DESIGN

We used an interrupted time-series analysis using a mixed effects zero-inflated Poisson model with random intercept by county and standardised by population to examine the associations between stay-at-home orders and county-level counts of daily new COVID-19 cases and deaths in rural versus urban counties between 22 January 2020 and 10 June 2020. We secondarily examined the association between stay-at-home orders and mobility in rural versus urban counties using Google Community Mobility Reports.

INTERVENTIONS

Issuance of stay-at-home orders.

PRIMARY AND SECONDARY OUTCOME MEASURES

Co-primary outcomes were COVID-19 daily incidence of cases (14-day lagged) and mortality (26-day lagged). Secondary outcome was mobility.

RESULTS

Stay-at-home orders were implemented later (median 30 March 2020 vs 28 March 2020) and were shorter in duration (median 35 vs 54 days) in rural compared with urban counties. Indoor mobility was, on average, 2.6%-6.9% higher in rural than urban counties both during and after stay-at-home orders. Compared with the baseline (pre-stay-at-home) period, the number of new COVID-19 cases increased under stay-at-home by incidence risk ratio (IRR) 1.60 (95% CI, 1.57 to 1.64) in rural and 1.36 (95% CI, 1.30 to 1.42) in urban counties, while the number of new COVID-19 deaths increased by IRR 14.21 (95% CI, 11.02 to 18.34) in rural and IRR 2.93 in urban counties (95% CI, 1.82 to 4.73). For each day under stay-at-home orders, the number of new cases changed by a factor of 0.982 (95% CI, 0.981 to 0.982) in rural and 0.952 (95% CI, 0.951 to 0.953) in urban counties compared with prior to stay-at-home, while number of new deaths changed by a factor of 0.977 (95% CI, 0.976 to 0.977) in rural counties and 0.935 (95% CI, 0.933 to 0.936) in urban counties. Each day after stay-at-home orders expired, the number of new cases changed by a factor of 0.995 (95% CI, 0.994 to 0.995) in rural and 0.997 (95% CI, 0.995 to 0.999) in urban counties compared with prior to stay-at-home, while number of new deaths changed by a factor of 0.969 (95% CI, 0.968 to 0.970) in rural counties and 0.928 (95% CI, 0.926 to 0.929) in urban counties.

CONCLUSION

Stay-at-home orders decreased mobility, slowed the spread of COVID-19 and mitigated COVID-19 mortality, but did so less effectively in rural than in urban counties. This necessitates a critical re-evaluation of how stay-at-home orders are designed, communicated and implemented in rural areas.