Quasi-experimental study designs series—paper 13: realizing the full potential of quasi-experiments for health research

Systematic Review Reveals Lack of Causal Methodology Applied to Pooled Longitudinal Observational Infectious Disease Studies

OBJECTIVES

Among ID studies seeking to make causal inferences and pooling individual-level longitudinal data from multiple infectious disease cohorts, we sought to assess what methods are being used, how those methods are being reported, and whether these factors have changed over time.

STUDY DESIGN AND SETTING

Systematic review of longitudinal observational infectious disease studies pooling individual-level patient data from 2+ studies published in English in 2009. 2014, or 2019. This systematic review protocol is registered with PROSPERO (CRD42020204104).

RESULTS

Our search yielded 1,462 unique articles. Of these, 16 were included in the final review. Our analysis showed a lack of causal inference methods and of clear reporting on methods and the required assumptions.

CONCLUSION

There are many approaches to causal inference which may help facilitate accurate inference in the presence of unmeasured and time-varying confounding. In observational ID studies leveraging pooled, longitudinal IPD, the absence of these causal inference methods and gaps in the reporting of key methodological considerations suggests there is ample opportunity to enhance the rigor and reporting of research in this field. Interdisciplinary collaborations between substantive and methodological experts would strengthen future work.

Opportunities and Challenges in HIV Treatment as Prevention Research: Results from the ANRS 12249 Cluster-Randomized Trial and Associated Population Cohort

PURPOSE OF REVIEW

The ANRS 12249 treatment as prevention (TasP) trial investigated the impact of a universal test and treat (UTT) approach on reducing HIV incidence in one of the regions of the world most severely affected by the HIV epidemic-KwaZulu-Natal, South Africa. We summarize key findings from this trial as well as recent findings from controlled studies conducted in the linked population cohort quantifying the long-term effects of expanding ART on directly measured HIV incidence (2004-2017).

RECENT FINDINGS

The ANRS TasP trial did not-and could not-demonstrate a reduction in HIV incidence, because the offer of UTT in the intervention communities did not increase ART coverage and population viral suppression compared to the standard of care in the control communities. Ten controlled studies from the linked population cohort-including several quasi-experimental study designs-exploit heterogeneity in ART exposure to show a consistent and substantial impact of expanding provision of ART and population viral suppression on reduction in HIV incidence at the couple, household, community, and population levels. In this setting, all of the evidence from large, population-based studies (inclusive of the ANRS TasP trial) is remarkably coherent and consistent-i.e., higher ART coverage and population viral suppression were repeatedly associated with clear, measurable decreases in HIV incidence. Thus, the expanded provision of ART has plausibly contributed in a major way toward the dramatic 43% decline in population-level HIV incidence in this typical rural African population. The outcome of the ANRS TasP trial constitutes a powerful null finding with important insights for overcoming implementation challenges in the population delivery of ART. This finding does not imply lack of ART effectiveness in blocking onward transmission of HIV nor its inability to reduce HIV incidence. Rather, it demonstrates that large increases in ART coverage over current levels will require health systems innovations to attract people living with HIV in early stages of the disease to participate in HIV treatment. Such innovations and new approaches are required for the true potential of UTT to be realized.

Using the Natural Experiment Study Design to Evaluate the Effect of a Change in Doctor's Roster on Patient Flow in an Emergency Department

Introduction

The effect of changes in doctors' rosters is rarely subjected to scientific evaluation. We describe how a natural experiment (NE) study design can be used to evaluate if a managerial decision about doctors' rosters has an effect on patient flow in an emergency department (ED). We hypothesized that an extra doctor each morning from 6 a.m. (i.e., a modified "casino shift") might improve the productivity of a hospital's ED.

Methods

This was an NE observational study using data on patient flow in the ED of Zealand University Hospital, Denmark, between April 1, 2016, and April 1, 2017. We compared days on which the 6 a.m. emergency physician called in sick (case days) with data from the same weekday a week later where staffing was as scheduled (control days).

Results

Patient caseload did not did differ significantly on days with and without the extra doctor from 6 a.m. (measured by number of admissions, triage scores and mean patient age). Door-to-doctor time was 70 minutes (mean, standard deviation [SD], 49) on days without the extra doctor and 56 minutes (mean, SD 41) on days with the early-morning doctor present (p > 0.05). ED length of stay was 250 minutes (mean, SD 119) on days without the extra doctor and 209 minutes (mean, SD 109) on days with the early-morning doctor present (p > 0.05).

Conclusion

In our setting, an extra doctor in the ED from 6 a.m. did not change patient flow. These results suggest that the workflow in the ED should be viewed as a connected supply chain. The study also demonstrates that a natural experiment study design can be used to evaluate ED managerial decisions.

Reducing inequity of cataract blindness and vision impairment is a global priority, but where is the evidence?

Throughout the world, people who are socially or economically disadvantaged disproportionately experience blindness and vision impairment caused by cataract. Reducing vision loss from cataract and its unequal distribution must be a priority if the WHO's aim of 'universal eye health' is to be realised. To help achieve this, decision-makers and service planners need evidence on which strategies improve access to cataract services among disadvantaged populations, and under what circumstances. Unfortunately, despite many strategies to improve cataract services being implemented in recent decades, evidence of what works, for who and in what circumstances is not readily available. This paper summarises the extent of the evidence on interventions to reduce inequity of vision loss from cataract and makes suggestions for how the evidence base can be strengthened.

Quasi-experimental study designs series-paper 1: introduction: two historical lineages

OBJECTIVES

The objective of this study was to contrast the historical development of experiments and quasi-experiments and provide the motivation for a journal series on quasi-experimental designs in health research.

STUDY DESIGN AND SETTING

A short historical narrative, with concrete examples, and arguments based on an understanding of the practice of health research and evidence synthesis.

RESULTS

Health research has played a key role in developing today's gold standard for causal inference-the randomized controlled multiply blinded trial. Historically, allocation approaches developed from convenience and purposive allocation to alternate and, finally, to random allocation. This development was motivated both by concerns for manipulation in allocation as well as statistical and theoretical developments demonstrating the power of randomization in creating counterfactuals for causal inference. In contrast to the sequential development of experiments, quasi-experiments originated at very different points in time, from very different scientific perspectives, and with frequent and long interruptions in their methodological development. Health researchers have only recently started to recognize the value of quasi-experiments for generating novel insights on causal relationships.

CONCLUSION

While quasi-experiments are unlikely to replace experiments in generating the efficacy and safety evidence required for clinical guidelines and regulatory approval of medical technologies, quasi-experiments can play an important role in establishing the effectiveness of health care practice, programs, and policies. The papers in this series describe and discuss a range of important issues in utilizing quasi-experimental designs for primary research and quasi-experimental results for evidence synthesis.