A Bayesian multivariate factor analysis model for causal inference using time-series observational data on mixed outcomes

Assessing the impact of an intervention by using time-series observational data on multiple units and outcomes is a frequent problem in many fields of scientific research. Here, we propose a novel Bayesian multivariate factor analysis model for estimating intervention effects in such settings and develop an efficient Markov chain Monte Carlo algorithm to sample from the high-dimensional and nontractable posterior of interest. The proposed method is one of the few that can simultaneously deal with outcomes of mixed type (continuous, binomial, count), increase efficiency in the estimates of the causal effects by jointly modeling multiple outcomes affected by the intervention, and easily provide uncertainty quantification for all causal estimands of interest. Using the proposed approach, we evaluate the impact that Local Tracing Partnerships had on the effectiveness of England's Test and Trace programme for COVID-19.

Evaluating the effect of direct-acting antiviral agent treatment scale-up on Hepatitis C virus prevalence among people who inject drugs in UK

BACKGROUND

There is limited empirical work assessing the effectiveness of treatment as prevention (TasP) in reducing HCV prevalence among people who inject drugs (PWID). Here, we used survey data from the UK during 2010-2020, to evaluate the impact of direct-acting antiviral agent (DAA) treatment scale-up, which started in 2015, on HCV prevalence among PWID.

METHODS

We fitted a logistic regression to time/location specific data on prevalence from the Needle Exchange Surveillance Initiative in Scotland and Unlinked Anonymous Monitoring programme in England. For each post-intervention year and location, we quantified the effect of TasP as the difference between estimated prevalence and its counterfactual (prevalence in the absence of scale-up). Progress to elimination was assessed by comparing most recent prevalence against one in 2015.

RESULTS

In 2015, prevalence ranged from 0.44 to 0.71 across the 23 locations (3 Scottish, 20 English). Compared to counterfactuals, there was an absolute reduction of 46% (95% credible interval [32%,59%]) in Tayside in 2020, 35% ([24%,44%]) in Glasgow in 2019, and 25% ([10%,39%]) in the Rest of Scotland in 2020. The English sites with highest estimated absolute reductions in 2021 were South Yorkshire (45%, [29%,58%]), Thames Valley (49%, [34%,59%]) and West London (41%, [14%,59%]). Compared to 2015, there was 80% probability that prevalence had fallen by 65% in Tayside, 53% in Glasgow and 36% in the Rest of Scotland. The English sites with highest % prevalence decrease compared to 2015, achieved with probability 80%, were Chesire & Merseyside (70%), South Yorkshire (65%) and Thames Valley (71%). Higher treatment intensity was associated with higher reductions in prevalence.

CONCLUSION

Conclusion. Real-world evidence showing substantial reductions in chronic HCV associated with increase of HCV treatment scale-up in the community thus supporting the effectiveness of HCV treatmen as prevention in people who inject drugs.

Evaluating mental health decision units in acute care pathways (DECISION): a quasi-experimental, qualitative and health economic evaluation

Background

People experiencing mental health crises in the community often present to emergency departments and are admitted to a psychiatric hospital. Because of the demands on emergency department and inpatient care, psychiatric decision units have emerged to provide a more suitable environment for assessment and signposting to appropriate care.

Objectives

The study aimed to ascertain the structure and activities of psychiatric decision units in England and to provide an evidence base for their effectiveness, costs and benefits, and optimal configuration.

Design

This was a mixed-methods study comprising survey, systematic review, interrupted time series, synthetic control study, cohort study, qualitative interview study and health economic evaluation, using a critical interpretive synthesis approach.

Setting

The study took place in four mental health National Health Service trusts with psychiatric decision units, and six acute hospital National Health Service trusts where emergency departments referred to psychiatric decision units in each mental health trust.

Participants

Participants in the cohort study (n = 2110) were first-time referrals to psychiatric decision units for two 5-month periods from 1 October 2018 and 1 October 2019, respectively. Participants in the qualitative study were first-time referrals to psychiatric decision units recruited within 1 month of discharge (n = 39), members of psychiatric decision unit clinical teams (n = 15) and clinicians referring to psychiatric decision units (n = 19).

Outcomes

Primary mental health outcome in the interrupted time series and cohort study was informal psychiatric hospital admission, and in the synthetic control any psychiatric hospital admission; primary emergency department outcome in the interrupted time series and synthetic control was mental health attendance at emergency department. Data for the interrupted time series and cohort study were extracted from electronic patient record in mental health and acute trusts; data for the synthetic control study were obtained through NHS Digital from Hospital Episode Statistics admitted patient care for psychiatric admissions and Hospital Episode Statistics Accident and Emergency for emergency department attendances. The health economic evaluation used data from all studies. Relevant databases were searched for controlled or comparison group studies of hospital-based mental health assessments permitting overnight stays of a maximum of 1 week that measured adult acute psychiatric admissions and/or mental health presentations at emergency department. Selection, data extraction and quality rating of studies were double assessed. Narrative synthesis of included studies was undertaken and meta-analyses were performed where sufficient studies reported outcomes.

Results

Psychiatric decision units have the potential to reduce informal psychiatric admissions, mental health presentations and wait times at emergency department. Cost savings are largely marginal and do not offset the cost of units. First-time referrals to psychiatric decision units use more inpatient and community care and less emergency department-based liaison psychiatry in the months following the first visit. Psychiatric decision units work best when configured to reduce either informal psychiatric admissions (longer length of stay, higher staff-to-patient ratio, use of psychosocial interventions), resulting in improved quality of crisis care or demand on the emergency department (higher capacity, shorter length of stay). To function well, psychiatric decision units should be integrated into the crisis care pathway alongside a range of community-based support.

Limitations

The availability and quality of data imposed limitations on the reliability of some analyses.

Future work

Psychiatric decision units should not be commissioned with an expectation of short-term financial return on investment but, if appropriately configured, they can provide better quality of care for people in crisis who would not benefit from acute admission or reduce pressure on emergency department.

Study registration

The systematic review was registered on the International Prospective Register of Systematic Reviews as CRD42019151043.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 17/49/70) and is published in full in Health and Social Care Delivery Research; Vol. 11, No. 25. See the NIHR Funding and Awards website for further award information.

Synthetic Control Methods for the Evaluation of Single-Unit Interventions in Epidemiology: A Tutorial

Evaluating the impacts of population-level interventions (e.g., changes to state legislation) can be challenging as conducting randomized experiments is often impractical and inappropriate, especially in settings where the intervention is implemented in a single, aggregate unit (e.g., a country or state). A common nonrandomized alternative is to compare outcomes in the treated unit(s) with unexposed controls both before and after the intervention. However, the validity of these designs depends on the use of controls that closely resemble the treated unit on before-intervention characteristics and trends on the outcome, and suitable controls may be difficult to find because the number of potential control regions is typically limited. The synthetic control method provides a potential solution to these problems by using a data-driven algorithm to identify an optimal weighted control unit—a “synthetic control”—based on data from before the intervention from available control units. While popular in the social sciences, the method has not garnered as much attention in health research, perhaps due to a lack of accessible texts aimed at health researchers. We address this gap by providing a comprehensive, nontechnical tutorial on the synthetic control method, using a worked example evaluating Florida’s “stand your ground” law to illustrate methodological and practical considerations.

A (Flexible) Synthetic Control Method for Count Data and Other Nonnegative Outcomes

The synthetic control method is a covariate balancing method that exploits data from untreated regions to construct a synthetic control that approximates a single, aggregate treatment unit on a time series of preintervention outcomes and covariates. The method is increasingly being used to evaluate population-level interventions in epidemiology. Although the original version can be used with bounded outcomes, it imposes strong constraints on the balancing weights to ensure that the counterfactuals are based solely on interpolation. This feature, while attractive from a causal inference perspective, is sometimes too conservative and can lead to unnecessary bias due to poor covariate balance. Alternatives exist that allow for extrapolation to improve balance but existing procedures may produce negative estimates of the counterfactual outcomes and are therefore inappropriate for count data. We propose an alternative way to allow for extrapolation, although ensuring that the estimated counterfactuals remain nonnegative. Following a related proposal, we add a penalty to the balancing procedure that favors interpolation over extrapolation whenever possible. As we demonstrate theoretically and using empirical examples, our proposal can serve as a useful alternative when existing approaches yield demonstrably poor or unrealistic counterfactuals. Finally, we provide functions to implement the method in R.

Modelling the impact of the tier system on SARS-CoV-2 transmission in the UK between the first and second national lockdowns

OBJECTIVE

To measure the effects of the tier system on the COVID-19 pandemic in the UK between the first and second national lockdowns, before the emergence of the B.1.1.7 variant of concern.

DESIGN

This is a modelling study combining estimates of real-time reproduction number Rt (derived from UK case, death and serological survey data) with publicly available data on regional non-pharmaceutical interventions. We fit a Bayesian hierarchical model with latent factors using these quantities to account for broader national trends in addition to subnational effects from tiers.

SETTING

The UK at lower tier local authority (LTLA) level. 310 LTLAs were included in the analysis.

PRIMARY AND SECONDARY OUTCOME MEASURES

Reduction in real-time reproduction number Rt .

RESULTS

Nationally, transmission increased between July and late September, regional differences notwithstanding. Immediately prior to the introduction of the tier system, Rt averaged 1.3 (0.9-1.6) across LTLAs, but declined to an average of 1.1 (0.86-1.42) 2 weeks later. Decline in transmission was not solely attributable to tiers. Tier 1 had negligible effects. Tiers 2 and 3, respectively, reduced transmission by 6% (5%-7%) and 23% (21%-25%). 288 LTLAs (93%) would have begun to suppress their epidemics if every LTLA had gone into tier 3 by the second national lockdown, whereas only 90 (29%) did so in reality.

CONCLUSIONS

The relatively small effect sizes found in this analysis demonstrate that interventions at least as stringent as tier 3 are required to suppress transmission, especially considering more transmissible variants, at least until effective vaccination is widespread or much greater population immunity has amassed.

Evaluating the power of the causal impact method in observational studies of HCV treatment as prevention

OBJECTIVES

The causal impact method (CIM) was recently introduced for evaluation of binary interventions using observational time-series data. The CIM is appealing for practical use as it can adjust for temporal trends and account for the potential of unobserved confounding. However, the method was initially developed for applications involving large datasets and hence its potential in small epidemiological studies is still unclear. Further, the effects that measurement error can have on the performance of the CIM have not been studied yet. The objective of this work is to investigate both of these open problems.

METHODS

Motivated by an existing dataset of HCV surveillance in the UK, we perform simulation experiments to investigate the effect of several characteristics of the data on the performance of the CIM. Further, we quantify the effects of measurement error on the performance of the CIM and extend the method to deal with this problem.

RESULTS

We identify multiple characteristics of the data that affect the ability of the CIM to detect an intervention effect including the length of time-series, the variability of the outcome and the degree of correlation between the outcome of the treated unit and the outcomes of controls. We show that measurement error can introduce biases in the estimated intervention effects and heavily reduce the power of the CIM. Using an extended CIM, some of these adverse effects can be mitigated.

CONCLUSIONS

The CIM can provide satisfactory power in public health interventions. The method may provide misleading results in the presence of measurement error.

Ecological responses to blue water MPAs

Marine protected areas (MPAs) can contribute to protecting biodiversity and managing ocean activities, including fishing. There is, however, limited evidence of ecological responses to blue water MPAs. We conducted the first comprehensive evaluation of impacts on fisheries production and ecological responses to pelagic MPAs of the Pacific Remote Islands Marine National Monument. A Bayesian time series-based counterfactual modelling approach using fishery-dependent data was used to compare the temporal response in the MPAs to three reference regions for standardized catch rates, lengths, trophic level of the catch and species diversity. Catch rates of bigeye tuna, the main target species (Kingman/Palmyra MPA, causal effect probability >99% of an 84% reduction; 95% credible interval: -143%, -25%), and blue shark (Johnston MPAs, causal effect probability >95%) were significantly lower and longnose lancetfish significantly higher (Johnston MPAs, causal effect probability >95%) than predicted had the MPAs not been established, possibly from closing areas near shallow features, which aggregate pelagic predators, and from ‘fishing-the-line’. There were no apparent causal impacts of the MPAs on species diversity, lengths and trophic level of the catch, perhaps because the MPAs were young, were too small, did not contain critical habitat for specific life-history stages, had been lightly exploited or experienced fishing-the-line. We also assessed model-standardized catch rates for species of conservation concern and mean trophic level of the catch within and outside of MPAs. Displaced effort produced multi-species conflicts: MPAs protect bycatch hotspots and hotspots of bycatch-to-target catch ratios for some at-risk species, but coldspots for others. Mean trophic level of the catch was significantly higher around MPAs, likely due to the aggregating effect of the shallow features and there having been light fishing pressure within MPAs. These findings demonstrate how exploring a wide range of ecological responses supports evidence-based evaluations of blue water MPAs.

Effects of the Convention on the Rights of the Child on child mortality and vaccination rates: a synthetic control analysis

BACKGROUND

Scholars have long been sceptical about the effectiveness of human rights treaties in changing the behaviour of states parties and prior empirical research has often justified that scepticism. However, only a few prior studies have considered the effects of adoption of core human rights treaties on health outcomes and only one prior study has analysed the effects of adoption of the Convention on the Rights of the Child (CRC) on children's health outcomes.

METHODS

In this study, we estimated the effects of CRC adoption on child mortality rates and vaccination rates in less developed countries. In particular, we compared 43 less developed countries that adopted the CRC in 1990 with synthetic control groups drawn from 21 less developed countries that adopted it after 1992.

RESULTS

We find that CRC adoption may be related to additional reductions in infant and under-5 mortality rates of about 1 to 2 deaths per 1000 live births, on average, during the first three years after adoption, although those relationships are not statistically significant. And we find that CRC adoption is related to additional increases in vaccination rates for the five vaccines that we considered of about 4 to 5%, on average, during the first three years after adoption and that those relationships remain significant for up to seven years after adoption.

CONCLUSION

From a policy perspective, our results further support the effectiveness of CRC adoption in promoting children's right to health in less developed countries. And from a research perspective, our results show the advantages of using synthetic control methods in these types of studies, because our analyses using other methods that have most commonly been used in these studies did not find any consistent, significant relationships between CRC adoption and mortality or vaccination rates.