Valid Adjustment of Randomized Comparisons for Binary Covariates

Effect of modified income assistance payment schedules on substance use service access: Evidence from an experimental study

BACKGROUND

Despite being critical to reducing the impacts of poverty internationally, synchronized monthly government income assistance payments are linked to intensified drug use and associated harms, including disrupted access to substance use-related services. This study evaluates whether alternative income assistance distribution schedules improve harm reduction (HR), pharmacotherapy and substance use service utilization.

METHODS

This exploratory, parallel group, unblinded, randomized controlled trial analyzed data from adults (n = 192) in Vancouver, Canada receiving income assistance, and reporting active, regular illicit drug use. Participants were randomly assigned on a 1:2:2 basis for six income assistance payment cycles to: (1) existing government schedules (control); (2) a "staggered" single monthly payment; or (3) "split & staggered" twice-monthly payments. Generalized linear mixed models analyzed secondary outcomes of HR, pharmacotherapy and substance use service utilization as well as barriers accessing these services.

RESULTS

Forty-five control, 71 staggered, and 76 split & staggered volunteers participated between 2015 and 2019. Multivariable modified per-protocol analyses demonstrate increased access to substance use services (Adjusted Odds Ratio [AOR] 1.64, 95% Confidence Interval [CI] 1.02-2.64) for split & staggered arm participants, and, conversely, increased barriers to HR for participants in the staggered (AOR 2.34, 95% CI 1.24-4.41) and split & staggered (AOR 2.16, 95% CI 1.08-4.35) arms. Results also showed decreased barriers to pharmacotherapy around government payments (AOR 0.23, 95% CI 0.06-0.90), pharmacotherapy around individual payments (AOR 0.12, 95% CI 0.02-0.58), and HR around individual payments (AOR 0.11, 95% CI 0.02-0.63) for staggered arm participants.

CONCLUSION

Modifying payments schedules demonstrate improved access to overall substance use services, and reduced barriers to HR and pharmacotherapy around income assistance payments. However, increased overall barriers to HR access were also shown. These complex, predominantly beneficial findings support the exploration of offering alternative payment schedules to support service access.

Effect of alternative income assistance schedules on drug use and drug-related harm: a randomised controlled trial

Background

The synchronised monthly disbursement of income assistance, whereby all recipients are paid on the same day, has been associated with increases in illicit drug use and serious associated harms. This phenomenon is often referred to as the cheque effect. Because payment variability can affect consumption patterns, this study aimed to assess whether these harms could be mitigated through a structural intervention that varied income assistance payment timing and frequency.

Methods

This randomised, parallel group trial was done in Vancouver, Canada, and enrolled recipients of income assistance whose drug use increased around payment days. The recipients were randomly assigned 1:2:2 to a control group that received monthly synchronised income assistance payments on government payment days, a staggered group in which participants received single desynchronised monthly income assistance payments, or a split and staggered group in which participants received desynchronised income assistance payments split into two instalments per month, 2 weeks apart, for six monthly payment cycles. Desynchronised payments in the intervention groups were made on individual payment days outside the week of the standard government schedules. Randomisation was through a pre-established stratified block procedure. Investigators and statisticians were masked to group allocation, but participants and front-line staff were not. Complete final results are reported after scheduled interim analyses and the resulting early stoppage of recruitment. Under intention-to-treat specifications, generalised linear mixed models were used to analyse the primary outcome, which was escalations in drug use, predefined as a 40% increase in at least one of: use frequency; use quantity; or number of substances used during the 3 days after government payments. Secondary analyses examined analogous drug use outcomes coinciding with individual payments as well as exposure to violence. This trial is registered with ClinicalTrials.gov, NCT02457949.

Findings

Between Oct 27, 2015, and Jan 2, 2019, 45 participants were enrolled to the control group, 72 to the staggered group, and 77 to the split and staggered group. Intention-to-treat analyses showed a significantly reduced likelihood of increased drug use coinciding with government payment days, relative to the control group, in the staggered (adjusted odds ratio 0·38, 95% CI 0·20–0·74; p=0·0044) and split and staggered (0·44, 0·23–0·83; p=0·012) groups. Findings were consistent in the secondary analyses of drug use coinciding with individual payment days (staggered group 0·50, 0·27–0·96, p=0·036; split and staggered group 0·49, 0·26–0·94, p=0·030). However, secondary outcome analyses of exposure to violence showed increased harm in the staggered group compared with the control group (2·71, 1·06–6·91, p=0·037). Additionally, 51 individuals had a severe or life-threatening adverse event and there were six deaths, none of which was directly attributed to study participation.

Interpretation

Complex results indicate the potential for modified income assistance payment schedules to mitigate escalations in drug use, provided measures to address unintended harms are also undertaken. Additional research is needed to clarify whether desynchronised schedules produce other unanticipated consequences and if additional measures could mitigate these harms.

Funding

Canadian Institutes of Health Research, Providence Health Care Research Institute, Peter Wall Institute for Advanced Research, Michael Smith Foundation for Health Research.

A comparison of methods to adjust for continuous covariates in the analysis of randomised trials

BACKGROUND

Although covariate adjustment in the analysis of randomised trials can be beneficial, adjustment for continuous covariates is complicated by the fact that the association between covariate and outcome must be specified. Misspecification of this association can lead to reduced power, and potentially incorrect conclusions regarding treatment efficacy.

METHODS

We compared several methods of adjustment to determine which is best when the association between covariate and outcome is unknown. We assessed (a) dichotomisation or categorisation; (b) assuming a linear association with outcome; (c) using fractional polynomials with one (FP1) or two (FP2) polynomial terms; and (d) using restricted cubic splines with 3 or 5 knots. We evaluated each method using simulation and through a re-analysis of trial datasets.

RESULTS

Methods which kept covariates as continuous typically had higher power than methods which used categorisation. Dichotomisation, categorisation, and assuming a linear association all led to large reductions in power when the true association was non-linear. FP2 models and restricted cubic splines with 3 or 5 knots performed best overall.

CONCLUSIONS

For the analysis of randomised trials we recommend (1) adjusting for continuous covariates even if their association with outcome is unknown; (2) keeping covariates as continuous; and (3) using fractional polynomials with two polynomial terms or restricted cubic splines with 3 to 5 knots when a linear association is in doubt.

The analysis of stratified 2 x 2 contingency tables

We consider the problem of testing for independence against the consistent superiority of one treatment over another when the response variable is binary and is compared across two treatments in each of several strata. Specifically, we consider the randomized clinical trial setting. A number of issues arise in this context. First, should tables be combined if there are small or zero margins? Second, should one assume a common odds ratio across strata? Third, if the odds ratios differ across strata, then how does the standard test (based on a common odds ratio) perform? Fourth, are there other analyzes that are more appropriate for handling a situation in which the odds ratios may differ across strata? In addressing these issues we find that the frequently used Cochran-Mantel-Haenszel test may have a poor power profile, despite being optimal when the odds ratios are common. We develop novel tests that are analogous to the Smirnov, modified Smirnov, convex hull, and adaptive tests that have been proposed for ordered categorical data.

A Cautionary Note on Design Implications when the Primary Analysis is a Stratified Analysis of a Binary Endpoint

When the primary analysis is a stratified analysis of a binary endpoint, the commonly used CMH procedure could produce results that are quite different from those obtained by ignoring the stratification factor altogether. This means that when designing a trial with a primary endpoint that is binary, it is important that one keep in mind how the data will be analyzed. Only when design and analysis are considered in tandem will the study possess the desired properties. The discussion in this paper is illustrated by a hypothetical example, which nevertheless resembles the findings from a confirmatory trial involving the evaluation of a biologic for severe sepsis.

Nonparametric adjustment techniques for binary covariates

Though a variety of reasons are often articulated for adjusting analyses for covariates, these reasons often fall into one of two general objectives, specifically to increase precision or to decrease bias. In practice, one does not generally choose between these objectives, because the methods that address one tend to address the other, as well. Because of this, no distinction is made in the methods used to correct for a baseline imbalance with respect to a prognostic covariate versus to ensure a fair comparison across treatment groups by making the comparisons within the levels of a prognostic covariate. Yet the literal translation of these two uses of covariate adjustment will lead to two distinct adjustment methods. We illustrate this divergence in the simplest case of a single binary covariate, a binary outcome, and two treatments, and we note that it is possible to combine the two approaches to derive yet a third approach. Each of these approaches is nonparametric and exact, and so it is the precise reason for adjusting that should dictate which would be used in any given situation.