1
|
Voldal EC, Kenny A, Xia F, Heagerty P, Hughes JP. Robust analysis of stepped wedge trials using composite likelihood models. Stat Med 2024. [PMID: 38837431 DOI: 10.1002/sim.10120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 05/02/2024] [Accepted: 05/09/2024] [Indexed: 06/07/2024]
Abstract
Stepped wedge trials (SWTs) are a type of cluster randomized trial that involve repeated measures on clusters and design-induced confounding between time and treatment. Although mixed models are commonly used to analyze SWTs, they are susceptible to misspecification particularly for cluster-longitudinal designs such as SWTs. Mixed model estimation leverages both "horizontal" or within-cluster information and "vertical" or between-cluster information. To use horizontal information in a mixed model, both the mean model and correlation structure must be correctly specified or accounted for, since time is confounded with treatment and measurements are likely correlated within clusters. Alternative non-parametric methods have been proposed that use only vertical information; these are more robust because between-cluster comparisons in a SWT preserve randomization, but these non-parametric methods are not very efficient. We propose a composite likelihood method that focuses on vertical information, but has the flexibility to recover efficiency by using additional horizontal information. We compare the properties and performance of various methods, using simulations based on COVID-19 data and a demonstration of application to the LIRE trial. We found that a vertical composite likelihood model that leverages baseline data is more robust than traditional methods, and more efficient than methods that use only vertical information. We hope that these results demonstrate the potential value of model-based vertical methods for SWTs with a large number of clusters, and that these new tools are useful to researchers who are concerned about misspecification of traditional models.
Collapse
Affiliation(s)
| | - Avi Kenny
- Department of Biostatistics & Bioinformatics, Duke University, Durham, North Carolina, USA
- Global Health Institute, Duke University, Durham, North Carolina, USA
| | - Fan Xia
- Department of Epidemiology & Biostatistics, University of California San Francisco, San Francisco, California, USA
| | - Patrick Heagerty
- Department of Biostatistics, University of Washington, Seattle, Washington, USA
| | - James P Hughes
- Department of Biostatistics, University of Washington, Seattle, Washington, USA
| |
Collapse
|
2
|
Liu J, Li F. Optimal designs using generalized estimating equations in cluster randomized crossover and stepped wedge trials. Stat Methods Med Res 2024:9622802241247717. [PMID: 38813761 DOI: 10.1177/09622802241247717] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/31/2024]
Abstract
Cluster randomized crossover and stepped wedge cluster randomized trials are two types of longitudinal cluster randomized trials that leverage both the within- and between-cluster comparisons to estimate the treatment effect and are increasingly used in healthcare delivery and implementation science research. While the variance expressions of estimated treatment effect have been previously developed from the method of generalized estimating equations for analyzing cluster randomized crossover trials and stepped wedge cluster randomized trials, little guidance has been provided for optimal designs to ensure maximum efficiency. Here, an optimal design refers to the combination of optimal cluster-period size and optimal number of clusters that provide the smallest variance of the treatment effect estimator or maximum efficiency under a fixed total budget. In this work, we develop optimal designs for multiple-period cluster randomized crossover trials and stepped wedge cluster randomized trials with continuous outcomes, including both closed-cohort and repeated cross-sectional sampling schemes. Local optimal design algorithms are proposed when the correlation parameters in the working correlation structure are known. MaxiMin optimal design algorithms are proposed when the exact values are unavailable, but investigators may specify a range of correlation values. The closed-form formulae of local optimal design and MaxiMin optimal design are derived for multiple-period cluster randomized crossover trials, where the cluster-period size and number of clusters are decimal. The decimal estimates from closed-form formulae can then be used to investigate the performances of integer estimates from local optimal design and MaxiMin optimal design algorithms. One unique contribution from this work, compared to the previous optimal design research, is that we adopt constrained optimization techniques to obtain integer estimates under the MaxiMin optimal design. To assist practical implementation, we also develop four SAS macros to find local optimal designs and MaxiMin optimal designs.
Collapse
Affiliation(s)
- Jingxia Liu
- Division of Public Health Sciences, Department of Surgery and Division of Biostatistics, Washington University School of Medicine, St. Louis, MO, USA
| | - Fan Li
- Department of Biostatistics, Yale University, New Haven, CT, USA
| |
Collapse
|
3
|
Ouyang Y, Taljaard M, Forbes AB, Li F. Maintaining the validity of inference from linear mixed models in stepped-wedge cluster randomized trials under misspecified random-effects structures. Stat Methods Med Res 2024:9622802241248382. [PMID: 38807552 DOI: 10.1177/09622802241248382] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/30/2024]
Abstract
Linear mixed models are commonly used in analyzing stepped-wedge cluster randomized trials. A key consideration for analyzing a stepped-wedge cluster randomized trial is accounting for the potentially complex correlation structure, which can be achieved by specifying random-effects. The simplest random effects structure is random intercept but more complex structures such as random cluster-by-period, discrete-time decay, and more recently, the random intervention structure, have been proposed. Specifying appropriate random effects in practice can be challenging: assuming more complex correlation structures may be reasonable but they are vulnerable to computational challenges. To circumvent these challenges, robust variance estimators may be applied to linear mixed models to provide consistent estimators of standard errors of fixed effect parameters in the presence of random-effects misspecification. However, there has been no empirical investigation of robust variance estimators for stepped-wedge cluster randomized trials. In this article, we review six robust variance estimators (both standard and small-sample bias-corrected robust variance estimators) that are available for linear mixed models in R, and then describe a comprehensive simulation study to examine the performance of these robust variance estimators for stepped-wedge cluster randomized trials with a continuous outcome under different data generators. For each data generator, we investigate whether the use of a robust variance estimator with either the random intercept model or the random cluster-by-period model is sufficient to provide valid statistical inference for fixed effect parameters, when these working models are subject to random-effect misspecification. Our results indicate that the random intercept and random cluster-by-period models with robust variance estimators performed adequately. The CR3 robust variance estimator (approximate jackknife) estimator, coupled with the number of clusters minus two degrees of freedom correction, consistently gave the best coverage results, but could be slightly conservative when the number of clusters was below 16. We summarize the implications of our results for the linear mixed model analysis of stepped-wedge cluster randomized trials and offer some practical recommendations on the choice of the analytic model.
Collapse
Affiliation(s)
- Yongdong Ouyang
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
| | - Monica Taljaard
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
| | - Andrew B Forbes
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia
| | - Fan Li
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
- Center for Methods in Implementation and Prevention Science, Yale School of Public Health, New Haven, CT, USA
| |
Collapse
|
4
|
Nevins P, Ryan M, Davis-Plourde K, Ouyang Y, Macedo JAP, Meng C, Tong G, Wang X, Ortiz-Reyes L, Caille A, Li F, Taljaard M. Adherence to key recommendations for design and analysis of stepped-wedge cluster randomized trials: A review of trials published 2016-2022. Clin Trials 2024; 21:199-210. [PMID: 37990575 PMCID: PMC11003836 DOI: 10.1177/17407745231208397] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2023]
Abstract
BACKGROUND/AIMS The stepped-wedge cluster randomized trial (SW-CRT), in which clusters are randomized to a time at which they will transition to the intervention condition - rather than a trial arm - is a relatively new design. SW-CRTs have additional design and analytical considerations compared to conventional parallel arm trials. To inform future methodological development, including guidance for trialists and the selection of parameters for statistical simulation studies, we conducted a review of recently published SW-CRTs. Specific objectives were to describe (1) the types of designs used in practice, (2) adherence to key requirements for statistical analysis, and (3) practices around covariate adjustment. We also examined changes in adherence over time and by journal impact factor. METHODS We used electronic searches to identify primary reports of SW-CRTs published 2016-2022. Two reviewers extracted information from each trial report and its protocol, if available, and resolved disagreements through discussion. RESULTS We identified 160 eligible trials, randomizing a median (Q1-Q3) of 11 (8-18) clusters to 5 (4-7) sequences. The majority (122, 76%) were cross-sectional (almost all with continuous recruitment), 23 (14%) were closed cohorts and 15 (9%) open cohorts. Many trials had complex design features such as multiple or multivariate primary outcomes (50, 31%) or time-dependent repeated measures (27, 22%). The most common type of primary outcome was binary (51%); continuous outcomes were less common (26%). The most frequently used method of analysis was a generalized linear mixed model (112, 70%); generalized estimating equations were used less frequently (12, 8%). Among 142 trials with fewer than 40 clusters, only 9 (6%) reported using methods appropriate for a small number of clusters. Statistical analyses clearly adjusted for time effects in 119 (74%), for within-cluster correlations in 132 (83%), and for distinct between-period correlations in 13 (8%). Covariates were included in the primary analysis of the primary outcome in 82 (51%) and were most often individual-level covariates; however, clear and complete pre-specification of covariates was uncommon. Adherence to some key methodological requirements (adjusting for time effects, accounting for within-period correlation) was higher among trials published in higher versus lower impact factor journals. Substantial improvements over time were not observed although a slight improvement was observed in the proportion accounting for a distinct between-period correlation. CONCLUSIONS Future methods development should prioritize methods for SW-CRTs with binary or time-to-event outcomes, small numbers of clusters, continuous recruitment designs, multivariate outcomes, or time-dependent repeated measures. Trialists, journal editors, and peer reviewers should be aware that SW-CRTs have additional methodological requirements over parallel arm designs including the need to account for period effects as well as complex intracluster correlations.
Collapse
Affiliation(s)
- Pascale Nevins
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Mary Ryan
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
| | - Kendra Davis-Plourde
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
- Yale Center for Analytical Sciences, Yale School of Public Health, New Haven, CT, USA
| | - Yongdong Ouyang
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada
| | | | - Can Meng
- Yale Center for Analytical Sciences, Yale School of Public Health, New Haven, CT, USA
| | - Guangyu Tong
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
- Center for Methods in Implementation and Prevention Science, Yale University, New Haven, CT, USA
| | - Xueqi Wang
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
- Section of Geriatrics, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, USA
| | - Luis Ortiz-Reyes
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Agnès Caille
- Université de Tours, Université de Nantes, INSERM, SPHERE U1246, Tours, France
- INSERM CIC 1415, CHRU de Tours, Tours, France
| | - Fan Li
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
- Center for Methods in Implementation and Prevention Science, Yale University, New Haven, CT, USA
| | - Monica Taljaard
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|
5
|
Tian Z, Li F. Information content of stepped wedge designs under the working independence assumption. J Stat Plan Inference 2024; 229:106097. [PMID: 37954217 PMCID: PMC10634667 DOI: 10.1016/j.jspi.2023.106097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2023]
Abstract
The stepped wedge design is increasingly popular in pragmatic trials and implementation science research studies for evaluating system-level interventions that are perceived to be beneficial to patient populations. An important step in planning a stepped wedge design is to understand the efficiency of the treatment effect estimator and hence the power of the study. We develop several novel analytical results for designing stepped wedge cluster randomized trials analyzed through generalized estimating equations under a misspecified working independence correlation structure. We first contribute a general variance expression of the treatment effect estimator when data collection is scheduled for each cluster-period. Because resource and patient-centered considerations may intentionally call for an incomplete design with outcome data being omitted for certain cluster-periods, we further derive the information content based on the robust sandwich variance to identify data elements that may be preferentially omitted with minimum loss of precision in estimating the treatment effect. We prove that centrosymmetric pairs of cluster-periods, treatment sequences and periods have identical information content and thus contribute equally to the treatment effect estimation, as long as the true covariance structure for the cluster-period means remains centrosymmetric. Finally, we provide an example of how to obtain an incomplete stepped wedge design that admits a more efficient independence GEE estimator but requires less data collection effort. Our results elegantly extend existing ones from linear mixed models coupled with model-based variances to accommodate a misspecified independence working correlation structure through the robust sandwich variances.
Collapse
Affiliation(s)
- Zibo Tian
- Department of Biostatistics, University of Florida, Gainesville, FL, USA
| | - Fan Li
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
- Center for Methods in Implementation and Prevention Science, Yale University, New Haven, CT, USA
| |
Collapse
|
6
|
Li F, Chen X, Tian Z, Wang R, Heagerty PJ. Planning stepped wedge cluster randomized trials to detect treatment effect heterogeneity. Stat Med 2024; 43:890-911. [PMID: 38115805 DOI: 10.1002/sim.9990] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2022] [Revised: 09/22/2023] [Accepted: 11/30/2023] [Indexed: 12/21/2023]
Abstract
Stepped wedge design is a popular research design that enables a rigorous evaluation of candidate interventions by using a staggered cluster randomization strategy. While analytical methods were developed for designing stepped wedge trials, the prior focus has been solely on testing for the average treatment effect. With a growing interest on formal evaluation of the heterogeneity of treatment effects across patient subpopulations, trial planning efforts need appropriate methods to accurately identify sample sizes or design configurations that can generate evidence for both the average treatment effect and variations in subgroup treatment effects. To fill in that important gap, this article derives novel variance formulas for confirmatory analyses of treatment effect heterogeneity, that are applicable to both cross-sectional and closed-cohort stepped wedge designs. We additionally point out that the same framework can be used for more efficient average treatment effect analyses via covariate adjustment, and allows the use of familiar power formulas for average treatment effect analyses to proceed. Our results further sheds light on optimal design allocations of clusters to maximize the weighted precision for assessing both the average and heterogeneous treatment effects. We apply the new methods to the Lumbar Imaging with Reporting of Epidemiology Trial, and carry out a simulation study to validate our new methods.
Collapse
Affiliation(s)
- Fan Li
- Department of Biostatistics, Yale University School of Public Health, New Haven, Connecticut, USA
- Center for Methods in Implementation and Prevention Science, Yale University School of Public Health, New Haven, Connecticut, USA
| | - Xinyuan Chen
- Department of Mathematics and Statistics, Mississippi State University, Mississippi State, Mississippi, USA
| | - Zizhong Tian
- Department of Public Health Sciences, Pennsylvania State University College of Medicine, Hershey, Pennsylvania, USA
| | - Rui Wang
- Department of Population Medicine, Harvard Pilgrim Health Care Institute and Harvard Medical School, Boston, Massachusetts, USA
- Department of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, USA
| | - Patrick J Heagerty
- Department of Biostatistics, University of Washington, Seattle, Washington, USA
| |
Collapse
|
7
|
Ouyang Y, Hemming K, Li F, Taljaard M. Estimating intra-cluster correlation coefficients for planning longitudinal cluster randomized trials: a tutorial. Int J Epidemiol 2023; 52:1634-1647. [PMID: 37196320 PMCID: PMC10555741 DOI: 10.1093/ije/dyad062] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2022] [Accepted: 04/26/2023] [Indexed: 05/19/2023] Open
Abstract
It is well-known that designing a cluster randomized trial (CRT) requires an advance estimate of the intra-cluster correlation coefficient (ICC). In the case of longitudinal CRTs, where outcomes are assessed repeatedly in each cluster over time, estimates for more complex correlation structures are required. Three common types of correlation structures for longitudinal CRTs are exchangeable, nested/block exchangeable and exponential decay correlations-the latter two allow the strength of the correlation to weaken over time. Determining sample sizes under these latter two structures requires advance specification of the within-period ICC and cluster autocorrelation coefficient as well as the intra-individual autocorrelation coefficient in the case of a cohort design. How to estimate these coefficients is a common challenge for investigators. When appropriate estimates from previously published longitudinal CRTs are not available, one possibility is to re-analyse data from an available trial dataset or to access observational data to estimate these parameters in advance of a trial. In this tutorial, we demonstrate how to estimate correlation parameters under these correlation structures for continuous and binary outcomes. We first introduce the correlation structures and their underlying model assumptions under a mixed-effects regression framework. With practical advice for implementation, we then demonstrate how the correlation parameters can be estimated using examples and we provide programming code in R, SAS, and Stata. An Rshiny app is available that allows investigators to upload an existing dataset and obtain the estimated correlation parameters. We conclude by identifying some gaps in the literature.
Collapse
Affiliation(s)
- Yongdong Ouyang
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
| | - Karla Hemming
- Institute of Applied Health Research, The University of Birmingham, Birmingham, UK
| | - Fan Li
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
- Center for Methods in Implementation and Prevention Science, Yale School of Public Health, New Haven, CT, USA
| | - Monica Taljaard
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
| |
Collapse
|
8
|
Li F, Kasza J, Turner EL, Rathouz PJ, Forbes AB, Preisser JS. Generalizing the information content for stepped wedge designs: A marginal modeling approach. Scand Stat Theory Appl 2023; 50:1048-1067. [PMID: 37601275 PMCID: PMC10434823 DOI: 10.1111/sjos.12615] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/10/2022] [Accepted: 09/02/2022] [Indexed: 11/30/2022]
Abstract
Stepped wedge trials are increasingly adopted because practical constraints necessitate staggered roll-out. While a complete design requires clusters to collect data in all periods, resource and patient-centered considerations may call for an incomplete stepped wedge design to minimize data collection burden. To study incomplete designs, we expand the metric of information content to discrete outcomes. We operate under a marginal model with general link and variance functions, and derive information content expressions when data elements (cells, sequences, periods) are omitted. We show that the centrosymmetric patterns of information content can hold for discrete outcomes with the variance-stabilizing link function. We perform numerical studies under the canonical link function, and find that while the patterns of information content for cells are approximately centrosymmetric for all examined underlying secular trends, the patterns of information content for sequences or periods are more sensitive to the secular trend, and may be far from centrosymmetric.
Collapse
Affiliation(s)
- Fan Li
- Department of Biostatistics, Yale University School of Public Health, New Haven, Connecticut, USA
- Center for Methods in Implementation and Prevention Science, Yale University, New Haven, Connecticut, USA
| | - Jessica Kasza
- School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia
| | - Elizabeth L. Turner
- Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, USA
| | - Paul J. Rathouz
- Department of Population Health, The University of Texas at Austin, Austin, Texas, USA
| | - Andrew B. Forbes
- School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia
| | - John S. Preisser
- Department of Epidemiology, Gillings School of Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| |
Collapse
|
9
|
Meng C, Ryan M, Rathouz PJ, Turner EL, Preisser JS, Li F. ORTH.Ord: An R package for analyzing correlated ordinal outcomes using alternating logistic regressions with orthogonalized residuals. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 237:107567. [PMID: 37207384 DOI: 10.1016/j.cmpb.2023.107567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/22/2023] [Revised: 03/31/2023] [Accepted: 04/21/2023] [Indexed: 05/21/2023]
Abstract
BACKGROUND AND OBJECTIVES Marginal models with generalized estimating equations (GEE) are usually recommended for analyzing correlated ordinal outcomes which are commonly seen in a longitudinal study or clustered randomized trial (CRT). Within-cluster association is often of interest in longitudinal studies or CRTs, and can be estimated with paired estimating equations. However, the estimators for within-cluster association parameters and variances may be subject to finite-sample biases when the number of clusters is small. The objective of this article is to introduce a newly developed R package ORTH.Ord for analyzing correlated ordinal outcomes using GEE models with finite-sample bias corrections. METHODS The R package ORTH.Ord implements a modified version of alternating logistic regressions with estimation based on orthogonalized residuals (ORTH), which use paired estimating equations to jointly estimate parameters in marginal mean and association models. The within-cluster association between ordinal responses is modeled by global pairwise odds ratios (POR). The R package also provides a finite-sample bias correction to POR parameter estimates based on matrix multiplicative adjusted orthogonalized residuals (MMORTH) for correcting estimating equations, and bias-corrected sandwich estimators with different options for covariance estimation. RESULTS A simulation study shows that MMORTH provides less biased global POR estimates and coverage of their 95% confidence intervals closer to the nominal level than uncorrected ORTH. An analysis of patient-reported outcomes from an orthognathic surgery clinical trial illustrates features of ORTH.Ord. CONCLUSIONS This article provides an overview of the ORTH method with bias-correction on both estimating equations and sandwich estimators for analyzing correlated ordinal data, describes the features of the ORTH.Ord R package, evaluates the performance of the package using a simulation study, and finally illustrates its application in an analysis of a clinical trial.
Collapse
Affiliation(s)
- Can Meng
- Yale Center for Analytical Sciences, Yale School of Public Health, New Haven, 06511, CT, USA; Department of Biostatistics, Yale School of Public Health, New Haven, 06511, CT, USA.
| | - Mary Ryan
- Yale Center for Analytical Sciences, Yale School of Public Health, New Haven, 06511, CT, USA; Department of Biostatistics, Yale School of Public Health, New Haven, 06511, CT, USA
| | - Paul J Rathouz
- Department of Population Health, University of Texas at Austin, Austin, 78712, TX, USA
| | - Elizabeth L Turner
- Department of Biostatistics and Bioinformatics, Duke University, Durham, 27710, NC, USA
| | - John S Preisser
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, 27599, NC, USA
| | - Fan Li
- Yale Center for Analytical Sciences, Yale School of Public Health, New Haven, 06511, CT, USA; Department of Biostatistics, Yale School of Public Health, New Haven, 06511, CT, USA; Center for Methods in Implementation and Prevention Science, Yale School of Public Health, New Haven, 06511, CT, USA
| |
Collapse
|
10
|
Tong J, Li F, Harhay MO, Tong G. Accounting for expected attrition in the planning of cluster randomized trials for assessing treatment effect heterogeneity. BMC Med Res Methodol 2023; 23:85. [PMID: 37024809 PMCID: PMC10077680 DOI: 10.1186/s12874-023-01887-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2022] [Accepted: 03/10/2023] [Indexed: 04/08/2023] Open
Abstract
BACKGROUND Detecting treatment effect heterogeneity is an important objective in cluster randomized trials and implementation research. While sample size procedures for testing the average treatment effect accounting for participant attrition assuming missing completely at random or missing at random have been previously developed, the impact of attrition on the power for detecting heterogeneous treatment effects in cluster randomized trials remains unknown. METHODS We provide a sample size formula for testing for a heterogeneous treatment effect assuming the outcome is missing completely at random. We also propose an efficient Monte Carlo sample size procedure for assessing heterogeneous treatment effect assuming covariate-dependent outcome missingness (missing at random). We compare our sample size methods with the direct inflation method that divides the estimated sample size by the mean follow-up rate. We also evaluate our methods through simulation studies and illustrate them with a real-world example. RESULTS Simulation results show that our proposed sample size methods under both missing completely at random and missing at random provide sufficient power for assessing heterogeneous treatment effect. The proposed sample size methods lead to more accurate sample size estimates than the direct inflation method when the missingness rate is high (e.g., ≥ 30%). Moreover, sample size estimation under both missing completely at random and missing at random is sensitive to the missingness rate, but not sensitive to the intracluster correlation coefficient among the missingness indicators. CONCLUSION Our new sample size methods can assist in planning cluster randomized trials that plan to assess a heterogeneous treatment effect and participant attrition is expected to occur.
Collapse
Affiliation(s)
- Jiaqi Tong
- Department of Biostatistics, Yale School of Public Health, 135 College Street, CT, New Haven, 06510, USA
| | - Fan Li
- Department of Biostatistics, Yale School of Public Health, 135 College Street, CT, New Haven, 06510, USA
- Center for Methods in Implementation and Prevention Science, Yale School of Public Health, New Haven, CT, USA
| | - Michael O Harhay
- Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Guangyu Tong
- Department of Biostatistics, Yale School of Public Health, 135 College Street, CT, New Haven, 06510, USA.
- Center for Methods in Implementation and Prevention Science, Yale School of Public Health, New Haven, CT, USA.
| |
Collapse
|
11
|
Zhang Y, Preisser JS, Li F, Turner EL, Toles M, Rathouz PJ. GEEMAEE: A SAS macro for the analysis of correlated outcomes based on GEE and finite-sample adjustments with application to cluster randomized trials. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 230:107362. [PMID: 36709555 PMCID: PMC10037297 DOI: 10.1016/j.cmpb.2023.107362] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Revised: 01/13/2023] [Accepted: 01/17/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVES Generalized estimating equations (GEE) are used to analyze correlated outcomes in marginal regression models with population-averaged interpretations of exposure effects. Limitations of popular software for GEE include: (i) user choice is restricted to a small set of within-cluster pairwise correlation (intra-class correlation; ICC) structures; and (ii) inference on ICC parameters is usually not possible because the precision of their estimates is not quantified. This is important because ICC values inform the design of cluster randomized trials. Beyond the standard GEE implementation, use of paired estimating equations (Prentice 1988) provides: (i) flexible specification of models for pairwise correlations and (ii) standard errors for ICC estimates. However, most GEEs give biased estimates of standard errors and correlations when the number of clusters is small (roughly, ≤40). Consequently, there is a need for software to provide GEE analysis with finite-sample bias-corrections. METHODS The SAS macro GEEMAEE implements paired estimating equations to simultaneously estimate parameters in marginal mean and ICC models. It provides bias-corrected standard errors and uses matrix-adjusted estimating equations (MAEE) for bias-corrected estimation of correlations. Several built-in correlation matrix options, rarely found in software, are offered for multi-period, cluster randomized trials and similarly structured longitudinal observational data structures. Additional options include user-specified correlation structures and deletion diagnostics, namely Cooks' Distance and DBETA statistics that estimate the influence of observations, cluster-periods (when applicable) and clusters. RESULTS GEEMAEE is illustrated for a binary and a count outcome in two stepped wedge cluster randomized trials and a binary outcome in a longitudinal study of disease surveillance. Use of MAEE resulted in larger values of correlation estimates compared to uncorrected estimating equations. Use of bias-corrected variance estimators resulted in (appropriately) larger values of standard errors compared to the usual sandwich estimators. Deletion diagnostics identified the clusters and cluster-periods having the most influence. CONCLUSIONS The SAS macro GEEMAEE provides regression analysis for clustered or longitudinal responses, and is particularly useful when the number of clusters is small. Flexible specification and bias-corrected estimation of pairwise correlation parameters and standard errors are key features of the software to provide valid inference in real-world settings.
Collapse
Affiliation(s)
- Ying Zhang
- Department of Biostatistics, University of North Carolina, Chapel Hill, NC, 27514, U.S.A.
| | - John S Preisser
- Department of Biostatistics, University of North Carolina, Chapel Hill, NC, 27514, U.S.A
| | - Fan Li
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, U.S.A; Center for Methods in Implementation and Prevention Science, Yale School of Public Health, New Haven, CT, U.S.A
| | - Elizabeth L Turner
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, U.S.A
| | - Mark Toles
- School of Nursing, University of North Carolina, Chapel Hill, NC, U.S.A
| | - Paul J Rathouz
- Department of Population Health, The University of Texas at Austin, Austin, TX, U.S.A
| |
Collapse
|
12
|
Zhang Y, Preisser JS, Turner EL, Rathouz PJ, Toles M, Li F. A general method for calculating power for GEE analysis of complete and incomplete stepped wedge cluster randomized trials. Stat Methods Med Res 2023; 32:71-87. [PMID: 36253078 DOI: 10.1177/09622802221129861] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
Stepped wedge designs have uni-directional crossovers at randomly assigned time points (steps) where clusters switch from control to intervention condition. Incomplete stepped wedge designs are increasingly used in cluster randomized trials of health care interventions and have periods without data collection due to logistical, resource and patient-centered considerations. The development of sample size formulae for stepped wedge trials has primarily focused on complete designs and continuous responses. Addressing this gap, a general, fast, non-simulation based power procedure is proposed for generalized estimating equations analysis of complete and incomplete stepped wedge designs and its predicted power is compared to simulated power for binary and continuous responses. An extensive set of simulations for six and twelve clusters is based upon the Connect-Home trial with an incomplete stepped wedge design. Results show that empirical test size is well controlled using a t-test with bias-corrected sandwich variance estimator for as few as six clusters. Analytical power agrees well with a simulated power in scenarios with twelve clusters. For six clusters, analytical power is similar to simulated power with estimation using the correctly specified model-based variance estimator. To explore the impact of study design choice on power, the proposed fast GEE power method is applied to the Connect-Home trial design, four alternative incomplete stepped wedge designs and one complete design.
Collapse
Affiliation(s)
- Ying Zhang
- Department of Biostatistics, University of North Carolina, Chapel Hill, NC, USA
| | - John S Preisser
- Department of Biostatistics, University of North Carolina, Chapel Hill, NC, USA
| | - Elizabeth L Turner
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - Paul J Rathouz
- Department of Population Health, The University of Texas at Austin, Austin, TX, USA
| | - Mark Toles
- School of Nursing, University of North Carolina, Chapel Hill, NC, USA
| | - Fan Li
- Department of Biostatistics, Yale School of Public Health, New Haven, CT, USA
- Center for Methods in Implementation and Prevention Science, Yale School of Public Health, New Haven, CT, USA
| |
Collapse
|
13
|
Gallis JA, Wang X, Rathouz PJ, Preisser JS, Li F, Turner EL. power swgee: GEE-based power calculations in stepped wedge cluster randomized trials. THE STATA JOURNAL 2022; 22:811-841. [PMID: 36968149 PMCID: PMC10035664 DOI: 10.1177/1536867x221140953] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/18/2023]
Abstract
Stepped wedge cluster randomized trials are increasingly being used to evaluate interventions in medical, public health, educational, and social science contexts. With the longitudinal and crossover nature of a SW-CRT, complex analysis techniques are often needed which makes appropriately powering SW-CRTs challenging. In this paper, we introduce a newly-developed SW-CRT power calculator, embedded within the power command in Stata. The power calculator assumes a marginal model (i.e., generalized estimating equations [GEE]) for the primary analysis of SW-CRTs, for which other currently available SW-CRT power calculators may not be suitable. The program accommodates complete cross-sectional and closed-cohort designs, and includes multilevel correlation structures appropriate for such designs. We discuss the methods and formulae underlying our SW-CRT calculator, and provide illustrative examples of the use of power swgee. We provide suggestions about the choice of parameters in power swgee, and conclude by discussing areas of future research which may improve the program.
Collapse
Affiliation(s)
- John A Gallis
- Department of Biostatistics, Duke University, Duke Global Health Institute, Durham, NC
| | - Xueqi Wang
- Department of Biostatistics, Duke University, Duke Global Health Institute, Durham, NC
| | - Paul J Rathouz
- Department of Population Health, University of Texas at Austin, Dell Medical School, Austin, TX
| | - John S Preisser
- Department of Biosttistics, University of North Carolina at Chapel Hill, Gillings School of Global Public Health, Chapel Hill, NC
| | - Fan Li
- Department of Biostatistics, Yale School of Public Health, Center for Methods in Implementation, Prevention Science, New Haven, CT
| | - Elizabeth L Turner
- Department of Biostatistics, Duke University, Duke Global Health Institute, Durham, NC
| |
Collapse
|
14
|
Rezaei-Darzi E, Kasza J, Forbes A, Bowden R. Use of information criteria for selecting a correlation structure for longitudinal cluster randomised trials. Clin Trials 2022; 19:316-325. [PMID: 35706343 DOI: 10.1177/17407745221082227] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
BACKGROUND When designing and analysing longitudinal cluster randomised trials, such as the stepped wedge, the similarity of outcomes from the same cluster must be accounted for through the choice of a form for the within-cluster correlation structure. Several choices for this structure are commonly considered for application within the linear mixed model paradigm. The first assumes a constant intra-cluster correlation for all pairs of outcomes from the same cluster (the exchangeable/Hussey and Hughes model); the second assumes that correlations of outcomes measured in the same period are higher than outcomes measured in different periods (the block exchangeable model) and the third is the discrete-time decay model, which allows the correlation between pairs of outcomes to decay over time. Currently, there is limited guidance on how to select the most appropriate within-cluster correlation structure. METHODS We simulated continuous outcomes under each of the three considered within-cluster correlation structures for a range of design and parameter choices, and, using the ASReml-R package, fit each linear mixed model to each simulated dataset. We evaluated the performance of the Akaike and Bayesian information criteria for selecting the correct within-cluster correlation structure for each dataset. RESULTS For smaller total sample sizes, neither criteria performs particularly well in selecting the correct within-cluster correlation structure, with the simpler exchangeable model being favoured. Furthermore, in general, the Bayesian information criterion favours the exchangeable model. When the cluster auto-correlation (which defines the degree of dependence between observations in adjacent time periods) is large and number of periods is small, neither criteria is able to distinguish between the block exchangeable and discrete time decay models. However, for increasing numbers of clusters, periods, and subjects per cluster period, both the Akaike and Bayesian information criteria perform increasingly well in the detection of the correct within-cluster correlation structure. CONCLUSIONS With increasing amounts of data, be they number of clusters, periods or subjects per cluster period, both the Akaike and Bayesian information criteria are increasingly likely to select the correct correlation structure. We recommend that if there are sufficient data available when planning a trial, that the Akaike or Bayesian information criterion is used to guide the choice of within-cluster correlation structure in the absence of other compelling justifications for a specific correlation structure. We also suggest that researchers conduct supplementary analyses under alternate correlation structures to gauge sensitivity to the initial choice.
Collapse
Affiliation(s)
- Ehsan Rezaei-Darzi
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia.,Monash University Accident Research Centre, Monash University, Clayton, VIC, Australia
| | - Jessica Kasza
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia
| | - Andrew Forbes
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia
| | - Rhys Bowden
- School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC, Australia
| |
Collapse
|
15
|
Stepped Wedge Cluster Randomized Trials: A Methodological Overview. World Neurosurg 2022; 161:323-330. [PMID: 35505551 PMCID: PMC9074087 DOI: 10.1016/j.wneu.2021.10.136] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Revised: 10/17/2021] [Accepted: 10/18/2021] [Indexed: 11/21/2022]
Abstract
BACKGROUND Stepped wedge cluster randomized trials enable rigorous evaluations of health intervention programs in pragmatic settings. In the present study, we aimed to update neurosurgeon scientists on the design of stepped wedge randomized trials. METHODS We have presented an overview of recent methodological developments for stepped wedge designs and included an update on the newer associated methodological tools to aid with future study designs. RESULTS We defined the stepped wedge trial design and reviewed the indications for the design in depth. In addition, key considerations, including mainstream methods of analysis and sample size determination, were discussed. CONCLUSIONS Stepped wedge designs can be attractive for study intervention programs aiming to improve the delivery of patient care, especially when examining a small number of heterogeneous clusters.
Collapse
|
16
|
Golzarri-Arroyo L, Dickinson SL, Jamshidi-Naeini Y, Zoh RS, Brown AW, Owora AH, Li P, Oakes JM, Allison DB. Evaluation of the type I error rate when using parametric bootstrap analysis of a cluster randomized controlled trial with binary outcomes and a small number of clusters. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 215:106654. [PMID: 35093646 PMCID: PMC8847311 DOI: 10.1016/j.cmpb.2022.106654] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Revised: 12/30/2021] [Accepted: 01/20/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND Cluster randomized controlled trials (cRCTs) are increasingly used but must be analyzed carefully. We conducted a simulation study to evaluate the validity of a parametric bootstrap (PB) approach with respect to the empirical type I error rate for a cRCT with binary outcomes and a small number of clusters. METHODS We simulated a case study with a binary (0/1) outcome, four clusters, and 100 subjects per cluster. To compare the validity of the test with respect to error rate, we simulated the same experiment with K=10, 20, and 30 clusters, each with 2,000 simulated datasets. To test the null hypothesis, we used a generalized linear mixed model including a random intercept for clusters and obtained p-values based on likelihood ratio tests (LRTs) using the parametric bootstrap method as implemented in the R package "pbkrtest". RESULTS The PB test produced error rates of 9.1%, 5.5%, 4.9%, and 5.0% on average across all ICC values for K=4, K=10, K=20, and K=30, respectively. The error rates were higher, ranging from 9.1% to 36.5% for K=4, in the models with singular fits (i.e., ignoring clustering) because the ICC was estimated to be zero. CONCLUSION Using the parametric bootstrap for cRCTs with a small number of clusters results in inflated error rates and is not valid.
Collapse
Affiliation(s)
- Lilian Golzarri-Arroyo
- Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington USA.
| | - Stephanie L Dickinson
- Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington USA
| | - Yasaman Jamshidi-Naeini
- Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington USA
| | - Roger S Zoh
- Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington USA
| | - Andrew W Brown
- Department of Applied Health Science, Indiana University School of Public Health-Bloomington USA
| | - Arthur H Owora
- Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington USA
| | - Peng Li
- School of Nursing, University of Alabama at Birmingham USA
| | | | - David B Allison
- Department of Epidemiology and Biostatistics, Indiana University School of Public Health-Bloomington USA
| |
Collapse
|
17
|
Tian Z, Preisser JS, Esserman D, Turner EL, Rathouz PJ, Li F. Impact of unequal cluster sizes for GEE analyses of stepped wedge cluster randomized trials with binary outcomes. Biom J 2021; 64:419-439. [PMID: 34596912 PMCID: PMC9292617 DOI: 10.1002/bimj.202100112] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2021] [Revised: 07/15/2021] [Accepted: 08/07/2021] [Indexed: 12/31/2022]
Abstract
The stepped wedge (SW) design is a type of unidirectional crossover design where cluster units switch from control to intervention condition at different prespecified time points. While a convention in study planning is to assume the cluster‐period sizes are identical, SW cluster randomized trials (SW‐CRTs) involving repeated cross‐sectional designs frequently have unequal cluster‐period sizes, which can impact the efficiency of the treatment effect estimator. In this paper, we provide a comprehensive investigation of the efficiency impact of unequal cluster sizes for generalized estimating equation analyses of SW‐CRTs, with a focus on binary outcomes as in the Washington State Expedited Partner Therapy trial. Several major distinctions between our work and existing work include the following: (i) we consider multilevel correlation structures in marginal models with binary outcomes; (ii) we study the implications of both the between‐cluster and within‐cluster imbalances in sizes; and (iii) we provide a comparison between the independence working correlation versus the true working correlation and detail the consequences of ignoring correlation estimation in SW‐CRTs with unequal cluster sizes. We conclude that the working independence assumption can lead to substantial efficiency loss and a large sample size regardless of cluster‐period size variability in SW‐CRTs, and recommend accounting for correlations in the analysis. To improve study planning, we additionally provide a computationally efficient search algorithm to estimate the sample size in SW‐CRTs accounting for unequal cluster‐period sizes, and conclude by illustrating the proposed approach in the context of the Washington State study.
Collapse
Affiliation(s)
- Zibo Tian
- Department of Biostatistics, Yale University School of Public Health, New Haven, CT, USA
| | - John S Preisser
- Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Denise Esserman
- Department of Biostatistics, Yale University School of Public Health, New Haven, CT, USA.,Yale Center for Analytical Sciences, New Haven, CT, USA
| | - Elizabeth L Turner
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA.,Duke Global Health Institute, Durham, NC, USA
| | - Paul J Rathouz
- Department of Population Health, The University of Texas at Austin, Austin, TX, USA
| | - Fan Li
- Department of Biostatistics, Yale University School of Public Health, New Haven, CT, USA.,Yale Center for Analytical Sciences, New Haven, CT, USA.,Center for Methods in Implementation and Prevention Science, Yale University, New Haven, CT, USA
| |
Collapse
|
18
|
Zhan D, Xu L, Ouyang Y, Sawatzky R, Wong H. Methods for dealing with unequal cluster sizes in cluster randomized trials: A scoping review. PLoS One 2021; 16:e0255389. [PMID: 34324593 PMCID: PMC8320970 DOI: 10.1371/journal.pone.0255389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2020] [Accepted: 07/15/2021] [Indexed: 11/18/2022] Open
Abstract
In a cluster-randomized trial (CRT), the number of participants enrolled often varies across clusters. This variation should be considered during both trial design and data analysis to ensure statistical performance goals are achieved. Most methodological literature on the CRT design has assumed equal cluster sizes. This scoping review focuses on methodology for unequal cluster size CRTs. EMBASE, Medline, Google Scholar, MathSciNet and Web of Science databases were searched to identify English-language articles reporting on methodology for unequal cluster size CRTs published until March 2021. We extracted data on the focus of the paper (power calculation, Type I error etc.), the type of CRT, the type and the range of parameter values investigated (number of clusters, mean cluster size, cluster size coefficient of variation, intra-cluster correlation coefficient, etc.), and the main conclusions. Seventy-nine of 5032 identified papers met the inclusion criteria. Papers primarily focused on the parallel-arm CRT (p-CRT, n = 60, 76%) and the stepped-wedge CRT (n = 14, 18%). Roughly 75% of the papers addressed trial design issues (sample size/power calculation) while 25% focused on analysis considerations (Type I error, bias, etc.). The ranges of parameter values explored varied substantially across different studies. Methods for accounting for unequal cluster sizes in the p-CRT have been investigated extensively for Gaussian and binary outcomes. Synthesizing the findings of these works is difficult as the magnitude of impact of the unequal cluster sizes varies substantially across the combinations and ranges of input parameters. Limited investigations have been done for other combinations of a CRT design by outcome type, particularly methodology involving binary outcomes-the most commonly used type of primary outcome in trials. The paucity of methodological papers outside of the p-CRT with Gaussian or binary outcomes highlights the need for further methodological development to fill the gaps.
Collapse
Affiliation(s)
- Denghuang Zhan
- School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada
- Centre for Health Evaluation and Outcomes Sciences, University of British Columbia, Vancouver, British Columbia, Canada
| | - Liang Xu
- School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada
- Centre for Health Evaluation and Outcomes Sciences, University of British Columbia, Vancouver, British Columbia, Canada
| | - Yongdong Ouyang
- School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada
- Centre for Health Evaluation and Outcomes Sciences, University of British Columbia, Vancouver, British Columbia, Canada
| | - Richard Sawatzky
- Centre for Health Evaluation and Outcomes Sciences, University of British Columbia, Vancouver, British Columbia, Canada
- School of Nursing, Trinity Western University, Langley City, British Columbia, Canada
| | - Hubert Wong
- School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada
- Centre for Health Evaluation and Outcomes Sciences, University of British Columbia, Vancouver, British Columbia, Canada
| |
Collapse
|