R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 10

In this latest update we discuss the transportability of comparative effectiveness evidence across countries. We highlight results of a survey indicating that European HTA agencies are reluctant to accept real-world data from other countries, review recent benefit assessments indicating a potential softening of a requirement for the use of local real-world data in Germany, and outline a recent review presenting approaches that can correct for a lack of transportability.

A calibration approach to transportability and data-fusion with observational data

Two important considerations in clinical research studies are proper evaluations of internal and external validity. While randomized clinical trials can overcome several threats to internal validity, they may be prone to poor external validity. Conversely, large prospective observational studies sampled from a broadly generalizable population may be externally valid, yet susceptible to threats to internal validity, particularly confounding. Thus, methods that address confounding and enhance transportability of study results across populations are essential for internally and externally valid causal inference, respectively. These issues persist for another problem closely related to transportability known as data-fusion. We develop a calibration method to generate balancing weights that address confounding and sampling bias, thereby enabling valid estimation of the target population average treatment effect. We compare the calibration approach to two additional doubly robust methods that estimate the effect of an intervention on an outcome within a second, possibly unrelated target population. The proposed methodologies can be extended to resolve data-fusion problems that seek to evaluate the effects of an intervention using data from two related studies sampled from different populations. A simulation study is conducted to demonstrate the advantages and similarities of the different techniques. We also test the performance of the calibration approach in a motivating real data example comparing whether the effect of biguanides vs sulfonylureas-the two most common oral diabetes medication classes for initial treatment-on all-cause mortality described in a historical cohort applies to a contemporary cohort of US Veterans with diabetes.

The Future of Causal Inference

The past several decades have seen exponential growth in causal inference approaches and their applications. In this commentary, we provide our top-10 list of emerging and exciting areas of research in causal inference. These include methods for high-dimensional data and precision medicine, causal machine learning, causal discovery, and others. These methods are not meant to be an exhaustive list; instead, we hope that this list will serve as a springboard for stimulating the development of new research.

Characterizing real world safety profile of oral Janus kinase inhibitors among adult atopic dermatitis patients: evidence transporting from the rheumatoid arthritis population

OBJECTIVE

To address potential safety concerns of Janus Kinase Inhibitors (JAK-Is), we characterized their safety profile in the atopic dermatitis (AD) patient population.

METHODS

In this retrospective observational study, we used propensity score-based methods and a Poisson modeling framework to estimate the incidence of health outcomes of interest (HOI) for the AD patient. To that end, two mutually exclusive cohorts were created using a real world data resource: a rheumatoid arthritis (RA) cohort, where we directly quantify the safety risk of JAK-Is on HOIs, and an AD cohort, that comprises the target population of interest and to whom we transport the results obtained from the RA cohort. The RA cohort included all adults who filled at least one prescription for a JAK-I (tofacitinib, baricitinib, or upadacitinib) between 1 January 2017 and 31 January 2020. The AD cohort consisted of all adults diagnosed with AD during the same period. We first estimated the incidence rate of each HOI in the RA cohort, and then transported the results to the AD population.

RESULTS

The RA and AD cohorts included 5,296 and 261,855 patients, respectively. On average, patients in the AD cohort were younger, more often male, more likely to be Asian, and had higher household income. They also had a lower prevalence of several comorbid conditions including hypertension, chronic kidney disease, obesity, and depression. Overall, the transported incidence rates of the HOIs to the AD cohort were lower than those obtained in the RA cohort by 13-50%.

CONCLUSION

We applied transportability methods to characterize the risk of the HOIs in the AD population and found absolute risks higher than that of the general population. Future work is needed to validate these conclusions in comparable populations.

Challenges in Systematic Reviews and Meta-Analyses of Mediation Analyses

Systematic reviews and meta-analyses of mediation studies are increasingly being implemented in practice. Nonetheless, the methodology for conducting such review and analysis is still in a development phase, with much room for improvement. In this paper, we highlight and discuss challenges that investigators face in systematic reviews and meta-analyses of mediation studies and propose ways of accommodating these in practice.

Emulating a target trial of intensive nurse home visiting in the policy-relevant population using linked administrative data

BACKGROUND

Populations willing to participate in randomized trials may not correspond well to policy-relevant target populations. Evidence of effectiveness that is complementary to randomized trials may be obtained by combining the 'target trial' causal inference framework with whole-of-population linked administrative data.

METHODS

We demonstrate this approach in an evaluation of the South Australian Family Home Visiting Program, a nurse home visiting programme targeting socially disadvantaged families. Using de-identified data from 2004-10 in the ethics-approved Better Evidence Better Outcomes Linked Data (BEBOLD) platform, we characterized the policy-relevant population and emulated a trial evaluating effects on child developmental vulnerability at 5 years (n = 4160) and academic achievement at 9 years (n = 6370). Linkage to seven health, welfare and education data sources allowed adjustment for 29 confounders using Targeted Maximum Likelihood Estimation (TMLE) with SuperLearner. Sensitivity analyses assessed robustness to analytical choices.

RESULTS

We demonstrated how the target trial framework may be used with linked administrative data to generate evidence for an intervention as it is delivered in practice in the community in the policy-relevant target population, and considering effects on outcomes years down the track. The target trial lens also aided in understanding and limiting the increased measurement, confounding and selection bias risks arising with such data. Substantively, we did not find robust evidence of a meaningful beneficial intervention effect.

CONCLUSIONS

This approach could be a valuable avenue for generating high-quality, policy-relevant evidence that is complementary to trials, particularly when the target populations are multiply disadvantaged and less likely to participate in trials.

A Bayesian approach for estimating the partial potential impact fraction with exposure measurement error under a main study/internal validation design

The partial potential impact fraction describes the proportion of disease cases that can be prevented if the distribution of modifiable continuous exposures is shifted in a population, while other risk factors are not modified. It is a useful quantity for evaluating the burden of disease in epidemiologic and public health studies. When exposures are measured with error, the partial potential impact fraction estimates may be biased, which necessitates methods to correct for the exposure measurement error. Motivated by the health professionals follow-up study, we develop a Bayesian approach to adjust for exposure measurement error when estimating the partial potential impact fraction under the main study/internal validation study design. We adopt the reclassification approach that leverages the strength of the main study/internal validation study design and clarifies transportability assumptions for valid inference. We assess the finite-sample performance of both the point and credible interval estimators via extensive simulations and apply the proposed approach in the health professionals follow-up study to estimate the partial potential impact fraction for colorectal cancer incidence under interventions exploring shifting the distributions of red meat, alcohol, and/or folate intake.

Transportability From Randomized Trials to Clinical Care: On Initial HIV Treatment With Efavirenz and Suicidal Thoughts or Behaviors

In an analysis of randomized trials, use of efavirenz for treatment of human immunodeficiency virus (HIV) infection was associated with increased suicidal thoughts/behaviors. However, analyses of observational data have found no evidence of increased risk. To assess whether population differences might explain this divergence, we transported the effect of efavirenz use from these trials to a specific target population. Using inverse odds weights and multiple imputation, we transported the effect of efavirenz on suicidal thoughts/behaviors in these randomized trials (participants were enrolled in 2001-2007) to a trials-eligible cohort of US adults initiating antiretroviral therapy while receiving HIV clinical care at medical centers between 1999 and 2015. Overall, 8,291 cohort participants and 3,949 trial participants were eligible. Prescription of antidepressants (19% vs. 13%) and injection drug history (16% vs. 10%) were more frequent in the cohort than in the trial participants. Compared with the effect in trials, the estimated hazard ratio for efavirenz on suicidal thoughts/behaviors was attenuated in our target population (trials: hazard ratio (HR) = 2.3 (95% confidence interval (CI): 1.2, 4.4); transported: HR = 1.8 (95% CI: 0.9, 4.4)), whereas the incidence rate difference was similar (trials: HR = 5.1 (95% CI: 1.6, 8.7); transported: HR = 5.4 (95% CI: -0.4, 11.4)). In our target population, there was greater than 20% attenuation of the hazard ratio estimate as compared with the trials-only estimate. Transporting results from trials to a target population is informative for addressing external validity.

Study Designs for Extending Causal Inferences From a Randomized Trial to a Target Population

In this article, we examine study designs for extending (generalizing or transporting) causal inferences from a randomized trial to a target population. Specifically, we consider nested trial designs, where randomized individuals are nested within a sample from the target population, and nonnested trial designs, including composite data-set designs, where observations from a randomized trial are combined with those from a separately obtained sample of nonrandomized individuals from the target population. We show that the counterfactual quantities that can be identified in each study design depend on what is known about the probability of sampling nonrandomized individuals. For each study design, we examine identification of counterfactual outcome means via the g-formula and inverse probability weighting. Last, we explore the implications of the sampling properties underlying the designs for the identification and estimation of the probability of trial participation.

Transporting Subgroup Analyses of Randomized Controlled Trials for Planning Implementation of New Interventions

Subgroup analyses of randomized controlled trials guide resource allocation and implementation of new interventions by identifying groups of individuals who are likely to benefit most from the intervention. Unfortunately, trial populations are rarely representative of the target populations of public health or clinical interest. Unless the relevant differences between trial and target populations are accounted for, subgroup results from trials might not reflect which groups in the target population will benefit most from the intervention. Transportability provides a rigorous framework for applying results derived in potentially highly selected study populations to external target populations. The method requires that researchers measure and adjust for all variables that 1) modify the effect of interest and 2) differ between the target and trial populations. To date, applications of transportability have focused on the external validity of overall study results and understanding within-trial heterogeneity; however, this approach has not yet been used for subgroup analyses of trials. Through an example from the Iniciativa Profilaxis Pre-Exposición (iPrEx) study (multiple countries, 2007-2010) of preexposure prophylaxis for human immunodeficiency virus, we illustrate how transporting subgroup analyses can produce target-specific subgroup effect estimates and numbers needed to treat. This approach could lead to more tailored and accurate guidance for resource allocation and cost-effectiveness analyses.

Efficient, doubly robust estimation of the effect of dose switching for switchers in a randomized clinical trial

Motivated by a clinical trial conducted by Janssen Pharmaceutica in which a flexible dosing regimen is compared to placebo, we evaluate how switchers in the treatment arm (i.e., patients who were switched to the higher dose) would have fared had they been kept on the low dose. This is done in order to understand whether flexible dosing is potentially beneficial for them. Simply comparing these patients' responses with those of patients who stayed on the low dose does not likely entail a satisfactory evaluation because the latter patients are usually in a better health condition. Because the available information in the considered trial is too limited to enable a reliable adjustment, we will instead transport data from a fixed dosing trial that has been conducted concurrently on the same target, albeit not in an identical patient population. In particular, we propose an estimator that relies on an outcome model, a model for switching, and a propensity score model for the association between study and patient characteristics. The proposed estimator is asymptotically unbiased if either the outcome or the propensity score model is correctly specified, and efficient (under the semiparametric model where the randomization probabilities are known and independent of baseline covariates) when all models are correctly specified. The proposed method for transporting information from an external study is more broadly applicable in studies where a classical confounding adjustment is not possible due to near positivity violation (e.g., studies where switching takes place in a (near) deterministic manner). Monte Carlo simulations and application to the motivating study demonstrate adequate performance.

Fusion designs and estimators for treatment effects

While randomized trials remain the best evidence for treatment effectiveness, lack of generalizability often remains an important concern. Additionally, when new treatments are compared against existing standards of care, the potentially small benefit of the new treatment may be difficult to detect in a trial without extremely large sample sizes and long follow-up times. Recent advances in "data fusion" provide a framework to combine results across studies that are applicable to a given population of interest and allow treatment comparisons that may not be feasible with traditional study designs. We propose a data fusion-based estimator that can be used to combine information from two studies: (1) a study comparing a new treatment to the standard of care in the local population of interest, and (2) a study comparing the standard of care to placebo in a separate, distal population. We provide conditions under which the parameter of interest can be identified from the two studies described and explore properties of the estimator through simulation. Finally, we apply the estimator to estimate the effect of triple- vs monotherapy for the treatment of HIV using data from two randomized trials. The proposed estimator can account for underlying population structures that induce differences in case mix, adherence, and outcome prevalence between the local and distal populations, and the estimator can also account for potentially informative loss to follow-up. Approaches like those detailed here are increasingly important to speed the approval and adoption of effective new therapies by leveraging multiple sources of information.

Implementation of the COVID-19 Vulnerability Index Across an International Network of Health Care Data Sets: Collaborative External Validation Study

Background

SARS-CoV-2 is straining health care systems globally. The burden on hospitals during the pandemic could be reduced by implementing prediction models that can discriminate patients who require hospitalization from those who do not. The COVID-19 vulnerability (C-19) index, a model that predicts which patients will be admitted to hospital for treatment of pneumonia or pneumonia proxies, has been developed and proposed as a valuable tool for decision-making during the pandemic. However, the model is at high risk of bias according to the “prediction model risk of bias assessment” criteria, and it has not been externally validated.

Objective

The aim of this study was to externally validate the C-19 index across a range of health care settings to determine how well it broadly predicts hospitalization due to pneumonia in COVID-19 cases.

Methods

We followed the Observational Health Data Sciences and Informatics (OHDSI) framework for external validation to assess the reliability of the C-19 index. We evaluated the model on two different target populations, 41,381 patients who presented with SARS-CoV-2 at an outpatient or emergency department visit and 9,429,285 patients who presented with influenza or related symptoms during an outpatient or emergency department visit, to predict their risk of hospitalization with pneumonia during the following 0-30 days. In total, we validated the model across a network of 14 databases spanning the United States, Europe, Australia, and Asia.

Results

The internal validation performance of the C-19 index had a C statistic of 0.73, and the calibration was not reported by the authors. When we externally validated it by transporting it to SARS-CoV-2 data, the model obtained C statistics of 0.36, 0.53 (0.473-0.584) and 0.56 (0.488-0.636) on Spanish, US, and South Korean data sets, respectively. The calibration was poor, with the model underestimating risk. When validated on 12 data sets containing influenza patients across the OHDSI network, the C statistics ranged between 0.40 and 0.68.

Conclusions

Our results show that the discriminative performance of the C-19 index model is low for influenza cohorts and even worse among patients with COVID-19 in the United States, Spain, and South Korea. These results suggest that C-19 should not be used to aid decision-making during the COVID-19 pandemic. Our findings highlight the importance of performing external validation across a range of settings, especially when a prediction model is being extrapolated to a different population. In the field of prediction, extensive validation is required to create appropriate trust in a model.

Transporting stochastic direct and indirect effects to new populations

Transported mediation effects may contribute to understanding how interventions work differently when applied to new populations. However, we are not aware of any estimators for such effects. Thus, we propose two doubly robust, efficient estimators of transported stochastic (also called randomized interventional) direct and indirect effects. We demonstrate their finite sample properties in a simulation study. We then apply the preferred substitution estimator to longitudinal data from the Moving to Opportunity Study, a large-scale housing voucher experiment, to transport stochastic indirect effect estimates of voucher receipt in childhood on subsequent risk of mental health or substance use disorder mediated through parental employment across sites, thereby gaining understanding of drivers of the site differences.

Calibrating validation samples when accounting for measurement error in intervention studies

Many lifestyle intervention trials depend on collecting self-reported outcomes, such as dietary intake, to assess the intervention's effectiveness. Self-reported outcomes are subject to measurement error, which impacts treatment effect estimation. External validation studies measure both self-reported outcomes and accompanying biomarkers, and can be used to account for measurement error. However, in order to account for measurement error using an external validation sample, an assumption must be made that the inferences are transportable from the validation sample to the intervention trial of interest. This assumption does not always hold. In this paper, we propose an approach that adjusts the validation sample to better resemble the trial sample, and we also formally investigate when bias due to poor transportability may arise. Lastly, we examine the performance of the methods using simulation, and illustrate them using PREMIER, a lifestyle intervention trial measuring self-reported sodium intake as an outcome, and OPEN, a validation study measuring both self-reported diet and urinary biomarkers.

Generalizing randomized trial findings to a target population using complex survey population data

Randomized trials are considered the gold standard for estimating causal effects. Trial findings are often used to inform policy and programming efforts, yet their results may not generalize well to a relevant target population due to potential differences in effect moderators between the trial and population. Statistical methods have been developed to improve generalizability by combining trials and population data, and weighting the trial to resemble the population on baseline covariates. Large-scale surveys in fields such as health and education with complex survey designs are a logical source for population data; however, there is currently no best practice for incorporating survey weights when generalizing trial findings to a complex survey. We propose and investigate ways to incorporate survey weights in this context. We examine the performance of our proposed estimator through simulations in comparison to estimators that ignore the complex survey design. We then apply the methods to generalize findings from two trials-a lifestyle intervention for blood pressure reduction and a web-based intervention to treat substance use disorders-to their respective target populations using population data from complex surveys. The work highlights the importance in properly accounting for the complex survey design when generalizing trial findings to a population represented by a complex survey sample.

Extending inferences from a randomized trial to a new target population

When treatment effect modifiers influence the decision to participate in a randomized trial, the average treatment effect in the population represented by the randomized individuals will differ from the effect in other populations. In this tutorial, we consider methods for extending causal inferences about time-fixed treatments from a trial to a new target population of nonparticipants, using data from a completed randomized trial and baseline covariate data from a sample from the target population. We examine methods based on modeling the expectation of the outcome, the probability of participation, or both (doubly robust). We compare the methods in a simulation study and show how they can be implemented in software. We apply the methods to a randomized trial nested within a cohort of trial-eligible patients to compare coronary artery surgery plus medical therapy versus medical therapy alone for patients with chronic coronary artery disease. We conclude by discussing issues that arise when using the methods in applied analyses.

Estimating treatment importance in multidrug-resistant tuberculosis using Targeted Learning: An observational individual patient data network meta-analysis

Persons with multidrug-resistant tuberculosis (MDR-TB) have a disease resulting from a strain of tuberculosis (TB) that does not respond to at least isoniazid and rifampicin, the two most effective anti-TB drugs. MDR-TB is always treated with multiple antimicrobial agents. Our data consist of individual patient data from 31 international observational studies with varying prescription practices, access to medications, and distributions of antibiotic resistance. In this study, we develop identifiability criteria for the estimation of a global treatment importance metric in the context where not all medications are observed in all studies. With stronger causal assumptions, this treatment importance metric can be interpreted as the effect of adding a medication to the existing treatments. We then use this metric to rank 15 observed antimicrobial agents in terms of their estimated add-on value. Using the concept of transportability, we propose an implementation of targeted maximum likelihood estimation, a doubly robust and locally efficient plug-in estimator, to estimate the treatment importance metric. A clustered sandwich estimator is adopted to compute variance estimates and produce confidence intervals. Simulation studies are conducted to assess the performance of our estimator, verify the double robustness property, and assess the appropriateness of the variance estimation approach.

Correcting for selective participation in cohort studies using auxiliary register data without identification of non-participants

Aims: Selective participation may hamper the validity of population-based cohort studies. The resulting bias can be alleviated by linking auxiliary register data to both the participants and the non-participants of the study, estimating propensity scores for participation and correcting for participation based on these. However, registry holders may not be allowed to disclose sensitive data on (invited) non-participants. Our aim is to provide guidance on how adequate bias correction can be achieved by using auxiliary register data but without disclosing information that could be linked to the subset of non-participants. Methods: We show how existing methods can be used to estimate generalisation weights under various data disclosure scenarios where invited non-participants are indistinguishable from uninvited ones. We also demonstrate how the methods can be implemented using Nordic register data. Results: Inverse-probability-of-sampling weights estimated within a random sample of the target population in which the non-respondents are disclosed are equivalent in expectation to analogous weights in a scenario where the non-participants and uninvited individuals from the population are indistinguishable. To minimise the risk of disclosure when the entire population is invited to participate, investigators should instead consider inverse-odds-of-sampling weights, a method that has previously been suggested for transporting study results to external populations. Conclusions: Generalisation weights can be estimated from auxiliary register data without disclosing information on invited non-participants.

A novel approach for identifying and addressing case-mix heterogeneity in individual participant data meta-analysis

Case-mix heterogeneity across studies complicates meta-analyses. As a result of this, treatments that are equally effective on patient subgroups may appear to have different effectiveness on patient populations with different case mix. It is therefore important that meta-analyses be explicit for what patient population they describe the treatment effect. To achieve this, we develop a new approach for meta-analysis of randomized clinical trials, which use individual patient data (IPD) from all trials to infer the treatment effect for the patient population in a given trial, based on direct standardization using either outcome regression (OCR) or inverse probability weighting (IPW). Accompanying random-effect meta-analysis models are developed. The new approach enables disentangling heterogeneity due to case mix from that due to beyond case-mix reasons.

Generalizing Intensive Blood Pressure Treatment to Adults With Diabetes Mellitus

BACKGROUND

Controversy over blood pressure (BP) treatment targets for individuals with diabetes is in part due to conflicting perspectives about generalizability of available trial data.

OBJECTIVE

The authors sought to estimate how results from the largest clinical trial of intensive BP treatment among adults with diabetes would generalize to the U.S.

POPULATION

METHODS

The authors used transportability methods to reweight individual patient data from the ACCORD (Action to Control Cardiovascular Risk in Diabetes) BP trial (N = 4,507) of intensive (goal systolic BP <120 mm Hg) versus standard (goal systolic BP <140 mm Hg) treatment to better represent the demographic and clinical risk factors of the U.S. population of adults with diabetes (data from NHANES [National Health and Nutrition Examination Survey] 2005 to 2014, n = 1,943). The primary outcome was the first occurrence of nonfatal myocardial infarction, nonfatal stroke, or cardiovascular death. Analysis used weighted Cox proportional hazards regression models with robust standard errors.

RESULTS

The ACCORD BP sample had less racial/ethnic diversity and more elevated cardiovascular risk factors than the NHANES participants. Weighted results significantly favored intensive BP treatment, unlike unweighted results (hazard ratio for primary outcome in intensive versus standard treatment in weighted analyses: 0.67, 95% confidence interval: 0.49 to 0.91; in unweighted analyses: hazard ratio: 0.88, 95% confidence interval: 0.73 to 1.07). Over 5 years, the weighted results estimate a number needed to treat of 34, and number needed to harm of 55.

CONCLUSIONS

After reweighting to better reflect the U.S. adult population with diabetes, intensive BP therapy was associated with significantly lower risk for cardiovascular events. However, data were limited among racial/ethnic minorities and those with lower cardiovascular risk.

Lessons Learned About Barriers to Implementing School-Based Interventions for Adolescents: Ideas for Enhancing Future Research and Clinical Projects

The majority of youth with mental health problems do not receive treatment, highlighting the critical need to transport evidence-based interventions into community settings, such as schools. Despite being able to reach a large number of adolescents and minority youth, the process of implementing evidence-based interventions to schools is challenging. This paper discusses some expected and unexpected challenges experienced during the implementation of an open trial and a pilot randomized controlled trial examining the acceptability and effectiveness of a school-based preventive intervention for adolescents at risk for internalizing disorders. First, we highlight key programs and findings on preventive interventions for adolescents at risk for depression and anxiety. Next, we provide a brief overview of the preventive intervention we implemented in schools. This provides a context for the section that describes implementation issues and highlights specific challenges and potential solutions for intervention implementation. Finally, the paper offers recommendations for researchers and clinicians interested in implementing school-based mental health services for adolescents.

Generalizing causal inferences from individuals in randomized trials to all trial-eligible individuals

We consider methods for causal inference in randomized trials nested within cohorts of trial-eligible individuals, including those who are not randomized. We show how baseline covariate data from the entire cohort, and treatment and outcome data only from randomized individuals, can be used to identify potential (counterfactual) outcome means and average treatment effects in the target population of all eligible individuals. We review identifiability conditions, propose estimators, and assess the estimators' finite-sample performance in simulation studies. As an illustration, we apply the estimators in a trial nested within a cohort of trial-eligible individuals to compare coronary artery bypass grafting surgery plus medical therapy vs. medical therapy alone for chronic coronary artery disease.

Diagnostic Assessment of Assumptions for External Validity: An Example Using Data in Metastatic Colorectal Cancer

BACKGROUND

Methods developed to estimate intervention effects in external target populations assume that all important effect measure modifiers have been identified and appropriately modeled. Propensity score-based diagnostics can be used to assess the plausibility of these assumptions for weighting methods.

METHODS

We demonstrate the use of these diagnostics when assessing the transportability of treatment effects from the standard of care for metastatic colorectal cancer control arm in a phase III trial (HORIZON III) to a target population of 1,942 Medicare beneficiaries age 65+ years.

RESULTS

In an unadjusted comparison, control arm participants had lower mortality compared with target population patients treated with the standard of care therapy (trial vs. target hazard ratio [HR] = 0.72, 95% confidence interval [CI], 0.58, 0.89). Applying inverse odds of sampling weights attenuated the trial versus target HR (weighted HR = 0.96, 95% CI = 0.73, 1.26). However, whether unadjusted or weighted, hazards did not appear proportional. At 6 months of follow-up, mortality was lower in the weighted trial population than the target population (weighted trial vs. target risk difference [RD] = -0.07, 95% CI = -0.13, -0.01), but not at 12 months (weighted RD = 0.00, 95% CI = -0.09, 0.09).

CONCLUSION

These diagnostics suggest that direct transport of treatment effects from HORIZON III to the Medicare population is not valid. However, the proposed sampling model might allow valid transport of the treatment effects on longer-term mortality from HORIZON III to the Medicare population treated in clinical practice. See video abstract at, http://links.lww.com/EDE/B435.

Composition or Context: Using Transportability to Understand Drivers of Site Differences in a Large-scale Housing Experiment

BACKGROUND

The Moving To Opportunity (MTO) experiment manipulated neighborhood context by randomly assigning housing vouchers to volunteers living in public housing to use to move to lower poverty neighborhoods in five US cities. This random assignment overcomes confounding limitations that challenge other neighborhood studies. However, differences in MTO's effects across the five cities have been largely ignored. Such differences could be due to population composition (e.g., differences in the racial/ethnic distribution) or to context (e.g., differences in the economy).

METHODS

Using a nonparametric omnibus test and a multiply robust, semiparametric estimator for transportability, we assessed the extent to which differences in individual-level compositional characteristics that may act as effect modifiers can account for differences in MTO's effects across sites. We examined MTO's effects on marijuana use, behavioral problems, major depressive disorder, and generalized anxiety disorder among black and Latino adolescent males, where housing voucher receipt was harmful for health in some sites but beneficial in others.

RESULTS

Comparing point estimates, differences in composition partially explained site differences in MTO effects on marijuana use and behavioral problems but did not explain site differences for major depressive disorder or generalized anxiety disorder.

CONCLUSIONS

Our findings provide quantitative, rigorous evidence for the importance of context or unmeasured individual-level compositional variables in modifying MTO's effects.