Generalizing Evidence from Randomized Trials using Inverse Probability of Sampling Weights

A Two-Step Framework for Validating Causal Effect Estimates

BACKGROUND

Comparing causal effect estimates obtained using observational data to those obtained from the gold standard (i.e., randomized controlled trials [RCTs]) helps assess the validity of these estimates. However, comparisons are challenging due to differences between observational data and RCT generated data. The unknown treatment assignment mechanism in the observational data and varying sampling mechanisms between the RCT and the observational data can lead to confounding and sampling bias, respectively.

AIMS

The objective of this study is to propose a two-step framework to validate causal effect estimates obtained from observational data by adjusting for both mechanisms.

MATERIALS AND METHODS

An estimator of causal effects related to the two mechanisms is constructed. A two-step framework for comparing causal effect estimates is derived from the estimator. An R package RCTrep is developed to implement the framework in practice.

RESULTS

A simulation study is conducted to show that using our framework observational data can produce causal effect estimates similar to those of an RCT. A real-world application of the framework to validate treatment effects of adjuvant chemotherapy obtained from registry data is demonstrated.

CONCLUSION

This  study constructs a framework for comparing causal effect estimates between observational data and RCT data, facilitating the assessment of the validity of causal effect estimates obtained from observational data.

Cross-Sectional Survey of Factors Contributing to COVID-19 Testing Hesitancy Among US Adults at Risk of Severe Outcomes from COVID-19

INTRODUCTION

The United States Centers for Disease Control and Prevention (CDC) advises testing individuals for COVID-19 after exposure or if they display symptoms. However, a deeper understanding of demographic factors associated with testing hesitancy is necessary.

METHODS

A US nationwide cross-sectional survey of adults with risk factors for developing severe COVID-19 ("high-risk" individuals) was conducted from August 18-September 5, 2023. Objectives included characterizing demographics and attitudes associated with COVID-19 testing. Inverse propensity weighting was used to weight the data to accurately reflect the high-risk adult US population as reflected in IQVIA medical claims data. We describe here the weighted results modeled to characterize demographic factors driving hesitancy.

RESULTS

In the weighted sample of 5019 respondents at high risk for severe COVID-19, 58.2% were female, 37.8% were ≥ 65 years old, 77.1% were White, and 13.9% had a postgraduate degree. Overall, 67% were Non-testers (who indicated that they were unlikely or unsure of their likelihood of being tested within the next 6 months); these respondents were significantly more likely than Testers (who indicated a higher probability of testing within 6 months) to be female (60.2 vs. 54.1%; odds ratio [OR] [95% confidence interval (CI)], 1.3 [1.1‒1.4]), aged ≥ 65 years old (41.5 vs. 30.3%; OR [95% CI] compared with ages 18‒34 years, 0.6 [0.5‒0.7]), White (82.1 vs. 66.8%; OR [95% CI], 1.4 [1.1‒1.8]), and to identify as politically conservative (40.9 vs. 18.1%; OR [95% CI], 2.6 [2.3‒2.9]). In contrast, Testers were significantly more likely than Non-testers to have previous experience with COVID-19 testing, infection, or vaccination; greater knowledge regarding COVID-19 and testing; greater healthcare engagement; and concerns about COVID-19.

CONCLUSIONS

Older, female, White, rural-dwelling, and politically conservative high-risk adults are the most likely individuals to experience COVID-19 testing hesitancy. Understanding these demographic factors will help guide strategies to improve US testing rates.

Transporting randomized trial results to estimate counterfactual survival functions in target populations

When the distributions of treatment effect modifiers differ between a randomized trial and an external target population, the sample average treatment effect in the trial may be substantially different from the target population average treatment, and accurate estimation of the latter requires adjusting for the differential distribution of effect modifiers. Despite the increasingly rich literature on transportability, little attention has been devoted to methods for transporting trial results to estimate counterfactual survival functions in target populations, when the primary outcome is time to event and subject to right censoring. In this article, we study inverse probability weighting and doubly robust estimators to estimate counterfactual survival functions and the target average survival treatment effect in the target population, and provide their respective approximate variance estimators. We focus on a common scenario where the target population information is observed only through a complex survey, and elucidate how the survey weights can be incorporated into each estimator we considered. Simulation studies are conducted to examine the finite-sample performances of the proposed estimators in terms of bias, efficiency and coverage, under both correct and incorrect model specifications. Finally, we apply the proposed method to assess transportability of the results in the Action to Control Cardiovascular Risk in Diabetes-Blood Pressure (ACCORD-BP) trial to all adults with Diabetes in the United States.

genRCT: a statistical analysis framework for generalizing RCT findings to real-world population

When evaluating the real-world treatment effect, the analysis based on randomized clinical trials (RCTs) often introduces generalizability bias due to the difference in risk factors between the trial participants and the real-world patient population. This problem of lack of generalizability associated with the RCT-only analysis can be addressed by leveraging observational studies with large sample sizes that are representative of the real-world population. A set of novel statistical methods, termed "genRCT", for improving the generalizability of the trial has been developed using calibration weighting, which enforces the covariates balance between the RCT and observational study. This paper aims to review statistical methods for generalizing the RCT findings by harnessing information from large observational studies that represent real-world patients. Specifically, we discuss the choices of data sources and variables to meet key theoretical assumptions and principles. We introduce and compare estimation methods for continuous, binary, and survival endpoints. We showcase the use of the R package genRCT through a case study that estimates the average treatment effect of adjuvant chemotherapy for the stage 1B non-small cell lung patients represented by a large cancer registry.

Estimating treatment effect in randomized trial after control to treatment crossover using external controls

In clinical trials, it is common to design a study that permits the administration of an experimental treatment to participants in the placebo or standard of care group post primary endpoint. This is often seen in the open-label extension phase of a phase III, pivotal study of the new medicine, where the focus is on assessing long-term safety and efficacy. With the availability of external controls, proper estimation and inference of long-term treatment effect during the open-label extension phase in the absence of placebo-controlled patients are now feasible. Within the framework of causal inference, we propose several difference-in-differences (DID) type methods and a synthetic control method (SCM) for the combination of randomized controlled trials and external controls. Our realistic simulation studies demonstrate the desirable performance of the proposed estimators in a variety of practical scenarios. In particular, DID methods outperform SCM and are the recommended methods of choice. An empirical application of the methods is demonstrated through a phase III clinical trial in rare disease.

Visualizing External Validity: Graphical Displays to Inform the Extension of Treatment Effects from Trials to Clinical Practice

BACKGROUND

In the presence of effect measure modification, estimates of treatment effects from randomized controlled trials may not be valid in clinical practice settings. The development and application of quantitative approaches for extending treatment effects from trials to clinical practice settings is an active area of research.

METHODS

In this article, we provide researchers with a practical roadmap and four visualizations to assist in variable selection for models to extend treatment effects observed in trials to clinical practice settings and to assess model specification and performance. We apply this roadmap and visualizations to an example extending the effects of adjuvant chemotherapy (5-fluorouracil vs. plus oxaliplatin) for colon cancer from a trial population to a population of individuals treated in community oncology practices in the United States.

RESULTS

The first visualization screens for potential effect measure modifiers to include in models extending trial treatment effects to clinical practice populations. The second visualization displays a measure of covariate overlap between the clinical practice populations and the trial population. The third and fourth visualizations highlight considerations for model specification and influential observations. The conceptual roadmap describes how the output from the visualizations helps interrogate the assumptions required to extend treatment effects from trials to target populations.

CONCLUSIONS

The roadmap and visualizations can inform practical decisions required for quantitatively extending treatment effects from trials to clinical practice settings.

Fusing trial data for treatment comparisons: Single vs multi-span bridging

While randomized controlled trials (RCTs) are critical for establishing the efficacy of new therapies, there are limitations regarding what comparisons can be made directly from trial data. RCTs are limited to a small number of comparator arms and often compare a new therapeutic to a standard of care which has already proven efficacious. It is sometimes of interest to estimate the efficacy of the new therapy relative to a treatment that was not evaluated in the same trial, such as a placebo or an alternative therapy that was evaluated in a different trial. Such dual-study comparisons are challenging because of potential differences between trial populations that can affect the outcome. In this article, two bridging estimators are considered that allow for comparisons of treatments evaluated in different trials, accounting for measured differences in trial populations. A "multi-span" estimator leverages a shared arm between two trials, while a "single-span" estimator does not require a shared arm. A diagnostic statistic that compares the outcome in the standardized shared arms is provided. The two estimators are compared in simulations, where both estimators demonstrate minimal empirical bias and nominal confidence interval coverage when the identification assumptions are met. The estimators are applied to data from the AIDS Clinical Trials Group 320 and 388 to compare the efficacy of two-drug vs four-drug antiretroviral therapy on CD4 cell counts among persons with advanced HIV. The single-span approach requires weaker identification assumptions and was more efficient in simulations and the application.

Cost-effectiveness analysis of mosunetuzumab for treatment of relapsed or refractory follicular lymphoma after two or more lines of systemic therapy in the United States

AIMS

Mosunetuzumab has received accelerated approval by the US Food and Drug Administration for adult patients with relapsed or refractory (R/R) follicular lymphoma (FL) after two or more lines of systemic therapy. We evaluated the cost-effectiveness of mosunetuzumab for the treatment of R/R FL from a US private payer perspective.

MATERIALS AND METHODS

A partitioned survival model simulated lifetime costs and outcomes of mosunetuzumab against seven comparators: axicabtagene ciloleucel (axi-cel), tisagenlecleucel (tisa-cel), tazemetostat (taz, EZH2 wild-type only), rituximab plus lenalidomide (R-Len) or bendamustine (R-Benda), obinutuzumab plus bendamustine (O-Benda), and a retrospective real-world cohort (RW) based on current patterns of care derived from US electronic health records (Flatiron Health). Efficacy data for mosunetuzumab were from the pivotal Phase II GO29781 trial (NCT02500407). Relative treatment efficacy was estimated from indirect treatment comparisons (ITCs). Costs included were related to treatment, adverse events, routine care, and terminal care. Except for drug costs (March 2023), all costs were inflated to 2022 US dollars. Costs and quality-adjusted life-years (QALYs) were used to calculate incremental cost-effectiveness ratios (ICERs). Net monetary benefit (NMB) was calculated using a willingness-to-pay (WTP) threshold of $150,000/QALY.

RESULTS

Mosunetuzumab dominated taz, tisa-cel, and axi-cel with greater QALYs and lower costs. Mosunetuzumab was projected to be cost-effective against R-Benda, O-Benda, and RW with ICERs of $78,607, $42,731, and $21,434, respectively. Mosunetuzumab incurred lower costs but lower QALYs vs. R-Len. NMBs showed that mosunetuzumab was cost-effective against comparators except R-Len.

LIMITATIONS

Without head-to-head comparative data, the model had to rely on ITCs, some of which were affected by residual bias. Model inputs were obtained from multiple sources. Extensive sensitivity analyses assessed the importance of these uncertainties.

CONCLUSION

Mosunetuzumab is estimated to be cost-effective compared with approved regimens except R-Len for the treatment of adults with R/R FL.

Entropy balancing for causal generalization with target sample summary information

In this paper, we focus on estimating the average treatment effect (ATE) of a target population when individual-level data from a source population and summary-level data (e.g., first or second moments of certain covariates) from the target population are available. In the presence of the heterogeneous treatment effect, the ATE of the target population can be different from that of the source population when distributions of treatment effect modifiers are dissimilar in these two populations, a phenomenon also known as covariate shift. Many methods have been developed to adjust for covariate shift, but most require individual covariates from a representative target sample. We develop a weighting approach based on the summary-level information from the target sample to adjust for possible covariate shift in effect modifiers. In particular, weights of the treated and control groups within a source sample are calibrated by the summary-level information of the target sample. Our approach also seeks additional covariate balance between the treated and control groups in the source sample. We study the asymptotic behavior of the corresponding weighted estimator for the target population ATE under a wide range of conditions. The theoretical implications are confirmed in simulation studies and a real-data application.

Examining the continuum of resistance model in two population-based screening studies in Sweden

In studies recruited on a voluntary basis, lack of representativity may impair the ability to generalize findings to the target population. Previous studies, primarily based on surveys, have suggested that generalizability may be improved by exploiting data on individuals who agreed to participate only after receiving one or several reminders, as such individuals may be more similar to non-participants than what early participants are. Assessing this idea in the context of screenings, we compared sociodemographic characteristics and health across early, late, and non-participants in two large population-based screening studies in Sweden: STROKESTOP II (screening for atrial fibrillation; 6,867 participants) and SCREESCO (screening for colorectal cancer; 39,363 participants). We also explored the opportunities to reproduce the distributions of characteristics in the full invited populations, either by assuming that the non-participants were similar to the late participants, or by applying a linear extrapolation model based on both early and late participants. Findings showed that early and late participants exhibited similar characteristics along most dimensions, including civil status, education, income, and health examination results. Both these types of participants in turn differed from the non-participants, with fewer married, lower educational attainments, and lower incomes. Compared to early participants, late participants were more likely to be born outside of Sweden and to have comorbidities, with non-participants similar or even more so. The two empirical models improved representativity in some cases, but not always. Overall, we found mixed support that data on late participation may be useful for improving representativeness of screening studies.

Reweighting estimators to extend the external validity of clinical trials: methodological considerations

Methods to extend the strong internal validity of randomized controlled trials to reliably estimate treatment effects in target populations are gaining attention. This paper enumerates steps recommended for undertaking such extended inference, discusses currently viable choices for each one, and provides recommendations. We demonstrate a complete extended inference from a clinical trial studying a pharmaceutical treatment for Alzheimer's disease (AD) to a realistic target population of European residents diagnosed with AD. This case study highlights approaches to overcoming practical difficulties and demonstrates limitations of reliably extending inference from a trial to a real-world population.

Keeping kin close? Geographies of family networks by race and income, 1981-2017

Objective

This study examined changes in geographic proximity to family members among race and income groups in the United States from 1981 to 2017.

Background

Close geographic proximity to family members can facilitate mutual support and strengthen family bonds. Some scholars argue that institutional sources of support have replaced many core family functions, which might mean that households are likely to live increasingly farther away from family. Advancing technology and changing labor market opportunities might reinforce this pattern. Yet, the ongoing cultural and emotional salience of family might curtail the effects of these factors on the increasing distance to family.

Method

We conducted a quantitative analysis of longitudinal data from the Panel Study of Income Dynamics (PSID). We utilized the multigenerational structure of the PSID and restricted-use geocodes to map kin proximity at every interview from 1981 to 2017. We cross-classified our sample by race and income, focusing on Black and White respondents across income quartiles (n = 171,501 person-periods).

Results

High-income White respondents showed the greatest increases in distance from kin over time, whereas proximity to kin among other race-income groups was relatively stable.

Conclusion

Proximate kin has become less central in the lives of high-income White households over time, whereas close proximity to kin has been the norm over time for other racial and income groups. These results have implications for racial and income differences in kin relations over time.

Elastic integrative analysis of randomised trial and real-world data for treatment heterogeneity estimation

We propose a test-based elastic integrative analysis of the randomised trial and real-world data to estimate treatment effect heterogeneity with a vector of known effect modifiers. When the real-world data are not subject to bias, our approach combines the trial and real-world data for efficient estimation. Utilising the trial design, we construct a test to decide whether or not to use real-world data. We characterise the asymptotic distribution of the test-based estimator under local alternatives. We provide a data-adaptive procedure to select the test threshold that promises the smallest mean square error and an elastic confidence interval with a good finite-sample coverage property.

Improving trial generalizability using observational studies

Complementary features of randomized controlled trials (RCTs) and observational studies (OSs) can be used jointly to estimate the average treatment effect of a target population. We propose a calibration weighting estimator that enforces the covariate balance between the RCT and OS, therefore improving the trial-based estimator's generalizability. Exploiting semiparametric efficiency theory, we propose a doubly robust augmented calibration weighting estimator that achieves the efficiency bound derived under the identification assumptions. A nonparametric sieve method is provided as an alternative to the parametric approach, which enables the robust approximation of the nuisance functions and data-adaptive selection of outcome predictors for calibration. We establish asymptotic results and confirm the finite sample performances of the proposed estimators by simulation experiments and an application on the estimation of the treatment effect of adjuvant chemotherapy for early-stage non-small-cell lung patients after surgery.

Improving efficiency of inference in clinical trials with external control data

Suppose we are interested in the effect of a treatment in a clinical trial. The efficiency of inference may be limited due to small sample size. However, external control data are often available from historical studies. Motivated by an application to Helicobacter pylori infection, we show how to borrow strength from such data to improve efficiency of inference in the clinical trial. Under an exchangeability assumption about the potential outcome mean, we show that the semiparametric efficiency bound for estimating the average treatment effect can be reduced by incorporating both the clinical trial data and external controls. We then derive a doubly robust and locally efficient estimator. The improvement in efficiency is prominent especially when the external control data set has a large sample size and small variability. Our method allows for a relaxed overlap assumption, and we illustrate with the case where the clinical trial only contains a treated group. We also develop doubly robust and locally efficient approaches that extrapolate the causal effect in the clinical trial to the external population and the overall population. Our results also offer a meaningful implication for trial design and data collection. We evaluate the finite-sample performance of the proposed estimators via simulation. In the Helicobacter pylori infection application, our approach shows that the combination treatment has potential efficacy advantages over the triple therapy.

Illustration of 2 Fusion Designs and Estimators

"Fusion" study designs combine data from different sources to answer questions that could not be answered (as well) by subsets of the data. Studies that augment main study data with validation data, as in measurement-error correction studies or generalizability studies, are examples of fusion designs. Fusion estimators, here solutions to stacked estimating functions, produce consistent answers to identified research questions using data from fusion designs. In this paper, we describe a pair of examples of fusion designs and estimators, one where we generalize a proportion to a target population and one where we correct measurement error in a proportion. For each case, we present an example motivated by human immunodeficiency virus research and summarize results from simulation studies. Simulations demonstrate that the fusion estimators provide approximately unbiased results with appropriate 95% confidence interval coverage. Fusion estimators can be used to appropriately combine data in answering important questions that benefit from multiple sources of information.

Data Integration in Causal Inference

Integrating data from multiple heterogeneous sources has become increasingly popular to achieve a large sample size and diverse study population. This paper reviews development in causal inference methods that combines multiple datasets collected by potentially different designs from potentially heterogeneous populations. We summarize recent advances on combining randomized clinical trial with external information from observational studies or historical controls, combining samples when no single sample has all relevant variables with application to two-sample Mendelian randomization, distributed data setting under privacy concerns for comparative effectiveness and safety research using real-world data, Bayesian causal inference, and causal discovery methods.

Transfer learning of individualized treatment rules from experimental to real-world data

Individualized treatment effect lies at the heart of precision medicine. Interpretable individualized treatment rules (ITRs) are desirable for clinicians or policymakers due to their intuitive appeal and transparency. The gold-standard approach to estimating the ITRs is randomized experiments, where subjects are randomized to different treatment groups and the confounding bias is minimized to the extent possible. However, experimental studies are limited in external validity because of their selection restrictions, and therefore the underlying study population is not representative of the target real-world population. Conventional learning methods of optimal interpretable ITRs for a target population based only on experimental data are biased. On the other hand, real-world data (RWD) are becoming popular and provide a representative sample of the target population. To learn the generalizable optimal interpretable ITRs, we propose an integrative transfer learning method based on weighting schemes to calibrate the covariate distribution of the experiment to that of the RWD. Theoretically, we establish the risk consistency for the proposed ITR estimator. Empirically, we evaluate the finite-sample performance of the transfer learner through simulations and apply it to a real data application of a job training program.

Generalizing trial evidence to target populations in non-nested designs: Applications to AIDS clinical trials

Comparative effectiveness evidence from randomized trials may not be directly generalizable to a target population of substantive interest when, as in most cases, trial participants are not randomly sampled from the target population. Motivated by the need to generalize evidence from two trials conducted in the AIDS Clinical Trials Group (ACTG), we consider weighting, regression and doubly robust estimators to estimate the causal effects of HIV interventions in a specified population of people living with HIV in the USA. We focus on a non-nested trial design and discuss strategies for both point and variance estimation of the target population average treatment effect. Specifically in the generalizability context, we demonstrate both analytically and empirically that estimating the known propensity score in trials does not increase the variance for each of the weighting, regression and doubly robust estimators. We apply these methods to generalize the average treatment effects from two ACTG trials to specified target populations and operationalize key practical considerations. Finally, we report on a simulation study that investigates the finite-sample operating characteristics of the generalizability estimators and their sandwich variance estimators.

When More Can Be Less: The Effect of Add-On Insurance on the Consumption of Professional Services

The emergence of online platforms for professional services (e.g., cosmetic procedures) represents a natural progression of e-commerce from search and experience goods to credence goods. Because of the deeply consequential nature of professional services and the large information asymmetries between customers and service providers, designing effective risk-reduction strategies is crucial for facilitating digital transactions of professional services. This paper studies whether and how the introduction of a novel risk-reduction strategy, the add-on insurance covering the potential cost of negative consequences (e.g., complications and unsatisfactory outcomes), affects the demand for professional services in online platforms. We leverage a policy change in an online platform for cosmetic procedures, which started to offer the add-on insurance for a subset of procedures in 2016. Our empirical analysis shows that the introduction of insurance increases the sales of low-risk procedures, but not those of high-risk ones. More importantly, the insurance has a negative spillover effect on uninsured competitors, regardless of their risk levels. The negative spillover effect on high-risk procedures is noteworthy because it hurts the sales of their uninsured competitors without increasing their own sales, reducing the overall demand. Our findings have important implications for platforms to design, deploy, and evaluate their risk-reduction strategies.

A note on semiparametric efficient generalization of causal effects from randomized trials to target populations

When effect modifiers influence the decision to participate in randomized trials, generalizing causal effect estimates to an external target population requires the knowledge of two scores - the propensity score for receiving treatment and the sampling score for trial participation. While the former score is known due to randomization, the latter score is usually unknown and estimated from data. Under unconfounded trial participation, we characterize the asymptotic efficiency bounds for estimating two causal estimands - the population average treatment effect and the average treatment effect among the non-participants - and examine the role of the scores. We also study semiparametric efficient estimators that directly balance the weighted trial sample toward the target population, and illustrate their operating characteristics via simulations.

Applicability of Transcatheter Aortic Valve Replacement Trials to Real-World Clinical Practice: Findings From EXTEND-CoreValve

OBJECTIVES

The aim of this study was to examine the applicability of pivotal transcatheter aortic valve replacement (TAVR) trials to the real-world population of Medicare patients undergoing TAVR.

BACKGROUND

It is unclear whether randomized controlled trial results of novel cardiovascular devices apply to patients encountered in clinical practice.

METHODS

Characteristics of patients enrolled in the U.S. CoreValve pivotal trials were compared with those of the population of Medicare beneficiaries who underwent TAVR in U.S. clinical practice between November 2, 2011, and December 31, 2017. Inverse probability weighting was used to reweight the trial cohort on the basis of Medicare patient characteristics, and a "real-world" treatment effect was estimated.

RESULTS

A total of 2,026 patients underwent TAVR in the U.S. CoreValve pivotal trials, and 135,112 patients underwent TAVR in the Medicare cohort. Trial patients were mostly similar to real-world patients at baseline, though trial patients were more likely to have hypertension (50% vs 39%) and coagulopathy (25% vs 17%), whereas real-world patients were more likely to have congestive heart failure (75% vs 68%) and frailty. The estimated real-world treatment effect of TAVR was an 11.4% absolute reduction in death or stroke (95% CI: 7.50%-14.92%) and an 8.7% absolute reduction in death (95% CI: 5.20%-12.32%) at 1 year with TAVR compared with conventional therapy (surgical aortic valve replacement for intermediate- and high-risk patients and medical therapy for extreme-risk patients).

CONCLUSIONS

The trial and real-world populations were mostly similar, with some notable differences. Nevertheless, the extrapolated real-world treatment effect was at least as high as the observed trial treatment effect, suggesting that the absolute benefit of TAVR in clinical trials is similar to the benefit of TAVR in the U.S. real-world setting.

Transportability From Randomized Trials to Clinical Care: On Initial HIV Treatment With Efavirenz and Suicidal Thoughts or Behaviors

In an analysis of randomized trials, use of efavirenz for treatment of human immunodeficiency virus (HIV) infection was associated with increased suicidal thoughts/behaviors. However, analyses of observational data have found no evidence of increased risk. To assess whether population differences might explain this divergence, we transported the effect of efavirenz use from these trials to a specific target population. Using inverse odds weights and multiple imputation, we transported the effect of efavirenz on suicidal thoughts/behaviors in these randomized trials (participants were enrolled in 2001-2007) to a trials-eligible cohort of US adults initiating antiretroviral therapy while receiving HIV clinical care at medical centers between 1999 and 2015. Overall, 8,291 cohort participants and 3,949 trial participants were eligible. Prescription of antidepressants (19% vs. 13%) and injection drug history (16% vs. 10%) were more frequent in the cohort than in the trial participants. Compared with the effect in trials, the estimated hazard ratio for efavirenz on suicidal thoughts/behaviors was attenuated in our target population (trials: hazard ratio (HR) = 2.3 (95% confidence interval (CI): 1.2, 4.4); transported: HR = 1.8 (95% CI: 0.9, 4.4)), whereas the incidence rate difference was similar (trials: HR = 5.1 (95% CI: 1.6, 8.7); transported: HR = 5.4 (95% CI: -0.4, 11.4)). In our target population, there was greater than 20% attenuation of the hazard ratio estimate as compared with the trials-only estimate. Transporting results from trials to a target population is informative for addressing external validity.

Generalizability and Transportability of the National Lung Screening Trial Data: Extending Trial Results to Different Populations

BACKGROUND

Randomized controlled trials (RCT) play a central role in evidence-based healthcare. However, the clinical and policy implications of implementing RCTs in clinical practice are difficult to predict as the studied population is often different from the target population where results are being applied. This study illustrates the concepts of generalizability and transportability, demonstrating their utility in interpreting results from the National Lung Screening Trial (NLST).

METHODS

Using inverse-odds weighting, we demonstrate how generalizability and transportability techniques can be used to extrapolate treatment effect from (i) a subset of NLST to the entire NLST population and from (ii) the entire NLST to different target populations.

RESULTS

Our generalizability analysis revealed that lung cancer mortality reduction by LDCT screening across the entire NLST [16% (95% confidence interval [CI]: 4-24)] could have been estimated using a smaller subset of NLST participants. Using transportability analysis, we showed that populations with a higher prevalence of females and current smokers had a greater reduction in lung cancer mortality with LDCT screening [e.g., 27% (95% CI, 11-37) for the population with 80% females and 80% current smokers] than those with lower prevalence of females and current smokers.

CONCLUSIONS

This article illustrates how generalizability and transportability methods extend estimation of RCTs' utility beyond trial participants, to external populations of interest, including those that more closely mirror real-world populations.

IMPACT

Generalizability and transportability approaches can be used to quantify treatment effects for populations of interest, which may be used to design future trials or adjust lung cancer screening eligibility criteria.

A generalizability score for aggregate causal effect

Scientists frequently generalize population level causal quantities such as average treatment effect from a source population to a target population. When the causal effects are heterogeneous, differences in subject characteristics between the source and target populations may make such a generalization difficult and unreliable. Reweighting or regression can be used to adjust for such differences when generalizing. However, these methods typically suffer from large variance if there is limited covariate distribution overlap between the two populations. We propose a generalizability score to address this issue. The score can be used as a yardstick to select target subpopulations for generalization. A simplified version of the score avoids using any outcome information and thus can prevent deliberate biases associated with inadvertent access to such information. Both simulation studies and real data analysis demonstrate convincing results for such selection.

Study Designs for Extending Causal Inferences From a Randomized Trial to a Target Population

In this article, we examine study designs for extending (generalizing or transporting) causal inferences from a randomized trial to a target population. Specifically, we consider nested trial designs, where randomized individuals are nested within a sample from the target population, and nonnested trial designs, including composite data-set designs, where observations from a randomized trial are combined with those from a separately obtained sample of nonrandomized individuals from the target population. We show that the counterfactual quantities that can be identified in each study design depend on what is known about the probability of sampling nonrandomized individuals. For each study design, we examine identification of counterfactual outcome means via the g-formula and inverse probability weighting. Last, we explore the implications of the sampling properties underlying the designs for the identification and estimation of the probability of trial participation.

Emulating Target Trials to Improve Causal Inference From Agent-Based Models

Agent-based models are a key tool for investigating the emergent properties of population health settings, such as infectious disease transmission, where the exposure often violates the key "no interference" assumption of traditional causal inference under the potential outcomes framework. Agent-based models and other simulation-based modeling approaches have generally been viewed as a separate knowledge-generating paradigm from the potential outcomes framework, but this can lead to confusion about how to interpret the results of these models in real-world settings. By explicitly incorporating the target trial framework into the development of an agent-based or other simulation model, we can clarify the causal parameters of interest, as well as make explicit the assumptions required for valid causal effect estimation within or between populations. In this paper, we describe the use of the target trial framework for designing agent-based models when the goal is estimation of causal effects in the presence of interference, or spillover.

A directed acyclic graph for interactions

BACKGROUND

Directed acyclic graphs (DAGs) are of great help when researchers try to understand the nature of causal relationships and the consequences of conditioning on different variables. One fundamental feature of causal relations that has not been incorporated into the standard DAG framework is interaction, i.e. when the effect of one variable (on a chosen scale) depends on the value that another variable is set to. In this paper, we propose a new type of DAG-the interaction DAG (IDAG), which can be used to understand this phenomenon.

METHODS

The IDAG works like any DAG but instead of including a node for the outcome, it includes a node for a causal effect. We introduce concepts such as confounded interaction and total, direct and indirect interaction, showing that these can be depicted in ways analogous to how similar concepts are depicted in standard DAGs. This also allows for conclusions on which treatment interactions to account for empirically. Moreover, since generalizability can be compromised in the presence of underlying interactions, the framework can be used to illustrate threats to generalizability and to identify variables to account for in order to make results valid for the target population.

CONCLUSIONS

The IDAG allows for a both intuitive and stringent way of illustrating interactions. It helps to distinguish between causal and non-causal mechanisms behind effect variation. Conclusions about how to empirically estimate interactions can be drawn-as well as conclusions about how to achieve generalizability in contexts where interest lies in estimating an overall effect.

Reweighting a Swedish health questionnaire survey using extensive population register and self-reported data for assessing and improving the validity of longitudinal associations

Background

In cohorts with voluntary participation, participants may not be representative of the underlying population, leading to distorted estimates. If the relevant sources of selective participation are observed, it is however possible to restore the representativeness by reweighting the sample to resemble the target population. So far, few studies in epidemiology have applied reweighting based on extensive register data on socio-demographics and disease history, or with self-reported data on health and health-related behaviors.

Methods

We examined selective participation at baseline and the first two follow-ups of the Scania Public Health Cohort (SPHC), a survey conducted in Southern Sweden in 1999/2000 (baseline survey; n = 13,581 participants, 58% participation rate), 2005 (first follow-up, n = 10,471), and 2010 (second follow-up; n = 9,026). Survey participants were reweighted to resemble the underlying population with respect to a broad range of socio-demographic, disease, and health-related characteristics, and we assessed how selective participation impacted the validity of associations between self-reported overall health and dimensions of socio-demographics and health.

Results

Participants in the baseline and follow-up surveys were healthier and more likely to be female, born in Sweden, middle-aged, and have higher socioeconomic status. However, the differences were not very large. In turn, reweighting the samples to match the target population had generally small or moderate impacts on associations. Most examined regression coefficients changed by less than 20%, with virtually no changes in the directions of the effects.

Conclusion

Overall, selective participation with respect to the observed factors was not strong enough to substantially alter the associations with self-assessed health. These results are consistent with an interpretation that SPHC has high validity, perhaps reflective of a relatively high participation rate. Since validity must be determined on a case-by-case basis, however, researchers should apply the same method to other health cohorts to assess and potentially improve the validity.

Methods for population adjustment with limited access to individual patient data: A review and simulation study

Population-adjusted indirect comparisons estimate treatment effects when access to individual patient data is limited and there are cross-trial differences in effect modifiers. Popular methods include matching-adjusted indirect comparison (MAIC) and simulated treatment comparison (STC). There is limited formal evaluation of these methods and whether they can be used to accurately compare treatments. Thus, we undertake a comprehensive simulation study to compare standard unadjusted indirect comparisons, MAIC and STC across 162 scenarios. This simulation study assumes that the trials are investigating survival outcomes and measure continuous covariates, with the log hazard ratio as the measure of effect. MAIC yields unbiased treatment effect estimates under no failures of assumptions. The typical usage of STC produces bias because it targets a conditional treatment effect where the target estimand should be a marginal treatment effect. The incompatibility of estimates in the indirect comparison leads to bias as the measure of effect is non-collapsible. Standard indirect comparisons are systematically biased, particularly under stronger covariate imbalance and interaction effects. Standard errors and coverage rates are often valid in MAIC but the robust sandwich variance estimator underestimates variability where effective sample sizes are small. Interval estimates for the standard indirect comparison are too narrow and STC suffers from bias-induced undercoverage. MAIC provides the most accurate estimates and, with lower degrees of covariate overlap, its bias reduction outweighs the loss in precision under no failures of assumptions. An important future objective is the development of an alternative formulation to STC that targets a marginal treatment effect.

Fusion designs and estimators for treatment effects

While randomized trials remain the best evidence for treatment effectiveness, lack of generalizability often remains an important concern. Additionally, when new treatments are compared against existing standards of care, the potentially small benefit of the new treatment may be difficult to detect in a trial without extremely large sample sizes and long follow-up times. Recent advances in "data fusion" provide a framework to combine results across studies that are applicable to a given population of interest and allow treatment comparisons that may not be feasible with traditional study designs. We propose a data fusion-based estimator that can be used to combine information from two studies: (1) a study comparing a new treatment to the standard of care in the local population of interest, and (2) a study comparing the standard of care to placebo in a separate, distal population. We provide conditions under which the parameter of interest can be identified from the two studies described and explore properties of the estimator through simulation. Finally, we apply the estimator to estimate the effect of triple- vs monotherapy for the treatment of HIV using data from two randomized trials. The proposed estimator can account for underlying population structures that induce differences in case mix, adherence, and outcome prevalence between the local and distal populations, and the estimator can also account for potentially informative loss to follow-up. Approaches like those detailed here are increasingly important to speed the approval and adoption of effective new therapies by leveraging multiple sources of information.

Rejoinder: Learning Optimal Distributionally Robust Individualized Treatment Rules

We thank the opportunity offered by editors for this discussion and the discussants for their insightful comments and thoughtful contributions. We also want to congratulate Kallus (2020) for his inspiring work in improving the effciency of policy learning by retargeting. Motivated from the discussion in Dukes and Vansteelandt (2020), we first point out interesting connections and distinctions between our work and Kallus (2020) in Section 1. In particular, the assumptions and sources of variation for consideration in these two papers lead to different research problems with different scopes and focuses. In Section 2, following the discussions in Li et al. (2020); Liang and Zhao (2020), we also consider the efficient policy evaluation problem when we have some data from the testing distribution available at the training stage. We show that under the assumption that the sample sizes from training and testing are growing in the same order, efficient value function estimates can deliver competitive performance. We further show some connections of these estimates with existing literature. However, when the growth of testing sample size available for training is in a slower order, efficient value function estimates may not perform well anymore. In contrast, the requirement of the testing sample size for DRITR is not as strong as that of efficient policy evaluation using the combined data. Finally, we highlight the general applicability and usefulness of DRITR in Section 3.

A comparison of reweighting estimators of average treatment effects in real world populations

Regulatory agencies typically evaluate the efficacy and safety of new interventions and grant commercial approval based on randomized controlled trials (RCTs). Other major healthcare stakeholders, such as insurance companies and health technology assessment agencies, while basing initial access and reimbursement decisions on RCT results, are also keenly interested in whether results observed in idealized trial settings will translate into comparable outcomes in real world settings-that is, into so-called "real world" effectiveness. Unfortunately, evidence of real world effectiveness for new interventions is not available at the time of initial approval. To bridge this gap, statistical methods are available to extend the estimated treatment effect observed in a RCT to a target population. The generalization is done by weighting the subjects who participated in a RCT so that the weighted trial population resembles a target population. We evaluate a variety of alternative estimation and weight construction procedures using both simulations and a real world data example using two clinical trials of an investigational intervention for Alzheimer's disease. Our results suggest an optimal approach to estimation depends on the characteristics of source and target populations, including degree of selection bias and treatment effect heterogeneity.

Target validity: Bringing treatment of external validity in line with internal validity

Purpose of Review

"Target bias" is the difference between an estimate of association from a study sample and the causal effect in the target population of interest. It is the sum of internal and external bias. Given the extensive literature on internal validity, here, we review threats and methods to improve external validity.

Recent findings

External bias may arise when the distribution of modifiers of the effect of treatment differs between the study sample and the target population. Methods including those based on modeling the outcome, modeling sample membership, and doubly robust methods are available, assuming data on the target population is available.

Summary

The relevance of information for making policy decisions is dependent on both the actions that were studied and the sample in which they were evaluated. Combining methods for addressing internal and external validity can improve the policy relevance of study results.

Can the validity of a cohort be improved by reweighting based on register data? Evidence from the Swedish MDC study

Background

In any study with voluntary participation, self-selection risks leading to invalid conclusions. If the determinants of selection are observed, it is however possible to restore the parameters of interest by reweighting the sample to match the population, but this approach has seldom been applied in epidemiological research.

Methods

We reweighted the Malmö Diet and Cancer (MDC) study based on population register data on background variables, including socio-demographics and hospital admissions for both participants and the background population. Following individuals from baseline in 1991–1996 and at most until 2016, we studied mortality (all-cause, cancer, and CVD), incidences (cancer and CVD), and associations between these outcomes and background variables. Results from the unweighted and reweighted participant sample were compared with those from the background population.

Results

Mortality was substantially lower in participants than in the background population, but reweighting the sample helped only little to make the numbers similar to those in the background population. For incidences and associations, numbers were generally similar between participants and the background population already without reweighting, rendering reweighting unnecessary.

Conclusion

Reweighting samples based on an extensive range of sociodemographic characteristics and previous hospitalizations does not necessarily yield results that are valid for the population as a whole. In the case of MDC, there appear to be important factors related to both mortality and selection into the study that are not observable in registry data, making it difficult to obtain accurate numbers on population mortality based on cohort participants. These issues seem less relevant for incidences and associations, however. Overall, our results suggest that representativeness must be judged on a case-by-case basis.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-020-10004-z.

Generalizing randomized trial findings to a target population using complex survey population data

Randomized trials are considered the gold standard for estimating causal effects. Trial findings are often used to inform policy and programming efforts, yet their results may not generalize well to a relevant target population due to potential differences in effect moderators between the trial and population. Statistical methods have been developed to improve generalizability by combining trials and population data, and weighting the trial to resemble the population on baseline covariates. Large-scale surveys in fields such as health and education with complex survey designs are a logical source for population data; however, there is currently no best practice for incorporating survey weights when generalizing trial findings to a complex survey. We propose and investigate ways to incorporate survey weights in this context. We examine the performance of our proposed estimator through simulations in comparison to estimators that ignore the complex survey design. We then apply the methods to generalize findings from two trials-a lifestyle intervention for blood pressure reduction and a web-based intervention to treat substance use disorders-to their respective target populations using population data from complex surveys. The work highlights the importance in properly accounting for the complex survey design when generalizing trial findings to a population represented by a complex survey sample.

Doubly robust inference when combining probability and non-probability samples with high dimensional data

We consider integrating a non-probability sample with a probability sample which provides high dimensional representative covariate information of the target population. We propose a two-step approach for variable selection and finite population inference. In the first step, we use penalized estimating equations with folded concave penalties to select important variables and show selection consistency for general samples. In the second step, we focus on a doubly robust estimator of the finite population mean and re-estimate the nuisance model parameters by minimizing the asymptotic squared bias of the doubly robust estimator. This estimating strategy mitigates the possible first-step selection error and renders the doubly robust estimator root n consistent if either the sampling probability or the outcome model is correctly specified.

Learning Optimal Distributionally Robust Individualized Treatment Rules

Recent development in the data-driven decision science has seen great advances in individualized decision making. Given data with individual covariates, treatment assignments and outcomes, policy makers best individualized treatment rule (ITR) that maximizes the expected outcome, known as the value function. Many existing methods assume that the training and testing distributions are the same. However, the estimated optimal ITR may have poor generalizability when the training and testing distributions are not identical. In this paper, we consider the problem of finding an optimal ITR from a restricted ITR class where there is some unknown covariate changes between the training and testing distributions. We propose a novel distributionally robust ITR (DR-ITR) framework that maximizes the worst-case value function across the values under a set of underlying distributions that are "close" to the training distribution. The resulting DR-ITR can guarantee the performance among all such distributions reasonably well. We further propose a calibrating procedure that tunes the DR-ITR adaptively to a small amount of calibration data from a target population. In this way, the calibrated DR-ITR can be shown to enjoy better generalizability than the standard ITR based on our numerical studies.

Assessing exposure effects on gene expression

In observational genomics data sets, there is often confounding of the effect of an exposure on gene expression. To adjust for confounding when estimating the exposure effect, a common approach involves including potential confounders as covariates with the exposure in a regression model of gene expression. However, when the exposure and confounders interact to influence gene expression, the fitted regression model does not necessarily estimate the overall effect of the exposure. Using inverse probability weighting (IPW) or the parametric g-formula in these instances is straightforward to apply and yields consistent effect estimates. IPW can readily be integrated into a genomics data analysis pipeline with upstream data processing and normalization, while the g-formula can be implemented by making simple alterations to the regression model. The regression, IPW, and g-formula approaches to exposure effect estimation are compared herein using simulations; advantages and disadvantages of each approach are explored. The methods are applied to a case study estimating the effect of current smoking on gene expression in adipose tissue.

Extending inferences from a randomized trial to a new target population

When treatment effect modifiers influence the decision to participate in a randomized trial, the average treatment effect in the population represented by the randomized individuals will differ from the effect in other populations. In this tutorial, we consider methods for extending causal inferences about time-fixed treatments from a trial to a new target population of nonparticipants, using data from a completed randomized trial and baseline covariate data from a sample from the target population. We examine methods based on modeling the expectation of the outcome, the probability of participation, or both (doubly robust). We compare the methods in a simulation study and show how they can be implemented in software. We apply the methods to a randomized trial nested within a cohort of trial-eligible patients to compare coronary artery surgery plus medical therapy versus medical therapy alone for patients with chronic coronary artery disease. We conclude by discussing issues that arise when using the methods in applied analyses.

Household food insecurity and antepartum depression in the National Children's Study

PURPOSE

The purpose of this study was to determine the association between household food insecurity (HFI) and elevated antepartum depressive symptoms (EADS) in the National Children's Study, 2009-2014, as well as standardize our results to the U.S. pregnant population.

METHODS

HFI was collected at participants' baseline visits using the U.S. Household Food Security Survey Module; antepartum depression symptoms were collected twice during pregnancy using the Center for Epidemiologic Study Depression scale. Generalized estimating equations for binary outcomes were used to estimate the association between HFI and EADS. Inverse probability weighting was used to generalize the effect to the U.S. population using the National Health and Nutrition Examination Survey.

RESULTS

Among 746 participants, 20.6% were food insecure. Women who were food insecure were 3.39 times (95% confidence interval: 1.73, 6.62) as likely to report EADS compared with women who were food secure. This estimate was marginally strengthened in a weighted analysis (odds ratio: 3.68; 95% confidence interval: 1.43, 9.43).

CONCLUSIONS

This study suggests that women who are food insecure are at a greater risk of EADS, and HFI should be evaluated when assessing antepartum depression.

Population-Based Registry Linkages to Improve Validity of Electronic Health Record-Based Cancer Research

BACKGROUND

There is tremendous potential to leverage the value gained from integrating electronic health records (EHR) and population-based cancer registry data for research. Registries provide diagnosis details, tumor characteristics, and treatment summaries, while EHRs contain rich clinical detail. A carefully conducted cancer registry linkage may also be used to improve the internal and external validity of inferences made from EHR-based studies.

METHODS

We linked the EHRs of a large, multispecialty, mixed-payer health care system with the statewide cancer registry and assessed the validity of our linked population. For internal validity, we identify patients that might be "missed" in a linkage, threatening the internal validity of an EHR study population. For generalizability, we compared linked cases with all other cancer patients in the 22-county EHR catchment region.

RESULTS

From an EHR population of 4.5 million, we identified 306,554 patients with cancer, 26% of the catchment region patients with cancer; 22.7% of linked patients were diagnosed with cancer after they migrated away from our health care system highlighting an advantage of system-wide linkage. We observed demographic differences between EHR patients and non-EHR patients in the surrounding region and demonstrated use of selection probabilities with model-based standardization to improve generalizability.

CONCLUSIONS

Our experiences set the foundation to encourage and inform researchers interested in working with EHRs for cancer research as well as provide context for leveraging linkages to assess and improve validity and generalizability.

IMPACT

Researchers conducting linkages may benefit from considering one or more of these approaches to establish and evaluate the validity of their EHR-based populations.See all articles in this CEBP Focus section, "Modernizing Population Science."

A Randomized Trial to Evaluate the Effects of Folic Acid and Zinc Supplementation on Male Fertility and Livebirth: Design and Baseline Characteristics

The Folic Acid and Zinc Supplementation Trial (FAZST) was a multicenter, double-blind, block-randomized, placebo-controlled trial to determine whether folic acid and zinc supplementation in men improves semen quality and increases livebirth rate among couples seeking infertility treatment (2013-2017). Eligible men were aged 18 years or older with female partners aged 18-45 years, seeking infertility treatment. Men were randomized (1:1) to 5 mg folic acid and 30 mg elemental zinc daily or matching placebo for 6 months. Randomization was stratified by site and intended infertility treatment (in vitro fertilization (IVF), non-IVF/study site, and non-IVF/outside clinic). Follow-up of men continued for 6 months, and female partners were passively followed for a minimum of 9 months. Women who conceived were followed throughout pregnancy. Overall, 2,370 men were randomized during 2013-2017 (1,185 folic acid and zinc, 1,185 placebo); they had a mean age of 33 years and body mass index (weight (kg)/height (m)2) of 29.8. Most participants were white (82%), well educated (83% with some college), and employed (72%). Participant characteristics were balanced across intervention arms. Study visits were completed by 89%, 77%, and 75% of men at months 2, 4, and 6, respectively. Here we describe the study design, recruitment, data collection, lessons learned, and baseline participant characteristics.

Correcting for selective participation in cohort studies using auxiliary register data without identification of non-participants

Aims: Selective participation may hamper the validity of population-based cohort studies. The resulting bias can be alleviated by linking auxiliary register data to both the participants and the non-participants of the study, estimating propensity scores for participation and correcting for participation based on these. However, registry holders may not be allowed to disclose sensitive data on (invited) non-participants. Our aim is to provide guidance on how adequate bias correction can be achieved by using auxiliary register data but without disclosing information that could be linked to the subset of non-participants. Methods: We show how existing methods can be used to estimate generalisation weights under various data disclosure scenarios where invited non-participants are indistinguishable from uninvited ones. We also demonstrate how the methods can be implemented using Nordic register data. Results: Inverse-probability-of-sampling weights estimated within a random sample of the target population in which the non-respondents are disclosed are equivalent in expectation to analogous weights in a scenario where the non-participants and uninvited individuals from the population are indistinguishable. To minimise the risk of disclosure when the entire population is invited to participate, investigators should instead consider inverse-odds-of-sampling weights, a method that has previously been suggested for transporting study results to external populations. Conclusions: Generalisation weights can be estimated from auxiliary register data without disclosing information on invited non-participants.

Effect heterogeneity and variable selection for standardizing causal effects to a target population

The participants in randomized trials and other studies used for causal inference are often not representative of the populations seen by clinical decision-makers. To account for differences between populations, researchers may consider standardizing results to a target population. We discuss several different types of homogeneity conditions that are relevant for standardization: Homogeneity of effect measures, homogeneity of counterfactual outcome state transition parameters, and homogeneity of counterfactual distributions. Each of these conditions can be used to show that a particular standardization procedure will result in an unbiased estimate of the effect in the target population, given assumptions about the relevant scientific context. We compare and contrast the homogeneity conditions, in particular their implications for selection of covariates for standardization and their implications for how to compute the standardized causal effect in the target population. While some of the recently developed counterfactual approaches to generalizability rely upon homogeneity conditions that avoid many of the problems associated with traditional approaches, they often require adjustment for a large (and possibly unfeasible) set of covariates.

An outcome model approach to transporting a randomized controlled trial results to a target population

OBJECTIVE

Participants enrolled into randomized controlled trials (RCTs) often do not reflect real-world populations. Previous research in how best to transport RCT results to target populations has focused on weighting RCT data to look like the target data. Simulation work, however, has suggested that an outcome model approach may be preferable. Here, we describe such an approach using source data from the 2 × 2 factorial NAVIGATOR (Nateglinide And Valsartan in Impaired Glucose Tolerance Outcomes Research) trial, which evaluated the impact of valsartan and nateglinide on cardiovascular outcomes and new-onset diabetes in a prediabetic population.

MATERIALS AND METHODS

Our target data consisted of people with prediabetes serviced at the Duke University Health System. We used random survival forests to develop separate outcome models for each of the 4 treatments, estimating the 5-year risk difference for progression to diabetes, and estimated the treatment effect in our local patient populations, as well as subpopulations, and compared the results with the traditional weighting approach.

RESULTS

Our models suggested that the treatment effect for valsartan in our patient population was the same as in the trial, whereas for nateglinide treatment effect was stronger than observed in the original trial. Our effect estimates were more efficient than the weighting approach and we effectively estimated subgroup differences.

CONCLUSIONS

The described method represents a straightforward approach to efficiently transporting an RCT result to any target population.