Synthetic data method to incorporate external information into a current study

Trial emulation to assess the effect of surgery on survival when there are competing risks, with application to patients with thoracic aortic aneurysms

OBJECTIVES

This study extends methods to estimate average causal effect of aneurysm repair surgery on (i) overall survival and (ii) aneurysm-related mortality, accounting for competing risks using data from the Effective Treatment for Thoracic Aortic Aneurysm (ETTAA) cohort.

STUDY DESIGN AND SETTING

ETTAA, a prospective cohort study, recruited 886 patients between 2014 and 2018. Patients were linked to UK national hospital and mortality databases by National Health Service digital and followed-up for later surgeries and deaths. We compared a strategy of open or endovascular surgery (whichever appropriate) within 12 months of enrollment to ETTAA with no surgery within 12 months using the trial emulation framework and cloning-censoring-weighting (CCW) analysis. Key confounders at baseline were controlled for using inverse probability weighting methods.

RESULTS

In complete case analysis, if everyone received surgery within a 12-month grace period, an estimated 7-year survival probability was 57.4% (95% CI: 47.3%, 67.4%) vs 49.9% (44.0%, 55.0%) if no one received surgery. This benefit was primarily attributable to reduction in aneurysm-related deaths (difference -8.7%, 95% CI: -14.0%, -3.9%), with no significant effect on deaths from other causes. The findings were consistent under sensitivity analyses, including multiple imputation of missing confounders. Our CCW approach addressed selection-for-treatment, allowed for surgery to be received within a grace period, and used appropriate methods to separate aneurysm-related mortality from competing risks.

CONCLUSION

The study demonstrates the utility of trial emulation and counterfactual methods in estimation of causal effects on competing risks using observational data. The findings suggest a benefit for aneurysm-related survival up to 7 years after enrollment.

PLAIN LANGUAGE SUMMARY

This study shows how to estimate effects of surgery on different causes of death, when we cannot do a clinical trial, and illustrates this using an example from heart surgery. The aorta is the main artery that carries oxygen-rich blood from the heart to the body. In some people, a part of the vessel wall becomes weak and loses its elastic properties, so it doesn't return to its normal shape after the blood has passed through. This can lead to swelling or bulging in the aorta, called an aneurysm. A thoracic aortic aneurysm, or TAA for short, is an aneurysm in the section of the aorta in the chest (https://www.bhf.org.uk/informationsupport/conditions/thoracic-aortic-aneurysm). We have used data from the Effective Treatment for Thoracic Aortic Aneurysm (ETTAA) study, which investigated aneurysm growth rates, patient outcomes, quality of life, and costs, in 886 patients diagnosed with TAA. ETTAA compared two surgical treatments, Open Heart Surgery, where the section of the aorta that contains the aneurysm is removed and replaced by a new aorta made from a synthetic material, and Stent Grafting, where tubes are inserted into arteries to allow blood to flow freely using less invasive "keyhole" surgery. ETTAA reviewed existing research evidence but data comparing the effectiveness of these two approaches to each other and to outcomes without surgery were of sparse or limited quality and outdated. The results of ETTAA up to 2020 have been published in a monograph. (https://pubmed.ncbi.nlm.nih.gov/35094747/). Two findings from ETTAA motivated this study. First, there were no clinical trials comparing surgery with no surgery and no studies that mimic clinical trials. Second, we had not considered whether surgery overall prevents deaths due to aneurysm or deaths from other causes. We call these two types of death, competing risks. It is unlikely that a clinical trial comparing surgery with no surgery will ever be completed because the number of people who are diagnosed with TAA is small. Also, TAA can become a serious problem if left untreated. On the other hand, surgery for TAA is difficult and can result in serious complications, including death. Therefore, it is important to know how much surgery improves survival related to the aneurysm and whether it improves survival overall. Recent developments in statistics provided methods for investigating survival in a way which increases confidence in the cause-effect relationship between surgery and outcomes. In this study, we show how these statistical methods can be used to estimate the proportion of patients who die from the competing risks, if all patients had surgery within 12 months compared with if no patients had surgery within 12 months. We take into account the different times between diagnosis of TAA and surgery and adjust for the main differences between surgery and no surgery patients. Using these methods, we estimate that surgery reduces deaths due to aneurysms at 7 years by 8.7%, with no effect on deaths from other causes. The benefit of surgery was significant by 3 years after diagnosis. We also provide discussion about using routine medical records to repeat this type of study.

Improving prediction of linear regression models by integrating external information from heterogeneous populations: James-Stein estimators

We consider the setting where (1) an internal study builds a linear regression model for prediction based on individual-level data, (2) some external studies have fitted similar linear regression models that use only subsets of the covariates and provide coefficient estimates for the reduced models without individual-level data, and (3) there is heterogeneity across these study populations. The goal is to integrate the external model summary information into fitting the internal model to improve prediction accuracy. We adapt the James-Stein shrinkage method to propose estimators that are no worse and are oftentimes better in the prediction mean squared error after information integration, regardless of the degree of study population heterogeneity. We conduct comprehensive simulation studies to investigate the numerical performance of the proposed estimators. We also apply the method to enhance a prediction model for patella bone lead level in terms of blood lead level and other covariates by integrating summary information from published literature.

Robust data integration from multiple external sources for generalized linear models with binary outcomes

We aim to estimate parameters in a generalized linear model (GLM) for a binary outcome when, in addition to the raw data from the internal study, more than 1 external study provides summary information in the form of parameter estimates from fitting GLMs with varying subsets of the internal study covariates. We propose an adaptive penalization method that exploits the external summary information and gains efficiency for estimation, and that is both robust and computationally efficient. The robust property comes from exploiting the relationship between parameters of a GLM and parameters of a GLM with omitted covariates and from downweighting external summary information that is less compatible with the internal data through a penalization. The computational burden associated with searching for the optimal tuning parameter for the penalization is reduced by using adaptive weights and by using an information criterion when searching for the optimal tuning parameter. Simulation studies show that the proposed estimator is robust against various types of population distribution heterogeneity and also gains efficiency compared to direct maximum likelihood estimation. The method is applied to improve a logistic regression model that predicts high-grade prostate cancer making use of parameter estimates from 2 external models.

A synthetic data integration framework to leverage external summary-level information from heterogeneous populations

There is a growing need for flexible general frameworks that integrate individual-level data with external summary information for improved statistical inference. External information relevant for a risk prediction model may come in multiple forms, through regression coefficient estimates or predicted values of the outcome variable. Different external models may use different sets of predictors and the algorithm they used to predict the outcome Y given these predictors may or may not be known. The underlying populations corresponding to each external model may be different from each other and from the internal study population. Motivated by a prostate cancer risk prediction problem where novel biomarkers are measured only in the internal study, this paper proposes an imputation-based methodology, where the goal is to fit a target regression model with all available predictors in the internal study while utilizing summary information from external models that may have used only a subset of the predictors. The method allows for heterogeneity of covariate effects across the external populations. The proposed approach generates synthetic outcome data in each external population, uses stacked multiple imputation to create a long dataset with complete covariate information. The final analysis of the stacked imputed data is conducted by weighted regression. This flexible and unified approach can improve statistical efficiency of the estimated coefficients in the internal study, improve predictions by utilizing even partial information available from models that use a subset of the full set of covariates used in the internal study, and provide statistical inference for the external population with potentially different covariate effects from the internal population.

Improvement of an External Predictive Model Based on New Information Using a Synthetic Data Approach: Application to CADASIL

Background and Objectives

Cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy (CADASIL) is the most frequent hereditary cerebral small vessel disease. It is caused by mutations of the NOTCH3 gene. The disease evolves progressively over decades leading to stroke, disability, cognitive decline, and functional dependency. The course and clinical severity of CADASIL seem heterogeneous. Predictive models are thus needed to improve prognostic evaluation and inform future clinical trials. A predictive model of the 3-year variation in the Mattis Dementia Rating Scale (MDRS), which reflects the global cognitive performance of patients with CADASIL, was previously proposed. This model made predictions based on demographic, clinical, and MRI data. We aimed to improve this existing predictive model by integrating a new potential factor, the location of the genetic mutation in the different epidermal growth factor (EGFr) domains of the NOTCH3 gene, dichotomized into EGFr domains 1 to 6 or 7 to 34.

Methods

We used a new synthetic data approach to improve the initial predictive model by incorporating additional genetic information. This method combined the predicted outcomes from the previous model and 5 "synthetic" data sets with the observed outcome in a new data set. We then applied a multiple imputation method for missing data on the mutation location.

Results

The new data set included 367 patients who were followed up for 30 to 42 months. In the multivariable model with synthetic data, patients with NOTCH3 mutations in EGFr domains 7 to 34 had an additional average decrease of -1.4 points (standard error 0.67, p = 0.035) in their MDRS score variation over 3 years compared with patients with mutations located in EGFr domains 1 to 6. Cross-validation results highlighted the improved predictive performance of the enhanced model. Moreover, the model estimation was found to be more robust than fitting a model without synthetic data.

Discussion

The use of synthetic data improved the predictive model of MDRS change over 3 years in CADASIL. The predictive performance and estimation robustness of the predictive model were enhanced using this approach, whether genetic information was used. A statistically significant association between the location of the mutation in the NOTCH3 gene and the 3-year MDRS score variation was detected.

Integrating Information from Existing Risk Prediction Models with No Model Details

Consider the setting where (i) individual-level data are collected to build a regression model for the association between an event of interest and certain covariates, and (ii) some risk calculators predicting the risk of the event using less detailed covariates are available, possibly as algorithmic black boxes with little information available about how they were built. We propose a general empirical-likelihood-based framework to integrate the rich auxiliary information contained in the calculators into fitting the regression model, to make the estimation of regression parameters more efficient. Two methods are developed, one using working models to extract the calculator information and one making a direct use of calculator predictions without working models. Theoretical and numerical investigations show that the calculator information can substantially reduce the variance of regression parameter estimation. As an application, we study the dependence of the risk of high grade prostate cancer on both conventional risk factors and newly identified molecular biomarkers by integrating information from the Prostate Biopsy Collaborative Group (PBCG) risk calculator, which was built based on conventional risk factors alone.

Data integration: exploiting ratios of parameter estimates from a reduced external model

We consider the situation of estimating the parameters in a generalized linear prediction model, from an internal dataset, where the outcome variable [Formula: see text] is binary and there are two sets of covariates, [Formula: see text] and [Formula: see text]. We have information from an external study that provides parameter estimates for a generalized linear model of [Formula: see text] on [Formula: see text]. We propose a method that makes limited assumptions about the similarity of the distributions in the two study populations. The method involves orthogonalizing the [Formula: see text] variables and then borrowing information about the ratio of the coefficients from the external model. The method is justified based on a new result relating the parameters in a generalized linear model to the parameters in a generalized linear model with omitted covariates. The method is applicable if the regression coefficients in the [Formula: see text] given [Formula: see text] model are similar in the two populations, up to an unknown scalar constant. This type of transportability between populations is something that can be checked from the available data. The asymptotic variance of the proposed method is derived. The method is evaluated in a simulation study and shown to gain efficiency compared to simple analysis of the internal dataset, and is robust compared to an alternative method of incorporating external information.

COMMUTE: Communication-efficient transfer learning for multi-site risk prediction

OBJECTIVES

We propose a communication-efficient transfer learning approach (COMMUTE) that effectively incorporates multi-site healthcare data for training a risk prediction model in a target population of interest, accounting for challenges including population heterogeneity and data sharing constraints across sites.

METHODS

We first train population-specific source models locally within each site. Using data from a given target population, COMMUTE learns a calibration term for each source model, which adjusts for potential data heterogeneity through flexible distance-based regularizations. In a centralized setting where multi-site data can be directly pooled, all data are combined to train the target model after calibration. When individual-level data are not shareable in some sites, COMMUTE requests only the locally trained models from these sites, with which, COMMUTE generates heterogeneity-adjusted synthetic data for training the target model. We evaluate COMMUTE via extensive simulation studies and an application to multi-site data from the electronic Medical Records and Genomics (eMERGE) Network to predict extreme obesity.

RESULTS

Simulation studies show that COMMUTE outperforms methods without adjusting for population heterogeneity and methods trained in a single population over a broad spectrum of settings. Using eMERGE data, COMMUTE achieves an area under the receiver operating characteristic curve (AUC) around 0.80, which outperforms other benchmark methods with AUC ranging from 0.51 to 0.70.

CONCLUSION

COMMUTE improves the risk prediction in a target population with limited samples and safeguards against negative transfer when some source populations are highly different from the target. In a federated setting, it is highly communication efficient as it only requires each site to share model parameter estimates once, and no iterative communication or higher-order terms are needed.

Improving main analysis by borrowing information from auxiliary data

In many clinical and observational studies, auxiliary data from the same subjects, such as repeated measurements or surrogate variables, will be collected in addition to the data of main interest. Not directly related to the main study, these auxiliary data in practice are rarely incorporated into the main analysis, though they may carry extra information that can help improve the estimation in the main analysis. Under the setting where part of or all subjects have auxiliary data available, we propose an effective weighting approach to borrow the auxiliary information by building a working model for the auxiliary data, where improvement of estimation precision over the main analysis is guaranteed regardless of the specification of the working model. An information index is also constructed to assess how well the selected working model works to improve the main analysis. Both theoretical and numerical studies show the excellent and robust performance of the proposed method in comparison to estimation without using the auxiliary data. Finally, we utilize the Atherosclerosis Risk in Communities study for illustration.

A meta-inference framework to integrate multiple external models into a current study

It is becoming increasingly common for researchers to consider incorporating external information from large studies to improve the accuracy of statistical inference instead of relying on a modestly sized data set collected internally. With some new predictors only available internally, we aim to build improved regression models based on individual-level data from an "internal" study while incorporating summary-level information from "external" models. We propose a meta-analysis framework along with two weighted estimators as the composite of empirical Bayes estimators, which combines the estimates from different external models. The proposed framework is flexible and robust in the ways that (i) it is capable of incorporating external models that use a slightly different set of covariates; (ii) it is able to identify the most relevant external information and diminish the influence of information that is less compatible with the internal data; and (iii) it nicely balances the bias-variance trade-off while preserving the most efficiency gain. The proposed estimators are more efficient than the naïve analysis of the internal data and other naïve combinations of external estimators.