Transparency of high-dimensional propensity score analyses: guidance for diagnostics and reporting

Identifying markers of health-seeking behaviour and healthcare access in UK electronic health records

OBJECTIVE

To assess the feasibility of identifying markers of health-seeking behaviour and healthcare access in UK electronic health records (EHR), for identifying populations at risk of poor health outcomes and adjusting for confounding in epidemiological studies.

DESIGN

Cross-sectional observational study using the Clinical Practice Research Datalink Aurum prelinked to Hospital Episode Statistics.

SETTING

Individual-level routine clinical data from 13 million patients across general practices (GPs) and secondary data in England.

PARTICIPANTS

Individuals aged ≥66 years on 1 September 2019.

MAIN OUTCOME MEASURES

We used the Theory of Planned Behaviour (TPB) model and the literature to iteratively develop criteria for markers selection. Based on this we selected 15 markers: those that represented uptake of public health interventions, markers of active healthcare access/use and markers of lack of access/underuse. We calculated the prevalence of each marker using relevant lookback periods prior to the index date (1 September 2019) and compared with national estimates. We assessed the correlation coefficients (phi) between markers with inferred hierarchical clustering.

RESULTS

We included 1 991 284 individuals (mean age: 75.9 and 54.0% women). The prevalence of markers ranged from <0.1% (low-value prescriptions) to 92.6% (GP visits), and most were in line with national estimates; for example, 73.3% for influenza vaccination in the 2018/2019 season, compared with 72.4% in national estimates. Screening markers, for example, abdominal aortic aneurysm screening were under-recorded even in age-eligible groups (54.3% in 65-69 years old vs 76.1% in national estimates in men). Overall, marker correlations were low (<0.5) and clustered into groups according to underlying determinants from the TPB model.

CONCLUSION

Overall, markers of health-seeking behaviour and healthcare access can be identified in UK EHRs. The generally low correlations between different markers of health-seeking behaviour and healthcare access suggest a range of variables are needed to capture different determinants of healthcare use.

Application of High-Dimensional Propensity Score Methods to the National Health and Aging Trends Study

BACKGROUND

High-dimensional propensity scoring (HDPS) is a method for empirically identifying potential confounders within large healthcare databases such as administrative claims data. However, this method has not yet been applied to large national health surveys such as the National Health and Aging Trends Study (NHATS), an ongoing nationally representative survey of older adults in the United States and important resource in gerontology research.

METHODS

In this Research Practice article, we present an overview of HDPS and describe the specific data transformation steps and analytic considerations needed to apply it to national health surveys. We applied HDPS within NHATS to investigate the association between self-reported visual difficulty and incident dementia, comparing HDPS to conventional confounder selection methods.

RESULTS

Among 7 207 dementia-free NHATS Wave 1 respondents, 528 (7.3%) had self-reported visual difficulty. In an unadjusted discrete time proportional hazards model accounting for the complex survey design of NHATS, self-reported visual difficulty was strongly associated with incident dementia (odds ratio [OR] 2.34, 95% confidence interval [CI]: 1.95-2.81). After adjustment for standard investigator-selected covariates via inverse probability weighting, the magnitude of this association decreased, but evidence of an association remained (OR 1.44, 95% CI: 1.11-1.85). Adding 75 HDPS-prioritized variables to the investigator-selected propensity score model resulted in further attenuation of the association between visual impairment and dementia (OR 0.94, 95% CI: 0.70-1.23).

CONCLUSIONS

HDPS can be successfully applied to national health surveys such as NHATS and may improve confounder adjustment. We hope developing this framework will encourage future consideration of HDPS in this setting.

Improving Inverse Probability Weighting by Post-calibrating Its Propensity Scores

Theoretical guarantees for causal inference using propensity scores are partially based on the scores behaving like conditional probabilities. However, prediction scores between zero and one do not necessarily behave like probabilities, especially when output by flexible statistical estimators. We perform a simulation study to assess the error in estimating the average treatment effect before and after applying a simple and well-established postprocessing method to calibrate the propensity scores. We observe that postcalibration reduces the error in effect estimation and that larger improvements in calibration result in larger improvements in effect estimation. Specifically, we find that expressive tree-based estimators, which are often less calibrated than logistic regression-based models initially, tend to show larger improvements relative to logistic regression-based models. Given the improvement in effect estimation and that postcalibration is computationally cheap, we recommend its adoption when modeling propensity scores with expressive models.

A methodological review of the high-dimensional propensity score in comparative-effectiveness and safety-of-interventions research finds incomplete reporting relative to algorithm development and robustness

OBJECTIVES

The use of secondary databases has become popular for evaluating the effectiveness and safety of interventions in real-life settings. However, the absence of important confounders in these databases is challenging. To address this issue, the high-dimensional propensity score (hdPS) algorithm was developed in 2009. This algorithm uses proxy variables for mitigating confounding by combining information available across several healthcare dimensions. This study assessed the methodology and reporting of the hdPS in comparative effectiveness and safety research.

STUDY DESIGN AND SETTING

In this methodological review, we searched PubMed and Google Scholar from July 2009 to May 2022 for studies that used the hdPS for evaluating the effectiveness or safety of healthcare interventions. Two reviewers independently extracted study characteristics and assessed how the hdPS was applied and reported. Risk of bias was evaluated with the Risk Of Bias In Non-randomised Studies - of Interventions (ROBINS-I) tool.

RESULTS

In total, 136 studies met the inclusion criteria; the median publication year was 2018 (Q1-Q3 2016-2020). The studies included 192 datasets, mostly North American databases (n = 132, 69%). The hdPS was used in primary analysis in 120 studies (88%). Dimensions were defined in 101 studies (74%), with a median of 5 (Q1-Q3 4-6) dimensions included. A median of 500 (Q1-Q3 200-500) empirically identified covariates were selected. Regarding hdPS reporting, only 11 studies (8%) reported all recommended items. Most studies (n = 81, 60%) had a moderate overall risk of bias.

CONCLUSION

There is room for improvement in the reporting of hdPS studies, especially regarding the transparency of methodological choices that underpin the construction of the hdPS.

A population-based cohort of drug exposures and adverse pregnancy outcomes in China (DEEP): rationale, design, and baseline characteristics

The DEEP cohort is the first population-based cohort of pregnant population in China that longitudinally documented drug uses throughout the pregnancy life course and adverse pregnancy outcomes. The main goal of the study aims to monitor and evaluate the safety of drug use through the pregnancy life course in the Chinese setting. The DEEP cohort is developed primarily based on the population-based data platforms in Xiamen, a municipal city of 5 million population in southeast China. Based on these data platforms, we developed a pregnancy database that documented health care services and outcomes in the maternal and other departments. For identifying drug uses, we developed a drug prescription database using electronic healthcare records documented in the platforms across the primary, secondary and tertiary hospitals. By linking these two databases, we developed the DEEP cohort. All the pregnant women and their offspring in Xiamen are provided with health care and followed up according to standard protocols, and the primary adverse outcomes - congenital malformations - are collected using a standardized Case Report Form. From January 2013 to December 2021, the DEEP cohort included 564,740 pregnancies among 470,137 mothers, and documented 526,276 live births, 14,090 miscarriages and 6,058 fetal deaths/stillbirths and 25,723 continuing pregnancies. In total, 13,284,982 prescriptions were documented, in which 2,096 chemicals drugs, 163 biological products, 847 Chinese patent medicines and 655 herbal medicines were prescribed. The overall incidence rate of congenital malformations was 2.0% (10,444/526,276), while there were 25,526 (4.9%) preterm births and 25,605 (4.9%) live births with low birth weight.

Associations Between Postdischarge Care and Cognitive Impairment-Related Hospital Readmissions for Ketoacidosis and Severe Hypoglycemia in Adults With Diabetes

OBJECTIVE

Patients with severe hypoglycemia (SH) or diabetic ketoacidosis (DKA) experience high hospital readmission after being discharged. Cognitive impairment (CI) may further increase the risk, especially in those experiencing an interruption of medical care after discharge. This study examined the effect modification role of postdischarge care (PDC) on CI-associated readmission risk among U.S. adults with diabetes initially admitted for DKA or SH.

RESEARCH DESIGN AND METHODS

We used the Nationwide Readmissions Database (NRD) (2016-2018) to identify individuals hospitalized with a diagnosis of DKA or SH. Multivariate Cox regression was used to compare the all-cause readmission risk at 30 days between those with and without CI identified during the initial hospitalization. We assessed the CI-associated readmission risk in the patients with and without PDC, an effect modifier with the CI status.

RESULTS

We identified 23,775 SH patients (53.3% women, mean age 65.9 ± 15.3 years) and 140,490 DKA patients (45.8% women, mean age 40.3 ± 15.4 years), and 2,675 (11.2%) and 1,261 (0.9%), respectively, had a CI diagnosis during their index hospitalization. For SH and DKA patients discharged without PDC, CI was associated with a higher readmission risk of 23% (adjusted hazard ratio [aHR] 1.23, 95% confidence interval 1.08-1.40) and 35% (aHR 1.35, 95% confidence interval 1.08-1.70), respectively. However, when patients were discharged with PDC, we found PDC was an effect modifier to mitigate CI-associated readmission risk for both SH and DKA patients (P < 0.05 for all).

CONCLUSIONS

Our results suggest that PDC can potentially mitigate the excessive readmission risk associated with CI, emphasizing the importance of postdischarge continuity of care for medically complex patients with comorbid diabetes and CI.

Prevalent new user designs: A literature review of current implementation practice

PURPOSE

Prevalent new user (PNU) designs extend the active comparator new user design by allowing for the inclusion of initiators of the study drug who were previously on a comparator treatment. We performed a literature review summarising current practice.

METHODS

PubMed was searched for studies applying the PNU design since its proposal in 2017. The review focused on three components. First, we extracted information on the overall study design, including the database used. We summarised information on implementation of the PNU design, including key decisions relating to exposure set definition and estimation of time-conditional propensity scores. Finally, we reviewed the analysis strategy of the matched cohort.

RESULTS

Nineteen studies met the criteria for inclusion. Most studies (73%) implemented the PNU design in electronic health record or registry databases, with the remaining using insurance claims databases. Of 15 studies including a class of prevalent users, 40% deviated from the original exposure set definition proposals in favour of a more complex definition. Four studies did not include prevalent new users but used other aspects of the PNU framework. Several studies lacked details on exposure set definition (n = 2), time-conditional propensity score model (n = 2) or integration of complex analytical techniques, such as the high-dimensional propensity score algorithm (n = 3).

CONCLUSION

PNU designs have been applied in a range of therapeutic and disease areas. However, to encourage more widespread use of this design and help shape best practice, there is a need for improved accessibility, specifically through the provision of analytical code alongside guidance to support implementation and transparent reporting.

High-dimensional propensity scores for empirical covariate selection in secondary database studies: Planning, implementation, and reporting

Real-world evidence used for regulatory, payer, and clinical decision-making requires principled epidemiology in design and analysis, applying methods to minimize confounding given the lack of randomization. One technique to deal with potential confounding is propensity score (PS) analysis, which allows for the adjustment for measured preexposure covariates. Since its first publication in 2009, the high-dimensional propensity score (hdPS) method has emerged as an approach that extends traditional PS covariate selection to include large numbers of covariates that may reduce confounding bias in the analysis of healthcare databases. hdPS is an automated, data-driven analytic approach for covariate selection that empirically identifies preexposure variables and proxies to include in the PS model. This article provides an overview of the hdPS approach and recommendations on the planning, implementation, and reporting of hdPS used for causal treatment-effect estimations in longitudinal healthcare databases. We supply a checklist with key considerations as a supportive decision tool to aid investigators in the implementation and transparent reporting of hdPS techniques, and to aid decision-makers unfamiliar with hdPS in the understanding and interpretation of studies employing this approach. This article is endorsed by the International Society for Pharmacoepidemiology.

Visualizations throughout pharmacoepidemiology study planning, implementation, and reporting

Transparency is increasingly promoted to instill trust in nonrandomized studies using real-world data. Graphics and data visualizations support transparency by aiding communication and understanding, and can inform study design and analysis decisions. However, other than graphical representation of a study design and flow diagrams (e.g., a Consolidated Standards of Reporting Trials [CONSORT] like diagram), specific standards on how to maximize validity and transparency with visualization are needed. This paper provides guidance on how to use visualizations throughout the life cycle of a pharmacoepidemiology study-from initial study design to final report-to facilitate rationalized and transparent decision-making about study design and implementation, and clear communication of study findings. Our intent is to help researchers align their practices with current consensus statements on transparency.