Point: Immortal Time Bias-What Are the Determinants of Its Magnitude?

Methodological biases in observational hospital studies of COVID-19 treatment effectiveness: pitfalls and potential

Introduction

This study aims to discuss and assess the impact of three prevalent methodological biases: competing risks, immortal-time bias, and confounding bias in real-world observational studies evaluating treatment effectiveness. We use a demonstrative observational data example of COVID-19 patients to assess the impact of these biases and propose potential solutions.

Methods

We describe competing risks, immortal-time bias, and time-fixed confounding bias by evaluating treatment effectiveness in hospitalized patients with COVID-19. For our demonstrative analysis, we use observational data from the registry of patients with COVID-19 who were admitted to the Bellvitge University Hospital in Spain from March 2020 to February 2021 and met our predefined inclusion criteria. We compare estimates of a single-dose, time-dependent treatment with the standard of care. We analyze the treatment effectiveness using common statistical approaches, either by ignoring or only partially accounting for the methodological biases. To address these challenges, we emulate a target trial through the clone-censor-weight approach.

Results

Overlooking competing risk bias and employing the naïve Kaplan-Meier estimator led to increased in-hospital death probabilities in patients with COVID-19. Specifically, in the treatment effectiveness analysis, the Kaplan-Meier estimator resulted in an in-hospital mortality of 45.6% for treated patients and 59.0% for untreated patients. In contrast, employing an emulated trial framework with the weighted Aalen-Johansen estimator, we observed that in-hospital death probabilities were reduced to 27.9% in the "X"-treated arm and 40.1% in the non-"X"-treated arm. Immortal-time bias led to an underestimated hazard ratio of treatment.

Conclusion

Overlooking competing risks, immortal-time bias, and confounding bias leads to shifted estimates of treatment effects. Applying the naïve Kaplan-Meier method resulted in the most biased results and overestimated probabilities for the primary outcome in analyses of hospital data from COVID-19 patients. This overestimation could mislead clinical decision-making. Both immortal-time bias and confounding bias must be addressed in assessments of treatment effectiveness. The trial emulation framework offers a potential solution to address all three methodological biases.

Comparing broad and narrow phenotype algorithms: differences in performance characteristics and immortal time incurred

Introduction: When developing phenotype algorithms for observational research, there is usually a trade-off between definitions that are sensitive or specific. The objective of this study was to estimate the performance characteristics of phenotype algorithms designed for increasing specificity and to estimate the immortal time associated with each algorithm. Materials and methods: We examined algorithms for 11 chronic health conditions. The analyses were from data from five databases. For each health condition, we created five algorithms to examine performance (sensitivity and positive predictive value (PPV)) differences: one broad algorithm using a single code for the health condition and four narrow algorithms where a second diagnosis code was required 1-30 days, 1-90 days, 1-365 days, or 1- all days in a subject's continuous observation period after the first code. We also examined the proportion of immortal time relative to time-at-risk (TAR) for four outcomes. The TAR's were: 0-30 days after the first condition occurrence (the index date), 0-90 days post-index, 0-365 days post-index, and 0-1,095 days post-index. Performance of algorithms for chronic health conditions was estimated using PheValuator (V2.1.4) from the OHDSI toolstack. Immortal time was calculated as the time from the index date until the first of the following: 1) the outcome; 2) the end of the outcome TAR; 3) the occurrence of the second code for the chronic health condition. Results: In the first analysis, the narrow phenotype algorithms, i.e., those requiring a second condition code, produced higher estimates for PPV and lower estimates for sensitivity compared to the single code algorithm. In all conditions, increasing the time to the required second code increased the sensitivity of the algorithm. In the second analysis, the amount of immortal time increased as the window used to identify the second diagnosis code increased. The proportion of TAR that was immortal was highest in the 30 days TAR analyses compared to the 1,095 days TAR analyses. Conclusion: Attempting to increase the specificity of a health condition algorithm by adding a second code is a potentially valid approach to increase specificity, albeit at the cost of incurring immortal time.

Immortal time bias for life-long conditions in retrospective observational studies using electronic health records

Background

Immortal time bias is common in observational studies but is typically described for pharmacoepidemiology studies where there is a delay between cohort entry and treatment initiation.

Methods

This study used the Clinical Practice Research Datalink (CPRD) and linked national mortality data in England from 2000 to 2019 to investigate immortal time bias for a specific life-long condition, intellectual disability. Life expectancy (Chiang’s abridged life table approach) was compared for 33,867 exposed and 980,586 unexposed individuals aged 10+ years using five methods: (1) treating immortal time as observation time; (2) excluding time before date of first exposure diagnosis; (3) matching cohort entry to first exposure diagnosis; (4) excluding time before proxy date of inputting first exposure diagnosis (by the physician); and (5) treating exposure as a time-dependent measure.

Results

When not considered in the design or analysis (Method 1), immortal time bias led to disproportionately high life expectancy for the exposed population during the first calendar period (additional years expected to live: 2000–2004: 65.6 [95% CI: 63.6,67.6]) compared to the later calendar periods (2005–2009: 59.9 [58.8,60.9]; 2010–2014: 58.0 [57.1,58.9]; 2015–2019: 58.2 [56.8,59.7]). Date of entry of diagnosis (Method 4) was unreliable in this CPRD cohort. The final methods (Method 2, 3 and 5) appeared to solve the main theoretical problem but residual bias may have remained.

Conclusions

We conclude that immortal time bias is a significant issue for studies of life-long conditions that use electronic health record data and requires careful consideration of how clinical diagnoses are entered onto electronic health record systems.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01581-1.

Pharmacoepidemiology for nephrologists (part 2): potential biases and how to overcome them

Observational pharmacoepidemiological studies using routinely collected healthcare data are increasingly being used in the field of nephrology to answer questions on the effectiveness and safety of medications. This review discusses a number of biases that may arise in such studies and proposes solutions to minimize them during the design or statistical analysis phase. We first describe designs to handle confounding by indication (e.g. active comparator design) and methods to investigate the influence of unmeasured confounding, such as the E-value, the use of negative control outcomes and control cohorts. We next discuss prevalent user and immortal time biases in pharmacoepidemiology research and how these can be prevented by focussing on incident users and applying either landmarking, using a time-varying exposure, or the cloning, censoring and weighting method. Lastly, we briefly discuss the common issues with missing data and misclassification bias. When these biases are properly accounted for, pharmacoepidemiological observational studies can provide valuable information for clinical practice.

Conditional survival in patients with esophageal or gastroesophageal junction cancer after receiving various treatment modalities

Background

To estimate the adjusted conditional overall survival (COS) in patients with esophageal cancer after receiving various treatment modalities via a national population‐based database, and to investigate the possible time‐dependent effects.

Materials and Methods

Eligible patients diagnosed with esophageal cancer between 2000 and 2016 were identified from the Surveillance, Epidemiology, and End Results (SEER) registry. The Kaplan‐Meier method was used to calculate conventional survival time. The inverse probability of treatment weighting method was used to estimate the adjusted COS in patients receiving different treatment modalities. Landmark analysis was employed to investigate the possible time‐dependent effects of different treatment modalities in patients who had survived a certain period of time.

Results

A total of 25,232 patients were included in the final analysis. The conventional 5‐year overall survival was 19.3%. The 5‐year adjusted COS increased most for the first 3 years, and increased slightly afterwards. In patients with regional esophageal or gastroesophageal junction cancer, stage‐specific analysis showed that surgery only and preoperative radiation therapy benefited most for patients with localized disease, preoperative radiation therapy plus surgery benefited regional, and preoperative radiation therapy plus surgery benefited distant disease, with the 5‐year adjusted COS given patients had survived 3 years being 67.0% (95% CI 65.2%–68.7%), 59.9% (95% CI 58.3%–61.5%), 58.4% (95% CI 56.3%–60.5%), and 61.8% (95% CI 59.5%–64.1%), respectively. In time‐dependent analysis, the benefits of surgery only in localized cases were prominent within 48 months after diagnosis. Preoperative radiation therapy showed long‐lasting benefits in patients with regional disease. In patients with distant disease, all treatment modalities showed similar and short‐term effects.

Conclusions

The adjusted COS in patients with esophageal cancer increased as time accrued after receiving various treatment modalities. The time‐dependent effects in specific tumor stage provided a dynamic view on optimization of treatment strategies.

Counterpoint: Keeping the Demons at Bay When Handling Time-Varying Exposures-Beyond Avoiding Immortal Person-Time

The potential for immortal time bias is pervasive in epidemiologic studies with left truncation or time-varying exposures. Unlike other biases in epidemiologic research (e.g., measurement bias, confounding due to unmeasured factors, and selection based on unmeasured predictors of the outcome), immortal time bias can and should be avoided by the correct assignment of person-time during follow up. However, even when handing person-time correctly, allowing late entry into a study or into an exposure group can open the door to more insidious sources of bias, some of which we explore here. Clear articulation of the study question, including the treatment plans of interest, can provide navigation around these sources of bias and elucidate the assumptions needed for inference given the available data. Here, we use simulated data to illustrate the assumptions required under various approaches to estimate the effect of a time-varying treatment and describe how these assumptions relate to the assumptions necessary to estimate single sample rates and risks in settings with censoring and truncation.