Validation of an Algorithm to Ascertain Late Breast Cancer Recurrence Using Danish Medical Registries

Evaluation of Algorithms Using Automated Health Plan Data to Identify Breast Cancer Recurrences

BACKGROUND

We updated algorithms to identify breast cancer recurrences from administrative data, extending previously developed methods.

METHODS

In this validation study, we evaluated pairs of breast cancer recurrence algorithms (vs. individual algorithms) to identify recurrences. We generated algorithm combinations that categorized discordant algorithm results as no recurrence [High Specificity and PPV (positive predictive value) Combination] or recurrence (High Sensitivity Combination). We compared individual and combined algorithm results to manually abstracted recurrence outcomes from a sample of 600 people with incident stage I-IIIA breast cancer diagnosed between 2004 and 2015. We used Cox regression to evaluate risk factors associated with age- and stage-adjusted recurrence rates using different recurrence definitions, weighted by inverse sampling probabilities.

RESULTS

Among 600 people, we identified 117 recurrences using the High Specificity and PPV Combination, 505 using the High Sensitivity Combination, and 118 using manual abstraction. The High Specificity and PPV Combination had good specificity [98%, 95% confidence interval (CI): 97-99] and PPV (72%, 95% CI: 63-80) but modest sensitivity (64%, 95% CI: 44-80). The High Sensitivity Combination had good sensitivity (80%, 95% CI: 49-94) and specificity (83%, 95% CI: 80-86) but low PPV (29%, 95% CI: 25-34). Recurrence rates using combined algorithms were similar in magnitude for most risk factors.

CONCLUSIONS

By combining algorithms, we identified breast cancer recurrences with greater PPV than individual algorithms, without additional review of discordant records.

IMPACT

Researchers should consider tradeoffs between accuracy and manual chart abstraction resources when using previously developed algorithms. We provided guidance for future studies that use breast cancer recurrence algorithms with or without supplemental manual chart abstraction.

Machine Learning Algorithm to Estimate Distant Breast Cancer Recurrence at the Population Level with Administrative Data

Purpose

High-quality population-based cancer recurrence data are scarcely available, mainly due to complexity and cost of registration. For the first time in Belgium, we developed a tool to estimate distant recurrence after a breast cancer diagnosis at the population level, based on real-world cancer registration and administrative data.

Methods

Data on distant cancer recurrence (including progression) from patients diagnosed with breast cancer between 2009-2014 were collected from medical files at 9 Belgian centers to train, test and externally validate an algorithm (i.e., gold standard). Distant recurrence was defined as the occurrence of distant metastases between 120 days and within 10 years after the primary diagnosis, with follow-up until December 31, 2018. Data from the gold standard were linked to population-based data from the Belgian Cancer Registry (BCR) and administrative data sources. Potential features to detect recurrences in administrative data were defined based on expert opinion from breast oncologists, and subsequently selected using bootstrap aggregation. Based on the selected features, classification and regression tree (CART) analysis was performed to construct an algorithm for classifying patients as having a distant recurrence or not.

Results

A total of 2507 patients were included of whom 216 had a distant recurrence in the clinical data set. The performance of the algorithm showed sensitivity of 79.5% (95% CI 68.8-87.8%), positive predictive value (PPV) of 79.5% (95% CI 68.8-87.8%), and accuracy of 96.7% (95% CI 95.4-97.7%). The external validation resulted in a sensitivity of 84.1% (95% CI 74.4-91.3%), PPV of 84.1% (95% CI 74.4-91.3%), and an accuracy of 96.8% (95% CI 95.4-97.9%).

Conclusion

Our algorithm detected distant breast cancer recurrences with an overall good accuracy of 96.8% for patients with breast cancer, as observed in the first multi-centric external validation exercise.

A Validated Algorithm for Register-Based Identification of Patients with Relapse of Clinical Stage I Testicular Cancer

Purpose

The Danish Testicular Cancer (DaTeCa) database aims to monitor and improve quality of care for testicular cancer patients. Relapse data registered in the DaTeCa database rely on manual registration. Currently, some safeguarding against missing registrations is attempted by a non-validated register-based algorithm. However, this algorithm is inaccurate and entails time-consuming medical record reviews. We aimed (1) to validate relapse data as registered in the DaTeCa database, and (2) to develop and validate an improved register-based algorithm identifying patients diagnosed with relapse of clinical stage I testicular cancer.

Patients and Methods

Patients registered in the DaTeCa database with clinical stage I testicular cancer from 2013 to 2018 were included. Medical record information on relapse data served as a gold standard. A pre-specified algorithm to identify relapse was tested and optimized on a random sample of 250 patients. Indicators of relapse were obtained from pathology codes in the Danish National Pathology Register and from diagnosis and procedure codes in the Danish National Patient Register. We applied the final algorithm to the remaining study population to validate its performance.

Results

Of the 1377 included patients, 284 patients relapsed according to the gold standard during a median follow-up time of 5.9 years. The completeness of relapse data registered in the DaTeCa database was 97.2% (95% confidence interval (CI): 95.2-99.1). The algorithm achieved a sensitivity of 99.6% (95% CI: 98.7-100), a specificity of 98.9% (95% CI: 98.2-99.6), and a positive predictive value of 95.9% (95% CI: 93.4-98.4) in the validation cohort (n = 1127, 233 relapses).

Conclusion

The registration of relapse data in the DaTeCa database is accurate, confirming the database as a reliable source for ongoing clinical quality assessments. Applying the provided algorithm to the DaTeCa database will optimize the accuracy of relapse data further, decrease time-consuming medical record review and contribute to important future clinical research.

A Validated Register-Based Algorithm to Identify Patients Diagnosed with Recurrence of Surgically Treated Stage I Lung Cancer in Denmark

Introduction

Recurrence of cancer is not routinely registered in Danish national health registers. This study aimed to develop and validate a register-based algorithm to identify patients diagnosed with recurrent lung cancer and to estimate the accuracy of the identified diagnosis date.

Material and Methods

Patients with early-stage lung cancer treated with surgery were included in the study. Recurrence indicators were diagnosis and procedure codes recorded in the Danish National Patient Register and pathology results recorded in the Danish National Pathology Register. Information from CT scans and medical records served as the gold standard to assess the accuracy of the algorithm.

Results

The final population consisted of 217 patients; 72 (33%) had recurrence according to the gold standard. The median follow-up time since primary lung cancer diagnosis was 29 months (interquartile interval: 18-46). The algorithm for identifying a recurrence reached a sensitivity of 83.3% (95% CI: 72.7-91.1), a specificity of 93.8% (95% CI: 88.5-97.1), and a positive predictive value of 87.0% (95% CI: 76.7-93.9). The algorithm identified 70% of the recurrences within 60 days of the recurrence date registered by the gold standard method. The positive predictive value of the algorithm decreased to 70% when the algorithm was simulated in a population with a recurrence rate of 15%.

Conclusion

The proposed algorithm demonstrated good performance in a population with 33% recurrences over a median of 29 months. It can be used to identify patients diagnosed with recurrent lung cancer, and it may be a valuable tool for future research in this field. However, a lower positive predictive value is seen when applying the algorithm in populations with low recurrence rates.

10-year survival in female breast cancer patients according to ER, PR and HER2 expression: a cancer registry population-based analysis

INTRODUCTION

Invasive breast cancer prognosis has significantly improved over time; however, there are few data about the long-term survival.

MATERIALS AND METHODS

We analysed the data on female breast cancer incident during 2004-2005 in the area of the Tuscan Cancer Registry, distinguishing them in five subtypes, according to ER, PgR, HER2, and Ki67 expression: luminal A, luminal B, luminal B/HER2 + , triple-negative, and HER2 + . Effects of subtypes and age on 10 years breast cancer specific survival were analysed by Kaplan-Meier and multivariate Cox analysis.

RESULTS

The majority of breast cancer were luminal B (57%), and 45% of them were diagnosed at pathological stage I. The 10-year survival rates (p < 0.001) were higher among luminal A (90.2%) and lower among HER-2 + patients (70.3%). Prognostic effect of age was statistically significant (p < 0.0004): the 10-year cancer specific survival rates were higher among 40-59 years of age patients (88.5%), lower among 0-39 (75.8%). Luminal A breast cancer patients had a constant low risk throughout 10 years of follow up, while luminal B/HER2 + and triple negative tumours showed a peak 5 years after the diagnosis and then declined.

DISCUSSION

Our study confirmed the prognostic effect of biological subtype also in a long term follow up study; moreover, age at diagnosis showed to influence the outcome, other than stage at diagnosis and treatment. The long term follow up showed a constant risk of death for luminal A and B tumours, whereas for non-luminal cancer a peak 5 years after the diagnosis was found.

Mortality After Late Breast Cancer Recurrence in Denmark

PURPOSE

Late breast cancer (BC) recurrence (ie, ≥ 10 years after primary diagnosis) may have a more favorable prognosis than earlier recurrence. We investigated the risk of BC death after late recurrence, identified prognostic factors, and compared survival after early and late recurrence.

METHODS

Using the Danish Breast Cancer Group and other nationwide databases, we identified women with early or late BC recurrence during 2004-2018, who were alive 6 months after recurrence. We followed them until BC death, death from other causes, emigration, 10 years, or December 31, 2018, whichever came first. We calculated mortality rates (MRs) per 1,000 person-years (PY) and cumulative BC mortality, for early versus late recurrence, and by characteristics of the primary tumor and the late recurrence. Using Cox regression, we calculated adjusted hazard ratios (HRs) for BC death, accounting for death from other causes as competing risks.

RESULTS

Among 2,004 patients with late recurrence, 721 died of BC with a median survival time of 10 years (MR = 84.8 per 1,000 PY; 10-year cumulative mortality = 50%). Among 1,528 patients with early recurrence, 1,092 BC deaths occurred with a median survival time of 4 years (MR = 173.9 per 1,000 PY; 10-year cumulative mortality = 72%). We observed a lower hazard of BC-specific death among patients who developed late compared with early recurrence (hazard ratio = 0.72; 95% CI, 0.62 to 0.85). Advanced stage at primary diagnosis, distant metastases, adjuvant treatment for locoregional recurrence, and systemic treatment for distant recurrence were associated with increased mortality after late recurrence. Breast-conserving surgery at primary diagnosis, locoregional recurrence, and surgery for recurrence were associated with lower mortality after late recurrence.

CONCLUSION

Patients with late recurrence had more favorable prognosis than patients with early recurrence. The localization of recurrent disease was the main prognostic factor for BC death.

The Incidence of Breast Cancer Recurrence 10-32 Years after Primary Diagnosis

Background

Extended, more effective breast cancer treatments have increased the prevalence of long-term survivors. We investigated the risk of late breast cancer recurrence (BCR), 10 years or more after primary diagnosis, and associations between patient and tumor characteristics at primary diagnosis and late BCR up to 32 years after primary breast cancer diagnosis.

Methods

Using the Danish Breast Cancer Group clinical database, we identified all women with an incident early breast cancer diagnosed during 1987-2004. We restricted to women who survived 10 years without a recurrence or second cancer (10-year disease-free survivors) and followed them from 10 years after breast cancer diagnosis date until late recurrence, death, emigration, second cancer, or December 31, 2018. We calculated incidence rates per 1000 person-years and cumulative incidences for late BCR, stratifying by patient and tumor characteristics. Using Cox regression, we calculated adjusted hazard ratios for late BCR accounting for competing risks.

Results

Among 36 924 women with breast cancer, 20 315 became 10-year disease-free survivors. Of these, 2595 developed late BCR (incidence rate = 15.53 per 1000 person-years, 95% confidence interval = 14.94 to 16.14; cumulative incidence = 16.6%, 95% confidence interval = 15.8% to 17.5%) from year 10 to 32 after primary diagnosis. Tumor size larger than 20 mm, lymph node–positive disease, and estrogen receptor–positive tumors were associated with increased cumulative incidences and hazards for late BCR.

Conclusions

Recurrences continued to occur up to 32 years after primary diagnosis. Women with high lymph node burden, large tumor size, and estrogen receptor–positive tumors had increased risk of late recurrence. Such patients may warrant extended surveillance, more aggressive treatment, or new therapy approaches.

A Validated Register-Based Algorithm to Identify Patients Diagnosed with Recurrence of Malignant Melanoma in Denmark

Purpose

Information on cancer recurrence is rarely available outside clinical trials. Wide exclusion criteria used in clinical trials tend to limit the generalizability of findings to the entire population of people living beyond a cancer disease. Therefore, population-level evidence is needed. The aim of this study was to develop and validate a register-based algorithm to identify patients diagnosed with recurrence after curative treatment of malignant melanoma.

Patients and Methods

Indicators of recurrence were diagnosis and procedure codes recorded in the Danish National Patient Register and pathology results recorded in the Danish National Pathology Register. Medical records on recurrence status and recurrence date in the Danish Melanoma Database served as the gold standard to assess the accuracy of the algorithm.

Results

The study included 1747 patients diagnosed with malignant melanoma; 95 (5.4%) were diagnosed with recurrence of malignant melanoma according to the gold standard. The algorithm reached a sensitivity of 93.7% (95% confidence interval (CI) 86.8–97.6), a specificity of 99.2% (95% CI: 98.6–99.5), a positive predictive value of 86.4% (95% CI: 78.2–92.4), and negative predictive value of 99.6% (95% CI: 99.2–99.9). Lin’s concordance correlation coefficient was 0.992 (95% CI: 0.989–0.996) for the agreement between the recurrence dates generated by the algorithm and by the gold standard.

Conclusion

The algorithm can be used to identify patients diagnosed with recurrence of malignant melanoma and to establish the timing of recurrence. This can generate population-level evidence on disease-free survival and diagnostic pathways for recurrence of malignant melanoma.