Can Administrative Data Identify Incident Cases of Colorectal Cancer? A Comparison of Two Health Plans

The impact of the lookback period and definition of confirmatory events on the identification of incident cancer cases in administrative data

Background

This cohort study examined the impact of the lengths of lookback and confirmation periods as well as the definition of confirmatory events on the number of incident cancer cases identified and age-standardized cumulative incidences (ACI) estimated in administrative data using German cancer registry data as a benchmark.

Methods

ACI per 100,000 insured persons for breast, prostate and colorectal cancer were estimated using BARMER Statutory Health Insurance claims data. Incident cancer cases were defined as having an in- or outpatient diagnosis in 2013, no diagnosis in a lookback period of 1 year and a second diagnosis (or death) in a confirmation period of 1 quarter. We varied lookback periods from 1 to 7 years, confirmation periods from 1 to 4 quarters as well as the definition of confirmatory events and compared ACI estimates to cancer registry data.

Results

ACI were higher for breast (138.7) and prostate (103.6) but lower for colorectal cancer (42.1) when compared to cancer registries (119.3, 98.0 and 45.5, respectively). Extending the lookback period to 7 years reduced ACI to 129.0, 95.1 and 38.3. An extended confirmation period of 4 quarters increased ACI to 151.3, 114.9 and 46.8. Including breast and colorectal surgeries as a confirmatory event reduced ACI to 114.9 and 37.1, respectively.

Conclusions

The choice of lookback and confirmation periods and the definition of confirmatory events have considerable impact on the number of incident cancer cases identified and ACI estimated. Researchers need to be aware of potential misclassification when identifying incident cancer cases in administrative data. Further validation studies as well as studies using administrative data to estimate cancer incidences should consider several choices of the lookback and confirmation periods and the definition of confirmatory events to show how these parameters impact the validity and robustness of their results.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-017-0407-4) contains supplementary material, which is available to authorized users.

Validation of Claims Algorithms for Progression to Metastatic Cancer in Patients with Breast, Non-small Cell Lung, and Colorectal Cancer

Background

Validated algorithms for identifying progression to metastatic cancer could permit the use of administrative claims databases for research in this area.

Objective

To identify simple algorithms that could accurately detect cancer progression to metastatic breast, non-small cell lung, and colorectal cancer (CRC) using medical and pharmacy claims data.

Methods

Adults with stage I–III breast, non-small cell lung cancer (NSCLC), or CRC in the Geisinger Health System from 2004 to 2011 were selected. Evidence of progression was extracted via manual chart review as the reference standard. In addition to secondary malignancy diagnosis (ICD-9 code for metastases), diagnoses, procedures, and treatments were selected with clinician input as indicators of cancer progression. Random forests models provided variable importance scores. In addition to codes for secondary malignancy, several more complex algorithms were constructed and performance measures calculated.

Results

Among those with breast cancer [17/502 (3.4%) progressed], the performance of a secondary malignancy code was suboptimal [sensitivity: 64.7%; specificity: 86.0%; positive predictive value (PPV): 13.9; negative predictive value (NPV): 98.6%]; requiring malignancy at another site or initiation of immunotherapy increased PPV and specificity but decreased sensitivity. For NSCLC [61/236 (25.8%) progressed], codes for secondary malignancy alone (PPV: 47.4%; NPV: 84.8%; sensitivity: 60.7%; specificity: 76.6%) performed similarly or better than more complex algorithms. For CRC [33/276 (12.0%) progressed], secondary malignancy codes had good specificity (92.7%) and NPV (92.3%) but low sensitivity (42.4%) and PPV (43.8%); an algorithm with change in chemotherapy increased sensitivity but decreased other metrics.

Conclusion

Selected algorithms performed similarly to the presence of a secondary tumor diagnosis code, with low sensitivity/PPV and higher specificity/NPV. Accurate identification of cancer progression likely requires verification through chart review.

Validation of administrative data sources for endoscopy utilization in colorectal cancer diagnosis

Background

Validation of administrative data is important to assess potential sources of bias in outcome evaluation and to prevent dissemination of misleading or inaccurate information. The purpose of the study was to determine the completeness and accuracy of endoscopy data in several administrative data sources in the year prior to colorectal cancer diagnosis as part of a larger project focused on evaluating the quality of pre-diagnostic care.

Methods

Primary and secondary data sources for endoscopy were collected from the Alberta Cancer Registry, cancer medical charts and three different administrative data sources. 1672 randomly sampled patients diagnosed with invasive colorectal cancer in years 2000–2005 in Alberta, Canada were included. A retrospective validation study of administrative data for endoscopy in the year prior to colorectal cancer diagnosis was conducted. A gold standard dataset was created by combining all the datasets. Number and percent identified, agreement and percent unique to a given data source were calculated and compared across each dataset and to the gold standard with respect to identifying all patients who underwent endoscopy and all endoscopies received by those patients.

Results

The combined administrative data and physician billing data identified as high or higher percentage of patients who had one or more endoscopy (84% and 78%, respectively) and total endoscopy procedures (89% and 81%, respectively) than the chart review (78% for both).

Conclusions

Endoscopy data has a high level of completeness and accuracy in physician billing data alone. Combined with hospital in/outpatient data it is more complete than chart review alone.

Evaluation of three algorithms to identify incident breast cancer in Medicare claims data

OBJECTIVE

To test the validity of three published algorithms designed to identify incident breast cancer cases using recent inpatient, outpatient, and physician insurance claims data.

DATA

The Surveillance, Epidemiology, and End Results (SEER) registry data linked with Medicare physician, hospital, and outpatient claims data for breast cancer cases diagnosed from 1995 to 1998 and a 5 percent control sample of Medicare beneficiaries in SEER areas.

STUDY DESIGN

We evaluate the sensitivity and specificity of three algorithms applied to new data compared with original reported results. Algorithms use health insurance diagnosis and procedure claims codes to classify breast cancer cases, with SEER as the reference standard. We compare algorithms by age, stage, race, and SEER region, and explore via logistic regression whether adding demographic variables improves algorithm performance.

PRINCIPAL FINDINGS

The sensitivity of two of three algorithms is significantly lower when applied to newer data, compared with sensitivity calculated during algorithm development (59 and 77.4 percent versus 90 and 80.2 percent, p<.00001). Sensitivity decreases as age increases, and false negative rates are higher for cases with in situ, metastatic, and unknown stage disease compared with localized or regional breast cancer. Substantial variation also exists by SEER registry. There was potential for improvement in algorithm performance when adding age, region, and race to an indicator variable for whether the algorithm determined a subject to be a breast cancer case (p<.00001).

CONCLUSIONS

Differential sensitivity of the algorithms by SEER region and age likely reflects variation in practice patterns, because the algorithms rely on administrative procedure codes. Depending on the algorithm, 3-5 percent of subjects overall are misclassified in 1998. Misclassification disproportionately affects older women and those diagnosed with in situ, metastatic, or unknown-stage disease. Algorithms should be applied cautiously to insurance claims databases to assess health care utilization outside SEER-Medicare populations because of uneven misclassification of subgroups that may be understudied already.