Estimating the cumulative risk of a false-positive test in a repeated screening program

A review of methods for the analysis of diagnostic tests performed in sequence

BACKGROUND

Many clinical pathways for the diagnosis of disease are based on diagnostic tests that are performed in sequence. The performance of the full diagnostic sequence is dictated by the diagnostic performance of each test in the sequence as well as the conditional dependence between them, given true disease status. Resulting estimates of performance, such as the sensitivity and specificity of the test sequence, are key parameters in health-economic evaluations. We conducted a methodological review of statistical methods for assessing the performance of diagnostic tests performed in sequence, with the aim of guiding data analysts towards classes of methods that may be suitable given the design and objectives of the testing sequence.

METHODS

We searched PubMed, Scopus and Web of Science for relevant papers describing methodology for analysing sequences of diagnostic tests. Papers were classified by the characteristics of the method used, and these were used to group methods into themes. We illustrate some of the methods using data from a cohort study of repeat faecal immunochemical testing for colorectal cancer in symptomatic patients, to highlight the importance of allowing for conditional dependence in test sequences and adjustment for an imperfect reference standard.

RESULTS

Five overall themes were identified, detailing methods for combining multiple tests in sequence, estimating conditional dependence, analysing sequences of diagnostic tests used for risk assessment, analysing test sequences in conjunction with an imperfect or incomplete reference standard, and meta-analysis of test sequences.

CONCLUSIONS

This methodological review can be used to help researchers identify suitable analytic methods for studies that use diagnostic tests performed in sequence.

Nonhomogeneous Markov chain for estimating the cumulative risk of multiple false positive screening tests

Screening tests are widely recommended for the early detection of disease among asymptomatic individuals. While detecting disease at an earlier stage has the potential to improve outcomes, screening also has negative consequences, including false positive results which may lead to anxiety, unnecessary diagnostic procedures, and increased healthcare costs. In addition, multiple false positive results could discourage participating in subsequent screening rounds. Screening guidelines typically recommend repeated screening over a period of many years, but little prior research has investigated how often individuals receive multiple false positive test results. Estimating the cumulative risk of multiple false positive results over the course of multiple rounds of screening is challenging due to the presence of censoring and competing risks, which may depend on the false positive risk, screening round, and number of prior false positive results. To address the general challenge of estimating the cumulative risk of multiple false positive test results, we propose a nonhomogeneous multistate model to describe the screening process including competing events. We developed alternative approaches for estimating the cumulative risk of multiple false positive results using this multistate model based on existing estimators for the cumulative risk of a single false positive. We compared the performance of the newly proposed models through simulation studies and illustrate model performance using data on screening mammography from the Breast Cancer Surveillance Consortium. Across most simulation scenarios, the multistate extension of a censoring bias model demonstrated lower bias compared to other approaches. In the context of screening mammography, we found that the cumulative risk of multiple false positive results is high. For instance, based on the censoring bias model, for a high-risk individual, the cumulative probability of at least two false positive mammography results after 10 rounds of annual screening is 40.4.

Statistical Methods for Estimating the Cumulative Risk of Screening Mammography Outcomes

BACKGROUND

This study illustrates alternative statistical methods for estimating cumulative risk of screening mammography outcomes in longitudinal studies.

METHODS

Data from the US Breast Cancer Surveillance Consortium (BCSC) and the Nijmegen Breast Cancer Screening Program in the Netherlands were used to compare four statistical approaches to estimating cumulative risk. We estimated cumulative risk of false-positive recall and screen-detected cancer after 10 screening rounds using data from 242,835 women ages 40 to 74 years screened at the BCSC facilities in 1993-2012 and from 17,297 women ages 50 to 74 years screened in Nijmegen in 1990-2012.

RESULTS

In the BCSC cohort, a censoring bias model estimated bounds of 53.8% to 59.3% for false-positive recall and 2.4% to 7.6% for screen-detected cancer, assuming 10% increased or decreased risk among women screened for one additional round. In the Nijmegen cohort, false-positive recall appeared to be associated with subsequent discontinuation of screening leading to overestimation of risk of a false-positive recall based on adjusted discrete-time survival models. Bounds estimated by the censoring bias model were 11.0% to 19.9% for false-positive recall and 4.2% to 9.7% for screen-detected cancer.

CONCLUSION

Choice of statistical methodology can substantially affect cumulative risk estimates. The censoring bias model is appropriate under a variety of censoring mechanisms and provides bounds for cumulative risk estimates under varying degrees of dependent censoring.

IMPACT

This article illustrates statistical methods for estimating cumulative risks of cancer screening outcomes, which will be increasingly important as screening test recommendations proliferate.

The cumulative risk of false-positive results in the Norwegian Breast Cancer Screening Program: updated results

BACKGROUND

Some false-positive results are inevitable in mammographic screening, but the impact of false-positive findings on the program and the participants is a disadvantage of screening. The objective of the current study was to estimate the cumulative risk of a false-positive result over 10 biennial screening examinations and the cumulative risk of undergoing an invasive procedure with a benign outcome in women screened between the ages of 50 years to 69 years.

METHODS

A retrospective cohort study was performed in 231,310 women aged 50 years to 51 years at the time of first mammography screening who underwent 715,311 screening mammograms in the Norwegian Breast Cancer Screening Program from 1996 through 2010. Generalized linear mixed models were used to estimate the probability of a false-positive screening result and to compute the cumulative false-positive risk for up to 10 biennial screening examinations.

RESULTS

The cumulative false-positive risk after 20 years of biennial screening for women who initiated screening aged 50 years to 51 years was 20.0% (95% confidence interval [95% CI], 19.7%-20.4%). The cumulative risk of undergoing an invasive procedure with a benign outcome for the same group of women was 4.1% (95% CI, 3.9%-4.3%). The cumulative risk of undergoing a fine-needle aspiration cytology, core needle biopsy, or open biopsy with a benign outcome was 1.4% (95% CI, 1.3%-1.5%), 2.0% (95% CI, 1.9%-2.1%), and 0.16% (95% CI, 0.13%-0.19%), respectively.

CONCLUSIONS

One in every 5 women will be recalled for further assessment with a negative outcome if they attend biennial mammographic screening between ages 50 years to 69 years. The risk of an invasive procedure with a benign outcome is approximately 4%. It is important to communicate the existence and extent of this risk to the target group and to reduce to a minimum the waiting times between screening and further assessment.

A semiparametric censoring bias model for estimating the cumulative risk of a false-positive screening test under dependent censoring

False-positive test results are among the most common harms of screening tests and may lead to more invasive and expensive diagnostic testing procedures. Estimating the cumulative risk of a false-positive screening test result after repeat screening rounds is, therefore, important for evaluating potential screening regimens. Existing estimators of the cumulative false-positive risk are limited by strong assumptions about censoring mechanisms and parametric assumptions about variation in risk across screening rounds. To address these limitations, we propose a semiparametric censoring bias model for cumulative false-positive risk that allows for dependent censoring without specifying a fixed functional form for variation in risk across screening rounds. Simulation studies demonstrated that the censoring bias model performs similarly to existing models under independent censoring and can largely eliminate bias under dependent censoring. We used the existing and newly proposed models to estimate the cumulative false-positive risk and variation in risk as a function of baseline age and family history of breast cancer after 10 years of annual screening mammography using data from the Breast Cancer Surveillance Consortium. Ignoring potential dependent censoring in this context leads to underestimation of the cumulative risk of false-positive results. Models that provide accurate estimates under dependent censoring are critical for providing appropriate information for evaluating screening tests.

Screening outcomes in older US women undergoing multiple mammograms in community practice: does interval, age, or comorbidity score affect tumor characteristics or false positive rates?

Background Uncertainty exists about the appropriate use of screening mammography among older women because comorbid illnesses may diminish the benefit of screening. We examined the risk of adverse tumor characteristics and false positive rates according to screening interval, age, and comorbidity. Methods From January 1999 to December 2006, data were collected prospectively on 2993 older women with breast cancer and 137 949 older women without breast cancer who underwent mammography at facilities that participated in a data linkage between the Breast Cancer Surveillance Consortium and Medicare claims. Women were aged 66 to 89 years at study entry to allow for measurement of 1 year of preexisting illnesses. We used logistic regression analyses to calculate the odds of advanced (IIb, III, IV) stage, large (>20 millimeters) tumors, and 10-year cumulative probability of false-positive mammography by screening frequency (1 vs 2 years), age, and comorbidity score. The comorbidity score was derived using the Klabunde approximation of the Charlson score. All statistical tests were two-sided. Results Adverse tumor characteristics did not differ statistically significantly by comorbidity, age, or interval. Cumulative probability of a false-positive mammography result was higher among annual screeners than biennial screeners irrespective of comorbidity: 48.0% (95% confidence interval [CI] = 46.1% to 49.9%) of annual screeners aged 66 to 74 years had a false-positive result compared with 29.0% (95% CI = 28.1% to 29.9%) of biennial screeners. Conclusion Women aged 66 to 89 years who undergo biennial screening mammography have similar risk of advanced-stage disease and lower cumulative risk of a false-positive recommendation than annual screeners, regardless of comorbidity.

Modelling the cumulative risk of a false-positive screening test

The goal of a screening test is to reduce morbidity and mortality through the early detection of disease; but the benefits of screening must be weighed against potential harms, such as false-positive (FP) results, which may lead to increased healthcare costs, patient anxiety, and other adverse outcomes associated with diagnostic follow-up procedures. Accurate estimation of the cumulative risk of an FP test after multiple screening rounds is important for program evaluation and goal setting, as well as informing individuals undergoing screening what they should expect from testing over time. Estimation of the cumulative FP risk is complicated by the existence of censoring and possible dependence of the censoring time on the event history. Current statistical methods for estimating the cumulative FP risk from censored data follow two distinct approaches, either conditioning on the number of screening tests observed or marginalizing over this random variable. We review these current methods, identify their limitations and possibly unrealistic assumptions, and propose simple extensions to address some of these limitations. We discuss areas where additional extensions may be useful. We illustrate methods for estimating the cumulative FP recall risk of screening mammography and investigate the appropriateness of modelling assumptions using 13 years of data collected by the Breast Cancer Surveillance Consortium (BCSC). In the BCSC data we found evidence of violations of modelling assumptions of both classes of statistical methods. The estimated risk of an FP recall after 10 screening mammograms varied between 58% and 77% depending on the approach used, with an estimate of 63% based on what we feel are the most reasonable modelling assumptions.

Improving the biomarker pipeline to develop and evaluate cancer screening tests

The biomarker pipeline to develop and evaluate cancer screening tests has three stages: identification of promising biomarkers for the early detection of cancer, initial evaluation of biomarkers for cancer screening, and definitive evaluation of biomarkers for cancer screening. Statistical and biological issues to improve this pipeline are discussed. Although various recommendations, such as identifying cases based on clinical symptoms, keeping biomarker tests simple, and adjusting for postscreening noise, have been made previously, they are not widely known. New recommendations include more frequent specimen collection to help identify promising biomarkers and the use of the paired availability design with interval cases (symptomatic cancers detected in the interval after screening) for initial evaluation of biomarkers for cancer screening.

Cumulative incidence of false-positive results in repeated, multimodal cancer screening

PURPOSE

Multiple cancer screening tests have been advocated for the general population; however, clinicians and patients are not always well-informed of screening burdens. We sought to determine the cumulative risk of a false-positive screening result and the resulting risk of a diagnostic procedure for an individual participating in a multimodal cancer screening program.

METHODS

Data were analyzed from the intervention arm of the ongoing Prostate, Lung, Colorectal, and Ovarian (PLCO) Cancer Screening Trial, a randomized controlled trial to determine the effects of prostate, lung, colorectal, and ovarian cancer screening on disease-specific mortality. The 68,436 participants, aged 55 to 74 years, were randomized to screening or usual care. Women received serial serum tests to detect cancer antigen 125 (CA-125), transvaginal sonograms, posteroanterior-view chest radiographs, and flexible sigmoidoscopies. Men received serial chest radiographs, flexible sigmoidoscopies, digital rectal examinations, and serum prostate-specific antigen tests. Fourteen screening examinations for each sex were possible during the 3-year screening period.

RESULTS

After 14 tests, the cumulative risk of having at least 1 false-positive screening test is 60.4% (95% CI, 59.8%-61.0%) for men, and 48.8% (95% CI, 48.1%-49.4%) for women. The cumulative risk after 14 tests of undergoing an invasive diagnostic procedure prompted by a false-positive test is 28.5% (CI, 27.8%-29.3%) for men and 22.1% (95% CI, 21.4%-22.7%) for women.

CONCLUSIONS

For an individual in a multimodal cancer screening trial, the risk of a false-positive finding is about 50% or greater by the 14th test. Physicians should educate patients about the likelihood of false positives and resulting diagnostic interventions when counseling about cancer screening.

Cumulative false positive recall rate and association with participant related factors in a population based breast cancer screening programme

STUDY OBJECTIVE

To investigate the cumulative false positive recall rate throughout the period of participation in a population based breast cancer screening programme and to examine its association with women related factors.

DESIGN

Analysis of a database to estimate the cumulative false positive recall rate after 10 biennial mammograms in a cohort of women. Cumulative risk after 10 rounds was calculated by projecting forward the information available on the four rounds. Logistic regression was used to evaluate the association between the cumulative risk of false positive recall and women related factors.

SETTING

Population based breast cancer screening programme in Barcelona City (Spain).

PARTICIPANTS

8502 women aged 50-69 years who participated in four consecutive screening rounds. Eligible women had received a mammogram in the first screening round between 1 December 1995 and 31 December 1996.

MAIN RESULTS

The false positive recall rate in the first screening for women who entered the screening programme at the age of 50-51 years was assessed at 10.6% (95% CI 8.9, 12.3). In the second screening this risk decreased to 3.8% (95% CI 2.7, 4.9) and remained almost constant in subsequent rounds. After 10 mammograms, the cumulative false positive recall rate was estimated at 32.4% (95% CI 29.7, 35.1). The factors associated with a higher cumulative risk of false positive recall were: previous benign breast disease (OR = 8.48; CI 7.39, 9.73), perimenopausal status (OR = 1.62; CI 1.12, 2.34), body mass index above 27.3 (OR = 1.17; CI 1.02, 1.34), and age 50-54 years (OR = 1.15; CI 1.00, 1.31).

CONCLUSIONS

One third of women could have at least one false positive recall over 10 biennial screens. Women participating in screening programmes should be informed about this risk, especially those with associated factors.

Evaluating markers for the early detection of cancer: overview of study designs and methods

BACKGROUND

The field of cancer biomarker development has been evolving rapidly. New developments both in the biologic and statistical realms are providing increasing opportunities for evaluation of markers for both early detection and diagnosis of cancer.

PURPOSE

To review the major conceptual and methodological issues in cancer biomarker evaluation, with an emphasis on recent developments in statistical methods together with practical recommendations.

METHODS

We organized this review by type of study: preliminary performance, retrospective performance, prospective performance and cancer screening evaluation.

RESULTS

For each type of study, we discuss methodologic issues, provide examples and discuss strengths and limitations.

CONCLUSION

Preliminary performance studies are useful for quickly winnowing down the number of candidate markers; however their results may not apply to the ultimate target population, asymptomatic subjects. If stored specimens from cohort studies with clinical cancer endpoints are available, retrospective studies provide a quick and valid way to evaluate performance of the markers or changes in the markers prior to the onset of clinical symptoms. Prospective studies have a restricted role because they require large sample sizes, and, if the endpoint is cancer on biopsy, there may be bias due to overdiagnosis. Cancer screening studies require very large sample sizes and long follow-up, but are necessary for evaluating the marker as a trigger of early intervention.

Abdominal aortic aneurysm screening: recommendations and controversies

Extensive level one evidence supports routine abdominal aortic aneurysm (AAA) screening in men aged 65 to 75 years, because AAAs are highly prevalent in this population. Physical examination is an insensitive means of detection. Ruptured AAAs are costly with respect to quality adjusted life years (QALY) lost and medical expenses. Large scale, randomized trials have demonstrated that AAA screening reduces all AAA-related mortality in the screened population and is cost-effective in mid-term follow-up. AAA screening by ultrasound has many advantages over other accepted medical screening programs in its simplicity in structure and the availability of an inexpensive, portable, and reliable means of screening. Additionally, AAA screening almost entirely avoids the negative consequences associated with other screening programs, including the adverse psychological effects and medical costs associated with false-positive examination results. There are subgroups of at-risk women who might benefit from AAA screening, and this issue should be further studied.