Use of a mathematical model to evaluate breast cancer screening policy

Cost effectiveness of breast cancer screening and prevention: a systematic review with a focus on risk-adapted strategies

OBJECTIVES

Benefit and cost effectiveness of breast cancer screening are still matters of controversy. Risk-adapted strategies are proposed to improve its benefit-harm and cost-benefit relations. Our objective was to perform a systematic review on economic breast cancer models evaluating primary and secondary prevention strategies in the European health care setting, with specific focus on model results, model characteristics, and risk-adapted strategies.

METHODS

Literature databases were systematically searched for economic breast cancer models evaluating the cost effectiveness of breast cancer screening and prevention strategies in the European health care context. Characteristics, methodological details and results of the identified studies are reported in evidence tables. Economic model outputs are standardized to achieve comparable cost-effectiveness ratios.

RESULTS

Thirty-two economic evaluations of breast cancer screening and seven evaluations of primary breast cancer prevention were included. Five screening studies and none of the prevention studies considered risk-adapted strategies. Studies differed in methodologic features. Only about half of the screening studies modeled overdiagnosis-related harms, most often indirectly and without reporting their magnitude. All models predict gains in life expectancy and/or quality-adjusted life expectancy at acceptable costs. However, risk-adapted screening was shown to be more effective and efficient than conventional screening.

CONCLUSIONS

Economic models suggest that breast cancer screening and prevention are cost effective in the European setting. All screening models predict gains in life expectancy, which has not yet been confirmed by trials. European models evaluating risk-adapted screening strategies are rare, but suggest that risk-adapted screening is more effective and efficient than conventional screening.

How hospitals can improve their public quality metrics: a decision-theoretic model

The public reporting of hospitals' quality of care is providing additional motivation for hospitals to deliver high-quality patient care. Hospital Compare, a consumer-oriented website by the Centers for Medicare and Medicaid Services (CMS), provides patients with detailed quality of care data on most US hospitals. Given that many quality metrics are the aggregate result of physicians' individual clinical decisions, the question arises if and how hospitals could influence their physicians so that their decisions positively contribute to hospitals' quality goals. In this paper, we develop a decision-theoretic model to explore how three different hospital interventions-incentivization, training, and nudging-may affect physicians' decisions. We focus our analysis on Outpatient Measure 14 (OP-14), which is an imaging quality metric that reports the percentage of outpatients with a brain computed tomography (CT) scan, who also received a same-day sinus CT scan. In most cases, same-day brain and sinus CT scans are considered unnecessary, and high utilizing hospitals aim to reduce their OP-14 metric. Our model captures the physicians' imaging decision process accounting for medical and behavioral factors, in particular the uncertainty in clinical assessment and a physician's diagnostic ability. Our analysis shows how hospital interventions of incentivization, training, and nudging affect physician decisions and consequently OP-14. This decision-theoretic model provides a foundation to develop insights for policy makers on the multi-level effects of their policy decisions.

Bayesian versus Empirical Calibration of Microsimulation Models: A Comparative Analysis

Calibration of a microsimulation model (MSM) is a challenging but crucial step for the development of a valid model. Numerous calibration methods for MSMs have been suggested in the literature, most of which are usually adjusted to the specific needs of the model and based on subjective criteria for the selection of optimal parameter values. This article compares 2 general approaches for calibrating MSMs used in medical decision making, a Bayesian and an empirical approach. We use as a tool the MIcrosimulation Lung Cancer (MILC) model, a streamlined, continuous-time, dynamic MSM that describes the natural history of lung cancer and predicts individual trajectories accounting for age, sex, and smoking habits. We apply both methods to calibrate MILC to observed lung cancer incidence rates from the Surveillance, Epidemiology and End Results (SEER) database. We compare the results from the 2 methods in terms of the resulting parameter distributions, model predictions, and efficiency. Although the empirical method proves more practical, producing similar results with smaller computational effort, the Bayesian method resulted in a calibrated model that produced more accurate outputs for rare events and is based on a well-defined theoretical framework for the evaluation and interpretation of the calibration outcomes. A combination of the 2 approaches is an alternative worth considering for calibrating complex predictive models, such as microsimulation models.

Methods for the evaluation of biomarkers in patients with kidney and liver diseases: multicentre research programme including ELUCIDATE RCT

Background

Protein biomarkers with associations with the activity and outcomes of diseases are being identified by modern proteomic technologies. They may be simple, accessible, cheap and safe tests that can inform diagnosis, prognosis, treatment selection, monitoring of disease activity and therapy and may substitute for complex, invasive and expensive tests. However, their potential is not yet being realised.

Design and methods

The study consisted of three workstreams to create a framework for research: workstream 1, methodology – to define current practice and explore methodology innovations for biomarkers for monitoring disease; workstream 2, clinical translation – to create a framework of research practice, high-quality samples and related clinical data to evaluate the validity and clinical utility of protein biomarkers; and workstream 3, the ELF to Uncover Cirrhosis as an Indication for Diagnosis and Action for Treatable Event (ELUCIDATE) randomised controlled trial (RCT) – an exemplar RCT of an established test, the ADVIA Centaur® Enhanced Liver Fibrosis (ELF) test (Siemens Healthcare Diagnostics Ltd, Camberley, UK) [consisting of a panel of three markers – (1) serum hyaluronic acid, (2) amino-terminal propeptide of type III procollagen and (3) tissue inhibitor of metalloproteinase 1], for liver cirrhosis to determine its impact on diagnostic timing and the management of cirrhosis and the process of care and improving outcomes.

Results

The methodology workstream evaluated the quality of recommendations for using prostate-specific antigen to monitor patients, systematically reviewed RCTs of monitoring strategies and reviewed the monitoring biomarker literature and how monitoring can have an impact on outcomes. Simulation studies were conducted to evaluate monitoring and improve the merits of health care. The monitoring biomarker literature is modest and robust conclusions are infrequent. We recommend improvements in research practice. Patients strongly endorsed the need for robust and conclusive research in this area. The clinical translation workstream focused on analytical and clinical validity. Cohorts were established for renal cell carcinoma (RCC) and renal transplantation (RT), with samples and patient data from multiple centres, as a rapid-access resource to evaluate the validity of biomarkers. Candidate biomarkers for RCC and RT were identified from the literature and their quality was evaluated and selected biomarkers were prioritised. The duration of follow-up was a limitation but biomarkers were identified that may be taken forward for clinical utility. In the third workstream, the ELUCIDATE trial registered 1303 patients and randomised 878 patients out of a target of 1000. The trial started late and recruited slowly initially but ultimately recruited with good statistical power to answer the key questions. ELF monitoring altered the patient process of care and may show benefits from the early introduction of interventions with further follow-up. The ELUCIDATE trial was an ‘exemplar’ trial that has demonstrated the challenges of evaluating biomarker strategies in ‘end-to-end’ RCTs and will inform future study designs.

Conclusions

The limitations in the programme were principally that, during the collection and curation of the cohorts of patients with RCC and RT, the pace of discovery of new biomarkers in commercial and non-commercial research was slower than anticipated and so conclusive evaluations using the cohorts are few; however, access to the cohorts will be sustained for future new biomarkers. The ELUCIDATE trial was slow to start and recruit to, with a late surge of recruitment, and so final conclusions about the impact of the ELF test on long-term outcomes await further follow-up. The findings from the three workstreams were used to synthesise a strategy and framework for future biomarker evaluations incorporating innovations in study design, health economics and health informatics.

Trial registration

Current Controlled Trials ISRCTN74815110, UKCRN ID 9954 and UKCRN ID 11930.

Funding

This project was funded by the NIHR Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 6, No. 3. See the NIHR Journals Library website for further project information.

Evaluation of a Stratified National Breast Screening Program in the United Kingdom: An Early Model-Based Cost-Effectiveness Analysis

OBJECTIVES

To identify the incremental costs and consequences of stratified national breast screening programs (stratified NBSPs) and drivers of relative cost-effectiveness.

METHODS

A decision-analytic model (discrete event simulation) was conceptualized to represent four stratified NBSPs (risk 1, risk 2, masking [supplemental screening for women with higher breast density], and masking and risk 1) compared with the current UK NBSP and no screening. The model assumed a lifetime horizon, the health service perspective to identify costs (£, 2015), and measured consequences in quality-adjusted life-years (QALYs). Multiple data sources were used: systematic reviews of effectiveness and utility, published studies reporting costs, and cohort studies embedded in existing NBSPs. Model parameter uncertainty was assessed using probabilistic sensitivity analysis and one-way sensitivity analysis.

RESULTS

The base-case analysis, supported by probabilistic sensitivity analysis, suggested that the risk stratified NBSPs (risk 1 and risk-2) were relatively cost-effective when compared with the current UK NBSP, with incremental cost-effectiveness ratios of £16,689 per QALY and £23,924 per QALY, respectively. Stratified NBSP including masking approaches (supplemental screening for women with higher breast density) was not a cost-effective alternative, with incremental cost-effectiveness ratios of £212,947 per QALY (masking) and £75,254 per QALY (risk 1 and masking). When compared with no screening, all stratified NBSPs could be considered cost-effective. Key drivers of cost-effectiveness were discount rate, natural history model parameters, mammographic sensitivity, and biopsy rates for recalled cases. A key assumption was that the risk model used in the stratification process was perfectly calibrated to the population.

CONCLUSIONS

This early model-based cost-effectiveness analysis provides indicative evidence for decision makers to understand the key drivers of costs and QALYs for exemplar stratified NBSP.

Improvement in risk prediction, early detection and prevention of breast cancer in the NHS Breast Screening Programme and family history clinics: a dual cohort study

Background

In the UK, women are invited for 3-yearly mammography screening, through the NHS Breast Screening Programme (NHSBSP), from the ages of 47–50 years to the ages of 69–73 years. Women with family histories of breast cancer can, from the age of 40 years, obtain enhanced surveillance and, in exceptionally high-risk cases, magnetic resonance imaging. However, no NHSBSP risk assessment is undertaken. Risk prediction models are able to categorise women by risk using known risk factors, although accurate individual risk prediction remains elusive. The identification of mammographic breast density (MD) and common genetic risk variants [single nucleotide polymorphisms (SNPs)] has presaged the improved precision of risk models.

Objectives

To (1) identify the best performing model to assess breast cancer risk in family history clinic (FHC) and population settings; (2) use information from MD/SNPs to improve risk prediction; (3) assess the acceptability and feasibility of offering risk assessment in the NHSBSP; and (4) identify the incremental costs and benefits of risk stratified screening in a preliminary cost-effectiveness analysis.

Design

Two cohort studies assessing breast cancer incidence.

Setting

High-risk FHC and the NHSBSP Greater Manchester, UK.

Participants

A total of 10,000 women aged 20–79 years [Family History Risk Study (FH-Risk); UK Clinical Research Network identification number (UKCRN-ID) 8611] and 53,000 women from the NHSBSP [aged 46–73 years; Predicting the Risk of Cancer At Screening (PROCAS) study; UKCRN-ID 8080].

Interventions

Questionnaires collected standard risk information, and mammograms were assessed for breast density by a number of techniques. All FH-Risk and 10,000 PROCAS participants participated in deoxyribonucleic acid (DNA) studies. The risk prediction models Manual method, Tyrer–Cuzick (TC), BOADICEA (Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm) and Gail were used to assess risk, with modelling based on MD and SNPs. A preliminary model-based cost-effectiveness analysis of risk stratified screening was conducted.

Main outcome measures

Breast cancer incidence.

Data sources

The NHSBSP; cancer registration.

Results

A total of 446 women developed incident breast cancers in FH-Risk in 97,958 years of follow-up. All risk models accurately stratified women into risk categories. TC had better risk precision than Gail, and BOADICEA accurately predicted risk in the 6268 single probands. The Manual model was also accurate in the whole cohort. In PROCAS, TC had better risk precision than Gail [area under the curve (AUC) 0.58 vs. 0.54], identifying 547 prospective breast cancers. The addition of SNPs in the FH-Risk case–control study improved risk precision but was not useful inBRCA1(breast cancer 1 gene) families. Risk modelling of SNPs in PROCAS showed an incremental improvement from using SNP18 used in PROCAS to SNP67. MD measured by visual assessment score provided better risk stratification than automatic measures, despite wide intra- and inter-reader variability. Using a MD-adjusted TC model in PROCAS improved risk stratification (AUC = 0.6) and identified significantly higher rates (4.7 per 10,000 vs. 1.3 per 10,000;p < 0.001) of high-stage cancers in women with above-average breast cancer risks. It is not possible to provide estimates of the incremental costs and benefits of risk stratified screening because of lack of data inputs for key parameters in the model-based cost-effectiveness analysis.

Conclusions

Risk precision can be improved by using DNA and MD, and can potentially be used to stratify NHSBSP screening. It may also identify those at greater risk of high-stage cancers for enhanced screening. The cost-effectiveness of risk stratified screening is currently associated with extensive uncertainty. Additional research is needed to identify data needed for key inputs into model-based cost-effectiveness analyses to identify the impact on health-care resource use and patient benefits.

Future work

A pilot of real-time NHSBSP risk prediction to identify women for chemoprevention and enhanced screening is required.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme. The DNA saliva collection for SNP analysis for PROCAS was funded by the Genesis Breast Cancer Prevention Appeal.

The influence of disease risk on the optimal time interval between screens for the early detection of cancer: a mathematical approach

The intervals between screens for the early detection of diseases such as breast and colon cancer suggested by screening guidelines are typically based on the average population risk of disease. With the emergence of ever more biomarkers for cancer risk prediction and the development of personalized medicine, there is a need for risk-specific screening intervals. The interval between successive screens should be shorter with increasing cancer risk. A risk-dependent optimal interval is ideally derived from a cost-effectiveness analysis using a validated simulation model. However, this is time-consuming and costly. We propose a simplified mathematical approach for the exploratory analysis of the implications of risk level on optimal screening interval. We develop a mathematical model of the optimal screening interval for breast cancer screening. We verified the results by programming the simplified model in the MISCAN-Breast microsimulation model and comparing the results. We validated the results by comparing them with the results of a full, published MISCAN-Breast cost-effectiveness model for a number of different risk levels. The results of both the verification and validation were satisfactory. We conclude that the mathematical approach can indicate the impact of disease risk on the optimal screening interval.

Empirically evaluating decision-analytic models

OBJECTIVES

Model-based cost-effectiveness analyses support decision-making. To augment model credibility, evaluation via comparison to independent, empirical studies is recommended.

METHODS

We developed a structured reporting format for model evaluation and conducted a structured literature review to characterize current model evaluation recommendations and practices. As an illustration, we applied the reporting format to evaluate a microsimulation of human papillomavirus and cervical cancer. The model's outputs and uncertainty ranges were compared with multiple outcomes from a study of long-term progression from high-grade precancer (cervical intraepithelial neoplasia [CIN]) to cancer. Outcomes included 5 to 30-year cumulative cancer risk among women with and without appropriate CIN treatment. Consistency was measured by model ranges overlapping study confidence intervals.

RESULTS

The structured reporting format included: matching baseline characteristics and follow-up, reporting model and study uncertainty, and stating metrics of consistency for model and study results. Structured searches yielded 2963 articles with 67 meeting inclusion criteria and found variation in how current model evaluations are reported. Evaluation of the cervical cancer microsimulation, reported using the proposed format, showed a modeled cumulative risk of invasive cancer for inadequately treated women of 39.6% (30.9-49.7) at 30 years, compared with the study: 37.5% (28.4-48.3). For appropriately treated women, modeled risks were 1.0% (0.7-1.3) at 30 years, study: 1.5% (0.4-3.3).

CONCLUSIONS

To support external and projective validity, cost-effectiveness models should be iteratively evaluated as new studies become available, with reporting standardized to facilitate assessment. Such evaluations are particularly relevant for models used to conduct comparative effectiveness analyses.

Calibration methods used in cancer simulation models and suggested reporting guidelines

Increasingly, computer simulation models are used for economic and policy evaluation in cancer prevention and control. A model's predictions of key outcomes, such as screening effectiveness, depend on the values of unobservable natural history parameters. Calibration is the process of determining the values of unobservable parameters by constraining model output to replicate observed data. Because there are many approaches for model calibration and little consensus on best practices, we surveyed the literature to catalogue the use and reporting of these methods in cancer simulation models. We conducted a MEDLINE search (1980 through 2006) for articles on cancer-screening models and supplemented search results with articles from our personal reference databases. For each article, two authors independently abstracted pre-determined items using a standard form. Data items included cancer site, model type, methods used for determination of unobservable parameter values and description of any calibration protocol. All authors reached consensus on items of disagreement. Reviews and non-cancer models were excluded. Articles describing analytical models, which estimate parameters with statistical approaches (e.g. maximum likelihood) were catalogued separately. Models that included unobservable parameters were analysed and classified by whether calibration methods were reported and if so, the methods used. The review process yielded 154 articles that met our inclusion criteria and, of these, we concluded that 131 may have used calibration methods to determine model parameters. Although the term 'calibration' was not always used, descriptions of calibration or 'model fitting' were found in 50% (n = 66) of the articles, with an additional 16% (n = 21) providing a reference to methods. Calibration target data were identified in nearly all of these articles. Other methodological details, such as the goodness-of-fit metric, were discussed in 54% (n = 47 of 87) of the articles reporting calibration methods, while few details were provided on the algorithms used to search the parameter space. Our review shows that the use of cancer simulation modelling is increasing, although thorough descriptions of calibration procedures are rare in the published literature for these models. Calibration is a key component of model development and is central to the validity and credibility of subsequent analyses and inferences drawn from model predictions. To aid peer-review and facilitate discussion of modelling methods, we propose a standardized Calibration Reporting Checklist for model documentation.

Breast Cancer Screening Services: Trade-offs in Quality, Capacity, Outreach, and Centralization

This work combines and extends previous work on breast cancer screening models by explicitly incorporating, for the first time, aspects of the dynamics of health care states, program outreach, and the screening volume-quality relationship in a service system model to examine the effect of public health policy and service capacity decisions on public health outcomes. We consider the impact of increasing standards for minimum reading volume to improve quality, expanding outreach with or without decentralization of service facilities, and the potential of queueing due to stochastic effects and limited capacity. The results indicate a strong relation between screening quality and the cost of screening and treatment, and emphasize the importance of accounting for service dynamics when assessing the performance of health care interventions. For breast cancer screening, increasing outreach without improving quality and maintaining capacity results in less benefit than predicted by standard models.

Withdrawing low risk women from cervical screening programmes: mathematical modelling study

OBJECTIVE

To evaluate the impact of policies for removing women before the recommended age of 64 from screening programmes for cervical cancer in the United Kingdom.

DESIGN

A mathematical model of the clinical course of precancerous lesions which accounts for the influence of infection with the human papillomavirus, the effects of screening on the progression of disease, and the accuracy of the testing procedures. Two policies are compared: one in which women are withdrawn from the programme if their current smear is negative and they have a recent history of regular, negative results and one in which women are withdrawn if their current smear test is negative and a simultaneous test is negative for exposure to high risk types of human papillomavirus.

SETTING

United Kingdom cervical screening programme.

MAIN OUTCOME MEASURES

The incidence of invasive cervical cancer and the use of resources.

RESULTS

Early withdrawal of selected women from the programme is predicted to give rise to resource savings of up to 25% for smear tests and 18% for colposcopies when withdrawal occurs from age 50, the youngest age considered in the study. An increase in the incidence of invasive cervical cancer, by up to 2 cases/100 000 women each year is predicted. Testing for human papillomavirus infection to determine which women should be withdrawn from the programme makes little difference to outcome.

CONCLUSIONS

This model systematically analyses the consequences of screening options using available data and the clinical course of precancerous lesions. If further audit studies confirm the model's forecasts, a policy of early withdrawal might be considered. This would be likely to release substantial resources which could be channelled into other aspects of health care or may be more effectively used within the cervical screening programme to counteract the possible increase in cancer incidence that early withdrawal might bring.