Assessing the cost effectiveness of using prognostic biomarkers with decision models: case study in prioritising patients waiting for coronary artery surgery

Long-Term Cost-Effectiveness of Fractional Flow Reserve-Based Percutaneous Coronary Intervention in Stable and Unstable Angina

Background

There are limited studies on the cost-effectiveness of fractional flow reserve (FFR)-based percutaneous coronary intervention (PCI) over angiography-based PCI.

Objectives

The current study sought to evaluate long-term cost-effectiveness of FFR-based PCI compared to angiography-based PCI.

Methods

A cost-effectiveness analysis was conducted using a nationwide cohort that consisted of patients with stable or unstable angina from the National Health Insurance Service (NHIS) and Health Insurance Review and Assessment (HIRA) database in Korea. The cost-effectiveness analysis was also performed by using a decision and Markov model with key values from the United States and the United Kingdom health care systems. Incremental cost-effectiveness ratio (ICER), an indicator of incremental cost on additional quality-adjusted life-years gained by FFR-based PCI, was evaluated.

Results

In the NHIS-HIRA data, FFR-based PCI was used during the index PCI in 5,116 patients (3.8%) among 134,613 eligible patients. FFR-based PCI showed significantly lower risk of all-cause death (5.8% vs 7.7%, P = 0.001) and spontaneous myocardial infarction (1.6% vs 2.2%, P = 0.022) than the angiography-based PCI at 4 years. In the NHIS-HIRA data, FFR-based PCI gained 0.039 quality-adjusted life-years at a lower cost ($303) than angiography-based PCI, yielding an ICER of -$7,748 during the 4-year follow-up. FFR-based PCI was dominant in the health care system of Korea (ICER = -$7,309), United States (ICER = -$31,267), and United Kingdom (ICER = -$1,341) during a 10-year time horizon. These results were consistently shown in probabilistic sensitivity analyses.

Conclusions

In the current cohort, FFR-based PCI was associated with higher quality of life at a lower cost than angiography-based PCI. FFR-based PCI was cost-effective in patients with stable or unstable angina undergoing PCI.

The impact of waiting for intervention on costs and effectiveness: the case of transcatheter aortic valve replacement

OBJECTIVES

The economic crisis in Europe might have limited access to some innovative technologies implying an increase of waiting time. The purpose of the study is to evaluate the impact of waiting time on the costs and benefits of transcatheter aortic valve replacement (TAVR) for the treatment of severe aortic stenosis.

METHODS

This is a cost-utility analysis from the perspective of the Spanish National Health Service. Results of two prospective hospital registries (158 and 273 consecutive patients) were incorporated into a probabilistic Markov model to compare quality adjusted life years (QALYs) and costs for TAVR after waiting for 3-12 months, relative to immediate TAVR. We simulated a cohort of 1000 patients, male, and 80 years old; other patient profiles were assessed in sensitivity analyses.

RESULTS

As waiting time increased, costs decreased at the expense of lower survival and loss of QALYs, leading to incremental cost-effectiveness ratios for eliminating waiting lists of about 12,500 € per QALY. In subgroup analyses prioritization of patients for whom higher benefit was expected led to a smaller loss of QALYs. Concerning budget impact, long waiting lists reduced spending considerably and permanently.

CONCLUSIONS

A shorter waiting time is likely to be cost-effective (considering commonly accepted willingness-to-pay thresholds in Europe) relative to 3 months or longer waiting periods. If waiting lists are nevertheless seen as unavoidable due to severe but temporary budgetary restrictions, prioritizing patients for whom higher benefit is expected appears to be a way of postponing spending without utterly sacrificing patients' survival and quality of life.

Cohort profile: the Coronary Artery disease Risk Determination In Innsbruck by diaGnostic ANgiography (CARDIIGAN) cohort

PURPOSE

The Coronary Artery disease Risk Determination In Innsbruck by diaGnostic ANgiography (CARDIIGAN) cohort is aimed to gain a better understanding of cardiovascular risk factors and their relation to the diagnosis and severity of coronary artery disease, as well as to the long-term prognosis in consecutive (including revascularised) patients referred for elective coronary angiography.

PARTICIPANTS

The included patients visited the University Clinic of Cardiology at Innsbruck (Austria), which fulfils a secondary and tertiary hospital function. Inclusion took place in the period between February 2004 and April 2008 and resulted in a total of 8296 patients aged 18-91 years; 65% of them were men.

FINDINGS TO DATE

There was one follow-up round on vital status through record linkage for 84% of the cohort (those with residence in Tyrol), resulting in a follow-up duration of over 5.5 to nearly 10.0 years among survivors. The data contain basic patient characteristics, cardiovascular risk factors, laboratory measurements, medications, detailed information on the extent and severity of coronary artery disease, revascularisation history, treatment strategy and mortality specifics. A few studies have already been published.

FUTURE PLANS

Various diagnostic and prognostic studies are planned, also concerning complications, competing risks and cost-effectiveness. Collaboration with other research groups is welcomed.

Preoperative serum ST2 level predicts acute kidney injury after adult cardiac surgery

OBJECTIVE

The purpose of this study was to evaluate the relationship between preoperative levels of serum soluble ST2 (ST2) and acute kidney injury (AKI) after cardiac surgery. Previous research has shown that biomarkers facilitate the prediction of AKI and other complications after cardiac surgery.

METHODS

Preoperative ST2 proteins were measured in 1498 patients undergoing isolated coronary artery bypass graft surgery at 8 hospitals participating in the Northern New England Biomarker Study from 2004 to 2007. AKI severity was defined using the Acute Kidney Injury Network (AKIN) definition. Preoperative ST2 levels were measured using multiplex assays. Ordered logistic regression was used to examine the relationship between ST2 levels and levels of AKI severity.

RESULTS

Participants in this study showed a significant association between elevated preoperative ST2 levels and acute kidney risk. Before adjustment, the odds of patients developing AKIN stage 2 or 3, compared with AKIN stage 1, are 2.43 times higher (95% confidence interval, 1.86-3.16; P < .001) for patients in the highest tercile of preoperative ST2. After adjustment, patients in the highest tercile of preoperative ST2 had significantly greater odds of developing AKIN stage 2 or 3 AKI (odds ratio, 1.99; 95% confidence interval, 1.50-2.65; P < .001) compared with patients with AKIN stage 1.

CONCLUSIONS

Preoperative ST2 levels are associated with postoperative AKI risk and can be used to identify patients at higher risk of developing AKI after cardiac surgery.

Personalising the decision for prolonged dual antiplatelet therapy: development, validation and potential impact of prognostic models for cardiovascular events and bleeding in myocardial infarction survivors

Aims

The aim of this study is to develop models to aid the decision to prolong dual antiplatelet therapy (DAPT) that requires balancing an individual patient's potential benefits and harms.

Methods and results

Using population-based electronic health records (EHRs) (CALIBER, England, 2000-10), of patients evaluated 1 year after acute myocardial infarction (MI), we developed (n = 12 694 patients) and validated (n = 5613) prognostic models for cardiovascular (cardiovascular death, MI or stroke) events and three different bleeding endpoints. We applied trial effect estimates to determine potential benefits and harms of DAPT and the net clinical benefit of individuals. Prognostic models for cardiovascular events (c-index: 0.75 (95% CI: 0.74, 0.77)) and bleeding (c index 0.72 (95% CI: 0.67, 0.77)) were well calibrated: 3-year risk of cardiovascular events was 16.5% overall (5.2% in the lowest- and 46.7% in the highest-risk individuals), while for major bleeding, it was 1.7% (0.3% in the lowest- and 5.4% in the highest-risk patients). For every 10 000 patients treated per year, we estimated 249 (95% CI: 228, 269) cardiovascular events prevented and 134 (95% CI: 87, 181) major bleeding events caused in the highest-risk patients, and 28 (95% CI: 19, 37) cardiovascular events prevented and 9 (95% CI: 0, 20) major bleeding events caused in the lowest-risk patients. There was a net clinical benefit of prolonged DAPT in 63-99% patients depending on how benefits and harms were weighted.

Conclusion

Prognostic models for cardiovascular events and bleeding using population-based EHRs may help to personalise decisions for prolonged DAPT 1-year following acute MI.

Systematic Review of Health Economic Impact Evaluations of Risk Prediction Models: Stop Developing, Start Evaluating

BACKGROUND

Although health economic evaluations (HEEs) are increasingly common for therapeutic interventions, they appear to be rare for the use of risk prediction models (PMs).

OBJECTIVES

To evaluate the current state of HEEs of PMs by performing a comprehensive systematic review.

METHODS

Four databases were searched for HEEs of PM-based strategies. Two reviewers independently selected eligible articles. A checklist was compiled to score items focusing on general characteristics of HEEs of PMs, model characteristics and quality of HEEs, evidence on PMs typically used in the HEEs, and the specific challenges in performing HEEs of PMs.

RESULTS

After screening 791 abstracts, 171 full texts, and reference checking, 40 eligible HEEs evaluating 60 PMs were identified. In these HEEs, PM strategies were compared with current practice (n = 32; 80%), to other stratification methods for patient management (n = 19; 48%), to an extended PM (n = 9; 23%), or to alternative PMs (n = 5; 13%). The PMs guided decisions on treatment (n = 42; 70%), further testing (n = 18; 30%), or treatment prioritization (n = 4; 7%). For 36 (60%) PMs, only a single decision threshold was evaluated. Costs of risk prediction were ignored for 28 (46%) PMs. Uncertainty in outcomes was assessed using probabilistic sensitivity analyses in 22 (55%) HEEs.

CONCLUSIONS

Despite the huge number of PMs in the medical literature, HEE of PMs remains rare. In addition, we observed great variety in their quality and methodology, which may complicate interpretation of HEE results and implementation of PMs in practice. Guidance on HEE of PMs could encourage and standardize their application and enhance methodological quality, thereby improving adequate use of PM strategies.

Tumour necrosis factor-α inhibitors for ankylosing spondylitis and non-radiographic axial spondyloarthritis: a systematic review and economic evaluation

BACKGROUND

Tumour necrosis factor (TNF)-α inhibitors (anti-TNFs) are typically used when the inflammatory rheumatologic diseases ankylosing spondylitis (AS) and non-radiographic axial spondyloarthritis (nr-AxSpA) have not responded adequately to conventional therapy. Current National Institute for Health and Care Excellence (NICE) guidance recommends treatment with adalimumab, etanercept and golimumab in adults with active (severe) AS only if certain criteria are fulfilled but it does not recommend infliximab for AS. Anti-TNFs for patients with nr-AxSpA have not previously been appraised by NICE.

OBJECTIVE

To determine the clinical effectiveness, safety and cost-effectiveness within the NHS of adalimumab, certolizumab pegol, etanercept, golimumab and infliximab, within their licensed indications, for the treatment of severe active AS or severe nr-AxSpA (but with objective signs of inflammation).

DESIGN

Systematic review and economic model.

DATA SOURCES

Fifteen databases were searched for relevant studies in July 2014.

REVIEW METHODS

Clinical effectiveness data from randomised controlled trials (RCTs) were synthesised using Bayesian network meta-analysis methods. Results from other studies were summarised narratively. Only full economic evaluations that compared two or more options and considered both costs and consequences were included in the systematic review of cost-effectiveness studies. The differences in the approaches and assumptions used across the studies, and also those in the manufacturer's submissions, were examined in order to explain any discrepancies in the findings and to identify key areas of uncertainty. A de novo decision model was developed with a generalised framework for evidence synthesis that pooled change in disease activity (BASDAI and BASDAI 50) and simultaneously synthesised information on function (BASFI) to determine the long-term quality-adjusted life-year and cost burden of the disease in the economic model. The decision model was developed in accordance with the NICE reference case. The model has a lifetime horizon (60 years) and considers costs from the perspective of the NHS and personal social services. Health effects were expressed in terms of quality-adjusted life-years.

RESULTS

In total, 28 eligible RCTs were identified and 26 were placebo controlled (mostly up to 12 weeks); 17 extended into open-label active treatment-only phases. Most RCTs were judged to have a low risk of bias overall. In both AS and nr-AxSpA populations, anti-TNFs produced clinically important benefits to patients in terms of improving function and reducing disease activity; for AS, the relative risks for ASAS 40 ranged from 2.53 to 3.42. The efficacy estimates were consistently slightly smaller for nr-AxSpA than for AS. Statistical (and clinical) heterogeneity was more apparent in the nr-AxSpA analyses than in the AS analyses; both the reliability of the nr-AxSpA meta-analysis results and their true relevance to patients seen in clinical practice are questionable. In AS, anti-TNFs are approximately equally effective. Effectiveness appears to be maintained over time, with around 50% of patients still responding at 2 years. Evidence for an effect of anti-TNFs delaying disease progression was limited; results from ongoing long-term studies should help to clarify this issue. Sequential treatment with anti-TNFs can be worthwhile but the drug survival response rates and benefits are reduced with second and third anti-TNFs. The de novo model, which addressed many of the issues of earlier evaluations, generated incremental cost-effectiveness ratios ranging from £19,240 to £66,529 depending on anti-TNF and modelling assumptions.

CONCLUSIONS

In both AS and nr-AxSpA populations anti-TNFs are clinically effective, although more so in AS than in nr-AxSpA. Anti-TNFs may be an effective use of NHS resources depending on which assumptions are considered appropriate.

FUTURE WORK RECOMMENDATIONS

Randomised trials are needed to identify the nr-AxSpA population who will benefit the most from anti-TNFs.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42014010182.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Assessing the value of cardiac biomarkers: going beyond diagnostic accuracy?

In this era of scrutinized resource utilization, providers and payers are focused on the value of healthcare interventions more than ever. Cost-effectiveness evaluations are required by some health authorities and requested by others in order to guide budget allocation decisions. In the past, these evaluations did not methodologically consider laboratory diagnostics. We set out to explore the current requirements of health technology agencies that include laboratory diagnostics and describe, through a review of the literature, alternative methods for establishing the value of a biomarker or labroatory diagnostic beyond assay specifications and performance. The aim of this study was to evaluate the current use of a linked evidence approach in cost-effectiveness studies for cardiac laboratory tests in the last 5 years.

What do providers, payers and patients need from comparative effectiveness research on diagnostics? The case of HER2/Neu testing in breast cancer

AIMS

Comparing effectiveness of diagnostic tests is one of the highest priorities for comparative effectiveness research (CER) set by the Institute of Medicine. Our study aims to identify what information providers, payers and patients need from CER on diagnostics, and what challenges they encounter implementing comparative information on diagnostic alternatives in practice and policy.

MATERIALS & METHODS

Using qualitative research methods and the example of two alternative protocols for HER2 testing in breast cancer, we conducted interviews with 45 stakeholders: providers (n = 25) from four academic and eight nonacademic institutions, executives (n = 13) from five major US private payers and representatives (n = 7) from two breast cancer patient advocacies.

RESULTS

The need for additional scientific evidence to determine the preferred HER2 protocol was more common for advocates than payers (100 vs 54%; p = 0.0515) and significantly more common for advocates than providers (100 vs 40%; p = 0.0077). The availability of information allowing assessment of the implementation impact from alternative diagnostic protocols on provider institutions may mitigate the need for additional scientific evidence for some providers and payers (24 and 46%, respectively). The cost-effectiveness of alternative protocols from the societal perspective is important to payers and advocates (69 and 71%, respectively) but not to providers (0%; p = 0.0001 and p = 0.0001). The lack of reporting laboratory practices is a more common implementation challenge for payers and advocates (77 and 86%, respectively) than for providers (32%). The absence of any mechanism for patient involvement was recognized as a challenge by payers and advocates (69 and 100%, respectively) but not by providers (0%; p = 0.0001 and p = 0.0001).

CONCLUSION

Comparative implementation research is needed to inform the stakeholders considering diagnostic alternatives. Transparency of laboratory practices is an important factor in enabling implementation of CER on diagnostics in practice and policy. The incongruent views of providers versus patient advocates and payers on involving patients in diagnostic decisions is a concerning challenge to utilizing the results of CER.

Prognostic models for stable coronary artery disease based on electronic health record cohort of 102 023 patients

Aims

The population with stable coronary artery disease (SCAD) is growing but validated models to guide their clinical management are lacking. We developed and validated prognostic models for all-cause mortality and non-fatal myocardial infarction (MI) or coronary death in SCAD.

Methods and results

Models were developed in a linked electronic health records cohort of 102 023 SCAD patients from the CALIBER programme, with mean follow-up of 4.4 (SD 2.8) years during which 20 817 deaths and 8856 coronary outcomes were observed. The Kaplan–Meier 5-year risk was 20.6% (95% CI, 20.3, 20.9) for mortality and 9.7% (95% CI, 9.4, 9.9) for non-fatal MI or coronary death. The predictors in the models were age, sex, CAD diagnosis, deprivation, smoking, hypertension, diabetes, lipids, heart failure, peripheral arterial disease, atrial fibrillation, stroke, chronic kidney disease, chronic pulmonary disease, liver disease, cancer, depression, anxiety, heart rate, creatinine, white cell count, and haemoglobin. The models had good calibration and discrimination in internal (external) validation with C-index 0.811 (0.735) for all-cause mortality and 0.778 (0.718) for non-fatal MI or coronary death. Using these models to identify patients at high risk (defined by guidelines as 3% annual mortality) and support a management decision associated with hazard ratio 0.8 could save an additional 13–16 life years or 15–18 coronary event-free years per 1000 patients screened, compared with models with just age, sex, and deprivation.

Conclusion

These validated prognostic models could be used in clinical practice to support risk stratification as recommended in clinical guidelines.

An integrated approach to evaluating alternative risk prediction strategies: a case study comparing alternative approaches for preventing invasive fungal disease

OBJECTIVES

This article proposes an integrated approach to the development, validation, and evaluation of new risk prediction models illustrated with the Fungal Infection Risk Evaluation study, which developed risk models to identify non-neutropenic, critically ill adult patients at high risk of invasive fungal disease (IFD).

METHODS

Our decision-analytical model compared alternative strategies for preventing IFD at up to three clinical decision time points (critical care admission, after 24 hours, and end of day 3), followed with antifungal prophylaxis for those judged "high" risk versus "no formal risk assessment." We developed prognostic models to predict the risk of IFD before critical care unit discharge, with data from 35,455 admissions to 70 UK adult, critical care units, and validated the models externally. The decision model was populated with positive predictive values and negative predictive values from the best-fitting risk models. We projected lifetime cost-effectiveness and expected value of partial perfect information for groups of parameters.

RESULTS

The risk prediction models performed well in internal and external validation. Risk assessment and prophylaxis at the end of day 3 was the most cost-effective strategy at the 2% and 1% risk threshold. Risk assessment at each time point was the most cost-effective strategy at a 0.5% risk threshold. Expected values of partial perfect information were high for positive predictive values or negative predictive values (£11 million-£13 million) and quality-adjusted life-years (£11 million).

CONCLUSIONS

It is cost-effective to formally assess the risk of IFD for non-neutropenic, critically ill adult patients. This integrated approach to developing and evaluating risk models is useful for informing clinical practice and future research investment.

From accuracy to patient outcome and cost-effectiveness evaluations of diagnostic tests and biomarkers: an exemplary modelling study

BACKGROUND

Proper evaluation of new diagnostic tests is required to reduce overutilization and to limit potential negative health effects and costs related to testing. A decision analytic modelling approach may be worthwhile when a diagnostic randomized controlled trial is not feasible. We demonstrate this by assessing the cost-effectiveness of modified transesophageal echocardiography (TEE) compared with manual palpation for the detection of atherosclerosis in the ascending aorta.

METHODS

Based on a previous diagnostic accuracy study, actual Dutch reimbursement data, and evidence from literature we developed a Markov decision analytic model. Cost-effectiveness of modified TEE was assessed for a life time horizon and a health care perspective. Prevalence rates of atherosclerosis were age-dependent and low as well as high rates were applied. Probabilistic sensitivity analysis was applied.

RESULTS

The model synthesized all available evidence on the risk of stroke in cardiac surgery patients. The modified TEE strategy consistently resulted in more adapted surgical procedures and, hence, a lower risk of stroke and a slightly higher number of life-years. With 10% prevalence of atherosclerosis the incremental cost-effectiveness ratio was € 4,651 and € 481 per quality-adjusted life year in 55-year-old men and women, respectively. In all patients aged 65 years or older the modified TEE strategy was cost saving and resulted in additional health benefits.

CONCLUSIONS

Decision analytic modelling to assess the cost-effectiveness of a new diagnostic test based on characteristics, costs and effects of the test itself and of the subsequent treatment options is both feasible and valuable. Our case study on modified TEE suggests that it may reduce the risk of stroke in cardiac surgery patients older than 55 years at acceptable cost-effectiveness levels.

Prognosis Research Strategy (PROGRESS) 3: prognostic model research

Prognostic models are abundant in the medical literature yet their use in practice seems limited. In this article, the third in the PROGRESS series, the authors review how such models are developed and validated, and then address how prognostic models are assessed for their impact on practice and patient outcomes, illustrating these ideas with examples.

Zinc protoporphyrin levels have added value in the prediction of low hemoglobin deferral in whole blood donors

BACKGROUND

Increased zinc protoporphyrin (ZPP) levels can indicate iron deficiency and may be predictive for low hemoglobin (Hb) deferral in blood donors. Prediction models for Hb deferral in whole blood donors have already been developed. In this study, we examined if addition of ZPP to these prediction models improves risk estimation of Hb deferral.

STUDY DESIGN AND METHODS

This study included 4598 Dutch whole blood donors. Information on ZPP levels measured at the previous visit was added to the existing prediction models to estimate the risk of Hb deferral. Models were compared using the following measures: concordance (c)-statistic, continuous net reclassification improvement (NRI), and clinical net benefit (NB).

RESULTS

Seventy-six male donors (2.9%) and 69 female donors (3.5%) were deferred because of a low Hb level. Previous ZPP level was associated with risk of Hb deferral (odds ratio for interquartile range of previous ZPP level, men 2.0 [95% confidence interval {CI}, 1.7-2.3]; women 2.2 [95% CI, 1.9-2.4]) in a multivariable risk model. Addition of ZPP into the models resulted in an increase of the c-statistic from 0.93 to 0.94 for men and from 0.80 to 0.85 for women. The added value of ZPP was confirmed by measures of clinical usefulness. NRI for men was 0.42, and for women, 0.56. At relevant threshold probabilities between 10 and 15%, NB was higher for models considering ZPP.

CONCLUSION

This study shows that ZPP measurements obtained at the previous visit may have added value in the risk prediction of Hb deferral in whole blood donors.

Cost-effectiveness of candesartan versus losartan in the primary preventive treatment of hypertension

Background

Although angiotensin receptor blockers have different receptor binding properties, no comparative randomized studies with cardiovascular event endpoints have been performed for this class of drugs. The aim of this study was to assess the long-term cost-effectiveness of candesartan (Atacand^®) versus generic losartan in the primary preventive treatment of hypertension.

Methods

A decision-analytic model was developed to estimate costs and health outcomes over a patient’s lifetime. Data from a clinical registry study were used to estimate event rates for cardiovascular complications, such as myocardial infarction and heart failure. Costs and quality of life data were from published sources. Costs were in Swedish kronor and the outcome was quality-adjusted life-years (QALYs).

Results

Due to reduced rates of cardiovascular complications, candesartan was associated with a QALY gain and lower health care costs compared with generic losartan (0.053 QALYs gained and reduced costs of approximately 4700 Swedish kronor for women; and 0.057 QALYs gained and reduced costs of approximately 4250 Swedish kronor for men). This result was robust in several sensitivity analyses.

Conclusion

When modeling costs and health outcomes based on event rates for cardiovascular complications from a real-world registry study, candesartan appears to bring a QALY gain and a reduction in costs compared with generic losartan in the primary preventive treatment of hypertension in Sweden.

Increased preoperative C-reactive protein levels are associated with inhospital death after coronary artery bypass surgery

Increased C-reactive protein (CRP) is a predictor of cardiovascular risk, but its influence on outcomes after coronary artery bypass grafting surgery (CABG) is still incompletely studied. We studied the association between preoperative CRP and inhospital death after CABG. Patients with acute or chronic infectious or inflammatory disorders, autoimmune diseases, cancer, and prior cardiac surgery were excluded. Seventy-six patients were studied [27.6% with elevated CRP (>3 mg/l)]. Elevated CRP was more frequently found in patients who died than in those who survived (83.3% vs 17.1%, p = 0.003); mean CRP levels were, respectively, 6.5 ± 3.4 vs 2.4 ± 3.5 mg/l (p = 0.03). The hazard ratio of death was 11.7 for elevated CRP, and the ROC curve for the discrimination of death with CRP had an area under the curve of 0.82. An improvement to mortality risk prediction following CABG may be offered by the preoperative analysis of CRP.

A method for the early health technology assessment of novel biomarker measurement in primary prevention programs

Many promising biomarkers for stratifying individuals at risk of developing a chronic disease or subsequent complications have been identified. Research into the potential cost-effectiveness of applying these biomarkers in actual clinical settings has however been lacking. Investors and analysts may improve their venture decision making should they have indicative estimates of the potential costs and effects associated with a new biomarker technology already at the early stages of its development. To assist in obtaining such estimates, this paper presents a general method for the early health technology assessment of a novel biomarker technology. The setting considered is that of primary prevention programs where initial screening to select high-risk individuals eligible for a subsequent intervention occurs, for example, prevention of type 2 diabetes. The method is based on quantifying the health outcomes and downstream healthcare consumption of all individuals who get reclassified as a result of moving from a screening variant based on traditional risk factors to a screening variant based on traditional risk factors plus a novel biomarker. As these individuals form well-defined subpopulations, a combination of disease progression modeling and sensitivity analysis can be used to perform an initial assessment of the maximum increase in screening cost for which the use of the new biomarker technology is still likely to be cost effective.

Assessing the incremental value of diagnostic and prognostic markers: a review and illustration

BACKGROUND

New markers may improve prediction of diagnostic and prognostic outcomes. We review various measures to quantify the incremental value of markers over standard, readily available characteristics.

METHODS

Widely used traditional measures include the improvement in model fit or in the area under the receiver operating characteristic (ROC) curve (AUC). New measures include the net reclassification index (NRI) and decision-analytic measures, such as the fraction of true-positive classifications penalized for false-positive classifications [net benefit (NB)]. For illustration, we discuss a case study on the presence of residual tumour vs. benign tissue in 544 patients with testicular cancer. We assessed three tumour markers [Alpha-fetoprotein (AFP), Human chorionic gonadotropin (HCG) and Lactate dehydrogenase (LDH)] for their incremental value over currently standard clinical predictors.

RESULTS

AUC and R(2) values suggested adding continuous LDH and AFP whereas NB only favoured HCG as a potentially promising marker at a clinically defendable decision threshold of 20% risk. The NRI suggested reclassification potential of all three markers.

CONCLUSIONS

The improvement in standard discrimination measures, which focus on finding variables that might be promising across all decision thresholds, may not detect the most informative markers at a specific threshold of particular clinical relevance. When a marker is intended to support decision-making, calculation of the improvement in a decision-analytic measure, such as NB, is preferable over an overall judgment as obtained from the AUC in ROC analysis.

[Performance measures for prediction models and markers: evaluation of predictions and classifications]

Prediction models are becoming more and more important in medicine and cardiology. Nowadays, specific interest focuses on ways in which models can be improved using new prognostic markers. We aim to describe the similarities and differences between performance measures for prediction models. We analyzed data from 3264 subjects to predict 10-year risk of coronary heart disease according to age, systolic blood pressure, diabetes, and smoking. We specifically study the incremental value of adding high-density lipoprotein cholesterol to this model. We emphasize that we need to separate the evaluation of predictions, where traditional performance measures such as the area under the receiver operating characteristic curve and calibration are useful, from the evaluation of classifications, where various other statistics are now available, including the net reclassification index and net benefit.

Evaluating the quality of research into a single prognostic biomarker: a systematic review and meta-analysis of 83 studies of C-reactive protein in stable coronary artery disease

BACKGROUND

Systematic evaluations of the quality of research on a single prognostic biomarker are rare. We sought to evaluate the quality of prognostic research evidence for the association of C-reactive protein (CRP) with fatal and nonfatal events among patients with stable coronary disease.

METHODS AND FINDINGS

We searched MEDLINE (1966 to 2009) and EMBASE (1980 to 2009) and selected prospective studies of patients with stable coronary disease, reporting a relative risk for the association of CRP with death and nonfatal cardiovascular events. We included 83 studies, reporting 61,684 patients and 6,485 outcome events. No study reported a prespecified statistical analysis protocol; only two studies reported the time elapsed (in months or years) between initial presentation of symptomatic coronary disease and inclusion in the study. Studies reported a median of seven items (of 17) from the REMARK reporting guidelines, with no evidence of change over time. The pooled relative risk for the top versus bottom third of CRP distribution was 1.97 (95% confidence interval [CI] 1.78-2.17), with substantial heterogeneity (I(2) = 79.5). Only 13 studies adjusted for conventional risk factors (age, sex, smoking, obesity, diabetes, and low-density lipoprotein [LDL] cholesterol) and these had a relative risk of 1.65 (95% CI 1.39-1.96), I(2) = 33.7. Studies reported ten different ways of comparing CRP values, with weaker relative risks for those based on continuous measures. Adjusting for publication bias (for which there was strong evidence, Egger's p<0.001) using a validated method reduced the relative risk to 1.19 (95% CI 1.13-1.25). Only two studies reported a measure of discrimination (c-statistic). In 20 studies the detection rate for subsequent events could be calculated and was 31% for a 10% false positive rate, and the calculated pooled c-statistic was 0.61 (0.57-0.66).

CONCLUSION

Multiple types of reporting bias, and publication bias, make the magnitude of any independent association between CRP and prognosis among patients with stable coronary disease sufficiently uncertain that no clinical practice recommendations can be made. Publication of prespecified statistical analytic protocols and prospective registration of studies, among other measures, might help improve the quality of prognostic biomarker research.