From accuracy to patient outcome and cost-effectiveness evaluations of diagnostic tests and biomarkers: an exemplary modelling study

Cost-effectiveness of one-stop-shop [18F]Fluorocholine PET/CT to localise parathyroid adenomas in patients suffering from primary hyperparathyroidism

PURPOSE

We conducted a cost-effectiveness analysis in which we compared a preoperative [18F]Fluorocholine PET/CT-based one-stop-shop imaging strategy with current best practice in which [18F]Fluorocholine PET/CT is only recommended after negative or inconclusive [99mTc]Tc-methoxy isobutyl isonitrile SPECT/CT for patients suffering from primary hyperparathyroidism. We investigated whether the one-stop-shop strategy performs as well as current best practice but at lower costs.

METHODS

We developed a cohort-level state transition model to evaluate both imaging strategies respecting an intraoperative parathyroid hormone monitored treatment setting as well as a traditional treatment setting. The model reflects patients' hospital journeys after biochemically diagnosed primary hyperparathyroidism. A cycle length of twelve months and a lifetime horizon were used. We conducted probabilistic analyses simulating 50,000 cohorts to assess joint parameter uncertainty. The incremental net monetary benefit and cost for each quality-adjusted life year were estimated. Furthermore, threshold analyses regarding the tariff of [18F]Fluorocholine PET/CT and the sensitivity of [99mTc]Tc-methoxy isobutyl isonitrile SPECT/CT were performed.

RESULTS

The simulated long-term health effects and costs were similar for both imaging strategies. Accordingly, there was no incremental net monetary benefit and the one-stop-shop strategy did not result in lower costs. These results applied to both treatment settings. The threshold analysis indicated that a tariff of €885 for [18F]Fluorocholine PET/CT was required to be cost-effective compared to current best practice.

CONCLUSION

Both preoperative imaging strategies can be used interchangeably. Daily clinical practice grounds such as available local resources and patient preferences should inform policy-making on whether a hospital should implement the one-stop-shop imaging strategy.

Clinical research study implementation of case-finding strategies for heart failure and chronic obstructive pulmonary disease in the elderly with reduced exercise tolerance or dyspnea: A cluster randomized trial

BACKGROUND

Heart failure (HF) and chronic obstructive pulmonary disease (COPD) often remain undiagnosed in older individuals, although both disorders inhibit functionality and impair health. The aim of the study was to assess the effectiveness of a case-finding strategy of these disorders.

METHODS

This is a clustered randomized trial; 18 general practices from the vicinity of Utrecht, the Netherlands, were randomly allocated to a case-finding strategy or usual care. Multimorbid community subjects (≥65 years) with dyspnea or reduced exercise tolerance were eligible for inclusion. The case-finding strategy consisted of history taking, physical examination, blood tests, electrocardiography, spirometry, and echocardiography. Subsequent treatment decisions were at the discretion of the general practitioner. Questionnaires regarding health status and functionality were filled out at baseline and after 6 months of follow-up. Information regarding changes in medication and health care use during the 6 months follow-up was extracted.

RESULTS

A total of 829 participants were randomized: 389 in the case-finding strategy group and 440 in the usual care group. More patients in the case-finding group received a new diagnosis of HF or COPD than the usual care group (cumulative incidence 34% vs 2% and 17% vs. 2%, respectively). Scores for health status, functionality, and health care use were similar between the 2 strategies after 6 months of follow-up.

CONCLUSIONS

A case-finding strategy applied in primary care to multimorbid older people with dyspnea or reduced exercise tolerance resulted in a number of new diagnoses of HF and COPD but did not result in short-term improvement of health status compared to usual care.

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

Clinical prediction rules (CPRs) that predict the absolute risk of a clinical condition or future outcome for individual patients are abundant in the medical literature; however, systematic reviews have demonstrated shortcomings in the methodological quality and reporting of prediction studies. To maximise the potential and clinical usefulness of CPRs, they must be rigorously developed and validated, and their impact on clinical practice and patient outcomes must be evaluated. This review aims to present a comprehensive overview of the stages involved in the development, validation and evaluation of CPRs, and to describe in detail the methodological standards required at each stage, illustrated with examples where appropriate. Important features of the study design, statistical analysis, modelling strategy, data collection, performance assessment, CPR presentation and reporting are discussed, in addition to other, often overlooked aspects such as the acceptability, cost-effectiveness and longer-term implementation of CPRs, and their comparison with clinical judgement. Although the development and evaluation of a robust, clinically useful CPR is anything but straightforward, adherence to the plethora of methodological standards, recommendations and frameworks at each stage will assist in the development of a rigorous CPR that has the potential to contribute usefully to clinical practice and decision-making and have a positive impact on patient care.

Toward Alignment in the Reporting of Economic Evaluations of Diagnostic Tests and Biomarkers: The AGREEDT Checklist

Objectives. General frameworks for conducting and reporting health economic evaluations are available but not specific enough to cover the intricacies of the evaluation of diagnostic tests and biomarkers. Such evaluations are typically complex and model-based because tests primarily affect health outcomes indirectly and real-world data on health outcomes are often lacking. Moreover, not all aspects relevant to the evaluation of a diagnostic test may be known and explicitly considered for inclusion in the evaluation, leading to a loss of transparency and replicability. To address this challenge, this study aims to develop a comprehensive reporting checklist. Methods. This study consisted of 3 main steps: 1) the development of an initial checklist based on a scoping review, 2) review and critical appraisal of the initial checklist by 4 independent experts, and 3) development of a final checklist. Each item from the checklist is illustrated using an example from previous research. Results. The scoping review followed by critical review by the 4 experts resulted in a checklist containing 44 items, which ideally should be considered for inclusion in a model-based health economic evaluation. The extent to which these items were included or discussed in the studies identified in the scoping review varied substantially, with 14 items not being mentioned in ≥47 (75%) of the included studies. Conclusions. The reporting checklist developed in this study may contribute to improved transparency and completeness of model-based health economic evaluations of diagnostic tests and biomarkers. Use of this checklist is therefore encouraged to enhance the interpretation, comparability, and—indirectly—the validity of the results of such evaluations.

The early economic evaluation of novel biomarkers to accelerate their translation into clinical applications

BACKGROUND

Translating prognostic and diagnostic biomarker candidates into clinical applications takes time, is very costly, and many candidates fail. It is therefore crucial to be able to select those biomarker candidates that have the highest chance of successfully being adopted in the clinic. This requires an early estimate of the potential clinical impact and commercial value. In this paper, we aim to demonstratively evaluate a set of novel biomarkers in terms of clinical impact and commercial value, using occurrence of cardiovascular disease (CVD) in type-2 diabetes (DM2) patients as a case study.

METHODS

We defined a clinical application for the novel biomarkers, and subsequently used data from a large cohort study in The Netherlands in a modeling exercise to assess the potential clinical impact and headroom for the biomarkers.

RESULTS

The most likely application of the biomarkers would be to identify DM2 patients with a low CVD risk and subsequently withhold statin treatment. As a result, one additional CVD event in every 75 patients may be expected. The expected downstream savings resulted in a headroom for a point-of-care device ranging from €119.09 at a willingness to accept of €0 for one additional CVD event, to €0 at a willingness to accept of €15,614 or more.

CONCLUSION

It is feasible to evaluate novel biomarkers on outcomes directly relevant to technological development and clinical adoption. Importantly, this may be attained at the same point in time and using the same data as used for the evaluation of association with disease and predictive power.

Biomarkers: Delivering on the expectation of molecularly driven, quantitative health

Biomarkers are the pillars of precision medicine and are delivering on expectations of molecular, quantitative health. These features have made clinical decisions more precise and personalized, but require a high bar for validation. Biomarkers have improved health outcomes in a few areas such as cancer, pharmacogenetics, and safety. Burgeoning big data research infrastructure, the internet of things, and increased patient participation will accelerate discovery in the many areas that have not yet realized the full potential of biomarkers for precision health. Here we review themes of biomarker discovery, current implementations of biomarkers for precision health, and future opportunities and challenges for biomarker discovery. Impact statement Precision medicine evolved because of the understanding that human disease is molecularly driven and is highly variable across patients. This understanding has made biomarkers, a diverse class of biological measurements, more relevant for disease diagnosis, monitoring, and selection of treatment strategy. Biomarkers' impact on precision medicine can be seen in cancer, pharmacogenomics, and safety. The successes in these cases suggest many more applications for biomarkers and a greater impact for precision medicine across the spectrum of human disease. The authors assess the status of biomarker-guided medical practice by analyzing themes for biomarker discovery, reviewing the impact of these markers in the clinic, and highlight future and ongoing challenges for biomarker discovery. This work is timely and relevant, as the molecular, quantitative approach of precision medicine is spreading to many disease indications.

Evaluating the impact of prediction models: lessons learned, challenges, and recommendations

An important aim of clinical prediction models is to positively impact clinical decision making and subsequent patient outcomes. The impact on clinical decision making and patient outcome can be quantified in prospective comparative-ideally cluster-randomized-studies, known as 'impact studies'. However, such impact studies often require a lot of time and resources, especially when they are (cluster-)randomized studies. Before envisioning such large-scale randomized impact study, it is important to ensure a reasonable chance that the use of the prediction model by the targeted healthcare professionals and patients will indeed have a positive effect on both decision making and subsequent outcomes. We recently performed two differently designed, prospective impact studies on a clinical prediction model to be used in surgical patients. Both studies taught us new valuable lessons on several aspects of prediction model impact studies, and which considerations may guide researchers in their decision to conduct a prospective comparative impact study. We provide considerations on how to prepare a prediction model for implementation in practice, how to present the model predictions, and how to choose the proper design for a prediction model impact study.

Evaluating the accuracy and economic value of a new test in the absence of a perfect reference test

BACKGROUND

Streptococcus pneumoniae (SP) pneumonia is often treated empirically as diagnosis is challenging because of the lack of a perfect test. Using BinaxNOW-SP, a urinary antigen test, as an add-on to standard cultures may not only increase diagnostic yield but also increase costs.

OBJECTIVE

To estimate the sensitivity and specificity of BinaxNOW-SP and subsequently estimate the cost-effectiveness of adding BinaxNOW-SP to the diagnostic work-up.

DESIGN

We fit a Bayesian latent-class meta-analysis model to obtain estimates of BinaxNOW-SP accuracy that adjust for the imperfect accuracy of culture. Meta-analysis results were combined with information on prevalence of SP pneumonia to estimate the number of patients who are correctly classified under competing diagnostic strategies. Taking into consideration the cost of antibiotics, we determined the incremental cost of adding BinaxNOW-SP to the work-up per case correctly diagnosed.

RESULTS

The BinaxNOW-SP test had a pooled sensitivity of 0.74 (95% credible interval [CrI], 0.67-0.83) and a pooled specificity of 0.96 (95% CrI, 0.92-0.99). An overall increase in diagnostic accuracy of 6.2% due to the addition of BinaxNOW-SP corresponded to an incremental cost per case correctly classified of $582 Canadian dollars.

CONCLUSIONS

The methods we have described allow us to evaluate the accuracy and economic value of a new test in the absence of a perfect reference test using an evidence-based approach.

Systematic Review of Health Economic Impact Evaluations of Risk Prediction Models: Stop Developing, Start Evaluating

BACKGROUND

Although health economic evaluations (HEEs) are increasingly common for therapeutic interventions, they appear to be rare for the use of risk prediction models (PMs).

OBJECTIVES

To evaluate the current state of HEEs of PMs by performing a comprehensive systematic review.

METHODS

Four databases were searched for HEEs of PM-based strategies. Two reviewers independently selected eligible articles. A checklist was compiled to score items focusing on general characteristics of HEEs of PMs, model characteristics and quality of HEEs, evidence on PMs typically used in the HEEs, and the specific challenges in performing HEEs of PMs.

RESULTS

After screening 791 abstracts, 171 full texts, and reference checking, 40 eligible HEEs evaluating 60 PMs were identified. In these HEEs, PM strategies were compared with current practice (n = 32; 80%), to other stratification methods for patient management (n = 19; 48%), to an extended PM (n = 9; 23%), or to alternative PMs (n = 5; 13%). The PMs guided decisions on treatment (n = 42; 70%), further testing (n = 18; 30%), or treatment prioritization (n = 4; 7%). For 36 (60%) PMs, only a single decision threshold was evaluated. Costs of risk prediction were ignored for 28 (46%) PMs. Uncertainty in outcomes was assessed using probabilistic sensitivity analyses in 22 (55%) HEEs.

CONCLUSIONS

Despite the huge number of PMs in the medical literature, HEE of PMs remains rare. In addition, we observed great variety in their quality and methodology, which may complicate interpretation of HEE results and implementation of PMs in practice. Guidance on HEE of PMs could encourage and standardize their application and enhance methodological quality, thereby improving adequate use of PM strategies.

Prediction models: the right tool for the right problem

PURPOSE OF REVIEW

Perioperative prediction models can help to improve personalized patient care by providing individual risk predictions to both patients and providers. However, the scientific literature on prediction model development and validation can be quite technical and challenging to understand. This article aims to provide the necessary insight for clinicians to assess the value of a prediction model that they intend to use in their clinical practice.

RECENT FINDINGS

Recent developments in prediction model research include the continuous development of new performance characteristics for prediction models, increasing insight into the limitations of old characteristics, as well as an improved understanding of the generalizability of prediction models to new populations and practices.

SUMMARY

Clinicians can assess the value of a prediction model for their practice by first identifying what the usage of the model will be. Second, they can recognize which performance characteristics are relevant to their assessment of the model. Finally, they need to decide whether the available scientific evidence sufficiently matches their clinical practice to proceed with implementation.

Using evidence to combat overdiagnosis and overtreatment: evaluating treatments, tests, and disease definitions in the time of too much

Ray Moynihan and colleagues outline suggestions for improving the way that medical evidence is produced, analysed, and interpreted to avoid problems of overdiagnosis and overtreatment. Please see later in the article for the Editors' Summary

Diagnostic accuracy of conventional or age adjusted D-dimer cut-off values in older patients with suspected venous thromboembolism: systematic review and meta-analysis

OBJECTIVE

To review the diagnostic accuracy of D-dimer testing in older patients (>50 years) with suspected venous thromboembolism, using conventional or age adjusted D-dimer cut-off values.

DESIGN

Systematic review and bivariate random effects meta-analysis.

DATA SOURCES

We searched Medline and Embase for studies published before 21 June 2012 and we contacted the authors of primary studies.

STUDY SELECTION

Primary studies that enrolled older patients with suspected venous thromboembolism in whom D-dimer testing, using both conventional (500 µg/L) and age adjusted (age × 10 µg/L) cut-off values, and reference testing were performed. For patients with a non-high clinical probability, 2 × 2 tables were reconstructed and stratified by age category and applied D-dimer cut-off level.

RESULTS

13 cohorts including 12,497 patients with a non-high clinical probability were included in the meta-analysis. The specificity of the conventional cut-off value decreased with increasing age, from 57.6% (95% confidence interval 51.4% to 63.6%) in patients aged 51-60 years to 39.4% (33.5% to 45.6%) in those aged 61-70, 24.5% (20.0% to 29.7% in those aged 71-80, and 14.7% (11.3% to 18.6%) in those aged >80. Age adjusted cut-off values revealed higher specificities over all age categories: 62.3% (56.2% to 68.0%), 49.5% (43.2% to 55.8%), 44.2% (38.0% to 50.5%), and 35.2% (29.4% to 41.5%), respectively. Sensitivities of the age adjusted cut-off remained above 97% in all age categories.

CONCLUSIONS

The application of age adjusted cut-off values for D-dimer tests substantially increases specificity without modifying sensitivity, thereby improving the clinical utility of D-dimer testing in patients aged 50 or more with a non-high clinical probability.