Outcome-based analytical performance specifications: current status and future challenges

Analytical performance specifications (APS) based on outcomes refer to how 'good' the analytical performance of a test needs to be to do more good than harm to the patient. Analytical performance of a measurand affects its clinical performance. Without first setting clinical performance requirements, it is difficult to define how good analytically the test needs to be to meet medical needs. As testing is indirectly linked to health outcomes through clinical decisions on patient management, often simulation-based studies are used to assess the impact of analytical performance on the probability of clinical outcomes which is then translated to Model 1b APS according to the Milan consensus. This paper discusses the related key definitions, concepts and considerations that should assist in finding the most appropriate methods for deriving Model 1b APS. We review the advantages and limitations of published methods and discuss the criteria for transferability of Model 1b APS to different settings. We consider that the definition of the clinically acceptable misclassification rate is central to Model 1b APS. We provide some examples and guidance on a more systematic approach for first defining the clinical performance requirements for tests and we also highlight a few ideas to tackle the future challenges associated with providing outcome-based APS for laboratory testing.

[Benefit assessment of diagnostics - Dealing with incomplete evidence]

Incomplete evidence in diagnostic studies results from missing or too few randomised test-treatment studies or from studies of too low quality. In order to be able to carry out a benefit assessment, it is helpful in the first step to design a hypothetical randomised test-treatment study. In the second step, the linked evidence approach can be used to link the evidence of the individual components of the test-treatment pathway and to assess the potential benefits and risks. In the third step, based on the linked evidence approach, decision analytic models can be used to quantify the benefit-risk ratio. In the case of incomplete evidence, the assessment can thus be made by linking the individual components of the test-treatment pathway, provided that their evidence is sufficient.

Decision-analytic modeling for early health technology assessment of medical devices - a scoping review

Objective

The goal of this review was to identify decision-analytic modeling studies in early health technology assessments (HTA) of high-risk medical devices, published over the last three years, and to provide a systematic overview of model purposes and characteristics. Additionally, the aim was to describe recent developments in modeling techniques.

Methods

For this scoping review, we performed a systematic literature search in PubMed and Embase including studies published in English or German. The search code consisted of terms describing early health technology assessment and terms for decision-analytic models. In abstract and full-text screening, studies were excluded that were not modeling studies for a high-risk medical device or an in-vitro diagnostic test. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram was used to report on the search and exclusion of studies. For all included studies, study purpose, framework and model characteristics were extracted and reported in systematic evidence tables and a narrative summary.

Results

Out of 206 identified studies, 19 studies were included in the review. Studies were either conducted for hypothetical devices or for existing devices after they were already available on the market. No study extrapolated technical data from early development stages to estimate potential value of devices in development. All studies except one included cost as an outcome. Two studies were budget impact analyses. Most studies aimed at adoption and reimbursement decisions. The majority of studies were on in-vitro diagnostic tests for personalized and targeted medicine. A timed automata model, to our knowledge a model type new to HTA, was tested by one study. It describes the agents in a clinical pathway in separate models and, by allowing for interaction between the models, can reflect complex individual clinical pathways and dynamic system interactions. Not all sources of uncertainty for in-vitro tests were explicitly modeled. Elicitation of expert knowledge and judgement was used for substitution of missing empirical data. Analysis of uncertainty was the most valuable strength of decision-analytic models in early HTA, but no model applied sensitivity analysis to optimize the test positivity cutoff with regard to the benefit-harm balance or cost-effectiveness. Value-of-information analysis was rarely performed. No information was found on the use of causal inference methods for estimation of effect parameters from observational data.

Conclusion

Our review provides an overview of the purposes and model characteristics of nineteen recent early evaluation studies on medical devices. The review shows the growing importance of personalized interventions and confirms previously published recommendations for careful modeling of uncertainties surrounding diagnostic devices and for increased use of value-of-information analysis. Timed automata may be a model type worth exploring further in HTA. In addition, we recommend to extend the application of sensitivity analysis to optimize positivity criteria for in-vitro tests with regard to benefit-harm or cost-effectiveness. We emphasize the importance of causal inference methods when estimating effect parameters from observational data.

Causal evidence in health decision making: methodological approaches of causal inference and health decision science

Objectives

Public health decision making is a complex process based on thorough and comprehensive health technology assessments involving the comparison of different strategies, values and tradeoffs under uncertainty. This process must be based on best available evidence and plausible assumptions. Causal inference and health decision science are two methodological approaches providing information to help guide decision making in health care. Both approaches are quantitative methods that use statistical and modeling techniques and simplifying assumptions to mimic the complexity of the real world. We intend to review and lay out both disciplines with their aims, strengths and limitations based on a combination of textbook knowledge and expert experience.

Methods

To help understanding and differentiating the methodological approaches of causal inference and health decision science, we reviewed both methods with the focus on aims, research questions, methods, assumptions, limitations and challenges, and software. For each methodological approach, we established a group of four experts from our own working group to carefully review and summarize each method, followed by structured discussion rounds and written reviews, in which the experts from all disciplines including HTA and medicine were involved. The entire expert group discussed objectives, strengths and limitations of both methodological areas, and potential synergies. Finally, we derived recommendations for further research and provide a brief outlook on future trends.

Results

Causal inference methods aim for drawing causal conclusions from empirical data on the relationship of pre-specified interventions on a specific target outcome and apply a counterfactual framework and statistical techniques to derive causal effects of exposures or interventions from these data. Causal inference is based on a causal diagram, more specifically, a directed acyclic graph (DAG), which encodes the assumptions regarding the causal relations between variables. Depending on the type of confounding and selection bias, traditional statistical methods or more complex g-methods are needed to derive valid causal effects. Besides the correct specification of the DAG and the statistical model, assumptions such as consistency, positivity, and exchangeability must be checked when aiming at causal inference. Health decision science aims for guiding policy decision making regarding health interventions considering and balancing multiple competing objectives of a decision based on data from multiple sources and studies, for example prevalence studies, clinical trials and long-term observational routine effectiveness studies, and studies on preferences and costs. It involves decision analysis, a systematic, explicit and quantitative framework to guide decisions under uncertainty. Decision analyses are based on decision-analytic models to mimic the course of disease as well as aspects and consequences of the intervention in order to quantitatively optimize the decision. Depending on the type of decision problem, decision trees, state-transition models, discrete event simulation models, dynamic transmission models, or other model types are applied. Models must be validated against observed data, and comprehensive sensitivity analyses must be performed to assess uncertainty. Besides the appropriate choice of the model type and the valid specification of the model structure, it must be checked if input parameters of effects can be interpreted as causal parameters in the model. Otherwise results will be biased.

Conclusions

Both causal inference and health decision science aim for providing best causal evidence for informed health decision making. The strengths and limitations of both methods differ and a good understanding of both methods is essential for correct application but also for correct interpretation of findings from the described methods. Importantly, decision-analytic modeling should be combined with causal inference when developing guidance and recommendations regarding decisions on health care interventions.

Assessing the Utility of Test-Based Prediction Models: Heart Failure as an Example

Chirinos et al.1 demonstrated that endotrophin levels are strongly associated with all-cause mortality and a composite outcome in patients with heart failure with preserved ejection fraction (HFpEF). The authors established the association in a subset of the TOPCAT trial (Treatment of Preserved Cardiac Function Heart Failure with an Aldosterone Antagonist)2 and validated it in external data sets. At a minimum, their observation is new knowledge that may advance our understanding of the pathophysiology of HFpEF. It also indicates that endotrophin-based risk prediction has utility for patients, the health system, and designing future research.

Step-adjusted tree-based reinforcement learning for evaluating nested dynamic treatment regimes using test-and-treat observational data

Dynamic treatment regimes (DTRs) include a sequence of treatment decision rules, in which treatment is adapted over time in response to the changes in an individual's disease progression and health care history. In medical practice, nested test-and-treat strategies are common to improve cost-effectiveness. For example, for patients at risk of prostate cancer, only patients who have high prostate-specific antigen (PSA) need a biopsy, which is costly and invasive, to confirm the diagnosis and help determine the treatment if needed. A decision about treatment happens after the biopsy, and is thus nested within the decision of whether to do the test. However, current existing statistical methods are not able to accommodate such a naturally embedded property of the treatment decision within the test decision. Therefore, we developed a new statistical learning method, step-adjusted tree-based reinforcement learning, to evaluate DTRs within such a nested multistage dynamic decision framework using observational data. At each step within each stage, we combined the robust semiparametric estimation via augmented inverse probability weighting with a tree-based reinforcement learning method to deal with the counterfactual optimization. The simulation studies demonstrated robust performance of the proposed methods under different scenarios. We further applied our method to evaluate the necessity of prostate biopsy and identify the optimal test-and-treat regimes for prostate cancer patients using data from the Johns Hopkins University prostate cancer active surveillance dataset.

Impact of the COVID-19 Pandemic on the Appropriateness of Diagnostic Pathways of Autoimmune Rheumatic Diseases

OBJECTIVE

Early diagnosis of autoimmune rheumatic diseases (ARDs) is key to achieving effective treatment and improving prognosis. The coronavirus disease 2019 (COVID-19) pandemic has led to major changes in clinical practice on a global scale. We aimed to evaluate the impact of the COVID-19 pandemic on rheumatological clinical practice and autoimmunity testing demands.

METHODS

Data regarding the first rheumatological visits and new diagnoses, together with the autoimmunity laboratory testing volumes related to the COVID-19 pandemic phase (January-December 2020), were collected from medical records and the laboratory information system of a regional reference hospital (Basilicata, Italy) and compared with those obtained during the corresponding period in 2019.

RESULTS

A significant decrease in the 2020 autoimmunity laboratory test volume was found when compared with the same period in 2019 (9912 vs 14,100; P < 0.05). A significant decrease in first rheumatological visits and diagnosis (1272 vs 2336; P < 0.05) was also observed. However, an equivalent or higher percentage of positive autoimmunity results from outpatient services was recorded during 2020 when compared to the prepandemic state. Of note, COVID-19-associated decline in new diagnoses affected mainly less severe diseases. In contrast, ARDs with systemic involvement were diagnosed at the same levels as in the prepandemic period.

CONCLUSION

The COVID-19 pandemic has affected access to health services. However, our study highlighted that during the outbreak, greater appropriateness of the requests for laboratory tests and visits emerged, as shown by a greater percentage of positive test results and new diagnoses of more severe ARDs compared to the prepandemic period.

Health economic evaluations of medical tests: Translating laboratory information into value - A case study example

Health-care providers and funders are focused on identifying value in all their services and that includes laboratories. This means that in order to gain a share of scarce resources, laboratory professionals must also understand and assess the value of tests and that includes their economic impact. This can be assessed using health economic modelling tools which, when used in conjunction with a detailed value proposition for the test, can translate laboratory information into value. While a variety of health economic assessment tools are available, this review will focus on the use of decision analytic models which essentially compare the outcomes from pathways with and without the new test, the value of which is being assessed. A step-by-step framework is provided to guide laboratory professionals through the essential steps of conducting the evaluation. Initial steps include mapping the clinical pathway, understanding the goal of the evaluation, identifying the key stakeholders and their needs and determining a suitable analytical model. Following collection of the actual data, the validity of the model must be checked, and the robustness of the outcomes tested through sensitivity analysis. The last step is to translate the findings into measures of value which can then inform appropriate decisions by the stakeholders. This review of basic health economic modelling should enable laboratory professionals to have an understanding of how modelling can be applied to tests in their own environment and help deliver their potential value.

Superabsorbent Wound Dressing for Management of Patients With Moderate-to-Highly Exuding Chronic Leg Ulcers: An Early Stage Model-Based Benefit-Harm Assessment

The aim of the research is to assess the benefit-harm of superabsorbent polymers wound dressings based on polyacrylate polymers (SAPs) compared with standard of care (SoC) dressing mix for patients with moderate-to-highly exuding hard-to-heal leg ulcers. The SoC dressings mix was composed of other superabsorbents in 29% of cases, antimicrobials 26%, foams 20%, alginates 5%, and other dressings 19% weighted according to their frequency. We have used the decision-analytic modeling method, Markov process, as an adequate analytical solution for medical prognosis. We have combined the systematic literature search to identify the most relevant inputs for the analysis, with available patient-level clinical data concerning benefits of superabsorbent to generate a robust prediction of patient-relevant outcomes, including healing rates and health-related quality of life. Besides, we have qualitatively described adverse events associated with those treatments. Our research indicates that SAPs when compared with SoC dressing mix in a patient with moderate-to-highly excluding leg ulcers are leading to an improved healing rate with an absolute risk difference of 2.20% in 6 months and a relative risk of 1.07 in favor of SAP dressings. The attributable fraction among those exposed to SAP dressings of 6.6%, meaning that 6.6% of the healed ulcers could be attributed to having had the SAP dressing treatment instead of the SoC dressing treatment. Besides, SAP dressings lead to improved quality of life measured as incremental quality-adjusted life weeks (QALWs) of 0.13 QALWs.

Laboratory-Supported Multiple System Atrophy beyond Autonomic Function Testing and Imaging: A Systematic Review by the MoDiMSA Study Group

Background

Neuroimaging has been used to support a diagnosis of possible multiple system atrophy (MSA). Only blood pressure changes upon standing are included in the second consensus criteria but other autonomic function tests (AFT) are also useful to diagnose widespread and progressive autonomic failure typical of MSA. Additional diagnostic tools are of interest to improve accuracy of MSA diagnosis.

Objectives

To assess the utility of diagnostic tools beyond brain imaging and AFT in enhancing a laboratory-supported diagnosis of MSA to support the upcoming revision of the consensus criteria.

Methods

The International Parkinson and Movement Disorders Society MSA Study Group (MoDiMSA) performed a systematic review of original papers on biomarkers, sleep studies, genetic, neuroendocrine, neurophysiological, neuropsychological and other tests including olfactory testing and acute levodopa challenge test published before August 2019.

Results

Evaluation of history of levodopa responsiveness and olfaction is useful in patients in whom MSA-parkinsonian subtype is suspected. Neuropsychological testing is useful to exclude dementia at time of diagnosis. Applicability of sphincter EMG is limited. When MSA-cerebellar subtype is suspected, a screening for the common causes of adult-onset progressive ataxia is useful, including spinocerebellar ataxias in selected patients. Diagnosing stridor and REM sleep behavior disorder is useful in both MSA subtypes. However, none of these tools are validated in large longitudinal cohorts of postmortem confirmed MSA cases.

Conclusions

Despite limited evidence, additional laboratory work-up of patients with possible MSA beyond imaging and AFT should be considered to optimize the clinical diagnostic accuracy.

Economic value of insertable cardiac monitors in unexplained syncope in the United States

INTRODUCTION

Early use of insertable cardiac monitor (ICM) is recommended for patients with unexplained syncope following initial clinical workup, due to its superior ability to establish symptom-rhythm correlation compared with conventional testing (CONV). However, ICMs incur higher upfront costs, and the impact of additional diagnoses and resulting treatment on downstream costs and outcomes is unclear. We aimed to evaluate the cost-effectiveness of ICM compared with CONV for the diagnosis of arrhythmia in patients with unexplained syncope, from a US payer perspective.

METHODS

A Markov model was developed to estimate lifetime costs and benefits of arrhythmia diagnosis with ICM versus CONV, considering all related diagnostic and arrhythmia-related treatment costs and consequences. Cohort characteristics and costs were informed by original claims database analyses. Risks of mortality, syncopal recurrence, injury due to syncope and quality of life consequences from syncopal events were identified from the literature.

RESULTS

ICM was less costly and more effective than CONV. Most of the observed US$4532 cost savings were attributed to reduced downstream diagnostic testing. For every 1000 patients, ICM was projected to yield an additional 253 arrhythmia diagnoses and lead to treatment in an additional 168 patients. The ICM strategy resulted in overall improved outcomes (0.30 quality-adjusted life years gained), due to a reduction in syncope recurrence and injury resulting from arrhythmia treatment. The results were robust to changes in the base case parameters but sensitive to the model time horizon, underlying probability of syncope recurrence and prevalence of arrhythmias.

CONCLUSIONS

Our model projected that early ICM for the diagnosis of unexplained syncope reduced long-term costs, and led to an improvement in overall clinical outcomes by shortening time to arrhythmia treatment. The cost of ICM was outweighed by savings arising from fewer downstream diagnostic episodes, and the increased cost of treatment was counterbalanced by fewer syncope-related event costs.

Health Economic Decision Tree Models of Diagnostics for Dummies: A Pictorial Primer

Health economics is a discipline of economics applied to health care. One method used in health economics is decision tree modelling, which extrapolates the cost and effectiveness of competing interventions over time. Such decision tree models are the basis of reimbursement decisions in countries using health technology assessment for decision making. In many instances, these competing interventions are diagnostic technologies. Despite a wealth of excellent resources describing the decision analysis of diagnostics, two critical errors persist: not including diagnostic test accuracy in the structure of decision trees and treating sequential diagnostics as independent. These errors have consequences for the accuracy of model results, and thereby impact on decision making. This paper sets out to overcome these errors using color to link fundamental epidemiological calculations to decision tree models in a visually and intuitively appealing pictorial format. The paper is a must-read for modelers developing decision trees in the area of diagnostics for the first time and decision makers reviewing diagnostic reimbursement models.

Reducing overtreatment associated with overdiagnosis in cervical cancer screening-A model-based benefit-harm analysis for Austria

A general concern exists that cervical cancer screening using human papillomavirus (HPV) testing may lead to considerable overtreatment. We evaluated the trade-off between benefits and overtreatment among different screening strategies differing by primary tests (cytology, p16/Ki-67, HPV alone or in combinations), interval, age and diagnostic follow-up algorithms. A Markov state-transition model calibrated to the Austrian epidemiological context was used to predict cervical cancer cases, deaths, overtreatments and incremental harm-benefit ratios (IHBR) for each strategy. When considering the same screening interval, HPV-based screening strategies were more effective compared to cytology or p16/Ki-67 testing (e.g., relative reduction in cervical cancer with biennial screening: 67.7% for HPV + Pap cotesting, 57.3% for cytology and 65.5% for p16/Ki-67), but were associated with increased overtreatment (e.g., 19.8% more conizations with biennial HPV + Papcotesting vs. biennial cytology). The IHBRs measured in unnecessary conizations per additional prevented cancer-related death were 31 (quinquennial Pap + p16/Ki-67-triage), 49 (triennial Pap + p16/Ki-67-triage), 58 (triennial HPV + Pap cotesting), 66 (biennial HPV + Pap cotesting), 189 (annual Pap + p16/Ki-67-triage) and 401 (annual p16/Ki-67 testing alone). The IHBRs increased significantly with increasing screening adherence rates and slightly with lower age at screening initiation, with a reduction in HPV incidence or with lower Pap-test sensitivity. Depending on the accepted IHBR threshold, biennial or triennial HPV-based screening in women as of age 30 and biennial cytology in younger women may be considered in opportunistic screening settings with low or moderate adherence such as in Austria. In organized settings with high screening adherence and in postvaccination settings with lower HPV prevalence, the interval may be prolonged.

Cost-Effectiveness of Advanced Imaging Technologies in the Presurgical Workup of Epilepsy

The cost-effectiveness and benefit of many diagnostic tests used in the presurgical evaluation for persons with epilepsy is for the most part uncertain as is their influence on decision-making. The options we have at our disposal are ever increasing. Advanced imaging modalities aim to improve surgical candidacy by helping us better define the epileptogenic zone and optimize surgical planning. However, judicious use is important. Randomized controlled trials delineating which mode of investigation is superior are lacking. Presurgical tests do have incremental value by increasing surgical candidacy and refining surgical planning. The yield of additional imaging will increase with complex localization. However, every case must be tailored by hypothesis, cost, and accessibility. Future studies using a quantitative cost–benefit framework are needed to determine the cost-effectiveness of advanced diagnostic tests (beyond magnetic resonance imaging) in the presurgical evaluation of those with epilepsy.

Cervical cancer prevention in Morocco: a model-based cost-effectiveness analysis

Objective: Cervical cancer is a huge public health issue in Morocco which represents the second most frequent and fatal cancer among women. Countries that have not yet introduced the HPV vaccine could benefit greatly, but before implementation it is necessary to perform country-specific economic assessments that include current screening practices. Methods: A Markov model was developed to simulate the natural history of HPV and cervical cancer so as to calculate the long-term health benefits and costs of HPV vaccination and current screening by visual inspection with acetic acid (VIA). Starting from a previous transition probability matrix used for a model from Spain, the present model was calibrated to cervical cancer incidence from Morocco. Cost survey data was used to estimate the cost of screening and clinical procedures from the public healthcare perspective. Incremental cost-effectiveness ratios were calculated as 2018US$ per additional year of life saved (YLS) and both costs and health outcomes were discounted at 3%. Results: The expected reduction in lifetime risk of cervical cancer for current screening would be 14% at a cost of US$551/YLS compared with no intervention, assuming VIA every 3 years in women aged 30-49 at 10% coverage. HPV vaccination of pre-adolescent girls at 70% coverage would reduce the lifetime risk of cervical cancer by 62% at a cost of US$1,150/YLS, compared with no intervention. When implementing HPV vaccination in combination with current screening, vaccination would be dominated, and the combined strategy would provide a 69% reduction at a cost of US$2,843/YLS, compared with screening alone. Current screening would be dominated by vaccination when screening coverage is higher than 15%, whereas the combined strategy rapidly exceeds US$4,000/YLS. Conclusions: HPV vaccination could be highly effective and cost-effective in Morocco. Current screening would be good value for money compared with no intervention, but scaling-up screening coverage would make it inefficient compared with vaccination.

Toward a Framework for Outcome-Based Analytical Performance Specifications: A Methodology Review of Indirect Methods for Evaluating the Impact of Measurement Uncertainty on Clinical Outcomes

BACKGROUND

For medical tests that have a central role in clinical decision-making, current guidelines advocate outcome-based analytical performance specifications. Given that empirical (clinical trial-style) analyses are often impractical or unfeasible in this context, the ability to set such specifications is expected to rely on indirect studies to calculate the impact of test measurement uncertainty on downstream clinical, operational, and economic outcomes. Currently, however, a lack of awareness and guidance concerning available alternative indirect methods is limiting the production of outcome-based specifications. Therefore, our aim was to review available indirect methods and present an analytical framework to inform future outcome-based performance goals.

CONTENT

A methodology review consisting of database searches and extensive citation tracking was conducted to identify studies using indirect methods to incorporate or evaluate the impact of test measurement uncertainty on downstream outcomes (including clinical accuracy, clinical utility, and/or costs). Eighty-two studies were identified, most of which evaluated the impact of imprecision and/or bias on clinical accuracy. A common analytical framework underpinning the various methods was identified, consisting of 3 key steps: (a) calculation of "true" test values; (b) calculation of measured test values (incorporating uncertainty); and (c) calculation of the impact of discrepancies between (a) and (b) on specified outcomes. A summary of the methods adopted is provided, and key considerations are discussed.

CONCLUSIONS

Various approaches are available for conducting indirect assessments to inform outcome-based performance specifications. This study provides an overview of methods and key considerations to inform future studies and research in this area.

What Drives Detection and Diagnosis of Autism Spectrum Disorder? Looking Under the Hood of a Multi-stage Screening Process in Early Intervention

U.S. guidelines for detecting autism emphasize screening and also incorporate clinical judgment. However, most research focuses on the former. Among 1,654 children participating in a multi-stage screening protocol for autism, we used mixed methods to evaluate: (1) the effectiveness of a clinical decision rule that encouraged further assessment based not only on positive screening results, but also on parent or provider concern, and (2) the influence of shared decision-making on screening administration. Referrals based on concern alone were cost-effective in the current study, and reported concerns were stronger predictors than positive screens of time-to-complete referrals. Qualitative analyses suggest a dynamic relationship between parents' concerns, providers' concerns, and screening results that is central to facilitating shared decision-making and influencing diagnostic assessment.

Different evidence summaries have implications for contextualizing findings of meta-analysis of diagnostic tests

OBJECTIVE

To evaluate diagnostic tests, analysts use meta-analyses to provide inputs to parameters in decision models. Choosing parameter estimands from meta-analyses requires understanding the meta-analytic and decision-making contexts.

STUDY DESIGN AND SETTING

We expand on an analysis comparing positron emission tomography (PET), PET with computed tomography (PET/CT), and conventional workup (CW) in women with suspected recurrent breast cancer. We discuss Bayesian meta-analytic summaries (posterior mean over a set of existing studies, posterior estimate in an existing study, posterior predictive mean in a new study) used to estimate diagnostic test parameters (prevalence, sensitivity, specificity) needed to calculate quality-adjusted life years in a decision model contextualizing PET, PET/CT, and CW.

RESULTS

The mean and predictive mean give similar estimates, but the latter displays greater uncertainty. Namely, PET/CT outperforms CW on average but may not do better than CW when implemented in future settings.

CONCLUSION

Selecting estimands for decision model parameters from meta-analyses requires understanding the relationship between decision settings and meta-analysis studies' settings, specifically whether the former resemble one or all study settings or represents new settings. We provide an algorithm recommending appropriate estimands as input parameters in decision models for diagnostic tests to obtain output parameters consistent with the decision context.

Early cost-effectiveness modeling for better decisions in public research investment of personalized medicine technologies

Millions of dollars are spent on the development of new personalized medicine technologies. While these research costs are often supported by public research funds, many diagnostic tests and biomarkers are not adopted by the healthcare system due to lack of evidence on their cost-effectiveness. We describe a stepwise approach to conducting cost-effectiveness analyses that are performed early in the technology's development process and can help mitigate the potential risks of investment. Decision analytic modeling can identify the key drivers of cost effectiveness and provide minimum criteria that the technology needs to meet for adoption by public and private healthcare systems. A value of information analysis can quantify the added value of conducting more research to provide further evidence for policy decisions. These steps will allow public research funders to make better decisions on their investments to maximize the health benefits and to minimize the number of suboptimal technologies.

Integrating Qualitative Techniques in Model Development: A Case Study Using the Framework Approach

BACKGROUND

Despite their potential, there is limited uptake of formal qualitative methods in model development by modellers and health economists. The aim of this case study was to highlight in a real-world context how a qualitative approach has been applied to gain insight into current practice (delineating existing care pathways) for typhoid fever in Ghana, which can then assist in model structure conceptualisation in a model-based cost-effectiveness analysis.

METHODS

The perspectives of a range of healthcare professionals working in different settings and across different practices in the Eastern region of Ghana were captured with a self-administered survey using open-ended questions and analysed using the framework method.

RESULTS

A total of 51 completed questionnaires were retrieved representing a 73% response rate. It was found that two main care pathways for typhoid fever exist in Ghana and there was no consensus on how a new test might be applied to the existing pathways.

CONCLUSION

The two settings in Ghana have different care pathways and any cost-effectiveness analysis should consider the alternative pathways separately. This study demonstrated that framework analysis is a qualitative methodology that is likely to be accessible and feasible across a wide range of health economic settings.

Economic evaluation of typhoid - a review

INTRODUCTION

To evaluate the potential economic value and likely impact of a hypothetical rapid test in its early stages of development requires the use of models. The model structure and the type of model (dynamic/static) to employ are key considerations. The aim of the review was to explore the literature on typhoid economic evaluations and to explore the types of models that have been previously adopted in this setting for test-treat evaluations and to capture data on model inputs that may be useful for a de novo model.

AREAS COVERED

A systematic review was conducted to identify economic evaluations focused on typhoid in established literature databases. Eight studies were identified and included for narrative synthesis. The review has revealed that there have been relatively few economic evaluations that have focused on typhoid fever, all of which have focused on the impact of interventions at the population level (vaccination) but not the individual level (test-treat strategies).

EXPERT COMMENTARY

Under certain circumstances, either a static model or a transmission dynamic model may be appropriate in the evaluation of an intervention for typhoid fever. Typhoid test-treat modeling represents a gray area where further work is needed.

Use of diagnostic tests and the appropriateness of the treatment decision in patients with suspected urinary tract infection in primary care in Denmark - observational study

Background

Inappropriate prescription of antibiotics is the leading driver of antimicrobial resistance (AMR). The majority of antibiotics are prescribed in primary care.

Understanding how general practitioners (GPs) use diagnostic tests and the effect on treatment decision under daily practice conditions is important to reduce inappropriate prescription of antibiotics. The aim of the study was to investigate the use of diagnostic tests in primary care patients with suspected urinary tract infection (UTI) and to assess the appropriateness of the treatment decision (TD) under daily practice conditions in Denmark.

Methods

Prospective observational study. Symptomatic adult patients consulting general practice with suspected UTI recruited over 12 months. The diagnostic workup was registered in a standardized form. The appropriateness of the TD was assessed based on the results of a culture performed at a reference microbiological laboratory. TD was considered appropriate if a patient had a positive culture and was prescribed antibiotics or had a negative culture and was not prescribed antibiotics. TD was considered inappropriate if a patient had a negative culture and was prescribed antibiotics (overtreatment) or had a positive culture and was not prescribed antibiotics (undertreatment).

Results

Four hundred and eighty-eight patients were included. Dipstick was used in 98% of the patients and urine culture was used in 89% of the patients; 317 had the culture performed in practice and 117 had the culture performed at the hospital. The appropriateness of the final TD was significantly (p = 0.04) lower in patients without culture (55%) than in patients with culture performed in practice (71%) or at hospital (69%).

Conclusion

In a context with wide availability of diagnostic tests, GPs use diagnostic tests for the decision-making process in all patients with suspected UTI. Urine culture is used in the majority of the patients and is associated with a higher proportion of appropriate treatment decisions. Performance of urine culture is therefore important in reducing inappropriate antibiotic prescribing in patients with suspected UTI seeking care in general practice in Denmark.

Trial registration

ClinicalTrials.gov NCT02249273.

Electronic supplementary material

The online version of this article (10.1186/s12875-018-0754-1) contains supplementary material, which is available to authorized users.

Epigenome-based cancer risk prediction: rationale, opportunities and challenges

The incidence of cancer is continuing to rise and risk-tailored early diagnostic and/or primary prevention strategies are urgently required. The ideal risk-predictive test should: integrate the effects of both genetic and nongenetic factors and aim to capture these effects using an approach that is both biologically stable and technically reproducible; derive a score from easily accessible biological samples that acts as a surrogate for the organ in question; and enable the effectiveness of risk-reducing measures to be monitored. Substantial evidence has accumulated suggesting that the epigenome and, in particular, DNA methylation-based tests meet all of these requirements. However, the development and implementation of DNA methylation-based risk-prediction tests poses considerable challenges. In particular, the cell type specificity of DNA methylation and the extensive cellular heterogeneity of the easily accessible surrogate cells that might contain information relevant to less accessible tissues necessitates the use of novel methods in order to account for these confounding issues. Furthermore, the engagement of the scientific community with health-care professionals, policymakers and the public is required in order to identify and address the organizational, ethical, legal, social and economic challenges associated with the routine use of epigenetic testing.

Cost-Effectiveness Models in Breast Cancer Screening in the General Population: A Systematic Review

BACKGROUND

Many Western countries have long-established population-based mammography screening programs. Prior to implementing these programs, decision-analytic modeling was widely used to inform decisions.

OBJECTIVE

The aim of this study was to perform a systematic review of cost-effectiveness models in breast cancer screening in the general population to analyze their structural and methodological approaches.

METHODS

A systematic literature search for health economic models was performed in the electronic databases MEDLINE (Ovid), EMBASE, CRD Databases, Cochrane Library, and EconLit in August 2011 with updates in June 2013, April 2015, and November 2016. To assess studies systematically, a standardized form was applied to extract relevant information that was then summarized in evidence tables.

RESULTS

Thirty-five studies were included; 27 state-transition models were analyzed using cohort (n = 12) and individual-level simulation (n = 15). Twenty-one studies modeled the natural history of breast cancer and predicted mortality as a function of the early detection modality. The models employed different assumptions regarding ductal carcinoma in situ. Thirteen studies performed cost-utility analyses with different sources for utility values, but assumptions were often made about utility weights. Twenty-two models did not report any validation.

CONCLUSION

State-transition modeling was the most frequently applied analytic approach. Different methods in modeling the progression of ductal carcinoma in situ to invasive cancer were identified because there is currently no agreement on the biological behavior of noninvasive breast cancer. Main weaknesses were the lack of precise utility estimates and insufficient reporting of validation. Sensitivity analyses of assumptions regarding ductal carcinoma in situ and in particular adequate validation are critical to minimize the risk of biased model outcomes.

[Benefit assessment of diagnostic procedures: quo vadimus?]

Not only therapeutic procedures, but also diagnostic procedures, have to demonstrate their patient-relevant benefits if they are to be reimbursed by public health insurance. Randomized trials comparing two diagnostic procedures allow us to assess these benefits directly if appropriate outcomes are used. However, owing to the widespread lack of such studies, it is now necessary to use the "linked evidence" approach as well, trying to predict the patient-relevant benefits from the results of comparative accuracy studies. Such a prediction is based on explicitly specifying our expectations with regard to the consequences of a change in diagnosis at the level of a single patient. We discuss the basic properties of these two approaches, which are relevant to the understanding of their possible role in the benefit assessment of diagnostic procedures. We try to predict the future roles of the two approaches and outline some of the issues on which a consensus is required to allow their successful use in benefit assessment. Furthermore, we indicate some of the developments related to the paradigm of individualized care that may influence the use of benefit assessments for diagnostic studies in the future.

Appropriateness of diagnostics tests

The evolution of the concept of 'appropriateness', in the three past decades, from 'no harm' and 'no waste' to 'medical decision-making' and 'determining outcomes' highlights two main points: its foundation is evidence-based medicine, and it is a quality of every phase of the total testing process, not only for the selection of tests. Nevertheless, appropriateness in Laboratory Hematology, as well as in Laboratory Medicine, is an elusive concept: 'Appropriateness' interplays with 'patient's safety', 'healthcare costs', 'clinical decision-making', and 'effectiveness', and the criteria for appropriateness, mainly adherence to clinical guidelines, are often not evidence-based and not always consensus-based. Moreover, practising appropriateness is a complex issue because of the ambiguity of the criteria and targets, the never-ending work of implementing guidelines and their audit, and the uniqueness of the clinical situation of the individual patient. Authors agree on some practical rules: establishing a multidisciplinary and multiprofessional team, choosing carefully clinical targets, finding or building evidences, sharing guidelines with clinicians, choosing adequate tools for changing, working hard on implementation, identifying the 'right' laboratory methods and processes, checking progress indefinitely, providing information, interpretations, and consultations, and promoting feedback and audits. The success depends on the 'right' combination of educational, operative, and reinforcing interventions. Competences in organization, in implementation science, and in interpersonal relationship management are essential as well as knowledge and experience in Hematology, not only in Laboratory Hematology.

Impact of Molecular Diagnostics for Tuberculosis on Patient-Important Outcomes: A Systematic Review of Study Methodologies

Background

Several reviews on the accuracy of Tuberculosis (TB) Nucleic Acid Amplification Tests (NAATs) have been performed but the evidence on their impact on patient-important outcomes has not been systematically reviewed. Given the recent increase in research evaluating such outcomes and the growing list of TB NAATs that will reach the market over the coming years, there is a need to bring together the existing evidence on impact, rather than accuracy. We aimed to assess the approaches that have been employed to measure the impact of TB NAATs on patient-important outcomes in adults with possible pulmonary TB and/or drug-resistant TB.

Methods

We first develop a conceptual framework to clarify through which mechanisms the improved technical performance of a novel TB test may lead to improved patient outcomes and outline which designs may be used to measure them. We then systematically review the literature on studies attempting to assess the impact of molecular TB diagnostics on such outcomes and provide a narrative synthesis of designs used, outcomes assessed and risk of bias across different study designs.

Results

We found 25 eligible studies that assessed a wide range of outcomes and utilized a variety of experimental and observational study designs. Many potentially strong design options have never been used. We found that much of the available evidence on patient-important outcomes comes from a small number of settings with particular epidemiological and operational context and that confounding, time trends and incomplete outcome data receive insufficient attention.

Conclusions

A broader range of designs should be considered when designing studies to assess the impact of TB diagnostics on patient outcomes and more attention needs to be paid to the analysis as concerns about confounding and selection bias become relevant in addition to those on measurement that are of greatest concern in accuracy studies.

Setting analytical performance specifications based on outcome studies - is it possible?

The 1st Strategic Conference of the European Federation of Clinical Chemistry and Laboratory Medicine proposed a simplified hierarchy for setting analytical performance specifications (APS). The top two levels of the 1999 Stockholm hierarchy, i.e., evaluation of the effect of analytical performance on clinical outcomes and clinical decisions have been proposed to be replaced by one outcome-based model. This model can be supported by: (1a) direct outcome studies; and (1b) indirect outcome studies investigating the impact of analytical performance of the test on clinical classifications or decisions and thereby on the probability of patient relevant clinical outcomes. This paper reviews the need for outcome-based specifications, the most relevant types of outcomes to be considered, and the challenges and limitations faced when setting outcome-based APS. The methods of Model 1a and b are discussed and examples are provided for how outcome data can be translated to APS using the linked evidence and simulation or decision analytic techniques. Outcome-based APS should primarily reflect the clinical needs of patients; should be tailored to the purpose, role and significance of the test in a well defined clinical pathway; and should be defined at a level that achieves net health benefit for patients at reasonable costs. Whilst it is acknowledged that direct evaluations are difficult and may not be possible for all measurands, all other forms of setting APS should be weighed against that standard, and regarded as approximations. Better definition of the relationship between the analytical performance of tests and health outcomes can be used to set analytical performance criteria that aim to improve the clinical and cost-effectiveness of laboratory tests.

Evidence used in model-based economic evaluations for evaluating pharmacogenetic and pharmacogenomic tests: a systematic review protocol

INTRODUCTION

Decision models can be used to conduct economic evaluations of new pharmacogenetic and pharmacogenomic tests to ensure they offer value for money to healthcare systems. These models require a great deal of evidence, yet research suggests the evidence used is diverse and of uncertain quality. By conducting a systematic review, we aim to investigate the test-related evidence used to inform decision models developed for the economic evaluation of genetic tests.

METHODS AND ANALYSIS

We will search electronic databases including MEDLINE, EMBASE and NHS EEDs to identify model-based economic evaluations of pharmacogenetic and pharmacogenomic tests. The search will not be limited by language or date. Title and abstract screening will be conducted independently by 2 reviewers, with screening of full texts and data extraction conducted by 1 reviewer, and checked by another. Characteristics of the decision problem, the decision model and the test evidence used to inform the model will be extracted. Specifically, we will identify the reported evidence sources for the test-related evidence used, describe the study design and how the evidence was identified. A checklist developed specifically for decision analytic models will be used to critically appraise the models described in these studies. Variations in the test evidence used in the decision models will be explored across the included studies, and we will identify gaps in the evidence in terms of both quantity and quality.

DISSEMINATION

The findings of this work will be disseminated via a peer-reviewed journal publication and at national and international conferences.

Assessing value of innovative molecular diagnostic tests in the concept of predictive, preventive, and personalized medicine

Molecular diagnostic tests drive the scientific and technological uplift in the field of predictive, preventive, and personalized medicine offering invaluable clinical and socioeconomic benefits to the key stakeholders. Although the results of diagnostic tests are immensely influential, molecular diagnostic tests (MDx) are still grudgingly reimbursed by payers and amount for less than 5 % of the overall healthcare costs. This paper aims at defining the value of molecular diagnostic test and outlining the most important components of "value" from miscellaneous assessment frameworks, which go beyond accuracy and feasibility and impact the clinical adoption, informing healthcare resource allocation decisions. The authors suggest that the industry should facilitate discussions with various stakeholders throughout the entire assessment process in order to arrive at a consensus about the depth of evidence required for positive marketing authorization or reimbursement decisions. In light of the evolving "value-based healthcare" delivery practices, it is also recommended to account for social and ethical parameters of value, since these are anticipated to become as critical for reimbursement decisions and test acceptance as economic and clinical criteria.

Thresholds and accuracy in screening tools for early detection of psychopathology

BACKGROUND

The accuracy of any screening instrument designed to detect psychopathology among children is ideally assessed through rigorous comparison to 'gold standard' tests and interviews. Such comparisons typically yield estimates of what we refer to as 'standard indices of diagnostic accuracy', including sensitivity, specificity, positive predictive value (PPV), and negative predictive value. However, whereas these statistics were originally designed to detect binary signals (e.g., diagnosis present or absent), screening questionnaires commonly used in psychology, psychiatry, and pediatrics typically result in ordinal scores. Thus, a threshold or 'cut score' must be applied to these ordinal scores before accuracy can be evaluated using such standard indices. To better understand the tradeoffs inherent in choosing a particular threshold, we discuss the concept of 'threshold probability'. In contrast to PPV, which reflects the probability that a child whose score falls at or above the screening threshold has the condition of interest, threshold probability refers specifically to the likelihood that a child whose score is equal to a particular screening threshold has the condition of interest.

METHOD

The diagnostic accuracy and threshold probability of two well-validated behavioral assessment instruments, the Child Behavior Checklist Total Problem Scale and the Strengths and Difficulties Questionnaire total scale were examined in relation to a structured psychiatric interview in three de-identified datasets.

RESULTS

Although both screening measures were effective in identifying groups of children at elevated risk for psychopathology in all samples (odds ratios ranged from 5.2 to 9.7), children who scored at or near the clinical thresholds that optimized sensitivity and specificity were unlikely to meet criteria for psychopathology on gold standard interviews.

CONCLUSIONS

Our results are consistent with the view that screening instruments should be interpreted probabilistically, with attention to where along the continuum of positive scores an individual falls.

Current Evidence and Future Perspectives on the Effective Practice of Patient-Centered Laboratory Medicine

BACKGROUND

Systematic evidence of the contribution made by laboratory medicine to patient outcomes and the overall process of healthcare is difficult to find. An understanding of the value of laboratory medicine, how it can be determined, and the various factors that influence it is vital to ensuring that the service is provided and used optimally.

CONTENT

This review summarizes existing evidence supporting the impact of laboratory medicine in healthcare and indicates the gaps in our understanding. It also identifies deficiencies in current utilization, suggests potential solutions, and offers a vision of a future in which laboratory medicine is used optimally to support patient care.

SUMMARY

To maximize the value of laboratory medicine, work is required in 5 areas: (a) improved utilization of existing and new tests; (b) definition of new roles for laboratory professionals that are focused on optimizing patient outcomes by adding value at all points of the diagnostic brain-to-brain cycle; (c) development of standardized protocols for prospective patient-centered studies of biomarker clinical effectiveness or extraanalytical process effectiveness; (d) benchmarking of existing and new tests in specified situations with commonly accepted measures of effectiveness; (e) agreed definition and validation of effectiveness measures and use of checklists for articles submitted for publication. Progress in these areas is essential if we are to demonstrate and enhance the value of laboratory medicine and prevent valuable information being lost in meaningless data. This requires effective collaboration with clinicians, and a determination to accept patient outcome and patient experience as the primary measure of laboratory effectiveness.

Assessing the value of cardiac biomarkers: going beyond diagnostic accuracy?

In this era of scrutinized resource utilization, providers and payers are focused on the value of healthcare interventions more than ever. Cost-effectiveness evaluations are required by some health authorities and requested by others in order to guide budget allocation decisions. In the past, these evaluations did not methodologically consider laboratory diagnostics. We set out to explore the current requirements of health technology agencies that include laboratory diagnostics and describe, through a review of the literature, alternative methods for establishing the value of a biomarker or labroatory diagnostic beyond assay specifications and performance. The aim of this study was to evaluate the current use of a linked evidence approach in cost-effectiveness studies for cardiac laboratory tests in the last 5 years.

Analyzing paired diagnostic studies by estimating the expected benefit

When the efficacy of a new medical drug is compared against that of an established competitor in a randomized controlled trial, the difference in patient-relevant outcomes, such as mortality, is usually measured directly. In diagnostic research, however, the impact of diagnostic procedures is of an indirect nature as test results do influence downstream clinical decisions, but test performance (as characterized by sensitivity, specificity, and the predictive values of a procedure) is, at best, only a surrogate endpoint for patient outcome and does not necessarily translate into it. Not many randomized controlled trials have been conducted so far in diagnostic research, and, hence, we need alternative approaches to close the gap between test characteristics and patient outcomes. Several informal approaches have been suggested in order to close this gap, and decision modeling has been advocated as a means of obtaining formal approaches. Recently, the expected benefit has been proposed as a quantity that allows a simple formal approach, and we take up this suggestion in this paper. We regard the expected benefit as an estimation problem and consider two approaches to statistical inference. Moreover, using data from a previously published study, we illustrate the possible insights to be gained from the application of formal inference techniques to determine the expected benefit.

Cost-effectiveness of PET and PET/computed tomography: a systematic review

The development of clinical diagnostic procedures comprises early-phase and late-phase studies to elucidate diagnostic accuracy and patient outcome. Economic assessments of new diagnostic procedures compared with established work-ups indicate additional cost for 1 additional unit of effectiveness measure by means of incremental cost-effectiveness ratios when considering the replacement of the standard regimen by a new diagnostic procedure. This article discusses economic assessments of PET and PET/computed tomography reported until mid-July 2014. Forty-seven studies on cancer and noncancer indications were identified but, because of the widely varying scope of the analyses, a substantial amount of work remains to be done.

The move from accuracy studies to randomized trials in PET: current status and future directions

Since the influential study by van Tinteren et al. published in The Lancet in 2002, there have been an increasing number of diagnostic randomized controlled trials (RCTs) investigating the benefit of PET. If they provide valid and useful information on the benefit, these studies can play an important role in informing guideline developers and policy makers. Our aim was to investigate how far the nuclear medicine community has come on its way from accuracy studies to RCTs and which issues we have to take into account in planning future studies.

METHODS

We conducted a systematic review of diagnostic randomized trials, in which PET was applied in only one arm. We covered published studies as well as registered unpublished and planned studies. We considered 3 quality indicators related to the usefulness of a trial to generate evidence for a clinical benefit: use of patient-important outcome, sufficient sample size, and current standard as comparator.

RESULTS

Fourteen published and 15 planned studies were identified. Five of the published studies and 12 of the planned studies did not use a patient-important outcome. Sample sizes were often so small that a significant result could be expected only under the assumption of a substantial reduction in the event rate. Comparators typically reflected the current standard.

CONCLUSION

If we consider the traditional areas of primary diagnosis, staging, and follow-up, then the number and quality of RCTs on PET is currently not sufficient to provide a major source for evidence-based decisions on the clinical benefit of PET. There will also be a future need in these traditional areas to deduce the clinical benefit of PET from the results of accuracy studies. The situation may be more favorable for the areas of treatment planning and response evaluation. Choice of patient-important outcomes and sufficient sample sizes are crucial issues in planning RCTs to demonstrate the clinical benefit of using PET.

The added clinical and economic value of diagnostic testing for epilepsy surgery

•

We assess the added value of diagnostic imaging for epilepsy surgery.

•

A cost-effectiveness evaluation is conducted considering a range of strategies.

•

We find that additional PET and invasive EEG investigations are most cost-effective.

•

The long-term effectiveness of surgery has a large impact on the optimal strategy.

The costs, benefits and risks associated with diagnostic imaging investigations for epilepsy surgery necessitate the identification of an optimal pathway in the pre-surgical workup. In order to assess the added value of additional investigations a full cost-effectiveness evaluation should be conducted, taking into account all of the life-time costs and benefits associated with undertaking additional investigations. This paper considers and applies the appropriate framework against which a full evaluation should be assessed.

We conducted a systematic review to evaluate the progression of the literature through this framework, finding that only isolated elements of added value have been appropriately evaluated. The results from applying the full added value framework are also presented, identifying an optimal strategy for pre-surgical evaluation for temporal lobe epilepsy surgery. Our results suggest that additional FDG–PET and invasive EEG investigations after an initially discordant MRI and video-EEG appears cost-effective, and that the value of subsequent invasive-EEGs is closely linked to the maintenance of longer-term benefits after surgery.

It is integral to the evaluation of imaging technologies in the work-up for epilepsy surgery that the impact of the use of these technologies on clinical decision-making, and on further treatment decisions, is considered fully when informing cost-effectiveness.

The validation of screening tests: meet the new screen same as the old screen?

The focus of this report is to examine the process of validation of new screening tests designed to detect the problem gambler in research and practice settings. A hierarchical or phases of evaluation model is presented as a conceptual framework to describe the basic features of the validation process and its implications for application and interpretation of test results. The report describes a number of threats to validity in the form of sources of unintended bias that when unrecognized may lead to incorrect interpretations of study results and the drawing of incorrect conclusions about the usefulness of the new screening tests. Examples drawn from the gambling literature on problem gambling are used to illustrate some of the more important concepts including spectrum bias and clinical variation in test accuracy. The concept of zones of severity and the bias inherent in selecting criterion thresholds are reviewed. A definition of reference or study gold standard is provided. The use of 2-stage designs to establish validity by efficiently using reference standards to determine indices of accuracy and prevalence is recommended.

PET in the clinical management of glioma: evidence map

OBJECTIVE

Several studies have assessed PET to complement the anatomic information obtained from other imaging modalities in various clinical contexts for the management of glioma. We constructed an evidence map of clinical evidence on the use of PET in glioma and identified research gaps.

MATERIALS AND METHODS

We searched PubMed and Scopus (from inception through June 30, 2011) to identify studies assessing the use of PET for glioma regardless of setting of care or indication. We extracted test objectives, study characteristics, and phases of diagnostic evidence and then assessed research diversity and temporal trends in the literature. We excluded studies assessing only technical feasibility and optimization of PET.

RESULTS

A total of 129 studies were considered eligible; the number of articles published annually has greatly increased over time (p for trend < 0.001). Most studies (n = 118, 91%) assessed diagnostic or prognostic performance; fewer studies reported on the impact of PET on diagnostic thinking (n = 4, 3%), therapeutic decisions (n = 4, 3%), or patient-relevant clinical outcomes (n = 3; 2%). Fluorine-18 FDG (n = 73, 57%) or (11)C-methionine (n = 44, 34%) were the two most commonly evaluated PET tracers. Pretherapy assessment (n = 72, 56%) and monitoring of treatment response (n = 48, 37%) were the most common settings of test use assessed in the research studies.

CONCLUSION

More primary studies, particularly studies of newer tracers focusing on biopsy or treatment planning, are needed to better characterize the role of PET in specific contexts.

Use of a decision-analytic model in a health technology assessment: beyond measuring value for money

The well-designed, model-based cost-utility analysis by Ginsberg and colleagues provides useful information on the value for money of universal GBS screening in Israel. An extended application of the model-based approach used in the study could provide policymakers additional practical information on the budget impact of a potential universal GBS screening program. Such an approach could also be used to guide future research priorities in the prevention of GBS in Israel, by measuring the value of seeking further information to reduce the uncertainty in the cost-effectiveness of universal GBS screening.

This is a commentary on http://www.ijhpr.org/content/2/1/6.

Economic evaluations and diagnostic testing: an illustrative case study approach

OBJECTIVE

The aim of this study was to present a clear process of synthesizing test accuracy data when conducting economic evaluations of diagnostic tests for health technology assessment (HTA) assessors and health economists.

METHODS

We appraised the methods advocated for using diagnostic test accuracy data in economic evaluations. We used a case study of fetal anemia in which data from a screening test are used in combination with a confirmatory test.

RESULTS

We developed a step-by-step guide and consider two scenarios: when data on test accuracy from several studies are based on (i) the same test threshold for positivity and (ii) different test thresholds.

CONCLUSIONS

We conclude that each approach has its strengths and limitations. We show that the optimal operating point of the test should be identified to determine the true cost-effectiveness of the test. We advocate that these issues require a multidisciplinary team of health economists, decision modelers and statisticians.

Improving the efficiency and relevance of evidence-based recommendations in the era of whole-genome sequencing: an EGAPP methods update

To provide an update on recent revisions to Evaluation of Genomic Applications in Practice and Prevention (EGAPP) methods designed to improve efficiency, and an assessment of the implications of whole genome sequencing for evidence-based recommendation development. Improvements to the EGAPP approach include automated searches for horizon scanning, a quantitative ranking process for topic prioritization, and the development of a staged evidence review and evaluation process. The staged process entails (i) triaging tests with minimal evidence of clinical validity, (ii) using and updating existing reviews, (iii) evaluating clinical validity prior to analytic validity or clinical utility, (iv) using decision modeling to assess potential clinical utility when direct evidence is not available. EGAPP experience to date suggests the following approaches will be critical for the development of evidence based recommendations in the whole genome sequencing era: (i) use of triage approaches and frameworks to improve efficiency, (ii) development of evidence thresholds that consider the value of further research, (iii) incorporation of patient preferences, and (iv) engagement of diverse stakeholders. The rapid advances in genomics present a significant challenge to traditional evidence based medicine, but also an opportunity for innovative approaches to recommendation development.

Adding decision models to systematic reviews: informing a framework for deciding when and how to do so

OBJECTIVE

Decision models are sometimes used alongside systematic reviews to synthesize evidence. Clarity, however, is lacking about when and how to conduct modeling studies in tandem with systematic reviews, as well as about how to evaluate and present model results. The objective of this study was to collect and analyze information from various sources to inform the development of a framework for deciding when and how a decision model should be added to a systematic review.

METHODS

We collected data through 1) review and analysis of evidence reports that used decision models; 2) review and synthesis of current best practices for the development of decision models; 3) interviews of Evidence-Based Practice Center directors and selected staff, United States Preventive Services Task Force members, and decision modelers who developed models used by the United States Preventive Services Task Force; and 4) a focus group of expert modelers.

RESULTS

Models are well suited to address gaps in the literature, better suited for certain types of research questions, and essential for determining the value of information relating to future research. Opinions differ regarding whether model outputs constitute evidence, but interviewees expressed concern over the lack of standards and directions in grading and reporting such "evidence." Interviews of stakeholders and modelers revealed the importance of communication and presentation of model results as well as the importance of model literacy and involvement of stakeholders.

CONCLUSIONS

The study demonstrates the need for a framework for deciding when and how to use models alongside systematic reviews and provides information to develop such a framework.