Concordance between decision analysis and matching systematic review of randomized controlled trials in assessment of treatment comparisons: a systematic review

Decision analysis in cardiac surgery: a scoping review and methodological primer

OBJECTIVES

Randomized controlled trials are the gold standard for evidence generation in medicine but are limited by their real-world generalizability, resource needs, shorter follow-up durations and inability to be conducted for all clinical questions. Decision analysis (DA) models may simulate trials and observational studies by using existing data and evidence- and expert-informed assumptions and extend analyses over longer time horizons, different study populations and specific scenarios, helping to translate population outcomes to patient-specific clinical and economic outcomes. Here, we present a scoping review and methodological primer on DA for cardiac surgery research.

METHODS

A scoping review was performed using the PubMed/MEDLINE, EMBASE and Web of Science databases for cardiac surgery DA studies published until December 2021. Articles were summarized descriptively to quantify trends and ascertain methodological consistency.

RESULTS

A total of 184 articles were identified, among which Markov models (N = 92, 50.0%) were the most commonly used models. The most common outcomes were costs (N = 107, 58.2%), quality-adjusted life-years (N = 96, 52.2%) and incremental cost-effectiveness ratios (N = 89, 48.4%). Most (N = 165, 89.7%) articles applied sensitivity analyses, most frequently in the form of deterministic sensitivity analyses (N = 128, 69.6%). Reporting of guidelines to inform the model development and/or reporting was present in 22.3% of articles.

CONCLUSION

DA methods are increasing but remain limited and highly variable in cardiac surgery. A methodological primer is presented and may provide researchers with the foundation to start with or improve DA, as well as provide readers and reviewers with the fundamental concepts to review DA studies.

Identifying neonatal early-onset sepsis test and treatment decision thresholds

OBJECTIVE

To derive testing and treatment thresholds for early-onset neonatal sepsis and compare them to thresholds used in the Kaiser-Permanente (KP) Sepsis Calculator.

METHODS

Using surveys distributed in the United States, Brazil and Italy, decision thresholds were derived via self-identified thresholds selected from structured lists (Method 1), and based on clinical vignette responses for testing and treatment with or without inclusion of associated relative risk (Methods 2 and 3).

RESULTS

Using Method 1, both testing and treatment thresholds were higher than the KP calculator thresholds. Test thresholds were lower (Method 2) or equivalent (Method 3) to KP using clinical vignettes. No vignette reached the 50% cutoff necessary to define a treatment threshold.

CONCLUSION

The test threshold used by the KP calculator is the same as the threshold chosen by clinicians given a vignette and risk estimate. The KP treatment threshold is lower than that derived using all 3 methods.

A framework for meaningful use of clinical decision model: A diabetic nephropathy prediction modeling based on real world data

This study aims to propose a framework for developing a sharable predictive model of diabetic nephropathy (DN) to improve the clinical efficiency of automatic DN detection in data intensive clinical scenario. Different classifiers have been developed for early detection, while the heterogeneity of data makes meaningful use of such developed models difficult. Decision tree (DT) and random forest (RF) were adopted as training classifiers in de-identified electronic medical record dataset from 6,745 patients with diabetes. After model construction, the obtained classification rules from classifier were coded in a standard PMML file. A total of 39 clinical features from 2159 labeled patients were included as risk factors in DN prediction after data preprocessing. The mean testing accuracy of the DT classifier was 0.8, which was consistent to that of the RF classifier (0.823). The DT classifier was choose to recode as a set of operable rules in PMML file that could be transferred and shared, which indicates the proposed framework of constructing a sharable prediction model via PMML is feasible and will promote the interoperability of trained classifiers among different institutions, thus achieving meaningful use of clinical decision making. This study will be applied to multiple sites to further verify feasibility.

Concordance between decision analysis and matching systematic review of randomized controlled trials in assessment of treatment comparisons: a systematic review

Background

Systematic review (SR) of randomized controlled trials (RCT) is the gold standard for informing treatment choice. Decision analyses (DA) also play an important role in informing health care decisions. It is unknown how often the results of DA and matching SR of RCTs are in concordance. We assessed whether the results of DA are in concordance with SR of RCTs matched on patient population, intervention, control, and outcomes.

Methods

We searched PubMed up to 2008 for DAs comparing at least two interventions followed by matching SRs of RCTs. Data were extracted on patient population, intervention, control, and outcomes from DAs and matching SRs of RCTs. Data extraction from DAs was done by one reviewer and from SR of RCTs by two independent reviewers.

Results

We identified 28 DAs representing 37 comparisons for which we found matching SR of RCTs. Results of the DAs and SRs of RCTs were in concordance in 73% (27/37) of cases. The sensitivity analyses conducted in either DA or SR of RCTs did not impact the concordance. Use of single (4/37) versus multiple data source (33/37) in design of DA model was statistically significantly associated with concordance between DA and SR of RCTs.

Conclusions

Our findings illustrate the high concordance of current DA models compared with SR of RCTs. It is shown previously that there is 50% concordance between DA and matching single RCT. Our study showing the concordance of 73% between DA and matching SR of RCTs underlines the importance of totality of evidence (i.e. SR of RCTs) in the design of DA models and in general medical decision-making.