Automated critiquing of medical decision trees

Patient decision aids: a content analysis based on a decision tree structure

INTRODUCTION

This paper presents the preliminary results of a decision-tree analysis of Patient Decision Aids (PDA). PDAs are online or offline tools used to structure health information, elicit relevant values and emphasize the decision as a process, in ways that help patients make more informed health decisions individually or with relevant others.

METHOD

Twenty PDAs are randomly selected from the International Patient Decision Aids Standards (IPDAS) ( https://decisionaid.ohri.ca/AZlist.html ) approved list. An evaluation tool is built bottom-up and top-down and results are described in terms of communicating uncertainty, completeness of the decision tree, ambiguous or misleading phrasing, overall strategies suggested within personal stories.

RESULTS

Twelve of the analyzed PDAs had branches of the decision tree which were not discussed in the tool and 6 had logically ambiguous phrasing. Many tools included dichotomous options, when the option range was wider. Several options were clustered within the "Do not take/Do not do" option and thus the PDA failed to provide all comparisons necessary to make a decision. Some tools employ expressions that do not differentiate between lack of information and known negative effects. Other tools provide unequal amounts or non-comparable bits of information about the options.

CONCLUSION

These results indicate a very loose range of interpretations of what constitutes an option, a treatment, and a treatment option. It thus emphasizes a gap between theory and practice in the evaluation of PDAs. Future developments of PDA evaluation tools should keep track of missing decision tree branches, accurate communication of uncertainty, ambiguity, and lack of knowledge and consider using measures for evaluating the completeness of the option spectrum at an agreed period in time.

From knowledge bases to decision models

In recent years there has been a growing interest among AI researchers in probabilistic and decision modelling, spurred by significant advances in representation and computation with network modelling formalisms. In applying these techniques to decision support tasks, fixed network models have proven to be inadequately expressive when a broad range of situations must be handled. Hence many researchers have sought to combine the strengths of flexible knowledge representation languages with the normative status and well-understood computational properties of decision-modelling formalisms and algorithms. One approach is to encode general knowledge in an expressive language, then dynamically construct a decision model for each particular situation or problem instance. We have developed several systems adopting this approach, which illustrate a variety of interesting techniques and design issues.

Developing and validating predictive decision tree models from mining chemical structural fingerprints and high-throughput screening data in PubChem

Background

Recent advances in high-throughput screening (HTS) techniques and readily available compound libraries generated using combinatorial chemistry or derived from natural products enable the testing of millions of compounds in a matter of days. Due to the amount of information produced by HTS assays, it is a very challenging task to mine the HTS data for potential interest in drug development research. Computational approaches for the analysis of HTS results face great challenges due to the large quantity of information and significant amounts of erroneous data produced.

Results

In this study, Decision Trees (DT) based models were developed to discriminate compound bioactivities by using their chemical structure fingerprints provided in the PubChem system . The DT models were examined for filtering biological activity data contained in four assays deposited in the PubChem Bioassay Database including assays tested for 5HT1a agonists, antagonists, and HIV-1 RT-RNase H inhibitors. The 10-fold Cross Validation (CV) sensitivity, specificity and Matthews Correlation Coefficient (MCC) for the models are 57.2~80.5%, 97.3~99.0%, 0.4~0.5 respectively. A further evaluation was also performed for DT models built for two independent bioassays, where inhibitors for the same HIV RNase target were screened using different compound libraries, this experiment yields enrichment factor of 4.4 and 9.7.

Conclusion

Our results suggest that the designed DT models can be used as a virtual screening technique as well as a complement to traditional approaches for hits selection.

An architecture for knowledge-based construction of decision models

Clinical application of decision analysis has been limited by unfamiliarity of clinicians with the technique, large data requirements, and the length of time needed to construct models. In order to make decision modeling more accessible to clinicians, the authors developed a computer program to construct decision models automatically. The system contains two separate knowledge bases. One contains frames encoding knowledge of the medical domain, the evaluation of pulmonary disease in patients infected with the human immunodeficiency virus (HIV). The other contains rules of correct decision model construction that guide the selection of items from the domain knowledge base and their insertion into the decision model. The system can create either a tree or an influence diagram that satisfies previously published critiquing rules. The system has the potential to enable novices to construct useful decision models and to provide individualized decision-analytic advice to clinicians in real time.

Medical diagnostic decision support systems--past, present, and future: a threaded bibliography and brief commentary

Articles about medical diagnostic decision support (MDDS) systems often begin with a disclaimer such as, "despite many years of research and millions of dollars of expenditures on medical diagnostic systems, none is in widespread use at the present time." While this statement remains true in the sense that no single diagnostic system is in widespread use, it is misleading with regard to the state of the art of these systems. Diagnostic systems, many simple and some complex, are now ubiquitous, and research on MDDS systems is growing. The nature of MDDS systems has diversified over time. The prospects for adoption of large-scale diagnostic systems are better now than ever before, due to enthusiasm for implementation of the electronic medical record in academic, commercial, and primary care settings. Diagnostic decision support systems have become an established component of medical technology. This paper provides a review and a threaded bibliography for some of the important work on MDDS systems over the years from 1954 to 1993.

DecisionNET: database/network support for clinical decision analysts

DecisionNET is a proposed remote database/wide-area computer network that should cultivate quantitative approaches to medicine by providing rapid access to medical facts, literature citations, decision models, and commentary of specific use to those involved with quantitative clinical decision making. Increasing accessibility to data and providing a forum for exchange of ideas should prove to be a significant asset to clinicians, either as modelers or as clients. Further work will address specific database implementation, development of a query engine, and coordination of the communications infrastructure. DecisionNET provides a unique model for the growth and maintenance of other medical knowledge bases.

Analysing clinical decision analyses

We present a critical review of aspects of clinical decision analysis which uses an application to screening for familial intracranial aneurysms. The analysis is reported together with methods for assessing decision trees. These methods appear to be powerful checks on the usually rather intuitive way in which decision trees are built. The problem of assessing the uncertainty in the results of a decision analysis is discussed in detail. In practice, sensitivity analysis covers nearly every calculation apart from the standard evaluation of the decision tree. Different forms of sensitivity analysis are distinguished and given appropriate names: influence analysis, threshold analysis, full Bayesian analysis, Bayesian influence analysis, attribute analysis, generalization analysis and scenario analysis. The biostatistical community may well contribute to the much needed methodological improvement in decision analysis and its different forms of sensitivity analysis, especially if prepared to look beyond the standard statistical techniques.