Pattern-Based Interactive Diagnosis of Multiple Disorders: The MEDAS System

A patient-similarity-based model for diagnostic prediction

OBJECTIVE

To simulate the clinical reasoning of doctors, retrieve analogous patients of an index patient automatically and predict diagnoses by the similar/dissimilar patients.

METHODS

We proposed a novel patient-similarity-based framework for diagnostic prediction, which is inspired by the structure-mapping theory about analogy reasoning in psychology. Patient similarity is defined as the similarity between two patients' diagnoses sets rather than a dichotomous (absence/presence of just one disease). The multilabel classification problem is converted to a single-value regression problem by integrating the pairwise patients' clinical features into a vector and taking the vector as the input and the patient similarity as the output. In contrast to the common k-NN method which only considering the nearest neighbors, we not only utilize similar patients (positive analogy) to generate diagnostic hypotheses, but also utilize dissimilar patients (negative analogy) are used to reject diagnostic hypotheses.

RESULTS

The patient-similarity-based models perform better than the one-vs-all baseline and traditional k-NN methods. The f-1 score of positive-analogy-based prediction is 0.698, significantly higher than the scores of baselines ranging from 0.368 to 0.661. It increases to 0.703 when the negative analogy method is applied to modify the prediction results of positive analogy. The performance of this method is highly promising for larger datasets.

CONCLUSION

The patient-similarity-based model provides diagnostic decision support that is more accurate, generalizable, and interpretable than those of previous methods and is based on heterogeneous and incomplete data. The model also serves as a new application for the use of clinical big data through artificial intelligence technology.

Approximation of Unknown Multivariate Probability Distributions by Using Mixtures of Product Components: A Tutorial

In literature the references to EM estimation of product mixtures are not very frequent. The simplifying assumption of product components, e.g. diagonal covariance matrices in case of Gaussian mixtures, is usually considered only as a compromise because of some computational constraints or limited dataset. We have found that the product mixtures are rarely used intentionally as a preferable approximating tool. Probably, most practitioners do not “trust” the product components because of their formal similarity to “naive Bayes models.” Another reason could be an unrecognized numerical instability of EM algorithm in multidimensional spaces. In this paper we recall that the product mixture model does not imply the assumption of independence of variables. It is even not restrictive if the number of components is large enough. In addition, the product components increase numerical stability of the standard EM algorithm, simplify the EM iterations and have some other important advantages. We discuss and explain the implementation details of EM algorithm and summarize our experience in estimating product mixtures. Finally we illustrate the wide applicability of product mixtures in pattern recognition and in other fields.

A knowledge-based clinical toxicology consultant for diagnosing multiple exposures

OBJECTIVE

This paper presents continued research toward the development of a knowledge-based system for the diagnosis of human toxic exposures. In particular, this research focuses on the challenging task of diagnosing exposures to multiple toxins. Although only 10% of toxic exposures in the United States involve multiple toxins, multiple exposures account for more than half of all toxin-related fatalities. Using simple medical mathematics, we seek to produce a practical decision support system capable of supplying useful information to aid in the diagnosis of complex cases involving multiple unknown substances.

METHODS

The system is automatically trained using data mining techniques to extract prior probabilities and likelihood ratios from a database managed by the Florida Poison Information Center (FPIC). When supplied with observed clinical effects, the system produces a ranked list of the most plausible toxic exposures. During testing, the system diagnosed toxins at three levels: identifying the substance, identifying the toxin's major and minor categories, and identifying the toxin's major category alone. To enable comparison between these three levels, accuracy was calculated as the percentage of exposures correctly identified in top 10% of trained diagnoses.

RESULTS

System evaluation utilized a dataset of 8901 multiple exposure cases and 37,617 single exposure cases. Initial system testing using only multiple exposure cases yielded poor results, with diagnosis accuracies ranging from 18.5% to 50.1%. Further investigation revealed that the system's inability to diagnose multiple disorders resulted from insufficient data and that the clinical effects observed in multiple exposures are dominated by a single substance. Including single exposures when training, the system achieved accuracies as high as 83.5% when diagnosing the primary contributors in multiple exposure cases by substance, 86.9% when diagnosing by major and minor categories, and 79.9% when diagnosing by major category alone.

CONCLUSIONS

Although the system failed to completely diagnose exposures to multiple toxins, the ability to identify the primary contributor in such cases may prove valuable in aiding medical personnel as they seek to diagnose and treat patients. As time passes and more cases are added to the FPIC database, we believe system accuracy will continue to improve, producing a viable decision support system for clinical toxicology.

Individualizing generic decision models using assessments as evidence

Complex decision models in expert systems often depend upon a number of utilities and subjective probabilities for an individual. Although these values can be estimated for entire populations or demographic subgroups, a model should be customized to the individual's specific parameter values. This process can be onerous and inefficient for practical decisions. We propose an interactive approach for incrementally improving our knowledge about a specific individual's parameter values, including utilities and probabilities, given a decision model and a prior joint probability distribution over the parameter values. We define the concept of value of elicitation and use it to determine dynamically the next most informative elicitation for a given individual. We evaluated the approach using an example model and demonstrate that we can improve the decision quality by focusing on those parameter values most material to the decision.

Desiderata for product labeling of medical expert systems

The proliferation and increasing complexity of medical expert systems raise ethical and legal concerns about the ability of practitioners to protect their patients from defective or misused software. Appropriate product labeling of expert systems can help clinical users to understand software indications and limitations. Mechanisms of action and knowledge representation schema should be explained in layperson's terminology. User qualifications and resources available for acquiring the skills necessary to understand and critique the system output should be listed. The processes used for building and maintaining the system's knowledge base are key determinants of the product's quality, and should be carefully documented. To meet these desiderata, a printed label is insufficient. The authors suggest a new, more active, model of product labeling for medical expert systems that involves embedding 'knowledge of the knowledge base', creating user-specific data, and sharing global information using the Internet.

PATHMASTER: modelling differential diagnosis as "dynamic competition" between systematic analysis and disease-directed deduction

PATHMASTER is an expert system under development to assist in teaching histopathologic differential diagnosis. The system incorporates two "orthogonal" representations of the knowledge required to perform differential diagnosis. One representation groups histopathologic features around the anatomic structures of liver tissue, the second representation groups the features around the diseases in which they occur. Using these two representations, PATHMASTER models the process of diagnosis as a "dynamic competition" between systematic analysis and disease-directed deduction. By varying two parameters of PATHMASTER's underlying mathematical model, the interplay between these factors can be varied.

A menu-driven knowledge base browsing tool

Conventional computer-assisted medical decision-making systems have had limited impact on routine clinical practice. This has stimulated an alternative approach to the utilization of medical knowledge bases. Centering on the storage and retrieval of medical information, it aims to provide clinicians with computerized medical reference systems. In this paper we describe the development of a prototype menu-driven browsing tool, which allows clinicians to browse through the contents of a knowledge base in a number of ways. Operations include interrogation via disease classes, names or attributes; hierarchical display of all or part of a disease profile; printing of a disease profile; construction of differential diagnosis lists and comparison of two diseases. We discuss how the use of a menu-driven interface can help to overcome some of the problems encountered with previous designs of medical reference systems.

Expert systems and the pancreatic cancer problem: decision support in the pre-operative diagnosis

In this paper, after reviewing the main issue in artificial intelligence, decision support systems, medical decision-making, expert systems and some of their applications in medicine, we focus on the diagnostic aspect of pancreatic cancer. We briefly examine the most significant applications both from the oncological and from the diagnostic point of view. We discuss the medical problems mentioning incidence and mortality, aetiological factors and diagnosis, considering the roles of surgery and adjuvant therapies. Finally we justify the decision to develop an expert system in such a medical domain and discuss the SPES (Surgical Pancreatic Expert System) project, its parts dealing with the different medical phases of pancreatic cancer diagnosis and therapy: pre-operative, intra-operative and adjuvant therapies. In particular we discuss diagnostic aspects of pancreatic cancer disease, pointing out the aims of the project, methodologies, tools used and future developments.

Revision of diagnostic logic using a clinical database

Statistical pattern-recognition techniques have been frequently applied to the problem of medical diagnosis. Sequential Bayesian approaches are appealing because of the possibility of generating the underlying sensitivities, specificities, and prevalence statistics from the estimates of medical experts. The accuracy of these estimates and the consequences of inaccuracies carry implications for the future development of this type of system. In an effort to explore these subjects, the authors used statistics derived from a clinical database to revise the diagnostic logic in a Bayesian system for generating a differential diagnostic list. Substantial changes in estimated a priori probabilities, sensitivities, and specificities were made to correct for significant under- and overestimations of these values by a group of medical experts. The system based on the derived values appears to perform better than the original system. It is concluded that the statistics used in a Bayesian diagnostic system should be derived from a database representative of the patient population for which the system is designed.

A methodology for generating computer-based explanations of decision-theoretic advice

Decision analysis is an appealing methodology with which to provide decision support to the practicing physician. However, its use in the clinical setting is impeded because computer-based explanations of decision-theoretic advice are difficult to generate without resorting to mathematical arguments. Nevertheless, human decision analysts generate useful and intuitive explanations based on decision trees. To facilitate the use of decision theory in a computer-based decision support system, the authors developed a computer program that uses symbolic reasoning techniques to generate nonquantitative explanations of the results of decision analyses. A combined approach has been implemented to explain the differences in expected utility among branches of a decision tree. First, the mathematical relationships inherent in the structure of the tree are used to find any asymmetries in tree structure or inequalities among analogous decision variables that are responsible for a difference in expected utility. Next, an explanation technique is selected and applied to the most significant variables, creating a symbolic expression that justifies the decision. Finally, the symbolic expression is converted to English-language text, thereby generating an explanation that justifies the desirability of the choice with the greater expected utility. The explanation does not refer to mathematical formulas, nor does it include probability or utility values. The results suggest that explanations produced by a combination of decision analysis and symbolic processing techniques may be more persuasive and acceptable to clinicians than those produced by either technique alone.

Intelligent dialogue based on statistical models of clinical decision-making

The independence Bayesian model has been used widely in computer programs designed to support clinical decision-making. A reasoning strategy has been developed to enable these programs to conduct clinically pertinent dialogue and explain their reasoning. It has been implemented in a program for the diagnosis of acute abdominal pain based on the Bayesian model of de Dombal et al. Several features of the dialogue design have been adopted from artificial intelligence research, including shared initiative and critiquing. The program adopts a flexible goal-driven strategy, attempting to confirm the clinician's diagnosis or rule out the likeliest alternative. Symptoms and signs are selected in order of their expected weights of evidence in favour of the hypothesized disease.

Answer justification in medical decision support systems based on Bayesian classification

Answer justification refers to the ability of a computer program to explain how or why it arrived at a particular conclusion. This paper presents a new method for automated answer justification that is suitable for use in computer-supported decision aids in medicine which are based on Bayesian classification. The factors most responsible for the relative ordering of posterior probabilities of outcomes are identified by analyzing the prior and conditional probabilities used to generate them. This approach is illustrated using a computer decision aid for stroke classification and is seen to produce understandable and clinically plausible explanations.

Derivation of rule-based knowledge from established medical outlines

The purpose of this paper is to describe the derivation of a rule base for an expert system from an existing medical auditing method called a criteria map. The criteria map represents the physician's logic in the specialty involved. The system described, EMERGE, is written in the standard Pascal programming language, operates on a microcomputer, and provides a convenient user-interface. All medical knowledge in EMERGE is contained in an independent set of production rules, which are derived from the criteria map.

EMERGE-A Data-Driven Medical Decision Making Aid

EMERGE is an expert system designed as a medical decision making aid. It is machine-independent, and is implemented in standard Pascal. It has modest memory requirements, and can operate on a microcomputer. EMERGE is rule-based, and its initial application is the analysis of chest pain in the emergency room. The knowledge base is maintained separately from the consultation program. Thus the application area can be changed without any modification to the software. This paper describes the control structures and rule searching procedures used in EMERGE.

Evaluating Multimembership Classifiers: A Methodology and Application to the MEDAS Diagnostic System

Performance evaluation measures for multimembership classifiers are presented and applied in a retrospective study on the diagnostic performance of the MEDAS (Medical Emergency Decision Assistance System) system. Admission and discharge diagnoses for 122 patients with one or more of 26 distinct disorders in five major disorder categories were gathered. The average number of disorders per patient was 2 with 36 (29.5 percent) patients having 3 or more disorders simultaneously. The features (symptoms, signs, and laboratory data) available at admission were entered into a multimembership Bayesian pattern recognition algorithm which permits for diagnosis of multiple disorders. When the top five computer-ranked diagnoses were considered, all of the correct diagnoses for 86.1 percent of the patients were displayed by the fifth position. In 71.6 percent of these cases, no false diagnosis preceded any correct diagnosis. In ten cases a discharge diagnosis which was suggested by the available findings was omitted by the admitting physician. In six of these ten cases, the overlooked diagnoses appeared at the computer ranked list above all false diagnoses. Considering the urgency of diagnosis in the Emergency Department, the high uncertainty involved due to the limited availability of data, and the high frequency with which multiple disorders coexist, this limited study encourages our confidence in the MEDAS knowledge base and algorithm as a useful diagnostic support tool.

Sensitivity analysis in bayesian classification models: multiplicative deviations

The sensitivity of Bayesian pattern recognition models to multiplicative deviations in the prior and conditional probabilities is investigated for the two-class case. Explicit formulas are obtained for the factor K by which the computed posterior probabilities should be divided in order to eliminate the deviation effect. Numerical results for the case of binary features indicate that the Bayesian model tolerates large deviations in the prior and conditional probabilities. In fact, the a priori ratio and the likelihood ratio may deviate within a range of 65-135 percent and still produce posterior probabilities in accurate proximity of at most ±0.10. The main implication is that Bayesian systems which are based on limited data or subjective probabilities are expected to have a high percentage of correct classification despite the fact that the prior and conditional probabilities they use may deviate rather significantly from the true values.