Data-Driven Anomaly Detection in Laboratory Medicine: Past, Present, and Future

BACKGROUND

Anomaly detection is an integral component of operating a clinical laboratory. It covers both the recognition of laboratory errors and the rapid reporting of clinically impactful results. Procedures for identifying laboratory errors and highlighting critical results can be improved by applying modern data-driven approaches.

CONTENT

This review will prepare the reader to appraise anomaly detection literature, identify common sources of anomalous results in the clinical laboratory, and offer potential solutions for common shortcomings in current laboratory practices.

SUMMARY

Laboratories should implement data-driven approaches to detect technical anomalies and keep them from entering the medical record, while also using the full array of clinical metadata available in the laboratory information system for context-dependent, patient-centered result interpretations.

A model to establish autoverification in the clinical laboratory

OBJECTIVES

Autoverification is the process of evaluating and validating laboratory results using predefined computer-based algorithms without human interaction. By using autoverification, all reports are validated according to the standard evaluation criteria with predefined rules, and the number of reports per laboratory specialist is reduced. However, creating and validating these rules are the most demanding steps for setting up an autoverification system. In this study, we aimed to develop a model for helping users establish autoverification rules and evaluate their validity and performance.

DESIGN & METHODS

The proposed model was established by analyzing white papers, previous study results, and national/international guidelines. An autoverification software (myODS) was developed to create rules according to the model and to evaluate the rules and autoverification rates. The simulation results that were produced by the software were used to demonstrate that the determined framework works as expected. Both autoverification rates and step-based evaluations were performed using actual patient results. Two algorithms defined according to delta check usage (Algorithm A and B) and three review limits were used for the evaluation.

RESULTS

Six hundred seventeen rules were created according to the proposed model. 1,976 simulation results were created for validation. Our results showed that manual review limits are the most critical step in determining the autoverification rate, and delta check evaluation is especially important for evaluating inpatients. Algorithm B, which includes consecutive delta check evaluation, had higher AV rates.

CONCLUSIONS

Systemic rule formation is a critical factor for successful AV. Our proposed model can help laboratories establish and evaluate autoverification systems. Rules created according to this model could be used as a starting point for different test groups.

Evaluating forensic biology results given source level propositions

The evaluation of forensic evidence can occur at any level within the hierarchy of propositions depending on the question being asked and the amount and type of information that is taken into account within the evaluation. Commonly DNA evidence is reported given propositions that deal with the sub-source level in the hierarchy, which deals only with the possibility that a nominated individual is a source of DNA in a trace (or contributor to the DNA in the case of a mixed DNA trace). We explore the use of information obtained from examinations, presumptive and discriminating tests for body fluids, DNA concentrations and some case circumstances within a Bayesian network in order to provide assistance to the Courts that have to consider propositions at source level. We use a scenario in which the presence of blood is of interest as an exemplar and consider how DNA profiling results and the potential for laboratory error can be taken into account. We finish with examples of how the results of these reports could be presented in court using either numerical values or verbal descriptions of the results.

Bayesian network models for error detection in radiotherapy plans

The purpose of this study is to design and develop a probabilistic network for detecting errors in radiotherapy plans for use at the time of initial plan verification. Our group has initiated a multi-pronged approach to reduce these errors. We report on our development of Bayesian models of radiotherapy plans. Bayesian networks consist of joint probability distributions that define the probability of one event, given some set of other known information. Using the networks, we find the probability of obtaining certain radiotherapy parameters, given a set of initial clinical information. A low probability in a propagated network then corresponds to potential errors to be flagged for investigation. To build our networks we first interviewed medical physicists and other domain experts to identify the relevant radiotherapy concepts and their associated interdependencies and to construct a network topology. Next, to populate the network's conditional probability tables, we used the Hugin Expert software to learn parameter distributions from a subset of de-identified data derived from a radiation oncology based clinical information database system. These data represent 4990 unique prescription cases over a 5 year period. Under test case scenarios with approximately 1.5% introduced error rates, network performance produced areas under the ROC curve of 0.88, 0.98, and 0.89 for the lung, brain and female breast cancer error detection networks, respectively. Comparison of the brain network to human experts performance (AUC of 0.90 ± 0.01) shows the Bayes network model performs better than domain experts under the same test conditions. Our results demonstrate the feasibility and effectiveness of comprehensive probabilistic models as part of decision support systems for improved detection of errors in initial radiotherapy plan verification procedures.

The ideal laboratory information system

CONTEXT

Laboratory information systems (LIS) are critical components of the operation of clinical laboratories. However, the functionalities of LIS have lagged significantly behind the capacities of current hardware and software technologies, while the complexity of the information produced by clinical laboratories has been increasing over time and will soon undergo rapid expansion with the use of new, high-throughput and high-dimensionality laboratory tests. In the broadest sense, LIS are essential to manage the flow of information between health care providers, patients, and laboratories and should be designed to optimize not only laboratory operations but also personalized clinical care.

OBJECTIVES

To list suggestions for designing LIS with the goal of optimizing the operation of clinical laboratories while improving clinical care by intelligent management of laboratory information.

DATA SOURCES

Literature review, interviews with laboratory users, and personal experience and opinion.

CONCLUSIONS

Laboratory information systems can improve laboratory operations and improve patient care. Specific suggestions for improving the function of LIS are listed under the following sections: (1) Information Security, (2) Test Ordering, (3) Specimen Collection, Accessioning, and Processing, (4) Analytic Phase, (5) Result Entry and Validation, (6) Result Reporting, (7) Notification Management, (8) Data Mining and Cross-sectional Reports, (9) Method Validation, (10) Quality Management, (11) Administrative and Financial Issues, and (12) Other Operational Issues.

Detection of preanalytic laboratory testing errors using a statistically guided protocol

Preanalytic laboratory testing errors are often difficult to identify. We demonstrate how laboratories can integrate statistical models with clinical judgment to develop protocols for preanalytic error detection. Specifically, we developed a protocol to identify spuriously elevated glucose values resulting from improper "line draws" or related phlebotomy errors. Using a decision tree-generating algorithm and an annotated set of training data, we generated decision trees to classify critically elevated glucose results as "real" or "spurious" based on available laboratory parameters. Decision trees revealed that a 30-day patient-specific average glucose concentration lower than 186.3 mg/dL (10.3 mmol/L), a current glucose concentration higher than 663 mg/dL (37 mmol/L), and an anion gap lower than 16.5 mEq/L (16.5 mmol/L) suggested a spurious result. We then used the results from the decision tree analysis to inform the implementation of a clinical protocol that significantly improved the laboratory's identification of spurious results. Similar approaches may be useful in developing protocols to identify other errors or to assist in clinical interpretation of results.