A highly accurate delta check method using deep learning for detection of sample mix-up in the clinical laboratory

Machine learning-based delta check method for detecting misidentification errors in tumor marker tests

OBJECTIVES

Misidentification errors in tumor marker tests can lead to serious diagnostic and treatment errors. This study aims to develop a method for detecting these errors using a machine learning (ML)-based delta check approach, overcoming limitations of conventional methods.

METHODS

We analyzed five tumor marker test results: alpha-fetoprotein (AFP), cancer antigen 19-9 (CA19-9), cancer antigen 125 (CA125), carcinoembryonic antigen (CEA), and prostate-specific antigen (PSA). A total of 246,261 records were used in the analysis. Of these, 179,929 records were used for model training and 66,332 records for performance evaluation. We developed a misidentification error detection model based on the random forest (RF) and deep neural network (DNN) methods. We performed an in silico simulation with 1 % random sample shuffling. The performance of the developed models was evaluated and compared to conventional delta check methods such as delta percent change (DPC), absolute DPC (absDPC), and reference change values (RCV).

RESULTS

The DNN model outperformed the RF, DPC, absDPC, and RCV methods in detecting sample misidentification errors. It achieved balanced accuracies of 0.828, 0.842, 0.792, 0.818, and 0.833 for AFP, CA19-9, CA125, CEA, and PSA, respectively. Although the RF method performed better than DPC and absDPC, it showed similar or lower performance compared to RCV.

CONCLUSIONS

Our research results demonstrate that an ML-based delta check method can more effectively detect sample misidentification errors compared to conventional delta check methods. In particular, the DNN model demonstrated superior and stable detection performance compared to the RF, DPC, absDPC, and RCV methods.

Artificial intelligence in the clinical laboratory

Laboratory medicine has become a highly automated medical discipline. Nowadays, artificial intelligence (AI) applied to laboratory medicine is also gaining more and more attention, which can optimize the entire laboratory workflow and even revolutionize laboratory medicine in the future. However, only a few commercially available AI models are currently approved for use in clinical laboratories and have drawbacks such as high cost, lack of accuracy, and the need for manual review of model results. Furthermore, there are a limited number of literature reviews that comprehensively address the research status, challenges, and future opportunities of AI applications in laboratory medicine. Our article begins with a brief introduction to AI and some of its subsets, then reviews some AI models that are currently being used in clinical laboratories or that have been described in emerging studies, and explains the existing challenges associated with their application and possible solutions, finally provides insights into the future opportunities of the field. We highlight the current status of implementation and potential applications of AI models in different stages of the clinical testing process.

Machine learning-based clinical decision support using laboratory data

Artificial intelligence (AI) and machine learning (ML) are becoming vital in laboratory medicine and the broader context of healthcare. In this review article, we summarized the development of ML models and how they contribute to clinical laboratory workflow and improve patient outcomes. The process of ML model development involves data collection, data cleansing, feature engineering, model development, and optimization. These models, once finalized, are subjected to thorough performance assessments and validations. Recently, due to the complexity inherent in model development, automated ML tools were also introduced to streamline the process, enabling non-experts to create models. Clinical Decision Support Systems (CDSS) use ML techniques on large datasets to aid healthcare professionals in test result interpretation. They are revolutionizing laboratory medicine, enabling labs to work more efficiently with less human supervision across pre-analytical, analytical, and post-analytical phases. Despite contributions of the ML tools at all analytical phases, their integration presents challenges like potential model uncertainties, black-box algorithms, and deskilling of professionals. Additionally, acquiring diverse datasets is hard, and models' complexity can limit clinical use. In conclusion, ML-based CDSS in healthcare can greatly enhance clinical decision-making. However, successful adoption demands collaboration among professionals and stakeholders, utilizing hybrid intelligence, external validation, and performance assessments.

Practical delta check limits for tumour markers in different clinical settings

OBJECTIVES

Few studies have reported on delta checks for tumour markers, even though these markers are often evaluated serially. Therefore, this study aimed to establish a practical delta check limit in different clinical settings for five tumour markers: alpha-fetoprotein, cancer antigen 19-9, cancer antigen 125, carcinoembryonic antigen, and prostate-specific antigen.

METHODS

Pairs of patients' results (current and previous) for five tumour markers between 2020 and 2021 were retrospectively collected from three university hospitals. The data were classified into three subgroups, namely: health check-up recipient (subgroup H), outpatient (subgroup O), and inpatient (subgroup I) clinics. The check limits of delta percent change (DPC), absolute DPC (absDPC), and reference change value (RCV) for each test were determined using the development set (the first 18 months, n=179,929) and then validated and simulated by applying the validation set (the last 6 months, n=66,332).

RESULTS

The check limits of DPC and absDPC for most tests varied significantly among the subgroups. Likewise, the proportions of samples requiring further evaluation, calculated by excluding samples with both current and previous results within the reference intervals, were 0.2-2.9% (lower limit of DPC), 0.2-2.7% (upper limit of DPC), 0.3-5.6% (absDPC), and 0.8-35.3% (RCV99.9%). Furthermore, high negative predictive values >0.99 were observed in all subgroups in the in silico simulation.

CONCLUSIONS

Using real-world data, we found that DPC was the most appropriate delta-check method for tumour markers. Moreover, Delta-check limits for tumour markers should be applied based on clinical settings.

Flowing through laboratory clinical data: the role of artificial intelligence and big data

During the last few years, clinical laboratories have faced a sea change, from facilities producing a high volume of low-cost test results, toward a more integrated and patient-centered service. Parallel to this paradigm change, the digitalization of healthcare data has made an enormous quantity of patients' data easily accessible, thus opening new scenarios for the utilization of artificial intelligence (AI) tools. Every day, clinical laboratories produce a huge amount of information, of which patients' results are only a part. The laboratory information system (LIS) may include other "relevant" compounding data, such as internal quality control or external quality assessment (EQA) results, as well as, for example, timing of test requests and of blood collection and exams transmission, these data having peculiar characteristics typical of big data, as volume, velocity, variety, and veracity, potentially being used to generate value in patients' care. Despite the increasing interest expressed in AI and big data in laboratory medicine, these topics are approaching the discipline slowly for several reasons, attributable to lack of knowledge and skills but also to poor or absent standardization, harmonization and problematic regulatory and ethical issues. Finally, it is important to bear in mind that the mathematical postulation of algorithms is not sufficient for obtaining useful clinical tools, especially when biological parameters are not evaluated in the appropriate context. It is therefore necessary to enhance cooperation between laboratory and AI experts, and to coordinate and govern processes, thus favoring the development of valuable clinical tools.