Delta Checks in the clinical laboratory

Machine Learning-Based Sample Misidentification Error Detection in Clinical Laboratory Tests: A Retrospective Multicenter Study

BACKGROUND

In clinical laboratories, the precision and sensitivity of autoverification technologies are crucial for ensuring reliable diagnostics. Conventional methods have limited sensitivity and applicability, making error detection challenging and reducing laboratory efficiency. This study introduces a machine learning (ML)-based autoverification technology to enhance tumor marker test error detection.

METHODS

The effectiveness of various ML models was evaluated by analyzing a large data set of 397 751 for model training and internal validation and 215 339 for external validation. Sample misidentification was simulated by random shuffling error-free test results with a 1% error rate to achieve a real-world approximation. The ML models were developed with Bayesian optimization for tuning. Model validation was performed internally at the primary institution and externally at other institutions, comparing the ML models' performance with conventional delta check methods.

RESULTS

Deep neural networks and extreme gradient boosting achieved an area under the receiver operating characteristic curve of 0.834 to 0.903, outperforming that of conventional methods (0.705 to 0.816). External validation by 3 independent laboratories showed that the balanced accuracy of the ML model ranged from 0.760 to 0.836, outperforming the balanced accuracy of 0.670 to 0.773 of the conventional models.

CONCLUSIONS

This study addresses limitations regarding the sensitivity of current delta check methods for detection of sample misidentification errors and provides versatile models that mitigate the operational challenges faced by smaller laboratories. Our findings offer a pathway toward more efficient and reliable clinical laboratory testing.

How many mislabelled samples go unidentified? Results of a pilot study to determine the occult mislabelled sample rate

BACKGROUND

Specimens with incorrect patient information are both a critical safety error and difficult to identify. Estimates of sample mislabelling rely on subjective identification of mislabelling, with the possibility that not all mislabelled samples are being caught.

METHODS

We determined the blood type of two or more complete blood count specimens with the same patient label and assessed for discrepancies. We additionally determined the rate of identified sample mislabelling for the study period.

RESULTS

We found a rate of 3.17 per 1000 discrepancies over the study period. These discrepancies most likely represent occult, or unidentified, mislabelled samples. In contrast, the rate of identified sample mislabelling was 1.15 per 1000.

CONCLUSIONS

This study suggests that specimens identified as, or known to be, mislabelled represent only a fraction of those mislabelled. These findings are currently being confirmed in our laboratory and are likely generalisable to other institutions.

Delta check limits for thyroid function tests adjusted for clinical settings

BACKGROUND

This study aimed to determine practical delta check limits (DCLs) for thyroid function tests (TFTs) to detect sample misidentifications across various clinical settings.

METHODS

Between 2020 and 2022, 610,437 paired TFT results were collected from six university hospitals. The absolute DCL (absDCL) was determined using the 95th percentile for each clinical setting from a random 60 % of the total data. These absDCLs were then tested within and across different settings using the remaining 40 % of the data, alongside mix-up datasets for result and sample comparisons. The sensitivities of absDCL were calculated within and across groups in the mix-up datasets.

RESULTS

Health screening absDCLs were notably lower than in other settings (2.58 vs. 5.93-7.08 for thyroid-stimulating hormone; 4.12 vs. 8.24-10.04 for free thyroxine; 0.49 vs. 0.82-0.91 for total triiodothyronine). The proportion of results exceeding absDCL of health screening differed from those of other clinical settings. Furthermore, sensitivity between health screening and other clinical settings was significantly different in both the result mix-up and sample mix-up datasets.

CONCLUSIONS

This study determined practical DCLs for TFTs and highlighted differences in absDCLs between health screening and other settings. These findings emphasize the importance of tailored DCLs in improving the accurate reporting of TFTs.

Critical test result management at Danish hospital laboratories: a national survey

Critical test results in clinical laboratories are crucial for timely patient care, serving as indicators of potentially life-threatening conditions. Despite their importance, a notable heterogeneity in management practices exists globally. This study investigates the current practices of managing critical results at Danish clinical biochemistry laboratories and identifies areas prone for improvement. A comprehensive online survey was distributed to all 21 Danish clinical biochemistry laboratories regarding their critical result management, including documentation practices, critical limit selection, and quality assurance measures. A total of 17 laboratories (81%) responded. The answers revealed a generally uniform approach to managing critical results, with all laboratories having 24-h reporting, local instructions and using the telephone as communication channel. However, variations were noted in documentation practices and critical limit selection. Notably, 23.5% of the laboratories reported that one out of every ten critical results was not reported, indicating a significant risk of delayed critical results. This is further complicated by the limited use of predefined timeframes for reporting and also, only few laboratories actively monitored response times. The findings emphasize the need for more standardized documentation and evaluation practices to align with international standards and to enhance patient safety. While the laboratories showed a commitment to standardized procedures, the study emphasizes the necessity of a National or Nordic guideline to supplement the ISO 15189:2022. This study is a step towards optimizing critical result management, not only in Danish clinical biochemistry laboratories but also across various laboratory specialties, thereby improving overall laboratory quality, efficiency, and patient safety.

Pre-analytical errors in coagulation testing: a case series

OBJECTIVES

Prevention of pre-analytical issues in coagulation testing is of paramount importance for good laboratory performance. In addition to common issues like hemolysed, icteric, or lipemic samples, some specific pre-analytical errors of coagulation testing include clotted specimens, improper blood-to-anticoagulant ratio, contamination with other anticoagulants, etc. Prothrombin time (PT) and activated partial thromboplastin time (aPTT) are very commonly affected tests due to pre-analytical variables. The impact these parameters possess on surgical decision-making and various life-saving interventions are substantial therefore we cannot afford laxity and casual mistakes in carrying out these critical investigations at all.

CASE PRESENTATION

In this case series, a total of 4 cases of unexpectedly deranged coagulation profiles have been described which were reported incorrectly due to the overall casual approach towards these critical investigations. We have also mentioned how the treating clinician and lab physician retrospectively accessed relevant information in the nick of time to bring back reassurance.

CONCLUSIONS

Like every other critical investigation, analytical errors can occur in coagulation parameters due to various avoidable pre-analytical variables. The release of spurious results for coagulation parameters sets alarm bells ringing causing much agony to the treating doctor and patient. Only a disciplined and careful approach taken by hospital and lab staff towards each sample regardless of its criticality can negate these stressful errors to a large extent.

Design of an algorithm for the detection of intravenous fluid contamination in clinical laboratory samples

OBJECTIVES

Contamination of blood samples from patients receiving intravenous fluids is a common error with potential risk to the patient. Algorithms based on the presence of aberrant results have been described but have the limitation that not all infusion fluids have the same composition. Our objective is to develop an algorithm based on the detection of the dilution observed on the analytes not usually included in infusion fluids.

METHODS

A group of 89 cases was selected from samples flagged as contaminated. Contamination was confirmed by reviewing the clinical history and comparing the results with previous and subsequent samples. A control group with similar characteristics was selected. Eleven common biochemical parameters not usually included in infusion fluids and with low intraindividual variability were selected. The dilution in relation to the immediate previous results was calculated for each analyte and a global indicator, defined as the percentage of analytes with significant dilution, was calculated. ROC curves were used to define the cut-off points.

RESULTS

A cut-off point of 20 % of dilutional effect requiring also a 60 % dilutional ratio achieved a high specificity (95 % CI 91-98 %) with an adequate sensitivity (64 % CI 54-74 %). The Area Under Curve obtained was 0.867 (95 % CI 0.819-0.915).

CONCLUSIONS

Our algorithm based on the global dilutional effect presents a similar sensitivity but greater specificity than the systems based on alarming results. The implementation of this algorithm in the laboratory information systems may facilitate the automated detection of contaminated samples.

Practical delta check limits for tumour markers in different clinical settings

OBJECTIVES

Few studies have reported on delta checks for tumour markers, even though these markers are often evaluated serially. Therefore, this study aimed to establish a practical delta check limit in different clinical settings for five tumour markers: alpha-fetoprotein, cancer antigen 19-9, cancer antigen 125, carcinoembryonic antigen, and prostate-specific antigen.

METHODS

Pairs of patients' results (current and previous) for five tumour markers between 2020 and 2021 were retrospectively collected from three university hospitals. The data were classified into three subgroups, namely: health check-up recipient (subgroup H), outpatient (subgroup O), and inpatient (subgroup I) clinics. The check limits of delta percent change (DPC), absolute DPC (absDPC), and reference change value (RCV) for each test were determined using the development set (the first 18 months, n=179,929) and then validated and simulated by applying the validation set (the last 6 months, n=66,332).

RESULTS

The check limits of DPC and absDPC for most tests varied significantly among the subgroups. Likewise, the proportions of samples requiring further evaluation, calculated by excluding samples with both current and previous results within the reference intervals, were 0.2-2.9% (lower limit of DPC), 0.2-2.7% (upper limit of DPC), 0.3-5.6% (absDPC), and 0.8-35.3% (RCV99.9%). Furthermore, high negative predictive values >0.99 were observed in all subgroups in the in silico simulation.

CONCLUSIONS

Using real-world data, we found that DPC was the most appropriate delta-check method for tumour markers. Moreover, Delta-check limits for tumour markers should be applied based on clinical settings.

A Machine Learning Model for the Routine Detection of "Wrong Blood in Complete Blood Count Tube" Errors

BACKGROUND

Current laboratory procedures may fail to detect wrong blood in tube (WBIT) errors. Machine learning models have the potential to improve WBIT error detection, as demonstrated by proof-of-concept studies. The models developed so far, however, are not appropriate for routine use because they are unable to handle missing values and have low positive predictive value (PPV). In this study, a machine learning model suitable for routine use was developed.

METHODS

A model was trained and a preliminary evaluation performed on a retrospective data set of 135 128 current and previous patient complete blood count (CBC) results. The model was then applied prospectively to routine samples tested in a public hospital laboratory over a period of 22 weeks. Each week, the 5 samples identified by the model as most likely to be WBIT errors underwent further investigation by testing blood group and red cell phenotype. The study assessed the number of WBIT errors that were missed by current procedures but detected by the model, as well as the PPV of the model.

RESULTS

The model was applied prospectively to 38 187 CBC results that had passed routine laboratory checks. One hundred and ten samples were identified for further testing and 12 WBIT errors were detected. The PPV of the model was 10.9%.

CONCLUSION

A machine learning model suitable for routine use was able to identify WBIT errors missed by the laboratory's current procedures. Machine learning models are valuable for the identification of WBIT errors, and their validation and deployment in clinical laboratories would improve patient safety.

Delta checks

Delta check is an electronic error detection tool. It compares the difference in sequential results within a patient against a predefined limit, and when exceeded, the delta check rule is considered triggered. The patient results should be withheld for review and troubleshooting before releasing to the clinical team for patient management. Delta check was initially developed as a tool to detect wrong-blood-in-tube (sample misidentification) errors. It is now applied to detect errors more broadly within the total testing process. Recent advancements in the theoretical understanding of delta check has allowed for more precise application of this tool to achieve the desired clinical performance and operational set up. In this Chapter, we review the different pre-implementation considerations, the foundation concepts of delta check, the process of setting up key delta check parameters, performance verification and troubleshooting of a delta check flag.

Technical quality assurance and quality control for medical laboratories: a review and proposal of a new concept to obtain integrated and validated QA/QC plans

Technical quality assurance (QA) and quality control (QA/QC) are important activities within medical laboratories to ensure the adequate quality of obtained test results. QA/QC tools available at medical laboratories include external QC and internal QC, patient-based real-time quality control (PBRTQC) tools such as moving average quality control (MAQC), limit checks, delta checks, and multivariate checks, and finally, analyzer flagging. Recently, for PBRTQC tools, new optimization and validation methods based on error detection simulation have been developed to obtain laboratory-specific insights into PBRTQC error detection. These developments have enabled implementation and application of these individual tools in routine clinical practice. As a next step, they also enable performance comparison of the individual QA/QC tools and integration of all the individual QA/QC tools in order to obtain the most powerful and efficient QA/QC plans. In this review, a brief overview of the individual QA/QC tools and their characteristics is provided and the error detection simulation approaches are explained. Finally, a new concept entitled integrated quality assurance and control (IQAC) is presented. To enable IQAC, a conceptual framework is suggested and demonstrated for sodium, based on available published data. The proposed IQAC framework provides ways and tools by which the performance of different QA/QC tools can be compared in a so-called QA/QC error detection table to enable optimization and validation of the overall QA/QC plan in terms of alarm rate as well as pre-analytical, analytical, and post-analytical error detection performance.

Designing and validating an autoverification system of biochemical test results in Hatay Mustafa Kemal University, clinical laboratory

Introduction

Autoverification (AV) is a postanalytical tool that uses algorithms to validate test results according to specified criteria. The Clinical and Laboratory Standard Institute (CLSI) document for AV of clinical laboratory test result (AUTO-10A) includes recommendations for laboratories needing guidance on implementation of AV algorithms. The aim was to design and validate the AV algorithm for biochemical tests.

Materials and methods

Criteria were defined according to AUTO-10A. Three different approaches for algorithm were used as result limit checks, which are reference range, reference range ± total allowable error, and 2nd and 98th percentile values. To validate the algorithm, 720 cases in middleware were tested. For actual cases, 3,188,095 results and 194,520 reports in laboratory information system (LIS) were evaluated using the AV system. Cohen’s kappa (κ) was calculated to determine the degree of agreement between seven independent reviewers and the AV system.

Results

The AV passing rate was found between 77% and 85%. The highest rates of AV were in alanine transaminase (ALT), direct bilirubin (DBIL), and magnesium (Mg), which all had AV rates exceeding 85%. The most common reason for non-validated results was the result limit check (41%). A total of 328 reports evaluated by reviewers were compared to AV system. The statistical analysis resulted in a κ value between 0.39 and 0.63 (P < 0.001) and an agreement rate between 79% and 88%.

Conclusions

Our improved model can help laboratories design, build, and validate AV systems and be used as starting point for different test groups.

A study on quality control using delta data with machine learning technique

Background

In the big data era, patient-based real-time quality control (PBRTQC), as an emerging quality control (QC) method, is expanding within the clinical laboratory industry. However, the main issue of current PBRTQC methodology is data stability. Our study is aimed to explore a novel protocol for data stability by combining delta data with machine learning (ML) technique to improve the capacity of QC event detection.

Methods

A data set of 423,290 laboratory results from Beijing Chao-yang Hospital 2019 patient results were used as a training set (n = 380960, 90%) and internal validation set (n = 42330, 10%). A further 22,460 results from Beijing Long-fu Hospital 2019 patient results were used as a test set. Three-type data (1) Single-type data processed by truncation limits; (2) delta-type data processed by truncation limits and (3)delta-type data processed by Isolated Forest (IF) algorithm were evaluated with accuracy, sensitivity, NPed, etc., and compared with previously published statistical methods.

Results

The optimal model was based on Random Forest (RF) algorithm by using delta-type data processed by IF algorithm. The model had a better accuracy (0.99), sensitivity (0.99) specificity (0.99) and AUC (0.99) with the dependent test set, surpassing the critical bias of PBRTQC by over 50%. For the LYMPH#, HGB, and PLT, the cumulative MNPed of MLQC were reduced by 95.43%, 97.39%, and 97.97% respectively when compared to the best of the PBRTQC.

Conclusion

Final results indicate that by integrating an innovative ML algorithm with the overall data processing protocol the detection of QC events is improved.

•

A protocol for data processing by using delta data together with machine learning algorithm, enables to improve data stability.

•

After data processing, the performance of QC event prediction surpassed over 50% clinical recognized PBRTQC method, especially for the hard-to-detect error in QC event prediction.

Biologically Augmented Suture for Ligament Bracing Procedures Positively Affects Human Ligamentocytes and Osteoblasts In Vitro

PURPOSE

The purpose was to evaluate the response of human ligamentocytes and osteoblasts after biological augmentation with thrombin, concentrated bone marrow aspirate (cBMA), or platelet-rich plasma (PRP) on two different types of nonresorbable flat braided suture used for ligament bracing.

METHODS

Uncoated (U) and collagen-coated (C) flat braided suture material was augmented with either thrombin (T), cBMA (B), PRP (P), or a combination of these three (A), while platelet-poor plasma was used as a source for fibrin (F) in each assay. Previously cultured ligamentocytes and osteoblasts were added with a defined density and assayed after the required time period for adhesion, proliferation, and alkaline phosphatase activity.

RESULTS

Biological augmentation of uncoated [(UFT, UFBT, UFA; P < .001), (UFPT; P = .017)] and collagen-coated suture (CFT, CFPT, CFBT, CFA; P < .001) led to a significantly higher ligamentocyte adhesion. Significantly higher adhesion was also observed for osteoblasts (UFT, UFPT, UFBT, UFA; P < .001; CFT, CFPT, CFBT, CFA; P < .001). Similarly, ligamentocyte proliferation was significantly higher [(UFT, UFPT, UFA; P = .009), (UFBT; P = .001), (CFT; P = .009), (CFBT; P = .001), and (CFA; P = .01)]. Osteoblasts showed significantly higher proliferation as well [(UFT, UFPT, UFA; P = .002), (UFBT; P = .001); (CFT: P = .003), and (CFPT, CFBT, CFA; P = .001)]. Augmentation with thrombin, PRP, and BMA for uncoated (UFT; P = .006, UFPT; P = .035, UFBT; P = .001) and BMA for coated suture (CFBT; P = .027) led to significantly higher alkaline phosphatase activity.

CONCLUSION

Biological enhancement of suture used for ligament bracing significantly increased ligamentocyte and osteoblast adhesion and proliferation, as well as alkaline phosphatase activity of osteoblasts in an in vitro model. After biological augmentation, cellular adhesion, proliferation, and alkaline phosphatase activity changed up to 1,077%, 190%, and 78%, respectively. Furthermore, no overall superiority between uncoated or collagen-coated suture material was observed for cellular adhesion, proliferation, or alkaline phosphatase activity.

CLINICAL RELEVANCE

This study provides in vitro data on a new treatment concept of biologic augmentation for acute ligamentous lesions treated with ligament bracing that has not been widely described. This concept may improve the healing of injured ligaments, in addition to providing immediate biomechanical stabilization.

A highly accurate delta check method using deep learning for detection of sample mix-up in the clinical laboratory

OBJECTIVES

Delta check (DC) is widely used for detecting sample mix-up. Owing to the inadequate error detection and high false-positive rate, the implementation of DC in real-world settings is labor-intensive and rarely capable of absolute detection of sample mix-ups. The aim of the study was to develop a highly accurate DC method based on designed deep learning to detect sample mix-up.

METHODS

A total of 22 routine hematology test items were adopted for the study. The hematology test results, collected from two hospital laboratories, were independently divided into training, validation, and test sets. By selecting six mainstream algorithms, the Deep Belief Network (DBN) was able to learn error-free and artificially (intentionally) mixed sample results. The model's analytical performance was evaluated using training and test sets. The model's clinical validity was evaluated by comparing it with three well-recognized statistical methods.

RESULTS

When the accuracy of our model in the training set reached 0.931 at the 22nd epoch, the corresponding accuracy in the validation set was equal to 0.922. The loss values for the training and validation sets showed a similar (change) trend over time. The accuracy in the test set was 0.931 and the area under the receiver operating characteristic curve was 0.977. DBN demonstrated better performance than the three comparator statistical methods. The accuracy of DBN and revised weighted delta check (RwCDI) was 0.931 and 0.909, respectively. DBN performed significantly better than RCV and EDC. Of all test items, the absolute difference of DC yielded higher accuracy than the relative difference for all methods.

CONCLUSIONS

The findings indicate that input of a group of hematology test items provides more comprehensive information for the accurate detection of sample mix-up by machine learning (ML) when compared with a single test item input method. The DC method based on DBN demonstrated highly effective sample mix-up identification performance in real-world clinical settings.

Decision support or autonomous artificial intelligence? The case of wrong blood in tube errors

OBJECTIVES

Artificial intelligence (AI) models are increasingly being developed for clinical chemistry applications, however, it is not understood whether human interaction with the models, which may occur once they are implemented, improves or worsens their performance. This study examined the effect of human supervision on an artificial neural network trained to identify wrong blood in tube (WBIT) errors.

METHODS

De-identified patient data for current and previous (within seven days) electrolytes, urea and creatinine (EUC) results were used in the computer simulation of WBIT errors at a rate of 50%. Laboratory staff volunteers reviewed the AI model's predictions, and the EUC results on which they were based, before making a final decision regarding the presence or absence of a WBIT error. The performance of this approach was compared to the performance of the AI model operating without human supervision.

RESULTS

Laboratory staff supervised the classification of 510 sets of EUC results. This workflow identified WBIT errors with an accuracy of 81.2%, sensitivity of 73.7% and specificity of 88.6%. However, the AI model classifying these samples autonomously was superior on all metrics (p-values<0.05), including accuracy (92.5%), sensitivity (90.6%) and specificity (94.5%).

CONCLUSIONS

Human interaction with AI models can significantly alter their performance. For computationally complex tasks such as WBIT error identification, best performance may be achieved by autonomously functioning AI models.

The development of autoverification system of lymphocyte subset assays on the flow cytometry platform

OBJECTIVES

Peripheral blood lymphocyte subsets are important parameters for monitoring immune status; however, lymphocyte subset detection is time-consuming and error-prone. This study aimed to explore a highly efficient and clinically useful autoverification system for lymphocyte subset assays performed on the flow cytometry platform.

METHODS

A total of 94,402 lymphocyte subset test results were collected. To establish the limited-range rules, 80,427 results were first used (69,135 T lymphocyte subset tests and 11,292 NK, B, T lymphocyte tests), of which 15,000 T lymphocyte subset tests from human immunodeficiency virus (HIV) infected patients were used to set customized limited-range rules for HIV infected patients. Subsequently, 13,975 results were used for historical data validation and online test validation.

RESULTS

Three key autoverification rules were established, including limited-range, delta-check, and logical rules. Guidelines for addressing the issues that trigger these rules were summarized. The historical data during the validation phase showed that the total autoverification passing rate of lymphocyte subset assays was 69.65% (6,941/9,966), with a 67.93% (5,268/7,755) passing rate for T lymphocyte subset tests and 75.67% (1,673/2,211) for NK, B, T lymphocyte tests. For online test validation, the total autoverification passing rate was 75.26% (3,017/4,009), with 73.23% (2,191/2,992) for the T lymphocyte subset test and 81.22% (826/1,017) for the NK, B, T lymphocyte test. The turnaround time (TAT) was reduced from 228 to 167 min using the autoverification system.

CONCLUSIONS

The autoverification system based on the laboratory information system for lymphocyte subset assays reduced TAT and the number of error reports and helped in the identification of abnormal cell populations that may offer clues for clinical interventions.

Improving the laboratory result release process in the light of ISO 15189:2012 standard

The ISO 15189:2012 standard section 5.9.1 requires laboratories to review results before release, considering quality control, previous results, and clinical information, if any, and to issue documented procedures about it. While laboratory result reporting is generally regarded as part of the post-analytical phase, the result release process requires a general view of the total examination process. Reviewing test results may follow with troubleshooting and test repetition, including reanalyzing an individual sample or resampling. A systematic understanding of the result release may help laboratory professionals carry out appropriate test repetition and ensure the plausibility of laboratory results. In this paper, we addressed the crucial steps in the result release process, including evaluation of sample quality, critical result notification, result reporting, and recommendations for the management of the result release, considering quality control alerts, instrument flags, warning messages, and interference indexes. Error detection tools and plausibility checks mentioned in the present paper can support the daily practice of results release.

Estimation of change limits (deltacheck) in clinical laboratory

Objectives

Change limits, more commonly called delta check, are those in which a change in a patient's measured result in relation to their corresponding preceding measurement is suspected of being erroneous and should be considered as a doubtful result. The aim of this study was to provide change limits for some biochemical and haematological quantities to detect doubtful measured results and to assess its effectiveness to detect erroneous results for their application in and the standardization of the plausibility control.

Methods

Change limits have been estimated for 13 biochemical and 6 haematological quantities. For each quantity, relative differences (D), expressed as a percentage between the two consecutive measured results from the same patient (from scheduled laboratory requests), were calculated. From these differences (D), the p5 and p95 percentiles of the data distribution were calculated. To assess the effectiveness of the change limits to detect laboratory errors, 43 erroneous laboratory reports, containing different biochemical and haematological quantities, were obtained from the standard laboratory plausibility control procedure.

Results

From the 43 erroneous laboratory reports, 31 (72%) were due to endovenous administration errors and 12 (28%) were due to mislabeling errors. All erroneous laboratory reports were detected when the change limits of the quantities were combined and applied together.

Conclusions

The best combination of quantities, which detect all the erroneous reports in the same specimen were: potassium, albumin, creatinine, glucose and haemoglobin.

Identifying mislabelled samples: Machine learning models exceed human performance

BACKGROUND

It is difficult for clinical laboratories to identify samples that are labelled with the details of an incorrect patient. Many laboratories screen for these errors with delta checks, with final decision-making based on manual review of results by laboratory staff. Machine learning models have been shown to outperform delta checks for identifying these errors. However, a comparison of machine learning models to human-level performance has not yet been made.

METHODS

Deidentified data for current and previous (within seven days) electrolytes, urea and creatinine results was used in the computer simulation of mislabelled samples. Eight different machine learning models were developed on 127,256 sets of results using different algorithms: artificial neural network, extreme gradient boosting, support vector machine, random forest, logistic regression, k-nearest neighbours and two decision trees (one complex and one simple). A separate test data-set (n = 14,140) was used to evaluate the performance of these models as well as laboratory staff volunteers, who manually reviewed a random subset of this data (n = 500).

RESULTS

The best performing machine learning model was the artificial neural network (92.1% accuracy), with the simple decision tree demonstrating the poorest accuracy (86.5%). The accuracy of laboratory staff for identifying mislabelled samples was 77.8%.

CONCLUSIONS

The results of this preliminary investigation suggest that even relatively simple machine learning models can exceed human performance for identifying mislabelled samples. Machine learning techniques should be considered for implementation in clinical laboratories to assist with error identification.

Managing biological variation data: modern approaches for study design and clinical application

For more than one half-century, variability observed in clinical test result measurements has been ascribed to three major independent factors: (i) pre-analytical variation, occurring at sample collection and processing steps; (ii) analytical variation of the test method for which measurements are taken, and; (iii) biological variation (BV). Appreciation of this last source of variability is the major goal of this review article. Several recent advances have been made to generate, collate, and utilize BV data of biomarker tests within the clinical laboratory setting. Consideration of both prospective and retrospective study designs will be addressed. The prospective/direct study design will be described in accordance with recent recommendations discussed in the framework of a newly-developed system of checklist items. Potential value of retrospective/indirect study design, modeled on data mining from cohort studies or pathology laboratory information systems (LIS), offers an alternative approach to obtain BV estimates for clinical biomarkers. Moreover, updates to BV databases have made these data more current and widely accessible. Principal aims of this review are to provide the clinical laboratory scientist with a historical framework of BV concepts, to highlight useful applications of BV data within the clinical laboratory environment, and to discuss key terms and concepts related to statistical treatment of BV data.

Average of Patient Deltas: Patient-Based Quality Control Utilizing the Mean Within-Patient Analyte Variation

BACKGROUND

Because traditional QC is discontinuous, laboratories use additional strategies to detect systematic error. One strategy, the delta check, is best suited to detect large systematic error. The moving average (MA) monitors the mean patient analyte value but cannot equitably detect systematic error in skewed distributions. Our study combines delta check and MA to develop an average of deltas (AoD) strategy that monitors the mean delta of consecutive, intrapatient results.

METHODS

Arrays of the differences (delta) between paired patient results collected within 20-28 h of each other were generated from historical data. AoD protocols were developed using a simulated annealing algorithm in MatLab (Mathworks) to select the number of patient delta values to average and truncation limits to eliminate large deltas. We simulated systematic error by adding bias to arrays for plasma albumin, alanine aminotransferase, alkaline phosphatase, amylase, aspartate aminotransferase, bicarbonate, bilirubin (total and direct), calcium, chloride, creatinine, lipase, sodium, phosphorus, potassium, total protein, and magnesium. The average number of deltas to detection (ANDED) was then calculated in response to induced systematic error.

RESULTS

ANDED varied by combination of assay and AoD protocol. Errors in albumin, lipase, and total protein were detected with a mean of 6 delta pairs. The highest ANDED was calcium, with a positive 0.6-mg/dL shift detected with an ANDED of 75. However, a negative 0.6-mg/dL calcium shift was detected with an ANDED of 25.

CONCLUSIONS

AoD detects systematic error with relatively few paired patient samples and is a patient-based QC technique that will enhance error detection.

Integrating quality control and external quality assurance

Effective management of clinical laboratories relies upon an understanding of Quality Control and External Quality Assurance principles. These processes, when applied effectively, reduce patient risk and drive quality improvement. In this Review, we will describe the purpose of QC and EQA and their role in identifying analytical and process error. The two concepts are linked, and we will illustrate that linkage. Some EQA providers offer far more than analytical surveillance. They facilitate training and education and extend quality improvement and identify areas where there is potential for patient harm into the pre-and post-analytical phases of the total testing process.

The Impact of COVID-19 Containment Actions on Extra-Analytical Phases of the Clinical Laboratory: A Case Report

Laboratory information systems need to adapt to new demands created by the COVID-19 pandemic, which has set up new normals like containment measures and social distancing. Some of these have negatively impacted the pre- and postanalytical phases of laboratory testing. Here, we present an intriguing finding related to the generation of the accession number/specimen number on the investigation module of a hospital management information system and its impact on the dissemination of reports resulting in the wrong release of reports on a female patient amidst the background of COVID-19 containment measures. We analyze the situation that led to this false reporting and the importance of the proper customization of information software in laboratories along with a robust postanalytical framework of laboratory work culture to avert such untoward incidents. This introspection has made us realize that COVID-19 has been a scientific, medical, and social challenge. We need to redefine our priorities in the days to come because SARS-CoV-2 is here to stay.

Biological variation: Understanding why it is so important?

This Review will describe the increasing importance of the concepts of biological variation to clinical chemists. The idea of comparison to 'reference' is fundamental in measurement. For the biological measurands, that reference is the relevant patient population, a clinical decision point based on a trial or an individual patient's previous results. The idea of using biological variation to set quality goals was then realised for setting Quality Control (QC) and External Quality Assurance (EQA) limits. The current phase of BV integration into practice is using Patient-Based Real-Time Quality Control (PBRTQC) and Patient Based Quality Assurance (PBQA) to detect a change in assay performance. The challenge of personalised medicine is to determine an individual reference interval. The Athletes Biological Passport may provide the solution.

The Hypnotic Analgesia Suggestion Mitigated the Effect of the Transcranial Direct Current Stimulation on the Descending Pain Modulatory System: A Proof of Concept Study

Objective

We evaluated whether active(a)-tDCS combined with hypnotic analgesia suggestion (HS) would be more effective than a single active(a)-tDCS, and/or sham-(s)-tDCS and s-tDCS/HS on the following outcomes: function of descending pain modulatory system (DPMS) during the conditioned pain modulation test (CPM-test) (primary outcome), heat pain threshold (HPT), heat pain tolerance (HPTo) and cold pressor test (CPT) (secondary outcomes). We also examined whether their effects are related to neuroplasticity state evaluated by serum brain-derived-neurotropic factor (BDNF).

Materials and Methods

Forty-eight females received one session of one of the four interventions (a-tDCS/HS, s-tDCS/HS, a-tDCS, and s-tDCS) in an incomplete randomized crossover sequence. The a-tDCS or s-tDCS was applied over the left dorsolateral prefrontal cortex (DLPFC) for 30 minutes at 2mA.

Results

A generalized linear model revealed a significant main effect for the intervention group (P <0.032). The delta-(Δ) pain score on the Numerical Pain Scale (NPS0-10) during CPM-test in the a-tDCS/HS group was -0.25 (0.43). The (Δ) pain score on NPS (0-10) during CPM-test in the other three groups was a-tDCS=-0.54 (0.41), HS -0.01 (0.41) and s-tDCS/HS=-0.19 (0.43). A-tDCS/HS intervention increased the CPT substantially compared to all other interventions. Also, higher baseline levels of BDNF were associated with a larger change in CPT and HPTo.

Conclusion

These findings indicate that the HS combined with a-tDCS mitigated the effect of the a-tDCS on the DPMS. The a-tDCS up-regulates the inhibition on DPMS, and the HS improved pain tolerance. And, together they enhanced the reaction time substantially upon the CPT.

Clinical Trial Registration

www.ClinicalTrials.gov, identifier NCT03744897.

Evidence-based approach to setting delta check rules

Delta checks are a post-analytical verification tool that compare the difference in sequential laboratory results belonging to the same patient against a predefined limit. This unique quality tool highlights a potential error at the individual patient level. A difference in sequential laboratory results that exceeds the predefined limit is considered likely to contain an error that requires further investigation that can be time and resource intensive. This may cause a delay in the provision of the result to the healthcare provider or entail recollection of the patient sample. Delta checks have been used primarily to detect sample misidentification (sample mix-up, wrong blood in tube), and recent advancements in laboratory medicine, including the adoption of protocolized procedures, information technology and automation in the total testing process, have significantly reduced the prevalence of such errors. As such, delta check rules need to be selected carefully to balance the clinical risk of these errors and the need to maintain operational efficiency. Historically, delta check rules have been set by professional opinion based on reference change values (biological variation) or the published literature. Delta check rules implemented in this manner may not inform laboratory practitioners of their real-world performance. This review discusses several evidence-based approaches to the optimal setting of delta check rules that directly inform the laboratory practitioner of the error detection capabilities of the selected rules. Subsequent verification of workflow for the selected delta check rules is also discussed. This review is intended to provide practical assistance to laboratories in setting evidence-based delta check rules that best suits their local operational and clinical needs.

Mass spectrometric sample identification with indicator compounds introduced via labeled sample tubes

Objectives

The risk of sample confusion continues to be a challenge for the pre-analytical part of the overall testing process. We here describe a novel system to track samples based on a chemical code labeling of test tubes with unique combinations of indicator compounds, which are naturally not present in specimens of human origin. As part of the sample vessel filling, the liquid specimens are permanently labeled with the compound code that can be tracked back to the primary tube.

Methods

As a proof of concept we used 10 stable-isotope-labeled derivates of medical drugs as indicator substances to create a combinatory 10-digit binary number ID for individual test tubes, i.e. presence/absence of the respective compound. For this purpose, combinations of indicator compounds were provided in evaporated form in polypropylene tubes prior to filling with anonymized patient whole blood and corresponding plasmas subjected to liquid chromatography tandem-mass spectrometry designed to detect the 10 indicator compounds.

Results

In the blind analysis, we correctly identified 307 different whole blood samples by readout of a 10-digit binary number ID based on the detection of indicator compounds with respect to their presence and number.

Conclusions

We have demonstrated the feasibility of an internal labeling procedure for diagnostic samples with mass spectrometry-based readout of dissolved indicator compound combinations as a binary number ID. With an increasing number of coding compounds (≫10) a vast number of combinations for sample labeling can be realized beyond the proof of concept setting studied herein.

An approach to selecting auto-verification limits and validating their error detection performance independently for pre-analytical and analytical errors

BACKGROUND

Auto-verification limits are widely used to trigger confirmatory actions to enable detection of pre-analytical, analytical and post-analytical errors. An approach is presented for validating auto-verification limit performance in a laboratory-specific manner, independently for pre-analytical and analytical error detection.

METHODS

To evaluate this approach, MA Generator (www.huvaros.com) was used to run error-detection simulations using various upper-limit checks (ULC) and lower-limit checks (LLC). Pre-analytical error detection was defined as triggering of a limit check alarm within one erroneous result. Analytical error detection was defined as triggering a limit check alarm within the scheduled internal QC measurement interval, both with ≥97.5% probability. Furthermore, the limit check alarm rates were obtained.

RESULTS

Pre-analytical error detection and rapid detection of larger analytical errors by limit checks outperformed moving average quality control at the cost of a significantly larger number of alarms. A pre-analytical error detection by LLC and ULC of ≥-55% and >60%, ≥-10% and ≥20%, and ≥-40% and ≥50% and an analytical error detection of ≥-4% and ≥15%, ≥-3% and ≥4% and ≥-30% and ≥25% were obtained for hemoglobin, sodium and calcium, respectively.

CONCLUSIONS

The obtained ULC and LLC alarm rate and error detection performance, enabled substantiated selection of optimal auto-verification limits and validation thereof.

National survey on delta checks in clinical laboratories in China

Background

This study aimed to understand the status quo of delta checks in Chinese clinical laboratories through a nationwide online survey.

Methods

The survey was divided into two parts. The first part was a general situation survey in which clinical laboratories had to provide information about the laboratories, including delta checks used. In the second part, clinical laboratories were asked to record the delta check alerts generated in their laboratories from June 1st, 2019 to June 30th, 2019.

Results

The most frequently used analytes in delta checks were potassium (K), glucose (Glu), creatinine (Cre) for clinical chemistry and hemoglobin (Hgb), platelet (PLT) count and white blood cell (WBC) count for clinical hematology. The median maximum time interval between specimens for all analytes was 5 days. The most commonly used delta check calculation modes in Chinese clinical laboratories were percentage change and absolute change. K and Hgb were the analytes most involved in clinical chemistry and clinical hematology delta check alerts. The most common causes of delta check alerts were that the patients had received treatment, which was followed by the change in the patient’s physiological state and interference from hemolysis, lipemia and icterus. The two most common outcomes of delta check alerts were ‘no problems found, standard report issued’ and ‘no problems found, report issued with comment’.

Conclusions

This study was the first nationwide survey of delta checks in China, the results of which help us to understand the current situation of delta checks in Chinese clinical laboratories.

Data-driven quality assurance to prevent erroneous test results

Increasing laboratory automation and efficiency requires quality assurance (QA) approaches to ensure that reported results are precise and accurate. Prerequisites for designing optimal QA strategies include an in-depth understanding of the laboratory processes, the expected results, and of the mechanisms that can cause erroneous results. Oftentimes, a laboratory's own data, extracted from the laboratory information system, electronic medical record, and/or clinical data warehouse are necessary to master the aforementioned requirements. Data-driven QA utilizes retrospective and/or prospective laboratory results to minimize errors in the clinical laboratory due to pre-analytical or analytical vulnerabilities. Additionally, exploitation of this data may improve result interpretation. The objective of this review is to illustrate specific examples of data-driven QA approaches for several areas of the clinical laboratory and for different phases of the testing cycle.

Design and evaluation of a LIS-based autoverification system for coagulation assays in a core clinical laboratory

Background

The autoverification system for coagulation consists of a series of rules that allow normal data to be released without manual verification. With new advances in medical informatics, the laboratory information system (LIS) has growing potential for the autoverification, allowing rapid and accurate verification of clinical laboratory tests. The purpose of the study is to develop and evaluate a LIS-based autoverification system for validation and efficiency.

Methods

Autoverification decision rules, including quality control, analytical error flag, critical value, limited range check, delta check and logical check, as well as patient’s historical information, were integrated into the LIS. Autoverification limited range was constructed based on 5 and 95% percentiles. The four most commonly used coagulation assays, prothrombin time (PT), activated partial thromboplastin time (APTT), thrombin time (TT), and fibrinogen (FBG), were followed by the autoverification protocols. The validation was assessed by the autoverification passing rate, the true-positive cases, the true-negative cases, the false-positive cases, the false-negative cases, the sensitivity and the specificity; the efficiency was evaluated in the turnaround time (TAT).

Results

A total of 157,079 historical test results of coagulation profiles from January 2016 to December 2016 were collected to determine the distribution intervals. The autoverification passing rate was 77.11% (29,165/37,821) based on historical patient data. In the initial test of the autoverification version in June 2017, the overall autoverification passing rate for the whole sample was 78.75% (11,257/14,295), with 892 true-positive cases, 11,257 true-negative cases, 2146 false-positive cases, no false-negative cases, sensitivity of 100% and specificity of 83.99%. After formal implementation of the autoverification system for 6 months, 83,699 samples were assessed. The average overall autoverification passing rate for the whole sample was 78.86% and the 95% confidence interval (CI) of the passing rate was [78.25, 79.59%]. TAT was reduced from 126 min to 101 min, which was statistically significant (P < 0.001, Mann-Whitney U test).

Conclusions

The autoverification system for coagulation assays based on LIS can halt the samples with abnormal values for manual verification, guarantee medical safety, minimize the requirements for manual work, shorten TAT and raise working efficiency.