The development of autoverification rules applied to urinalysis performed on the AutionMAX-SediMAX platform

Current urinalysis practices in Belgian laboratories towards the 2023 EFLM European urinalysis guideline

OBJECTIVES/BACKGROUND

We aimed to investigate routine urinalysis practices in Belgian laboratories and verify these findings against the 2023 European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) European Urinalysis Guideline.

METHODS

A questionnaire was developed to collect information on pre- to postanalytical aspects of urine test strip and particle analysis. The questionnaire was distributed by Sciensano to all Belgian laboratories, licensed to perform urine particle analysis.

RESULTS

Sixty-six percent of the Belgian laboratories (75/113) participated. The responding laboratories served physicians in private (25%), hospital (60%) and university hospital (15%) setting. All laboratories performed test strip and particle analysis, predominantly automatically (97% and 96%, respectively). In addition, most laboratories (87%) used intelligent verification criteria to optimize diagnostic accuracy. Almost all laboratories (≥90%) screened and reported a minimal biochemistry panel (glucose, protein, pH, ketones) and particle count (red and white blood cells). Independent of the technology, a notable variability was observed regarding medical cut-off values and advanced particle differentiation and reporting. Internal quality control was extensively performed for urine test strip (91%) and particle analysis (96%), while external QC was less common (32% and 36%, respectively). Consequently, only few laboratories were ISO15189 accredited for urine test strip (15%) and particle analysis (17%).

CONCLUSION

There is considerable variability in current urinalysis performed in Belgian laboratories. The 2023 EFLM urinalysis guideline has the potential to guide clinical laboratories towards improving their urinalysis practices. Additional efforts are required to implement these recommendations into clinical practice in Belgium.

Applications of Artificial Intelligence in Urinalysis: Is the Future Already Here?

BACKGROUND

Artificial intelligence (AI) has emerged as a promising and transformative tool in the field of urinalysis, offering substantial potential for advancements in disease diagnosis and the development of predictive models for monitoring medical treatment responses.

CONTENT

Through an extensive examination of relevant literature, this narrative review illustrates the significance and applicability of AI models across the diverse application area of urinalysis. It encompasses automated urine test strip and sediment analysis, urinary tract infection screening, and the interpretation of complex biochemical signatures in urine, including the utilization of cutting-edge techniques such as mass spectrometry and molecular-based profiles.

SUMMARY

Retrospective studies consistently demonstrate good performance of AI models in urinalysis, showcasing their potential to revolutionize clinical practice. However, to comprehensively evaluate the real clinical value and efficacy of AI models, large-scale prospective studies are essential. Such studies hold the potential to enhance diagnostic accuracy, improve patient outcomes, and optimize medical treatment strategies. By bridging the gap between research and clinical implementation, AI can reshape the landscape of urinalysis, paving the way for more personalized and effective patient care.

Pitfalls in the diagnosis of hematuria

Detection of hemoglobin (Hb) and red blood cells in urine (hematuria) is characterized by a large number of pitfalls. Clinicians and laboratory specialists must be aware of these pitfalls since they often lead to medical overconsumption or incorrect diagnosis. Pre-analytical issues (use of vacuum tubes or urine tubes containing preservatives) can affect test results. In routine clinical laboratories, hematuria can be assayed using either chemical (test strips) or particle-counting techniques. In cases of doubtful results, Munchausen syndrome or adulteration of the urine specimen should be excluded. Pigmenturia (caused by the presence of dyes, urinary metabolites such as porphyrins and homogentisic acid, and certain drugs in the urine) can be easily confused with hematuria. The peroxidase activity (test strip) can be positively affected by the presence of non-Hb peroxidases (e.g. myoglobin, semen peroxidases, bacterial, and vegetable peroxidases). Urinary pH, haptoglobin concentration, and urine osmolality may affect specific peroxidase activity. The implementation of expert systems may be helpful in detecting preanalytical and analytical errors in the assessment of hematuria. Correcting for dilution using osmolality, density, or conductivity may be useful for heavily concentrated or diluted urine samples.

Improving clinical performance of urine sediment analysis by implementation of intelligent verification criteria

OBJECTIVES

Urinary test strip and sediment analysis integrated with intelligent verification criteria can help to select samples that need manual review. This study aimed to evaluate the improvement in the diagnostic performance of combined urinary test strip and urinary sediment analysis using intelligent verification criteria on the latest generation automated test strip and urinary fluoresce flow cytometry (UFFC) analysers.

METHODS

Urine test strip and sediment analysis were performed using the Sysmex UC-3500 and UF-5000 (Kobe, Japan) on 828 urinary samples at the clinical laboratory of the Ghent University Hospital. The results were compared to manual microscopy using phase-contrast microscopy as a reference. After the application of the intelligent verification criteria, we determined whether the diagnostic performance of urine sediment analysis could be improved.

RESULTS

Application of intelligent verification criteria resulted in an increase in specificity from 88.5 to 96.8% and from 88.2 to 94.9% for red blood cells and white blood cells, respectively. Implementing review rules for renal tubular epithelial cells and pathological casts increased the specificity from 66.7 to 74.2% and from 96.2 to 100.0%, respectively; and improved the diagnostic performance of urinary crystals and atypical cells.

CONCLUSIONS

The implementation of review rules improved the diagnostic performance of UFFC, thereby increasing the reliability and quality of urine sediment results.

The development of autoverification system of lymphocyte subset assays on the flow cytometry platform

OBJECTIVES

Peripheral blood lymphocyte subsets are important parameters for monitoring immune status; however, lymphocyte subset detection is time-consuming and error-prone. This study aimed to explore a highly efficient and clinically useful autoverification system for lymphocyte subset assays performed on the flow cytometry platform.

METHODS

A total of 94,402 lymphocyte subset test results were collected. To establish the limited-range rules, 80,427 results were first used (69,135 T lymphocyte subset tests and 11,292 NK, B, T lymphocyte tests), of which 15,000 T lymphocyte subset tests from human immunodeficiency virus (HIV) infected patients were used to set customized limited-range rules for HIV infected patients. Subsequently, 13,975 results were used for historical data validation and online test validation.

RESULTS

Three key autoverification rules were established, including limited-range, delta-check, and logical rules. Guidelines for addressing the issues that trigger these rules were summarized. The historical data during the validation phase showed that the total autoverification passing rate of lymphocyte subset assays was 69.65% (6,941/9,966), with a 67.93% (5,268/7,755) passing rate for T lymphocyte subset tests and 75.67% (1,673/2,211) for NK, B, T lymphocyte tests. For online test validation, the total autoverification passing rate was 75.26% (3,017/4,009), with 73.23% (2,191/2,992) for the T lymphocyte subset test and 81.22% (826/1,017) for the NK, B, T lymphocyte test. The turnaround time (TAT) was reduced from 228 to 167 min using the autoverification system.

CONCLUSIONS

The autoverification system based on the laboratory information system for lymphocyte subset assays reduced TAT and the number of error reports and helped in the identification of abnormal cell populations that may offer clues for clinical interventions.

Development and implementation of an LIS-based validation system for autoverification toward zero defects in the automated reporting of laboratory test results

Background

Validation of the autoverification function is one of the critical steps to confirm its effectiveness before use. It is crucial to verify whether the programmed algorithm follows the expected logic and produces the expected results. This process has always relied on the assessment of human–machine consistency and is mostly a manually recorded and time-consuming activity with inherent subjectivity and arbitrariness that cannot guarantee a comprehensive, timely and continuous effectiveness evaluation of the autoverification function. To overcome these inherent limitations, we independently developed and implemented a laboratory information system (LIS)-based validation system for autoverification.

Methods

We developed a correctness verification and integrity validation method (hereinafter referred to as the "new method") in the form of a human–machine dialog. The system records personnel review steps and determines whether the human–machine review results are consistent. Laboratory personnel then analyze the reasons for any inconsistency according to system prompts, add to or modify rules, reverify, and finally improve the accuracy of autoverification.

Results

The validation system was successfully established and implemented. For a dataset consisting of 833 rules for 30 assays, 782 rules (93.87%) were successfully verified in the correctness verification phase, and 51 rules were deleted due to execution errors. In the integrity validation phase, 24 projects were easily verified, while the other 6 projects still required the additional rules or changes to the rule settings. Taking the Hepatitis B virus test as an example, from the setting of 65 rules to the automated releasing of 3000 reports, the validation time was reduced from 452 (manual verification) to 275 h (new method), a reduction in validation time of 177 h. Furthermore, 94.6% (168/182) of laboratory users believed the new method greatly reduced the workload, effectively controlled the report risk and felt satisfied. Since 2019, over 3.5 million reports have been automatically reviewed and issued without a single clinical complaint.

Conclusion

To the best of our knowledge, this is the first report to realize autoverification validation as a human–machine interaction. The new method effectively controls the risks of autoverification, shortens time consumption, and improves the efficiency of laboratory verification.

Establishment of the intelligent verification criteria for a routine urinalysis analyzer in a multi-center study

Background Although laboratory information system (LIS) is widely used nowadays, the results of routine urinalysis still need 100% manual verification. We established intelligent verification criteria to perform the automated verification process and reduce manual labor. Methods A total of 4610 urine specimens were obtained from the patients of three hospitals in Beijing, China. Firstly, 895 specimens were measured to establish the reference intervals of formed-element parameters in UF5000. Secondly, 2803 specimens were analyzed for setting up the intelligent verification criteria (including the microscopic review rules and manual verification rules). Lastly, 912 specimens were used to verify the efficacy and accuracy of the intelligent verification criteria. Phase-contrast microscopes were used for the microscopic review. Results Employing a results level corresponding relationship in specific parameters including hemoglobin (red blood cell [RBC]), leukocyte esterase (white blood cell [WBC]) and protein (cast) between the dry-chemistry analysis and formed-element analysis, as well as instrument flags, we established seven WBC verification rules, eight RBC verification rules and four cast verification rules. Based on the microscopy results, through analyzing the pre-set rules mentioned earlier, we finally determined seven microscopic review rules, nine manual verification rules and three auto-verification rules. The microscopic review rate was 21.98% (616/2803), the false-negative rate was 4.32% (121/2803), the total manual verification rate was 35.71% (1001/2803) and the auto-verification rate was 64.29% (1802/2803). The validation results were consistent. Conclusions The intelligent verification criteria for urinary dry-chemistry and urinary formed-element analysis can improve the efficiency of the results verification process and ensure the reliability of the test results.

Increased effectiveness of urinalysis testing via the integration of automated instrumentation, the lean management approach, and autoverification

Background

In 2014, the Department of Clinical Pathology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand developed and implemented a new process that uses fully automated instrumentation, the lean management approach, and autoverification to improve the productivity and efficiency of the urinalysis workflow process. The aim of this study was to evaluate analytical turnaround time compared with our old urinalysis workflow process and our new urinalysis workflow process that was launched in 2014.

Methods

This study was performed at the Central Laboratory of our center during June 2017 using data collected from the July 2012 (old process) and July 2014 (new process) study periods. We used our laboratory information system to compute and analyze turnaround time of urinalysis tests, and those results were compared between processes.

Results

The 90th percentile turnaround time in overall data was dramatically decreased from approximately 60 minutes in 2012 to <50 minutes in 2014. The mean during both 6:00 am to 9:00 am and 9:00 am to 12:00 pm was approximately 42 minutes in 2012; however, that duration was reduced to approximately 30 minutes for both of those time periods in 2014. Specimens within 60 minutes in both intervals increase from approximately 80% to more than 90%.

Conclusion

The results of this study revealed our new urinalysis workflow process that incorporates fully automated instrumentation, the lean management approach, and autoverification to be effective for significantly increasing productivity as measured by analytical turnaround time and removing 1 staff to another section.

Design and evaluation of a LIS-based autoverification system for coagulation assays in a core clinical laboratory

Background

The autoverification system for coagulation consists of a series of rules that allow normal data to be released without manual verification. With new advances in medical informatics, the laboratory information system (LIS) has growing potential for the autoverification, allowing rapid and accurate verification of clinical laboratory tests. The purpose of the study is to develop and evaluate a LIS-based autoverification system for validation and efficiency.

Methods

Autoverification decision rules, including quality control, analytical error flag, critical value, limited range check, delta check and logical check, as well as patient’s historical information, were integrated into the LIS. Autoverification limited range was constructed based on 5 and 95% percentiles. The four most commonly used coagulation assays, prothrombin time (PT), activated partial thromboplastin time (APTT), thrombin time (TT), and fibrinogen (FBG), were followed by the autoverification protocols. The validation was assessed by the autoverification passing rate, the true-positive cases, the true-negative cases, the false-positive cases, the false-negative cases, the sensitivity and the specificity; the efficiency was evaluated in the turnaround time (TAT).

Results

A total of 157,079 historical test results of coagulation profiles from January 2016 to December 2016 were collected to determine the distribution intervals. The autoverification passing rate was 77.11% (29,165/37,821) based on historical patient data. In the initial test of the autoverification version in June 2017, the overall autoverification passing rate for the whole sample was 78.75% (11,257/14,295), with 892 true-positive cases, 11,257 true-negative cases, 2146 false-positive cases, no false-negative cases, sensitivity of 100% and specificity of 83.99%. After formal implementation of the autoverification system for 6 months, 83,699 samples were assessed. The average overall autoverification passing rate for the whole sample was 78.86% and the 95% confidence interval (CI) of the passing rate was [78.25, 79.59%]. TAT was reduced from 126 min to 101 min, which was statistically significant (P < 0.001, Mann-Whitney U test).

Conclusions

The autoverification system for coagulation assays based on LIS can halt the samples with abnormal values for manual verification, guarantee medical safety, minimize the requirements for manual work, shorten TAT and raise working efficiency.