Comparative study on the quality control effectiveness of AI-PBRTQC and traditional PBRTQC model in identifying quality risks

Introduction

We compared the quality control efficiency of artificial intelligence-patient-based real-time quality control (AI-PBRTQC) and traditional PBRTQC in laboratories to create favorable conditions for the broader application of PBRTQC in clinical laboratories.

Materials and methods

In the present study, the data of patients with total thyroxine (TT4), anti-Müllerian hormone (AMH), alanine aminotransferase (ALT), total cholesterol (TC), urea, and albumin (ALB) over five months were categorized into two groups: AI-PBRTQC group and traditional PBRTQC group. The Box-Cox transformation method estimated truncation ranges in the conventional PBRTQC group. In contrast, in the AI-PBRTQC group, the PBRTQC software platform intelligently selected the truncation ranges. We developed various validation models by incorporating different weighting factors, denoted as λ. Error detection, false positive rate, false negative rate, average number of the patient sample until error detection, and area under the curve were employed to evaluate the optimal PBRTQC model in this study. This study provides evidence of the effectiveness of AI-PBRTQC in identifying quality risks by analyzing quality risk cases.

Results

The optimal parameter setting scheme for PBRTQC is TT4 (78-186), λ = 0.03; AMH (0.02-2.96), λ = 0.02; ALT (10-25), λ = 0.02; TC (2.84-5.87), λ = 0.02; urea (3.5-6.6), λ = 0.02; ALB (43-52), λ = 0.05.

Conclusions

The AI-PBRTQC group was more efficient in identifying quality risks than the conventional PBRTQC. AI-PBRTQC can also effectively identify quality risks in a small number of samples. AI-PBRTQC can be used to determine quality risks in both biochemistry and immunology analytes. AI-PBRTQC identifies quality risks such as reagent calibration, onboard time, and brand changes.

Application of Patient-Based Real-Time Quality Control Based on Artificial Intelligence Monitoring Platform in Continuously Quality Risk Monitoring of Down Syndrome Serum Screening

BACKGROUND

Patient-based real-time quality control (PBRTQC) has gained attention because of its potential to continuously monitor the analytical quality in situations wherein internal quality control (IQC) is less effective. Therefore, we tried to investigate the application of PBRTQC method based on an artificial intelligence monitoring (AI-MA) platform in quality risk monitoring of Down syndrome (DS) serum screening.

METHODS

The DS serum screening item determination data and relative IQC data from January 4 to September 7 in 2021 were collected. Then, PBRTQC exponentially weighted moving average (EWMA) and moving average (MA) procedures were built and optimized in the AI-MA platform. The efficiency of the EWMA and MA procedures with intelligent and traditional control rules were compared. Next, the optimal EWMA procedures that contributed to the quality assurance of serum screening were run and generated early warning cases were investigated.

RESULTS

Optimal EWMA and MA procedures on the AI-MA platform were built. Comparison results showed the EWMA procedure with intelligent QC rules but not traditional quality rules contained the best efficiency. Based on the AI-MA platform, two early warning cases were generated by using the optimal EWMA procedure, which finally found were caused by instrument failure. Moreover, the EWMA procedure could truly reflect the detection accuracy and quality in situations wherein traditional IQC products were unstable or concentrations were inappropriate.

CONCLUSIONS

The EWMA procedure built by the AI-MA platform could be a good complementary control tool for the DS serum screening by truly and timely reflecting the detection quality risks.

Advances in internal quality control

Quality control practices in the modern laboratory are the result of significant advances over the many years of the profession. Major advance in conventional internal quality control has undergone a philosophical shift from a focus solely on the statistical assessment of the probability of error identification to more recent thinking on the capability of the measurement procedure (e.g. sigma metrics), and most recently, the risk of harm to the patient (the probability of patient results being affected by an error or the number of patient results with unacceptable analytical quality). Nonetheless, conventional internal quality control strategies still face significant limitations, such as the lack of (proven) commutability of the material with patient samples, the frequency of episodic testing, and the impact of operational and financial costs, that cannot be overcome by statistical advances. In contrast, patient-based quality control has seen significant developments including algorithms that improve the detection of specific errors, parameter optimization approaches, systematic validation protocols, and advanced algorithms that require very low numbers of patient results while retaining sensitive error detection. Patient-based quality control will continue to improve with the development of new algorithms that reduce biological noise and improve analytical error detection. Patient-based quality control provides continuous and commutable information about the measurement procedure that cannot be easily replicated by conventional internal quality control. Most importantly, the use of patient-based quality control helps laboratories to improve their appreciation of the clinical impact of the laboratory results produced, bringing them closer to the patients.Laboratories are encouraged to implement patient-based quality control processes to overcome the limitations of conventional internal quality control practices. Regulatory changes to recognize the capability of patient-based quality approaches, as well as laboratory informatics advances, are required for this tool to be adopted more widely.

Technical quality assurance and quality control for medical laboratories: a review and proposal of a new concept to obtain integrated and validated QA/QC plans

Technical quality assurance (QA) and quality control (QA/QC) are important activities within medical laboratories to ensure the adequate quality of obtained test results. QA/QC tools available at medical laboratories include external QC and internal QC, patient-based real-time quality control (PBRTQC) tools such as moving average quality control (MAQC), limit checks, delta checks, and multivariate checks, and finally, analyzer flagging. Recently, for PBRTQC tools, new optimization and validation methods based on error detection simulation have been developed to obtain laboratory-specific insights into PBRTQC error detection. These developments have enabled implementation and application of these individual tools in routine clinical practice. As a next step, they also enable performance comparison of the individual QA/QC tools and integration of all the individual QA/QC tools in order to obtain the most powerful and efficient QA/QC plans. In this review, a brief overview of the individual QA/QC tools and their characteristics is provided and the error detection simulation approaches are explained. Finally, a new concept entitled integrated quality assurance and control (IQAC) is presented. To enable IQAC, a conceptual framework is suggested and demonstrated for sodium, based on available published data. The proposed IQAC framework provides ways and tools by which the performance of different QA/QC tools can be compared in a so-called QA/QC error detection table to enable optimization and validation of the overall QA/QC plan in terms of alarm rate as well as pre-analytical, analytical, and post-analytical error detection performance.

The next wave of innovation in laboratory automation: systems for auto-verification, quality control and specimen quality assurance

Laboratory automation in clinical laboratories has made enormous differences in patient outcomes, with a wide range of tests now available that are accurate and have a rapid turnaround. Total laboratory automation (TLA) has mechanised tube handling, sample preparation and storage in general chemistry, immunoassay, haematology, and microbiology and removed most of the tedious tasks involved in those processes. However, there are still many tasks that must be performed by humans who monitor the automation lines. We are seeing an increase in the complexity of the automated laboratory through further platform consolidation and expansion of the reach of molecular genetics into the core laboratory space. This will likely require rapid implementation of enhanced real time quality control measures and these solutions will generate a significantly greater number of failure flags. To capitalise on the benefits that an improved quality control process can deliver, it will be important to ensure that an automation process is implemented simultaneously with enhanced, real time quality control measures and auto-verification of patient samples in middleware. Therefore, it appears that the best solution may be to automate those critical decisions that still require human intervention and therefore include quality control as an integral part of total laboratory automation.

Moving average procedures as an additional tool for real-time analytical quality control: challenges and opportunities of implementation in small-volume medical laboratories

Introduction

Moving average (MA) is one possible way to use patient results for analytical quality control in medical laboratories. The aims of this study were to: (1) implement previously optimized MA procedures for 10 clinical chemistry analytes into the laboratory information system (LIS); (2) monitor their performance as a real-time quality control tool, and (3) define an algorithm for MA alarm management in a small-volume laboratory to suit the specific laboratory.

Materials and methods

Moving average alarms were monitored and analysed over a period of 6 months on all patient results (total of 73,059) obtained for 10 clinical chemistry parameters. The optimal MA procedures were selected previously using an already described technique called the bias detection simulation method, considering the ability of bias detection the size of total allowable error as the key parameter for optimization.

Results

During 6 months, 17 MA alarms were registered, which is 0.023% of the total number of generated MA values. In 65% of cases, their cause was of pre-analytical origin, in 12% of analytical origin, and in 23% the cause was not found. The highest alarm rate was determined on sodium (0.10%), and the lowest on calcium and chloride.

Conclusions

This paper showed that even in a small-volume laboratory, previously optimized MA procedures could be successfully implemented in the LIS and used for continuous quality control. Review of patient results, re-analysis of samples from the stable period, analysis of internal quality control samples and assessment of the analyser malfunctions and maintenance log have been proposed for the algorithm for managing MA alarms.

Multirule procedures vs moving average algorithms for IQC: An appropriate comparison reveals how best to combine their strengths

BACKGROUND

Moving Average Algorithms (MAA) have been widely recommended for use in Patient Based Real Time Quality Control applications (PBRTQC) to supplement or replace traditional Internal Quality Control (IQC) techniques. A recent "proof of concept" study recommends applying MAAs to IQC data to replace traditional IQC procedures because they "outperform Westgard Rules," which is a current standard of practice for IQC.

METHODS

We generated power curves for multi-rule procedures with 2 and 4 control measurements per QC event, as well as a Simple Moving Average having block sizes of 5, 10, and 20 control measurements. We also assessed time to detection in terms of the Average Number of QC Events required to detect different sizes of systematic errors.

RESULTS

As expected, the more control measurements included in the control technique, the better the error detection. However, when QC performance is considered on the Sigma Scale, high Sigma methods require only 1 or 2 control measurements to detect medically important systematic errors. MAAs have very low ability to detect error at the first few QC events following shift, so they suffer a lag phase in detecting medically important errors. MAAs are most useful for methods having 4.0 Sigma performance or less. Even then, large systematic shifts are more quickly detected by simple single and multirule procedures.

CONCLUSIONS

Choice of control techniques (rules, means, ranges, etc.) should consider the Sigma-metric of the method. For methods having Sigmas of 4 or greater, traditional single rule and multirule procedures with Ns up to 4 are most effective; below 4 Sigma, a multirule coupled with a Simple Moving Average (SMA) rule with Ns of 4 to 8 can improve error detection.

Regression-Adjusted Real-Time Quality Control

BACKGROUND

Patient-based real-time quality control (PBRTQC) has gained increasing attention in the field of clinical laboratory management in recent years. Despite the many upsides that PBRTQC brings to the laboratory management system, it has been questioned for its performance and practical applicability for some analytes. This study introduces an extended method, regression-adjusted real-time quality control (RARTQC), to improve the performance of real-time quality control protocols.

METHODS

In contrast to the PBRTQC, RARTQC has an additional regression adjustment step before using a common statistical process control algorithm, such as the moving average, to decide whether an analytical error exists. We used all patient test results of 4 analytes in 2019 from Zhongshan Hospital, Fudan University, to compare the performance of the 2 frameworks. Three types of analytical error were added in the study to compare the performance of PBRTQC and RARTQC protocols: constant, random, and proportional errors. The false alarm rate and error detection charts were used to assess the protocols.

RESULTS

The study showed that RARTQC outperformed PBRTQC. RARTQC, compared with the PBRTQC, improved the trimmed average number of patients affected before detection (tANPed) at total allowable error by about 50% for both constant and proportional errors.

CONCLUSIONS

The regression step in the RARTQC framework removes autocorrelation in the test results, allows researchers to add additional variables, and improves data transformation. RARTQC is a powerful framework for real-time quality control research.

Integrating quality control and external quality assurance

Effective management of clinical laboratories relies upon an understanding of Quality Control and External Quality Assurance principles. These processes, when applied effectively, reduce patient risk and drive quality improvement. In this Review, we will describe the purpose of QC and EQA and their role in identifying analytical and process error. The two concepts are linked, and we will illustrate that linkage. Some EQA providers offer far more than analytical surveillance. They facilitate training and education and extend quality improvement and identify areas where there is potential for patient harm into the pre-and post-analytical phases of the total testing process.

Benefits, limitations, and controversies on patient-based real-time quality control (PBRTQC) and the evidence behind the practice

BACKGROUND

In recent years, there has been renewed interest in the "old" average of normals concept, now generally referred to as moving average quality control (MA QC) or patient-based real-time quality control (PBRTQC). However, there are some controversies regarding PBRTQC which this review aims to address while also indicating the current status of PBRTQC.

CONTENT

This review gives the background of certain newly described optimization and validation methods. It also indicates how QC plans incorporating PBRTQC can be designed for greater effectiveness and/or (cost) efficiency. Furthermore, it discusses controversies regarding the complexity of obtaining PBRTQC settings, the replacement of iQC, and software functionality requirements. Finally, it presents evidence of the added value and practicability of PBRTQC.

OUTLOOK

Recent developments in, and availability of, simulation methods to optimize and validate laboratory-specific PBRTQC procedures have enabled medical laboratories to implement PBRTQC in their daily practice. Furthermore, these methods have made it possible to demonstrate the practicability and added value of PBRTQC by means of two prospective "clinical" studies and other investigations. Although internal QC will remain an essential part of any QC plan, applying PBRTQC can now significantly improve its performance and (cost) efficiency.