Next-Generation Patient-Based Real-Time Quality Control Models

Patient-based real-time QC (PBRTQC) uses patient-derived data to assess assay performance. PBRTQC algorithms have advanced in parallel with developments in computer science and the increased availability of more powerful computers. The uptake of Artificial Intelligence in PBRTQC has been rapid, with many stated advantages over conventional approaches. However, until this review, there has been no critical comparison of these. The PBRTQC algorithms based on moving averages, regression-adjusted real-time QC, neural networks and anomaly detection are described and contrasted. As Artificial Intelligence tools become more available to laboratories, user-friendly and computationally efficient, the major disadvantages, such as complexity and the need for high computing resources, are reduced and become attractive to implement in PBRTQC applications.

Establishment and evaluation of Voting algorithm-based internal quality control (ViQC), a patient-based real-time quality control

BACKGROUND

Patient-Based Real-Time Quality Control (PBRTQC) has emerged as a supplementary programme to traditional internal quality control (iQC) mechanisms. Despite its growing popularity, practical applications in clinical settings reveal several challenges. The primary objective of this research is to introduce and develop an Artificial Intelligence (AI)-based method, named Voting algorithm based iQC (ViQC), designed to enhance the precision and reliability of existing PBRTQC systems.

METHODS

In this study, we conducted a retrospective analysis of 111,925 inpatient serum glucose test results from Nanjing Drum Tower Hospital, Nanjing, China, to provide an unbiased data set. The Voting iQC (ViQC) algorithm, established by the principles of the Voting algorithm, was then developed. Its analytical performance was evaluated through the calculation of random errors (RE). Subsequently, its clinical efficacy was assessed by comparison with five statistical algorithms: Moving Average (MA), Exponentially Weighted Moving Average (EWMA), Moving Median (movMed, MM), Moving Quartile (MQ), and Moving Standard Deviation (MovSD).

RESULTS

The ViQC model incorporates a variety of machine learning models, including logistic regression, Bayesian methods, K-Nearest Neighbor, decision trees, random forests, and gradient boosting decision trees, to establish a robust predictive framework. This model consistently maintains a false positive rate below 0.002 across all six evaluated error factors, showcasing exceptional precision. Notably, its performance further excels with an error factor of 3.0, where the false positive rate drops below 0.001, and achieves an accuracy rate as high as 0.965 at an error factor of 2.0. The classification effectiveness of ViQC model is evaluated by an area under the curve (AUC) exceeding 0.97 for all error factors. In comparison to five conventional PBRTQC statistical methods, ViQC significantly enhances error detection efficiency, maximum reducing the trimmed average number of patient samples required for detecting errors from 724 to 168, thereby affirming its superior error detection capability.

CONCLUSION

The new established PBRTQC using artificial intelligence yielded satisfactory performance compared to the traditional PBBTQC in real world setting.

Circulating 25-hydroxyvitamin D concentration can predict bowel resection risk among individuals with inflammatory bowel disease in a longitudinal cohort with 13 years of follow-up

BACKGROUND

Although the beneficial properties of vitamin D in anti-inflammation and immunity-modulation are promising in the management of inflammatory bowel disease (IBD), data were limited for the critical IBD prognosis. The association between serum vitamin D levels and the risk of bowel resection in individuals with IBD remains largely unknown.

MATERIALS AND METHODS

The authors performed a longitudinal cohort study among 5474 individuals with IBD in the UK Biobank. Serum 25-hydroxyvitamin D [25(OH)D] was measured using direct competitive chemiluminescent immunoassay. Bowel resection events were ascertained via national inpatient data. Multivariable-adjusted Cox proportional hazard regression was used to examine the association between serum 25(OH)D and bowel resection risk, presented with hazard ratios (HRs) and 95% CIs. Restricted cubic spline (RCS) was used to evaluate dose-response associations.

RESULTS

During a mean follow-up of 13.1 years, the authors documented 513 incident bowel resection cases. Compared to participants with vitamin D deficiency, nondeficient participants showed a significantly reduced bowel resection risk in IBD (HR 0.72, 95% CI: 0.59-0.87, P =0.001), Crohn's disease (CD, HR 0.74, 95% CI: 0.56-0.98, P =0.038), and ulcerative colitis (UC, HR 0.73, 95% CI: 0.57-0.95, P =0.020). When comparing extreme quintiles of 25(OH)D level, participants with IBD showed a 34% reduced risk of bowel resection (95% CI: 11-51%, P =0.007) and participants with UC showed a 46% reduced risk (95% CI: 19-64%, P =0.003), while this association was not significant in CD (HR 0.93, 95% CI: 0.59-1.45, P =0.740). Linear dose-response associations were observed using the RCS curve (all P -nonlinearity>0.05).

CONCLUSION

Increased serum level of 25(OH)D is independently associated with reduced bowel resection risk in IBD. This association was significant in UC but may not be stable in CD. Vitamin D deficiency is a risk factor for bowel resection in individuals with IBD, and may be an effective metric in predicting and risk-screening surgical events.

QC Constellation: a cutting-edge solution for risk and patient-based quality control in clinical laboratories

OBJECTIVES

Clinical laboratories face limitations in implementing advanced quality control (QC) methods with existing systems. This study aimed to develop a web-based application to addresses this gap, and improve QC practices.

METHODS

QC Constellation, a web application built using Python 3.11, integrates various statistical QC modules. These include Levey-Jennings charts with Westgard rules, sigma-metric calculations, exponentially weighted moving average (EWMA) and cumulative sum (CUSUM) charts, and method decision charts. Additionally, it offers a risk-based QC section and a patient-based QC module aligning with modern QC practices. The codes and the web application links for QC Constellation were shared at https://github.com/hikmetc/QC_Constellation, and http://qcconstellation.com, respectively.

RESULTS

Using synthetic data, QC Constellation demonstrated effective implementation of Levey-Jennings charts with user-friendly features like checkboxes for Westgard rules and customizable moving averages graphs. Sigma-metric calculations for hypothetical performance values of serum total cholesterol were successfully performed using allowable total error and maximum allowable measurement uncertainty goals, and displayed on method decision charts. The utility of the risk-based QC module was exemplified by assessing QC plans for serum total cholesterol, showcasing the application's capability in calculating risk-based QC parameters including maximum unreliable final patient results, risk management index, and maximum run size and offering risk-based QC recommendations. Similarly, the patient-based QC and optimization modules were demonstrated using simulated sodium results.

CONCLUSIONS

In conclusion, QC Constellation emerges as a pivotal tool for laboratory professionals, streamlining the management of quality control and analytical performance monitoring, while enhancing patient safety through optimized QC processes.

Time-of-day dependence of running averages for some components of metabolic panels at our hospital: Relation to strategy for patient-based quality control

BACKGROUND AND OBJECTIVES

We assessed properties of running averages for our hospital's most common chemistry analytes, for use in real-time patient-based quality control (PBQC). We determined whether there was dependence of any running averages on 24-h clock time (time-of-day, TOD).

MATERIALS AND METHODS

We analyzed 3-months' data for measurements of 13 metabolic panel components. Running averages for 20 consecutive results (20-mers) were computed for data restricted to results within reference intervals. This produced an overall mean (X) and standard-deviation (SD) of 20-mers for each analyte. We then computed the average 20-mer result (Y) reported within 1-h bins across 24-hour clock time (t). Y(t) was regarded as having TOD-dependence if either nadir or apex values for |Y-X| exceeded 0.5 SD, occurring within a contiguous series of at least 4 Y(t) values on one side of the mean.

RESULTS

Seven analytes (albumin, aspartate aminotransferase, calcium, chloride, CO2, potassium, total protein) demonstrated TOD-dependence of running means for 20-mers.

CONCLUSIONS

At our hospital, TOD-dependence of running means was identified for 7 of 13 metabolic panel analytes. TOD-dependence is likely to be hospital-specific. Utilization of TOD-dependent targets for PBQC, rather than fixed targets, would be appropriate in these cases.

Integrating Patient-Based Real-Time Quality Control (PBRTQC) in a New Field: Inter-Comparison between Biochemical Instrumentations with LDL-C

BACKGROUND

Patient-based real-time quality control (PBRTQC) can be a valuable tool in clinical laboratories due to its cost-effectiveness and constant monitoring. More focus is placed on discovering and improving algorithms that compliment conventional internal control techniques. The practical implementation of PBRTQC with a biochemical instrument comparison is lacking. We aim to evaluate PBRTQC's efficacy and practicality by comparing low-density lipoprotein cholesterol (LDL-C) test results to ensure consistent real-time monitoring across biochemical instrumentations in clinical laboratories.

METHOD

From 1 September 2021 to 30 August 2022, the First Affiliated Hospital of Xi'an Jiaotong University collected data from 158,259 both healthy and diseased patients, including 84,187 male and 74,072 female patients, and examined their LDL-C results. This dataset encompassed a group comprising 50,556 individuals undergoing health examinations, a group comprising 42,472 inpatients (IP), and a group comprising 75,490 outpatients (OP) for the PBRTQC intelligent monitoring platform to conduct daily tests, parameter configuration, program development, real-time execution, and performance validation of the patients' data. Moreover 40 patients' LDL-C levels were assessed using two biochemical analyzers, designated as the reference and comparator instruments. A total of 160 LDL-C results were obtained from 40 both healthy and diseased patients, including 14 OP, 16 IP, and 10 health examination attendees, who were selected to represent LDL-C levels broadly. Two biochemical instruments measured LDL-C measurements from the same individuals to investigate consistency and reproducibility across patient statuses and settings. We employed exponentially weighted moving average (EWMA) and moving median (MM) methods to calculate inter-instrument bias and ensure analytical accuracy. Inter-instrument bias for LDL-C measurements was determined by analyzing fresh serum samples, different concentrations of quality control (QC), and commercialized calibrators, employing both EWMA and MM within two assay systems. The assessment of inter-instrumental bias with five different methods adhered to the external quality assessment standards of the Clinical Laboratory Center of the Health Planning Commission, which mandates a bias within ±15.0%.

RESULT

We calculated inter-instrument comparison bias with each of the five methods based on patient big data. The comparison of fresh serum samples, different concentrations of QC, commercialized calibrators, and EWMA were all in the permissive range, except for MM. MM showed that the bias between two biochemical instruments in the concentration ranges of 1.5 mmoL/L-6.2 mmoL/L exceeded the permissible range. This was mainly due to the small number of specimens, affected by variations among individual patients, leading to increased false alarms and reduced effectiveness in monitoring the consistency of the inter-instrumental results. Moreover, the inter-comparison bias derived from EWMA was less than 3.01%, meeting the 15% range assessment criteria. The bias result for MM was lower than 24.66%, which was much higher than EWMA. Thus, EWMA is better than MM for monitoring inter-instrument comparability. PBRTQC can complement the use of inter-comparison bias between biochemical analyzers. EWMA has comparable inter-instrument comparability monitoring efficacy.

CONCLUSIONS

The utilization of AI-based PBRTQC enables the automated real-time comparison of test results across different biochemical instruments, leading to a reduction in laboratory operating costs, enhanced work efficiency, and improved QC. This advanced technology facilitates seamless data integration and analysis, ultimately contributing to a more streamlined and efficient laboratory workflow in the biomedical field.

The effect of ratios upon improving patient-based real-time quality control (PBRTQC) performance

OBJECTIVES

Large biological variation hinders application of patient-based real-time quality control (PBRTQC). The effect of analyte ratios on the ability of PBRTQC to improve error detection was investigated.

METHODS

Four single analyte-ratio pairs (alanine aminotransferase [ALT] vs. ALT to aspartate aminotransferase ratio [RALT]; creatinine [Cr] vs. Cr to cystatin C ratio [RCr]; lactate dehydrogenase [LDH] vs. LDH to hydroxybutyrate dehydrogenase ratio [RLDH]; total bilirubin [TB] vs. TB to direct bilirubin ratio [RTB]) were chosen for comparison. Various procedures, including four conventional algorithms (moving average [MA], moving median [MM], exponentially weighted moving average [EWMA] and moving standard deviation [MSD]) were assessed. A new algorithm that monitors the number of defect reports per analytical run (NDR) was also evaluated.

RESULTS

When a single analyte and calculated ratio used the same PBRTQC parameters, fewer samples were needed to detect systematic errors (SE) by taking ratios (p<0.05). Application of ratios in MA, MM and EWMA significantly enhanced their ability to detect SE. The influence of ratio on random error (RE) detection depended upon the analytes and PBRTQC parameters, as consistent advantage was not demonstrated. The NDR method performed well when appropriate parameters were used, but was only effective for unilateral SE. Rearrangement of sample order led to a significant deterioration of conventional algorithms' performance, while NDR remained almost unaffected.

CONCLUSIONS

For analytes with large variation and poor PBRTQC performance, using ratios as PBRTQC indexes may significantly improve performance and achieve better anti-interference ability, providing a new class of monitoring indicators for PBRTQC.

Machine learning-based nonlinear regression-adjusted real-time quality control modeling: a multi-center study

OBJECTIVES

Patient-based real-time quality control (PBRTQC), a laboratory tool for monitoring the performance of the testing process, has gained increasing attention in recent years. It has been questioned for its generalizability among analytes, instruments, laboratories, and hospitals in real-world settings. Our purpose was to build a machine learning, nonlinear regression-adjusted, patient-based real-time quality control (mNL-PBRTQC) with wide application.

METHODS

Using computer simulation, artificial biases were added to patient population data of 10 measurands. An mNL-PBRTQC was created using eight hospital laboratory databases as a training set and validated by three other hospitals' independent patient datasets. Three different Patient-based models were compared on these datasets, the IFCC PBRTQC model, linear regression-adjusted real-time quality control (L-RARTQC), and the mNL-PBRTQC model.

RESULTS

Our study showed that in the three independent test data sets, mNL-PBRTQC outperformed the IFCC PBRTQC and L-RARTQC for all measurands and all biases. Using platelets as an example, it was found that for 20 % bias, both positive and negative, the uncertainty of error detection for mNL-PBRTQC was smallest at the median and maximum values.

CONCLUSIONS

mNL-PBRTQC is a robust machine learning framework, allowing accurate error detection, especially for analytes that demonstrate instability and for detecting small biases.

Application of Patient-Based Real-Time Quality Control Based on Artificial Intelligence Monitoring Platform in Continuously Quality Risk Monitoring of Down Syndrome Serum Screening

BACKGROUND

Patient-based real-time quality control (PBRTQC) has gained attention because of its potential to continuously monitor the analytical quality in situations wherein internal quality control (IQC) is less effective. Therefore, we tried to investigate the application of PBRTQC method based on an artificial intelligence monitoring (AI-MA) platform in quality risk monitoring of Down syndrome (DS) serum screening.

METHODS

The DS serum screening item determination data and relative IQC data from January 4 to September 7 in 2021 were collected. Then, PBRTQC exponentially weighted moving average (EWMA) and moving average (MA) procedures were built and optimized in the AI-MA platform. The efficiency of the EWMA and MA procedures with intelligent and traditional control rules were compared. Next, the optimal EWMA procedures that contributed to the quality assurance of serum screening were run and generated early warning cases were investigated.

RESULTS

Optimal EWMA and MA procedures on the AI-MA platform were built. Comparison results showed the EWMA procedure with intelligent QC rules but not traditional quality rules contained the best efficiency. Based on the AI-MA platform, two early warning cases were generated by using the optimal EWMA procedure, which finally found were caused by instrument failure. Moreover, the EWMA procedure could truly reflect the detection accuracy and quality in situations wherein traditional IQC products were unstable or concentrations were inappropriate.

CONCLUSIONS

The EWMA procedure built by the AI-MA platform could be a good complementary control tool for the DS serum screening by truly and timely reflecting the detection quality risks.

Exploring optimization algorithms for establishing patient-based real-time quality control models

BACKGROUND

Patient-based real-time quality control (PBRTQC) models must be optimized for use in different clinical laboratories, but the grid search (GS) algorithm explored in recent studies for this purpose is inefficient. Thus, finding an efficient optimization algorithm is critical for future research and implementation of the PBRTQC.

METHODS

We compared the efficiency and performance of five commonly used optimization algorithms, including GS, simulated annealing (SA), genetic algorithms (GA), differential evolution (DE), and particle swarm optimization (PSO), to optimize conventional PBRTQC and regression-adjusted real-time quality control (RARTQC) models for serum alanine aminotransferase and sodium.

RESULTS

The GS, GA, DE, and PSO provided models with similar performances. However, GA and DE required significantly less computation time than GS. The results also demonstrate a general tradeoff between the optimization method's chance of discovering the optimum and the computation time required.

CONCLUSION

More efficient optimization methods should be adopted when establishing PBRTQC or RARTQC models to save time and computing power that will enable the development of more complex models and increase the scalability of extensive PBRTQC applications.

Advances in clinical chemistry patient-based real-time quality control (PBRTQC)

Patient-Based Real-Time Quality Control involves monitoring an assay using patient samples rather than external material. If the patient population does not change, then a shift in the long-term assay population results represents the introduction of a change in the assay. The advantages of this approach are that the sample(s) are commutable, it is inexpensive, the rules are simple to interpret and there is virtually continuous monitoring of the assay. The disadvantages are that the laboratory needs to understand their patient population and how they may change during the day, week or year and the initial change of mindset required to adopt the system. The concept is not new, having been used since the 1960s and widely adopted on hematology analyzers in the mid-1970s. It was not widely used in clinical chemistry as there were other stable quality control materials available. However, the limitations of conventional quality control approaches have become more evident. There is a greater understanding of how to collect and use patient data in real time and a range of powerful algorithms which can identify changes in assays. There are more assays on more samples being run. There is also a greater interest in providing a theoretical basis for the validation and integration of these techniques into routine practice.

Advances in internal quality control

Quality control practices in the modern laboratory are the result of significant advances over the many years of the profession. Major advance in conventional internal quality control has undergone a philosophical shift from a focus solely on the statistical assessment of the probability of error identification to more recent thinking on the capability of the measurement procedure (e.g. sigma metrics), and most recently, the risk of harm to the patient (the probability of patient results being affected by an error or the number of patient results with unacceptable analytical quality). Nonetheless, conventional internal quality control strategies still face significant limitations, such as the lack of (proven) commutability of the material with patient samples, the frequency of episodic testing, and the impact of operational and financial costs, that cannot be overcome by statistical advances. In contrast, patient-based quality control has seen significant developments including algorithms that improve the detection of specific errors, parameter optimization approaches, systematic validation protocols, and advanced algorithms that require very low numbers of patient results while retaining sensitive error detection. Patient-based quality control will continue to improve with the development of new algorithms that reduce biological noise and improve analytical error detection. Patient-based quality control provides continuous and commutable information about the measurement procedure that cannot be easily replicated by conventional internal quality control. Most importantly, the use of patient-based quality control helps laboratories to improve their appreciation of the clinical impact of the laboratory results produced, bringing them closer to the patients.Laboratories are encouraged to implement patient-based quality control processes to overcome the limitations of conventional internal quality control practices. Regulatory changes to recognize the capability of patient-based quality approaches, as well as laboratory informatics advances, are required for this tool to be adopted more widely.

Patient-based pre-classified real-time quality control (PCRTQC)

BACKGROUND

Patient-based real-time quality control (PBRTQC) has gained increasing attention in clinical laboratory management. Although its valuable characteristics complement traditional quality control measures, its performance and practical application have faced scrutiny. In this study, patient-based pre-classified real-time quality control (PCRTQC), an extended approach was devised to enhance real-time quality control protocols.

METHODS

PCRTQC distinguishes itself by incorporating an additional patient pre-classification step utilising the OPTICS algorithm, thus addressing interference from diverse patient types. The complete set of patient test results obtained from a clinical chemistry analyser at The First Hospital of China Medical University in 2021 was utilised. Constant error (CE) and proportional error (PE) were introduced as analytical errors. Four analytes were selected to evaluate the PCRTQC, measuring probability for false rejection (Pfr) and the average number of patient samples until error detection (ANPed). Relevant error detection charts were generated.

RESULTS

The PCRTQC outperformed regression-adjusted real-time quality control (RARTQC) based on the ANPed by approximately 50% for both the CE and PE, compared to the RARTQC, particularly for the total allowable error threshold.

CONCLUSION

The pre-classification step effectively reduced inter-individual variation and improved data preprocessing, filtering, and modelling. The PCRTQC is a robust framework for real-time quality control research.

Delta checks

Delta check is an electronic error detection tool. It compares the difference in sequential results within a patient against a predefined limit, and when exceeded, the delta check rule is considered triggered. The patient results should be withheld for review and troubleshooting before releasing to the clinical team for patient management. Delta check was initially developed as a tool to detect wrong-blood-in-tube (sample misidentification) errors. It is now applied to detect errors more broadly within the total testing process. Recent advancements in the theoretical understanding of delta check has allowed for more precise application of this tool to achieve the desired clinical performance and operational set up. In this Chapter, we review the different pre-implementation considerations, the foundation concepts of delta check, the process of setting up key delta check parameters, performance verification and troubleshooting of a delta check flag.

Developing an evidence-based approach to quality control

Effective Quality Control remains one of the pillars of Clinical Biochemistry. An understanding of the possible analytical errors that may occur, how to detect them efficiently and how to prevent them from causing patient harm are critical components of a Quality System. For some time, there have been questions about the theoretical basis of the models used to describe and detect analytical error. The current theory recognises two types of error, systematic and random and a system based on sampling the analytical process using a synthetic material to detect these errors. However, there are at least two other errors that are present. One is related to the QC material and the other, irregular errors. In this Opinion Piece, some of the underlying assumptions of Quality Control systems are described and analysed.

Technical quality assurance and quality control for medical laboratories: a review and proposal of a new concept to obtain integrated and validated QA/QC plans

Technical quality assurance (QA) and quality control (QA/QC) are important activities within medical laboratories to ensure the adequate quality of obtained test results. QA/QC tools available at medical laboratories include external QC and internal QC, patient-based real-time quality control (PBRTQC) tools such as moving average quality control (MAQC), limit checks, delta checks, and multivariate checks, and finally, analyzer flagging. Recently, for PBRTQC tools, new optimization and validation methods based on error detection simulation have been developed to obtain laboratory-specific insights into PBRTQC error detection. These developments have enabled implementation and application of these individual tools in routine clinical practice. As a next step, they also enable performance comparison of the individual QA/QC tools and integration of all the individual QA/QC tools in order to obtain the most powerful and efficient QA/QC plans. In this review, a brief overview of the individual QA/QC tools and their characteristics is provided and the error detection simulation approaches are explained. Finally, a new concept entitled integrated quality assurance and control (IQAC) is presented. To enable IQAC, a conceptual framework is suggested and demonstrated for sodium, based on available published data. The proposed IQAC framework provides ways and tools by which the performance of different QA/QC tools can be compared in a so-called QA/QC error detection table to enable optimization and validation of the overall QA/QC plan in terms of alarm rate as well as pre-analytical, analytical, and post-analytical error detection performance.

Nonlinear Association between Serum 25-Hydroxyvitamin D and All-Cause Mortality in Adults with Inflammatory Bowel Disease in a Prospective Cohort Study

BACKGROUND

Nonlinear association between serum 25-hydroxyvitamin D [25(OH)D] concentration and all-cause mortality has been widely reported for the general population, but this association has not been quantified for individuals with inflammatory bowel disease (IBD).

OBJECTIVES

The aim was to explore the association between serum 25(OH)D and all-cause mortality in individuals with IBD.

METHODS

We identified 2690 females and 2532 males aged 40-69 y with diagnosed IBD at baseline in the UK Biobank. Serum 25(OH)D concentration was measured by direct competitive chemiluminescent immunoassay. The outcome was all-cause mortality, ascertained via the death registry. Cox proportional hazard regression was used to evaluate associations between serum 25(OH)D in quintiles and all-cause mortality among individuals with IBD [Crohn disease (CD; n = 1760) and ulcerative colitis (UC; n = 3462)]. Restricted cubic splines were used to investigate potential nonlinearity.

RESULTS

During the mean follow-up period of 11.9 y, 529 deaths (198 in CD and 331 in UC) were documented among 5222 individuals with IBD. Compared with the lowest quintile of serum 25(OH)D, HRs for the second to the highest quintiles were 0.82 (95% CI: 0.63, 1.06), 0.63 (95% CI: 0.47, 0.83), 0.64 (95% CI: 0.48, 0.85), and 0.74 (95% CI: 0.55, 0.99), respectively. Nonlinearity was detected in the dose-response association between serum 25(OH)D concentration and all-cause mortality (P-nonlinearity < 0.001), and 25(OH)D concentrations of 44-78 nmol/L were associated with a 50% lower risk of all-cause mortality (than 10 nmol/L). Subgroup analyses showed that the nonlinear association mostly applied to females (P-nonlinearity < 0.001 compared with 0.080 in males).

CONCLUSIONS

We observed a nonlinear association, mostly applicable to females, between serum 25(OH)D concentrations and all-cause mortality among individuals with IBD. A 25(OH)D concentration range of 44-78 nmol/L can serve as a starting point for future research to confirm recommended 25(OH)D concentrations for individuals with IBD.

Assessment of patient based real-time quality control on comparative assays for common clinical analytes

Background

It is critical for laboratories to conduct multianalyzer comparisons as a part of daily routine work to strengthen the quality management of test systems. Here, we explored the application of patient‐based real‐time quality controls (PBRTQCs) on comparative assays to monitor the consistency among clinical laboratories.

Methods

The present study included 11 commonly tested analytes that were detected using three analyzers. PBRTQC procedures were set up with exponentially weighted moving average (EWMA) algorithms and evaluated using the AI‐MA artificial intelligence platform. Comparative assays were carried out on serum samples, and patient data were collected. Patients were divided into total patient (TP), inpatient (IP), and outpatient (OP) groups.

Results

Optimal PBRTQC protocols were evaluated and selected with appropriate truncation limits and smoothing factors. Generally, similar comparative assay performance was achieved using both the EWMA and median methods. Good consistency between the results from patients' data and serum samples was obtained, and unacceptable bias was detected for alkaline phosphatase (ALP) and gamma‐glutamyl transferase (GGT) when using analyzer C. Categorizing patients' data and applying specific groups for comparative assays could significantly improve the performance of PBRTQCs. When monitoring the inter‐ and intraanalyzer stability on a daily basis, EWMA was superior in detecting very small quality‐related changes with lower false‐positive alarms.

Conclusions

We found that PBRTQCs have the potential to efficiently assess multianalyzer comparability. Laboratories should be aware of population variations concerning both analytes and analyzers to build more suitable PBRTQC protocols.

A study on quality control using delta data with machine learning technique

Background

In the big data era, patient-based real-time quality control (PBRTQC), as an emerging quality control (QC) method, is expanding within the clinical laboratory industry. However, the main issue of current PBRTQC methodology is data stability. Our study is aimed to explore a novel protocol for data stability by combining delta data with machine learning (ML) technique to improve the capacity of QC event detection.

Methods

A data set of 423,290 laboratory results from Beijing Chao-yang Hospital 2019 patient results were used as a training set (n = 380960, 90%) and internal validation set (n = 42330, 10%). A further 22,460 results from Beijing Long-fu Hospital 2019 patient results were used as a test set. Three-type data (1) Single-type data processed by truncation limits; (2) delta-type data processed by truncation limits and (3)delta-type data processed by Isolated Forest (IF) algorithm were evaluated with accuracy, sensitivity, NPed, etc., and compared with previously published statistical methods.

Results

The optimal model was based on Random Forest (RF) algorithm by using delta-type data processed by IF algorithm. The model had a better accuracy (0.99), sensitivity (0.99) specificity (0.99) and AUC (0.99) with the dependent test set, surpassing the critical bias of PBRTQC by over 50%. For the LYMPH#, HGB, and PLT, the cumulative MNPed of MLQC were reduced by 95.43%, 97.39%, and 97.97% respectively when compared to the best of the PBRTQC.

Conclusion

Final results indicate that by integrating an innovative ML algorithm with the overall data processing protocol the detection of QC events is improved.

•

A protocol for data processing by using delta data together with machine learning algorithm, enables to improve data stability.

•

After data processing, the performance of QC event prediction surpassed over 50% clinical recognized PBRTQC method, especially for the hard-to-detect error in QC event prediction.

Traceable machine learning real-time quality control based on patient data

Objectives

Patient-based real-time quality control (PBRTQC) has gained attention as an alternative/integrative tool for internal quality control (iQC). However, it is still doubted for its performance and its application in real clinical settings. We aim to generate a newly and easy-to-access patient-based real-time QC by machine learning (ML) traceable to standard reference data with assigned values by National Institute of Metrology of China (NIM), and to compare it with PBRTQC for clinical validity evaluation.

Methods

For five representative biochemistry analytes, 1,195 000 patient testing results each were collected. After data processing, independent training and test sets were divided. Machine learning internal quality control (MLiQC) was set up by Random Forest in ML and was validated by way of both metrology algorithm traceability and 4 PBRTQC methods recommended by IFCC analytical working group.

Results

MLiQC were established. As an example of albumin (ALB) at the critical bias, the uncertainty of MLiQC was 0.14%, which was evaluated by standard reference data produced by NIM. Compared with four optimal PBRTQC methods at critical bias, the average of the number of patient samples from a bias introduced until detected (ANPed) of MLiQC averagely decreased from 600 to 20. The median and 95 quantiles of NPeds (MNPed and 95NPed) of MLiQC were superior to all optimal PBRTQCs above 90% for all test items.

Conclusions

MLiQC is highly superior to PBRTQC and well-suited in real settings. The validation of the model from two aspects of algorithm traceability and clinical effectiveness confirms its satisfactory performance.

A multi-model fusion algorithm as a real-time quality control tool for small shift detection

BACKGROUND

Patient-based real-time quality control (PBRTQC), a complement to traditional QC, may eliminate matrix effect from QC materials, realize real-time monitoring as well as cut costs. However, the accuracy of PBRTQC has not been satisfactory as physicians expect till now. Our aim is to set up a artificial intelligence-based QC for small error detection in real laboratory settings. Taking tPSA as our unique research subject, data extraction, data stimulation, data partition, model construction and evaluation were designed.

METHODS

84241 deidentified results for tPSA were extracted from Laboratory Information System of Aviation General Hospital. The data set was accumulated by way of data simulation. Independent training and test datasets were separated. After three classification models (RF, SVM and DNN) in ML constructed and weighted by information entropy, a multi-model fusion algorithm was generated. Performance of the fusion model was evaluated by comparing with optimal PBRTQC.

RESULTS

For 4 PBRTQC methods, MovSO showed overall better performance for 0.2 μg/L bias and optimal MNPed was equal to 200. For the fusion model, MNPeds were less than 12 for all biases, and ACC surpassed MovSO nearly 100 times. Except for 0.01 μg/L bias, ACC was more than 0.9 for the rest of biases. FPR was apparently lower than MovSO, only 0.2% and 0.1%.

CONCLUSION

The fusion model shows outstanding performance and reduces incorrect and omitting error detection, adaptable for the real settings.

Patient-based real-time quality control for quantitative hepatitis B virus DNA test using moving rate of positive and negative patient results

OBJECTIVES

Patient-based real-time quality control (PBRTQC) has gained increasing attention in the field of laboratory quality management in recent years. However, PBRTQC has not been reported for use in molecular diagnostics. This study introduces PBRTQC to quantitative hepatitis B virus (HBV) DNA test using moving rate (MR) of positive and negative patient results.

METHODS

In contrast to the MR protocols described in other literature, MR protocol for HBV-DNA test has an additional logarithmic transformation and binary conversion steps before using a common statistical process control algorithm, such as the MR. We used all patient test results of HBV-DNA assay from August 2018 to August 2021 at the Second Affiliated Hospital of Guangzhou University of Chinese Medicine, for parameters setting, optimization, and performance validation. The false rejection rate, error detection curves and validation charts were used to assess the MR protocols.

RESULTS

The false rejection rates of two MR protocols were both <0.7%. The optimal block sizes for positive and negative errors in each cut-off value were not the same, so we first proposed a combined protocol that used different block size to detect negative and positive errors. It turned out that the combined protocols outperformed the simple protocols for each cut-off value, especially detecting positive errors.

CONCLUSIONS

The performances of MR protocols using positive or negative patient results to detect constant errors of HBV-DNA test could meet laboratory requirements. Therefore, we have provided an effective alternative tool for internal quality control in the field of molecular diagnostics.

Optimization of patient-based real-time quality control based on the Youden index

AIM

This study sets out to investigate the utility of exponentially weighted moving average (EWMA) as patient-based real-time quality control (PBRTQC) by conducting a simulation study and subsequent real-patient data implementation to determine optimal EWMA features (weighting factors, control limits, and truncation methods) based on the Youden index.

METHODS

A simulation experiment was conducted in the first stage to investigate optimal EWMA features for the tests, including aspartate aminotransferase, blood urea nitrogen, and glucose, calcium, creatinine, potassium, sodium, triglycerides, thyroid - stimulating hormone (TSH), and vitamin B12 tests. In the second stage of the study, EWMA was applied to real patient data to elucidate practical utility and achieve final optimal EWMA features. Different degrees of systematic errors (SE) including total allowable error (TEa) as a maximum error level were added to both simulation and patient results, and then the EWMA performance was assessed for different EWMA features. We calculated Youden's index for each combination of EWMA features to find their optimal features to achieve minimum false positive rate (FPR) and maximum error detection rate at the SE level corresponding to TEa.

RESULTS

EWMA implementation on real patient data revealed optimal EWMA features for each test. FPR values of creatinine and glucose were 18.48% and 10.17%, respectively, which exceeded the acceptable criteria for FPR (10%). The remaining six analytes showed acceptable FPR.

CONCLUSIONS

We showed the implementation of EWMA as PBRTQC, and optimization of its features based on the Youden index by conducting extensive performance evaluations and simulations in this study.

Association between urinary thallium exposure and cardiovascular disease in U.S. adult population

Exposure to environmental metals, especially heavy metals, can damage human health. However, the association between metals and the prevalence of cardiovascular disease (CVD) remains controversial. The objective of this study was to determine the relationship of urinary metals to CVD in a general population of U.S. adults. We analyzed the cross-sectional data from 6867 adult (age ≥20 years) participants with 12 urinary metals in the National Health and Nutrition Examination Survey (NHANES) (2011-2016). Multivariate logistic regression and restricted cubic spline (RCS) regression were conducted to explore the association between urinary metals and CVD outcomes. Sensitivity analyses were performed to test the robustness of the results. Compared to the lowest quartile, the odds ratios with 95% confidence intervals for CVD across the quartiles were 0.73 (0.38, 1.42), 0.58 (0.42, 0.81), and 0.71 (0.59, 0.84) for urinary thallium (U-Tl) (P for trend <0.001). RCS plot showed the nonlinear association between log₂-transformed U-Tl levels and CVD (P for nonlinearity = 0.001). Sensitivity analyses confirmed the robustness. Higher concentrations of urinary cobalt, manganese and tungsten were associated with an increased risk of CVD. In summary, the large sample data suggests U-Tl is nonlinearly and negatively associated with the prevalence of CVD in the U.S. general adults with low exposure levels. Considering the shortcomings of cross-sectional study design, further studies are warranted to verify our results and to clarify the potential mechanisms.

A study of the moving rate of positive results for use in a patient-based real-time quality control program on a procalcitonin point-of-care testing analyzer

Objective

To establish an applicable and highly sensitive patient‐based real‐time quality control (PBRTQC) program based on a data model constructed with patients’ results of a procalcitonin point‐of‐care testing (POCT) analyzer.

Methods

Patients’ results were retrospectively collected within one year. The Excel software was used to establish quality control (QC) programs of the moving average (MA) and the moving rate of positive results (MR). A Monte Carlo simulation was used to introduce positive and negative biases between 0.01 and 1 ng/ml at random points of the testing data set. Different parameters were used to detect the biases, and the detection efficiency was expressed using the median number of patient samples affected until error detection (MNPed). After comparing the MNPeds of different programs, MA and MR programs with appropriate parameters were selected, and validation plots were generated using MNPeds and maximum number of the patient samples affected (MAX). β curves were generated using the power function of the programs, the performances were compared with that of the conventional QC program.

Results

Neither the conventional QC nor MA program was sensitive to small bias, While MR program can detect the minimum positive bias of 0.06 ng/ml and negative of 0.4 ng/ml at an average daily run size of 10 specimens, with FRs < 1.0%, βs < 1%.

Conclusion

The MR program, which is more sensitive to small biases than conventional QC and MA programs, with low FR and β. As such, it can be used as a PBRTQC program with high performance.

Comparison and optimization of various moving patient-based real-time quality control procedures for serum sodium

Background

Patient‐based real‐time quality control (PBRTQC) is a valuable tool for monitoring the performance of testing processes. We aimed to compare and optimize various PBRTQC procedures for serum sodium.

Methods

In a computer simulation, artificial errors were added to 680,000 real patients’ results. The characteristics of error detection of various algorithms—moving average, moving median, moving SD and moving proportion of normal results including different control limits (CLs)—were assessed on their ability to detect critical errors early.

Results

The moving average and moving median were sensitive to system error, and the moving SD tended to detect random error. P_3SD (moving proportion of normal results, CLs based on mean and SD of proportion of normal results) demonstrated excellent performance for both system error and random error. The increase of block sizes (N) leads to the delay of error detection and the decrease of false rejection, except for QC procedures with minimum and maximum as CLs. CLs calculation with “0.1% false alarm rate” had more effective performance than that set false alarm to zero (minimum and maximum as CLs). The impact of truncation on QC performance depended on truncation limits, algorithms and the types of error. The significant improvement in QC performance due to truncation was only found in moving SD.

Conclusion

“P_3SD,N = 50, without truncation” and “moving SD, N = 25, set 0.1% false alarm as CLs and set 1% outliers exclusion as truncation limits” were recommended as the optimized procedures for serum sodium to monitor system error and random error, respectively.

Regression-Adjusted Real-Time Quality Control

BACKGROUND

Patient-based real-time quality control (PBRTQC) has gained increasing attention in the field of clinical laboratory management in recent years. Despite the many upsides that PBRTQC brings to the laboratory management system, it has been questioned for its performance and practical applicability for some analytes. This study introduces an extended method, regression-adjusted real-time quality control (RARTQC), to improve the performance of real-time quality control protocols.

METHODS

In contrast to the PBRTQC, RARTQC has an additional regression adjustment step before using a common statistical process control algorithm, such as the moving average, to decide whether an analytical error exists. We used all patient test results of 4 analytes in 2019 from Zhongshan Hospital, Fudan University, to compare the performance of the 2 frameworks. Three types of analytical error were added in the study to compare the performance of PBRTQC and RARTQC protocols: constant, random, and proportional errors. The false alarm rate and error detection charts were used to assess the protocols.

RESULTS

The study showed that RARTQC outperformed PBRTQC. RARTQC, compared with the PBRTQC, improved the trimmed average number of patients affected before detection (tANPed) at total allowable error by about 50% for both constant and proportional errors.

CONCLUSIONS

The regression step in the RARTQC framework removes autocorrelation in the test results, allows researchers to add additional variables, and improves data transformation. RARTQC is a powerful framework for real-time quality control research.

Average of Patient Deltas: Patient-Based Quality Control Utilizing the Mean Within-Patient Analyte Variation

BACKGROUND

Because traditional QC is discontinuous, laboratories use additional strategies to detect systematic error. One strategy, the delta check, is best suited to detect large systematic error. The moving average (MA) monitors the mean patient analyte value but cannot equitably detect systematic error in skewed distributions. Our study combines delta check and MA to develop an average of deltas (AoD) strategy that monitors the mean delta of consecutive, intrapatient results.

METHODS

Arrays of the differences (delta) between paired patient results collected within 20-28 h of each other were generated from historical data. AoD protocols were developed using a simulated annealing algorithm in MatLab (Mathworks) to select the number of patient delta values to average and truncation limits to eliminate large deltas. We simulated systematic error by adding bias to arrays for plasma albumin, alanine aminotransferase, alkaline phosphatase, amylase, aspartate aminotransferase, bicarbonate, bilirubin (total and direct), calcium, chloride, creatinine, lipase, sodium, phosphorus, potassium, total protein, and magnesium. The average number of deltas to detection (ANDED) was then calculated in response to induced systematic error.

RESULTS

ANDED varied by combination of assay and AoD protocol. Errors in albumin, lipase, and total protein were detected with a mean of 6 delta pairs. The highest ANDED was calcium, with a positive 0.6-mg/dL shift detected with an ANDED of 75. However, a negative 0.6-mg/dL calcium shift was detected with an ANDED of 25.

CONCLUSIONS

AoD detects systematic error with relatively few paired patient samples and is a patient-based QC technique that will enhance error detection.

Impact of combining data from multiple instruments on performance of patient-based real-time quality control

Introduction

It is unclear what is the best strategy for applying patient-based real-time quality control (PBRTQC) algorithm in the presence of multiple instruments. This simulation study compared the error detection capability of applying PBRTQC algorithms for instruments individually and in combination using serum sodium as an example.

Materials and methods

Four sets of random serum sodium measurements were generated with differing means and standard deviations to represent four simulated instruments. Moving median with winsorization was selected as the PBRTQC algorithm. The PBRTQC parameters (block size and control limits) were optimized and applied to the four simulated laboratory data sets individually and in combination.

Results

When the PBRTQC algorithm were individually optimized and applied to the data of the individual simulated instruments, it was able to detect bias several folds faster than when they were combined. Similarly, the individually applied algorithms had perfect error detection rates across different magnitudes of bias, whereas the error detection rates of the algorithm applied on the combined data missed smaller biases. The performance of the individually applied PBRTQC algorithm performed more consistently among the simulated instruments compared to when the data were combined.

Discussion

While combining data from different instruments can increase the data stream and hence, increase the speed of error detection, it may widen the control limits and compromising the probability of error detection. The presence of multiple instruments in the data stream may dilute the effect of the error when it only affects a selected instrument.

Benefits, limitations, and controversies on patient-based real-time quality control (PBRTQC) and the evidence behind the practice

BACKGROUND

In recent years, there has been renewed interest in the "old" average of normals concept, now generally referred to as moving average quality control (MA QC) or patient-based real-time quality control (PBRTQC). However, there are some controversies regarding PBRTQC which this review aims to address while also indicating the current status of PBRTQC.

CONTENT

This review gives the background of certain newly described optimization and validation methods. It also indicates how QC plans incorporating PBRTQC can be designed for greater effectiveness and/or (cost) efficiency. Furthermore, it discusses controversies regarding the complexity of obtaining PBRTQC settings, the replacement of iQC, and software functionality requirements. Finally, it presents evidence of the added value and practicability of PBRTQC.

OUTLOOK

Recent developments in, and availability of, simulation methods to optimize and validate laboratory-specific PBRTQC procedures have enabled medical laboratories to implement PBRTQC in their daily practice. Furthermore, these methods have made it possible to demonstrate the practicability and added value of PBRTQC by means of two prospective "clinical" studies and other investigations. Although internal QC will remain an essential part of any QC plan, applying PBRTQC can now significantly improve its performance and (cost) efficiency.