A multi-test planning model for risk based statistical quality control strategies

An assessment of analytical performance using the six sigma scale in second-trimester maternal prenatal screening practices in China

Objectives

We aimed to evaluate the analytical performance of second-trimester maternal serum screening in China, and to compare if there are differences in sigma levels across different methods and months.

Methods

A retrospective study was conducted to assess the analytical quality levels of laboratories by calculating the Sigma metrics with prenatal screening biomarkers: AFP, Total β-hCG, free β-hCG, uE3. Data from 591 laboratories were selected. Sigma metrics were computed using the formula: Sigma metrics(σ) = (%TEa - |%Bias|)/%CV. The Friedman test and Mann-Whitney test were used to compare differences across various methods and different months. The Hodges-Lehmann was used for determining 95 % confidence intervals of pseudo-medians.

Results

Only uE3 showed significant monthly variations in sigma calculations. However, around 8 % of laboratories across all four analytes demonstrated sigma levels both above 6 and below 3 in different months. Laboratories utilizing time-resolved fluorescence methods significantly outperformed those using chemiluminescence in sigma level. For AFP, the pseudo-median difference between these methods lies within a 95 % confidence interval of (-3.22, -1.93), while for uE3, it is at (-2.30, -1.40). Notably, the median sigma levels for all analytes reached the 4-sigma threshold, with free β-hCG even attaining the 6-sigma level.

Conclusion

With current standards, China's second-trimester maternal serum screening is of relatively high analytical quality, and variations in sigma levels exist across different months and methods.

The value of Sigma-metrics in laboratory medicine

While Six Sigma is used in different disciplines to improve quality, Tony Badric and Elvar Theodorsson in a recent paper in CCLM have questioned Six Sigma application in medical laboratory concluding Six Sigma has provided no value to medical laboratory. In addition, the authors have expanded their criticism to Total Analytical Error (TAE) model and statistical quality control. To address their arguments, we have explained the basics of TAE model and Six Sigma and have shown the value of Six Sigma to medical laboratory.

QC Constellation: a cutting-edge solution for risk and patient-based quality control in clinical laboratories

OBJECTIVES

Clinical laboratories face limitations in implementing advanced quality control (QC) methods with existing systems. This study aimed to develop a web-based application to addresses this gap, and improve QC practices.

METHODS

QC Constellation, a web application built using Python 3.11, integrates various statistical QC modules. These include Levey-Jennings charts with Westgard rules, sigma-metric calculations, exponentially weighted moving average (EWMA) and cumulative sum (CUSUM) charts, and method decision charts. Additionally, it offers a risk-based QC section and a patient-based QC module aligning with modern QC practices. The codes and the web application links for QC Constellation were shared at https://github.com/hikmetc/QC_Constellation, and http://qcconstellation.com, respectively.

RESULTS

Using synthetic data, QC Constellation demonstrated effective implementation of Levey-Jennings charts with user-friendly features like checkboxes for Westgard rules and customizable moving averages graphs. Sigma-metric calculations for hypothetical performance values of serum total cholesterol were successfully performed using allowable total error and maximum allowable measurement uncertainty goals, and displayed on method decision charts. The utility of the risk-based QC module was exemplified by assessing QC plans for serum total cholesterol, showcasing the application's capability in calculating risk-based QC parameters including maximum unreliable final patient results, risk management index, and maximum run size and offering risk-based QC recommendations. Similarly, the patient-based QC and optimization modules were demonstrated using simulated sodium results.

CONCLUSIONS

In conclusion, QC Constellation emerges as a pivotal tool for laboratory professionals, streamlining the management of quality control and analytical performance monitoring, while enhancing patient safety through optimized QC processes.

A spreadsheet tool for designing statistical quality control programs based on patient risk parameters

BACKGROUND

Quality control (QC) in the laboratory aims to reduce the risk of harm to a patient due to erroneous results, as highlighted by the Clinical Laboratory Standards Institute (CLSI) guidance for Statistical Quality Control (SQC) (C24-Ed4). To effectively reduce patient risk, a convenient spreadsheet tool was developed to assist laboratories in SQC design based on patient risk parameters.

METHODS

In accordance with Parvin's patient risk model and the mathematical formula for calculating the expected number of unreliable final patient results [E(N_uf)], the function is edited using Excel software, and the maximum E(N_uf) [MaxE(N_uf)] value and other risk parameters based on the current QC strategy are calculated to assess the risk of the QC strategy.

RESULTS

A convenient spreadsheet tool is proposed in this study. After the quality requirements, performance parameters, practical run size, QC rules and the number of QC results of test items are input, the laboratory is enabled to quickly obtain MaxE(N_uf) value, maximum run size and other data based on the strategy. The QC strategy conforming to the risk requirements can be developed by changing the QC rules or the quantity of run size. Moreover, the Power Function Graph of the QC strategy and two risk diagrams are presented simultaneously.

CONCLUSIONS

Convenient spreadsheet tools can be adopted by laboratories to assess the risks of QC strategies and design appropriate risk-based SQC strategies to reduce patient risk to acceptable levels.

Risk Analysis for Quality Control Part 2: Theoretical Foundations for Risk Analysis

BACKGROUND

Risk analysis can be used to determine control limits for quality control (QC). The Parvin model is the most commonly used method for risk analysis; however; the Parvin model rests on assumptions that have been shown to produce paradoxical results and to underestimate risk. There is a need for an improved framework for risk analysis.

METHODS

We developed a dynamic model (Markov Reward Model) to analyze the long-term behavior of an assay under the influence of a QC monitoring system. The model is flexible and accounts for different patterns of assay behavior (shift frequency, shift distribution) and the impact of error on patient outcomes. The model determines the distribution of undetected reported errors and the frequency of false-positive laboratory results as a function of QC settings. The model accounts for the competing risks (false detections, shifts in the mean) that cause an assay to move from an in-control state to an out-of-control state.

RESULTS

The model provides a tradeoff curve that expresses the cost to prevent an unacceptable reported result in terms of laboratory cost (false-positive QC). The model can be used to optimize settings of a particular QC method or to compare the performance of different methods.

CONCLUSIONS

We developed a method to evaluate that determines the cost to reduce the risk to patients (reported results with unacceptable errors) in terms of laboratory costs (false-positive QC).

Setting up an own laboratory performance-based internal quality control plan - a model for complete blood count

Quality Control (QC) in Romania is regulated by the Order of the Minister of Health no. 1608/2022 that modifies the previous Order 1301/2007. The new version of the Order introduces a more scientific approach by requesting the laboratories to assess test performance and then elaborate an appropriate internal QC plan. The aim of this study was to demonstrate how to design a QC plan for complete blood count (CBC) in an Emergency Laboratory with continuous activity, in order to comply with the new Order 1608/2022. QC data obtained over a three-month period (April-June 2022) from the Sysmex XN-1000 instrument of the Emergency Laboratory of the County Emergency Clinical Hospital of Târgu Mureș were included. In order to establish an appropriate QC plan, two models were applied and the following parameters were calculated: the number of daily QC runs (N), the probability of false rejection (Pfr), the QC frequency (run size), and the required QC rules. White blood cells achieved high performance, while Hematocrit performance was poor. Different levels of performance were achieved for Platelets. We emphasize that, when all parameters are measured on the same instrument, QC frequency and Pfr should be adjusted in order to develop a QC plan that “fits” all the parameters of the CBC as a whole. In our Emergency Laboratory, the calculated QC plan for CBC is N=2, Pfr=0.03, multi-rule 1:3s/2:2s/R:4s, and a run size of 95 samples which is approximately the same as the number of CBCs performed during one 12-hour shift.

Integrating moving average control procedures into the risk-based quality control plan in small-volume medical laboratories

The modern approach to quality control (QC) in medical laboratories implies the development of a risk-based control plan. This paper aims to develop a risk-based QC plan for a laboratory with a small daily testing volume and to integrate the already optimized moving average (MA) control procedures into this plan. A multistage bracketed QC plan for ten clinical chemistry analytes was made using a Westgard QC frequency calculator. Previously, MA procedures were optimized by the bias detection simulation method. Aspartate aminotransferase, HDL-cholesterol and potassium had patient-risk sigma metrics greater than 6, albumin and cholesterol greater than 5, creatinine, chlorides, calcium and total proteins between 4 and 5, and sodium less than 4. Based on the calculated run sizes and characteristics of optimized MA procedures, for 6 tests, it was possible to replace the monitoring QC procedure with an MA procedure. For the remaining 4 tests, it was necessary to keep the monitoring QC procedure and introduce MA control for added security. This study showed that even in a laboratory with a small volume of daily testing, it is possible to make a risk-based QC plan and integrate MA control procedures into that plan.

Sigma Metrics in Laboratory Medicine: A Call for Harmonization

BACKGROUND AND AIM

Sigma metrics are applied in clinical laboratories to assess the quality of analytical processes. A parameter associated to a Sigma >6 is considered "world class" whereas a Sigma <3 is "poor" or "unacceptable". The aim of this retrospective study was to quantify the impact of different approaches for Sigma metrics calculation.

MATERIAL AND METHODS

Two IQC levels of 20 different parameters were evaluated for a 12-month period. Sigma metrics were calculated using the formula: (allowable total error (TEa) (%) - bias (%))/(coefficient of variation (CV) (%)). Method precision was calculated monthly or annually. The bias was obtained from peer comparison program (PCP) or external quality assessment program (EQAP), and 9 different TEa sources were included.

RESULTS

There was a substantial monthly variation of Sigma metrics for all combinations, with a median variation of 32% (IQR, 25.6-41.3%). Variation across multiple analyzers and IQC levels were also observed. Furthermore, TEa source had the highest impact on Sigma calculation with proportions of Sigma >6 ranging from 17.5% to 84.4%. The nature of bias was less decisive.

CONCLUSION

In absence of a clear consensus, we recommend that laboratories calculate Sigma metrics on a sufficiently long period of time (>6 months) and carefully evaluate the choice of TEa source.