Sigma metrics for assessing the analytical quality of clinical chemistry assays: a comparison of two approaches: Electronic supplementary material available online for this article

Analysis of biochemical analytes using six sigma metrics with two analyzers at an Indian lab setting

A zero defects goal was implemented in the clinical laboratory settings using a six-sigma model. Daily Internal Quality Control (IQC) and external quality control data from April-September 2023 was extracted to calculate the sigma metrics of 21 biochemical analytes based on Total Error Allowable (TEa), % bias and co-efficient of variation percent (CV%). A retrospective comparative study was conducted in the department of Clinical Biochemistry at Kanva Diagnostic Services Pvt. Ltd, Bengaluru, India. The analytical performance of the 21 biochemical analytes was tested on Cobas 6000 and C311 analyzers. Quality Goal Index (QGI) and root cause analysis was calculated to infer the reason for the deviation of six sigma. Method decision charts were plotted to show the comparison of the problem analytes on both the analyzers. On Cobas 6000 at level 1 IQC, out of 21 analytes, 10 analytes showed σ>6 and 10 analytes showed σ 3-6 and on C311, 15 analytes which showed σ>6 and 6 analytes that showed σ 3-6. On Cobas 6000 at level 2 IQC, out of 21 analytes, 12 analytes showed σ>6 and 8 analytes showed σ 3-6 and on C311 17 analytes showed σ>6 and 4 analytes showed σ 3-6. Creatinine failed to meet minimal sigma performance at both levels of IQC on Cobas 6000.

Quality analysis of the clinical laboratory literature and its effectiveness on clinical quality improvement: a systematic review

Quality improvement in clinical laboratories is crucial to ensure accurate and reliable test results. With increasing awareness of the potential adverse effects of errors in laboratory practice on patient outcomes, the need for continual improvement of laboratory services cannot be overemphasized. A literature search was conducted on PubMed and a web of science core collection between October and February 2021 to evaluate the scientific literature quality of clinical laboratory quality improvement; only peer-reviewed articles written in English that met quality improvement criteria were included. A structured template was used to extract data, and the papers were rated on a scale of 0-16 using the Quality Improvement Minimum Quality Criteria Set (QI-MQCS). Out of 776 studies, 726 were evaluated for clinical laboratory literature quality analysis. Studies were analyzed according to the quality improvement and control methods and interventions, such as training, education, task force, and observation. Results showed that the average score of QI-MQCS for quality improvement papers from 1981-2000 was 2.5, while from 2001-2020, it was 6.8, indicating continuous high-quality improvement in the clinical laboratory sector. However, there is still room to establish a proper system to judge the quality of clinical laboratory literature and improve accreditation programs within the sector.

Comparison of Sigma metrics computed by three bias estimation approaches for 33 chemistry and 26 immunoassay analytes

Objectives

Sigma metric can be calculated using a simple equation. However, there are multiple sources for the elements in the equation that may produce different Sigma values. This study aimed to investigate the importance of different bias estimation approaches for Sigma metric calculation.

Methods

Sigma metrics were computed for 33 chemistry and 26 immunoassay analytes on the Roche Cobas 6000 analyzer. Bias was estimated by three approaches: (1) averaging the monthly bias values obtained from the external quality assurance (EQA) studies; (2) calculating the bias values from the regression equation derived from the EQA data; and (3) averaging the monthly bias values from the internal quality control (IQC) events. Sigma metrics were separately calculated for the two levels of the IQC samples using three bias estimation approaches. The resulting Sigma values were classified into five categories considering Westgard Sigma Rules as ≥6, <6 and ≥5, <5 and ≥4, <4 and ≥3, and <3.

Results

When classifying Sigma metrics estimated by three bias estimation approaches for each assay, 16 chemistry assays at the IQC level 1 and 2 were observed to fall into different Sigma categories under at least one bias estimation approach. Similarly, for 12 immunoassays at the IQC level 1 and 2, Sigma category was different depending on bias estimation approach.

Conclusions

Sigma metrics may differ depending on bias estimation approaches. This should be considered when using Six Sigma for assessing analytical performance or scheduling the IQC events.

Comparación de la métrica Sigma calculada con tres métodos de estimación del sesgo en 33 magnitudes químicas y 26 de inmunoensayo

Objetivos

Aunque la métrica Sigma se puede calcular mediante una sencilla ecuación, la diversidad de fuentes de las que se extraen los elementos de la ecuación pueden arrojar diferentes valores Sigma. El objetivo de este estudio era investigar la importancia de las distintas estrategias de estimación del sesgo para el cálculo de la métrica Sigma.

Métodos

Se calculó la métrica Sigma de 33 magnitudes químicas y 26 magnitudes de inmunoensayo en un analizador Roche Cobas 6,000. El sesgo se calculó mediante tres métodos: a) calculando la media del sesgo mensual obtenida en los estudios de control de calidad externo (EQA, por sus siglas en inglés); 2) calculando los valores de sesgo mediante una ecuación de regresión a partir de datos obtenidos del EQA; y 3) calculando la media de los valores de sesgo mensual de los eventos de control de calidad internos (IQC, por sus siglas en inglés). Se realizó una métrica Sigma para cada uno de los dos niveles de muestras de IQC empleando tres métodos para calcular el sesgo. Los valores Sigma obtenidos se clasificaron en cinco categorías, en función de las reglas Sigma de Westgard, siendo ≥6, <6 y ≥5, <5 y ≥4, <4 y ≥3, y <3.

Resultados

Al clasificar la métrica Sigma, calculada aplicando tres métodos de estimación del sesgo para cada magnitud, se observó que 16 magnitudes químicas en los niveles 1 y 2 de IQC fueron clasificadas en categorías Sigma diferentes por al menos uno de los métodos de estimación de la desviación. Del mismo modo, dependiendo del método de estimación del sesgo empleado, se clasificaba en diferentes categorías a 12 magnitudes de inmunoensayo con niveles 1 y 2 de IQC.

Conclusiones

La métrica Sigma puede variar dependiendo del método empleado para calcular el sesgo, lo cual debe ser tenido en cuenta a la hora de evaluar el rendimiento analítico o programar eventos de IQC aplicando el método Seis Sigma.

The application of sigma metrics in the laboratory to assess quality control processes in South Africa

Background

Laboratories use quality control processes to monitor and evaluate analytical performance in terms of precision and bias. Sigma metrics provide an objective assessment of laboratory quality using the total allowable error as an additional parameter.

Objective

This study aimed to determine the sigma metrics of analytes when using different total allowable error guidelines.

Methods

A retrospective analysis was performed on 19 general chemistry analytes at Charlotte Maxeke Johannesburg Academic Hospital in South Africa between January 2017 and December 2017. Sigma metrics were calculated on two identical analysers, using internal quality control data and total allowable error guidelines from the Ricos biological variation database and three alternative sources (the Royal College of Pathologists of Australasia, the Clinical Laboratory Improvements Amendment, and the European Federation of Clinical Chemistry and Laboratory Medicine).

Results

The sigma performance was similar on both analysers but varied based on the guideline used, with the Clinical Laboratory Improvements Amendment guidelines resulting in the best sigma metrics (53% of analytes on one analyser and 46% on the other had acceptable sigma metrics) and the Royal College of Pathologists of Australia guidelines being the most stringent (21% and 23%). Sodium and chloride performed poorly across all guidelines (sigma < 3). There were also month-to-month variations that may result in acceptable sigma despite poor performance during certain months.

Conclusion

The sigma varies greatly depending on the total allowable error, but could be a valuable tool to save time and decrease costs in high-volume laboratories. Sigma metrics calculations need to be standardised.

Sigma Metrics in Laboratory Medicine: A Call for Harmonization

BACKGROUND AND AIM

Sigma metrics are applied in clinical laboratories to assess the quality of analytical processes. A parameter associated to a Sigma >6 is considered "world class" whereas a Sigma <3 is "poor" or "unacceptable". The aim of this retrospective study was to quantify the impact of different approaches for Sigma metrics calculation.

MATERIAL AND METHODS

Two IQC levels of 20 different parameters were evaluated for a 12-month period. Sigma metrics were calculated using the formula: (allowable total error (TEa) (%) - bias (%))/(coefficient of variation (CV) (%)). Method precision was calculated monthly or annually. The bias was obtained from peer comparison program (PCP) or external quality assessment program (EQAP), and 9 different TEa sources were included.

RESULTS

There was a substantial monthly variation of Sigma metrics for all combinations, with a median variation of 32% (IQR, 25.6-41.3%). Variation across multiple analyzers and IQC levels were also observed. Furthermore, TEa source had the highest impact on Sigma calculation with proportions of Sigma >6 ranging from 17.5% to 84.4%. The nature of bias was less decisive.

CONCLUSION

In absence of a clear consensus, we recommend that laboratories calculate Sigma metrics on a sufficiently long period of time (>6 months) and carefully evaluate the choice of TEa source.

Application of a six sigma model to evaluate the analytical performance of urinary biochemical analytes and design a risk-based statistical quality control strategy for these assays: A multicenter study

Background

The six sigma model has been widely used in clinical laboratory quality management. In this study, we first applied the six sigma model to (a) evaluate the analytical performance of urinary biochemical analytes across five laboratories, (b) design risk‐based statistical quality control (SQC) strategies, and (c) formulate improvement measures for each of the analytes when needed.

Methods

Internal quality control (IQC) and external quality assessment (EQA) data for urinary biochemical analytes were collected from five laboratories, and the sigma value of each analyte was calculated based on coefficients of variation, bias, and total allowable error (TEa). Normalized sigma method decision charts for these urinary biochemical analytes were then generated. Risk‐based SQC strategies and improvement measures were formulated for each laboratory according to the flowchart of Westgard sigma rules, including run sizes and the quality goal index (QGI).

Results

Sigma values of urinary biochemical analytes were significantly different at different quality control levels. Although identical detection platforms with matching reagents were used, differences in these analytes were also observed between laboratories. Risk‐based SQC strategies for urinary biochemical analytes were formulated based on the flowchart of Westgard sigma rules, including run size and analytical performance. Appropriate improvement measures were implemented for urinary biochemical analytes with analytical performance lower than six sigma according to the QGI calculation.

Conclusions

In multilocation laboratory systems, a six sigma model is an excellent quality management tool and can quantitatively evaluate analytical performance and guide risk‐based SQC strategy development and improvement measure implementation.

Assessment of Sigma Metrics for Routine Chemistry Testing in 4 Laboratories in Kwa-Zulu Natal, South Africa

BACKGROUND

Sigma metrics is a quantitative management tool. This study assessed the Six Sigma score for 26 chemistry analytes, compared scores with different total allowable errors (TEa) and use of scores for internal quality control (IQC) rules in 4 Laboratories in Kwa-Zulu Natal, South Africa.

METHODS

Utilizing 6 months of IQC SD, CV, and bias data on albumin, alkaline phosphatase, alanine aminotransferase, amylase, aspartate aminotransferase, bicarbonate, calcium, total cholesterol, creatine kinase, chloride, creatinine, gamma glutamyl transferase, glucose, HDL-cholesterol, potassium, lactate dehydrogenase, magnesium, sodium, inorganic phosphate, direct bilirubin, total bilirubin, triglycerides, total protein, urea nitrogen, uric acid, and C-reactive protein (CRP) Six Sigma scores were calculated using Microsoft Excel 2016 and ideal IQC rules were determined. Six Sigma scores using Ricos et al. 2014, Royal College of Pathologists Australasia, and Clinical Laboratory Improvement Amendments TEas were compared.

RESULTS

For levels 1, 2, and 3 respectively, analytes scoring >3 sigma was 9 (35%), 12 (46%), and 14 (54%) in Laboratory A; Laboratory B had 15 (58%), 19 (73%), and 17 (65%); Laboratory C had 12 (46%), 13 (50%), and 15 (58%); and Laboratory D had 13 (50%), 18 (69%), and 18 (69%). Albumin, calcium, sodium, magnesium, bicarbonate, and chloride scored <3; CRP scored >6 for all. In Laboratories A, B, C, and D, 7 (27%), 7 (27%), 6 (23%), and 8 (31%) analytes, respectively, required only 1 IQC rule. One of 21 analytes for Laboratories C and D, 3 for Laboratory A, and 0 for Laboratory B had the same sigma score with all 3 databases.

CONCLUSION

Despite South Africa being a developing nation, many analytes are able to achieve >3 sigma.

Application of Sigma metrics in the quality control strategies of immunology and protein analytes

Background

Six Sigma (6σ) is an efficient laboratory management method. We aimed to analyze the performance of immunology and protein analytes in terms of Six Sigma.

Methods

Assays were evaluated for these 10 immunology and protein analytes: Immunoglobulin G (IgG), Immunoglobulin A (IgA), Immunoglobulin M (IgM), Complement 3 (C3), Complement 4 (C4), Prealbumin (PA), Rheumatoid factor (RF), Anti streptolysin O (ASO), C‐reactive protein (CRP), and Cystatin C (Cys C). The Sigma values were evaluated based on bias, four different allowable total error (TEa) and coefficient of variation (CV) at QC materials levels 1 and 2 in 2020. Sigma Method Decision Charts were established. Improvement measures of analytes with poor performance were recommended according to the quality goal index (QGI), and appropriate quality control rules were given according to the Sigma values.

Results

While using the TEa_NCCL, 90% analytes had a world‐class performance with σ>6, Cys C showed marginal performance with σ<4. While using minimum, desirable, and optimal biological variation of TEa, only three (IgG, IgM, and CRP), one (CRP), and one (CRP) analytes reached 6σ level, respectively. Based on σ_NCCL that is calculated from TEa_NCCL, Sigma Method Decision Charts were constructed. For Cys C, five multi‐rules (1_3s/2_2s/R_4s/4_1s/6_X, N = 6, R = 1, Batch length: 45) were adopted for QC management. The remaining analytes required only one QC rule (1_3s, N = 2, R = 1, Batch length: 1000). Cys C need to improve precision (QGI = 0.12).

Conclusions

The laboratories should choose appropriate TEa goals and make judicious use of Sigma metrics as a quality improvement tool.

Sigma metrics of Alinity ci system - a study on thirty-nine clinical chemistry and immunoassay parameters

Objectives

Sigma metrics in an invaluable and inexpensive tool used in laboratories to monitor analytical quality of the assays. Alinity ci platform is a relatively recent analytical system launched by Abbott Diagnostics, and as such performance studies on it are few. We have calculated sigma metrics of 39 clinical chemistry and immunoassay analytes on two Alinity ci systems.

Methods

Sigma metrics were calculated using results of method validation studies. Coefficient of variation (CV) was calculated according to CLSI EP 15 guidelines. Bias was calculated using three different methods i.e., proficiency testing material, alternate method comparison with existent analyzers and linearity experiment. Total allowable error limits were kept similar to or less than the ones used in reference studies.

Results

All analytes except blood urea nitrogen (BUN) demonstrated greater than six sigma value across one or more levels and methods. No analyte amongst clinical chemistry and immunoassays was at below three sigma class. Amongst electrolytes, sodium was below three sigma class at two levels by proficiency testing method, although it was above four sigma class by other two methods. Sigma levels obtained were comparable to those reported in previously published studies.

Conclusions

Acceptable sigma metrics were achieved for all clinical chemistry, immunoassays and electrolytes on Alinity ci. Sigma metrics is an objective and well established cost effective tool to tailor internal quality control practices. This study determines sigma metrics for a wide range of high throughput assays. Long term assay performance needs to be monitored.

Application of a six sigma model to the evaluation of the analytical performance of serum enzyme assays and the design of a quality control strategy for these assays: A multicentre study

BACKGROUND

Six medical testing laboratories at six different sites in China participated in this study. We applied a six sigma model for (a) the evaluation of the analytical performance of serum enzyme assays at each of the laboratories, (b) the design of individualized quality control programs and (c) the development of improvement measures for each of the assays, as appropriate.

METHODS

Internal quality control (IQC) and external quality assessment (EQA) data for selected serum enzyme assays were collected from each of the laboratories. Sigma values for these assays were calculated using coefficients of variation, bias, and total allowable error (TEa). Normalized sigma method decision charts were generated using these parameters. IQC design and improvement measures were defined using the Westgard sigma rules. The quality goal index (QGI) was used to assist with identification of deficiencies (bias problems, precision problems, or their combination) affecting the analytical performance of assays with sigma values <6.

RESULTS

Sigma values for the selected serum enzyme assays were significantly different at different levels of enzyme activity. Differences in assay quality in different laboratories were also seen, despite the use of identical testing instruments and reagents. Based on the six sigma data, individualized quality control programs were outlined for each assay with sigma <6 at each laboratory.

CONCLUSIONS

In multi-location laboratory systems, a six sigma model can evaluate the quality of the assays being performed, allowing management to design individualized IQC programs and strategies for continuous improvement as appropriate for each laboratory. This will improve patient care, especially for patients transferred between sites within multi-hospital systems.

Application of the sigma metrics to evaluate the analytical performance of cystatin C and design a quality control strategy

BACKGROUND

Sigma metrics are commonly used to evaluate laboratory management. In this study, we aimed to evaluate the analytical performance of cystatin C using sigma metrics and to develop an individualized quality control scheme for cystatin C concentrations.

METHODS

Bias was calculated based on the samples used for the external quality assessment. The coefficient of variation was calculated using six months of internal quality control measurements at two levels, and desirable specification derived from biological variation was used as the quality goal. The sigma value for cystatin C was calculated using the above data. The internal quality control scheme and improvement measures were formulated according to the Westgard sigma standards for batch size and quality goal index.

RESULTS

The sigma values for cystatin C, for quality control levels 1 and 2, were 3.04 and 4.95, respectively. The 1_3s/2_2s/R_4s/4_1s/6_x multirules (n = 6, R = 1), with a batch size of 45 patient samples, were selected as the internal quality control schemes for cystatin C. With different concentrations of cystatin C, the power function graph showed a probability for error detection of 94% and 100% and a probability for false rejection of 4% and 2%, respectively. According to the quality goal index of cystatin C, its precision needs to be improved.

CONCLUSIONS

With a 'desirable' biological variation of 6.50%, the Westgard rule 1_3s/2_2s/R_4s/4_1s/6_x (n = 6, R = 1, batch size of 45) with high efficacy for determining the detection error is recommended for individualized quality control schemes of cystatin C.

The application of Six Sigma to perform quality analyses of plasma proteins

Background

The Six Sigma theory is an important tool for laboratory quality management. It has been widely used in clinical chemistry, haematology and other disciplines. The aim of our study was to evaluate the analytical performance of plasma proteins by application of Sigma metric and to compare the differences among three different allowable total errors in evaluating the analytical performance of plasma proteins.

Methods

Three different allowable total error values were used as quality goals. Data from an external quality assessment were used as bias, and the cumulative coefficient of variation in internal quality control data was used to represent the amount of imprecision during the same period. Sigma metric of analytes was calculated using the above data. The quality goal index was calculated to provide corrected measures for continuous improvements in analytical quality.

Results

The Sigma metric was highest using the external quality assessment standards of China: it was sigma ≥6 or higher in 57.1% of plasma proteins. But Sigma metric was lower by using RiliBÄK or biological variation standards. IgG, C3 and C-reactive protein all required quality improvements in imprecision. A single-rule 13s for internal quality control was recommended for IgA, IgM, C4 and rheumatoid factor, whereas multiple rules (13s/22s/R4s) were recommended for IgG, C3 and C-reactive protein, according to the external quality assessment standards of China.

Conclusions

Different quality goals can lead to different Sigma metric for the same analyte. As the lowest acceptable standard in clinical practice, the external quality assessment standard of China can guide laboratories to formulate reasonable quality improvement programmes.

Comparative analysis of calculating sigma metrics by a trueness verification proficiency testing-based approach and an internal quality control data inter-laboratory comparison-based approach

Introduction

Two methods were compared for evaluating the sigma metrics of clinical biochemistry tests using two different allowable total error (TEa) specifications.

Materials and methods

The imprecision (CV%) and bias (bias%) of 19 clinical biochemistry analytes were calculated using a trueness verification proficiency testing (TPT)‐based approach and an internal quality control data inter‐laboratory comparison (IQC)‐based approach, respectively. Two sources of total allowable error (TEa), the Clinical Laboratory Improvement Amendments of 1988 (CLIA '88) and the People's Republic of China Health Industry Standard (WS/T 403‐2012), were used to calculate the sigma metrics (σ_CLIA, σ_WS/T). Sigma metrics were calculated to provide a single value for assessing the quality of each test based on a single concentration level.

Results

For both approaches, σ_CLIA > σ_WS/T in 18 out of 19 assays. For the TPT‐based approach, 16 assays showed σ_CLIA > 3, and 12 assays showed σ_WS/T > 3. For the IQC‐based approach, 19 and 16 assays showed σ_CLIA > 3 and σ_WS/T > 3, respectively.

Conclusions

Both methods can be used as references for calculating sigma metrics and designing QC schedules in clinical laboratories. Sigma metrics should be evaluated comprehensively by different approaches.