Assessment of quality control system by sigma metrics and quality goal index ratio: A roadmap towards preparation for NABL

Six SIGMA evaluation of 17 biochemistry parameters using bias calculated from internal quality control and external quality assurance data

Background

Six Sigma is a popular quality management system that enables continuous monitoring and improvement of analytical performance in the clinical laboratory. We aimed to calculate sigma metrics and quality goal index (QGI) for 17 biochemical analytes and compare the use of bias from internal quality control (IQC) and external quality assurance (EQA) data in the calculation of sigma metrics.

Methods

This retrospective study was conducted in Marmara University Pendik E&R Hospital Biochemistry Laboratory. Sigma metrics calculation was performed as (TEa-bias)/CV). CV was calculated from IQC data from June 2018 - February 2019. EQA bias was calculated as the mean of % deviation from the peer group means in the last seven surveys, and IQC bias was calculated as (laboratory control result mean-manufacturer control mean)/ manufacturer control mean) x100. In parameters where sigma metrics were <5; QGI=bias/1.5 CV) score of <0.8 indicated imprecision, >1.2 pointed inaccuracy, and 0.8-1.2 showed both imprecision and inaccuracy.

Results

Creatine kinase (both levels), iron and magnesium (pathologic levels) showed an ideal performance with ≥6 sigma level for both bias determinations. Eight of the 17 parameters had different sigma levels when we compared sigma values calculated from EQA and IQC derived bias% while the rest were grouped at the same levels.

Conclusions

Sigma metrics is a good quality tool to assess a laboratory's analytical performance and facilitate the comparison of the assay performances in the same manner across multiple systems. However, we might need to design a tight internal quality control protocol for analytes showing poor assay performance.

Sigma metric used to evaluate the performance of haematology analysers: choosing an internal reference analyser for the laboratory

INTRODUCTION

The sigma metric offers a quantitative framework for evaluating process performance in clinical laboratories. This study aimed to evaluate the analytical performance of automated analysers in haematology laboratories, using the sigma metric to choose the best analyser as an internal reference analyser.

MATERIALS AND METHODS

internal quality control (IQC) data were collected for 6 months from SNCS, and the sigma value was calculated for 9 haematology analysers in the laboratory.

RESULTS

For the normal control level, a satisfactory mean sigma value ≥3 was observed for all of the studied parameters of all automated analysers. For the low control level, platelet (PLT) count by Instrument (Inst.) G performed poorly, with a mean sigma value <3. Inst. H, with all parameters' sigma values >4, performed best and was chosen as the internal reference analyser.

CONCLUSION

The sigma metric can be used as a guide to choose the QC strategy and plan QC frequency. It can facilitate the comparison of the same assay performed by multiple systems.

R, Yassir S, Prasad V. G, Ansar M. A comparative evaluation of six sigma metrics and quality goal index ratio 3 months prior to first lockdown due to COVID-19 pandemic and 3 months during lockdown in a NABL accredited central laboratory

Introduction and Aim: Sigma represents Standard Deviation (SD) which indicates the degree of variation in a process, where the higher sigma value implies that less likely the laboratory reports false test results. Using a newer parameter called Quality Goal Index (QGI) we can find the reason behind the lower sigma value. Our study aimed to compare the six-sigma metric and QGI ratio 3 months prior to first lockdown due to COVID-19 pandemic and 3 months during the first lockdown.   Methodology: A retrospective study was used to compare the six-sigma metric and QGI ratio 3 months prior to first lockdown due to COVID-19 pandemic and 3 months during the first lockdown for the selected ten analytes from 1st of January 2020 to 30th of June 2020 from the clinical biochemistry section of Yenepoya Medical College Hospital, Deralakatte, Mangalore.   Results: The sigma metrics from January to March (level 1) indicated that urea, TSH, beta-HCG fell short of meeting Six Sigma quality performance and from April to June, glucose, creatinine, urea and ALT had metrics less than 3 at both the Internal Quality Control levels. QGI ratio indicated that from January to March, the problem was imprecision for urea, TSH and beta-HCG (QGI < 0.8). From April to June, urea and creatinine showed imprecision, glucose and ALT showed inaccuracy, urea and ALT showed both imprecision and inaccuracy.   Conclusion: This study highlights the necessity for stringent Internal Quality Control and External Quality Assurance monitoring even during the lockdown period of the pandemic. By implementing six sigma and finding QGI ratio, quality of laboratory services can be improved immensely.

The application of sigma metrics in the laboratory to assess quality control processes in South Africa

Background

Laboratories use quality control processes to monitor and evaluate analytical performance in terms of precision and bias. Sigma metrics provide an objective assessment of laboratory quality using the total allowable error as an additional parameter.

Objective

This study aimed to determine the sigma metrics of analytes when using different total allowable error guidelines.

Methods

A retrospective analysis was performed on 19 general chemistry analytes at Charlotte Maxeke Johannesburg Academic Hospital in South Africa between January 2017 and December 2017. Sigma metrics were calculated on two identical analysers, using internal quality control data and total allowable error guidelines from the Ricos biological variation database and three alternative sources (the Royal College of Pathologists of Australasia, the Clinical Laboratory Improvements Amendment, and the European Federation of Clinical Chemistry and Laboratory Medicine).

Results

The sigma performance was similar on both analysers but varied based on the guideline used, with the Clinical Laboratory Improvements Amendment guidelines resulting in the best sigma metrics (53% of analytes on one analyser and 46% on the other had acceptable sigma metrics) and the Royal College of Pathologists of Australia guidelines being the most stringent (21% and 23%). Sodium and chloride performed poorly across all guidelines (sigma < 3). There were also month-to-month variations that may result in acceptable sigma despite poor performance during certain months.

Conclusion

The sigma varies greatly depending on the total allowable error, but could be a valuable tool to save time and decrease costs in high-volume laboratories. Sigma metrics calculations need to be standardised.

Sigma Metrics in Laboratory Medicine: A Call for Harmonization

BACKGROUND AND AIM

Sigma metrics are applied in clinical laboratories to assess the quality of analytical processes. A parameter associated to a Sigma >6 is considered "world class" whereas a Sigma <3 is "poor" or "unacceptable". The aim of this retrospective study was to quantify the impact of different approaches for Sigma metrics calculation.

MATERIAL AND METHODS

Two IQC levels of 20 different parameters were evaluated for a 12-month period. Sigma metrics were calculated using the formula: (allowable total error (TEa) (%) - bias (%))/(coefficient of variation (CV) (%)). Method precision was calculated monthly or annually. The bias was obtained from peer comparison program (PCP) or external quality assessment program (EQAP), and 9 different TEa sources were included.

RESULTS

There was a substantial monthly variation of Sigma metrics for all combinations, with a median variation of 32% (IQR, 25.6-41.3%). Variation across multiple analyzers and IQC levels were also observed. Furthermore, TEa source had the highest impact on Sigma calculation with proportions of Sigma >6 ranging from 17.5% to 84.4%. The nature of bias was less decisive.

CONCLUSION

In absence of a clear consensus, we recommend that laboratories calculate Sigma metrics on a sufficiently long period of time (>6 months) and carefully evaluate the choice of TEa source.

Sigma and Risk in the Quality Control Routine: Analysis in Chilean Clinical Laboratories

BACKGROUND

The Six Sigma methodology is focused toward improvement, based on the Total Quality Management. It has been implemented in analytical procedures for clinical laboratories in the form of Sigma Metrics. This method is used in the evaluation of analytical procedures, providing evidence for risk-based management.

METHODS

A descriptive study was carried using data from 18 Chilean clinical laboratories. The information of their performance and quality specifications used in their routine work was obtained from UNITY, an internal quality comparison program.

RESULTS

A total of 3461 sigma evaluations was gathered, mostly from biyearly controls. The general distribution shows a median of 5.5 with positive asymmetry similar to other publications. The reported quality specifications are based in CLIA for 51.2% of the cases, 30.2% from biological variation, and 10.7% from other programs for the external quality evaluation. Significant differences (P < 0.05) were found between medians against their specification source.

CONCLUSIONS

In the studied series, it would be feasible to implement a risk-based quality control system with simple rules and minimal control materials for 55.5% of the evaluated sigmas. 19.6% of the sigmas require improvement mainly in precision. The variety in specifications reveals a lack of harmonization in the specification's selections.

Assessment of Sigma Metrics for Routine Chemistry Testing in 4 Laboratories in Kwa-Zulu Natal, South Africa

BACKGROUND

Sigma metrics is a quantitative management tool. This study assessed the Six Sigma score for 26 chemistry analytes, compared scores with different total allowable errors (TEa) and use of scores for internal quality control (IQC) rules in 4 Laboratories in Kwa-Zulu Natal, South Africa.

METHODS

Utilizing 6 months of IQC SD, CV, and bias data on albumin, alkaline phosphatase, alanine aminotransferase, amylase, aspartate aminotransferase, bicarbonate, calcium, total cholesterol, creatine kinase, chloride, creatinine, gamma glutamyl transferase, glucose, HDL-cholesterol, potassium, lactate dehydrogenase, magnesium, sodium, inorganic phosphate, direct bilirubin, total bilirubin, triglycerides, total protein, urea nitrogen, uric acid, and C-reactive protein (CRP) Six Sigma scores were calculated using Microsoft Excel 2016 and ideal IQC rules were determined. Six Sigma scores using Ricos et al. 2014, Royal College of Pathologists Australasia, and Clinical Laboratory Improvement Amendments TEas were compared.

RESULTS

For levels 1, 2, and 3 respectively, analytes scoring >3 sigma was 9 (35%), 12 (46%), and 14 (54%) in Laboratory A; Laboratory B had 15 (58%), 19 (73%), and 17 (65%); Laboratory C had 12 (46%), 13 (50%), and 15 (58%); and Laboratory D had 13 (50%), 18 (69%), and 18 (69%). Albumin, calcium, sodium, magnesium, bicarbonate, and chloride scored <3; CRP scored >6 for all. In Laboratories A, B, C, and D, 7 (27%), 7 (27%), 6 (23%), and 8 (31%) analytes, respectively, required only 1 IQC rule. One of 21 analytes for Laboratories C and D, 3 for Laboratory A, and 0 for Laboratory B had the same sigma score with all 3 databases.

CONCLUSION

Despite South Africa being a developing nation, many analytes are able to achieve >3 sigma.

Application of Sigma metrics in the quality control strategies of immunology and protein analytes

Background

Six Sigma (6σ) is an efficient laboratory management method. We aimed to analyze the performance of immunology and protein analytes in terms of Six Sigma.

Methods

Assays were evaluated for these 10 immunology and protein analytes: Immunoglobulin G (IgG), Immunoglobulin A (IgA), Immunoglobulin M (IgM), Complement 3 (C3), Complement 4 (C4), Prealbumin (PA), Rheumatoid factor (RF), Anti streptolysin O (ASO), C‐reactive protein (CRP), and Cystatin C (Cys C). The Sigma values were evaluated based on bias, four different allowable total error (TEa) and coefficient of variation (CV) at QC materials levels 1 and 2 in 2020. Sigma Method Decision Charts were established. Improvement measures of analytes with poor performance were recommended according to the quality goal index (QGI), and appropriate quality control rules were given according to the Sigma values.

Results

While using the TEa_NCCL, 90% analytes had a world‐class performance with σ>6, Cys C showed marginal performance with σ<4. While using minimum, desirable, and optimal biological variation of TEa, only three (IgG, IgM, and CRP), one (CRP), and one (CRP) analytes reached 6σ level, respectively. Based on σ_NCCL that is calculated from TEa_NCCL, Sigma Method Decision Charts were constructed. For Cys C, five multi‐rules (1_3s/2_2s/R_4s/4_1s/6_X, N = 6, R = 1, Batch length: 45) were adopted for QC management. The remaining analytes required only one QC rule (1_3s, N = 2, R = 1, Batch length: 1000). Cys C need to improve precision (QGI = 0.12).

Conclusions

The laboratories should choose appropriate TEa goals and make judicious use of Sigma metrics as a quality improvement tool.

Evaluation of Sigma metric approach for monitoring the performance of automated analyzers in hematology unit of Alexandria Main University Hospital

INTRODUCTION

Sigma metric offers a quantitative framework for evaluating process performance in clinical laboratories. This study aimed to evaluate the analytical performance of automated analyzers in hematology unit of Alexandria Main University Hospital using the sigma metric approach.

MATERIALS AND METHODS

Quality control data were collected for 6 months, and sigma value was calculated from hematology analyzers SYSMEX (XN 1000, XT 1800i), ADVIA (2120i, 2120), and coagulation analyzers SYSMEX CA 1500 (3610, 6336).

RESULTS

For the normal control level, satisfactory mean sigma value ≥3 was observed for all of the studied parameters by all analyzers. For the high control level, red blood cell count by ADVIA 2120, and hematocrit by ADVIA (2120i and 2120) performed poorly with a mean sigma value <3. For the low control level, red blood cell count by ADVIA (2120i and 2120), hemoglobin by ADVIA 2120, hematocrit by ADVIA (2120i and 2120) and SYSMEX XN 1000, platelet count by the SYSMEX XT 1800i also performed poorly with a mean sigma value <3. Satisfactory mean sigma value of ≥3 was observed for prothrombin time and activated partial thromboplastin time for both normal and pathological control levels and analyzers.

CONCLUSION

Sigma metrics can be used as a guide to make QC strategy and plan QC frequency and can facilitate the comparison of the same assay performance across multiple systems. Harmonization for TEa source is recommended to standardize sigma value calculation.

Practical application of Westgard Sigma rules with run size in analytical biochemistry processes in clinical settings

BACKGROUND

The performance of 18 routine chemical detection methods was evaluated by the sigma (σ) metric, and Westgard Sigma rules with run size were used to establish internal quality control (IQC) standards to reduce patient risks.

MATERIALS AND METHODS

External quality assessment (EQA) and internal quality control data from 18 assays in a biochemical laboratory were collected from January to June 2020. The sigma values of each assay were calculated, based on the bias, total error allowable, and coefficient of variation, appropriate quality control rules were selected. According to the quality goal index, the main causes of poor performance were determined to guide quality improvement.

RESULTS

At IQC material level 1, seven of the 18 assays achieved five sigma (excellent), and five assays (UA, Crea, AMY, TC and Na) showed world-class performance. At IQC material level 2, 14 of the 18 assays achieved 5 sigma (excellent), and thirteen assays (UA, ALT, CK, Crea, AMY, K, AST, ALP, Na, LDH, Mg, TC and GGT) showed world-class performance. The quality goal index (QGI) was calculated for items with analysis performance <5 sigma, and the main causes of poor performance were determined to guide quality improvement.

CONCLUSIONS

Westgard sigma rules with run size are an effective tool for evaluating the performance of biochemical assays. These rules can be used to more simply and intuitively select the quality control strategy of related items and reduce the risk to patients.

Application of the sigma metrics to evaluate the analytical performance of cystatin C and design a quality control strategy

BACKGROUND

Sigma metrics are commonly used to evaluate laboratory management. In this study, we aimed to evaluate the analytical performance of cystatin C using sigma metrics and to develop an individualized quality control scheme for cystatin C concentrations.

METHODS

Bias was calculated based on the samples used for the external quality assessment. The coefficient of variation was calculated using six months of internal quality control measurements at two levels, and desirable specification derived from biological variation was used as the quality goal. The sigma value for cystatin C was calculated using the above data. The internal quality control scheme and improvement measures were formulated according to the Westgard sigma standards for batch size and quality goal index.

RESULTS

The sigma values for cystatin C, for quality control levels 1 and 2, were 3.04 and 4.95, respectively. The 1_3s/2_2s/R_4s/4_1s/6_x multirules (n = 6, R = 1), with a batch size of 45 patient samples, were selected as the internal quality control schemes for cystatin C. With different concentrations of cystatin C, the power function graph showed a probability for error detection of 94% and 100% and a probability for false rejection of 4% and 2%, respectively. According to the quality goal index of cystatin C, its precision needs to be improved.

CONCLUSIONS

With a 'desirable' biological variation of 6.50%, the Westgard rule 1_3s/2_2s/R_4s/4_1s/6_x (n = 6, R = 1, batch size of 45) with high efficacy for determining the detection error is recommended for individualized quality control schemes of cystatin C.

Analysis of performance of clinical biochemistry laboratory using Sigma metrics and Quality Goal Index

Background

Unreliable and ingenuine results issued by clinical laboratories have serious consequences for the patients. Sigma metrics is a standardized tool for Quality assessment for test performance in a laboratory.

Objective

To evaluate the performance of routine biochemistry laboratory at MMIMSR, Mullana in terms of Sigma metrics and Quality Goal Index.

Material and methods

This cross sectional study evaluated performance of 14 routine chemistry parameters using retrospective Internal Quality Control data of two levels on Siemens Dimension Rxl from Feb to Jul 2019 for CV% and EQAS reports from CMC, Vellore for Bias%. Sigma metrics was calculated using total allowable error targets as per CLIA and Biological Variability database guidelines.

Results

For level-2 IQC; TG, Chol, ALP showed excellent performance with σ ​> ​6 while σ ​< ​3 was observed for AST, Total Protein, Glucose, BUN and ALT using CLIA guidelines while in IQC Level-3 poor performers were only BUN and ALT with Ca, TG and Chol showing σ ​> ​6. Further by using Biological Variability data guidelines; 10 parameters of IQC Level-2 and 5 of IQC level-3 were poor performers with σ ​< ​3.

Conclusion

Sigma metrics is an excellent tool for performance analysis of tests performed in a clinical laboratory. Lack of precision in terms of CV% was seen for majority of the poor performers. Total allowable error targets using Biological Variability data revealed σ ​< ​3 for 10 parameters while using CLIA guidelines σ ​< ​3 was seen for only 5 parameters of IQC level-2.

The application of Six Sigma to perform quality analyses of plasma proteins

Background

The Six Sigma theory is an important tool for laboratory quality management. It has been widely used in clinical chemistry, haematology and other disciplines. The aim of our study was to evaluate the analytical performance of plasma proteins by application of Sigma metric and to compare the differences among three different allowable total errors in evaluating the analytical performance of plasma proteins.

Methods

Three different allowable total error values were used as quality goals. Data from an external quality assessment were used as bias, and the cumulative coefficient of variation in internal quality control data was used to represent the amount of imprecision during the same period. Sigma metric of analytes was calculated using the above data. The quality goal index was calculated to provide corrected measures for continuous improvements in analytical quality.

Results

The Sigma metric was highest using the external quality assessment standards of China: it was sigma ≥6 or higher in 57.1% of plasma proteins. But Sigma metric was lower by using RiliBÄK or biological variation standards. IgG, C3 and C-reactive protein all required quality improvements in imprecision. A single-rule 13s for internal quality control was recommended for IgA, IgM, C4 and rheumatoid factor, whereas multiple rules (13s/22s/R4s) were recommended for IgG, C3 and C-reactive protein, according to the external quality assessment standards of China.

Conclusions

Different quality goals can lead to different Sigma metric for the same analyte. As the lowest acceptable standard in clinical practice, the external quality assessment standard of China can guide laboratories to formulate reasonable quality improvement programmes.

Practical application of Six Sigma management in analytical biochemistry processes in clinical settings

BACKGROUND

Six Sigma methodology with a zero-defect goal has long been applied in commercial settings and was utilized in this study to assure/improve the quality of various analytes.

METHODS

Daily internal quality control (QC) and external quality assessment data were collected and analyzed by calculating the sigma (σ) values for 19 analytes based on the coefficient of variation, bias, and total error allowable. Standardized QC sigma charts were established with these parameters. Quality goal index (QGI) analysis and root cause analysis (RCA) were used to discover potential problems for the analytes.

RESULTS

Five analytes with σ ≥ 6 achieved world-class performance, and only the Westgard rule (1_3s ) with one control measurement at two QC material levels (N2) per QC event and a run size of 1000 patient samples between QC events (R1000) was needed for QC. In contrast, more control rules (2_2s /R_4s /4_1s ) along with high N values and low R values were needed for quality assurance for five analytes with 4 ≤ σ < 6. However, the sigma levels of nine analytes were σ < 4 at one or more QC levels, and a more rigorous QC procedure (1_3s /2_2s /R_4s /4_1s /8_x with N4 and R45) was implemented. The combination of QGI analysis and RCA further revealed inaccuracy or imprecision problems for these analytes with σ < 4 and discovered five aspects of potential causes considered for quality improvement.

CONCLUSIONS

Six Sigma methodology is an effective tool for evaluating the performance of biochemical analytes and is conducive to quality assurance and improvement.

Evaluating the analytical quality control of urinary albumin measurements using sigma metrics

BACKGROUND

There is no worldwide recognized reference system and standard for urinary albumin measurement until now, so the analytical quality from different laboratories has always varied. In this study, we aimed to evaluate the analytical performance of a urinary albumin assay system using Sigma-metric, and thereby choose a suitable control rule to guarantee the analytical quality of the assays.

METHOD

Two levels of diluted reference material (ERM-DA47OK/IFCC) were used to calculate the biases, the coefficient of variation (CV) were calculated from six months of internal quality control measurements at two levels, and the external quality assessment standard of China for urinary albumin (30%) was used as the total allowable error(TEa).

RESULTS

The Sigma values for quality control levels 1 and 2 were 4.28 and 6.14, leading to recommended Westgard rules of 1_3s/2_2s/R_4s/4_1s (N = 2, R = 2) and 1_3s(N = 2, R = 1), respectively. Westgard rule 1_3s/2_2s/R_4s/4_1s(N = 2, R = 2) was selected for the quality control of the urinary albumin measurements, and with it, the power function graph showed a high efficacy for determining the detection errors with a probability of false rejection of 1.004% and a probability of error detection of 98.80%.

CONCLUSION

With a TEa of 30% recommended by the external quality assessment standard of China, Westgard rule 1_3s/2_2s/R_4s/4_1s(N = 2, R = 2) with a high efficacy for determining the detection error is recommended for the quality control of urinary albumin measurements.