Quality specifications of routine clinical chemistry methods based on sigma metrics in performance evaluation

Comparison of ionised calcium measured using a portable analyser to a reference method in healthy dogs

OBJECTIVES

To compare the ionised calcium measured on a portable analyser (iSTAT, Abbott) to a reference method.

MATERIALS AND METHODS

Blood samples from 39 apparently healthy dogs were analysed in duplicate using a portable analyser and a reference method (Radiometer ABL800 FLEX). Bland-Altman plots and Passing-Bablok regression were used to assess constant and proportional bias between the two instruments. A within-assay percentage coefficient of variation and total error (TE) was calculated for both analysers. The reference interval was calculated for the portable analyser using the robust method with confidence interval bootstrapping.

RESULTS

The Bland-Altman plot showed a -0.036 mmol/L difference between the two instruments (95% confidence limit -0.08 to 0.01 mmol/L; limits of agreement -0.07 to 0.006 mmol/L). Neither the Bland-Altman plot nor the Passing-Bablock regression (slope -0.03; 95% confidence interval -0.08 to 0.19 and intercept 1; 95% confidence interval 0.83 to 1.2) showed significant proportional bias. The coefficient of variation for the portable analyser was 1.08%, compared to 0.78% for the reference method with a total error of 3.5% for the portable analyser. The estimated population-based reference interval for ionised calcium using the portable analyser is 1.23 to 1.42 mmol/L.

CLINICAL SIGNIFICANCE

For the healthy dogs in this study, compared to the reference method, the portable analyser showed no significant bias for measurement of ionised calcium. Further studies including hyper and hypocalcaemic dogs are required to determine clinical impact of the use of this analyser.

Use of Uncertainty of Measurement for Traceability of Test Results and Setting up of own Quality Goal for Methods having Lower Stability- A Tertiary Care Hospital study

Uncertainty of measurement (UM) provides a quantitative estimate for traceability of test results. The Nordtest guide was applied for calculating UM of 26 analytes. For this, internal and external quality control data from July 2019 to April 2020 was used. UM of test results were compared to %TEa values of CLIA '2019, RiliBÄK, and Ricos. It was observed that UM for all analytes were below %TEa values of RiliBÄK. UM value of Albumin, Calcium and Sodium could not meet CLIA '2019 and Ricos guidelines. For results of Albumin, Calcium and Sodium to be traceable, more frequent quality control protocols resulted in decrease in bias. Quality goals were set for these three parameters. This helped in reduction of quality control cycles and optimum utilization of resources.

The application of sigma metrics in the laboratory to assess quality control processes in South Africa

Background

Laboratories use quality control processes to monitor and evaluate analytical performance in terms of precision and bias. Sigma metrics provide an objective assessment of laboratory quality using the total allowable error as an additional parameter.

Objective

This study aimed to determine the sigma metrics of analytes when using different total allowable error guidelines.

Methods

A retrospective analysis was performed on 19 general chemistry analytes at Charlotte Maxeke Johannesburg Academic Hospital in South Africa between January 2017 and December 2017. Sigma metrics were calculated on two identical analysers, using internal quality control data and total allowable error guidelines from the Ricos biological variation database and three alternative sources (the Royal College of Pathologists of Australasia, the Clinical Laboratory Improvements Amendment, and the European Federation of Clinical Chemistry and Laboratory Medicine).

Results

The sigma performance was similar on both analysers but varied based on the guideline used, with the Clinical Laboratory Improvements Amendment guidelines resulting in the best sigma metrics (53% of analytes on one analyser and 46% on the other had acceptable sigma metrics) and the Royal College of Pathologists of Australia guidelines being the most stringent (21% and 23%). Sodium and chloride performed poorly across all guidelines (sigma < 3). There were also month-to-month variations that may result in acceptable sigma despite poor performance during certain months.

Conclusion

The sigma varies greatly depending on the total allowable error, but could be a valuable tool to save time and decrease costs in high-volume laboratories. Sigma metrics calculations need to be standardised.

Sigma Metrics in Laboratory Medicine: A Call for Harmonization

BACKGROUND AND AIM

Sigma metrics are applied in clinical laboratories to assess the quality of analytical processes. A parameter associated to a Sigma >6 is considered "world class" whereas a Sigma <3 is "poor" or "unacceptable". The aim of this retrospective study was to quantify the impact of different approaches for Sigma metrics calculation.

MATERIAL AND METHODS

Two IQC levels of 20 different parameters were evaluated for a 12-month period. Sigma metrics were calculated using the formula: (allowable total error (TEa) (%) - bias (%))/(coefficient of variation (CV) (%)). Method precision was calculated monthly or annually. The bias was obtained from peer comparison program (PCP) or external quality assessment program (EQAP), and 9 different TEa sources were included.

RESULTS

There was a substantial monthly variation of Sigma metrics for all combinations, with a median variation of 32% (IQR, 25.6-41.3%). Variation across multiple analyzers and IQC levels were also observed. Furthermore, TEa source had the highest impact on Sigma calculation with proportions of Sigma >6 ranging from 17.5% to 84.4%. The nature of bias was less decisive.

CONCLUSION

In absence of a clear consensus, we recommend that laboratories calculate Sigma metrics on a sufficiently long period of time (>6 months) and carefully evaluate the choice of TEa source.

Sigma and Risk in the Quality Control Routine: Analysis in Chilean Clinical Laboratories

BACKGROUND

The Six Sigma methodology is focused toward improvement, based on the Total Quality Management. It has been implemented in analytical procedures for clinical laboratories in the form of Sigma Metrics. This method is used in the evaluation of analytical procedures, providing evidence for risk-based management.

METHODS

A descriptive study was carried using data from 18 Chilean clinical laboratories. The information of their performance and quality specifications used in their routine work was obtained from UNITY, an internal quality comparison program.

RESULTS

A total of 3461 sigma evaluations was gathered, mostly from biyearly controls. The general distribution shows a median of 5.5 with positive asymmetry similar to other publications. The reported quality specifications are based in CLIA for 51.2% of the cases, 30.2% from biological variation, and 10.7% from other programs for the external quality evaluation. Significant differences (P < 0.05) were found between medians against their specification source.

CONCLUSIONS

In the studied series, it would be feasible to implement a risk-based quality control system with simple rules and minimal control materials for 55.5% of the evaluated sigmas. 19.6% of the sigmas require improvement mainly in precision. The variety in specifications reveals a lack of harmonization in the specification's selections.

Assessment of Sigma Metrics for Routine Chemistry Testing in 4 Laboratories in Kwa-Zulu Natal, South Africa

BACKGROUND

Sigma metrics is a quantitative management tool. This study assessed the Six Sigma score for 26 chemistry analytes, compared scores with different total allowable errors (TEa) and use of scores for internal quality control (IQC) rules in 4 Laboratories in Kwa-Zulu Natal, South Africa.

METHODS

Utilizing 6 months of IQC SD, CV, and bias data on albumin, alkaline phosphatase, alanine aminotransferase, amylase, aspartate aminotransferase, bicarbonate, calcium, total cholesterol, creatine kinase, chloride, creatinine, gamma glutamyl transferase, glucose, HDL-cholesterol, potassium, lactate dehydrogenase, magnesium, sodium, inorganic phosphate, direct bilirubin, total bilirubin, triglycerides, total protein, urea nitrogen, uric acid, and C-reactive protein (CRP) Six Sigma scores were calculated using Microsoft Excel 2016 and ideal IQC rules were determined. Six Sigma scores using Ricos et al. 2014, Royal College of Pathologists Australasia, and Clinical Laboratory Improvement Amendments TEas were compared.

RESULTS

For levels 1, 2, and 3 respectively, analytes scoring >3 sigma was 9 (35%), 12 (46%), and 14 (54%) in Laboratory A; Laboratory B had 15 (58%), 19 (73%), and 17 (65%); Laboratory C had 12 (46%), 13 (50%), and 15 (58%); and Laboratory D had 13 (50%), 18 (69%), and 18 (69%). Albumin, calcium, sodium, magnesium, bicarbonate, and chloride scored <3; CRP scored >6 for all. In Laboratories A, B, C, and D, 7 (27%), 7 (27%), 6 (23%), and 8 (31%) analytes, respectively, required only 1 IQC rule. One of 21 analytes for Laboratories C and D, 3 for Laboratory A, and 0 for Laboratory B had the same sigma score with all 3 databases.

CONCLUSION

Despite South Africa being a developing nation, many analytes are able to achieve >3 sigma.

Internal Quality Control Data of Urine Reagent Strip Tests and Derivation of Control Rules Based on Sigma Metrics

Background

Urine reagent strip test (URST) results are semi-quantitative; therefore, the precision of URSTs is evaluated as the proportion of categorical results from repeated measurements of a sample that are concordant with an expected result. However, URSTs have quantitative readout values before ordinal results challenging statistical monitoring for internal quality control (IQC) with control rules. This study aimed to determine the sigma metric of URSTs and derive appropriate control rules for IQC.

Methods

The URiSCAN Super Plus fully automated urine analyzer (YD Diagnostics, Yongin, Korea) was used for URSTs. Change in reflectance rate (change %R) data from IQC for URSTs performed between November 2018 and May 2020 were analyzed. Red blood cells, bilirubin, urobilinogen, ketones, protein, glucose, leukocytes, and pH were measured from 2-3 levels of control materials. The total allowable error (TE_a) for a grade was the difference in midpoints of a predefined change %R range between two adjacent grades. The sigma metric was calculated as TE_a/SD. Sigma metric-based control rules were determined with Westgard EZ Rules 3 software (Westgard QC, Madison, WI, USA).

Results

Seven out of the eight analytes had a sigma metric >4 in the control materials with a negative grade (-), which were closer to the cut-offs. Corresponding control rules ranged from 1_2.5s to 1_3.5s.

Conclusions

Although the URST is a semi-quantitative test, statistical IQC can be performed using the readout values. According to the sigma metric, control rules recommended for URST IQC in routine clinical practice are 1_2.5s to 1_3.5s.

Practical application of Westgard Sigma rules with run size in analytical biochemistry processes in clinical settings

BACKGROUND

The performance of 18 routine chemical detection methods was evaluated by the sigma (σ) metric, and Westgard Sigma rules with run size were used to establish internal quality control (IQC) standards to reduce patient risks.

MATERIALS AND METHODS

External quality assessment (EQA) and internal quality control data from 18 assays in a biochemical laboratory were collected from January to June 2020. The sigma values of each assay were calculated, based on the bias, total error allowable, and coefficient of variation, appropriate quality control rules were selected. According to the quality goal index, the main causes of poor performance were determined to guide quality improvement.

RESULTS

At IQC material level 1, seven of the 18 assays achieved five sigma (excellent), and five assays (UA, Crea, AMY, TC and Na) showed world-class performance. At IQC material level 2, 14 of the 18 assays achieved 5 sigma (excellent), and thirteen assays (UA, ALT, CK, Crea, AMY, K, AST, ALP, Na, LDH, Mg, TC and GGT) showed world-class performance. The quality goal index (QGI) was calculated for items with analysis performance <5 sigma, and the main causes of poor performance were determined to guide quality improvement.

CONCLUSIONS

Westgard sigma rules with run size are an effective tool for evaluating the performance of biochemical assays. These rules can be used to more simply and intuitively select the quality control strategy of related items and reduce the risk to patients.

Analysis of performance of clinical biochemistry laboratory using Sigma metrics and Quality Goal Index

Background

Unreliable and ingenuine results issued by clinical laboratories have serious consequences for the patients. Sigma metrics is a standardized tool for Quality assessment for test performance in a laboratory.

Objective

To evaluate the performance of routine biochemistry laboratory at MMIMSR, Mullana in terms of Sigma metrics and Quality Goal Index.

Material and methods

This cross sectional study evaluated performance of 14 routine chemistry parameters using retrospective Internal Quality Control data of two levels on Siemens Dimension Rxl from Feb to Jul 2019 for CV% and EQAS reports from CMC, Vellore for Bias%. Sigma metrics was calculated using total allowable error targets as per CLIA and Biological Variability database guidelines.

Results

For level-2 IQC; TG, Chol, ALP showed excellent performance with σ ​> ​6 while σ ​< ​3 was observed for AST, Total Protein, Glucose, BUN and ALT using CLIA guidelines while in IQC Level-3 poor performers were only BUN and ALT with Ca, TG and Chol showing σ ​> ​6. Further by using Biological Variability data guidelines; 10 parameters of IQC Level-2 and 5 of IQC level-3 were poor performers with σ ​< ​3.

Conclusion

Sigma metrics is an excellent tool for performance analysis of tests performed in a clinical laboratory. Lack of precision in terms of CV% was seen for majority of the poor performers. Total allowable error targets using Biological Variability data revealed σ ​< ​3 for 10 parameters while using CLIA guidelines σ ​< ​3 was seen for only 5 parameters of IQC level-2.

Sigma metrics application for validated and non-validated detecting systems performance assessment

Background

Sigma metrics provide an objective and quantitative methodology for analytical quality evaluation of clinical laboratory. This study investigated the testing performance of validated systems and non‐validated systems based on sigma metrics, and explored the major parameters affecting the system performance.

Methods

Sigma metrics were evaluated by six biochemistry assays based on Beckman and Mindray validated and non‐validated systems through crossing the reagents and analyzers. Imprecision and bias were assessed for all assays based on trueness programs organized by National Centre for Clinical Laboratory. Total error allowance obtained from the Chinese Ministry of Health Clinical Laboratory Centre Industry Standard (WS/T403‐2012).

Results

The imprecision for all systems meets the quality specifications except TP assay (2.19%) detected by Mindray non‐validated system, and the bias for four assays measured by non‐validated systems cannot fulfill the criterion, including lactate dehydrogenase (LDH), total protein (TP), triglycerides (TG), and glucose (GLU). Higher biases were detected in six assays at different levels among non‐validated and validated systems. Systems performed poorly or unacceptably for TP assay with sigma metrics lower than 3 except Mindray non‐validated system. The sigma metrics for other assays with four systems were greater than 3 except the LDH evaluated on Mindray non‐validated systems.

Conclusion

Non‐validated systems may introduce performance uncertainty compared with validated systems based on sigma metrics evaluation, and lower bias was provided by validated systems. The performance of non‐validated systems should be evaluated thoroughly in the clinical laboratory before they were adopted for routine use.

Comparative analysis of calculating sigma metrics by a trueness verification proficiency testing-based approach and an internal quality control data inter-laboratory comparison-based approach

Introduction

Two methods were compared for evaluating the sigma metrics of clinical biochemistry tests using two different allowable total error (TEa) specifications.

Materials and methods

The imprecision (CV%) and bias (bias%) of 19 clinical biochemistry analytes were calculated using a trueness verification proficiency testing (TPT)‐based approach and an internal quality control data inter‐laboratory comparison (IQC)‐based approach, respectively. Two sources of total allowable error (TEa), the Clinical Laboratory Improvement Amendments of 1988 (CLIA '88) and the People's Republic of China Health Industry Standard (WS/T 403‐2012), were used to calculate the sigma metrics (σ_CLIA, σ_WS/T). Sigma metrics were calculated to provide a single value for assessing the quality of each test based on a single concentration level.

Results

For both approaches, σ_CLIA > σ_WS/T in 18 out of 19 assays. For the TPT‐based approach, 16 assays showed σ_CLIA > 3, and 12 assays showed σ_WS/T > 3. For the IQC‐based approach, 19 and 16 assays showed σ_CLIA > 3 and σ_WS/T > 3, respectively.

Conclusions

Both methods can be used as references for calculating sigma metrics and designing QC schedules in clinical laboratories. Sigma metrics should be evaluated comprehensively by different approaches.

Application of Six Sigma for evaluating the analytical quality of tumor marker assays

CONTEXT

The results of detection assays for the same specimen are usually quite different in different laboratories or when tested with different detection systems.

OBJECTIVE

This study was designed to investigate the value of applying sigma metrics derived from different standards for allowable total error (TEa) in evaluating the analytical quality of tumor marker assays.

METHODS

Assays were evaluated for these six tumor markers: total prostate-specific antigen (tPSA), carcinoembryonic antigen (CEA), alpha-fetoprotein (AFP), carbohydrate antigen 199 (CA199), carbohydrate antigen 125 (CA125), and carbohydrate antigen 153 (CA153). Sigma values were calculated for two concentrations of quality control products to assess differences in quality of tumor marker assays. Improvement measures were recommended according to the quality goal index, and appropriate quality control rules were selected according to the sigma value.

RESULTS

The sigma value was highest using the higher biological variation-derived "appropriate" TEa standard: it was sigma ≥6 or higher in 16.7% of tumor markers. Sigma was below 6 for all tumor markers using the other three TEa. CEA, AFP, CA199, CA125, and CA153 required improved precision. The marker tPSA required improve precision and accuracy. According to sigma values by using China's external quality assessment standards, CEA, AFP, CA125, and CA153 require 1_3s /2_2s /R_4s /4_1s multirules for internal quality control, CA199 requires use of 1_3s /2_2s /R_4s /4_1s /8_x multirules, and tPSA requires maximum quality control rules.

CONCLUSION

Six Sigma is useful for evaluating performance of tumor markers assays and has important application value in the quality control of these assays.

Quality specifications of routine clinical chemistry methods based on sigma metrics in performance evaluation

INTRODUCTION

Sigma metrics were applied to evaluate the performance of 20 routine chemistry assays, and individual quality control criteria were established based on the sigma values of different assays.

METHODS

Precisions were expressed as the average coefficient variations (CVs) of long-term two-level chemistry controls. The biases of the 20 assays were obtained from the results of trueness programs organized by National Center for Clinical Laboratories (NCCL, China) in 2016. Four different allowable total error (TEa) targets were chosen from biological variation (minimum, desirable, optimal), Clinical Laboratory Improvements Amendments (CLIA, US), Analytical Quality Specification for Routine Analytes in Clinical Chemistry (WS/T 403-2012, China) and the National Cholesterol Education Program (NECP).

RESULTS

The sigma values from different TEa targets varied. The TEa targets for ALT, AMY, Ca, CHOL, CK, Crea, GGT, K, LDH, Mg, Na, TG, TP, UA and Urea were chosen from WS/T 403-2012; the targets for ALP, AST and GLU were chosen from CLIA; the target for K was chosen from desirable biological variation; and the targets for HDL and LDL were chosen from the NECP. Individual quality criteria were established based on different sigma values.

CONCLUSIONS

Sigma metrics are an optimal tool to evaluate the performance of different assays. An assay with a high value could use a simple internal quality control rule, while an assay with a low value should be monitored strictly.