Assessment of enzyme measurement procedures in China through a trueness verification program

Comparable analysis of six immunoassays for carcinoembryonic antigen detection

Objective

This study aimed to assess the current status of carcinoembryonic antigen (CEA) detection. We evaluated the correlation, consistency, and comparability of CEA results among six automated immunoassays, and combined with the results of CEA trueness verification of the Beijing Center for Clinical Laboratories (BCCL) for further analysis.

Methods

Abbott Architect i2000, Beckman DxI800, Roche Cobas E601, Diasorin Liaison XL, Maccura IS1200, and Autolumo A2000 were used to detect 40 individual serum CEA samples. Taking the optimal analytical quality specifications calculated from data on biological variation as the evaluation criterion. Passing-Bablok regression and Bland-Altman analysis were performed between each assay and all-assays median values to evaluate the correlation and relative difference. The concordance correlation coefficient (CCC) was used for consistency analysis. Additionally, the trueness verification program used samples at three concentration levels to assess the bias, coefficient of variation (CV), and total error (TE) between the average measured values and the target value.

Results

The Spearman's rank correlation coefficient (rs) was ≥0.996 and the CCC ranged between 0.9448 and 0.9990 for each assay vs. all-assays median. Considering the all-assays median value of each sample as a reference, there were proportional and systematic differences according to the Passing-bablok regression analysis. The relative difference of the four assays (Abbott Architect i2000, Autolumo A2000, Diasorin Liaison XL, and Maccura IS1200) met the optimal analytical quality specifications. On the other hand, Beckman DxI800 (13.2 %) and Roche Cobas E601 (-9.0 %) were only able to fulfill the desirable analytical quality specifications. The average pass rates for bias, CV, and TE of the trueness verification program were 80 %, 98 %, and 96 %, respectively.

Conclusions

The six automated immunoassays vs. all-assays median have a good correlation in CEA detection. However, there is a lack of comparability of CEA results. Further improvements are needed in harmonization among CEA detections.

Model of implementing proficiency testing in Vietnam, a developing country

Backgrounds and aims

The aim of this study is to provide a good approach for a quantitative EQA scheme assigned value with limited resources.

Materials and methods

Twelve lyophilized EQA items were distributed to participants in 2021 from North to Southeast Vietnam to measure the concentration of nine parameters, including glucose, urea, creatinine, cholesterol, triglyceride, uric acid, AST, ALT, and GGT. The consensus value of the expert group and all participants were calculated and statistically compared to choose the most appropriate consensus value.

Results

Fifty-nine laboratories attended the EQA scheme, including an expert group using automatic biochemistry analyzers (AAs) and all participants with auto and semi-auto biochemistry (SAA) analyzers. Consensus values of six per nine parameters were different between the two groups for at least two EQA items, including glucose, creatinine, cholesterol, uric acid, AST, and ALT. The coefficients of variation of glucose, urea, creatinine, triglycerides, uric acid, and GGT in the expert group were significantly lower than those in all the participants.

Conclusion

Using the consensus values of expert groups as the assigned values of the EQA program is a relevant strategy to increase testing quality in developing countries with limited resources, such as Vietnam.

Sigma metrics application for validated and non-validated detecting systems performance assessment

Background

Sigma metrics provide an objective and quantitative methodology for analytical quality evaluation of clinical laboratory. This study investigated the testing performance of validated systems and non‐validated systems based on sigma metrics, and explored the major parameters affecting the system performance.

Methods

Sigma metrics were evaluated by six biochemistry assays based on Beckman and Mindray validated and non‐validated systems through crossing the reagents and analyzers. Imprecision and bias were assessed for all assays based on trueness programs organized by National Centre for Clinical Laboratory. Total error allowance obtained from the Chinese Ministry of Health Clinical Laboratory Centre Industry Standard (WS/T403‐2012).

Results

The imprecision for all systems meets the quality specifications except TP assay (2.19%) detected by Mindray non‐validated system, and the bias for four assays measured by non‐validated systems cannot fulfill the criterion, including lactate dehydrogenase (LDH), total protein (TP), triglycerides (TG), and glucose (GLU). Higher biases were detected in six assays at different levels among non‐validated and validated systems. Systems performed poorly or unacceptably for TP assay with sigma metrics lower than 3 except Mindray non‐validated system. The sigma metrics for other assays with four systems were greater than 3 except the LDH evaluated on Mindray non‐validated systems.

Conclusion

Non‐validated systems may introduce performance uncertainty compared with validated systems based on sigma metrics evaluation, and lower bias was provided by validated systems. The performance of non‐validated systems should be evaluated thoroughly in the clinical laboratory before they were adopted for routine use.

Evaluation of serum electrolytes measurement through the 6-year trueness verification program in China

Objectives

The National Center for Clinical Laboratories (NCCL) in China initiated a serum electrolyte trueness verification (ETV) program in 2014 for measurement standardization.

Methods

Every year, two levels of fresh frozen commutable serum samples determined by inductively coupled plasma mass spectrometry (ICP-MS) reference methods were transported to participating clinical laboratories for the measurement of sodium, potassium, calcium and magnesium. Both samples were measured 15 times in 3 days, and the mean values and coefficient variations (CVs) were calculated from the results. The tolerance limits of trueness (bias), precision (CV) and accuracy (TE) based on the biological variation database were used as the evaluation criteria. The overall trend of the ETV program over 6 years was surveyed by calculating the pass rates of the participating laboratories. The mean bias, inter-laboratory CV, and TE of all laboratory results were analysed. Furthermore, homogeneous and heterogeneous systems were compared, and the bias and CV results of mainstream analysis systems were analysed.

Results

Pass rates of the three quality specifications increased, and the overall mean bias and inter-laboratory CVs decreased. The homogeneous system was superior to the heterogeneous system for calcium and magnesium measurements. For sodium, potassium, calcium and magnesium, the minimum bias corresponded to Hitachi, Siemens, Beckman AU and Roche, respectively. For inter-laboratory robust CVs, no obvious differences were observed between each peer group.

Conclusions

The commutable ETV materials assigned via reference methods can evaluate the accuracy and reproducibility of an individual laboratory and the calibration traceability and uniformity between laboratories for measurements.

Evaluation of serum alkaline phosphatase measurement through the 4-year trueness verification program in China

Background Alkaline phosphatase (ALP) is critical for various diseases. The International Federation of Clinical Chemistry and Laboratory Medicine had recommended the new reference procedure in 2011, but many manufacturers did not trace results to the higher procedure. Since 2012, the National Center for Clinical Laboratories (NCCL) in China has organized the trueness verification program (TV) with commutable materials. The present study summarizes the 4-year TV program to give an overview of the measurement standardization for ALP results. Methods Commutable serum-based materials with different concentrations were prepared and sent to participating laboratories. The target values were assigned by the reference lab network. Results The analytical performance was evaluated according to three indexes: trueness (bias), imprecision (CV) and accuracy (total error [TE]). The number of participating laboratories increased from 115 in 2012 to 287 in 2016. The pass rates of precision for homogeneous and heterogeneous systems were all above 85% over the 4 years; however, the pass rates of bias were much lower (<50%). Among the homogeneous systems, Roche Cobas/Modular had an obvious negative bias, whereas the mean positive bias for Beckman AU was prominent. As to the heterogeneous systems, the pass rates of bias for Sichuan Maccura (57.1%-78.6%) were higher than Roche Cobas/Modular (4.4%-33.9%) and Beckman AU (35.7%-64.8%). Conclusions The PT/EQA program with commutable materials can be used to assess the trueness against target values assigned by reference procedures. For ALP, homogeneous systems did not perform better than heterogeneous systems. The bias for ALP performance was notable and was the main obstacle to its standardization in China.

Comparative analysis of calculating sigma metrics by a trueness verification proficiency testing-based approach and an internal quality control data inter-laboratory comparison-based approach

Introduction

Two methods were compared for evaluating the sigma metrics of clinical biochemistry tests using two different allowable total error (TEa) specifications.

Materials and methods

The imprecision (CV%) and bias (bias%) of 19 clinical biochemistry analytes were calculated using a trueness verification proficiency testing (TPT)‐based approach and an internal quality control data inter‐laboratory comparison (IQC)‐based approach, respectively. Two sources of total allowable error (TEa), the Clinical Laboratory Improvement Amendments of 1988 (CLIA '88) and the People's Republic of China Health Industry Standard (WS/T 403‐2012), were used to calculate the sigma metrics (σ_CLIA, σ_WS/T). Sigma metrics were calculated to provide a single value for assessing the quality of each test based on a single concentration level.

Results

For both approaches, σ_CLIA > σ_WS/T in 18 out of 19 assays. For the TPT‐based approach, 16 assays showed σ_CLIA > 3, and 12 assays showed σ_WS/T > 3. For the IQC‐based approach, 19 and 16 assays showed σ_CLIA > 3 and σ_WS/T > 3, respectively.

Conclusions

Both methods can be used as references for calculating sigma metrics and designing QC schedules in clinical laboratories. Sigma metrics should be evaluated comprehensively by different approaches.

Investigation and Analysis of Hemoglobin A1c Measurement Systems' Performance for 135 Laboratories in China

Background:

Hemoglobin A1c (HbA1c) measurement is of great value for the diagnosis and monitoring of diabetes. Many manufacturers have developed various experiments to determine the HbA1c concentration. However, the longitudinal use of these tests requires strict quality management. This study aimed to analyze the quality of HbA1c measurement systems in China using six sigma techniques to help improve their performances.

Methods:

A total of 135 laboratories were involved in this investigation in 2015. Bias values and coefficients of variation were collected from an HbA1c trueness verification external quality assessment program and an internal quality control program organized by the National Center of Clinical Laboratories in China. The sigma (σ) values and the quality goal index (QGI) were used to evaluate the performances of different groups, which were divided according to principles and instruments.

Results:

The majority of participants (88, 65.2%) were scored as “improvement needed (σ < 3)”, suggesting that the laboratories needed to improve their measurement performance. Only 8.2% (11/135) of the laboratories were scored as “world class (σ ≥ 6)”. Among all the 88 laboratories whose σ values were below 3, 52 (59.1%) and 23 (26.1%) laboratories needed to improve measurement precision (QGI <8.0) and trueness (QGI >1.2), respectively; the remaining laboratories (13, 14.8%) needed to improve both measurement precision and trueness. In addition, 16.1% (5/31) and 15.0% (3/20) of the laboratories in “TOSOH” and “ARKRAY” groups, respectively, were scored as “world class”, whereas none of the laboratories in “BIO-RAD” group were scored as “world class”.

Conclusions:

This study indicated that, although participating laboratories were laboratories with better performance in China, the performances were still unsatisfactory. Actions should be taken to improve HbA1c measurement performance before we can include HbA1c assays in diabetes diagnosis in China.

Standardization of clinical enzyme analysis using frozen human serum pools with values assigned by the International Federation of Clinical Chemistry and Laboratory Medicine reference measurement procedures

Variation in clinical enzyme analysis, particularly across different measuring systems and laboratories, represents a critical but long-lasting problem in diagnosis. Calibrators with traceability and commutability are imminently needed to harmonize analysis in laboratory medicine. Fresh frozen human serum pools were assigned values for alanine aminotransferase (ALT), aspartate aminotransferase (AST), gamma-glutamyltransferase (GGT), creatine kinase (CK) and lactate dehydrogenase (LDH) by six laboratories with established International Federation of Clinical Chemistry and Laboratory Medicine reference measurement procedures. These serum pools were then used across 76 laboratories as a calibrator in the analysis of five enzymes. Bias and imprecision in the measurement of the five enzymes tested were significantly reduced by using the value-assigned serum in analytical systems with open and single-point calibration. The median (interquartile range) of the relative biases of ALT, AST, GGT, CK and LDH were 2.0% (0.6-3.4%), 0.8% (-0.8-2.3%), 1.0% (-0.5-2.0%), 0.2% (-0.3-1.0%) and 0.2% (-0.9-1.1%), respectively. Before calibration, the interlaboratory coefficients of variation (CVs) in the analysis of patient serum samples were 8.0-8.2%, 7.3-8.5%, 8.1-8.7%, 5.1-5.9% and 5.8-6.4% for ALT, AST, GGT, CK and LDH, respectively; after calibration, the CVs decreased to 2.7-3.3%, 3.0-3.6%, 1.6-2.1%, 1.8-1.9% and 3.3-3.5%, respectively. The results suggest that the use of fresh frozen serum pools significantly improved the comparability of test results in analytical systems with open and single-point calibration.