Analytical performance specifications for trace elements in biological fluids derived from six countries federated external quality assessment schemes over 10 years

OBJECTIVES

This article defines analytical performance specifications (APS) for evaluating laboratory proficiency through an external quality assessment scheme.

METHODS

Standard deviations for proficiency assessment were derived from Thompson's characteristic function applied to robust data calculated from participants' submissions in the Occupational and Environmental Laboratory Medicine (OELM) external quality assurance scheme for trace elements in serum, whole blood and urine. Characteristic function was based on two parameters: (1) β - the average coefficient of variation (CV) at high sample concentrations; (2) α - the average standard deviation (SD) at low sample concentrations. APSs were defined as 1.65 standard deviations calculated by Thompson's approach. Comparison between OELM robust data and characteristic function were used to validate the model.

RESULTS

Application of the characteristic function allowed calculated APS for 18 elements across three matrices. Some limitations were noted, particularly for elements (1) with no sample concentrations near analytical technique limit of detection; (2) exhibiting high robust CV at high concentration; (3) exhibiting high analytical variability such as whole blood Tl and urine Pb; (4) with an unbalanced number of robust SD above and under the characteristic function such as whole blood Mn and serum Al and Zn.

CONCLUSIONS

The characteristic function was a useful means of deriving APS for trace elements in biological fluids where biological variation data or outcome studies were not available. However, OELM external quality assurance scheme data suggests that the characteristic functions are not appropriate for all elements.

Recommendations on the measurement and use of the alcohol consumption biomarker CDT. A position paper from the IFCC Working Group on CDT standardisation

BACKGROUND

Carbohydrate deficient transferrin (CDT) is a biomarker for excessive alcohol consumption utilized in clinical and forensic medicine and workplace testing. Previously, many different analytical methods for CDT were used and the measurand varied considerably, making direct comparison of test results difficult. To end this confusion, the IFCC established a working group on CDT standardisation (WG-CDT) which completed its tasks in 2017.

METHODS

This IFCC position paper by the WG-CDT summarizes state of the art information about the measurand and the analytical methods and gives concise recommendations for its utilization.

RESULTS

The results achieved by the CDT standardisation process led to accuracy improvements in national external quality assessment schemes over the years. A brief review of ROC based comparison studies with the traditional biomarkers (GGT, MCV, ALT and AST) discusses the bias resulting from inadequate study populations. In large groups of the general population the superior diagnostic performance of CDT is confirmed.

CONCLUSION

The relationship between alcohol intake versus resulting CDT is discussed as well as the cutoff and measurement uncertainty. Concerning the application in practice, potential pitfalls are considered and recommendations handling both analytical and preanalytical caveats are given. Finally, some examples of serious misunderstandings in publications about CDT are addressed.

The risk of clinical misinterpretation of HbA1c: Modelling the impact of biological variation and analytical performance on HbA1c used for diagnosis and monitoring of diabetes

BACKGROUND

The validity of clinical interpretation of HbA1c depends on the analytical performance of the method and the biological variation of HbA1c in patients. The contribution of non-glucose related factors to the biological variation of HbA1c (NGBVA1c) is not known. This paper explores the cumulative impact of analytical errors and NGBVA1c on the risk of misinterpretation.

METHODS

A model has been developed to predict the risk of misinterpretation of HbA1c for diagnosis and monitoring with variables for analytical performance and levels of NGBVA1c.

RESULTS

The model results in probabilities of misinterpretation for a given HbA1c.

EXAMPLE

for an HbA1c 43 mmol/mol (6.1%), bias 1 mmol/mol (0.09%), CV 3% (2%) used for diagnosis, the probabilities of misinterpretation range from 1 to 19% depending on the contribution of NGBVA1c to the biological variation of HbA1c.

CONCLUSIONS

In addition to analytical bias and imprecision, NGBVA1c contributes to the risk of misinterpretation, but the relative impact is different per clinical application of HbA1c. For monitoring, imprecision is the predominating factor, for diagnosis both biological variation and analytical bias. Given the increasing use of HbA1c for diagnosis, increase of knowledge on NGBVA1c, decrease of analytical bias, and awareness of the risk of misinterpretation are required.

IFCC Working Group Recommendations for Correction of Bias Caused by Noncommutability of a Certified Reference Material Used in the Calibration Hierarchy of an End-User Measurement Procedure

Establishing metrological traceability to an assigned value of a matrix-based certified reference material (CRM) that has been validated to be commutable among available end-user measurement procedures (MPs) is central to producing equivalent results for the measurand in clinical samples (CSs) irrespective of the clinical laboratory MPs used. When a CRM is not commutable with CSs, the bias due to noncommutability will be propagated to the CS results causing incorrect metrological traceability to the CRM and nonequivalent CS results among different MPs. In a commutability assessment, a conclusion that a CRM is commutable or noncommutable for use with a specific MP is made when the difference in bias between the CRM and CSs meets or does not meet a criterion for that specific MP when compared to other MPs. A conclusion regarding commutability or noncommutability requires that the magnitude of the difference in bias observed in the commutability assessment remains unchanged over time. This conclusion requires the CRM to be stable and no substantive changes in the MPs. These conditions should be periodically reverified. If an available CRM is determined to be noncommutable for a specific MP, that CRM can be used in the calibration hierarchy for that MP when an appropriately validated MP-specific correction for the noncommutability bias is included. We describe with examples how a MP-specific correction and its uncertainty can be developed and applied in a calibration hierarchy to achieve metrological traceability of results for CSs to the CRM's assigned value.

Feasibility for aggregation of commutable external quality assessment results to evaluate metrological traceability and agreement among results

Objectives

External quality assessment (EQA) with commutable samples is used for assessing agreement of results for patients' samples. We investigated the feasibility to aggregate results from four different EQA schemes to determine the bias between different measurement procedures and a reference target value.

Methods

We aggregated EQA results for creatinine from programs that used commutable EQA material by calculating the relative difference between individual participant results and the reference target value for each sample. The means and standard errors of the means were calculated for the relative differences. Results were partitioned by methods, manufacturers and instrument platforms to evaluate the biases for the measurement procedures.

Results

Data aggregated for enzymatic methods had biases that varied from -8.2 to 3.8% among seven instrument platforms for creatinine at normal concentrations (61-85 μmol/L). EQA schemes differed in the evidence provided about the commutability of their samples, and in the amount of detail collected from participants regarding the measurement procedures which limited the ability to sub-divide aggregated data by instrument platforms and models.

Conclusions

EQA data could be aggregated from four different programs using different commutable samples to determine bias among different measurement procedures. Criteria for commutability for EQA samples as well as standardization of reporting the measurement methods, reagents, instrument platforms and models used by participants are needed to improve the ability to aggregate the results for optimal assessment of performance of measurement procedures. Aggregating data from a larger number of EQA schemes is feasible to assess trueness on a global scale.

External quality assessment schemes for inorganic elements in the clinical laboratory: Lessons from the OELM scheme

Measurements of inorganic elements in clinical laboratories produce results used for the diagnosis, the treatment and the monitoring of deficiencies or overloads. The main objective of External Quality Assessment Schemes is to verify, on a regular frequency, that clinical laboratory results correspond to the quality requirement for patient care. Therefore, External Quality Assessment Schemes represent an essential component of a laboratory's quality management system. However, External Quality Assessment Schemes within the same analytical field remain heterogeneous for different reasons such as samples, determination of assigned value, acceptable limits, content of the reports. The aim of this review was to describe and illustrate some major critical aspects of External Quality Assessment Schemes based on Occupational and Environmental Laboratory Medicine external quality assessment scheme experience.

The global impact of the International Federation of Clinical Chemistry and Laboratory Medicine, Education and Management Division: engaging stakeholders and assessing HbA1c quality in a multicentre study across China

Background Diabetes mellitus is a major global issue and high quality testing is essential for the diagnosis and treatment of the disease. The IFCC Committee for the Education in the Utility of Biomarkers in Diabetes (C-EUBD) plays a global role in improving knowledge and understanding around diabetes testing. This paper describes a multi-stakeholder approach, to improving diagnostic and therapeutic testing for diabetes, using a multicentre study in China as an example of the global impact of the group. Methods Educational workshops were developed to support the scientific aims of the study in which 30 centres around China received identical, fresh frozen whole blood samples with values assigned using IFCC secondary reference methods and undertook precision (EP-5) and trueness studies. Performance was assessed using sigma metrics. Results A successful multi-stakeholder group was developed and sustained throughout the study through several educational workshops, which enabled the formation of a long-term collaboration with key opinion leaders and policy makers in China. All 30 centres showed good performance with within and between laboratory coefficient of variations (CVs) below 3% in SI units at both low and high haemoglobin A1c (HbA1c) levels. All individual laboratories met the criteria of a sigma of two or more at a total allowable error (TAE) of 5 mmol/mol (0.46% NGSP). Conclusions The study led to a successful multi-partner approach to improving diabetes testing in China. All centres involved in the study meeting the published IFCC quality criteria, paving the way for future clinical trials and an expanded role for HbA1c testing across the country.

EurA1c: The European HbA1c Trial to Investigate the Performance of HbA1c Assays in 2166 Laboratories across 17 Countries and 24 Manufacturers by Use of the IFCC Model for Quality Targets

BACKGROUND

A major objective of the IFCC Committee on Education and Use of Biomarkers in Diabetes is to generate awareness and improvement of HbA1c assays through evaluation of the performance by countries and manufacturers.

METHODS

Fresh whole blood and lyophilized hemolysate specimens manufactured from the same pool were used by 17 external quality assessment organizers to evaluate analytical performance of 2166 laboratories. Results were evaluated per country, per manufacturer, and per manufacturer and country combined according to criteria of the IFCC model for quality targets.

RESULTS

At the country level with fresh whole blood specimens, 6 countries met the IFCC criterion, 2 did not, and 2 were borderline. With lyophilized hemolysates, 5 countries met the criterion, 2 did not, and 3 were borderline. At the manufacturer level using fresh whole blood specimens, 13 manufacturers met the criterion, 8 did not, and 3 were borderline. Using lyophilized hemolysates, 7 manufacturers met the criterion, 6 did not, and 3 were borderline. In both country and manufacturer groups, the major contribution to total error derived from between-laboratory variation. There were no substantial differences in performance between groups using fresh whole blood or lyophilized hemolysate samples.

CONCLUSIONS

The state of the art is that 1 of 20 laboratories does not meet the IFCC criterion, but there are substantial differences between country and between manufacturer groups. Efforts to further improve quality should focus on reducing between-laboratory variation. With some limitations, fresh whole blood and well-defined lyophilized specimens are suitable for purpose.

Evaluation of Performance of Laboratories and Manufacturers Within the Framework of the IFCC model for Quality Targets of HbA1c

HbA1c is a key parameter in diabetes management. For years the test has been used exclusively for monitoring of long-term diabetic control. However, due to improvement of the performance, HbA1c is considered more and more for diagnosis and screening. With this new application, quality demands further increase. A task force of the International Federation of Clinical Chemistry and Laboratory Medicine developed a model to set and evaluate quality targets for HbA1c. The model is based on the concept of total error and takes into account the major sources of analytical errors in the medical laboratory: bias and imprecision. Performance criteria are derived from sigma-metrics and biological variation. This review shows 2 examples of the application of the model: at the level of single laboratories, and at the level of a group of laboratories. In the first example data of 125 individual laboratories of a recent external quality assessment program in the Netherlands are evaluated. Differences between laboratories as well as their relation to method principles are shown. The second example uses recent and 3-year-old data of the proficiency test of the College of American Pathologists. The differences in performance between 26 manufacturer-related groups of laboratories are shown. Over time these differences are quite consistent although some manufacturers improved substantially either by better standardization or by replacing a test. The IFCC model serves all who are involved in HbA1c testing in the ongoing process of better performance and better patient care.

Clinically relevant discrepancies between different rheumatoid factor assays

Background:

Accurate measurements of rheumatoid factors (RFs), autoantibodies binding IgG, are important for diagnosing rheumatoid arthritis (RA) and for predicting disease course. Worldwide, various RF assays are being used that differ in technique and target antigens. We studied whether assay choice leads to clinically important discrepancies in RF status and level.

Methods:

RF measurements using four commercial RF assays were compared in 32 RF+ samples. Using enzyme-linked immunosorbent assays (ELISAs), the influence of the target antigen source – human IgG (hIgG) versus rabbit IgG (rIgG) – on measured RF levels was investigated in arthralgia patients and RA patients.

Results:

Substantial discrepancies were found between RF levels measured in the four commercial assays. Six samples (19%) with RF levels below or slightly above the cutoff in the rIgG-based Phadia assay were RF+ in three assays using hIgG as the target antigen, some with very high levels. Direct ELISA comparisons of RF reactivity against hIgG and rIgG estimated that among 173 ACPA+ arthralgia patients, originally RF negative in rIgG-based assays, up to 10% were single positive against hIgG. Monoclonal RFs binding to hIgG and rIgG or hIgG only supported these findings. In a cohort of 69 early RA patients, virtually all RF responses reacted with both targets, although levels were still variable.

Conclusions:

The use of RF assays that differ in technique and target antigen, together with the different specificities of RF responses, leads to discrepancies in RF status and levels. This has important consequences for patient care if RA diagnosis and disease progression assessments are based on RF test results.

IFCC Working Group Recommendations for Assessing Commutability Part 1: General Experimental Design

Commutability is a property of a reference material (RM) that relates to the closeness of agreement between results for an RM and results for clinical samples (CSs) when measured by ≥2 measurement procedures (MPs). Commutability of RMs used in a calibration traceability scheme is an essential property for them to be fit for purpose. Similarly, commutability of trueness controls or external quality assessment samples is essential when those materials are used to assess trueness of results for CSs. This report is part 1 of a 3-part series describing how to assess commutability of RMs. Part 1 defines commutability and addresses critical components of the experimental design for commutability assessment, including selection of individual CSs, use of pooled CSs, qualification of MPs for inclusion, establishing criteria for the determination that an RM is commutable, generalization of commutability conclusions to future measurements made with the MPs included in the assessment, and information regarding commutability to be included in the certificate for an RM. Parts 2 and 3 in the series present 2 different statistical approaches to commutability assessment that use fixed criteria related to the medical decisions that will be made using the laboratory test results.

IFCC Working Group Recommendations for Assessing Commutability Part 2: Using the Difference in Bias between a Reference Material and Clinical Samples

A process is described to assess the commutability of a reference material (RM) intended for use as a calibrator, trueness control, or external quality assessment sample based on the difference in bias between an RM and clinical samples (CSs) measured using 2 different measurement procedures (MPs). This difference in bias is compared with a criterion based on a medically relevant difference between an RM and CS results to make a conclusion regarding commutability. When more than 2 MPs are included, the commutability is assessed pairwise for all combinations of 2 MPs. This approach allows the same criterion to be used for all combinations of MPs included in the assessment. The assessment is based on an error model that allows estimation of various random and systematic sources of error, including those from sample-specific effects of interfering substances. An advantage of this approach is that the difference in bias between an RM and the average bias of CSs at the concentration (i.e., amount of substance present or quantity value) of the RM is determined and its uncertainty estimated. An RM is considered fit for purpose for those MPs for which commutability is demonstrated.

IFCC Working Group Recommendations for Assessing Commutability Part 3: Using the Calibration Effectiveness of a Reference Material

A process is described to assess the commutability of a reference material (RM) intended for use as a calibrator based on its ability to fulfill its intended use in a calibration traceability scheme to produce equivalent clinical sample (CS) results among different measurement procedures (MPs) for the same measurand. Three sources of systematic error are elucidated in the context of creating the calibration model for translating MP signals to measurand amounts: calibration fit, calibrator level trueness, and commutability. An example set of 40 CS results from 7 MPs is used to illustrate estimation of bias and variability for each MP. The candidate RM is then used to recalibrate each MP, and its effectiveness in reducing the systematic error among the MPs within an acceptable level of equivalence based on medical requirements confirms its commutability for those MPs. The RM is declared noncommutable for MPs for which, after recalibration, the CS results do not agree with those from other MPs. When a lack of agreement is found, other potential causes, including lack of calibration fit, should be investigated before concluding the RM is noncommutable. The RM is considered fit for purpose for those MPs where commutability is demonstrated.

The quest for equivalence of test results: the pilgrimage of the Dutch Calibration 2.000 program for metrological traceability

Calibration 2.000 was initiated 20 years ago for standardization and harmonization of medical tests. The program also intended to evaluate adequate implementation of the In Vitro Diagnostics (IVD) 98/79/EC directive, in order to ensure that medical tests are fit-for-clinical purpose. The Calibration 2.000 initiative led to ongoing verification of test standardization and harmonization in the Netherlands using commutable external quality assessment (EQA)-tools and a type 1 EQA-design, where feasible. National support was guaranteed by involving all laboratory professionals as well as laboratory technicians responsible for EQA and quality officers. A category 1 EQA-system for general chemistry analytes, harmonizers for specific analytes like hGH and IGF-1, and commutable materials for other EQA-sections have been developed and structurally introduced in the EQA-schemes. The type 1 EQA-design facilitates the dialogue between individual specialists in laboratory medicine and the IVD-industry to reduce lot-to-lot variation and to improve standardization. In such a way, Calibration 2.000 sheds light on the metrological traceability challenges that we are facing and helps the laboratory community to get the issues on the table and resolved. The need for commutable trueness verifiers and/or harmonizers for other medical tests is now seen as paramount. Much knowledge is present in the Netherlands and for general chemistry, humoral immunology and protein chemistry, a few endocrinology tests, and various therapeutic drug monitoring (TDM) tests, commutable materials are available. Also the multi sample evaluation scoring system (MUSE) and the category 1 EQA-design offer many possibilities for permanent education of laboratory professionals to further improve the between and within laboratory variation and the test equivalence.

Analytical performance of 17 general chemistry analytes across countries and across manufacturers in the INPUtS project of EQA organizers in Italy, the Netherlands, Portugal, United Kingdom and Spain

Background:

Optimum patient care in relation to laboratory medicine is achieved when results of laboratory tests are equivalent, irrespective of the analytical platform used or the country where the laboratory is located. Standardization and harmonization minimize differences and the success of efforts to achieve this can be monitored with international category 1 external quality assessment (EQA) programs.

Methods:

An EQA project with commutable samples, targeted with reference measurement procedures (RMPs) was organized by EQA institutes in Italy, the Netherlands, Portugal, UK, and Spain. Results of 17 general chemistry analytes were evaluated across countries and across manufacturers according to performance specifications derived from biological variation (BV).

Results:

For K, uric acid, glucose, cholesterol and high-density density (HDL) cholesterol, the minimum performance specification was met in all countries and by all manufacturers. For Na, Cl, and Ca, the minimum performance specifications were met by none of the countries and manufacturers. For enzymes, the situation was complicated, as standardization of results of enzymes toward RMPs was still not achieved in 20% of the laboratories and questionable in the remaining 80%.

Conclusions:

The overall performance of the measurement of 17 general chemistry analytes in European medical laboratories met the minimum performance specifications. In this general picture, there were no significant differences per country and no significant differences per manufacturer. There were major differences between the analytes. There were six analytes for which the minimum quality specifications were not met and manufacturers should improve their performance for these analytes. Standardization of results of enzymes requires ongoing efforts.

Diagnostic performance of serological assays for anti-HBs testing: Results from a quality assessment program

BACKGROUND

Post-vaccination testing after hepatitis B vaccination is indispensable to evaluate long-term immunological protection. Using a threshold level of antibodies against hepatitis B surface antigen (anti-HBs) to define serological protection, implies reproducible and valid measurements of different diagnostic assays.

OBJECTIVES

In this study we assess the performance of currently used anti-HBs assays.

STUDY DESIGN

In 2013, 45 laboratories participated in an external quality assessment program using pooled anti-HBs serum samples around the cutoff values 10IU/l and 100IU/l. Laboratories used either Axsym (Abbott Laboratories), Architect (Abbott Laboratories), Access (Beckman-Coulter), ADVIA Centaur anti-HBs2 (Siemens Healthcare Diagnostics), Elecsys, Modular or Cobas (Roche Diagnostics) or Vidas Total Quick (Biomerieux) for anti-HBs titre quantification. We analysed covariance using mixed-model repeated measures. To assess sensitivity/specificity and agreement, a true positive or true negative result was defined as an anti-HBs titre respectively above or below the cutoff value by ≥4 of 6 assays.

RESULTS

Different anti-HBs assays were associated with statistically significant (P<0.05) differences in anti-HBs titres in all dilutions. Sensitivity and specificity ranged respectively from 64%-100% and 95%-100%. Agreement between assays around an anti-HBs titre cutoff value of 10IU/l ranged from 93%-100% and was 44% for a cutoff value of 100IU/l.

CONCLUSIONS

Around a cutoff value of 10IU/l use of the Access assay may result in false-negative results. Concerning the cutoff value of 100IU/l, a sample being classified below or above this cutoff relied heavily on the specific assay used, with both the Architect and the Access resulting in false-negative results.

Post-standardization of routine creatinine assays: are they suitable for clinical applications

Introduction

Reliable serum creatinine measurements are of vital importance for the correct classification of chronic kidney disease and early identification of kidney injury. The National Kidney Disease Education Programme working group and other groups have defined clinically acceptable analytical limits for creatinine methods. The aim of this study was to re-evaluate the performance of routine creatinine methods in the light of these defined limits so as to assess their suitability for clinical practice.

Method

In collaboration with the Dutch External Quality Assurance scheme, six frozen commutable samples, with a creatinine concentration ranging from 80 to 239  μmol/L and traceable to isotope dilution mass spectrometry, were circulated to 91 laboratories in four European countries for creatinine measurement and estimated glomerular filtration rate calculation. Two out of the six samples were spiked with glucose to give high and low final concentrations of glucose.

Results

Results from 89 laboratories were analysed for bias, imprecision (%CV) for each creatinine assay and total error for estimated glomerular filtration rate. The participating laboratories used analytical instruments from four manufacturers; Abbott, Beckman, Roche and Siemens. All enzymatic methods in this study complied with the National Kidney Disease Education Programme working group recommended limits of bias of 5% above a creatinine concentration of 100  μmol/L. They also did not show any evidence of interference from glucose. In addition, they also showed compliance with the clinically recommended %CV of ≤4% across the analytical range. In contrast, the Jaffe methods showed variable performance with regard to the interference of glucose and unsatisfactory bias and precision.

Conclusion

Jaffe-based creatinine methods still exhibit considerable analytical variability in terms of bias, imprecision and lack of specificity, and this variability brings into question their clinical utility. We believe that clinical laboratories and manufacturers should work together to phase out the use of relatively non-specific Jaffe methods and replace them with more specific methods that are enzyme based.

Effects of hemoglobin C, D, E and S traits on measurements of hemoglobin A1c by twelve methods

BACKGROUND

Hemoglobin C, D Punjab, E or S trait can interfere with hemoglobin A1c (HbA1c) results. We assessed whether they affect results obtained with 12 current assay methods.

METHODS

Hemoglobin AA (HbAA), HbAC, HbAD Punjab, HbAE and HbAS samples were analyzed on one enzymatic, nine ion-exchange HPLC and two Capillary Electrophoresis methods. Trinity ultra(2) boronate affinity HPLC was the comparative method. An overall test of coincidence of least-squared linear regression lines was performed to determine if HbA1c results were statistically significantly different from those of HbAA samples. Clinically significant interference was defined as >7% difference from HbAA at 6 or 9% HbA1c compared to ultra(2) using Deming regression.

RESULTS

All methods showed statistically significant effects for one or more variants. Clinically significant effects were observed for the Tosoh G8 variant mode and GX (all variants), GX V1.22 (all but HbAE) and G11 variant mode (HbAC). All other methods (Abbott Architect c Enzymatic, Bio-Rad D-100, Variant II NU and Variant II Turbo 2.0, Menarini HA-8180T thalassemia mode and HA-8180V variant mode, Sebia Capillarys 2 and Capillarys 3) showed no clinically significant differences.

CONCLUSIONS

Several methods showed clinically significant interference with HbA1c results from one or more variants which could adversely affect patient care.

HbA1c: EQA in Germany, Belgium and the Netherlands using fresh whole blood samples with target values assigned with the IFCC reference system

Background:

External quality assessment/proficiency test (EQA/PT) organizers play an important role in monitoring the performance of HbA

Methods:

Throughout 2015, ten fresh whole blood samples were supplied to the participants. Aggregated results were evaluated according to the IFCC model for quality targets at four levels: overall, per country, per manufacturer, and per country per manufacturer.

Results:

Robust results in summer and winter demonstrated the feasibility of organizing an EQA with fresh whole blood samples in three countries. The overall performances, as well as the performance for each country were very similar: results fell within the IFCC criteria. Although substantial differences between results from different manufacturers were present, the performances of laboratories using tests of the same manufacturer were strikingly similar in the three countries, suggesting that the quality of HbA