Establishing, evaluating and monitoring analytical quality in the traceability era

Poor analytical quality may be the bane of medical use of laboratory tests, and the fight against excessive analytical variability presents a daily struggle. Laboratories should prioritize the perspectives and needs of their customers (the patients and healthcare personnel). Among them, comparability of results from the same patient sample when measured by different laboratories using different in vitro diagnostic (IVD) medical devices is a logical priority to avoid result misinterpretation and potential patient harm. Harmonization (standardization) of laboratory measurements can be achieved by establishing metrological traceability of the results on clinical samples to stated higher-order references and providing an estimate of the uncertainty of measurement (MU). This estimate should be based on an MU budget including all known MU contributions generated by the employed calibration hierarchy, which in turn should be validated against fit-for-purpose maximum allowable MU derived according to internationally recommended models. In this report, we review the available strategies for establishing, evaluating, and monitoring analytical quality, drawing on three decades experience in the field. We discuss the most important aspects that may influence obtaining and maintaining analytical standardization in laboratory medicine, and offer practical solutions aimed at educating all stakeholders for the achievement of harmonized laboratory results. To fully implement the recommended approaches, all involved parties-i.e. reference providers, IVD manufacturers, medical laboratories, and External Quality Assessment organizers-must agree on their importance and enhance their specific knowledge.

What the Milan conference has taught us about analytical performance specification model definition and measurand allocation

Analytical performance specifications (APS) represent the criteria that specify the quality required for laboratory test information to satisfy clinical needs. In 2014 the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) considered timely to update the topic of APS by organizing a conference in Milan in which some strategic concepts were proposed. Here I summarize the essential points representing the EFLM Strategic Conference heritage and discuss the approaches that will permit us to become more concrete, including roles and main actions expected from each of involved stakeholders for contributing a quantum leap forward in the way of practicality of Milan consensus about APS.

Applying the Milan models to setting analytical performance specifications - considering all the information

Analytical performance specifications (APS) are used for decisions about the required analytical quality of pathology tests to meet clinical needs. The Milan models, based on clinical outcome, biological variation, or state of the art, were developed to provide a framework for setting APS. An approach has been proposed to assign each measurand to one of the models based on a defined clinical use, physiological control, or an absence of quality information about these factors. In this paper we propose that in addition to such assignment, available information from all models should be considered using a risk-based approach that considers the purpose and role of the actual test in a clinical pathway and its impact on medical decisions and clinical outcomes in addition to biological variation and the state-of-the-art. Consideration of APS already in use and the use of results in calculations may also need to be considered to determine the most appropriate APS for use in a specific setting.

State-of-the-art model for derivation of analytical performance specifications: how to define the highest level of analytical performance technically achievable

To be accurate and equivalent among assays, laboratory results should be traceable to higher-order references and their quality should fulfill maximum allowable measurement uncertainty (MU) as defined to fit the intended clinical use. Accordingly, laboratory professionals should estimate and validate MU of performed tests using appropriate analytical performance specifications (APS). Current consensus supports the derivation of APS by using one of the three models established by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Strategic Conference held in Milan in 2014. It is recognized that some models are better suited for certain measurands than for others and the attention should be primarily directed towards their biological and clinical characteristics. Among others, model 3 should reflect the state of the art of the measurements that can be defined as the best analytical performance that is technically achievable. Taking serum C-reactive protein and ferritin as examples, here we describe the theoretical premises and the experimental protocol to be used to derive APS for MU when a measurand is allocated to this model. Although the model lacks a direct relationship with clinical outcomes, useful information about the in vitro diagnostic medical device performance and the average quality of provided results may be obtained.

Analytical performance and user-friendliness of four commercially available point-of-care devices for C-reactive protein

INTRODUCTION

Proper implementation of Point-of-Care testing (POCT) for C-reactive protein (CRP) in primary care can decrease the inappropriate use of antibiotics, thereby tackling the problem of growing antimicrobial resistance.

OBJECTIVE

The analytical performance and user-friendliness of four POCT-CRP assays were evaluated: QuikRead go easy, LumiraDx, cobas b 101 and Afinion 2.

MATERIALS AND METHODS

Imprecision was evaluated using plasma pools in addition to manufacturer-specific control material. Trueness was assessed by verification of traceability to ERM-DA474/IFCC in parallel to method comparison towards the central laboratory CRP method (cobas c 503) using i) retrospectively selected plasma samples (n = 100) and ii) prospectively collected capillary whole blood samples (n = 50). User-friendliness was examined using a questionnaire.

RESULTS

Between-day imprecision on plasma pools varied from 4.5 % (LumiraDx) to 11.5 % (QuikRead). Traceability verification revealed no significant difference between cobas c 503 CRP results and the ERM-DA474/IFCC certified value. cobas b 101 and Afinion achieved the best agreement with the central laboratory method. LumiraDx and QuikRead revealed a negative mean difference, with LumiraDx violating the criterion of > 95 % of POCT-CRP-results within ± 20 % of the comparison method. Regarding user-friendliness, Afinion obtained the highest Likert-scores.

CONCLUSION

The analytical performance and user-friendliness of POCT-CRP devices varies among manufacturers, emphasizing the need for quality assurance supervised by a central laboratory.

Are analytical performance specifications derived from reference intervals of any use in the veterinary clinical laboratory? A preliminary study on the empirical biological variation model

BACKGROUND

Analytical performance specifications (APS) are vital for method evaluation and quality control validation. However, the limited availability of biological variation (BV) data, regulatory guidelines, and expert opinion (EO) may present challenges in veterinary medicine. The empirical biological variation (EBV) approach, based on population reference intervals (pRI), has emerged as an alternative method to derive APS in human medicine.

OBJECTIVES

This study aimed to assess the practicality and usefulness of the EBV approach in deriving performance limits for various measurands in dogs and cats.

METHODS

Eight hematology and 13 biochemistry measurands were analyzed in dogs and cats. Estimates of combined biologic variation based on traditional biological (CVB ) and EBV-derived (CVE *) formulas were calculated and assessed for evidence of correlation. Performance limits for expanded uncertainty/total error and imprecision were compared among EO, BV, and EBV.

RESULTS

Strong and significant correlations were found between CVB and CVE * for both dogs (r = .86, p < .00001) and cats (r = 0.95, p < .00001). The EBV-derived APS were generally comparable to EO and BV, with a subjective criterion of 1.5% difference for imprecision and 3% for total error/expanded uncertainty.

CONCLUSION

The EBV approach, using pRI, shows promise as a surrogate marker for biological variation and as a practical tool for determining performance limits in dogs and cats. Assuming accurate pRI generated on analyzers with stable analytical performance, this approach could offer benefits when expert recommendations or robust BV studies are lacking or yield conflicting results. Further research is needed to explore the applicability and advantages of the EBV in veterinary medicine.

Current performance of C-reactive protein determination and derivation of quality specifications for its measurement uncertainty

From External Quality Assessment data, current harmonization of CRP measuring systems appears to be satisfactory, the inter-assay CV being well below 10%. The inter-method variability is even better (close to 3%) when the widely used measuring systems are compared at CRP concentrations employed as cut-off for detecting sub-clinical infection (i.e., 10.0 mg/L) and measurement variability estimated, according to ISO 20914:2019 Technical Specification, from the intermediate within-lab reproducibility of 6-month consecutive measurement data. According to the state-of-the-art model (which is better suited for CRP), the maximum allowable measurement uncertainty (MAU) for CRP measurement on clinical samples with 10.0 mg/L concentrations is 3.76% (desirable quality). As measurement uncertainty (MU) of the only available reference material (ERM-DA474/IFCC) is ∼3%, to fulfil desirable MAU on clinical samples, IVD manufacturers should work to keep the contribution of remaining MU sources (commercial calibrator and intermediate within-lab reproducibility) lower than 2.3%.

C-reactive protein and clinical outcome in COVID-19 patients: the importance of harmonized measurements

C-reactive protein (CRP) is a cytokine-mediated acute phase reactant with a recognized role in inflammatory conditions and infectious disease. In coronavirus disease 2019 (COVID-19), elevated CRP concentrations in serum were frequently detected and significantly associated with poor outcome in terms of disease severity, need for intensive care, and in-hospital death. For these reasons, the marker was proposed as a powerful test for prognostic classification of COVID-19 patients. In most of available publications, there was however confounding information about how interpretative criteria for CRP in COVID-19 should be derived, including quality of employed assays and optimal cut-off definition. Assuring result harmonization and controlling measurement uncertainty in terms of performance specifications are fundamental to allow worldwide application of clinical information according to specific CRP thresholds and to avoid risk of patient misclassification.

Analytical performance of 17 commercially available point-of-care tests for CRP to support patient management at lower levels of the health system

Accurate and precise point-of-care (POC) testing for C-reactive protein (CRP) can help support healthcare providers in the clinical management of patients. Here, we compared the analytical performance of 17 commercially available POC CRP tests to enable more decentralized use of the tool. The following CRP tests were evaluated. Eight quantitative tests: QuikRead go (Aidian), INCLIX (Sugentech), Spinit (Biosurfit), LS4000 (Lansionbio), GS 1200 (Gensure Biotech), Standard F200 (SD Biosensor), Epithod 616 (DxGen), IFP-3000 (Xincheng Biological); and nine semi-quantitative tests: Actim CRP (ACTIM), NADAL Dipstick (nal von minden), NADAL cassette (nal von minden), ALLTEST Dipstick (Hangzhou Alltest Biotech), ALLTEST Cassette cut-off 10-40-80 (Hangzhou Alltest Biotech), ALLTEST Cassette cut-off 10-30 (Hangzhou Alltest Biotech), Biotest (Hangzhou Biotest Biotech), BTNX Quad Line (BTNX), BTNX Tri Line (BTNX). Stored samples (n = 660) had previously been tested for CRP using Cobas 8000 Modular analyzer (Roche Diagnostics International AG, Rotkreuz, Switzerland (reference standards). CRP values represented the clinically relevant range (10-100 mg/L) and were grouped into four categories (<10 mg/L, 10-40 mg/L or 10-30 mg/L, 40-80 mg/L or 30-80 mg/L, and > 80mg/L) for majority of the semi-quantitative tests. Among the eight quantitative POC tests evaluated, QuikRead go and Spinit exhibited better agreement with the reference method, showing slopes of 0.963 and 0.921, respectively. Semi-quantitative tests with the four categories showed a poor percentage agreement for the intermediate categories and higher percentage agreement for the lower and upper limit categories. Analytical performance varied considerably for the semi-quantitative tests, especially among the different categories of CRP values. Our findings suggest that quantitative tests might represent the best choice for a variety of use cases, as they can be used across a broad range of CRP categories.

Redesigning the surveillance of in vitro diagnostic medical devices and of medical laboratory performance by quality control in the traceability era

IVD manufacturers have total responsibility in terms of the traceability of marketed in vitro diagnostic medical devices (IVD-MD). This includes the provision of a quality control (QC) material as a part of the measuring system, suitable for traceability verification and alignment surveillance by end-users in daily practice. This material [to be used for the internal QC (IQC) component I as described in this paper] should have unbiased target values and an acceptability range corresponding to analytical performance specifications (APS) for suitable (expanded) measurement uncertainty (MU) on clinical samples. On the other hand, medical laboratories (by the IQC component II as described in this paper) should improve the IQC process and its judging criteria to establish a direct link between their performance, estimated as MU of provided results, and APS defined according to recommended models to apply corrective actions if the performance is worsening with the risk to jeopardize the clinical validity of test results. The participation to external quality assessment (EQA) programs that meet specific metrological criteria is also central to the evaluation of performance of IVD-MDs and of medical laboratories in terms of harmonization and clinical suitability of their measurements. In addition to the use of commutable materials, in this type of EQA it is necessary to assign values to them with selected reference procedures and to define and apply maximum allowable APS to substantiate the suitability of laboratory measurements in the clinical setting.

Optimizing Available Tools for Achieving Result Standardization: Value Added by Joint Committee on Traceability in Laboratory Medicine (JCTLM)

BACKGROUND

The JCTLM created a Task Force on Reference Measurement System Implementation (TF-RMSI) to provide guidance on metrological traceability implementation for the in vitro diagnostics (IVD) community.

CONTENT

TF-RMSI investigated the reference measurement systems (RMS) for 13 common measurands by applying the following procedural steps: (a) extracting data from the JCTLM database of available certified reference materials (CRMs) and reference measurement procedures (RMPs); (b) describing the RMS to which each recruited CRM or RMP belongs; (c) identifying the intended use of the CRMs, and, if used as a common calibrator for IVD measuring systems and/or trueness assessment of field methods was included, checking the CRM's certificate for information about commutability with clinical samples; and (d) checking if the CRM or RMP measurement uncertainty (MU) has the potential to be small enough to avoid significantly affecting the analytical performance specifications (APS) for MU of clinical sample results when the MU from the IVD calibrator and from the end-user measuring system were combined.

SUMMARY

We produced a synopsis of JCTLM-listed higher-order CRMs and RMPs for the selected measurands, including their main characteristics for implementing traceability and fulfilling (or not) the APS for suitable MU. Results showed that traceability to higher-order references can be established by IVD manufacturers within the defined APS for most of the 13 selected measurands. However, some measurands do not yet have suitable CRMs for use as common calibrators. For these measurands, splitting clinical samples with a laboratory performing the RMP may provide a practical alternative for establishing a calibration hierarchy.

Sources and clinical significance of aspartate aminotransferase increases in COVID-19

BACKGROUND

Aspartate aminotransferase (AST) is often increased in COVID-19 and, in some studies, AST abnormalities were associated with mortality risk.

METHODS

2054 hospitalized COVID-19 patients were studied. To identify sources of AST release, correlations between AST peak values and other biomarkers of tissue damage, i.e., alanine aminotransferase (ALT) for hepatocellular damage, creatine kinase (CK) for muscle damage, lactate dehydrogenase for multiorgan involvement, alkaline phosphatase and γ-glutamyltransferase for cholestatic injury, and C-reactive protein (CRP) for systemic inflammation, were performed and coefficients of determination estimated. The role of AST to predict death and intensive care unit admission during hospitalization was also evaluated. All measurements were performed using standardized assays.

RESULTS

AST was increased in 69% of patients. Increases could be fully explained by summing the effects of hepatocellular injury [AST dependence from ALT, 66.8% [95% confidence interval (CI): 64.5-69.1)] and muscle damage [AST dependence from CK, 42.6% (CI: 39.3-45.8)]. We were unable to demonstrate an independent association of AST increases with worse outcomes.

CONCLUSION

The mechanisms for abnormal AST in COVID-19 are likely multifactorial and a status related to tissue suffering could play a significant role. The clinical significance of AST elevations remains unclear.

The European Biological Variation Study (EuBIVAS): a summary report

Biological variation (BV) data have many important applications in laboratory medicine. Concerns about quality of published BV data led the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) 1st Strategic Conference to indicate need for new studies to generate BV estimates of required quality. In response, the EFLM Working Group on BV delivered the multicenter European Biological Variation Study (EuBIVAS). This review summarises the EuBIVAS and its outcomes. Serum/plasma samples were taken from 91 ostensibly healthy individuals for 10 consecutive weeks at 6 European centres. Analysis was performed by Siemens ADVIA 2400 (clinical chemistry), Cobas Roche 8000, c702 and e801 (proteins and tumor markers/hormones respectively), ACL Top 750 (coagulation parameters), and IDS iSYS or DiaSorin Liaison (bone biomarkers). A strict preanalytical and analytical protocol was applied. To determine BV estimates with 95% CI, CV-ANOVA after analysis of outliers, homogeneity and trend analysis or a Bayesian model was applied. EuBIVAS has so far delivered BV estimates for 80 different measurands. Estimates for 10 measurands (Non-HDL Cholesterol, S100-β protein, neuron-specific enolase, soluble transferrin receptor, intact fibroblast growth-factor-23, uncarboxylated-unphosphorylated matrix-Gla protein, human epididymis protein-4, free, conjugated and %free prostate-specific antigen), prior to EuBIVAS, have not been available. BV data for creatinine and troponin I were obtained using two analytical methods in each case. The EuBIVAS has delivered high-quality BV data for a wide range of measurands. The BV estimates are for many measurands lower than those previously reported, having an impact on the derived analytical performance specifications and reference change values.

Performance specifications for measurement uncertainty of common biochemical measurands according to Milan models

OBJECTIVES

Definition and fullfillment of analytical performance specifications (APS) for measurement uncertainty (MU) allow to make laboratory determinations clinically usable. The 2014 Milan Strategic Conference have proposed models to objectively derive APS based on: (a) the effect of analytical performance on clinical outcome; (b) biological variation components; and (3) the state of the art of the measurement, defined as the highest level of analytical performance technically achievable. Using these models appropriately, we present here a proposal for defining APS for standard MU for some common biochemical measurands.

METHODS

We allocated a group of 13 measurands selected among the most commonly laboratory requested tests to each of the three Milan models on the basis of their biological and clinical characteristics. Both minimum and desirable levels of quality of APS for standard MU of clinical samples were defined by using information obtained from available studies.

RESULTS

Blood total hemoglobin, plasma glucose, blood glycated hemoglobin, and serum 25-hydroxyvitamin D3 were allocated to the model 1 and the corresponding desirable APS were 2.80, 2.00, 3.00, and 10.0%, respectively. Plasma potassium, sodium, chloride, total calcium, alanine aminotransferase, creatinine, urea, and total bilirubin were allocated to the model 2 and the corresponding desirable APS were 1.96, 0.27, 0.49, 0.91, 4.65, 2.20, 7.05, and 10.5%, respectively. For C-reactive protein, allocated to the model 3, a desirable MU of 3.76% was defined.

CONCLUSIONS

APS for MU of clinical samples derived in this study are essential to objectively evaluate the reliability of results provided by medical laboratories.

A Comprehensive Appraisal of Laboratory Biochemistry Tests as Major Predictors of COVID-19 Severity

CONTEXT.—

A relevant portion of coronavirus disease 2019 (COVID-19) patients develop severe disease with negative outcomes. Several biomarkers have been proposed to predict COVID-19 severity, but no definite interpretative criteria have been established to date for stratifying risk.

OBJECTIVE.—

To evaluate 6 serum biomarkers (C-reactive protein, lactate dehydrogenase, D-dimer, albumin, ferritin, and cardiac troponin T) for predicting COVID-19 severity and to define related cutoffs able to aid clinicians in risk stratification of hospitalized patients.

DESIGN.—

A retrospective study of 427 COVID-19 patients was performed. Patients were divided into groups based on their clinical outcome: nonsurvivors versus survivors and patients admitted to an intensive care unit versus others. Receiver operating characteristic curves and likelihood ratios were employed to define predictive cutoffs for evaluated markers.

RESULTS.—

Marker concentrations at peak were significantly different between groups for both selected outcomes. At univariate logistic regression analysis, all parameters were significantly associated with higher odds of death and intensive care. At the multivariate analysis, high concentrations of lactate dehydrogenase and low concentrations of albumin in serum remained significantly associated with higher odds of death, whereas only low lactate dehydrogenase activities remained associated with lower odds of intensive care admission. The best cutoffs for death prediction were greater than 731 U/L for lactate dehydrogenase and 18 g/L or lower for albumin, whereas a lactate dehydrogenase activity lower than 425 U/L was associated with a negative likelihood ratio of 0.10 for intensive treatment.

CONCLUSIONS.—

Our study identifies which biochemistry tests represent major predictors of COVID-19 severity and defines the best cutoffs for their use.

Searching for a role of procalcitonin determination in COVID-19: a study on a selected cohort of hospitalized patients

Objectives

Procalcitonin (PCT) has been proposed for differentiating viral vs. bacterial infections. In COVID-19, some preliminary results have shown that PCT testing could act as a predictor of bacterial co-infection and be a useful marker for assessment of disease severity.

Methods

We studied 83 COVID-19 hospitalized patients in whom PCT was specifically ordered by attending physicians. PCT results were evaluated according to the ability to accurately predict bacterial co-infections and death in comparison with other known biomarkers of infection and with major laboratory predictors of COVID-19 severity.

Results

Thirty-three (39.8%) patients suffered an in-hospital bacterial co-infection and 44 (53.0%) patients died. In predicting bacterial co-infection, PCT showed a relatively low accuracy (area under receiver-operating characteristic [ROC] curve [AUC]: 0.757; 95% confidence interval [CI]: 0.651-0.845), with a strength for detecting the outcome not significantly different from that of white blood cell count and C-reactive protein (CRP). In predicting patient death, PCT showed an AUC of 0.815 (CI: 0.714-0.892), not better than those of other more common laboratory tests, such as blood lymphocyte percentage (AUC: 0.874, p=0.19), serum lactate dehydrogenase (AUC: 0.860, p=0.47), blood neutrophil count (AUC: 0.845, p=0.59), and serum albumin (AUC: 0.839, p=0.73).

Conclusions

Procalcitonin (PCT) testing, even when appropriately ordered, did not provide a significant added value in COVID-19 patients when compared with more consolidated biomarkers of infection and poor clinical outcome. The major application of PCT in COVID-19 is its ability, associated with a negative predictive value >90%, to exclude a bacterial co-infection when a rule-out cut-off (<0.25 μg/L) is applied.