Commutability of possible external quality assessment materials for cardiac troponin measurement

An Update to the Kaiser Permanente Inpatient Risk Adjustment Methodology Accurately Predicts In-Hospital Mortality: a Retrospective Cohort Study

BACKGROUND

Methods to accurately predict the risk of in-hospital mortality are important for applications including quality assessment of healthcare institutions and research.

OBJECTIVE

To update and validate the Kaiser Permanente inpatient risk adjustment methodology (KP method) to predict in-hospital mortality, using open-source tools to measure comorbidity and diagnosis groups, and removing troponin which is difficult to standardize across modern clinical assays.

DESIGN

Retrospective cohort study using electronic health record data from GEMINI. GEMINI is a research collaborative that collects administrative and clinical data from hospital information systems.

PARTICIPANTS

Adult general medicine inpatients at 28 hospitals in Ontario, Canada, between April 2010 and December 2022.

MAIN MEASURES

The outcome was in-hospital mortality, modeled by diagnosis group using 56 logistic regressions. We compared models with and without troponin as an input to the laboratory-based acute physiology score. We fit and validated the updated method using internal-external cross-validation at 28 hospitals from April 2015 to December 2022.

KEY RESULTS

In 938,103 hospitalizations with 7.2% in-hospital mortality, the updated KP method accurately predicted the risk of mortality. The c-statistic at the median hospital was 0.866 (see Fig. 3) (25th-75th 0.848-0.876, range 0.816-0.927) and calibration was strong for nearly all patients at all hospitals. The 95th percentile absolute difference between predicted and observed probabilities was 0.038 at the median hospital (25th-75th 0.024-0.057, range 0.006-0.118). Model performance was very similar with and without troponin in a subset of 7 hospitals, and performance was similar with and without troponin for patients hospitalized for heart failure and acute myocardial infarction.

CONCLUSIONS

An update to the KP method accurately predicted in-hospital mortality for general medicine inpatients in 28 hospitals in Ontario, Canada. This updated method can be implemented in a wider range of settings using common open-source tools.

Exploration of suitable external quality assessment materials for serum C-peptide measurement

OBJECTIVES

To find suitable external quality assessment (EQA) materials for serum C-peptide, we evaluated the commutability of five types of processed materials.

METHODS

Seventy-four individual serum samples and 12 processed samples including three EQA samples currently in use, frozen human serum pools (FHSP), and three other kinds of processed samples were prepared by dissolving WHO International Standard Reagent for C-peptide (WHO ISR 13/146) in three different matrixes: 0.05 % bovine serum albumin, fetal bovine serum and human serum pools. Samples were analyzed using the isotope dilution liquid chromatography-tandem mass spectrometry (ID-LC-MS/MS) method and six widely used immunoassays. The commutabilities of processed materials were assessed according to the difference in bias approach recommended by the IFCC. And the short- and long-term stability of FHSP samples at different temperatures were also evaluated.

RESULTS

Out of the five kinds of processed materials, FHSP samples were commutable on most assays. In contrast, the EQA materials currently in use were only commutable on a few immunoassays. Additionally, processed materials derived from WHO ISR 13/146 were found to be un-commutable on over half of immunoassays. The FHSP samples could be stably stored at 4 and -20 °C for at least 16 days, and at -80 °C for at least 1 year, but at room temperature only for 12 h.

CONCLUSIONS

With clarified commutability and stability information, the human serum pool samples along with the developed ID-LC-MS/MS method could be used in the EQA program to promote the comparability among laboratories for C-peptide measurement in China.

Commutability evaluation of candidate reference materials and ERM-DA470k/IFCC for immunoglobulin M using two international approaches

BACKGROUND

This study aimed to assess the commutability of frozen pooled human serum (PHS), high concentration of Immunoglobulin M (IgM) pure diluted materials (HPDM), commercialized pure materials (CPM), and dilutions of ERM-DA470k/IFCC in IgM detection using the CLSI and IFCC approaches, to support standardization or harmonization of IgM measurement.

METHODS

Twenty-four serum samples, relevant reference materials (PHS, HPDM, CPM), and different ERM-DA470k/IFCC dilutions were analyzed in triplicate using six routine methods. The commutability of the relevant reference materials was carried out following CLSI EP30-A and IFCC bias analysis.

RESULTS

According to the CLSI approach, low, medium, and high concentrations of PHS, HPDM, and CPM were commutable on 10, 13, 15, 13, and 8 of 15 assay combinations, respectively. Using the IFCC approach, low, medium, and high concentrations of PHS, HPDM, and CPM were commutable on 10, 11, 9, 15, and 10 of 15 assay combinations, respectively. The ERM-DA470k/IFCC dilutions with D-PBS and RPMI-1640 Medium were commutable on 13 of 15 assay combinations according to CLSI and were commutable on all 15 assay combinations using IFCC approach.

CONCLUSIONS

High concentration of PHS were commutable on all six detection systems using the CLSI approach. Low and medium concentration of PHS showed unsatisfied commutability. HPDM, not CPM have good commutability, has the potential to become reference materials. ERM-DA470k/IFCC diluted with different medium showed different commutability.

Commutability Assessment of Processed Human Plasma Samples for Normetanephrine and Metanephrine Measurements Based on the Candidate Reference Measurement Procedure

Background

To identify candidate external quality assessment (EQA) materials for normetanephrine and metanephrine measurements, we assessed the commutability of eight processed human plasma samples. The agreement between routine assays and the candidate reference measurement procedure (cRMP) was also evaluated.

Methods

Fifty-three clinical samples and eight processed plasma samples were prepared. The processed samples included pooled and individual plasma samples spiked with pure normetanephrine and metanephrine and non-spiked pooled and individual plasma samples. The clinical and processed samples were subjected to four routine isotope dilution tandem mass spectrometry assays and cRMP. Commutability was assessed based on two approaches recommended by the CLSI and International Federation of Clinical Chemistry (IFCC). Passing–Bablok regression and Bland–Altman analysis were used to evaluate the agreement between the routine assays and cRMP.

Results

The commutability results of the CLSI approach were better than those of the IFCC approach. For the CLSI approach, spiked individual plasma samples and spiked high-concentration pooled plasma samples were commutable for all routine assays for both analytes. The non-spiked pooled plasma sample was commutable for two out of four routine assays for metanephrine and three out of four routine assays for normetanephrine. The agreement between the routine assays and the cRMP was satisfactory, except for one routine assay showing significant bias.

Conclusions

High-concentration spiked pooled plasma samples and spiked individual plasma samples are candidate EQA materials for normetanephrine and metanephrine measurements.

Comparability of thyroid-stimulating hormone immunoassays using fresh frozen human sera and external quality assessment data

Background

This study aimed to assess the comparability among assays using freshly frozen human sera and external quality assessment (EQA) data in China.

Methods

Twenty-nine serum samples and two commercial EQA materials, obtained from the National Center for Clinical Laboratories (NCCL), were analyzed in triplicate using eight routine TSH assays. The commutability of commercial EQA materials (NCCL materials) was evaluated in accordance with the CLSI EP30-A and IFCC bias analysis. Median values obtained for the NCCL EQA materials were used to determine the systematic and commutability-related biases among immunoassays through back-calculation. The comparability of TSH measurements from a panel of clinical samples and NCCL EQA data was determined on the basis of Passing–Bablok regression. Furthermore, human serum pools were used to perform commutable EQA.

Results

NCCL EQA materials displayed commutability among three or five of seven assay combinations according CLSI or IFCC approach, respectively. The mean of systematic bias ranged from -13.78% to 9.85% for the eight routine TSH assays. After correcting for systematic bias, averaged commutability-related biases ranged between -42.26% and 12.19%. After correction for systematic and commutability -related biases, the slopes indicating interassay relatedness ranged from 0.801 to 1.299 using individual human sera, from 0.735 to 1.254 using NCCL EQA data, and from 0.729 to 1.115 using pooled human serum EQA(the commutable EQA).

Conclusions

The harmonization of TSH measurement is challenging; hence, systematic and commutability-related biases should be determined and corrected for accurate comparisons among assays when using human individual serum and the commercial EQA materials.

Commutability Assessment of Candidate External Quality Assessment Materials for Aminotransferase Activity Measurements Based on Different Approaches in China

Background

Using commutable external quality assessment (EQA) materials is important for monitoring successful harmonization efforts. We assessed the commutability of four human serum pool (HSP) preparations to identify candidate EQA materials for alanine aminotransferase (ALT) and aspartate aminotransferase (AST) activity measurement.

Methods

One set each of 85 clinical samples (CSs) was collected for ALT and AST activity measurement. The 15 candidate EQA materials included four types of HSP preparations (A to D): materials A, C, and D contained human original recombinant (HOR) aminotransferases; materials B was mixed leftover samples. The CSs and 15 candidate EQA materials were analyzed using seven routine assays, and the ln-transformed results were analyzed in 21 assay pairs. Commutability was assessed using Deming regression, with a 95% prediction interval (CLSI approach) and the difference in bias with an error component model (International Federation of Clinical Chemistry and Laboratory Medicine [IFCC] approach).

Results

For ALT, all materials were commutable for 14–21 assay pairs according to the CLSI and IFCC approaches. For AST, B01–03 showed commutability for 14–21 assay pairs, and C01–03 and D01–03 showed commutability for no less than 10 assay pairs according to the two approaches. A01–06 were commutable for 9–16 assay pairs according to the CLSI approach, but for 6–9 assay pairs according to the IFCC approach.

Conclusions

Mixed leftover samples showed desirable commutability characteristics as candidate EQA materials for routine aminotransferase activity measurements. Human serum bases supplemented with HOR were commutable for most routine ALT activity measurements.

Commutability of external quality assessment materials for point-of-care glucose testing using the Clinical and Laboratory Standards Institute and International Federation of Clinical Chemistry approaches

Objectives

The aim of this study was to assess the commutability of three external quality assessment (EQA) materials for point‐of‐care (POC) glucose testing using two approaches, to identify suitable EQA materials to evaluate and monitor the quality of POC testing.

Methods

Commercial control materials (CCMs), pooled human serum samples (PHSs), and homemade human whole‐blood samples (HWBs) were measured along with 33 individual clinical samples using five POC instruments and a Hitachi 7600 analyzer. Data were analyzed by Deming regression analysis with a 95% prediction interval as described in Clinical and Laboratory Standards Institute (CLSI) EP30‐A, and by difference in bias analysis as described by the International Federation of Clinical Chemistry (IFCC) Working Group on Commutability.

Results

Using the CLSI approach, HWBs, CCMs, and PHSs were commutable with five, one, and two instruments, respectively. With the IFCC approach, HWBs were commutable with two instruments, while CCMs and PHSs were largely inconclusive or non‐commutable on five instruments.

Conclusions

HWBs were commutable on all instruments by the CLSI approach and may be a suitable EQA material for POC testing. Although some results differed between the IFCC and CLSI approaches, both indicated that HWBs were far superior to CCMs and PHSs in commutability.

Commutability assessment of reference materials for the enumeration of lymphocyte subsets

Background Flow cytometric enumeration of lymphocyte subsets in peripheral blood can provide important information about immune status. Commutable reference materials (RM) are crucial for maintaining accurate and comparable measurement results over time and space. Commutability assessment of RMs for lymphocyte subsets enumeration has not been reported elsewhere. Methods Lymphocyte subsets were measured in triplicate on 56 patient samples and eight RMs using two measuring systems commonly used in laboratories (FACS Canto II and Cytomics FC500). The first step was to determine the suitability of RMs and comparability of different systems with patient samples. After the requirements of suitability and comparability were met, the second step was to assess commutability following regression approach and difference in bias approach. Results Two RMs were not measurable on FC500 system for CD3-CD16/56+ and CD3-CD19+ percentages. The results of comparability showed no significant difference in the two systems. Eight RMs for CD3+CD4+ cell count, six RMs for CD3+ and CD3+CD8+ percentages, five RMs for CD3-CD16/56+ percentage, and three RMs for CD3-CD19+ percentage were commutable using the two approaches. For CD3+, CD3+CD8+ and CD3-CD19+ percentages, the results of regression approach showed that one RM was non-commutable for each parameter, while the other approach showed that the RM was commutable. Conclusions The suitability of RM and comparability of different measuring systems are prerequisites for assessing commutability. This study indicated that different approaches led to different results. The difference in bias approach is recommended for criteria relating to medical requirements and performance characteristics of measuring systems in use.

Commutability of control materials for external quality assessment of serum apolipoprotein A-I measurement

BACKGROUND

The aim of the current study was to evaluate the commutability of commercial control materials and human serum pools and to investigate the suitability of the materials for the external quality assessment (EQA) of serum apolipoprotein A-I (apo A-I) measurement.

METHODS

The Clinical and Laboratory Standards Institute (CLSI) EP14-A3 protocol was used for the commutability study. Apo A-I concentrations in two levels of commercial control materials used in EQA program, two fresh-frozen human serum pools (FSPs) and two frozen human serum pools prepared from residual clinical specimens (RSPs) were measured along with 50 individual samples using nine commercial assays. Measurement results of the 50 individual samples obtained with different assays were pairwise analyzed by Deming regression, and 95% prediction intervals (PIs) were calculated. The commutability of the processed materials was evaluated by comparing the measurement results of the materials with the limits of the PIs.

RESULTS

The FSP-1 was commutable for all the 36 assay pairs, and FSP-2 was commutable for 30 pairs; RSP-1 and RSP-2 showed commutability for 27/36 and 22/36 assay pairs, respectively, whereas the two EQA materials were commutable only for 4/36 and 5/36 assay pairs, respectively.

CONCLUSIONS

Non-commutability of the tested EQA materials has been observed among current apo A-I assays. EQA programs need either to take into account the commutability-related biases in the interpretation of the EQA results or to use more commutable materials. Frozen human serum pools were commutable for most of the assays.

Inductively Coupled Plasma Mass Spectrometry as a Reference Method to Evaluate Serum Calcium Measurement Bias and the Commutability of Processed Materials during Routine Measurements

Background:

Measuring total serum calcium is important for the diagnosis of diseases. Currently, results from commercial kits for calcium measurement are variable. Generally, the performance of serum calcium measurements is monitored by external quality assessment (EQA) or proficiency testing schemes. However, the commutability of the EQA samples and calibrators is often unknown, which limits the effectiveness of EQA schemes. The aim of this study was to evaluate the bias of serum calcium measurements and the commutability of processed materials.

Methods:

Inductively coupled plasma mass spectrometry was applied as a comparative method, and 14 routine methods were chosen as test methods. Forty-eight serum samples from individual patients and 25 processed materials were quantified. A scatter plot was generated from patient samples, and 95% prediction intervals were calculated to evaluate the commutability of the processed materials and measurement bias at three concentration levels was used to determine the accuracy of routine assays.

Results:

All assays showed high precision (total coefficient of variation [CV] <2.26%) and correlation coefficients (r > 0.99). For all assays, the mean bias for the 48 patient samples ranged from −0.13 mmol/L to 0.00 mmol/L (−5.61–0.01%), and the ranges for the three concentrations were −0.10–0.04 mmol/L (−5.71–2.35%), −0.14–−0.01 mmol/L (−5.80–−0.30%), and −0.19–0.04 mmol/L (−6.24–1.22%). The EQA samples, calibrators, and animal sera exhibited matrix effects in some assays; human serum pools were commutable in all assays; certificate reference materials were commutable in most assays, and only GBW09152 exhibited a matrix effect in one assay; and aqueous reference materials exhibited matrix effects in most assays.

Conclusions:

Biases for most assays were within the acceptable range, although the accuracy of some assays needs improvement. Human serum pools prepared from patient samples were commutable, and the other tested materials exhibited a matrix effect.

Standardization of clinical enzyme analysis using frozen human serum pools with values assigned by the International Federation of Clinical Chemistry and Laboratory Medicine reference measurement procedures

Variation in clinical enzyme analysis, particularly across different measuring systems and laboratories, represents a critical but long-lasting problem in diagnosis. Calibrators with traceability and commutability are imminently needed to harmonize analysis in laboratory medicine. Fresh frozen human serum pools were assigned values for alanine aminotransferase (ALT), aspartate aminotransferase (AST), gamma-glutamyltransferase (GGT), creatine kinase (CK) and lactate dehydrogenase (LDH) by six laboratories with established International Federation of Clinical Chemistry and Laboratory Medicine reference measurement procedures. These serum pools were then used across 76 laboratories as a calibrator in the analysis of five enzymes. Bias and imprecision in the measurement of the five enzymes tested were significantly reduced by using the value-assigned serum in analytical systems with open and single-point calibration. The median (interquartile range) of the relative biases of ALT, AST, GGT, CK and LDH were 2.0% (0.6-3.4%), 0.8% (-0.8-2.3%), 1.0% (-0.5-2.0%), 0.2% (-0.3-1.0%) and 0.2% (-0.9-1.1%), respectively. Before calibration, the interlaboratory coefficients of variation (CVs) in the analysis of patient serum samples were 8.0-8.2%, 7.3-8.5%, 8.1-8.7%, 5.1-5.9% and 5.8-6.4% for ALT, AST, GGT, CK and LDH, respectively; after calibration, the CVs decreased to 2.7-3.3%, 3.0-3.6%, 1.6-2.1%, 1.8-1.9% and 3.3-3.5%, respectively. The results suggest that the use of fresh frozen serum pools significantly improved the comparability of test results in analytical systems with open and single-point calibration.

Commutability of Reference Materials for α-Fetoprotein in Human Serum

CONTEXT

- Reliable quantification of α-fetoprotein (AFP) is critical for clinical diagnosis. Accuracy in AFP analysis relies on traceability to reference materials with confirmed commutability.

OBJECTIVE

- To assess the commutability of the reference materials for AFP. We screened for appropriate reference materials for the calibration of clinical AFP analysis and for application in an external quality assessment scheme. The feasibility of using water to dilute a reference material from the World Health Organization was also evaluated.

DESIGN

- Patient serum samples with various levels of AFP were randomly interspersed among AFP reference materials from the World Health Organization, the Beijing Center for Clinical Laboratories, and Beijing Controls and Standards Biotechnology and quality controls from Bio-Rad. The samples were analyzed on 5 different platforms to assess the comparability of the results and commutability of the reference materials.

RESULTS

- Significant variations in AFP measurement were observed among the 5 instrument platforms. The Beijing Center for Clinical Laboratories and Beijing Controls and Standards Biotechnology reference materials were commutable across all the instrument platforms. The World Health Organization AFP 72/225 reference material diluted with distilled water was also commutable at high concentrations. The Bio-Rad quality control materials for AFP were commutable among 4 out of 5 instrument platforms.

CONCLUSIONS

- Our results suggested that the Beijing Center for Clinical Laboratories and Beijing Controls and Standards Biotechnology materials were commutable across all 5 instrument platforms, whereas the Bio-Rad quality controls were limited by the concentration of AFP and the instrument platforms used. Caution needs to be taken in using water to dilute the World Health Organization 72/225 reference material because its commutability is limited to high concentrations.

Analytical Bias Exceeding Desirable Quality Goal in 4 out of 5 Common Immunoassays: Results of a Native Single Serum Sample External Quality Assessment Program for Cobalamin, Folate, Ferritin, Thyroid-Stimulating Hormone, and Free T4 Analyses

BACKGROUND

We undertook this study to evaluate method differences for 5 components analyzed by immunoassays, to explore whether the use of method-dependent reference intervals may compensate for method differences, and to investigate commutability of external quality assessment (EQA) materials.

METHODS

Twenty fresh native single serum samples, a fresh native serum pool, Nordic Federation of Clinical Chemistry Reference Serum X (serum X) (serum pool), and 2 EQA materials were sent to 38 laboratories for measurement of cobalamin, folate, ferritin, free T4, and thyroid-stimulating hormone (TSH) by 5 different measurement procedures [Roche Cobas (n = 15), Roche Modular (n = 4), Abbott Architect (n = 8), Beckman Coulter Unicel (n = 2), and Siemens ADVIA Centaur (n = 9)]. The target value for each component was calculated based on the mean of method means or measured by a reference measurement procedure (free T4). Quality specifications were based on biological variation. Local reference intervals were reported from all laboratories.

RESULTS

Method differences that exceeded acceptable bias were found for all components except folate. Free T4 differences from the uncommonly used reference measurement procedure were large. Reference intervals differed between measurement procedures but also within 1 measurement procedure. The serum X material was commutable for all components and measurement procedures, whereas the EQA materials were noncommutable in 13 of 50 occasions (5 components, 5 methods, 2 EQA materials).

CONCLUSIONS

The bias between the measurement procedures was unacceptably large in 4/5 tested components. Traceability to reference materials as claimed by the manufacturers did not lead to acceptable harmonization. Adjustment of reference intervals in accordance with method differences and use of commutable EQA samples are not implemented commonly.