Harmonization of indirect reference intervals calculation by the Bhattacharya method

OBJECTIVES

The aim of this study was to harmonize the criteria for the Bhattacharya indirect method Microsoft Excel Spreadsheet for reference intervals calculation to reduce between-user variability and use these criteria to calculate and evaluate reference intervals for eight analytes in two different years.

METHODS

Anonymized laboratory test results from outpatients were extracted from January 1st 2018 to December 31st 2019. To assure data quality, we examined the monthly results from an external quality control program. Reference intervals were determined by the Bhattacharya method with the St Vincent's hospital Spreadsheet firstly using original criteria and then using additional harmonized criteria defined in this study. Consensus reference intervals using the additional harmonized criteria were calculated as the mean of four users' lower and upper reference interval results. To further test the operation criteria and robustness of the obtained reference intervals, an external user validated the Spreadsheet procedure.

RESULTS

The extracted test results for all selected laboratory tests fulfilled the quality criteria and were included in the present study. Differences between users in calculated reference intervals were frequent when using the Spreadsheet. Therefore, additional criteria for the Spreadsheet were proposed and applied by independent users, such as: to set central bin as the mean of all the data, bin size as small as possible, at least three consecutive bins and a high proportion of bins within the curve.

CONCLUSIONS

The proposed criteria contributed to the harmonization of reference interval calculation between users of the Bhattacharya indirect method Spreadsheet.

Redesigning the surveillance of in vitro diagnostic medical devices and of medical laboratory performance by quality control in the traceability era

IVD manufacturers have total responsibility in terms of the traceability of marketed in vitro diagnostic medical devices (IVD-MD). This includes the provision of a quality control (QC) material as a part of the measuring system, suitable for traceability verification and alignment surveillance by end-users in daily practice. This material [to be used for the internal QC (IQC) component I as described in this paper] should have unbiased target values and an acceptability range corresponding to analytical performance specifications (APS) for suitable (expanded) measurement uncertainty (MU) on clinical samples. On the other hand, medical laboratories (by the IQC component II as described in this paper) should improve the IQC process and its judging criteria to establish a direct link between their performance, estimated as MU of provided results, and APS defined according to recommended models to apply corrective actions if the performance is worsening with the risk to jeopardize the clinical validity of test results. The participation to external quality assessment (EQA) programs that meet specific metrological criteria is also central to the evaluation of performance of IVD-MDs and of medical laboratories in terms of harmonization and clinical suitability of their measurements. In addition to the use of commutable materials, in this type of EQA it is necessary to assign values to them with selected reference procedures and to define and apply maximum allowable APS to substantiate the suitability of laboratory measurements in the clinical setting.

Control externo de la calidad en medicina del laboratorio. Avances y futuro

Objectivos

Un programa de control externo distribuye las mismas muestras control entre varios laboratorios y evalúa los resultados obtenidos con un criterio común. El objetivo de este trabajo es resumir la evolución de los programas externos, poner de manifiesto los progresos conseguidos y deducir consecuencias prácticas para el laboratorio participante.

Métodos

El material es una breve revisión de los diferentes tipos de programas externos utilizados a lo largo de cuarenta años. El método es el análisis crítico de las ventajas e inconvenientes de cada modelo, a la luz de nuestra experiencia.

Resultados

A mitad del siglo XX se iniciaron los programas EQA, detectándose gran discrepancia entre resultados emitidos por distintos laboratorios. Se desarrollaron EQA en muchos países y se propusieron mecanismos para armonizarlos, como: establecer especificaciones derivadas de la variación biológica, promover el uso de métodos analíticos homogéneos, usar el EQA como herramienta educacional. A partir del 2000 se hacen importantes avances: asegurar el adecuado uso clínico de las pruebas del laboratorio, utilizar material control conmutable con el espécimen humano, armonizar los distintos modelos de EQA, promover una organización de cooperación entre organizadores de programas EQA.

Conclusiones

Participar en un EQA con controles conmutables y valores asignados por método de referencia certificado permite conocer la inexactitud real de los resultados y el impacto en las muestras de pacientes. Si se participa en programas con controles no conmutables solo se conoce si la prestación del laboratorio es similar a la de otros usuarios del mismo método analítico.

External quality control in laboratory medicine. Progresses and future

Objectives

An external quality control program distributes same control samples to various laboratories and evaluates results obtained with a common criterion. The aim of this work is to summarize the evolution of various types of external programs, to point out the progresses ant to preclude practical consequences of the participant laboratories.

Content

The material consists on a brief revision of the different types of external programs that have been used for the last forty years. The method is the critical analysis of the strong and weak points of each program model, from the light of our experience. External quality assessment (EQA) programs were initiated at half the XX century, evidencing big discrepancies among laboratory results. EQA were developed in various countries and some mechanisms to harmonize them were proposed: to establish common performance specifications derived from biological variation, to use EQS as educational tool. Since the 2000 important advances were seen: to focus EQA to assure the adequate clinical use of laboratory tests, to use commutable controls, to harmonize the different EQA models, to promote a forum for co-operation and exchange of knowledge on quality-related matters for EQA organizers.

Summary and Outlook

To participate in an EQA with commutable-reference method assigned values controls allows to know the real inaccuracy of results and their impact on patient' samples. To participate in a EQA with non commutable controls allows to know whether the individual laboratory performance agrees with that from other laboratories using same analytical method.

Indirect determination of biochemistry reference intervals using outpatient data

The aim of this study was to determine reference intervals in an outpatient population from Vall d'Hebron laboratory using an indirect approach previously described in a Dutch population (NUMBER project). We used anonymized test results from individuals visiting general practitioners and analysed during 2018. Analytical quality was assured by EQA performance, daily average monitoring and by assessing longitudinal accuracy between 2018 and 2020 (using trueness verifiers from Dutch EQA). Per test, outliers by biochemically related tests were excluded, data were transformed to a normal distribution (if necessary) and means and standard deviations were calculated, stratified by age and sex. In addition, the reference limit estimator method was also used to calculate reference intervals using the same dataset. Finally, for standardized tests reference intervals obtained were compared with the published NUMBER results. Reference intervals were calculated using data from 509,408 clinical requests. For biochemical tests following a normal distribution, similar reference intervals were found between Vall d'Hebron and the Dutch study. For creatinine and urea, reference intervals increased with age in both populations. The upper limits of Gamma-glutamyl transferase were markedly higher in the Dutch study compared to Vall d'Hebron results. Creatine kinase and uric acid reference intervals were higher in both populations compared to conventional reference intervals. Medical test results following a normal distribution showed comparable and consistent reference intervals between studies. Therefore a simple indirect method is a feasible and cost-efficient approach for calculating reference intervals. Yet, for generating standardized calculated reference intervals that are traceable to higher order materials and methods, efforts should also focus on test standardization and bias assessment using commutable trueness verifiers.

An approach for determining allowable between reagent lot variation

Clinicians trust medical laboratories to provide reliable results on which they rely for clinical decisions. Laboratories fulfil their responsibility for accurate and consistent results by utilizing an arsenal of approaches, ranging from validation and verification experiments to daily quality control procedures. All these procedures verify, on different moments, that the results of a certain examination procedure have analytical performance characteristics (APC) that meet analytical performance specifications (APS) set for a particular intended use. The APC can in part be determined by estimating the measurement uncertainty component under conditions of within-laboratory precision (uRw), which comprises all components influencing the measurement uncertainty of random sources. To maintain the adequacy of their measurement procedures, laboratories need to distinguish aspects that are manageable vs. those that are not. One of the aspects that may influence uRw is the momentary significant bias caused by shifts in reagent and/or calibrator lots, which, when accepted or unnoticed, become a factor of the APC. In this paper, we postulate a model for allocating a part of allowable uRw to between-reagent lot variation, based on the need for long-term consistency of the measurement variability for that specific measurand. The allocation manages the ratio between short-term and long-term variation and indicates laboratories when to reject or correct certain variations due to reagent lots.

An updated protocol based on CLSI document C37 for preparation of off-the-clot serum from individual units for use alone or to prepare commutable pooled serum reference materials

Manufacturers of in vitro diagnostic medical devices, clinical laboratories, research laboratories and calibration laboratories require commutable reference materials that can be used in the calibration hierarchies of medical laboratory measurement procedures used for human specimens to establish metrological traceability to higher order reference systems. Commutable materials are also useful in external quality assessment surveys. In order to achieve these goals, matrix-based reference materials with long-term stability, appropriate measurand concentrations and commutability with individual human specimens are required. The Clinical and Laboratory Standards Institute (CLSI) guideline C37-A (now archived) provided guidance to prepare commutable pooled serum reference materials for use in the calibration hierarchies of cholesterol measurement procedures. Experience using the C37-A guideline has identified a number of technical enhancements as well as applications to measurands other than cholesterol. This experience is incorporated into this updated protocol to ensure the procedure will continue to meet the needs of the medical laboratory. The updated protocol describes a procedure for preparing frozen human serum units or pools with minimal matrix alterations that are likely to be commutable with individual human serum samples. The protocol provides step-by-step guidance for the planning phase, collection of individual serum units, processing the units, qualifying the units for use in a pool and frozen storage of aliquots of pooled sera to manufacture frozen serum pools. Guidance on how to perform quality control of the final product and suggestions on documentation are also provided.

Big data and reference intervals: rationale, current practices, harmonization and standardization prerequisites and future perspectives of indirect determination of reference intervals using routine data

Reference intervals are commonly used as a decision-making tool. In this review, we provide an overview on "big data" and reference intervals, describing the rationale, current practices including statistical methods, essential prerequisites concerning data quality, including harmonization and standardization, and future perspectives of the indirect determination of reference intervals using routine laboratory data.

Harmonization and Standardization: Where Are We Now?

BACKGROUND

The purpose of a medical laboratory test is to provide information on the pathophysiologic condition of an individual patient as an aid in diagnosis, therapy, or assessment of risk for a disease. For optimal laboratory service, results from different measurement procedures (MPs) for the same measurand should be equivalent (harmonized) within stated specifications, enabling the results to be used reliably for medical decisions. The term "harmonization" refers to any process that enables establishing equivalence of reported values among different end-user MPs. The term "standardization" refers to achieving harmonization by metrological traceability of patients' results to higher order reference materials and/or reference measurement procedures.

CONTENT

New procedures for harmonization and standardization were published in 2020 by the International Organization for Standardization (ISO) and by the IFCC. ISO 17511:2020 provides revised requirements for establishing metrologically traceable calibration hierarchies for end-user MPs used in clinical laboratories. ISO 21151:2020 provides new requirements to implement a harmonization protocol to address the situation when there are no fit-for-purpose certified reference materials or reference MPs available for a measurand. The IFCC Working Group on Commutability published recommendations for applying a correction for noncommutability of a certified reference material to enable using that material in a metrologically traceable calibration hierarchy for an end-user MP.

SUMMARY

We review metrological traceability and how these new approaches will improve the capability to achieve harmonized results for clinical samples.

Programas de garantía externa de la calidad SEQCML. Evolución de las prestaciones analíticas de los laboratorios clínicos a lo largo de 30 años y comparación con otros programas

Objetivos

El objetivo de este estudio es conocer la evolución de la prestación analítica de los laboratorios participantes en los programas EQA de la SEQCML durante los 30 años de funcionamiento y compararla con la prestación obtenida en otros programas EQA para saber si los resultados son similares.

Métodos

Se evalúan los resultados obtenidos durante este periodo, aplicando las especificaciones de la calidad derivadas de la VB y del estado del arte. Además, se realiza una comparación con los resultados obtenidos por otras organizaciones de programas EQA.

Resultados

Se observa que los laboratorios participantes en los programas EQA-SEQCML han mejorado su prestación durante los 30 años de experiencia y que las especificaciones derivadas de la variación biológica son alcanzables. La comparación entre programas EQA es difícil, debido a: la falta de accesibilidad y a las diferencias en el diseño de estos programas (materiales control, cálculos empleados y especificaciones analíticas establecidas).

Conclusiones

Los datos de este estudio ponen de manifiesto que para algunas magnitudes biológicas los resultados obtenidos en los programas todavía no están armonizados, aunque se estan realizando esfuerzos para alcanzar la armonización. Los organizadores de programas EQA deberían sumarse al esfuerzo de armonización, facilitando la información sobre sus resultados para permitir su comparación.

Spanish society of laboratory medicine external quality assurance programmes: evolution of the analytical performance of clinical laboratories over 30 years and comparison with other programmes

The purpose of this study is to understand the evolution of the analytical performance of the laboratories participating in the Spanish society of laboratory medicine (SEQC^ML) external quality assurance (EQA) programmes during its 30 years of operation and to compare it with the performance of other EQA programmes to establish whether the results are similar. The results obtained during this period are evaluated by applying the biological variability (BV) and state of the art-derived quality specifications. In addition, the results are compared with those obtained by other EQA programme organisations. It is noted that the laboratories participating in the EQA-SEQC^ML programmes have improved their performance over 30 years of experience and that the specifications derived from biological variation are achievable. It is difficult to compare EQA programmes, due to lack of accessibility and the differences in the design of these programmes (control materials, calculations used and analytical specifications established). The data from this study show that for some biological magnitudes the results obtained by the programmes are not yet harmonised, although efforts are being made to achieve this. Organisers of EQA programmes should also join the harmonisation effort by providing information on their results to enable comparison.

Alanine Aminotransferase Results Differ by Analyzer Manufacturer in a National Integrated Health Setting, 2012-2017

CONTEXT.—

Disease guidelines specify universal alanine aminotransferase (ALT) thresholds for clinical decision-making, yet the effect of variability among ALT analyzers remains unclear.

OBJECTIVE.—

To compare ALT results from different analyzers from 2012-2017.

DESIGN.—

Veterans Health Administration (VHA) laboratories perform external ALT proficiency testing using standardized College of American Pathologists (CAP) samples in analyzers by 5 manufacturers. In this operational analysis, we evaluated 22 950 ALT values from 80 independent CAP samples tested at 223 laboratories. Using mixed effects modeling, we estimated the association between analyzer manufacturer and CAP outcome, adjusting for manufacturer, facility, and calendar year. We performed subgroup analyses on CAP samples with overall means near clinical guideline-specified thresholds, including less than 50 U/L (n = 10) and less than 35 U/L (n = 5).

RESULTS.—

The VHA used Abbott Laboratories (n = 3175; 14%), Beckman Coulter Diagnostics (n = 8723; 38%), Roche Diagnostics (n = 2595; 11%), Siemens Healthineers USA (n = 5713; 25%), and Vitros/Ortho Clinical Diagnostics (n = 2744; 12%) analyzers. The CAP samples (n = 80 samples, n = 22 950 tests) covered a wide range of mean ALT values (21-268 U/L). The average difference in mean ALT value per sample between the highest-reading and lowest-reading manufacturers was 15.4 U/L (SD = 1.8) for the 10 samples with mean ALT less than 50 U/L, and it was 10.4 U/L (SD = 3.6) overall (n = 80). In linear mixed effects modeling, we found statistically significant differences in ALT values between the different manufacturers in each year.

CONCLUSIONS.—

We found statistically and clinically meaningful differences between analyzers across the ALT spectrum in each year, including at ALT levels lower than 50 U/L and lower than 35 U/L. Universal ALT thresholds should be avoided as a trigger for clinical action until differences between analyzers can be resolved.

Commutability of proficiency testing material containing amitriptyline and nortriptyline: A study within the framework of the Dutch Calibration 2.000 project

BACKGROUND

External quality assessment schemes (EQAS) can provide important information regarding accuracy and comparability of different measurement methods if the sample matrices are composed of commutable material. The aim of this study was to assess the commutability of different matrices for the material used in an EQAS for amitriptyline and nortriptyline.

METHODS

Proficiency testing material (PTM) and patient samples containing amitriptyline and nortriptyline were prepared, collected, pooled, and distributed to participating laboratories for analysis. Low, medium and high concentrations of both drugs in liquid pooled human, lyophilized human and lyophilized bovine serum were tested in this study. The measurement deviation of the PTM results to the patient serum regression line were normalized by dividing trough the average within-laboratory SD (SD_wl) derived from the results reported in the official EQAS, resulting in a relative residual. The commutability decision limit was set at 3 SD_wl.

RESULTS

With 10 laboratories participating in this study, 45 laboratory couples were formed. All matrix types delivered several relative residuals outside the commutability decision limit. The number and the magnitude of relative residuals for both drugs were lower for liquid human sera as compared to lyophilized human and bovine sera.

CONCLUSIONS

The PTM used for amitriptyline and nortriptyline is preferably prepared with human serum, although not all relative residuals are within the commutability decision limit.

Metrological traceability and harmonization of medical tests: a quantum leap forward is needed to keep pace with globalization and stringent IVD-regulations in the 21st century!

In our efforts to advance the profession and practice of clinical laboratory medicine, strong coordination and collaboration are needed more than ever before. At the dawn of the 21st century, medical laboratories are facing many unmet clinical needs, a technological revolution promising a plethora of better biomarkers, financial constraints, a growing scarcity of well-trained laboratory technicians and a sharply increasing number of International Organization for Standardization guidelines and new regulations to which medical laboratories should comply in order to guarantee safety and effectiveness of medical test results. Although this is a global trend, medical laboratories across continents and countries are in distinct phases and experience various situations. A universal underlying requirement for safe and global use of medical test results is the standardization and harmonization of test results. Since two decades and after a number of endeavors on standardization/harmonization of medical tests, it is time to reflect on the effectiveness of the approaches used. To keep laboratory medicine sustainable, viable and affordable, clarification of the promises of metrological traceability of test results for improving sick and health care, realization of formal commitment among all stakeholders of the metrological traceability chain and preparation of a joint and global plan for action are essential prerequisites. Policy makers and regulators should not only overwhelm the diagnostic sector with oversight and regulations but should also create the conditions by establishing a global professional forum for anchoring the metrological traceability concept in the medical test domain. Even so, professional societies should have a strong voice in their (inter-) national governments to negotiate long-lasting public policy commitment and funds for global standardization of medical tests.

Harmonization of External Quality Assessment Schemes and their role - clinical chemistry and beyond

The article tries to reply to the following three questions: Are External Quality Assessment Schemes (EQAS) really fit for purpose? Are all schemes equivalent and sufficiently harmonized? Is the role of EQAS similar and necessary in all branches of laboratory medicine? Although the reply to the first two questions is, unfortunately, negative for several reasons (lack of commutable material with reference method values, EQAS with different scopes, etc.), the reply to the third one is positive: EQAS are a necessary source of information on trueness and accuracy and must be fully developed for all the branches of the clinical laboratory.

NUMBER: standardized reference intervals in the Netherlands using a ‘big data’ approach

Background

External quality assessment (EQA) programs for general chemistry tests have evolved from between laboratory comparison programs to trueness verification surveys. In the Netherlands, the implementation of such programs has reduced inter-laboratory variation for electrolytes, substrates and enzymes. This allows for national and metrological traceable reference intervals, but these are still lacking. We have initiated a national endeavor named NUMBER (Nederlandse UniforMe Beslisgrenzen En Referentie-intervallen) to set up a sustainable system for the determination of standardized reference intervals in the Netherlands.

Methods

We used an evidence-based ‘big-data’ approach to deduce reference intervals using millions of test results from patients visiting general practitioners from clinical laboratory databases. We selected 21 medical tests which are either traceable to SI or have Joint Committee for Traceability in Laboratory Medicine (JCTLM)-listed reference materials and/or reference methods. Per laboratory, per test, outliers were excluded, data were transformed to a normal distribution (if necessary), and means and standard deviations (SDs) were calculated. Then, average means and SDs per test were calculated to generate pooled (mean±2 SD) reference intervals. Results were discussed in expert meetings.

Results

Sixteen carefully selected clinical laboratories across the country provided anonymous test results (n=7,574,327). During three expert meetings, participants found consensus about calculated reference intervals for 18 tests and necessary partitioning in subcategories, based on sex, age, matrix and/or method. For two tests further evaluation of the reference interval and the study population were considered necessary. For glucose, the working group advised to adopt the clinical decision limit.

Conclusions

Using a ‘big-data’ approach we were able to determine traceable reference intervals for 18 general chemistry tests. Nationwide implementation of these established reference intervals has the potential to improve unequivocal interpretation of test results, thereby reducing patient harm.