Interpretation of EQA results and EQA-based trouble shooting

Assessment of accuracy of laboratory testing results, relative to peer group consensus values in external quality control, by bivariate z-score analysis: the example of D-Dimer

OBJECTIVES

The aim of this study is to develop a practical method for bivariate z-score analysis which can be applied to the survey of an external quality assessment programme.

METHODS

To develop the bivariate z-score analysis, the results of four surveys of the international D-Dimer external quality assessment programme of 2022 of the ECAT Foundation were used. The proposed methodology starts by identifying the bivariate outliers, using a Supervised Sequential Hotelling T2 control chart. The outlying data are removed, and all the remaining data are used to provide robust estimates of the parameters of the assumed underlying bivariate normal distribution. Based on these estimates two nested homocentric ellipses are drawn, corresponding to confidence levels of 95 and 99.7 %. The bivariate z-score plot described provides the laboratory with an indication of both systematic and random deviations from zero z-score values. The bivariate z-score analysis was examined within survey 2022-D4 across the three most frequently used methods.

RESULTS

The number of z-score pairs included varied between 830 and 857 and the number of bivariate outliers varied between 20 and 28. The correlation between the z-score pairs varied between 0.431 and 0.647. The correlation between the z-score pairs for the three most frequently used varied between 0.208 and 0.636.

CONCLUSIONS

The use of the bivariate z-score analysis is of major importance when multiple samples are distributed around in the same survey and dependency of the results is likely. Important lessons can be drawn from the shape of the ellipse with respect to random and systematic deviations, while individual laboratories have been informed about their position in the state-of-the-art distribution and whether they have to deal with systematic and/or random deviations.

Longitudinal evaluation of laboratory results and method precision in worldwide erythropoietin external quality assessments

Introduction: This study presents a longitudinal analysis of external quality assessment (EQA) results for erythropoietin (EPO) determinations conducted between 2017 and 2022 with a continuously increasing number of participating laboratories. The aim of this work was to evaluate participant performance and methodological aspects. Methods: In each of the eleven EQA surveys, a blinded sample set of lyophilized human serum containing one sample with lower EPO concentrations (L) and one with higher EPO concentrations (H) was sent to the participating laboratories. Results: A total of 1,256 measurements were included. The median (interquartile range) fraction of participants not meeting the criteria of acceptance set at 20% around the robust mean of the respective survey was 9.5% (6.1%-10.7%) (sample L) and 9.1% (5.8%-11.8%) (sample H) but lacked a clear trend in the observed period. Some surveys exhibited unusually high interlaboratory variation, suggesting interfering components in the EQA samples. Different immunological methods and reagent manufacturers also showed variability in measurement outcomes to some extent. Conclusion: These findings highlight the need for continuous quality assessment in EPO measurements to ensure patient safety and identify areas for further research and investigation.

Navigating Quality Assessment Hurdles in Clinical Laboratory Services: A Comprehensive Review in Resource-Limited Settings

External quality assessment is the process of evaluating the performance of a laboratory and the competence of professionals. Participation in EQA and standardizing the quality of laboratory services are a mandatory requirements for accreditation. This review is aimed at identifying and discussing challenges that hinder the effective implementation of an EQA program in countries with resource limited setting. To obtain abundant information, articles were identified by searching the literature publishedin English using the National Library of Medicine, PubMed, Science Direct and AMC digital library databases. The articles identified in the references were manually searched and included. After the article was identified, it was imported to Endnote version 8.1 and exported to Microsoft Word for citation. Based on this review, the major identified challenges that hinder the implementation of an EQA program include the high cost of control materials, malfunction and lack of maintenance for equipment failure and breakdown, a knowledge gap among laboratory professionals, noncommutability of control samples, and difficulty in assigning target values. In addition, failing to participate in EQA and failing to take corrective action are the major challenges identified. As a result, applying to an EQA program in resource-limited counties was highly challenging. To attain high performance in the laboratory and to provide quality laboratory service for patient care, the EQA supplier and the user laboratory must pay attention to these issues and take appropriate corrective actions for ongoing quality improvement and accreditation.

Establishment of a stable proficiency testing matrix in transfusion microbiology in South Africa

Background

All medical laboratories must participate in proficiency testing (PT) programmes to ensure high-quality results. Proficiency testing samples mimic clinical samples; however, PT programmes for detection of bacteria in blood products are not routinely performed due to unavailability of matrix-equivalent samples.

Objective

The aim of this study was to develop and test a matrix-equivalent PT programme using blood products as the basis matrix.

Methods

A prospective cross-sectional study was conducted from April 2021 until June 2021, using 52 blood products comprising 36 pooled platelet and 16 red blood cell products at the South African National Blood Service PT laboratory in Gauteng. Products were manipulated into matrix-equivalent PT samples by spiking 42 products with known bacterial strains at specific concentrations and treating the remaining 10 products with preserving fluid containing antibiotics. The level of agreement between the researcher results and participating laboratories' results was assessed.

Results

Of the prepared matrices, 568 out of 572 (99%) were stable for 30 days. Bacteria could correctly be identified in spiked samples for up to 23 days. Samples treated with preserving fluid remained negative until day 30. For spiked samples, an average of 98% agreement (153/156) was achieved between the three participating laboratories when compared with the researcher's results; 100% agreement was achieved for unspiked samples. The kappa scores obtained from all tested variables presented with scores between 0.856 and 1.000, and the p-value was < 0.001 throughout.

Conclusion

The developed PT matrix was therefore stable and suitable to be implemented in transfusion microbiology.

What this study adds

This study demonstrated that a stable microbiology PT programme using platelets and red blood cells can be developed for use on bacterial detection analysers and could help to close the gap presented by unavailability of a blood PT matrix for transfusion microbiology.

Impact of randomized blinded rechecking program on the performance of the AFB Microscopy Laboratory Network in Uganda: a decadal retrospective study

BACKGROUND

Smear microscopy has remained the initial diagnostic test for presumptive tuberculosis (TB) patients in health facilities without the World Health Organization (WHO) recommended rapid diagnostic tools. In the Uganda TB laboratory network, the technique remains the only tool to monitor response to treatment among drug susceptible TB patients, with the country currently having over 1,600 microscopy TB testing units. It has been evidenced that acid-fast bacilli (AFB) microscopy's yield highly depends on the staining technique and reading ability of the laboratory personnel. For the quality of TB testing in the country, the TB control program set up a Randomized Blinded Rechecking (RBRC) program in 2008 to monitor the testing performance of laboratories to continuously improve the reliability and efficiency of results. This is the first study to determine the effectiveness and impact of the RBRC program on the performance of the participating laboratories in Uganda.

METHODS

This was a retrospective cross-sectional study based on a record review of the RBRC's annual results compilations between January 2008 and December 2017.

RESULTS

Between January 2008 and December 2017, a total of 265,523 smears were re-checked during the RBRC program. The number of enrolled laboratories in the RBRC program rose from 660 to 2008 to 1,406 in 2017. The RBRC program resulted in a statistically significant reduction in microscopy errors, with false positives decreasing from 12.8% to 2008 to 7.6% in 2017, false positive errors decreasing from 10 to 6.3%, false negative errors decreasing from 2.9 to 0.7%, quantification errors decreasing from 6.0 to 1.8%, and the overall sensitivity of smear microscopy compared to the controllers increased with statistical significance from 93 to 97%.

CONCLUSION

The study reveals an overall significant error reduction and an improved sensitivity of smear microscopy upon continuous implementation of the RBRC program in an AFB microscopy TB laboratory network. Implementation of a RBRC program is crucial and essential to maintaining a reliable TB laboratory service that can facilitate accurate diagnosis and offset the disadvantages of using smear microscopy.

An external quality assessment feasibility study; cross laboratory comparison of haemagglutination inhibition assay and microneutralization assay performance for seasonal influenza serology testing: A FLUCOP study

Introduction

External Quality Assessment (EQA) schemes are designed to provide a snapshot of laboratory proficiency, identifying issues and providing feedback to improve laboratory performance and inter-laboratory agreement in testing. Currently there are no international EQA schemes for seasonal influenza serology testing. Here we present a feasibility study for conducting an EQA scheme for influenza serology methods.

Methods

We invited participant laboratories from industry, contract research organizations (CROs), academia and public health institutions who regularly conduct hemagglutination inhibition (HAI) and microneutralization (MN) assays and have an interest in serology standardization. In total 16 laboratories returned data including 19 data sets for HAI assays and 9 data sets for MN assays.

Results

Within run analysis demonstrated good laboratory performance for HAI, with intrinsically higher levels of intra-assay variation for MN assays. Between run analysis showed laboratory and strain specific issues, particularly with B strains for HAI, whilst MN testing was consistently good across labs and strains. Inter-laboratory variability was higher for MN assays than HAI, however both assays showed a significant reduction in inter-laboratory variation when a human sera pool is used as a standard for normalization.

Discussion

This study has received positive feedback from participants, highlighting the benefit such an EQA scheme would have on improving laboratory performance, reducing inter laboratory variation and raising awareness of both harmonized protocol use and the benefit of biological standards for seasonal influenza serology testing.

The Togo national proficiency test pilot programme for basic clinical chemistry tests

Background

A national proficiency test (PT) programme is not currently implemented in most low-income countries. However, participation in such PT programmes assists improves test performance and result accuracy.

Objective

This study assessed how well 11 government hospital laboratories performed 18 basic clinical chemistry tests and identified areas needing improvement.

Methods

A cross-sectional study was carried out by the Division of Laboratories of the Ministry of Health of Togo from 01 July 2016 to 31 December 2016. The test performance was evaluated using panels provided by One World Accuracy, Canada (Vancouver). The Clinical Laboratory Improvement Amendments criteria were used in evaluating the laboratories, and their success rates were compared with the World Health Organization Regional Office for Africa’s target of 80%.

Results

The overall rate of acceptable results at the laboratories was over 80% for glucose, alanine aminotransferase, aspartate aminotransferase, gamma-glutamyltransferase, alkaline phosphatase and triglycerides tests. The laboratories using fully automated spectrophotometers had an acceptable results rate of 89% (p = 0.001). The overall performance of the laboratories by cycles varied from 71% to 82%.

Conclusion

This national PT programme identified the tests, which laboratories must improve their performance (urea, creatinine, uric acid, bilirubin, cholesterol, total protein, calcium, magnesium, phosphorus). It demonstrated the need for the use of routine appropriate internal quality control in all laboratories. The proficiency test programme should be extended to all clinical laboratories and target all biology disciplines.

An approach for determining allowable between reagent lot variation

Clinicians trust medical laboratories to provide reliable results on which they rely for clinical decisions. Laboratories fulfil their responsibility for accurate and consistent results by utilizing an arsenal of approaches, ranging from validation and verification experiments to daily quality control procedures. All these procedures verify, on different moments, that the results of a certain examination procedure have analytical performance characteristics (APC) that meet analytical performance specifications (APS) set for a particular intended use. The APC can in part be determined by estimating the measurement uncertainty component under conditions of within-laboratory precision (uRw), which comprises all components influencing the measurement uncertainty of random sources. To maintain the adequacy of their measurement procedures, laboratories need to distinguish aspects that are manageable vs. those that are not. One of the aspects that may influence uRw is the momentary significant bias caused by shifts in reagent and/or calibrator lots, which, when accepted or unnoticed, become a factor of the APC. In this paper, we postulate a model for allocating a part of allowable uRw to between-reagent lot variation, based on the need for long-term consistency of the measurement variability for that specific measurand. The allocation manages the ratio between short-term and long-term variation and indicates laboratories when to reject or correct certain variations due to reagent lots.

Formation and Evaluation of Complete Blood Count Proficiency Testing Program

Introduction: The haematology external quality assessment (EQA) scheme is the most commonly used service of quality assurance. The provision of complete blood count (CBC) materials must meet the quality requirements at a reasonable cost. These requirements are the most significant challenges for EQA organisers in Vietnam. This study’s objective was to evaluate the homogeneity, long-term stability, and peer-group performance of 10-parameter stabilised CBC EQA samples. Methods: The CBC EQA material was prepared using the following steps, including (1) adjusting levels of stabilised erythrocyte, leukocyte, and platelet samples, (2) mixing those cells into batches at three levels, and (3) dispensing and storing them at 2–6 °C. A set of 10 and 30 specimens were randomly chosen from each batch to study the homogeneity and long-term stability following ISO 13528:2015. In total, 166 samples at two levels were randomly distributed to 40 participants, which reported 83 automatic cell counters among six automated analyser models in the CBC EQA program. Results: The 10-parameter stabilised CBC EQA materials at three levels became homogeneous and stable in 12 weeks when preserved at 2–6 °C. Meanwhile, for five parameters (RBC, Hb, MCH, MCV, and MPV), this process was prolonged for up to 16 weeks in stock condition. In terms of peer-group performance, the CV (%) values increased at the low concentration for almost all parameters, especially in platelet counts. Conclusions: The stabilised CBC EQA samples prepared using the partial fixation method with aldehyde and gutaraldehyde in this study meet the ISO 13528:2015 requirements of homogeneity and long-term stability for the CBC EQA scheme. Analytical performance evaluation should categorise participant methods into peer groups.

Integrating quality control and external quality assurance

Effective management of clinical laboratories relies upon an understanding of Quality Control and External Quality Assurance principles. These processes, when applied effectively, reduce patient risk and drive quality improvement. In this Review, we will describe the purpose of QC and EQA and their role in identifying analytical and process error. The two concepts are linked, and we will illustrate that linkage. Some EQA providers offer far more than analytical surveillance. They facilitate training and education and extend quality improvement and identify areas where there is potential for patient harm into the pre-and post-analytical phases of the total testing process.

Errors within the total laboratory testing process, from test selection to medical decision-making - A review of causes, consequences, surveillance and solutions

Laboratory analyses are crucial for diagnosis, follow-up and treatment decisions. Since mistakes in every step of the total testing process may potentially affect patient safety, a broad knowledge and systematic assessment of laboratory errors is essential for future improvement. In this review, we aim to discuss the types and frequencies of potential errors in the total testing process, quality management options, as well as tentative solutions for improvement. Unlike most currently available reviews on this topic, we also include errors in test-selection, reporting and interpretation/action of test results. We believe that laboratory specialists will need to refocus on many process steps belonging to the extra-analytical phases, intensifying collaborations with clinicians and supporting test selection and interpretation. This would hopefully lead to substantial improvements in these activities, but may also bring more value to the role of laboratory specialists within the health care setting.

Causes behind error rates for predictive biomarker testing: the utility of sending post-EQA surveys

External quality assessment (EQA) schemes assess the performance of predictive biomarker testing in lung and colorectal cancer and have previously demonstrated variable error rates. No information is currently available on the underlying causes of incorrect EQA results in the laboratories. Participants in EQA schemes by the European Society of Pathology between 2014 and 2018 for lung and colorectal cancer were contacted to complete a survey if they had at least one analysis error or test failure in the provided cases. Of the 791 surveys that were sent, 325 were completed including data from 185 unique laboratories on 514 incorrectly analyzed or failed cases. For the digital cases and immunohistochemistry, the majority of errors were interpretation-related. For fluorescence in situ hybridization, problems with the EQA materials were reported frequently. For variant analysis, the causes were mainly methodological for lung cancer but variable for colorectal cancer. Post-analytical (clerical and interpretation) errors were more likely detected after release of the EQA results compared to pre-analytical and analytical issues. Accredited laboratories encountered fewer reagent problems and more often responded to the survey. A recent change in test methodology resulted in method-related problems. Testing more samples annually introduced personnel errors and lead to a lower performance in future schemes. Participation to quality improvement projects is important to reduce deviating test results in laboratories, as the different error causes differently affect the test performance. EQA providers could benefit from requesting root cause analyses behind errors to offer even more tailored feedback, subschemes, and cases.

Managing Deviating EQA Results: A Survey to Assess the Corrective and Preventive Actions of Medical Laboratories Testing for Oncological Biomarkers

Laboratories testing predictive biomarkers in lung and colorectal cancer are advised to participate in external quality assessment (EQA) schemes. This study aimed to investigate which corrective actions were taken by laboratories if predetermined performance criteria were not met, to ultimately improve current test practices. EQA participants from the European Society of Pathology between 2014 and 2018 for lung and colorectal cancer were contacted, if they had at least one analysis error or test failure in the provided cases, to complete a survey. For 72.4% of 514 deviating EQA results, an appropriate action was performed, most often including staff training (15.2%) and protocol revisions (14.6%). Main assigned persons were the molecular biologist (40.0%) and pathologist (46.5%). A change in test method or the use of complex techniques, such as next-generation sequencing, required more training and the involvement of dedicated personnel to reduce future test failures. The majority of participants adhered to ISO 15189 and implemented suitable actions by designated staff, not limited to accredited laboratories. However, for 27.6% of cases (by 20 laboratories) no corrective action was taken, especially for pre-analytic problems and complex techniques. The surveys were feasible to request information on results follow-up and further recommendations were provided.

External quality assessment schemes for inorganic elements in the clinical laboratory: Lessons from the OELM scheme

Measurements of inorganic elements in clinical laboratories produce results used for the diagnosis, the treatment and the monitoring of deficiencies or overloads. The main objective of External Quality Assessment Schemes is to verify, on a regular frequency, that clinical laboratory results correspond to the quality requirement for patient care. Therefore, External Quality Assessment Schemes represent an essential component of a laboratory's quality management system. However, External Quality Assessment Schemes within the same analytical field remain heterogeneous for different reasons such as samples, determination of assigned value, acceptable limits, content of the reports. The aim of this review was to describe and illustrate some major critical aspects of External Quality Assessment Schemes based on Occupational and Environmental Laboratory Medicine external quality assessment scheme experience.

A Quarter Century of PCR-Applied Techniques and Their Still-Increasing Fields of Use

Quantitative polymerase chain reaction (PCR) is the basis of a variety of scientific applications and publications in a broad range of interests. It also plays a fundamental role in nucleic acid sequencing applications, including Next Generation Sequencing (NGS)-based ones. The potential of PCR diagnostics is enormous, particularly for the early diagnosis of life-threatening infections. Some other fields of applications that use PCR on a regular basis include oncology, genetics, microbiology, biochemistry, immunogenetics, NGS, ecology, comparative genome evolution, ancestry DNA, pharmacogenomics, personalized medicine, and even general medicine.

Methodological and statistical issues in developing an External Quality Assessment scheme in laboratory medicine: Focus on biomarker research

External Quality Assessment (EQA) schemes are well-established tools with which to evaluate, monitor and improve the output quality of clinical laboratories, recognising that high quality laboratory medicine is essential for patient care. EQA programs involve the testing of multiple laboratories and the statistical comparison of their results, according to a multistep workflow. New clinical laboratory activities, such as biomarker research, require new EQA schemes. Critical elements in designing EQA programs are choosing the statistical methods and defining reference values and control limits. This article summarizes the key features of an EQA scheme, including designing the study, identifying reference values and control limits for qualitative and quantitative data, and graphically reporting laboratory performance statistics. These steps are illustrated with examples taken from the authors' experience in national and international quality assessment schemes for biomarker research.

The importance of commutability in material used for quality control purposes

External Quality Assessment (EQA) is an important part of laboratory quality assurance. Spiking of normal plasma is sometimes employed to mimic clinical samples. It is important that spiked material gives similar results to clinical samples (ie, is commutable) to ensure appropriate conclusions can be drawn from EQA exercises. We describe here data from UK National External Quality Assessment Scheme for Blood Coagulation (NEQAS BC) exercises where spiked samples were tested alongside samples from patients to explore commutability of artificial material. Normal plasma was spiked with unfractionated heparin (UFH), low molecular weight heparin (LMWH), Dabigatran or Rivaroxaban. Factor VIII (FVIII) deficient plasma was spiked with FVIII concentrate. Spiked samples and ex vivo patient material were sent to laboratories for testing. For LMWH, good agreement was seen between results for samples from patient plasma and plasma spiked with heparin. For UFH, APTT ratios differed between spiked and patient samples for the same drug concentration, with no correlation in ranking of reagents. Precision for patient and spiked material for Rivaroxaban and Dabigatran assays was comparable. However, the pattern of results for some Dabigatran assay kits differed between spiked and patient samples. For FVIII assays, results obtained with spiked and postinfusion samples gave comparable results. Spiked material is suitable for EQA if commutability is demonstrated. Our data show commutability for plasma spiked with Rivaroxaban, LWMH and some FVIII concentrates. For some tests, notably APTT for UFH, marked differences between patient and spiked samples indicate not all tests can be evaluated using artificial samples.

Analytical quality goals-a review

Analytical quality goals indicate how laboratory tests must perform to be clinically useful for their intended purpose. These goals have historically focused on analytical error assessment for quantitative methods and vary with measurand concentration or activity, and species. Although formalized quality goal models have been developed in human medicine, quality goals in veterinary medicine, to date, have not been formalized; use of human regulatory-based goals, consensus-based goals, or biologic variation-based goals have been reported most often. This review provides an overview of how quality goals are derived, how these may be used, and highlights challenges. Pending formal recommendations, individual veterinary laboratories should select quality goals that make the most sense clinically, logistically, and financially based on their individual needs and the needs of the clients that they serve.

Measurement uncertainty of blood ethanol concentration in drink-driving cases in an emergency laboratory

Introduction

The quality of blood ethanol concentration (BEC) determination is important because of its legal ramifications. Measurement uncertainty provides quantitative information about the quality and reliability of test results. In this study, we aim to calculate the measurement uncertainty for the ethanol test in our laboratory measured with a Synchron Systems Ethanol assay kit by employing an enzymatic rate method on the Beckman-Coulter Olympus AU400 auto analyzer (Beckman Coulter Inc, Melville, USA).

Materials and methods

The measurement uncertainty values were calculated in accordance to the Nordtest guidelines. All vehicle drivers involved in a traffic accident were retrospectively inspected for the BEC test conducted during July to December 2016 in our emergency laboratory.

Results

A 1034 vehicle drivers had their BEC tested. The results for 181 drivers were > 0.50 g/L and reported as positive. The serum ethanol concentration in those showing a positive result was 2.04 ± 1.01 g/L, over four times the legal limit. The median BEC in those showing a negative result was 0.03 (IQR: 0.03) g/L. The expanded uncertainty obtained was 19.74%. When measurement uncertainty values were added to the results of the 1034 drivers who were retrospectively screened, eight vehicle drivers had results with 95% confidence intervals that exceeded the legal limit 0.50 g/L.

Conclusions

The BEC test results for vehicle drivers with values close to legal limits should be reported as the obtained ethanol concentration with corresponding measurement uncertainty.