Serological diagnostic for SARS-CoV-2: an experimental External Quality Assessment Scheme

External Quality Assessment (EQA) scheme for serological diagnostic test for SARS-CoV-2 detection in Sicily Region (Italy), in the period 2020-2022

OBJECTIVES

Since December 2019, worldwide public health has been exposed to a severe acute respiratory syndrome caused by Coronavirus-2. Serological testing is necessary for retrospective assessment of seroprevalence rates, and the determination of vaccine response and duration of immunity. For this reason, it was necessary to introduce a panel of tests able to identify and quantify Covid-19 antibodies.

METHODS

As a Regional Reference Centre, the CRQ Laboratory (Regional Laboratory for the Quality Control) developed and conducted an External Quality Assessment (EQA) panel of assays, to evaluate the quality of various methods, that were used by 288 Sicilian laboratories, previously authorized on behalf of the Public Health Service.

RESULTS

The performance test was based on pooled samples with different levels of concentration of antibodies. 97 , 98, and 95 % of the participating laboratories tested all samples correctly in 2020, 2021, and 2022 respectively. The best performance was observed in the test of total Ig. The general performance of laboratories improved over the years.

CONCLUSIONS

The incorrect diagnosis had and could still have important implications on vaccination cycles. Only through the effort of laboratory professionals, and the extension of the EQA scheme, a better harmonization of methods, protocols, and thus results, to guarantee a better healthcare system, will be possible.

Benchmarking medical laboratory performance on a global scale

Background and aims

Laboratory performance as a relative concept needs repetitive benchmarking for continuous improvement of laboratory procedures and medical processes. Benchmarking as such establishes reference levels as a basis for improvements efforts for healthcare institutions along the diagnosis cycle, with the patient at its center. But while this concept seems to be generally acknowledged in laboratory medicine, a lack of practical implementation hinders progress at a global level. The aim of this study was to examine the utility of a specific combination of indicators and survey-based data collection approach, and to establish a global benchmarking dataset of laboratory performance for decision makers in healthcare institutions.

Methods

The survey consisted of 44 items relating to laboratory operations in general and three subscales identified in previous studies. A global sample of laboratories was approached by trained professionals. Results were analyzed with standard descriptive statistics and exploratory factor analysis. Dimensional reduction of specific items was performed using confirmatory factor analysis, resulting in individual laboratory scores for the three subscales of "Operational performance," "Integrated clinical care performance," and "Financial sustainability" for the high-level concept of laboratory performance.

Results and conclusions

In total, 920 laboratories from 55 countries across the globe participated in the survey, of which 401 were government hospital laboratories, 296 private hospital laboratories, and 223 commercial laboratories. Relevant results include the need for digitalization and automation along the diagnosis cycle. Formal quality management systems (ISO 9001, ISO 15189 etc.) need to be adapted more broadly to increase patient safety. Monitoring of key performance indicators (KPIs) relating to healthcare performance was generally low (in the range of 10-30% of laboratories overall), and as a particularly salient result, only 19% of laboratories monitored KPIs relating to speeding up diagnosis and treatment. Altogether, this benchmark elucidates current practice and has the potential to guide improvement efforts and standardization in quality & safety for patients and employees alike as well as sustainability of healthcare systems around the globe.

Ensuring diagnostic testing accuracy for patient care and public health- COVID-19 testing scale-up from an EQA provider's perspective

In response to the coronavirus pandemic (COVID-19) and scale up of diagnostic testing, the Canadian Microbiology Proficiency Testing program created a new proficiency testing (PT) program for the molecular and antigen detection of SARS-CoV-2. The program was geared to point of care testing (POCT) sites located in each of the eight provincial Health Authorities across British Columbia, Canada, with the intention to monitor testing quality. The PT program consisted of 6 shipments in a year, each containing a set of 4 samples either positive for SARS-CoV-2 virus or negative. The program began with initial 23 sites enrolling in March 2021, expanding to >100 participants by December 2021. After the first two surveys, it was observed that testing performance (accuracy) was consistently acceptable for sites using nucleic acid technology (NAT), however performance by sites using rapid antigen detection (RAD) methods was poor, especially when testing the weakly positive samples. A root cause investigation of poor testing performance revealed gaps in the execution of testing methods and also in results interpretation. These quality issues were most commonly associated with new testers who lacked experience with diagnostic testing. Tester training and mentoring was reinforced as was retraining of personnel; sample processing instructions were modified, and a training video was also created for testing sites. As a result of these interventions, sites improved their testing accuracy and the performance of POCT sites using RAD methods came to more closely match the performance of sites utilizing NAT. Overall, the PT program was highly successfully and improved quality of testing in the province. This work demonstrates the critical value of an external quality assessment (EQA) partner towards improving patient and public health and safety, especially when testing is conducted outside of an accredited medical laboratory setting.

Three rounds of a national external quality assessment reveal a link between disharmonic anti-SARS-CoV-2 antibody quantifications and the infection stage

OBJECTIVES

The WHO's standardized measuring unit, "binding antibody units per milliliter (BAU/mL)," should allow the harmonization of quantitative results by different commercial Anti-SARS-CoV-2 immunoassays. However, multiple studies demonstrate inter-assay discrepancies. The antigenic changes of the Omicron variant affect the performance of Spike-specific immunoassays. This study evaluated the variation of quantitative Anti-SARS-CoV-2-Spike antibody measurements among 46, 50, and 44 laboratories in three rounds of a national external quality assessment (EQA) prior to and after the emergence of the Omicron variant in a diagnostic near-to-real-life setting.

METHODS

We analyzed results reported by the EQA participant laboratories from single and sequential samples from SARS-CoV-2 convalescent, acutely infected, and vaccinated individuals, including samples obtained after primary and breakthrough infections with the Omicron variant.

RESULTS

The three immunoassays most commonly used by the participants displayed a low intra-assay and inter-laboratory variation with excellent reproducibility using identical samples sent to the participants in duplicates. In contrast, the inter-assay variation was very high with all samples. Notably, the ratios of BAU/mL levels quantified by different immunoassays were not equal among all samples but differed between vaccination, past, and acute infection, including primary infection with the Omicron variant. The antibody kinetics measured in vaccinated individuals strongly depended on the applied immunoassay.

CONCLUSIONS

Measured BAU/mL levels are only inter-changeable among different laboratories when the same assay was used for their assessment. Highly variable ratios of BAU/mL quantifications among different immunoassays and infection stages argue against the usage of universal inter-assay conversion factors.

External quality assessment of SARS-CoV-2 serology in European expert laboratories, April 2021

Background

Countries worldwide are focusing to mitigate the ongoing SARS-CoV-2 pandemic by employing public health measures. Laboratories have a key role in the control of SARS-CoV-2 transmission. Serology for SARS-CoV-2 is of critical importance to support diagnosis, define the epidemiological framework and evaluate immune responses to natural infection and vaccine administration.

Aim

The aim of this study was the assessment of the actual capability among laboratories involved in sero-epidemiological studies on COVID-19 in EU/EEA and EU enlargement countries to detect SARS-CoV-2 antibodies through an external quality assessment (EQA) based on proficiency testing.

Methods

The EQA panels were composed of eight different, pooled human serum samples (all collected in 2020 before the vaccine roll-out), addressing sensitivity and specificity of detection. The panels and two EU human SARS-CoV-2 serological standards were sent to 56 laboratories in 30 countries.

Results

The overall performance of laboratories within this EQA indicated a robust ability to establish past SARS-CoV-2 infections via detection of anti-SARS-CoV-2 antibodies, with 53 of 55 laboratories using at least one test that characterised all EQA samples correctly. IgM-specific test methods provided most incorrect sample characterisations (24/208), while test methods detecting total immunoglobulin (0/119) and neutralising antibodies (2/230) performed the best. The semiquantitative assays used by the EQA participants also showed a robust performance in relation to the standards.

Conclusion

Our EQA showed a high capability across European reference laboratories for reliable diagnostics for SARS-CoV-2 antibody responses. Serological tests that provide robust and reliable detection of anti-SARS-CoV-2 antibodies are available.

Results of a European-Wide External Quality Assessment (EQA) Scheme for Serological Detection of Anti-SARS-CoV-2 (CoVimm)—Pitfalls of Routine Application

Background: During the last two years, a variety of assays for the serological detection of antibodies to the new SARS-CoV-2 virus have been launched and used as part of standard care in many laboratories. The pace with which these tests have been introduced into routine care emphasizes the importance of quality measures for analytical methods, particularly with regard to the implications of results for clinical and epidemiologic decisions. Accuracy, reliability and comparability of analytical test results are thus essential, and here external quality assessment (EQA) is the most important quality assurance tool. It allows us to achieve harmonization of test methods as a prerequisite for a high standard of performance for laboratory and analytical techniques and their interpretation. Methods: This EQA scheme consisted of pre-characterized clinical biospecimens dedicated to the analysis of anti-SARS-CoV-2 IgG total antibodies and differentiation into spike protein-specific IgG antibodies against SARS-CoV-2 (anti-S-SARS-CoV-2) and nucleocapsid-specific IgG antibodies against SARS-CoV-2 (anti-N-SARS-CoV-2). Results: A total of 239 laboratories across Europe participated in this scheme, called CoVimm. In detail, 536 results for anti-SARS-CoV-2 IgG, 431 results for anti-S-SARS-CoV-2 IgG, and 200 results for anti-N-SARS-CoV-2 IgG were reported. Based on the pre-defined thresholds, the success rates for the determination of anti-S-SARS-CoV-2 IgG and anti-N-SARS-CoV-2 IgG were 96% and 90%, respectively. Interestingly, only 64% of the participating laboratories successfully passed the EQA scheme for the determination of total anti-SARS-CoV-2 IgG. Conclusions: This EQA revealed serious concerns regarding the reliability and appropriate use of anti-SARS-CoV-2 antibody assays in routine care. In addition to the wide heterogeneity of different assays used by participating laboratories, a lack of standardization and harmonization is also evident. This is of particular importance for reliable and clinically meaningful interpretation of test results.

Benchmarking medical laboratory performance: survey validation and results for Europe, Middle East, and Africa

OBJECTIVES

Medical laboratory performance is a relative concept, as are quality and safety in medicine. Therefore, repetitive benchmarking appears to be essential for sustainable improvement in health care. The general idea in this approach is to establish a reference level, upon which improvement may be strived for and quantified. While the laboratory community traditionally is highly aware of the need for laboratory performance and public scrutiny is more intense than ever due to the SARS-CoV-2 pandemic, few initiatives span the globe. The aim of this study was to establish a good practice approach towards benchmarking on a high abstraction level for three key dimensions of medical laboratory performance, generate a tentative snapshot of the current state of the art in the region of Europe, Middle East, and Africa (EMEA), and thus set the stage for global follow-up studies.

METHODS

The questionnaire used and previously published in this initiative consisted of 50 items, roughly half relating to laboratory operations in general with the other half addressing more specific topics. An international sample of laboratories from EMEA was approached to elicit high fidelity responses with the help of trained professionals. Individual item results were analyzed using standard descriptive statistics. Dimensional reduction of specific items was performed using exploratory factor analysis and assessed with confirmatory factor analysis, resulting in individual laboratory scores for the three subscales of "Operational performance", "Integrated clinical care performance", and "Financial sustainability".

RESULTS

Altogether, 773 laboratories participated in the survey, of which 484 were government hospital laboratories, 129 private hospital laboratories, 146 commercial laboratories, and 14 were other types of laboratories (e.g. research laboratories). Respondents indicated the need for digitalization (e.g. use of IT for order management, auto-validation), automation (e.g. pre-analytics, automated sample transportation), and establishment of formal quality management systems (e.g. ISO 15189, ISO 9001) as well as sustainably embedding them in the fabric of laboratory operations. Considerable room for growth also exists for services provided to physicians, such as "Diagnostic pathways guidance", "Proactive consultation on complex cases", and "Real time decision support" which were provided by less than two thirds of laboratories. Concordantly, the most important kind of turn-around time (TAT) for clinicians, sample-to-result TAT, was monitored by only 40% of respondents.

CONCLUSIONS

Altogether, the need for stronger integration of laboratories into the clinical care process became apparent and should be a main trajectory of future laboratory management. Factor analysis confirmed the theoretical constructs of the questionnaire design phase, resulting in a reasonably valid tool for further benchmarking activities on the three aimed-for key dimensions.