The versatility of external quality assessment for the surveillance of laboratory and in vitro diagnostic performance: SARS-CoV-2 viral genome detection in Austria

Concordance between MITS and conventional autopsies for pathological and virological diagnoses

In pandemics or to further study highly contagious infectious diseases, new strategies are needed for the collection of post-mortem tissue samples to identify the pathogen as well as its morphological impact. In this study, an ultrasound-guided minimally invasive tissue sampling (MITS) protocol was developed and validated for post-mortem use. The histological and microbiological qualities of post-mortem specimens were evaluated and compared between MITS and conventional autopsy (CA) in a series of COVID-19 deaths. Thirty-six ultrasound-guided MITS were performed. In five cases more, specimens for histological and virological examination were also obtained and compared during the subsequently performed CA. Summary statistics and qualitative interpretations (positive, negative) were calculated for each organ tissue sample from MITS and CA, and target genes were determined for both human cell count (beta-globin) and virus (SARS-CoV-2 specific E gene). There are no significant differences between MITS and CA with respect to the detectability of viral load in individual organs, which is why MITS can be of utmost importance and an useful alternative, especially during outbreaks of infectious diseases.

High quality of SARS-CoV-2 molecular diagnostics in a diverse laboratory landscape through supported benchmark testing and External Quality Assessment

A two-step strategy combining assisted benchmark testing (entry controls) and External Quality Assessments (EQAs) with blinded simulated clinical specimens to enhance and maintain the quality of nucleic acid amplification testing was developed. This strategy was successfully applied to 71 diagnostic laboratories in The Netherlands when upscaling the national diagnostic capacity during the SARS-CoV-2 pandemic. The availability of benchmark testing in combination with advice for improvement substantially enhanced the quality of the laboratory testing procedures for SARS-CoV-2 detection. The three subsequent EQA rounds demonstrated high quality testing with regard to specificity (99.6% correctly identified) and sensitivity (93.3% correctly identified). Even with the implementation of novel assays, changing workflows using diverse equipment and a high degree of assay heterogeneity, the overall high quality was maintained using this two-step strategy. We show that in contrast to the limited value of Cq value for absolute proxies of viral load, these Cq values can, in combination with metadata on strategies and techniques, provide valuable information for laboratories to improve their procedures. In conclusion, our two-step strategy (preparation phase followed by a series of EQAs) is a rapid and flexible system capable of scaling, improving, and maintaining high quality diagnostics even in a rapidly evolving (e.g. pandemic) situation.

Performance of SARS-CoV-2 nucleic acid amplification testing in Austria as measured by external quality assessment schemes during 3 years of the COVID-19 pandemic: an observational retrospective study

BACKGROUND

The aim of external quality assessment (EQA) schemes is to evaluate the analytical performance of laboratories and test systems in a near-to-real-life setting. This monitoring service provides feedback to participant laboratories and serves as a control measure for the epidemiological assessment of the regional incidence of a pathogen, particularly during epidemics. Using data from EQA schemes implemented as a result of the intensive effort to monitor SARS-CoV-2 infections in Austria, we aimed to identify factors that explained the variation in laboratory performance for SARS-CoV-2 detection over the course of the COVID-19 pandemic.

METHODS

For this observational study, we retrospectively analysed 6308 reverse transcriptase quantitative PCR (RT-qPCR) test results reported by 191 laboratories on 71 samples during 14 rounds of three SARS-CoV-2 pathogen detection EQA schemes in Austria between May 18, 2020, and Feb 20, 2023. We calculated the overall rates of false and true-negative, false and true-positive, and inconclusive results. We then assessed laboratory performance by estimating the sensitivity by testing whether significant variation in the odds of obtaining a true-positive result could be explained by virus concentration, laboratory type, or assay format. We also assessed whether laboratory performance changed over time.

FINDINGS

4371 (93·7%) of 4663 qPCR test results were true-positive, 241 (5·2%) were false-negative, and 51 (1·1%) were inconclusive. The mean per-sample sensitivity was 99·7% in samples with high virus concentrations (1383 [99·4%] true-positive, three [0·2%] false-negative, and five [0·4%] inconclusive results for 1391 tests in which the sample cycle threshold was ≤32), whereas detection rates were lower in samples with low virus concentrations (mean per-sample sensitivity 92·5%; 2988 [91·3%] true-positive, 238 [7·3%] false-negative, and 46 [1·4%] inconclusive results for 3272 tests in which the cycle threshold was >32). Of the 1645 results expected to be negative, 1561 (94·9%) were correctly reported as negative, 10 (0·6%) were incorrectly reported as positive, and 74 (4·5%) were reported as inconclusive. Notably, the overall performance of the tests did not change significantly over time. The odds of reporting a correct result were 2·94 (95% CI 1·75-4·96) times higher for a medical laboratory than for a non-medical laboratory, and 4·60 (2·91-7·41) times greater for automated test systems than for manual test systems. Automated test systems within medical laboratories had the highest sensitivity when compared with systems requiring manual intervention in both medical and non-medical laboratories.

INTERPRETATION

High rates of false-negativity in all PCR analyses evaluated in comprehensive, multiple, and repeated EQA schemes outline a clear path for improvement in the future. The performance of some laboratories (eg, non-medical laboratories or those using non-automated test systems) should receive additional scrutiny-for example, by requiring additional EQA schemes for certification or accreditation-if the aggregated data from EQA rounds suggest lower sensitivity than that recorded by others. This strategy will provide assurances that epidemiological data as a whole are reliable when testing on such a large scale. Although performance did not improve over time, we cannot exclude extenuating circumstances-such as shortages and weakened supply chains-that could have prevented laboratories from seeking alternative methods to improve performance.

FUNDING

None.

Clinical comparison of three SARS-CoV-2 nucleic acid amplification tests for routine diagnostic testing

Background

Cycle threshold (Ct) values from SARS-CoV-2 nucleic acid amplification tests have been used to estimate viral load for treatment decisions. Additionally, there is a need for high-throughput testing, consolidating a variety of assays on one random-access analyzer.

Objectives

In this study, the clinical performance of the Alinity m SARS-CoV-2, RealTime SARS-CoV-2, and GeneXpert Xpress SARS-CoV-2/Flu/RSV assays was assessed.

Methods

Alinity precision and detection rates were evaluated using a dilution series of the Alinity m SARS-CoV-2 positive control. In a retrospective study, 7 remnant external quality assessment (EQA) specimens and 200 remnant nasopharyngeal swab specimens (100 positive and 100 negative) were tested in the three assays.

Results

Alinity had 100 % detection rate at 50 copies/mL and high reproducibility (Ct value coefficient of variation ≤3.1 %). All three assays correctly detected positive and negative EQA samples with comparable Ct values (max difference 2.38) and high linearity. In patient samples, positive percent agreement was 95 % (95 % CI 89-98 %) and negative percent agreement was 100 % (95 % CI 96-100 %) for Alinity, compared to the other two assays. Four specimens detected on Alinity m but not RealTime or Xpert had Ct values above 40. Assay results were highly correlated (r ≥ 0.94). Ct values (after addition of 10 unread cycles to the reported Ct of RealTime) were comparable across the three assays.

Conclusions

Alinity m had high precision and accuracy and Ct values comparable to those of the RealTime and Xpert assays. The assays could be used interchangeably, with no need for adjustment of patient management decisions based on Ct values from each assay.

Design of external quality assessment schemes and definition of the roles of their providers in future epidemics

During an epidemic, individual test results form the basis of epidemiological indicators such as case numbers or incidence. Therefore, the accuracy of measures derived from these indicators depends on the reliability of individual results. In the COVID-19 pandemic, monitoring and evaluating the performance of the unprecedented number of testing facilities in operation, and novel testing systems in use, was urgently needed. External quality assessment (EQA) schemes are unique sources of data reporting on testing performance, and their providers are recognised contacts and support for test facilities (for technical-analytical topics) and health authorities (for planning the monitoring of infection diagnostics). To identify information provided by SARS-CoV-2 genome detection EQA schemes that is relevant for public health microbiology, we reviewed the current literature published in PubMed between January, 2020, and July, 2022. We derived recommendations for EQA providers and their schemes for best practices to monitor pathogen-detection performance in future epidemics. We also showed laboratories, test facilities, and health authorities the information and benefits they can derive from EQA data, and from the non-EQA services of their providers.

Converting to an international unit system improves harmonization of results for SARS-CoV-2 quantification: Results from multiple external quality assessments

BACKGROUND

The detection of SARS-CoV-2 vRNA in clinical samples has relied almost exclusively on RT-qPCR as the gold standard test. Published results from various external quality assessments ("ring trials") worldwide have shown that there is still a large variability in results reported for the same samples. As reference standards of SARS-CoV-2 RNA are available, we tested whether using standard curves to convert Ct values into copies/mL (cp/mL) improved harmonization.

METHODS

Nine laboratories using 23 test systems (15 of which were unique) prepared standard dilution curves to convert C_t values of 13 SARS-CoV-2 positive samples to cp/mL (hereafter IU/mL). The samples were provided in three rounds of a virus genome detection external quality assessment (EQA) scheme. We tested the precision and accuracy of results reported in IU/mL, and attempted to identify the sources of variability.

RESULTS

Reporting results as IU/mL improved the precision of the estimated concentrations of all samples compared to reporting Ct values, although some inaccuracy remained. Variance analysis showed that nearly all variability in data was explained by individual test systems within individual laboratories. When controlling for this effect, there was no significant difference between all other factors tested (test systems, EQA rounds, sample material).

CONCLUSIONS

Converting results to copies/mL improved precision across laboratory test systems. However, it seems the results are still very specific to test systems within laboratories. Further efforts could be made to improve accuracy and achieve full harmonization across diagnostic laboratories.

SARS-CoV-2 RNA Testing Using Different Assays—Impact on Testing Strategies in a Clinical Setting

In order to assess SARS-CoV-2 real time quantitative polymerase chain reaction (RT-qPCR) results in a real-life setting, three independent laboratories in Graz (Austria) set up a continuous cross comparison schedule. The following test systems were used: The QIAGEN NeuMoDx SARS-CoV-2 Assay, the Allplex™ 2019-nCoV Assay (Seegene) on a MicroLab Nimbus (Hamilton) platform combined with RealStar SARS-CoV-2 RT-PCR Assay (Altona Diagnostics GmbH), and the cobas SARS-CoV-2 test on a fully automated cobas 6800 system (Roche). A total of 200 samples were analysed, 184 (92%) were found to be concordant with all testing platforms, 14 (7%) discordant. Two (1%) samples tested invalid on a single platform and were excluded from further analysis. Discordant results were distributed randomly across the assays. The Ct values from all assays correlated closely with each other. All discordant samples showed Ct values ≥ 26. SARS-CoV-2 RT-qPCR assays may show considerable variability, especially in samples with low viral RNA concentrations. Decision makers should thus balance the advantages and disadvantages of RT-qPCR for mass screening and adopt suitable strategies that ensure a rational management of positive samples with high Ct values.

OUP accepted manuscript

Objective

This study aimed to examine the intra- and interlaboratory variations of cycle threshold (Ct) values using the nationwide proficiency testing for SARS-CoV-2.

Methods

Triplicated strong-positive contrived samples duplicated weak-positive contrived samples, and 2 negative samples were transported to participating laboratories in October 2021.

Results

A total of 232 laboratories responded. All except 4 laboratories correctly answered. Six false-negative results, including 2 false-negatives with Ct values beyond the threshold and 1 clerical error, were noted from weak-positive samples. Intralaboratory variations of Ct values of weak-positive and strong-positive samples were not acceptable (Ct > 1.66) in 17 and 7 laboratories, respectively. High interlaboratory variations of Ct values (up to 7 cycles) for the 2 commonly used polymerase chain reaction (PCR) reagents were observed.

Conclusion

The overall qualitative performance was acceptable; intralaboratory variation was acceptable. However, interlaboratory variations of Ct values were remarkable even when the same PCR reagents were used.

Inadequate design of mutation detection panels prevents interpretation of variants of concern: results of an external quality assessment for SARS-CoV-2 variant detection

OBJECTIVES

Mutation-specific PCR assays have quickly found their way into laboratory diagnostics due to their capacity to be a fast, easy to implement and high-throughput method for the detection of known SARS-CoV-2 variants of concern (VoCs). However, little is known about the performance of such assays in routine laboratory analysis.

METHODS

The results reported in a recent round of an external quality assessment (EQA) scheme for SARS-CoV-2 mutation-specific PCR were retrospectively analyzed. For the determination of individual variant-specific sequences as well as for the interpretation results for certain virus variants, correct, incorrect, and unreported results were evaluated, and their possible causes were investigated.

RESULTS

A total of 34 laboratories participated in this study. For five samples containing the VoC Alpha + E484K, Beta, Gamma, Delta, or B.1.1.318 (as a variant of interest), 848 results for SARS-2-CoV mutation detection were reported, 824 (97.2%, range per sample 88-100%) of which were correct. Melting curve assays gave 99% correct results, real-time RT-qPCR 94%, microarray-based assays 100%, and MALDI-TOF MS 96%. A total of 122/167 (73%) reported results for SARS-CoV-2 variant determination were correct. Of the 45 inconclusive or incorrect results, 33 (73%) were due to inadequate selection of targets that did not allow identification of contemporary VoC, 11 (24%) were due to incorrect results, and one (3%) was due to correct results of mutation-specific PCR.

CONCLUSIONS

Careful and up-to-date selection of the targets used in mutation-specific PCR is essential for successful detection of current SARS-CoV-2 variants.