1
|
Bigio J, MacLean ELH, Das R, Sulis G, Kohli M, Berhane S, Dinnes J, Deeks JJ, Brümmer LE, Denkinger CM, Pai M. Accuracy of package inserts of SARS-CoV-2 rapid antigen tests: a secondary analysis of manufacturer versus systematic review data. THE LANCET. MICROBE 2023; 4:e875-e882. [PMID: 37844595 DOI: 10.1016/s2666-5247(23)00222-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 05/05/2023] [Accepted: 07/17/2023] [Indexed: 10/18/2023]
Abstract
BACKGROUND Rapid antigen tests (RATs) were crucial during the COVID-19 pandemic. Information provided by the test manufacturer in product package inserts, also known as instructions for use (IFUs), is often the only data available to clinicians, public health professionals, and individuals on the diagnostic accuracy of these tests. We aimed to assess whether manufacturer IFU accuracy data aligned with evidence from independent research. METHODS We searched company websites for package inserts for RATs that were included in the July 2022 update of the Cochrane meta-analysis of SARS-CoV-2 RATs, which served as a benchmark for research evidence. We fitted bivariate hierarchical models to obtain absolute differences in sensitivity and specificity between IFU and Cochrane Review estimates for each test, as well as overall combined differences. FINDINGS We found 22 (100%) of 22 IFUs of the RATs included in the Cochrane Review. IFUs for 12 (55%) of 22 RATs reported statistically significantly higher sensitivity estimates than the Cochrane Review, and none reported lower estimates. The mean difference between IFU and Cochrane Review sensitivity estimates across tests was 12·0% (95% CI 7·5-16·6). IFUs in three (14%) of 22 diagnostic tests had significantly higher specificity estimates than the Cochrane Review and two (9%) of 22 had lower estimates. The mean difference between IFU and Cochrane Review specificity estimates across tests was 0·3% (95% CI 0·1-0·5). If 100 people with SARS-CoV-2 infection were tested with each of the tests in this study, on average 12 fewer people would be correctly diagnosed than is suggested by the package inserts. INTERPRETATION Health professionals and the public should be aware that package inserts for SARS-CoV-2 RATs might provide an overly optimistic picture of the sensitivity of a test. Regulatory bodies should strengthen their requirements for the reporting of diagnostic accuracy data in package inserts and policy makers should demand independent validation data for decision making. FUNDING None.
Collapse
Affiliation(s)
- Jacob Bigio
- Research Institute of the McGill University Health Centre, Montreal General Hospital, Montreal, QC, Canada; McGill International TB Centre, Montreal, QC, Canada.
| | - Emily L-H MacLean
- Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia
| | - Rishav Das
- Research Institute of the McGill University Health Centre, Montreal General Hospital, Montreal, QC, Canada; McGill International TB Centre, Montreal, QC, Canada
| | - Giorgia Sulis
- School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada; Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada
| | | | - Sarah Berhane
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Lukas E Brümmer
- Division of Infectious Diseases and Tropical Medicine, Center of Infectious Diseases, Heidelberg University Hospital, Germany
| | - Claudia M Denkinger
- Division of Infectious Diseases and Tropical Medicine, Center of Infectious Diseases, Heidelberg University Hospital, Germany; German Centre for Infection Research, Partner Site Heidelberg University Hospital, Heidelberg, Germany
| | - Madhukar Pai
- McGill International TB Centre, Montreal, QC, Canada; Department of Epidemiology, Biostatistics and Occupational Health, School of Population and Global Health, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
| |
Collapse
|
2
|
Brümmer LE, Katzenschlager S, Gaeddert M, Erdmann C, Schmitz S, Bota M, Grilli M, Larmann J, Weigand MA, Pollock NR, Macé A, Carmona S, Ongarello S, Sacks JA, Denkinger CM. Accuracy of novel antigen rapid diagnostics for SARS-CoV-2: A living systematic review and meta-analysis. PLoS Med 2021; 18:e1003735. [PMID: 34383750 PMCID: PMC8389849 DOI: 10.1371/journal.pmed.1003735] [Citation(s) in RCA: 159] [Impact Index Per Article: 53.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/26/2021] [Revised: 08/26/2021] [Accepted: 07/14/2021] [Indexed: 12/11/2022] Open
Abstract
BACKGROUND SARS-CoV-2 antigen rapid diagnostic tests (Ag-RDTs) are increasingly being integrated in testing strategies around the world. Studies of the Ag-RDTs have shown variable performance. In this systematic review and meta-analysis, we assessed the clinical accuracy (sensitivity and specificity) of commercially available Ag-RDTs. METHODS AND FINDINGS We registered the review on PROSPERO (registration number: CRD42020225140). We systematically searched multiple databases (PubMed, Web of Science Core Collection, medRvix, bioRvix, and FIND) for publications evaluating the accuracy of Ag-RDTs for SARS-CoV-2 up until 30 April 2021. Descriptive analyses of all studies were performed, and when more than 4 studies were available, a random-effects meta-analysis was used to estimate pooled sensitivity and specificity in comparison to reverse transcription polymerase chain reaction (RT-PCR) testing. We assessed heterogeneity by subgroup analyses, and rated study quality and risk of bias using the QUADAS-2 assessment tool. From a total of 14,254 articles, we included 133 analytical and clinical studies resulting in 214 clinical accuracy datasets with 112,323 samples. Across all meta-analyzed samples, the pooled Ag-RDT sensitivity and specificity were 71.2% (95% CI 68.2% to 74.0%) and 98.9% (95% CI 98.6% to 99.1%), respectively. Sensitivity increased to 76.3% (95% CI 73.1% to 79.2%) if analysis was restricted to studies that followed the Ag-RDT manufacturers' instructions. LumiraDx showed the highest sensitivity, with 88.2% (95% CI 59.0% to 97.5%). Of instrument-free Ag-RDTs, Standard Q nasal performed best, with 80.2% sensitivity (95% CI 70.3% to 87.4%). Across all Ag-RDTs, sensitivity was markedly better on samples with lower RT-PCR cycle threshold (Ct) values, i.e., <20 (96.5%, 95% CI 92.6% to 98.4%) and <25 (95.8%, 95% CI 92.3% to 97.8%), in comparison to those with Ct ≥ 25 (50.7%, 95% CI 35.6% to 65.8%) and ≥30 (20.9%, 95% CI 12.5% to 32.8%). Testing in the first week from symptom onset resulted in substantially higher sensitivity (83.8%, 95% CI 76.3% to 89.2%) compared to testing after 1 week (61.5%, 95% CI 52.2% to 70.0%). The best Ag-RDT sensitivity was found with anterior nasal sampling (75.5%, 95% CI 70.4% to 79.9%), in comparison to other sample types (e.g., nasopharyngeal, 71.6%, 95% CI 68.1% to 74.9%), although CIs were overlapping. Concerns of bias were raised across all datasets, and financial support from the manufacturer was reported in 24.1% of datasets. Our analysis was limited by the included studies' heterogeneity in design and reporting. CONCLUSIONS In this study we found that Ag-RDTs detect the vast majority of SARS-CoV-2-infected persons within the first week of symptom onset and those with high viral load. Thus, they can have high utility for diagnostic purposes in the early phase of disease, making them a valuable tool to fight the spread of SARS-CoV-2. Standardization in conduct and reporting of clinical accuracy studies would improve comparability and use of data.
Collapse
Affiliation(s)
- Lukas E. Brümmer
- Division of Tropical Medicine, Center for Infectious Diseases, Heidelberg University Hospital, Heidelberg, Germany
| | | | - Mary Gaeddert
- Division of Tropical Medicine, Center for Infectious Diseases, Heidelberg University Hospital, Heidelberg, Germany
| | | | - Stephani Schmitz
- Division of Tropical Medicine, Center for Infectious Diseases, Heidelberg University Hospital, Heidelberg, Germany
| | - Marc Bota
- Agaplesion Bethesda Hospital, Hamburg, Germany
| | - Maurizio Grilli
- Library, University Medical Center Mannheim, Mannheim, Germany
| | - Jan Larmann
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - Markus A. Weigand
- Department of Anesthesiology, Heidelberg University Hospital, Heidelberg, Germany
| | - Nira R. Pollock
- Department of Laboratory Medicine, Boston Children’s Hospital, Boston, Massachusetts, United States of America
| | | | | | | | | | - Claudia M. Denkinger
- Division of Tropical Medicine, Center for Infectious Diseases, Heidelberg University Hospital, Heidelberg, Germany
- Partner Site Heidelberg University Hospital, German Center for Infection Research (DZIF), Heidelberg, Germany
| |
Collapse
|
3
|
Quantification of circulating Mycobacterium tuberculosis antigen peptides allows rapid diagnosis of active disease and treatment monitoring. Proc Natl Acad Sci U S A 2017; 114:3969-3974. [PMID: 28348223 DOI: 10.1073/pnas.1621360114] [Citation(s) in RCA: 79] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/01/2023] Open
Abstract
Tuberculosis (TB) is a major global health threat, resulting in an urgent unmet need for a rapid, non-sputum-based quantitative test to detect active Mycobacterium tuberculosis (Mtb) infections in clinically diverse populations and quickly assess Mtb treatment responses for emerging drug-resistant strains. We have identified Mtb-specific peptide fragments and developed a method to rapidly quantify their serum concentrations, using antibody-labeled and energy-focusing porous discoidal silicon nanoparticles (nanodisks) and high-throughput mass spectrometry (MS) to enhance sensitivity and specificity. NanoDisk-MS diagnosed active Mtb cases with high sensitivity and specificity in a case-control study with cohorts reflecting the complexity of clinical practice. Similar robust sensitivities were obtained for cases of culture-positive pulmonary TB (PTB; 91.3%) and extrapulmonary TB (EPTB; 92.3%), and the sensitivities obtained for culture-negative PTB (82.4%) and EPTB (75.0%) in HIV-positive patients significantly outperformed those reported for other available assays. NanoDisk-MS also exhibited high specificity (87.1-100%) in both healthy and high-risk groups. Absolute quantification of serum Mtb antigen concentration was informative in assessing responses to antimycobacterial treatment. Thus, a NanoDisk-MS assay approach could significantly improve the diagnosis and management of active TB cases, and perhaps other infectious diseases as well.
Collapse
|
4
|
Denkinger CM, Dolinger D, Schito M, Wells W, Cobelens F, Pai M, Zignol M, Cirillo DM, Alland D, Casenghi M, Gallarda J, Boehme CC, Perkins MD. Target product profile of a molecular drug-susceptibility test for use in microscopy centers. J Infect Dis 2015; 211 Suppl 2:S39-49. [PMID: 25765105 DOI: 10.1093/infdis/jiu682] [Citation(s) in RCA: 32] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
BACKGROUND Current phenotypic testing for drug resistance in patients with tuberculosis is inadequate primarily with respect to turnaround time. Molecular tests hold the promise of an improved time to diagnosis. METHODS A target product profile for a molecular drug-susceptibility test (DST) was developed on the basis of a collaborative effort that included opinions gathered from researchers, clinicians, policy makers, and test developers on optimal clinical and operational characteristics in settings of intended use. In addition, the current diagnostic ecosystem and the diagnostic development landscape were mapped. RESULTS Molecular DSTs for detecting tuberculosis in microscopy centers should ideally evaluate for resistance to rifampin, fluoroquinolones, isoniazid, and pyrazinamide and enable the selection of the most appropriate treatment regimen. Performance characteristics of DSTs need to be optimized, but compromises can be made that depend on the trade-off between a false-positive result and a false-negative result. The operational requirements of a test will vary depending on the site of implementation. However, the most-important considerations pertain to quality control, maintenance and calibration, and the ability to export data. CONCLUSION This target product profile defines the needs as perceived by the tuberculosis stakeholder community and attempts to provide a means of communication with test developers to ensure that fit-for-purpose DSTs are being developed.
Collapse
Affiliation(s)
- Claudia M Denkinger
- FIND Division of Infectious Disease, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | | | - Marco Schito
- Division of AIDS, Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, Maryland
| | | | - Frank Cobelens
- KNCV Tuberculosis Foundation, the Hague Amsterdam Institute for Global Health and Development, Academic Medical Center, Amsterdam, The Netherlands
| | - Madhukar Pai
- McGill International TB Centre Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Canada
| | | | | | | | | | - Jim Gallarda
- Bill and Melinda Gates Foundation, Seattle, Washington
| | | | | |
Collapse
|
5
|
Results from early programmatic implementation of Xpert MTB/RIF testing in nine countries. BMC Infect Dis 2014; 14:2. [PMID: 24383553 PMCID: PMC3898850 DOI: 10.1186/1471-2334-14-2] [Citation(s) in RCA: 119] [Impact Index Per Article: 11.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2013] [Accepted: 12/20/2013] [Indexed: 11/24/2022] Open
Abstract
Background The Xpert MTB/RIF assay has garnered significant interest as a sensitive and rapid diagnostic tool to improve detection of sensitive and drug resistant tuberculosis. However, most existing literature has described the performance of MTB/RIF testing only in study conditions; little information is available on its use in routine case finding. TB REACH is a multi-country initiative focusing on innovative ways to improve case notification. Methods We selected a convenience sample of nine TB REACH projects for inclusion to cover a range of implementers, regions and approaches. Standard quarterly reports and machine data from the first 12 months of MTB/RIF implementation in each project were utilized to analyze patient yields, rifampicin resistance, and failed tests. Data was collected from September 2011 to March 2013. A questionnaire was implemented and semi-structured interviews with project staff were conducted to gather information on user experiences and challenges. Results All projects used MTB/RIF testing for people with suspected TB, as opposed to testing for drug resistance among already diagnosed patients. The projects placed 65 machines (196 modules) in a variety of facilities and employed numerous case-finding strategies and testing algorithms. The projects consumed 47,973 MTB/RIF tests. Of valid tests, 7,195 (16.8%) were positive for MTB. A total of 982 rifampicin resistant results were found (13.6% of positive tests). Of all tests conducted, 10.6% failed. The need for continuous power supply was noted by all projects and most used locally procured solutions. There was considerable heterogeneity in how results were reported and recorded, reflecting the lack of standardized guidance in some countries. Conclusions The findings of this study begin to fill the gaps among guidelines, research findings, and real-world implementation of MTB/RIF testing. Testing with Xpert MTB/RIF detected a large number of people with TB that routine services failed to detect. The study demonstrates the versatility and impact of the technology, but also outlines various surmountable barriers to implementation. The study is not representative of all early implementer experiences with MTB/RIF testing but rather provides an overview of the shared issues as well as the many different approaches to programmatic MTB/RIF implementation.
Collapse
|
6
|
Denkinger CM, Kalantri Y, Schumacher SG, Michael JS, Shankar D, Saxena A, Sriram N, Balamugesh T, Luo R, Pollock NR, Pai M, Christopher DJ. Challenges in the development of an immunochromatographic interferon-gamma test for diagnosis of pleural tuberculosis. PLoS One 2013; 8:e85447. [PMID: 24376880 PMCID: PMC3871622 DOI: 10.1371/journal.pone.0085447] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2013] [Accepted: 12/05/2013] [Indexed: 11/18/2022] Open
Abstract
Existing diagnostic tests for pleural tuberculosis (TB) have inadequate accuracy and/or turnaround time. Interferon-gamma (IFNg) has been identified in many studies as a biomarker for pleural TB. Our objective was to develop a lateral flow, immunochromatographic test (ICT) based on this biomarker and to evaluate the test in a clinical cohort. Because IFNg is commonly present in non-TB pleural effusions in low amounts, a diagnostic IFNg-threshold was first defined with an enzyme-linked immunosorbent assay (ELISA) for IFNg in samples from 38 patients with a confirmed clinical diagnosis (cut-off of 300 pg/ml; 94% sensitivity and 93% specificity). The ICT was then designed; however, its achievable limit of detection (5000 pg/ml) was over 10-fold higher than that of the ELISA. After several iterations in development, the prototype ICT assay for IFNg had a sensitivity of 69% (95% confidence interval (CI): 50-83) and a specificity of 94% (95% CI: 81-99%) compared to ELISA on frozen samples. Evaluation of the prototype in a prospective clinical cohort (72 patients) on fresh pleural fluid samples, in comparison to a composite reference standard (including histopathological and microbiologic test results), showed that the prototype had 65% sensitivity (95% CI: 44-83) and 89% specificity (95% CI: 74-97). Discordant results were observed in 15% of samples if testing was repeated after one freezing and thawing step. Inter-rater variability was limited (3%; 1 out of 32). In conclusion, despite an iterative development and optimization process, the performance of the IFNg ICT remained lower than what could be expected from the published literature on IFNg as a biomarker in pleural fluid. Further improvements in the limit of detection of an ICT for IFNg, and possibly combination of IFNg with other biomarkers such as adenosine deaminase, are necessary for such a test to be of value in the evaluation of pleural tuberculosis.
Collapse
Affiliation(s)
- Claudia M. Denkinger
- Division of Infectious Diseases, Beth Israel Deaconess Medical Center, Boston, Massachusetts, United States of America
- McGill International TB Centre & Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Quebec, Canada
- *
| | | | - Samuel G. Schumacher
- McGill International TB Centre & Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Quebec, Canada
| | - Joy S. Michael
- Department of Microbiology, Christian Medical College, Vellore, India
| | - Deepa Shankar
- Department of Pulmonary Medicine, Christian Medical College, Vellore, India
| | | | | | | | - Robert Luo
- Department of Pathology, Stanford University, Stanford, California, United States of America
| | - Nira R. Pollock
- Division of Infectious Diseases, Beth Israel Deaconess Medical Center, Boston, Massachusetts, United States of America
- Department of Laboratory Medicine, Boston Children’s Hospital, Boston, Massachusetts, United States of America
| | - Madhukar Pai
- McGill International TB Centre & Department of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Quebec, Canada
| | | |
Collapse
|