Promise versus reality: optimism bias in package inserts for tuberculosis diagnostics

Accuracy of package inserts of SARS-CoV-2 rapid antigen tests: a secondary analysis of manufacturer versus systematic review data

BACKGROUND

Rapid antigen tests (RATs) were crucial during the COVID-19 pandemic. Information provided by the test manufacturer in product package inserts, also known as instructions for use (IFUs), is often the only data available to clinicians, public health professionals, and individuals on the diagnostic accuracy of these tests. We aimed to assess whether manufacturer IFU accuracy data aligned with evidence from independent research.

METHODS

We searched company websites for package inserts for RATs that were included in the July 2022 update of the Cochrane meta-analysis of SARS-CoV-2 RATs, which served as a benchmark for research evidence. We fitted bivariate hierarchical models to obtain absolute differences in sensitivity and specificity between IFU and Cochrane Review estimates for each test, as well as overall combined differences.

FINDINGS

We found 22 (100%) of 22 IFUs of the RATs included in the Cochrane Review. IFUs for 12 (55%) of 22 RATs reported statistically significantly higher sensitivity estimates than the Cochrane Review, and none reported lower estimates. The mean difference between IFU and Cochrane Review sensitivity estimates across tests was 12·0% (95% CI 7·5-16·6). IFUs in three (14%) of 22 diagnostic tests had significantly higher specificity estimates than the Cochrane Review and two (9%) of 22 had lower estimates. The mean difference between IFU and Cochrane Review specificity estimates across tests was 0·3% (95% CI 0·1-0·5). If 100 people with SARS-CoV-2 infection were tested with each of the tests in this study, on average 12 fewer people would be correctly diagnosed than is suggested by the package inserts.

INTERPRETATION

Health professionals and the public should be aware that package inserts for SARS-CoV-2 RATs might provide an overly optimistic picture of the sensitivity of a test. Regulatory bodies should strengthen their requirements for the reporting of diagnostic accuracy data in package inserts and policy makers should demand independent validation data for decision making.

FUNDING

None.

Accuracy of novel antigen rapid diagnostics for SARS-CoV-2: A living systematic review and meta-analysis

BACKGROUND

SARS-CoV-2 antigen rapid diagnostic tests (Ag-RDTs) are increasingly being integrated in testing strategies around the world. Studies of the Ag-RDTs have shown variable performance. In this systematic review and meta-analysis, we assessed the clinical accuracy (sensitivity and specificity) of commercially available Ag-RDTs.

METHODS AND FINDINGS

We registered the review on PROSPERO (registration number: CRD42020225140). We systematically searched multiple databases (PubMed, Web of Science Core Collection, medRvix, bioRvix, and FIND) for publications evaluating the accuracy of Ag-RDTs for SARS-CoV-2 up until 30 April 2021. Descriptive analyses of all studies were performed, and when more than 4 studies were available, a random-effects meta-analysis was used to estimate pooled sensitivity and specificity in comparison to reverse transcription polymerase chain reaction (RT-PCR) testing. We assessed heterogeneity by subgroup analyses, and rated study quality and risk of bias using the QUADAS-2 assessment tool. From a total of 14,254 articles, we included 133 analytical and clinical studies resulting in 214 clinical accuracy datasets with 112,323 samples. Across all meta-analyzed samples, the pooled Ag-RDT sensitivity and specificity were 71.2% (95% CI 68.2% to 74.0%) and 98.9% (95% CI 98.6% to 99.1%), respectively. Sensitivity increased to 76.3% (95% CI 73.1% to 79.2%) if analysis was restricted to studies that followed the Ag-RDT manufacturers' instructions. LumiraDx showed the highest sensitivity, with 88.2% (95% CI 59.0% to 97.5%). Of instrument-free Ag-RDTs, Standard Q nasal performed best, with 80.2% sensitivity (95% CI 70.3% to 87.4%). Across all Ag-RDTs, sensitivity was markedly better on samples with lower RT-PCR cycle threshold (Ct) values, i.e., <20 (96.5%, 95% CI 92.6% to 98.4%) and <25 (95.8%, 95% CI 92.3% to 97.8%), in comparison to those with Ct ≥ 25 (50.7%, 95% CI 35.6% to 65.8%) and ≥30 (20.9%, 95% CI 12.5% to 32.8%). Testing in the first week from symptom onset resulted in substantially higher sensitivity (83.8%, 95% CI 76.3% to 89.2%) compared to testing after 1 week (61.5%, 95% CI 52.2% to 70.0%). The best Ag-RDT sensitivity was found with anterior nasal sampling (75.5%, 95% CI 70.4% to 79.9%), in comparison to other sample types (e.g., nasopharyngeal, 71.6%, 95% CI 68.1% to 74.9%), although CIs were overlapping. Concerns of bias were raised across all datasets, and financial support from the manufacturer was reported in 24.1% of datasets. Our analysis was limited by the included studies' heterogeneity in design and reporting.

CONCLUSIONS

In this study we found that Ag-RDTs detect the vast majority of SARS-CoV-2-infected persons within the first week of symptom onset and those with high viral load. Thus, they can have high utility for diagnostic purposes in the early phase of disease, making them a valuable tool to fight the spread of SARS-CoV-2. Standardization in conduct and reporting of clinical accuracy studies would improve comparability and use of data.

Quantification of circulating Mycobacterium tuberculosis antigen peptides allows rapid diagnosis of active disease and treatment monitoring

Tuberculosis (TB) is a major global health threat, resulting in an urgent unmet need for a rapid, non-sputum-based quantitative test to detect active Mycobacterium tuberculosis (Mtb) infections in clinically diverse populations and quickly assess Mtb treatment responses for emerging drug-resistant strains. We have identified Mtb-specific peptide fragments and developed a method to rapidly quantify their serum concentrations, using antibody-labeled and energy-focusing porous discoidal silicon nanoparticles (nanodisks) and high-throughput mass spectrometry (MS) to enhance sensitivity and specificity. NanoDisk-MS diagnosed active Mtb cases with high sensitivity and specificity in a case-control study with cohorts reflecting the complexity of clinical practice. Similar robust sensitivities were obtained for cases of culture-positive pulmonary TB (PTB; 91.3%) and extrapulmonary TB (EPTB; 92.3%), and the sensitivities obtained for culture-negative PTB (82.4%) and EPTB (75.0%) in HIV-positive patients significantly outperformed those reported for other available assays. NanoDisk-MS also exhibited high specificity (87.1-100%) in both healthy and high-risk groups. Absolute quantification of serum Mtb antigen concentration was informative in assessing responses to antimycobacterial treatment. Thus, a NanoDisk-MS assay approach could significantly improve the diagnosis and management of active TB cases, and perhaps other infectious diseases as well.

Target product profile of a molecular drug-susceptibility test for use in microscopy centers

BACKGROUND

Current phenotypic testing for drug resistance in patients with tuberculosis is inadequate primarily with respect to turnaround time. Molecular tests hold the promise of an improved time to diagnosis.

METHODS

A target product profile for a molecular drug-susceptibility test (DST) was developed on the basis of a collaborative effort that included opinions gathered from researchers, clinicians, policy makers, and test developers on optimal clinical and operational characteristics in settings of intended use. In addition, the current diagnostic ecosystem and the diagnostic development landscape were mapped.

RESULTS

Molecular DSTs for detecting tuberculosis in microscopy centers should ideally evaluate for resistance to rifampin, fluoroquinolones, isoniazid, and pyrazinamide and enable the selection of the most appropriate treatment regimen. Performance characteristics of DSTs need to be optimized, but compromises can be made that depend on the trade-off between a false-positive result and a false-negative result. The operational requirements of a test will vary depending on the site of implementation. However, the most-important considerations pertain to quality control, maintenance and calibration, and the ability to export data.

CONCLUSION

This target product profile defines the needs as perceived by the tuberculosis stakeholder community and attempts to provide a means of communication with test developers to ensure that fit-for-purpose DSTs are being developed.

Results from early programmatic implementation of Xpert MTB/RIF testing in nine countries

Background

The Xpert MTB/RIF assay has garnered significant interest as a sensitive and rapid diagnostic tool to improve detection of sensitive and drug resistant tuberculosis. However, most existing literature has described the performance of MTB/RIF testing only in study conditions; little information is available on its use in routine case finding. TB REACH is a multi-country initiative focusing on innovative ways to improve case notification.

Methods

We selected a convenience sample of nine TB REACH projects for inclusion to cover a range of implementers, regions and approaches. Standard quarterly reports and machine data from the first 12 months of MTB/RIF implementation in each project were utilized to analyze patient yields, rifampicin resistance, and failed tests. Data was collected from September 2011 to March 2013. A questionnaire was implemented and semi-structured interviews with project staff were conducted to gather information on user experiences and challenges.

Results

All projects used MTB/RIF testing for people with suspected TB, as opposed to testing for drug resistance among already diagnosed patients. The projects placed 65 machines (196 modules) in a variety of facilities and employed numerous case-finding strategies and testing algorithms. The projects consumed 47,973 MTB/RIF tests. Of valid tests, 7,195 (16.8%) were positive for MTB. A total of 982 rifampicin resistant results were found (13.6% of positive tests). Of all tests conducted, 10.6% failed. The need for continuous power supply was noted by all projects and most used locally procured solutions. There was considerable heterogeneity in how results were reported and recorded, reflecting the lack of standardized guidance in some countries.

Conclusions

The findings of this study begin to fill the gaps among guidelines, research findings, and real-world implementation of MTB/RIF testing. Testing with Xpert MTB/RIF detected a large number of people with TB that routine services failed to detect. The study demonstrates the versatility and impact of the technology, but also outlines various surmountable barriers to implementation. The study is not representative of all early implementer experiences with MTB/RIF testing but rather provides an overview of the shared issues as well as the many different approaches to programmatic MTB/RIF implementation.

Challenges in the development of an immunochromatographic interferon-gamma test for diagnosis of pleural tuberculosis

Existing diagnostic tests for pleural tuberculosis (TB) have inadequate accuracy and/or turnaround time. Interferon-gamma (IFNg) has been identified in many studies as a biomarker for pleural TB. Our objective was to develop a lateral flow, immunochromatographic test (ICT) based on this biomarker and to evaluate the test in a clinical cohort. Because IFNg is commonly present in non-TB pleural effusions in low amounts, a diagnostic IFNg-threshold was first defined with an enzyme-linked immunosorbent assay (ELISA) for IFNg in samples from 38 patients with a confirmed clinical diagnosis (cut-off of 300 pg/ml; 94% sensitivity and 93% specificity). The ICT was then designed; however, its achievable limit of detection (5000 pg/ml) was over 10-fold higher than that of the ELISA. After several iterations in development, the prototype ICT assay for IFNg had a sensitivity of 69% (95% confidence interval (CI): 50-83) and a specificity of 94% (95% CI: 81-99%) compared to ELISA on frozen samples. Evaluation of the prototype in a prospective clinical cohort (72 patients) on fresh pleural fluid samples, in comparison to a composite reference standard (including histopathological and microbiologic test results), showed that the prototype had 65% sensitivity (95% CI: 44-83) and 89% specificity (95% CI: 74-97). Discordant results were observed in 15% of samples if testing was repeated after one freezing and thawing step. Inter-rater variability was limited (3%; 1 out of 32). In conclusion, despite an iterative development and optimization process, the performance of the IFNg ICT remained lower than what could be expected from the published literature on IFNg as a biomarker in pleural fluid. Further improvements in the limit of detection of an ICT for IFNg, and possibly combination of IFNg with other biomarkers such as adenosine deaminase, are necessary for such a test to be of value in the evaluation of pleural tuberculosis.