Study designs for comparative diagnostic test accuracy: A methodological review and classification scheme

OBJECTIVES

(1) To identify and classify comparative diagnostic test accuracy (DTA) study designs; (2) to describe study design labels used by authors of comparative DTA studies.

METHODS

We performed a methodological review of 100 comparative DTA studies published between 2015 and 2017, randomly sampled from studies included in 238 comparative DTA systematic reviews indexed in MEDLINE in 2017. From each study report, we extracted six design elements characterizing participant flow and the labels used by authors.

RESULTS

We identified a total of 46 unique combinations of study design features in our sample, based on six design elements characterizing participant flow. We classified the studies into five study design categories based on how participants were allocated to receive each index test: 'fully paired' (n=79), 'partially paired, random subset' (n=0), 'partially paired, nonrandom subset' (n=2), 'unpaired randomized' (n=1) and 'unpaired nonrandomized' (n=3). The allocation method used in 15 studies was unclear. Sixty-one studies reported, in total, 29 unique study design labels but only four labels referred to specific design features of comparative studies.

CONCLUSION

Our classification scheme can help systematic review authors define study eligibility criteria, assess risk of bias, and communicate the strength of the evidence. A standardized labelling scheme could be developed to facilitate communication of specific design features.

GRADE Guidance: 31. Assessing the certainty across a body of evidence for comparative test accuracy

OBJECTIVES

This article provides GRADE guidance on how authors of evidence syntheses and health decision makers, including guideline developers, can rate the certainty across a body of evidence for comparative test accuracy questions.

STUDY DESIGN AND SETTING

This guidance extends the previously published GRADE guidance for assessing certainty of evidence for test accuracy to scenarios in which two or more index tests are compared. Through an iterative brainstorm-discussion-feedback process within the GRADE working group, we developed a guidance accompanied by practical examples.

RESULTS

Rating the certainty of evidence for comparative test accuracy shares many concepts and ideas with the existing GRADE guidance for test accuracy. The rating in comparisons of test accuracy requires additional considerations, such as the selection of appropriate comparative study designs, additional criteria for judging risk of bias, and the consequences of using comparative measures of test accuracy. Distinct approaches to rating certainty are required for comparative test accuracy studies and between-study (indirect) comparisons.

CONCLUSION

This GRADE guidance will support transparent assessment of the certainty for a body of comparative test accuracy evidence.

Comparison of commercial and in-house real-time PCR platforms for 15 parasites and microsporidia in human stool samples without a gold standard

INTRODUCTION

A test comparison of in-house and commercial real-time PCR (qPCR) kits for the detection of human parasites and microsporidia in stool samples was conducted without a gold standard. Three different commercial kits were included in the comparison, with a range of 3-15 different PCR targets, while 14 targets were covered by in-house testing, so not all 16 target pathogens were covered by all assays.

METHODS

Residual materials from nucleic acid extractions of stool samples with very high likelihood of being colonized or infected by at least one enteric parasite species or microsporidia were tested. In all, 500 DNA samples were analyzed, but due to limited sample volume, only 250 of the 500 samples were tested per assay. Each sample was assessed with the qPCR platforms being compared and cycle threshold (Ct) values were included in a descriptive comparison.

RESULTS

Depending on the assay applied, qPCR detected per 250 tested samples Giardia duodenalis (184-205), Blastocystis spp. (174-183), Trichuris trichiura (118-120), Ascaris lumbricoides (79-96), Necator americanus (78-106), Hymenolepis nana (40-42), Cryptosporidium spp. (27-36), Dientamoeba fragilis (26-28), Schistosoma spp. (13-23), Enterobius vermicularis (8-14), Entamoeba histolytica (7-16), Strongyloides stercoralis (6-38), Cyclospora spp. (6-13), Taenia spp. (1-4), microsporidia (1-5), and Ancylostoma spp. (1-2). Inter-assay agreement kappa was almost perfect (0.81-1) for Dientamoeba fragilis, Hymenolepis nana, Cryptosporidium spp., and Ascaris lumbricoides, substantial (0.61-0.8) for Necator americanus, Blastocystis spp., Ancylostoma spp., Giardia duodenalis, Schistosoma spp., Trichuris trichiura, and Enterobius vermicularis, moderate (0.41-0.6) for Entamoeba histolytica, fair (0.21-0.4) for microsporidia, slight (0-0.2) for Cyclospora spp. and Strongyloides stercoralis, and poor (<0) for Taenia spp.

CONCLUSIONS

Varying inter-assay agreement makes interpretation of microsporidia and parasite PCR in stool samples challenging. Intra-assay agreement had been controlled during the developing of the assays. Future studies, e.g., with optimized nucleic acid procedures and including microscopically characterized samples, are advisable.

Methods and reporting of systematic reviews of comparative accuracy were deficient: a methodological survey and proposed guidance

OBJECTIVE

The objective of this study was to examine methodological and reporting characteristics of systematic reviews and meta-analyses which compare diagnostic test accuracy (DTA) of multiple index tests, identify good practice, and develop guidance for better reporting.

STUDY DESIGN AND SETTING

Methodological survey of 127 comparative or multiple tests reviews published in 74 different general medical and specialist journals. We summarized methods and reporting characteristics that are likely to differ between reviews of a single test and comparative reviews. We then developed guidance to enhance reporting of test comparisons in DTA reviews.

RESULTS

Of 127 reviews, 16 (13%) reviews restricted study selection and test comparisons to comparative accuracy studies while the remaining 111 (87%) reviews included any study type. Fifty-three reviews (42%) statistically compared test accuracy with only 18 (34%) of these using recommended methods. Reporting of several items-in particular the role of the index tests, test comparison strategy, and limitations of indirect comparisons (i.e., comparisons involving any study type)-was deficient in many reviews. Five reviews with exemplary methods and reporting were identified.

CONCLUSION

Reporting quality of reviews which evaluate and compare multiple tests is poor. The guidance developed, complemented with the exemplars, can assist review authors in producing better quality comparative reviews.

A comparison of two PCR protocols for the differentiation of Plasmodium ovale species and implications for clinical management in travellers returning to Germany: a 10-year cross-sectional study

BACKGROUND

To assess the occurrence of Plasmodium ovale wallikeri and Plasmodium ovale curtisi species in travellers returning to Germany, two real-time PCR protocols for the detection and differentiation of the two P. ovale species were compared. Results of parasite differentiation were correlated with patient data.

METHODS

Residual nucleic acid extractions from EDTA blood samples of patients with P. ovale spp. malaria, collected between 2010 and 2019 at the National Reference Centre for Tropical Pathogens in Germany, were subjected to further parasite discrimination in a retrospective assessment. All samples had been analysed by microscopy and by P. ovale spp.-specific real-time PCR without discrimination on species level. Two different real-time PCR protocols for species discrimination of P. o. curtisi and P. o. wallikeri were carried out. Results were correlated with patient data on gender, age, travel destination, thrombocyte count, and duration of parasite latency.

RESULTS

Samples from 77 P. ovale spp. malaria patients were assessed, with a male:female ratio of about 2:1 and a median age of 30 years. Parasitaemia was low, ranging from few visible parasites up to 1% infected erythrocytes. Discriminative real-time PCRs revealed 41 cases of P. o. curtisi and 36 cases of P. o. wallikeri infections. Concordance of results by the two PCR approaches was 100%. Assessment of travel destinations confirmed co-existence of P. o. curtisi and P. o. wallikeri over a wide range of countries in sub-Saharan Africa. Latency periods for the two P. ovale species were similar, with median values of 56.0 days for P. o. curtisi and 58.0 days for P. o. wallikeri; likewise, there was no statistically significant difference in thrombocyte count with median values of 138.5/µL for patients with P. o. curtisi and 152.0/µL for P. o. wallikeri-infected patients.

CONCLUSIONS

Two different real-time PCR protocols were found to be suitable for the discrimination of P. o. curtisi and P. o. wallikeri with only minor differences in sensitivity. Due to the overall low parasitaemia and the lack of differences in severity-related aspects like parasite latency periods or thrombocyte counts, this study supports the use of P. ovale spp. PCR without discrimination on species level to confirm the diagnosis and to inform clinical management of malaria in these patients.

Serological approaches for the diagnosis of schistosomiasis - A review

Schistosomiasis is a common disease in endemic areas of Sub-Saharan Africa, South America and Asia. It is rare in Europe, mainly imported from endemic countries due to travelling or human migration. Available methods for the diagnosis of schistosomiasis comprise microscopic, molecular and serological approaches, with the latter detecting antigens or antibodies associated with Schistosoma spp. infection. The serological approach is a valuable screening tool in low-endemicity settings and for travel medicine, though the interpretation of any diagnostic results requires knowledge of test characteristics and a patient's history. Specific antibody detection by most currently used assays is only possible in a relatively late stage of infection and does not allow for the differentiation of acute from previous infections for therapeutic control or the discrimination between persisting infection and re-infection. Throughout the last decades, new target antigens have been identified, and assays with improved performance and suitability for use in the field have been developed. For numerous assays, large-scale studies are still required to reliably characterise assay characteristics alone and in association with other available methods for the diagnosis of schistosomiasis. Apart from S. mansoni, S. haematobium and S. japonicum, for which most available tests were developed, other species of Schistosoma that occur less frequently need to be taken into account. This narrative review describes and critically discusses the results of published studies on the evaluation of serological assays that detect antibodies against different Schistosoma species of humans. It provides insights into the diagnostic performance and an overview of available assays and their suitability for large-scale use or individual diagnosis, and thus sets the scene for serological diagnosis of schistosomiasis and the interpretation of results.