Biases in the assessment of diagnostic tests

Optimal two-phase sampling for comparing correlated areas under the ROC curves of two screening tests in the presence of verification bias

The accuracy of a screening test is often measured by the area under the receiver characteristic (ROC) curve (AUC) of a screening test. Two-phase designs have been widely used in diagnostic studies for estimating one single AUC and comparing two AUCs where the screening test results are measured for a large sample (Phase one sample) while the disease status is only verified for a subset of Phase one sample (Phase two sample) by a gold standard. In this paper, we consider the optimal two-phase sampling design for comparing the performance of two ordinal screening tests in classifying disease status. Specifically, we derive an analytical variance formula for the AUC difference estimator and use it to find the optimal sampling probabilities that minimize the variance formula for the AUC difference estimator. According to the proposed optimal two-phase design, the strata with the levels of two tests far apart from each other should be over-sampled while the strata with the levels of two tests close to each other should be under-sampled. Simulation results indicate that two-phase sampling under optimal allocation (OA) achieves a substantial amount of variance reduction, compared with two-phase sampling under proportional allocation (PA). Furthermore, in comparison with a one-phase random sampling, two-phase sampling under OA or PA has a clear advantage in reducing the variance of AUC difference estimator when the variances of the two screening test results in the disease population differ greatly from their counterparts in non-disease population.

Revolutionizing the diagnosis of irreversible pulpitis - Current strategies and future directions

BACKGROUND

Pulpitis primarily arises from the pulp space infection by oral microbiota. Vital pulp therapy is a minimally invasive approach that relies on assessing the severity of pulpal inflammation to facilitate repair. However, the current evaluation methods prescribed by the American Association of Endodontics are subjective, leading to ambiguity in assessment. Therefore, this review aims to explore molecular strategies for evaluating the severity of pulpal inflammation to accurately predict the success of pulp vitality preservation in clinical settings.

METHODOLOGY

This review was conducted by searching relevant keywords, such as irreversible pulpitis, pulpitis biomarkers, molecular diagnosis, inflammation, and genomic strategies, in databases such as PubMed, Web of Science, and Scopus to address the subjective nature of diagnosis. The data included in this review were collected up to April 2023. The literature search revealed well-documented limitations in clinically assessing the pulp inflammatory. Molecular approaches that aid in clinical differentiation between irreversible and reversible pulpitis may potentially enhance favorable outcomes in vital pulp therapy. Non-invasive diagnostic methods for pulpal assessment would also be valuable for determining whether the inflamed pulp is reversible, irreversible, or necrotic.

CONCLUSION

The present review examines the various molecular diagnostic approaches that have revolutionized the medical field and are considered the most promising empirical methodologies for the proactive detection of pulpal diseases. It also provides comprehensive insights into the current diagnostic methods, associated challenges, next-generation strategies, and future directions for diagnosing the severity of pulp inflammation.

Identification of Subtypes of Dry Eye Disease, Including a Candidate Corneal Neuropathic Pain Subtype Through the Use of a Latent Class Analysis

PURPOSE

In the absence of a gold-standard diagnostic test for different subtypes of dry eye disease (DED), we aimed to identify latent subtypes of DED within a well-characterized cohort.

DESIGN

This is a cross-sectional study of participants enrolled in the Sjögren International Collaborative Clinical Alliance (SICCA).

METHODS

A latent class analysis was applied to different dry eye-related signs/tests and symptoms of ocular pain (particularly those that aligned with corneal neuropathic pain) giving relative specificities and sensitivities of each diagnostic test or symptom in the SICCA population.

RESULTS

Four subtypes of DED were identified with putative designations including normal, asymptomatic dry eye, symptomatic dry eye, and corneal neuropathic pain.

CONCLUSIONS

More specific classification criteria are needed for DED. Latent class analysis applied to the signs and symptoms captured in the SICCA cohort may allow for the development and refinement of classification criteria for specific subtypes of dry eye.

Relative diagnostic accuracy of point-of-care tests to rule-in Dirofilaria immitis infection in clinically suspect dogs: A systematic review and meta-analysis

Canine heartworm, Dirofilaria immitis, can cause severe disease and sometimes death of the host. Associated clinical signs, lack of preventative usage and regional endemicity are unlikely sufficient by themselves to reach a definitive diagnosis. Several point-of-care (POC) diagnostic tests are commercially available to aid in-clinic diagnosis, however, there is variable diagnostic accuracy reported and no synthesis of published evidence. This systematic review aims at meta-analysing the likelihood ratio of a positive result (LR⁺) to inform the selection and interpretation of POC tests in practice to rule-in heartworm infection when there is clinical suspicion. Three literature index interfaces (Web of Science, PubMed, Scopus) were searched on November 11th, 2022, for diagnostic test evaluation (DTE) articles assessing at least one currently commercialised POC test. Risk of bias was assessed adapting the QUADAS-2 protocol and articles with no evidence of high risk of bias were meta-analysed if deemed applicable to our review objective. Substantial between DTE heterogeneity was investigated including potential threshold or covariate effects. A total of 324 primary articles were sourced and 18 were retained for full text review of which only three had low risk of bias in all four QUADAS-2 domains. Of the nine heartworm POC tests evaluated, only three, IDEXX SNAP (n DTEs = 6), Zoetis WITNESS (n DTEs = 3) and Zoetis VETSCAN (n DTEs = 5) could be analysed. Both WITNESS and VETSCAN DTEs showed substantial heterogeneity due to a putative threshold effect and no summary point estimates could be reported. SNAP DTEs showed acceptable heterogeneity, and a summary LR⁺ was estimated at 559.0 (95%CI: 24.3-12,847.4). The quality and heterogeneity of heartworm POC test DTEs was highly variable which restricted our summary of the diagnostic accuracy to only the SNAP test. A positive result from the SNAP test provides strong evidence of the presence of an infection with adult heartworm(s) in a dog patient and this test is warranted to rule-in clinical suspicion(s) in clinics. However, our review did not appraise the literature to assess the fitness of SNAP test, or any other POC tests, to rule-out heartworm infection in dogs without clinical suspicion or following heartworm therapy.

AI/ML in Precision Medicine: A Look Beyond the Hype

Artificial Intelligence (AI) and Machine Learning (ML) are making headlines in medical research, especially in drug discovery, digital imaging, disease diagnostics, genetic testing, and optimal care pathway (personalized care). However, the potential uses and benefits of AI/ML applications need to be distinguished from hype. In the 2022 American Statistical Association Biopharmaceutical Section Regulatory-Industry Statistical Workshop, we convened a panel of experts from FDA and industry to talk about the challenges of successfully applying AI/ML in precision medicine and how to overcome those challenges. This paper provides a summary and expansion on the topics discussed in the panel: the application of AI/ML, bias, and data quality.

The 14-item short health anxiety inventory (SHAI-14) used as a screening tool: appropriate interpretation and diagnostic accuracy of the Swedish version

BACKGROUND

The 14-item Short Health Anxiety Inventory (SHAI-14) is a common measure of health anxiety but its screening properties have not been studied. The aims of this study were to evaluate the SHAI-14 as a screening instrument, identify cut-offs for clinically significant health anxiety and investigate which scores correspond to different severity levels.

METHOD

The study included 1729 psychiatric patients and 85 healthy controls. Participants completed the SHAI-14 and underwent a diagnostic interview. Cut-off scores were evaluated in three scenarios to approximate screening 1) in a psychiatric clinic, 2) in a low prevalence setting and, 3) of healthy volunteers (cut-off for remission). Receiver operating characteristics were used. Classification of severity was based on the distribution of SHAI-14 scores reported by patients with clinically significant health anxiety.

RESULTS

The area under the curve (AUC) values were high in all scenarios (above 0.95). The optimal cut-off scores on the SHAI-14 were 22 in the psychiatric context, 29 in a setting with low prevalence of psychiatric disorders and 18 versus healthy controls. SHAI-14 scores of 0-27 represented no or mild health anxiety, 28-32 moderate health anxiety and 33-42 substantial health anxiety.

CONCLUSION

Brief self-report measures used as screening instruments are a simple way of gathering information about the presence of specific symptoms and thus a way to detect the likelihood of a diagnosis. The SHAI-14 shows evidence of good diagnostic utility in both clinical and non-clinical settings. However, which cut-off score is to be used, depends on the intended purpose and the setting where the cut-off is used.

Commentary: Statistical analysis of 2 x 2 tables in Biomarker studies 3) Design, Interpretation and Guidelines

Abstract/SummaryThis article discusses issues associated with the design and interpretation of biomarker studies, points to various guidelines and lists points to look out for in assessing studies.

Application of a complete blood count to screening lethargic and anorectic cats for pancreatitis

Background

Feline pancreatitis (FP) is an important health problem of cats. Its diagnostics is based on the combination of quantification of serum pancreatic lipase immunoreactivity (fPLI) and abdominal ultrasonography (AUS). These modalities allow for establishing highly specific diagnosis, however they are relatively expensive and time-consuming. On the other hand, a screening test of high sensitivity which would allow to rule out FP on the first visit without a considerable increase of costs would be clinically useful. To evaluate accuracy of nonspecific inflammatory biomarkers based on complete blood count (CBC) in diagnosing FP 73 client-owned cats with signs of lethargy and reduced appetite lasting for at least 2 days before presentation were enrolled in the cross-sectional study. They were examined with fPLI assay and AUS and classified as cats with very low risk of FP when fPLI ≤3.5 μg/L and AUS negative for FP, or as cats with increased risk of FP in the case of any other combination of results. Then, 7 various CBC measurements were measured in each cat and linked to the risk of FP using the multivariable logistic regression.

Results

Five CBC measurements turned out to be significantly associated with the risk of FP – total leukocyte count (WBC; crude odds ratio(OR_crude) = 12.2; CI 95%: 1.52, 98.5), total neutrophil count (OR_crude = 5.84; CI 95%: 1.22, 27.9), band neutrophil count (BNC; OR_crude = 6.67; CI 95%: 1.98, 22.4), neutrophil-to-lymphocyte ratio (OR_crude = 3.68; CI 95%: 1.25, 10.9), and eosinophil count (EC; OR_crude = 0.34; CI 95%: 0.12, 0.96). The model based on WBC, BNC, and EC proved to have at least fair diagnostic potential (area under ROC curve 82.7%; CI 95%: 72.8%, 92.5%). When WBC <  18 G/L, BNC <  0.27 G/L, and EC >  0.3 G/L was considered as a negative result, and any other combination as the positive result, the CBC model had high sensitivity (91.8%; CI 95%: 80.8%, 96.8%) at a relatively low specificity (58.3%; CI 95%: 38.8%, 75.5%).

Conclusion

The combination of three CBC measurements is an immediately available and fairly accurate screening method for identification of lethargic and anorectic cats with increased risk of FP.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12917-021-03098-z.

Predictions of attrition among US Marine Corps: Comparison of four predictive methods

The present study compared the performance of logistic regression models with that of machine learning classification models (classification trees and random forests) in the context of predicting training attrition from the Delayed Enlistment Program in the United States Marine Corps (USMC) with scores from the Tailored Adaptive Personality Assessment System (TAPAS). Performance was assessed according to the type of misclassification error and across a variety of different reasons for attrition. The base rate of attrition was low, which impeded the training process, but the machine learning models outperformed logistic regression in predicting voluntary attrition in a stratified 50% attrition sample.

Application of Artificial Intelligence to Clinical Practice in Inflammatory Bowel Disease - What the Clinician Needs to Know

Artificial intelligence [AI] techniques are quickly spreading across medicine as an analytical method to tackle challenging clinical questions. What were previously thought of as highly complex data sources, such as images or free text, are now becoming manageable. Novel analytical methods merge the latest developments in information technology infrastructure with advances in computer science. Once primarily associated with Silicon Valley, AI techniques are now making their way into medicine, including in the field of inflammatory bowel diseases [IBD]. Understanding potential applications and limitations of these techniques can be difficult, in particular for busy clinicians. In this article, we explain the basic terminologies and provide a particular focus on the foundations behind state-of-the-art AI methodologies in both imaging and text. We explore the growing applications of AI in medicine, with a specific focus on IBD to inform the practising gastroenterologist and IBD specialist. Finally, we outline possible future uses of these technologies in daily clinical practice.

How to Analyze the Diagnostic Performance of a New Test? Explained with Illustrations

Diagnostic tests are pivotal in modern medicine due to their applications in statistical decision-making regarding confirming or ruling out the presence of a disease in patients. In this regard, sensitivity and specificity are two most important and widely utilized components that measure the inherent validity of a diagnostic test for dichotomous outcomes against a gold standard test. Other diagnostic indices like positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, accuracy of a diagnostic test, and the effect of prevalence on various diagnostic indices have also been discussed. We have tried to present the performance of a classification model at all classification thresholds by reviewing the receiver operating characteristic (ROC) curve and the depiction of the tradeoff between sensitivity and (1–specificity) across a series of cutoff points when the diagnostic test is on a continuous scale. The area under the ROC (AUROC) and comparison of AUROCs of different tests have also been discussed. Reliability of a test is defined in terms of the repeatability of the test such that the test gives consistent results when repeated more than once on the same individual or material, under the same conditions. In this article, we have presented the calculation of kappa coefficient, which is the simplest way of finding the agreement between two observers by calculating the overall percentage of agreement. When the prevalence of disease in the population is low, prospective study becomes increasingly difficult to handle through the conventional design. Hence, we chose to describe three more designs along with the conventional one and presented the sensitivity and specificity calculations for those designs. We tried to offer some guidance in choosing the best possible design among these four designs, depending on a number of factors. The ultimate aim of this article is to provide the basic conceptual framework and interpretation of various diagnostic test indices, ROC analysis, comparison of diagnostic accuracy of different tests, and the reliability of a test so that the clinicians can use it effectively. Several R packages, as mentioned in this article, can prove handy during quantitative synthesis of clinical data related to diagnostic tests.

Systematic Reviews: Understanding the Best Evidence For Clinical Decision-making in Health Care: Pros and Cons

In the era of evidence-based decision-making, systematic reviews (SRs) are being widely used in many health care policies, government programs, and academic disciplines. SRs are detailed and comprehensive literature review of a specific research topic with a view to identifying, appraising, and synthesizing the research findings from various relevant primary studies. A SR therefore extracts the relevant summary information from the selected studies without bias by strictly adhering to the review procedures and protocols. This paper presents all underlying concepts, stages, steps, and procedures in conducting and publishing SRs. Unlike the findings of narrative reviews, the synthesized results of any SRs are reproducible, not subjective and bias free. However, there are a number of issues related to SRs that directly impact on the quality of the end results. If the selected studies are of high quality, the criteria of the SRs are fully satisfied, and the results constitute the highest level of evidence. It is therefore essential that the end users of SRs are aware of the weaknesses and strengths of the underlying processes and techniques so that they could assess the results in the correct perspective within the context of the research question.

Optimal two-phase sampling for estimating the area under the receiver operating characteristic curve

Statistical methods are well developed for estimating the area under the receiver operating characteristic curve (AUC) based on a random sample where the gold standard is available for every subject in the sample, or a two-phase sample where the gold standard is ascertained only at the second phase for a subset of subjects sampled using fixed sampling probabilities. However, the methods based on a two-phase sample do not attempt to optimize the sampling probabilities to minimize the variance of AUC estimator. In this paper, we consider the optimal two-phase sampling design for evaluating the performance of an ordinal test in classifying disease status. We derived the analytic variance formula for the AUC estimator and used it to obtain the optimal sampling probabilities. The efficiency of the two-phase sampling under the optimal sampling probabilities (OA) is evaluated by a simulation study, which indicates that two-phase sampling under OA achieves a substantial amount of variance reduction with an over-sample of subjects with low and high ordinal levels, compared with two-phase sampling under proportional allocation (PA). Furthermore, in comparison with an one-phase random sampling, two-phase sampling under OA or PA have a clear advantage in reducing the variance of AUC estimator when the variance of diagnostic test results in the disease population is small relative to its counterpart in nondisease population. Finally, we applied the optimal two-phase sampling design to a real-world example to evaluate the performance of a questionnaire score in screening for childhood asthma.

Ocular Clinical Signs and Diagnostic Tests Most Compatible With Keratoconjunctivitis Sicca: A Latent Class Approach

PURPOSE

To evaluate the ocular signs and tests for keratoconjunctivitis sicca (KCS) in the absence of a gold standard.

METHODS

Cross-sectional study of participants from the Sjögren's International Collaborative Clinical Alliance (SICCA) registry. Participants had oral/ocular/rheumatologic examinations, blood/saliva samples collected, and salivary gland biopsy. Latent class analysis (LCA) identified clusters of patients based on 3 to 4 predictor variables relating to signs or tests of KCS. The resulting model-based "gold standard" classification formed the basis for estimated sensitivity and specificity associated with these predictors.

RESULTS

A total of 3514 participants were enrolled into SICCA, with 52.9% classified as SS. LCA revealed a best-fit model with 2 groups. For the gold standard-positive group, an abnormal tear breakup time, ocular staining score (OSS), and Schirmer I had a sensitivity of 99.5%, 91.0%, and 47.4%, respectively. For the gold standard-negative group, an abnormal tear breakup time, OSS, and Schirmer I had a specificity of 32.0%, 84.0%, and 88.5%, respectively. OSS components (fluorescein and lissamine staining), exhibited a sensitivity of 82.6% and 90.5%, respectively, in the gold standard-positive group, whereas these signs in the gold standard-negative group had a specificity of 88.8% and 73.0%, respectively.

CONCLUSIONS

OSS and its components (fluorescein and lissamine staining) differentiated 2 groups from each other better than other KCS parameters and had relatively high sensitivity and specificity.

High-throughput multiplex qPCRs for the surveillance of zoonotic species of canine hookworms

The canine hookworms Ancylostoma braziliense, Ancylostoma ceylanicum, Ancylostoma caninum and Uncinaria stenocephala are not only capable of producing morbidity and mortality in dogs but are also neglected tropical zoonoses. Each hookworm species differs considerably in its geographical distribution, life cycle, biology, pathogenic impacts on both canine and human hosts, zoonotic potential, and response to treatment with anthelminthics. Here we describe the development and validation of two Taq-Man based multiplex PCR assays capable of detecting and differentiating all four canine hookworm species in faeces of naturally infected dogs. The analytical sensitivity of both assays was assessed using 10-fold serial dilutions of synthetic gene block fragments containing individual sequence targets of each hookworm species. The sensitivity of the assays and ability to detect mixed species infections were compared to a conventional PCR-Restriction Fragment Length Polymorphism based-approach when applied to laboratory and field samples from endemic areas. The qPCRs detected at least one species of hookworms in 82.4% of PCR-RFLP-negative but microscopy-positive samples. The qPCRs detected an additional 68% mixed infections with different species of canine hookworms, and additional single species infection with A. caninum (47%), U. stenocephala (33%) and A. ceylanicum (0.02%) that were missed by PCR-RFLP. These multiplex qPCR assays will assist field based epidemiological surveillance studies towards an accurate and sensitive monitoring of canine hookworm infections in dogs, to inform their species-specific zoonotic risks to populations living in endemic areas, globally.

Validity of the Mood Disorder Questionnaire (MDQ) as a screening tool for bipolar spectrum disorders in anabaptist populations

The Mood Disorder Questionnaire (MDQ) is an established screening tool for bipolar spectrum disorders (BSD), but has not been validated in diverse populations and the best scoring method remains uncertain. This study assessed diagnostic validity of the MDQ among Anabaptists, an underserved population frequently involved in genetic research. 161 participants completed the MDQ and were diagnosed by a best-estimate final diagnosis (BEFD). Diagnostic agreements between alternate MDQ scoring methods and the BEFD were quantified using Cohen's Kappa (κ), sensitivity (α), and specificity (β). Scoring criteria evaluated included >7 simultaneous symptoms and at least moderate impairment, >7 simultaneous symptoms, with at least mild impairment, >7 symptoms only, with no further requirement, and three novel scoring methods that require >5 symptoms or fewer. Diagnostic agreement varied. The original method proved most specific but had the lowest κ and sensitivity. κ increased with more liberal scoring criteria, reaching a maximum under the lower-threshold symptom methods, with little loss of specificity in the 5-symptom method. Decreasing the symptom threshold below 5 conferred little or no benefit. These results support the diagnostic validity of the MDQ among this Anabaptist sample and suggest that a 5-symptom scoring method may increase diagnostic sensitivity in populations at high risk for bipolar disorder.

Comparison of Two Bayesian Methods in Evaluation of the Absence of the Gold Standard Diagnostic Tests

Objective

The Bayesian model plays an important role in diagnostic test evaluation in the absence of the gold standard, which used the external prior distribution of a parameter combined with sample data to yield the posterior distribution of the test characteristics. However, the correlation between diagnostic tests has always been a problem that cannot be ignored in the Bayesian model evaluation. This study will discuss how different Bayesian model, correlation scenarios, and prior distribution affect the outcome.

Methods

The data analyzed in this study was gathered during studies of patients presenting to the Nanjing Chest Hospital with suspected tuberculosis. The diagnostic character of T-SPOT.Tb and KD38 tuberculosis antibody test were evaluated in different Bayesian model, and discharge diagnosis as a gold standard was used to verify the model results in the end.

Result

The comparison of four models under the conditional independence situation found that Bayesian probabilistic constraint model was consistent with the Conditional Covariance Bayesian model. The results were mainly affected by prior information. The sensitivity and specificity of the two tests in Conditional Covariance Bayesian model in prior constraint situation were considerably higher than the Bayesian probabilistic constraint model in prior constraint situation. The results of the four models under the conditional dependence situation were similar to the conditional independence situation; p_D was also negative with no prior constraint situation in both model Bayesian probabilistic constraint model and Conditional Covariance Bayesian model. The Deviance Information Criterion of Bayesian probabilistic constraint model was close to model Conditional Covariance Bayesian model, but p_D of Conditional Covariance Bayesian model in Prior constraint situation (p_D=2.40) was higher than the Bayesian probabilistic constraint model in Prior constraint situation (p_D=1.66).

Conclusion

The result of Conditional Covariance Bayesian model in prior constraint with conditional independence situation was closest to the result of gold standard evaluation in our data. Both of the two Bayesian methods are the feasible way for the evaluation of diagnostic test in the absence of the gold standard diagnostic. Prior source, priority number, and conditional dependencies should be considered in the method selection, the accuracy of posterior estimation mainly depending on the prior distribution.

Resting Heart Rate and the Risk of Atrial Fibrillation

In a previous meta-analysis, it was demonstrated that the resting heart rate (RHR) is a potential risk factor for atrial fibrillation (AF). However, the results of that meta-analysis were conflicting, and the relationship between the RHR and AF is still not well established. In the current meta-analysis, our aim is to update evidence with a better statistical model. We searched the Cochrane Library, PubMed, and Embase databases for relevant studies and used a "one-stage approach" with a restricted cubic spline model to summarize the dose-specific relationships between the RHR and AF. Relative risk (RR) was used to measure the effects. In total, 10 studies were included, with a total of 18,630 cases of AF among 431,432 participants. In the dose-response analysis, there was evidence of a nonlinear association between the RHR and the risk of AF (nonlinearity, P < 0.0001), which exhibited a significant J-shaped association between the two factors. An RHR between 68 and 80 bpm had the lowest risk of AF. Among people who had RHR < 70 bpm, the summary RR was 1.09 per 10-RHR decrease (95% confidence interval [CI] = 1.06-1.12; P < 0.001). The results were similar for participants with RHR > 70 bpm (per 10 bpm increase) (RR = 1.06, 95% CI = 1.03-1.08; P < 0.001). Our dose-response meta-analysis revealed a significant J-shaped association between the RHR and AF. Both low RHR and high RHR were associated with an increased risk of AF compared with a modest RHR of 68-80 bpm.

The effect of uncertainty in patient classification on diagnostic performance estimations

Background

The performance of a new diagnostic test is typically evaluated against a comparator which is assumed to correspond closely to some true state of interest. Judgments about the new test’s performance are based on the differences between the outputs of the test and comparator. It is commonly assumed that a small amount of uncertainty in the comparator’s classifications will negligibly affect the measured performance of a diagnostic test.

Methods

Simulated datasets were generated to represent typical diagnostic scenarios. Comparator noise was introduced in the form of random misclassifications, and the effect on the apparent performance of the diagnostic test was determined. An actual dataset from a clinical trial on a new diagnostic test for sepsis was also analyzed.

Results

We demonstrate that as little as 5% misclassification of patients by the comparator can be enough to statistically invalidate performance estimates such as sensitivity, specificity and area under the receiver operating characteristic curve, if this uncertainty is not measured and taken into account. This distortion effect is found to increase non-linearly with comparator uncertainty, under some common diagnostic scenarios. For clinical populations exhibiting high degrees of classification uncertainty, failure to measure and account for this effect will introduce significant risks of drawing false conclusions. The effect of classification uncertainty is magnified further for high performing tests that would otherwise reach near-perfection in diagnostic evaluation trials. A requirement of very high diagnostic performance for clinical adoption, such as a 99% sensitivity, can be rendered nearly unachievable even for a perfect test, if the comparator diagnosis contains even small amounts of uncertainty. This paper and an accompanying online simulation tool demonstrate the effect of classification uncertainty on the apparent performance of tests across a range of typical diagnostic scenarios. Both simulated and real datasets are used to show the degradation of apparent test performance as comparator uncertainty increases.

Conclusions

Overall, a 5% or greater misclassification rate by the comparator can lead to significant underestimation of true test performance. An online simulation tool allows researchers to explore this effect using their own trial parameters (https://imperfect-gold-standard.shinyapps.io/classification-noise/) and the source code is freely available (https://github.com/ksny/Imperfect-Gold-Standard).

The diagnostic accuracy of 1.5 T versus 3 T non-echo-planar diffusion-weighted imaging in the detection of residual or recurrent cholesteatoma in the middle ear and mastoid

PURPOSE AND BACKGROUND

This study retrospectively compares diagnostic performance of 1.5 T versus 3 T non-echo planar diffusion weighted imaging with or without additional T1 and T2 sequences in the detection of residual and/or recurrent cholesteatoma.

METHODS

Patients with clinically suspected recurrent cholesteatoma or postoperative routine survey MR who subsequently underwent surgical procedure were retrospectively included (135 patients, 164 operated ears) from a large database. Patients underwent 1.5 T (128 ears) or 3 T MRI (36 ears), with non-echo planar DWI, T1 and T2 acquisitions. Two radiologists independently reassessed the images. Definitive surgical diagnosis was used as gold standard. Sensitivity, specificity and diagnostic odds ratio were evaluated.

RESULTS

According to surgical diagnosis a cholesteatoma was present in 124 of 164 ears, corresponding with a prevalence of 75%. Sensitivity and specificity were lower for 3 T compared to 1.5 T, irrespective of whether additional T1 and T2-weighted sequences were used or not. Diagnostic odds ratios were higher for 1.5 T (34 and 12 for reader 1 and 2, respectively) compared to 3 T (3 and 4 for reader 1 and 2, respectively). Adding T1 and T2 sequences lowers sensitivity but increases specificity.

CONCLUSION

Non-epi DWI for the detection of residual/recurrent cholesteatoma is preferably performed on 1.5 T scanners over 3 T. The use of additional sequences regarding detection of cholesteatoma is debatable as it lowers sensitivity but increases specificity. However, these sequences may also be of use in diagnosing complications and planning surgical procedures in some hospitals.

Value of Contrast-Enhanced Ultrasound in the Diagnosis of Renal Cancer and in Comparison With Contrast-Enhanced Computed Tomography: A Meta-analysis

OBJECTIVES

The purpose of this meta-analysis was to assess the diagnostic accuracy of contrast-enhanced ultrasound (CEUS) in renal cancer and to compare it with contrast-enhanced computed tomography (CECT).

METHODS

A systematic search was performed to recruit eligible original studies published until December 2017. Two reviewers independently extracted data. A meta-analysis was performed, and the pooled sensitivity, specificity, positive and negative likelihood ratios, positive and negative predictive values, diagnostic odds ratio, summary receiver operating characteristic curve, and area under the curve were calculated. The extent and potential sources of heterogeneity were further explored. Moreover, a head-to-head comparison was also performed to compare the diagnostic value between CEUS and CECT in renal cancer.

RESULTS

Twenty-two studies were included in this meta-analysis. The summary sensitivity and specificity of CEUS for detecting renal cancer were 0.96 (95% confidence interval [CI], 0.94-0.97) and 0.82 (95% CI, 0.74-0.88), respectively. The summary diagnostic odds ratio was 102.04 (95% CI, 49.55-210.13). The area under the summary receiver operating characteristic curve was 0.97 (95% CI, 0.95-0.98). In the head-to-head comparison, CEUS showed higher diagnostic sensitivity than CECT (0.94 versus 0.85) for renal cancer, whereas the specificities were comparable between CEUS and CECT (0.77 versus 0.75).

CONCLUSIONS

Contrast-enhanced US has high sensitivity and moderate specificity in the differential diagnosis of renal cancer. The diagnostic sensitivity of CEUS in renal cancer was higher than that of CECT, suggesting that CEUS could be used as a preferred diagnostic tool for renal cancer.

Different evidence summaries have implications for contextualizing findings of meta-analysis of diagnostic tests

OBJECTIVE

To evaluate diagnostic tests, analysts use meta-analyses to provide inputs to parameters in decision models. Choosing parameter estimands from meta-analyses requires understanding the meta-analytic and decision-making contexts.

STUDY DESIGN AND SETTING

We expand on an analysis comparing positron emission tomography (PET), PET with computed tomography (PET/CT), and conventional workup (CW) in women with suspected recurrent breast cancer. We discuss Bayesian meta-analytic summaries (posterior mean over a set of existing studies, posterior estimate in an existing study, posterior predictive mean in a new study) used to estimate diagnostic test parameters (prevalence, sensitivity, specificity) needed to calculate quality-adjusted life years in a decision model contextualizing PET, PET/CT, and CW.

RESULTS

The mean and predictive mean give similar estimates, but the latter displays greater uncertainty. Namely, PET/CT outperforms CW on average but may not do better than CW when implemented in future settings.

CONCLUSION

Selecting estimands for decision model parameters from meta-analyses requires understanding the relationship between decision settings and meta-analysis studies' settings, specifically whether the former resemble one or all study settings or represents new settings. We provide an algorithm recommending appropriate estimands as input parameters in decision models for diagnostic tests to obtain output parameters consistent with the decision context.

Statistical Considerations for Bias and Protocol Deviation in Medical Device Pivotal Clinical Study

BACKGROUND

The gold standard in conducting clinical trials/studies is to follow what is prespecified in the study protocol. However, deviations from the study protocol may occur. This article discusses the issues of protocol deviation in pivotal clinical trials or studies for medical device and provides statistical approaches to mitigating bias such as selection bias specifically for diagnostic test clinical trials or studies.

METHOD

Bias correction methods are developed for 2 specific types of selection biases, prescreening bias and verification bias. Statistical approaches are discussed on how to estimate device performance adjusted for enrollment enrichment and discrepant testing results. We use an FDA-approved Roche Cobas Human Papillomavirus (HPV) test for detecting high-grade cervical disease (>CIN2) as an example to illustrate how to correct for verification bias. A recently FDA-cleared Microarray Assay in detecting copy number variation is used to illustrate how to properly estimate sensitivity and specificity for the discrepancy analysis.

RESULTS

The unadjusted sensitivity and specificity based on verified samples were 83.2% and 60.4% for the Roche's HPV test. However, using the correction method with the missing-at-random assumption, the verification bias-adjusted sensitivity and specificity were 34.5% and 93.6%, respectively.

CONCLUSION

Protocol deviations can lead to biased estimates of device clinical performance if not handled appropriately. Statistical methods correcting for bias and protocol deviations are recommended in estimating device performance.

Design, statistical analysis and reporting standards for test accuracy studies for infectious diseases in animals: Progress, challenges and recommendations

The quality of diagnostic accuracy studies (DAS) for infectious diseases of animals has improved over the last 20 years because of international educational efforts, use of design and reporting standards to guide researchers and test developers, and acceptance of the use of latent class models to account for imperfect reference tests. In this review, we focus on measurement of diagnostic sensitivity and specificity as a measure of clinical validity, describe the leadership role of the World Organisation of Animal Health (OIE) in setting standards for test validation in the context of fitness-for-purpose, and describe how design and reporting quality have facilitated the increased use of systematic reviews and meta-analysis of DAS. Ongoing challenges for design, conduct, analysis and reporting of DAS are identified; and we make recommendations for improvements in these areas for OIE-listed and non-listed infectious diseases.

Performance and usability evaluation of the INSTI HIV self-test in Kenya for qualitative detection of antibodies to HIV

BACKGROUND

HIV testing is often undermined by lack of confidentiality, stigma, shortage of counselors and long distances to testing centers. Self-testing has the potential to circumvent these constraints.

OBJECTIVE

To determine the performance and usability characteristics of the INSTI® HIV-1/HIV-2 Self-Test.

METHODS

The performance evaluation was a cross sectional study and the usability a mixed methods study. For method comparison, Bioelisa HIV-1+2 Ag/Ab test was used as the reference test. When the test results were discrepant, results from Alere Determine™ HIV-1/2 and First Response HIV-1-2 Antibody tests were used for confirmation of status.

RESULTS

Sensitivity of the INSTI HIV Self-Test was 98.99% (95% CI 96.05-99.75%), and specificity 98.15% (95% CI 95.63-99.23%). The concordance was therefore 97.27%. A total of 354 participants took part in the usability study. Of those, 343 (98.00%) found instructions for use easy to follow, 330 (94.29%) found the finger prick device easy to use, 303 (86.57%) were confident while performing the test, 342 (97.71%) felt result interpretation was easy, while 304 (86.86%) declared results within the recommended five minutes. Three hundred and forty two (342, 97.71%) were willing to use the test again while 344 (98.29%) would recommend the kit to a sexual partner. None of the 350 participants quit the process at any stage. Three hundred and eighteen (318, 91.12%) participants felt the test needed no further improvement. All 91 lay users correctly identified cartridges that showed positive, negative and invalid results. Only 31 (34.07%) participants correctly identified weak positive dummy test results.

CONCLUSION

The excellent performance and usability characteristics of INSTI HIV-1/HIV-2 self-test make the kit a viable option for HIV self-testing. To improve the identification of weak positive results, the manufacturer should indicate on the IFU that even a faint test spot should be interpreted as positive.

Application of Bayes' Theorem in Valuating Depression Tests Performance

The validity of clinical diagnoses is a fundamental topic in clinical psychology, because now there are some political administrations, as the IOM or the U.K. government, which are focusing on best evidence-based practice in clinical psychology. The most problematic issue in clinical psychology is to avoid wrong diagnoses which can have negative consequences on individual life and on the utility of clinical treatments. In the case of diagnoses based on self-report tests, the diagnostic decision about individual health is based on the comparison between its score and the cutoff, according to the frequentist approach to probability. However, the frequentist approach underestimates the possible risks of incorrect diagnoses based on cutoffs only. The Bayesian approach is a valid alternative to make diagnoses on the basis of the scores from psychological tests. The Bayes' theorem estimates the posterior probability of the presence of a pathology on the basis of the knowledge about the diffusion of this pathology (prior probability) and of the knowledge of sensitivity and specificity values of the test. With all this information, it is possible to estimate the diagnostic accuracy of some self-report tests used for assessing depression. We analyzed the diagnostic accuracy of the most used psychological tests of depression (Zung's Self-Rating Depression Scale, Hamilton Rating Scale for Depression, Center for Epidemiological Studies for Depression and the Beck Depression Inventory), together with a new scale (Teate Depression Inventory) developed with the IRT procedure, by analyzing the published works in which data about sensitivity and specificity of these scales are reported. Except the TDI, none of these scales can reach a satisfactory level of diagnostic accuracy, probably for the absence of an optimal procedure to select test items and subjects with clearly defined pathological symptoms which could allow the reduction of false positives in test scoring.

The role of sciatic nerve block to complement femoral nerve block in total knee arthroplasty: a meta-analysis of randomized controlled trials

INTRODUCTION

Femoral nerve block (FNB) appears to have higher postoperative analgesic benefits compared with the patient-controlled analgesia (PCA) in total knee arthroplasty (TKA). However, the role of sciatic nerve block (SNB) as a complement to FNB remains controversial. We performed a meta-analysis assessing the benefits of the SNB as a complement to FNB, as well as comparing the efficacy of single-injection versus continuous SNB in TKA.

METHODS

Our group conducted a systematic literature search in PubMed, EMBASE and Google Scholar. We retrieved randomized trials comparing either SNB versus placebo or continuous versus single-injection SNB. The intervention group was the use of SNB as a complement to FNB, while the control group was FNB alone. Pain score at rest and movement (at 4, 12, 24, 48 and 72 h), patient-controlled intravenous opioid consumption, length of hospital stay, and incidence of nausea were extracted from each study. Random-effects model was used for meta-analysis and standardized mean difference (SMD) was used as the effect size.

RESULTS

Ten articles comprising 514 patients were included to compare the effects of SNB combined with FNB versus FNB alone. Interventional group was found to significantly reduce pain score at for 4 h (SMD = - 0.94, 95% CI - 1.42 to - 0.47, P < 0.001, I² = 76.5%) compared with the control group. Pain score at rest was significantly reduced at movement for 12 h (SMD = - 0.29, 95% CI - 0.54 to - 0.04, P = 0.02, I² = 0%). Opioid consumption was significantly reduced at 24 (SMD = - 0.60, 95% CI - 1.01 to - 0.17, P = 0.01, I² = 69.1%) and 48 h (SMD = - 1.04, 95% CI - 1.46 to - 0.61, P < 0.001, I² = 43.4%) after TKA using SNB as a complement to FNB. Three articles were additionally meta-analyzed to compare the efficacy of single-injection (n = 79) versus continuous SNB (n = 79), being the latter one significantly associated with less pain score at 24 (SMD = -0.77, 95% CI - 1.10 to - 0.45, P < 0.001, I² = 0%) and 48 h (SMD = - 0.69, 95% CI - 1.01 to - 0.36, P < 0.001, I² = 0%), but not at 12 h (SMD = - 0.34, 95% CI - 0.73 to - 0.06, P = 0.10, I² = 0%).

CONCLUSIONS

This meta-analysis provides evidence-based supports to the benefits of SNB as a complement to FNB in TKA. The combination sciatic-femoral nerve block appears to be the optimal choice for patients in high risk of postoperative opioids consumption or acute pain after TKA.

The Reproducibility of Changes in Diagnostic Figures of Merit Across Laboratory and Clinical Imaging Reader Studies

RATIONALE AND OBJECTIVES

In this paper we examine which comparisons of reading performance between diagnostic imaging systems made in controlled retrospective laboratory studies may be representative of what we observe in later clinical studies. The change in a meaningful diagnostic figure of merit between two diagnostic modalities should be qualitatively or quantitatively comparable across all kinds of studies.

MATERIALS AND METHODS

In this meta-study we examine the reproducibility of relative measures of sensitivity, false positive fraction (FPF), area under the receiver operating characteristic (ROC) curve, and expected utility across laboratory and observational clinical studies for several different breast imaging modalities, including screen film mammography, digital mammography, breast tomosynthesis, and ultrasound.

RESULTS

Across studies of all types, the changes in the FPFs yielded very small probabilities of having a common mean value. The probabilities of relative sensitivity being the same across ultrasound and tomosynthesis studies were low. No evidence was found for different mean values of relative area under the ROC curve or relative expected utility within any of the study sets.

CONCLUSION

The comparison demonstrates that the ratios of areas under the ROC curve and expected utilities are reproducible across laboratory and clinical studies, whereas sensitivity and FPF are not.

Accuracy of Physical Examination, Ankle-Brachial Index, and Ultrasonography in the Diagnosis of Arterial Injury in Patients With Penetrating Extremity Trauma: A Systematic Review and Meta-analysis

BACKGROUND

Penetrating Extremity Trauma (PET) may result in arterial injury, a rare but limb- and life-threatening surgical emergency. Timely, accurate diagnosis is essential for potential intervention in order to prevent significant morbidity.

OBJECTIVES

Using a systematic review/meta-analytic approach, we determined the utility of physical examination, Ankle-Brachial Index (ABI), and Ultrasonography (US) in the diagnosis of arterial injury in emergency department (ED) patients who have sustained PET. We applied a test-treatment threshold model to determine which evaluations may obviate CT Angiography (CTA).

METHODS

We searched PubMed, Embase, and Scopus from inception to November 2016 for studies of ED patients with PET. We included studies on adult and pediatric subjects. We defined the reference standard to include CTA, catheter angiography, or surgical exploration. When low-risk patients did not undergo the reference standard, trials must have specified that patients were observed for at least 24 hours. We used the Quality Assessment Tool for Diagnostic Accuracy Studies (QUADAS-2) to evaluate bias and applicability of the included studies. We calculated positive and negative likelihood ratios (LR+ and LR-) of physical examination ("hard signs" of vascular injury), US, and ABI. Using established CTA test characteristics (sensitivity = 96.2%, specificity = 99.2%) and applying the Pauker-Kassirer method, we developed a test-treatment threshold model (testing threshold = 0.14%, treatment threshold = 72.9%).

RESULTS

We included eight studies (n = 2,161, arterial injury prevalence = 15.5%). Studies had variable quality with most at high risk for partial and double verification bias. Some studies investigated multiple index tests: physical examination (hard signs) in three studies (n = 1,170), ABI in five studies (n = 1,040), and US in four studies (n = 173). Due to high heterogeneity (I²  > 75%) of the results, we could not calculate LR+ or LR- for hard signs or LR+ for ABI. The weighted prevalence of arterial injury for ABI was 14.3% and LR- was 0.59 (95% confidence interval [CI] = 0.48-0.71) resulting in a posttest probability of 9% for arterial injury. Ultrasonography had weighted prevalence of 18.9%, LR+ of 35.4 (95% CI = 8.3-151), and LR- of 0.24 (95% CI = 0.08-0.72); posttest probabilities for arterial injury were 89% and 5% after positive or negative US, respectively. The posttest probability of arterial injury with positive US (89%) exceeded the CTA treatment threshold (72.9%). The posttest probabilities of arterial injury with negative US (5%) and normal ABI (9%) exceeded the CTA testing threshold (0.14%). Normal examination (no hard or soft signs) with normal ABI in combination had LR- of 0.01 (95% CI = 0.0-0.10) resulting in an arterial injury posttest probability of 0%.

CONCLUSIONS

In PET patients, positive US may obviate CTA. In patients with a normal examination (no hard or soft signs) and a normal ABI, arterial injury can be ruled out. However, a normal ABI or negative US cannot independently exclude arterial injury. Due to high study heterogeneity, we cannot make recommendations when hard signs are present or absent or when ABI is abnormal. In these situations, one should use clinical judgment to determine the need for further observation, CTA or catheter angiography, or surgical exploration.

Assessment of nasalance and nasality in patients with a repaired cleft palate

In patients with a repaired cleft palate, nasality is typically diagnosed by speech language pathologists. In addition, there are various instruments to objectively diagnose nasalance. To explore the potential of nasalance measurements after cleft palate repair by NasalView®, we correlated perceptual nasality and instrumentally measured nasalance of eight speech items and determined the relationship between sensitivity and specificity of the nasalance measures by receiver-operating characteristics (ROC) analyses and AUC (area under the curve) computation for each single test item and specific item groups. We recruited patients with a primarily repaired cleft palate receiving speech therapy during follow-up. During a single day visit, perceptive and instrumental assessments were obtained in 36 patients and analyzed. The individual perceptual nasality was assigned to one of four categories; the corresponding instrumental nasalance measures for the eight specific speech items were expressed on a metric scale (1-100). With reference to the perceptual diagnoses, we observed 3 nasal and one oral test item with high sensitivity. However, the specificity of the nasality indicating measures was rather low. The four best speech items with the highest sensitivity provided scores ranging from 96.43 to 100%, while the averaged sensitivity of all eight items was below 90%. We conclude that perceptive evaluation of nasality remains state of the art. For clinical follow-up, instrumental nasalance assessment can objectively document subtle changes by analysis of four speech items only. Further studies are warranted to determine the applicability of instrumental nasalance measures in the clinical routine, using discriminative items only.

Systematic review: Can non-mydriatic cameras accurately detect diabetic retinopathy?

AIMS

Diabetes puts many individuals at risk for developing diabetic retinopathy (DR). Non-mydriatic (NM) retinal photography without pharmacological pupil dilation can be used to detect DR, however, its value in a general clinical setting has not been established. The objective of this review is to evaluate the validity of NM retinal photos, as compared to the reference standard of seven standard stereoscopic 30 degree field photographs (7SF), for their use of detecting DR in community-dwelling adults.

METHODS

English articles were identified through MEDLINE (1992-2016), EMBASE (1992-2016), Center for Disease Control and Prevention, Food and Drug Administration, Health Canada, and Google Scholar (1992-2016). Only studies that examined the validity of NM retinal photos, without pharmacological pupil dilation, as compared to 7SF, used in a community-dwelling adult population were included. Risk of bias was assessed using the QUADAS-2.

RESULTS

Of the 368 articles identified in the search, 6 articles met eligibility requirements. Sensitivities ranged from 64 to 97.9% and specificities ranged from 65.6 to 98%. Studies that used multiple field images to detect DR in a younger population generally produced higher quality photos and more valid results.

CONCLUSIONS

The strongest validity indicators were reported in studies capturing multiple field images.

STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration

Diagnostic accuracy studies are, like other clinical studies, at risk of bias due to shortcomings in design and conduct, and the results of a diagnostic accuracy study may not apply to other patient groups and settings. Readers of study reports need to be informed about study design and conduct, in sufficient detail to judge the trustworthiness and applicability of the study findings. The STARD statement (Standards for Reporting of Diagnostic Accuracy Studies) was developed to improve the completeness and transparency of reports of diagnostic accuracy studies. STARD contains a list of essential items that can be used as a checklist, by authors, reviewers and other readers, to ensure that a report of a diagnostic accuracy study contains the necessary information. STARD was recently updated. All updated STARD materials, including the checklist, are available at http://www.equator-network.org/reporting-guidelines/stard Here, we present the STARD 2015 explanation and elaboration document. Through commented examples of appropriate reporting, we clarify the rationale for each of the 30 items on the STARD 2015 checklist, and describe what is expected from authors in developing sufficiently informative study reports.

Statistical innovations in the medical device world sparked by the FDA

The world of medical devices while highly diverse is extremely innovative, and this facilitates the adoption of innovative statistical techniques. Statisticians in the Center for Devices and Radiological Health (CDRH) at the Food and Drug Administration (FDA) have provided leadership in implementing statistical innovations. The innovations discussed include: the incorporation of Bayesian methods in clinical trials, adaptive designs, the use and development of propensity score methodology in the design and analysis of non-randomized observational studies, the use of tipping-point analysis for missing data, techniques for diagnostic test evaluation, bridging studies for companion diagnostic tests, quantitative benefit-risk decisions, and patient preference studies.

Clinical Performance Evaluation of Molecular Diagnostic Tests

Molecular diagnostic tests with application to clinical diagnostics involve studies in infectious diseases, inherited diseases, oncology, predisposition to disease, or the description of polymorphisms linked to disease states. General considerations in the design of evaluation of diagnostic test trials and statistical principles for reporting the results are discussed. A brief overview of the general statistical considerations related to the intent of use, test development versus validation, different types of biases, and issues with missing data are provided. Furthermore, issues related to commonly used but not necessarily correct methods to characterize the performance in the presence and absence of a clinical reference standard are discussed. These issues are broadly applicable to any molecular diagnostic test with a dichotomous result. This overview may help the clinical molecular diagnostic community to evaluate tests that provide a dichotomous result.

Nearest-Neighbor Estimation for ROC Analysis under Verification Bias

For a continuous-scale diagnostic test, the receiver operating characteristic (ROC) curve is a popular tool for displaying the ability of the test to discriminate between healthy and diseased subjects. In some studies, verification of the true disease status is performed only for a subset of subjects, possibly depending on the test result and other characteristics of the subjects. Estimators of the ROC curve based only on this subset of subjects are typically biased; this is known as verification bias. Methods have been proposed to correct verification bias, in particular under the assumption that the true disease status, if missing, is missing at random (MAR). MAR assumption means that the probability of missingness depends on the true disease status only through the test result and observed covariate information. However, the existing methods require parametric models for the (conditional) probability of disease and/or the (conditional) probability of verification, and hence are subject to model misspecification: a wrong specification of such parametric models can affect the behavior of the estimators, which can be inconsistent. To avoid misspecification problems, in this paper we propose a fully nonparametric method for the estimation of the ROC curve of a continuous test under verification bias. The method is based on nearest-neighbor imputation and adopts generic smooth regression models for both the probability that a subject is diseased and the probability that it is verified. Simulation experiments and an illustrative example show the usefulness of the new method. Variance estimation is also discussed.

Taenia solium taeniosis/cysticercosis and the co-distribution with schistosomiasis in Africa

Background

This study aimed to map the distribution of Taenia solium taeniosis/cysticercosis and the co-distribution with schistosomiasis in Africa. These two major neglected tropical diseases are presumed to be widely distributed in Africa, but currently the level of co-distribution is unclear.

Methods

A literature search on T. solium taeniosis/cysticercosis was performed to compile all known studies on the presence of T. solium and apparent prevalence of taeniosis and porcine cysticercosis in Africa. Studies were geo-referenced using an online gazetteer. A Bayesian framework was used to combine the epidemiological data on the apparent prevalence with external information on test characteristics to estimate informed district-level prevalence of taeniosis and porcine cysticercosis. Districts with T. solium taeniosis/cysticercosis presence were cross-referenced with the Global Neglected Tropical Diseases Database for schistosomiasis presence.

Results

The search strategies identified 141 reports of T. solium in Africa from 1985 to 2014 from a total of 476 districts in 29 countries, 20 with porcine cysticercosis, 22 with human cysticercosis, and 16 with taeniosis, in addition to 2 countries identified from OIE reports. All 31 countries were considered, on national scale, to have co-distribution with schistosomiasis. Presence of both parasites was confirmed in 124 districts in 17 countries. The informed prevalence of taeniosis and porcine cysticercosis were estimated for 14 and 41 districts in 10 and 13 countries, respectively.

Conclusions

With the paucity of data, T. solium infection is grossly under-reported and expected to be more widespread than this study suggests. In areas where co-distribution occurs there is a need for increased emphasis on evaluation of integrated intervention approaches for these two helminth infections and allocation of resources for evaluating the extent of adverse effects caused by mass drug administration.

Electronic supplementary material

The online version of this article (doi:10.1186/s13071-015-0938-7) contains supplementary material, which is available to authorized users.

An introduction to systematic reviews in animal health, animal welfare, and food safety

In this paper, we provide an introduction to systematic reviews and discuss the process for conducting systematic reviews in animal health, animal welfare, and food safety. The research synthesis need that can be addressed by a systematic review is discussed. The use of systematic reviews to address questions about intervention effects, etiology, diagnostic tests evaluation and disease burden are discussed. The steps included in a systematic review are described.

Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration

The TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) Statement includes a 22-item checklist, which aims to improve the reporting of studies developing, validating, or updating a prediction model, whether for diagnostic or prognostic purposes. The TRIPOD Statement aims to improve the transparency of the reporting of a prediction model study regardless of the study methods used. This explanation and elaboration document describes the rationale; clarifies the meaning of each item; and discusses why transparent reporting is important, with a view to assessing risk of bias and clinical usefulness of the prediction model. Each checklist item of the TRIPOD Statement is explained in detail and accompanied by published examples of good reporting. The document also provides a valuable reference of issues to consider when designing, conducting, and analyzing prediction model studies. To aid the editorial process and help peer reviewers and, ultimately, readers and systematic reviewers of prediction model studies, it is recommended that authors include a completed checklist in their submission. The TRIPOD checklist can also be downloaded from www.tripod-statement.org.

Verification and classification bias interactions in diagnostic test accuracy studies for fine-needle aspiration biopsy

BACKGROUND

Reliable estimates of accuracy are important for any diagnostic test. Diagnostic accuracy studies are subject to unique sources of bias. Verification bias and classification bias are 2 sources of bias that commonly occur in diagnostic accuracy studies. Statistical methods are available to estimate the impact of these sources of bias when they occur alone. The impact of interactions when these types of bias occur together has not been investigated.

METHODS

We developed mathematical relationships to show the combined effect of verification bias and classification bias. A wide range of case scenarios were generated to assess the impact of bias components and interactions on total bias.

RESULTS

Interactions between verification bias and classification bias caused overestimation of sensitivity and underestimation of specificity. Interactions had more effect on sensitivity than specificity. Sensitivity was overestimated by at least 7% in approximately 6% of the tested scenarios. Specificity was underestimated by at least 7% in less than 0.1% of the scenarios.

CONCLUSIONS

Interactions between verification bias and classification bias create distortions in accuracy estimates that are greater than would be predicted from each source of bias acting independently.

Opportunities and challenges of proteomics in pediatric patients: circulating biomarkers after hematopoietic stem cell transplantation as a successful example

Biomarkers have the potential to improve diagnosis and prognosis, facilitate-targeted treatment, and reduce health care costs. Thus, there is great hope that biomarkers will be integrated in all clinical decisions in the near future. A decade ago, the biomarker field was launched with great enthusiasm because MS revealed that blood contains a rich library of candidate biomarkers. However, biomarker research has not yet delivered on its promise due to several limitations: (i) improper sample handling and tracking as well as limited sample availability in the pediatric population, (ii) omission of appropriate controls in original study designs, (iii) lability and low abundance of interesting biomarkers in blood, and (iv) the inability to mechanistically tie biomarker presence to disease biology. These limitations as well as successful strategies to overcome them are discussed in this review. Several advances in biomarker discovery and validation have been made in hematopoietic stem cell transplantation, the current most effective tumor immunotherapy, and these could serve as examples for other conditions. This review provides fresh optimism that biomarkers clinically relevant in pediatrics are closer to being realized based on: (i) a uniform protocol for low-volume blood collection and preservation, (ii) inclusion of well-controlled independent cohorts, (iii) novel technologies and instrumentation with low analytical sensitivity, and (iv) integrated animal models for exploring potential biomarkers and targeted therapies.

Clinical practice guidelines for the management of hypothyroidism

INTRODUCTION

Hypothyroidism has long been known for its effects on different organ systems, leading to hypometabolism. However, subclinical hypothyroidism, its most prevalent form, has been recently related to cardiovascular risk and also to maternal-fetal complications in pregnant women.

OBJECTIVES

In these clinical practice guidelines, several aspects of this field have been discussed with the clear objectives of helping physicians treat patients with hypothyroidism, and of sharing some of our Latin American-based clinical experience.

MATERIALS AND METHODS

The Latin American Thyroid Society commissioned a Task Force on Hypothyroidism to develop evidence-based clinical guidelines on hypothyroidism. A systematic review of the available literature, focused on the primary databases of MedLine/PubMed and Lilacs/SciELO was performed. Filters to assess methodological quality were applied to select the best quality studies. The strength of recommendation on a scale from A-D was based on the Oxford Centre for Evidence--based Medicine, Levels of Evidence 2009, allowing an unbiased opinion devoid of subjective viewpoints. The areas of interest for the studies comprised diagnosis, screening, treatment and a special section for hypothyroidism in pregnancy.

RESULTS

Several questions based on diagnosis, screening, treatment of hypothyroidism in adult population and specifically in pregnant women were posed. Twenty six recommendations were created based on the answers to these questions. Despite the fact that evidence in some areas of hypothyroidism, such as therapy, is lacking, out of 279 references, 73% were Grade A and B, 8% Grade C and 19% Grade D.

CONCLUSIONS

These evidence-based clinical guidelines on hypothyroidism will provide unified criteria for management of hypothyroidism throughout Latin America. Although most of the studies referred to are from all over the world, the point of view of thyroidologists from Latin America is also given.

Pediatric Pleural Empyema in the Province of Quebec: Analysis of a 10-Fold Increase Between 1990 and 2007

BACKGROUND

Although the frequency of pneumonia has decreased over time, an increase in pleural empyema has been observed in different settings worldwide. This study assessed the epidemiology of community-acquired pediatric pleural empyema in the province of Quebec through validation of cases found in a hospitalization discharge database.

METHODS

We used the national administrative database of hospitalization to identify children (6 months-14 years) hospitalized for pleural empyema or pleural effusion with drainage from January 1990 until December 2007 and reviewed their medical charts. Patients with pleural effusion secondary to chest trauma, thoracic surgery, malignancies, cardiac failure, or metabolic disorders were excluded.

RESULTS

Predictive positive value (PPV) of empyema code in any position among discharge diagnostics in the administrative database was 86.5% (95% confidence interval: 81.9%-90.3%). After chart revision, 292 met the inclusion criteria. Age-adjusted incidence of pleural empyema in the pediatric population increased from 0.23 in 1990 to 4.01/100,000 person-years in 2007. A bacterial pathogen was identified in 46.5%; Streptococcus pneumoniae (Sp) (42%) and S pyogenes (30%) were most frequent. There was no obvious change in the PPV and proportions of children with chronic disease or asthma and in identified pathogens over time, but an increase in pre-admission respiratory symptoms duration (from 3.8 days to 5.7) and nonsteroidal anti-inflammatory drug use (from 0% to 19%) was observed.

CONCLUSIONS

From 1990 to 2007, we observed a 10-fold increase in the incidence of pediatric hospitalizations associated with pleural empyema. This increase preceded the introduction of a pneumococcal conjugated vaccine program in Quebec. Sp remained the major pathogen identified.

The accuracy of thyroid nodule ultrasound to predict thyroid cancer: systematic review and meta-analysis

CONTEXT

Significant uncertainty remains surrounding the diagnostic accuracy of sonographic features used to predict the malignant potential of thyroid nodules.

OBJECTIVE

The objective of the study was to summarize the available literature related to the accuracy of thyroid nodule ultrasound (US) in the prediction of thyroid cancer.

METHODS

We searched multiple databases and reference lists for cohort studies that enrolled adults with thyroid nodules with reported diagnostic measures of sonography. A total of 14 relevant US features were analyzed.

RESULTS

We included 31 studies between 1985 and 2012 (number of nodules studied 18,288; average size 15 mm). The frequency of thyroid cancer was 20%. The most common type of cancer was papillary thyroid cancer (84%). The US nodule features with the highest diagnostic odds ratio for malignancy was being taller than wider [11.14 (95% confidence interval 6.6-18.9)]. Conversely, the US nodule features with the highest diagnostic odds ratio for benign nodules was spongiform appearance [12 (95% confidence interval 0.61-234.3)]. Heterogeneity across studies was substantial. Estimates of accuracy depended on the experience of the physician interpreting the US, the type of cancer and nodule (indeterminate), and type of reference standard. In a threshold model, spongiform appearance and cystic nodules were the only two features that, if present, could have avoided the use of fine-needle aspiration biopsy.

CONCLUSIONS

Low- to moderate-quality evidence suggests that individual ultrasound features are not accurate predictors of thyroid cancer. Two features, cystic content and spongiform appearance, however, might predict benign nodules, but this has limited applicability to clinical practice due to their infrequent occurrence.

Estimates of sensitivity and specificity of electric pulp testing depend on pulp disease spectrum: a modelling study

AIM

To demonstrate how the spectrum of diseased pulps may influence sensitivity and specificity in diagnostic studies on pulp status.

METHODOLOGY

An original sample from a previous study consisting of 59 teeth scheduled for root canal treatment was used where the relationship between the response to electric pulp testing and the visual status of the pulp was evaluated. To alter the spectrum of diseased pulps, a hypothetical sample of asymptomatic teeth with deep caries lesions was added to the original sample. Sensitivity and specificity were then compared for the two samples.

RESULTS

In the original sample of 59 teeth, sensitivity was 72% and specificity 90%. When the spectrum of diseased pulps was altered, sensitivity decreased to 67% and specificity increased to 97%. The change in disease spectrum also decreased the prevalence of necrotic pulps.

CONCLUSIONS

The spectrum of diseased pulps included in a diagnostic study on the accuracy of electric pulp testing, and indirectly also disease prevalence (here pulp necrosis), influences estimates of sensitivity and specificity. This implies that estimates of diagnostic accuracy from one study with a particular tooth population spectrum may not apply to another tooth population with a different disease spectrum.