Partial verification bias and incorporation bias affected accuracy estimates of diagnostic studies for biomarkers that were part of an existing composite gold standard

The screening performance of glucose challenge test for gestational diabetes in twin pregnancies: a systematic review and meta-analysis

BACKGROUND

The screening accuracy of the 50 g-glucose challenge test (50 g-GCT) for gestational diabetes (GDM) has been described in singleton pregnancies. Given the physiologic differences and greater increase in insulin resistance in twin compared with singleton pregnancies, the performance of the 50 g-GCT in twin pregnancies may differ.

OBJECTIVES

To perform a systematic review on the screening performance of the 50 g-GCT for gestational diabetes in twin pregnancies.

DATA SOURCES

Ovid Medline, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL).

STUDY ELIGIBILITY CRITERIA, PARTICIPANTS, AND INTERVENTIONS

We included randomized controlled trials or cohort studies that evaluated the screening accuracy of the 50 g-GCT for GDM in twin pregnancies using the two-step approach. The primary outcome was the positive predictive value of the 50 g-GCT for GDM using the 140 mg/dL (7.8 mmol/L) threshold.

STUDY APPRAISAL AND SYNTHESIS METHODS

Methodological quality of included studies was assessed using the QUADAS-2 tool. The positive predictive value (PPV) was pooled for studies that used similar test characteristics.

RESULTS

From 2044 citations, 7 retrospective cohort studies with a total of 55,597 participants were included (6.5% twins and 93.5% singletons). The majority of studies evaluated a 50-g GCT cutoff point of 140 mg/dL. The pooled PPV for a threshold of 140 mg/dL (7.8 mmol/L) for twins was 22.58% (95% CI: 0.1912-0.2647, I²=34.1%). The 50-g GCT screen positive rate in twin pregnancies was higher than that in singleton pregnancies. None of the studies performed routine OGTT.

CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS

The PPV of 50 g-GCT for GDM in twin pregnancies when using a threshold of 140 mg/dL (7.8 mmol/L) is approximately 23%. There is currently no data on the sensitivity and specificity of the 50 g-GCT in twins.

Biomarkers and diagnostic guidelines for sporadic Creutzfeldt-Jakob disease

Sporadic Creutzfeldt-Jakob disease is a fatal neurodegenerative disease caused by misfolded prion proteins (PrP^Sc). Effective therapeutics are currently not available and accurate diagnosis can be challenging. Clinical diagnostic criteria use a combination of characteristic neuropsychiatric symptoms, CSF proteins 14-3-3, MRI, and EEG. Supportive biomarkers, such as high CSF total tau, could aid the diagnostic process. However, discordant studies have led to controversies about the clinical value of some established surrogate biomarkers. Development and clinical application of disease-specific protein aggregation and amplification assays, such as real-time quaking induced conversion (RT-QuIC), have constituted major breakthroughs for the confident pre-mortem diagnosis of sporadic Creutzfeldt-Jakob disease. Updated criteria for the diagnosis of sporadic Creutzfeldt-Jakob disease, including application of RT-QuIC, should improve early clinical confirmation, surveillance, assessment of PrP^Sc seeding activity in different tissues, and trial monitoring. Moreover, emerging blood-based, prognostic, and potentially pre-symptomatic biomarker candidates are under investigation.

Development and Validation of a Multidimensional Frailty Scale for Clinical Geriatric Assessment

OBJECTIVES

This study aims to develop a multidimensional frailty assessment tool including physical, psychological, and social frailty and to examine its psychometric property in a clinical setting.

METHODS

Through a literature review and gathered expert perspectives, the Korean Frailty Scale (KFS, 0-6 points) was developed with six questions about weight loss, self-assessment of health status, energy loss, fatigue, social network, and support. The psychometric property was then validated in community-dwelling older adults using the Korean Frailty and Aging Cohort Survey (2016-2019). Receiver operator characteristic (ROC) analyses determined optimal cutoff scores for frail, pre-frail, and robust. The predictive validity for mortality, hospitalization, fall events, and disability, such as declines in activities of daily living (ADL) and instrumental activities of daily living (IADL) was assessed after adjusting for age, sex, education level, household income, body-mass index, smoking status, alcohol consumption, physical activity, marital status, current employment, and residence.

RESULTS

The 2,923 participants (mean age: 76.0±3.9 years, female: 52.2%) were categorized into frail (≥3 points, 21.1%), pre-frail (1-2 points, 50.2%) and robust (0 points, 28.7%) groups. Sensitivity (88.3%), specificity (91.6%), and the area under the ROC curve (0.730) for the frail group were sufficient to be accepted as a new frailty scale. The frail group had increased risk of falling (relative risk [RR]: 1.810, 95% confidence interval: 1.354-2.420), mortality (hazard ratio, 6.596, 1.802-24.137), and IADL decline (RR, 1.638, 1.231-2.178).

CONCLUSION

The KFS is a new multidimensional frailty scale for clinical geriatric assessment that was confirmed as effective for predicting adverse health outcomes in Korean community-dwelling older adults.

The Added Value of Radiographs in Diagnosing Knee Osteoarthritis Is Similar for General Practitioners and Secondary Care Physicians; Data from the CHECK Early Osteoarthritis Cohort

Objective: The purpose of this study was to evaluate the added value of radiographs for diagnosing knee osteoarthritis (KOA) by general practitioners (GPs) and secondary care physicians (SPs). Methods: Seventeen GPs and nineteen SPs were recruited to evaluate 1185 knees from the CHECK cohort (presenters with knee pain in primary care) for the presence of clinically relevant osteoarthritis (OA) during follow-up. Experts were required to make diagnoses independently, first based on clinical data only and then on clinical plus radiographic data, and to provide certainty scores (ranging from 1 to 100, where 1 was “certainly no OA” and 100 was “certainly OA”). Next, experts held consensus meetings to agree on the final diagnosis. With the final diagnosis as gold standard, diagnostic indicators were calculated (sensitivity, specificity, positive/negative predictive value, accuracy and positive/negative likelihood ratio) for all knees, as well as for clinically “certain” and “uncertain” knees, respectively. Student paired t-tests compared certainty scores. Results: Most diagnoses of GPs (86%) and SPs (82%) were “consistent” after assessment of radiographic data. Diagnostic indicators improved similarly for GPs and SPs after evaluating the radiographic data, but only improved relevantly in clinically “uncertain” knees. Radiographs added some certainty to “consistent” OA knees (GP 69 vs. 72, p < 0.001; SP 70 vs. 77, p < 0.001), but not to the consistent no OA knees (GP 21 vs. 22, p = 0.16; SP 20 vs. 21, p = 0.04). Conclusions: The added value of radiographs is similar for GP and SP, in terms of diagnostic accuracy and certainty. Radiographs appear to be redundant when clinicians are certain of their clinical diagnosis.

Development of clinical decision rules for traumatic intracranial injuries in patients with mild traumatic brain injury in a developing country

Background

The majority of clinical decision rules for prediction of intracranial injury in patients with mild traumatic brain injury (TBI) were developed from high-income countries. The application of these rules in low or middle-income countries, where the primary mechanism of injury was traffic accidents, is questionable.

Methods

We developed two practical decision rules from a secondary analysis of a multicenter, prospective cohort of 1,164 patients with mild TBI who visited the emergency departments from 2013 to 2016. The clinical endpoints were the presence of any intracranial injury on CT scans and the requirement of neurosurgical interventions within seven days of onset.

Results

Thirteen predictors were included in both models, which were age ≥60 years, dangerous mechanism of injury, diffuse headache, vomiting >2 episodes, loss of consciousness, posttraumatic amnesia, posttraumatic seizure, history of anticoagulant use, presence of neurological deficits, significant wound at the scalp, signs of skull base fracture, palpable stepping at the skull, and GCS <15 at 2 hours. For the model-based score, the area under the receiver operating characteristic curve (AuROC) was 0.85 (95%CI 0.82, 0.87) for positive CT results and 0.87 (95%CI 0.83, 0.91) for requirement of neurosurgical intervention. For the clinical-based score, the AuROC for positive CT results and requirement of neurosurgical intervention was 0.82 (95%CI 0.79, 0.85) and 0.84 (95%CI 0.80, 0.88), respectively.

Conclusions

The models delivered good calibration and excellent discrimination both in the development and internal validation cohort. These rules can be used as assisting tools in risk stratification of patients with mild TBI to be sent for CT scans or admitted for clinical observation.

Diagnosing COVID-19 in the Emergency Department: A Scoping Review of Clinical Examinations, Laboratory Tests, Imaging Accuracy, and Biases

OBJECTIVE

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) emerged as a global pandemic in early 2020 with rapidly evolving approaches to diagnosing the clinical illness called coronavirus disease (COVID-19). The primary objective of this scoping review is to synthesize current research of the diagnostic accuracy of history, physical examination, routine laboratory tests, real-time reverse transcription-polymerase chain reaction (rRT-PCR), immunology tests, and computed tomography (CT) for the emergency department (ED) diagnosis of COVID-19. Secondary objectives included a synopsis of diagnostic biases likely with current COVID-19 research as well as corresponding implications of false-negative and false-positive results for clinicians and investigators.

METHODS

A Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Scoping Review (PRISMA-ScR)-adherent synthesis of COVID-19 diagnostic accuracy through May 5, 2020, was conducted. The search strategy was designed by a medical librarian and included studies indexed by PubMed and Embase since January 2020.

RESULTS

A total of 1,907 citations were screened for relevance. Patients without COVID-19 are rarely reported, so specificity and likelihood ratios were generally unavailable. Fever is the most common finding, while hyposmia and hypogeusia appear useful to rule in COVID-19. Cough is not consistently present. Lymphopenia is the mostly commonly reported laboratory abnormality and occurs in over 50% of COVID-19 patients. rRT-PCR is currently considered the COVID-19 criterion standard for most diagnostic studies, but a single test sensitivity ranges from 60% to 78%. Multiple reasons for false-negatives rRT-PCR exist, including sample site tested and disease stage during which sample was obtained. CT may increase COVID-19 sensitivity in conjunction with rRT-PCR, but guidelines for imaging patients most likely to benefit are emerging. IgM and IgG serology levels are undetectable in the first week of COVID-19, but sensitivity (range = 82% to 100%) and specificity (range = 87% to 100%) are promising. Whether detectable COVID-19 antibodies correspond to immunity remains unanswered. Current studies do not adhere to accepted diagnostic accuracy reporting standards and likely report significantly biased results if the same tests were to be applied to general ED populations with suspected COVID-19.

CONCLUSIONS

With the exception of fever and disorders of smell/taste, history and physical examination findings are unhelpful to distinguish COVID-19 from other infectious conditions that mimic SARS-CoV-2 like influenza. Routine laboratory tests are also nondiagnostic, although lymphopenia is a common finding and other abnormalities may predict severe disease. Although rRT-PCR is the current criterion standard, more inclusive consensus-based criteria will likely emerge because of the high false-negative rate of PCR tests. The role of serology and CT in ED assessments remains undefined.

The utility of endobronchial ultrasound-transbronchial needle aspiration in patients with suspected extra-pulmonary sarcoidosis without thoracic lymphadenopathy

BACKGROUND

Diagnosis of extra-pulmonary sarcoidosis can be difficult, and a biopsy is usually required. We evaluated the utility of endobronchial ultrasound-transbronchial needle aspiration (EBUS-TBNA) in patients with suspected extra-pulmonary sarcoidosis with thoracic lymph nodes ≤10 mm on chest computed tomography (CT) and no or minimal pulmonary infiltrates.

METHODS

The Cleveland Clinic bronchoscopy registry was screened. Patients with thoracic lymph nodes >10 mm on short axis or significant pulmonary infiltrates in the chest CT scan were excluded. Two separate analyses using expert consensus (before and after release of bronchoscopy results) were the reference standard.

RESULTS

15 patients met the inclusion criteria. 40% had suspected ocular, 33% cardiac and 27% neurologic sarcoidosis. Six patients (40%) had EBUS-TBNA compatible with sarcoidosis. When the reference standard was the consensus diagnosis blinded to bronchoscopy results, the sensitivity, specificity, positive predictive value and negative predictive value of EBUS-TBNA were 56%, 83%, 83%, and 56% respectively. The combination of a positive EBUS-TBNA and BAL CD4/CD8 improved the specificity from 83 to 100%, but the difference was not statistically significant (p = 0.074). When the reference standard was the consensus diagnosis with the bronchoscopic results, the sensitivity, specificity, positive predictive value and negative predictive value of EBUS-TBNA were 75%, 100%, 100%, and 78% respectively.

CONCLUSIONS

In patients with suspected extra-pulmonary sarcoidosis, the EBUS-TBNA may be useful in the diagnosis of patients with thoracic lymph nodes ≤10 mm and no or minimal pulmonary infiltrates on chest CT. Larger and prospective studies are needed to validate our findings.

Value of CT to detect radiographically occult injuries of the proximal femur in elderly patients after low-energy trauma: determination of non-inferiority margins of CT in comparison with MRI

PURPOSE

To determine the margins of non-inferiority of the sensitivity of CT and the sample size needed to test the non-inferiority of CT in comparison with MRI.

MATERIALS AND METHODS

During a 2-year period, elderly patients with suspected radiographically occult post-traumatic bone injuries were investigated by CT and MRI in two institutions. Four radiologists analyzed separately the CT and MRI examinations to detect post-traumatic femoral injuries. Their sensitivities at CT (SeCT) and MRI (SeMRI) were calculated with the reference being a best valuable comparator (consensus reading of the MRI and clinical follow-up). ROC analysis followed by an exact test (Newcombe's approach) was performed to assess the 95% confidence interval (CI) for the difference SeCT-SeMRI for each reader. A sample size calculation was performed based on our observed results by using a one-sided McNemar's test.

RESULTS

Twenty-nine out of 102 study participants had a post-traumatic femoral injury. SeCT ranged between 83 and 93% and SeMRI ranged between 97 and 100%. The 95% CIs for (SeCT-SeMRI) were [- 5.3%, + 0.8%], (p^R1 = 0.1250), [- 4.5%; + 1.2%] (p^R2 = 0.2188), [- 3.4%; + 1.1%] (p^R3 = 0.2500) to [- 3.8%; + 1.6%] (p^R4 = 0.3750) according to readers, with a lowest limit for 95% CIs superior to a non-inferiority margin of (- 6%) for all readers. A population of 440 patients should be analyzed to test the non-inferiority of CT in comparison with MRI.

CONCLUSION

CT and MRI are sensitive for the detection of radiographically occult femoral fractures in elderly patients after low-energy trauma. The choice between both these modalities is a compromise between the most available and the most sensitive technique.

KEY POINTS

• The sensitivity of four separate readers to detect radiographically occult post-traumatic femoral injuries in elderly patients after low-energy trauma ranged between 83 and 93% at CT and between 97 and 100% at MRI according to a best valuable comparator including MRI and clinical follow-up. • CT is a valuable alternative method to MRI for the detection of post-traumatic femoral injuries in elderlies after low-energy trauma if a 6% loss in sensitivity can be accepted in comparison with MRI. • The choice between CT and MRI is a compromise between the most available and the most sensitive technique.

Diagnostic test evaluation methodology: A systematic review of methods employed to evaluate diagnostic tests in the absence of gold standard - An update

OBJECTIVE

To systematically review methods developed and employed to evaluate the diagnostic accuracy of medical test when there is a missing or no gold standard.

STUDY DESIGN AND SETTINGS

Articles that proposed or applied any methods to evaluate the diagnostic accuracy of medical test(s) in the absence of gold standard were reviewed. The protocol for this review was registered in PROSPERO (CRD42018089349).

RESULTS

Identified methods were classified into four main groups: methods employed when there is a missing gold standard; correction methods (which make adjustment for an imperfect reference standard with known diagnostic accuracy measures); methods employed to evaluate a medical test using multiple imperfect reference standards; and other methods, like agreement studies, and a mixed group of alternative study designs. Fifty-one statistical methods were identified from the review that were developed to evaluate medical test(s) when the true disease status of some participants is unverified with the gold standard. Seven correction methods were identified and four methods were identified to evaluate medical test(s) using multiple imperfect reference standards. Flow-diagrams were developed to guide the selection of appropriate methods.

CONCLUSION

Various methods have been proposed to evaluate medical test(s) in the absence of a gold standard for some or all participants in a diagnostic accuracy study. These methods depend on the availability of the gold standard, its' application to the participants in the study and the availability of alternative reference standard(s). The clinical application of some of these methods, especially methods developed when there is missing gold standard is however limited. This may be due to the complexity of these methods and/or a disconnection between the fields of expertise of those who develop (e.g. mathematicians) and those who employ the methods (e.g. clinical researchers). This review aims to help close this gap with our classification and guidance tools.

Sporadic Creutzfeldt-Jakob disease

Sporadic Creutzfeldt-Jakob disease (CJD), the most common human prion disease, is generally regarded as a spontaneous neurodegenerative illness, arising either from a spontaneous PRNP somatic mutation or a stochastic PrP structural change. Alternatively, the possibility of an infection from animals or other source remains to be completely ruled out. Sporadic CJD is clinically characterized by rapidly progressive dementia with ataxia, myoclonus, or other neurologic signs and, neuropathologically, by the presence of aggregates of abnormal prion protein, spongiform change, neuronal loss, and gliosis. Despite these common features the disease shows a wide phenotypic variability which was recognized since its early descriptions. In the late 1990s the identification of key molecular determinants of phenotypic expression and the availability of a large series of neuropathologically verified cases led to the characterization of definite clinicopathologic and molecular disease subtypes and to an internationally recognized disease classification. By showing that these disease subtypes correspond to specific agent strain-host genotype combinations, recent transmission studies have confirmed the biologic basis of this classification. The introduction of brain magnetic resonance imaging techniques such as fluid-attenuated inversion recovery and diffusion-weighted imaging sequences and cerebrospinal fluid biomarker assays for the detection of brain-derived proteins as well as real-time quaking-induced conversion assay, allowing the specific detection of prions in accessible biologic fluids and tissues, has significantly contributed to the improved accuracy of the clinical diagnosis of sporadic CJD in recent years.

Cerebrospinal Fluid Biomarker-Based Diagnosis of Sporadic Creutzfeldt-Jakob Disease: A Validation Study for Previously Established Cutoffs

BACKGROUND

Several biomarkers have been proposed to discriminate sporadic Creutzfeldt-Jakob disease (sCJD) from other dementias and control cases. However, their clinical accuracy depends on the PRNP codon 129 genotype, leaving it unclear how well established markers behave in untested conditions.

METHODS

We analyzed 14-3-3, tau, p-tau levels, and the p-tau/tau ratio in a population sample collected from Polish hospitals including nondementia, dementia, and definite sCJD cases and validated their parameters according to previously established cutoffs. Additionally, the correlation between biomarkers and disease duration as well as the influence of the PRNP129 polymorphism are reported.

RESULTS

The tau levels and p-tau/tau ratios differed considerably between sCJD and clinically characterized non-CJD cases (p < 0.001). p-tau was only elevated in sCJD when compared to cases without dementia (p < 0.05). Tau and the p-tau/tau ratio showed a sensitivity of 95 and 100%, respectively, in detecting sCJD cases. A negative correlation between tau levels and disease duration, but not the timing of lumbar puncture was observed.

CONCLUSION

The present findings confirmed the value of the p-tau/tau ratio as a robust sCJD biomarker and suggest a role for tau as prognostic marker.

Determining sample size for a binary diagnostic test in the presence of verification bias

To compare a new binary diagnostic test with the gold standard, sensitivity and specificity are the two common measurements used to evaluate the new test. When not all the patients are verified by the gold standard due to time, budget, or cost considerations, several approaches have been proposed to compute sample size for such studies under the assumption of missing completely at random. However, the majority of them are based on asymptotic approaches that generally do not guarantee the type I and II error rates, and the remaining approaches use exact binomial distributions in sample size calculation but only the verified samples are used. In this article, for a study with verification bias, we propose computing exact sample sizes by using all the samples. The proposed approach is compared with the existing exact approach that compute sample size by using verified samples only, and the results show that the proposed approach requires fewer participants than the competitor.