Verification and incorporation biases in studies assessing screening tests: prostate-specific antigen as an example

A single centre service evaluation of the pre-biopsy mpMRI pathway for prostate cancer diagnosis

Aim:

To study the efficacy and impact of the local pre-biopsy multiparametric magnetic resonance imaging (mpMRI) pathway for prostate cancer diagnosis.

Methods:

In this tertiary centre, 570 patients had prostate mpMRI across a 6-month period in 2019. A total of 511 patients met inclusion criteria for retrospective analysis. MRI reporting used the Prostate Imaging-Reporting and Data System (PI-RADS) v2.1. These were assessed alongside histological outcomes and diagnostic times. PI-RADS ⩾ 3 were recommended for biopsy consideration. Gleason scoring ⩾ 3 + 4 and 3 + 3 were used to define clinically and non-clinically significant prostate cancer (csPCa and nsPCa), respectively.

Results:

Overall prostate cancer prevalence was 40% (204/511, csPCa in 31.1%) with an overall biopsy avoidance of 32.1% (164/511). Around 69.7% (356/511) scored PI-RADS ⩾ 3 and 30.3% (155/511) scored PI-RADS 1–2. About 22.6% (35/155) of PI-RADS 1–2 patients proceeded to biopsy, demonstrating a negative predictive value of 91.43% for csPCa. For PI-RADS ⩾ 3 patients, 63.4% (197/312) of those biopsied had cancer (Gleason ⩾ 3 + 3), with 50% (156/312) demonstrating csPCa. Around 76.7% (102/133) of PI-RADS 5, 35.3% (48/136) of PI-RADS 4, 14.0% (6/43) of PI-RADS 3 and 8.6% (3/35) of PI-RADS 1–2 scores demonstrated csPCa. Overall median prostate-specific antigen (PSA) density was 0.15 ng/mL2 (IQR: 0.10–0.27/mL2). PSA density were significantly different across PI-RADS cohorts ( H = 118.8, p < 0.0001) and across all three biopsy outcomes ( H = 99.72, p < 0.0001). Only 34.3% (119/347) of biopsied patients met the NHS 28-day standard. MRI acquisition and reporting met the 14-day local standard in 96.1% (491/511). The biopsy was the most delayed component with a median of 20 days (IQR: 8–43).

Conclusion:

Pre-biopsy mpMRI with PI-RADS scoring safely avoided biopsy in almost one-third (32.1%) of patients. The use of PSA-density in risk stratifying PI-RADS 3 lesions has informed local practice in the period 2020–2021, with implementation of a PSA-density threshold of 0.12 ng/mL2. Biopsy scheduling issues and anaesthetic requirements need to be overcome to improve diagnostic waiting times.

Level of evidence:

2

Reporting Quality of Diagnostic Accuracy Studies in Laboratory Medicine: Adherence to Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015

Background

Poor reporting quality in diagnostic accuracy studies hampers an adequate judgment of the validity of the study. The Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement was published to improve the reporting quality of diagnostic accuracy studies. This study aimed to evaluate the adherence of diagnostic accuracy studies published in Annals of Laboratory Medicine (ALM) to STARD 2015 and to identify directions for improvement in the reporting quality of these studies.

Methods

Two independent authors assessed articles published in ALM between 2012–2018 for compliance with 30 STARD 2015 checklist items to identify all eligible diagnostic accuracy studies published during this period. We included 66 diagnostic accuracy studies. A total of the fulfilled STARD items were calculated, and adherence was analyzed on an individual-item basis.

Results

The overall mean±SD number of STARD items reported for the included studies was 11.2±2.7. Only five (7.6%) studies adhered to more than 50% of the 30 items. No study satisfied more than 80% of the items. Large variability in adherence to reporting standards was detected across items, ranging from 0% to 100%.

Conclusions

Adherence to STARD 2015 is suboptimal among diagnostic accuracy studies published in ALM. Our study emphasizes the necessity of adherence to STARD to improve the reporting quality of future diagnostic accuracy studies to be published in ALM.

Partial verification bias and incorporation bias affected accuracy estimates of diagnostic studies for biomarkers that were part of an existing composite gold standard

OBJECTIVE

To investigate how choice of gold standard biases estimates of sensitivity and specificity in studies reassessing the diagnostic accuracy of biomarkers that are already part of a lifetime composite gold standard (CGS).

STUDY DESIGN AND SETTING

We performed a simulation study based on the real-life example of the biomarker "protein 14-3-3" used for diagnosing Creutzfeldt-Jakob disease. Three different types of gold standard were compared: perfect gold standard "autopsy" (available in a small fraction only; prone to partial verification bias), lifetime CGS (including the biomarker under investigation; prone to incorporation bias), and "best available" gold standard (autopsy if available, otherwise CGS).

RESULTS

Sensitivity was unbiased when comparing 14-3-3 with autopsy but overestimated when using CGS or "best available" gold standard. Specificity of 14-3-3 was underestimated in scenarios comparing 14-3-3 with autopsy (up to 24%). In contrast, overestimation (up to 20%) was observed for specificity compared with CGS; this could be reduced to 0-10% when using the "best available" gold standard.

CONCLUSION

Choice of gold standard affects considerably estimates of diagnostic accuracy. Using the "best available" gold standard (autopsy where available, otherwise CGS) leads to valid estimates of specificity, whereas sensitivity is estimated best when tested against autopsy alone.

A primer on clinical trial design

A well-designed and executed clinical trial is the gold standard of evidence-based medicine. It is important for readers to understand the rationale for the study design, identify common pitfalls, and scrutinize limitations. Herein, we present a brief overview of types of designs used for clinical trials and discuss the use of appropriate end points, the selection of study participants, randomization, sample size calculation, blinding, and analysis of data. Finally, we emphasize the importance of accurate and transparent reporting. Our goal is to provide a primer for practicing urologists to enhance their understanding of the clinical trial literature.

A systematic review classifies sources of bias and variation in diagnostic test accuracy studies

OBJECTIVE

To classify the sources of bias and variation and to provide an updated summary of the evidence of the effects of each source of bias and variation.

STUDY DESIGN AND SETTING

We conducted a systematic review of studies of any design with the main objective of addressing bias or variation in the results of diagnostic accuracy studies. We searched MEDLINE, EMBASE, BIOSIS, the Cochrane Methodology Register, and Database of Abstracts of Reviews of Effects (DARE) from 2001 to October 2011. Citation searches based on three key papers were conducted, and studies from our previous review (search to 2001) were eligible. One reviewer extracted data on the study design, objective, sources of bias and/or variation, and results. A second reviewer checked the extraction.

RESULTS

We summarized the number of studies providing evidence of an effect arising from each source of bias and variation on the estimates of sensitivity, specificity, and overall accuracy.

CONCLUSIONS

We found consistent evidence for the effects of case-control design, observer variability, availability of clinical information, reference standard, partial and differential verification bias, demographic features, and disease prevalence and severity. Effects were generally stronger for sensitivity than for specificity. Evidence for other sources of bias and variation was limited.

Adult small bowel obstruction

BACKGROUND

Small bowel obstruction (SBO) is a clinical condition that is often initially diagnosed and managed in the emergency department (ED). The high rates of potential complications that are associated with an SBO make it essential for the emergency physician (EP) to make a timely and accurate diagnosis.

OBJECTIVES

The primary objective was to perform a systematic review and meta-analysis of the history, physical examination, and imaging modalities associated with the diagnosis of SBO. The secondary objectives were to identify the prevalence of SBO in prospective ED-based studies of adult abdominal pain and to apply Pauker and Kassirer's threshold approach to clinical decision-making to the diagnosis and management of SBO.

METHODS

MEDLINE, EMBASE, major emergency medicine (EM) textbooks, and the bibliographies of selected articles were scanned for studies that assessed one or more components of the history, physical examination, or diagnostic imaging modalities used for the diagnosis of SBO. The selected articles underwent a quality assessment by two of the authors using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool. Data used to compile sensitivities and specificities were obtained from these studies and a meta-analysis was performed on those that examined the same historical component, physical examination technique, or diagnostic test. Separate information on the prevalence and management of SBO was used in conjunction with the meta-analysis findings of computed tomography (CT) to determine the test and treatment threshold.

RESULTS

The prevalence of SBO in the ED was determined to be approximately 2% of all patients who present with abdominal pain. Having a previous history of abdominal surgery, constipation, abnormal bowel sounds, and/or abdominal distention on examination were the best history and physical examination predictors of SBO. X-ray was determined to be the least useful imaging modality for the diagnosis of SBO, with a pooled positive likelihood ratio (+LR) of 1.64 (95% confidence interval [CI] = 1.07 to 2.52). On the other hand, CT and magnetic resonance imaging (MRI) were both quite accurate in diagnosing SBO with +LRs of 3.6 (5- to 10-mm slices, 95% CI = 2.3 to 5.4) and 6.77 (95% CI = 2.13 to 21.55), respectively. Although limited to only a select number of studies, the use of ultrasound (US) was determined to be superior to all other imaging modalities, with a +LR of 14.1 (95% CI = 3.57 to 55.66) and a negative likelihood ratio (-LR) of 0.13 (95% CI = 0.08 to 0.20) for formal scans and a +LR of 9.55 (95% CI = 2.16 to 42.21) and a -LR of 0.04 (95% CI = 0.01 to 0.13) for beside scans. Using the CT results of the meta-analysis for the 5- to 10-mm slice subgroup as well as information on intravenous (IV) contrast reactions and nasogastric (NG) intubation management, the pretest probability threshold for further testing was determined to be 1.5%, and the pretest probability threshold for beginning treatment was determined to be 20.7%.

CONCLUSIONS

The potentially useful aspects of the history and physical examination were limited to a history of abdominal surgery, constipation, and the clinical examination findings of abnormal bowel sounds and abdominal distention. CT, MRI, and US are all adequate imaging modalities to make the diagnosis of SBO. Bedside US, which can be performed by EPs, had very good diagnostic accuracy and has the potential to play a larger role in the ED diagnosis of SBO. More ED-focused research into this area will be necessary to bring about this change.

Bayesian estimation for performance measures of two diagnostic tests in the presence of verification bias

Sensitivity and specificity are measures that allow us to evaluate the performance of a diagnostic test. In practice, it is common to have situations where a proportion of selected individuals cannot have the real state of the disease verified, since the verification could be an invasive procedure, as occurs with biopsy. This happens, as a special case, in the diagnosis of prostate cancer, or in any other situation related to risks, that is, not practicable, nor ethical, or in situations with high cost. For this case, it is common to use diagnostic tests based only on the information of verified individuals. This procedure can lead to biased results or workup bias. In this paper, we introduce a Bayesian approach to estimate the sensitivity and the specificity for two diagnostic tests considering verified and unverified individuals, a result that generalizes the usual situation based on only one diagnostic test.