Correcting for Partial Verification Bias: A Comparison of Methods

Changes in illness perception, pain catastrophizing, and psychological distress following hand surgeon consultation: A prospective study

BACKGROUND

Baseline mindset factors are important factors that influence treatment decisions and outcomes. Theoretically, improving the mindset prior to treatment may improve treatment decisions and outcomes. This prospective cohort study evaluated changes in patients' mindset following hand surgeon consultation. Additionally, we assessed if the change in illness perception differed between surgical and nonsurgical patients.

METHODS

The primary outcome was illness perception, measured using the total score of the Brief Illness Perception Questionnaire (B-IPQ, range 0-80). Secondary outcomes were the B-IPQ subscales, pain catastrophizing (measured using the Pain Catastrophizing Scale (PCS)), and psychological distress (measured using the Patient Health Questionnaire-4).

RESULTS

A total of 276 patients with various hand and wrist conditions completed the mindset questionnaires before and after hand surgeon consultation (median time interval: 15 days). The B-IPQ total score improved from 39.7 (±10.6) before to 35.8 (±11.3) after consultation (p < 0.0001, Cohen's d = 0.36); scores also improved for the B-IPQ subscales Coherence, Concern, Emotional Response, Timeline, Treatment Control, and Identity and the PCS. There were no changes in the other outcomes. Surgical patients improved on the B-IPQ subscales Treatment Control and Timeline, while nonsurgical patients did not.

CONCLUSIONS

Illness perception and pain catastrophizing improved following hand surgeon consultation, suggesting that clinicians may actively influence the patients' mindset during consultations, and that they may try to enhance this effect to improve outcomes. Furthermore, surgical patients improved more in illness perceptions, indicating that nonsurgical patients may benefit from a more targeted strategy for changing mindset.

Diagnostic accuracy of MRI, CT, and [18F]FDG-PET-CT in detecting lymph node metastases in clinically early-stage cervical cancer - a nationwide Dutch cohort study

OBJECTIVES

Imaging is increasingly used to assess lymph node involvement in clinically early-stage cervical cancer. This retrospective study aimed to evaluate the diagnostic accuracy of MRI, CT, and [18F]FDG-PET-CT.

METHODS

Women with International Federation of Gynaecology and Obstetrics (FIGO) 2009 stage IA2-IIA cervical cancer and pretreatment imaging between 2009 and 2017 were selected from the Netherlands Cancer Registry. Patient-based and region-based (i.e. pelvic and common iliac) nodal status was extracted from radiology reports. Pathology results were considered the reference standard for calculating accuracy indices. Multiple imputation was used for missing pathology to limit verification bias risk.

RESULTS

Nodal assessment was performed in 1676 patients with MRI, 926 with CT, and 379 with [18F]FDG-PET-CT, with suspicious nodes detected in 17%, 16%, and 48%, respectively. [18F]FDG-PET-CT was used to confirm MRI/CT results in 95% of patients. Pathology results were imputed for 30% of patients. [18F]FDG-PET-CT outperformed MRI and CT in detecting patient-based nodal metastases with sensitivities of 80%, 48%, and 40%, and AUCs of 0.814, 0.706, and 0.667, respectively, but not in specificity: 79%, 92%, and 92%. Region-based analyses showed similar indices in the pelvic region, but worse performance in the common iliac region with AUCs of 0.575, 0.554, and 0.517, respectively.

CONCLUSIONS

[18F]FDG-PET-CT outperformed MRI and CT in detecting nodal metastases, which may be related to its use as a verification modality. However, MRI and CT had the highest specificity. As MRI is generally performed routinely to assess local and regional spread of cervical cancer, [18F]FDG-PET-CT can be used to confirm suspicious nodes.

CRITICAL RELEVANCE STATEMENT

Accurate assessment of the nodal status in clinically early-stage cervical cancer is essential for tumour staging, treatment decision making and prognosis.

KEY POINTS

• The accuracy of MRI, CT or [18F]FDG-PET-CT for nodal staging in early cervical cancer is a subject of discussion. • Overall, [18F]FDG-PET-CT outperformed MRI, followed by CT, when used as a verification modality. • Staging with MRI and the addition of [18F]FDG-PET-CT to verify high-risk cases seems to be a good approach.

Long-Term Outcomes of Nonsurgical Treatment of Thumb Carpometacarpal Osteoarthritis: A Cohort Study

BACKGROUND

Although nonsurgical treatment of thumb carpometacarpal (CMC-1) osteoarthritis (OA) provides short-term improvement, the durability of these effects beyond 1 year is unknown. In this study, we investigated patient-reported pain and limitations in activities of daily living (ADL) at >5 years following nonsurgical treatment (i.e., exercise therapy and use of an orthosis) for CMC-1 OA. We hypothesized that pain and limitations in ADL would not worsen after 12 months. Secondary outcomes were satisfaction with treatment results and health-related quality of life at >5 years of follow-up and the rate of conversion to surgery.

METHODS

This was a multicenter, prospective cohort study using 2 overlapping samples. The change in the Michigan Hand Outcomes Questionnaire (MHQ) subscales of pain and ADL between 12 months and >5 years was the primary outcome as measured in the first sample (n = 170), which consisted of patients who did not undergo conversion to surgery. Additional measurement time points included baseline and 3 months. We evaluated conversion to surgery in a second sample, which included all patients who responded to the invitation for this follow-up study (n = 217).

RESULTS

At a median follow-up of 6.6 years (range, 5.1 to 8.7 years), the score on the MHQ pain subscale did not differ significantly from that at 12 months. The score on the MHQ ADL improved by 4.4 points (95% confidence interval [CI],1.5 to 7.2) compared with 12 months, but this was not clinically relevant. At >5 years, 5% of the patients rated their satisfaction as "poor," 14% as "moderate," 26% as "fair," 39% as "good," and 16% as "excellent." The median EuroQol-5 Dimensions-5 Levels (EQ-5D-5L) index score was 0.852 (range, 0.135 to 1). The rate of conversion to surgery was 22% (95% CI,16.4% to 27.7%) at a median follow-up of 7 years (range, 5.5 to 9.0 years).

CONCLUSIONS

We found positive outcomes at >5 years of follow-up for nonsurgical treatment of CMC-1 OA, with no worsening of pain or of limitations in ADL after 12 months. Our findings support nonsurgical treatment as the first treatment choice and suggest that treatment effects are sustainable.

LEVEL OF EVIDENCE

Therapeutic Level II . See Instructions for Authors for a complete description of levels of evidence.

Diagnostic and prognostic value of cardiac stress testing before major noncardiac surgery-A cohort study

OBJECTIVE

To assess the incremental contribution of preoperative stress test results toward a diagnosis of obstructive coronary artery disease (CAD), prediction of mortality, or prediction of perioperative myocardial infarction in patients considering noncardiac, nonophthalmologic surgery.

DESIGN, SETTING, PARTICIPANTS

A retrospective cohort study of visits to a preoperative risk assessment and optimization clinic in a large health system between 2008 and 2018.

MEASUREMENTS

To assess diagnostic information of preoperative stress testing, we used the Begg and Greenes method to calculate test characteristics adjusted for referral bias, with a gold standard of angiography. To assess prognostic information, we first created multiply-imputed logistic regression models to predict 90-day mortality and perioperative myocardial infarction (MI), starting with two tools commonly used to assess perioperative cardiac risk, Revised Cardiac Risk Index (RCRI) and Myocardial Infarction or Cardiac Arrest (MICA). We then added stress test results and compared the discrimination for models with and without stress test results.

MAIN RESULTS

Among 136,935 visits by patients without an existing diagnosis of CAD, the decision to obtain preoperative stress testing identified around 4.0% of likely new diagnoses. Stress testing increased the likelihood of CAD (likelihood ratio: 1.31), but for over 99% of patients, stress testing should not change a decision on whether to proceed to angiography. In 117,445 visits with subsequent noncardiac surgery, stress test results failed to improve predictions of either perioperative MI or 90-day mortality. Reweighting the models and adding hemoglobin improved the prediction of both outcomes.

CONCLUSIONS

Cardiac stress testing before noncardiac, nonophthalmologic surgery does not improve predictions of either perioperative mortality or myocardial infarction. Very few patients considering noncardiac, nonophthalmologic surgery have a pretest probability of CAD in a range where stress testing could usefully select patients for angiography. Better use of existing patient data could improve predictions of perioperative adverse events without additional patient testing.

Changes in illness perception, pain catastrophizing, and psychological distress following hand surgeon consultation: A prospective study

BACKGROUND

Baseline mindset factors are important factors that influence treatment decisions and outcomes. Theoretically, improving the mindset prior to treatment may improve treatment decisions and outcomes. This prospective cohort study evaluated changes in patients' mindset following hand surgeon consultation. Additionally, we assessed if the change in illness perception differed between surgical and nonsurgical patients.

METHODS

The primary outcome was illness perception, measured using the total score of the Brief Illness Perception Questionnaire (B-IPQ, range 0-80). Secondary outcomes were the B-IPQ subscales, pain catastrophizing (measured using the Pain Catastrophizing Scale (PCS)), and psychological distress (measured using the Patient Health Questionnaire-4).

RESULTS

A total of 276 patients with various hand and wrist conditions completed the mindset questionnaires before and after hand surgeon consultation (median time interval: 15 days). The B-IPQ total score improved from 39.7 (±10.6) before to 35.8 (±11.3) after consultation (p < 0.0001, Cohen's d = 0.36); scores also improved for the B-IPQ subscales Coherence, Concern, Emotional Response, Timeline, Treatment Control, and Identity and the PCS. There were no changes in the other outcomes. Surgical patients improved on the B-IPQ subscales Treatment Control and Timeline, while nonsurgical patients did not.

CONCLUSIONS

Illness perception and pain catastrophizing improved following hand surgeon consultation, suggesting that clinicians may actively influence the patients' mindset during consultations, and that they may try to enhance this effect to improve outcomes. Furthermore, surgical patients improved more in illness perceptions, indicating that nonsurgical patients may benefit from a more targeted strategy for changing mindset.

Outcomes of 218 primary single-finger flexor tendon repairs up to 1 year after surgery: a multicentre cohort study

In this retrospective multicentre cohort study, we present the results of primary single-finger flexor tendon repairs in zones 1 to 3 between 2014 and 2021. Data from 218 patients were retrieved with their demographics, injury and surgery characteristics and therapy outcomes. The data were systematically collected and analysed at predefined time points up to 1 year after surgery. A good to excellent return of motion was achieved by 77% (Tang classification) and 92% (American Association for Surgery of the Hand classification) of patients at 1 year. The tendon rupture rate was 8.7%. Time had a significant impact on the recovery of finger motion and grip strength of up to 1 year, of patient satisfaction and upper extremity function up to 26 weeks, and of pain up to 13 weeks after surgery. Our results demonstrate that it is worthwhile to assess therapy outcomes over various measurement periods, as some outcomes may continue to improve up to 1 year after flexor tendon repair surgery.Level of evidence: III.

Three-dimensional printed customized versus conventional plaster brace for trapeziometacarpal osteoarthritis: a randomized controlled crossover trial

We investigated the non-operative management of trapeziometacarpal osteoarthritis with a three-dimensional (3-D) printed patient-customized brace compared with a conventional plaster brace. Fifty-two patients with symptomatic trapeziometacarpal osteoarthritis were enrolled in a 9-week crossover study, which was designed as a randomized controlled trial of two periods of 4-week brace therapies. The primary outcome was patient satisfaction measured with the Dutch version of the Quebec User Evaluation of Satisfaction with Assistive Technology questionnaire survey. Secondary outcomes included pain, patient-reported function, functional hand strength measured by pinch and grip strength, and compliance assessed through a daily log of self-reported brace usage. The 3-D printed patient-customized brace had higher patient satisfaction and compliance than the conventional plaster brace. Patients preferred the 3-D printed customized brace (93%) rather than the conventional plaster brace (7%). This suggests that the 3-D printed patient-customized brace is effective in the non-operative management of trapeziometacarpal osteoarthritis.Level of evidence: I.

Partial Verification Bias Correction Using Inverse Probability Bootstrap Sampling for Binary Diagnostic Tests

In medical care, it is important to evaluate any new diagnostic test in the form of diagnostic accuracy studies. These new tests are compared to gold standard tests, where the performance of binary diagnostic tests is usually measured by sensitivity (Sn) and specificity (Sp). However, these accuracy measures are often biased owing to selective verification of the patients, known as partial verification bias (PVB). Inverse probability bootstrap (IPB) sampling is a general method to correct sampling bias in model-based analysis and produces debiased data for analysis. However, its utility in PVB correction has not been investigated before. The objective of this study was to investigate IPB in the context of PVB correction under the missing-at-random assumption for binary diagnostic tests. IPB was adapted for PVB correction, and tested and compared with existing methods using simulated and clinical data sets. The results indicated that IPB is accurate for Sn and Sp estimation as it showed low bias. However, IPB was less precise than existing methods as indicated by the higher standard error (SE). Despite this issue, it is recommended to use IPB when subsequent analysis with full data analytic methods is expected. Further studies must be conducted to reduce the SE.

Which Factors Are Associated With Satisfaction With Treatment Results in Patients With Hand and Wrist Conditions? A Large Cohort Analysis

BACKGROUND

Satisfaction with treatment results is an important outcome domain in striving for patient-centered and value-based healthcare. Although numerous studies have investigated factors associated with satisfaction with treatment results, most studies used relatively small samples. Additionally, many studies have only investigated univariable associations instead of multivariable associations; to our knowledge, none have investigated the independent association of baseline sociodemographics, quality of life, improvement in pain and function, experiences with healthcare delivery, and baseline measures of mental health with satisfaction with treatment results.

QUESTIONS/PURPOSES

(1) What factors are independently associated with satisfaction with treatment results at 3 months post-treatment in patients treated for common hand and wrist conditions? (2) What factors are independently associated with the willingness to undergo the treatment again at 3 months post-treatment in patients treated for common hand and wrist conditions? Among the factors under study were baseline sociodemographics, quality of life, improvement in pain and function, experiences with healthcare delivery, and baseline measures of mental health.

METHODS

Between August 2018 and May 2020, we included patients who underwent carpal tunnel release, nonsurgical or surgical treatment for thumb-base osteoarthritis, trigger finger release, limited fasciectomy for Dupuytren contracture, or nonsurgical treatment for midcarpal laxity in one of the 28 centers of Xpert Clinics in the Netherlands. We screened 5859 patients with complete sociodemographics and data at baseline. Thirty-eight percent (2248 of 5859) of these patients had complete data at 3 months. Finally, participants were eligible for inclusion if they provided a relevant answer to the three patient-reported experience measure (PREM) items. A total of 424 patients did not do this because they answered "I don't know" or "not applicable" to a PREM item, leaving 31% (1824 of 5859) for inclusion in the final sample. A validated Satisfaction with Treatment Result Questionnaire was administered at 3 months, which identified the patients' level of satisfaction with treatment results so far on a 5-point Likert scale (research question 1, with answers of poor, moderate, fair, good, or excellent) and the patients' willingness to undergo the treatment again under similar circumstances (research question 2, with answers of yes or no). A hierarchical logistic regression model was used to identify whether baseline sociodemographics, quality of life, change in outcome (patient-reported outcome measures for hand function and pain), baseline measures of mental health (including treatment credibility [the extent to which a patient attributes credibility to a treatment] and expectations, illness perception, pain catastrophizing, anxiety, and depression), and PREMs were associated with each question of the Satisfaction with Treatment Result Questionnaire at 3 months post-treatment. We dichotomized responses to our first question as good and excellent, which were considered more satisfied, and poor, moderate, and fair, which were considered less satisfied. After dichotomization, 57% (1042 of 1824) of patients were classified as more satisfied with the treatment results.

RESULTS

The following variables were independently associated with satisfaction with treatment results, with an area under the curve of 0.82 (95% confidence interval 0.80 to 0.84) (arranged from the largest to the smallest standardized odds ratio [SOR]): greater decrease in pain during physical load (standardized odds ratio 2.52 [95% CI 2.18 to 2.92]; p < 0.001), patient's positive experience with the explanation of the pros and cons of the treatment (determined with the question: "Have you been explained the pros and cons of the treatment or surgery?") (SOR 1.83 [95% CI 1.41 to 2.38]; p < 0.001), greater improvement in hand function (SOR 1.76 [95% CI 1.54 to 2.01]; p < 0.001), patients' positive experience with the advice for at-home care (determined with the question: "Were you advised by the healthcare providers on how to deal with your illness or complaints in your home situation?") (SOR 1.57 [95% CI 1.21 to 2.04]; p < 0.001), patient's better personal control (determined with the question: "How much control do you feel you have over your illness?") (SOR 1.24 [95% CI 1.1 to 1.40]; p < 0.001), patient's more positive treatment expectations (SOR 1.23 [95% CI 1.04 to 1.46]; p = 0.02), longer expected illness duration by the patient (SOR 1.20 [95% CI 1.04 to 1.37]; p = 0.01), a smaller number of symptoms the patient saw as part of the illness (SOR 0.84 [95% CI 0.72 to 0.97]; p = 0.02), and less concern about the illness the patient experiences (SOR 0.84 [95% CI 0.72 to 0.99]; p = 0.04). For willingness to undergo the treatment again, the following variables were independently associated with an AUC of 0.81 (95% CI 0.78 to 0.83) (arranged from the largest to the smallest standardized OR): patient's positive experience with the information about the pros and cons (determined with the question: "Have you been explained the pros and cons of the treatment or surgery?") (SOR 2.05 [95% CI 1.50 to 2.80]; p < 0.001), greater improvement in hand function (SOR 1.80 [95% CI 1.54 to 2.11]; p < 0.001), greater decrease in pain during physical load (SOR 1.74 [95% CI 1.48 to 2.07]; p < 0.001), patient's positive experience with the advice for at home (determined with the question: "Were you advised by the healthcare providers on how to deal with your illness or complaints in your home situation?") (SOR 1.52 [95% CI 1.11 to 2.07]; p = 0.01), patient's positive experience with shared decision-making (determined with the question: "Did you decide together with the care providers which care or treatment you will receive?") (SOR 1.45 [95% CI 1.06 to 1.99]; p = 0.02), higher credibility the patient attributes to the treatment (SOR 1.44 [95% CI 1.20 to 1.73]; p < 0.001), longer symptom duration (SOR 1.27 [95% CI 1.09 to 1.52]; p < 0.01), and patient's better understanding of the condition (SOR 1.17 [95% CI 1.01 to 1.34]; p = 0.03).

CONCLUSION

Our findings suggest that to directly improve satisfaction with treatment results, clinicians might seek to: (1) improve the patient's experience with healthcare delivery, (2) try to influence illness perception, and (3) boost treatment expectations and credibility. Future research should confirm whether these suggestions are valid and perhaps also investigate whether satisfaction with treatment results can be predicted (instead of explained, as was done in this study). Such prediction models, as well as other decision support tools that investigate patient-specific needs, may influence experience with healthcare delivery, expectations, or illness perceptions, which in turn may improve satisfaction with treatment results.

LEVEL OF EVIDENCE

Level III, therapeutic study.

Detection of atrial fibrillation in primary care with radial pulse palpation, electronic blood pressure measurement and handheld single-lead electrocardiography: a diagnostic accuracy study

OBJECTIVE

To determine the diagnostic accuracy of three tests-radial pulse palpation, an electronic blood pressure monitor and a handheld single-lead ECG device-for opportunistic screening for unknown atrial fibrillation (AF).

DESIGN

We performed a diagnostic accuracy study in the intention-to-screen arm of a cluster randomised controlled trial aimed at opportunistic screening for AF in general practice. We performed radial pulse palpation, followed by electronic blood pressure measurement (WatchBP Home A) and handheld ECG (MyDiagnostick) in random order. If one or more index tests were positive, we performed a 12-lead ECG at shortest notice. Similarly, to limit verification bias, a random sample of patients with three negative index tests received this reference test. Additionally, we analysed the dataset using multiple imputation. We present pooled diagnostic parameters.

SETTING

47 general practices participated between September 2015 and August 2018.

PARTICIPANTS

In the electronic medical record system of the participating general practices (n=47), we randomly marked 200 patients of ≥65 years without AF. When they visited the practice for any reason, we invited them to participate. Exclusion criteria were terminal illness, inability to give informed consent or visit the practice or having a pacemaker or an implantable cardioverter-defibrillator.

OUTCOMES

Diagnostic accuracy of individual tests and test combinations to detect unknown AF.

RESULTS

We included 4339 patients; 0.8% showed new AF. Sensitivity and specificity were 62.8% (range 43.1%-69.7%) and 91.8% (91.7%-91.8%) for radial pulse palpation, 70.0% (49.0%-80.6%) and 96.5% (96.3%-96.7%) for electronic blood pressure measurement and 90.1% (60.8%-100%) and 97.9% (97.8%-97.9%) for handheld ECG, respectively. Positive predictive values were 5.8% (5.3%-6.1%), 13.8% (12.2%-14.8%) and 25.2% (24.2%-25.8%), respectively. All negative predictive values were ≥99.7%.

CONCLUSION

In detecting AF, electronic blood pressure measurement (WatchBP Home A), but especially handheld ECG (MyDiagnostick) showed better diagnostic accuracy than radial pulse palpation.

TRIAL REGISTRATION NUMBER

Netherlands Trial Register No. NL4776 (old NTR4914).

Adjusting for verification bias in diagnostic accuracy measures when comparing multiple screening tests - an application to the IP1-PROSTAGRAM study

Introduction

Novel screening tests used to detect a target condition are compared against either a reference standard or other existing screening methods. However, as it is not always possible to apply the reference standard on the whole population under study, verification bias is introduced. Statistical methods exist to adjust estimates to account for this bias. We extend common methods to adjust for verification bias when multiple tests are compared to a reference standard using data from a prospective double blind screening study for prostate cancer.

Methods

Begg and Greenes method and multiple imputation are extended to include the results of multiple screening tests which determine condition verification status. These two methods are compared to the complete case analysis using the IP1-PROSTAGRAM study data. IP1-PROSTAGRAM used a paired-cohort double-blind design to evaluate the use of imaging as alternative tests to screen for prostate cancer, compared to a blood test called prostate specific antigen (PSA). Participants with positive imaging (index) and/or PSA (control) underwent a prostate biopsy (reference standard).

Results

When comparing complete case results to Begg and Greenes and methods of multiple imputation there is a statistically significant increase in the specificity estimates for all screening tests. Sensitivity estimates remained similar across the methods, with completely overlapping 95% confidence intervals. Negative predictive value (NPV) estimates were higher when adjusting for verification bias, compared to complete case analysis, even though the 95% confidence intervals overlap. Positive predictive value (PPV) estimates were similar across all methods.

Conclusion

Statistical methods are required to adjust for verification bias in accuracy estimates of screening tests. Expanding Begg and Greenes method to include multiple screening tests can be computationally intensive, hence multiple imputation is recommended, especially as it can be modified for low prevalence of the target condition.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01481-w.

Statistical methods for evaluating the fine needle aspiration cytology procedure in breast cancer diagnosis

Background

Statistical issues present while evaluating a diagnostic procedure for breast cancer are non rare but often ignored, leading to biased results. We aimed to evaluate the diagnostic accuracy of the fine needle aspiration cytology(FNAC), a minimally invasive and rapid technique potentially used as a rule-in or rule-out test, handling its statistical issues: suspect test results and verification bias.

Methods

We applied different statistical methods to handle suspect results by defining conditional estimates. When considering a partial verification bias, Begg and Greenes method and multivariate imputation by chained equations were applied, however, and a Bayesian approach with respect to each gold standard was used when considering a differential verification bias. At last, we extended the Begg and Greenes method to be applied conditionally on the suspect results.

Results

The specificity of the FNAC test above 94%, was always higher than its sensitivity regardless of the proposed method. All positive likelihood ratios were higher than 10, with variations among methods. The positive and negative yields were high, defining precise discriminating properties of the test.

Conclusion

The FNAC test is more likely to be used as a rule-in test for diagnosing breast cancer. Our results contributed in advancing our knowledge regarding the performance of FNAC test and the methods to be applied for its evaluation.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12874-022-01506-y).

Correcting for partial verification bias in diagnostic accuracy studies: A tutorial using R

Diagnostic tests play a crucial role in medical care. Thus any new diagnostic tests must undergo a thorough evaluation. New diagnostic tests are evaluated in comparison with the respective gold standard tests. The performance of binary diagnostic tests is quantified by accuracy measures, with sensitivity and specificity being the most important measures. In any diagnostic accuracy study, the estimates of these measures are often biased owing to selective verification of the patients, which is referred to as partial verification bias. Several methods for correcting partial verification bias are available depending on the scale of the index test, target outcome, and missing data mechanism. However, these are not easily accessible to the researchers due to the complexity of the methods. This article aims to provide a brief overview of the methods available to correct for partial verification bias involving a binary diagnostic test and provide a practical tutorial on how to implement the methods using the statistical programming language R.

Psychological Characteristics, Female Sex, and Opioid Use Predict Acute Postoperative Pain in Patients Surgically Treated for Thumb Base Osteoarthritis: A Cohort Study

BACKGROUND

It is unclear which factors predict acute postoperative pain in patients surgically treated for thumb base osteoarthritis. The authors investigated the influence of type of surgery, preoperative sociodemographics, preoperative patient-reported outcome measures, psychological characteristics, and postoperative opioid use on acute postoperative pain 24 hours postoperatively following surgery for thumb carpometacarpal osteoarthritis. In addition, preoperative and acute postoperative pain were compared.

METHODS

In this prospective cohort study, 215 patients surgically treated for thumb carpometacarpal osteoarthritis were included. Data were collected in 16 clinics for hand surgery and therapy in The Netherlands. Hierarchical regression was used to identify whether type of surgery, preoperative sociodemographics, preoperative patient-reported outcome measures, psychological characteristics (including treatment credibility and expectations, illness perception, pain catastrophizing, anxiety, and depression), and postoperative opioid use predicted acute postoperative pain 24 hours postoperatively, measured using the Numeric Pain Rating Scale (range, 0 to 10).

RESULTS

Female sex, opioid use, higher preoperative satisfaction with hand, and higher self-reported consequences and coherence predicted greater postoperative pain, with 31 percent explained variance in the final model including psychological factors. Mean postoperative Numeric Pain Rating Scale score was lower (5.1 ± 2.4) than preoperative pain, measured using visual analogue scales (during the past week, 6.7 ± 1.7; physical load, 7.5 ± 1.7) and the Michigan Hand Outcomes Questionnaire (6.4 ± 1.4; p < 0.001).

CONCLUSIONS

Psychological factors, female sex, and opioid use enhance the prediction of acute postoperative pain beyond surgery type, preoperative sociodemographics, and patient-reported outcome measures. Female sex and opioid use were the strongest predictors, even after controlling for psychological factors. Future studies may investigate sex-based approaches and patient education for reducing acute postoperative pain.

CLINICAL QUESTION/LEVEL OF EVIDENCE

Risk, II.

Newborn Screening for Long-Chain 3-Hydroxyacyl-CoA Dehydrogenase and Mitochondrial Trifunctional Protein Deficiencies Using Acylcarnitines Measurement in Dried Blood Spots-A Systematic Review of Test Accuracy

Background: Long-chain 3-hydroxyacyl-CoA dehydrogenase (LCHAD) and mitochondrial trifunctional protein (MTP) deficiencies are rare autosomal recessive fatty acid β-oxidation disorders. Their clinical presentations are variable, and premature death is common. They are included in newborn blood spot screening programs in many countries around the world. The current process of screening, through the measurement of acylcarnitines (a metabolic by-product) in dried blood spots with tandem mass spectrometry, is subject to uncertainty regarding test accuracy. Methods: We conducted a systematic review of literature published up to 19th June 2018. We included studies that investigated newborn screening for LCHAD or MTP deficiencies by tandem mass spectrometry of acylcarnitines in dried blood spots. The reference standards were urine organic acids, blood acylcarnitine profiles, enzyme analysis in cultured fibroblasts or lymphocytes, mutation analysis, or at least 10-year follow-up. The outcomes of interest were sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Assessment of titles, abstracts, and full-text papers and quality appraisal were carried out independently by two reviewers. One reviewer extracted study data. This was checked by a second reviewer. Results: Ten studies provided data on test accuracy. LCHAD or MTP deficiencies were identified in 23 babies. No cases of LCHAD/MTP deficiencies were identified in four studies. PPV ranged from 0% (zero true positives and 28 false positives from 276,565 babies screened) to 100% (13 true positives and zero false positives from 2,037,824 babies screened). Sensitivity, specificity, and NPV could not be calculated as there was no systematic follow-up of babies who screened negative. Conclusions: Test accuracy estimates of screening for LCHAD and MTP deficiencies with tandem mass spectrometry measurement of acylcarnitines in dried blood were variable in terms of PPVs. Screening methods (including markers and thresholds) varied between studies, and sensitivity, specificity, and NPVs are unknown.

Identifying Patients With Axial Spondyloarthritis in Large Datasets: Expanding Possibilities for Observational Research

OBJECTIVE

Observational research of axial spondyloarthritis (axSpA) is limited by a lack of methods for identifying diverse axSpA phenotypes in large datasets. Algorithms were previously designed to identify a broad spectrum of patients with axSpA, including patients not identifiable with diagnosis codes. The study objective was to estimate the performance of axSpA identification methods in the general Veterans Affairs (VA) population.

METHODS

A patient sample with known axSpA status (n = 300) was established with chart review. For feasibility, this sample was enriched with veterans with axSpA risk factors. Algorithm performance outcomes included sensitivities, positive predictive values (PPV), and F1 scores (an overall performance metric combining sensitivity and PPV). Performance was estimated with unweighted outcomes for the axSpA-enriched sample and inverse probability weighted (IPW) outcomes for the general VA population. These outcomes were also assessed for traditional identification methods using diagnosis codes for the ankylosing spondylitis (AS) subtype of axSpA.

RESULTS

The mean age was 54.7 and 92% were male. Unweighted F1 scores (0.59-0.74) were higher than IPW F1 scores (0.48-0.65). The full algorithm had the best overall performance (F1_IPW 0.65). The Early Algorithm was the most inclusive (sensitivity_IPW 0.90, PPV_IPW 0.38). The traditional method using ≥ 2 AS diagnosis codes from rheumatology had the highest PPV (PPV_IPW 0.84, sensitivity_IPW 0.34).

CONCLUSION

The axSpA identification methods demonstrated a range of performance attributes in the general VA population that may be appropriate for various types of studies. The novel identification algorithms may expand the scope of research by enabling identification of more diverse axSpA populations.

Routine Health Outcome Measurement: Development, Design, and Implementation of the Hand and Wrist Cohort

Routine measurement of outcome of clinical care is increasingly considered important, but implementation in practice is challenging. This article describes (1) how the authors created and implemented a routine outcome measurement cohort of patients with hand and wrist conditions and (2) how these data are used to improve the quality of care and facilitate scientific research. Starting in 2011, routine outcome measurement was implemented at all practice sites (currently 22) of a specialized treatment center for hand and wrist conditions across The Netherlands. The authors developed five "measurement tracks," including measurements administered at predetermined time points covering all hand and wrist disorders and treatments. An online system automatically distributes measurements among patients, which can be accessed by health care professionals. Using this system, the total number of yearly assigned tracks increased up to over 16,500 in 2018, adding up to 85,000 tracks in 52,000 patients in total. All surgeons, therapists, and other staff have direct access to individual patient data and patients have access to their treatment information using a secure patient portal. The data serve as a basis for studies on, among others, comparative effectiveness, prediction modeling, and clinimetric analyses. In conclusion, the authors present the design and successful implementation of a routine outcome measurement system that was made feasible using a highly automated data collection infrastructure, tightly linked to the patient journey and the workflow of health care professionals. The system serves not only as a tool to improve care but also as a basis for scientific research studies.

Diagnostic test evaluation methodology: A systematic review of methods employed to evaluate diagnostic tests in the absence of gold standard - An update

OBJECTIVE

To systematically review methods developed and employed to evaluate the diagnostic accuracy of medical test when there is a missing or no gold standard.

STUDY DESIGN AND SETTINGS

Articles that proposed or applied any methods to evaluate the diagnostic accuracy of medical test(s) in the absence of gold standard were reviewed. The protocol for this review was registered in PROSPERO (CRD42018089349).

RESULTS

Identified methods were classified into four main groups: methods employed when there is a missing gold standard; correction methods (which make adjustment for an imperfect reference standard with known diagnostic accuracy measures); methods employed to evaluate a medical test using multiple imperfect reference standards; and other methods, like agreement studies, and a mixed group of alternative study designs. Fifty-one statistical methods were identified from the review that were developed to evaluate medical test(s) when the true disease status of some participants is unverified with the gold standard. Seven correction methods were identified and four methods were identified to evaluate medical test(s) using multiple imperfect reference standards. Flow-diagrams were developed to guide the selection of appropriate methods.

CONCLUSION

Various methods have been proposed to evaluate medical test(s) in the absence of a gold standard for some or all participants in a diagnostic accuracy study. These methods depend on the availability of the gold standard, its' application to the participants in the study and the availability of alternative reference standard(s). The clinical application of some of these methods, especially methods developed when there is missing gold standard is however limited. This may be due to the complexity of these methods and/or a disconnection between the fields of expertise of those who develop (e.g. mathematicians) and those who employ the methods (e.g. clinical researchers). This review aims to help close this gap with our classification and guidance tools.

Exercise Therapy in Addition to an Orthosis Reduces Pain More Than an Orthosis Alone in Patients With Thumb Base Osteoarthritis: A Propensity Score Matching Study

OBJECTIVE

To compare the effect of exercises and orthotics with orthotics alone on pain and hand function in patients with first carpometacarpal joint (CMC-1) osteoarthritis (OA) and to predict outcomes on pain and hand function of exercises and orthotics.

DESIGN

Prospective cohort study with propensity score matching.

SETTING

Data collection took place in 13 outpatient clinics for hand surgery and hand therapy in The Netherlands.

PARTICIPANTS

A consecutive, population-based sample of patients with CMC-1 OA (N=173) was included in this study, of which 84 were matched on baseline demographics and baseline primary outcomes.

INTERVENTIONS

Exercises and orthotics versus orthotics alone.

MAIN OUTCOME MEASURES

Primary outcomes included pain and hand function at 3 months, measured using visual analog scale (VAS, 0-100) and the Michigan Hand Outcomes Questionnaire (MHQ, 0-100).

RESULTS

A larger decrease in VAS pain at rest (11.1 points difference; 95% confidence interval, 1.9-20.3; P=.002) and during physical load (22.7 points difference; 95% confidence interval, 13.6-31.0; P<.001) was found in the exercise + orthotic group compared to the orthotic group. In addition, larger improvement was found for the MHQ subscales pain, work performance, aesthetics, and satisfaction in the exercise + orthotic group. No differences were found on other outcomes. Baseline scores of metacarpophalangeal flexion, presence of scaphotrapeziotrapezoid OA, VAS pain at rest, heavy physical labor, and MHQ total predicted primary outcomes for the total exercise + orthotic group (N=131).

CONCLUSIONS

Non-surgical treatment of patients with CMC-1 OA should include exercises, since there is a relatively large treatment effect compared to using an orthosis alone. Future research should study exercises and predictors in a more standardized setting to confirm this finding.

A Bayesian Analysis With Informative Prior on Disease Prevalence for Predicting Missing Values Due To Verification Bias

AIM

Verification bias is one of the major problems encountered in diagnostic accuracy studies. It occurs when a standard test performed on a non-representative subsample of subjects which have undergone the diagnostic test. In this study we extend a Bayesian model to correct this bias.

METHODS

The study population is patients that have undergone at least two repeated failed IVF/ICSI (in vitro fertilization/intra cytoplasmic sperm injection) cycles. Patients were screened using ultrasonography and those with polyps were recommended for hysteroscopy. A Bayesian modeling was applied on mechanism of missing data using an informative prior on disease prevalence. The parameters of the model were estimated through Markov Chain Monte Carlo methods.

RESULTS

A total of 238 patients were screened, 47 of which had polyps. Those with polyps were strongly recommended to undergo hysteroscopy, 47/47 decide to have a hysteroscopy and in 37/47 polyps confirmed. None of the 191 patients with no polyps detected in ultrasonography underwent a hysteroscopy. A model using Bayesian approach was applied with informative prior on polyp prevalence. False and true negatives were estimated in the Bayesian framework. The false negative was obtained 14 and 177 true negatives were obtained, so sensitivity and specificity was estimated easily after estimating the missing data. Sensitivity and specificity were equal to 74% and 94% respectively.

CONCLUSION

Bayesian analyses with informative prior seem to be powerful tools in the simulation of experimental space.

Clinical phenotypes of obstructive airway diseases in an outpatient population

BACKGROUND AND OBJECTIVES

Historically, obstructive airway diseases such as asthma and COPD are classified as different diseases. Although the definitions are clearly described, classification of patients into these traditional, clinical disease entity can be difficult. Recent evidence that there are complex, overlapping phenotypes of obstructive lung disease. Our aim was to capture clinical phenotypes of obstructive diseases through the use of cluster analysis in a representative patient population at a common Dutch pulmonary outpatient clinic. Clinical physiological and cellular/ molecular markers were used in the analysis.

METHODS

To carry out the cluster analysis, an imputed dataset was created from a random sample of 191 adult patients chosen from a pulmonary outpatient clinic. The selection criteria from the sample included patients with a doctor's diagnosis for asthma or COPD. Detailed assessment of patient pulmonary function, blood eosinophil counts, allergic sensitisation and smoking history was collected.

RESULTS

We observed four distinct clusters with different clinical characteristics of obstructive lung diseases. Cluster 1: patients with a history of extensive cigarette smoking, airway obstruction without signs of emphysema; cluster 2: patients with features of the emphysematous type of COPD; cluster 3: patients with characteristics of allergic asthma; cluster 4: patients with features suggesting an overlap syndrome of atopic asthma and COPD.

CONCLUSION

Four phenotypes of obstructive lung disease were identified amongst patients clinically labelled as asthma or COPD. These findings emphasize the concept that there are different phenotypes of obstructive lung diseases, including overlapping and complementary disease entities. These phenotypes of chronic airways disease can serve to tailor disease management.

Case-finding for cognitive impairment among people with Type 2 diabetes in primary care using the Test Your Memory and Self-Administered Gerocognitive Examination questionnaires: the Cog-ID study

AIM

To evaluate two cognitive tests for case-finding for cognitive impairment in older patients with Type 2 diabetes.

METHODS

Of 1243 invited patients with Type 2 diabetes, aged ≥70 years, 228 participated in a prospective cohort study. Exclusion criteria were: diagnosis of dementia; previous investigation at a memory clinic; and inability to write or read. Patients first filled out two self-administered cognitive tests (Test Your Memory and Self-Administered Gerocognitive Examination). Secondly, a general practitioner, blinded to Test Your Memory and Self-Administered Gerocognitive Examination scores, performed a structured evaluation using the Mini-Mental State Examination. Subsequently, patients suspected of cognitive impairment (on either the cognitive tests or general practitioner evaluation) and a random sample of 30% of patients not suspected of cognitive impairment were evaluated at a memory clinic. Diagnostic accuracy and area under the curve were determined for the Test Your Memory, Self-Administered Gerocognitive Examination and general practitioner evaluation compared with a memory clinic evaluation to detect cognitive impairment (mild cognitive impairment or dementia).

RESULTS

A total of 44 participants were diagnosed with cognitive impairment. The Test Your Memory and Self-Administered Gerocognitive Examination questionnaires had negative predictive values of 81 and 85%, respectively. Positive predictive values were 39 and 40%, respectively. The general practitioner evaluation had a negative predictive value of 83% and positive predictive value of 64%. The area under the curve was ~0.70 for all tests.

CONCLUSIONS

Both the tests evaluated in the present study can easily be used in case-finding strategies for cognitive impairment in patients with Type 2 diabetes in primary care. The Self-Administered Gerocognitive Examination had the best diagnostic accuracy and therefore we would have a slight preference for this test. Applying the Self-Administered Gerocognitive Examination would considerably reduce the number of patients in whom the general practitioner needs to evaluate cognitive functioning to tailor diabetes treatment.

Partial verification bias and incorporation bias affected accuracy estimates of diagnostic studies for biomarkers that were part of an existing composite gold standard

OBJECTIVE

To investigate how choice of gold standard biases estimates of sensitivity and specificity in studies reassessing the diagnostic accuracy of biomarkers that are already part of a lifetime composite gold standard (CGS).

STUDY DESIGN AND SETTING

We performed a simulation study based on the real-life example of the biomarker "protein 14-3-3" used for diagnosing Creutzfeldt-Jakob disease. Three different types of gold standard were compared: perfect gold standard "autopsy" (available in a small fraction only; prone to partial verification bias), lifetime CGS (including the biomarker under investigation; prone to incorporation bias), and "best available" gold standard (autopsy if available, otherwise CGS).

RESULTS

Sensitivity was unbiased when comparing 14-3-3 with autopsy but overestimated when using CGS or "best available" gold standard. Specificity of 14-3-3 was underestimated in scenarios comparing 14-3-3 with autopsy (up to 24%). In contrast, overestimation (up to 20%) was observed for specificity compared with CGS; this could be reduced to 0-10% when using the "best available" gold standard.

CONCLUSION

Choice of gold standard affects considerably estimates of diagnostic accuracy. Using the "best available" gold standard (autopsy where available, otherwise CGS) leads to valid estimates of specificity, whereas sensitivity is estimated best when tested against autopsy alone.

Extended prediction rule to optimise early detection of heart failure in older persons with non-acute shortness of breath: a cross-sectional study

OBJECTIVES

There is a need for a practical tool to aid general practitioners in early detection of heart failure in the elderly with shortness of breath. In this study, such a screening rule was developed based on an existing rule for detecting heart failure in older persons with a diagnosis of chronic obstructive pulmonary disease. The original rule included a history of ischaemic heart disease, body mass index, laterally displaced apex beat, heart rate, elevated N-terminal pro B-type natriuretic peptide and an abnormal ECG.

DESIGN

Cross-sectional data were used to validate, update and extend the original prediction rule according to a standardised state-of-the-art stepwise approach.

SETTING

Primary care with 30 participating general practices.

PARTICIPANTS

Community-dwelling people aged ≥ 65 years with shortness of breath on exertion.

METHODS AND RESULTS

Validation of the existing screening rule in our population showed satisfying discrimination with a concordance statistic of 0.84 (range 0.80-0.85), but poor calibration. Performance measures were most improved by adding the predictors age >75 years, peripheral oedema and systolic murmur, resulting in a concordance statistic of 0.88 (range 0.85-0.90) and a net reclassification improvement of 31%. A risk score was computed, which showed high accuracy with a negative predictive value of 87% and a positive predictive value of 73%. Evaluating the improved rule in the derivation set and an independent set of patients with type 2 diabetes aged 60 years or older showed satisfying generalisability of the rule.

CONCLUSIONS

Our rule resulted in excellent prediction of heart failure in the large domain of the elderly with shortness of breath, and would help general practitioners to select those needing echocardiography.

TRIAL REGISTRATION NUMBER

NCT01202006.

Anticipating missing reference standard data when planning diagnostic accuracy studies

Results obtained using a reference standard may be missing for some participants in diagnostic accuracy studies. This paper looks at methods for dealing with such missing data when designing or conducting a prospective diagnostic accuracy study

HPV DNA testing with cytology triage in cervical cancer screening: Influence of revealing HPV infection status

BACKGROUND

Knowledge of cervical human papillomavirus (HPV) status might influence a cytotechnician's assessment of cellular abnormalities. The authors compared original cytotechnicians' Papanicolaou (Pap) readings for which HPV status was concealed with Pap rereads for which HPV status was revealed separately for 3 screening populations.

METHODS

Previously collected cervical Pap smears and clinical data were obtained from the Canadian Cervical Cancer Screening Trial (study A), the Democratic Republic of Congo Community-Based Screening Study (study B), and the Brazilian Investigation into Nutrition and Cervical Cancer Prevention (study C). Smears were reread with knowledge of HPV status for all HPV-positive women as well as a sample of HPV-negative women. Diagnostic performance of Pap cytology was compared between original readings and rereads.

RESULTS

A total of 1767 Pap tests were reread. Among 915 rereads for HPV-positive women, the contrast between "revealed" and "concealed" Pap readings demonstrated revisions from negative to positive results for 109 women (cutoff was atypical squamous cells of undetermined significance or worse) and 124 women (cutoff was low-grade squamous intraepithelial lesions [LSIL] or worse). For a disease threshold of cervical intraepithelial neoplasia of grade 2 or worse, specificity significantly declined at the atypical squamous cells of undetermined significance cutoff for studies A (86.6% to 75.3%) and C (42.5% to 15.5%), and at the LSIL cutoff for study C (61.9% to 37.6%). Sensitivity remained nearly unchanged between readings, except in study C, in which reread performance was superior (91.3% vs 71.9% for the LSIL cutoff).

CONCLUSIONS

A reduction in the diagnostic accuracy of Pap cytology was observed when revealing patients' cervical HPV status, possibly due to a heightened awareness of potential abnormalities, which led to more false-positive results.

Cognitive Impairment in Diabetes: Rationale and Design Protocol of the Cog-ID Study

Background

Cognitive impairment frequently co-occurs with type 2 diabetes but is often undiagnosed. Cognitive impairment affects self-management leading to treatment-related complications.

Objective

The aim of this study is to develop a stepped diagnostic procedure, consisting of a screening test complemented by an evaluation by a general practitioner (GP), to detect undiagnosed cognitive impairment in older people with type 2 diabetes.

Methods

The accuracy of two self-administered cognitive tests, the “Test Your Memory” (TYM) and “Self-Administered Gerocognitive Examination” (SAGE) alone, and in combination with an evaluation by a GP will be assessed. A diagnosis of mild cognitive impairment (MCI) or dementia at a memory clinic will serve as reference standard. This cognitive impairment in diabetes (Cog-ID) study will include 513 people from primary care facilities aged ≥70 with type 2 diabetes. The participants will first fill out the TYM and SAGE tests, followed by a standardized GP evaluation for cognitive impairment, including a mini mental state examination (MMSE). Subsequently, participants suspected of cognitive impairment (on either test or the GP assessment) and a random sample of 15% (65/435) of participants without suspected cognitive impairment will be referred to the memory clinic. At the memory clinic, a medical examination, neuropsychological examination, and magnetic resonance imaging (MRI) of the brain will be performed. Participants will also fill out questionnaires assessing health status and depressive symptoms at baseline and after 6 and 24 months.

Results

This research obtained funding and ethical approval. Enrolment started in August, 2012, and all study-related activities will be completed in September, 2016.

Conclusions

With the results from this study, physicians will be able to detect cognitive impairment affecting type 2 diabetes patients through case-finding, and can use tailored care to reduce associated complications. Additionally, the results may stimulate discussions about cognitive impairment and whether early recognition is desirable.

Evaluating predictive screening for children's post-injury mental health: New data and a replication

BACKGROUND

Recommended approaches for secondary prevention of posttrauma mental health difficulties in children require empirically sound predictive screening to determine which children require more intensive monitoring or targeted intervention. Although there are several promising screening tools for injured children, none has emerged as the gold standard, and little replication data are available regarding their performance.

OBJECTIVE

To evaluate a predictive screening protocol for risk of later posttraumatic stress (PTS) and depression outcomes and address a crucial lack of replication studies by examining performance of two previously published screening tools (Screening Tool for Early Predictors of PTSD [STEPP] and Child Trauma Screening Questionnaire [CTSQ]).

METHOD

The study enrolled 290 children hospitalized after acute injury. A three-part screening protocol, including acute PTS and depression symptoms and other empirically derived risk factors, was administered in hospital as part of a stepped care study. PTS and depression symptoms and health-related quality of life (HRQoL) were assessed 6 months post-injury.

Quantifying the added value of a diagnostic test or marker

In practice, the diagnostic workup usually starts with a patient with particular symptoms or signs, who is suspected of having a particular target disease. In a sequence of steps, an array of diagnostic information is commonly documented. The diagnostic information conveyed by different results from patient history, physical examination, and subsequent testing is to varying extents overlapping and thus mutually dependent. This implies that the diagnostic potential of a test or biomarker is conditional on the information obtained from previous tests. A key question about the accuracy of a diagnostic test/biomarker is whether that test improves the diagnostic workup beyond already available diagnostic test results. This second report in a series of 4 gives an overview of several methods to quantify the added value of a new diagnostic test or biomarker, including the area under the ROC curve, net reclassification improvement, integrated discrimination improvement, predictiveness curve, and decision curve analysis. Each of these methods is illustrated with the use of empirical data. We reiterate that reporting on the relative increase in discrimination and disease classification is relevant to obtain insight into the incremental value of a diagnostic test or biomarker. We also recommend the use of decision-analytic measures to express the accuracy of an entire diagnostic workup in an informative way.

Quantifying the accuracy of a diagnostic test or marker

BACKGROUND

In recent years, increasing focus has been directed to the methodology for evaluating (new) tests or biomarkers. A key step in the evaluation of a diagnostic test is the investigation into its accuracy.

CONTENT

We reviewed the literature on how to assess the accuracy of diagnostic tests. Accuracy refers to the amount of agreement between the results of the test under evaluation (index test) and the results of a reference standard or test. The generally recommended approach is to use a prospective cohort design in patients who are suspected of having the disease of interest, in which each individual undergoes the index and same reference standard tests. This approach presents several challenges, including the problems that can arise with the verification of the index test results by the preferred reference standard test, the choice of cutoff value in case of a continuous index test result, and the determination of how to translate accuracy results to recommendations for clinical use. This first in a series of 4 reports presents an overview of the designs of single-test accuracy studies and the concepts of specificity, sensitivity, posterior probabilities (i.e., predictive values) for the presence of target disease, ROC curves, and likelihood ratios, all illustrated with empirical data from a study on the diagnosis of suspected deep venous thrombosis. Limitations of the concept of the diagnostic accuracy for a single test are also highlighted.

CONCLUSIONS

The prospective cohort design in patients suspected of having the disease of interest is the optimal approach to estimate the accuracy of a diagnostic test. However, the accuracy of a diagnostic index test is not constant but varies across different clinical contexts, disease spectrums, and even patient subgroups.

Adjusting for partial verification or workup bias in meta-analyses of diagnostic accuracy studies

A key requirement in the design of diagnostic accuracy studies is that all study participants receive both the test under evaluation and the reference standard test. For a variety of practical and ethical reasons, sometimes only a proportion of patients receive the reference standard, which can bias the accuracy estimates. Numerous methods have been described for correcting this partial verification bias or workup bias in individual studies. In this article, the authors describe a Bayesian method for obtaining adjusted results from a diagnostic meta-analysis when partial verification or workup bias is present in a subset of the primary studies. The method corrects for verification bias without having to exclude primary studies with verification bias, thus preserving the main advantages of a meta-analysis: increased precision and better generalizability. The results of this method are compared with the existing methods for dealing with verification bias in diagnostic meta-analyses. For illustration, the authors use empirical data from a systematic review of studies of the accuracy of the immunohistochemistry test for diagnosis of human epidermal growth factor receptor 2 status in breast cancer patients.

How to deal with double partial verification when evaluating two index tests in relation to a reference test?

Research into the diagnostic accuracy of clinical tests is often hampered by single or double partial verification mechanisms, that is, not all patients have their disease status verified by a reference test, neither do all patients receive all tests under evaluation (index tests). We show methods that reduce verification bias introduced when omitting data from partially tested patients. Adjustment techniques are well established when there are no missing index tests and when the reference test is 'missing at random'. However, in practice, index tests tend to be omitted, and the choice of applying a reference test may depend on unobserved variables related to disease status, that is, verification may be missing not at random (MNAR). We study double partial verification in a clinical example from reproductive medicine in which we analyse the diagnostic values of the chlamydia antibody test and the hysterosalpingography in relation to a diagnostic laparoscopy. First, we plot all possible combinations of sensitivity and specificity of both index tests in two test ignorance regions. Then, we construct models in which we impose different assumptions for the verification process. We allow for missing index tests, study the influence of patient characteristics and study the accuracy estimates if an MNAR mechanism would operate. It is shown that data on tests used in the diagnostic process of the same population are preferably studied jointly and that the influence of an MNAR verification process was limited in a clinical study where more than half of the patients did not have the reference test.