An evidence-based approach to test accuracy studies in gynecologic oncology: the 'STARD' checklist

Emergency Ultrasound Literature and Adherence to Standards for Reporting of Diagnostic Accuracy Criteria

BACKGROUND

Given the wide usage of emergency point-of-care ultrasound (EUS) among emergency physicians (EPs), rigorous study surrounding its accuracy is essential. The Standards for Reporting of Diagnostic Accuracy (STARD) criteria were established to ensure robust reporting methodology for diagnostic studies. Adherence to the STARD criteria among EUS diagnostic studies has yet to be reported.

OBJECTIVES

Our objective was to evaluate a body of EUS literature shortly after STARD publication for its baseline adherence to the STARD criteria.

METHODS

EUS studies in 5 emergency medicine journals from 2005-2010 were evaluated for their adherence to the STARD criteria. Manuscripts were selected for inclusion if they reported original research and described the use of 1 of 10 diagnostic ultrasound modalities designated as "core emergency ultrasound applications" in the 2008 American College of Emergency Physicians Ultrasound Guidelines. Literature search identified 307 studies; of these, 45 met inclusion criteria for review.

RESULTS

The median STARD score was 15 (interquartile range [IQR] 12-17), representing 60% of the 25 total STARD criteria. The median STARD score among articles that reported diagnostic accuracy was significantly higher than those that did not report accuracy (17 [IQR 15-19] vs. 11 [IQR 9-13], respectively; p < 0.0001). Seventy-one percent of articles met ≥50% of the STARD criteria (56-84%) and 4% met >80% of the STARD criteria.

CONCLUSIONS

Significant opportunities exist to improve methodological reporting of EUS research. Increased adherence to the STARD criteria among diagnostic EUS studies will improve reporting and improve our ability to compare outcomes.

A systematic review of the role of C4d in the diagnosis of acute antibody-mediated rejection

In this study, we conducted a systematic review of the literature to re-evaluate the role of C4d in the diagnosis of acute antibody-mediated rejection of kidney allografts. Electronic databases were searched until September 2013. Eligible studies allowed derivation of diagnostic tables for the performance of C4d by immunofluorescence or immunohistochemistry with comparison to histopathological features of acute antibody-mediated rejection and/or donor-specific antibody (DSA) assays. Of 3492 unique abstracts, 29 studies encompassing 3485 indication and 868 surveillance biopsies were identified. Assessment of C4d by immunofluorescence and immunohistochemistry exhibited slight to moderate agreement with glomerulitis, peritubular capillaritis, solid-phase DSA assays, DSA with glomerulitis, and DSA with peritubular capillaritis. The sensitivity and specificity of C4d varied as a function of C4d and comparator test thresholds. Prognostically, the presence of C4d was associated with inferior allograft survival compared with DSA or histopathology alone. Thus, our findings support the presence of complement-dependent and -independent phenotypes of acute antibody-mediated rejection. Whether the presence of C4d in combination with histopathology or DSA should be considered for the diagnosis of acute antibody-mediated rejection warrants further study.

The quality of reporting of primary test accuracy studies in obstetrics and gynaecology: application of the STARD criteria

Background

In obstetrics and gynaecology there has been a rapid growth in the development of new tests and primary studies of their accuracy. It is imperative that such studies are reported with transparency allowing the detection of any potential bias that may invalidate the results. The objective of this study was to determine the quality of reporting in diagnostic test accuracy studies in obstetrics and gynaecology using the Standards for Reporting of Diagnostic Accuracy - STARD checklist.

Methods

The included studies of ten systematic reviews were assessed for compliance with each of the reporting criteria. Using appropriate statistical tests we investigated whether there was an improvement in reporting quality since the introduction of the STARD checklist, whether a correlation existed between study sample size, country of origin of study and reporting quality.

Results

A total of 300 studies were included (195 for obstetrics, 105 for gynaecology). The overall reporting quality of included studies to the STARD criteria was poor. Obstetric studies reported adequately > 50% of the time for 62.1% (18/29) of the items while gynaecologic studies did the same 51.7% (15/29). There was a greater mean compliance with STARD criteria in the included obstetric studies than the gynaecological (p < 0.0001). There was a positive correlation, in both obstetrics (p < 0.0001) and gynaecology (p = 0.0123), between study sample size and reporting quality. No correlation between geographical area of publication and compliance with the reporting criteria could be demonstrated.

Conclusions

The reporting quality of papers in obstetrics and gynaecology is improving. This may be due to initiatives such as the STARD checklist as well as historical progress in awareness among authors of the need to accurately report studies. There is however considerable scope for further improvement.

Quality of reporting of diagnostic accuracy studies: no change since STARD statement publication--before-and-after study

PURPOSE

To determine the quality of reporting of diagnostic accuracy studies before and after the Standards for Reporting of Diagnostic Accuracy (STARD) statement publication and to determine whether there is a difference in the quality of reporting by comparing STARD (endorsing) and non-STARD (nonendorsing) journals.

MATERIALS AND METHODS

Diagnostic accuracy studies were identified by hand searching six STARD and six non-STARD journals for 2001, 2002, 2004, and 2005. Diagnostic accuracy studies (n = 240) were assessed by using a checklist of 13 of 25 STARD items. The change in the mean total score on the modified STARD checklist was evaluated with analysis of covariance. The change in proportion of times that each individual STARD item was reported before and after STARD statement publication was evaluated (chi(2) tests for linear trend).

RESULTS

With mean total score as dependent factor, analysis of covariance showed that the interaction between the two independent factors (STARD or non-STARD journal and year of publication) was not significant (F = 0.664, df = 3, partial eta(2) = 0.009, P = .58). Additionally, the frequency with which individual items on the STARD checklist were reported before and after STARD statement publication has remained relatively constant, with little difference between STARD and non-STARD journals.

CONCLUSION

After publication of the STARD statement in 2003, the quality of reporting of diagnostic accuracy studies remained similar to pre-STARD statement publication levels, and there was no meaningful difference (ie, one additional item on the checklist of 13 of 25 STARD items being reported) in the quality of reporting between those journals that published the STARD statement and those that did not.

A systematic review of tests for lymph node status in primary endometrial cancer

BACKGROUND

The lymph node status of a patient is a key determinate in staging, prognosis and adjuvant treatment of endometrial cancer. Despite this, the potential additional morbidity associated with lymphadenectomy makes its role controversial. This study systematically reviews the accuracy literature on sentinel node biopsy; ultra sound scanning, magnetic resonance imaging (MRI) and computer tomography (CT) for determining lymph node status in endometrial cancer.

METHODS

Relevant articles were identified form MEDLINE (1966-2006), EMBASE (1980-2006), MEDION, the Cochrane library, hand searching of reference lists from primary articles and reviews, conference abstracts and contact with experts in the field. The review included 18 relevant primary studies (693 women). Data was extracted for study characteristics and quality. Bivariate random-effect model meta-analysis was used to estimate diagnostic accuracy of the various index tests.

RESULTS

MRI (pooled positive LR 26.7, 95% CI 10.6 - 67.6 and negative LR 0.29 95% CI 0.17 - 0.49) and successful sentinel node biopsy (pooled positive LR 18.9 95% CI 6.7 - 53.2 and negative LR 0.22, 95% CI 0.1 - 0.48) were the most accurate tests. CT was not as accurate a test (pooled positive LR 3.8, 95% CI 2.0 - 7.3 and negative LR of 0.62, 95% CI 0.45 - 0.86. There was only one study that reported the use of ultrasound scanning.

CONCLUSION

MRI and sentinel node biopsy have shown similar diagnostic accuracy in confirming lymph node status among women with primary endometrial cancer than CT scanning, although the comparisons made are indirect and hence subject to bias. MRI should be used in preference, in light of the ASTEC trial, because of its non invasive nature.

Diagnostic accuracy of tests for lymph node status in primary cervical cancer: a systematic review and meta-analysis

BACKGROUND

Lymph node status is the key to determining the prognosis and treatment of cervical cancer. However, it cannot be assessed clinically, and testing for nodal metastasis is controversial. We sought to systematically review the diagnostic accuracy literature on sentinel node biopsy, positron emission tomography, magnetic resonance imaging and computed tomography to evaluate the accuracy of each index test in determining lymph node status in patients with cervical cancer.

METHODS

We searched MEDLINE (1966-2006), EMBASE (1980-2006), Medion (1980-2006) and the Cochrane library (Issue 2, 2006) for relevant articles. We also manually searched the reference lists from primary articles and reviews, and we contacted experts in the field for conference abstracts and unpublished studies. We performed random-effects meta-analysis of accuracy indices, and we performed meta-regression analysis to test the effect of study quality on diagnostic accuracy and to identify other sources of heterogeneity.

RESULTS

We included 72 relevant primary studies, involving a total of 5042 women, in our analysis. We found that, in determining lymph node status, sentinel node biopsy had a pooled positive likelihood ratio of 40.8 (95% confidence interval [CI] 24.6-67.6) and a pooled negative likelihood ratio of 0.18 (95% CI 0.14-0.24). The pooled positive likelihood ratios (and 95% CI) were 15.3 (7.9-29.6) for positron emission tomography, 6.4 (4.9-8.3) for magnetic resonance imaging and 4.3 (3.0-6.2) for computed tomography. The pooled negative likelihood ratios (and 95% CIs) were 0.27 (0.11-0.66) for positron emission tomography, 0.50 (0.39-0.64) for magnetic resonance imaging and 0.58 (0.48-0.70) for computed tomography. Using a 27% pretest probability of lymph node metastasis among all cases (regardless of stage), we found that a positive sentinel node biopsy result increased post-test probability to 94% (95% CI 90%-96%), whereas a positive finding on positron emission tomography increased it to 85% (75%-92%).

INTERPRETATION

Sentinel node biopsy has greater accuracy in determining lymph node status among women with primary cervical cancer than current commonly used imaging methods.

The diagnostic accuracy of magnetic resonance imaging in detecting cervical involvement in endometrial cancer

OBJECTIVES

The objective of this study was to determine the diagnostic accuracy of magnetic resonance imaging (MRI) in detecting cervical involvement by endometrial cancer.

METHODS

A retrospective accuracy study of 135 consecutive women who underwent preoperative MRI and surgery for endometrial cancer at a single gynaecological cancer centre between 1st February 2003 and 30th November 2004.

RESULTS

For the detection of any cervical involvement by MRI, the sensitivity was 72%, specificity 93.2%, positive predictive value (PPV) 89.8%, negative predictive value (NPV) 80.2%, positive likelihood ratio (+LR) 10.7 and negative likelihood ratio (-LR) 0.3. When cervical stromal invasion was considered alone, the sensitivity was 84.4%, specificity 87.4%, PPV 67.5%, NPV 94.7%, +LR 6.7 and -LR 0.18.

CONCLUSION

We believe that MRI is able to accurately predict cervical involvement in endometrial cancer and allows a decision to be made on the type of hysterectomy to be offered.

Quality of reporting of test accuracy studies in reproductive medicine: impact of the Standards for Reporting of Diagnostic Accuracy (STARD) initiative

OBJECTIVE

To evaluate the extent to which test accuracy studies published in two leading reproductive medicine journals in the years 1999 and 2004 adhered to the Standards for Reporting of Diagnostic Accuracy (STARD) initiative parameters, and to explore whether the introduction of the STARD statement has led to an improved quality of reporting.

DESIGN

Structured literature search. Articles that reported on the diagnostic performance of a test in comparison with a reference standard were eligible for inclusion. For each article we scored how well the 25 items of the STARD checklist were reported. These items deal with the study question, study participants, study design, test methods, reference standard, statistical methods, reporting of results, and conclusions. We calculated the total number of reported STARD items per article, summary scores for each STARD item, and the average number of reported STARD items per publication year.

SETTING

Not applicable.

PATIENT(S)

Not applicable.

INTERVENTION(S)

Not applicable.

MAIN OUTCOME MEASURE(S)

Quality of reporting.

RESULT(S)

We found 24 studies reporting on test accuracy in reproductive medicine in 1999 and 27 studies in 2004. The mean number of reported STARD items for articles published in 1999 was 12.1 +/- 3.3 (range 6.5-20) and 12.4 +/- 3.2 (range 7-17.5) in 2004, after publication of the STARD statement. Overall, less than half of the studies reported adequately on 50% or more of the STARD items. The reporting of individual items showed a wide variation. There was no significant improvement in mean number of reported items for the articles published after the introduction of the STARD statement.

CONCLUSION(S)

Authors of test accuracy studies in the two leading fertility journals poorly report the design, conduct, methodology, and statistical analysis of their study. Strict adherence to the STARD guidelines should be encouraged.

A systematic review of the accuracy of diagnostic tests for inguinal lymph node status in vulvar cancer

OBJECTIVE

To determine the accuracy of minimally and non-invasive tests to assess the groin node status in squamous cell vulvar cancer.

METHODS

A systematic review of published research from 1979 to 2004 that compares the results of tests to determine groin node status with histology at inguinofemoral lymphadenectomy was made. Studies included in the review were those that compared the index test to the standard surgical intervention of inguinofemoral lymphadenectomy and allowed the construction of two-by-two tables. From these tables, sensitivity, specificity, and the likelihood ratios (with 95% confidence intervals) were reported and, where feasible, meta-analysis was used to pool results for each test separately. Sentinel node biopsy using technetium-99m-labelled nanocolloid ((99m)Tc) had a pooled sensitivity and negative LR of 97% (91-100 95% CI) and 0.12 (0.053-0.28 95% CI), respectively, and was the most accurate test reviewed.

CONCLUSION

Five diagnostic tests were identified in a total of 29 studies (961 groins). Although the studies were small and the design often poor, this represents the best summary of the data to date. Sentinel node identification using (99m)Tc appeared to be the most promising test for accurately excluding lymph node metastases in squamous cell vulvar cancer and potentially reducing the radicality of surgery. Its efficacy as a tool in reducing the need for radical surgery and associated patient morbidity without reducing survival needs further assessment probably in a randomised control trial.