Sample size and precision of estimates in studies of depression screening tool accuracy: A meta-research review of studies published in 2018-2021

'Optimal' cutoff selection in studies of depression screening tool accuracy using the PHQ-9, EPDS, or HADS-D: A meta-research study

OBJECTIVES

Optimal cutoff thresholds are selected to separate 'positive' from 'negative' screening results. We evaluated how depression screening tool studies select optimal cutoffs.

METHODS

We included studies from previously conducted meta-analyses of Patient Health Questionnaire-9, Edinburgh Postnatal Depression Scale, or Hospital Anxiety and Depression Scale-Depression accuracy. Outcomes included whether an optimal cutoff was selected, method used, recommendations made, and reporting guideline and protocol citation.

RESULTS

Of 212 included studies, 172 (81%) attempted to identify an optimal cutoff, and 147 of these 172 (85%) reported one or more methods. Methods were heterogeneous with Youden's J (N = 35, 23%) most common. Only 23 of 147 (16%) studies described a rationale for their method. Rationales focused on balancing sensitivity and specificity without describing why desirable. 131 of 172 studies (76%) identified an optimal cutoff other than the standard; most did not make use recommendations (N = 56; 43%) or recommended using a non-standard cutoff (N = 53; 40%). Only 4 studies cited a reporting guideline, and 4 described a protocol with optimal cutoff selection methods, but none used the protocol method in the published study.

CONCLUSIONS

Research is needed to guide how selection of cutoffs for depression screening tools can be standardized and reflect clinical considerations.

Transparency and completeness of reporting of depression screening tool accuracy studies: A meta-research review of adherence to the Standards for Reporting of Diagnostic Accuracy Studies statement

OBJECTIVES

Accurate and complete study reporting allows evidence users to critically appraise studies, evaluate possible bias, and assess generalizability and applicability. We evaluated the extent to which recent studies on depression screening accuracy were reported consistent with Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement requirements.

METHODS

MEDLINE was searched from January 1, 2018 through May 21, 2021 for depression screening accuracy studies.

RESULTS

106 studies were included. Of 34 STARD items or sub-items, the number of adequately reported items per study ranged from 7 to 18 (mean = 11.5, standard deviation [SD] = 2.5; median = 11.5), and the number inadequately reported ranged from 3 to 17 (mean = 10.1, SD = 2.5; median = 10.0). There were eight items adequately reported, seven partially reported, 11 inadequately reported, and four not applicable in ≥50% of studies; the remaining four items had mixed reporting. Items inadequately reported in ≥70% of studies related to the rationale for index test cut-offs examined, missing data management, analyses of variability in accuracy results, sample size determination, participant flow, study registration, and study protocol.

CONCLUSION

Recently published depression screening accuracy studies are not optimally reported. Journals should endorse and implement STARD adherence.