Ruijter BEW, Bik CA, Schofield I, Niessen SJM. External validation of a United Kingdom primary-care Cushing's prediction tool in a population of referred Dutch dogs.
J Vet Intern Med 2023;
37:2052-2063. [PMID:
37665189 PMCID:
PMC10658492 DOI:
10.1111/jvim.16848]
[Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Accepted: 08/23/2023] [Indexed: 09/05/2023] Open
Abstract
BACKGROUND
A prediction tool was developed and internally validated to aid the diagnosis of Cushing's syndrome in dogs attending UK primary-care practices. External validation is an important part of model validation to assess model performance when used in different populations.
OBJECTIVES
To assess the original prediction model's transportability, applicability, and diagnostic performance in a secondary-care practice in the Netherlands.
ANIMALS
Two hundred thirty client-owned dogs.
METHODS
Retrospective observational study. Medical records of dogs under investigation of Cushing's syndrome between 2011 and 2020 were reviewed. Dogs diagnosed with Cushing's syndrome by the attending internists and fulfilling ALIVE criteria were defined as cases, others as non-cases. All dogs were scored using the aforementioned prediction tool. Dog characteristics and predictor-outcome effects in development and validation data sets were compared to assess model transportability. Calibration and discrimination were examined to assess model performance.
RESULTS
Eighty of 230 dogs were defined as cases. Significant differences in dog characteristics were found between UK primary-care and Dutch secondary-care populations. Not all predictors from the original model were confirmed to be significant predictors in the validation sample. The model systematically overestimated the probability of having Cushing's syndrome (a = -1.10, P < .001). Calibration slope was 1.35 and discrimination proved excellent (area under the receiver operating curve = 0.83).
CONCLUSIONS AND CLINICAL IMPORTANCE
The prediction model had moderate transportability, excellent discriminatory ability, and overall overestimated probability of having Cushing's syndrome. This study confirms its utility, though emphasizes that ongoing validation efforts of disease prediction tools are a worthwhile effort.
Collapse