Development and internal validation of a prediction tool to aid the diagnosis of Cushing's syndrome in dogs attending primary-care practice

Frequency of low-dose dexamethasone suppression test (LDDST) response patterns and their correlation with clinicopathologic signs in dogs suspected of having Cushing's syndrome: A retrospective study

A retrospective cross-sectional study was conducted to assess the frequency of low-dose dexamethasone suppression test (LDDST) patterns in canine patients that had clinicopathologic signs consistent with Cushing's syndrome (CS). Medical records for patients of interest (N = 128) were reviewed between January 2014 and December 2020 to analyse and classify LDDST results based upon the following patterns: lack of suppression, partial suppression, complete suppression, escape, or inverse. Complete suppression, lack of suppression, partial suppression, escape, and inverse patterns were identified in 39.1%, 31.2%, 14.1%, 10.1% and 5.5% of cases respectively. LDDST results were also evaluated with respect to clinical signs, serum alkaline phosphatase (ALP) activity, urine specific gravity (USG) and adrenal ultrasonographic findings. There was no association between LDDST patterns and clinical signs (p = 0.11), increased ALP (p = 0.32), USG (p = 0.33) or adrenal ultrasonographic findings (p = 0.19). In all dogs that demonstrated complete suppression or an inverse pattern, CS was excluded by the attending clinician. The diagnosis of CS was also excluded without further exploration in 23.1%, 7.5% and 5.6% of dogs that demonstrated an escape pattern, lack of suppression and partial suppression pattern, respectively. These results suggest that the clinical significance of LDDST patterns, particularly escape and inverse patterns, are misunderstood by some clinicians, leading them to prematurely exclude the diagnosis of CS.

Development and internal validation of diagnostic prediction models using machine-learning algorithms in dogs with hypothyroidism

Introduction

Hypothyroidism can be easily misdiagnosed in dogs, and prediction models can support clinical decision-making, avoiding unnecessary testing and treatment. The aim of this study is to develop and internally validate diagnostic prediction models for hypothyroidism in dogs by applying machine-learning algorithms.

Methods

A single-institutional cross-sectional study was designed searching the electronic database of a Veterinary Teaching Hospital for dogs tested for hypothyroidism. Hypothyroidism was diagnosed based on suggestive clinical signs and thyroid function tests. Dogs were excluded if medical records were incomplete or a definitive diagnosis was lacking. Predictors identified after data processing were dermatological signs, alopecia, lethargy, hematocrit, serum concentrations of cholesterol, creatinine, total thyroxine (tT4), and thyrotropin (cTSH). Four models were created by combining clinical signs and clinicopathological variables expressed as quantitative (models 1 and 2) and qualitative variables (models 3 and 4). Models 2 and 4 included tT4 and cTSH, models 1 and 3 did not. Six different algorithms were applied to each model. Internal validation was performed using a 10-fold cross-validation. Apparent performance was evaluated by calculating the area under the receiver operating characteristic curve (AUROC).

Results

Eighty-two hypothyroid and 233 euthyroid client-owned dogs were included. The best performing algorithms were naive Bayes in model 1 (AUROC = 0.85; 95% confidence interval [CI] = 0.83-0.86) and in model 2 (AUROC = 0.98; 95% CI = 0.97-0.99), logistic regression in model 3 (AUROC = 0.88; 95% CI = 0.86-0.89), and random forest in model 4 (AUROC = 0.99; 95% CI = 0.98-0.99). Positive predictive value was 0.76, 0.84, 0.93, and 0.97 in model 1, 2, 3, and 4, respectively. Negative predictive value was 0.89, 0.89, 0.99, and 0.99 in model 1, 2, 3, and 4, respectively.

Discussion

Machine learning-based prediction models were accurate in predicting and quantifying the likelihood of hypothyroidism in dogs based on internal validation performed in a single-institution, but external validation is required to support the clinical applicability of these models.

External validation of a United Kingdom primary-care Cushing's prediction tool in a population of referred Dutch dogs

BACKGROUND

A prediction tool was developed and internally validated to aid the diagnosis of Cushing's syndrome in dogs attending UK primary-care practices. External validation is an important part of model validation to assess model performance when used in different populations.

OBJECTIVES

To assess the original prediction model's transportability, applicability, and diagnostic performance in a secondary-care practice in the Netherlands.

ANIMALS

Two hundred thirty client-owned dogs.

METHODS

Retrospective observational study. Medical records of dogs under investigation of Cushing's syndrome between 2011 and 2020 were reviewed. Dogs diagnosed with Cushing's syndrome by the attending internists and fulfilling ALIVE criteria were defined as cases, others as non-cases. All dogs were scored using the aforementioned prediction tool. Dog characteristics and predictor-outcome effects in development and validation data sets were compared to assess model transportability. Calibration and discrimination were examined to assess model performance.

RESULTS

Eighty of 230 dogs were defined as cases. Significant differences in dog characteristics were found between UK primary-care and Dutch secondary-care populations. Not all predictors from the original model were confirmed to be significant predictors in the validation sample. The model systematically overestimated the probability of having Cushing's syndrome (a = -1.10, P < .001). Calibration slope was 1.35 and discrimination proved excellent (area under the receiver operating curve = 0.83).

CONCLUSIONS AND CLINICAL IMPORTANCE

The prediction model had moderate transportability, excellent discriminatory ability, and overall overestimated probability of having Cushing's syndrome. This study confirms its utility, though emphasizes that ongoing validation efforts of disease prediction tools are a worthwhile effort.

Evaluation of a machine learning tool to screen for hypoadrenocorticism in dogs presenting to a teaching hospital

BACKGROUND

Dogs with hypoadrenocorticism (HA) have clinical signs and clinicopathologic abnormalities that can be mistaken as other diseases. In dogs with a differential diagnosis of HA, a machine learning model (MLM) has been validated to discriminate between HA and other diseases. This MLM has not been evaluated as a screening tool for a broader group of dogs.

HYPOTHESIS

An MLM can accurately screen dogs for HA.

ANIMALS

Dogs (n = 1025) examined at a veterinary hospital.

METHODS

Dogs that presented to a tertiary referral hospital that had a CBC and serum chemistry panel were enrolled. A trained MLM was applied to clinicopathologic data and in dogs that were MLM positive for HA, diagnosis was confirmed by measurement of serum cortisol.

RESULTS

Twelve dogs were MLM positive for HA and had further cortisol testing. Five had HA confirmed (true positive), 4 of which were treated for mineralocorticoid and glucocorticoid deficiency, and 1 was treated for glucocorticoid deficiency alone. Three MLM positive dogs had baseline cortisol ≤2 μg/dL but were euthanized or administered glucocorticoid treatment without confirming the diagnosis with an ACTH-stimulation test (classified as "undetermined"), and in 4, HA was ruled out (false positives). The positive likelihood ratio of the MLM was 145 to 254. All dogs diagnosed with HA by attending clinicians tested positive by the MLM.

CONCLUSIONS AND CLINICAL IMPORTANCE

This MLM can robustly predict HA status when indiscriminately screening all dogs with blood work. In this group of dogs with a low prevalence of HA, the false positive rates were clinically acceptable.

Evaluation of the ACTH stimulation test using a low dose of a depot formulation in healthy dogs and in dogs with untreated Cushing's syndrome

The sensitivity of the adrenocorticotropic hormone (ACTH) stimulation test to detect Cushing's Syndrome (CS) using a depot formulation needs to be evaluated. The aims of this study were to propose a reference interval (RI) for cortisol values 1-hour after administration of a low-dose of depot ACTH in healthy dogs, and to evaluate the sensitivity of this test to detect CS, differentiating among types of CS based on ultrasound findings. Forty-one healthy dogs (20 males, 21 females) were prospectively included. Additionally, 90 dogs with CS (31 males, 59 females) were retrospectively included. Dogs with CS were ultrasonographically classified as follows: 44 dogs with symmetrical adrenomegaly consistent with pituitary-dependent hypercortisolism (PDH), 8 dogs with unilateral adrenomegaly and atrophy of the contralateral adrenal gland or unilateral or bilateral adrenomegaly with malignancy features consistent with adrenal-dependent hypercortisolism (ADH), 34 dogs with equivocal adrenal asymmetry (EAA) and 4 dogs with normal adrenal thickness. In healthy dogs, lower and upper limit of the 95% RI for 1-hour post-ACTH cortisol concentration and their 90% confidence intervals, were 4.4 (2.7-5.8) μg/dl and 18.4 (16.5-20.0) μg/dl, respectively. Post-ACTH cortisol concentration was above the RI in 90.0% (ci95%, 76.1-100) of dogs with CS. An elevated post-ACTH cortisol concentration was detected in 95.5% (ci95%, 76.1-100) of dogs with PDH, 62.5% (ci95%, 46.1-78.9) of dogs with ADH and 88.2% (ci95%, 69.1-100) of dogs with EAA. The sensitivity of the ACTH stimulation test using a low-dose of depot ACTH in high in dogs with CS.

Comparison of methods to monitor dogs with hypercortisolism treated with trilostane

Background

The use of adrenocorticotropic hormone stimulation test as method to monitor efficacy of trilostane treatment of hypercortisolism (HC) in dogs has been questioned.

Objectives

To evaluate and compare 12 methods with which to monitor efficacy of trilostane treatment in dogs with HC.

Animals

Forty‐five client‐owned dogs with HC treated with trilostane q12h.

Methods

Prospective cross‐sectional observational study. The dogs were categorized as well‐controlled, undercontrolled, and unwell through a clinical score obtained from an owner questionnaire. The ability to correctly identify trilostane‐treatment control of dogs with HC with the following variables was evaluated: before trilostane serum cortisol (prepill), before‐ACTH serum cortisol, post‐ACTH serum cortisol, plasma endogenous ACTH concentrations, prepill/eACTH ratio, serum haptoglobin (Hp) concentration, serum alanine aminotransferase (ALT), gamma‐glutamyl transferase (γGT) and alkaline phosphatase activity, urine specific gravity, and urinary cortisol : creatinine ratio.

Results

Ninety‐four re‐evaluations of 44 dogs were included; 5 re‐evaluations of 5 unwell dogs were excluded. Haptoglobin was significantly associated with the clinical score (P < .001) and in the receiver operating characteristic analysis, Hp cutoff of 151 mg/dL correctly identified 90.0% of well‐controlled dogs (specificity) and 65.6% of undercontrolled dogs (sensitivity). Alanine aminotransferase (P = .01) and γGT (P = .009) were significantly higher in undercontrolled dogs. Cutoff of ALT and γGT greater than or equal to 86 U/L and 5.8 U/L, respectively, were significantly associated with poor control of HC by trilostane.

Conclusions and Clinical Importance

Of all the 12 variables, Hp, and to a lesser degree ALT and γGT, could be considered additional tools to the clinical picture to identify well‐controlled and undercontrolled trilostane‐treated dogs.

Machine-learning based prediction of Cushing's syndrome in dogs attending UK primary-care veterinary practice

Cushing’s syndrome is an endocrine disease in dogs that negatively impacts upon the quality-of-life of affected animals. Cushing’s syndrome can be a challenging diagnosis to confirm, therefore new methods to aid diagnosis are warranted. Four machine-learning algorithms were applied to predict a future diagnosis of Cushing's syndrome, using structured clinical data from the VetCompass programme in the UK. Dogs suspected of having Cushing's syndrome were included in the analysis and classified based on their final reported diagnosis within their clinical records. Demographic and clinical features available at the point of first suspicion by the attending veterinarian were included within the models. The machine-learning methods were able to classify the recorded Cushing’s syndrome diagnoses, with good predictive performance. The LASSO penalised regression model indicated the best overall performance when applied to the test set with an AUROC = 0.85 (95% CI 0.80–0.89), sensitivity = 0.71, specificity = 0.82, PPV = 0.75 and NPV = 0.78. The findings of our study indicate that machine-learning methods could predict the future diagnosis of a practicing veterinarian. New approaches using these methods could support clinical decision-making and contribute to improved diagnosis of Cushing’s syndrome in dogs.

Development and internal validation of a prediction tool to aid the diagnosis of Cushing's syndrome in dogs attending primary-care practice

BACKGROUND

Novel methods to aid identification of dogs with spontaneous Cushing's syndrome are warranted to optimize case selection for diagnostics, avoid unnecessary testing, and ultimately aid decision-making for veterinarians.

HYPOTHESIS/OBJECTIVES

To develop and internally validate a prediction tool for dogs receiving a diagnosis of Cushing's syndrome using primary-care electronic health records.

ANIMALS

Three hundred and ninety-eight dogs diagnosed with Cushing's syndrome and 541 noncase dogs, tested for but not diagnosed with Cushing's syndrome, from a cohort of 905 544 dogs attending VetCompass participating practices.

METHODS

A cross-sectional study design was performed. A prediction model was developed using multivariable binary logistic regression taking the demography, presenting clinical signs and some routine laboratory results into consideration. Predictive performance of each model was assessed and internally validated through bootstrap resampling. A novel clinical prediction tool was developed from the final model.

RESULTS

The final model included predictor variables sex, age, breed, polydipsia, vomiting, potbelly/hepatomegaly, alopecia, pruritus, alkaline phosphatase, and urine specific gravity. The model demonstrated good discrimination (area under the receiver operating curve [AUROC] = 0.78 [95% CI = 0.75-0.81]; optimism-adjusted AUROC = 0.76) and calibration (C-slope = 0.86). A tool was developed from the model which calculates the predicted likelihood of a dog having Cushing's syndrome from 0% (score = -13) to 96% (score = 10).

CONCLUSIONS AND CLINICAL IMPORTANCE

A tool to predict a diagnosis of Cushing's syndrome at the point of first suspicion in dogs was developed, with good predictive performance. This tool can be used in practice to support decision-making and increase confidence in diagnosis.