Evaluation of inter-rater agreement of the clinical signs used to diagnose bovine respiratory disease in individually housed veal calves

Performance of various interpretations of clinical scoring systems for diagnosis of respiratory disease in dairy calves in a temperate climate using Bayesian latent class analysis

Bovine respiratory disease (BRD) presents a challenge to farmers all over the globe, not only because it can have significant impacts on welfare and productivity, but also because diagnosis can prove challenging. Several clinical scoring systems have been developed to aid farmers in making consistent early diagnosis, 2 examples being the Wisconsin (WCS) and the California (CALIF) systems. Neither of these systems were developed in or for use in a temperate environment. As environment may lead to changes in BRD presentation, the weightings and cutoffs designed for one environmental presentation of BRD may not be appropriate when used in a temperate climate. Additionally, the interpretation of the scores recorded varies between studies; this may also influence conclusions. Hence, the objective of this work was to investigate the sensitivity (Se) and specificity (Sp) of these tests in a temperate climate and investigate the influence of varying the interpretation on the performance of the WCS. In this prospective study, 98 commercial spring-calving dairy farms were recruited (40 randomly, 58 targeted) and visited. Thoracic ultrasound and WCS were performed on 20 randomly sampled calves between 4 and 6 wk of age on each farm. On a subset of 32 farms, the CALIF score was also undertaken. The data were then used in a hierarchical Bayesian latent class model to estimate the Se and Sp of 5 different interpretations of the Wisconsin clinical score and 1 interpretation of the California clinical score. In total, 1,936 calves were examined. The Se of the Wisconsin score varied from 0.336 to 0.577 depending on the interpretation used, and the Sp varied from 0.943 to 0.977. The Se of the California score was 0.563 (95% Bayesian credible interval [BCI]: 0.452, 0.681) and the Sp was 0.919 (95% BCI: 0.899, 0.937). In conclusion, the performances of the clinical scores in a temperate environment were similar to previously published work from more extreme climates; however, the performance varied widely depending on the score interpretation. Authors should justify their use of a particular clinical score interpretation to improve clarity in publications.

Comparison of physiological markers, behavior monitoring, and clinical illness scoring as indicators of an inflammatory response in beef cattle

Clinical illness (CI) scoring using visual observation is the most widely applied method of detecting respiratory disease in cattle but has limited effectiveness in practice. In contrast, body-mounted sensor technology effectively facilitates disease detection. To evaluate whether a combination of movement behavior and CI scoring is effective for disease detection, cattle were vaccinated to induce a temporary inflammatory immune response. Cattle were evaluated before and after vaccination to identify the CI variables that are most indicative of sick cattle. Respiratory rate (H2 = 43.08, P < 0.0001), nasal discharge (H2 = 8.35, P = 0.015), and ocular discharge (H2 = 16.38, P = 0.0003) increased after vaccination, and rumen fill decreased (H2 = 20.10, P < 0.0001). Locomotor activity was measured via leg-mounted sensors for the four days preceding and seven days following vaccination. A statistical model that included temperature, steps, lying time, respiratory rate, rumen fill, head position, and excess saliva was developed to distinguish between scores from before and after vaccination with a sensitivity of 0.898 and specificity of 0.915. Several clinical illness signs were difficult to measure in practice. Binoculars were required for scoring respiratory rate and eye-related metrics, and cattle had to be fitted with colored collars for individual identification. Scoring each animal took up to three minutes in a small research pen; therefore, technologies that can automate both behavior monitoring and identification of clinical illness signs are key to improving capacity for BRD detection and treatment.

Accuracy and inter-rater agreement among practitioners using quick thoracic ultrasonography to diagnose calf pneumonia

BACKGROUND

Thoracic ultrasonography (TUS) is a commonly used tool for on-farm detection of pneumonia in calves. Different scanning methods have been described, but the performance of novice practitioners after training has not been documented.

METHODS

In this study, 38 practitioners performed quick TUS (qTUS) on 18-23 calves each. Pneumonia was defined as lung consolidation 1 cm or more in depth. Diagnostic parameters (accuracy [Acc], sensitivity [Se] and specificity [Sp]) were compared to those of an experienced operator. Cohen's kappa and Krippendorff's alpha (Kalpha) were determined. The potential effects of training and exam sessions on performance were evaluated.

RESULTS

The average relative Se and Sp were 0.66 (standard deviation [SD] = 0.26; minimum [Min.]-Maximum [Max.] = 0-1) and 0.71 (SD = 0.19; Min.-Max. = 0.25-1), respectively. The average relative Acc was 0.73 (SD = 0.11; Min.-Max. = 0.52-0.96). Over all sessions, Cohen's kappa averaged 0.40 (SD = 0.24; Min.-Max. = 0.014-0.90) and Kalpha was 0.24 (95% confidence interval [CI]: 0.20-0.27), indicating 'fair' agreement. Calf age and housing influenced Se and Sp. Supervised practical training improved Se by 17.5% (95% CI: 0.01-0.34).

LIMITATIONS

The separate effects of calf age and housing could not be determined.

CONCLUSION

This study showed that qTUS, like any other clinical skill, has a learning curve, and variability in performance can be substantial. Adequate training and certification of one's skill are recommended to assure good diagnostic accuracy.

Strategies for Bovine Respiratory Disease (BRD) Diagnosis and Prognosis: A Comprehensive Overview

Despite significant advances in vaccination strategies and antibiotic therapy, bovine respiratory disease (BRD) continues to be the leading disease affecting the global cattle industry. The etiology of BRD is complex, often involving multiple microbial agents, which lead to intricate interactions between the host immune system and pathogens during various beef production stages. These interactions present environmental, social, and geographical challenges. Accurate diagnosis is essential for effective disease management. Nevertheless, correct identification of BRD cases remains a daunting challenge for animal health technicians in feedlots. In response to current regulations, there is a growing interest in refining clinical diagnoses of BRD to curb the overuse of antimicrobials. This shift marks a pivotal first step toward establishing a structured diagnostic framework for this disease. This review article provides an update on recent developments and future perspectives in clinical diagnostics and prognostic techniques for BRD, assessing their benefits and limitations. The methods discussed include the evaluation of clinical signs and animal behavior, biomarker analysis, molecular diagnostics, ultrasound imaging, and prognostic modeling. While some techniques show promise as standalone diagnostics, it is likely that a multifaceted approach-leveraging a combination of these methods-will yield the most accurate diagnosis of BRD.

Literature Review of the Principal Diagnostic Tests to Detect Bovine Respiratory Disease in Pre-Weaned Dairy and Veal Calves

Bovine respiratory disease (BRD) is an infection of the upper and lower respiratory tract, characterized by an inflammation of the lung. Different diagnostic tests can be used to detect BRD, including clinical respiratory scoring systems, thoracic auscultation, and imaging tests like thoracic ultrasonography and thoracic radiography. Although commonly used, none of these diagnostic tests are perfect for detecting BRD. This article reviews the advantages and drawbacks of these techniques and their performance in detecting BRD in pre-weaned dairy and veal calves.

Short communication: Circadian variations and day-to-day variability of clinical signs used for the early diagnosis of pneumonia within and between calves

For rational antimicrobial use, a timely and correct diagnosis of bovine respiratory disease, especially pneumonia, in calves is required. Current approaches often rely on clinical signs observed at a single time point, and do not take potential diurnal patterns in the manifestation of these clinical signs into account. Therefore, the aim of this pilot study was to investigate how clinical signs utilized for the (early) detection of pneumonia vary both within and between calves, throughout the day and across days. A longitudinal study was conducted in which 36 pre-weaned Holstein-Friesian calves were clinically examined eight times over the course of 48 h. The following parameters were considered: respiratory rate, type of respiration, dyspnea, stridor, induced cough (trachea reflex), spontaneous cough, eye and nasal discharge, ear positions, head tilt, rectal temperature, diarrhea, milk residue, body posture, Wisconsin and Davis BRD scorecard. The advent of thoracic ultrasonography (TUS) enables detection of (sub)clinical pneumonia in a more reliable way, compared to the diagnosis based solely on clinical signs. In this study, 14% (5/36) of the calves had an ultrasound confirmed pneumonia (consolidation ≥1 cm in depth). No variations were observed in the prevalence of clinical signs at the various time points of the day. However, we did observe a difference in the manifestation of clinical signs in individual calves (intra) and between (inter) them. Due to the significant intra-calf variability, diagnosing pneumonia based solely on a single observation of clinical signs, is likely to be insufficient. Hence, misdiagnosis might lead to incorrect use of antimicrobials.

Initial validation of an intelligent video surveillance system for automatic detection of dairy cattle lameness

Introduction

Lameness is a major welfare challenge facing the dairy industry worldwide. Monitoring herd lameness prevalence, and early detection and therapeutic intervention are important aspects of lameness control in dairy herds. The objective of this study was to evaluate the performance of a commercially available video surveillance system for automatic detection of dairy cattle lameness (CattleEye Ltd).

Methods

This was achieved by first measuring mobility score agreement between CattleEye and two veterinarians (Assessor 1 and Assessor 2), and second, by investigating the ability of the CattleEye system to detect cows with potentially painful foot lesions. We analysed 6,040 mobility scores collected from three dairy farms. Inter-rate agreement was estimated by calculating percentage agreement (PA), Cohen's kappa (κ) and Gwet's agreement coefficient (AC). Data regarding the presence of foot lesions were also available for a subset of this dataset. The ability of the system to predict the presence of potentially painful foot lesions was tested against that of Assessor 1 by calculating measures of accuracy, using lesion records during the foot trimming sessions as reference.

Results

In general, inter-rater agreement between CattleEye and either human assessor was strong and similar to that between the human assessors, with PA and AC being consistently above 80% and 0.80, respectively. Kappa agreement between CattleEye and the human scorers was in line with previous studies (investigating agreement between human assessors) and within the fair to moderate agreement range. The system was more sensitive than Assessor 1 in identifying cows with potentially painful lesions, with 0.52 sensitivity and 0.81 specificity compared to the Assessor's 0.29 and 0.89 respectively.

Discussion

This pilot study showed that the CattleEye system achieved scores comparable to that of two experienced veterinarians and was more sensitive than a trained veterinarian in detecting painful foot lesions.

Recording of calf diseases for potential use in breeding programs: a case study on calf respiratory illness and diarrhea

Calf diseases remain a challenge for dairy producers from both an economic and welfare perspective. Genetically selecting for disease resistance in calves is a promising approach that could contribute to sustainable dairy farming. Genetic evaluations, however, require well-defined and consistently recorded phenotypes to be successful. Therefore, this study aimed to understand the current state of calf disease recording on Ontario farms. Calf disease records of respiratory illness and diarrhea were available from the national milk recording organization (Lactanet Canada, Guelph, Ontario, Canada) from 2009 to 2020. A case study was conducted to describe calf disease diagnoses and recording practices by surveying a subset of 13 Ontario dairy producers. The percentage of milk recorded farms that recorded calf respiratory illness and calf diarrhea increased from 2.6% in 2009 to 11.1% in 2020. Potential sources of data loss were identified along the information chain from farm to genetic evaluation database. Clear definitions and thresholds to diagnose calf disease, standard operating procedures for data recording, as well as a data transfer pipeline, which includes exchange formats, are needed to facilitate the inclusion of calf health traits in genetic evaluations.

Prevalence of respiratory disease in Irish preweaned dairy calves using hierarchical Bayesian latent class analysis

Introduction

Bovine respiratory disease (BRD) has a significant impact on the health and welfare of dairy calves. It can result in increased antimicrobial usage, decreased growth rate and reduced future productivity. There is no gold standard antemortem diagnostic test for BRD in calves and no estimates of the prevalence of respiratory disease in seasonal calving dairy herds.

Methods

To estimate BRD prevalence in seasonal calving dairy herds in Ireland, 40 dairy farms were recruited and each farm was visited once during one of two calving seasons (spring 2020 & spring 2021). At that visit the prevalence of BRD in 20 calves between 4 and 6 weeks of age was determined using thoracic ultrasound score (≥3) and the Wisconsin respiratory scoring system (≥5). Hierarchical Bayesian latent class analysis was used to estimate the calf-level true prevalence of BRD, and the within-herd prevalence distribution, accounting for the imperfect nature of both diagnostic tests.

Results

In total, 787 calves were examined, of which 58 (7.4%) had BRD as defined by a Wisconsin respiratory score ≥5 only, 37 (4.7%) had BRD as defined by a thoracic ultrasound score of ≥3 only and 14 (1.8%) calves had BRD based on both thoracic ultrasound and clinical scoring. The primary model assumed both tests were independent and used informed priors for test characteristics. Using this model the true prevalence of BRD was estimated as 4%, 95% Bayesian credible interval (BCI) (1%, 8%). This prevalence estimate is lower or similar to those found in other dairy production systems. Median within herd prevalence varied from 0 to 22%. The prevalence estimate was not sensitive to whether the model was constructed with the tests considered conditionally dependent or independent. When the case definition for thoracic ultrasound was changed to a score ≥2, the prevalence estimate increased to 15% (95% BCI: 6%, 27%).

Discussion

The prevalence of calf respiratory disease, however defined, was low, but highly variable, in these seasonal calving dairy herds.

Diagnostic accuracy of Wisconsin and California scoring systems to detect bovine respiratory disease in preweaning dairy calves under subtropical environmental conditions

Bovine respiratory disease (BRD) is a multifactorial disease which causes short- and long-term negative effects. Early detection is crucial for a prompt response to therapy, as well as to decrease mortality risk. Clinical scoring systems have been developed mostly in North America for screening calves at risk or suspected of having BRD, and these tools have also been applied in subtropical and tropical countries. However, it has been unknown whether these scoring systems had the same accuracy in tropical environmental conditions. Therefore, this study evaluated the accuracy of 4 different field techniques, as well as serum haptoglobin (HAP), to diagnose BRD in Holstein dairy calves in subtropical conditions. The tests used to diagnose BRD were thoracic ultrasound (TUS; positive if consolidation depth ≥1 cm), thoracic auscultation (AUSC; positive if crackles, wheezes, or silent areas were present), Wisconsin score (WISC; ≥2 categories with scores of ≥2), and California score (CALIF; positive if total score ≥5). Also, HAP was measured and classified as positive if ≥15 mg/dL. Heifers between 30 d of age and weaning (n = 482), residing on 17 commercial dairies in São Paulo state, were enrolled in this study. Bayesian latent class models were used with informative priors to evaluate the accuracy of TUS, AUSC, and HAP, and noninformative priors for the accuracy of WISC and CALIF. The percentage of calves positive for each test on each farm ranged from 0 to 56% for WISC, 11-51% for CALIF, 0-72% for TUS, 0-32% for AUSC, and 0-100% for HAP. The sensitivity (Se; 95% credible interval) and specificity (Sp) for WISC were 77.9% (64.8-90.2) and 81.9% (76.3-88.2). For CALIF, the Se was 67.1% (53.6-80.1) and Sp 79.1% (73.9-84.6). For TUS Se was 59.8% (46.5-73.1) and Sp was 84.8% (80.0-89.5), and for AUSC, Se was 58.8% (41.3-79.8) and Sp was 98.6% (95.7-99.9). The Se and Sp of HAP was 67.6% (55.3-78.8) and 46.7% (41.4-52.2), respectively. The performance of the scoring systems was similar to, or better than, the performance found in North American studies, despite the fact that calves were in a tropical environment.

Clinical Scores in Veterinary Medicine: What Are the Pitfalls of Score Construction, Reliability, and Validation? A General Methodological Approach Applied in Cattle

Simple Summary

Clinical scores are practical tools that can be used in the daily management of cattle. Score building and validation are a challenge involving various methodological and statistical issues. This article provides a specific framework for clinical score building where the target condition can be assessed directly or indirectly. Practical examples are given throughout the manuscript in order to build new scores or to assess score robustness.

Abstract

Clinical scores are commonly used for cattle. They generally contain a mix of categorical and numerical variables that need to be assessed by scorers, such as farmers, animal caretakers, scientists, and veterinarians. This article examines the key concepts that need to be accounted for when developing the test for optimal outcomes. First, the target condition or construct that the scale is supposed to measure should be defined, and if possible, an adequate proxy used for classification should be determined. Then, items (e.g., clinical signs) of interest that are either caused by the target condition (reflective items) or that caused the target condition (formative items) are listed, and reliable items (inter and intra-rater reliability) are kept for the next step. A model is then developed to determine the relative weight of the items associated with the target condition. A scale is then built after validating the model and determining the optimal threshold in terms of sensitivity (ability to detect the target condition) and specificity (ability to detect the absence of the target condition). Its robustness to various scenarios of the target condition prevalence and the impact of the relative cost of false negatives to false positives can also be assessed to tailor the scale used based on specific application conditions.

The Agreement between Feline Pancreatic Lipase Immunoreactivity and DGGR-Lipase Assay in Cats-Preliminary Results

Simple Summary

Feline pancreatitis is a common disease with diagnostics requiring a combination of clinical, laboratory and imaging methods. Measuring of pancreatic lipase concentration is the laboratory mainstay of feline pancreatitis diagnostics. It can be done using immunoenzymatic assay (feline pancreatic immunoreactivity, fPLI) or colorimetric assay based on hydrolyzing a special substrate (DGGR) to a color product. A DGGR-lipase assay is manufactured by various companies. Studies carried out in cats on the DGGR-lipase assay of a single manufacturer have shown a high agreement with fPLI, however studies in dogs indicate that the performance of assays of different manufacturers may vary considerably. One of the largest Polish veterinary laboratories working 24/7 in the largest Polish cities offers the DGGR-lipase assay of a large Polish laboratory solutions provider. The values obtained in this assay are higher than those yielded by the traditional DGGR-lipase assay validated in studies carried out so far. Therefore, we decided to evaluate the agreement between the DGGR-lipase assay of the Polish manufacturer and fPLI. The agreement proved to be as high as for the traditional assay.

Abstract

The colorimetric catalytic assay based on the use of 1,2-o-dilauryl-rac-glycero-3-glutaric acid-(6′-methylresorufin) (DGGR) ester as a substrate for pancreatic lipase activity is commonly used for the diagnosis of pancreatitis in dogs and cats. Even though the assay has generally been shown to yield consistent results with feline pancreatic lipase immunoreactivity (fPLI) assay, the agreement may vary between assays of different manufacturers. In this study, the chance-corrected agreement between a DGGR-lipase assay offered by one of the biggest providers of diagnostic solutions in Poland and fPLI assay was investigated. The study was carried out on 50 cats in which DGGR-lipase activity and fPLI were tested in the same blood sample. The chance-corrected agreement was determined using Gwet’s AC₁ coefficient separately for the fPLI assay’s cut-off values of >3.5 μg/L and >5.3 μg/L. The DGGR-lipase activity significantly positively correlated with fPLI (R_s = 0.665; CI 95%: 0.451, 0.807, p < 0.001). The chance-corrected agreement between the fPLI assay and DGGR-lipase assay differed considerably depending on the cut-off values of the DGGR-lipase assay. When the cut-off value reported in the literature (>26 U/L) was used, it was poor to fair. It was moderate at the cut-off value recommended by the laboratory (>45 U/L), and good at the cut-off value recommended by the assay’s manufacturer (>60 U/L). The highest agreement was obtained between the fPLI assay at the cut-off value of 3.5 μg/L and the DGGR-lipase assay at the cut-off value of 55 U/L (AC₁ = 0.725; CI 95%: 0.537, 0.914) and between the fPLI assay at the cut-off value of 5.3 μg/L and the DGGR-lipase assay at the cut-off value of 70 U/L (AC₁ = 0.749; CI 95%: 0.577, 0.921). The study confirms that the chance-corrected agreement between the two assays is good. Prospective studies comparing both assays to a diagnostic gold standard are needed to determine which of them is more accurate.