Evaluation of Inter-Observer Reliability of Animal Welfare Indicators: Which Is the Best Index to Use?

Refinement and revalidation of the Equine Ophthalmic Pain Scale: R-EOPS a new scale for ocular pain assessment in horses

This study addresses the refinement and revalidation of a composite pain scale that focuses on equine facial expressions and behavioural indicators as exhibitions of ophthalmic pain. This scale included only Behavioural and Facial and Ocular expression indicators and, compared to the first version of Equine Ophthalmic Pain Scale (EOPS), item descriptors and related ratings were changed. Thirteen horses with ocular diseases that required medical or surgical treatment were enroled (group P). In each animal, the refined EOPS (R-EOPS) was applied prior to any treatment (T0) and one week later (T7). The R-EOPS was applied twice, 7 days apart, to 16 healthy control horses (group C). Two 30-second videos were recorded each time to allow the retrospective analysis by eight observers. Inter-observer reliability of items was moderate or substantial (Krippendorff's alpha, Kα>0.40) while their intra-observer reliability was substantial or almost perfect for most items (Kα ≥0.61). Both inter- and intra-observer reliability of Total Score (TS) were however excellent (Intraclass Correlation Coefficients, ICC>0.75). The TS also showed good reproducibility (Kendall coefficient=0.786, ICC=0.684) and high consistency of its items (Cronbach's α=0.847). The comparison between groups as well as the sensitivity and specificity values supported the validity of the R-EOPS. In particular, for each extra point added to the TS, the risk of the horse having pain increased by more than two times (Odds Ratio=2.079, 95%CI=1.542-2.804; P<0.001). The Receiver Operating Characteristic analysis identified 6 as the threshold value of R-EOPS for discriminating horses with ocular pathology (sensitivity=83%, specificity=100%). This scale may be an effective tool for reliably assessing the pain level in horses with ophthalmic diseases and potentially guiding pain management although it still requires large-scale application and external validation.

A comparison of online and live training of livestock farmers for an on-farm self-assessment of animal welfare

One approach to strengthening the involvement of farmers or stockpersons in the evaluation and improvement of animal welfare is the implementation of an on-farm self-assessment. A valid comparison of the results with reference values, between or within farms, requires that training of the farmers and reliability testing have taken place. We investigated two different training methods (online vs. live) with a total of 146 livestock farmers from farms with dairy cows and calves, beef cattle, sows and suckling piglets, weaners and fattening pigs, laying hens, broiler chickens, and turkeys from all over Germany. Online tests were conducted by assessing photos/videos of each indicator of the assessment scheme to estimate the inter-rater reliability (prevalence-adjusted and bias-adjusted kappa, PABAK). The farmers were requested to provide information on their professional background and rate their motivation to participate in the training and their subjective training success, meaning their confidence in assessing each indicator later on-farm. They evaluated the feasibility of the training and its impact on their views and attitudes. In general, farmers achieved at least substantial inter-rater reliability (PABAK ≥ 0.61) in 86.8% of all initial tests; 13.4% of the tests were repeated once or more times, resulting in a significant improvement of the agreement, with 90.9% of the tests reaching a PABAK ≥ 0.61. However, reliability was higher for indicators with a lower number of score levels. The subjective evaluation of training success was, on average, positive (score = 74.8 out of 100). No effects of the training method or the farmers’ professional background on the inter-rater reliability or the subjective training success were detected. Furthermore, for both methods, farmers moderately agreed that the training had sharpened their views on the animals, encouraged them to implement the assessment on their farm, and made it clear that self-assessment supports animal management. Although the reported costs and time investment for the online training were significantly lower, the effort required for both methods and the ease of integration into the workflow were ranked as similarly acceptable. Overall, both training methods appear feasible for the training of farmers/stockpersons on the assessment of animal-based indicators.

Validating owner-reporting of feather condition of pet Psittaciformes using photographs

Reporting of outcome variables by caregivers in welfare studies is commonplace but is open to subjective bias and so requires validation. Biases can occur in either direction: familiarity with an animal allows a deeper insight into welfare problems, but also can lead to reticence in admitting that an animal in one's care is experiencing problems. Here, we aim to validate owner-reporting of plumage condition of pet parrots, including those with self-inflicted feather-damaging behaviour (FDB), by comparing owners' scores of feather condition with those of two independent raters, blind to the owners' and each other's assessments. We surveyed pet parrot owners to collect data on basic demographics and feather condition, and requested four standardised photographs of birds. We received 259 responses (17% of the 1,521 people contacted); 78 sets of images of appropriate quality for assessment by raters were provided. Mean percentage agreement between owners' and raters' scores was mostly fair to substantial using Cohen's kappa; however, raters scored a greater proportion of feather damage than did owners. Overall, our results indicate owner-reporting of feather condition, including FDB, to be generally reliable and consistent with independent assessment of photographs. As the use of photographs can be limited by image quality, a failure to represent the long-term state of a parrot, and the potential for incorrect recording if assessed without relevant information (eg on moulting), this evidence that owner-reports can be reliable opens the door for larger-scale surveys of the extent of welfare-relevant problems.

Feasibility and Reliability of the AWIN Welfare Assessment Protocol for Dairy Goats in Semi-extensive Farming Conditions

The aim of this study was to test the feasibility and reliability of the Animal Welfare Indicators (AWIN) protocol for welfare assessment of dairy goats when applied to semi-extensive farming conditions. We recruited 13 farms located in the NW Italian Alps where three assessors individually and independently applied a modified version of the AWIN welfare assessment protocol for goats integrated with some indicators derived from the AWIN welfare assessment protocol for sheep. The applied protocol consisted of nine individual-level (body condition score, hair coat condition, abscesses, overgrown claws, udder asymmetry, fecal soiling, nasal discharge, ocular discharge, and improper disbudding) and seven group-level (severe lameness, Qualitative Behavior Assessment-QBA, thermal stress, oblivion, Familiar Human Approach Test-FHAT, synchrony at grazing, synchrony at resting) animal-based indicators. On most farms, the level of welfare was good. Many of the considered welfare problems (overgrown claws, fecal soiling, discharges, and thermal stress) were never recorded. However, oblivion, severe lameness, hair coat condition and abscesses were detected on some farms, with percentages ranging from 5 to 35%. The mean percentage of animals with normal body condition was 67.9 ± 5.7. The level of synchronization during resting was on average low (14.3 ± 7.2%). The application of the whole protocol required more than 4 h/farm and 3 min/goat. The inter-observer reliability varied from excellent (udder asymmetry, overgrown claws, discharges, synchrony at resting, use of shelter) to acceptable (abscesses, fecal soiling, and oblivion), but insufficient for hair coat condition, improper disbudding, synchrony at grazing, QBA. Differences in background of the assessors and feasibility constraints (i.e., use of binoculars in unfenced pastures, individual-level assessment conducted during the morning milking in narrow and dark pens, difficulties when using the scan and instantaneous sampling method due to the high number of animals that moved at the same time) can affect the reliability of data collection. Extensive training seems necessary for properly scoring animals when applying the QBA, whereas the FHAT to evaluate the Human-Animal Relationship of goats at pasture seems promising but needs to be validated. Indicators that evaluate the synchrony of activities require to be validated to identify the best moment to perform the observations during the day.

The Agreement between Feline Pancreatic Lipase Immunoreactivity and DGGR-Lipase Assay in Cats-Preliminary Results

Simple Summary

Feline pancreatitis is a common disease with diagnostics requiring a combination of clinical, laboratory and imaging methods. Measuring of pancreatic lipase concentration is the laboratory mainstay of feline pancreatitis diagnostics. It can be done using immunoenzymatic assay (feline pancreatic immunoreactivity, fPLI) or colorimetric assay based on hydrolyzing a special substrate (DGGR) to a color product. A DGGR-lipase assay is manufactured by various companies. Studies carried out in cats on the DGGR-lipase assay of a single manufacturer have shown a high agreement with fPLI, however studies in dogs indicate that the performance of assays of different manufacturers may vary considerably. One of the largest Polish veterinary laboratories working 24/7 in the largest Polish cities offers the DGGR-lipase assay of a large Polish laboratory solutions provider. The values obtained in this assay are higher than those yielded by the traditional DGGR-lipase assay validated in studies carried out so far. Therefore, we decided to evaluate the agreement between the DGGR-lipase assay of the Polish manufacturer and fPLI. The agreement proved to be as high as for the traditional assay.

Abstract

The colorimetric catalytic assay based on the use of 1,2-o-dilauryl-rac-glycero-3-glutaric acid-(6′-methylresorufin) (DGGR) ester as a substrate for pancreatic lipase activity is commonly used for the diagnosis of pancreatitis in dogs and cats. Even though the assay has generally been shown to yield consistent results with feline pancreatic lipase immunoreactivity (fPLI) assay, the agreement may vary between assays of different manufacturers. In this study, the chance-corrected agreement between a DGGR-lipase assay offered by one of the biggest providers of diagnostic solutions in Poland and fPLI assay was investigated. The study was carried out on 50 cats in which DGGR-lipase activity and fPLI were tested in the same blood sample. The chance-corrected agreement was determined using Gwet’s AC₁ coefficient separately for the fPLI assay’s cut-off values of >3.5 μg/L and >5.3 μg/L. The DGGR-lipase activity significantly positively correlated with fPLI (R_s = 0.665; CI 95%: 0.451, 0.807, p < 0.001). The chance-corrected agreement between the fPLI assay and DGGR-lipase assay differed considerably depending on the cut-off values of the DGGR-lipase assay. When the cut-off value reported in the literature (>26 U/L) was used, it was poor to fair. It was moderate at the cut-off value recommended by the laboratory (>45 U/L), and good at the cut-off value recommended by the assay’s manufacturer (>60 U/L). The highest agreement was obtained between the fPLI assay at the cut-off value of 3.5 μg/L and the DGGR-lipase assay at the cut-off value of 55 U/L (AC₁ = 0.725; CI 95%: 0.537, 0.914) and between the fPLI assay at the cut-off value of 5.3 μg/L and the DGGR-lipase assay at the cut-off value of 70 U/L (AC₁ = 0.749; CI 95%: 0.577, 0.921). The study confirms that the chance-corrected agreement between the two assays is good. Prospective studies comparing both assays to a diagnostic gold standard are needed to determine which of them is more accurate.