A Review: Development of Computer Vision-Based Lameness Detection for Dairy Cows and Discussion of the Practical Applications

Descriptive evaluation of a camera-based dairy cattle lameness detection technology

Lameness in dairy cattle is a clinical sign of impaired locomotion, mainly caused by painful foot lesions, compromising the US dairy industry's economic, environmental, and social sustainability goals. Combining technology and on farm data may be a more precise and less labor-intensive lameness detection tool, particularly for early detection. The objective of this observational study was to describe the association between average weekly autonomous camera-based (AUTO) locomotion scores and hoof trimming (HT) data. The AUTO data were collected from 3 farms from April 2022 to March 2023. Historical farm HT data were collected from March 2016 to March 2023 and used to determine cow lesion history and date of HT event. The HT events were categorized as a regular HT (TRIM; n = 2290) or a HT with a lesion recorded (LESION; n = 670). Events with LESION were sub-categorized based on lesion category: digital dermatitis (DD; n = 276), sole ulcer (SU; n = 79), white line disease (WLD; n = 141), and other (n = 174). The data also contained the leg of the LESION, classified as front left (FL; n = 54), front right (FR; n = 146), rear left (RL; n = 281), or rear right (RR; n = 183) leg with 6 events missing the leg. Cows' HT histories were classified as follows: cows with no previous recorded instance of any lesion were classified as TRIM0 (n = 1554). The first instance of any hoof lesion was classified as LESION1 (n = 238). This classification was retained until a subsequent TRIM occurred - recorded as TRIM1 (n = 632). The next unique instance of any lesion following a TRIM1 was classified as LESION2 (n = 86). Any LESION events occurring after LESION1 or LESION2 without a subsequent TRIM were considered a hoof lesion recurrence and classified as LESIONRE1 (n = 164) and LESIONRE2 (n = 22), respectively. TRIM events after LESION2 or LESION2RE (n = 104) or LESION events after LESIONRE1 or LESIONRE2 were classified as LESION_OTHER (n = 160). The AUTO scores from -28 to -1 days prior to the HT event were summarized into weekly scores and included if cows had at least 1 observation per week in the 4 weeks before the event. For all weeks, LESION cows had a higher median AUTO score than TRIM cows. Cows with TRIM0 had the lowest and most consistent median weekly score compared to LESION and other TRIM classifications. Before HT cows with TRIM0 and TRIM1, both had median score increases of 1 across the 4 weeks, while the LESION categories had an increase of 4 to 8. Scores increased with each subsequent LESION event compared to the previous LESION event. Cows with SU lesions had the highest median score across the 4 weeks, WLD had the largest score increase, and DD had the lowest median score and score increase. When grouping a LESION event by leg the hoof lesion was found on, the AUTO scores for four groups displayed comparable median values. Due to the difference between TRIM and LESION events, this technology shows potential for the early detection of hoof lesions.

Evaluating machine learning algorithms to predict lameness in dairy cattle

Dairy cattle lameness represents one of the common concerns in intensive and commercial dairy farms. Lameness is characterized by gait-related behavioral changes in cows and multiple approaches are being utilized to associate these changes with lameness conditions including data from accelerometers, and other precision technologies. The objective was to evaluate the use of machine learning algorithms for the identification of lameness conditions in dairy cattle. In this study, 310 multiparous Holstein dairy cows from a herd in Northern Colorado were affixed with a leg-based accelerometer (Icerobotics® Inc, Edinburg, Scotland) to obtain the lying time (min/d), daily steps count (n/d), and daily change (n/d). Subsequently, study cows were monitored for 4 months and cows submitted for claw trimming (CT) were differentiated as receiving corrective claw trimming (CCT) or as being diagnosed with a lameness disorder and consequent therapeutic claw trimming (TCT) by a certified hoof trimmer. Cows not submitted to CT were considered healthy controls. A median filter was applied to smoothen the data by reducing inherent variability. Three different machine learning (ML) models were defined to fit each algorithm which included the conventional features (containing daily lying, daily steps, and daily change derived from the accelerometer), slope features (containing features extracted from each variable in Conventional feature), or all features (3 simple features and 3 slope features). Random forest (RF), Naive Bayes (NB), Logistic Regression (LR), and Time series (ROCKET) were used as ML predictive approaches. For the classification of cows requiring CCT and TCT, ROCKET classifier performed better with accuracy (> 90%), ROC-AUC (> 74%), and F1 score (> 0.61) as compared to other algorithms. Slope features derived in this study increased the efficiency of algorithms as the better-performing models included All features explored. However, further classification of diseases into infectious and non-infectious events was not effective because none of the algorithms presented satisfactory model accuracy parameters. For the classification of observed cow locomotion scores into severely lame and moderately lame conditions, the ROCKET classifier demonstrated satisfactory accuracy (> 0.85), ROC-AUC (> 0.68), and F1 scores (> 0.44). We conclude that ML models using accelerometer data are helpful in the identification of lameness in cows but need further research to increase the granularity and accuracy of classification.

Lameness in Cattle-Etiopathogenesis, Prevention and Treatment

The aim of this review was to analyse the health problem of lameness in dairy cows by assessing the health and economic losses. This review also presents in detail the etiopathogenesis of lameness in dairy cattle and examples of its treatment and prevention. This work is based on a review of available publications. In selecting articles for the manuscript, the authors focused on issues observed in cattle herds during their clinical work. Lameness in dairy cattle is a serious health and economic problem around the world. Production losses result from reduced milk yield, reduced feed intake, reproductive disorders, treatment costs, and costs associated with early culling. A significant difficulty in the control and treatment of lameness is the multifactorial nature of the disease; causes may be individual or species-specific and may be associated with the environment, nutrition, or the presence of concomitant diseases. An important role is ascribed to infectious agents of both systemic and local infections, which can cause problems with movement in animals. It is also worth noting the long treatment process, which can last up to several months, thus significantly affecting yield and production. Given the high economic losses resulting from lameness in dairy cows, reaching even >40% (depending on the scale of production), there seems to be a need to implement extensive preventive measures to reduce the occurrence of limb infections in animals. The most important effective preventive measures to reduce the occurrence of limb diseases with symptoms of lameness are periodic hoof examinations and correction, nutritional control, and bathing with disinfectants. A clean and dry environment for cows should also be a priority.

Opportunities for Regulatory Authorities to Assess Animal-Based Measures at the Slaughterhouse Using Sensor Technology and Artificial Intelligence: A Review

Animal-based measures (ABMs) are the preferred way to assess animal welfare. However, manual scoring of ABMs is very time-consuming during the meat inspection. Automatic scoring by using sensor technology and artificial intelligence (AI) may bring a solution. Based on review papers an overview was made of ABMs recorded at the slaughterhouse for poultry, pigs and cattle and applications of sensor technology to measure the identified ABMs. Also, relevant legislation and work instructions of the Dutch Regulatory Authority (RA) were scanned on applied ABMs. Applications of sensor technology in a research setting, on farm or at the slaughterhouse were reported for 10 of the 37 ABMs identified for poultry, 4 of 32 for cattle and 13 of 41 for pigs. Several applications are related to aspects of meat inspection. However, by European law meat inspection must be performed by an official veterinarian, although there are exceptions for the post mortem inspection of poultry. The examples in this study show that there are opportunities for using sensor technology by the RA to support the inspection and to give more insight into animal welfare risks. The lack of external validation for multiple commercially available systems is a point of attention.

Crowd sourcing remote comparative lameness assessments for dairy cattle

Lameness assessments are rarely conducted routinely on dairy farms and when completed typically underestimate lameness prevalence, hampering early diagnosis and treatment. A well-known feature of many perceptual tasks is that relative assessments are more accurate than absolute assessments, suggesting that creating methods that allow for the relative scoring of which cow is more lame will allow for reliable lameness assessments. Here we developed and tested a remote comparative lameness assessment method: we recruited nonexperienced crowd workers via an online platform and asked them to watch 2 videos side-by-side, each showing a cow walking, and to identify which cow was more lame and by how much (on a scale of -3 to 3). We created 11 tasks, each with 10 video pairs for comparison, and recruited 50 workers per task. All tasks were also completed by 5 experienced cattle lameness assessors. We evaluated data filtering and clustering methods based on worker responses and determined the agreement among workers, among experienced assessors, and between these groups. A moderate to high interobserver reliability was observed (intraclass correlation coefficient, ICC = 0.46 to 0.77) for crowd workers and agreement was high among the experienced assessors (ICC = 0.87). Average crowd-worker responses showed excellent agreement with the average of experienced assessor responses (ICC = 0.89 to 0.91), regardless of data processing method. To investigate if we could use fewer workers per task while still retaining high agreement with experienced assessors, we randomly subsampled 2 to 43 (1 less than the minimum number of workers retained per task after data cleaning) workers from each task. The agreement with experienced assessors increased substantially as we increased the number of workers from 2 to 10, but little increase was observed after 10 or more workers were used (ICC > 0.80). The proposed method provides a fast and cost-effective way to assess lameness in commercial herds. In addition, this method allows for large-scale data collection useful for training computer vision algorithms that could be used to automate lameness assessments on farm.

Innovations in Cattle Farming: Application of Innovative Technologies and Sensors in the Diagnosis of Diseases

Precision livestock farming has a crucial function as farming grows in significance. It will help farmers make better decisions, alter their roles and perspectives as farmers and managers, and allow for the tracking and monitoring of product quality and animal welfare as mandated by the government and industry. Farmers can improve productivity, sustainability, and animal care by gaining a deeper understanding of their farm systems as a result of the increased use of data generated by smart farming equipment. Automation and robots in agriculture have the potential to play a significant role in helping society fulfill its future demands for food supply. These technologies have already enabled significant cost reductions in production, as well as reductions in the amount of intensive manual labor, improvements in product quality, and enhancements in environmental management. Wearable sensors can monitor eating, rumination, rumen pH, rumen temperature, body temperature, laying behavior, animal activity, and animal position or placement. Detachable or imprinted biosensors that are adaptable and enable remote data transfer might be highly important in this quickly growing industry. There are already multiple gadgets to evaluate illnesses such as ketosis or mastitis in cattle. The objective evaluation of sensor methods and systems employed on the farm is one of the difficulties presented by the implementation of modern technologies on dairy farms. The availability of sensors and high-precision technology for real-time monitoring of cattle raises the question of how to objectively evaluate the contribution of these technologies to the long-term viability of farms (productivity, health monitoring, welfare evaluation, and environmental effects). This review focuses on biosensing technologies that have the potential to change early illness diagnosis, management, and operations for livestock.

Precision Livestock Farming: What Does It Contain and What Are the Perspectives?

Precision Livestock Farming (PLF) describes the combined use of sensor technology, the related algorithms, interfaces, and applications in animal husbandry. PLF technology is used in all animal production systems and most extensively described in dairy farming. PLF is developing rapidly and is moving beyond health alarms towards an integrated decision-making system. It includes animal sensor and production data but also external data. Various applications have been proposed or are available commercially, only a part of which has been evaluated scientifically; the actual impact on animal health, production and welfare therefore remains largely unknown. Although some technology has been widely implemented (e.g., estrus detection and calving detection), other systems are adopted more slowly. PLF offers opportunities for the dairy sector through early disease detection, capturing animal-related information more objectively and consistently, predicting risks for animal health and welfare, increasing the efficiency of animal production and objectively determining animal affective states. Risks of increasing PLF usage include the dependency on the technology, changes in the human-animal relationship and changes in the public perception of dairy farming. Veterinarians will be highly affected by PLF in their professional life; they nevertheless must adapt to this and play an active role in further development of technology.

Variability of gait characteristics in lameness elephant

Lameness has a significant impact not only on the economy but also on elephant welfare. Several gait characteristics are altered to compensate for the discomfort. The traditional approach to detecting lameness has relied on mahout and veterinarian observation. The study aimed to determine how lameness affected the variability of an elephant's gait by using a three-dimensional inertial measurement unit (IMU) with Wi-Fi sensors. Five elephants with lameness, as determined by an experienced veterinarian and two, non-lamed elephants, aged between 58-80 years were included in the study. Gait biomechanics including limb segment motion, obtained from individually gyrometric- and accelero-based parameters and demonstrated as a graphical pattern showing changes in absolute rotation angle over time. The result revealed some character changes in gait kinematics parameters, but it was heterogeneity with an inconclusive pattern. The interlimb coordination could be a part of maintaining the actual locomotion pattern, or it could be a result of the mild degree of lameness for which all of the clients have compensated. This study introduces a new objective method for quantifying gait changes caused by lameness, additional research is required to make this objective more clinically applicable.

Estimating body weight and body condition score of mature beef cows using depth images

Obtaining accurate body weight (BW) is crucial for management decisions yet can be a challenge for cow-calf producers. Fast-evolving technologies such as depth sensing have been identified as low-cost sensors for agricultural applications but have not been widely validated for U.S. beef cattle. This study aimed to (1) estimate the body volume of mature beef cows from depth images, (2) quantify BW and metabolic weight (MBW) from image-projected body volume, and (3) classify body condition scores (BCS) from image-obtained measurements using a machine-learning-based approach. Fifty-eight crossbred cows with a mean BW of 410.0 ± 60.3 kg and were between 4 and 6 yr of age were used for data collection between May and December 2021. A low-cost, commercially available depth sensor was used to collect top-view depth images. Images were processed to obtain cattle biometric measurements, including MBW, body length, average height, maximum body width, dorsal area, and projected body volume. The dataset was partitioned into training and testing datasets using an 80%:20% ratio. Using the training dataset, linear regression models were developed between image-projected body volume and BW measurements. Results were used to test BW predictions for the testing dataset. A machine-learning-based multivariate analysis was performed with 29 algorithms from eight classifiers to classify BCS using multiple inputs conveniently obtained from the cows and the depth images. A feature selection algorithm was performed to rank the relevance of each input to the BCS. Results demonstrated a strong positive correlation between the image-projected cow body volume and the measured BW (r = 0.9166). The regression between the cow body volume and the measured BW had a co-efficient of determination (R2) of 0.83 and a 19.2 ± 13.50 kg mean absolute error (MAE) of prediction. When applying the regression to the testing dataset, an increase in the MAE of the predicted BW (22.7 ± 13.44 kg) but a slightly improved R2 (0.8661) was noted. Among all algorithms, the Bagged Tree model in the Ensemble class had the best performance and was used to classify BCS. Classification results demonstrate the model failed to predict any BCS lower than 4.5, while it accurately classified the BCS with a true prediction rate of 60%, 63.6%, and 50% for BCS between 4.75 and 5, 5.25 and 5.5, and 5.75 and 6, respectively. This study validated using depth imaging to accurately predict BW and classify BCS of U.S. beef cow herds.

Automatic Detection Method of Dairy Cow Feeding Behaviour Based on YOLO Improved Model and Edge Computing

The feeding behaviour of cows is an essential sign of their health in dairy farming. For the impression of cow health status, precise and quick assessment of cow feeding behaviour is critical. This research presents a method for monitoring dairy cow feeding behaviour utilizing edge computing and deep learning algorithms based on the characteristics of dairy cow feeding behaviour. Images of cow feeding behaviour were captured and processed in real time using an edge computing device. A DenseResNet-You Only Look Once (DRN-YOLO) deep learning method was presented to address the difficulties of existing cow feeding behaviour detection algorithms’ low accuracy and sensitivity to the open farm environment. The deep learning and feature extraction enhancement of the model was improved by replacing the CSPDarknet backbone network with the self-designed DRNet backbone network based on the YOLOv4 algorithm using multiple feature scales and the Spatial Pyramid Pooling (SPP) structure to enrich the scale semantic feature interactions, finally achieving the recognition of cow feeding behaviour in the farm feeding environment. The experimental results showed that DRN-YOLO improved the accuracy, recall, and mAP by 1.70%, 1.82%, and 0.97%, respectively, compared to YOLOv4. The research results can effectively solve the problems of low recognition accuracy and insufficient feature extraction in the analysis of dairy cow feeding behaviour by traditional methods in complex breeding environments, and at the same time provide an important reference for the realization of intelligent animal husbandry and precision breeding.

Intelligent Perception-Based Cattle Lameness Detection and Behaviour Recognition: A Review

Simple Summary

Cattle lameness detection as well as behaviour recognition are the two main objectives in the applications of precision livestock farming (PLF). Over the last five years, the development of smart sensors, big data, and artificial intelligence has offered more automatic tools. In this review, we discuss over 100 papers that used automated techniques to detect cattle lameness and to recognise animal behaviours. To assist researchers and policy-makers in promoting various livestock technologies for monitoring cattle welfare and productivity, we conducted a comprehensive investigation of intelligent perception for cattle lameness detection and behaviour analysis in the PLF domain. Based on the literature review, we anticipate that PLF will develop in an objective, autonomous, and real-time direction. Additionally, we suggest that further research should be dedicated to improving the data quality, modeling accuracy, and commercial availability.

Abstract

The growing world population has increased the demand for animal-sourced protein. However, animal farming productivity is faced with challenges from traditional farming practices, socioeconomic status, and climate change. In recent years, smart sensors, big data, and deep learning have been applied to animal welfare measurement and livestock farming applications, including behaviour recognition and health monitoring. In order to facilitate research in this area, this review summarises and analyses some main techniques used in smart livestock farming, focusing on those related to cattle lameness detection and behaviour recognition. In this study, more than 100 relevant papers on cattle lameness detection and behaviour recognition have been evaluated and discussed. Based on a review and a comparison of recent technologies and methods, we anticipate that intelligent perception for cattle behaviour and welfare monitoring will develop towards standardisation, a larger scale, and intelligence, combined with Internet of things (IoT) and deep learning technologies. In addition, the key challenges and opportunities of future research are also highlighted and discussed.