Comparing the predictive validity of behavioral codings and behavioral ratings in a working-dog breeding program

Development of a modified C-BARQ for evaluating behavior in working dogs

Introduction

Current high demand for effective odor detection dogs calls for the development of reliable methods for measuring performance-related behavioral phenotypes in these highly specialized working animals. The Canine Behavioral Assessment & Research Questionnaire (C-BARQ) is a widely used behavioral assessment tool among working dog organizations with a demonstrated ability to predict success/failure of dogs in training. However, this instrument was developed originally to study the prevalence of behavior problems in the pet dog population, and it therefore lacks the capacity to measure specific behavioral propensities that may also be important predictors of working dog success. The current paper examines the factor structure, internal reliability, and content validity of a modified version of the C-BARQ designed to evaluate four new domains of canine behavior in addition to those encompassed by the original C-BARQ. These domains, labeled Playfulness, Impulsivity, Distractibility, and Basophobia (fear of falling), respectively, describe aspects of canine behavior or temperament which are believed to contribute substantially to working dog performance.

Methods

Exploratory factor analysis (EFA) of owner/handler questionnaire responses based on a sample of 1,117 working odor detection dogs.

Results

A total of 15 factors were extracted by EFA, 10 of which correspond to original C-BARQ factors. The remaining 5 comprise the four new domains- Playfulness, Impulsivity, Distractibility, and Basophobia- as well as a fifth new factor labeled Food focus.

Discussion

The resulting Working Dog Canine Behavioral Assessment & Research Questionnaire (WDC-BARQ) successfully expands the measurement capacities of the original C-BARQ to include dimensions of behavior/temperament of particular relevance to many working dog populations.

Explosive Detection Dogs: A Perspective from the Personality Profile, Selection, Training Methods, Employment, and Performance to Mitigate a Real Threat

Terrorist organizations have compelled security authorities of every nation to make an increasingly significant commitment toward mitigating the risk of mass casualties and severe financial and property damages. As a result, various security measures have been implemented, including the use of advanced equipment and an uptick in intelligence activities. One of the most effective tools that has yielded outstanding results is the use of explosive detection dogs (EDDs). The nature of EDDs demands a high level of sensitivity given the inherent danger and severity of real threat situations that may involve the risk of explosion. Moreover, the operating procedures for EDDs are unique and distinguishable from other forms of detection. We conducted a review to ensure a comprehensive understanding of the subject, highlighting the EDDs' personality profile, selection, training methods, performance, and employment, incorporating insights from diverse fields, conducting an analysis, and presenting a perspective on using EDDs to prevent explosion threats.

Digitally-enhanced dog behavioral testing

Behavioral traits in dogs are assessed for a wide range of purposes such as determining selection for breeding, chance of being adopted or prediction of working aptitude. Most methods for assessing behavioral traits are questionnaire or observation-based, requiring significant amounts of time, effort and expertise. In addition, these methods might be also susceptible to subjectivity and bias, negatively impacting their reliability. In this study, we proposed an automated computational approach that may provide a more objective, robust and resource-efficient alternative to current solutions. Using part of a 'Stranger Test' protocol, we tested n = 53 dogs for their response to the presence and neutral actions of a stranger. Dog coping styles were scored by three dog behavior experts. Moreover, data were collected from their owners/trainers using the Canine Behavioral Assessment and Research Questionnaire (C-BARQ). An unsupervised clustering of the dogs' trajectories revealed two main clusters showing a significant difference in the stranger-directed fear C-BARQ category, as well as a good separation between (sufficiently) relaxed dogs and dogs with excessive behaviors towards strangers based on expert scoring. Based on the clustering, we obtained a machine learning classifier for expert scoring of coping styles towards strangers, which reached an accuracy of 78%. We also obtained a regression model predicting C-BARQ scores with varying performance, the best being Owner-Directed Aggression (with a mean average error of 0.108) and Excitability (with a mean square error of 0.032). This case study demonstrates a novel paradigm of 'machine-based' dog behavioral assessment, highlighting the value and great promise of AI in this context.

Machine learning prediction and classification of behavioral selection in a canine olfactory detection program

There is growing interest in canine behavioral research specifically for working dogs. Here we take advantage of a dataset of a Transportation Safety Administration olfactory detection cohort of 628 Labrador Retrievers to perform Machine Learning (ML) prediction and classification studies of behavioral traits and environmental effects. Data were available for four time points over a 12 month foster period after which dogs were accepted into a training program or eliminated. Three supervised ML algorithms had robust performance in correctly predicting which dogs would be accepted into the training program, but poor performance in distinguishing those that were eliminated (~ 25% of the cohort). The 12 month testing time point yielded the best ability to distinguish accepted and eliminated dogs (AUC = 0.68). Classification studies using Principal Components Analysis and Recursive Feature Elimination using Cross-Validation revealed the importance of olfaction and possession-related traits for an airport terminal search and retrieve test, and possession, confidence, and initiative traits for an environmental test. Our findings suggest which tests, environments, behavioral traits, and time course are most important for olfactory detection dog selection. We discuss how this approach can guide further research that encompasses cognitive and emotional, and social and environmental effects.

Defining the Characteristics of Successful Biosecurity Scent Detection Dogs

To perform their role effectively, scent detection dogs require certain characteristics. Identifying these characteristics will inform the selection of prospective dogs and preferred approaches to their training. The current study drew upon the perspectives of industry stakeholders to identify the behavioural traits considered relevant for detection dogs in biosecurity screening roles. Dog handlers, trainers, and supervisors (n = 25) in Australian biosecurity operations participated in focus group interviews to determine the perceived characteristics that, in their experience, influence detection performance. Their descriptions were used to create a questionnaire which was then administered to handlers to assess the working behaviours of current biosecurity dogs. Responses were collected for 88% of the operational dogs (n = 36). An exploratory factor analysis revealed seven tentative dimensions: search motivation, emotional stability, search arousal, food motivation, play motivation, search independence, and search focus. Search motivation and search arousal were both positively associated with handler ratings of detection performance (p ≤ 0.006). In general, biosecurity dogs were scored consistently high in ratings of search motivation, emotional stability, and food motivation. Our approach has advanced our understanding of the working behaviours and characteristic profile of biosecurity detector dogs and will be used to inform candidate selection processes.

Phenotypic correlates of the working dog microbiome

Dogs have a key role in law enforcement and military work, and research with the goal of improving working dog performance is ongoing. While there have been intriguing studies from lab animal models showing a potential connection between the gut microbiome and behavior or mental health there is a dearth of studies investigating the microbiome-behavior relationship in working dogs. The overall objective of this study was to characterize the microbiota of working dogs and to determine if the composition of the microbiota is associated with behavioral and performance outcomes. Freshly passed stools from each working canine (Total n = 134) were collected and subject to shotgun metagenomic sequencing using Illumina technology. Behavior, performance, and demographic metadata were collected. Descriptive statistics and prediction models of behavioral/phenotypic outcomes using gradient boosting classification based on Xgboost were used to study associations between the microbiome and outcomes. Regarding machine learning methodology, only microbiome features were used for training and predictors were estimated in cross-validation. Microbiome markers were statistically associated with motivation, aggression, cowardice/hesitation, sociability, obedience to one trainer vs many, and body condition score (BCS). When prediction models were developed based on machine learning, moderate predictive power was observed for motivation, sociability, and gastrointestinal issues. Findings from this study suggest potential gut microbiome markers of performance and could potentially advance care for working canines.

Development of a Performance Monitoring Instrument for Rating Explosives Search Dog Performance

The growing body of working dog literature includes many examples of scales robustly developed to measure aspects of dog behavior. However, when comparing behavior to working dog ability, most studies rely on training organizations' own long-established ratings of performance, or simply pass/fail at selection or certification as measures of success. Working ability is multifaceted, and it is likely that different aspects of ability are differentially affected by external factors. In order to understand how specific aspects of selection, training, and operations influence a dog's working ability, numerous facets of performance should be considered. An accurate and validated method for quantifying multiple aspects of performance is therefore required. Here, we describe the first stages of formulating a meaningful performance measurement tool for two types of working search dogs. The systematic methodology used was: (1) interviews and workshops with a representative cross-section of stakeholders to produce a shortlist of behaviors integral to current operational performance of vehicle (VS) and high assurance (HAS) search dogs; (2) assessing the reliability and construct validity of the shortlisted behavioral measures (at the behavior and the individual rater level) using ratings of diverse videoed searches by experienced personnel; and (3) selecting the most essential and meaningful behaviors based on their reliability/validity and importance. The resulting performance measurement tool was composed of 12 shortlisted behaviors, most of which proved reliable and valid when assessed by a group of raters. At the individual rater level, however, there was variability between raters in the ability to use and interpret behavioral measures, in particular, more abstract behaviors such as Independence. This illustrates the importance of examining individual rater scores rather than extrapolating from group consensus (as is often done), especially when designing a tool that will ultimately be used by single raters. For ratings to be practically valuable, individual rater reliability needs to be improved, especially for behaviors deemed as essential (e.g., control and confidence). We suggest that the next steps are to investigate why individuals vary in their ratings and to undertake efforts to increase the likelihood that they reach a common conceptualization of each behavioral construct. Plausible approaches are improving the format in which behaviors are presented, e.g., by adding benchmarks and utilizing rater training.

Enhancing the Selection and Performance of Working Dogs

Dogs perform a variety of integral roles in our society, engaging in work ranging from assistance (e.g., service dogs, guide dogs) and therapy to detection (e.g., search-and-rescue dogs, explosive detection dogs) and protection (e.g., military and law enforcement dogs). However, success in these roles, which requires dogs to meet challenging behavioral criteria and to undergo extensive training, is far from guaranteed. Therefore, enhancing the selection process is critical for the effectiveness and efficiency of working dog programs and has the potential to optimize how resources are invested in these programs, increase the number of available working dogs, and improve working dog welfare. In this paper, we review two main approaches for achieving this goal: (1) developing selection tests and criteria that can efficiently and effectively identify ideal candidates from the overall pool of candidate dogs, and (2) developing approaches to enhance performance, both at the individual and population level, via improvements in rearing, training, and breeding. We summarize key findings from the empirical literature regarding best practices for assessing, selecting, and improving working dogs, and conclude with future steps and recommendations for working dog organizations, breeders, trainers, and researchers.

Validation of a Behavior Test for Predicting Puppies' Suitability as Detection Dogs

Behavioral characteristics are the most influential factor in the success of a working dog. The need for highly capable detection dogs continues to rise; but reliable methods for early selection are lacking. The current study aimed to assess the reliability and validity of a behavioral test for assessing detection dog suitability. A cohort of candidate detection dog puppies (n = 60) were tested at 3; 5; and 11 months of age; as well as at the completion of training at approximately one year. Tests were designed to assess important detection dog behavioral characteristics such as search ability and fearfulness. Inter-rater reliability was high between independent observers. Convergent validity was demonstrated by comparing Principal Component Analysis (PCA) scores from the behavior test to trainer ratings using the Canine Behavioral Assessment and Research Questionnaire (C-BARQ) and a survey of detection dog traits. Performance on the behavior test predicted adult selection as a detection dog as early as 3 months. The methods reported will be valuable for improving selection measures and enhancing collaborations across breeding programs in order to increase the availability of highly capable detection dogs.

Does Benchmarking of Rating Scales Improve Ratings of Search Performance Given by Specialist Search Dog Handlers?

Rating scales are widely used to rate working dog behavior and performance. Whilst behaviour scales have been extensively validated, instruments used to rate ability have usually been designed by training and practitioner organizations, and often little consideration has been given to how seemingly insignificant aspects of the scale design might alter the validity of the results obtained. Here we illustrate how manipulating one aspect of rating scale design, the provision of verbal benchmarks or labels (as opposed to just a numerical scale), can affect the ability of observers to distinguish between differing levels of search dog performance in an operational environment. Previous studies have found evidence for range restriction (using only part of the scale) in raters' use of the scales and variability between raters in their understanding of the traits used to measures performance. As provision of verbal benchmarks has been shown to help raters in a variety of disciplines to select appropriate scale categories (or scores), it may be predicted that inclusion of verbal benchmarks will bring raters' conceptualization of the traits closer together, increasing agreement between raters, as well as improving the ability of observers to distinguish between differing levels of search dog performance and reduce range restriction. To test the value of verbal benchmarking we compared inter-rater reliability, raters' ability to discriminate between different levels of search dog performance, and their use of the whole scale before and after being presented with benchmarked scales for the same traits. Raters scored the performance of two separate types of explosives search dog (High Assurance Search (HAS) and Vehicle Search (VS) dogs), from short (~30 s) video clips, using 11 previously validated traits. Taking each trait in turn, for the first five clips raters were asked to give a score from 1, representing the lowest amount of the trait evident to 5, representing the highest. Raters were given a list of adjective-based benchmarks (e.g., very low, low, intermediate, high, very high) and scored a further five clips for each trait. For certain traits, the reliability of scoring improved when benchmarks were provided (e.g., Motivation and Independence), indicating that their inclusion may potentially reduce ambivalence in scoring, ambiguity of meanings, and cognitive difficulty for raters. However, this effect was not universal, with the ratings of some traits remaining unchanged (e.g., Control), or even reducing in reliability (e.g., Distraction). There were also some differences between VS and HAS (e.g., Confidence reliability increased for VS raters and decreased for HAS raters). There were few improvements in the spread of scores across the range, but some indication of more favorable scoring. This was a small study of operational handlers and trainers utilizing training video footage from realistic operational environments, and there are potential cofounding effects. We discuss possible causal factors, including issues specific to raters and possible deficiencies in the chosen benchmarks, and suggest ways to further improve the effectiveness of rating scales. This study illustrates why it is vitally important to validate all aspects of rating scale design, even if they may seem inconsequential, as relatively small changes to the amount and type of information provided to raters can have both positive and negative impacts on the data obtained.

Selecting Dogs for Explosives Detection: Behavioral Characteristics

Detection dogs are widely considered the most effective and adaptive method for explosives detection. Increases in emerging sophisticated threats are accelerating the demand for highly capable explosives detection, causing a strain on available supplies of quality canines worldwide. These strains are further compounded by rigorous behavioral standards required to meet mission-specific capabilities, leading to high rates of dogs disqualified from training or deployment. Ample research has explored the behavioral characteristics important for assistance, guide, and other traditional working roles, while those corresponding to more specialized tasks such as detection of explosives are not as well-understood. In this review we aim to identify the behavioral characteristics important for operational tasks of explosives detection dogs, contrasting with that of other working roles and highlighting key differences between explosives and other types of detection dogs. Further, we review the available research on methods for assessing and selecting candidate detection dogs and make recommendations for future directions and applications to the industry. Improvements and standardization in assessment technology allowing for the identification and enhancement of behavioral characteristics will be key to advancing canine detection technology in general.

Attachment security in companion dogs: adaptation of Ainsworth's strange situation and classification procedures to dogs and their human caregivers

This exploratory study describes the development of a classification system for dogs’ attachment security to caregivers that adheres closely to Ainsworth’s seminal methodology. Fifty-nine adult dogs and caregivers participated in a mildly threatening laboratory encounter with a stranger (TS) and the Strange Situation (SSP). Dog and attachment experts adapted Ainsworth’s classification system  for the behavioral repertoire of the dog. Four potentially comparable patterns of attachment were identified. The proportions of secure and insecure classifications (61% and 39%) were similar to those found in human toddlers. Caregivers’ sensitivity to their dogs during the TS procedure significantly differentiated dogs with secure vs. insecure classifications Lower scores on the Active/excited personality scale on the Monash Canine Personality Questionnaire-Revised (MCPQ-R) also were related to secure classification. This system now makes it possible to compare directly the effects of human and dog attachment patterns on the health and emotional well-being of humans and dogs.

A Systematic Review of the Reliability and Validity of Behavioural Tests Used to Assess Behavioural Characteristics Important in Working Dogs

Background

Working dogs are selected based on predictions from tests that they will be able to perform specific tasks in often challenging environments. However, withdrawal from service in working dogs is still a big problem, bringing into question the reliability of the selection tests used to make these predictions.

Methods

A systematic review was undertaken aimed at bringing together available information on the reliability and predictive validity of the assessment of behavioural characteristics used with working dogs to establish the quality of selection tests currently available for use to predict success in working dogs.

Results

The search procedures resulted in 16 papers meeting the criteria for inclusion. A large range of behaviour tests and parameters were used in the identified papers, and so behaviour tests and their underpinning constructs were grouped on the basis of their relationship with positive core affect (willingness to work, human-directed social behaviour, object-directed play tendencies) and negative core affect (human-directed aggression, approach withdrawal tendencies, sensitivity to aversives). We then examined the papers for reports of inter-rater reliability, within-session intra-rater reliability, test-retest validity and predictive validity.

Conclusions

The review revealed a widespread lack of information relating to the reliability and validity of measures to assess behaviour and inconsistencies in terminologies, study parameters and indices of success. There is a need to standardise the reporting of these aspects of behavioural tests in order to improve the knowledge base of what characteristics are predictive of optimal performance in working dog roles, improving selection processes and reducing working dog redundancy. We suggest the use of a framework based on explaining the direct or indirect relationship of the test with core affect.

An evidence-based decision assistance model for predicting training outcome in juvenile guide dogs

Working dog organisations, such as Guide Dogs, need to regularly assess the behaviour of the dogs they train. In this study we developed a questionnaire-style behaviour assessment completed by training supervisors of juvenile guide dogs aged 5, 8 and 12 months old (n = 1,401), and evaluated aspects of its reliability and validity. Specifically, internal reliability, temporal consistency, construct validity, predictive criterion validity (comparing against later training outcome) and concurrent criterion validity (comparing against a standardised behaviour test) were evaluated. Thirty-nine questions were sourced either from previously published literature or created to meet requirements identified via Guide Dogs staff surveys and staff feedback. Internal reliability analyses revealed seven reliable and interpretable trait scales named according to the questions within them as: Adaptability; Body Sensitivity; Distractibility; Excitability; General Anxiety; Trainability and Stair Anxiety. Intra-individual temporal consistency of the scale scores between 5–8, 8–12 and 5–12 months was high. All scales excepting Body Sensitivity showed some degree of concurrent criterion validity. Predictive criterion validity was supported for all seven scales, since associations were found with training outcome, at at-least one age. Thresholds of z-scores on the scales were identified that were able to distinguish later training outcome by identifying 8.4% of all dogs withdrawn for behaviour and 8.5% of all qualified dogs, with 84% and 85% specificity. The questionnaire assessment was reliable and could detect traits that are consistent within individuals over time, despite juvenile dogs undergoing development during the study period. By applying thresholds to scores produced from the questionnaire this assessment could prove to be a highly valuable decision-making tool for Guide Dogs. This is the first questionnaire-style assessment of juvenile dogs that has shown value in predicting the training outcome of individual working dogs.

Development of the "Highly Sensitive Dog" questionnaire to evaluate the personality dimension "Sensory Processing Sensitivity" in dogs

In humans, the personality dimension 'sensory processing sensitivity (SPS)', also referred to as "high sensitivity", involves deeper processing of sensory information, which can be associated with physiological and behavioral overarousal. However, it has not been studied up to now whether this dimension also exists in other species. SPS can influence how people perceive the environment and how this affects them, thus a similar dimension in animals would be highly relevant with respect to animal welfare. We therefore explored whether SPS translates to dogs, one of the primary model species in personality research. A 32-item questionnaire to assess the "highly sensitive dog score" (HSD-s) was developed based on the "highly sensitive person" (HSP) questionnaire. A large-scale, international online survey was conducted, including the HSD questionnaire, as well as questions on fearfulness, neuroticism, "demographic" (e.g. dog sex, age, weight; age at adoption, etc.) and "human" factors (e.g. owner age, sex, profession, communication style, etc.), and the HSP questionnaire. Data were analyzed using linear mixed effect models with forward stepwise selection to test prediction of HSD-s by the above-mentioned factors, with country of residence and dog breed treated as random effects. A total of 3647 questionnaires were fully completed. HSD-, fearfulness, neuroticism and HSP-scores showed good internal consistencies, and HSD-s only moderately correlated with fearfulness and neuroticism scores, paralleling previous findings in humans. Intra- (N = 447) and inter-rater (N = 120) reliabilities were good. Demographic and human factors, including HSP score, explained only a small amount of the variance of HSD-s. A PCA analysis identified three subtraits of SPS, comparable to human findings. Overall, the measured personality dimension in dogs showed good internal consistency, partial independence from fearfulness and neuroticism, and good intra- and inter-rater reliability, indicating good construct validity of the HSD questionnaire. Human and demographic factors only marginally affected the HSD-s suggesting that, as hypothesized for human SPS, a genetic basis may underlie this dimension within the dog species.