1
|
Zinkovskaia E, Tahary O, Loewenstern Y, Benaroya-Milshtein N, Bar-Gad I. Temporally aligned segmentation and clustering (TASC) framework for behavior time series analysis. Sci Rep 2024; 14:14952. [PMID: 38942770 PMCID: PMC11213853 DOI: 10.1038/s41598-024-63669-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/05/2023] [Accepted: 05/30/2024] [Indexed: 06/30/2024] Open
Abstract
Behavior exhibits a complex spatiotemporal structure consisting of discrete sub-behaviors, or motifs. Continuous behavior data requires segmentation and clustering to reveal these embedded motifs. The popularity of automatic behavior quantification is growing, but existing solutions are often tailored to specific needs and are not designed for the time scale and precision required in many experimental and clinical settings. Here we propose a generalized framework with an iterative approach to refine both segmentation and clustering. Temporally aligned segmentation and clustering (TASC) uses temporal linear alignment to compute distances between and align the recurring behavior motifs in a multidimensional time series, enabling precise segmentation and clustering. We introduce an alternating-step process: evaluation of temporal neighbors against current cluster centroids using linear alignment, alternating with selecting the best non-overlapping segments and their subsequent re-clustering. The framework is evaluated on semi-synthetic and real-world experimental and clinical data, demonstrating enhanced segmentation and clustering, offering a better foundation for consequent research. The framework may be used to extend existing tools in the field of behavior research and may be applied to other domains requiring high precision of time series segmentation.
Collapse
Affiliation(s)
- Ekaterina Zinkovskaia
- Gonda Multidisciplinary Brain Research Center, Bar-Ilan University, Ramat Gan, Israel
| | - Orel Tahary
- Gonda Multidisciplinary Brain Research Center, Bar-Ilan University, Ramat Gan, Israel
| | - Yocheved Loewenstern
- Gonda Multidisciplinary Brain Research Center, Bar-Ilan University, Ramat Gan, Israel
| | - Noa Benaroya-Milshtein
- Department of Psychological Medicine, The Neuropsychiatric Tourette Clinic, Schneider Children's Medical Center of Israel, Petah Tikva, Israel
- School of Medicine, Faculty of Medical and Health Sciences, Tel Aviv University, Tel Aviv, Israel
| | - Izhar Bar-Gad
- Gonda Multidisciplinary Brain Research Center, Bar-Ilan University, Ramat Gan, Israel.
| |
Collapse
|
2
|
Lin S, Gillis WF, Weinreb C, Zeine A, Jones SC, Robinson EM, Markowitz J, Datta SR. Characterizing the structure of mouse behavior using Motion Sequencing. Nat Protoc 2024:10.1038/s41596-024-01015-w. [PMID: 38926589 DOI: 10.1038/s41596-024-01015-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Accepted: 04/12/2024] [Indexed: 06/28/2024]
Abstract
Spontaneous mouse behavior is composed from repeatedly used modules of movement (e.g., rearing, running or grooming) that are flexibly placed into sequences whose content evolves over time. By identifying behavioral modules and the order in which they are expressed, researchers can gain insight into the effect of drugs, genes, context, sensory stimuli and neural activity on natural behavior. Here we present a protocol for performing Motion Sequencing (MoSeq), an ethologically inspired method that uses three-dimensional machine vision and unsupervised machine learning to decompose spontaneous mouse behavior into a series of elemental modules called 'syllables'. This protocol is based upon a MoSeq pipeline that includes modules for depth video acquisition, data preprocessing and modeling, as well as a standardized set of visualization tools. Users are provided with instructions and code for building a MoSeq imaging rig and acquiring three-dimensional video of spontaneous mouse behavior for submission to the modeling framework; the outputs of this protocol include syllable labels for each frame of the video data as well as summary plots describing how often each syllable was used and how syllables transitioned from one to the other. In addition, we provide instructions for analyzing and visualizing the outputs of keypoint-MoSeq, a recently developed variant of MoSeq that can identify behavioral motifs from keypoints identified from standard (rather than depth) video. This protocol and the accompanying pipeline significantly lower the bar for users without extensive computational ethology experience to adopt this unsupervised, data-driven approach to characterize mouse behavior.
Collapse
Affiliation(s)
- Sherry Lin
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | | | - Caleb Weinreb
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Ayman Zeine
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Samuel C Jones
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Emma M Robinson
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
| | - Jeffrey Markowitz
- Department of Neurobiology, Harvard Medical School, Boston, MA, USA
- Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA
| | | |
Collapse
|
3
|
Ravan A, Feng R, Gruebele M, Chemla YR. Rapid automated 3-D pose estimation of larval zebrafish using a physical model-trained neural network. PLoS Comput Biol 2023; 19:e1011566. [PMID: 37871114 PMCID: PMC10621986 DOI: 10.1371/journal.pcbi.1011566] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Revised: 11/02/2023] [Accepted: 10/02/2023] [Indexed: 10/25/2023] Open
Abstract
Quantitative ethology requires an accurate estimation of an organism's postural dynamics in three dimensions plus time. Technological progress over the last decade has made animal pose estimation in challenging scenarios possible with unprecedented detail. Here, we present (i) a fast automated method to record and track the pose of individual larval zebrafish in a 3-D environment, applicable when accurate human labeling is not possible; (ii) a rich annotated dataset of 3-D larval poses for ethologists and the general zebrafish and machine learning community; and (iii) a technique to generate realistic, annotated larval images in different behavioral contexts. Using a three-camera system calibrated with refraction correction, we record diverse larval swims under free swimming conditions and in response to acoustic and optical stimuli. We then employ a convolutional neural network to estimate 3-D larval poses from video images. The network is trained against a set of synthetic larval images rendered using a 3-D physical model of larvae. This 3-D model samples from a distribution of realistic larval poses that we estimate a priori using a template-based pose estimation of a small number of swim bouts. Our network model, trained without any human annotation, performs larval pose estimation three orders of magnitude faster and with accuracy comparable to the template-based approach, capturing detailed kinematics of 3-D larval swims. It also applies accurately to other datasets collected under different imaging conditions and containing behavioral contexts not included in our training.
Collapse
Affiliation(s)
- Aniket Ravan
- Center for Biophysics and Quantitative Biology, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
- Center for the Physics of Living Cells, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
| | - Ruopei Feng
- Center for the Physics of Living Cells, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
- Department of Chemistry, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
| | - Martin Gruebele
- Center for Biophysics and Quantitative Biology, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
- Center for the Physics of Living Cells, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
- Department of Chemistry, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
- Department of Physics, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
| | - Yann R. Chemla
- Center for Biophysics and Quantitative Biology, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
- Center for the Physics of Living Cells, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
- Department of Physics, University of Illinois at Urbana-Champaign, Urbana, Illinois, United States of America
| |
Collapse
|
4
|
Maselli A, Gordon J, Eluchans M, Lancia GL, Thiery T, Moretti R, Cisek P, Pezzulo G. Beyond simple laboratory studies: Developing sophisticated models to study rich behavior. Phys Life Rev 2023; 46:220-244. [PMID: 37499620 DOI: 10.1016/j.plrev.2023.07.006] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Accepted: 07/06/2023] [Indexed: 07/29/2023]
Abstract
Psychology and neuroscience are concerned with the study of behavior, of internal cognitive processes, and their neural foundations. However, most laboratory studies use constrained experimental settings that greatly limit the range of behaviors that can be expressed. While focusing on restricted settings ensures methodological control, it risks impoverishing the object of study: by restricting behavior, we might miss key aspects of cognitive and neural functions. In this article, we argue that psychology and neuroscience should increasingly adopt innovative experimental designs, measurement methods, analysis techniques and sophisticated computational models to probe rich, ecologically valid forms of behavior, including social behavior. We discuss the challenges of studying rich forms of behavior as well as the novel opportunities offered by state-of-the-art methodologies and new sensing technologies, and we highlight the importance of developing sophisticated formal models. We exemplify our arguments by reviewing some recent streams of research in psychology, neuroscience and other fields (e.g., sports analytics, ethology and robotics) that have addressed rich forms of behavior in a model-based manner. We hope that these "success cases" will encourage psychologists and neuroscientists to extend their toolbox of techniques with sophisticated behavioral models - and to use them to study rich forms of behavior as well as the cognitive and neural processes that they engage.
Collapse
Affiliation(s)
- Antonella Maselli
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Jeremy Gordon
- University of California, Berkeley, Berkeley, CA, 94704, United States
| | - Mattia Eluchans
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Gian Luca Lancia
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Thomas Thiery
- Department of Psychology, University of Montréal, Montréal, Québec, Canada
| | - Riccardo Moretti
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Paul Cisek
- Department of Neuroscience, University of Montréal, Montréal, Québec, Canada
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| |
Collapse
|
5
|
Van Dam EA, Noldus LPJJ, Van Gerven MAJ. Disentangling rodent behaviors to improve automated behavior recognition. Front Neurosci 2023; 17:1198209. [PMID: 37496740 PMCID: PMC10366600 DOI: 10.3389/fnins.2023.1198209] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 06/12/2023] [Indexed: 07/28/2023] Open
Abstract
Automated observation and analysis of behavior is important to facilitate progress in many fields of science. Recent developments in deep learning have enabled progress in object detection and tracking, but rodent behavior recognition struggles to exceed 75-80% accuracy for ethologically relevant behaviors. We investigate the main reasons why and distinguish three aspects of behavior dynamics that are difficult to automate. We isolate these aspects in an artificial dataset and reproduce effects with the state-of-the-art behavior recognition models. Having an endless amount of labeled training data with minimal input noise and representative dynamics will enable research to optimize behavior recognition architectures and get closer to human-like recognition performance for behaviors with challenging dynamics.
Collapse
Affiliation(s)
- Elsbeth A. Van Dam
- Department of Artificial Intelligence, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
- Noldus Information Technology BV, Wageningen, Netherlands
| | - Lucas P. J. J. Noldus
- Noldus Information Technology BV, Wageningen, Netherlands
- Department of Biophysics, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
| | - Marcel A. J. Van Gerven
- Department of Artificial Intelligence, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
| |
Collapse
|
6
|
Shemesh Y, Chen A. A paradigm shift in translational psychiatry through rodent neuroethology. Mol Psychiatry 2023; 28:993-1003. [PMID: 36635579 PMCID: PMC10005947 DOI: 10.1038/s41380-022-01913-z] [Citation(s) in RCA: 21] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/06/2022] [Revised: 12/05/2022] [Accepted: 12/07/2022] [Indexed: 01/14/2023]
Abstract
Mental disorders are a significant cause of disability worldwide. They profoundly affect individuals' well-being and impose a substantial financial burden on societies and governments. However, despite decades of extensive research, the effectiveness of current therapeutics for mental disorders is often not satisfactory or well tolerated by the patient. Moreover, most novel therapeutic candidates fail in clinical testing during the most expensive phases (II and III), which results in the withdrawal of pharma companies from investing in the field. It also brings into question the effectiveness of using animal models in preclinical studies to discover new therapeutic agents and predict their potential for treating mental illnesses in humans. Here, we focus on rodents as animal models and propose that they are essential for preclinical investigations of candidate therapeutic agents' mechanisms of action and for testing their safety and efficiency. Nevertheless, we argue that there is a need for a paradigm shift in the methodologies used to measure animal behavior in laboratory settings. Specifically, behavioral readouts obtained from short, highly controlled tests in impoverished environments and social contexts as proxies for complex human behavioral disorders might be of limited face validity. Conversely, animal models that are monitored in more naturalistic environments over long periods display complex and ethologically relevant behaviors that reflect evolutionarily conserved endophenotypes of translational value. We present how semi-natural setups in which groups of mice are individually tagged, and video recorded continuously can be attainable and affordable. Moreover, novel open-source machine-learning techniques for pose estimation enable continuous and automatic tracking of individual body parts in groups of rodents over long periods. The trajectories of each individual animal can further be subjected to supervised machine learning algorithms for automatic detection of specific behaviors (e.g., chasing, biting, or fleeing) or unsupervised automatic detection of behavioral motifs (e.g., stereotypical movements that might be harder to name or label manually). Compared to studies of animals in the wild, semi-natural environments are more compatible with neural and genetic manipulation techniques. As such, they can be used to study the neurobiological mechanisms underlying naturalistic behavior. Hence, we suggest that such a paradigm possesses the best out of classical ethology and the reductive behaviorist approach and may provide a breakthrough in discovering new efficient therapies for mental illnesses.
Collapse
Affiliation(s)
- Yair Shemesh
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot, 7610001, Israel
- Department of Molecular Neuroscience, Weizmann Institute of Science, Rehovot, 7610001, Israel
| | - Alon Chen
- Department of Brain Sciences, Weizmann Institute of Science, Rehovot, 7610001, Israel.
- Department of Molecular Neuroscience, Weizmann Institute of Science, Rehovot, 7610001, Israel.
- Department of Stress Neurobiology and Neurogenetics, Max Planck Institute of Psychiatry, 80804, Munich, Germany.
| |
Collapse
|
7
|
Barrett LF. Context reconsidered: Complex signal ensembles, relational meaning, and population thinking in psychological science. AMERICAN PSYCHOLOGIST 2022; 77:894-920. [PMID: 36409120 PMCID: PMC9683522 DOI: 10.1037/amp0001054] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/26/2023]
Abstract
This article considers the status and study of "context" in psychological science through the lens of research on emotional expressions. The article begins by updating three well-trod methodological debates on the role of context in emotional expressions to reconsider several fundamental assumptions lurking within the field's dominant methodological tradition: namely, that certain expressive movements have biologically prepared, inherent emotional meanings that issue from singular, universal processes which are independent of but interact with contextual influences. The second part of this article considers the scientific opportunities that await if we set aside this traditional understanding of "context" as a moderator of signals with inherent psychological meaning and instead consider the possibility that psychological events emerge in ecosystems of signal ensembles, such that the psychological meaning of any individual signal is entirely relational. Such a fundamental shift has radical implications not only for the science of emotion but for psychological science more generally. It offers opportunities to improve the validity and trustworthiness of psychological science beyond what can be achieved with improvements to methodological rigor alone. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
Collapse
|
8
|
Bermudez Contreras E, Sutherland RJ, Mohajerani MH, Whishaw IQ. Challenges of a small world analysis for the continuous monitoring of behavior in mice. Neurosci Biobehav Rev 2022; 136:104621. [PMID: 35307475 DOI: 10.1016/j.neubiorev.2022.104621] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2021] [Revised: 02/14/2022] [Accepted: 03/11/2022] [Indexed: 12/18/2022]
Abstract
Documenting a mouse's "real world" behavior in the "small world" of a laboratory cage with continuous video recordings offers insights into phenotypical expression of mouse genotypes, development and aging, and neurological disease. Nevertheless, there are challenges in the design of a small world, the behavior selected for analysis, and the form of the analysis used. Here we offer insights into small world analyses by describing how acute behavioral procedures can guide continuous behavioral methodology. We show how algorithms can identify behavioral acts including walking and rearing, circadian patterns of action including sleep duration and waking activity, and the organization of patterns of movement into home base activity and excursions, and how they are altered with aging. We additionally describe how specific tests can be incorporated within a mouse's living arrangement. We emphasize how machine learning can condense and organize continuous activity that extends over extended periods of time.
Collapse
Affiliation(s)
| | - Robert J Sutherland
- Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Canada
| | - Majid H Mohajerani
- Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Canada.
| | - Ian Q Whishaw
- Canadian Centre for Behavioural Neuroscience, University of Lethbridge, Canada
| |
Collapse
|
9
|
|
10
|
León A, Hernandez V, Lopez J, Guzman I, Quintero V, Toledo P, Avendaño-Garrido ML, Hernandez-Linares CA, Escamilla E. Beyond Single Discrete Responses: An Integrative and Multidimensional Analysis of Behavioral Dynamics Assisted by Machine Learning. Front Behav Neurosci 2021; 15:681771. [PMID: 34737691 PMCID: PMC8562345 DOI: 10.3389/fnbeh.2021.681771] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Accepted: 08/26/2021] [Indexed: 11/13/2022] Open
Abstract
Understanding behavioral systems as emergent systems comprising the environment and organism subsystems, include spatial dynamics as a primary dimension in natural settings. Nevertheless, under the standard approaches, the experimental analysis of behavior is based on the single response paradigm and the temporal distribution of discrete responses. Thus, the continuous analysis of spatial behavioral dynamics is a scarcely studied field. The technological advancements in computer vision have opened new methodological perspectives for the continuous sensing of spatial behavior. With the application of such advancements, recent studies suggest that there are multiple features embedded in the spatial dynamics of behavior, such as entropy, and that they are affected by programmed stimuli (e.g., schedules of reinforcement) at least as much as features related to discrete responses. Despite the progress, the characterization of behavioral systems is still segmented, and integrated data analysis and representations between discrete responses and continuous spatial behavior are exiguous in the experimental analysis of behavior. Machine learning advancements, such as t-distributed stochastic neighbor embedding and variable ranking, provide invaluable tools to crystallize an integrated approach for analyzing and representing multidimensional behavioral data. Under this rationale, the present work (1) proposes a multidisciplinary approach for the integrative and multilevel analysis of behavioral systems, (2) provides sensitive behavioral measures based on spatial dynamics and helpful data representations to study behavioral systems, and (3) reveals behavioral aspects usually ignored under the standard approaches in the experimental analysis of behavior. To exemplify and evaluate our approach, the spatial dynamics embedded in phenomena relevant to behavioral science, namely, water-seeking behavior and motivational operations, are examined, showing aspects of behavioral systems hidden until now.
Collapse
Affiliation(s)
- Alejandro León
- Comparative Psychology Laboratory, Centro de Estudios e Investigaciones en Conocimiento y Aprendizaje Humano, Universidad Veracruzana, Xalapa, Mexico
| | - Varsovia Hernandez
- Comparative Psychology Laboratory, Centro de Estudios e Investigaciones en Conocimiento y Aprendizaje Humano, Universidad Veracruzana, Xalapa, Mexico
| | - Juan Lopez
- Facultad de Estadística e Informática, Universidad Veracruzana, Xalapa, Mexico
| | - Isiris Guzman
- Comparative Psychology Laboratory, Centro de Estudios e Investigaciones en Conocimiento y Aprendizaje Humano, Universidad Veracruzana, Xalapa, Mexico
| | - Victor Quintero
- Comparative Psychology Laboratory, Centro de Estudios e Investigaciones en Conocimiento y Aprendizaje Humano, Universidad Veracruzana, Xalapa, Mexico
| | - Porfirio Toledo
- Facultad de Matemáticas, Universidad Veracruzana, Xalapa, Mexico
| | | | | | | |
Collapse
|
11
|
MouseVenue3D: A Markerless Three-Dimension Behavioral Tracking System for Matching Two-Photon Brain Imaging in Free-Moving Mice. Neurosci Bull 2021; 38:303-317. [PMID: 34637091 PMCID: PMC8975979 DOI: 10.1007/s12264-021-00778-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/01/2021] [Accepted: 06/23/2021] [Indexed: 10/20/2022] Open
Abstract
Understanding the connection between brain and behavior in animals requires precise monitoring of their behaviors in three-dimensional (3-D) space. However, there is no available three-dimensional behavior capture system that focuses on rodents. Here, we present MouseVenue3D, an automated and low-cost system for the efficient capture of 3-D skeleton trajectories in markerless rodents. We improved the most time-consuming step in 3-D behavior capturing by developing an automatic calibration module. Then, we validated this process in behavior recognition tasks, and showed that 3-D behavioral data achieved higher accuracy than 2-D data. Subsequently, MouseVenue3D was combined with fast high-resolution miniature two-photon microscopy for synchronous neural recording and behavioral tracking in the freely-moving mouse. Finally, we successfully decoded spontaneous neuronal activity from the 3-D behavior of mice. Our findings reveal that subtle, spontaneous behavior modules are strongly correlated with spontaneous neuronal activity patterns.
Collapse
|
12
|
A review of 28 free animal-tracking software applications: current features and limitations. Lab Anim (NY) 2021; 50:246-254. [PMID: 34326537 DOI: 10.1038/s41684-021-00811-1] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2020] [Accepted: 06/24/2021] [Indexed: 11/09/2022]
Abstract
Well-quantified laboratory studies can provide a fundamental understanding of animal behavior in ecology, ethology and ecotoxicology research. These types of studies require observation and tracking of each animal in well-controlled and defined arenas, often for long timescales. Thus, these experiments produce long time series and a vast amount of data that require the use of software applications to automate the analysis and reduce manual annotation. In this review, we examine 28 free software applications for animal tracking to guide researchers in selecting the software that might best suit a particular experiment. We also review the algorithms in the tracking pipeline of the applications, explain how specific techniques can fit different experiments, and finally, expose each approach's weaknesses and strengths. Our in-depth review includes last update, type of platform, user-friendliness, off- or online video acquisition, calibration method, background subtraction and segmentation method, species, multiple arenas, multiple animals, identity preservation, manual identity correction, data analysis and extra features. We found, for example, that out of 28 programs, only 3 include a calibration algorithm to reduce image distortion and perspective problems that affect accuracy and can result in substantial errors when analyzing trajectories and extracting mobility or explored distance. In addition, only 4 programs can directly export in-depth tracking and analysis metrics, only 5 are suited for tracking multiple unmarked animals for more than a few seconds and only 11 have been updated in the period 2019-2021.
Collapse
|
13
|
Lost in translation: no effect of repeated optogenetic cortico-striatal stimulation on compulsivity in rats. Transl Psychiatry 2021; 11:315. [PMID: 34031365 PMCID: PMC8144623 DOI: 10.1038/s41398-021-01448-x] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/28/2021] [Revised: 04/30/2021] [Accepted: 05/11/2021] [Indexed: 11/30/2022] Open
Abstract
The orbitofrontal cortex-ventromedial striatum (OFC-VMS) circuitry is widely believed to drive compulsive behavior. Hyperactivating this pathway in inbred mice produces excessive and persistent self-grooming, which has been considered a model for human compulsivity. We aimed to replicate these findings in outbred rats, where there are few reliable compulsivity models. Male Long-Evans rats implanted with optical fibers into VMS and with opsins delivered into OFC received optical stimulation at parameters that produce OFC-VMS plasticity and compulsive grooming in mice. We then evaluated rats for compulsive self-grooming at six timepoints: before, during, immediately after, and 1 h after each stimulation, 1 and 2 weeks after the ending of a 6-day stimulation protocol. To further test for effects of OFC-VMS hyperstimulation, we ran animals in three standard compulsivity assays: marble burying, nestlet shredding, and operant attentional set-shifting. OFC-VMS stimulation did not increase self-grooming or induce significant changes in nestlet shredding, marble burying, or set-shifting in rats. Follow-on evoked potential studies verified that the stimulation protocol altered OFC-VMS synaptic weighting. In sum, although we induced physiological changes in the OFC-VMS circuitry, we could not reproduce in a strongly powered study in rats a model of compulsive behavior previously reported in mice. This suggests possible limitations to translation of mouse findings to species higher on the phylogenetic chain.
Collapse
|
14
|
Huang K, Han Y, Chen K, Pan H, Zhao G, Yi W, Li X, Liu S, Wei P, Wang L. A hierarchical 3D-motion learning framework for animal spontaneous behavior mapping. Nat Commun 2021; 12:2784. [PMID: 33986265 PMCID: PMC8119960 DOI: 10.1038/s41467-021-22970-y] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2020] [Accepted: 04/06/2021] [Indexed: 02/03/2023] Open
Abstract
Animal behavior usually has a hierarchical structure and dynamics. Therefore, to understand how the neural system coordinates with behaviors, neuroscientists need a quantitative description of the hierarchical dynamics of different behaviors. However, the recent end-to-end machine-learning-based methods for behavior analysis mostly focus on recognizing behavioral identities on a static timescale or based on limited observations. These approaches usually lose rich dynamic information on cross-scale behaviors. Here, inspired by the natural structure of animal behaviors, we address this challenge by proposing a parallel and multi-layered framework to learn the hierarchical dynamics and generate an objective metric to map the behavior into the feature space. In addition, we characterize the animal 3D kinematics with our low-cost and efficient multi-view 3D animal motion-capture system. Finally, we demonstrate that this framework can monitor spontaneous behavior and automatically identify the behavioral phenotypes of the transgenic animal disease model. The extensive experiment results suggest that our framework has a wide range of applications, including animal disease model phenotyping and the relationships modeling between the neural circuits and behavior.
Collapse
Grants
- This work was supported in part by Key Area R&D Program of Guangdong Province (2018B030338001 P.W., 2018B030331001 L.W.), National Key R&D Program of China (2018YFA0701403 P.W.), National Natural Science Foundation of China (NSFC 31500861 P.W., NSFC 31630031 L.W., NSFC 91732304 L.W., NSFC 31930047 L.W.), Chang Jiang Scholars Program (L.W.), the International Big Science Program Cultivating Project of CAS (172644KYS820170004 L.W.), the Strategic Priority Research Program of Chinese Academy of Science (XDB32030100, L.W.), the Youth Innovation Promotion Association of the Chinese Academy of Sciences (2017413 P.W.), CAS Key Laboratory of Brain Connectome and Manipulation (2019DP173024), Shenzhen Government Basic Research Grants (JCYJ20170411140807570 P.W., JCYJ20170413164535041 L.W.), Science, Technology and Innovation Commission of Shenzhen Municipality (JCYJ20160429185235132 K.H.), Helmholtz-CAS joint research grant (GJHZ1508 L.W.), Guangdong Provincial Key Laboratory of Brain Connectome and Behavior (2017B030301017 L.W.), the Ten Thousand Talent Program (L.W.), the Guangdong Special Support Program (L.W.), Key Laboratory of SIAT (2019DP173024 L.W.), Shenzhen Key Science and Technology Infrastructure Planning Project (ZDKJ20190204002 L.W.).
Collapse
Affiliation(s)
- Kang Huang
- Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Yaning Han
- Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Ke Chen
- Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Hongli Pan
- Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Gaoyang Zhao
- Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Wenling Yi
- Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Xiaoxi Li
- Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
- University of Chinese Academy of Sciences, Beijing, China
| | - Siyuan Liu
- Pennsylvania State University, University Park, PA, USA
| | - Pengfei Wei
- Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
- University of Chinese Academy of Sciences, Beijing, China.
| | - Liping Wang
- Shenzhen Key Lab of Neuropsychiatric Modulation and Collaborative Innovation Center for Brain Science, Guangdong Provincial Key Laboratory of Brain Connectome and Behavior, CAS Center for Excellence in Brain Science and Intelligence Technology, Brain Cognition and Brain Disease Institute (BCBDI), Shenzhen-Hong Kong Institute of Brain Science-Shenzhen Fundamental Research Institutions, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
- University of Chinese Academy of Sciences, Beijing, China.
| |
Collapse
|
15
|
Singh SH, Peterson SM, Rao RPN, Brunton BW. Mining naturalistic human behaviors in long-term video and neural recordings. J Neurosci Methods 2021; 358:109199. [PMID: 33910024 DOI: 10.1016/j.jneumeth.2021.109199] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2020] [Revised: 04/07/2021] [Accepted: 04/19/2021] [Indexed: 10/21/2022]
Abstract
BACKGROUND Recent technological advances in brain recording and machine learning algorithms are enabling the study of neural activity underlying spontaneous human behaviors, beyond the confines of cued, repeated trials. However, analyzing such unstructured data lacking a priori experimental design remains a significant challenge, especially when the data is multi-modal and long-term. NEW METHOD Here we describe an automated, behavior-first approach for analyzing simultaneously recorded long-term, naturalistic electrocorticography (ECoG) and behavior video data. We identify and characterize spontaneous human upper-limb movements by combining computer vision, discrete latent-variable modeling, and string pattern-matching on the video. RESULTS Our pipeline discovers and annotates over 40,000 instances of naturalistic arm movements in long term (7-9 day) behavioral videos, across 12 subjects. Analysis of the simultaneously recorded brain data reveals neural signatures of movement that corroborate previous findings. Our pipeline produces large training datasets for brain-computer interfacing applications, and we show decoding results from a movement initiation detection task. COMPARISON WITH EXISTING METHODS Spontaneous movements capture real-world neural and behavior variability that is missing from traditional cued tasks. Building beyond window-based movement detection metrics, our unsupervised discretization scheme produces a queryable pose representation, allowing localization of movements with finer temporal resolution. CONCLUSIONS Our work addresses the unique analytic challenges of studying naturalistic human behaviors and contributes methods that may generalize to other neural recording modalities beyond ECoG. We publish our curated dataset and believe that it will be a valuable resource for future studies of naturalistic movements.
Collapse
Affiliation(s)
- Satpreet H Singh
- Department of Electrical and Computer Engineering, University of Washington, Seattle, USA
| | - Steven M Peterson
- Department of Biology, University of Washington, Seattle, USA; eScience Institute, University of Washington, Seattle, USA
| | - Rajesh P N Rao
- Department of Electrical and Computer Engineering, University of Washington, Seattle, USA; Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, USA; Center for Neurotechnology, University of Washington, Seattle, USA; University of Washington Institute for Neuroengineering, Seattle, USA
| | - Bingni W Brunton
- Department of Biology, University of Washington, Seattle, USA; eScience Institute, University of Washington, Seattle, USA; University of Washington Institute for Neuroengineering, Seattle, USA.
| |
Collapse
|
16
|
Dennis EJ, El Hady A, Michaiel A, Clemens A, Tervo DRG, Voigts J, Datta SR. Systems Neuroscience of Natural Behaviors in Rodents. J Neurosci 2021; 41:911-919. [PMID: 33443081 PMCID: PMC7880287 DOI: 10.1523/jneurosci.1877-20.2020] [Citation(s) in RCA: 31] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2020] [Revised: 10/15/2020] [Accepted: 10/20/2020] [Indexed: 11/21/2022] Open
Abstract
Animals evolved in complex environments, producing a wide range of behaviors, including navigation, foraging, prey capture, and conspecific interactions, which vary over timescales ranging from milliseconds to days. Historically, these behaviors have been the focus of study for ecology and ethology, while systems neuroscience has largely focused on short timescale behaviors that can be repeated thousands of times and occur in highly artificial environments. Thanks to recent advances in machine learning, miniaturization, and computation, it is newly possible to study freely moving animals in more natural conditions while applying systems techniques: performing temporally specific perturbations, modeling behavioral strategies, and recording from large numbers of neurons while animals are freely moving. The authors of this review are a group of scientists with deep appreciation for the common aims of systems neuroscience, ecology, and ethology. We believe it is an extremely exciting time to be a neuroscientist, as we have an opportunity to grow as a field, to embrace interdisciplinary, open, collaborative research to provide new insights and allow researchers to link knowledge across disciplines, species, and scales. Here we discuss the origins of ethology, ecology, and systems neuroscience in the context of our own work and highlight how combining approaches across these fields has provided fresh insights into our research. We hope this review facilitates some of these interactions and alliances and helps us all do even better science, together.
Collapse
Affiliation(s)
- Emily Jane Dennis
- Princeton University and Howard Hughes Medical Institute, Princeton, New Jersey, 08540
| | - Ahmed El Hady
- Princeton University and Howard Hughes Medical Institute, Princeton, New Jersey, 08540
| | | | - Ann Clemens
- University of Edinburgh, Edinburgh, Scotland, EH8 9JZ
| | | | - Jakob Voigts
- Massachusetts Institute of Technology, Cambridge, Massachusets, 02139
| | | |
Collapse
|
17
|
Donnarumma F, Prevete R, Maisto D, Fuscone S, Irvine EM, van der Meer MAA, Kemere C, Pezzulo G. A framework to identify structured behavioral patterns within rodent spatial trajectories. Sci Rep 2021; 11:468. [PMID: 33432100 PMCID: PMC7801653 DOI: 10.1038/s41598-020-79744-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2020] [Accepted: 11/10/2020] [Indexed: 11/09/2022] Open
Abstract
Animal behavior is highly structured. Yet, structured behavioral patterns-or "statistical ethograms"-are not immediately apparent from the full spatiotemporal data that behavioral scientists usually collect. Here, we introduce a framework to quantitatively characterize rodent behavior during spatial (e.g., maze) navigation, in terms of movement building blocks or motor primitives. The hypothesis that we pursue is that rodent behavior is characterized by a small number of motor primitives, which are combined over time to produce open-ended movements. We assume motor primitives to be organized in terms of two sparsity principles: each movement is controlled using a limited subset of motor primitives (sparse superposition) and each primitive is active only for time-limited, time-contiguous portions of movements (sparse activity). We formalize this hypothesis using a sparse dictionary learning method, which we use to extract motor primitives from rodent position and velocity data collected during spatial navigation, and successively to reconstruct past trajectories and predict novel ones. Three main results validate our approach. First, rodent behavioral trajectories are robustly reconstructed from incomplete data, performing better than approaches based on standard dimensionality reduction methods, such as principal component analysis, or single sparsity. Second, the motor primitives extracted during one experimental session generalize and afford the accurate reconstruction of rodent behavior across successive experimental sessions in the same or in modified mazes. Third, in our approach the number of motor primitives associated with each maze correlates with independent measures of maze complexity, hence showing that our formalism is sensitive to essential aspects of task structure. The framework introduced here can be used by behavioral scientists and neuroscientists as an aid for behavioral and neural data analysis. Indeed, the extracted motor primitives enable the quantitative characterization of the complexity and similarity between different mazes and behavioral patterns across multiple trials (i.e., habit formation). We provide example uses of this computational framework, showing how it can be used to identify behavioural effects of maze complexity, analyze stereotyped behavior, classify behavioral choices and predict place and grid cell displacement in novel environments.
Collapse
Affiliation(s)
- Francesco Donnarumma
- Institute of Cognitive Sciences and Technologies (ISTC), National Research Council (CNR), Via San Martino della Battaglia 44, 00185, Rome, Italy
| | - Roberto Prevete
- Department of Electric Engineering and Information Technologies (DIETI), Università degli Studi di Napoli Federico II, Naples, Italy
| | - Domenico Maisto
- Institute for High Performance Computing and Networking (ICAR), National Research Council (CNR), Via Pietro Castellino 111, 80131, Naples, Italy
| | | | - Emily M Irvine
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | | | - Caleb Kemere
- Electrical and Computer Engineering, Rice University, Houston, TX, USA
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies (ISTC), National Research Council (CNR), Via San Martino della Battaglia 44, 00185, Rome, Italy.
| |
Collapse
|
18
|
Revealing the structure of pharmacobehavioral space through motion sequencing. Nat Neurosci 2020; 23:1433-1443. [PMID: 32958923 PMCID: PMC7606807 DOI: 10.1038/s41593-020-00706-3] [Citation(s) in RCA: 107] [Impact Index Per Article: 26.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2019] [Accepted: 08/10/2020] [Indexed: 12/15/2022]
Abstract
Understanding how genes, drugs and neural circuits influence behavior requires the ability to effectively organize information about similarities and differences within complex behavioral datasets. Motion Sequencing (MoSeq) is an ethologically-inspired behavioral analysis method that identifies modular components of 3D mouse body language called “syllables.” Here we show that MoSeq effectively parses behavioral differences and captures similarities elicited by a panel of neuro- and psychoactive drugs administered to a cohort of nearly 700 mice. MoSeq identifies syllables that are characteristic of individual drugs; we leverage this finding to reveal specific on- and off-target effects of both established and candidate therapeutics in a mouse model of autism spectrum disorder. These results demonstrate that MoSeq can meaningfully organize large-scale behavioral data, illustrate the power of a fundamentally modular description of behavior, and suggest that behavioral syllables represent a new class of druggable target.
Collapse
|
19
|
Procacci NM, Allen KM, Robb GE, Ijekah R, Lynam H, Hoy JL. Context-dependent modulation of natural approach behaviour in mice. Proc Biol Sci 2020; 287:20201189. [PMID: 32873203 PMCID: PMC7542797 DOI: 10.1098/rspb.2020.1189] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2020] [Accepted: 08/07/2020] [Indexed: 01/09/2023] Open
Abstract
Specific features of visual objects innately draw approach responses in animals, and provide natural signals of potential reward. However, visual sampling behaviours and the detection of salient, rewarding stimuli are context and behavioural state-dependent and it remains unclear how visual perception and orienting responses change with specific expectations. To start to address this question, we employed a virtual stimulus orienting paradigm based on prey capture to quantify the conditional expression of visual stimulus-evoked innate approaches in freely moving mice. We found that specific combinations of stimulus features selectively evoked innate approach or freezing responses when stimuli were unexpected. We discovered that prey capture experience, and therefore the expectation of prey in the environment, selectively modified approach frequency, as well as altered those visual features that evoked approach. Thus, we found that mice exhibit robust and selective orienting responses to parameterized visual stimuli that can be robustly and specifically modified via natural experience. This work provides critical insight into how natural appetitive behaviours are driven by both specific features of visual motion and internal states that alter stimulus salience.
Collapse
Affiliation(s)
| | - Kelsey M. Allen
- Department of Biology, University of Nevada, Reno, NV 89557, USA
| | - Gael E. Robb
- Department of Neuroscience, University of St Thomas, St Paul, MN 55105, USA
| | - Rebecca Ijekah
- Department of Physiology and Cell Biology, University of Nevada, Reno, NV 89557, USA
| | - Hudson Lynam
- Department of Computer Science and Engineering, University of Nevada, Reno, NV 89557, USA
| | - Jennifer L. Hoy
- Department of Biology, University of Nevada, Reno, NV 89557, USA
| |
Collapse
|
20
|
Corcoran CM, Cecchi GA. Using Language Processing and Speech Analysis for the Identification of Psychosis and Other Disorders. BIOLOGICAL PSYCHIATRY: COGNITIVE NEUROSCIENCE AND NEUROIMAGING 2020; 5:770-779. [PMID: 32771179 DOI: 10.1016/j.bpsc.2020.06.004] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/16/2020] [Revised: 06/09/2020] [Accepted: 06/09/2020] [Indexed: 01/12/2023]
Abstract
Increasingly, data-driven methods have been implemented to understand psychopathology. Language is the main source of information in psychiatry and represents "big data" at the level of the individual. Language and behavior are amenable to computational natural language processing (NLP) analytics, which may help operationalize the mental status examination. In this review, we highlight the application of NLP to schizophrenia and its risk states as an exemplar of its use, operationalizing tangential and concrete speech as reductions in semantic coherence and syntactic complexity, respectively. Other clinical applications are reviewed, including forecasting suicide risk and detecting intoxication. Challenges and future directions are discussed, including biomarker development, harmonization, and application of NLP more broadly to behavior, including intonation/prosody, facial expression and gesture, and the integration of these in dyads and during discourse. Similar NLP analytics can also be applied beyond humans to behavioral motifs across species, important for modeling psychopathology in animal models. Finally, clinical neuroscience can inform the development of artificial intelligence.
Collapse
Affiliation(s)
- Cheryl Mary Corcoran
- Icahn School of Medicine at Mount Sinai, New York; James J. Peters Veterans Administration Medical Center, Bronx.
| | - Guillermo A Cecchi
- Thomas J. Watson Research Center, IBM Corporation, Yorktown Heights, New York
| |
Collapse
|
21
|
Datta SR, Anderson DJ, Branson K, Perona P, Leifer A. Computational Neuroethology: A Call to Action. Neuron 2019; 104:11-24. [PMID: 31600508 PMCID: PMC6981239 DOI: 10.1016/j.neuron.2019.09.038] [Citation(s) in RCA: 191] [Impact Index Per Article: 38.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2019] [Revised: 09/16/2019] [Accepted: 09/23/2019] [Indexed: 12/11/2022]
Abstract
The brain is worthy of study because it is in charge of behavior. A flurry of recent technical advances in measuring and quantifying naturalistic behaviors provide an important opportunity for advancing brain science. However, the problem of understanding unrestrained behavior in the context of neural recordings and manipulations remains unsolved, and developing approaches to addressing this challenge is critical. Here we discuss considerations in computational neuroethology-the science of quantifying naturalistic behaviors for understanding the brain-and propose strategies to evaluate progress. We point to open questions that require resolution and call upon the broader systems neuroscience community to further develop and leverage measures of naturalistic, unrestrained behavior, which will enable us to more effectively probe the richness and complexity of the brain.
Collapse
Affiliation(s)
| | - David J Anderson
- Division of Biology and Biological Engineering 156-29, California Institute of Technology, Pasadena, CA 91125, USA; Howard Hughes Medical Institute, Pasadena, CA, 91125, USA; Tianqiao and Chrissy Chen Institute for Neuroscience, California Institute of Technology, Pasadena, CA 91125, USA
| | - Kristin Branson
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, VA 20147, USA
| | - Pietro Perona
- Division of Engineering & Applied Sciences 136-93, California Institute of Technology, Pasadena, CA 91125, USA
| | - Andrew Leifer
- Department of Physics, Princeton University, Princeton, NJ 08544, USA; Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, USA.
| |
Collapse
|