1
|
Silva N, Zhang D, Kulvicius T, Gail A, Barreiros C, Lindstaedt S, Kraft M, Bölte S, Poustka L, Nielsen-Saines K, Wörgötter F, Einspieler C, Marschik PB. The future of General Movement Assessment: The role of computer vision and machine learning - A scoping review. RESEARCH IN DEVELOPMENTAL DISABILITIES 2021; 110:103854. [PMID: 33571849 PMCID: PMC7910279 DOI: 10.1016/j.ridd.2021.103854] [Citation(s) in RCA: 52] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/30/2020] [Revised: 01/03/2021] [Accepted: 01/05/2021] [Indexed: 05/03/2023]
Abstract
BACKGROUND The clinical and scientific value of Prechtl general movement assessment (GMA) has been increasingly recognised, which has extended beyond the detection of cerebral palsy throughout the years. With advancing computer science, a surging interest in developing automated GMA emerges. AIMS In this scoping review, we focused on video-based approaches, since it remains authentic to the non-intrusive principle of the classic GMA. Specifically, we aimed to provide an overview of recent video-based approaches targeting GMs; identify their techniques for movement detection and classification; examine if the technological solutions conform to the fundamental concepts of GMA; and discuss the challenges of developing automated GMA. METHODS AND PROCEDURES We performed a systematic search for computer vision-based studies on GMs. OUTCOMES AND RESULTS We identified 40 peer-reviewed articles, most (n = 30) were published between 2017 and 2020. A wide variety of sensing, tracking, detection, and classification tools for computer vision-based GMA were found. Only a small portion of these studies applied deep learning approaches. A comprehensive comparison between data acquisition and sensing setups across the reviewed studies, highlighting limitations and advantages of each modality in performing automated GMA is provided. CONCLUSIONS AND IMPLICATIONS A "method-of-choice" for automated GMA does not exist. Besides creating large datasets, understanding the fundamental concepts and prerequisites of GMA is necessary for developing automated solutions. Future research shall look beyond the narrow field of detecting cerebral palsy and open up to the full potential of applying GMA to enable an even broader application.
Collapse
Affiliation(s)
- Nelson Silva
- iDN - Interdisciplinary Developmental Neuroscience, Division of Phoniatrics, Medical University of Graz, Graz, Austria; Know-Center GmbH, Graz, Austria
| | - Dajie Zhang
- iDN - Interdisciplinary Developmental Neuroscience, Division of Phoniatrics, Medical University of Graz, Graz, Austria; Child and Adolescent Psychiatry and Psychotherapy, University Medical Center Göttingen, Göttingen, Germany; Leibniz-ScienceCampus Primate Cognition, Göttingen, Germany
| | - Tomas Kulvicius
- Department for Computational Neuroscience, Third Institute of Physics-Biophysics, Georg-August-University of Göttingen, Göttingen, Germany
| | - Alexander Gail
- Leibniz-ScienceCampus Primate Cognition, Göttingen, Germany; German Primate Center - Leibniz Institute for Primate Research, Göttingen, Germany
| | - Carla Barreiros
- Know-Center GmbH, Graz, Austria; Institute of Interactive Systems and Data Science, Graz University of Technology, Graz, Austria
| | - Stefanie Lindstaedt
- Know-Center GmbH, Graz, Austria; Institute of Interactive Systems and Data Science, Graz University of Technology, Graz, Austria
| | - Marc Kraft
- Department of Medical Engineering, Technical University Berlin, Berlin, Germany
| | - Sven Bölte
- Center of Neurodevelopmental Disorders (KIND), Centre for Psychiatry Research, Department of Women's and Children's Health, Karolinska Institutet, Stockholm, Sweden; Child and Adolescent Psychiatry, Stockholm Health Care Services, Region Stockholm, Stockholm, Sweden; Curtin Autism Research Group, School of Occupational Therapy, Social Work and Speech Pathology, Curtin University, Perth, Western Australia, Australia
| | - Luise Poustka
- Child and Adolescent Psychiatry and Psychotherapy, University Medical Center Göttingen, Göttingen, Germany; Leibniz-ScienceCampus Primate Cognition, Göttingen, Germany
| | - Karin Nielsen-Saines
- Division of Pediatric Infectious Diseases, David Geffen UCLA School of Medicine, USA
| | - Florentin Wörgötter
- Leibniz-ScienceCampus Primate Cognition, Göttingen, Germany; Department for Computational Neuroscience, Third Institute of Physics-Biophysics, Georg-August-University of Göttingen, Göttingen, Germany; Institute of Physics, Department for Computational Neuroscience at the Bernstein Center Göttingen, Georg-August-University of Göttingen, Göttingen, Germany
| | - Christa Einspieler
- iDN - Interdisciplinary Developmental Neuroscience, Division of Phoniatrics, Medical University of Graz, Graz, Austria
| | - Peter B Marschik
- iDN - Interdisciplinary Developmental Neuroscience, Division of Phoniatrics, Medical University of Graz, Graz, Austria; Child and Adolescent Psychiatry and Psychotherapy, University Medical Center Göttingen, Göttingen, Germany; Leibniz-ScienceCampus Primate Cognition, Göttingen, Germany; Center of Neurodevelopmental Disorders (KIND), Centre for Psychiatry Research, Department of Women's and Children's Health, Karolinska Institutet, Stockholm, Sweden.
| |
Collapse
|
2
|
Doroniewicz I, Ledwoń DJ, Affanasowicz A, Kieszczyńska K, Latos D, Matyja M, Mitas AW, Myśliwiec A. Writhing Movement Detection in Newborns on the Second and Third Day of Life Using Pose-Based Feature Machine Learning Classification. SENSORS 2020; 20:s20215986. [PMID: 33105787 PMCID: PMC7660095 DOI: 10.3390/s20215986] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 09/27/2020] [Revised: 10/16/2020] [Accepted: 10/20/2020] [Indexed: 11/16/2022]
Abstract
Observation of neuromotor development at an early stage of an infant’s life allows for early diagnosis of deficits and the beginning of the therapeutic process. General movement assessment is a method of spontaneous movement observation, which is the foundation for contemporary attempts at objectification and computer-aided diagnosis based on video recordings’ analysis. The present study attempts to automatically detect writhing movements, one of the normal general movement categories presented by newborns in the first weeks of life. A set of 31 recordings of newborns on the second and third day of life was divided by five experts into videos containing writhing movements (with occurrence time) and poor repertoire, characterized by a lower quality of movement in relation to the norm. Novel, objective pose-based features describing the scope, nature, and location of each limb’s movement are proposed. Three machine learning algorithms are evaluated in writhing movements’ detection in leave-one-out cross-validation for different feature extraction time windows and overlapping time. The experimental results make it possible to indicate the optimal parameters for which 80% accuracy was achieved. Based on automatically detected writhing movement percent in the video, infant movements are classified as writhing movements or poor repertoire with an area under the ROC (receiver operating characteristics) curve of 0.83.
Collapse
Affiliation(s)
- Iwona Doroniewicz
- Institute of Physiotherapy and Health Science, Academy of Physical Education in Katowice, 40-065 Katowice, Poland; (I.D.); (A.A.); (K.K.); (D.L.); (M.M.); (A.M.)
| | - Daniel J. Ledwoń
- Faculty of Biomedical Engineering, Silesian University of Technology, 41-800 Zabrze, Poland;
- Correspondence:
| | - Alicja Affanasowicz
- Institute of Physiotherapy and Health Science, Academy of Physical Education in Katowice, 40-065 Katowice, Poland; (I.D.); (A.A.); (K.K.); (D.L.); (M.M.); (A.M.)
| | - Katarzyna Kieszczyńska
- Institute of Physiotherapy and Health Science, Academy of Physical Education in Katowice, 40-065 Katowice, Poland; (I.D.); (A.A.); (K.K.); (D.L.); (M.M.); (A.M.)
| | - Dominika Latos
- Institute of Physiotherapy and Health Science, Academy of Physical Education in Katowice, 40-065 Katowice, Poland; (I.D.); (A.A.); (K.K.); (D.L.); (M.M.); (A.M.)
| | - Małgorzata Matyja
- Institute of Physiotherapy and Health Science, Academy of Physical Education in Katowice, 40-065 Katowice, Poland; (I.D.); (A.A.); (K.K.); (D.L.); (M.M.); (A.M.)
| | - Andrzej W. Mitas
- Faculty of Biomedical Engineering, Silesian University of Technology, 41-800 Zabrze, Poland;
| | - Andrzej Myśliwiec
- Institute of Physiotherapy and Health Science, Academy of Physical Education in Katowice, 40-065 Katowice, Poland; (I.D.); (A.A.); (K.K.); (D.L.); (M.M.); (A.M.)
| |
Collapse
|
3
|
Mathis A, Pack AR, Maeda RS, McDougle SD. Highlights from the 29th Annual Meeting of the Society for the Neural Control of Movement. J Neurophysiol 2019; 122:1777-1783. [PMID: 31461364 PMCID: PMC6843106 DOI: 10.1152/jn.00484.2019] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2019] [Revised: 08/21/2019] [Accepted: 08/21/2019] [Indexed: 11/22/2022] Open
Affiliation(s)
- Alexander Mathis
- Department of Molecular & Cellular Biology, Harvard University, Cambridge, Massachusetts
| | - Andrea R Pack
- Department of Biology, Emory University, Atlanta, Georgia
| | - Rodrigo S Maeda
- Brain and Mind Institute, Western University, London, Ontario, Canada
- Robarts Research Institute, Western University, London, Ontario, Canada
- Department of Psychology, Western University, London, Ontario, Canada
| | - Samuel D McDougle
- Department of Psychology, University of California, Berkeley, California
| |
Collapse
|