Movidea: A Software Package for Automatic Video Analysis of Movements in Infants at Risk for Neurodevelopmental Disorders

Automated Prediction of Infant Cognitive Development Risk by Video: A Pilot Study

OBJECTIVE

Cognition is an essential human function, and its development in infancy is crucial. Traditionally, pediatricians used clinical observation or medical imaging to assess infants' current cognitive development (CD) status. The object of pediatricians' greater concern is however their future outcomes, because high-risk infants can be identified early in life for intervention. However, this opportunity has not yet been realized. Fortunately, some recent studies have shown that the general movement (GM) performance of infants around 3-4 months after birth might reflect their future CD status, which gives us an opportunity to achieve this goal by cameras and artificial intelligence.

METHODS

First, infants' GM videos were recorded by cameras, from which a series of features reflecting their bilateral movement symmetry (BMS) were extracted. Then, after at least eight months of natural growth, the infants' CD status was evaluated by the Bayley Infant Development Scale, and they were divided into high-risk and low-risk groups. Finally, the BMS features extracted from the early recorded GM videos were fed into the classifiers, using late infant CD risk assessment as the prediction target.

RESULTS

The area under the curve, recall and precision values reached 0.830, 0.832, and 0.823 for two-group classification, respectively.

CONCLUSION

This pilot study demonstrates that it is possible to automatically predict the CD of infants around the age of one year based on their GMs recorded early in life.

SIGNIFICANCE

This study not only helps clinicians better understand infant CD mechanisms, but also provides an economical, portable and non-invasive way to screen infants at high-risk early to facilitate their recovery.

Mean 3D Dispersion for Automatic General Movement Assessment of Preterm Infants

The General Movement assessment (GMA) is a validated assessment of brain maturation primarily based on the qualitative analysis of the complexity and the variation of spontaneous motor activity. The GMA can identify preterm infants presenting an early abnormal developmental trajectory before term-equivalent age, which permits a personalized early developmental intervention. However, GMA is time-consuming and relies on a qualitative analysis; these limitations restrict the implementation of GMA in clinical practice. In this study based on a validated dataset of 183 videos from 92 premature infants (54 males, 38 females) born <33 weeks of gestational age (GA) and acquired between 32 and 40 weeks of GA, we introduce the mean 3D dispersion (M3D) for objective quantification and classification of normal and abnormal GMA. Moreover, we have created a new 3D representation of skeleton joints which allows an objective comparison of spontaneous movements of infants of different ages and sizes. Preterm infants with normal versus abnormal GMA had a distinct M3D distribution (p <0.001). The M3D has shown a good classification performance for GMA (AUC=0.7723) and presented an accuracy of 74.1%, a sensitivity of 75.8%, and a specificity of 70.1% when using an M3D of 0.29 as a classification threshold.Clinical relevance- Our study paves the way for the development of quantitative analysis of GMA within the Neonatal Unit.

NeoCam: An Edge-Cloud Platform for Non-Invasive Real-Time Monitoring in Neonatal Intensive Care Units

In this work we introduce NeoCam, an open source hardware-software platform for video-based monitoring of preterms infants in Neonatal Intensive Care Units (NICUs). NeoCam includes an edge computing device that performs video acquisition and processing in real-time. Compared to other proposed solutions, it has the advantage of handling data more efficiently by performing most of the processing on the device, including proper anonymisation for better compliance with privacy regulations. In addition, it allows to perform various video analysis tasks of clinical interest in parallel at speeds of between 20 and 30 frames-per-second. We introduce algorithms to measure without contact the breathing rate, motor activity, body pose and emotional status of the infants. For breathing rate, our system shows good agreement with existing methods provided there is sufficient light and proper imaging conditions. Models for motor activity and stress detection are new to the best of our knowledge. NeoCam has been tested on preterms in the NICU of the University Hospital Puerta del Mar (Cádiz, Spain), and we report the lessons learned from this trial.

Identifying activity level related movement features of children with ASD based on ADOS videos

Autism spectrum disorder (ASD) is a neurodevelopmental disorder that affects about 2% of children. Due to the shortage of clinicians, there is an urgent demand for a convenient and effective tool based on regular videos to assess the symptom. Computer-aided technologies have become widely used in clinical diagnosis, simplifying the diagnosis process while saving time and standardizing the procedure. In this study, we proposed a computer vision-based motion trajectory detection approach assisted with machine learning techniques, facilitating an objective and effective way to extract participants' movement features (MFs) to identify and evaluate children's activity levels that correspond to clinicians' professional ratings. The designed technique includes two key parts: (1) Extracting MFs of participants' different body key points in various activities segmented from autism diagnostic observation schedule (ADOS) videos, and (2) Identifying the most relevant MFs through established correlations with existing data sets of participants' activity level scores evaluated by clinicians. The research investigated two types of MFs, i.e., pixel distance (PD) and instantaneous pixel velocity (IPV), three participants' body key points, i.e., neck, right wrist, and middle hip, and five activities, including Table-play, Birthday-party, Joint-attention, Balloon-play, and Bubble-play segmented from ADOS videos. Among different combinations, the high correlations with the activity level scores evaluated by the clinicians (greater than 0.6 with p < 0.001) were found in Table-play activity for both the PD-based MFs of all three studied key points and the IPV-based MFs of the right wrist key point. These MFs were identified as the most relevant ones that could be utilized as an auxiliary means for automating the evaluation of activity levels in the ASD assessment.

A markerless pipeline to analyze spontaneous movements of preterm infants

BACKGROUND AND OBJECTIVE

The analysis of spontaneous movements of preterm infants is important because anomalous motion patterns can be a sign of neurological disorders caused by lesions in the developing brain. A diagnosis in the first weeks of child's life is crucial to plan timely and appropriate rehabilitative interventions. An accurate visual assessment of infants' spontaneous movements requires highly specialized personnel, not always available, and it is operator dependent. Motion capture systems, markers and wearable sensors are commonly used for human motion analysis, but they can be cumbersome, limiting their use in the study of infants' movements.

METHODS

In this paper we propose a computer-aided pipeline to characterize and classify infants' motion from 2D video recordings. The final goal is detecting anomalous motion patterns. The implemented pipeline is based on computer vision and machine learning algorithms and includes a specific step to increase the interpretability of the results. Specifically, it can be summarized by the following steps: (i) body keypoints detection: we rely on a deep learning-based semantic features detector to localize the positions of meaningful landmark points on infants' bodies; (ii) parameters extraction: starting from the trajectories of the detected landmark points, we extract quantitative parameters describing infants motion patterns; (iii) classification: we implement different classifiers (Support Vector Machines, Random Forest, fully connected Neural Network, Long Short Term Memory) that, starting from the motion parameters, classify between normal or abnormal motion patterns.

RESULTS

We tested the proposed pipeline on a dataset, recorded at the 40th gestational week, of 142 infants, 59 with evidence of neuromotor disorders according to a medical assessment carried out a posteriori. Our procedure successfully discriminates normal and anomalous motion patterns with a maximum accuracy of 85.7%.

CONCLUSIONS

In conclusion, our pipeline has the potential to be adopted as a tool to support the early detection of abnormal motion patterns in preterm infants.

Prediction of autistic tendencies at 18 months of age via markerless video analysis of spontaneous body movements in 4-month-old infants

Early intervention is now considered the core treatment strategy for autism spectrum disorders (ASD). Thus, it is of significant clinical importance to establish a screening tool for the early detection of ASD in infants. To achieve this goal, in a longitudinal design, we analyzed spontaneous bodily movements of 4-month-old infants from general population and assessed their ASD-like behaviors at 18 months of age. A total of 26 movement features were calculated from video-recorded bodily movements of infants at 4 months of age. Their risk of ASD was assessed at 18 months of age with the Modified Checklist for Autism in Toddlerhood, a widely used screening questionnaire. Infants at high risk for ASD at 18 months of age exhibited less rhythmic and weaker bodily movement patterns at 4 months of age than low-risk infants. When the observed bodily movement patterns were submitted to a machine learning-based analysis, linear and non-linear classifiers successfully predicted ASD-like behavior at 18 months of age based on the bodily movement patterns at 4 months of age, at the level acceptable for practical use. This study analyzed the relationship between spontaneous bodily movements at 4 months of age and the ASD risk at 18 months of age. Experimental results suggested the utility of the proposed method for the early screening of infants at risk for ASD. We revealed that the signs of ASD risk could be detected as early as 4 months after birth, by focusing on the infant's spontaneous bodily movements.

Changes in the Complexity of Limb Movements during the First Year of Life across Different Tasks

Infants’ limb movements evolve from disorganized to more selectively coordinated during the first year of life as they learn to navigate and interact with an ever-changing environment more efficiently. However, how these coordination patterns change during the first year of life and across different contexts is unknown. Here, we used wearable motion trackers to study the developmental changes in the complexity of limb movements (arms and legs) at 4, 6, 9 and 12 months of age in two different tasks: rhythmic rattle-shaking and free play. We applied Multidimensional Recurrence Quantification Analysis (MdRQA) to capture the nonlinear changes in infants’ limb complexity. We show that the MdRQA parameters (entropy, recurrence rate and mean line) are task-dependent only at 9 and 12 months of age, with higher values in rattle-shaking than free play. Since rattle-shaking elicits more stable and repetitive limb movements than the free exploration of multiple objects, we interpret our data as reflecting an increase in infants’ motor control that allows for stable body positioning and easier execution of limb movements. Infants’ motor system becomes more stable and flexible with age, allowing for flexible adaptation of behaviors to task demands.

Deep learning-based quantitative analyses of spontaneous movements and their association with early neurological development in preterm infants

This study aimed to develop quantitative assessments of spontaneous movements in high-risk preterm infants based on a deep learning algorithm. Video images of spontaneous movements were recorded in very preterm infants at the term-equivalent age. The Hammersmith Infant Neurological Examination (HINE) was performed in infants at 4 months of corrected age. Joint positional data were extracted using a pretrained pose-estimation model. Complexity and similarity indices of joint angle and angular velocity in terms of sample entropy and Pearson correlation coefficient were compared between the infants with HINE < 60 and ≥ 60. Video images of spontaneous movements were recorded in 65 preterm infants at term-equivalent age. Complexity indices of joint angles and angular velocities differed between the infants with HINE < 60 and ≥ 60 and correlated positively with HINE scores in most of the joints at the upper and lower extremities (p < 0.05). Similarity indices between each joint angle or joint angular velocity did not differ between the two groups in most of the joints at the upper and lower extremities. Quantitative assessments of spontaneous movements in preterm infants are feasible using a deep learning algorithm and sample entropy. The results indicated that complexity indices of joint movements at both the upper and lower extremities can be potential candidates for detecting developmental outcomes in preterm infants.

Video-Based Automatic Baby Motion Analysis for Early Neurological Disorder Diagnosis: State of the Art and Future Directions

Neurodevelopmental disorders (NDD) are impairments of the growth and development of the brain and/or central nervous system. In the light of clinical findings on early diagnosis of NDD and prompted by recent advances in hardware and software technologies, several researchers tried to introduce automatic systems to analyse the baby's movement, even in cribs. Traditional technologies for automatic baby motion analysis leverage contact sensors. Alternatively, remotely acquired video data (e.g., RGB or depth) can be used, with or without active/passive markers positioned on the body. Markerless approaches are easier to set up and maintain (without any human intervention) and they work well on non-collaborative users, making them the most suitable technologies for clinical applications involving children. On the other hand, they require complex computational strategies for extracting knowledge from data, and then, they strongly depend on advances in computer vision and machine learning, which are among the most expanding areas of research. As a consequence, also markerless video-based analysis of movements in children for NDD has been rapidly expanding but, to the best of our knowledge, there is not yet a survey paper providing a broad overview of how recent scientific developments impacted it. This paper tries to fill this gap and it lists specifically designed data acquisition tools and publicly available datasets as well. Besides, it gives a glimpse of the most promising techniques in computer vision, machine learning and pattern recognition which could be profitably exploited for children motion analysis in videos.

An Automated Approach for General Movement Assessment: A Pilot Study

Objective: The objective of the study was to develop an automatic quantitative approach to identify infants with abnormal movements of the limbs at term equivalent age (TEA) compared with general movement assessment (GMA). Methods: GMA was performed at TEA by a trained operator in neonates with neurological risk. GMs were classified as normal (N) or abnormal (Ab), which included poor repertoire and cramped synchronized movements. The signals from four micro-accelerometers placed on all limbs were recorded for 10 min simultaneously. A global index (KC_index), quantifying the characteristics of individual limb movements and the coordination among the limbs, was obtained by adding normalized kurtosis of the distribution of the first principal component of the acceleration signals to the cross-correlation of the jerk for the upper and lower limbs. Results: Sixty-eight infants were studied. A KC_index cut-off of 201.5 (95% CI: 199.9-205.0) provided specificity = 0.86 and sensitivity = 0.88 in identifying infants with Ab movements. Conclusions: KC_index provides an automatic and quantitative measure that may allow the identification of infants who require further neurological evaluation.

Abnormal visual attention to simple social stimuli in 4-month-old infants at high risk for Autism

Despite an increasing interest in detecting early signs of Autism Spectrum Disorders (ASD), the pathogenesis of the social impairments characterizing ASD is still largely unknown. Atypical visual attention to social stimuli is a potential early marker of the social and communicative deficits of ASD. Some authors hypothesized that such impairments are present from birth, leading to a decline in the subsequent typical functioning of the learning-mechanisms. Others suggested that these early deficits emerge during the transition from subcortically to cortically mediated mechanisms, happening around 2–3 months of age. The present study aimed to provide additional evidence on the origin of the early visual attention disturbance that seems to characterize infants at high risk (HR) for ASD. Four visual preference tasks were used to investigate social attention in 4-month-old HR, compared to low-risk (LR) infants of the same age. Visual attention differences between HR and LR infants emerged only for stimuli depicting a direct eye-gaze, compared to an adverted eye-gaze. Specifically, HR infants showed a significant visual preference for the direct eye-gaze stimulus compared to LR infants, which may indicate a delayed development of the visual preferences normally observed at birth in typically developing infants. No other differences were found between groups. Results are discussed in the light of the hypotheses on the origins of early social visual attention impairments in infants at risk for ASD.

The Challenging Heterogeneity of Autism: Editorial for Brain Sciences Special Issue "Advances in Autism Research"

My personal experience as Guest Editor of the Special Issue (SI) entitled "Advances in Autism Research" began with a nice correspondence with Andrew Meltzoff, from the University of Washington, Seattle (WA, USA), which, in hindsight, I consider as a good omen for the success of this Special Issue: "Dear Antonio… [...].

Early Motor Development Predicts Clinical Outcomes of Siblings at High-Risk for Autism: Insight from an Innovative Motion-Tracking Technology

Atypical motor patterns are potential early markers and predictors of later diagnosis of Autism Spectrum Disorder (ASD). This study aimed to investigate the early motor trajectories of infants at high-risk (HR) of ASD through MOVIDEA, a semi-automatic software developed to analyze 2D and 3D videos and provide objective kinematic features of their movements. MOVIDEA was developed within the Italian Network for early detection of Autism Spectrum Disorder (NIDA Network), which is currently coordinating the most extensive surveillance program for infants at risk for neurodevelopmental disorders (NDDs). MOVIDEA was applied to video recordings of 53 low-risk (LR; siblings of typically developing children) and 50 HR infants’ spontaneous movements collected at 10 days and 6, 12, 18, and 24 weeks. Participants were grouped based on their clinical outcome (18 HR received an NDD diagnosis, 32 HR and 53 LR were typically developing). Results revealed that early developmental trajectories of specific motor parameters were different in HR infants later diagnosed with NDDs from those of infants developing typically. Since MOVIDEA was useful in the association of quantitative measures with specific early motor patterns, it should be applied to the early detection of ASD/NDD markers.