Deep learning-based quantitative analyses of spontaneous movements and their association with early neurological development in preterm infants

Artificial intelligence detects awareness of functional relation with the environment in 3 month old babies

A recent experiment probed how purposeful action emerges in early life by manipulating infants' functional connection to an object in the environment (i.e., tethering an infant's foot to a colorful mobile). Vicon motion capture data from multiple infant joints were used here to create Histograms of Joint Displacements (HJDs) to generate pose-based descriptors for 3D infant spatial trajectories. Using HJDs as inputs, machine and deep learning systems were tasked with classifying the experimental state from which snippets of movement data were sampled. The architectures tested included k-Nearest Neighbour (kNN), Linear Discriminant Analysis (LDA), Fully connected network (FCNet), 1D-Convolutional Neural Network (1D-Conv), 1D-Capsule Network (1D-CapsNet), 2D-Conv and 2D-CapsNet. Sliding window scenarios were used for temporal analysis to search for topological changes in infant movement related to functional context. kNN and LDA achieved higher classification accuracy with single joint features, while deep learning approaches, particularly 2D-CapsNet, achieved higher accuracy on full-body features. For each AI architecture tested, measures of foot activity displayed the most distinct and coherent pattern alterations across different experimental stages (reflected in the highest classification accuracy rate), indicating that interaction with the world impacts the infant behaviour most at the site of organism~world connection.

Reduction in limb-movement complexity at term-equivalent age is associated with motor developmental delay in very-preterm or very-low-birth-weight infants

Reduced complexity during the writhing period can be crucial in the spontaneous movements of high-risk infants for neurologic impairment. This study aimed to verify the association between quantified complexity of upper and lower-limb movements at term-equivalent age and motor development in very-preterm or very-low-birth-weight infants. Video images of spontaneous movements at term-equivalent age were collected from very-preterm or very-low-birth-weight infants. A pretrained pose-estimation model and sample entropy (SE) quantified the complexity of the upper- and lower-limb movements. Motor development was evaluated at 9 months of corrected age using Bayley Scales of Infant and Toddler Development, Third Edition. The SE measures were compared between infants with and without motor developmental delay (MDD). Among 90 infants, 11 exhibited MDD. SE measures at most of the upper and lower limbs were significantly reduced in infants with MDD compared to those without MDD (p < 0.05). Composite scores in the motor domain showed significant positive correlations with SE measures at most upper and lower limbs (p < 0.05). The results show that limb-movement complexity at term-equivalent age is reduced in infants with MDD at 9 months of corrected age. SE of limb movements can be a potentially useful kinematic parameter to detect high-risk infants for MDD.

Spontaneous movements as prognostic tool of neurodevelopmental outcomes in preterm infants: a narrative review

An estimated 15 million infants are born prematurely each year. Although the survival rate of preterm infants has increased with advances in perinatal and neonatal care, many still experience various complications. Since improving the neurodevelopmental outcomes of preterm births is a crucial topic, accurate evaluations should be performed to detect infants at high risk of cerebral palsy. General movements are spontaneous movements involving the whole body as the expression of neural activity and can be an excellent biomarker of neural dysfunction caused by brain impairment in preterm infants. The predictive value of general movements with respect to cerebral palsy increases with continuous observation. Automated approaches to examining general movements based on machine learning can help overcome the limited utilization of assessment tools owing to their qualitative or semiquantitative nature and high dependence on assessor skills and experience. This review covers each of these topics by summarizing normal and abnormal general movements as well as recent advances in automatic approaches based on infantile spontaneous movements.

Early prediction of need for invasive mechanical ventilation in the neonatal intensive care unit using artificial intelligence and electronic health records: a clinical study

BACKGROUND

Respiratory support is crucial for newborns with underdeveloped lung. The clinical outcomes of patients depend on the clinician's ability to recognize the status underlying the presented symptoms and signs. With the increasing number of high-risk infants, artificial intelligence (AI) should be considered as a tool for personalized neonatal care. Continuous monitoring of vital signs is essential in cardiorespiratory care. In this study, we developed deep learning (DL) prediction models for rapid and accurate detection of mechanical ventilation requirements in neonates using electronic health records (EHR).

METHODS

We utilized data from the neonatal intensive care unit in a single center, collected between March 3, 2012, and March 4, 2022, including 1,394 patient records used for model development, consisting of 505 and 889 patients with and without invasive mechanical ventilation (IMV) support, respectively. The proposed model architecture includes feature embedding using feature-wise fully connected (FC) layers, followed by three bidirectional long short-term memory (LSTM) layers.

RESULTS

A mean gestational age (GA) was 36.61 ± 3.25 weeks, and the mean birth weight was 2,734.01 ± 784.98 g. The IMV group had lower GA, birth weight, and longer hospitalization duration than the non-IMV group (P < 0.05). Our proposed model, tested on a dataset from March 4, 2019, to March 4, 2022. The mean AUROC of our proposed model for IMV support prediction performance demonstrated 0.861 (95%CI, 0.853-0.869). It is superior to conventional approaches, such as newborn early warning score systems (NEWS), Random Forest, and eXtreme gradient boosting (XGBoost) with 0.611 (95%CI, 0.600-0.622), 0.837 (95%CI, 0.828-0.845), and 0.0.831 (95%CI, 0.821-0.845), respectively. The highest AUPRC value is shown in the proposed model at 0.327 (95%CI, 0.308-0.347). The proposed model performed more accurate predictions as gestational age decreased. Additionally, the model exhibited the lowest alarm rate while maintaining the same sensitivity level.

CONCLUSION

Deep learning approaches can help accurately standardize the prediction of invasive mechanical ventilation for neonatal patients and facilitate advanced neonatal care. The results of predictive, recall, and alarm performances of the proposed model outperformed the other models.

2D Capsule Networks Detect Perceived Changes in Infant∼Environment Relationship Reflected in 3D Movement Dynamics

Can infant exploration and causal discovery be detected using Artificial Intelligence (AI)? A recent experiment probed how purposeful action emerges in early life by manipulating infants' functional connection to an object in the environment (i.e., tethering one foot to a colorful mobile). Vicon motion capture data from multiple infant joints were used here to create Histograms of Joint Displacements (HJDs) to generate pose-based descriptors for 3D infant spatial trajectories. Using HJDs as inputs, machine and deep learning systems were tasked with classifying the experimental state from which snippets of movement data were sampled. The architectures tested included k-Nearest Neighbour (kNN), Linear Discriminant Analysis (LDA), Fully connected network (FCNet), 1D-Convolutional Neural Network (1D-Conv), 1D-Capsule Network (1D-CapsNet), 2D-Conv and 2D-CapsNet. Sliding window scenarios were used for temporal analysis to search for topological changes in infant movement related to functional context. kNN and LDA achieved higher classification accuracy with single joint features, while deep learning approaches, particularly 2D-CapsNet, achieved higher accuracy on full-body features. For each AI architecture tested, measures of foot activity displayed the most distinct and coherent pattern alterations across different experimental stages (reflected in the highest classification accuracy rate), indicating that interaction with the world impacts the infant behaviour most at the site of organism∼world connection. Pairing theory-driven experimentation with AI tools thus opens a path to developing functionally-relevant assessments of infant behaviour that are likely to be useful in clinical settings.

Mean 3D Dispersion for Automatic General Movement Assessment of Preterm Infants

The General Movement assessment (GMA) is a validated assessment of brain maturation primarily based on the qualitative analysis of the complexity and the variation of spontaneous motor activity. The GMA can identify preterm infants presenting an early abnormal developmental trajectory before term-equivalent age, which permits a personalized early developmental intervention. However, GMA is time-consuming and relies on a qualitative analysis; these limitations restrict the implementation of GMA in clinical practice. In this study based on a validated dataset of 183 videos from 92 premature infants (54 males, 38 females) born <33 weeks of gestational age (GA) and acquired between 32 and 40 weeks of GA, we introduce the mean 3D dispersion (M3D) for objective quantification and classification of normal and abnormal GMA. Moreover, we have created a new 3D representation of skeleton joints which allows an objective comparison of spontaneous movements of infants of different ages and sizes. Preterm infants with normal versus abnormal GMA had a distinct M3D distribution (p <0.001). The M3D has shown a good classification performance for GMA (AUC=0.7723) and presented an accuracy of 74.1%, a sensitivity of 75.8%, and a specificity of 70.1% when using an M3D of 0.29 as a classification threshold.Clinical relevance- Our study paves the way for the development of quantitative analysis of GMA within the Neonatal Unit.

NeoCam: An Edge-Cloud Platform for Non-Invasive Real-Time Monitoring in Neonatal Intensive Care Units

In this work we introduce NeoCam, an open source hardware-software platform for video-based monitoring of preterms infants in Neonatal Intensive Care Units (NICUs). NeoCam includes an edge computing device that performs video acquisition and processing in real-time. Compared to other proposed solutions, it has the advantage of handling data more efficiently by performing most of the processing on the device, including proper anonymisation for better compliance with privacy regulations. In addition, it allows to perform various video analysis tasks of clinical interest in parallel at speeds of between 20 and 30 frames-per-second. We introduce algorithms to measure without contact the breathing rate, motor activity, body pose and emotional status of the infants. For breathing rate, our system shows good agreement with existing methods provided there is sufficient light and proper imaging conditions. Models for motor activity and stress detection are new to the best of our knowledge. NeoCam has been tested on preterms in the NICU of the University Hospital Puerta del Mar (Cádiz, Spain), and we report the lessons learned from this trial.

Development and Validation of a Deep Learning Method to Predict Cerebral Palsy From Spontaneous Movements in Infants at High Risk

IMPORTANCE

Early identification of cerebral palsy (CP) is important for early intervention, yet expert-based assessments do not permit widespread use, and conventional machine learning alternatives lack validity.

OBJECTIVE

To develop and assess the external validity of a novel deep learning-based method to predict CP based on videos of infants' spontaneous movements at 9 to 18 weeks' corrected age.

DESIGN, SETTING, AND PARTICIPANTS

This prognostic study of a deep learning-based method to predict CP at a corrected age of 12 to 89 months involved 557 infants with a high risk of perinatal brain injury who were enrolled in previous studies conducted at 13 hospitals in Belgium, India, Norway, and the US between September 10, 2001, and October 25, 2018. Analysis was performed between February 11, 2020, and September 23, 2021. Included infants had available video recorded during the fidgety movement period from 9 to 18 weeks' corrected age, available classifications of fidgety movements ascertained by the general movement assessment (GMA) tool, and available data on CP status at 12 months' corrected age or older. A total of 418 infants (75.0%) were randomly assigned to the model development (training and internal validation) sample, and 139 (25.0%) were randomly assigned to the external validation sample (1 test set).

EXPOSURE

Video recording of spontaneous movements.

MAIN OUTCOMES AND MEASURES

The primary outcome was prediction of CP. Deep learning-based prediction of CP was performed automatically from a single video. Secondary outcomes included prediction of associated functional level and CP subtype. Sensitivity, specificity, positive and negative predictive values, and accuracy were assessed.

RESULTS

Among 557 infants (310 [55.7%] male), the median (IQR) corrected age was 12 (11-13) weeks at assessment, and 84 infants (15.1%) were diagnosed with CP at a mean (SD) age of 3.4 (1.7) years. Data on race and ethnicity were not reported because previous studies (from which the infant samples were derived) used different study protocols with inconsistent collection of these data. On external validation, the deep learning-based CP prediction method had sensitivity of 71.4% (95% CI, 47.8%-88.7%), specificity of 94.1% (95% CI, 88.2%-97.6%), positive predictive value of 68.2% (95% CI, 45.1%-86.1%), and negative predictive value of 94.9% (95% CI, 89.2%-98.1%). In comparison, the GMA tool had sensitivity of 70.0% (95% CI, 45.7%-88.1%), specificity of 88.7% (95% CI, 81.5%-93.8%), positive predictive value of 51.9% (95% CI, 32.0%-71.3%), and negative predictive value of 94.4% (95% CI, 88.3%-97.9%). The deep learning method achieved higher accuracy than the conventional machine learning method (90.6% [95% CI, 84.5%-94.9%] vs 72.7% [95% CI, 64.5%-79.9%]; P < .001), but no significant improvement in accuracy was observed compared with the GMA tool (85.9%; 95% CI, 78.9%-91.3%; P = .11). The deep learning prediction model had higher sensitivity among infants with nonambulatory CP (100%; 95% CI, 63.1%-100%) vs ambulatory CP (58.3%; 95% CI, 27.7%-84.8%; P = .02) and spastic bilateral CP (92.3%; 95% CI, 64.0%-99.8%) vs spastic unilateral CP (42.9%; 95% CI, 9.9%-81.6%; P < .001).

CONCLUSIONS AND RELEVANCE

In this prognostic study, a deep learning-based method for predicting CP at 9 to 18 weeks' corrected age had predictive accuracy on external validation, which suggests possible avenues for using deep learning-based software to provide objective early detection of CP in clinical settings.

Automated Movement Analysis to Predict Cerebral Palsy in Very Preterm Infants: An Ambispective Cohort Study

The General Movements Assessment requires extensive training. As an alternative, a novel automated movement analysis was developed and validated in preterm infants. Infants < 31 weeks’ gestational age or birthweight ≤ 1500 g evaluated at 3−5 months using the general movements assessment were included in this ambispective cohort study. The C-statistic, sensitivity, specificity, positive predictive value, and negative predictive value were calculated for a predictive model. A total of 252 participants were included. The median gestational age and birthweight were 274/7 weeks (range 256/7−292/7 weeks) and 960 g (range 769−1215 g), respectively. There were 29 cases of cerebral palsy (11.5%) at 18−24 months, the majority of which (n = 22) were from the retrospective cohort. Mean velocity in the vertical direction, median, standard deviation, and minimum quantity of motion constituted the multivariable model used to predict cerebral palsy. Sensitivity, specificity, positive, and negative predictive values were 55%, 80%, 26%, and 93%, respectively. C-statistic indicated good fit (C = 0.74). A cluster of four variables describing quantity of motion and variability of motion was able to predict cerebral palsy with high specificity and negative predictive value. This technology may be useful for screening purposes in very preterm infants; although, the technology likely requires further validation in preterm and high-risk term populations.