1
|
Improving Performance and Quantifying Uncertainty of Body-Rocking Detection Using Bayesian Neural Networks. INFORMATION 2022. [DOI: 10.3390/info13070338] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/04/2022] Open
Abstract
Body-rocking is an undesired stereotypical motor movement performed by some individuals, and its detection is essential for self-awareness and habit change. We envision a pipeline that includes inertial wearable sensors and a real-time detection system for notifying the user so that they are aware of their body-rocking behavior. For this task, similarities of body rocking to other non-related repetitive activities may cause false detections which prevent continuous engagement, leading to alarm fatigue. We present a pipeline using Bayesian Neural Networks with uncertainty quantification for jointly reducing false positives and providing accurate detection. We show that increasing model capacity does not consistently yield higher performance by itself, while pairing it with the Bayesian approach does yield significant improvements. Disparities in uncertainty quantification are better quantified by calibrating them using deep neural networks. We show that the calibrated probabilities are effective quality indicators of reliable predictions. Altogether, we show that our approach provides additional insights on the role of Bayesian techniques in deep learning as well as aids in accurate body-rocking detection, improving our prior work on this subject.
Collapse
|
2
|
Automatic Assessment of Motor Impairments in Autism Spectrum Disorders: A Systematic Review. Cognit Comput 2022. [DOI: 10.1007/s12559-021-09940-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/06/2023]
|
3
|
Rahman S, Ahmed SF, Shahid O, Arrafi MA, Ahad MAR. Automated Detection Approaches to Autism Spectrum Disorder Based on Human Activity Analysis: A Review. Cognit Comput 2021. [DOI: 10.1007/s12559-021-09895-w] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|
4
|
Kumm AJ, Viljoen M, de Vries PJ. The Digital Divide in Technologies for Autism: Feasibility Considerations for Low- and Middle-Income Countries. J Autism Dev Disord 2021; 52:2300-2313. [PMID: 34121159 PMCID: PMC8200284 DOI: 10.1007/s10803-021-05084-8] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/11/2021] [Indexed: 12/27/2022]
Abstract
Digital technologies have the potential to empower individuals with autism and their families. The COVID-19 pandemic emphasized and accelerated the drive towards technology for information, communication, training, clinical care and research, also in the autism community. However, 95% of individuals with autism live in low- and middle-income countries (LMIC) where access to electricity, internet and the ever-increasing range of digital devices may be highly limited. The World Bank coined the term ‘the digital divide’ to describe the disparities in access to digital technologies between high-income and LMIC contexts. Here we evaluated the feasibility of six emerging technologies for autism spectrum disorders, and reflected on key considerations for implementation in LMIC contexts to ensure that we do not inadvertently widen the pre-existing digital divide.
Collapse
Affiliation(s)
- Aubrey J Kumm
- Division of Child & Adolescent Psychiatry, Centre for Autism Research in Africa (CARA), University of Cape Town, 46 Sawkins Road, Rondebosch, 7700, Cape Town, South Africa
| | - Marisa Viljoen
- Division of Child & Adolescent Psychiatry, Centre for Autism Research in Africa (CARA), University of Cape Town, 46 Sawkins Road, Rondebosch, 7700, Cape Town, South Africa
| | - Petrus J de Vries
- Division of Child & Adolescent Psychiatry, Centre for Autism Research in Africa (CARA), University of Cape Town, 46 Sawkins Road, Rondebosch, 7700, Cape Town, South Africa.
| |
Collapse
|
5
|
Simeoli R, Milano N, Rega A, Marocco D. Using Technology to Identify Children With Autism Through Motor Abnormalities. Front Psychol 2021; 12:635696. [PMID: 34113283 PMCID: PMC8186533 DOI: 10.3389/fpsyg.2021.635696] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2020] [Accepted: 04/19/2021] [Indexed: 12/11/2022] Open
Abstract
Autism is a neurodevelopmental disorder typically assessed and diagnosed through observational analysis of behavior. Assessment exclusively based on behavioral observation sessions requires a lot of time for the diagnosis. In recent years, there is a growing need to make assessment processes more motivating and capable to provide objective measures of the disorder. New evidence showed that motor abnormalities may underpin the disorder and provide a computational marker to enhance assessment and diagnostic processes. Thus, a measure of motor patterns could provide a means to assess young children with autism and a new starting point for rehabilitation treatments. In this study, we propose to use a software tool that through a smart tablet device and touch screen sensor technologies could be able to capture detailed information about children's motor patterns. We compared movement trajectories of autistic children and typically developing children, with the aim to identify autism motor signatures analyzing their coordinates of movements. We used a smart tablet device to record coordinates of dragging movements carried out by 60 children (30 autistic children and 30 typically developing children) during a cognitive task. Machine learning analysis of children's motor patterns identified autism with 93% accuracy, demonstrating that autism can be computationally identified. The analysis of the features that most affect the prediction reveals and describes the differences between the groups, confirming that motor abnormalities are a core feature of autism.
Collapse
Affiliation(s)
- Roberta Simeoli
- Department of Humanistic Studies, University of Naples Federico II, Naples, Italy
| | - Nicola Milano
- Institute of Cognitive Sciences and Technologies, National Research Council (CNR), Rome, Italy
| | - Angelo Rega
- Department of Humanistic Studies, University of Naples Federico II, Naples, Italy
- Neapolisanit S.R.L. Rehabilitation Center, Ottaviano, Italy
| | - Davide Marocco
- Department of Humanistic Studies, University of Naples Federico II, Naples, Italy
| |
Collapse
|
6
|
Bondioli M, Chessa S, Narzisi A, Pelagatti S, Zoncheddu M. Towards Motor-Based Early Detection of Autism Red Flags: Enabling Technology and Exploratory Study Protocol. SENSORS 2021; 21:s21061971. [PMID: 33799643 PMCID: PMC7998381 DOI: 10.3390/s21061971] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/23/2021] [Revised: 03/05/2021] [Accepted: 03/08/2021] [Indexed: 11/27/2022]
Abstract
Observing how children manipulate objects while they are playing can help detect possible autism spectrum disorders (ASD) at an early stage. For this purpose, specialists seek the so-called “red-flags” of motor signature of ASD for more precise diagnostic tests. However, a significant drawback to achieve this is that the observation of object manipulation by the child very often is not naturalistic, as it involves the physical presence of the specialist and is typically performed in hospitals. In this framework, we present a novel Internet of Things support in the form factory of a smart toy that can be used by specialists to perform indirect and non-invasive observations of the children in naturalistic conditions. While they play with the toy, children can be observed in their own environment and without the physical presence of the specialist. We also present the technical validation of the technology and the study protocol for the refinement of the diagnostic practice based on this technology.
Collapse
Affiliation(s)
- Mariasole Bondioli
- Department of Computer Science, University of Pisa, Largo Pontecorvo 3, 56127 Pisa, Italy; (M.B.); (S.C.); (S.P.); (M.Z.)
| | - Stefano Chessa
- Department of Computer Science, University of Pisa, Largo Pontecorvo 3, 56127 Pisa, Italy; (M.B.); (S.C.); (S.P.); (M.Z.)
| | - Antonio Narzisi
- Department of Child Psychiatry and Psychopharmacology, IRCCS Stella Maris Foundation, 56018 Pisa, Italy
- Correspondence:
| | - Susanna Pelagatti
- Department of Computer Science, University of Pisa, Largo Pontecorvo 3, 56127 Pisa, Italy; (M.B.); (S.C.); (S.P.); (M.Z.)
| | - Michele Zoncheddu
- Department of Computer Science, University of Pisa, Largo Pontecorvo 3, 56127 Pisa, Italy; (M.B.); (S.C.); (S.P.); (M.Z.)
| |
Collapse
|
7
|
Camada MY, Cerqueira JJ, Lima AM. Computational model for identifying stereotyped behaviors and determining the activation level of pseudo-autistic. Appl Soft Comput 2021. [DOI: 10.1016/j.asoc.2020.106877] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
8
|
Zhang Y, Tian Y, Wu P, Chen D. Application of Skeleton Data and Long Short-Term Memory in Action Recognition of Children with Autism Spectrum Disorder. SENSORS (BASEL, SWITZERLAND) 2021; 21:E411. [PMID: 33430118 PMCID: PMC7827022 DOI: 10.3390/s21020411] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/07/2020] [Revised: 12/25/2020] [Accepted: 01/06/2021] [Indexed: 11/16/2022]
Abstract
The recognition of stereotyped action is one of the core diagnostic criteria of Autism Spectrum Disorder (ASD). However, it mainly relies on parent interviews and clinical observations, which lead to a long diagnosis cycle and prevents the ASD children from timely treatment. To speed up the recognition process of stereotyped actions, a method based on skeleton data and Long Short-Term Memory (LSTM) is proposed in this paper. In the first stage of our method, the OpenPose algorithm is used to obtain the initial skeleton data from the video of ASD children. Furthermore, four denoising methods are proposed to eliminate the noise of the initial skeleton data. In the second stage, we track multiple ASD children in the same scene by matching distance between current skeletons and previous skeletons. In the last stage, the neural network based on LSTM is proposed to classify the ASD children's actions. The performed experiments show that our proposed method is effective for ASD children's action recognition. Compared to the previous traditional schemes, our scheme has higher accuracy and is almost non-invasive for ASD children.
Collapse
Affiliation(s)
- Yunkai Zhang
- School of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, China; (Y.Z.); (Y.T.)
| | - Yinghong Tian
- School of Communication and Electronic Engineering, East China Normal University, Shanghai 200241, China; (Y.Z.); (Y.T.)
| | - Pingyi Wu
- Experimental Teaching Center for Teacher Education, East China Normal University, Shanghai 200241, China
| | - Dongfan Chen
- Department of Rehabilitation Sciences, East China Normal University, Shanghai 200062, China;
| |
Collapse
|
9
|
Dufour MM, Lanovaz MJ, Cardinal P. Artificial intelligence for the measurement of vocal stereotypy. J Exp Anal Behav 2020; 114:368-380. [PMID: 33145781 PMCID: PMC7756764 DOI: 10.1002/jeab.636] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2020] [Revised: 09/26/2020] [Accepted: 10/12/2020] [Indexed: 11/11/2022]
Abstract
Both researchers and practitioners often rely on direct observation to measure and monitor behavior. When these behaviors are too complex or numerous to be measured in vivo, relying on direct observation using human observers increases the amount of resources required to conduct research and to monitor the effects of interventions in practice. To address this issue, we conducted a proof of concept examining whether artificial intelligence could measure vocal stereotypy in individuals with autism. More specifically, we used an artificial neural network with over 1,500 minutes of audio data from 8 different individuals to train and test models to measure vocal stereotypy. Our results showed that the artificial neural network performed adequately (i.e., session-by-session correlation near or above .80 with a human observer) in measuring engagement in vocal stereotypy for 6 of 8 participants. Additional research is needed to further improve the generalizability of the approach.
Collapse
Affiliation(s)
- Marie-Michèle Dufour
- École de psychoéducation, Université de Montréal.,Centre de recherche de l'Institut universitaire en santé mentale de Montréal
| | - Marc J Lanovaz
- École de psychoéducation, Université de Montréal.,Centre de recherche de l'Institut universitaire en santé mentale de Montréal
| | - Patrick Cardinal
- Département de génie logiciel et des TI, École de technologie supérieure
| |
Collapse
|
10
|
Maharaj AV, Gutierrez A, Cueto C, Cadavid S. Automated measurement of repetitive behavior using the Microsoft Kinect: a proof of concept. BEHAVIORAL INTERVENTIONS 2020. [DOI: 10.1002/bin.1746] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Affiliation(s)
- André V. Maharaj
- Department of Counseling and School Psychology University of Massachusetts Boston Boston Massachusetts USA
| | - Anibal Gutierrez
- Department of Psychology University of Miami Coral Gables Florida USA
| | - Carlos Cueto
- Department of Psychology Florida International University Miami Florida USA
| | | |
Collapse
|
11
|
Cantin-Garside KD, Srinivasan D, Ranganathan S, White SW, Nussbaum MA. Multi-level modeling with nonlinear movement metrics to classify self-injurious behaviors in autism spectrum disorder. Sci Rep 2020; 10:16699. [PMID: 33028829 PMCID: PMC7542156 DOI: 10.1038/s41598-020-73155-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2020] [Accepted: 09/14/2020] [Indexed: 11/09/2022] Open
Abstract
Self-injurious behavior (SIB) is among the most dangerous concerns in autism spectrum disorder (ASD), often requiring detailed and tedious management methods. Sensor-based behavioral monitoring could address the limitations of these methods, though the complex problem of classifying variable behavior should be addressed first. We aimed to address this need by developing a group-level model accounting for individual variability and potential nonlinear trends in SIB, as a secondary analysis of existing data. Ten participants with ASD and SIB engaged in free play while wearing accelerometers. Movement data were collected from > 200 episodes and 18 different types of SIB. Frequency domain and linear movement variability measures of acceleration signals were extracted to capture differences in behaviors, and metrics of nonlinear movement variability were used to quantify the complexity of SIB. The multi-level logistic regression model, comprising of 12 principal components, explained > 65% of the variance, and classified SIB with > 75% accuracy. Our findings imply that frequency-domain and movement variability metrics can effectively predict SIB. Our modeling approach yielded superior accuracy than commonly used classifiers (~ 75 vs. ~ 64% accuracy) and had superior performance compared to prior reports (~ 75 vs. ~ 69% accuracy) This work provides an approach to generating an accurate and interpretable group-level model for SIB identification, and further supports the feasibility of developing a real-time SIB monitoring system.
Collapse
Affiliation(s)
| | - Divya Srinivasan
- Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA, USA
| | | | - Susan W White
- Center for Youth Development and Intervention, Department of Psychology, University of Alabama, Tuscaloosa, AL, USA
| | - Maury A Nussbaum
- Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA, USA.
| |
Collapse
|
12
|
Alcañiz Raya M, Marín-Morales J, Minissi ME, Teruel Garcia G, Abad L, Chicchi Giglioli IA. Machine Learning and Virtual Reality on Body Movements' Behaviors to Classify Children with Autism Spectrum Disorder. J Clin Med 2020; 9:jcm9051260. [PMID: 32357517 PMCID: PMC7287942 DOI: 10.3390/jcm9051260] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2020] [Revised: 04/21/2020] [Accepted: 04/23/2020] [Indexed: 12/18/2022] Open
Abstract
Autism spectrum disorder (ASD) is mostly diagnosed according to behavioral symptoms in sensory, social, and motor domains. Improper motor functioning, during diagnosis, involves the qualitative evaluation of stereotyped and repetitive behaviors, while quantitative methods that classify body movements' frequencies of children with ASD are less addressed. Recent advances in neuroscience, technology, and data analysis techniques are improving the quantitative and ecological validity methods to measure specific functioning in ASD children. On one side, cutting-edge technologies, such as cameras, sensors, and virtual reality can accurately detect and classify behavioral biomarkers, as body movements in real-life simulations. On the other, machine-learning techniques are showing the potential for identifying and classifying patients' subgroups. Starting from these premises, three real-simulated imitation tasks have been implemented in a virtual reality system whose aim is to investigate if machine-learning methods on movement features and frequency could be useful in discriminating ASD children from children with typical neurodevelopment. In this experiment, 24 children with ASD and 25 children with typical neurodevelopment participated in a multimodal virtual reality experience, and changes in their body movements were tracked by a depth sensor camera during the presentation of visual, auditive, and olfactive stimuli. The main results showed that ASD children presented larger body movements than TD children, and that head, trunk, and feet represent the maximum classification with an accuracy of 82.98%. Regarding stimuli, visual condition showed the highest accuracy (89.36%), followed by the visual-auditive stimuli (74.47%), and visual-auditive-olfactory stimuli (70.21%). Finally, the head showed the most consistent performance along with the stimuli, from 80.85% in visual to 89.36% in visual-auditive-olfactory condition. The findings showed the feasibility of applying machine learning and virtual reality to identify body movements' biomarkers that could contribute to improving ASD diagnosis.
Collapse
Affiliation(s)
- Mariano Alcañiz Raya
- Instituto de Investigación e Innovación en Bioingeniería (i3B), Universitat Politécnica de Valencia, 46022 Valencia, Spain; (M.A.R.); (J.M.-M.); (M.E.M.); (G.T.G.)
| | - Javier Marín-Morales
- Instituto de Investigación e Innovación en Bioingeniería (i3B), Universitat Politécnica de Valencia, 46022 Valencia, Spain; (M.A.R.); (J.M.-M.); (M.E.M.); (G.T.G.)
| | - Maria Eleonora Minissi
- Instituto de Investigación e Innovación en Bioingeniería (i3B), Universitat Politécnica de Valencia, 46022 Valencia, Spain; (M.A.R.); (J.M.-M.); (M.E.M.); (G.T.G.)
| | - Gonzalo Teruel Garcia
- Instituto de Investigación e Innovación en Bioingeniería (i3B), Universitat Politécnica de Valencia, 46022 Valencia, Spain; (M.A.R.); (J.M.-M.); (M.E.M.); (G.T.G.)
| | - Luis Abad
- Red Cenit, Centros de Desarrollo Cognitivo, 46020 Valencia, Spain;
| | - Irene Alice Chicchi Giglioli
- Instituto de Investigación e Innovación en Bioingeniería (i3B), Universitat Politécnica de Valencia, 46022 Valencia, Spain; (M.A.R.); (J.M.-M.); (M.E.M.); (G.T.G.)
- Correspondence:
| |
Collapse
|
13
|
Cantin-Garside KD, Kong Z, White SW, Antezana L, Kim S, Nussbaum MA. Detecting and Classifying Self-injurious Behavior in Autism Spectrum Disorder Using Machine Learning Techniques. J Autism Dev Disord 2020; 50:4039-4052. [PMID: 32219634 DOI: 10.1007/s10803-020-04463-x] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
Traditional self-injurious behavior (SIB) management can place compliance demands on the caregiver and have low ecological validity and accuracy. To support an SIB monitoring system for autism spectrum disorder (ASD), we evaluated machine learning methods for detecting and distinguishing diverse SIB types. SIB episodes were captured with body-worn accelerometers from children with ASD and SIB. The highest detection accuracy was found with k-nearest neighbors and support vector machines (up to 99.1% for individuals and 94.6% for grouped participants), and classification efficiency was quite high (offline processing at ~ 0.1 ms/observation). Our results provide an initial step toward creating a continuous and objective smart SIB monitoring system, which could in turn facilitate the future care of a pervasive concern in ASD.
Collapse
Affiliation(s)
| | - Zhenyu Kong
- Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA, 24060, USA
| | - Susan W White
- Department of Psychology, The University of Alabama, Tuscaloosa, AB, USA.,Department of Psychology, Virginia Tech, Blacksburg, VA, 24060, USA
| | - Ligia Antezana
- Department of Psychology, Virginia Tech, Blacksburg, VA, 24060, USA
| | - Sunwook Kim
- Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA, 24060, USA
| | - Maury A Nussbaum
- Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA, 24060, USA. .,Department of Industrial and System Engineering, Virginia Tech, 250 Durham Hall (0118), Blacksburg, VA, 24061, USA.
| |
Collapse
|
14
|
Abstract
Numerous technologies have been introduced for the diagnosis, treatment, and management of patients with neurologic disorders, offering the promise of early diagnosis, tailored and individualized interventions, improvement in quality of life, and restoration of neurologic function. Many of these technologies have become available commercially without having been evaluated by rigorous clinical trials and regulatory reviews, or at the least by peer review of results submitted for publication. A subset is intended to assess, assist, and monitor cognitive functions, motor skills, and autonomic functions and as such may be applicable to persons with developmental disabilities. Barriers that have previously limited the use of technologies by persons with neurodevelopmental disabilities are disappearing as new technologies that have the potential to substantially augment diagnosis and interventions to enhance the daily lives of persons with these disorders are emerging. While recent and future advances in technology have the potential to transform their lives, cautious and thoughtful evaluation is needed to ensure the technologies provide maximal value. As such, further work is needed to demonstrate feasibility, efficacy, and cost-effectiveness, and technologies should be designed to be optimized for individual use.
Collapse
Affiliation(s)
- Steven C Schachter
- Department of Neurology, Harvard Medical School, Boston, MA, United States.
| |
Collapse
|
15
|
Gilchrist KH, Hegarty-Craver M, Christian RB, Grego S, Kies AC, Wheeler AC. Automated Detection of Repetitive Motor Behaviors as an Outcome Measurement in Intellectual and Developmental Disabilities. J Autism Dev Disord 2019; 48:1458-1466. [PMID: 29164446 DOI: 10.1007/s10803-017-3408-6] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
Repetitive sensory motor behaviors are a direct target for clinical treatment and a potential treatment endpoint for individuals with intellectual or developmental disabilities. By removing the burden associated with video annotation or direct observation, automated detection of stereotypy would allow for longer term monitoring in ecologic settings. We report automated detection of common stereotypical motor movements using commercially available accelerometers affixed to the body and a generalizable detection algorithm. The method achieved a sensitivity of 80% for body rocking and 93% for hand flapping without individualized algorithm training or foreknowledge of subject's specific movements. This approach is well-suited for implementation in a continuous monitoring system outside of a clinical setting.
Collapse
Affiliation(s)
- Kristin H Gilchrist
- RTI International, 3040 E. Cornwallis Road, Research Triangle Park, NC, 27709, USA.
| | | | - Robert B Christian
- Carolina Institute for Developmental Disabilities, University of North Carolina, Chapel Hill, NC, USA
| | - Sonia Grego
- RTI International, 3040 E. Cornwallis Road, Research Triangle Park, NC, 27709, USA
| | - Ashley C Kies
- Carolina Institute for Developmental Disabilities, University of North Carolina, Chapel Hill, NC, USA.,Child and Family Development, Inc., Charlotte, NC, USA
| | - Anne C Wheeler
- RTI International, 3040 E. Cornwallis Road, Research Triangle Park, NC, 27709, USA
| |
Collapse
|
16
|
Bangerter A, Manyakov NV, Lewin D, Boice M, Skalkin A, Jagannatha S, Chatterjee M, Dawson G, Goodwin MS, Hendren R, Leventhal B, Shic F, Ness S, Pandina G. Caregiver Daily Reporting of Symptoms in Autism Spectrum Disorder: Observational Study Using Web and Mobile Apps. JMIR Ment Health 2019; 6:e11365. [PMID: 30912762 PMCID: PMC6454343 DOI: 10.2196/11365] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/28/2018] [Revised: 12/05/2018] [Accepted: 12/31/2018] [Indexed: 12/22/2022] Open
Abstract
BACKGROUND Currently, no medications are approved to treat core symptoms of autism spectrum disorder (ASD). One barrier to ASD medication development is the lack of validated outcome measures able to detect symptom change. Current ASD interventions are often evaluated using retrospective caregiver reports that describe general clinical presentation but often require recall of specific behaviors weeks after they occur, potentially reducing accuracy of the ratings. My JAKE, a mobile and Web-based mobile health (mHealth) app that is part of the Janssen Autism Knowledge Engine-a dynamically updated clinical research system-was designed to help caregivers of individuals with ASD to continuously log symptoms, record treatments, and track progress, to mitigate difficulties associated with retrospective reporting. OBJECTIVE My JAKE was deployed in an exploratory, noninterventional clinical trial to evaluate its utility and acceptability to monitor clinical outcomes in ASD. Hypotheses regarding relationships among daily tracking of symptoms, behavior, and retrospective caregiver reports were tested. METHODS Caregivers of individuals with ASD aged 6 years to adults (N=144) used the My JAKE app to make daily reports on their child's sleep quality, affect, and other self-selected specific behaviors across the 8- to 10-week observational study. The results were compared with commonly used paper-and-pencil scales acquired over a concurrent period at regular 4-week intervals. RESULTS Caregiver reporting of behaviors in real time was successfully captured by My JAKE. On average, caregivers made reports 2-3 days per week across the study period. Caregivers were positive about their use of the system, with over 50% indicating that they would like to use My JAKE to track behavior outside of a clinical trial. More positive average daily reporting of overall type of day was correlated with 4 weekly reports of lower caregiver burden made at 4-week intervals (r=-0.27, P=.006, n=88) and with ASD symptoms (r=-0.42, P<.001, n=112). CONCLUSIONS My JAKE reporting aligned with retrospective Web-based or paper-and-pencil scales. Use of mHealth apps, such as My JAKE, has the potential to increase the validity and accuracy of caregiver-reported outcomes and could be a useful way of identifying early changes in response to intervention. Such systems may also assist caregivers in tracking symptoms and behavior outside of a clinical trial, help with personalized goal setting, and monitoring of progress, which could collectively improve understanding of and quality of life for individuals with ASD and their families. TRIAL REGISTRATION ClinicalTrials.gov NCT02668991; https://clinicaltrials.gov/ct2/show/NCT02668991.
Collapse
Affiliation(s)
- Abigail Bangerter
- Neuroscience Therapeutic Area, Janssen Research & Development, LLC, Titusville, NJ, United States
| | - Nikolay V Manyakov
- Computational Biology, Discovery Sciences, Janssen Research & Development, Beerse, Belgium
| | - David Lewin
- Clinical Biostatistics, Janssen Research & Development, LLC, Titusville, NJ, United States
| | - Matthew Boice
- Neuroscience Therapeutic Area, Janssen Research & Development, LLC, Titusville, NJ, United States
| | - Andrew Skalkin
- Informatics, Janssen Research & Development, LLC, Spring House, PA, United States
| | - Shyla Jagannatha
- Statistical Decision Sciences, Janssen Research & Development, LLC, Titusville, NJ, United States
| | - Meenakshi Chatterjee
- Computational Biology, Discovery Sciences, Janssen Research & Development, LLC, Spring House, PA, United States
| | - Geraldine Dawson
- Duke Center for Autism and Brain Development, Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, NC, United States
| | - Matthew S Goodwin
- Department of Health Sciences, Northeastern University, Boston, MA, United States
| | - Robert Hendren
- Department of Psychiatry, School of Medicine, University of California, San Francisco, San Francisco, CA, United States
| | - Bennett Leventhal
- Department of Psychiatry, School of Medicine, University of California, San Francisco, San Francisco, CA, United States
| | - Frederick Shic
- Center for Child Health, Behavior and Development, Seattle Children's Research Institute, Seattle, WA, United States
| | - Seth Ness
- Neuroscience Therapeutic Area, Janssen Research & Development, LLC, Titusville, NJ, United States
| | - Gahan Pandina
- Neuroscience Therapeutic Area, Janssen Research & Development, LLC, Titusville, NJ, United States
| |
Collapse
|
17
|
Ness SL, Bangerter A, Manyakov NV, Lewin D, Boice M, Skalkin A, Jagannatha S, Chatterjee M, Dawson G, Goodwin MS, Hendren R, Leventhal B, Shic F, Frazier JA, Janvier Y, King BH, Miller JS, Smith CJ, Tobe RH, Pandina G. An Observational Study With the Janssen Autism Knowledge Engine (JAKE ®) in Individuals With Autism Spectrum Disorder. Front Neurosci 2019; 13:111. [PMID: 30872988 PMCID: PMC6402449 DOI: 10.3389/fnins.2019.00111] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2018] [Accepted: 01/30/2019] [Indexed: 11/13/2022] Open
Abstract
Objective: The Janssen Autism Knowledge Engine (JAKE®) is a clinical research outcomes assessment system developed to more sensitively measure treatment outcomes and identify subpopulations in autism spectrum disorder (ASD). Here we describe JAKE and present results from its digital phenotyping (My JAKE) and biosensor (JAKE Sense) components. Methods: An observational, non-interventional, prospective study of JAKE in children and adults with ASD was conducted at nine sites in the United States. Feedback on JAKE usability was obtained from caregivers. JAKE Sense included electroencephalography, eye tracking, electrocardiography, electrodermal activity, facial affect analysis, and actigraphy. Caregivers of individuals with ASD reported behaviors using My JAKE. Results from My JAKE and JAKE Sense were compared to traditional ASD symptom measures. Results: Individuals with ASD (N = 144) and a cohort of typically developing (TD) individuals (N = 41) participated in JAKE Sense. Most caregivers reported that overall use and utility of My JAKE was "easy" (69%, 74/108) or "very easy" (74%, 80/108). My JAKE could detect differences in ASD symptoms as measured by traditional methods. The majority of biosensors included in JAKE Sense captured sizable amounts of quality data (i.e., 93-100% of eye tracker, facial affect analysis, and electrocardiogram data was of good quality), demonstrated differences between TD and ASD individuals, and correlated with ASD symptom scales. No significant safety events were reported. Conclusions: My JAKE was viewed as easy or very easy to use by caregivers participating in research outside of a clinical study. My JAKE sensitively measured a broad range of ASD symptoms. JAKE Sense biosensors were well-tolerated. JAKE functioned well when used at clinical sites previously inexperienced with some of the technologies. Lessons from the study will optimize JAKE for use in clinical trials to assess ASD interventions. Additionally, because biosensors were able to detect features differentiating TD and ASD individuals, and also were correlated with standardized symptom scales, these measures could be explored as potential biomarkers for ASD and as endpoints in future clinical studies. Clinical Trial Registration: https://clinicaltrials.gov/ct2/show/NCT02668991 identifier: NCT02668991.
Collapse
Affiliation(s)
- Seth L. Ness
- Neuroscience Therapeutic Area, Janssen Research & Development, Titusville, FL, United States
| | - Abigail Bangerter
- Neuroscience Therapeutic Area, Janssen Research & Development, Titusville, FL, United States
| | - Nikolay V. Manyakov
- Computational Biology, Discovery Sciences, Janssen Research & Development, Beerse, Belgium
| | - David Lewin
- Statistically Speaking Consulting, LLC, Chicago, IL, United States
| | - Matthew Boice
- Neuroscience Therapeutic Area, Janssen Research & Development, Titusville, FL, United States
| | - Andrew Skalkin
- Informatics, Janssen Research & Development, Spring House, PA, United States
| | - Shyla Jagannatha
- Statistical Decision Sciences, Janssen Research & Development, Titusville, NJ, United States
| | - Meenakshi Chatterjee
- Computational Biology, Discovery Sciences, Janssen Research & Development, Spring House, PA, United States
| | - Geraldine Dawson
- Departments of Psychiatry and Behavioral Sciences, Duke Center for Autism and Brain Development, Duke University School of Medicine, Durham, NC, United States
| | - Matthew S. Goodwin
- Department of Health Sciences, Northeastern University, Boston, MA, United States
| | - Robert Hendren
- Department of Psychiatry, School of Medicine, University of California, San Francisco, San Francisco, CA, United States
| | - Bennett Leventhal
- Department of Psychiatry, School of Medicine, University of California, San Francisco, San Francisco, CA, United States
| | - Frederick Shic
- Center for Child Health, Behavior and Development, Seattle Children's Research Institute, Seattle, WA, United States
- Department of Pediatrics, University of Washington, Seattle, WA, United States
| | - Jean A. Frazier
- Eunice Kennedy Shriver Center and Department of Psychiatry, University of Massachusetts Medical School, Worcester, MA, United States
| | - Yvette Janvier
- Department of Developmental-Behavioral Pediatrics, Children's Specialized Hospital, Toms River, NJ, United States
| | - Bryan H. King
- Department of Psychiatry, School of Medicine, University of California, San Francisco, San Francisco, CA, United States
| | - Judith S. Miller
- Center for Autism Research, Perelman School of Medicine, Children's Hospital of Philadelphia, University of Pennsylvania, Philadelphia, PA, United States
| | | | - Russell H. Tobe
- Department of Outpatient Research, Nathan Kline Institute for Psychiatric Research, Orangeburg, NY, United States
| | - Gahan Pandina
- Neuroscience Therapeutic Area, Janssen Research & Development, Pennington, NJ, United States
| |
Collapse
|
18
|
A Novel Deep Learning Approach for Recognizing Stereotypical Motor Movements within and across Subjects on the Autism Spectrum Disorder. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2018; 2018:7186762. [PMID: 30111994 PMCID: PMC6077579 DOI: 10.1155/2018/7186762] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/28/2018] [Accepted: 06/10/2018] [Indexed: 11/17/2022]
Abstract
Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder characterized by persistent difficulties including repetitive patterns of behavior known as stereotypical motor movements (SMM). So far, several techniques have been implemented to track and identify SMMs. In this context, we propose a deep learning approach for SMM recognition, namely, convolutional neural networks (CNN) in time and frequency-domains. To solve the intrasubject SMM variability, we propose a robust CNN model for SMM detection within subjects, whose parameters are set according to a proper analysis of SMM signals, thereby outperforming state-of-the-art SMM classification works. And, to solve the intersubject variability, we propose a global, fast, and light-weight framework for SMM detection across subjects which combines a knowledge transfer technique with an SVM classifier, therefore resolving the "real-life" medical issue associated with the lack of supervised SMMs per testing subject in particular. We further show that applying transfer learning across domains instead of transfer learning within the same domain also generalizes to the SMM target domain, thus alleviating the problem of the lack of supervised SMMs in general.
Collapse
|
19
|
Cameron C, Khalil I, Castle D. Determining Anxiety in Obsessive Compulsive Disorder through Behavioural Clustering and Variations in Repetition Intensity. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2018; 160:65-74. [PMID: 29728248 DOI: 10.1016/j.cmpb.2018.03.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/08/2017] [Revised: 02/18/2018] [Accepted: 03/20/2018] [Indexed: 06/08/2023]
Abstract
BACKGROUND AND OBJECTIVES Over the last decade, the application of computer vision techniques to the analysis of behavioural patterns has seen a considerable increase in research interest. One such interesting and recent application is the visual behavioural analysis of mental disorders. Despite the very recent surge in interest in this area, relatively little has been done thus far to assist individuals living with Obsessive Compulsive Disorder. The work proposed herein represents a proof of concept system designed to demonstrate the efficacy of such an approach, from the computational perspective. The specific focus of this work lies in demonstrating a mechanism for clustering different kinds of Obsessive Compulsive Disorder behaviours and then comparing new behaviours to existing behaviours to determine the approximate level of anxiety represented by a compulsive behaviour. METHODS The proposed system uses Temporal Motion Heat Maps, SURF descriptors, a visual bag of words model and SVM-based classification to categorise representations of various behaviours commonly seen in OCD. Moreover, we apply a set of statistical measures to the images in a given category in order to derive an approximate anxiety level for a given compulsive behaviour. This proof of concept is an essential step in the direction of integrating computational approaches into the treatment of psychiatric conditions such as Obsessive Compulsive Disorder, for more effective recovery. RESULTS Results gleaned from experimental simulations indicate that the proposed system is capable of correctly classifying different types of simulated Obsessive Compulsive Disorder behaviour classes 75% of the time, with the misclassifications almost exclusively occurring when two behavioural clusters appear highly similar. Based on this information the proposed system is then able to assign an approximate behavioural anxiety level to the compulsive behaviours that meets the approval of a mental health professional. CONCLUSIONS The proposed system demonstrates a good ability to categorise various types of simulated OCD behaviour, in addition to establishing an approximate anxiety level for a given compulsive behaviour. This research demonstrates strong potential for the use of such systems in assisting mental health professionals looking to better understand their patients' condition and treatment progress across time.
Collapse
Affiliation(s)
- Conor Cameron
- School of Computer Science and Information Technology, RMIT University, Melbourne, Victoria, Australia.
| | - Ibrahim Khalil
- School of Computer Science and Information Technology, RMIT University, Melbourne, Victoria, Australia
| | - David Castle
- Department of Psychiatry, The University of Melbourne, Australia
| |
Collapse
|
20
|
Ouss L, Le Normand MT, Bailly K, Leitgel Gille M, Gosme C, Simas R, Wenke J, Jeudon X, Thepot S, Da Silva T, Clady X, Thoueille E, Afshar M, Golse B, Guergova-Kuras M. Developmental Trajectories of Hand Movements in Typical Infants and Those at Risk of Developmental Disorders: An Observational Study of Kinematics during the First Year of Life. Front Psychol 2018. [PMID: 29515472 PMCID: PMC5826068 DOI: 10.3389/fpsyg.2018.00083] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022] Open
Abstract
HighlightsThe kinematics of hand movements (spatial use, curvature, acceleration, and velocity) of infants with their mothers in an interactive setting
are significantly associated with age in cohorts of typical and at-risk infants differ significantly at 5–6 months of age, depending on the context: relating either with an object or a person.
Environmental and developmental factors shape the developmental trajectories of hand movements in different cohorts: environment for infants with VIMs; stage of development for premature infants and those with West syndrome; and both factors for infants with orality disorders. The curvature of hand movements specifically reflects atypical development in infants with West syndrome when developmental age is considered.
We aimed to discriminate between typical and atypical developmental trajectory patterns of at-risk infants in an interactive setting in this observational and longitudinal study, with the assumption that hand movements (HM) reflect preverbal communication and its disorders. We examined the developmental trajectories of HM in five cohorts of at-risk infants and one control cohort, followed from ages 2 to 10 months: 25 West syndrome (WS), 13 preterm birth (PB), 16 orality disorder (OD), 14 with visually impaired mothers (VIM), 7 early hospitalization (EH), and 19 typically developing infants (TD). Video-recorded data were collected in three different structured interactive contexts. Descriptors of the hand motion were used to examine the extent to which HM were associated with age and cohort. We obtained four principal results: (i) the kinematics of HM (spatial use, curvature, acceleration, and velocity) were significantly associated with age in all cohorts; (ii) HM significantly differed at 5–6 months of age in TD infants, depending on the context; (iii) environmental and developmental factors shaped the developmental trajectories of HM in different cohorts: environment for VIM, development for PB and WS, and both factors for OD and; (iv) the curvatures of HM showed atypical development in WS infants when developmental age was considered. These findings support the importance of using kinematics of HM to identify very early developmental disorders in an interactive context and would allow early prevention and intervention for at-risk infants.
Collapse
Affiliation(s)
- Lisa Ouss
- Department of Child and Adolescent Psychiatry, Necker-Enfants-Malades Hospital, APHP Assistance Publique-Hopitaux De Paris, Université Paris Descartes, Université Sorbonne Paris Cité, Paris, France.,UMR 1129 Infantile Epilepsies and Brain Plasticity, Institut National de la Santé et de la Recherche Médicale, CEA, Université Paris Descartes, Paris, France
| | - Marie-Thérèse Le Normand
- Institut National de la Santé et de la Recherche Médicale & Laboratoire de Psychopathologie et Processus de Santé, Université Paris Descartes, Université Sorbonne Paris Cité, Paris, France
| | - Kevin Bailly
- Institut des Systèmes Intelligents et de Robotique, Centre National de la Recherche Scientifique UMR 7222, Université Pierre et Marie Curie Univ Paris 06, Sorbonne Universités, Paris, France
| | - Marluce Leitgel Gille
- Department of Child and Adolescent Psychiatry, Necker-Enfants-Malades Hospital, APHP Assistance Publique-Hopitaux De Paris, Université Paris Descartes, Université Sorbonne Paris Cité, Paris, France
| | - Christelle Gosme
- Department of Child and Adolescent Psychiatry, Necker-Enfants-Malades Hospital, APHP Assistance Publique-Hopitaux De Paris, Université Paris Descartes, Université Sorbonne Paris Cité, Paris, France
| | - Roberta Simas
- Department of Child and Adolescent Psychiatry, Necker-Enfants-Malades Hospital, APHP Assistance Publique-Hopitaux De Paris, Université Paris Descartes, Université Sorbonne Paris Cité, Paris, France
| | - Julia Wenke
- EA 3522, CRPMS, ED 450 Recherches en Psychanalyse et Psychopathologie, Université Paris Diderot, Université Sorbonne Paris Cité, Paris, France
| | - Xavier Jeudon
- Cellule Vidéo de l'Hôpital Necker Enfants Malades, Association A l'Aube de la Vie, Paris, France
| | | | | | - Xavier Clady
- Department of Visual Information, Vision Institute, Centre National de la Recherche Scientifique UMR S968, Institut National de la Santé et de la Recherche Médicale UMRS 7210, Université Pierre et Marie Curie Univ Paris 06, Sorbonne Universités, Paris, France
| | | | | | - Bernard Golse
- Department of Child and Adolescent Psychiatry, Necker-Enfants-Malades Hospital, APHP Assistance Publique-Hopitaux De Paris, Université Paris Descartes, Université Sorbonne Paris Cité, Paris, France
| | | |
Collapse
|
21
|
Dy ME, Waugh JL, Sharma N, O’Leary H, Kapur K, D’Gama AM, Sahin M, Urion DK, Kaufmann WE. Defining Hand Stereotypies in Rett Syndrome: A Movement Disorders Perspective. Pediatr Neurol 2017; 75:91-95. [PMID: 28838622 PMCID: PMC5624791 DOI: 10.1016/j.pediatrneurol.2017.05.025] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/19/2017] [Accepted: 05/28/2017] [Indexed: 10/19/2022]
Abstract
INTRODUCTION Hand stereotypies (HS) are a primary diagnostic criterion for Rett syndrome (RTT) but are difficult to characterize and quantify systematically. METHODS We collected video on 27 girls (2-12 years of age) with classic RTT who participated in a mecasermin trial. The present study focused exclusively on video analyses, by reviewing two five-minute windows per subject to identify the two most common HS. Three raters with expertise in movement disorders independently rated the five-minute windows using standardized terminology to determine the level of agreement. We iteratively refined the protocol in three stages to improve descriptive accuracy, categorizing HS as "central" or "peripheral," "simple" or "complex," scoring each hand separately. Inter-rater agreement was analyzed using Kappa statistics. RESULTS In the initial protocol evaluating HS by video, inter-rater agreement was 20.7%. In the final protocol, inter-rater agreement for the two most frequent HS was higher than the initial protocol at 50%. CONCLUSION Phenotypic variability makes standardized evaluation of HS in RTT a challenge; we achieved only 50% level of agreement and only for the most frequent HS. Therefore, objective measures are needed to evaluate HS.
Collapse
Affiliation(s)
- Marisela E. Dy
- Boston Children’s Hospital,Massachusetts General Hospital,Harvard Medical School
| | - Jeff L. Waugh
- Boston Children’s Hospital,Massachusetts General Hospital,Harvard Medical School
| | - Nutan Sharma
- Boston Children’s Hospital,Massachusetts General Hospital,Harvard Medical School
| | | | - Kush Kapur
- Boston Children’s Hospital,Harvard Medical School
| | | | | | | | - Walter E. Kaufmann
- Boston Children’s Hospital,Harvard Medical School,Greenwood Genetic Center
| |
Collapse
|
22
|
Apprendre en s’observant faire ce que l’on n’a jamais fait : une méthode innovante pour enfants non verbaux avec trouble du spectre de l’autisme. ENFANCE 2017. [DOI: 10.4074/s0013754517002063] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
23
|
Nadel J, Sagnier É, Tijus C. Apprendre en s’observant faire ce que l’on n’a jamais fait : une méthode innovante pour enfants non verbaux avec trouble du spectre de l’autisme. ENFANCE 2017. [DOI: 10.3917/enf1.172.0263] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
|
24
|
WearSense: Detecting Autism Stereotypic Behaviors through Smartwatches. Healthcare (Basel) 2017; 5:healthcare5010011. [PMID: 28264474 PMCID: PMC5371917 DOI: 10.3390/healthcare5010011] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2016] [Revised: 02/16/2017] [Accepted: 02/21/2017] [Indexed: 11/16/2022] Open
Abstract
Autism is a complex developmental disorder that affects approximately 1 in 68 children (according to the recent survey conducted by the Centers for Disease Control and Prevention—CDC) in the U.S., and has become the fastest growing category of special education. Each student with autism comes with her or his own unique needs and an array of behaviors and habits that can be severe and which interfere with everyday tasks. Autism is associated with intellectual disability, impairments in social skills, and physical health issues such as sleep and abdominal disturbances. We have designed an Internet-of-Things (IoT) framework named WearSense that leverages the sensing capabilities of modern smartwatches to detect stereotypic behaviors in children with autism. In this work, we present a study that used the inbuilt accelerometer of a smartwatch to detect three behaviors, including hand flapping, painting, and sibbing that are commonly observed in children with autism. In this feasibility study, we recruited 14 subjects to record the accelerometer data from the smartwatch worn on the wrist. The processing part extracts 34 different features in each dimension of the three-axis accelerometer, resulting in 102 features. Using and comparing various classification techniques revealed that an ensemble of 40 decision trees has the best accuracy of around 94.6%. This accuracy shows the quality of the data collected from the smartwatch and feature extraction methods used in this study. The recognition of these behaviors by using a smartwatch would be helpful in monitoring individuals with autistic behaviors, since the smartwatch can send the data to the cloud for comprehensive analysis and also to help parents, caregivers, and clinicians make informed decisions.
Collapse
|
25
|
Großekathöfer U, Manyakov NV, Mihajlović V, Pandina G, Skalkin A, Ness S, Bangerter A, Goodwin MS. Automated Detection of Stereotypical Motor Movements in Autism Spectrum Disorder Using Recurrence Quantification Analysis. Front Neuroinform 2017; 11:9. [PMID: 28261082 PMCID: PMC5311048 DOI: 10.3389/fninf.2017.00009] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2016] [Accepted: 01/23/2017] [Indexed: 11/24/2022] Open
Abstract
A number of recent studies using accelerometer features as input to machine learning classifiers show promising results for automatically detecting stereotypical motor movements (SMM) in individuals with Autism Spectrum Disorder (ASD). However, replicating these results across different types of accelerometers and their position on the body still remains a challenge. We introduce a new set of features in this domain based on recurrence plot and quantification analyses that are orientation invariant and able to capture non-linear dynamics of SMM. Applying these features to an existing published data set containing acceleration data, we achieve up to 9% average increase in accuracy compared to current state-of-the-art published results. Furthermore, we provide evidence that a single torso sensor can automatically detect multiple types of SMM in ASD, and that our approach allows recognition of SMM with high accuracy in individuals when using a person-independent classifier.
Collapse
Affiliation(s)
| | | | | | | | | | - Seth Ness
- Janssen Research & Development Titusville, NJ, USA
| | | | - Matthew S Goodwin
- Department of Health Sciences, Northeastern University Boston, MA, USA
| |
Collapse
|
26
|
Cabibihan JJ, Javed H, Aldosari M, Frazier TW, Elbashir H. Sensing Technologies for Autism Spectrum Disorder Screening and Intervention. SENSORS 2016; 17:s17010046. [PMID: 28036004 PMCID: PMC5298619 DOI: 10.3390/s17010046] [Citation(s) in RCA: 34] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/03/2016] [Revised: 12/15/2016] [Accepted: 12/16/2016] [Indexed: 12/22/2022]
Abstract
This paper reviews the state-of-the-art in sensing technologies that are relevant for Autism Spectrum Disorder (ASD) screening and therapy. This disorder is characterized by difficulties in social communication, social interactions, and repetitive behaviors. It is diagnosed during the first three years of life. Early and intensive interventions have been shown to improve the developmental trajectory of the affected children. The earlier the diagnosis, the sooner the intervention therapy can begin, thus, making early diagnosis an important research goal. Technological innovations have tremendous potential to assist with early diagnosis and improve intervention programs. The need for careful and methodological evaluation of such emerging technologies becomes important in order to assist not only the therapists and clinicians in their selection of suitable tools, but to also guide the developers of the technologies in improving hardware and software. In this paper, we survey the literatures on sensing technologies for ASD and we categorize them into eye trackers, movement trackers, electrodermal activity monitors, tactile sensors, vocal prosody and speech detectors, and sleep quality assessment devices. We assess their effectiveness and study their limitations. We also examine the challenges faced by this growing field that need to be addressed before these technologies can perform up to their theoretical potential.
Collapse
Affiliation(s)
- John-John Cabibihan
- Mechanical and Industrial Engineering Department, Qatar University, Doha 2713, Qatar.
| | - Hifza Javed
- Biomedical Engineering Department, George Washington University, Washington, DC 20052, USA.
| | - Mohammed Aldosari
- Center for Pediatric Neurology, Cleveland Clinic, and Case Western Reserve University, Cleveland, OH 44106, USA.
| | - Thomas W Frazier
- Center for Autism Pediatric Institute, Cleveland Clinic and Cleveland Clinic Lerner College of Medicine, Cleveland, OH 44195, USA.
| | - Haitham Elbashir
- Al Jalila Children's Specialty Hospital, Dubai, United Arab Emirates.
| |
Collapse
|
27
|
Wolff JJ, Boyd BA, Elison JT. A quantitative measure of restricted and repetitive behaviors for early childhood. J Neurodev Disord 2016; 8:27. [PMID: 27486483 PMCID: PMC4970296 DOI: 10.1186/s11689-016-9161-x] [Citation(s) in RCA: 25] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/05/2016] [Accepted: 07/17/2016] [Indexed: 01/01/2023] Open
Abstract
Background Restricted and repetitive behaviors are characteristic phenotypic features of many neurodevelopmental, psychiatric, and neurological conditions. During early childhood, such behaviors are considered normative. More research is needed to delineate the dimensions of restricted and repetitive behavior across typical and atypical development during this period. Methods We developed the 34-item parent-rated Repetitive Behavior Scale for Early Childhood (RBS-EC) to capture quantitative, dimensional features across a broad range of behaviors contributing to this domain. We evaluated its psychometric properties and factor structure in a community sample of 914 toddlers. Results The RBS-EC showed excellent overall internal consistency (α = 0.90), strong test-retest reliability (ICC = 0.87 for topographies and 0.90 for frequency) and evidence of convergent and discriminative validity. Using a split-half approach to factor analysis, we identified that a three- or four-factor structure best fit the data and confirmatory factor analysis indicated acceptable fit for both models. The empirically derived four-factor model was consistent with our conceptual model and included repetitive motor, restricted interests and behavior, ritual and routine, and self-directed behavior. Conclusions This initial study indicates that the RBS-EC is a reliable and valid instrument for characterizing quantitative, dimensional aspects of restricted and repetitive behaviors in young children. Electronic supplementary material The online version of this article (doi:10.1186/s11689-016-9161-x) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Jason J Wolff
- Department of Educational Psychology, University of Minnesota, Minneapolis, MN USA
| | - Brian A Boyd
- Department of Allied Health Sciences, University of North Carolina, Chapel Hill, NC USA
| | - Jed T Elison
- Institute of Child Development, University of Minnesota, Minneapolis, MN USA
| |
Collapse
|
28
|
Ni Q, García Hernando AB, de la Cruz IP. The Elderly's Independent Living in Smart Homes: A Characterization of Activities and Sensing Infrastructure Survey to Facilitate Services Development. SENSORS (BASEL, SWITZERLAND) 2015; 15:11312-62. [PMID: 26007717 PMCID: PMC4481963 DOI: 10.3390/s150511312] [Citation(s) in RCA: 62] [Impact Index Per Article: 6.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/02/2015] [Accepted: 05/08/2015] [Indexed: 11/17/2022]
Abstract
Human activity detection within smart homes is one of the basis of unobtrusive wellness monitoring of a rapidly aging population in developed countries. Most works in this area use the concept of "activity" as the building block with which to construct applications such as healthcare monitoring or ambient assisted living. The process of identifying a specific activity encompasses the selection of the appropriate set of sensors, the correct preprocessing of their provided raw data and the learning/reasoning using this information. If the selection of the sensors and the data processing methods are wrongly performed, the whole activity detection process may fail, leading to the consequent failure of the whole application. Related to this, the main contributions of this review are the following: first, we propose a classification of the main activities considered in smart home scenarios which are targeted to older people's independent living, as well as their characterization and formalized context representation; second, we perform a classification of sensors and data processing methods that are suitable for the detection of the aforementioned activities. Our aim is to help researchers and developers in these lower-level technical aspects that are nevertheless fundamental for the success of the complete application.
Collapse
Affiliation(s)
- Qin Ni
- Departamento de Ingeniería Telemática y Electrónica, Universidad Politécnica de Madrid, Carretera de Valencia km. 7, Madrid 28031, Spain.
| | - Ana Belén García Hernando
- Departamento de Ingeniería Telemática y Electrónica, Universidad Politécnica de Madrid, Carretera de Valencia km. 7, Madrid 28031, Spain.
| | - Iván Pau de la Cruz
- Departamento de Ingeniería Telemática y Electrónica, Universidad Politécnica de Madrid, Carretera de Valencia km. 7, Madrid 28031, Spain.
| |
Collapse
|
29
|
Marcroft C, Khan A, Embleton ND, Trenell M, Plötz T. Movement recognition technology as a method of assessing spontaneous general movements in high risk infants. Front Neurol 2015; 5:284. [PMID: 25620954 PMCID: PMC4288331 DOI: 10.3389/fneur.2014.00284] [Citation(s) in RCA: 55] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2014] [Accepted: 12/15/2014] [Indexed: 11/17/2022] Open
Abstract
Preterm birth is associated with increased risks of neurological and motor impairments such as cerebral palsy. The risks are highest in those born at the lowest gestations. Early identification of those most at risk is challenging meaning that a critical window of opportunity to improve outcomes through therapy-based interventions may be missed. Clinically, the assessment of spontaneous general movements is an important tool, which can be used for the prediction of movement impairments in high risk infants. Movement recognition aims to capture and analyze relevant limb movements through computerized approaches focusing on continuous, objective, and quantitative assessment. Different methods of recording and analyzing infant movements have recently been explored in high risk infants. These range from camera-based solutions to body-worn miniaturized movement sensors used to record continuous time-series data that represent the dynamics of limb movements. Various machine learning methods have been developed and applied to the analysis of the recorded movement data. This analysis has focused on the detection and classification of atypical spontaneous general movements. This article aims to identify recent translational studies using movement recognition technology as a method of assessing movement in high risk infants. The application of this technology within pediatric practice represents a growing area of inter-disciplinary collaboration, which may lead to a greater understanding of the development of the nervous system in infants at high risk of motor impairment.
Collapse
Affiliation(s)
- Claire Marcroft
- Neonatal Service, Royal Victoria Infirmary (RVI), Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK
- MoveLab, The Medical School, Newcastle University, Newcastle upon Tyne, UK
| | - Aftab Khan
- Culture Lab, School of Computing Science, Newcastle University, Newcastle upon Tyne, UK
| | - Nicholas D. Embleton
- Neonatal Service, Royal Victoria Infirmary (RVI), Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK
| | - Michael Trenell
- MoveLab, The Medical School, Newcastle University, Newcastle upon Tyne, UK
| | - Thomas Plötz
- Culture Lab, School of Computing Science, Newcastle University, Newcastle upon Tyne, UK
| |
Collapse
|
30
|
Computer vision tools for low-cost and noninvasive measurement of autism-related behaviors in infants. AUTISM RESEARCH AND TREATMENT 2014; 2014:935686. [PMID: 25045536 PMCID: PMC4090521 DOI: 10.1155/2014/935686] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/19/2013] [Revised: 04/30/2014] [Accepted: 05/13/2014] [Indexed: 01/21/2023]
Abstract
The early detection of developmental disorders is key to child outcome, allowing interventions to be initiated which promote development and improve prognosis. Research on autism spectrum disorder (ASD) suggests that behavioral signs can be observed late in the first year of life. Many of these studies involve extensive frame-by-frame video observation and analysis of a child's natural behavior. Although nonintrusive, these methods are extremely time-intensive and require a high level of observer training; thus, they are burdensome for clinical and large population research purposes. This work is a first milestone in a long-term project on non-invasive early observation of children in order to aid in risk detection and research of neurodevelopmental disorders. We focus on providing low-cost computer vision tools to measure and identify ASD behavioral signs based on components of the Autism Observation Scale for Infants (AOSI). In particular, we develop algorithms to measure responses to general ASD risk assessment tasks and activities outlined by the AOSI which assess visual attention by tracking facial features. We show results, including comparisons with expert and nonexpert clinicians, which demonstrate that the proposed computer vision tools can capture critical behavioral observations and potentially augment the clinician's behavioral observations obtained from real in-clinic assessments.
Collapse
|
31
|
Ousley O, Cermak T. Autism Spectrum Disorder: Defining Dimensions and Subgroups. CURRENT DEVELOPMENTAL DISORDERS REPORTS 2014; 1:20-28. [PMID: 25072016 PMCID: PMC4111262 DOI: 10.1007/s40474-013-0003-1] [Citation(s) in RCA: 81] [Impact Index Per Article: 8.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
Abstract
Autism spectrum disorder (ASD) is a behaviorally defined neurodevelopmental disorder associated with the presence of social-communication deficits and restricted and repetitive behaviors. In the latest conceptualization of ASD, these two behavioral dimensions represent the core defining features of ASD, whereas associated dimensions, such as intellectual and language ability, provide a means for describing the ASD heterogeneity. In addition, the characterization of ASD subgroups, defined by the presence of known medical, genetic, or other psychiatric disorders, furthers our understanding of ASD heterogeneity. This paper reviews the history of autism, describes its core defining features, and provides an overview of the clinically and etiologically relevant subgroups that add to the complexity of this condition.
Collapse
Affiliation(s)
- Opal Ousley
- Emory Autism Center, Department of Psychiatry and Behavioral
Sciences, Emory University School of Medicine, 1551 Shoup Court, Atlanta, GA
30322, USA
| | - Tracy Cermak
- Marcus Autism Center, Children’s Healthcare of Atlanta,
1920 Briarcliff Road, Atlanta, GA 30329, USA
| |
Collapse
|
32
|
Cohen IL, Gardner JM, Karmel BZ, Kim SY. Rating scale measures are associated with Noldus EthoVision-XT video tracking of behaviors of children on the autism spectrum. Mol Autism 2014; 5:15. [PMID: 24548743 PMCID: PMC3974120 DOI: 10.1186/2040-2392-5-15] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2013] [Accepted: 02/03/2014] [Indexed: 12/11/2022] Open
Abstract
Background Children with Autism Spectrum Disorder (ASD) show unusual social behaviors and repetitive behaviors. Some of these behaviors, e.g., time spent in an area or turning rate/direction, can be automatically tracked. Automated tracking has several advantages over subjective ratings including reliability, amount of information provided, and consistency across laboratories, and is potentially of importance for diagnosis, animal models and objective assessment of treatment efficacy. However, its validity for ASD has not been examined. In this exploratory study, we examined associations between rating scale data with automated tracking of children’s movements using the Noldus EthoVision XT system; i.e., tracking not involving a human observer. Based on our observations and previous research, we predicted that time spent in the periphery of the room would be associated with autism severity and that rate and direction of turning would be associated with stereotypies. Methods Children with and without ASD were observed in a free-play situation for 3 min before and 3 min after Autism Diagnostic Observation Scale – Generic (ADOS-G) testing. The Noldus system provided measures of the rate and direction of turning, latency to approach and time spend near the periphery or the parent. Results Ratings of the severity of maladaptive social behaviors, stereotypies, autism severity, and arousal problems were positively correlated with increases in percent time spent in the periphery in the total sample and in the ASD subset. Adaptive social communication skills decreased with increases in the percentage of time spent in the periphery and increases in the latency to approach the parent in the ASD group. The rate and direction of turning was linked with stereotypies only in the group without ASD (the faster the rate of a turn to the left, the worse the rating). In the ASD group, there was a shift from a neutral turning bias prior to the ADOS assessment to a strong left turn bias after the ADOS assessment. In the entire sample, this left turn bias was associated with measures of autism severity. Conclusion Results suggest that automated tracking yields valid and unbiased information for assessing children with autism. Turning bias is an interesting and unexplored measure related to autism.
Collapse
Affiliation(s)
- Ira L Cohen
- Department of Psychology, New York State Institute for Basic Research in Developmental Disabilities, 1050 Forest Hill Road, Staten Island, NY 10314, USA.
| | | | | | | |
Collapse
|
33
|
LeMonda BC, Holtzer R, Goldman S. Relationship between executive functions and motor stereotypies in children with Autistic Disorder. RESEARCH IN AUTISM SPECTRUM DISORDERS 2012; 6:1099-1106. [PMID: 23637708 PMCID: PMC3638958 DOI: 10.1016/j.rasd.2012.03.001] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/02/2023]
Abstract
This study reports on the relationship between motor stereotypies and impairments in executive functions (EF) in children with Autistic Disorder (AD) and in children with Developmental Language Disorders (DLD). We hypothesized that low EF performance would predict higher frequency and longer durations of stereotypies in the AD group only. Twenty-two children (age range = 7-9 years, 6 months, girls = 5) with AD were recruited from a longitudinal multi-site study and compared to twenty-two non-autistic children with DLD (age range = 7-9 years, 6 months, girls = 5). The two groups were matched on non-verbal IQ and demographic characteristics. Frequency and duration of stereotypies were coded from videotaped semi-structured play sessions. EF measures included the Wisconsin Card Sorting Task (WCST) Categories, Wechsler Intelligence Scale for Children-Revised (WISC-R) Mazes, and Stanford-Binet Fourth Edition (SB-IV) Matrices. The scores for frequency and duration of stereotypies were higher in the AD group. Separate linear regressions revealed that group status, EF, and their interactions predict stereotypies. Specifically, lower EF scores predicted higher frequencies and longer durations of stereotypies in the AD group only. Analyses controlled for age, gender, and parent education. Findings suggest that in AD, EF impairments and stereotypies may be linked to shared brain pathways.
Collapse
Affiliation(s)
- Brittany C. LeMonda
- Ferkauf Graduate School of Psychology, Yeshiva University, 1165 Morris Park Road, Bronx, NY 10461, United States
| | - Roee Holtzer
- Ferkauf Graduate School of Psychology, Yeshiva University, 1165 Morris Park Road, Bronx, NY 10461, United States
- Department of Neurology, Albert Einstein College of Medicine, Jack and Pearl Resnick Campus, Yeshiva University, 1300 Morris Park Avenue, Bronx, NY 10461, United States
| | - Sylvie Goldman
- Department of Neurology, Albert Einstein College of Medicine, Jack and Pearl Resnick Campus, Yeshiva University, 1300 Morris Park Avenue, Bronx, NY 10461, United States
- Department of Pediatrics, Albert Einstein College of Medicine, Jack and Pearl Resnick Campus, Yeshiva University, 1300 Morris Park Avenue, Bronx, NY 10461, United States
| |
Collapse
|
34
|
Karch D, Kang KS, Wochner K, Philippi H, Hadders-Algra M, Pietz J, Dickhaus H. Kinematic assessment of stereotypy in spontaneous movements in infants. Gait Posture 2012; 36:307-11. [PMID: 22503388 DOI: 10.1016/j.gaitpost.2012.03.017] [Citation(s) in RCA: 40] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/08/2011] [Revised: 02/13/2012] [Accepted: 03/19/2012] [Indexed: 02/02/2023]
Abstract
Movement variation constitutes a crucial feature of infant motor development. Reduced variation of spontaneous infant movements, i.e. stereotyped movements, may indicate severe neurological deficit at an early stage. Hitherto evaluation of movement variation has been mainly restricted to subjective assessment based on observation. This article introduces a method for quantitative assessment yielding an objective definition of stereotyped movements which may be used for the prognosis of neurological deficits such as cerebral palsy (CP). Movements of 3-month-old infants were recorded with an electromagnetic tracking system facilitating the analysis of joint angles of the upper and lower limb. A stereotypy score based on dynamic time warping has been developed describing movements which are self-similar in multiple degrees of freedom. For clinical evaluation, this measure was calculated in a group of infants at risk for neurological disorders (n=54) and a control group of typically developing children (n=21) on the basis of spontaneous movements at the age of 3 months. The stereotypy score was related to outcome at the age of 24 months in terms of CP (n=10) or no-CP (n=53). Using the stereotypy score of upper limb movements CP cases could be identified with a sensitivity of 90% and a specificity of 96%. The corresponding score of the leg movements did not allow for valid discrimination of the groups. The presented stereotypy feature is a promising candidate for a marker that may be used as a simple and noninvasive quantitative measure in the prediction of CP. The method can be adopted for the assessment of infant movement variation in research and clinical applications.
Collapse
Affiliation(s)
- Dominik Karch
- Institute of Medical Biometry and Informatics, University of Heidelberg, Germany.
| | | | | | | | | | | | | |
Collapse
|