Automatic emotion and attention analysis of young children at home: a ResearchKit autism feasibility study

Provider perspectives on equity in use of mobile health autism screening tools

LAY ABSTRACT

Families may find information about autism online, and health care and education providers may use online tools to screen for autism. However, we do not know if online autism screening tools are easily used by families and providers. We interviewed primary care and educational providers, asking them to review results from online tools that screen for autism. Providers had concerns about how usable and accessible these tools are for diverse families and suggested changes to make tools easier to use.

Objective Measurement of Social Gaze and Smile Behaviors in Children with Suspected Autism Spectrum Disorder During Administration of the Autism Diagnostic Observation Schedule, 2nd Edition

Best practice for the assessment of autism spectrum disorder (ASD) symptom severity relies on clinician ratings of the Autism Diagnostic Observation Schedule, 2nd Edition (ADOS-2), but the association of these ratings with objective measures of children's social gaze and smiling is unknown. Sixty-six preschool-age children (49 boys, M = 39.97 months, SD = 10.58) with suspected ASD (61 confirmed ASD) were administered the ADOS-2 and provided social affect calibrated severity scores (SA CSS). Children's social gaze and smiling during the ADOS-2, captured with a camera contained in eyeglasses worn by the examiner and parent, were obtained via a computer vision processing pipeline. Children who gazed more at their parents (p = .04) and whose gaze at their parents involved more smiling (p = .02) received lower social affect severity scores, indicating fewer social affect symptoms, adjusted R2 = .15, p = .003.

Computer Vision Analysis of Caregiver-Child Interactions in Children with Neurodevelopmental Disorders: A Preliminary Report

We report preliminary results of computer vision analysis of caregiver-child interactions during free play with children diagnosed with autism (N = 29, 41-91 months), attention-deficit/hyperactivity disorder (ADHD, N = 22, 48-100 months), or combined autism + ADHD (N = 20, 56-98 months), and neurotypical children (NT, N = 7, 55-95 months). We conducted micro-analytic analysis of 'reaching to a toy,' as a proxy for initiating or responding to a toy play bout. Dyadic analysis revealed two clusters of interaction patterns, which differed in frequency of 'reaching to a toy' and caregivers' contingent responding to the child's reach for a toy by also reaching for a toy. Children in dyads with higher caregiver responsiveness had less developed language, communication, and socialization skills. Clusters were not associated with diagnostic groups. These results hold promise for automated methods of characterizing caregiver responsiveness in dyadic interactions for assessment and outcome monitoring in clinical trials.

Using mobile health technology to assess childhood autism in low-resource community settings in India: An innovation to address the detection gap

LAY ABSTRACT

Autism is diagnosed by highly trained professionals- but most autistic people live in parts of the world that harbour few or no such autism specialists and little autism awareness. So many autistic people go undiagnosed, misdiagnosed, and misunderstood. We designed an app (START) to identify autism and related conditions in such places, in an attempt to address this global gap in access to specialists. START uses computerised games and activities for children and a questionnaire for parents to measure social, sensory, and motor skills. To check whether START can flag undiagnosed children likely to have neurodevelopmental conditions, we tested START with children whose diagnoses already were known: Non-specialist health workers with just a high-school education took START to family homes in poor neighbourhoods of Delhi, India to work with 131 two-to-seven-year-olds. Differences between typically and atypically developing children were highlighted in all three types of skills that START assesses: children with neurodevelopmental conditions preferred looking at geometric patterns rather than social scenes, were fascinated by predictable, repetitive sensory stimuli, and had more trouble with precise hand movements. Parents' responses to surveys further distinguished autistic from non-autistic children. An artificial-intelligence technique combining all these measures demonstrated that START can fairly accurately flag atypically developing children. Health workers and families endorsed START as attractive to most children, understandable to health workers, and adaptable within sometimes chaotic home and family environments. This study provides a proof of principle for START in digital screening of autism and related conditions in community settings.

Self-administered questionnaires enhance emotion estimation of individuals with autism spectrum disorders in a robotic interview setting

Background

Robots offer many unique opportunities for helping individuals with autism spectrum disorders (ASD). Determining the optimal motion of robots when interacting with individuals with ASD is important for achieving more natural human-robot interactions and for exploiting the full potential of robotic interventions. Most prior studies have used supervised machine learning (ML) of user behavioral data to enable robot perception of affective states (i.e., arousal and valence) and engagement. It has previously been suggested that including personal demographic information in the identification of individuals with ASD is important for developing an automated system to perceive individual affective states and engagement. In this study, we hypothesized that assessing self-administered questionnaire data would contribute to the development of an automated estimation of the affective state and engagement when individuals with ASD are interviewed by an Android robot, which will be linked to implementing long-term interventions and maintaining the motivation of participants.

Methods

Participants sat across a table from an android robot that played the role of the interviewer. Each participant underwent a mock job interview. Twenty-five participants with ASD (males 22, females 3, average chronological age = 22.8, average IQ = 94.04) completed the experiment. We collected multimodal data (i.e., audio, motion, gaze, and self-administered questionnaire data) to train a model to correctly classify the state of individuals with ASD when interviewed by an android robot. We demonstrated the technical feasibility of using ML to enable robot perception of affect and engagement of individuals with ASD based on multimodal data.

Results

For arousal and engagement, the area under the curve (AUC) values of the model estimates and expert coding were relatively high. Overall, the AUC values of arousal, valence, and engagement were improved by including self-administered questionnaire data in the classification.

Discussion

These findings support the hypothesis that assessing self-administered questionnaire data contributes to the development of an automated estimation of an individual's affective state and engagement. Given the efficacy of including self-administered questionnaire data, future studies should confirm the effectiveness of such long-term intervention with a robot to maintain participants' motivation based on the proposed method of emotion estimation.

Development of webcam-collected and artificial-intelligence-derived social and cognitive performance measures for neurodevelopmental genetic syndromes

This study focused on the development and initial psychometric evaluation of a set of online, webcam-collected, and artificial intelligence-derived patient performance measures for neurodevelopmental genetic syndromes (NDGS). Initial testing and qualitative input was used to develop four stimulus paradigms capturing social and cognitive processes, including social attention, receptive vocabulary, processing speed, and single-word reading. The paradigms were administered to a sample of 375 participants, including 163 with NDGS, 56 with idiopathic neurodevelopmental disability (NDD), and 156 neurotypical controls. Twelve measures were created from the four stimulus paradigms. Valid completion rates varied from 87 to 100% across measures, with lower but adequate completion rates in participants with intellectual disability. Adequate to excellent internal consistency reliability (α = 0.67 to 0.95) was observed across measures. Test-retest reproducibility at 1-month follow-up and stability at 4-month follow-up was fair to good (r = 0.40-0.73) for 8 of the 12 measures. All gaze-based measures showed evidence of convergent and discriminant validity with parent-report measures of other cognitive and behavioral constructs. Comparisons across NDGS groups revealed distinct patterns of social and cognitive functioning, including people with PTEN mutations showing a less impaired overall pattern and people with SYNGAP1 mutations showing more attentional, processing speed, and social processing difficulties relative to people with NFIX mutations. Webcam-collected performance measures appear to be a reliable and potentially useful method for objective characterization and monitoring of social and cognitive processes in NDGS and idiopathic NDD. Additional validation work, including more detailed convergent and discriminant validity analyses and examination of sensitivity to change, is needed to replicate and extend these observations.

A Review of and Roadmap for Data Science and Machine Learning for the Neuropsychiatric Phenotype of Autism

Autism spectrum disorder (autism) is a neurodevelopmental delay that affects at least 1 in 44 children. Like many neurological disorder phenotypes, the diagnostic features are observable, can be tracked over time, and can be managed or even eliminated through proper therapy and treatments. However, there are major bottlenecks in the diagnostic, therapeutic, and longitudinal tracking pipelines for autism and related neurodevelopmental delays, creating an opportunity for novel data science solutions to augment and transform existing workflows and provide increased access to services for affected families. Several efforts previously conducted by a multitude of research labs have spawned great progress toward improved digital diagnostics and digital therapies for children with autism. We review the literature on digital health methods for autism behavior quantification and beneficial therapies using data science. We describe both case-control studies and classification systems for digital phenotyping. We then discuss digital diagnostics and therapeutics that integrate machine learning models of autism-related behaviors, including the factors that must be addressed for translational use. Finally, we describe ongoing challenges and potential opportunities for the field of autism data science. Given the heterogeneous nature of autism and the complexities of the relevant behaviors, this review contains insights that are relevant to neurological behavior analysis and digital psychiatry more broadly.

A Survey on Autism Care, Diagnosis, and Intervention Based on Mobile Apps Focusing on Usability and Software Design

This article presents a systematic review on autism care, diagnosis, and intervention based on mobile apps running on smartphones and tablets. Here, the term "intervention" means a carefully planned set of activities with the objective of improving autism symptoms. We guide our review on related studies using five research questions. First, who benefits the most from these mobile apps? Second, what are the primary purposes of these mobile apps? Third, what mechanisms have been incorporated in these mobiles apps to improve usability? Fourth, what guidelines have been used in the design and implementation of these mobile apps? Fifth, what theories and frameworks have been used as the foundation for these mobile apps to ensure the intervention effectiveness? As can be seen from these research questions, we focus on the usability and software development of the mobile apps. Informed by the findings of these research questions, we propose a taxonomy for the mobile apps and their users. The mobile apps can be categorized into autism support apps, educational apps, teacher training apps, parental support apps, and data collection apps. The individuals with autism spectrum disorder (ASD) are the primary users of the first two categories of apps. Teachers of children with ASD are the primary users of the teacher training apps. Parents are the primary users of the parental support apps, while individuals with ASD are usually the primary users of the data collection apps and clinicians and autism researchers are the beneficiaries. Gamification, virtual reality, and autism-specific mechanisms have been used to improve the usability of the apps. User-centered design is the most popular approach for mobile app development. Augmentative and alternative communication, video modeling, and various behavior change practices have been used as the theoretical foundation for intervention efficacy.

A tablet-based game for the assessment of visual motor skills in autistic children

Increasing evidence suggests that early motor impairments are a common feature of autism. Thus, scalable, quantitative methods for measuring motor behavior in young autistic children are needed. This work presents an engaging and scalable assessment of visual-motor abilities based on a bubble-popping game administered on a tablet. Participants are 233 children ranging from 1.5 to 10 years of age (147 neurotypical children and 86 children diagnosed with autism spectrum disorder [autistic], of which 32 are also diagnosed with co-occurring attention-deficit/hyperactivity disorder [autistic+ADHD]). Computer vision analyses are used to extract several game-based touch features, which are compared across autistic, autistic+ADHD, and neurotypical participants. Results show that younger (1.5-3 years) autistic children pop the bubbles at a lower rate, and their ability to touch the bubble's center is less accurate compared to neurotypical children. When they pop a bubble, their finger lingers for a longer period, and they show more variability in their performance. In older children (3-10-years), consistent with previous research, the presence of co-occurring ADHD is associated with greater motor impairment, reflected in lower accuracy and more variable performance. Several motor features are correlated with standardized assessments of fine motor and cognitive abilities, as evaluated by an independent clinical assessment. These results highlight the potential of touch-based games as an efficient and scalable approach for assessing children's visual-motor skills, which can be part of a broader screening tool for identifying early signs associated with autism.

Digitally assisted diagnostics of autism spectrum disorder

Digital technologies have the potential to support psychiatric diagnostics and, in particular, differential diagnostics of autism spectrum disorder in the near future, making clinical decisions more objective, reliable and evidence-based while reducing clinical resources. Multimodal automatized measurement of symptoms at cognitive, behavioral, and neuronal levels combined with artificial intelligence applications offer promising strides toward personalized prognostics and treatment strategies. In addition, these new technologies could enable systematic and continuous assessment of longitudinal symptom development, beyond the usual scope of clinical practice. Early recognition of exacerbation and simplified, as well as detailed, progression control would become possible. Ultimately, digitally assisted diagnostics will advance early recognition. Nonetheless, digital technologies cannot and should not substitute clinical decision making that takes the comprehensive complexity of individual longitudinal and cross-section presentation of autism spectrum disorder into account. Yet, they might aid the clinician by objectifying decision processes and provide a welcome relief to resources in the clinical setting.

Early diagnosis of autism using indian autism grading tool

Autism spectrum disorder is a neuro-developmental disorder that affects communication and social skills in individuals. Screening and diagnosis of autism using conventional methods, such as interviews with parents or caregivers and observational assessments takes a long time. The accurate diagnosis of autism by physicians and healthcare professionals seems to be challenging. By analyzing data on autistic children, medical professionals can learn about autism screening assessment decision making. The present study aims to develop a parental autism screening tool termed the Indian Autism Grading Tool (IAGT) for early screening of autism. Data are collected using the Indian Autism Parental Questionnaire and assigned with grades. This dataset is employed to test five supervised machine learning models, which compare classification performance based on accuracy, precision and recall. The most effective model should be used to implement the autism screening application. MLR is known to be more robust and to support fewer data sets, so it can be employed for the implementation of ML-powered mobile applications. MLR achieves the overall accuracy of 97.85%, which equates to 0.72%, 2.37%, 0.84% and 1.54% better than SVM, DT, KNN and GNB respectively. The proposed tool is developed in both Tamil and English. The pilot study is conducted with 30 children and the predictability of the tool is compared with the clinician. Therefore, the tool consistently achieves the same level of accuracy as clinicians.

The feasibility of using Apple's ResearchKit for recruitment and data collection: Considerations for mental health research

In 2015, Apple launched an open-source software framework called ResearchKit. ResearchKit provides an infrastructure for conducting remote, smartphone-based research trials through the means of Apple's App Store. Such trials may have several advantages over conventional trial methods including the removal of geographic barriers, frequent assessments of participants in real-life settings, and increased inclusion of seldom-heard communities. The aim of the current study was to explore the feasibility of participant recruitment and the potential for data collection in the non-clinical population in a smartphone-based trial using ResearchKit. As a case example, an app called eMovit, a behavioural activation (BA) app with the aim of helping users to build healthy habits was used. The study was conducted over a 9-month period. Any iPhone user with access to the App Stores of The Netherlands, Belgium, and Germany could download the app and participate in the study. During the study period, the eMovit app was disseminated amongst potential users via social media posts (Twitter, Facebook, LinkedIn), paid social media advertisements (Facebook), digital newsletters and newspaper articles, blogposts and other websites. In total, 1,788 individuals visited the eMovit landing page. A total of 144 visitors subsequently entered Apple's App Store through that landing page. The eMovit product page was viewed 10,327 times on the App Store. With 79 installs, eMovit showed a conversion rate of 0.76% from product view to install of the app. Of those 79 installs, 53 users indicated that they were interested to participate in the research study and 36 subsequently consented and completed the demographics and the participants quiz. Fifteen participants completed the first PHQ-8 assessment and one participant completed the second PHQ-8 assessment. We conclude that from a technological point of view, the means provided by ResearchKit are well suited to be integrated into the app process and thus facilitate conducting smartphone-based studies. However, this study shows that although participant recruitment is technically straightforward, only low recruitment rates were achieved with the dissemination strategies applied. We argue that smartphone-based trials (using ResearchKit) require a well-designed app dissemination process to attain a sufficient sample size. Guidelines for smartphone-based trial designs and recommendations on how to work with challenges of mHealth research will ensure the quality of these trials, facilitate researchers to do more testing of mental health apps and with that enlarge the evidence-base for mHealth.

A systematic review of telehealth screening, assessment, and diagnosis of autism spectrum disorder

There is a significant delay between parents having concerns and receiving a formal assessment and Autism Spectrum Disorder (ASD) diagnosis. Telemedicine could be an effective alternative that shortens the waiting time for parents and primary health providers in ASD screening and diagnosis. We conducted a systematic review examining the uses of telemedicine technology for ASD screening, assessment, or diagnostic purposes and to what extent sample characteristics and psychometric properties were reported. This study searched four databases from 2000 to 2022 and obtained 26 studies that met the inclusion criteria. The 17 applications used in these 26 studies were divided into three categories based on their purpose: screening, diagnostic, and assessment. The results described the data extracted, including study characteristics, applied methods, indicators seen, and psychometric properties. Among the 15 applications with psychometric properties reported, the sensitivity ranged from 0.70 to 1, and the specificity ranged from 0.38 to 1. The present study highlights the strengths and weaknesses of current telemedicine approaches and provides a basis for future research. More rigorous empirical studies with larger sample sizes are needed to understand the feasibility, strengths, and limitations of telehealth technologies for screening, assessing, and diagnosing ASD.

A Versatile and Scalable Platform That Streamlines Data Collection for Patient-Centered Studies: Usability and Feasibility Study

Background

The Food and Drug Administration Center for Biologics Evaluation and Research (CBER) established the Biologics Effectiveness and Safety (BEST) Initiative with several objectives, including the expansion and enhancement of CBER’s access to fit-for-purpose data sources, analytics, tools, and infrastructures to improve the understanding of patient experiences with conditions related to CBER-regulated products. Owing to existing challenges in data collection, especially for rare disease research, CBER recognized the need for a comprehensive platform where study coordinators can engage with study participants and design and deploy studies while patients or caregivers could enroll, consent, and securely participate as well.

Objective

This study aimed to increase awareness and describe the design, development, and novelty of the Survey of Health and Patient Experience (SHAPE) platform, its functionality and application, quality improvement efforts, open-source availability, and plans for enhancement.

Methods

SHAPE is hosted in a Google Cloud environment and comprises 3 parts: the administrator application, participant app, and application programming interface. The administrator can build a study comprising a set of questionnaires and self-report entries through the app. Once the study is deployed, the participant can access the app, consent to the study, and complete its components. To build SHAPE to be scalable and flexible, we leveraged the open-source software development kit, Ionic Framework. This enabled the building and deploying of apps across platforms, including iOS, Android, and progressive web applications, from a single codebase by using standardized web technologies. SHAPE has been integrated with a leading Health Level 7 (HL7®) Fast Healthcare Interoperability Resources (FHIR®) application programming interface platform, 1upHealth, which allows participants to consent to 1-time data pull of their electronic health records. We used an agile-based process that engaged multiple stakeholders in SHAPE’s design and development.

Results

SHAPE allows study coordinators to plan, develop, and deploy questionnaires to obtain important end points directly from patients or caregivers. Electronic health record integration enables access to patient health records, which can validate and enhance the accuracy of data-capture methods. The administrator can then download the study data into HL7® FHIR®–formatted JSON files. In this paper, we illustrate how study coordinators can use SHAPE to design patient-centered studies. We demonstrate its broad applicability through a hypothetical type 1 diabetes cohort study and an ongoing pilot study on metachromatic leukodystrophy to implement best practices for designing a regulatory-grade natural history study for rare diseases.

Conclusions

SHAPE is an intuitive and comprehensive data-collection tool for a variety of clinical studies. Further customization of this versatile and scalable platform allows for multiple use cases. SHAPE can capture patient perspectives and clinical data, thereby providing regulators, clinicians, researchers, and patient advocacy organizations with data to inform drug development and improve patient outcomes.

Test–Retest Reliability in Automated Emotional Facial Expression Analysis: Exploring FaceReader 8.0 on Data from Typically Developing Children and Children with Autism

Automated emotional facial expression analysis (AEFEA) is used widely in applied research, including the development of screening/diagnostic systems for atypical human neurodevelopmental conditions. The validity of AEFEA systems has been systematically studied, but their test–retest reliability has not been researched thus far. We explored the test–retest reliability of a specific AEFEA software, Noldus FaceReader 8.0 (FR8; by Noldus Information Technology). We collected intensity estimates for 8 repeated emotions through FR8 from facial video recordings of 60 children: 31 typically developing children and 29 children with autism spectrum disorder. Test–retest reliability was imperfect in 20% of cases, affecting a substantial proportion of data points; however, the test–retest differences were small. This shows that the test–retest reliability of FR8 is high but not perfect. A proportion of cases which initially failed to show perfect test–retest reliability reached it in a subsequent analysis by FR8. This suggests that repeated analyses by FR8 can, in some cases, lead to the “stabilization” of emotion intensity datasets. Under ANOVA, the test–retest differences did not influence the pattern of cross-emotion and cross-group effects and interactions. Our study does not question the validity of previous results gained by AEFEA technology, but it shows that further exploration of the test–retest reliability of AEFEA systems is desirable.

Large multicenter randomized trials in autism: key insights gained from the balovaptan clinical development program

BACKGROUND

Autism spectrum disorder (ASD) is a common and heterogeneous neurodevelopmental condition that is characterized by the core symptoms of social communication difficulties and restricted and repetitive behaviors. At present, there is an unmet medical need for therapies to ameliorate these core symptoms in order to improve quality of life of autistic individuals. However, several challenges are currently faced by the ASD community relating to the development of pharmacotherapies, namely in the conduct of clinical trials. Balovaptan is a V1a receptor antagonist that has been investigated to improve social communication difficulties in individuals with ASD. In this viewpoint, we draw upon our recent first-hand experiences of the balovaptan clinical development program to describe current challenges of ASD trials.

DISCUSSION POINTS

The balovaptan trials were conducted in a wide age range of individuals with ASD with the added complexities associated with international trials. When summarizing all three randomized trials of balovaptan, a placebo response was observed across several outcome measures. Placebo response was predicted by greater baseline symptom severity, online recruitment of participants, and less experienced or non-academic trial sites. We also highlight challenges relating to selection of outcome measures in ASD, the impact of baseline characteristics, and the role of expectation bias in influencing trial results.

CONCLUSION

Taken together, the balovaptan clinical development program has advanced our understanding of the key challenges facing ASD treatment research. The insights gained can be used to inform and improve the design of future clinical trials with the collective aim of developing efficacious therapies to support individuals with ASD.

Attention Measurement of an Autism Spectrum Disorder User Using EEG Signals: A Case Study

Autism Spectrum Disorder (ASD) is a neurodevelopmental life condition characterized by problems with social interaction, low verbal and non-verbal communication skills, and repetitive and restricted behavior. People with ASD usually have variable attention levels because they have hypersensitivity and large amounts of environmental information are a problem for them. Attention is a process that occurs at the cognitive level and allows us to orient ourselves towards relevant stimuli, ignoring those that are not, and act accordingly. This paper presents a methodology based on electroencephalographic (EEG) signals for attention measurement in a 13-year-old boy diagnosed with ASD. The EEG signals are acquired with an Epoc+ Brain–Computer Interface (BCI) via the Emotiv Pro platform while developing several learning activities and using Matlab 2019a for signal processing. For this article, we propose to use electrodes F3, F4, P7, and P8. Then, we calculate the band power spectrum density to detect the Theta Relative Power (TRP), Alpha Relative Power (ARP), Beta Relative Power (BRP), Theta–Beta Ratio (TBR), Theta–Alpha Ratio (TAR), and Theta/(Alpha+Beta), which are features related to attention detection and neurofeedback. We train and evaluate several machine learning (ML) models with these features. In this study, the multi-layer perceptron neural network model (MLP-NN) has the best performance, with an AUC of 0.9299, Cohen’s Kappa coefficient of 0.8597, Matthews correlation coefficient of 0.8602, and Hamming loss of 0.0701. These findings make it possible to develop better learning scenarios according to the person’s needs with ASD. Moreover, it makes it possible to obtain quantifiable information on their progress to reinforce the perception of the teacher or therapist.

Identification of Social Engagement Indicators Associated With Autism Spectrum Disorder Using a Game-Based Mobile App: Comparative Study of Gaze Fixation and Visual Scanning Methods

BACKGROUND

Autism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Standard diagnostic mechanisms for ASD, which involve lengthy parent questionnaires and clinical observation, often result in long waiting times for results. Recent advances in computer vision and mobile technology hold potential for speeding up the diagnostic process by enabling computational analysis of behavioral and social impairments from home videos. Such techniques can improve objectivity and contribute quantitatively to the diagnostic process.

OBJECTIVE

In this work, we evaluate whether home videos collected from a game-based mobile app can be used to provide diagnostic insights into ASD. To the best of our knowledge, this is the first study attempting to identify potential social indicators of ASD from mobile phone videos without the use of eye-tracking hardware, manual annotations, and structured scenarios or clinical environments.

METHODS

Here, we used a mobile health app to collect over 11 hours of video footage depicting 95 children engaged in gameplay in a natural home environment. We used automated data set annotations to analyze two social indicators that have previously been shown to differ between children with ASD and their neurotypical (NT) peers: (1) gaze fixation patterns, which represent regions of an individual's visual focus and (2) visual scanning methods, which refer to the ways in which individuals scan their surrounding environment. We compared the gaze fixation and visual scanning methods used by children during a 90-second gameplay video to identify statistically significant differences between the 2 cohorts; we then trained a long short-term memory (LSTM) neural network to determine if gaze indicators could be predictive of ASD.

RESULTS

Our results show that gaze fixation patterns differ between the 2 cohorts; specifically, we could identify 1 statistically significant region of fixation (P<.001). In addition, we also demonstrate that there are unique visual scanning patterns that exist for individuals with ASD when compared to NT children (P<.001). A deep learning model trained on coarse gaze fixation annotations demonstrates mild predictive power in identifying ASD.

CONCLUSIONS

Ultimately, our study demonstrates that heterogeneous video data sets collected from mobile devices hold potential for quantifying visual patterns and providing insights into ASD. We show the importance of automated labeling techniques in generating large-scale data sets while simultaneously preserving the privacy of participants, and we demonstrate that specific social engagement indicators associated with ASD can be identified and characterized using such data.

Digital Phenotyping in Clinical Neurology

Internet-connected devices, including personal computers, smartphones, smartwatches, and voice assistants, have evolved into powerful multisensor technologies that billions of people interact with daily to connect with friends and colleagues, access and share information, purchase goods, play games, and navigate their environment. Digital phenotyping taps into the data streams captured by these devices to characterize and understand health and disease. The purpose of this article is to summarize opportunities for digital phenotyping in neurology, review studies using everyday technologies to obtain motor and cognitive information, and provide a perspective on how neurologists can embrace and accelerate progress in this emerging field.

Machine learning algorithms as new screening approach for patients with endometriosis

Endometriosis-a systemic and chronic condition occurring in women of childbearing age-is a highly enigmatic disease with unresolved questions. While multiple biomarkers, genomic analysis, questionnaires, and imaging techniques have been advocated as screening and triage tests for endometriosis to replace diagnostic laparoscopy, none have been implemented routinely in clinical practice. We investigated the use of machine learning algorithms (MLA) in the diagnosis and screening of endometriosis based on 16 key clinical and patient-based symptom features. The sensitivity, specificity, F1-score and AUCs of the MLA to diagnose endometriosis in the training and validation sets varied from 0.82 to 1, 0-0.8, 0-0.88, 0.5-0.89, and from 0.91 to 0.95, 0.66-0.92, 0.77-0.92, respectively. Our data suggest that MLA could be a promising screening test for general practitioners, gynecologists, and other front-line health care providers. Introducing MLA in this setting represents a paradigm change in clinical practice as it could replace diagnostic laparoscopy. Furthermore, this patient-based screening tool empowers patients with endometriosis to self-identify potential symptoms and initiate dialogue with physicians about diagnosis and treatment, and hence contribute to shared decision making.

A scalable computational approach to assessing response to name in toddlers with autism

BACKGROUND

This study is part of a larger research program focused on developing objective, scalable tools for digital behavioral phenotyping. We evaluated whether a digital app delivered on a smartphone or tablet using computer vision analysis (CVA) can elicit and accurately measure one of the most common early autism symptoms, namely failure to respond to a name call.

METHODS

During a pediatric primary care well-child visit, 910 toddlers, 17-37 months old, were administered an app on an iPhone or iPad consisting of brief movies during which the child's name was called three times by an examiner standing behind them. Thirty-seven toddlers were subsequently diagnosed with autism spectrum disorder (ASD). Name calls and children's behavior were recorded by the camera embedded in the device, and children's head turns were coded by both CVA and a human.

RESULTS

CVA coding of response to name was found to be comparable to human coding. Based on CVA, children with ASD responded to their name significantly less frequently than children without ASD. CVA also revealed that children with ASD who did orient to their name exhibited a longer latency before turning their head. Combining information about both the frequency and the delay in response to name improved the ability to distinguish toddlers with and without ASD.

CONCLUSIONS

A digital app delivered on an iPhone or iPad in real-world settings using computer vision analysis to quantify behavior can reliably detect a key early autism symptom-failure to respond to name. Moreover, the higher resolution offered by CVA identified a delay in head turn in toddlers with ASD who did respond to their name. Digital phenotyping is a promising methodology for early assessment of ASD symptoms.

Computational Methods to Measure Patterns of Gaze in Toddlers With Autism Spectrum Disorder

Importance

Atypical eye gaze is an early-emerging symptom of autism spectrum disorder (ASD) and holds promise for autism screening. Current eye-tracking methods are expensive and require special equipment and calibration. There is a need for scalable, feasible methods for measuring eye gaze.

Objective

Using computational methods based on computer vision analysis, we evaluated whether an app deployed on an iPhone or iPad that displayed strategically designed brief movies could elicit and quantify differences in eye-gaze patterns of toddlers with ASD vs typical development.

Design, Setting, and Participants

A prospective study in pediatric primary care clinics was conducted from December 2018 to March 2020, comparing toddlers with and without ASD. Caregivers of 1564 toddlers were invited to participate during a well-child visit. A total of 993 toddlers (63%) completed study measures. Enrollment criteria were aged 16 to 38 months, healthy, English- or Spanish-speaking caregiver, and toddler able to sit and view the app. Participants were screened with the Modified Checklist for Autism in Toddlers-Revised With Follow-up during routine care. Children were referred by their pediatrician for diagnostic evaluation based on results of the checklist or if the caregiver or pediatrician was concerned. Forty toddlers subsequently were diagnosed with ASD.

Exposures

A mobile app displayed on a smartphone or tablet.

Main Outcomes and Measures

Computer vision analysis quantified eye-gaze patterns elicited by the app, which were compared between toddlers with ASD vs typical development.

Results

Mean age of the sample was 21.1 months (range, 17.1-36.9 months), and 50.6% were boys, 59.8% White individuals, 16.5% Black individuals, 23.7% other race, and 16.9% Hispanic/Latino individuals. Distinctive eye-gaze patterns were detected in toddlers with ASD, characterized by reduced gaze to social stimuli and to salient social moments during the movies, and previously unknown deficits in coordination of gaze with speech sounds. The area under the receiver operating characteristic curve discriminating ASD vs non-ASD using multiple gaze features was 0.90 (95% CI, 0.82-0.97).

Conclusions and Relevance

The app reliably measured both known and new gaze biomarkers that distinguished toddlers with ASD vs typical development. These novel results may have potential for developing scalable autism screening tools, exportable to natural settings, and enabling data sets amenable to machine learning.

A Scalable Off-the-Shelf Framework for Measuring Patterns of Attention in Young Children and its Application in Autism Spectrum Disorder

Autism spectrum disorder (ASD) is associated with deficits in the processing of social information and difficulties in social interaction, and individuals with ASD exhibit atypical attention and gaze. Traditionally, gaze studies have relied upon precise and constrained means of monitoring attention using expensive equipment in laboratories. In this work we develop a low-cost off-the-shelf alternative for measuring attention that can be used in natural settings. The head and iris positions of 104 16-31 months children, an age range appropriate for ASD screening and diagnosis, 22 of them diagnosed with ASD, were recorded using the front facing camera in an iPad while they watched on the device screen a movie displaying dynamic stimuli, social stimuli on the left and nonsocial stimuli on the right. The head and iris position were then automatically analyzed via computer vision algorithms to detect the direction of attention. Children in the ASD group paid less attention to the movie, showed less attention to the social as compared to the nonsocial stimuli, and often fixated their attention to one side of the screen. The proposed method provides a low-cost means of monitoring attention to properly designed stimuli, demonstrating that the integration of stimuli design and automatic response analysis results in the opportunity to use off-the-shelf cameras to assess behavioral biomarkers.

The Digital Divide in Technologies for Autism: Feasibility Considerations for Low- and Middle-Income Countries

Digital technologies have the potential to empower individuals with autism and their families. The COVID-19 pandemic emphasized and accelerated the drive towards technology for information, communication, training, clinical care and research, also in the autism community. However, 95% of individuals with autism live in low- and middle-income countries (LMIC) where access to electricity, internet and the ever-increasing range of digital devices may be highly limited. The World Bank coined the term ‘the digital divide’ to describe the disparities in access to digital technologies between high-income and LMIC contexts. Here we evaluated the feasibility of six emerging technologies for autism spectrum disorders, and reflected on key considerations for implementation in LMIC contexts to ensure that we do not inadvertently widen the pre-existing digital divide.

Is Smiling the Key? Machine Learning Analytics Detect Subtle Patterns in Micro-Expressions of Infants with ASD

Time is a key factor to consider in Autism Spectrum Disorder. Detecting the condition as early as possible is crucial in terms of treatment success. Despite advances in the literature, it is still difficult to identify early markers able to effectively forecast the manifestation of symptoms. Artificial intelligence (AI) provides effective alternatives for behavior screening. To this end, we investigated facial expressions in 18 autistic and 15 typical infants during their first ecological interactions, between 6 and 12 months of age. We employed Openface, an AI-based software designed to systematically analyze facial micro-movements in images in order to extract the subtle dynamics of Social Smiles in unconstrained Home Videos. Reduced frequency and activation intensity of Social Smiles was computed for children with autism. Machine Learning models enabled us to map facial behavior consistently, exposing early differences hardly detectable by non-expert naked eye. This outcome contributes to enhancing the potential of AI as a supportive tool for the clinical framework.

Crowdsourced privacy-preserved feature tagging of short home videos for machine learning ASD detection

Standard medical diagnosis of mental health conditions requires licensed experts who are increasingly outnumbered by those at risk, limiting reach. We test the hypothesis that a trustworthy crowd of non-experts can efficiently annotate behavioral features needed for accurate machine learning detection of the common childhood developmental disorder Autism Spectrum Disorder (ASD) for children under 8 years old. We implement a novel process for identifying and certifying a trustworthy distributed workforce for video feature extraction, selecting a workforce of 102 workers from a pool of 1,107. Two previously validated ASD logistic regression classifiers, evaluated against parent-reported diagnoses, were used to assess the accuracy of the trusted crowd's ratings of unstructured home videos. A representative balanced sample (N = 50 videos) of videos were evaluated with and without face box and pitch shift privacy alterations, with AUROC and AUPRC scores > 0.98. With both privacy-preserving modifications, sensitivity is preserved (96.0%) while maintaining specificity (80.0%) and accuracy (88.0%) at levels comparable to prior classification methods without alterations. We find that machine learning classification from features extracted by a certified nonexpert crowd achieves high performance for ASD detection from natural home videos of the child at risk and maintains high sensitivity when privacy-preserving mechanisms are applied. These results suggest that privacy-safeguarded crowdsourced analysis of short home videos can help enable rapid and mobile machine-learning detection of developmental delays in children.

Digital Behavioral Phenotyping Detects Atypical Pattern of Facial Expression in Toddlers with Autism

Commonly used screening tools for autism spectrum disorder (ASD) generally rely on subjective caregiver questionnaires. While behavioral observation is more objective, it is also expensive, time-consuming, and requires significant expertise to perform. As such, there remains a critical need to develop feasible, scalable, and reliable tools that can characterize ASD risk behaviors. This study assessed the utility of a tablet-based behavioral assessment for eliciting and detecting one type of risk behavior, namely, patterns of facial expression, in 104 toddlers (ASD N = 22) and evaluated whether such patterns differentiated toddlers with and without ASD. The assessment consisted of the child sitting on his/her caregiver's lap and watching brief movies shown on a smart tablet while the embedded camera recorded the child's facial expressions. Computer vision analysis (CVA) automatically detected and tracked facial landmarks, which were used to estimate head position and facial expressions (Positive, Neutral, All Other). Using CVA, specific points throughout the movies were identified that reliably differentiate between children with and without ASD based on their patterns of facial movement and expressions (area under the curves for individual movies ranging from 0.62 to 0.73). During these instances, children with ASD more frequently displayed Neutral expressions compared to children without ASD, who had more All Other expressions. The frequency of All Other expressions was driven by non-ASD children more often displaying raised eyebrows and an open mouth, characteristic of engagement/interest. Preliminary results suggest computational coding of facial movements and expressions via a tablet-based assessment can detect differences in affective expression, one of the early, core features of ASD. LAY SUMMARY: This study tested the use of a tablet in the behavioral assessment of young children with autism. Children watched a series of developmentally appropriate movies and their facial expressions were recorded using the camera embedded in the tablet. Results suggest that computational assessments of facial expressions may be useful in early detection of symptoms of autism.

Information and Communication Technologies to Support Early Screening of Autism Spectrum Disorder: A Systematic Review

&nbsp;The aim of this systematic review is to identify recent digital technologies used to detect early signs of autism spectrum disorder (ASD) in preschool children (i.e., up to six years of age). A systematic literature search was performed for English language articles and conference papers indexed in Pubmed, PsycInfo, ERIC, CINAHL, WoS, IEEE, and ACM digital libraries up until January 2020. A follow-up search was conducted to cover the literature published until December 2020 for the usefulness and interest in this area of research during the Covid-19 emergency. In total, 2427 articles were initially retrieved from databases search. Additional 481 articles were retrieved from follow-up search. Finally, 28 articles met the inclusion criteria and were included in the review. The studies included involved four main interface modalities: Natural User Interface (e.g., eye trackers), PC or mobile, Wearable, and Robotics. Most of the papers included (n = 20) involved the use of Level 1 screening tools. Notwithstanding the variability of the solutions identified, psychometric information points to considering available technologies as promising supports in clinical practice to detect early sign of ASD in young children. Further research is needed to understand the acceptability and increase use rates of technology-based screenings in clinical settings.&nbsp.

Computer Vision Analysis for Quantification of Autism Risk Behaviors

Observational behavior analysis plays a key role for the discovery and evaluation of risk markers for many neurodevelopmental disorders. Research on autism spectrum disorder (ASD) suggests that behavioral risk markers can be observed at 12 months of age or earlier, with diagnosis possible at 18 months. To date, these studies and evaluations involving observational analysis tend to rely heavily on clinical practitioners and specialists who have undergone intensive training to be able to reliably administer carefully designed behavioural-eliciting tasks, code the resulting behaviors, and interpret such behaviors. These methods are therefore extremely expensive, time-intensive, and are not easily scalable for large population or longitudinal observational analysis. We developed a self-contained, closed-loop, mobile application with movie stimuli designed to engage the child's attention and elicit specific behavioral and social responses, which are recorded with a mobile device camera and then analyzed via computer vision algorithms. Here, in addition to presenting this paradigm, we validate the system to measure engagement, name-call responses, and emotional responses of toddlers with and without ASD who were presented with the application. Additionally, we show examples of how the proposed framework can further risk marker research with fine-grained quantification of behaviors. The results suggest these objective and automatic methods can be considered to aid behavioral analysis, and can be suited for objective automatic analysis for future studies.

The FDA MyStudies app: a reusable platform for distributed clinical trials and real-world evidence studies

We developed a mobile application and secure patient data storage platform, FDA MyStudies, to address privacy, engagement, and extensibility challenges in mobile clinical research. The system extends the capabilities of the mobile frameworks Apple ResearchKit and ResearchStack through an intuitive front-end application and secure storage environment that can support health research studies. The platform supports single or multisite studies via role-based access and can be implemented within highly secure data environments. As a proof-of-concept, pregnant women participated in a descriptive study via the app in which data not routinely captured in electronic health records (EHR) were collected and linked with existing patient data to provide a more wholistic view of the patient and illustrate how patient data combined with EHR data could be used to support public health research.

Present and Future of Digital Health in Diabetes and Metabolic Disease

The use of information and communication technology (ICT) in medical and healthcare services goes beyond everyday life. Expectations of a new medical environment, not previously experienced by ICT, exist in the near future. In particular, chronic metabolic diseases such as diabetes and obesity, have a high prevalence and high social and economic burden. In addition, the continuous evaluation and monitoring of daily life is important for effective treatment and management. Therefore, the wide use of ICTbased digital health systems is required for the treatment and management of these diseases. In this article, we compiled a variety of digital health technologies introduced to date in the field of diabetes and metabolic diseases.

Accurate detection of cerebellar smooth pursuit eye movement abnormalities via mobile phone video and machine learning

Eye movements are disrupted in many neurodegenerative diseases and are frequent and early features in conditions affecting the cerebellum. Characterizing eye movements is important for diagnosis and may be useful for tracking disease progression and response to therapies. Assessments are limited as they require an in-person evaluation by a neurology subspecialist or specialized and expensive equipment. We tested the hypothesis that important eye movement abnormalities in cerebellar disorders (i.e., ataxias) could be captured from iPhone video. Videos of the face were collected from individuals with ataxia (n = 102) and from a comparative population (Parkinson's disease or healthy participants, n = 61). Computer vision algorithms were used to track the position of the eye which was transformed into high temporal resolution spectral features. Machine learning models trained on eye movement features were able to identify abnormalities in smooth pursuit (a key eye behavior) and accurately distinguish individuals with abnormal pursuit from controls (sensitivity = 0.84, specificity = 0.77). A novel machine learning approach generated severity estimates that correlated well with the clinician scores. We demonstrate the feasibility of capturing eye movement information using an inexpensive and widely accessible technology. This may be a useful approach for disease screening and for measuring severity in clinical trials.

Using Medical Big Data to Develop Personalized Medicine for Dry Eye Disease

Dry eye disease (DED) is a chronic, multifactorial ocular surface disorder with multiple etiologies that results in tear film instability. Globally, the prevalence of DED is expected to increase with an aging society and daily use of digital devices. Unfortunately, the medical field is currently unprepared to meet the medical needs of patients with DED. Noninvasive, reliable, and readily reproducible biomarkers have not yet been identified, and the current mainstay treatment for DED relies on symptom alleviation using eye drops with no effective preventative therapies available. Medical big data analyses, mining information from multiomics studies and mobile health applications, may offer a solution for managing chronic conditions such as DED. Omics-based data on individual physiologic status may be leveraged to prevent high-risk diseases, accurately diagnose illness, and improve patient prognosis. Mobile health applications enable the portable collection of real-world medical data and biosignals through personal devices. Together, these data lay a robust foundation for personalized treatments for various ocular surface diseases and other pathologies that currently lack the components of precision medicine. To fully implement personalized and precision medicine, traditional aggregate medical data should not be applied directly to individuals without adjustments for personal etiology, phenotype, presentation, and symptoms.

New medical big data for P4 medicine on allergic conjunctivitis

Allergic conjunctivitis affects approximately 15-20% of the global population and can permanently deteriorate one's quality of life (QoL) and work productivity, leading to societal work force costs. Although not fully understood, allergic conjunctivitis is a multifactorial disease with a complex network of environmental, lifestyle, and host contributory risk factors. To effectively enhance the quality of treatment for patients with allergic conjunctivitis, as well as other allergic diseases, the field must first comprehend the pathology underlying various individualized subjective symptoms and stratify the disease according to risk factors and presentations. Such competent stratification and societal reconstruction that targets the alleviation of the damage due to allergic diseases would greatly help ramify personalized treatments and prevent the projected increase in societal costs imposed by allergic diseases. Owing to the rapid advancements in the information and technology sector, medical big data are greatly accessible and useful to decipher the pathophysiology of many diseases. Such data collected through multi-omics and mobile health have been effective for research on chronic diseases including allergic and immune-mediated diseases. Novel big data containing vast and continuous information on individuals with allergic conjunctivitis and other allergic symptoms are being used to search for causative genes of diseases, gain insights into new biomarkers, prevent disease progression, and, ultimately, improve QoL. The individualized and holistic data accrued from new angles using technological innovations are helping the field realize the principles of P4 medicine: predictive, preventive, personalized, and participatory medicine.

Computer vision in autism spectrum disorder research: a systematic review of published studies from 2009 to 2019

The current state of computer vision methods applied to autism spectrum disorder (ASD) research has not been well established. Increasing evidence suggests that computer vision techniques have a strong impact on autism research. The primary objective of this systematic review is to examine how computer vision analysis has been useful in ASD diagnosis, therapy and autism research in general. A systematic review of publications indexed on PubMed, IEEE Xplore and ACM Digital Library was conducted from 2009 to 2019. Search terms included ['autis*' AND ('computer vision' OR 'behavio* imaging' OR 'behavio* analysis' OR 'affective computing')]. Results are reported according to PRISMA statement. A total of 94 studies are included in the analysis. Eligible papers are categorised based on the potential biological/behavioural markers quantified in each study. Then, different computer vision approaches that were employed in the included papers are described. Different publicly available datasets are also reviewed in order to rapidly familiarise researchers with datasets applicable to their field and to accelerate both new behavioural and technological work on autism research. Finally, future research directions are outlined. The findings in this review suggest that computer vision analysis is useful for the quantification of behavioural/biological markers which can further lead to a more objective analysis in autism research.

Precision Telemedicine through Crowdsourced Machine Learning: Testing Variability of Crowd Workers for Video-Based Autism Feature Recognition

Mobilized telemedicine is becoming a key, and even necessary, facet of both precision health and precision medicine. In this study, we evaluate the capability and potential of a crowd of virtual workers-defined as vetted members of popular crowdsourcing platforms-to aid in the task of diagnosing autism. We evaluate workers when crowdsourcing the task of providing categorical ordinal behavioral ratings to unstructured public YouTube videos of children with autism and neurotypical controls. To evaluate emerging patterns that are consistent across independent crowds, we target workers from distinct geographic loci on two crowdsourcing platforms: an international group of workers on Amazon Mechanical Turk (MTurk) (N = 15) and Microworkers from Bangladesh (N = 56), Kenya (N = 23), and the Philippines (N = 25). We feed worker responses as input to a validated diagnostic machine learning classifier trained on clinician-filled electronic health records. We find that regardless of crowd platform or targeted country, workers vary in the average confidence of the correct diagnosis predicted by the classifier. The best worker responses produce a mean probability of the correct class above 80% and over one standard deviation above 50%, accuracy and variability on par with experts according to prior studies. There is a weak correlation between mean time spent on task and mean performance (r = 0.358, p = 0.005). These results demonstrate that while the crowd can produce accurate diagnoses, there are intrinsic differences in crowdworker ability to rate behavioral features. We propose a novel strategy for recruitment of crowdsourced workers to ensure high quality diagnostic evaluations of autism, and potentially many other pediatric behavioral health conditions. Our approach represents a viable step in the direction of crowd-based approaches for more scalable and affordable precision medicine.

A Six-Minute Measure of Vocalizations in Toddlers with Autism Spectrum Disorder

To improve early identification of autism spectrum disorder (ASD), we need objective, reliable, and accessible measures. To that end, a previous study demonstrated that a tablet-based application (app) that assessed several autism risk behaviors distinguished between toddlers with ASD and non-ASD toddlers. Using vocal data collected during this study, we investigated whether vocalizations uttered during administration of this app can distinguish among toddlers aged 16-31 months with typical development (TD), language or developmental delay (DLD), and ASD. Participant's visual and vocal responses were recorded using the camera and microphone in a tablet while toddlers watched movies designed to elicit behaviors associated with risk for ASD. Vocalizations were then coded offline. Results showed that (a) children with ASD and DLD were less likely to produce words during app administration than TD participants; (b) the ratio of syllabic vocalizations to all vocalizations was higher among TD than ASD or DLD participants; and (c) the rates of nonsyllabic vocalizations were higher in the ASD group than in either the TD or DLD groups. Those producing more nonsyllabic vocalizations were 24 times more likely to be diagnosed with ASD. These results lend support to previous findings that early vocalizations might be useful in identifying risk for ASD in toddlers and demonstrate the feasibility of using a scalable tablet-based app for assessing vocalizations in the context of a routine pediatric visit. LAY SUMMARY: Although parents often report symptoms of autism spectrum disorder (ASD) in infancy, we are not yet reliably diagnosing ASD until much later in development. A previous study tested a tablet-based application (app) that recorded behaviors we know are associated with ASD to help identify children at risk for the disorder. Here we measured how children vocalize while they watched the movies presented on the tablet. Children with ASD were less likely to produce words, less likely to produce speechlike sounds, and more likely to produce atypical sounds while watching these movies. These measures, combined with other behaviors measured by the app, might help identify which children should be evaluated for ASD. Autism Res 2020, 13: 1373-1382. © 2020 International Society for Autism Research, Wiley Periodicals, Inc.

Data-Driven Diagnostics and the Potential of Mobile Artificial Intelligence for Digital Therapeutic Phenotyping in Computational Psychiatry

Data science and digital technologies have the potential to transform diagnostic classification. Digital technologies enable the collection of big data, and advances in machine learning and artificial intelligence enable scalable, rapid, and automated classification of medical conditions. In this review, we summarize and categorize various data-driven methods for diagnostic classification. In particular, we focus on autism as an example of a challenging disorder due to its highly heterogeneous nature. We begin by describing the frontier of data science methods for the neuropsychiatry of autism. We discuss early signs of autism as defined by existing pen-and-paper-based diagnostic instruments and describe data-driven feature selection techniques for determining the behaviors that are most salient for distinguishing children with autism from neurologically typical children. We then describe data-driven detection techniques, particularly computer vision and eye tracking, that provide a means of quantifying behavioral differences between cases and controls. We also describe methods of preserving the privacy of collected videos and prior efforts of incorporating humans in the diagnostic loop. Finally, we summarize existing digital therapeutic interventions that allow for data capture and longitudinal outcome tracking as the diagnosis moves along a positive trajectory. Digital phenotyping of autism is paving the way for quantitative psychiatry more broadly and will set the stage for more scalable, accessible, and precise diagnostic techniques in the field.

Learning endometriosis phenotypes from patient-generated data

Endometriosis is a systemic and chronic condition in women of childbearing age, yet a highly enigmatic disease with unresolved questions: there are no known biomarkers, nor established clinical stages. We here investigate the use of patient-generated health data and data-driven phenotyping to characterize endometriosis patient subtypes, based on their reported signs and symptoms. We aim at unsupervised learning of endometriosis phenotypes using self-tracking data from personal smartphones. We leverage data from an observational research study of over 4000 women with endometriosis that track their condition over more than 2 years. We extend a classical mixed-membership model to accommodate the idiosyncrasies of the data at hand, i.e., the multimodality and uncertainty of the self-tracked variables. The proposed method, by jointly modeling a wide range of observations (i.e., participant symptoms, quality of life, treatments), identifies clinically relevant endometriosis subtypes. Experiments show that our method is robust to different hyperparameter choices and the biases of self-tracking data (e.g., the wide variations in tracking frequency among participants). With this work, we show the promise of unsupervised learning of endometriosis subtypes from self-tracked data, as learned phenotypes align well with what is already known about the disease, but also suggest new clinically actionable findings. More generally, we argue that a continued research effort on unsupervised phenotyping methods with patient-generated health data via new mobile and digital technologies will have significant impact on the study of enigmatic diseases in particular, and health in general.

Characterizing physiological and symptomatic variation in menstrual cycles using self-tracked mobile-health data

The menstrual cycle is a key indicator of overall health for women of reproductive age. Previously, menstruation was primarily studied through survey results; however, as menstrual tracking mobile apps become more widely adopted, they provide an increasingly large, content-rich source of menstrual health experiences and behaviors over time. By exploring a database of user-tracked observations from the Clue app by BioWink GmbH of over 378,000 users and 4.9 million natural cycles, we show that self-reported menstrual tracker data can reveal statistically significant relationships between per-person cycle length variability and self-reported qualitative symptoms. A concern for self-tracked data is that they reflect not only physiological behaviors, but also the engagement dynamics of app users. To mitigate such potential artifacts, we develop a procedure to exclude cycles lacking user engagement, thereby allowing us to better distinguish true menstrual patterns from tracking anomalies. We uncover that women located at different ends of the menstrual variability spectrum, based on the consistency of their cycle length statistics, exhibit statistically significant differences in their cycle characteristics and symptom tracking patterns. We also find that cycle and period length statistics are stationary over the app usage timeline across the variability spectrum. The symptoms that we identify as showing statistically significant association with timing data can be useful to clinicians and users for predicting cycle variability from symptoms, or as potential health indicators for conditions like endometriosis. Our findings showcase the potential of longitudinal, high-resolution self-tracked data to improve understanding of menstruation and women's health as a whole.

Automated recognition of spontaneous facial expression in individuals with autism spectrum disorder: parsing response variability

BACKGROUND

Reduction or differences in facial expression are a core diagnostic feature of autism spectrum disorder (ASD), yet evidence regarding the extent of this discrepancy is limited and inconsistent. Use of automated facial expression detection technology enables accurate and efficient tracking of facial expressions that has potential to identify individual response differences.

METHODS

Children and adults with ASD (N = 124) and typically developing (TD, N = 41) were shown short clips of "funny videos." Using automated facial analysis software, we investigated differences between ASD and TD groups and within the ASD group in evidence of facial action unit (AU) activation related to the expression of positive facial expression, in particular, a smile.

RESULTS

Individuals with ASD on average showed less evidence of facial AUs (AU12, AU6) relating to positive facial expression, compared to the TD group (p < .05, r = - 0.17). Using Gaussian mixture model for clustering, we identified two distinct distributions within the ASD group, which were then compared to the TD group. One subgroup (n = 35), termed "over-responsive," expressed more intense positive facial expressions in response to the videos than the TD group (p < .001, r = 0.31). The second subgroup (n = 89), ("under-responsive"), displayed fewer, less intense positive facial expressions in response to videos than the TD group (p < .001; r = - 0.36). The over-responsive subgroup differed from the under-responsive subgroup in age and caregiver-reported impulsivity (p < .05, r = 0.21). Reduced expression in the under-responsive, but not the over-responsive group, was related to caregiver-reported social withdrawal (p < .01, r = - 0.3).

LIMITATIONS

This exploratory study does not account for multiple comparisons, and future work will have to ascertain the strength and reproducibility of all results. Reduced displays of positive facial expressions do not mean individuals with ASD do not experience positive emotions.

CONCLUSIONS

Individuals with ASD differed from the TD group in their facial expressions of positive emotion in response to "funny videos." Identification of subgroups based on response may help in parsing heterogeneity in ASD and enable targeting of treatment based on subtypes.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02299700. Registration date: November 24, 2014.

Towards the automatic detection of social biomarkers in autism spectrum disorder: introducing the simulated interaction task (SIT)

Social interaction deficits are evident in many psychiatric conditions and specifically in autism spectrum disorder (ASD), but hard to assess objectively. We present a digital tool to automatically quantify biomarkers of social interaction deficits: the simulated interaction task (SIT), which entails a standardized 7-min simulated dialog via video and the automated analysis of facial expressions, gaze behavior, and voice characteristics. In a study with 37 adults with ASD without intellectual disability and 43 healthy controls, we show the potential of the tool as a diagnostic instrument and for better description of ASD-associated social phenotypes. Using machine-learning tools, we detected individuals with ASD with an accuracy of 73%, sensitivity of 67%, and specificity of 79%, based on their facial expressions and vocal characteristics alone. Especially reduced social smiling and facial mimicry as well as a higher voice fundamental frequency and harmony-to-noise-ratio were characteristic for individuals with ASD. The time-effective and cost-effective computer-based analysis outperformed a majority vote and performed equal to clinical expert ratings.

Computational Analysis of Deep Visual Data for Quantifying Facial Expression Production

The computational analysis of facial expressions is an emerging research topic that could overcome the limitations of human perception and get quick and objective outcomes in the assessment of neurodevelopmental disorders (e.g., Autism Spectrum Disorders, ASD). Unfortunately, there have been only a few attempts to quantify facial expression production and most of the scientific literature aims at the easier task of recognizing if either a facial expression is present or not. Some attempts to face this challenging task exist but they do not provide a comprehensive study based on the comparison between human and automatic outcomes in quantifying children’s ability to produce basic emotions. Furthermore, these works do not exploit the latest solutions in computer vision and machine learning. Finally, they generally focus only on a homogeneous (in terms of cognitive capabilities) group of individuals. To fill this gap, in this paper some advanced computer vision and machine learning strategies are integrated into a framework aimed to computationally analyze how both ASD and typically developing children produce facial expressions. The framework locates and tracks a number of landmarks (virtual electromyography sensors) with the aim of monitoring facial muscle movements involved in facial expression production. The output of these virtual sensors is then fused to model the individual ability to produce facial expressions. Gathered computational outcomes have been correlated with the evaluation provided by psychologists and evidence has been given that shows how the proposed framework could be effectively exploited to deeply analyze the emotional competence of ASD children to produce facial expressions.

Assessment of knowledge about childhood autism spectrum disorder among healthcare workers in Makkah- Saudi Arabia

OBJECTIVE

To measure the knowledge of healthcare professionals about increasingly prevalent Autism Spectrum Disorder (ASD) along with perceptions around its management and prognosis and comparison across various specialties.

METHODS

This Cross sectional survey based comparative analysis took place at Maternity and Children Hospital and King Faisal Hospital Makkah from December 2017 to May 2018. The validated self-administered "Knowledge about childhood autism among health workers" questionnaire was used along with additional questions regarding perceptions about ASD. The mean and mean percent scores were calculated. Chi squared test and ANOVA were applied to find the association between quantitative and qualitative variables respectively.

RESULTS

Out of 162 participants, 153 returned the questionnaire and 147 were included in final analysis. Physicians constituted 81.6% (120) of participants. The mean score for participants was 9.80(S.E.M ±0.32) where non-physicians yielded higher mean score (11.2±4.41) as compared to physicians (9.6±3.28) (p=0.113). Psychiatrists had highest score of 16/19 while general physicians had lowest (6/19). Participants with more years of experience had higher mean scores (p-value = 0.01). About 72.10% (106) of participants opted for medication as a treatment option. Nearly 38.1% (56) of participants were skeptical about improvement of ASD with early interventions.

CONCLUSION

There is a lack of knowledge about ASD amongst healthcare professionals in Saudi Arabia. Experienced professionals working with ASD children can be utilized to deliver targeted trainings nationwide.

Time to be "smart"-Opportunities Arising From Smartphone-Based Behavioral Analysis in Daily Patient Care

While pathologies of the central nervous system (CNS) are often associated with neuropsychological deficits, adequate quantification and monitoring of such deficits remains challenging. Due to their complex nature, comprehensive neuropsychological evaluations are needed, which are time-consuming, resource-intensive and do not adequately account for daily or hourly fluctuations of a patient's condition. Innovative approaches are required to improve the diagnostics and continuous monitoring of brain function, ideally in the form of a simple, objective, time-saving and inexpensive tool that overcomes the aforementioned weaknesses of conventional assessments. As smartphones are widely used and integrated in virtually every aspect of our lives, their potential regarding the acquisition of data representing an individual's behavior and health is enormous. Alterations in a patient's physical or mental health state may be recognized as behavioral deviation from the physiological range of the normal population, but also in comparison to the patient's individual baseline assessment. As smartphone-based assessment allows for continuous monitoring and therefore accounts for possible fluctuations or transiently occurring abnormalities in a patient's neurologic state, it may serve as a surveillance tool in the acute setting for early recognition of complications, or in the long-term outpatient setting to quantify rehabilitation or disease progress. This may be particularly interesting for regions of the world where healthcare resources for comprehensive clinical/neuropsychological examinations are insufficient or distances to healthcare providers are long. Here, we highlight the potential of smartphone-based behavioral monitoring in healthcare. Clinical Trial Registration: www.clinicaltrials.gov, identifier NCT03516162.