Mothers' use of touch across infants' development and its implications for word learning: Evidence from Korean dyadic interactions

Caregivers' touches that occur alongside words and utterances could aid in the detection of word/utterance boundaries and the mapping of word forms to word meanings. We examined changes in caregivers' use of touches with their speech directed to infants using a multimodal cross-sectional corpus of 35 Korean mother-child dyads across three age groups of infants (8, 14, and 27 months). We tested the hypothesis that caregivers' frequency and use of touches with speech change with infants' development. Results revealed that the frequency of word/utterance-touch alignment as well as word + touch co-occurrence is highest in speech addressed to the youngest group of infants. Thus, this study provides support for the hypothesis that caregivers' use of touch during dyadic interactions is sensitive to infants' age in a way similar to caregivers' use of speech alone and could provide cues useful to infants' language learning at critical points in early development.

Survey on natural language processing in medical image analysis

Recent advancement in natural language processing (NLP) and medical imaging empowers the wide applicability of deep learning models. These developments have increased not only data understanding, but also knowledge of state-of-the-art architectures and their real-world potentials. Medical imaging researchers have recognized the limitations of only targeting images, as well as the importance of integrating multimodal inputs into medical image analysis. The lack of comprehensive surveys of the current literature, however, impedes the progress of this domain. Existing research perspectives, as well as the architectures, tasks, datasets, and performance measures examined in the present literature, are reviewed in this work, and we also provide a brief description of possible future directions in the field, aiming to provide researchers and healthcare professionals with a detailed summary of existing academic research and to provide rational insights to facilitate future research.

English Phrase Learning With Multimodal Input

Although multimodal input has the potential to lead to more sound learning outcomes, it carries the risk of causing cognitive overload, making it difficult to determine the exact effects of multimodal input on the second language (L2) phrase learning. This study tests the efficacy of multimodal input on L2 phrase learning. It adopts a mixed-method approach by utilizing both quantitative and qualitative data. The experimental design is a 2 × 3 mixed model, with a group [the experimental group (EG) and the control group (CG)] as the between-subject factor and time (pretest, midtest, and posttest) as the within-subject factor. A total of 66 participants were divided into two groups. All materials incorporated three aspects of phrase knowledge (form, meaning, and use), but the materials of the CG were unimodal in that they were offered only on paper, and of the EG were multimodal in that they included pictures, audio recordings, and video clips. After the treatment, a questionnaire and a semi-structured interview were given to the EG learners to explore their perceptions of using multimodal materials to learn L2 phrases. The results indicate that both groups had significant gains in learning phrases, but students with the multimodal input achieved significantly better results than those with the unimodal input. Moreover, the EG students had a generally positive attitude toward the use of multimodal resources. This study validates the efficacy of multimodal input on the acquisition of English phrases and shows that cognitive overload was avoided by sequencing the information.

Shared Multimodal Input Through Social Coordination: Infants With Monolingual and Bilingual Learning Experiences

A growing number of children in the United States are exposed to multiple languages at home from birth. However, relatively little is known about the early process of word learning—how words are mapped to the referent in their child-centered learning experiences. The present study defined parental input operationally as the integrated and multimodal learning experiences as an infant engages with his/her parent in an interactive play session with objects. By using a head-mounted eye tracking device, we recorded visual scenes from the infant’s point of view, along with the parent’s social input with respect to gaze, labeling, and actions of object handling. Fifty-one infants and toddlers (aged 6–18 months) from an English monolingual or a diverse bilingual household were recruited to observe the early multimodal learning experiences in an object play session. Despite that monolingual parents spoke more and labeled more frequently relative to bilingual parents, infants from both language groups benefit from a comparable amount of socially coordinated experiences where parents name the object while the object is looked at by the infant. Also, a sequential path analysis reveals multiple social coordinated pathways that facilitate infant object looking. Specifically, young children’s attention to the referent objects is directly influenced by parent’s object handling. These findings point to the new approach to early language input and how multimodal learning experiences are coordinated socially for young children growing up with monolingual and bilingual learning contexts.

Lack of Cross-Modal Effects in Dual-Modality Implicit Statistical Learning

A current controversy in the area of implicit statistical learning (ISL) is whether this process consists of a single, central mechanism or multiple modality-specific ones. To provide insight into this question, the current study involved three ISL experiments to explore whether multimodal input sources are processed separately in each modality or are integrated together across modalities. In Experiment 1, visual and auditory ISL were measured under unimodal conditions, with the results providing a baseline level of learning for subsequent experiments. Visual and auditory sequences were presented separately, and the underlying grammar used for both modalities was the same. In Experiment 2, visual and auditory sequences were presented simultaneously with each modality using the same artificial grammar to investigate whether redundant multisensory information would result in a facilitative effect (i.e., increased learning) compared to the baseline. In Experiment 3, visual and auditory sequences were again presented simultaneously but this time with each modality employing different artificial grammars to investigate whether an interference effect (i.e., decreased learning) would be observed compared to the baseline. Results showed that there was neither a facilitative learning effect in Experiment 2 nor an interference effect in Experiment 3. These findings suggest that participants were able to track simultaneously and independently two sets of sequential regularities under dual-modality conditions. These findings are consistent with the theories that posit the existence of multiple, modality-specific ISL mechanisms rather than a single central one.

Seizure Forecasting from Idea to Reality. Outcomes of the My Seizure Gauge Epilepsy Innovation Institute Workshop

The Epilepsy Innovation Institute (Ei²) is a new research program of the Epilepsy Foundation designed to be an innovation incubator for epilepsy. Ei² research areas are selected based on community surveys that ask people impacted by epilepsy what they would like researchers to focus on. In their 2016 survey, unpredictability was selected as a top issue regardless of seizure frequency or severity. In response to this need, Ei² launched the My Seizure Gauge challenge, with the end goal of creating a personalized seizure advisory system device. Prior to moving forward, Ei² convened a diverse group of stakeholders from people impacted by epilepsy and clinicians, to device developers and data scientists, to basic science researchers and regulators, for a state of the science assessment on seizure forecasting. From the discussions, it was clear that we are at an exciting crossroads. With the advances in bioengineering, we can utilize digital markers, wearables, and biosensors as parameters for a seizure-forecasting algorithm. There are also over a thousand individuals who have been implanted with ambulatory intracranial EEG recording devices. Pairing up peripheral measurements to brain states could identify new relationships and insights. Another key component is the heterogeneity of the relationships indicating that pooling findings across groups is suboptimal, and that data collection will need to be done on longer time scales to allow for individualization of potential seizure-forecasting algorithms.