Multimodal Multiresolution Data Fusion Using Convolutional Neural Networks for IoT Wearable Sensing

A gated recurrent unit model based on ultrasound images of dynamic tongue movement for determining the severity of obstructive sleep apnea

Obstructive sleep apnea (OSA) presents as a respiratory disorder characterized by recurrent upper pharyngeal airway collapse during sleep. Dynamic tongue movement (DTM) analysis emerges as a promising avenue for elucidating the pathophysiological underpinnings of OSA, thereby facilitating its diagnosis. Recent endeavors have utilized artificial intelligence techniques to categorize OSA severity leveraging electrocardiography and blood oxygen saturation data. Nonetheless, the integration of ultrasound (US) imaging of the tongue remains largely untapped in the development of machine learning models aimed at determining the severity of OSA. This study endeavors to bridge this gap by capturing US images of DTM dynamics during wakefulness, encompassing transitions from normal breathing (NB) to the performance of the Müller maneuver (MM) in a cohort of 53 patients. Leveraging the modified optical flow method (MOFM), the trajectories of patients' DTM were tracked, facililtating the extraction of 27 parameters vital for model training. These parameters encompassed nine-point lateral movement, nine-point axial movement, and nine-point total displacement of the tongue, resulting in a dataset of 186,030 samples. The gated recurrent unit (GRU) method, renowned for its efficacy in motion tracking, was employed for model development in this study. Validation of the developed model was conducted via stratified k-fold cross-validation (SCV). The systems' overall performance in classifying OSA severity, as quantified by mean accuracy (MA), yielded a value of 43.49%. This pilot investigation marks an exploratory endeavor into the utilization of artificial intelligence for the classification of OSA severity based on US images and dynamic movement patterns. This novel model holds potential to assist clinicians in categorizing OSA severity and guiding the selection of pertinent treatment modalities tailored to the individual needs of patients afflicted with OSA.

Research and application of deep learning-based sleep staging: Data, modeling, validation, and clinical practice

Over the past few decades, researchers have attempted to simplify and accelerate the process of sleep stage classification through various approaches; however, only a few such approaches have gained widespread acceptance. Artificial intelligence technology, particularly deep learning, is promising for earning the trust of the sleep medicine community in automated sleep-staging systems, thus facilitating its application in clinical practice and integration into daily life. We aimed to comprehensively review the latest methods that are applying deep learning for enhancing sleep staging efficiency and accuracy. Starting from the requisite "data" for constructing deep learning algorithms, we elucidated the current landscape of this domain and summarized the fundamental modeling process, encompassing signal selection, data pre-processing, model architecture, classification tasks, and performance metrics. Furthermore, we reviewed the applications of automated sleep staging in scenarios such as sleep-disorder screening, diagnostic procedures, and health monitoring and management. Finally, we conducted an in-depth analysis and discussion of the challenges and future in intelligent sleep staging, particularly focusing on large-scale sleep datasets, interdisciplinary collaborations, and human-computer interactions.

Learning-Based Multimodal Information Fusion and Behavior Recognition of Vascular Interventionists' Operating Skills

The operating skills of vascular interventionists have an important impact on the effect of surgery. However, current research on behavior recognition and skills learning of interventionists' operating skills is limited. In this study, an innovative deep learning-based multimodal information fusion architecture is proposed for recognizing and analyzing eight common operating behaviors of interventionists. An experimental platform integrating four modal sensors is used to collect multimodal data from interventionists. The ANOVA and Manner-Whitney tests is used for relevance analysis of the data. The analysis results demonstrate that there is almost no significant difference ( p <0.001) between the actions related to the unimodal data, which cannot be used for accurate behavior recognition. Therefore, a study of the fusion architecture based on the existing machine learning classifier and the proposed deep learning fusion architecture is carried out. The research findings indicate that the proposed deep learning-based fusion architecture achieves an impressive overall accuracy of 98.5%, surpassing both the machine learning classifier (93.51%) and the unimodal data (90.05%). The deep learning-based multimodal information fusion architecture proves the feasibility of behavior recognition and skills learning of interventionist's operating skills. Furthermore, the application of deep learning-based multimodal fusion technology of surgeon's operating skills will help to improve the autonomy and intelligence of surgical robotic systems.