Cerebral Hemodynamic Response During Concealment of Information About a Mock Crime: Application of a General Linear Model With an Adaptive Hemodynamic Response Function

CAS(ME)3: A Third Generation Facial Spontaneous Micro-Expression Database With Depth Information and High Ecological Validity

Micro-expression (ME) is a significant non-verbal communication clue that reveals one person's genuine emotional state. The development of micro-expression analysis (MEA) has just gained attention in the last decade. However, the small sample size problem constrains the use of deep learning on MEA. Besides, ME samples distribute in six different databases, leading to database bias. Moreover, the ME database development is complicated. In this article, we introduce a large-scale spontaneous ME database: CAS(ME) 3. The contribution of this article is summarized as follows: (1) CAS(ME) 3 offers around 80 hours of videos with over 8,000,000 frames, including manually labeled 1,109 MEs and 3,490 macro-expressions. Such a large sample size allows effective MEA method validation while avoiding database bias. (2) Inspired by psychological experiments, CAS(ME) 3 provides the depth information as an additional modality unprecedentedly, contributing to multi-modal MEA. (3) For the first time, CAS(ME) 3 elicits ME with high ecological validity using the mock crime paradigm, along with physiological and voice signals, contributing to practical MEA. (4) Besides, CAS(ME) 3 provides 1,508 unlabeled videos with more than 4,000,000 frames, i.e., a data platform for unsupervised MEA methods. (5) Finally, we demonstrate the effectiveness of depth information by the proposed depth flow algorithm and RGB-D information.

Increasing motor cortex activation during grasping via novel robotic mirror hand therapy: a pilot fNIRS study

Background

Mirror therapy (MT) has been used for functional recovery of the affected hand by providing the mirrored image of the unaffected hand movement, which induces neural activation of the cortical hemisphere contralateral to the affected hand. Recently, many wearable robots assisting the movement of the hand have been developed, and several studies have proposed robotic mirror therapy (RMT) that uses a robot to provide mirrored movements of the unaffected hand to the affected hand with the robot controlled by measuring electromyography or posture of the unaffected hand. In some cases of RMT a mirror is placed to allow the person to observe only the unaffected hand but in others users simply observe the robotically assisted hand performing the mirrored movements, as was the case in this study. There have been limited evaluations of the cortical activity during RMT compared to MT and robotic therapy (RT) providing passive movements despite the difference in the modality of sensory feedback and the involvement of motor intention, respectively.

Methods

This paper analyzes bilateral motor cortex activation in nine healthy subjects and five chronic stroke survivors during a pinching task performed in MT, RT, and RMT conditions using functional near infrared spectroscopy (fNIRS). In the MT condition, the person moved the unaffected hand and observed it in a mirror while the affected hand remained still. In RT condition passive movements were provided to the affected hand with a cable-driven soft robotic glove, while, in RMT condition, the posture of the unaffected hand was measured by a sensing glove and the soft robotic glove mirrored its movement on the affected hand.

Results

For both groups, the RMT condition showed the greatest mean cortical activation on the motor cortex contralateral to the affected (non-dominant for the healthy group) hand compared to other conditions. Individual results indicate that RMT induces similar or greater neural activation on the motor cortex compared to MT and RT conditions. The interhemispheric activations of both groups were balanced in RMT condition. In MT condition, significantly greater activation was shown on the hemisphere ipsilateral to the affected (dominant for the healthy group) hand for both subject groups, while the contralateral side showed significantly greater activation for the healthy group in RT condition.

Conclusion

The experimental results indicate that combining visual feedback, somatosensory feedback, and motor intention are important for greater stimulation on the contralateral motor cortex of the affected hand. RMT that includes these factors is hypothesized to achieve a more effective functional rehabilitation due to greater and more balanced cortical activation.

Language Familiarity and Proficiency Leads to Differential Cortical Processing During Translation Between Distantly Related Languages

In the midst of globalization, English is regarded as an international language, or Lingua Franca, but learning it as a second language (L2) remains still difficult to speakers of other languages. This is true especially for the speakers of languages distantly related to English such as Japanese. In this sense, exploring neural basis for translation between the first language (L1) and L2 is of great interest. There have been relatively many previous researches revealing brain activation patterns during translations between L1 and English as L2. These studies, which focused on language translation with close or moderate linguistic distance (LD), have suggested that the Broca area (BA 44/45) and the dorsolateral prefrontal cortex (DLPFC; BA 46) may play an important role on translation. However, the neural mechanism of language translation between Japanese and English, having large LD, has not been clarified. Thus, we used functional near infrared spectroscopy (fNIRS) to investigate the brain activation patterns during word translation between Japanese and English. We also assessed the effects of translation directions and word familiarity. All participants’ first language was Japanese and they were learning English. Their English proficiency was advanced or elementary. We selected English and Japanese words as stimuli based on the familiarity for Japanese people. Our results showed that the brain activation patterns during word translation largely differed depending on their English proficiency. The advanced group elicited greater activation on the left prefrontal cortex around the Broca’s area while translating words with low familiarity, but no activation was observed while translating words with high familiarity. On the other hand, the elementary group evoked greater activation on the left temporal area including the superior temporal gyrus (STG) irrespective of the word familiarity. These results suggested that different cognitive process could be involved in word translation corresponding to English proficiency in Japanese learners of English. These difference on the brain activation patterns between the advanced and elementary group may reflect the difference on the cognitive loads depending on the levels of automatization in one’s language processing.