Infusing Expert Knowledge Into a Deep Neural Network Using Attention Mechanism for Personalized Learning Environments

Machine learning models are biased toward data seen during the training steps. The models will tend to give good results in classes where there are many examples and poor results in those with few examples. This problem generally occurs when the classes to predict are imbalanced and this is frequent in educational data where for example, there are skills that are very difficult or very easy to master. There will be less data on students that correctly answered questions related to difficult skills and who incorrectly answered those related to skills easy to master. In this paper, we tackled this problem by proposing a hybrid architecture combining Deep Neural Network architectures— especially Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN)—with expert knowledge for user modeling. The proposed solution uses attention mechanism to infuse expert knowledge into the Deep Neural Network. It has been tested in two contexts: knowledge tracing in an intelligent tutoring system (ITS) called Logic-Muse and prediction of socio-moral reasoning in a serious game called MorALERT. The proposed solution is compared to state-of-the-art machine learning solutions and experiments show that the resulting model can accurately predict the current student's knowledge state (in Logic-Muse) and thus enable an accurate personalization of the learning process. Other experiments show that the model can also be used to predict the level of socio-moral reasoning skills (in MorALERT). Our findings suggest the need for hybrid neural networks that integrate prior expert knowledge (especially when it is necessary to compensate for the strong dependency—of deep learning methods—on data size or the possible unbalanced datasets). Many domains can benefit from such an approach to building models that allow generalization even when there are small training data.

Assessing and Optimizing Socio-Moral Reasoning Skills: Findings From the MorALERT Serious Video Game

Background

Social cognition and competence are a key part of daily interactions and essential for satisfying relationships and well-being. Pediatric neurological and psychological conditions can affect social cognition and require assessment and remediation of social skills. To adequately approximate the complex and dynamic nature of real-world social interactions, innovative tools are needed. The aim of this study was to document the performance of adolescents on two versions of a serious video game presenting realistic, everyday, socio-moral conflicts, and to explore whether their performance is associated with empathy or sense of presence, factors known to influence social cognition.

Methods

Participants (12–17 years, M = 14.39; SD = 1.35) first completed a pre-test measure of socio-moral reasoning based on three dilemmas from a previously validated computer task. Then, they either played an evaluative version (n = 24) or an adaptive (n = 33) version of a video game presenting nine social situations in which they made socio-moral decisions and provided justifications. In the evaluative version, participants’ audio justifications were recorded verbatim and coded manually to obtain a socio-moral reasoning maturity score. In the adaptive version (AV), tailored feedback and social reinforcements were provided based on participant responses. An automatic coding algorithm developed using artificial intelligence was used to determine socio-moral maturity level in real-time and to provide a basis for the feedback and reinforcements in the game. All participants then completed a three-dilemma post-test assessment.

Results

Those who played the adaptive version showed improved SMR across the pre-test, in-game and post-test moral maturity scores, F(1.97,63.00) = 9.81, p_HF < 0.001, ϵ² = 0.21, but those who played the Evaluative version did not. Socio-moral reasoning scores from both versions combined did not correlate with empathy or sense of presence during the game, though results neared significance. The study findings support preliminary validation of the game as a promising method for assessing and remediating social skills during adolescence.