Can You Ink While You Blink? Assessing Mental Effort in a Sensor-Based Calligraphy Trainer

Exploring cerebral laterality of writing and the relationship to handedness: a functional transcranial Doppler ultrasound investigation

Cerebral lateralization of oral language has been investigated in a plethora of studies and it is well established that the left hemisphere is dominant for production tasks in the majority of individuals. However, few studies have focused on written language and even fewer have sampled left-handers. Writing comprises language and motor components, both of which contribute to cerebral activation, yet previous research has not disentangled. The aim of this study was to disentangle the language and motor components of writing lateralization. This was achieved through the comparison of cerebral activation during (i) written word generation and (ii) letter copying, as assessed by functional Transcranial Doppler (fTCD) ultrasound. We further assessed cerebral laterality of oral language. The sample was balanced for handedness. We preregistered the hypotheses that (i) cerebral lateralization of the linguistic component of writing would be weaker in left-handers compared to right-handers and (ii) oral language and the linguistic component of written language would not be correlated in terms of cerebral lateralization. No compelling evidence for either of our hypotheses was found. Findings highlight the complexity of the processes subserving written and oral language as well as the methodological challenges to isolate the linguistic component of writing.

Smart Sensors for Augmented Electrical Experiments

With the recent increase in the use of augmented reality (AR) in educational laboratory settings, there is a need for new intelligent sensor systems capturing all aspects of the real environment. We present a smart sensor system meeting these requirements for STEM (science, technology, engineering, and mathematics) experiments in electrical circuits. The system consists of custom experiment boxes and cables combined with an application for the Microsoft HoloLens 2, which creates an AR experiment environment. The boxes combine sensors for measuring the electrical voltage and current at the integrated electrical components as well as a reconstruction of the currently constructed electrical circuit and the position of the sensor box on a table. Combing these data, the AR application visualizes the measurement data spatially and temporally coherent to the real experiment boxes, thus fulfilling demands derived from traditional multimedia learning theory. Following an evaluation of the accuracy and precision of the presented sensors, the usability of the system was evaluated with n=20 pupils in a German high school. In this evaluation, the usability of the system was rated with a system usability score of 94 out of 100.

Table Tennis Tutor: Forehand Strokes Classification Based on Multimodal Data and Neural Networks

Beginner table-tennis players require constant real-time feedback while learning the fundamental techniques. However, due to various constraints such as the mentor’s inability to be around all the time, expensive sensors and equipment for sports training, beginners are unable to get the immediate real-time feedback they need during training. Sensors have been widely used to train beginners and novices for various skills development, including psychomotor skills. Sensors enable the collection of multimodal data which can be utilised with machine learning to classify training mistakes, give feedback, and further improve the learning outcomes. In this paper, we introduce the Table Tennis Tutor (T3), a multi-sensor system consisting of a smartphone device with its built-in sensors for collecting motion data and a Microsoft Kinect for tracking body position. We focused on the forehand stroke mistake detection. We collected a dataset recording an experienced table tennis player performing 260 short forehand strokes (correct) and mimicking 250 long forehand strokes (mistake). We analysed and annotated the multimodal data for training a recurrent neural network that classifies correct and incorrect strokes. To investigate the accuracy level of the aforementioned sensors, three combinations were validated in this study: smartphone sensors only, the Kinect only, and both devices combined. The results of the study show that smartphone sensors alone perform sub-par than the Kinect, but similar with better precision together with the Kinect. To further strengthen T3’s potential for training, an expert interview session was held virtually with a table tennis coach to investigate the coach’s perception of having a real-time feedback system to assist beginners during training sessions. The outcome of the interview shows positive expectations and provided more inputs that can be beneficial for the future implementations of the T3.

Multimodal Data Fusion in Learning Analytics: A Systematic Review

Multimodal learning analytics (MMLA), which has become increasingly popular, can help provide an accurate understanding of learning processes. However, it is still unclear how multimodal data is integrated into MMLA. By following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, this paper systematically surveys 346 articles on MMLA published during the past three years. For this purpose, we first present a conceptual model for reviewing these articles from three dimensions: data types, learning indicators, and data fusion. Based on this model, we then answer the following questions: 1. What types of data and learning indicators are used in MMLA, together with their relationships; and 2. What are the classifications of the data fusion methods in MMLA. Finally, we point out the key stages in data fusion and the future research direction in MMLA. Our main findings from this review are (a) The data in MMLA are classified into digital data, physical data, physiological data, psychometric data, and environment data; (b) The learning indicators are behavior, cognition, emotion, collaboration, and engagement; (c) The relationships between multimodal data and learning indicators are one-to-one, one-to-any, and many-to-one. The complex relationships between multimodal data and learning indicators are the key for data fusion; (d) The main data fusion methods in MMLA are many-to-one, many-to-many and multiple validations among multimodal data; and (e) Multimodal data fusion can be characterized by the multimodality of data, multi-dimension of indicators, and diversity of methods.

Advanced Sensors Technology in Education

The topic presented will show how different kinds of sensors can help to improve our skills in learning environments. When we open the mind and let it take the control to be creative, we can think how a martial art would be improved with registered sensors, or how a person may dance with machines to improve their technique, or how you may improve your soccer kick for a penalties round. The use of sensors seems easy to imagine in these examples, but their use is not limited to these types of learning environments. Using depth cameras to detect patterns in oral presentations, or improving the assessment of agility through low cost-sensors with multimodal learning analytics, or using computing devices as sensors to measure their impact on primary and secondary students’ performances are the focus of this study as well. We hope readers will find original ideas that allow them to improve and advance in their own researches.