Contributions of mixed reality in a calligraphy learning task: Effects of supplementary visual feedback and expertise on cognitive load, user experience and gestural performance

Does modifying visual feedback facilitate learning to write new pseudoletters?

Learning to write relies on the efficient integration of visual and proprioceptive feedback, with a transition from a control, based on the visual inspection of the written trace at the beginning of the learning process to a more predictive control, based mainly on handwriting movement, in proficient writers. The aim of this study was to test the effect of a partial deletion of the written trace, as well as the effect of supplementary visual information, on handwriting kinematics in a learning task. Twenty-four adults learned to write six new pseudoletters using their non-dominant hand on a touch screen digital tablet. Three pseudoletters were trained with modified visual feedback conditions and the other three, in the control condition, i.e. without any visual modification. Results revealed that, in the short-term, the pseudoletters learned with modified visual feedback were traced faster and more fluently than those learned in the control condition, without spatial accuracy reduction. This method seems to be efficient in adults, which is a prerequisite before testing a method with children.

Learning Performance Prediction-Based Personalized Feedback in Online Learning via Machine Learning

Online learning has become a vital option for ensuring daily instruction in response to the emergence of the COVID-19 epidemic. However, different from conventional massive online learning, inadequate available data bring challenges for instructors to identify underachieving students in school-based online learning, which may obstruct timely guidance and impede learning performance. Exploring small-sample-supported learning performance prediction and personalized feedback methods is an urgent need to mitigate these shortcomings. Consequently, considering the problem of insufficient data, this study proposes a machine learning model for learning performance prediction with additional pre-training and fine-tuning phases, and constructs a personalized feedback generation method to improve the online learning effect. With a quasi-experiment involving 62 participants (33 in experimental group and 29 in control group), the validity of the prediction model and personalized feedback generation, and the impact of the personalized feedback on learning performance and cognitive load, were evaluated. The results revealed that the proposed model reached a relatively high level of accuracy compared to the baseline models. Additionally, the students who learned with personalized feedback performed significantly better in terms of learning performance and showed a lower cognitive load.

Multimodal Data Fusion in Learning Analytics: A Systematic Review

Multimodal learning analytics (MMLA), which has become increasingly popular, can help provide an accurate understanding of learning processes. However, it is still unclear how multimodal data is integrated into MMLA. By following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, this paper systematically surveys 346 articles on MMLA published during the past three years. For this purpose, we first present a conceptual model for reviewing these articles from three dimensions: data types, learning indicators, and data fusion. Based on this model, we then answer the following questions: 1. What types of data and learning indicators are used in MMLA, together with their relationships; and 2. What are the classifications of the data fusion methods in MMLA. Finally, we point out the key stages in data fusion and the future research direction in MMLA. Our main findings from this review are (a) The data in MMLA are classified into digital data, physical data, physiological data, psychometric data, and environment data; (b) The learning indicators are behavior, cognition, emotion, collaboration, and engagement; (c) The relationships between multimodal data and learning indicators are one-to-one, one-to-any, and many-to-one. The complex relationships between multimodal data and learning indicators are the key for data fusion; (d) The main data fusion methods in MMLA are many-to-one, many-to-many and multiple validations among multimodal data; and (e) Multimodal data fusion can be characterized by the multimodality of data, multi-dimension of indicators, and diversity of methods.

A Review on Mixed Reality: Current Trends, Challenges and Prospects

Currently, new technologies have enabled the design of smart applications that are used as decision-making tools in the problems of daily life. The key issue in designing such an application is the increasing level of user interaction. Mixed reality (MR) is an emerging technology that deals with maximum user interaction in the real world compared to other similar technologies. Developing an MR application is complicated, and depends on the different components that have been addressed in previous literature. In addition to the extraction of such components, a comprehensive study that presents a generic framework comprising all components required to develop MR applications needs to be performed. This review studies intensive research to obtain a comprehensive framework for MR applications. The suggested framework comprises five layers: the first layer considers system components; the second and third layers focus on architectural issues for component integration; the fourth layer is the application layer that executes the architecture; and the fifth layer is the user interface layer that enables user interaction. The merits of this study are as follows: this review can act as a proper resource for MR basic concepts, and it introduces MR development steps and analytical models, a simulation toolkit, system types, and architecture types, in addition to practical issues for stakeholders such as considering MR different domains.

Can You Ink While You Blink? Assessing Mental Effort in a Sensor-Based Calligraphy Trainer

Sensors can monitor physical attributes and record multimodal data in order to provide feedback. The application calligraphy trainer, exploits these affordances in the context of handwriting learning. It records the expert's handwriting performance to compute an expert model. The application then uses the expert model to provide guidance and feedback to the learners. However, new learners can be overwhelmed by the feedback as handwriting learning is a tedious task. This paper presents the pilot study done with the calligraphy trainer to evaluate the mental effort induced by various types of feedback provided by the application. Ten participants, five in the control group and five in the treatment group, who were Ph.D. students in the technology-enhanced learning domain, took part in the study. The participants used the application to learn three characters from the Devanagari script. The results show higher mental effort in the treatment group when all types of feedback are provided simultaneously. The mental efforts for individual feedback were similar to the control group. In conclusion, the feedback provided by the calligraphy trainer does not impose high mental effort and, therefore, the design considerations of the calligraphy trainer can be insightful for multimodal feedback designers.