An eye-tracking analysis of instructor presence in video lectures

Increasing the realism of on-screen embodied instructors creates more looking but less learning

BACKGROUND

Although adding embodied instructors on the screen is considered an effective way to improve online multimedia learning, its effectiveness is still controversial. The level of realism of embodied on-screen instructors may be an influencing factor, but it is unclear how it affects multimedia learning.

AIMS

We explored whether and how embodied on-screen instructors rendered with different levels of realism in multimedia lessons affect learning process and learning outcomes.

SAMPLES

We recruited 125 college students as participants.

METHODS

Students learned about neural transmission in an online multimedia lesson that included a real human, cartoon human, cartoon animal or no instructor.

RESULTS

Students learning with cartoon human or cartoon animal instructors tended to fixate more on the relevant portions of the screen and performed better on retention and transfer tests than no instructor group. The real human group fixated more on the instructor, fixated less on the relevant portion of the screen and performed worse on a retention test in comparison to the cartoon human group. Fixation time on the instructor fully mediated the relationship between instructor realism and retention score.

CONCLUSIONS

The addition of embodied on-screen instructors can promote multimedia learning, but the promotion effect would be better if the embodied instructor was a cartoon animal or cartoon human rather than a real human. This suggests an important boundary condition in which less realism of on-screen embodied instructors produces better learning processes and outcomes.

Onscreen presence of instructors in video lectures affects learners' neural synchrony and visual attention during multimedia learning

COVID-19 forced students to rely on online learning using multimedia tools, and multimedia learning continues to impact education beyond the pandemic. In this study, we combined behavioral, eye-tracking, and neuroimaging paradigms to identify multimedia learning processes and outcomes. College students viewed four video lectures including slides with either an onscreen human instructor, an animated instructor, or no onscreen instructor. Brain activity was recorded via fMRI, visual attention was recorded via eye-tracking, and learning outcome was assessed via post-tests. Onscreen presence of instructor, compared with no instructor presence, resulted in superior post-test performance, less visual attention on the slide, more synchronized eye movements during learning, and higher neural synchronization in cortical networks associated with socio-emotional processing and working memory. Individual variation in cognitive and socio-emotional abilities and intersubject neural synchronization revealed different levels of cognitive and socio-emotional processing in different learning conditions. The instructor-present condition evoked increased synchronization, likely reflecting extra processing demands in attentional control, working memory engagement, and socio-emotional processing. Although human instructors and animated instructors led to comparable learning outcomes, the effects were due to the dynamic interplay of information processing vs. attentional distraction. These findings reflect a benefit-cost trade-off where multimedia learning outcome is enhanced only when the cognitive benefits motivated by the social presence of onscreen instructor outweigh the cognitive costs brought about by concurrent attentional distraction unrelated to learning.

Visual fixations during processing of time-compressed audiovisual presentations

Time-compression is a technique that allows users to adjust the playback speed of audio recordings, but comprehension declines at higher speeds. Previous research has shown that under challenging auditory conditions people have a greater tendency to fixate regions closer to a speaker's mouth. In the current study, we investigated whether there is a similar tendency to fixate the mouth region for time-compressed stimuli. Participants were presented with a brief audiovisual lecture at different speeds, while eye fixations were recorded, and comprehension was tested. Results showed that the 50% compressed lecture group looked more at the nose compared to eye fixations for the normal lecture, and those in the 75% compressed group looked more towards the mouth. Greater compression decreased comprehension, but audiovisual information did not reduce this deficit. These results indicate that people seek out audiovisual information to overcome time-compression, demonstrating the flexibility of the multimodal attentional system.

Exploring the Untapped Potential of Neuromarketing in Online Learning: Implications and Challenges for the Higher Education Sector in Europe

This research investigates the impact of applying neuromarketing techniques to three practical examples of higher education (HE) branding: an official college website page, an official college Facebook page, and recorded online video lectures used for teaching at HE institutions. The study was conducted in three different HE institutions with a representative sample of 720 participants, with n = 529 used for testing the CARE college website, n = 59 for testing the HAZEF Facebook page, and n = 132 for testing the emotional response of students studying online. To assess the participants' emotional responses, this study utilized automated facial coding through a webcam (15 Hz) and considered mood intensities. Additionally, a sentiment analysis was employed to verify the survey results and determine any discrepancies in the cognitive response. By analyzing gaze activity, movement patterns, and emotional responses, valuable insights were gained into students' behaviors and preferences. This study recommends incorporating neuromarketing research into HE branding and online teaching to enhance students' learning experiences. Overall, this study contributes to the understanding of human expectations and behaviors in response to online teaching and provides valuable insights for HE institutions in Europe.

Individual differences in selective attention and engagement shape students’ learning from visual cues and instructor presence during online lessons

Although some researchers recommend minimizing extraneous visual information in multimedia lessons, others have demonstrated that features such as visual cues and instructor videos can enhance learning. However, variability in selective attention skills may influence students’ ability to benefit from these additional features. This study investigated links between college students’ selective attention skills and their learning from video lessons that varied in the use of visual cues and the instructor video. Learning outcomes depended on both the visual features available and students’ effort and selective attention skills. Among students who reported increased effort during the lessons, those with more efficient selective attention benefited most when a single additional feature (i.e., either visual cues or the instructor video) was used. All students, regardless of attention skills, benefited when both visual cues and the instructor were combined. These findings suggest that learning during multimedia lessons may depend on the visual features of the lessons and the student’s effort and attention skills.

Confused or not: decoding brain activity and recognizing confusion in reasoning learning using EEG

Objective.Confusion is the primary epistemic emotion in the learning process, influencing students' engagement and whether they become frustrated or bored. However, research on confusion in learning is still in its early stages, and there is a need to better understand how to recognize it and what electroencephalography (EEG) signals indicate its occurrence. The present work investigates confusion during reasoning learning using EEG, and aims to fill this gap with a multidisciplinary approach combining educational psychology, neuroscience and computer science.Approach.First, we design an experiment to actively and accurately induce confusion in reasoning. Second, we propose a subjective and objective joint labeling technique to address the label noise issue. Third, to confirm that the confused state can be distinguished from the non-confused state, we compare and analyze the mean band power of confused and unconfused states across five typical bands. Finally, we present an EEG database for confusion analysis, together with benchmark results from conventional (Naive Bayes, Support Vector Machine, Random Forest, and Artificial Neural Network) and end-to-end (Long Short Term Memory, Residual Network, and EEGNet) machine learning methods.Main results.Findings revealed: 1. Significant differences in the power of delta, theta, alpha, beta and lower gamma between confused and non-confused conditions; 2. A higher attentional and cognitive load when participants were confused; and 3. The Random Forest algorithm with time-domain features achieved a high accuracy/F1 score (88.06%/0.88 for the subject-dependent approach and 84.43%/0.84 for the subject-independent approach) in the binary classification of the confused and non-confused states.Significance.The study advances our understanding of confusion and provides practical insights for recognizing and analyzing it in the learning process. It extends existing theories on the differences between confused and non-confused states during learning and contributes to the cognitive-affective model. The research enables researchers, educators, and practitioners to monitor confusion, develop adaptive systems, and test recognition approaches.

Instructors' presence in instructional videos: A systematic review

The discussion about how to present instructors in instructional videos has become a hot topic in recent years. This systematic review explores how the instructors' presence affects affective, cognitive, and social aspects of learning in different conditions and with different video types. The review analyses 41 empirical studies indexed in Web of Science, ERIC, Scopus, and Education Source research databases from 2014 to 2022. The results indicated that (i) many instructor-present videos were in picture-in-picture format and included direct gaze as a social cue, (ii) learners had positive feelings for instructor-present videos, (iii) the on-screen instructor could not be beneficial for gathering positive learning outcomes, but social and attentional cues provided by the on-screen instructor could foster learning, and (iv) findings regarding the social aspect of learning were inconclusive. This study also emphasizes the need for further studies to clearly explore the role of the instructor in different learning conditions.

Supplementary information

The online version contains supplementary material available at 10.1007/s10639-022-11532-4.

A review of eye tracking research on video-based learning

Eye tracking technology is increasingly used to understand individuals' non-conscious, moment-to-moment processes during video-based learning. This review evaluated 44 eye tracking studies on video-based learning conducted between 2010 and 2021. Specifically, the review sought to uncover how the utilisation of eye tracking technology has advanced understandings of the mechanisms underlying effective video-based learning and what type of caution should be exercised when interpreting the findings of these studies. Four important findings emerged from the analysis: (1) not all the studies explained the mechanisms underlying effective video-based learning through employing eye tracking technology, and few studies disentangled the complex relationship between eye tracking metrics and cognitive activities these metrics represent; (2) emotional factors potentially serve to explain the processes that facilitate video-based learning, but few studies captured learners' emotional processes or evaluated their affective gains; (3) ecological validity should be improved for eye tracking research on video-based learning through methods such as using eye tracking systems that have high tolerance for head movements, allowing learners to take control of the pacing of the video, and communicating the learning objectives of the video to participants; and (4) boundary conditions, including personal (e.g. age, prior knowledge) and environmental factors (e.g. the topic of videos, type of knowledge), must be considered when interpreting research findings. The findings of this review inspire a number of propositions for designing and interpreting eye tracking research on video-based learning.

Impact of instructor on-slide presence in synchronous e-learning

As a consequence of the COVID-19 pandemic, many higher education programs had to switch to synchronous online teaching. Teachers suddenly faced pressing unaddressed challenges, such as how to better transfer their "presence" from the traditional classroom to the online space in a way that keeps students engaged. This paper explores new venues for increasing the quality of synchronous online learning. We propose the notion of broad on-slide presence, pillared on an increased instructor expressiveness and an elevated instructor slide-content interaction. We conducted four studies to investigate the benefits of delivering lectures in this format, using a mixed methods research approach. We combined survey methodology with transversal design and structural equation modelling with qualitative methodology using discourse analysis of teacher interviews. Results revealed a significant increase in perceived knowledge gain and attentional engagement, and an improved and more personal student experience. At the same time, the instructor's broader on-slide presence also resulted in an increased teacher satisfaction.

Instructors’ expressive nonverbal behavior hinders learning when learners’ prior knowledge is low

This study investigated the influence of instructors’ expressive nonverbal behavior and nonexpressive nonverbal behavior in video lectures on students’ learning performance and affective experience. We conducted two rounds of experiments using the same materials and procedures, the only difference being the participants. In each round of experiments, participants were randomly assigned to expressive condition or nonexpressive condition. 227 rural primary school sixth-graders took part in experiment 1, participants in expressive condition had better affective experiences and perceived tasks as less difficult, but had lower learning performance than participants in nonexpressive condition. 175 sixth-graders from urban primary schools participated in experiment 2. The results showed that instructors’ expressive nonverbal behavior also improved students’ affective experience and reduced students’ perception of task difficulty, but there was no significant difference in learning performance between the two groups. Comparing the pretest scores of students in the two experiments, it was found that the pretest scores of participants in experiment 2 were higher than those in experiment 1. Overall, instructors’ expressive nonverbal behavior can improve students’ affective experience and reduce their perception of task difficulty. However, when students’ prior knowledge is relatively low, instructors’ expressive nonverbal behavior hinders students’ learning performance. We suggest that teachers adopt expressive nonverbal behavior when lecturing because it is beneficial to maintain students’ long-term interest in learning. However, it should be noted that the difficulty of learning material should be determined by students’ prior knowledge.

Exploring Gender Differences in the Instructor Presence Effect in Video Lectures: An Eye-Tracking Study

The instructor’s presence on the screen has become a popular feature in the video lectures of online learning and has drawn increasing research interest. Studies on the instructor presence effect of video lectures mainly focused on the features of the instructor, and few have taken learners’ differences, such as gender, into consideration. The current study examined whether male and female learners differed in their learning performance and eye movement features when learning video lectures with and without the instructor’s presence. All participants (N = 64) were asked to watch three different types of video lectures: audio-video without instructor presence (AV), picture-video with instructor presence (PV), and video-video with instructor presence (VV). They watched nine videos, three of each condition, and completed a reading comprehension test after each video. Their eye movement data were simultaneously collected when they watched these videos. Results showed that learners gained better outcomes after watching the videos with a talking instructor (VV) than those with the instructor’s picture (PV) or without the instructor (AV). This finding suggests that the dynamic presence of the instructor in video lectures could enhance learning through increased social presence and agency. Gender differences were found in their attention allocation, but not behavioral learning performance. When watching the videos with a talking instructor (VV), female learners dwelt longer on the instructor, while males transited more between the instructor and the text. Our results highlight the value of instructor presence in video lectures and call for more comprehensive explorations of gender differences in online learning outcomes and attention distribution.

Laugh before You Study: Does Watching Funny Videos before Study Facilitate Learning?

Emotions exist widely in the entire process of learning and affect students' motivation as well as academic performance. In multimedia learning, academics usually focus on the impact of teachers' emotions or the emotional design of multimedia learning materials on students' emotions and learning results. Few studies have investigated how to enhance learning by regulating students' pre-learning emotions. This study focused on whether playing funny videos before learning could promote students' positive emotions to enhance their motivation, satisfaction, and learning outcomes. We randomly divided 81 elementary school students into two groups: experimental group and control group. While the experimental group watched funny video clips, the control group watched neutral video clips before starting the video learning. The experimental group had more positive pre-learning emotions than the control group. After the course, the emotion of the experimental group declined while that of the control group enhanced. However, positive pre-learning emotions still promoted students' understanding and transfer of learning materials. Moreover, no significant differences were observed between the two groups in learning motivation, satisfaction, and retention tests. Furthermore, this paper analyzed the causes of the experimental results and discussed the insights for teaching.

Analysis of the Learning Process through Eye Tracking Technology and Feature Selection Techniques

In recent decades, the use of technological resources such as the eye tracking methodology is providing cognitive researchers with important tools to better understand the learning process. However, the interpretation of the metrics requires the use of supervised and unsupervised learning techniques. The main goal of this study was to analyse the results obtained with the eye tracking methodology by applying statistical tests and supervised and unsupervised machine learning techniques, and to contrast the effectiveness of each one. The parameters of fixations, saccades, blinks and scan path, and the results in a puzzle task were found. The statistical study concluded that no significant differences were found between participants in solving the crossword puzzle task; significant differences were only detected in the parameters saccade amplitude minimum and saccade velocity minimum. On the other hand, this study, with supervised machine learning techniques, provided possible features for analysis, some of them different from those used in the statistical study. Regarding the clustering techniques, a good fit was found between the algorithms used (k-means ++, fuzzy k-means and DBSCAN). These algorithms provided the learning profile of the participants in three types (students over 50 years old; and students and teachers under 50 years of age). Therefore, the use of both types of data analysis is considered complementary.

Curriculum Innovation in Times of the COVID-19 Pandemic: The Thinking-Based Instruction Theory and Its Application

At the beginning of 2020, to stop the spread of the coronavirus disease (COVID-19) to the campus, the Ministry of Education of China launched a policy “Suspension of classes without suspending schooling” for the spring semester of 2020. However, the drawbacks of online teaching (e.g., students’ inadequate autonomous learning, the lack of effective online instruction) forced us to modify teaching strategies during this special period, especially developing courses that are suitable for student learning at home and improving their key competencies. In order to solve these problems, this study introduces some theoretical exploration and practical work of curriculum design under the guidance of thinking-based instruction theory (TBIT) during the pandemic. We firstly introduce TBIT, and elaborate on the curriculum design under the TBIT theoretical frame. Then we describe a series of TBIT-based micro-courses with the pandemic as background. A descriptive study is reported to illustrate the effects of three micro-courses. Results showed that, compared to national curricula, the TBIT-based micro-courses not only improved the course quality but also enhanced students’ motivation and facilitated their online learning behavior (such as interactive communication) for the online courses. The current study has important implications for how to design effective and interesting online courses suitable under pandemic and capable of improving students’ thinking abilities and key competencies.

Lifelong Learning from Sustainable Education: An Analysis with Eye Tracking and Data Mining Techniques

The use of learning environments that apply Advanced Learning Technologies (ALTs) and Self-Regulated Learning (SRL) is increasingly frequent. In this study, eye-tracking technology was used to analyze scan-path differences in a History of Art learning task. The study involved 36 participants (students versus university teachers with and without previous knowledge). The scan-paths were registered during the viewing of video based on SRL. Subsequently, the participants were asked to solve a crossword puzzle, and relevant vs. non-relevant Areas of Interest (AOI) were defined. Conventional statistical techniques (ANCOVA) and data mining techniques (string-edit methods and k-means clustering) were applied. The former only detected differences for the crossword puzzle. However, the latter, with the Uniform Distance model, detected the participants with the most effective scan-path. The use of this technique successfully predicted 64.9% of the variance in learning results. The contribution of this study is to analyze the teaching–learning process with resources that allow a personalized response to each learner, understanding education as a right throughout life from a sustainable perspective.

What we learned from creating one of the world's most popular MOOCs

Learning How to Learn (LHTL) is currently one of the world's most popular massive open online course (MOOC), with nearly 2.5 million registered learners in its first 4 years. Here, we "reverse engineer" the design of the course's videos to show how creative application of well-known principles of multimedia learning in an MOOC context appear to have fueled the course's popularity. Gaps in knowledge of multimedia learning are also noted. There have been some 50 years of experience researching effective classroom teaching, but less there have been only 5 years since MOOCs became widespread. The success of LHTL may provide further insight into the importance of the principles of multimedia learning, and how those principles might be practically implemented to improve MOOC making and the general design of instructional videos.