Effectiveness of Multimedia Pedagogical Agents Predicted by Diverse Theories: a Meta-Analysis

"A Friendly Conversation." Developing an eHealth Intervention to Increase COVID-19 Testing and Vaccination Literacy Among Women with Criminal and Legal System Involvement

Many women leaving jails are ill-prepared to follow recommended COVID-19 mitigation practices, including testing and vaccination. Low COVID-19-related health literacy, exposure to disinformation, and mistrust in authorities put women at increased risk. Research on this population has shown significant use of mobile devices for communication and web access and public Wi-fi for the internet. Using inductive (formative empirical research with the community) and deductive (theory-based) practices, we designed, developed, and pilot-tested a multimedia, culturally tailored web-based electronic health (eHealth) application to increase COVID-19-specific health literacy and promote testing and vaccination among women with criminal and legal system involvement (CLSI). The intervention included a serialized animated multimedia component and a telenovela-style series, complementing each other and addressing knowledge needs identified in the formative research phase of the project. The eHealth intervention was pilot-tested with 13 CLSI women by using online activity logs and semi-structured telephone interviews. Findings confirmed that eHealth interventions employing multimodal information delivery had increased chances of engaging audiences, especially when developed with input from the target population and are culturally tailored. In addition, using a web-based delivery optimized for mobile made the intervention accessible on various devices and decreased the risk of technical problems.

Do a pedagogical agent's clothing and an animated video's setting affect learning?

Pedagogical agents are often used to enhance social cues in learning materials. The inclusion of pedagogical agents raises several design questions, for example on what kind of clothing the agent should wear. Further, it is not yet clear how the setting of an animated learning video (i.e., the digitally created background) affects learning. In an online experiment (N = 200), we investigated whether creating thematically appropriate clothing and setting has some added value in that it improves learning outcomes in comparison to more neutral assets. Whereas all participants acquired knowledge from the animated video, there were no main effects of clothing and setting for any of the dependent variables, but an interaction for learning outcomes (η p 2 = 0.02 ), indicating that the appropriately dressed agent worked better combined with the inappropriate setting than with the appropriate setting. Overall, given those non-significant main effects and the small effect size of the interaction, there seem to be some degrees of freedom for designers of pedagogical agents in animated learning videos. However, these degrees of freedom may be limited to at least moderate (i.e., neutral) levels of appropriateness.

Using Learner Reviews to Inform Instructional Video Design in MOOCs

Videos are arguably the most important and frequently used instructional resource in massive open online courses (MOOCs). Recent research has explored learners' perceptions and preferences regarding MOOC instructional videos. However, these studies are often limited to a small number of specific courses, and few grounded theory studies have been undertaken to investigate this topic. In the present study, a multiple-coder research methodology was adopted to analyze 4534 learner reviews of MOOCs in 14 categories. The study aimed to identify key characteristics associated with learners' favorable perceptions of MOOC videos, types of supplemental or in-video resources learners perceive helpful to support MOOC video use, and video production features learners value. Results revealed that (a) "organized", "detailed", "comprehensible", "interesting", and "practical" were the top five important characteristics associated with learners' favorable perceptions of MOOC videos; (b) learners perceived "presentation slides", "reading materials", "post-video assessments", "embedded questions", and "case studies" as helpful resources to support their utilization of MOOC videos; and (c) learners found "duration" a more salient production feature than "editing", "resolution", "subtitles", "music", or "voice". The findings present implications for MOOC video design and foundations for future research avenues.

Measuring Personalization, Embodiment, and Congruence in Online Learning: A Validation Study

PURPOSE

Instructional designs promoting personalization (conversational style and visible author), embodiment (human-like mannerisms and presence), and social and cognitive congruence have been shown to enhance learning. However, measurements of these instructional design features are lacking. The authors sought to develop and validate an instrument to measure personalization, embodiment, and congruence in an online course, collecting evidence of content, internal structure, and relationships with other variables.

METHOD

Through literature review, expert review, and pilot testing, the authors developed a 15-item, 5-domain instrument. Participants completed this instrument along with assessments of motivation (Instructional Materials Motivation Survey and Motivated Strategies for Learning Questionnaire), knowledge, and course satisfaction immediately after a 30-minute online multimedia learning module comprising slides with voiceover and (for some participants) short video clips, from January to April 2021. Participants included pharmacists, physicians, and advanced practice providers at academic and community hospitals. Confirmatory factor analysis (CFA) was used to examine model fit, followed by structural revisions guided by exploratory factor analysis (EFA) and tested in a new CFA using a split-sample approach. The authors evaluated associations between the revised instrument scores and motivation, knowledge, and satisfaction.

RESULTS

There were 406 participants. CFA of the original 15-item instrument revealed poor model fit. EFA using 214 randomly selected participants suggested a 2-domain structure. CFA of the revised structure using the other 192 participants revealed good fit for a 9-item, 2-domain instrument: the Congruence-Personalization Questionnaire (CPQ). Linear regression confirmed positive, statistically significant associations for most hypothesized relationships (R 2 ≥ 0.12 and P < .001 for 14 of 18 associations with motivation and satisfaction). Associations with knowledge were weak and nonsignificant (R 2 ≤ 0.004, P > .23).

CONCLUSIONS

Evidence of content, internal structure, and relationships with other variables support the validity of CPQ scores for measuring personalization and congruence in an online module.

The Impact of Emotional Feedback and Elaborated Feedback of a Pedagogical Agent on Multimedia Learning

This study aimed to explore the impact of emotional feedback and elaborated feedback provided by a pedagogical agent (PA) on learners' emotions, intrinsic motivation, agent perception, cognitive load, and transfer performance in multimedia learning. The experiment was conducted based on an actual undergraduate course. Undergraduate students (N = 117) were randomly assigned to one of the four conditions, where PA's feedback differed by emotional feedback (with vs. without) and elaborated feedback (elaborated feedback vs. knowledge of results). Results revealed that emotional feedback reduced learners' confusion, activated intrinsic motivation, and enhanced agent perception. In addition, elaborated feedback improved intrinsic motivation, agent perception, and transfer performance but reduced germane cognitive load. Surprisingly, there was no significant interaction between emotional feedback and elaborated feedback. These findings had implications for designing a PA with a feedback fulfilling learners' emotional and cognitive needs to maximize multimedia learning.

Can an angry pedagogical agent enhance mental effort and learning performance in a multimedia learning environment?

Purpose

This study aims to examine if a pedagogical agent’s expressed anger, when framed as a feedback cue, can enhance mental effort and learning performance in a multimedia learning environment than expressed happiness.

Design/methodology/approach

A between-subjects experiment was conducted in which learners engaged with a multimedia learning material that taught programming algorithms, featuring a pedagogical agent who expressed anger or happiness as a feedback cue in response to the learners’ prior performance. Learners completed a self-reported scale and post-test for measuring mental effort and learning performance, respectively.

Findings

Female learners reported higher mental effort and had better learning performance when the pedagogical agent expressed anger than happiness. Male learners reported marginally lower mental effort when the pedagogical agent expressed anger than happiness.

Originality/value

This study focuses on a pedagogical agent’s expressed emotion as social information to learners. Extending from research advocating a pedagogical agent’s positive emotional expression, this study highlights the potential benefits of a pedagogical agent’s negative emotional expression, such as anger, as a cue for learners to enhance learning effort and performance in a multimedia learning environment.