RETRACTED ARTICLE: Multimedia Effect in Problem Solving: a Meta-analysis

The effectiveness of web-based Mathematics instruction (WBMI) on K-16 students' mathematics learning: a meta-analytic research

Given the increasing prevalence of web technology, web-based mathematics environments have been increasingly widely used in mathematics education for the past two decades. The COVID-19 pandemic has led to an urgent transition from traditional mathematics instruction (TMI) to web-based mathematics instruction (WBMI) at all levels of mathematics education. At this point, it is crucial to scrutinize the effects of WBMI on K-16 students' mathematics learning comprehensively. This meta-analysis research contained a total of 63 studies with 115 effect sizes, which aimed to investigate the effectiveness of WBMI on K-16 students' mathematics learning by incorporating potential moderators, namely mathematics topics, mathematical content standards, feedback status, type of instructional features, age (i.e., grade level), and assessment methods. Based on findings, WBMI has a significantly strong effect on K-16 students' mathematics learning (g = 1.10, p = 0.01, 95% CI [0.95, 1.27]). Moderator analyses reveal that the effect sizes of WBMI on K-16 students' mathematics learning varied significantly depending on all these potential moderators. Additionally, higher-level mathematical concepts, statistics and probability, WBMI with providing feedback, tutorial systems, undergraduate students, and traditional paper-pencil assessment are the strongest moderators in their context. The most notable results of this research are that WBMI is significantly more effective on students' mathematics learning than TMI, while even in the context of WBMI, traditional paper-pencil assessment is significantly more effective than online assessment. This meta-analytic research provides a comprehensive and up-to-date perspective on the effectiveness of WBMI on K-16 students' mathematics learning.

Assessment of Diagnostic Competences With Standardized Patients Versus Virtual Patients: Experimental Study in the Context of History Taking

BACKGROUND

Standardized patients (SPs) have been one of the popular assessment methods in clinical teaching for decades, although they are resource intensive. Nowadays, simulated virtual patients (VPs) are increasingly used because they are permanently available and fully scalable to a large audience. However, empirical studies comparing the differential effects of these assessment methods are lacking. Similarly, the relationships between key variables associated with diagnostic competences (ie, diagnostic accuracy and evidence generation) in these assessment methods still require further research.

OBJECTIVE

The aim of this study is to compare perceived authenticity, cognitive load, and diagnostic competences in performance-based assessment using SPs and VPs. This study also aims to examine the relationships of perceived authenticity, cognitive load, and quality of evidence generation with diagnostic accuracy.

METHODS

We conducted an experimental study with 86 medical students (mean 26.03 years, SD 4.71) focusing on history taking in dyspnea cases. Participants solved three cases with SPs and three cases with VPs in this repeated measures study. After each case, students provided a diagnosis and rated perceived authenticity and cognitive load. The provided diagnosis was scored in terms of diagnostic accuracy; the questions asked by the medical students were rated with respect to their quality of evidence generation. In addition to regular null hypothesis testing, this study used equivalence testing to investigate the absence of meaningful effects.

RESULTS

Perceived authenticity (1-tailed t81=11.12; P<.001) was higher for SPs than for VPs. The correlation between diagnostic accuracy and perceived authenticity was very small (r=0.05) and neither equivalent (P=.09) nor statistically significant (P=.32). Cognitive load was equivalent in both assessment methods (t82=2.81; P=.003). Intrinsic cognitive load (1-tailed r=-0.30; P=.003) and extraneous load (1-tailed r=-0.29; P=.003) correlated negatively with the combined score for diagnostic accuracy. The quality of evidence generation was positively related to diagnostic accuracy for VPs (1-tailed r=0.38; P<.001); this finding did not hold for SPs (1-tailed r=0.05; P=.32). Comparing both assessment methods with each other, diagnostic accuracy was higher for SPs than for VPs (2-tailed t85=2.49; P=.01).

CONCLUSIONS

The results on perceived authenticity demonstrate that learners experience SPs as more authentic than VPs. As higher amounts of intrinsic and extraneous cognitive loads are detrimental to performance, both types of cognitive load must be monitored and manipulated systematically in the assessment. Diagnostic accuracy was higher for SPs than for VPs, which could potentially negatively affect students' grades with VPs. We identify and discuss possible reasons for this performance difference between both assessment methods.