Unskilled and unaware in the classroom: College students' desired grades predict their biased grade predictions

Grade prediction in the middle east: a post-pandemic case study of the optimism bias

Evidence exists that the pandemic has brought about stress, and altered study habits and academic performance. No evidence exists regarding whether metacognition has also been altered. The present field study examined the accuracy and confidence with which college students make grade predictions in a general education course after the pandemic. It tested whether one of three types of biases affected students' predictions as a way to cope with the uncertainty of a final exam's outcome: illusion-of-knowing, optimism, and pessimistic bracing. Students made predictions both before and after completing the final exam (summative assessment) to determine the impact of each of the hypothesized biases on estimates made in a context of varying uncertainty. Accuracy was computed as the difference between expected and actual grades on the final exam. Confidence in the predictions made was measured on a Likert scale. Exam performance was categorized as good, poor, or inadequate. In this study, less-than-desirable performance was accompanied by overestimations. However, overestimations were made with little confidence and benefited from the information acquired from completing the exam. This pattern of results suggests that students who are not doing well are not under the spell of the illusion-of-knowing phenomenon. Indeed, their optimistic predictions are punctured by the awareness of a likely undesirable outcome (as indicated by their weak confidence in the predictions made). Implications and applications of these findings are discussed.

Low-Performing Students Confidently Overpredict Their Grade Performance throughout the Semester

When asked to predict how they will perform on an upcoming exam, students are often poorly calibrated, typically in the direction of overpredicting their performance. Research shows that low-performing students' calibration tends to remain poor across multiple tests over the course of a semester. We tested whether these students remain confident in these erroneously high grade predictions across the semester or whether their confidence wanes, suggesting some degree of metacognitive awareness. In two studies, students made grade predictions prior to taking four in-class exams and then rated their level of confidence in their predictions. Results from both studies showed that miscalibration and confidence remained stable across tests, suggesting that low-performing students continued to believe that they would perform well on upcoming exams despite prior evidence to the contrary.

Blocked Presentation Leads Participants to Overutilize Domain Familiarity as a Cue for Judgments of Learning (JOLs)

The accuracy of judgments of learning (JOLs) is vital for efficient self-regulated learning. We examined a situation in which participants overutilize their prior knowledge of a topic ("domain familiarity") as a basis for JOLs, resulting in substantial overconfidence in topics they know the most about. College students rank ordered their knowledge across ten different domains and studied, judged, and then completed a test on facts from those domains. Recall and JOLs were linearly related to self-rated knowledge, as was overconfidence: participants were most overconfident for topics they knew more about, indicating the overutilization of domain familiarity as a cue for JOLs. We examined aspects of the task that might contribute to this pattern, including the order of the task phases and whether participants studied the facts blocked by topic. Although participants used domain familiarity as a cue for JOLs regardless of task design, we found that studying facts from multiple topics blocked by topic led them to overutilize this cue. In contrast, whether participants completed the rank ordering before studying the facts or received a warning about this tendency did not alter the pattern. The relative accuracy of participants' JOLs, however, was not related to domain familiarity under any conditions.

Probing gaps in educational outcomes within the U.S.: A dual moderation multiple mediator latent growth model

Racial/ethnic disparities in math achievement are especially troubling because math proficiency predicts long-term educational outcomes, but the mechanisms underlying these disparities remain unclear. Previous research has demonstrated that across diverse samples, both within and outside the United States, the relation between students' academic aspirations and later postsecondary attainment is mediated by initial levels of math ability and by growth in that ability across time. The key issue examined in this investigation is the extent to which students' underestimation or overestimation of their math ability (i.e., calibration bias) moderates those mediated effects and whether this moderation varies as a function of race/ethnicity. Using data from two longitudinal national surveys (i.e., NELS:88 and HSLS:09), these hypotheses were tested in samples of East Asian American, Mexican American, and Non-Hispanic White American high school students. In both studies and in all groups, the model explained large portions of the variance in postsecondary attainment. In East Asian Americans and non-Hispanic White Americans, calibration bias moderated the effect mediated by 9th grade math achievement. The strength of this effect was greatest at high levels of underconfidence and steadily weakened as self-confidence grew, suggesting that some degree of underconfidence may be achievement-promoting. Indeed, in the East Asian American sample, this effect became negative at high levels of overconfidence (i.e., academic aspirations actually predicted the lowest postsecondary attainment levels). Educational implications of these findings are discussed and possible reasons for the failure to find moderation effects in the Mexican American sample are explored.

Impact of the Appalachian Career Training in Oncology (ACTION) Program on Undergraduate Participants

Kentucky ranks first in the nation in cancer incidence and mortality. The Appalachian region of the state experiences the highest cancer disparities due to inequities in many social determinants of health. As a strategy for addressing cancer and education disparities in the region, the Appalachian Career Training in Oncology (ACTION) Program at the University of Kentucky Markey Cancer Center engaged 16 Appalachian-native undergraduate student participants annually in cancer-related activities. Students were recruited on an annual basis for the 2-year ACTION program. Entry, mid-point, and exit surveys were administered to participants. Classical test theory and exploratory factor analysis were used to examine the instruments used for program evaluation, whereas repeated measures ANOVA, paired t-tests, chi-squared, and post hoc analyses were used to analyze 6 years of survey data. There was a significant increase in participants' understanding of cancer-related topics among the entry, midpoint, and exit surveys (p < .001) and a significant increase in comfort with research, clinical, and outreach activities between entry and midpoint and entry and exit (p < .001), but not between midpoint and exit. With respect to research skills and perceptions of the program, increases in mean scores were observed between midpoint and exit, but these increases were not statistically significant (p = .167, p = 223, respectively). ACTION increased participants' understanding of cancer-related topics; comfort with research, clinical, and outreach activities; and research skills. These data suggest that ACTION has a significant impact on participants.

Too easy, too hard, or just right: Lifespan age differences and gender effects on task difficulty choices

In everyday life, individuals often need to make choices about the difficulty level of tasks they wish to perform. Here, we investigate age- and gender-related differences in the monitoring of discrepancies between the difficulty of a given task and one’s own performance level, and in the likelihood to select task difficulties that match one’s performance level. Male and female children, teenagers, younger adults, and older adults (total N = 160) were asked to play a modified version of the BINGO game. Task difficulty was operationalized as the number of cards played simultaneously. We expected that (a) discrepancies between individuals’ self-selected difficulty levels and their objectively assessed maximum manageable task difficulty (MMTD) would be lowest in early adulthood; (b) children and teenagers, on average, would select relatively difficult task difficulties; and (c) males would overestimate their performance levels, on average, to a greater extent than females. As predicted, younger adults selected task difficulties closest to their MMTD. All other age groups, including older adults, chose task difficulties above their MMTD. The expected gender differences were restricted to children, with boys showing more pronounced performance overestimations than girls. Children and teenagers fluctuated more in their difficulty choices than adults, and many of them, especially boys, occasionally chose difficulty levels far beyond their performance capabilities. We conclude that task-difficulty choices are an interesting topic for lifespan studies. Future research should systematically vary the physical risk involved in a task, and also include the presence of peers.

A Student-Centered, Entrepreneurship Development (ASCEND) Undergraduate Summer Research Program: Foundational Training for Health Research

Increasing the participation of students of African descent and other minoritized populations in the scientific workforce is imperative in generating a more equitable biomedical research infrastructure and increasing national research creativity and productivity. Undergraduate research training programs have shown to be essential tools in retaining underrepresented minority (URM) students in the sciences and attracting them into STEM and biomedical careers. This paper describes an innovative approach to harness students' entrepreneurial desire for autonomy and creativity in a Summer Research Institute (SRI) that has served as an entry point into a multiyear, National Institutes of Health Building Infrastructure Leading to Diversity (NIH BUILD)-funded research training program. The SRI was designed as an 8-week, student-centered and course-based research model in which students select their own research topics. We test here the effects of SRI training on students' science self-efficacy and science identity, along with several other constructs often associated with academic outcomes in the sciences. The data shown here comprise analysis of four different training cohorts throughout four subsequent summers. We show significant gains in students' science self-efficacy and science identity at the conclusion of SRI training, as well as academic adjustment and sense of belonging. SRI participants also displayed substantially improved retention in their science majors and graduation rates.

Non-traditional students' preferences for learning technologies and impacts on academic self-efficacy

Blended Learning (BL) as a pedagogical approach has increased in significance during the COVID-19 pandemic, with blended and online learning environments becoming the new digital norm for higher educational institutions around the globe. While BL has been discussed in the literature for thirty years, a common approach has been to categorise learner cohorts to support educators in better understanding students' relationships with learning technologies. This approach, largely unsupported by empirical evidence, has failed to adequately address the challenges of integrating learning technologies to fit with non-traditional students' preferences, their BL self-efficacy and the associated pedagogical implications. Focusing on student preference, our study presents findings from a pre-COVID survey of undergraduate students across four campuses of an Australian regional university where students shared their learning technology preferences and the self-regulated learning that influenced their academic self-efficacy in a BL context. Findings show students want consistency, relevance, and effectiveness with the use of BL tools, with a preference for lecture recordings and video resources to support their learning, while email and Facebook Messenger were preferred for communicating with peers and academic staff. Our study suggests a quality BL environment facilitates self-regulated learning using fit-for-purpose technological applications. Academic self-efficacy for BL can increase when students perceive the educational technologies used by their institution are sufficient for their learning needs.

Metacognitive errors in the classroom: The role of variability of past performance on exam prediction accuracy

Students often make incorrect predictions about their exam performance, with the lowest-performing students showing the greatest inaccuracies in their predictions. The reasons why low-performing students make inaccurate predictions are not fully understood. In two studies, we tested the hypothesis that low-performing students erroneously predict their exam performance in part because their past performance varies considerably, yielding unreliable data from which to make their predictions. In contrast, high-performing students tend to have consistently high past performance that they can rely on to make relatively accurate predictions of future test performance. Results showed that across different exams (Study 1) and different courses (Study 2), low-performing students had more variable past performance than high-performing students. Further, results from Study 2 showed that variability in past course performance (but not past exam performance) was associated with poor calibration. Results suggest that variability in past performance may be one factor that contributes to low-performing students' erroneous performance predictions.

The effect of simulation-based education before a cadaver dissection course

Although the methods for medical education continue to evolve due to the development of medicines, the cadaver dissection course still plays a fundamental role. The cadaver dissection course allows students to learn to handle instruments correctly while actively exploring three-dimensional anatomy. However, dissection comes with the risk of accidental injury. In recent years, the number of classes offered for the cadaver dissection course has decreased while the amount of knowledge required in clinical medicine has increased. Simulation-based education (SBE) has been proven to be an effective educational method that enhances the development of practical skills by integrating learners' knowledge and skills. This study aimed to investigate the effect of SBE as a preparatory education course when taken prior to a medical student's enrollment in the cadaver dissection course. In the present study, an SBE assuming practical cadaver dissection course was performed in the Clinical Simulation Center. The frequency of injury rates per 1000 h of cadaver dissection course was significantly less in 2017 and 2018 compared to that in 2016. Two years after the implementation of the SBE, average student self-efficacy scores and written examination scores significantly increased, whereas self-contentment scores were relatively unchanged. The results showed that the implementation of SBE decreased the incidence of injuries and improved students' overall self-efficacy scores and increased acquisition of knowledge evident on written examination score. Therefore, SBE as a preparatory education course may effectively promote the combined development of dissection skills and anatomical knowledge in the subsequent fundamental cadaver dissection course.

The Impact of Educational Resources and Perceived Preparedness on Medical Education Performance

INTRODUCTION

Undergraduate medical education has evolved necessarily with the increasing utilization of technology and the availability of ancillary resources developed for medical students. However, medical educational resources are expensive and there have been few studies validating these resources for their ability to significantly modify student exam performance.

METHODS

A post-exam survey was devised to evaluate medical students for resource usage, student-perceived preparedness, and exam performance.

RESULTS

Students who felt more prepared for exams performed better than students who felt less prepared (p = .017). Students who watched didactic lectures online and those who utilized peer-to-peer tutoring outperformed students who did not use these resources (p = .035, p = .008). Analyses of the data show that none of the purchased resources utilized significantly improved student exam performance. The majority of students used between six and eight resources for exam preparation. There may be a slightly negative association with the quantity of resources used and exam scores (p = .18).

DISCUSSION

Contrary to traditional confidence studies that correlate overconfidence with underperformance, medical students who reported feeling more prepared for exams performed better than students who felt less prepared.

CONCLUSION

Medical students may have a more complete grasp of their knowledge base and deficits, which may enable a more accurate match between exam expectations and academic performance. This post-exam survey method can be customized and applied to evaluate resource utility as it pertains to specific undergraduate medical education curricula at individual institutions.

Persistent Miscalibration for Low and High Achievers despite Practice Test Feedback in an Introductory Biology Course

Students' ability to accurately judge their knowledge is crucial for effective learning. However, students' perception of their current knowledge is often misaligned with their actual performance. The relationship between learners' perception of their performance and their actual performance on a task is defined as calibration. Previous studies have shown significant student miscalibration in an introductory biology course: students' predicted exam scores were, on average, significantly higher than their actual scores. The goal of this study was to determine whether completion of a practice test before exams would result in better performance and calibration. The hypothesis was that students who completed a practice test would perform better and be better predictors of their performance on exams than students who did not engage in practice testing. As predicted, students who voluntarily completed a practice test, on average, performed better and were more calibrated than students who did not. Importantly, however, many of the lowest-performing students continued to significantly overestimate their knowledge, predicting higher scores on the exam than they actually earned, despite feedback from practice tests. In contrast, practice testing was associated with underconfidence in high-performing students. These findings indicate that practice tests may enhance calibration for many students. However, additional interventions may be required for the lowest-performing students to become better predictors of their performance.

Self-Enhancement Error Motivates Social Projection

Social projection is the tendency to assume that others are similar to the self, whereas self-enhancement is the tendency to see them as inferior. Although these concepts appear to be in conflict, we suggest that both can stem from the same motive of self-protection. In three studies, we show that respondents overestimate the prevalence of self-enhancement bias in others and predict that most self-enhancement entails an error in judgment. Critically, we find that social projection is strongest among those who learn that they have committed a self-enhancement error. Those who receive feedback saying that their positive self-evaluations are false project this outcome onto others. We also find that self-enhancement errors degrade perceptual accuracy. We discuss how self-protection motives affect both self-evaluation and social projection, and how social judgments that appear inductively rational may stem from arational processes of need satisfaction.

Students’ Self-Efficacy, Causal Attribution Habits and Test Grades

Why do students vary in their performance on exams? It may be that their test preparation is insufficient because they overestimate their anticipated grade. Our study investigates four issues related to performance on a final examination. First, we analyze whether students’ ability to accurately predict their grade and their subjective confidence in this prediction may account for their grade. Second, we ask whether students at different levels of performance vary in their ability to accurately predict their grade, and if so, whether subjective confidence also differs. Third, we ask whether the accuracy and confidence of learners’ predictions are conditioned by self-efficacy beliefs and causal attribution habits, which serve as indices of motivation for test preparation. Fourth, we ask whether different causal attribution preferences contribute to self-efficacy. We use statistical analysis of data from a general education course at a large public university in the United States. Our results indicate that poor performers’ overestimates are likely to be wishful thinking as they are expressed with low subjective confidence. Self-efficacy is a significant contributor to the inaccuracy of students’ predicted grades and subjective confidence in such predictions. Professors’ understanding of learners’ forecasting mechanisms informs strategies devoted to academic success.

Effect of an E-Learning Module on Personal Protective Equipment Proficiency Among Prehospital Personnel: Web-Based Randomized Controlled Trial

Background

To avoid misuse of personal protective equipment (PPE), ensure health care workers’ safety, and avoid shortages, effective communication of up-to-date infection control guidelines is essential. As prehospital teams are particularly at risk of contamination given their challenging work environment, a specific gamified electronic learning (e-learning) module targeting this audience might provide significant advantages as it requires neither the presence of learners nor the repetitive use of equipment for demonstration.

Objective

The aim of this study was to evaluate whether a gamified e-learning module could improve the rate of adequate PPE choice by prehospital personnel in the context of the coronavirus disease (COVID-19) pandemic.

Methods

This was an individual-level randomized, controlled, quadruple-blind (investigators, participants, outcome assessors, and data analysts) closed web-based trial. All emergency prehospital personnel working in Geneva, Switzerland, were eligible for inclusion, and were invited to participate by email in April 2020. Participants were informed that the study aim was to assess their knowledge regarding PPE, and that they would be presented with both the guidelines and the e-learning module, though they were unaware that there were two different study paths. All participants first answered a preintervention quiz designed to establish their profile and baseline knowledge. The control group then accessed the guidelines before answering a second set of questions, and were then granted access to the e-learning module. The e-learning group was shown the e-learning module right after the guidelines and before answering the second set of questions.

Results

Of the 291 randomized participants, 176 (60.5%) completed the trial. There was no significant difference in baseline knowledge between groups. Though the baseline proportion of adequate PPE choice was high (75%, IQR 50%-75%), participants’ description of the donning sequence was in most cases incorrect. After either intervention, adequate choice of PPE increased significantly in both groups (P<.001). Though the median of the difference in the proportion of correct answers was slightly higher in the e-learning group (17%, IQR 8%-33% versus 8%, IQR 8%-33%), the difference was not statistically significant (P=.27). Confidence in the ability to use PPE was maintained in the e-learning group (P=.27) but significantly decreased in the control group (P=.04).

Conclusions

Among prehospital personnel with an already relatively high knowledge of and experience with PPE use, both web-based study paths increased the rate of adequate choice of PPE. There was no major added value of the gamified e-learning module apart from preserving participants' confidence in their ability to correctly use PPE.

Exploring students’ learning efficacy

Purpose

The purpose of this paper is to show the relevance of teachers’ extra role behaviour towards improving students’ learning efficacy status. This study examines the intervening role of art-based teaching pedagogies, i.e. involvement of different forms of art during the traditional teaching session between extra role behaviour and students’ learning efficacy.

Design/methodology/approach

The statistical test results showed that teachers’ extra role behaviour is significant for improving and strengthening students’ learning efficacy. Further, the moderation analysis showed that if art is integrated with teachers’ extra role behaviour, the effect on learning efficacy of students will increase. Art-based teaching pedagogies suggest involvement of art in teaching practices. Multiple regression analysis was conducted to evaluate the direct effect of extra role behaviour on students’ learning efficacy with the intervening role of art-based teaching pedagogies.

Findings

Results indicated a linear effect of teachers’ extra role behaviour on students’ learning efficacy and that art-based teaching pedagogies had an indirect effect (mediation) on students’ learning efficacy.

Originality/value

The study will bridge the gap between academic initiatives taken and its overall implementation in primary and secondary schools.

Opportunities for Self-Evaluation Increase Student Calibration in an Introductory Biology Course

Accurate self-evaluation is critical for learning. Calibration describes the relationship between learners' perception of their performance and their actual performance on a task. Here, we describe two studies aimed at assessing and improving student calibration in a first-semester introductory biology course at a 4-year public institution. Study 1 investigated students' (n = 310) calibration (the difference between estimated and actual exam performance) across one semester. Students were significantly miscalibrated for the first exam: their predicted scores were, on average, significantly higher than their actual scores. The lowest-performing students had the most inaccurate estimates. Calibration improved with each exam. By the final exam, students underestimated their scores. We initiated a second study in the following semester to examine whether explicitly teaching students about self-evaluation strategies would improve their calibration and performance. Instruction in the experimental section (n = 290) focused on students' tendency to overestimate their abilities and provided retrieval-practice opportunities. Students in the experimental section showed better calibration and performance on the first exam compared with students in a control section taught by a different instructor during the same semester (n = 251). These findings suggest that simple instructional strategies can increase students' metacognitive awareness and improve their performance.

Practicing metacognition through the use of repeated posttest predictions

BACKGROUND

Development of metacognitive skills is one method to increase self-awareness of pharmacy students. This study sought to assess students' posttest prediction (postdiction) of performance on a series of multiple-choice examinations to determine if feedback regarding predicted and actual performance could improve personal predictive abilities over time.

IMPACT

While there was a statistically significant change in the students' predictive abilities from examination one to examination three, lower scores in examination two disrupted the trend we had hoped to see. When broken down by overall course score, the highest performing students rarely overestimated their score (5-21% of the time, depending on examination), while the lowest performing students were more varied (22-56% over prediction, depending on examination).

RECOMMENDATIONS

This study used a novel assessment method of postdictions without additional data points such as predictions or grade point average (GPA), which could have helped confirm the value of the method. Additionally, we realized assessing the impact of the qualitative feedback students received could elucidate why and recommend this for future studies.

DISCUSSION

While students were generally poor predictors of their performance, repeated use of this skill helped them to reduce the number of over predictions made by the end of the course. This change was greatest for the lowest performing students indicating that they may receive more benefit from this exercise than higher performing students. This method of using postdictions adds to the collection of tools that can be used to measure student metacognitive skills.

Factors Affecting Student Time to Examination Completion

Objective. To investigate factors (prior or current knowledge, metacognitive accuracy, and personality) that might impact the time it takes students to complete an examination. Methods. On the final examination, the time to complete the examination was recorded. Prior to the course, students completed the five-factor personality assessment. During the semester, students completed four cumulative assessments that included prospective judgments of performance to improve their metacognitive accuracy. Measures of metacognitive accuracy were calculated from the difference between the students' prospective judgments of performance and their actual assessment performance for the final examination. Two weeks prior to the final examination, students completed a cumulative assessment, which served as prior knowledge; this was similar in content to the final examination. Results. The time to complete the final examination was significantly negatively correlated with examination score and positively correlated with Agreeableness, and degree of metacognitive bias. However, only current knowledge (β=-.35) and Agreeableness (β=.12) predicted the time to complete the final examination. These two factors explained about 14% of the variability in completion times. Examining the scale for the time to complete the examination, there were some regional differences between the slowest, intermediate and fastest completers. Conclusion. Current knowledge and to a lesser extent, pro-social behavior (agreeableness) influenced examination completion time. Metacognitive accuracy had limited predictability in time to complete the examination.

Grade Expectations: Rationality and Overconfidence

Confidence and overconfidence are essential aspects of human nature, but measuring (over)confidence is not easy. Our approach is to consider students' forecasts of their exam grades. Part of a student's grade expectation is based on the student's previous academic achievements; what remains can be interpreted as (over)confidence. Our results are based on a sample of about 500 second-year undergraduate students enrolled in a statistics course in Moscow. The course contains three exams and each student produces a forecast for each of the three exams. Our models allow us to estimate overconfidence quantitatively. Using these models we find that students' expectations are not rational and that most students are overconfident, in agreement with the general literature. Less obvious is that overconfidence helps: given the same academic achievement students with larger confidence obtain higher exam grades. Female students are less overconfident than male students, their forecasts are more rational, and they are also faster learners in the sense that they adjust their expectations more rapidly.

Moving from Novice to Expertise and Its Implications for Instruction

Objective: To address the stages of expertise development, what differentiates a novice from an expert, and how the development and differences impact how we teach our classes or design the curriculum. This paper will also address the downside of expertise and discuss the importance of teaching expertise relative to domain expertise. Summary: Experts develop through years of experience and by progressing from novice, advance beginner, proficient, competent, and finally expert. These stages are contingent on progressive problem solving, which means individuals must engage in increasingly complex problems, strategically aligned with the learner's stage of development. Thus, several characteristics differentiate experts from novices. Experts know more, their knowledge is better organized and integrated, they have better strategies for accessing knowledge and using it, and they are self-regulated and have different motivations.

Not by desire alone: The role of cognitive consistency in the desirability bias

We demonstrate that the desirability bias, the elevation of the estimated likelihood of a preferred event, can be due in part to the desire for consistency between the preference for the favored event and its predicted likelihood. An experiment uses a participant’s favorite team in Major League Baseball games and a recently devised method for priming the consistency goal. When preference is the first response, priming cognitive consistency moves prediction toward greater agreement with that preference, thereby increasing the desirability bias. In contrast, when prediction is the first response, priming cognitive consistency facilitates greater agreement with the factual information for each game. This increases the accuracy of the prediction and reduces the desirability bias.