Student Perceptions of a Framework for Facilitating Transfer from Lessons to Exams, and the Relevance of This Framework to Published Lessons

Exam reform: an opportunity for the redistribution of academic power

Students in STEM know well the stress, challenge, and effort that accompany college exams. As a widely recognizable feature of the STEM classroom experience, high-stakes assessments serve as crucial cultural gateways in shaping both preparation and motivation for careers. In this essay, we identify and discuss issues of power around STEM exams to further the understanding of exam practices that can unjustly hold students back. Through theory and practical examples, we consider the numerous ways in which power manifests both on and off the pages of the exams themselves, as well as ways in which power is consolidated away from students through logistical norms and tradeoffs. Centering the "rules of the culture of power" as delineated by Dr. Lisa Delpit, we reflect on exam practices that prioritize faculty voice and faculty convenience above student learning and student identity. We share some of what we have learned from our students as part of a call to improve STEM education by relinquishing some of our exam-related power over students, redistributing it to students so that they have more power to shape their own education.

Chatbot responses suggest that hypothetical biology questions are harder than realistic ones

The biology education literature includes compelling assertions that unfamiliar problems are especially useful for revealing students' true understanding of biology. However, there is only limited evidence that such novel problems have different cognitive requirements than more familiar problems. Here, we sought additional evidence by using chatbots based on large language models as models of biology students. For human physiology and cell biology, we developed sets of realistic and hypothetical problems matched to the same lesson learning objectives (LLOs). Problems were considered hypothetical if (i) known biological entities (molecules and organs) were given atypical or counterfactual properties (redefinition) or (ii) fictitious biological entities were introduced (invention). Several chatbots scored significantly worse on hypothetical problems than on realistic problems, with scores declining by an average of 13%. Among hypothetical questions, redefinition questions appeared especially difficult, with many chatbots scoring as if guessing randomly. These results suggest that, for a given LLO, hypothetical problems may have different cognitive demands than realistic problems and may more accurately reveal students' ability to apply biology core concepts to diverse contexts. The Test Question Templates (TQT) framework, which explicitly connects LLOs with examples of assessment questions, can help educators generate problems that are challenging (due to their novelty), yet fair (due to their alignment with pre-specified LLOs). Finally, ChatGPT's rapid improvement toward expert-level answers suggests that future educators cannot reasonably expect to ignore or outwit chatbots but must do what we can to make assessments fair and equitable.

Using Test Question Templates to teach physiology core concepts

The past ∼15 years have seen increasing interest in defining disciplinary core concepts. Within the field of physiology, Michael, McFarland, Modell, and colleagues have published studies that defined physiology core concepts and have elaborated many of these as detailed conceptual frameworks. With such helpful definitions now in place, attention is turning to the related issue of how to maximize student understanding of the core concepts by linking these "big ideas" to concrete student-facing resources for active learning and assessment. Our practitioner-based view begins with the recognition that in many if not most undergraduate physiology courses assessment drives learning. We have therefore linked published conceptual frameworks to Test Question Templates (TQTs), whose structure promotes transparent assessments as well as the active learning needed to prepare for such assessments. We provide examples of conceptual framework-linked TQTs for the physiology core concepts of Homeostasis, Flow Down Gradients, the Cell Membrane, and Cell-Cell Communication. We argue that this deployment of TQTs has at least two distinct benefits for the teaching and learning of core concepts. First, documenting the connections between conceptual frameworks and TQTs may clarify coverage and assessment of the core concepts for both instructors and students. Second, misconceptions about core concepts may be directly targeted and dispelled via thoughtful construction, arrangement, and iteration of TQTs. We propose that the TQT framework or similar approaches may be applied fruitfully to any sufficiently articulated physiology core concept for high school, undergraduate, or graduate students.NEW & NOTEWORTHY Our students often focus on the grades they need to advance through academic programs. How can instructors harness this understandable interest in grades to help students gain a true understanding of core concepts? The new framework of Test Question Templates (TQTs) shows promise in linking student priorities like test scores to instructor priorities like core concepts.