Teaching the control-of-variables strategy: A meta-analysis

Designing Activities to Teach Higher-Order Skills: How Feedback and Constraint Affect Learning of Experimental Design

Active learning approaches to biology teaching, including simulation-based activities, are known to enhance student learning, especially of higher-order skills; nonetheless, there are still many open questions about what features of an activity promote optimal learning. Here we designed three versions of a simulation-based tutorial called Understanding Experimental Design that asks students to design experiments and collect data to test their hypotheses. The three versions vary the experimental design task along the axes of feedback and constraint, where constraint measures how much choice students have in performing a task. Using a variety of assessments, we ask whether each of those features affects student learning of experimental design. We find that feedback has a direct positive effect on learning. We further find that small changes in constraint have only subtle and mostly indirect effects on learning. This work suggests that designers of tools for teaching higher-order skills should strive to include feedback to increase impact and may feel freer to vary the degree of constraint within a range to optimize for other features such as the ability to provide immediate feedback and time-on-task.

Using process features to investigate scientific problem-solving in large-scale assessments

Introduction

This study investigates the process data from scientific inquiry tasks of fair tests [requiring test-takers to manipulate a target variable while keeping other(s) constant] and exhaustive tests (requiring test-takers to construct all combinations of given variables) in the National Assessment of Educational Progress program.

Methods

We identify significant associations between item scores and temporal features of preparation time, execution time, and mean execution time.

Results

Reflecting, respectively, durations of action planning and execution, and execution efficiency, these process features quantitatively differentiate the high- and low-performing students: in the fair tests, high-performing students tended to exhibit shorter execution time than low-performing ones, but in the exhaustive tests, they showed longer execution time; and in both types of tests, high-performing students had shorter mean execution time than low-performing ones.

Discussion

This study enriches process features reflecting scientific problem-solving process and competence and sheds important light on how to improve performance in large-scale, online delivered scientific inquiry tasks.

The complex associations between scientific reasoning and advanced theory of mind

This 6-wave longitudinal study (2014-2018) of 161 German 5- to 10-year-olds from a midsized city and rural area in southern Germany (89 females, 72 males; predominantly White; mostly middle class) found that scientific-reasoning abilities first develop at 6 years. Abilities were highly stable, with the kindergarten score predicting 25% of end-of-elementary-school variance. Individual but not developmental differences were related to language abilities (0.39), mindreading skills (0.33), and parental education (0.36). In early elementary school, mindreading skills predicted scientific reasoning (0.15), but not vice versa; in late elementary school, bidirectional associations emerged (0.11-0.33). Our findings suggest that mindreading is a precursor for the development of scientific reasoning and that older children use scientific reasoning to revise their advanced theories of mind.

Children's failure to control variables may reflect adaptive decision-making

Changing one variable at a time while controlling others is a key aspect of scientific experimentation and a central component of STEM curricula. However, children reportedly struggle to learn and implement this strategy. Why do children's intuitions about how best to intervene on a causal system conflict with scientific practices? Mathematical analyses have shown that controlling variables is not always the most efficient learning strategy, and that its effectiveness depends on the "causal sparsity" of the problem, i.e., how many variables are likely to impact the outcome. We tested the degree to which 7- to 13-year-old children (n = 104) adapt their learning strategies based on expectations about causal sparsity. We report new evidence demonstrating that some previous work may have undersold children's causal learning skills: Children can perform and interpret controlled experiments, are sensitive to causal sparsity, and use this information to tailor their testing strategies, demonstrating adaptive decision-making.

The Role of Prior Knowledge and Intelligence in Gaining from a Training on Proportional Reasoning

We explored the mediating role of prior knowledge on the relation between intelligence and learning proportional reasoning. What students gain from formal instruction may depend on their intelligence, as well as on prior encounters with proportional concepts. We investigated whether a basic curriculum unit on the concept of density promoted students’ learning in a training on proportional reasoning. A 2 × 2 design with the factors basic curriculum unit (with, without) and intervention context to introduce proportional reasoning (speed, density) was applied in two consecutive, randomized classroom studies (N1 = 251, N2 = 566 fourth- and fifth-graders; 49%/56% female). We controlled for intelligence and mathematical achievement. We expected the combination of having received the basic curriculum unit on floating and sinking and proportional reasoning introduced via density (a familiar problem-solving context for this group) to be especially favorable. Whereas this hypothesis was not supported, we showed that mathematical achievement mediated the relation between intelligence and proportional reasoning and enabled learners to better exploit the learning opportunities.

Determining an Evidence Base for Particular Fields of Educational Practice: A Systematic Review of Meta-Analyses on Effective Mathematics and Science Teaching

The call for evidence-based practice in education emphasizes the need for research to provide evidence for particular fields of educational practice. With this systematic literature review we summarize and analyze aggregated effectiveness information from 41 meta-analyses published between 2004 and 2019 to inform evidence-based practice in a particular field. In line with target specifications in education that are provided for a certain school subject and educational level, we developed and adopted a selection heuristic for filtering aggregated effect sizes specific to both science and mathematics education and the secondary student population. The results include 78 context-specific aggregated effect sizes based on data from over one million students. The findings encompass a multitude of different teaching strategies, most of which offer a measurable advantage to alternatives. Findings demonstrate that context-specific effect size information may often differ from more general effect size information on teaching effectiveness and adherence to quality standards varies in sampled meta-analyses. Thus, although meta-analytic research has strongly developed over the last few years, providing context-specific and high-quality evidence still needs to be a focus in the field of secondary mathematics and science teaching and beyond.

Modellierung der Struktur der Variablenkontrollstrategie und Abbildung von Veränderungen in der Grundschule

Die Variablenkontrolle ist bei der Planung und Durchführung von Experimenten von besonderer Bedeutung, weil sie eindeutige Aussagen über Beziehungen zwischen Ursache und Wirkung zulässt. Ihre Anwendung ist daher ein eigenständiges Lernziel des naturwissenschaftlichen Sachunterrichts und Gegenstand zahlreicher empirischer Studien. Entsprechende Fähigkeiten werden unter dem Begriff Variablenkontrollstrategie (VKS) zusammengefasst und beinhalten die vier Teilfähigkeiten: 1) Planung kontrollierter Experimente, 2) Identifizierung kontrollierter Experimente, 3) Interpretation der Ergebnisse kontrollierter Experimente und 4) Verständnis der fehlenden Aussagekraft unkontrollierter Experimente. Bisherige Studien zeigen starke positive Veränderungen bezüglich der VKS während der Grundschulzeit. Allerdings erfassen sie oft nur eine Teilfähigkeit bzw. differenzieren in ihren Analysen nicht zwischen unterschiedlichen Teilfähigkeiten oder dem Einfluss der Fachkontexte der Aufgaben. Wir haben zur Erfassung der VKS in der Grundschule ein Testinstrument im Multiple-Choice-Format entwickelt, welches Aufgaben zu den Teilfähigkeiten Identifizierung und Interpretation in unterschiedlichen Fachkontexten enthält. Das Instrument wurde in einer Querschnittstudie mit N = 415 Zweit- bis Viertklässler*innen eingesetzt. Entgegen bisherigen Befunden zeigen die Ergebnisse einer Rasch-Analyse eine mehrdimensionale Struktur der VKS entsprechend den Teilfähigkeiten. Die Fachkontexte der Aufgaben haben keinen Einfluss auf die Dimensionalität. Die Schwierigkeitsstruktur von Aufgaben wird durch die angesprochene Teilfähigkeit (Identifizierung ist einfacher als Interpretation) und den gewählten Aufgabentyp (z. B. Wahl der Distraktoren nach Schülervorstellungen) beeinflusst. Darüber hinaus wurde eine unterrichtliche Förderung der VKS untersucht (N = 44), um abzuschätzen, inwiefern das entwickelte Testinstrument erwartete Veränderungen hinsichtlich der VKS abbildet. Die gemessenen Veränderungen werden in diesem Beitrag in Relation zur Querschnittsstudie gesetzt. Abschließend werden die Konsequenzen unserer Befunde für die Messung und Förderung der VKS in der Grundschule diskutiert.

Developmental Trajectories in Diagnostic Reasoning: Understanding Data Are Confounded Develops Independently of Choosing Informative Interventions to Resolve Confounded Data

Two facets of diagnostic reasoning related to scientific thinking are recognizing the difference between confounded and unconfounded evidence and selecting appropriate interventions that could provide learners the evidence necessary to make an appropriate causal conclusion (i.e., the control-of-variables strategy). The present study investigates both these abilities in 3- to 6-year-old children (N = 57). We found both competence and developmental progress in the capacity to recognize that evidence is confounded. Similarly, children performed above chance in some tasks testing for the selection of a controlled test of a hypothesis. However, these capacities were unrelated, suggesting that preschoolers' nascent understanding of the control-of-variables strategy may not be driven by a metacognitive understanding that confounded evidence does not support a unique causal conclusion, and requires further investigation.

Measuring and Fostering Preservice Chemistry Teachers’ Scientific Reasoning Competency

Developing scientific reasoning (SR) is a central goal of science-teacher education worldwide. On a fine-grained level, SR competency can be subdivided into at least six skills: formulating research questions, generating hypotheses, planning experiments, observing and measuring, preparing data for analysis, and drawing conclusions. In a study focusing on preservice chemistry teachers, an organic chemistry lab course was redesigned using problem-solving experiments and SR video lessons to foster SR skills. To evaluate the intervention, a self-assessment questionnaire was developed, and a performance-based instrument involving an experimental problem-solving task was adapted to the target group of undergraduates. The treatment was evaluated in a pre-post design with control group (cook-book experiments, no SR video lessons) and alternative treatment group (problem-solving experiments, unrelated video lessons). Interrater reliability was excellent (ρ from 0.915 to 1.000; ICC (A1)). Data analysis shows that the adapted instrument is suitable for university students. First insights from the pilot study indicate that the cook-book lab (control group) only fosters students’ skill in observing and measuring, while both treatment groups show an increase in generating hypotheses and planning experiments. No pretest-posttest differences were found in self-assessed SR skills in the treatment groups. Instruments and data are presented and discussed.

Scientific reasoning and counterfactual reasoning in development

In this chapter, we bridge research on scientific and counterfactual reasoning. We review findings that children struggle with many aspects of scientific experimentation in the absence of formal instruction, but show sophistication in the ability to reason about counterfactual possibilities. We connect these two sets of findings by reviewing relevant theories on the relation between causal, scientific, and counterfactual reasoning before describing a growing body of work that indicates that prompting children to consider counterfactual alternatives can scaffold both the scientific inquiry process (hypothesis-testing and evidence evaluation) and science concept learning. This work suggests that counterfactual thought experiments are a promising pedagogical tool. We end by discussing several open questions for future research.

The Predictive Value of Children's Understanding of Indeterminacy and Confounding for Later Mastery of the Control-of-Variables Strategy

Prior research has identified age 9–11 as a critical period for the development of the control-of-variables strategy (CVS). We examine the stability of interindividual differences in children's CVS skills with regard to their precursor skills during this critical developmental period. To this end, we relate two precursor skills of CVS at age 9 to four skills constituting fully developed CVS more than 2 years later, controlling for children's more general cognitive development. Note that N = 170 second- to fourth-graders worked on multiple choice-assessments of their understanding of indeterminacy of evidence and of confounding. We find relations between these two precursor skills and children's CVS skills 2 years later at age 11 in planning, identifying, and interpreting controlled experiments, and in recognizing the inconclusiveness of confounded comparisons (understanding). In accordance with the perspective that both indeterminacy and confounding constitute critical, related yet distinct elements of CVS, both precursor skills contribute to the prediction of later CVS. Together, the two precursor skills can explain 39% of students' later CVS mastery. Overall, the understanding of indeterminacy is a stronger predictor of fully developed CVS than that of confounding. The understanding of confounding, however, is a better predictor of the more difficult CVS sub-skills of understanding the inconclusiveness of confounded comparisons, and of planning a correctly controlled experiment. Importantly, both precursor skills maintain interactive predictive strength when controlling for children's general cognitive abilities and reading comprehension, showing that the developmental dynamics of CVS and its precursor skills cannot be fully ascribed to general cognitive development. We discuss implications of these findings for theories about the development of CVS and broader scientific reasoning.

Developing an Understanding of Science

Young children are adept at several types of scientific reasoning, yet older children and adults have difficulty mastering formal scientific ideas and practices. Why do “little scientists” often become scientifically illiterate adults? We address this question by examining the role of intuition in learning science, both as a body of knowledge and as a method of inquiry. Intuition supports children's understanding of everyday phenomena but conflicts with their ability to learn physical and biological concepts that defy firsthand observation, such as molecules, forces, genes, and germs. Likewise, intuition supports children's causal learning but provides little guidance on how to navigate higher-order constraints on scientific induction, such as the control of variables or the coordination of theory and data. We characterize the foundations of children's intuitive understanding of the natural world, as well as the conceptual scaffolds needed to bridge these intuitions with formal science.

Individual Differences in Children's Development of Scientific Reasoning Through Inquiry-Based Instruction: Who Needs Additional Guidance?

Scientific reasoning involves a person's ability to think and act in ways that help advance their understanding of the natural world. Young children are naturally inclined to engage in scientific reasoning and display an emerging competence in the component skills of, for example, hypothesizing, experimenting and evaluating evidence. Developmental psychology research has shown that same-age children often differ considerably in their proficiency to perform these skills. Part of this variation comes from individual differences in cognition; another part is due to the fact that the component skills of scientific reasoning emerge at a different age and mature at a different pace. Significantly less attention has been paid to children's capacity to improve in scientific reasoning through instruction and deliberate practice. Although elementary science lessons are generally effective to raise the skill level of a group of learners, not all children benefit equally from the instructional treatment they receive. Knowing what causes this differential effectiveness is important as it can inform the design of adaptive instruction and support. The present study therefore aimed to identify and explain how fifth-graders (N = 138) improve their scientific reasoning skills over the course of a 5-week inquiry-based physics unit. In line with our expectations, significant progress was observed in children's achievements on a written scientific reasoning test, which was administered prior to and after the lessons, as well as in their responses to the questions and assignments that appeared on the worksheets they filled out during each lesson. Children's reading comprehension and mathematical skillfulness explained a portion of the variance in children's pretest-posttest gain. As these overall results did not apply equally to all component skills of scientific reasoning, we recommend science teachers to adapt their lessons based on children's past performance in reading and math and their actual performance of each scientific reasoning skill. The orchestration and relative effectiveness of both adaptive science teaching approaches is an interesting topic for future research.

Lower secondary school students' scientific literacy and their proficiency in identifying and appraising health claims in news media: a secondary analysis using large-scale survey data

OBJECTIVES

Scientific literacy is assumed necessary for appraising the reliability of health claims. Using a national science achievement test, we explored whether students located at the lower quartile on the latent trait (scientific literacy) scale were likely to identify a health claim in a fictitious brief news report, and whether students located at or above the upper quartile were likely to additionally request information relevant for appraising that claim.

DESIGN

Secondary analysis of cross-sectional survey data.

SETTING AND PARTICIPANTS

2229 Norwegian 10th grade students (50% females) from 97 randomly sampled lower secondary schools who performed the test during April-May 2013.

OUTCOME MEASURES

Using Rasch modelling, we linked item difficulty and student proficiency in science to locate the proficiencies associated with different percentiles on the latent trait scale. Estimates of students' proficiency, the difficulty of identifying the claim and the difficulty of making at least one request for information to appraise that claim, were reported in logits.

RESULTS

Students who reached the lower quartile (located at -0.5 logits) on the scale were not likely to identify the health claim as their proficiency was below the difficulty estimate of that task (0.0 logits). Students who reached the upper quartile (located at 1.4 logits) were likely to identify the health claim but barely proficient at making one request for information (task difficulty located at 1.5 logits). Even those who performed at or above the 90th percentile typically made only one request for information, predominantly methodological aspects.

CONCLUSIONS

When interpreting the skill to request relevant information as expressing students' proficiency in critical appraisal of health claims, we found that only students with very high proficiency in science possessed that skill. There is a need for teachers, healthcare professionals and researchers to collaborate to create learning resources for developing these lifelong learning skills.

Guided Hands-On Activities Can Improve Student Learning in a Lecture-Based Qualitative Biomechanics Course

A qualitative biomechanics (functional anatomy) course is a typical course in kinesiology curriculum. Most evidence suggests that biomechanics learning could be improved with the inclusion of laboratory experiences. However, implementing laboratories into biomechanics curriculum is difficult due to cost and time constraints. This study was conducted to evaluate whether hands-on activities in lecture improve qualitative biomechanics learning. A lecture format was compared to the same course with guided and unguided hands-on activities included during lecture. Test performance and student evaluations were compared between lecture formats to determine if hands-on experiences improve learning. The hands-on group performed better on the same test questions and they evaluated their overall course activities as beneficial to their learning. The findings suggest that guided hands-on experiences may improve learning compared to unguided activities. The hands-on experiences seem to provide an embodied cognitive learning experience, facilitating retention of learned material through three-dimensional and tactile mental representations. Findings from this research are currently shaping how biomechanics is taught to students at this university and could at other universities as well.

Self-Generation in the Context of Inquiry-Based Learning

Self-generation of knowledge can activate deeper cognitive processing and improve long-term retention compared to the passive reception of information. It plays a distinctive role within the concept of inquiry-based learning, which is an activity-oriented, student-centered collaborative learning approach in which students become actively involved in knowledge construction by following an idealized hypothetico-deductive method. This approach allows students to not only acquire content knowledge, but also an understanding of investigative procedures/inquiry skills - in particular the control-of-variables strategy (CVS). From the perspective of cognitive load theory, generating answers and solutions during inquiry-based learning is inefficient as it imposes an intrinsic and extraneous load on learners. Previous research on self-generation of content knowledge in inquiry-based learning has demonstrated that (1) a high cognitive load impairs retention of the generated information, (2) feedback is a fundamental requirement for self-generation of complex content knowledge, (3) self-generation success is key to long-term retention, and (4) generating and rereading place different demands on learners. However, there is still no research on the self-generation of scientific reasoning skills (procedural knowledge) and no knowledge of interaction between the (long-term) retention of these skills with prior knowledge, feedback and self-generation success. That is why this experiment was conducted. The focus of this research is to analyze the distinctive role of self-generation of scientific reasoning skills within the concept of inquiry-based learning and to identify the influence of prior knowledge and self-generation success on short-term and long-term retention. For this purpose, an experiment involving 133 6th and 7th graders was conducted. An inquiry activity that included the self-generation of scientific reasoning skills was compared to an inquiry task that had students simply read information about the experimental design. We used both an immediate and a delayed test to examine which treatment better developed a deeper understanding of CVS and an ability to apply this knowledge to novel problems (transfer). Direct instruction was clearly superior to self-generation in facilitating students' acquisition of CVS immediately after the inquiry task. However, after a period of 1 week had elapsed, both treatment conditions turned out to be equally effective. A generation effect was only found among students with high self-generation success after a 1-week delay.