A critical analysis of assessment quality in genomics and bioinformatics education research

"It's More Of A Me-Thing Than An Evolution Thing": Exploring The Validity Of Evolution Acceptance Measures Using Student Interviews

Researchers who study student acceptance of evolution rely on surveys that are designed to measure evolution acceptance. It is important for these surveys to measure evolution acceptance accurately and in isolation from other constructs, so that researchers can accurately determine what leads to low acceptance. The Inventory of Student Evolution Acceptance (I-SEA) and the Generalized Acceptance of EvolutioN Evaluation (GAENE) are two surveys that were developed to improve upon the limitations of earlier surveys. Yet neither survey has been extensively tested for response process validity, which can assess the extent to which students use constructs other than their acceptance of evolution to answer survey items. In this study, we examined the response-process validity of the I-SEA and GAENE by conducting cognitive interviews with 60 undergraduate students. Interviews revealed that both surveys retain certain response-process issues. The I-SEA conflated knowledge about and acceptance of evolution for a subset of students. The GAENE measured evolution acceptance inconsistently because students interpreted "evolution" in different ways; it also measured willingness to advocate for evolution in addition to acceptance. Researchers can use these findings to better inform their survey choice when designing future studies, and to further improve the measurement of evolution acceptance.

The EvMed Assessment: A test for measuring student understanding of core concepts in evolutionary medicine

Background and objectives

Universities throughout the USA increasingly offer undergraduate courses in evolutionary medicine (EvMed), which creates a need for pedagogical resources. Several resources offer course content (e.g. textbooks) and a previous study identified EvMed core principles to help instructors set learning goals. However, assessment tools are not yet available. In this study, we address this need by developing an assessment that measures students' ability to apply EvMed core principles to various health-related scenarios.

Methodology

The EvMed Assessment (EMA) consists of questions containing a short description of a health-related scenario followed by several likely/unlikely items. We evaluated the assessment's validity and reliability using a variety of qualitative (expert reviews and student interviews) and quantitative (Cronbach's α and classical test theory) methods. We iteratively revised the assessment through several rounds of validation. We then administered the assessment to undergraduates in EvMed and Evolution courses at multiple institutions.

Results

We used results from the pilot to create the EMA final draft. After conducting quantitative validation, we deleted items that failed to meet performance criteria and revised items that exhibited borderline performance. The final version of the EMA consists of six core questions containing 25 items, and five supplemental questions containing 20 items.

Conclusions and implications

The EMA is a pedagogical tool supported by a wide range of validation evidence. Instructors can use it as a pre/post measure of student learning in an EvMed course to inform curriculum revision, or as a test bank to draw upon when developing in-class assessments, quizzes or exams.

A lack of population structure characterizes the invasive Lonicera japonica in West Virginia and across eastern North America

Invasive plant species cause massive ecosystem damage globally, yet represent powerful case studies in population genetics and rapid adaptation to new habitats. The availability of digitized herbarium collections data, and the ubiquity of invasive species across the landscape make them highly accessible for studies of invasion history and population dynamics associated with their introduction, establishment, spread, and ecological interactions. Here we focus on Lonicera japonica, one of the most damaging invasive vine species in North America. We leveraged digitized collections data and contemporary field collections to reconstruct the invasion history and characterize patterns of genomic variation in the eastern USA, using a straightforward method for generating nucleotide polymorphism data and a recently published, chromosome-level genome for the species. We found an overall lack of population structure among sites in northern West Virginia, USA, as well as across sites in the central and eastern USA. Heterozygosity and population differentiation were both low based on Fst, analysis of molecular variance, principal components analysis, and cluster-based analyses. We also found evidence of high inbreeding coefficients and significant linkage disequilibrium, in line with the ability of this otherwise outcrossing, perennial species to propagate vegetatively. Our findings corroborate earlier studies based on allozyme data, and suggest that intentional, human-assisted spread explains the lack of population structure, as this species was planted for erosion control and as an ornamental, escaping cultivation repeatedly across the USA. Finally, we discuss how plant invasion genomics can be incorporated into experiential undergraduate education as a way to integrate teaching and research.

The history and organization of the Workshop on Population and Speciation Genomics

With the advent of high-throughput genome sequencing, bioinformatics training has become essential for research in evolutionary biology and related fields. However, individual research groups are often not in the position to teach students about the most up-to-date methodology in the field. To fill this gap, extended bioinformatics courses have been developed by various institutions and provide intense training over the course of two or more weeks. Here, we describe our experience with the organization of a course in one of the longest-running extended bioinformatics series of workshops, the Evomics Workshop on Population and Speciation Genomics that takes place biennially in the UNESCO world heritage town of Český Krumlov, Czech Republic. We list the key ingredients that make this workshop successful in our view, explain the routine for workshop organization that we have optimized over the years, and describe the most important lessons that we have learned from it. We report the results of a survey conducted among past workshop participants that quantifies measures of effective teaching and provide examples of how the workshop setting has led to the cross-fertilisation of ideas and ultimately scientific progress. We expect that our account may be useful for other groups aiming to set up their own extended bioinformatics courses.

Developing the BETTSI: A tree-thinking diagnostic tool to assess individual elements of representational competence

Phylogenies are a ubiquitous visual representation of core concepts in evolutionary biology and it is important that students develop an ability to read and correctly interpret these diagrams. However, as with any representation of complex disciplinary information, learning to correctly interpret phylogenies can be challenging, requiring that a diversity of educational strategies be deployed. Representational competence is the ability to develop and effectively use abstract representations. Accurately interpreting a phylogenetic tree as a presentation of evolutionary relationships requires that students develop general representational competence as well as knowledge of specific technical aspects of tree interpretation, such as knowing the graphical components of trees and what they represent. Here, we report on the development of a basic diagnostic tool of students' representational competence and technical skills with phylogenies, the Basic Evolutionary Tree-Thinking Skills Instrument (BETTSI). This short, multiple-choice instrument was designed to provide instructors with a quick diagnostic of students' ability to read and interpret phylogenies. It has been checked for reliability and validity and provides a convenient formative and summative assessment of students' understanding of evolutionary trees.

There Is More than Multiple Choice: Crowd-Sourced Assessment Tips for Online, Hybrid, and Face-to-Face Environments

Developing effective assessments of student learning is a challenging task for faculty and even more difficult for those in emerging disciplines that lack readily available resources and standards. With the power of technology-enhanced education and accessible digital learning platforms, instructors are also looking for assessments that work in an online format. This article will be useful for all teachers, but especially for entry-level instructors, in addition to more mature instructors who are looking to become more well versed in assessment, who seek a succinct summary of assessment types to springboard the integration of new forms of assessment of student learning into their courses. In this paper, ten assessment types, all appropriate for face-to-face, blended, and online modalities, are discussed. The assessments are mapped to a set of bioinformatics core competencies with examples of how they have been used to assess student learning. Although bioinformatics is used as the focus of the assessment types, the question types are relevant to many disciplines.

Are Faculty Changing? How Reform Frameworks, Sampling Intensities, and Instrument Measures Impact Inferences about Student-Centered Teaching Practices

Although recent studies have used the Classroom Observation Protocol for Undergraduate STEM (COPUS) to make claims about faculty reform, important questions remain: How should COPUS measures be situated within existing reform frameworks? Is there a universal sampling intensity that allows for valid inferences about the frequency of student-centered instruction within a semester or across semesters of a course? These questions were addressed using longitudinal COPUS observations (128 classes, three faculty, 4 years). COPUS behaviors were used to categorize classes into didactic, interactive lecture, or student-centered instructional styles. Sampling intensities (one to 11 classes) were simulated (1000 times) within a course and across semesters. The sampling intensities required for generating valid inferences about 1) the presence of student-centered instruction and 2) the proportion of instructional styles in a course and through time were calculated. Results indicated that the sampling intensity needed to characterize courses and instructors varied and was much higher than previously recommended for instructors with: 1) rare instances of student-centered classes, 2) variability in instructional style, and 3) longitudinal changes in instructional patterns. These conditions are common in early reform contexts. This study highlights the risks of broad, decontextualized sampling protocol recommendations and illustrates how reform frameworks, sampling intensities, and COPUS measures interact to impact inferences about faculty change.

Illuminating the complexities of conflict with evolution: validation of the scales of evolutionary conflict measure (SECM)

BACKGROUND

Although personal, familial, and community conflict with evolution have been documented in the literature, these scales require conceptualization as a construct and operationalization as a measure. The Scales of Conflict with Evolution Measure (SECM) instrument was developed in response to these needs. Using a construct validity framework, the content, internal structure, convergent, and substantive validity of the SECM were evaluated using Rasch analysis, Structural Equation Modeling (SEM), and follow up questioning. The conceptual utility of the instrument was explored by examining whether it added explanatory insights into evolution acceptance above and beyond religiosity, evolution knowledge, and background variables.

RESULTS

A literature review and expert consultation indicated that construct of evolutionary conflict perception should (i) encompass the hierarchical nature of human social structures (personal, family, community) and (ii) probe conflict as it relates to human values, cultures, and beliefs. A three-dimensional construct was operationalized as a nine-item rating scale measure. Using Rasch analyses of SECM responses from a diverse sample of > 1000 students studying evolution, the instrument met criteria of robust measurement, including: fit to model expectations; three-dimensional structure; high reliability; good rating scale function; measurement invariance with time; and convergence with a similar construct. SEM showed that: (i) family and community conflict had unique causal contributions to personal conflict, with family showing a stronger and modest impact, and (ii) personal conflict had a significant and modest causal impact on evolution acceptance above and beyond the contributions of religiosity, evolution knowledge, and background variables.

CONCLUSION

The SECM is an easy-to-administer instrument to measure conflict with evolution and is supported by several forms of validity evidence. The SECM has potential for facilitating measurement of evolutionary conflict in educational settings, thereby raising instructor awareness of conflict levels in students, promoting rigorous evaluations of educational interventions designed to reduce conflict, and fostering conceptual advances in the field of evolution education. Future work is needed to gather additional forms of validity evidence and to test current validity claims in additional participant samples. SECM measures should also be incorporated into more complex SEM models that treat evolution knowledge and religiosity as part of the structural paths to evolution acceptance. Such models could provide insights into the most worthwhile targets for the development of educational interventions to mitigate conflict at multiple scales.

Bioinformatics-Based Activities in High School: Fostering Students' Literacy, Interest, and Attitudes on Gene Regulation, Genomics, and Evolution

The key role of bioinformatics in explaining biological phenomena calls for the need to rethink didactic approaches at high school aligned with a new scientific reality. Despite several initiatives to introduce bioinformatics in the classroom, there is still a lack of knowledge on their impact on students' learning gains, engagement, and motivation. In this study, we detail the effects of four bioinformatics laboratories tailored for high school biology classes named "Mining the Genome: Using Bioinformatics Tools in the Classroom to Support Student Discovery of Genes" on literacy, interest, and attitudes on 387 high school students. By exploring these laboratories, students get acquainted with bioinformatics and acknowledge that many bioinformatics tools can be intuitive for beginners. Furthermore, introducing comparative genomics in their learning practices contributed for a better understanding of curricular contents regarding the identification of genes, their regulation, and how to make evolutionary assumptions. Following the intervention, students were able to pinpoint bioinformatics tools required to identify genes in a genomics sequence, and most importantly, they were able to solve genomics-related misconceptions. Overall, students revealed a positive attitude regarding the integration of bioinformatics-based approaches in their learning practices, reinforcing their added value in educational approaches.

Development of the Biological Variation In Experimental Design And Analysis (BioVEDA) assessment

Variation is an important concept that underlies experimental design and data analysis. Incomplete understanding of variation can preclude students from designing experiments that adequately manage organismal and experimental variation, and from accurately conducting and interpreting statistical analyses of data. Because of the lack of assessment instruments that measure students' ideas about variation in the context of biological investigations, we developed the Biological Variation in Experimental Design and Analysis (BioVEDA) assessment. Psychometric analyses indicate that BioVEDA assessment scores are reliable/precise. We provide evidence that the BioVEDA instrument can be used to evaluate students' understanding of biological variation in the context of experimental design and analysis relative to other students and to their prior scores.

Entering Research Learning Assessment (ERLA): Validity Evidence for an Instrument to Measure Undergraduate and Graduate Research Trainee Development

Expanding the scope of previous undergraduate research assessment tools, the Entering Research Learning Assessment (ERLA) measures undergraduate and graduate research trainee learning gains in the seven areas of trainee development in the evidence-based Entering Research conceptual framework: Research Comprehension and Communication Skills, Practical Research Skills, Research Ethics, Researcher Identity, Researcher Confidence and Independence, Equity and Inclusion Awareness and Skills, and Professional and Career Development Skills. In this paper, we present multiple sources of validity evidence for the ERLA trainee self-assessment and mentor assessment of trainee learning gains. Evidence of internal structure of the initial scales via exploratory factor analysis (Ntrainees = 193; Nmentors = 130) revealed seven factors that align with the Entering Research conceptual framework. Validity evidence for internal structure using confirmatory factor analysis, convergent validity, and evidence of internal consistency for the revised scale were examined with a larger sample (Ntrainees = 489; Nmentors = 256). Evidence of internal structure and alignment for a paired version of the ERLA was also examined with a subset of the original sample (N = 121 pairs). Each analysis revealed acceptable model-data fit. Guidance on using the ERLA instruments and interpreting their scores is presented.

Degrees of freedom analysis in educational research and decision-making: leveraging qualitative data to promote excellence in bioinformatics training and education

Qualitative data are commonly collected in higher, graduate and postgraduate education; however, perhaps especially in the quantitative sciences, utilization of these qualitative data for decision-making can be challenging. A method for the analysis of qualitative data is the degrees of freedom analysis (DoFA), published in 1975. Given its origins in political science and its application in mainly business contexts, the DoFA method is unlikely to be discoverable or used to understand survey or other educational data obtained from teaching, training or evaluation. This article therefore introduces and demonstrates the DoFA with modifications specifically to support educational research and decision-making with examples in bioinformatics. DoFA identifies and aligns theoretical or applied principles with qualitative evidence. The demonstrations include two hypothetical examples, and a case study of the role of scaffolding in an independent project ('capstone') of a graduate course in biostatistics. Included to promote inquiry, inquiry-based learning and the development of research skills, the capstone is often scaffolded (instructor-supported and therefore, formative), although it is intended to be summative. The case analysis addresses the question of whether the scaffolding provided for a capstone assignment affects its utility for formative or summative assessment. The DoFA is also used to evaluate the relative efficacies of other models for scaffolding the capstone project. These examples are intended to both explain this method and to demonstrate how it can be used to make decisions within a curriculum or for bioinformatics training.

A Biology Core Concept Instrument (BCCI) to Teach and Assess Student Conceptual Understanding

Instruments for teaching and assessing student understanding of the five core concepts in biology from Vision and Change are needed. We developed four Biology Core -Concept Instruments (BCCIs) that teach and assess students' ability to describe a concept in their own words, identify concepts represented in biological phenomena, and make connections between concepts. The BCCI includes a narrative, followed by a series of 10 true-false/identify (TF/I) and three open-ended questions. The TF/I questions are aligned with Cary and Branchaw's Conceptual Elements Framework and were iteratively developed with feedback from biology experts and student performance and feedback obtained during think-aloud interviews. A component scoring system was developed to discriminate between a student's ability to apply and identify each core concept from his or her ability to make connections between concepts. We field-tested the BCCIs (n = 152-191) with students in a first-year course focused on learning the five core concepts in biology and collected evidence of interrater reliability (α = 0.70) and item validity. With component scoring, we identified examples in which students were able to identify concepts singularly, but not make connections between concepts, or were better able to apply concepts to one biological phenomenon than another. Identifying these nuanced differences in learning can guide instruction to improve students' conceptual understanding.

Big Data to the Bench: Transcriptome Analysis for Undergraduates

Next-generation sequencing (NGS)-based methods are revolutionizing biology. Their prevalence requires biologists to be increasingly knowledgeable about computational methods to manage the enormous scale of data. As such, early introduction to NGS analysis and conceptual connection to wet-lab experiments is crucial for training young scientists. However, significant challenges impede the introduction of these methods into the undergraduate classroom, including the need for specialized computer programs and knowledge of computer coding. Here, we describe a semester-long, course-based undergraduate research experience at a liberal arts college combining RNA-sequencing (RNA-seq) analysis with student-driven, wet-lab experiments to investigate plant responses to light. Students derived hypotheses based on analysis of RNA-seq data and designed follow-up studies of gene expression and plant growth. Our assessments indicate that students acquired knowledge of big data analysis and computer coding; however, earlier exposure to computational methods may be beneficial. Our course requires minimal prior knowledge of plant biology, is easy to replicate, and can be modified to a shorter, directed-inquiry module. This framework promotes exploration of the links between gene expression and phenotype using examples that are clear and tractable and improves computational skills and bioinformatics self-efficacy to prepare students for the "big data" era of modern biology.

Classroom as Genome: Using the Tools of Genomics and Bioinformatics to Illuminate Classroom Observation Data

Classroom observation protocols can provide an exceedingly rich form of data. However, this is a double-edged sword, as researchers often struggle to take full advantage of the detailed data outputs. In this essay, we introduce a new approach to the analysis of classroom observation data, termed "classroom as genome" (CAG). We illustrate how real-time classroom observation data and genomic data can be viewed as quite analogous, both conceptually and in terms of downstream analysis. We provide both abstract and concrete examples of how the tools of genomics and bioinformatics can be applied to classroom observation outputs. We also show how this philosophy of analysis allows for the layering of information from multiple observation protocols onto the same classroom data. The CAG approach enables biology education researchers to explore detailed patterns within observed classrooms in a highly scalable manner.

Error-Discovery Learning Boosts Student Engagement and Performance, while Reducing Student Attrition in a Bioinformatics Course

We sought to test a hypothesis that systemic blind spots in active learning are a barrier both for instructors-who cannot see what every student is actually thinking on each concept in each class-and for students-who often cannot tell precisely whether their thinking is right or wrong, let alone exactly how to fix it. We tested a strategy for eliminating these blind spots by having students answer open-ended, conceptual problems using a Web-based platform, and measured the effects on student attrition, engagement, and performance. In 4 years of testing both in class and using an online platform, this approach revealed (and provided specific resolution lessons for) more than 200 distinct conceptual errors, dramatically increased average student engagement, and reduced student attrition by approximately fourfold compared with the original lecture course format (down from 48.3% to 11.4%), especially for women undergraduates (down from 73.1% to 7.4%). Median exam scores increased from 53% to 72-80%, and the bottom half of students boosted their scores to the range in which the top half had scored before the pedagogical switch. By contrast, in our control year with the same active-learning content (but without this "zero blind spots" approach), these gains were not observed.

EcoEvo-MAPS: An Ecology and Evolution Assessment for Introductory through Advanced Undergraduates

A new assessment tool, Ecology and Evolution-Measuring Achievement and Progression in Science or EcoEvo-MAPS, measures student thinking in ecology and evolution during an undergraduate course of study. EcoEvo-MAPS targets foundational concepts in ecology and evolution and uses a novel approach that asks students to evaluate a series of predictions, conclusions, or interpretations as likely or unlikely to be true given a specific scenario. We collected evidence of validity and reliability for EcoEvo-MAPS through an iterative process of faculty review, student interviews, and analyses of assessment data from more than 3000 students at 34 associate's-, bachelor's-, master's-, and doctoral-granting institutions. The 63 likely/unlikely statements range in difficulty and target student understanding of key concepts aligned with the Vision and Change report. This assessment provides departments with a tool to measure student thinking at different time points in the curriculum and provides data that can be used to inform curricular and instructional modifications.

Young Adults' Belief in Genetic Determinism, and Knowledge and Attitudes towards Modern Genetics and Genomics: The PUGGS Questionnaire

In this paper we present the development and validation a comprehensive questionnaire to assess college students’ knowledge about modern genetics and genomics, their belief in genetic determinism, and their attitudes towards applications of modern genetics and genomic-based technologies. Written in everyday language with minimal jargon, the Public Understanding and Attitudes towards Genetics and Genomics (PUGGS) questionnaire is intended for use in research on science education and public understanding of science, as a means to investigate relationships between knowledge, determinism and attitudes about modern genetics, which are to date little understood. We developed a set of core ideas and initial items from reviewing the scientific literature on genetics and previous studies on public and student knowledge and attitudes about genetics. Seventeen international experts from different fields (e.g., genetics, education, philosophy of science) reviewed the initial items and their feedback was used to revise the questionnaire. We validated the questionnaire in two pilot tests with samples of university freshmen students. The final questionnaire contains 45 items, including both multiple choice and Likert scale response formats. Cronbach alpha showed good reliability for each section of the questionnaire. In conclusion, the PUGGS questionnaire is a reliable tool for investigating public understanding and attitudes towards modern genetics and genomic-based technologies.

Development of a Biological Science Quantitative Reasoning Exam (BioSQuaRE)

Multiple reports highlight the increasingly quantitative nature of biological research and the need to innovate means to ensure that students acquire quantitative skills. We present a tool to support such innovation. The Biological Science Quantitative Reasoning Exam (BioSQuaRE) is an assessment instrument designed to measure the quantitative skills of undergraduate students within a biological context. The instrument was developed by an interdisciplinary team of educators and aligns with skills included in national reports such as BIO2010, Scientific Foundations for Future Physicians, and Vision and Change Undergraduate biology educators also confirmed the importance of items included in the instrument. The current version of the BioSQuaRE was developed through an iterative process using data from students at 12 postsecondary institutions. A psychometric analysis of these data provides multiple lines of evidence for the validity of inferences made using the instrument. Our results suggest that the BioSQuaRE will prove useful to faculty and departments interested in helping students acquire the quantitative competencies they need to successfully pursue biology, and useful to biology students by communicating the importance of quantitative skills. We invite educators to use the BioSQuaRE at their own institutions.

Making authentic science accessible-the benefits and challenges of integrating bioinformatics into a high-school science curriculum

Despite the central place held by bioinformatics in modern life sciences and related areas, it has only recently been integrated to a limited extent into high-school teaching and learning programs. Here we describe the assessment of a learning environment entitled 'Bioinformatics in the Service of Biotechnology'. Students' learning outcomes and attitudes toward the bioinformatics learning environment were measured by analyzing their answers to questions embedded within the activities, questionnaires, interviews and observations. Students' difficulties and knowledge acquisition were characterized based on four categories: the required domain-specific knowledge (declarative, procedural, strategic or situational), the scientific field that each question stems from (biology, bioinformatics or their combination), the associated cognitive-process dimension (remember, understand, apply, analyze, evaluate, create) and the type of question (open-ended or multiple choice). Analysis of students' cognitive outcomes revealed learning gains in bioinformatics and related scientific fields, as well as appropriation of the bioinformatics approach as part of the students' scientific 'toolbox'. For students, questions stemming from the 'old world' biology field and requiring declarative or strategic knowledge were harder to deal with. This stands in contrast to their teachers' prediction. Analysis of students' affective outcomes revealed positive attitudes toward bioinformatics and the learning environment, as well as their perception of the teacher's role. Insights from this analysis yielded implications and recommendations for curriculum design, classroom enactment, teacher education and research. For example, we recommend teaching bioinformatics in an integrative and comprehensive manner, through an inquiry process, and linking it to the wider science curriculum.

A vision for collaborative training infrastructure for bioinformatics

In biology, a missing link connecting data generation and data-driven discovery is the training that prepares researchers to effectively manage and analyze data. National and international cyberinfrastructure along with evolving private sector resources place biologists and students within reach of the tools needed for data-intensive biology, but training is still required to make effective use of them. In this concept paper, we review a number of opportunities and challenges that can inform the creation of a national bioinformatics training infrastructure capable of servicing the large number of emerging and existing life scientists. While college curricula are slower to adapt, grassroots startup-spirited organizations, such as Software and Data Carpentry, have made impressive inroads in training on the best practices of software use, development, and data analysis. Given the transformative potential of biology and medicine as full-fledged data sciences, more support is needed to organize, amplify, and assess these efforts and their impacts.

Contemporary Test Validity in Theory and Practice: A Primer for Discipline-Based Education Researchers

Most discipline-based education researchers (DBERs) were formally trained in the methods of scientific disciplines such as biology, chemistry, and physics, rather than social science disciplines such as psychology and education. As a result, DBERs may have never taken specific courses in the social science research methodology--either quantitative or qualitative--on which their scholarship often relies so heavily. One particular aspect of (quantitative) social science research that differs markedly from disciplines such as biology and chemistry is the instrumentation used to quantify phenomena. In response, this Research Methods essay offers a contemporary social science perspective on test validity and the validation process. The instructional piece explores the concepts of test validity, the validation process, validity evidence, and key threats to validity. The essay also includes an in-depth example of a validity argument and validation approach for a test of student argument analysis. In addition to DBERs, this essay should benefit practitioners (e.g., lab directors, faculty members) in the development, evaluation, and/or selection of instruments for their work assessing students or evaluating pedagogical innovations.

A Primer for Developing Measures of Science Content Knowledge for Small-Scale Research and Instructional Use

The credibility of conclusions made about the effectiveness of educational interventions depends greatly on the quality of the assessments used to measure learning gains. This essay, intended for faculty involved in small-scale projects, courses, or educational research, provides a step-by-step guide to the process of developing, scoring, and validating high-quality content knowledge assessments. We illustrate our discussion with examples from our assessments of high school students' understanding of concepts in cell biology and epigenetics. Throughout, we emphasize the iterative nature of the development process, the importance of creating instruments aligned to the learning goals of an intervention or curricula, and the importance of collaborating with other content and measurement specialists along the way.

Redesigning a General Education Science Course to Promote Critical Thinking

Recent studies question the effectiveness of a traditional university curriculum in helping students improve their critical thinking and scientific literacy. We developed an introductory, general education (gen ed) science course to overcome both deficiencies. The course, titled Foundations of Science, differs from most gen ed science offerings in that it is interdisciplinary; emphasizes the nature of science along with, rather than primarily, the findings of science; incorporates case studies, such as the vaccine-autism controversy; teaches the basics of argumentation and logical fallacies; contrasts science with pseudoscience; and addresses psychological factors that might otherwise lead students to reject scientific ideas they find uncomfortable. Using a pretest versus posttest design, we show that students who completed the experimental course significantly improved their critical-thinking skills and were more willing to engage scientific theories the general public finds controversial (e.g., evolution), while students who completed a traditional gen ed science course did not. Our results demonstrate that a gen ed science course emphasizing the process and application of science rather than just scientific facts can lead to improved critical thinking and scientific literacy.

A campus-wide study of STEM courses: new perspectives on teaching practices and perceptions

At the University of Maine, middle and high school science, technology, engineering, and mathematics (STEM) teachers observed 51 STEM courses across 13 different departments and collected information on the active-engagement nature of instruction. The results of these observations show that faculty members teaching STEM courses cannot simply be classified into two groups, traditional lecturers or instructors who teach in a highly interactive manner, but instead exhibit a continuum of instructional behaviors between these two classifications. In addition, the observation data reveal that student behavior differs greatly in classes with varied levels of lecture. Although faculty members who teach large-enrollment courses are more likely to lecture, we also identified instructors of several large courses using interactive teaching methods. Observed faculty members were also asked to complete a survey about how often they use specific teaching practices, and we find that faculty members are generally self-aware of their own practices. Taken together, these findings provide comprehensive information about the range of STEM teaching practices at a campus-wide level and how such information can be used to design targeted professional development for faculty.

A survey of scholarly literature describing the field of bioinformatics education and bioinformatics educational research

Bioinformatics education can be broadly defined as the teaching and learning of the use of computer and information technology, along with mathematical and statistical analysis for gathering, storing, analyzing, interpreting, and integrating data to solve biological problems. The recent surge of genomics, proteomics, and structural biology in the potential advancement of research and development in complex biomedical systems has created a need for an educated workforce in bioinformatics. However, effectively integrating bioinformatics education through formal and informal educational settings has been a challenge due in part to its cross-disciplinary nature. In this article, we seek to provide an overview of the state of bioinformatics education. This article identifies: 1) current approaches of bioinformatics education at the undergraduate and graduate levels; 2) the most common concepts and skills being taught in bioinformatics education; 3) pedagogical approaches and methods of delivery for conveying bioinformatics concepts and skills; and 4) assessment results on the impact of these programs, approaches, and methods in students' attitudes or learning. Based on these findings, it is our goal to describe the landscape of scholarly work in this area and, as a result, identify opportunities and challenges in bioinformatics education.