Assessing teachers' intervention-related expectations and self-efficacy: An examination of the factor structure of the implementation beliefs assessment

Individual-level determinants are hypothesized to enable or prevent successful implementation of evidence-based practices, yet there are limited options for measuring theory-informed, individual-level determinants that influence teachers' and other implementers' delivery of school-based interventions. The goal of this study was to develop a self-report scale that measures variables that have been associated with initial and sustained behavior change related to school-based intervention implementation according to the health action process approach (HAPA). Participants were a nationally representative sample of kindergarten through Grade 12 public school teachers, stratified by grade level and geographical region. Item generation was based on a systematic review of the literature on outcome expectations and self-efficacy, the core constructs related to initiating and sustaining behavior change from the HAPA and in consultation with the theory developer. The sample was randomly split; half of the sample was used for exploratory factor analysis (EFA), and the other half was used for confirmatory factor analysis (CFA). The EFA resulted in a final factor structure of three dimensions of the Implementation Beliefs Assessment (IBA): (a) implementation self-efficacy, (b) positive outcome expectations, and (c) negative outcome expectations. This structure was supported in the other half of the sample using CFA. Additional analyses supported the reliability of IBA data. The IBA represents a step forward toward psychometrically sound measurement of factors associated with initial and sustained behavior change. Implications for future research are discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Evaluating a gated screening approach for identifying risk in kindergarten mathematics

The purpose of the present study was to examine the diagnostic accuracy of different universal screening approaches for identifying mathematics difficulties in kindergarteners. We used extant data from 2,010 kindergarten students from 23 schools across two states. First, we identified the diagnostic accuracy of two individual screeners: (a) a set of three curriculum-based measures (CBM) and (b) a diagnostic measure, the Number Sense Brief (NSB). Then, we determined the diagnostic accuracy of a gated screening approach considering both the CBM and NSB. The criterion measure was a norm-referenced mathematics achievement test. The results indicated that area under the curve values met or exceeded .80 for both individual screeners; however, the number of false positives was high. The gated screening approach yielded fewer false positives at the expense of increasing false negatives. Directions for future research and the practical implications of the results for screening in kindergarten mathematics are discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Effects of portable interventions on school psychologists' graph-rating inconsistency

The visual analysis of data presented in time-series graphs are common in single-case design (SCD) research and applied practice in school psychology. A growing body of research suggests that visual analysts' ratings are often influenced by construct-irrelevant features including Y-axis truncation and compression of the number of data points per X- to Y-axis ratio. We developed and tested two brief interventions, based on the research in cognitive and visual science, to reduce visual analysts' inconsistency when viewing unstandardized graphs. Two hundred practicing school psychologists visually analyzed data presented on standardized graphs and the same data again on unstandardized graphs. Across all conditions, participants were more willing to identify meaningful effects on unstandardized graphs and rated the data as showing significantly larger effects than on the corresponding standardized graphs. However, participants who answered additional (task-relevant) questions about the level or trend of graphed data showed greater rating consistency across the types of graphs in comparison to participants who answered task-irrelevant but challenging questions or control participants. Our results replicated prior research demonstrating the impact of SCD graph construction on practicing school psychologists' interpretations and provide initial support for an intervention to minimize the impact of construct-irrelevant factors. Limitations and future directions for research are discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Identifying profiles of school climate in high schools

This cross-sectional study analyzed data from 364,143 students in 492 high schools who completed the Georgia School Climate Survey during the 2017-2018 school year. Through latent profile analysis, we identified that student perceptions of school climate could be classified into three distinct profiles, including positive, moderate, and negative climate. Using multinomial logistic regression, we then identified school and student characteristics that predicted student classification in the student profiles using the total sample and subsamples by race/ethnicity. Among the key results, we found that most of the school characteristics (e.g., percent of students receiving free or reduced lunch, schools with higher percentages of minoritized students) predicting classification in the negative and positive school climate profiles were different for White students compared to minoritized students. For example, Black students in primarily non-White schools were more likely to view school climate positively, whereas the opposite was the case for White students. We also found that Black and Other (e.g., multiracial) students were more likely to be classified in the negative school climate profile and less likely to be classified in the positive school climate profile compared to White students. In contrast, Latino/a/e students were more likely to be classified in the positive school climate profile and less likely to be classified in the negative school climate profile. Implications for research and practice are discussed. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Spatial alignment supports comparison of life science images

Visual comparisons are pervasive in science, technology, engineering, and mathematics (STEM) instruction and practice. In previous work, adults' visual comparisons of simple stimuli were faster and more accurate when the layout of a display facilitated alignment of corresponding elements-the spatial alignment principle (Matlen et al., 2020). Here, we asked whether the spatial alignment principle extends to rich, educationally relevant stimuli, and how prior experience and spatial skill relate to spatial alignment effects. Participants were asked to find an incorrect bone within a skeleton, presented individually or paired with a correct skeleton in a layout that did (direct placement) or did not (impeded placement) support alignment (Kurtz & Gentner, 2013). Consistent with the spatial alignment principle, undergraduates (Study 1) showed an advantage of direct over impeded placement. Middle schoolers (Study 2) showed a direct advantage on items presented in atypical orientations. That atypical items showed the strongest effects suggests that direct placement may help most when materials are less familiar. However, neither individual differences in undergraduates' STEM course history, nor undergraduates' or middle schoolers' spatial skills moderated spatial alignment effects. Thus, applying the spatial alignment principle in science, technology, engineering, and mathematics has potential to improve visual comparisons, especially those that are challenging, for students of all spatial skill levels. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The effects of generating examples on comprehension and metacomprehension

Teachers and students often report using examples to support learning. Research has shown benefits of viewing provided examples and generating examples during declarative concept learning; however, there is less work showing clear benefits when learners generate their own examples on comprehension measures while students are attempting to learn from expository science texts. The present study tested whether generating examples would be useful for improving comprehension and comprehension monitoring in the context of an undergraduate science course. In a pre-post design, students completed an initial reading activity, followed by taking practice tests on each topic. Some students were assigned to complete an additional example generation activity after taking the practice tests. Some students also evaluated the quality of generated examples and received explanatory feedback. While there was an overall improvement in comprehension for all students, those who generated examples without the opportunity to evaluate the quality had the smallest overall learning gains. Students who evaluated the quality of examples showed the greatest learning gains on application-based test questions. And, although overall overconfidence decreased, there were no differences between conditions. These results suggest that example generation may not always be as helpful for improving learning as many students and teachers assume. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Number lines can be more effective at facilitating adults' performance on health-related ratio problems than risk ladders and icon arrays

Visual displays, such as icon arrays and risk ladders, are often used to communicate numerical health information. Number lines improve reasoning with rational numbers but are seldom used in health contexts. College students solved ratio problems related to COVID-19 (e.g., number of deaths and number of cases) in one of four randomly assigned conditions: icon arrays, risk ladders, number lines, or no accompanying visual display. As predicted, number lines facilitated performance on these problems-the number line condition outperformed the other visual display conditions, which did not perform any better than the no visual display condition. In addition, higher performance on the health-related ratio problems was associated with higher COVID-19 worry for oneself and others, higher perceptions of COVID-19 severity, and higher endorsement of intentions to engage in preventive health behaviors, even when controlling for baseline math skills. These findings have important implications for effectively presenting health statistics. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Using social network analysis in applied psychological research: A tutorial

Social network analysis (SNA) is a highly flexible research method that allows for novel exploration of a wide variety of research phenomena. Evidence from fields as disparate as public health, education, informatics, sociology, and medicine has demonstrated the importance of recognizing the complexity inherent in individuals' connections with others. In this article, we provide a brief conceptual overview of social network theory and methodology, and then demonstrate how to apply SNA to an applied psychological research context studying students embedded in classrooms. We also provide numerous supporting materials on our OSF page, including R code for all analyses, a dataset containing social network data, and a glossary of key terms in social network analysis. We conclude with a set of recommendations for researchers interested in applying SNA to their own contexts and content areas. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Correspondence measures for assessing replication success

Given recent evidence challenging the replicability of results in the social and behavioral sciences, critical questions have been raised about appropriate measures for determining replication success in comparing effect estimates across studies. At issue is the fact that conclusions about replication success often depend on the measure used for evaluating correspondence in results. Despite the importance of choosing an appropriate measure, there is still no widespread agreement about which measures should be used. This article addresses these questions by describing formally the most commonly used measures for assessing replication success, and by comparing their performance in different contexts according to their replication probabilities-that is, the probability of obtaining replication success given study-specific settings. The measures may be characterized broadly as conclusion-based approaches, which assess the congruence of two independent studies' conclusions about the presence of an effect, and distance-based approaches, which test for a significant difference or equivalence of two effect estimates. We also introduce a new measure for assessing replication success called the correspondence test, which combines a difference and equivalence test in the same framework. To help researchers plan prospective replication efforts, we provide closed formulas for power calculations that can be used to determine the minimum detectable effect size (and thus, sample sizes) for each study so that a predetermined minimum replication probability can be achieved. Finally, we use a replication data set from the Open Science Collaboration (2015) to demonstrate the extent to which conclusions about replication success depend on the correspondence measure selected. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Single-item principal stress and coping measures: Concurrent and predictive validity and comparisons to teacher measures

The present study analyzed concurrent and predictive validity of single-item scales for assessing principal stress and coping. We examined concurrent and prospective relations among stress and coping single-items with principal job satisfaction, overall health, perceptions of school safety, and principal leadership self-efficacy. We also compared principals and teachers on their stress and coping levels using the same single-item scales. Consistent with the literature on teacher stress and coping, the correlations of principal coping with different outcomes-job satisfaction, overall health, leadership efficacy, and safety perceptions-were stronger in comparison to the correlations between principal stress and those same outcomes. In regression models with both stress and coping, only principal coping predicted concurrent and future principal job satisfaction and overall health, as well as change in those outcomes. Coping also predicted concurrent but not future perceptions of school safety. Stress and coping did not consistently predict concurrent or future measures of leadership self-efficacy. Last, we found that principals reported even higher levels of stress than the well-documented high levels reported by teachers. We discuss areas for further research and potential use of these measures. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Measuring college belongingness: Structure and measurement of the Sense of Social Fit Scale

Sense of belonging is theorized to be a fundamental human need and has been shown to have important implications in many domains of life, including academic achievement. The Sense of Social Fit scale (SSF; Walton & Cohen, 2007) is widely used to assess college belongingness, particularly to study differences in academic experiences along lines of gender and race. Despite its wide use, the instrument's latent factor structure and measurement invariance properties have not been reported in the published literature to date. Consequently, researchers regularly use subsets of the SSF's items without psychometric justification. Here, we explore and validate the SSF's factor structure and other psychometric properties, and we provide recommendations about how to score the measure. A one-factor model in Study 1 showed poor fit, and exploratory factor analyses extracted a four-factor solution. Study 2's confirmatory factor analyses demonstrated superior fit of a bifactor model with four specific factors (from Study 1) and one general factor. Ancillary analyses supported a total scale scoring method for the SSF and did not support computing raw subscale scores. We also tested the bifactor model's measurement invariance across gender and race, compared latent mean scores between groups, and established the model's criterion and concurrent validity. We discuss implications and suggestions for future research. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Emotional complexity under high stress: Do protective associations for risk behaviors persist even during a pandemic?

Risk behaviors like substance use and binge eating are often used to cope with negative emotions. Engagement in these behaviors has been exacerbated by the COVID-19 pandemic. Past research suggests that complex emotion conceptualizations captured as emotion differentiation (ability to discriminate between emotional states) and polarity (ability to integrate positive and negative features of emotional experience) may be protective. We examined associations of mean affect intensity, emotion differentiation, and emotion polarity with frequency of daily substance use and binge eating across 10 days in a demographically diverse sample of U.S. adults (N = 353) recruited between March 24 and April 9, 2020, when stay-at-home orders were initiated. Owing to the nested data structure and excessive zero values, analyses were conducted using multilevel zero-inflated negative binomial regression. Consistent with past research, negative affect was positively associated with frequency of substance use and binge eating. Importantly, results indicated that negative emotion differentiation was protective, predicting greater likelihood of not using substances and binge eating at all across the sampling period. These effects remained even after controlling for mean affect intensity, emotion polarity, and positive emotion differentiation. Neither positive emotion differentiation nor emotion polarity were significantly associated with either behavior. Our results suggest that greater complexity in conceptualization of negative emotions facilitates some protection against risk behaviors such as substance use and binge eating, even during periods of high environmental stress. These findings have important implications for optimizing interventions to reduce engagement in risk behaviors. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A longitudinal study of home-based involvement and dyadic adjustment during the transition to early elementary school

Home-based involvement refers to caregivers' active efforts to create learning opportunities for their children at home and in the community. Across child development, home-based involvement is a positive influence on children's social-emotional and academic functioning. Findings have suggested that home-based involvement tends to decline during elementary and middle school, but the extent to which home-based involvement changes over time during the transition to early elementary school is less clear. Dyadic adjustment is the quality of the relationship between two partners. Grounded in family systems theory, the spillover hypothesis suggests that dyadic adjustment is an important influence on home-based involvement. However, there is limited research on the extent to which dyadic adjustment predicts home-based involvement. The present study used latent growth curve analysis to examine the trajectory of home-based involvement during the transition to early elementary school, as well as the extent to which dyadic adjustment predicts home-based involvement during this transition. Participants were 157 primary caregivers of children in kindergarten through second grade. Results suggest that home-based involvement has a negative, linear trajectory between kindergarten and second grade, and that dyadic adjustment predicts higher levels of home-based involvement at kindergarten, first grade, and second grade. Implications of study findings for research and practice are discussed, with a focus on preventive interventions that aim to promote dyadic adjustment and home-based involvement during the transition to early elementary school. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Associations between preschool cognitive and behavioral skills and college enrollment: Evidence from the Chicago School Readiness Project

The current article examines associations between preschool cognitive and behavioral skills and indicators of college enrollment in a sample (n = 379) of primarily Black and Hispanic youth growing up in low-income areas of Chicago. Although we found that most early cognitive and behavioral skills were only weakly or moderately related to later college enrollment, a rating of preschool attention and impulsivity control was a relatively strong predictor. Across most models tested, attention and impulsivity control, executive functioning, and effortful control produced predicted probabilities that were similar in magnitude, or larger, than the effects produced by early math and literacy. There was no indication that early behavioral difficulties were substantive predictors of college enrollment. These descriptive findings suggest that in a low-income sample of children, some early cognitive capabilities related to attention and EF predict longer term college enrollment. We discuss implications for developmental theory and suggest that caution should be applied when projecting likely effects of early skill-focused interventions. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Experimental tests of hypothetical lottery incentives on unvaccinated adults' COVID-19 vaccination intentions

OBJECTIVE

In May 2021, U.S. states began implementing "vaccination lotteries" encouraging COVID-19 vaccination. Drawing from Prospect Theory and math cognition research, we tested several monetary lottery structures and their framing to determine which would best motivate unvaccinated adults.

METHOD

In two online experiments, U.S. adults were asked to imagine that their state implemented a vaccination lottery. In Experiment 1, participants (N = 589) were randomly assigned to 1 of 12 conditions varying the monetary amount and number of winners, holding constant a $5 million total payout. In Experiment 2, participants (N = 274) were randomly assigned to one of four conditions in a 2 (Message Framing: Gain versus Loss) by 2 (Numeric Framing: Big versus Small) factorial design; in all conditions, five people would each win $1 million. Participants rated their baseline vaccination willingness (1 = not at all to 4 = very) and postmanipulation COVID-19 vaccination intentions "if their state offered this incentive" (0 = definitely would not to 100 = definitely would).

RESULTS

Intentions did not differ across conditions (Experiment 1: F[11, 561] = 1.29, p = .224, ηp² = .03; Experiment 2: Message Framing, F[1, 266)] = .01, p = .940, ηp² = .000; Numeric Framing, F[1, 266] = 1.40, p = .237, ηp² = .01; Interaction, F[1, 266] = 1.40, p = .238, ηp² = .01). When participants were shown a list of 12 lottery structures and asked which they preferred, participants on average preferred options that awarded less money to more people. However, 41.9% of participants across both experiments indicated they would not vaccinate for any lottery-based monetary incentive.

CONCLUSIONS

Multiple lottery structures could be equally (un)motivating for unvaccinated adults. Structures that distribute incentives across more people or alternative public health strategies should be considered. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Self-care behaviors and affect during the early stages of the COVID-19 pandemic

OBJECTIVE

Self-care behaviors aimed at maintaining physical and mental health are often recommended during stressful contexts. We tested emotional predictors of self-care behaviors (healthy eating, exercise, engaging in a hobby, relaxation/meditation, time spent with a supportive person, talking online with friends/family) during the COVID-19 pandemic and their emotional consequences. We hypothesized a reciprocal within-person process whereby positive affect increases self-care behaviors (Hypothesis 1) and self-care behaviors increase positive affect while decreasing negative affect (Hypothesis 2).

METHOD

A 10-day daily diary was completed by 289 adult participants in the United States during spring 2020 when counties in 40 out of 50 states had some form of stay-at-home orders.

RESULTS

Lagged analyses for Hypothesis 1 suggested that positive affect did not significantly predict residualized change in self-care behaviors; however, more intense negative affect predicted increased self-care behaviors from one day to the next. Concurrent analyses for Hypothesis 2 indicated most self-care behaviors were associated with more positive affect and some with less negative affect on the same day. Lagged analyses for Hypothesis 2 indicated that self-care behaviors largely did not predict residualized change in positive or negative affect from one day to the next. At the between-person level, people who experienced more positive affect engaged in more self-care behaviors across the sampling period.

CONCLUSION

Self-care behaviors continue to have mental health benefits during stressful environments such as the COVID-19 pandemic and stay-at-home orders. Negative affect can play an adaptive role during times of stress by facilitating self-care. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Distributional causal effects: Beyond an "averagarian" view of intervention effects

The usefulness of mean aggregates in the analysis of intervention effectiveness is a matter of considerable debate in the psychological, educational, and social sciences. In addition to studying "average treatment effects," the evaluation of "distributional treatment effects," (i.e., effects that go beyond means), has been suggested to obtain a broader picture of how an intervention affects the study outcome. We continue this discussion by considering distributional causal effects. We present formal definitions of causal effects that go beyond means and utilize a distributional regression framework known as generalized additive models for location, scale, and shape (GAMLSS). GAMLSS allows one to characterize an intervention effect in its totality through simultaneously modeling means, variances, skewnesses, kurtoses, as well as ceiling and floor effects of outcome distributions. Based on data from a large-scale randomized controlled trial, we use GAMLSS to evaluate the impact of a teacher classroom management program on student academic performance. Results suggest the teacher classroom management training increased mean academic competence as well as the chance to obtain the maximum score on the academic competence scale. These effects would have been completely overlooked in a traditional evaluation of mean aggregates. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Regression with reduced rank predictor matrices: A model of trade-offs

A regression model of predictor trade-offs is described. Each regression parameter equals the expected change in Y obtained by trading 1 point from one predictor to a second predictor. The model applies to predictor variables that sum to a constant T for all observations; for example, proportions summing to T = 1.0 or percentages summing to T = 100 for each observation. If predictor variables sum to a constant T for all observations and if a least squares solution exists, the predicted values for the criterion variable Y will be uniquely determined, but there will be an infinite set of linear regression weights and the familiar interpretation of regression weights does not apply. However, the regression weights are determined up to an additive constant and thus differences in regression weights βv-βv∗ are uniquely determined, readily estimable, and interpretable. βv-βv∗ is the expected increase in Y given a transfer of 1 point from variable v∗ to variable v. The model is applied to multiple-choice test items that have four response categories, one correct and three incorrect. Results indicate that the expected outcome depends, not just on the student's number of correct answers, but also on how the student's incorrect responses are distributed over the three incorrect response types. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Multilevel meta-analysis of single-case experimental designs using robust variance estimation

Single-case experimental designs (SCEDs) are used to study the effects of interventions on the behavior of individual cases, by making comparisons between repeated measurements of an outcome under different conditions. In research areas where SCEDs are prevalent, there is a need for methods to synthesize results across multiple studies. One approach to synthesis uses a multilevel meta-analysis (MLMA) model to describe the distribution of effect sizes across studies and across cases within studies. However, MLMA relies on having accurate sampling variances of effect size estimates for each case, which may not be possible due to auto-correlation in the raw data series. One possible solution is to combine MLMA with robust variance estimation (RVE), which provides valid assessments of uncertainty even if the sampling variances of effect size estimates are inaccurate. Another possible solution is to forgo MLMA and use simpler, ordinary least squares (OLS) methods with RVE. This study evaluates the performance of effect size estimators and methods of synthesizing SCEDs in the presence of auto-correlation, for several different effect size metrics, via a Monte Carlo simulation designed to emulate the features of real data series. Results demonstrate that the MLMA model with RVE performs properly in terms of bias, accuracy, and confidence interval coverage for estimating overall average log response ratios. The OLS estimator corrected with RVE performs the best in estimating overall average Tau effect sizes. None of the available methods perform adequately for meta-analysis of within-case standardized mean differences. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Bidirectional spillover across days between family assistance and physical health experiences during adolescence

Helping the family may either promote or undermine adolescents' physical health and well-being. Adolescents (N = 396, 58% female, Mage = 14.57 years) completed diary checklists for 14 days, reporting whether they provided instrumental assistance (e.g., tangible tasks) and emotional support (e.g., listening, giving advice) to family, as well as their amount of physical activity, sleep, and physical symptoms (e.g., headache, backpain) each day. After providing emotional support, adolescents slept more that night and experienced fewer physical symptoms the next day, over and above prior day levels. When adolescents provided instrumental assistance on school days (but not nonschool days), they engaged in less physical activity that day. These results were consistent across individual differences in gender, age, socioeconomic status, and ethnicity. In addition, bidirectional associations emerged such that adolescents were more likely to provide instrumental assistance on days after they slept more. (PsycInfo Database Record (c) 2021 APA, all rights reserved).