Evaluating SEM Model Fit with Small Degrees of Freedom

Research has revealed that the performance of root mean square error of approximation (RMSEA) in assessing structural equation models with small degrees of freedom (df) is suboptimal, often resulting in the rejection of correctly specified or closely fitted models. This study investigates the performance of standardized root mean square residual (SRMR) and comparative fit index (CFI) in small df models with various levels of factor loadings, sample sizes, and model misspecifications. We find that, in comparison with RMSEA, population SRMR and CFI are less susceptible to the effects of df. In small df models, the sample SRMR and CFI could provide more useful information to differentiate models with various levels of misfit. The confidence intervals and p-values of a close fit were generally accurate for all three fit indices. We recommend researchers use caution when interpreting RMSEA for models with small df and to rely more on SRMR and CFI.

Impact of a year-round school calendar on children's BMI and fitness: Final outcomes from a natural experiment

BACKGROUND

Structure may mitigate children's accelerated summer BMI gain and cardiorespiratory-fitness (CRF) loss.

OBJECTIVES

Examine BMI and CRF change during school and summer for year-round and traditional calendar school children.

METHODS

Three schools (N = 2279, 1 year-round) participated in this natural experiment. Children's BMI z-score (zBMI) and CRF (PACER laps) were measured from 2017 to 2019 each May/August. Mixed effects regression estimated monthly zBMI and CRF change during school/summer. Secondary analyses examined differences by weight status and race. Spline regression models estimated zBMI and CRF growth from kindergarten-sixth grade.

RESULTS

Compared to traditional school, children attending a year-round school gained more zBMI (difference = 0.015; 95CI = 0.002, 0.028) during school, and less zBMI (difference = -0.029; 95CI = -0.041, -0.018), and more CRF (difference = 0.834; 95CI = 0.575, 1.093) monthly during summer. Differences by weight status and race were observed during summer and school. Growth models demonstrated that the magnitude of overall zBMI and CRF change from kindergarten-sixth grade was similar for year-round or traditional school children.

CONCLUSIONS

Contrary to traditional school children zBMI increased during the traditional 9-month school calendar and zBMI decreased during the traditional summer vacation for year-round school children. Structured summer programming may mitigate accelerated summer BMI gain and CRF loss especially for overweight or obese, and/or Black children.

Assessing the Accuracy of Errors of Measurement. Implications for Assessing Reliable Change in Clinical settings

Item response theory (IRT) models are non-linear latent variable models for discrete measures, whereas factor analysis (FA) is a latent variable model for continuous measures. In FA, the standard error (SE) of individuals' scores is common for all individuals. In IRT, the SE depends on the individual's score, and the SE function is to be provided. The empirical standard deviation of the scores across discrete ranges should also be computed to inform the extent to which IRT SEs overestimate or underestimate the variability of the scores. Within the target range of scores the test was designed to measure, one should expect IRT SEs to be smaller and more precise than FA SEs, and therefore preferable to assess clinical change. Outside the target range, IRT SEs may be too large and more imprecise than FA SEs, and FA more precise to assess change. As a result, whether FA or IRT characterize reliable change more accurately in a sample will depend on the proportion of individuals within or outside the IRT target score range. An application is provided to illustrate these concepts.

Dynamics of sleep, sedentary behavior, and moderate-to-vigorous physical activity on school versus nonschool days

STUDY OBJECTIVES

Studies examining time-use activity behaviors (sleep, sedentary behavior, and physical activity) on school days compared with nonschool days have examined these behaviors independently, ignoring their interrelated nature, limiting our ability to optimize the health benefits of these behaviors. This study examines the associations of school-day (vs. nonschool day) with time-use activity behaviors.

METHODS

Time series data (6,642 days) from Fitbits (Charge-2) were collected (n = 196, 53% female, 5-10 years). We used a variable-centered dynamic structural equation modeling approach to estimate day-to-day associations of time-use activity behaviors on school days for each child. We then used person-centered cluster analyses to group individuals based on these estimates.

RESULTS

Within-participant analysis showed that on school days (vs. nonschool days), children (1) slept less (β = -0.17, 95% CI = -0.21, -0.13), (2) were less sedentary (β = -0.05, 95% CI = -0.09, -0.02), and (3) had comparable moderate-to-vigorous physical activity (MVPA; β = -0.05, 95% CI = -0.11, 0.00). Between-participant analysis showed that, on school days, children with higher sleep carryover experienced greater decreases in sleep (β = 0.44, 95% CI = 0.08, 0.71), children with higher body mass index z-score decreased sedentary behavior more (β = -0.41, 95% CI = -0.64, -0.13), and children with lower MVPA increased MVPA more (β = -0.41, 95% CI -0.64, -0.13). Cluster analysis demonstrated four distinct patterns of connections between time-use activity behaviors and school (High Activity, Sleep Resilient, High Sedentary, and Dysregulated Sleep).

CONCLUSIONS

Using a combination of person-centered and more traditional variable-centered approaches, we identified patterns of interrelated behaviors that differed on school, and nonschool days. Findings can inform targeted intervention strategies tailored to children's specific behavior patterns.

Using the Standardized Root Mean Squared Residual (SRMR) to Assess Exact Fit in Structural Equation Models

We examine the accuracy of p values obtained using the asymptotic mean and variance (MV) correction to the distribution of the sample standardized root mean squared residual (SRMR) proposed by Maydeu-Olivares to assess the exact fit of SEM models. In a simulation study, we found that under normality, the MV-corrected SRMR statistic provides reasonably accurate Type I errors even in small samples and for large models, clearly outperforming the current standard, that is, the likelihood ratio (LR) test. When data shows excess kurtosis, MV-corrected SRMR p values are only accurate in small models (p = 10), or in medium-sized models (p = 30) if no skewness is present and sample sizes are at least 500. Overall, when data are not normal, the MV-corrected LR test seems to outperform the MV-corrected SRMR. We elaborate on these findings by showing that the asymptotic approximation to the mean of the SRMR sampling distribution is quite accurate, while the asymptotic approximation to the standard deviation is not.

The impact of summer vacation on children's obesogenic behaviors and body mass index: a natural experiment

BACKGROUND

Children's BMI gain accelerates during summer. The Structured Days Hypothesis posits that the lack of the school day during summer vacation negatively impacts children's obesogenic behaviors (i.e., physical activity, screen time, diet, sleep). This natural experiment examined the impact of summer vacation on children's obesogenic behaviors and body mass index (BMI).

METHODS

Elementary-aged children (n = 285, 5-12 years, 48.7% male, 57.4% African American) attending a year-round (n = 97) and two match-paired traditional schools (n = 188) in the United States participated in this study. Rather than taking a long break from school during the summer like traditional schools, year-round schools take shorter and more frequent breaks from school. This difference in school calendars allowed for obesogenic behaviors to be collected during three conditions: Condition 1) all children attend school, Condition 2) year-round children attend school while traditional children were on summer vacation, and Condition 3) summer vacation for all children. Changes in BMI z-score were collected for the corresponding school years and summers. Multi-level mixed effects regressions estimated obesogenic behaviors and monthly zBMI changes. It was hypothesized that children would experience unhealthy changes in obesogenic behaviors when entering summer vacation because the absence of the school day (i.e., Condition 1 vs. 2 for traditional school children and 2 vs. 3 for year-round school children).

RESULTS

From Condition 1 to 2 traditional school children experienced greater unhealthy changes in daily minutes sedentary (∆ = 24.2, 95CI = 10.2, 38.2), screen time minutes (∆ = 33.7, 95CI = 17.2, 50.3), sleep midpoint time (∆ = 73:43, 95CI = 65:33, 81:53), and sleep efficiency percentage (-∆ = 0.7, 95CI = -1.1, - 0.3) when compared to year-round school children. Alternatively, from Condition 2 to 3 year-round school children experienced greater unhealthy changes in daily minutes sedentary (∆ = 54.5, 95CI = 38.0, 70.9), light physical activity minutes (∆ = - 42.2, 95CI = -56.2, - 28.3) MVPA minutes (∆ = - 11.4, 95CI = -3.7, - 19.1), screen time minutes (∆ = 46.5, 95CI = 30.0, 63.0), and sleep midpoint time (∆ = 95:54, 95CI = 85:26, 106:22) when compared to traditional school children. Monthly zBMI gain accelerated during summer for traditional (∆ = 0.033 95CI = 0.019, 0.047) but not year-round school children (∆ = 0.004, 95CI = -0.014, 0.023).

CONCLUSIONS

This study suggests that the lack of the school day during summer vacation negatively impacts sedentary behaviors, sleep timing, and screen time. Changes in sedentary behaviors, screen time, and sleep midpoint may contribute to accelerated summer BMI gain. Providing structured programming during summer vacation may positively impact these behaviors, and in turn, mitigate accelerated summer BMI gain.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03397940 . Registered January 12th 2018.

The Effect of Estimation Methods on SEM Fit Indices

We examined the effect of estimation methods, maximum likelihood (ML), unweighted least squares (ULS), and diagonally weighted least squares (DWLS), on three population SEM (structural equation modeling) fit indices: the root mean square error of approximation (RMSEA), the comparative fit index (CFI), and the standardized root mean square residual (SRMR). We considered different types and levels of misspecification in factor analysis models: misspecified dimensionality, omitting cross-loadings, and ignoring residual correlations. Estimation methods had substantial impacts on the RMSEA and CFI so that different cutoff values need to be employed for different estimators. In contrast, SRMR is robust to the method used to estimate the model parameters. The same criterion can be applied at the population level when using the SRMR to evaluate model fit, regardless of the choice of estimation method.

Fitting Ordinal Factor Analysis Models With Missing Data: A Comparison Between Pairwise Deletion and Multiple Imputation

This study compares two missing data procedures in the context of ordinal factor analysis models: pairwise deletion (PD; the default setting in Mplus) and multiple imputation (MI). We examine which procedure demonstrates parameter estimates and model fit indices closer to those of complete data. The performance of PD and MI are compared under a wide range of conditions, including number of response categories, sample size, percent of missingness, and degree of model misfit. Results indicate that both PD and MI yield parameter estimates similar to those from analysis of complete data under conditions where the data are missing completely at random (MCAR). When the data are missing at random (MAR), PD parameter estimates are shown to be severely biased across parameter combinations in the study. When the percentage of missingness is less than 50%, MI yields parameter estimates that are similar to results from complete data. However, the fit indices (i.e., χ2, RMSEA, and WRMR) yield estimates that suggested a worse fit than results observed in complete data. We recommend that applied researchers use MI when fitting ordinal factor models with missing data. We further recommend interpreting model fit based on the TLI and CFI incremental fit indices.

The potential of a year-round school calendar for maintaining children's weight status and fitness: Preliminary outcomes from a natural experiment

Purpose

To evaluate the potential of a year-round school calendar (180-day school year distributed across 12 months) as an intervention compared to a traditional school calendar (180-day school year distributed across 9 months) for mitigating children's weight gain and fitness loss via a natural experiment.

Methods

Height, weight, and cardiorespiratory fitness (CRF) (i.e., Fitnessgram Progressive Aerobic Cardiovascular Endurance Run) were measured in children (5-12 years old) in 3 schools (2 traditional, 1 year-round, n = 990 students, age = 8.6 ± 2.4 years, 53.1% male, 68.9% African American) from 1 school district. Structure (represented by the presence of a school day) was the independent variable. Changes in body mass index (BMI), age- and sex-specific BMI z-scores (zBMI), BMI percentile, percent of overweight or obese children, and CRF (Progressive Aerobic Cardiovascular Endurance Run laps completed) were assessed for summer 2017 (May-August 2017), school year 2017/2018 (August 2017-May 2018), and summer 2018 (May-August 2018). Primary analyses examined the overall change in weight and CRF from summer 2017 until summer 2018 via multilevel mixed effects regression, with group (traditional vs. year-round calendar), time, and a group-by-time interaction as the independent variables. Secondary regression analyses estimated differences in change within and between groups during each time period, separately.

Results

Year-round students gained less BMI (difference in ∆ = -0.44, 95% confidence interval (CI): -0.67 to -0.03) and less CRF (difference in ∆ = -1.92, 95%CI: -3.56 to -0.28) than students attending a traditional school overall. Compared with traditional students, during both summers, year-round students gained less BMI (summer 2017 difference in ∆ = -0.15, 95%CI: -0.21 to -0.08; summer 2018 difference in ∆ = -0.16, 95%CI: -0.24 to -0.07) and zBMI (summer 2017 difference in ∆ = -0.032, 95%CI: -0.050 to -0.010; summer 2018 difference in ∆ = -0.033, 95%CI: -0.056 to -0.009), and increased CRF (summer 2017 difference in ∆ = 0.40, 95%CI: 0.02-0.85; summer 2018 difference in ∆ = 0.23, 95%CI: -0.25 to 0.74). However, the opposite was observed for the school year, with traditional students gaining less BMI and zBMI and increasing CRF compared with year-round students (difference in BMI ∆ = 0.05, 95%CI: 0.03-0.07; difference in zBMI ∆ = 0.012, 95%CI: 0.005-0.019; difference in Progressive Aerobic Cardiovascular Endurance Run laps ∆ = -0.43, 95%CI: -0.58 to -0.28).

Conclusion

The year-round school calendar had a small beneficial impact on children's weight status but not CRF. It is unclear if this benefit to children's weight would be maintained because gains made in the summer were largely erased during the school year. Trajectories of weight and CRF gain/loss were consistent with the structured days hypothesis.

Restricted Recalibration of Item Response Theory Models

In item response theory (IRT), it is often necessary to perform restricted recalibration (RR) of the model: A set of (focal) parameters is estimated holding a set of (nuisance) parameters fixed. Typical applications of RR include expanding an existing item bank, linking multiple test forms, and associating constructs measured by separately calibrated tests. In the current work, we provide full statistical theory for RR of IRT models under the framework of pseudo-maximum likelihood estimation. We describe the standard error calculation for the focal parameters, the assessment of overall goodness-of-fit (GOF) of the model, and the identification of misfitting items. We report a simulation study to evaluate the performance of these methods in the scenario of adding a new item to an existing test. Parameter recovery for the focal parameters as well as Type I error and power of the proposed tests are examined. An empirical example is also included, in which we validate the pediatric fatigue short-form scale in the Patient-Reported Outcome Measurement Information System (PROMIS), compute global and local GOF statistics, and update parameters for the misfitting items.

Understanding the Model Size Effect on SEM Fit Indices

This study investigated the effect the number of observed variables (p) has on three structural equation modeling indices: the comparative fit index (CFI), the Tucker-Lewis index (TLI), and the root mean square error of approximation (RMSEA). The behaviors of the population fit indices and their sample estimates were compared under various conditions created by manipulating the number of observed variables, the types of model misspecification, the sample size, and the magnitude of factor loadings. The results showed that the effect of p on the population CFI and TLI depended on the type of specification error, whereas a higher p was associated with lower values of the population RMSEA regardless of the type of model misspecification. In finite samples, all three fit indices tended to yield estimates that suggested a worse fit than their population counterparts, which was more pronounced with a smaller sample size, higher p, and lower factor loading.

Evaluating Factorial Invariance: An Interval Estimation Approach Using Bayesian Structural Equation Modeling

In this study, we introduce an interval estimation approach based on Bayesian structural equation modeling to evaluate factorial invariance. For each tested parameter, the size of noninvariance with an uncertainty interval (i.e. highest density interval [HDI]) is assessed via Bayesian parameter estimation. By comparing the most credible values (i.e. 95% HDI) with a region of practical equivalence (ROPE), the Bayesian approach allows researchers to (1) support the null hypothesis of practical invariance, and (2) examine the practical importance of the noninvariant parameter. Compared to the traditional likelihood ratio test, simulation results suggested that the proposed Bayesian approach could offer additional insight into evaluating factorial invariance, thus, leading to more informative conclusions. We provide an empirical example to demonstrate the procedures necessary to implement the proposed method in applied research. The importance of and influences on the choice of an appropriate ROPE are discussed.

Changes in children's sleep and physical activity during a 1-week versus a 3-week break from school: a natural experiment

Study Objectives

To examine changes in elementary aged children's sleep and physical activity during a 1-week and a 3-week school break.

Methods

Sleep and physical activity of elementary children (n = 154, age = 5-9 years, 44.8% female, 65.5% African American) were collected over 7 weeks that included a 1-week break in two schools and a 3-week break in a single school. Mixed regression models estimated sleep and physical activity changes within and between groups (i.e. 1-week vs. 3-weeks) during school and school break weeks.

Results

Compared to school weeks, bed times shifted 72.7 (95% CI = 57.5, 87.9) and 75.4 (95% CI = 58.1, 92.7) minutes later on weekdays during the 1-week and 3-week break, respectively. Wake times shifted 111.6 (95% CI = 94.3, 128.9) and 99.8 (95% CI = 80.5, 119.1) minutes later on weekdays during 1-week and 3-week breaks. On weekdays during the 3-week break, children engaged in 33.1 (95% CI = 14.1, 52.2) more sedentary minutes and -12.2 (-20.2, -4.2) fewer moderate-to-vigorous physical activity minutes/day. No statistically significant changes in children's sedentary, light, or moderate-to-vigorous physical activity (MVPA) minutes were observed on weekdays during the 1-week break. Between-group differences in the change in time sedentary (32.1-95% CI = 5.8, 58.4), and moderate-to-vigorous (-13.0-95% CI = -23.9, -2.0) physical activity were observed.

Conclusions

Children's sleep shifted later on both 1-week and 3-week breaks. Children's activity changed minimally on weekdays during a 1-week school break and more during a 3-week school break. Displaced sleep and reductions in activity are intervention targets for mitigating unhealthy weight gain during extended breaks from school.

The Relationship Between the Standardized Root Mean Square Residual and Model Misspecification in Factor Analysis Models

We argue that the definition of close fitting models should embody the notion of substantially ignorable misspecifications (SIM). A SIM model is a misspecified model that might be selected, based on parsimony, over the true model should knowledge of the true model be available. Because in applications the true model (i.e., the data generating mechanism) is unknown, we investigate the relationship between the population standardized root mean square residual (SRMR) values and various model misspecifications in factor analysis models to better understand the magnitudes of the SRMR. Summary effect sizes of misfit such as the SRMR are necessarily insensitive to some non-ignorable localized misspecifications (i.e., the presence of a few large residual correlations in large models). Localized misspecifications may be identified by examining the largest standardized residual covariance. Based on the findings, our population reference values for close fit are based on a two-index strategy: (1) largest absolute value of standardized residual covariance ≤0.10, and (2) SRMR ≤0.05× R¯2 the average R² of the manifest variables; for acceptable fit our values are 0.15 and 0.10× R¯2 , respectively.

Confirmatory Factor Analytic Structure and Measurement Invariance of the Emotion Regulation Questionnaire for Children and Adolescents in a Longitudinal Sample of Adolescents

While adolescence is a critical stage of development marked by heightened bottom-up emotional reactivity and immature top-down regulatory control, research on emotion regulation has relatively neglected middle childhood to adolescence years. This may be attributed to the limited number and scope of age-appropriate, reliable, and valid measures of emotion regulation. This study examines the confirmatory factor analytic structure and measurement invariance of the Emotion Regulation Questionnaire for Children and Adolescents (ERQ-CA), a 10-item self-report measure designed to measure habitual use of cognitive reappraisal and expressive suppression, across a 1-year time interval in school samples of adolescents. Results indicate low test–retest reliability but high to acceptable internal consistency over a 1-year time period. The two-factor model has an approximate but close fit to the data collected, which is consistent with underlying theoretical framework and prior empirical findings. Tests of measurement equivalence support strong invariance, indicating that there were no statistically significant differences in factor means, variances, and correlations over a 1-year interval.

Effect Sizes of Model Misfit in Structural Equation Models

Abstract. Residual correlations and covariances provide effect sizes of the misfit of covariance structure models. In a simulation study, we found that accurate confidence intervals (CIs) for standardized residual covariances are obtained even in small samples (N = 100), regardless of model size, degree of model misspecification, and data distribution. Standardized residual covariances also provide information about the source of misfit in poorly fitting models. From this viewpoint, they may be considered an alternative to modification indices. We compared the empirical Type I errors and power rates of standardized residual covariances and modification indices and found that both procedures provide nearly identical rates across the simulation conditions investigated. Residual correlations and standardized residual covariances provide very similar results.

Assessing the Size of Model Misfit in Structural Equation Models

When a statistically significant mean difference is found, the magnitude of the difference is judged qualitatively using an effect size such as Cohen's d. In contrast, in a structural equation model (SEM), the result of the statistical test of model fit is often disregarded if significant, and inferences are drawn using "close" models retained based on point estimates of sample statistics (goodness-of-fit indices). However, when a SEM cannot be retained using a test of exact fit, all substantive inferences drawn from it are suspect. It is therefore important to determine the size of the model misfit. Standardized residual covariances and residual correlations provide standardized effect sizes of the misfit of SEM models. They can be summarized using the Standardized Root Mean Squared Residual (SRMSR) and the Correlation Root Mean Squared Residual (CRMSR) which can be used as overall effect sizes of the misfit. Statistical theory is provided that allows the construction of confidence intervals and tests of close fit based on the SRMSR and CRMSR. It is hoped that the use of standardized effect sizes of misfit will help reconcile current practices in SEM and elsewhere in statistics.

Improving Psychological Measurement: Does It Make a Difference? A Comment on Nesselroade and Molenaar (2016)

Nesselroade and Molenaar advocate the use of an idiographic filter approach. This is a fixed-effects approach, which may limit the number of individuals that can be simultaneously modeled, and it is not clear how to model the presence of subpopulations. Most important, Nesselroade and Molenaar's proposal appears to be best suited for modeling long time series on a few variables for a few individuals. Long time series are not common in psychological applications. Can it be applied to the usual longitudinal data we face? These are characterized by short time series (four to five points in time), hundreds of individuals, and dozens of variables. If so, what do we gain? Applied settings most often involve between-individual decisions. I conjecture that their approach will not outperform common, simpler, methods. However, when intraindividual decisions are involved, their approach may have an edge.

Item diagnostics in multivariate discrete data

Researchers who evaluate the fit of psychometric models to binary or multinomial items often look at univariate and bivariate residuals to determine how a poorly fitting model can be improved. There is a class of z statistics and also a class of generalized X₂ statistics that can be used for examining these marginal fits. We describe these statistics and compare them with regard to the control of Type I error and statistical power. We show how the class of z statistics can be extended to accommodate items with multinomial response options. We provide guidelines for the use of these statistics, including how to control for multiple testing, and present 2 detailed examples. Using the root mean square error of approximation (RMSEA) for discrete data to adjudge fit, the examples illustrate how the use of these methods can dramatically improve the fit of item response theory models to widely used measures in personality and clinical psychology.

Social problem solving in chronic pain: An integrative model of coping predicts mental health in chronic pain patients

Despite several models of coping have been proposed in chronic pain, research is not integrative and has not yet identified a reliable set of beneficial coping strategies. We intend to offer a comprehensive view of coping using the social problem-solving model. Participants were 369 chronic pain patients (63.78% women; mean age 58.89 years; standard deviation = 15.12 years). Correlation analyses and the structural equation model for mental health revealed potentially beneficial and harmful problem-solving components. This integrative perspective on general coping could be used to promote changes in the way patients deal with stressful conditions other than pain.

Identifying the Source of Misfit in Item Response Theory Models

When an item response theory model fails to fit adequately, the items for which the model provides a good fit and those for which it does not must be determined. To this end, we compare the performance of several fit statistics for item pairs with known asymptotic distributions under maximum likelihood estimation of the item parameters: (a) a mean and variance adjustment to bivariate Pearson's X(2), (b) a bivariate subtable analog to Reiser's (1996) overall goodness-of-fit test, (c) a z statistic for the bivariate residual cross product, and (d) Maydeu-Olivares and Joe's (2006) M2 statistic applied to bivariate subtables. The unadjusted Pearson's X(2) with heuristically determined degrees of freedom is also included in the comparison. For binary and ordinal data, our simulation results suggest that the z statistic has the best Type I error and power behavior among all the statistics under investigation when the observed information matrix is used in its computation. However, if one has to use the cross-product information, the mean and variance adjusted X(2) is recommended. We illustrate the use of pairwise fit statistics in 2 real-data examples and discuss possible extensions of the current research in various directions.

Assessing Approximate Fit in Categorical Data Analysis

A family of Root Mean Square Error of Approximation (RMSEA) statistics is proposed for assessing the goodness of approximation in discrete multivariate analysis with applications to item response theory (IRT) models. The family includes RMSEAs to assess the approximation up to any level of association of the discrete variables. Two members of this family are RMSEA2, which uses up to bivariate moments, and the full information RMSEAn. The RMSEA2 is estimated using the M2 statistic of Maydeu-Olivares and Joe (2005, 2006), whereas for maximum likelihood estimation, RMSEAn is estimated using Pearson's X(2) statistic. Using IRT models, we provide cutoff criteria of adequate, good, and excellent fit using the RMSEA2. When the data are ordinal, we find a strong linear relationship between the RMSEA2 and the Standardized Root Mean Squared Residual goodness-of-fit index. We are unable to offer cutoff criteria for the RMSEAn as its population values decrease as the number of variables and categories increase.

Modeling fMRI data: challenges and opportunities

We offer an introduction to the five papers that make up this special section. These papers deal with a range of the methodological challenges that face researchers analyzing fMRI data--the spatial, multilevel, and longitudinal nature of the data, the sources of noise, and so on. The papers all provide analyses of data collected by a multi-site consortium, the Function Biomedical Informatics Research Network. Due to the sheer volume of data, univariate procedures are often applied, which leads to a multiple comparisons problem (since the data are necessarily multivariate). The papers in this section include interesting applications, such as a state-space model applied to these data, and conclude with a reflection on basic measurement problems in fMRI. All in all, they provide a good overview of the challenges that fMRI data present to the standard psychometric toolbox, but also to the opportunities they offer for new psychometric modeling.

How should we assess the fit of Rasch-type models? Approximating the power of goodness-of-fit statistics in categorical data analysis

We investigate the performance of three statistics, R1, R2 (Glas in Psychometrika 53:525-546, 1988), and M2 (Maydeu-Olivares & Joe in J. Am. Stat. Assoc. 100:1009-1020, 2005, Psychometrika 71:713-732, 2006) to assess the overall fit of a one-parameter logistic model (1PL) estimated by (marginal) maximum likelihood (ML). R1 and R2 were specifically designed to target specific assumptions of Rasch models, whereas M2 is a general purpose test statistic. We report asymptotic power rates under some interesting violations of model assumptions (different item discrimination, presence of guessing, and multidimensionality) as well as empirical rejection rates for correctly specified models and some misspecified models. All three statistics were found to be more powerful than Pearson's X(2) against two- and three-parameter logistic alternatives (2PL and 3PL), and against multidimensional 1PL models. The results suggest that there is no clear advantage in using goodness-of-fit statistics specifically designed for Rasch-type models to test these models when marginal ML estimation is used.

How IRT can solve problems of ipsative data in forced-choice questionnaires

In multidimensional forced-choice (MFC) questionnaires, items measuring different attributes are presented in blocks, and participants have to rank order the items within each block (fully or partially). Such comparative formats can reduce the impact of numerous response biases often affecting single-stimulus items (aka rating or Likert scales). However, if scored with traditional methodology, MFC instruments produce ipsative data, whereby all individuals have a common total test score. Ipsative scoring distorts individual profiles (it is impossible to achieve all high or all low scale scores), construct validity (covariances between scales must sum to zero), criterion-related validity (validity coefficients must sum to zero), and reliability estimates. We argue that these problems are caused by inadequate scoring of forced-choice items and advocate the use of item response theory (IRT) models based on an appropriate response process for comparative data, such as Thurstone's law of comparative judgment. We show that when Thurstonian IRT modeling is applied (Brown & Maydeu-Olivares, 2011), even existing forced-choice questionnaires with challenging features can be scored adequately and that the IRT-estimated scores are free from the problems of ipsative data.

Item Response Modeling of Paired Comparison and Ranking Data

The comparative format used in ranking and paired comparisons tasks can significantly reduce the impact of uniform response biases typically associated with rating scales. Thurstone's (1927, 1931) model provides a powerful framework for modeling comparative data such as paired comparisons and rankings. Although Thurstonian models are generally presented as scaling models, that is, stimuli-centered models, they can also be used as person-centered models. In this article, we discuss how Thurstone's model for comparative data can be formulated as item response theory models so that respondents' scores on underlying dimensions can be estimated. Item parameters and latent trait scores can be readily estimated using a widely used statistical modeling program. Simulation studies show that item characteristic curves can be accurately estimated with as few as 200 observations and that latent trait scores can be recovered to a high precision. Empirical examples are given to illustrate how the model may be applied in practice and to recommend guidelines for designing ranking and paired comparisons tasks in the future.

Estimation of IRT graded response models: limited versus full information methods

The performance of parameter estimates and standard errors in estimating F. Samejima's graded response model was examined across 324 conditions. Full information maximum likelihood (FIML) was compared with a 3-stage estimator for categorical item factor analysis (CIFA) when the unweighted least squares method was used in CIFA's third stage. CIFA is much faster in estimating multidimensional models, particularly with correlated dimensions. Overall, CIFA yields slightly more accurate parameter estimates, and FIML yields slightly more accurate standard errors. Yet, across most conditions, differences between methods are negligible. FIML is the best election in small sample sizes (200 observations). CIFA is the best election in larger samples (on computational grounds). Both methods failed in a number of conditions, most of which involved 200 observations, few indicators per dimension, highly skewed items, or low factor loadings. These conditions are to be avoided in applications.

Estimation of IRT graded response models: limited versus full information methods

The performance of parameter estimates and standard errors in estimating F. Samejima's graded response model was examined across 324 conditions. Full information maximum likelihood (FIML) was compared with a 3-stage estimator for categorical item factor analysis (CIFA) when the unweighted least squares method was used in CIFA's third stage. CIFA is much faster in estimating multidimensional models, particularly with correlated dimensions. Overall, CIFA yields slightly more accurate parameter estimates, and FIML yields slightly more accurate standard errors. Yet, across most conditions, differences between methods are negligible. FIML is the best election in small sample sizes (200 observations). CIFA is the best election in larger samples (on computational grounds). Both methods failed in a number of conditions, most of which involved 200 observations, few indicators per dimension, highly skewed items, or low factor loadings. These conditions are to be avoided in applications.

Disentangling impulsiveness, aggressiveness and impulsive aggression: an empirical approach using self-report measures

There is confusion in the literature concerning the concept of impulsive aggression. Based on previous research, we hypothesize that impulsivity and aggression may be related, though not as closely as to consider them the same construct. So, our aim was to provide empirical evidence of the relationship between the impulsivity and aggressiveness constructs when considered as traits. Two widely used questionnaires [Barratt's Impulsiveness Scale (BIS) and Aggression Questionnaire-Refined (AQ-R)] were administered to 768 healthy respondents. Product-moment and canonical correlations were then calculated. In addition, a principal components analysis was conducted to explore whether impulsive aggression can be defined phenotypically as the expression of a single trait. The common variance between impulsivity and aggressiveness was never higher than 42%. The principal components analysis reveals that one component is not enough to represent all the variables. In conclusion, our results show that impulsivity and aggressiveness are two separate, although related constructs. This is particularly important in view of the misconceptions in the literature.

[The development of antisocial behavior: psychobiological and environmental factors and gene-environment interactions]

INTRODUCTION

Antisocial behavior is a complex phenomenon with strong implications in neurology and psychiatry. In order to study the ontogenetic development of antisocial behavior, we must check for the existence of physiological mechanisms related to it, and to understand its environmentally-modulated functioning.

AIM

To review the state-of-the-art of the development of antisocial behavior, and especially, of the interaction between environmental and genetic factors.

DEVELOPMENT

Recent research has highlighted certain brain alterations linked to violent behavior, either at structural, or functional or biochemical levels. Genetic research has also made some advances in this field, discovering some genes--i.e. monoamineoxidase A (MAOA)--related to antisocial behavior. However, the importance of environmental factors in its development must not be left behind. Recent studies have shown that individuals carrying a low transcriptional activity allele of the MAOA gene, and that also suffered severe maltreatment are more prone to antisocial behavior. This interaction is biologically relevant, as there are underlying biological mechanisms that may be able to explain the ethiopathogeny of antisocial behavior.

CONCLUSIONS

Although the works herein presented pioneered the field, they are limited by the fact that all the reviewed variables are associated to antisocial behavior, but they lack direct causal evidence of their effects on antisocial behavior. Undoubtedly, future research on psychobiological mechanisms and the understanding of their environmental modulation will help finding therapeutic targets and preventive strategies for antisocial behavior.

Testing Categorized Bivariate Normality With Two-Stage Polychoric Correlation Estimates

Structural equation modeling (SEM) with ordinal indicators rely on an assumption of categorized normality. This assumption may be tested for pairs of variables using the likelihood ratio G2 or Pearson’s X2 statistics. For increased computational efficiency, SEM programs usually estimate polychoric correlations in two stages. However, two-stage polychoric estimates are not asymptotically efficient and G2 and X2 need not be asymptotically chi-square when the estimator is not efficient. Recently, Maydeu-Olivares and Joe (2005) have introduced a new statistic, Mn , that is asymptotically chi-square even for estimators that are not efficient. We investigate the behavior of G2, X2, and Mn when testing underlying bivariate normality with polychoric correlations estimated in two stages.

Asymptotic Distribution Free Interval Estimation

Abstract. Confidence intervals for the intraclass correlation coefficient (ICC) have been proposed under the assumption of multivariate normality. We propose confidence intervals which do not require distributional assumptions. We performed a simulation study to assess the coverage rates of normal theory (NT) and asymptotically distribution free (ADF) intervals. We found that the ADF intervals performed better than the NT intervals when kurtosis was greater than 4. When violations of distributional assumptions were not too severe, both the intervals performed about the same. The point estimate of the ICC was robust to distributional violations. We provide R code for computing the ADF confidence intervals for the ICC.

Asymptotically distribution-free (ADF) interval estimation of coefficient alpha

The point estimate of sample coefficient alpha may provide a misleading impression of the reliability of the test score. Because sample coefficient alpha is consistently biased downward, it is more likely to yield a misleading impression of poor reliability. The magnitude of the bias is greatest precisely when the variability of sample alpha is greatest (small population reliability and small sample size). Taking into account the variability of sample alpha with an interval estimator may lead to retaining reliable tests that would be otherwise rejected. Here, the authors performed simulation studies to investigate the behavior of asymptotically distribution-free (ADF) versus normal-theory interval estimators of coefficient alpha under varied conditions. Normal-theory intervals were found to be less accurate when item skewness >1 or excess kurtosis >1. For sample sizes over 100 observations, ADF intervals are preferable, regardless of item skewness and kurtosis. A formula for computing ADF confidence intervals for coefficient alpha for tests of any size is provided, along with its implementation as an SAS macro.

Identification and Small Sample Estimation of Thurstone's Unrestricted Model for Paired Comparisons Data

The interpretation of a Thurstonian model for paired comparisons where the utilities' covariance matrix is unrestricted proved to be difficult due to the comparative nature of the data. We show that under a suitable constraint the utilities' correlation matrix can be estimated, yielding a readily interpretable solution. This set of identification constraints can recover any true utilities' covariance matrix, but it is not unique. Indeed, we show how to transform the estimated correlation matrix into alternative correlation matrices that are equally consistent with the data but may be more consistent with substantive theory. Also, we show how researchers can investigate the sample size needed to estimate a particular model by exploiting the simulation capabilities of a popular structural equation modeling statistical package.