A note on identification constraints and information criteria in Bayesian latent variable models

It is well known that, in traditional SEM applications, a scale must be set for each latent variable: typically, either the latent variance or a factor loading is fixed to one. While this has no impact on the fit metrics in ML estimation, it can potentially lead to varying Bayesian model comparison metrics due to the use of different prior distributions under each parameterization. This is a problem, because a researcher could artificially improve one's preferred model simply by changing the identification constraint. Using a single-factor CFA as motivation for study, we first show that Bayesian model comparison metrics can systematically change depending on constraints used. We then study principled methods for setting the scale of the latent variable that stabilize the model comparison metrics. These methods involve (i) the placement of priors on ratios of factor loadings, as opposed to individual loadings; and (ii) use of effect coding. We illustrate the methods via simulation and application.

Dangers of the Defaults: A Tutorial on the Impact of Default Priors When Using Bayesian SEM With Small Samples

When Bayesian estimation is used to analyze Structural Equation Models (SEMs), prior distributions need to be specified for all parameters in the model. Many popular software programs offer default prior distributions, which is helpful for novel users and makes Bayesian SEM accessible for a broad audience. However, when the sample size is small, those prior distributions are not always suitable and can lead to untrustworthy results. In this tutorial, we provide a non-technical discussion of the risks associated with the use of default priors in small sample contexts. We discuss how default priors can unintentionally behave as highly informative priors when samples are small. Also, we demonstrate an online educational Shiny app, in which users can explore the impact of varying prior distributions and sample sizes on model results. We discuss how the Shiny app can be used in teaching; provide a reading list with literature on how to specify suitable prior distributions; and discuss guidelines on how to recognize (mis)behaving priors. It is our hope that this tutorial helps to spread awareness of the importance of specifying suitable priors when Bayesian SEM is used with small samples.

Psychometric evaluation of the Working Alliance Inventory-Therapist version: Current and new short forms

Background: The Working Alliance Inventory (WAI) and its short forms are widely used, although the properties of the therapists' versions have been little studied. Method: We examined the psychometric properties of two short forms (WAI-S-T, WAI-SR-T), and explored the creation of a psychometrically stronger short form using contemporary measure development techniques. Well-fitting items from the full 36-item WAI were identified in a development sample (131 therapists, 688 patients) using multi-level Bayesian Structural Equation Modeling, accounting for therapist rated effects. Multi-level Item Response Theory (IRT) methods aided creation of a revised short form (WAI-S-T-IRT). Factor structures of the three forms were assessed using multi-level ML estimation with robust standard errors. Results: Collinearity problems for the Goal and Task dimensions led to testing a two-factor model (Goal-Task, Bond). All three measures showed satisfactory fit; the WAI-S-T-IRT fit slightly better but differences were minor. Testing the structures in an independent sample (N = 1117) yielded essentially the same results. No version showed strong measurement invariance. Discussion: Continued use of current therapist forms is supported; differentiation of theoretical dimensions is difficult with current measures, and may not be possible with self-report forms.

Evaluating Factorial Invariance: An Interval Estimation Approach Using Bayesian Structural Equation Modeling

In this study, we introduce an interval estimation approach based on Bayesian structural equation modeling to evaluate factorial invariance. For each tested parameter, the size of noninvariance with an uncertainty interval (i.e. highest density interval [HDI]) is assessed via Bayesian parameter estimation. By comparing the most credible values (i.e. 95% HDI) with a region of practical equivalence (ROPE), the Bayesian approach allows researchers to (1) support the null hypothesis of practical invariance, and (2) examine the practical importance of the noninvariant parameter. Compared to the traditional likelihood ratio test, simulation results suggested that the proposed Bayesian approach could offer additional insight into evaluating factorial invariance, thus, leading to more informative conclusions. We provide an empirical example to demonstrate the procedures necessary to implement the proposed method in applied research. The importance of and influences on the choice of an appropriate ROPE are discussed.