Recognize the Value of the Sum Score, Psychometrics' Greatest Accomplishment

The sum score on a psychological test is, and should continue to be, a tool central in psychometric practice. This position runs counter to several psychometricians' belief that the sum score represents a pre-scientific conception that must be abandoned from psychometrics in favor of latent variables. First, we reiterate that the sum score stochastically orders the latent variable in a wide variety of much-used item response models. In fact, item response theory provides a mathematically based justification for the ordinal use of the sum score. Second, because discussions about the sum score often involve its reliability and estimation methods as well, we show that, based on very general assumptions, classical test theory provides a family of lower bounds several of which are close to the true reliability under reasonable conditions. Finally, we argue that eventually sum scores derive their value from the degree to which they enable predicting practically relevant events and behaviors. None of our discussion is meant to discredit modern measurement models; they have their own merits unattainable for classical test theory, but the latter model provides impressive contributions to psychometrics based on very few assumptions that seem to have become obscured in the past few decades. Their generality and practical usefulness add to the accomplishments of more recent approaches.

In models we trust: preregistration, large samples, and replication may not suffice

Despite discussions about the replicability of findings in psychological research, two issues have been largely ignored: selection mechanisms and model assumptions. Both topics address the same fundamental question: Does the chosen statistical analysis tool adequately model the data generation process? In this article, we address both issues and show, in a first step, that in the face of selective samples and contrary to common practice, the validity of inferences, even when based on experimental designs, can be claimed without further justification and adaptation of standard methods only in very specific situations. We then broaden our perspective to discuss consequences of violated assumptions in linear models in the context of psychological research in general and in generalized linear mixed models as used in item response theory. These types of misspecification are oftentimes ignored in the psychological research literature. It is emphasized that the above problems cannot be overcome by strategies such as preregistration, large samples, replications, or a ban on testing null hypotheses. To avoid biased conclusions, we briefly discuss tools such as model diagnostics, statistical methods to compensate for selectivity and semi- or non-parametric estimation. At a more fundamental level, however, a twofold strategy seems indispensable: (1) iterative, cumulative theory development based on statistical methods with theoretically justified assumptions, and (2) empirical research on variables that affect (self-) selection into the observed part of the sample and the use of this information to compensate for selectivity.

A Test to Distinguish Monotone Homogeneity from Monotone Multifactor Models

The goodness-of-fit of the unidimensional monotone latent variable model can be assessed using the empirical conditions of nonnegative correlations (Mokken in A theory and procedure of scale-analysis, Mouton, The Hague, 1971), manifest monotonicity (Junker in Ann Stat 21:1359-1378, 1993), multivariate total positivity of order 2 (Bartolucci and Forcina in Ann Stat 28:1206-1218, 2000), and nonnegative partial correlations (Ellis in Psychometrika 79:303-316, 2014). We show that multidimensional monotone factor models with independent factors also imply these empirical conditions; therefore, the conditions are insensitive to multidimensionality. Conditional association (Rosenbaum in Psychometrika 49(3):425-435, 1984) can detect multidimensionality, but tests of it (De Gooijer and Yuan in Comput Stat Data Anal 55:34-44, 2011) are usually not feasible for realistic numbers of items. The only existing feasible test procedures that can reveal multidimensionality are Rosenbaum's (Psychometrika 49(3):425-435, 1984) Case 2 and Case 5, which test the covariance of two items or two subtests conditionally on the unweighted sum of the other items. We improve this procedure by conditioning on a weighted sum of the other items. The weights are estimated in a training sample from a linear regression analysis. Simulations show that the Type I error rate is under control and that, for large samples, the power is higher if one dimension is more important than the other or if there is a third dimension. In small samples and with two equally important dimensions, using the unweighted sum yields greater power.

A Comparison of Confirmatory Factor Analysis and Network Models for Measurement Invariance Assessment When Indicator Residuals are Correlated

Social science research is heavily dependent on the use of standardized assessments of a variety of phenomena, such as mood, executive functioning, and cognitive ability. An important assumption when using these instruments is that they perform similarly for all members of the population. When this assumption is violated, the validity evidence of the scores is called into question. The standard approach for assessing the factorial invariance of the measures across subgroups within the population involves multiple groups confirmatory factor analysis (MGCFA). CFA models typically, but not always, assume that once the latent structure of the model is accounted for, the residual terms for the observed indicators are uncorrelated (local independence). Commonly, correlated residuals are introduced after a baseline model shows inadequate fit and inspection of modification indices ensues to remedy fit. An alternative procedure for fitting latent variable models that may be useful when local independence does not hold is based on network models. In particular, the residual network model (RNM) offers promise with respect to fitting latent variable models in the absence of local independence via an alternative search procedure. This simulation study compared the performances of MGCFA and RNM for measurement invariance assessment when local independence is violated, and residual covariances are themselves not invariant. Results revealed that RNM had better Type I error control and higher power compared to MGCFA when local independence was absent. Implications of the results for statistical practice are discussed.

Incomplete Tests of Conditional Association for the Assessment of Model Assumptions

Many of the models that have been proposed for response data share the assumptions that define the monotone homogeneity (MH) model. Observable properties that are implied by the MH model allow for these assumptions to be tested. For binary response data, the most restrictive of these properties is called conditional association (CA). All the other properties considered can be considered incomplete tests of CA that alleviate the practical limitations encountered when assessing the MH model assumptions using CA. It is found that the assessment of the MH model assumptions with an incomplete test of CA, rather than CA, is generally associated with a substantial loss of information. We also look at the sensitivity of the observable properties to model violation and discuss the implications of the results. It is argued that more research is required about the extent to which the assumptions and the model specifications influence the inferences made from response data.

Item Response Thresholds Models: A General Class of Models for Varying Types of Items

A comprehensive class of models is proposed that can be used for continuous, binary, ordered categorical and count type responses. The difficulty of items is described by difficulty functions, which replace the item difficulty parameters that are typically used in item response models. They crucially determine the response distribution and make the models very flexible with regard to the range of distributions that are covered. The model class contains several widely used models as the binary Rasch model and the graded response model as special cases, allows for simplifications, and offers a distribution free alternative to count type items. A major strength of the models is that they can be used for mixed item formats, when different types of items are combined to measure abilities or attitudes. It is an immediate consequence of the comprehensive modeling approach that allows that difficulty functions automatically adapt to the response distribution. Basic properties of the model class are shown. Several real data sets are used to illustrate the flexibility of the models.

Idiographic Ising and Divide and Color Models: Encompassing Networks for Heterogeneous Binary Data

The Ising model is a graphical model that has played an essential role in network psychometrics. It has been used as a theoretical model to conceptualize psychological concepts and as a statistical model to analyze psychological data. Using graphical models such as the Ising model to analyze psychological data has been heavily critiqued since these data often come from cross-sectional applications. An often voiced concern is the inability of the Ising model to express heterogeneity in the population. The idiographic approach has been posed as an alternative and aims to infer individual network structures. While idiographic networks overcome population heterogeneity, it is unclear how they aggregate into established cross-sectional phenomena. This paper establishes a formal bridge between idiographic and cross-sectional network approaches of the Ising model. We ascertain unique topological structures that characterize individuals and aggregate into an Ising model cross-sectionally. This new formulation supports population heterogeneity while being consistent with cross-sectional phenomena. The proposed theory also establishes a new statistical framework for analyzing populations of idiographic networks for binary variables. The Ising model and the divide and color model are special cases of this new framework. We introduce a Gibbs sampling algorithm to estimate models from this new framework.

Relations between Networks, Regression, Partial Correlation, and the Latent Variable Model

The Gaussian graphical model (GGM) has become a popular tool for analyzing networks of psychological variables. In a recent article in this journal, Forbes, Wright, Markon, and Krueger (FWMK) voiced the concern that GGMs that are estimated from partial correlations wrongfully remove the variance that is shared by its constituents. If true, this concern has grave consequences for the application of GGMs. Indeed, if partial correlations only capture the unique covariances, then the data that come from a unidimensional latent variable model ULVM should be associated with an empty network (no edges), as there are no unique covariances in a ULVM. We know that this cannot be true, which suggests that FWMK are missing something with their claim. We introduce a connection between the ULVM and the GGM and use that connection to prove that we find a fully-connected and not an empty network associated with a ULVM. We then use the relation between GGMs and linear regression to show that the partial correlation indeed does not remove the common variance.

Quantifying and addressing the impact of measurement error in network models

Network psychometric models are often estimated using a single indicator for each node in the network, thus failing to consider potential measurement error. In this study, we investigate the impact of measurement error on cross-sectional network models. First, we conduct a simulation study to evaluate the performance of models based on single indicators as well as models that utilize information from multiple indicators per node, including average scores, factor scores, and latent variables. Our results demonstrate that measurement error impairs the reliability and performance of network models, especially when using single indicators. The reliability and performance of network models improves substantially with increasing sample size and when using methods that combine information from multiple indicators per node. Second, we use empirical data from the STAR*D trial (n = 3,731) to further evaluate the impact of measurement error. In the STAR*D trial, depression symptoms were assessed via three questionnaires, providing multiple indicators per symptom. Consistent with our simulation results, we find that when using sub-samples of this dataset, the discrepancy between the three single-indicator networks (one network per questionnaire) diminishes with increasing sample size. Together, our simulated and empirical findings provide evidence that measurement error can hinder network estimation when working with smaller samples and offers guidance on methods to mitigate measurement error.

Why are beliefs in different conspiracy theories positively correlated across individuals? Testing monological network versus unidimensional factor model explanations

A substantial minority of the public express belief in conspiracy theories. A robust phenomenon in this area is that people who believe one conspiracy theory are more likely to believe in others. But the reason for this "positive manifold" of belief in conspiracy theories is unclear. One possibility is that a single underlying latent factor (e.g. "conspiracism") causes variation in belief in specific conspiracy theories. Another possibility is that beliefs in various conspiracy theories support one another in a mutually reinforcing network of beliefs (the "monological belief system" theory). While the monological theory has been influential in the literature, the fact that it can be operationalised as a statistical network model has not previously been recognised. In this study, we therefore tested both the unidimensional factor model and a network model. Participants were 1553 American adults recruited via Prolific. Belief in conspiracies was measured using an adapted version of the Belief in Conspiracy Theories Inventory. The fit of the two competing models was evaluated both by using van Bork et al.'s (Psychometrika, 83, 2018, 443, Multivariate Behavioral Research, 56, 2019, 175) method for testing network versus unidimensional factor models, as well as by evaluating goodness of fit to the sample covariance matrix. In both cases, evaluation of fit according to our pre-registered inferential criteria favoured the network model.

Evaluating the Test Validity of the Comprehensive Assessment of Psychopathic Personality Symptom Rating Sale (CAPP SRS)

The Comprehensive Assessment of Psychopathic Personality Symptom Rating Scale (CAPP SRS) is a relatively new measure of psychopathic personality disorder (PPD) based on the CAPP concept map of psychopathy. To investigate the CAPP SRS, we identified the most plausible formal test structure for the test using the framework proposed by Slaney and Maraun, identified an appropriate quantitative characterization of that test structure, and then statistically evaluated it based on analysis of CAPP SRS data collected from a multisite sample of 314 adult male correctional offenders and secure hospital patients in Scotland and England. Overall, the CAPP SRS survived falsification when observed test data were compared to expectations based on the unidimensional monotone latent variable or UMLV model of Holland and Rosenbaum. CAPP SRS composite scores calculated consistent with the UMLV model had good measurement precision and good external validity with respect to scores on an established test of PPD. The findings provide provisional support for the test validity of the CAPP SRS and highlight the importance of theory-driven evaluations of test validity.

Latent Variable Models and Networks: Statistical Equivalence and Testability

Networks are gaining popularity as an alternative to latent variable models for representing psychological constructs. Whereas latent variable approaches introduce unobserved common causes to explain the relations among observed variables, network approaches posit direct causal relations between observed variables. While these approaches lead to radically different understandings of the psychological constructs of interest, recent articles have established mathematical equivalences that hold between network models and latent variable models. We argue that the fact that for any model from one class there is an equivalent model from the other class does not mean that both models are equally plausible accounts of the data-generating mechanism. In many cases the constraints that are meaningful in one framework translate to constraints in the equivalent model that lack a clear interpretation in the other framework. Finally, we discuss three diverging predictions for the relation between zero-order correlations and partial correlations implied by sparse network models and unidimensional factor models. We propose a test procedure that compares the likelihoods of these models in light of these diverging implications. We use an empirical example to illustrate our argument.

Studying Enhanced Recovery After Surgery (ERAS®) Core Items in Colorectal Surgery: A Causal Model with Latent Variables

Background

Previous Enhanced Recovery After Surgery (ERAS®) studies have not always taken into account that ERAS interventions depend on baseline covariates and that several confounding variables affect the composite outcomes.

Method

A causal latent variable model is proposed to analyze data obtained prospectively concerning 1261 patients undergoing elective colorectal surgery within the ERAS protocol. Primary outcomes (composite of any complication, surgical site infection, medical complications, early ready for discharge (TRD), early actual discharge) and secondary outcomes (composite of late bowel function recovery, IV fluid resumption, nasogastric tube replacement, postoperative nausea and vomiting, re-intervention, re-admission, death) are considered along with their multiple dimensions.

Results

Concerning the primary outcomes, our results evidence three subpopulations of patients: one with probable good outcome, one with possibly prolonged TRD and discharge without complications, and the other one with probable complications and prolonged TRD and discharge. Epidural anesthesia, waiving surgical drainage, and early ambulation, IV fluid stop and urinary catheter removal act favorably, while preoperative hospital stay and blood transfusion act negatively. Concerning the secondary outcomes our results evidence two subpopulations of patients: one with high probability of good outcome and one with high probability of complications. Epidural anesthesia, waiving surgical drainage, early ambulation and IV fluid stop act favorably, while blood transfusion acts negatively also with respect to these secondary outcomes.

Conclusion

The multivariate causal latent class two-parameter logistic model, a modern statistical method overcoming drawbacks of traditional models to estimate the average causal effects on the treated, allows us to disentangle subpopulations of patients and to evaluate ERAS interventions.

Why we should move from reductionism and embrace a network approach to parental burnout

Network science has allowed varied scientific fields to investigate and visualize complex relations between many variables, and psychology research has begun to adopt a network perspective. In this paper, we consider how leaving behind reductionist approaches and instead embracing a network perspective can advance the field of parental burnout. Although research into parental burnout is in its early stages, we argue that a network approach to parental burnout could set the scene for radically new vistas in parental burnout research. We claim that such an approach can allow simultaneous investigations (and clear visualizations) of many variables related to parental burnout and their interactions, integrates smoothly with prior family systems theories, and prioritizes dynamic research questions. We likewise discuss potential future clinical applications, such as interventions targeting central nodes and treatment personalized to a specific family's network system. We also review practical considerations, limitations, and future directions for researchers interested in applying a network approach to parental burnout research.

Using Medicare claims in identifying Alzheimer's disease and related dementias

INTRODUCTION

This study develops a measure of Alzheimer's disease and related dementias (ADRD) using Medicare claims.

METHODS

Validation resembles the approach of the American Psychological Association, including (1) content validity, (2) construct validity, and (3) predictive validity.

RESULTS

We found that four items-a Medicare claim recording ADRD 1 year ago, 2 years ago, 3 years ago, and a total stay of 6 months in a nursing home-exhibit a pattern of association consistent with a single underlying ADRD construct, and presence of any two of these four items predict a direct measure of cognitive function and also future claims for ADRD.

DISCUSSION

Our four items are internally consistent with the measurement of a single quantity. The presence of any two items do a better job than a single claim when predicting both a direct measure of cognitive function and future ADRD claims.

Gaining power in multiple testing of interval hypotheses via conditionalization

In this article, we introduce a novel procedure for improving power of multiple testing procedures (MTPs) of interval hypotheses. When testing interval hypotheses the null hypothesis $P$-values tend to be stochastically larger than standard uniform if the true parameter is in the interior of the null hypothesis. The new procedure starts with a set of $P$-values and discards those with values above a certain pre-selected threshold, while the rest are corrected (scaled-up) by the value of the threshold. Subsequently, a chosen family-wise error rate (FWER) or false discovery rate MTP is applied to the set of corrected $P$-values only. We prove the general validity of this procedure under independence of $P$-values, and for the special case of the Bonferroni method, we formulate several sufficient conditions for the control of the FWER. It is demonstrated that this "filtering" of $P$-values can yield considerable gains of power.

Toward an Integrative Psychometric Model of Emotions

Emotions are part and parcel of the human condition, but their nature is debated. Three broad classes of theories about the nature of emotions can be distinguished: affect-program theories, constructionist theories, and appraisal theories. Integrating these broad classes of theories into a unifying theory is challenging. An integrative psychometric model of emotions can inform such a theory because psychometric models are intertwined with theoretical perspectives about constructs. To identify an integrative psychometric model, we delineate properties of emotions stated by emotion theories and investigate whether psychometric models account for these properties. Specifically, an integrative psychometric model of emotions should allow (a) identifying distinct emotions (central in affect-program theories), (b) between- and within-person variations of emotions (central in constructionist theories), and (c) causal relationships between emotion components (central in appraisal theories). Evidence suggests that the popular reflective and formative latent variable models-in which emotions are conceptualized as unobservable causes or consequences of emotion components-cannot account for all properties. Conversely, a psychometric network model-in which emotions are conceptualized as systems of causally interacting emotion components-accounts for all properties. The psychometric network model thus constitutes an integrative psychometric model of emotions, facilitating progress toward a unifying theory.

False discovery rate control for multiple testing based on discrete p-values

For multiple testing based on discrete p-values, we propose a false discovery rate (FDR) procedure "BH+" with proven conservativeness. BH+ is at least as powerful as the BH (i.e., Benjamini-Hochberg) procedure when they are applied to superuniform p-values. Further, when applied to mid-p-values, BH+ can be more powerful than it is applied to conventional p-values. An easily verifiable necessary and sufficient condition for this is provided. BH+ is perhaps the first conservative FDR procedure applicable to mid-p-values and to p-values with general distributions. It is applied to multiple testing based on discrete p-values in a methylation study, an HIV study and a clinical safety study, where it makes considerably more discoveries than the BH procedure. In addition, we propose an adaptive version of the BH+ procedure, prove its conservativeness under certain conditions, and provide evidence on its excellent performance via simulation studies.

The Wiring of Intelligence

The positive manifold of intelligence has fascinated generations of scholars in human ability. In the past century, various formal explanations have been proposed, including the dominant g factor, the revived sampling theory, and the recent multiplier effect model and mutualism model. In this article, we propose a novel idiographic explanation. We formally conceptualize intelligence as evolving networks in which new facts and procedures are wired together during development. The static model, an extension of the Fortuin-Kasteleyn model, provides a parsimonious explanation of the positive manifold and intelligence's hierarchical factor structure. We show how it can explain the Matthew effect across developmental stages. Finally, we introduce a method for studying growth dynamics. Our truly idiographic approach offers a new view on a century-old construct and ultimately allows the fields of human ability and human learning to coalesce.

Diagnostic Test Score Validation With a Fallible Criterion

Criterion-related validation of diagnostic test scores for a construct of interest is complicated by the unavailability of the construct directly. The standard method, Known Group Validation, assumes an infallible reference test in place of the construct, but infallible reference tests are rare. In contrast, Mixed Group Validation allows for a fallible reference test, but has been found to make strong assumptions not appropriate for the majority of diagnostic test validation studies. The Neighborhood model is adapted for the purpose of diagnostic test validation, which makes alternate, but also strong, assumptions. The statistical properties of the Neighborhood model are evaluated and the assumptions are reviewed in the context of diagnostic test validation. Alternatively, strong assumptions may be avoided by estimating only intervals for the validity estimates, instead of point estimates. The Method of Bounds is also adapted for the purpose of diagnostic test validation, and an extension, Method of Bounds-Test Validation, is introduced here for the first time. All three point-estimate methods were found to make strong assumptions concerning the conditional relationships between the tests and the construct of interest, and all three lack robustness to assumption violation. The Method of Bounds-Test Validation was found to perform well across a range of plausible simulated datasets where the point-estimate methods failed. The point-estimate methods are recommended in special cases where the assumptions can be justified, while the interval methods are appropriate more generally.

Effects of Discontinue Rules on Psychometric Properties of Test Scores

This paper provides results on a form of adaptive testing that is used frequently in intelligence testing. In these tests, items are presented in order of increasing difficulty. The presentation of items is adaptive in the sense that a session is discontinued once a test taker produces a certain number of incorrect responses in sequence, with subsequent (not observed) responses commonly scored as wrong. The Stanford-Binet Intelligence Scales (SB5; Riverside Publishing Company, 2003) and the Kaufman Assessment Battery for Children (KABC-II; Kaufman and Kaufman, 2004), the Kaufman Adolescent and Adult Intelligence Test (Kaufman and Kaufman 2014) and the Universal Nonverbal Intelligence Test (2nd ed.) (Bracken and McCallum 2015) are some of the many examples using this rule. He and Wolfe (Educ Psychol Meas 72(5):808-826, 2012. https://doi.org/10.1177/0013164412441937 ) compared different ability estimation methods in a simulation study for this discontinue rule adaptation of test length. However, there has been no study, to our knowledge, of the underlying distributional properties based on analytic arguments drawing on probability theory, of what these authors call stochastic censoring of responses. The study results obtained by He and Wolfe (Educ Psychol Meas 72(5):808-826, 2012. https://doi.org/10.1177/0013164412441937 ) agree with results presented by DeAyala et al. (J Educ Meas 38:213-234, 2001) as well as Rose et al. (Modeling non-ignorable missing data with item response theory (IRT; ETS RR-10-11), Educational Testing Service, Princeton, 2010) and Rose et al. (Psychometrika 82:795-819, 2017. https://doi.org/10.1007/s11336-016-9544-7 ) in that ability estimates are biased most when scoring the not observed responses as wrong. This scoring is used operationally, so more research is needed in order to improve practice in this field. The paper extends existing research on adaptivity by discontinue rules in intelligence tests in multiple ways: First, an analytical study of the distributional properties of discontinue rule scored items is presented. Second, a simulation is presented that includes additional scoring rules and uses ability estimators that may be suitable to reduce bias for discontinue rule scored intelligence tests.

Symptoms of social anxiety disorder and major depressive disorder: A network perspective

BACKGROUND

We used network analyses to examine symptoms that may play a role in the co-occurrence of social anxiety disorder (SAD) and major depressive disorder (MDD). Whereas latent variable models examine relations among latent constructs, network analyses have the advantage of characterizing direct relations among the symptoms themselves.

METHOD

We conducted network modeling on symptoms of social anxiety and depression in a clinical sample of 130 women who met criteria for SAD, MDD, both disorders, or had no lifetime history of mental illness.

RESULTS

In the resulting network, the core symptoms of social fear and depressed mood appeared at opposite ends of the network and were weakly related; so-called "bridges" between these symptoms appeared to occur via intervening variables. In particular, the worthless variable appeared to play a central role in the network.

LIMITATIONS

Because our data were cross-sectional, we are unable to draw conclusions about the direction of these effects or whether these variables are related to each other prospectively.

CONCLUSIONS

Continued testing of these pathways using longitudinal data will help facilitate the development of more effective clinical interventions for these disorders.

Psychometric Models of Small Group Collaborations

The social combination theory of group problem solving is used to extend existing psychometric models to collaborative settings. A model for pairwise group work is proposed, the implications of the model for assessment design are considered, and its estimation is addressed. The results are illustrated with an empirical example in which dyads work together on a twelfth-grade level mathematics assessment. In conclusion, attention is given to avenues of research that seem most fruitful for advancing current initiatives concerning the assessment of collaboration, teamwork, and related constructs.

The Gaussian Graphical Model in Cross-Sectional and Time-Series Data

We discuss the Gaussian graphical model (GGM; an undirected network of partial correlation coefficients) and detail its utility as an exploratory data analysis tool. The GGM shows which variables predict one-another, allows for sparse modeling of covariance structures, and may highlight potential causal relationships between observed variables. We describe the utility in three kinds of psychological data sets: data sets in which consecutive cases are assumed independent (e.g., cross-sectional data), temporally ordered data sets (e.g., n = 1 time series), and a mixture of the 2 (e.g., n > 1 time series). In time-series analysis, the GGM can be used to model the residual structure of a vector-autoregression analysis (VAR), also termed graphical VAR. Two network models can then be obtained: a temporal network and a contemporaneous network. When analyzing data from multiple subjects, a GGM can also be formed on the covariance structure of stationary means-the between-subjects network. We discuss the interpretation of these models and propose estimation methods to obtain these networks, which we implement in the R packages graphicalVAR and mlVAR. The methods are showcased in two empirical examples, and simulation studies on these methods are included in the supplementary materials.

Unidimensional factor models imply weaker partial correlations than zero-order correlations

In this paper we present a new implication of the unidimensional factor model. We prove that the partial correlation between two observed variables that load on one factor given any subset of other observed variables that load on this factor lies between zero and the zero-order correlation between these two observed variables. We implement this result in an empirical bootstrap test that rejects the unidimensional factor model when partial correlations are identified that are either stronger than the zero-order correlation or have a different sign than the zero-order correlation. We demonstrate the use of the test in an empirical data example with data consisting of fourteen items that measure extraversion.

Deconstructing trait anxiety: a network perspective

BACKGROUND AND OBJECTIVES

For decades, the dominant paradigm in trait anxiety research has regarded the construct as signifying the underlying cause of the thoughts, feelings, and behaviors that supposedly reflect its presence. Recently, a network theory of personality has appeared. According to this perspective, trait anxiety is a formative construct emerging from interactions among its constitutive features (e.g., thought, feelings, behaviors); it is not a latent cause of these features.

DESIGN

In this study, we characterized trait anxiety as a network system of interacting elements.

METHODS

To do so, we estimated a graphical gaussian model via the computation of a regularized partial correlation network in an unselected sample (N = 611). We also implemented modularity-based community detection analysis to test whether the features of trait anxiety cohere as a single network system.

RESULTS

We find that trait anxiety can indeed be conceptualized as a single, coherent network system of interacting elements.

CONCLUSIONS

This radically new approach to visualizing trait anxiety may offer an especially informative view of the interplay between its constitutive features. As prior research has implicated trait anxiety as a risk factor for the development of anxiety-related psychopathology, our findings also set the scene for novel research directions.

Generalized Network Psychometrics: Combining Network and Latent Variable Models

We introduce the network model as a formal psychometric model, conceptualizing the covariance between psychometric indicators as resulting from pairwise interactions between observable variables in a network structure. This contrasts with standard psychometric models, in which the covariance between test items arises from the influence of one or more common latent variables. Here, we present two generalizations of the network model that encompass latent variable structures, establishing network modeling as parts of the more general framework of structural equation modeling (SEM). In the first generalization, we model the covariance structure of latent variables as a network. We term this framework latent network modeling (LNM) and show that, with LNM, a unique structure of conditional independence relationships between latent variables can be obtained in an explorative manner. In the second generalization, the residual variance-covariance structure of indicators is modeled as a network. We term this generalization residual network modeling (RNM) and show that, within this framework, identifiable models can be obtained in which local independence is structurally violated. These generalizations allow for a general modeling framework that can be used to fit, and compare, SEM models, network models, and the RNM and LNM generalizations. This methodology has been implemented in the free-to-use software package lvnet, which contains confirmatory model testing as well as two exploratory search algorithms: stepwise search algorithms for low-dimensional datasets and penalized maximum likelihood estimation for larger datasets. We show in simulation studies that these search algorithms perform adequately in identifying the structure of the relevant residual or latent networks. We further demonstrate the utility of these generalizations in an empirical example on a personality inventory dataset.

Dimensionality of the Chinese hospital anxiety depression scale in psychiatric outpatients: Mokken scale and factor analyses

OBJECTIVE

The hospital anxiety and depression scale (HADS) is a widely used scale of anxiety and depression. However, recent studies have challenged the bi-dimensional scoring of the HADS. The present study was to examine the dimensionality of the Chinese HADS.

METHODS

We recruited a convenience sample of 214 adult psychiatric outpatients at a medical centre in Taiwan, and they completed the Chinese HADS. We used Mokken scale analysis (MSA), exploratory factor analysis (EFA), exploratory bifactor analysis (EBA) and confirmatory factor analysis (CFA) to examine the dimensionality of the Chinese HADS.

RESULTS

The Chinese HADS was a moderate Mokken scale (H_s = 0.44), and had a two-factor structure. EBA showed that one general factor, emotional distress, explained 68% of the common variance of the Chinese HADS. CFA confirmed that the bifactor model had the best fit statistics. The items 5 and 7 of the Chinese HADS contributed to structural ambiguity in the Chinese HADS subscales.

CONCLUSIONS

The sum scores of the Chinese HADS were a reliable and valid unidimensional measure of emotional distress. The Chinese HADS subscales were incapable of differentiating between anxiety and depression. Clinicians and researchers should choose other scales that are specifically designed for measuring anxiety and depression.

Psychometric analysis of the Generalized Anxiety Disorder scale (GAD-7) in primary care using modern item response theory

OBJECTIVE

The Generalized Anxiety Disorder scale (GAD-7) is one of the most frequently used diagnostic self-report scales for screening, diagnosis and severity assessment of anxiety disorder. Its psychometric properties from the view of the Item Response Theory paradigm have rarely been investigated. We aimed to close this gap by analyzing the GAD-7 within a large sample of primary care patients with respect to its psychometric properties and its implications for scoring using Item Response Theory.

METHODS

Robust, nonparametric statistics were used to check unidimensionality of the GAD-7. A graded response model was fitted using a Bayesian approach. The model fit was evaluated using posterior predictive p-values, item information functions were derived and optimal predictions of anxiety were calculated.

RESULTS

The sample included N = 3404 primary care patients (60% female; mean age, 52,2; standard deviation 19.2) The analysis indicated no deviations of the GAD-7 scale from unidimensionality and a decent fit of a graded response model. The commonly suggested ultra-brief measure consisting of the first two items, the GAD-2, was supported by item information analysis. The first four items discriminated better than the last three items with respect to latent anxiety.

CONCLUSION

The information provided by the first four items should be weighted more heavily. Moreover, estimates corresponding to low to moderate levels of anxiety show greater variability. The psychometric validity of the GAD-2 was supported by our analysis.

A tutorial on how to do a Mokken scale analysis on your test and questionnaire data

Over the past decade, Mokken scale analysis (MSA) has rapidly grown in popularity among researchers from many different research areas. This tutorial provides researchers with a set of techniques and a procedure for their application, such that the construction of scales that have superior measurement properties is further optimized, taking full advantage of the properties of MSA. First, we define the conceptual context of MSA, discuss the two item response theory (IRT) models that constitute the basis of MSA, and discuss how these models differ from other IRT models. Second, we discuss dos and don'ts for MSA; the don'ts include misunderstandings we have frequently encountered with researchers in our three decades of experience with real-data MSA. Third, we discuss a methodology for MSA on real data that consist of a sample of persons who have provided scores on a set of items that, depending on the composition of the item set, constitute the basis for one or more scales, and we use the methodology to analyse an example real-data set.

Refinements of Stout’s Procedure for Assessing Latent Trait Unidimensionality

This article provides a detailed investigation of Stout’s statistical procedure (the computer program DIMTEST) for testing the hypothesis that an essentially unidimensional latent trait model fits observed binary item response data from a psychological test. One finding was that DIMTEST may fail to perform as desired in the presence of guessing when coupled with many high-discriminating items. A revision of DIMTEST is proposed to overcome this limitation. Also, an automatic approach is devised to determine the size of the assessment subtests. Further, an adjustment is made on the estimated standard error of the statistic on which DIMTEST depends. These three refinements have led to an improved procedure that is shown in simulation studies to adhere closely to the nominal level of signficance while achieving considerably greater power. Finally, DIMTEST is validated on a selection of real data sets.

Using Conditional Association to Identify Locally Independent Item Sets

Abstract. The ordinal, unidimensional monotone latent variable model assumes unidimensionality, local independence, and monotonicity, and implies the observable property of conditional association. We investigated three special cases of conditional association and implemented them in a new procedure that aims at identifying locally dependent items, removing these items from the initial item set, and producing an item subset that is locally independent. A simulation study showed that the new procedure correctly identified 89.5% of the model-consistent items and up to 90% of the model-inconsistent items. We recommend using this procedure for selecting locally independent item sets. The procedure may be used in combination with Mokken scale analysis.

On conditional stochastic ordering of distributions

Conditional stochastic ordering is concerned with the stochastic ordering of a pair of probability measures conditional on certain subsets or sub-σ -algebras. Some basic results of conditional stochastic ordering were proved by Whitt. We extend some of Whitt's results and prove a basic relation between stochastic ordering conditional on subsets and stochastic ordering conditional on σ -algebras. In the second part of the paper we consider the ordering of conditional expectations. There are several different formulations of this problem motivated by different types of applications.

On conditional stochastic ordering of distributions

Conditional stochastic ordering is concerned with the stochastic ordering of a pair of probability measures conditional on certain subsets or sub-σ-algebras. Some basic results of conditional stochastic ordering were proved by Whitt. We extend some of Whitt's results and prove a basic relation between stochastic ordering conditional on subsets and stochastic ordering conditional onσ-algebras. In the second part of the paper we consider the ordering of conditional expectations. There are several different formulations of this problem motivated by different types of applications.

A Mokken Scale for Transitive Reasoning Suited for Longitudinal Research

A set of transitivity tasks was solved on two occasions by 634 2nd through 6th grade primary school children. Transitivity scales were constructed for both occasions using Mokken scale analysis. Combining the results, one final scale was constructed for use in future research. The order of the scale tasks according to their difficulty level was the same on both occasions. Analysis of the scale scores of one group retested after 16 weeks and two control groups revealed an absence of effects due to memorisation or test experience. It was concluded that the final scale is useful as a transitivity measure in longitudinal research.

Remarks and a Correction of Ligtvoet's Treatment of the Isotonic Partial Credit Model

This note contains some remarks on Ligtvoet's (Psychometrika, 77:479-494, 2012) treatment of the isotonic partial credit model. The Proposition relating to the observable property MWI is shown to be false.

Problematic assumptions have slowed down depression research: why symptoms, not syndromes are the way forward

Major depression (MD) is a highly heterogeneous diagnostic category. Diverse symptoms such as sad mood, anhedonia, and fatigue are routinely added to an unweighted sum-score, and cutoffs are used to distinguish between depressed participants and healthy controls. Researchers then investigate outcome variables like MD risk factors, biomarkers, and treatment response in such samples. These practices presuppose that (1) depression is a discrete condition, and that (2) symptoms are interchangeable indicators of this latent disorder. Here I review these two assumptions, elucidate their historical roots, show how deeply engrained they are in psychological and psychiatric research, and document that they contrast with evidence. Depression is not a consistent syndrome with clearly demarcated boundaries, and depression symptoms are not interchangeable indicators of an underlying disorder. Current research practices lump individuals with very different problems into one category, which has contributed to the remarkably slow progress in key research domains such as the development of efficacious antidepressants or the identification of biomarkers for depression. The recently proposed network framework offers an alternative to the problematic assumptions. MD is not understood as a distinct condition, but as heterogeneous symptom cluster that substantially overlaps with other syndromes such as anxiety disorders. MD is not framed as an underlying disease with a number of equivalent indicators, but as a network of symptoms that have direct causal influence on each other: insomnia can cause fatigue which then triggers concentration and psychomotor problems. This approach offers new opportunities for constructing an empirically based classification system and has broad implications for future research.

On the explaining-away phenomenon in multivariate latent variable models

Many probabilistic models for psychological and educational measurements contain latent variables. Well-known examples are factor analysis, item response theory, and latent class model families. We discuss what is referred to as the 'explaining-away' phenomenon in the context of such latent variable models. This phenomenon can occur when multiple latent variables are related to the same observed variable, and can elicit seemingly counterintuitive conditional dependencies between latent variables given observed variables. We illustrate the implications of explaining away for a number of well-known latent variable models by using both theoretical and real data examples.

An inequality for correlations in unidimensional monotone latent variable models for binary variables

It is shown that a unidimensional monotone latent variable model for binary items implies a restriction on the relative sizes of item correlations: The negative logarithm of the correlations satisfies the triangle inequality. This inequality is not implied by the condition that the correlations are nonnegative, the criterion that coefficient H exceeds 0.30, or manifest monotonicity. The inequality implies both a lower bound and an upper bound for each correlation between two items, based on the correlations of those two items with every possible third item. It is discussed how this can be used in Mokken's (A theory and procedure of scale-analysis, Mouton, The Hague, 1971) scale analysis.