A Monte Carlo evaluation of three methods to detect local dependence in binary data latent class models

Identification of sources of DIF using covariates in patient-reported outcome measures: a simulation study comparing two approaches based on Rasch family models

When analyzing patient-reported outcome (PRO) data, sources of differential item functioning (DIF) can be multiple and there may be more than one covariate of interest. Hence, it could be of great interest to disentangle their effects. Yet, in the literature on PRO measures, there are many studies where DIF detection is applied separately and independently for each covariate under examination. With such an approach, the covariates under investigation are not introduced together in the analysis, preventing from simultaneously studying their potential DIF effects on the questionnaire items. One issue, among others, is that it may lead to the detection of false-positive effects when covariates are correlated. To overcome this issue, we developed two new algorithms (namely ROSALI-DIF FORWARD and ROSALI-DIF BACKWARD). Our aim was to obtain an iterative item-by-item DIF detection method based on Rasch family models that enable to adjust group comparisons for DIF in presence of two binary covariates. Both algorithms were evaluated through a simulation study under various conditions aiming to be representative of health research contexts. The performance of the algorithms was assessed using: (i) the rates of false and correct detection of DIF, (ii) the DIF size and form recovery, and (iii) the bias in the latent variable level estimation. We compared the performance of the ROSALI-DIF algorithms to the one of another approach based on likelihood penalization. For both algorithms, the rate of false detection of DIF was close to 5%. The DIF size and form influenced the rates of correct detection of DIF. Rates of correct detection was higher with increasing DIF size. Besides, the algorithm fairly identified homogeneous differences in the item threshold parameters, but had more difficulties identifying non-homogeneous differences. Over all, the ROSALI-DIF algorithms performed better than the penalized likelihood approach. Integrating several covariates during the DIF detection process may allow a better assessment and understanding of DIF. This study provides valuable insights regarding the performance of different approaches that could be undertaken to fulfill this aim.

Evaluating Covariate Effects on ESM Measurement Model Changes with Latent Markov Factor Analysis: A Three-Step Approach

Invariance of the measurement model (MM) between subjects and within subjects over time is a prerequisite for drawing valid inferences when studying dynamics of psychological factors in intensive longitudinal data. To conveniently evaluate this invariance, latent Markov factor analysis (LMFA) was proposed. LMFA combines a latent Markov model with mixture factor analysis: The Markov model captures changes in MMs over time by clustering subjects' observations into a few states and state-specific factor analyses reveal what the MMs look like. However, to estimate the model, Vogelsmeier, Vermunt, van Roekel, and De Roover (2019) introduced a one-step (full information maximum likelihood; FIML) approach that is counterintuitive for applied researchers and entails cumbersome model selection procedures in the presence of many covariates. In this paper, we simplify the complex LMFA estimation and facilitate the exploration of covariate effects on state memberships by splitting the estimation in three intuitive steps: (1) obtain states with mixture factor analysis while treating repeated measures as independent, (2) assign observations to the states, and (3) use these states in a discrete- or continuous-time latent Markov model taking into account classification errors. A real data example demonstrates the empirical value.

Use of the Lagrange Multiplier Test for Assessing Measurement Invariance Under Model Misspecification

This article studies the Type I error, false positive rates, and power of four versions of the Lagrange multiplier test to detect measurement noninvariance in item response theory (IRT) models for binary data under model misspecification. The tests considered are the Lagrange multiplier test computed with the Hessian and cross-product approach, the generalized Lagrange multiplier test and the generalized jackknife score test. The two model misspecifications are those of local dependence among items and nonnormal distribution of the latent variable. The power of the tests is computed in two ways, empirically through Monte Carlo simulation methods and asymptotically, using the asymptotic distribution of each test under the alternative hypothesis. The performance of these tests is evaluated by means of a simulation study. The results highlight that, under mild model misspecification, all tests have good performance while, under strong model misspecification, the tests performance deteriorates, especially for false positive rates under local dependence and power for small sample size under misspecification of the latent variable distribution. In general, the Lagrange multiplier test computed with the Hessian approach and the generalized Lagrange multiplier test have better performance in terms of false positive rates while the Lagrange multiplier test computed with the cross-product approach has the highest power for small sample sizes. The asymptotic power turns out to be a good alternative to the classic empirical power because it is less time consuming. The Lagrange tests studied here have been also applied to a real data set.

TAID-LCA: Segmentation Algorithm Based on Ternary Trees

In this work, a statistical method for the segmentation of samples and/or populations is presented, which is based on a ternary tree structure. This approach overcomes known limitations of other segmentation methods such as CHAID, concerning the multivariate response and the non-symmetric relationship between explanatory and response variables. The multivariate response segmentation problem is handled through latent class models, while the factorial decomposition of the explanatory capability of variables is based on the Non-Symmetrical Correspondence Analysis. Stop criteria based on the CATANOVA index and impurity measures are proposed. A Simulated Annealing based post-pruning strategy is considered to avoid over-fitting relative to the training set and guarantee a better generalization capability for the method.

Practitioner's Guide to Latent Class Analysis: Methodological Considerations and Common Pitfalls

Latent class analysis is a probabilistic modeling algorithm that allows clustering of data and statistical inference. There has been a recent upsurge in the application of latent class analysis in the fields of critical care, respiratory medicine, and beyond. In this review, we present a brief overview of the principles behind latent class analysis. Furthermore, in a stepwise manner, we outline the key processes necessary to perform latent class analysis including some of the challenges and pitfalls faced at each of these steps. The review provides a one-stop shop for investigators seeking to apply latent class analysis to their data.

Detecting Conditional Dependence Using Flexible Bayesian Latent Class Analysis

A fundamental assumption underlying latent class analysis (LCA) is that class indicators are conditionally independent of each other, given latent class membership. Bayesian LCA enables researchers to detect and accommodate violations of this assumption by estimating any number of correlations among indicators with proper prior distributions. However, little is known about how the choice of prior may affect the performance of Bayesian LCA. This article presents a Monte Carlo simulation study that investigates (1) the utility of priors in a range of prior variances (i.e., strongly non-informative to strongly informative priors) in terms of Type I error and power for detecting conditional dependence and (2) the influence of imposing approximate independence on model fit of Bayesian LCA. Simulation results favored the use of a weakly informative prior with large variance-model fit (posterior predictive p-value) was always satisfactory when the class indicators were either independent or dependent. Based on the current findings and the additional literature, this article offers methodological guidelines and suggestions for applied researchers.

Patterns of polysubstance use and simultaneous co-use in high risk young adults

BACKGROUND

Polysubstance use (PSU) is associated with worse prognosis and poorer physical and mental health compared to single substance use. The current study provides information about PSU patterns by examining a diverse range of alcohol/substance use behaviors ranging from low-level experimentation to indicators of severe abuse. In addition, the current study, for the first time, examines how simultaneous co-use of multiple substances cluster with other more commonly studied PSU behaviors.

METHODS

Latent Class Analysis was used to identify patterns of substance use, in a sample of young-adults (n = 2098), using 25 items from the Semi-Structured Assessment for the Genetics of Alcoholism (SSAGA-II) including: items assessing severity of problems with alcohol, cannabis, stimulants, opiates, and sedatives; items assessing # of lifetime uses; items assessing simultaneous co-use of all combinations of substances. Then the association of class membership and age, antisocial and impulsive personality, experience seeking, anxiety, and neuroticism was examined using Multinomial Regression.

RESULTS

Fit indices (i.e. AIC, SSABIC, and entropy) and interpretability of classes supported a five-class solution: "Low Problems" (32% of sample), "Alcohol Primary" (11%), "Alcohol and Cannabis" (25%), "Moderate PSU" (23%), and "Severe PSU" (9%). Simultaneous co-use behaviors discriminated between lower and higher severity groups. Externalizing personality constructs robustly predicted membership in the "Moderate" and "Severe" PSU classes compared to the "Alcohol Primary" class.

CONCLUSIONS

PSU patterns followed an additive pattern of use with lower severity classes using alcohol/cannabis and more severe classes using other illicit substances in addition. Co-use items provided valuable information about PSU severity.

A model-driven approach to better identify older people at risk of depression

Depression in later life is one of the most common mental disorders. Several instruments have been developed to detect the presence or the absence of certain symptoms or emotional disorders, based on cut-off points. However, the use of a cut-off does not allow identification of depression sub-types or distinguish between mild and severe depression. As a result, depression may be under- or over-diagnosed in older people. This paper aims to apply a model-driven approach to classify individuals into distinct sub-groups, based on different combinations of depressive and emotional conditions. This approach is based on two distinct statistical solutions: first, a latent class analysis is applied to the items collected by the depression scale and, according to the final model, the probability of belonging to each class is calculated for every individual. Second, a factor analysis of these classes is performed to obtain a reduced number of clusters for easy interpretation. We use data collected through the EURO-D scale in a large sample of older individuals, participants of the sixth wave of the Survey of Health, Ageing and Retirement in Europe. We show that by using such a model-based approach it is possible to classify individuals in a more accurate way than the simple dichotomisation ‘depressed’ versus ‘non-depressed’.

On Lagrange Multiplier Tests in Multidimensional Item Response Theory: Information Matrices and Model Misspecification

Lagrange multiplier (LM) or score tests have seen renewed interest for the purpose of diagnosing misspecification in item response theory (IRT) models. LM tests can also be used to test whether parameters differ from a fixed value. We argue that the utility of LM tests depends on both the method used to compute the test and the degree of misspecification in the initially fitted model. We demonstrate both of these points in the context of a multidimensional IRT framework. Through an extensive Monte Carlo simulation study, we examine the performance of LM tests under varying degrees of model misspecification, model size, and different information matrix approximations. A generalized LM test designed specifically for use under misspecification, which has apparently not been previously studied in an IRT framework, performed the best in our simulations. Finally, we reemphasize caution in using LM tests for model specification searches.

Building Latent Class Trees, With an Application to a Study of Social Capital

Abstract. Researchers use latent class (LC) analysis to derive meaningful clusters from sets of categorical variables. However, especially when the number of classes required to obtain a good fit is large, interpretation of the latent classes may not be straightforward. To overcome this problem, we propose an alternative way of performing LC analysis, Latent Class Tree (LCT) modeling. For this purpose, a recursive partitioning procedure similar to divisive hierarchical cluster analysis is used: classes are split until a certain criterion indicates that the fit does not improve. The advantage of the LCT approach compared to the standard LC approach is that it gives a clear insight into how the latent classes are formed and how solutions with different numbers of classes relate. We also propose measures to evaluate the relative importance of the splits. The practical use of the approach is illustrated by the analysis of a data set on social capital.

Problems in detecting misfit of latent class models in diagnostic research without a gold standard were shown

OBJECTIVES

The objective of this study was to evaluate the performance of goodness-of-fit testing to detect relevant violations of the assumptions underlying the criticized "standard" two-class latent class model. Often used to obtain sensitivity and specificity estimates for diagnostic tests in the absence of a gold reference standard, this model relies on assuming that diagnostic test errors are independent. When this assumption is violated, accuracy estimates may be biased: goodness-of-fit testing is often used to evaluate the assumption and prevent bias.

STUDY DESIGN AND SETTING

We investigate the performance of goodness-of-fit testing by Monte Carlo simulation. The simulation scenarios are based on three empirical examples.

RESULTS

Goodness-of-fit tests lack power to detect relevant misfit of the standard two-class latent class model at sample sizes that are typically found in empirical diagnostic studies. The goodness-of-fit tests that are based on asymptotic theory are not robust to the sparseness of data. A parametric bootstrap procedure improves the evaluation of goodness of fit in the case of sparse data.

CONCLUSION

Our simulation study suggests that relevant violation of the local independence assumption underlying the standard two-class latent class model may remain undetected in empirical diagnostic studies, potentially leading to biased estimates of sensitivity and specificity.

Assessing Model Fit in Latent Class Analysis When Asymptotics Do Not Hold

The application of latent class (LC) analysis involves evaluating the LC model using goodness-of-fit statistics. To assess the misfit of a specified model, say with the Pearson chi-squared statistic, a p-value can be obtained using an asymptotic reference distribution. However, asymptotic p-values are not valid when the sample size is not large and/or the analyzed contingency table is sparse. Another problem is that for various other conceivable global and local fit measures, asymptotic distributions are not readily available. An alternative way to obtain the p-value for the statistic of interest is by constructing its empirical reference distribution using resampling techniques such as the parametric bootstrap or the posterior predictive check (PPC). In the current paper, we show how to apply the parametric bootstrap and two versions of the PPC to obtain empirical p-values for a number of commonly used global and local fit statistics within the context of LC analysis. The main difference between the PPC using test statistics and the parametric bootstrap is that the former takes into account parameter uncertainty. The PPC using discrepancies has the advantage that it is computationally much less intensive than the other two resampling methods. In a Monte Carlo study we evaluated Type I error rates and power of these resampling methods when used for global and local goodness-of-fit testing in LC analysis. Results show that both the bootstrap and the PPC using test statistics are generally good alternatives to asymptotic p-values and can also be used when (asymptotic) distributions are not known. Nominal Type I error rates were not met when sample size was small and the contingency table has many cells. Overall the PPC using test statistics was somewhat more conservative than the parametric bootstrap. We have also replicated previous research suggesting that the Pearson χ2 statistic should in many cases be preferred over the likelihood-ratio G2 statistic. Power to reject a model for which the number of LCs was one less than in the population was very high, unless sample size was small. When the contingency tables are very sparse, the total bivariate residual (TBVR) statistic, which is based on bivariate relationships, still had very high power, signifying its usefulness in assessing model fit.