IRT Modeling in the Presence of Zero-Inflation With Application to Psychiatric Disorder Severity

Long-term Effects of the Family Check-Up on Suicidality in Childhood and Adolescence: Integrative Data Analysis of Three Randomized Trials

This study employed integrative data analysis techniques to examine the long-term effects of the family check-up (FCU) on changes in youth suicide risk using three randomized prevention trials, including one trial initiated in early childhood and two initiated in early adolescence. Data were harmonized across studies using moderated nonlinear factor analysis, and intervention effects were tested using an autoregressive latent trajectory model examining changes in suicide risk across long-term follow-up. Across trials, significant long-term effects of the FCU on reductions in suicide risk were observed, although differences between intervention and control group trajectories declined over time. No moderation of intervention effects was observed by youth gender or race/ethnicity or across samples. While results offer further support for the benefits of the FCU for suicide risk reduction, they also suggest that such effects may wane over time, underscoring the need for continued development of the FCU to enhance longer-term durability of effects on suicide-related behaviors.

Introduction to the Special Issue on Innovations and Applications of Integrative Data Analysis (IDA) and Related Data Harmonization Procedures in Prevention Science

This paper serves as an introduction to the special issue of Prevention Science entitled, "Innovations and Applications of Integrative Data Analysis (IDA) and Related Data Harmonization Procedures in Prevention Science." This special issue includes a collection of original papers from multiple disciplines that apply individual-level data synthesis methodologies, including IDA, individual participant meta-analysis, and other related methods to harmonize and integrate multiple datasets from intervention trials of the same or similar interventions. This work builds on a series of papers appearing in a prior Prevention Science special issue, entitled "Who Benefits from Programs to Prevent Adolescent Depression?" (Howe, Pantin, & Perrino, 2018). Since the publication of this prior work, the use of individual-level data synthesis has increased considerably in and outside of prevention. As such, there is a need for an update on current and future directions in IDA, with careful consideration of innovations and applications of these methods to fill important research gaps in prevention science. The papers in this issue are organized into two broad categories of (1) evidence synthesis papers that apply best practices in data harmonization and individual-level data synthesis and (2) new and emerging design, psychometric, and methodological issues and solutions. This collection of original papers is followed by two invited commentaries which provide insight and important reflections on the field and future directions for prevention science.

Mitigating Multiple Sources of Bias in a Quasi-Experimental Integrative Data Analysis: Does Treating Childhood Anxiety Prevent Substance Use Disorders in Late Adolescence/Young Adulthood?

Psychiatric epidemiologists, developmental psychopathologists, prevention scientists, and treatment researchers have long speculated that treating child anxiety disorders could prevent alcohol and other drug use disorders in young adulthood. A primary challenge in examining long-term effects of anxiety disorder treatment from randomized controlled trials is that all participants receive an immediate or delayed study-related treatment prior to long-term follow-up assessment. Thus, if a long-term follow-up is conducted, a comparison condition no longer exists within the trial. Quasi-experimental designs (QEDs) pairing such clinical samples with comparable untreated epidemiological samples offer a method of addressing this challenge. Selection bias, often a concern in QEDs, can be mitigated by propensity score weighting. A second challenge may arise because the clinical and epidemiological studies may not have used identical measures, necessitating Integrative Data Analysis (IDA) for measure harmonization and scale score estimation. The present study uses a combination of propensity score weighting, zero-inflated mixture moderated nonlinear factor analysis (ZIM-MNLFA), and potential outcomes mediation in a child anxiety treatment QED/IDA (n = 396). Under propensity score-weighted potential outcomes mediation, CBT led to reductions in substance use disorder severity, the effects of which were mediated by reductions in anxiety severity in young adulthood. Sensitivity analyses highlighted the importance of attending to multiple types of bias. This study illustrates how hybrid QED/IDAs can be used in secondary prevention contexts for improved measurement and causal inference, particularly when control participants in clinical trials receive study-related treatment prior to long-term assessment.

Hungarian PROMIS-29+2: psychometric properties and population reference values

OBJECTIVES

This study aims to assess psychometric properties of the Hungarian PROMIS-29+2 profile measure and provide general population reference values for Hungary.

METHODS

An adult general population sample (n = 1700) completed PROMIS-29+2 v2.1 in an online survey. The following psychometric properties were assessed: floor and ceiling effect, convergent validity with SF-36v1 domains, internal consistency (McDonald's omega), unidimensionality, local independence, monotonicity, graded response model (GRM) fit and differential item functioning (DIF). Age- and gender-specific reference values were established using the US item calibrations.

RESULTS

Depending on scale orientation, high floor or ceiling effects were observed for all domains (25.2-60.7%) except for sleep disturbance. McDonald's omega for domains ranged from 0.87-0.97. Unidimensionality, local independence and monotonicity were supported and the GRM adequately fitted for all but one domains. The sleep disturbance domain demonstrated item misfit, response level disordering and low discrimination ability, particularly for item Sleep116 ('refreshing sleep'). Strong correlations were observed between PROMIS-29+2 and corresponding SF-36 domains (r_s=│0.60│ to │0.78│). No DIF was detected for most sociodemographic characteristics. Problems with physical function, pain interference and social roles tended to increase, whereas problems with anxiety, depression, fatigue and cognitive function declined with age (p < 0.01). In all domains except for cognitive function, more health problems occurred in females than in males (p < 0.001).

CONCLUSION

The Hungarian PROMIS-29+2 shows satisfactory psychometric properties; however, the sleep disturbance domain substantially underperforms that requires further attention. Population reference values were generated that facilitate the interpretation of health outcomes in various patient populations.

A Common Measurement Scale for Self-Report Instruments in Mental Health Care

Abstract. The diversity of measures in clinical psychology hampers a straightforward interpretation of test results, complicates communication with the patient, and constitutes a challenge to the implementation of measurement-based care. In educational research and assessment, it is common practice to convert test scores to a common metric, such as T scores. We recommend applying this also in clinical psychology and propose and test a procedure to arrive at T scores approximating a normal distribution that can be applied to individual test scores. We established formulas to estimate normalized T scores from raw scale scores by regressing IRT-based θ scores on raw scores. With data from a large population and clinical samples, we established crosswalk formulas. Their validity was investigated by comparing calculated T scores with IRT-based T scores. IRT and formulas yielded very similar T scores, supporting the validity of the latter approach. Theoretical and practical advantages and disadvantages of both approaches to convert scores to common metrics and alternative approaches are discussed. Provided that scale characteristics allow for their computation, T scores will help to better understand measurement results, which makes it easier for patients and practitioners to use test results in joint decision-making about the course of treatment.

A stochastic approximation expectation maximization algorithm for estimating Ramsay-curve three-parameter normal ogive model with non-normal latent trait distributions

In the estimation of item response models, the normality of latent traits is frequently assumed. However, this assumption may be untenable in real testing. In contrast to the conventional three-parameter normal ogive (3PNO) model, a 3PNO model incorporating Ramsay-curve item response theory (RC-IRT), denoted as the RC-3PNO model, allows for flexible latent trait distributions. We propose a stochastic approximation expectation maximization (SAEM) algorithm to estimate the RC-3PNO model with non-normal latent trait distributions. The simulation studies of this work reveal that the SAEM algorithm produces more accurate item parameters for the RC-3PNO model than those of the 3PNO model, especially when the latent density is not normal, such as in the cases of a skewed or bimodal distribution. Three model selection criteria are used to select the optimal number of knots and the degree of the B-spline functions in the RC-3PNO model. A real data set from the PISA 2018 test is used to demonstrate the application of the proposed algorithm.

Symptom Presence and Symptom Severity as Unique Indicators of Psychopathology: An Application of Multidimensional Zero-Inflated and Hurdle Graded Response Models

Questionnaires inquiring about psychopathology symptoms often produce data with excess zeros or the equivalent (e.g., none, never, and not at all). This type of zero inflation is especially common in nonclinical samples in which many people do not exhibit psychopathology, and if unaccounted for, can result in biased parameter estimates when fitting latent variable models. In the present research, we adopt a maximum likelihood approach in fitting multidimensional zero-inflated and hurdle graded response models to data from a psychological distress measure. These models include two latent variables: susceptibility, which relates to the probability of endorsing the symptom at all, and severity, which relates to the frequency of the symptom, given its presence. After estimating model parameters, we compute susceptibility and severity scale scores and include them as explanatory variables in modeling health-related criterion measures (e.g., suicide attempts, diagnosis of major depressive disorder). Results indicate that susceptibility and severity uniquely and differentially predict other health outcomes, which suggests that symptom presence and symptom severity are unique indicators of psychopathology and both may be clinically useful. Psychometric and clinical implications are discussed, including scale score reliability.

DIF Detection With Zero-Inflation Under the Factor Mixture Modeling Framework

Response data containing an excessive number of zeros are referred to as zero-inflated data. When differential item functioning (DIF) detection is of interest, zero-inflation can attenuate DIF effects in the total sample and lead to underdetection of DIF items. The current study presents a DIF detection procedure for response data with excess zeros due to the existence of unobserved heterogeneous subgroups. The suggested procedure utilizes the factor mixture modeling (FMM) with MIMIC (multiple-indicator multiple-cause) to address the compromised DIF detection power via the estimation of latent classes. A Monte Carlo simulation was conducted to evaluate the suggested procedure in comparison to the well-known likelihood ratio (LR) DIF test. Our simulation study results indicated the superiority of FMM over the LR DIF test in terms of detection power and illustrated the importance of accounting for latent heterogeneity in zero-inflated data. The empirical data analysis results further supported the use of FMM by flagging additional DIF items over and above the LR test.

Should suicidal ideation be regarded as a dimension, a unipolar trait or a mixture? A model-based analysis at the score level

Screening questionnaires administered in community samples may allow to early identify suicidal ideation (S.I.). Although the results found in these samples suggest that S.I. behaves like a unipolar trait or a quasi-trait, it is routinely assessed using procedures developed for bipolar traits. Therefore, the main aim of this study is to determine whether there is a basis for modelling S.I. as a bipolar trait, a unipolar trait, or a quasi-trait with two classes of individuals (symptomatic and asymptomatic). In a community sample and mainly at the scoring level, we compare the results provided by fitting three models based on different assumptions: GRM (bipolar traits), LL-GRM (unipolar traits) and FMA (quasi-traits). 773 Spanish participants answered a S.I. and a life satisfaction questionnaires. GRM and LL-GRM provided equivalent results at the structural level, but not at the scoring level, especially in the conditional and marginal accuracy of the estimated scores. While the GRM scores are highly accurate only in a narrow range well above the mean, the LL-GRM scores are highly accurate in a much wider range around the mean. They also have different implications for the prediction of life satisfaction. FMA results suggest that an asymptomatic and a symptomatic class could not be clearly differentiated. In conclusion, LL-GRM would make it possible to accurately measure a larger number of subjects in a community sample than GRM, leaving fewer cases of vulnerable people unidentified. These results should be considered by researchers and professionals when deciding which modellings to use for screening purposes.

Assessing Dimensionality in Dichotomous Items When Many Subjects Have All-Zero Responses: An Example From Psychiatry and a Solution Using Mixture Models

Common methods for determining the number of latent dimensions underlying an item set include eigenvalue analysis and examination of fit statistics for factor analysis models with varying number of factors. Given a set of dichotomous items, the authors demonstrate that these empirical assessments of dimensionality often incorrectly estimate the number of dimensions when there is a preponderance of individuals in the sample with all-zeros as their responses, for example, not endorsing any symptoms on a health battery. Simulated data experiments are conducted to demonstrate when each of several common diagnostics of dimensionality can be expected to under- or over-estimate the true dimensionality of the underlying latent variable. An example is shown from psychiatry assessing the dimensionality of a social anxiety disorder battery where 1, 2, 3, or more factors are identified, depending on the method of dimensionality assessment. An all-zero inflated exploratory factor analysis model (AZ-EFA) is introduced for assessing the dimensionality of the underlying subgroup corresponding to those possessing the measurable trait. The AZ-EFA approach is demonstrated using simulation experiments and an example measuring social anxiety disorder from a large nationally representative survey. Implications of the findings are discussed, in particular, regarding the potential for different findings in community versus patient populations.

Design and methodology for an integrative data analysis of coping power: Direct and indirect effects on adolescent suicidality

As suicide rates have risen in the last decade, there has been greater emphasis on targeting early risk conditions for suicidality among youth and adolescents as a form of suicide "inoculation". Two particular needs that have been raised in this nascent literature are a) the dearth of examination of early intervention effects on distal suicide risk that target externalizing behaviors and b) the need to harmonize multiple existing intervention datasets for greater precision in modeling intervention effects on low base rate outcomes such as suicidal behaviors. This project, entitled "Integrative Data Analysis of Coping Power (CP): Effects on Adolescent Suicidality", funded by the National Institute of Mental Health (NIMH), will harmonize and analyze data from 11 randomized controlled trials of CP (total individual-level N = 3183, total school-level N = 189). CP is an empirically-supported, child- and family-focused preventive intervention that focuses on reducing externalizing more broadly among youth who exhibit early aggression, which makes it ideally suited to targeting externalizing pathways to suicidality. The project utilizes three measurement and data analysis frameworks that have emerged across multiple independent disciplines: integrative data analysis (IDA), random treatment effects multilevel modeling (RTE-MLM), and propensity score weighting (PSW). If successful, the project will a) provide initial evidence that CP would have gender-specific indirect effects on suicidality through reductions in externalizing for boys and reductions in internalizing for girls and b) identify optimal conditions under which CP is delivered (e.g., groups, individuals, online) across participants on reductions in suicidality and other key intermediate endpoints.

Matching IRT Models to Patient-Reported Outcomes Constructs: The Graded Response and Log-Logistic Models for Scaling Depression

Item response theory (IRT) model applications extend well beyond cognitive ability testing, and various patient-reported outcomes (PRO) measures are among the more prominent examples. PRO (and like) constructs differ from cognitive ability constructs in many ways, and these differences have model fitting implications. With a few notable exceptions, however, most IRT applications to PRO constructs rely on traditional IRT models, such as the graded response model. We review some notable differences between cognitive and PRO constructs and how these differences can present challenges for traditional IRT model applications. We then apply two models (the traditional graded response model and an alternative log-logistic model) to depression measure data drawn from the Patient-Reported Outcomes Measurement Information System project. We do not claim that one model is "a better fit" or more "valid" than the other; rather, we show that the log-logistic model may be more consistent with the construct of depression as a unipolar phenomenon. Clearly, the graded response and log-logistic models can lead to different conclusions about the psychometrics of an instrument and the scaling of individual differences. We underscore, too, that, in general, explorations of which model may be more appropriate cannot be decided only by fit index comparisons; these decisions may require the integration of psychometrics with theory and research findings on the construct of interest.

Linking Scores with Patient-Reported Health Outcome Instruments:A VALIDATION STUDY AND COMPARISON OF THREE LINKING METHODS

The psychometric process used to establish a relationship between the scores of two (or more) instruments is generically referred to as linking. When two instruments with the same content and statistical test specifications are linked, these instruments are said to be equated. Linking and equating procedures have long been used for practical benefit in educational testing. In recent years, health outcome researchers have increasingly applied linking techniques to patient-reported outcome (PRO) data. However, these applications have some noteworthy purposes and associated methodological questions. Purposes for linking health outcomes include the harmonization of data across studies or settings (enabling increased power in hypothesis testing), the aggregation of summed score data by means of score crosswalk tables, and score conversion in clinical settings where new instruments are introduced, but an interpretable connection to historical data is needed. When two PRO instruments are linked, assumptions for equating are typically not met and the extent to which those assumptions are violated becomes a decision point around how (and whether) to proceed with linking. We demonstrate multiple linking procedures-equipercentile, unidimensional IRT calibration, and calibrated projection-with the Patient-Reported Outcomes Measurement Information System Depression bank and the Patient Health Questionnaire-9. We validate this link across two samples and simulate different instrument correlation levels to provide guidance around which linking method is preferred. Finally, we discuss some remaining issues and directions for psychometric research in linking PRO instruments.

Item Parameter Estimation in Multistage Designs: A Comparison of Different Estimation Approaches for the Rasch Model

There is some debate in the psychometric literature about item parameter estimation in multistage designs. It is occasionally argued that the conditional maximum likelihood (CML) method is superior to the marginal maximum likelihood method (MML) because no assumptions have to be made about the trait distribution. However, CML estimation in its original formulation leads to biased item parameter estimates. Zwitser and Maris (2015, Psychometrika) proposed a modified conditional maximum likelihood estimation method for multistage designs that provides practically unbiased item parameter estimates. In this article, the differences between different estimation approaches for multistage designs were investigated in a simulation study. Four different estimation conditions (CML, CML estimation with the consideration of the respective MST design, MML with the assumption of a normal distribution, and MML with log-linear smoothing) were examined using a simulation study, considering different multistage designs, number of items, sample size, and trait distributions. The results showed that in the case of the substantial violation of the normal distribution, the CML method seemed to be preferable to MML estimation employing a misspecified normal trait distribution, especially if the number of items and sample size increased. However, MML estimation using log-linear smoothing lea to results that were very similar to the CML method with the consideration of the respective MST design.

Psychometric properties of the PROMIS-57 questionnaire, Norwegian version

Purpose

The aims of this cross-sectional study were to explore reliability and validity of the Norwegian version of the Patient-Reported Outcome Measurement System^®—Profile 57 (PROMIS-57) questionnaire in a general population sample, n = 408, and to examine Item Response properties and factor structure.

Methods

Reliability measures were obtained from factor analysis and item response theory (IRT) methods. Correlations between PROMIS-57 and RAND-36-item health survey (RAND36) were examined for concurrent and discriminant validity. Factor structure and IRT assumptions were examined with factor analysis methods. IRT Item and model fit and graphic plots were inspected, and differential item functioning (DIF) for language, age, gender, and education level were examined.

Results

PROMIS-57 demonstrated excellent reliability and satisfactory concurrent and discriminant validity. Factor structure of seven domains was supported. IRT assumptions were met for unidimensionality, local independence, monotonicity, and invariance with no DIF of consequence for language or age groups. Estimated common variance (ECV) per domain and confirmatory factor analysis (CFA) model fit supported unidimensionality for all seven domains. The GRM IRT Model demonstrates acceptable model fit.

Conclusions

The psychometric properties and factor structure of Norwegian PROMIS-57 were satisfactory. Hence, the 57-item questionnaire along with PROMIS-29, and the corresponding 8 and 4 item short forms for physical function, anxiety, depression, fatigue, sleep disturbance, social participation ability and pain interference, are considered suitable for use in research and clinical care in Norwegian populations. Further studies on longitudinal reliability and sensitivity in patient populations and for Norwegian item calibration and/or reference scores are needed.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11136-021-02906-1.

Does Functional Somatic Symptoms Measurement Differ Across Sex and Age? Differential Item Functioning in Somatic Symptoms Measured With the CIDI

Functional Somatic Symptoms (FSS) are physical symptoms that cannot be attributed to underlying pathology. Their severity is often measured with sum scores on questionnaires; however, this may not adequately reflect FSS severity in subgroups of patients. We aimed to identify the items of the somatization section of the Composite International Diagnostic Interview that best discriminate FSS severity levels, and to assess their functioning in sex and age subgroups. We applied the two-parameter logistic model to 19 items in a population-representative cohort of 962 participants. Subsequently, we examined differential item functioning (DIF). "Localized (muscle) weakness" was the most discriminative item of FSS severity. "Abdominal pain" consistently showed DIF by sex, with males reporting it at higher FSS severity. There was no consistent DIF by age, however, "Joint pain" showed poor discrimination of FSS severity in older adults. These findings could be helpful for the development of better assessment instruments for FSS, which can improve both future research and clinical care.

An Item Response Theory-Based Scoring of the South Oaks Gambling Screen-Revised Adolescents

The South Oaks Gambling Screen-Revised Adolescent (SOGS-RA) is one of the most widely used screening tools for problem gambling among adolescents. In this study, item response theory was used for computing measures of problem gambling severity that took into account how much information the endorsed items provided about the presence of problem gambling. A zero-inflated mixture two-parameter logistic model was estimated on the responses of 4,404 adolescents to the South Oaks Gambling Screen-Revised Adolescent to compute the difficulty and discrimination of each item, and the problem gambling severity level (θ score) of each respondent. Receiver operating characteristic curve analysis was used to identify the cutoff on the θ scores that best distinguished daily and nondaily gamblers. This cutoff outperformed the common cutoff defined on the sum scores in identifying daily gamblers but fell behind it in identifying nondaily gamblers. When screening adolescents to be subjected to further investigations, the cutoff on the θ scores must be preferred to that on the sum scores.

Measurement of the problematic usage of the Internet unidimensional quasitrait continuum with item response theory

Problematic usage of the internet (PUI) describes maladaptive use of online resources and is recognized as a growing worldwide issue. Here, we refined the Internet Addiction Test (IAT) for use as a screening tool to measure generalized internet use problems in normative samples. Analysis of response data with parametric unidimensional item response theory identified 10 items of the IAT that measured most of the PUI latent trait continuum with high precision in a subsample of 816 participants with meaningful variance in internet use problems. Selected items may characterize minor, or early stages of, PUI by measuring a preoccupation with the Internet, motivations to use online activities to escape aversive emotional experiences and regulate mood, as well as secrecy, defensiveness, and interpersonal conflict associated with internet use. Summed scores on these 10 items demonstrated a strong correlation with full-length IAT scores and comparable, or better, convergence with measures of impulsivity and compulsivity. Proposed cut-off scores differentiated between individuals potentially at risk of developing PUI from those with few self-reported internet use problems with good sensitivity and specificity. Differential item function testing revealed measurement equivalence between the sexes, Caucasians and non-Caucasians. However, evidence for differential test functioning between independent samples drawn from South Africa and the United States of America suggests that raw scores cannot be meaningfully compared between different geographic regions. These findings have implications for conceptualization and measurement of PUI in normative samples.

We provide recommendations for measuring symptoms of problematic usage of the internet, which can be identified in a subset of the population using our refined version of the IAT and suggested cut-off scores. Relevant self-reported internet use problems include a preference for online over face-to-face social interactions, use of the internet to regulate emotions, excessive online engagement, interpersonal conflict, and emotional withdrawal following cessation of internet use.

Latent Class Analysis: Insights about design and analysis of schistosomiasis diagnostic studies

Various global health initiatives are currently advocating the elimination of schistosomiasis within the next decade. Schistosomiasis is a highly debilitating tropical infectious disease with severe burden of morbidity and thus operational research accurately evaluating diagnostics that quantify the epidemic status for guiding effective strategies is essential. Latent class models (LCMs) have been generally considered in epidemiology and in particular in recent schistosomiasis diagnostic studies as a flexible tool for evaluating diagnostics because assessing the true infection status (via a gold standard) is not possible. However, within the biostatistics literature, classical LCM have already been criticised for real-life problems under violation of the conditional independence (CI) assumption and when applied to a small number of diagnostics (i.e. most often 3-5 diagnostic tests). Solutions of relaxing the CI assumption and accounting for zero-inflation, as well as collecting partial gold standard information, have been proposed, offering the potential for more robust model estimates. In the current article, we examined such approaches in the context of schistosomiasis via analysis of two real datasets and extensive simulation studies. Our main conclusions highlighted poor model fit in low prevalence settings and the necessity of collecting partial gold standard information in such settings in order to improve the accuracy and reduce bias of sensitivity and specificity estimates.

Accurate schistosomiasis diagnosis is essential to assess the impact of large scale and repeated mass drug administration to control or even eliminate this disease. However, in schistosomiasis diagnostic studies, several inherent study design issues pose a real challenge for the currently available statistical tools used for diagnostic modelling and associated data analysis and conclusions. More specifically, those study design issues are: 1) the inclusion of small number of diagnostic tests (i.e. most often five), 2) non formal consensus about a schistosomiasis gold standard, 3) the contemporary use of relatively small sample sizes in relevant studies due to lack of research funding, 4) the differing levels of prevalence of the studied disease even within the same area of one endemic country and 5) other real world factors such as: the lack of appropriate equipment, the variability of certain methods due to biological phenomena and training of technicians across the endemic countries because of scarce financial resources contributing to the existing lack of a schistosomiasis gold standard. The current study aims to caution practitioners from blindly applying statistical models with small number of diagnostic tests and sample sizes, proposing design guidelines of future schistosomiasis diagnostic accuracy studies with recommendations for further research. While our study is centred around the diagnosis of schistosomiasis, we feel that the recommendations can be adapted to other major tropical infectious diseases as well.

A study of alternative approaches to non-normal latent trait distributions in item response theory models used for health outcome measurement

It is often unrealistic to assume normally distributed latent traits in the measurement of health outcomes. If normality is violated, the item response theory (IRT) models that are used to calibrate questionnaires may yield parameter estimates that are biased. Recently, IRT models were developed for dealing with specific deviations from normality, such as zero-inflation (“excess zeros”) and skewness. However, these models have not yet been evaluated under conditions representative of item bank development for health outcomes, characterized by a large number of polytomous items. A simulation study was performed to compare the bias in parameter estimates of the graded response model (GRM), polytomous extensions of the zero-inflated mixture IRT (ZIM-GRM), and Davidian Curve IRT (DC-GRM). In the case of zero-inflation, the GRM showed high bias overestimating discrimination parameters and yielding estimates of threshold parameters that were too high and too close to one another, while ZIM-GRM showed no bias. In the case of skewness, the GRM and DC-GRM showed little bias with the GRM showing slightly better results. Consequences for the development of health outcome measures are discussed.

A data mining and item response mixture modeling method to retrospectively measure Diagnostic and Statistical Manual of Mental Disorders-5 attention deficit hyperactivity disorder in the 1970 British Cohort Study

OBJECTIVE

To facilitate future outcome studies, we aimed to develop a robust and replicable method for estimating a categorical and dimensional measure of Diagnostic and Statistical Manual of Mental Disorders-5 (DSM-5) attention deficit hyperactivity disorder (ADHD) in the 1970 British Cohort Study (BCS70).

METHOD

Following a data mining framework, we mapped DSM-5 ADHD symptoms to age 10 BCS70 data (N = 11,426) and derived a 16-item scale (α = 0.85). Mapping was validated by an expert panel. A categorical subgroup was derived (n = 594, 5.2%), and a zero-inflated item response theory (IRT) mixture model fitted to estimate a dimensional measure.

RESULTS

Subgroup composition was comparable with other ADHD samples. Relative risk ratios (ADHD/not ADHD) included boys = 1.38, unemployed fathers = 2.07, below average reading = 2.58, and depressed parent = 3.73. Our estimated measures correlated with two derived reference scales: Strengths and Difficulties Questionnaire hyperactivity (r = 0.74) and a Rutter/Conners-based scale (r = 0.81), supporting construct validity. IRT model items (symptoms) had moderate to high discrimination (0.90-2.81) and provided maximum information at average to moderate theta levels of ADHD (0.5-1.75).

CONCLUSION

We extended previous work to identify ADHD in BCS70, derived scales from existing data, modeled ADHD items with IRT, and adjusted for a zero-inflated distribution. Psychometric properties were promising, and this work will enable future studies of causal mechanisms in ADHD.

A Zero-Inflated Box-Cox Normal Unipolar Item Response Model for Measuring Constructs of Psychopathology

This research introduces a latent class item response theory (IRT) approach for modeling item response data from zero-inflated, positively skewed, and arguably unipolar constructs of psychopathology. As motivating data, the authors use 4,925 responses to the Patient Health Questionnaire (PHQ-9), a nine Likert-type item depression screener that inquires about a variety of depressive symptoms. First, Lucke's log-logistic unipolar item response model is extended to accommodate polytomous responses. Then, a nontrivial proportion of individuals who do not endorse any of the symptoms are accounted for by including a nonpathological class that represents those who may be absent on or at some floor level of the latent variable that is being measured by the PHQ-9. To enhance flexibility, a Box-Cox normal distribution is used to empirically determine a transformation parameter that can help characterize the degree of skewness in the latent variable density. A model comparison approach is used to test the necessity of the features of the proposed model. Results suggest that (a) the Box-Cox normal transformation provides empirical support for using a log-normal population density, and (b) model fit substantially improves when a nonpathological latent class is included. The parameter estimates from the latent class IRT model are used to interpret the psychometric properties of the PHQ-9, and a method of computing IRT scale scores that reflect unipolar constructs is described, focusing on how these scores may be used in clinical contexts.

Robustness of Parameter Estimation to Assumptions of Normality in the Multidimensional Graded Response Model

A central assumption that is implicit in estimating item parameters in item response theory (IRT) models is the normality of the latent trait distribution, whereas a similar assumption made in categorical confirmatory factor analysis (CCFA) models is the multivariate normality of the latent response variables. Violation of the normality assumption can lead to biased parameter estimates. Although previous studies have focused primarily on unidimensional IRT models, this study extended the literature by considering a multidimensional IRT model for polytomous responses, namely the multidimensional graded response model. Moreover, this study is one of few studies that specifically compared the performance of full-information maximum likelihood (FIML) estimation versus robust weighted least squares (WLS) estimation when the normality assumption is violated. The research also manipulated the number of nonnormal latent trait dimensions. Results showed that FIML consistently outperformed WLS when there were one or multiple skewed latent trait distributions. More interestingly, the bias of the discrimination parameters was non-ignorable only when the corresponding factor was skewed. Having other skewed factors did not further exacerbate the bias, whereas biases of boundary parameters increased as more nonnormal factors were added. The item parameter standard errors recovered well with both estimation algorithms regardless of the number of nonnormal dimensions.

Refining the assessment of disrupted maternal communication: Using item response models to identify central indicators of disrupted behavior

The Atypical Maternal Behavior Instrument for Assessment and Classification (AMBIANCE; Bronfman, Madigan, &amp; Lyons-Ruth, 2009-2014; Bronfman, Parsons, &amp; Lyons-Ruth, 1992-2004) is a widely used and well-validated measure for assessing disrupted forms of caregiver responsiveness within parent-child interactions. However, it requires evaluating approximately 150 behavioral items from videotape and extensive training to code, thus making its use impractical in most clinical contexts. Accordingly, the primary aim of the current study was to identify a reduced set of behavioral indicators most central to the AMBIANCE coding system using latent-trait item response theory (IRT) models. Observed mother-infant interaction data previously coded with the AMBIANCE was pooled from laboratories in both North America and Europe (N = 343). Using 2-parameter logistic IRT models, a reduced set of 45 AMBIANCE items was identified. Preliminary convergent and discriminant validity was evaluated in relation to classifications of maternal disrupted communication assigned using the full set of AMBIANCE indicators, to infant attachment disorganization, and to maternal sensitivity. The results supported the construct validity of the refined item set, opening the way for development of a brief screening measure for disrupted maternal communication. IRT models in clinical scale refinement and their potential for bridging clinical and research objectives in developmental psychopathology are discussed.

Alternative Approaches to Addressing Non-Normal Distributions in the Application of IRT Models to Personality Measures

It is generally assumed that the latent trait is normally distributed in the population when estimating logistic item response theory (IRT) model parameters. This assumption requires that the latent trait be fully continuous and the population homogenous (i.e., not a mixture). When this normality assumption is violated, models are misspecified, and item and person parameter estimates are inaccurate. When normality cannot be assumed, it might be appropriate to consider alternative modeling approaches: (a) a zero-inflated mixture, (b) a log-logistic, (c) a Ramsay curve, or (d) a heteroskedastic-skew model. The first 2 models were developed to address modeling problems associated with so-called quasi-continuous or unipolar constructs, which apply only to a subset of the population, or are meaningful at one end of the continuum only. The second 2 models were developed to address non-normal latent trait distributions and violations of homogeneity of error variance, respectively. To introduce these alternative IRT models and illustrate their strengths and weaknesses, we performed real data application comparing results to those from a graded response model. We review both statistical and theoretical challenges in applying these models and choosing among them. Future applications of these and other alternative models (e.g., unfolding, diffusion) are needed to advance understanding about model choice in particular situations.

Item response theory and the measurement of psychiatric constructs: some empirical and conceptual issues and challenges

Item response theory (IRT) measurement models are now commonly used in educational, psychological, and health-outcomes measurement, but their impact in the evaluation of measures of psychiatric constructs remains limited. Herein we present two, somewhat contradictory, theses. The first is that, when skillfully applied, IRT has much to offer psychiatric measurement in terms of scale development, psychometric analysis, and scoring. The second argument, however, is that psychiatric measurement presents some unique challenges to the application of IRT - challenges that may not be easily addressed by application of conventional IRT models and methods. These challenges include, but are not limited to, the modeling of conceptually narrow constructs and their associated limited item pools, and unipolar constructs where the expected latent trait distribution is highly skewed.