New Perspectives in the Adaptive Assessment of Depression: The ATS-PD Version of the QuEDS

Reliability of a probabilistic knowledge structure

Indexes for estimating the overall reliability of a test in the framework of knowledge space theory (KST) are proposed and analyzed. First, the possibility of applying in KST the existing classical test theory (CTT) methods, based on the ratio between the true score variance and the total variance of the measure, has been explored. However, these methods are not suitable because in KST error and true score are not independent. Therefore, two new indexes based on the concepts of entropy and conditional entropy are developed. One index is used to estimate the reliability of the response pattern given the knowledge state, while the second one refers to the reliability of the estimated knowledge state of a person. Some theoretical considerations as well as simulations and an empirical example on real data are provided within a study of the behavior of these indexes under a certain number of different conditions.

Validation of the structured interview section of the penn computerized adaptive test for neurocognitive and clinical psychopathology assessment (CAT GOASSESS)

Large-scale studies and burdened clinical settings require precise, efficient measures that assess multiple domains of psychopathology. Computerized adaptive tests (CATs) can reduce administration time without compromising data quality. We examined feasibility and validity of an adaptive psychopathology measure, GOASSESS, in a clinical community-based sample (N = 315; ages 18-35) comprising three groups: healthy controls, psychosis, mood/anxiety disorders. Assessment duration was compared between the Full and CAT GOASSESS. External validity was tested by comparing how the CAT and Full versions related to demographic variables, study group, and socioeconomic status. The relationships between scale scores and criteria were statistically compared within a mixed-model framework to account for dependency between relationships. Convergent validity was assessed by comparing scores of the CAT and the Full GOASSESS using Pearson correlations. The CAT GOASSESS reduced interview duration by more than 90 % across study groups and preserved relationships to external criteria and demographic variables as the Full GOASSESS. All CAT GOASSESS scales could replace those of the Full instrument. Overall, the CAT GOASSESS showed acceptable psychometric properties and demonstrated feasibility by markedly reducing assessment time compared to the Full GOASSESS. The adaptive version could be used in large-scale studies or clinical settings for intake screening.

Extracting partially ordered clusters from ordinal polytomous data

In practical applications of knowledge space theory, knowledge states can be conceived as partially ordered clusters of individuals. Existing extensions of the theory to polytomous data lack methods for building "polytomous" structures. To this aim, an adaptation of the k-median clustering algorithm is proposed. It is an extension of k-modes to ordinal data in which the Hamming distance is replaced by the Manhattan distance, and the central tendency measure is the median, rather than the mode. The algorithm is tested in a series of simulation studies and in an application to empirical data. Results show that there are theoretical and practical reasons for preferring the k-median to the k-modes algorithm, whenever the responses to the items are measured on an ordinal scale. This is because the Manhattan distance is sensitive to the order on the levels, while the Hamming distance is not. Overall, k-median seems to be a promising data-driven procedure for building polytomous structures.

On the Implementation of Computerized Adaptive Observations for Psychological Assessment

The use of observational tools in psychological assessment has decreased in recent years, mainly due to its personnel and time costs, and researchers have not explored methodological innovations like adaptive algorithms in observational assessment. In the present study, we introduce the behavior-driven observation procedure to develop, test, and implement observational adaptive instruments. In Study 1, we use a preexisting observational checklist to evaluate nonverbal behaviors related to psychotic symptoms and to specify the adaptive algorithm's model. We fit the model to observational data collected from 114 participants. The results support the model's goodness of fit. In Study 2, we use the estimated model parameters to calibrate the adaptive procedure and test the algorithm for accuracy and efficiency in adaptively reconstructing 58 nonadaptively collected response patterns. The results show the algorithm's good accuracy and efficiency, with a 40% average reduction in the number of administered items. In Study 3, we used real raters to test the adaptive checklist built with behavior-driven observation. The results indicate adequate intrarater agreement and good consistency of the observed response patterns. In conclusion, the results support the possibility of using behavior-driven observation to create accurate and affordable (in terms of resources) observational assessment tools.