1
|
Schneider S, Hernandez R, Junghaenel DU, Jin H, Lee PJ, Gao H, Maupin D, Orriens B, Meijer E, Stone AA. Can you tell people's cognitive ability level from their response patterns in questionnaires? Behav Res Methods 2024:10.3758/s13428-024-02388-2. [PMID: 38528247 DOI: 10.3758/s13428-024-02388-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/02/2024] [Indexed: 03/27/2024]
Abstract
Questionnaires are ever present in survey research. In this study, we examined whether an indirect indicator of general cognitive ability could be developed based on response patterns in questionnaires. We drew on two established phenomena characterizing connections between cognitive ability and people's performance on basic cognitive tasks, and examined whether they apply to questionnaires responses. (1) The worst performance rule (WPR) states that people's worst performance on multiple sequential tasks is more indicative of their cognitive ability than their average or best performance. (2) The task complexity hypothesis (TCH) suggests that relationships between cognitive ability and performance increase with task complexity. We conceptualized items of a questionnaire as a series of cognitively demanding tasks. A graded response model was used to estimate respondents' performance for each item based on the difference between the observed and model-predicted response ("response error" scores). Analyzing data from 102 items (21 questionnaires) collected from a large-scale nationally representative sample of people aged 50+ years, we found robust associations of cognitive ability with a person's largest but not with their smallest response error scores (supporting the WPR), and stronger associations of cognitive ability with response errors for more complex than for less complex questions (supporting the TCH). Results replicated across two independent samples and six assessment waves. A latent variable of response errors estimated for the most complex items correlated .50 with a latent cognitive ability factor, suggesting that response patterns can be utilized to extract a rough indicator of general cognitive ability in survey research.
Collapse
Affiliation(s)
- Stefan Schneider
- Dornsife Center for Self-Report Science, and Center for Economic & Social Research, University of Southern California, 635 Downey Way, Los Angeles, CA, 90089-3332, USA.
- Department of Psychology, University of Southern California, Los Angeles, CA, USA.
- Leonard Davis School of Gerontology, University of Southern California, Los Angeles, CA, USA.
| | - Raymond Hernandez
- Dornsife Center for Self-Report Science, and Center for Economic & Social Research, University of Southern California, 635 Downey Way, Los Angeles, CA, 90089-3332, USA
| | - Doerte U Junghaenel
- Dornsife Center for Self-Report Science, and Center for Economic & Social Research, University of Southern California, 635 Downey Way, Los Angeles, CA, 90089-3332, USA
- Department of Psychology, University of Southern California, Los Angeles, CA, USA
- Leonard Davis School of Gerontology, University of Southern California, Los Angeles, CA, USA
| | - Haomiao Jin
- School of Health Sciences, Faculty of Health and Medical Sciences, University of Surrey, Guildford, UK
| | - Pey-Jiuan Lee
- Dornsife Center for Self-Report Science, and Center for Economic & Social Research, University of Southern California, 635 Downey Way, Los Angeles, CA, 90089-3332, USA
| | - Hongxin Gao
- School of Health Sciences, Faculty of Health and Medical Sciences, University of Surrey, Guildford, UK
| | - Danny Maupin
- School of Health Sciences, Faculty of Health and Medical Sciences, University of Surrey, Guildford, UK
| | - Bart Orriens
- Center for Economic and Social Research, University of Southern California, Los Angeles, CA, USA
| | - Erik Meijer
- Center for Economic and Social Research, University of Southern California, Los Angeles, CA, USA
| | - Arthur A Stone
- Dornsife Center for Self-Report Science, and Center for Economic & Social Research, University of Southern California, 635 Downey Way, Los Angeles, CA, 90089-3332, USA
- Department of Psychology, University of Southern California, Los Angeles, CA, USA
| |
Collapse
|