How knowing the rules affects solving the Raven Advanced Progressive Matrices Test

Responses to Raven matrices: Governed by visual complexity and centrality

Raven matrices are widely considered a pure test of cognitive abilities. Previous research has examined the extent to which cognitive strategies are predictive of the number of correct responses to Raven items. This study examined whether response times can be explained directly from the centrality and visual complexity of the matrix cells (edge density and perceived complexity). A total of 159 participants completed a 12-item version of the Raven Advanced Progressive Matrices. In addition to item number (an index of item difficulty), the findings demonstrated a positive correlation between the visual complexity of Raven items and both the mean response time and the number of fixations on the matrix (a strong correlate of response time). Moreover, more centrally placed cells as well as more complex cells received more fixations. It is concluded that response times on Raven matrices are impacted by low-level stimulus attributes, namely, visual complexity and eccentricity.

Knowledge Representations: Individual Differences in Novel Problem Solving

The present study investigates how the quality of knowledge representations contributes to rule transfer in a problem-solving context and how working memory capacity (WMC) might contribute to the subsequent failure or success in transferring the relevant information. Participants were trained on individual figural analogy rules and then asked to rate the subjective similarity of the rules to determine how abstract their rule representations were. This rule representation score, along with other measures (WMC and fluid intelligence measures), was used to predict accuracy on a set of novel figural analogy test items, of which half included only the trained rules, and half were comprised of entirely new rules. The results indicated that the training improved performance on the test items and that WMC largely explained the ability to transfer rules. Although the rule representation scores did not predict accuracy on the trained items, rule representation scores did uniquely explain performance on the figural analogies task, even after accounting for WMC and fluid intelligence. These results indicate that WMC plays a large role in knowledge transfer, even when transferring to a more complex problem-solving context, and that rule representations may be important for novel problem solving.

A Log File Analysis on the Validity of Partial Solutions in Figural Matrices Tests

As a component of many intelligence test batteries, figural matrices tests are an effective way to assess reasoning, which is considered a core ability of intelligence. Traditionally, the sum of correct items is used as a performance indicator (total solution procedure). However, recent advances in the development of computer-based figural matrices tests allow additional indicators to be considered for scoring. In two studies, we focused on the added value of a partial solution procedure employing log file analyses from a computer-based figural matrices test. In the first study (n = 198), we explored the internal validity of this procedure by applying both an exploratory bottom-up approach (using sequence analyses) and a complementary top-down approach (using rule jumps, an indicator taken from relevant studies). Both approaches confirmed that higher scores in the partial solution procedure were associated with higher structuredness in participants' response behavior. In the second study (n = 169), we examined the external validity by correlating the partial solution procedure in addition to the total solution procedure with a Grade Point Average (GPA) criterion. The partial solution procedure showed an advantage over the total solution procedure in predicting GPA, especially at lower ability levels. The implications of the results and their applicability to other tests are discussed.

Cognitive strategies in matrix-reasoning tasks: State of the art

Cognitive strategies in matrix-reasoning tasks have been investigated for the last decade and a half. Several steps were made since the first paper in the field, but the advances have been sparse and with little connection. Here we present a review of the state of the art in this subject. We introduce how this topic was born and how to measure these strategies, covering the diverse methods and measures that are presented in the literature to progress on this subject and the applications that were developed, as well as the knowledge that resulted from these applications. Furthermore, the future directions are discussed with the intention to engage new researchers in this topic, as well as to bring awareness to limitations that were found given the available scientific literature.

Graph Mapping: A novel and simple test to validly assess fluid reasoning

We present Graph Mapping - a simple and effective computerized test of fluid intelligence (reasoning ability). The test requires structure mapping - a key component of the reasoning process. Participants are asked to map a pair of corresponding nodes across two mathematically isomorphic but visually different graphs. The test difficulty can be easily manipulated - the more complex structurally and dissimilar visually the graphs, the higher response error rate. Graph Mapping offers high flexibility in item generation, ranging from trivial to extremally difficult items, supporting progressive item sequences suitable for correlational studies. It also allows multiple item instances (clones) at a fixed difficulty level as well as full item randomization, both particularly suitable for within-subject experimental designs, longitudinal studies, and adaptive testing. The test has short administration times and is unfamiliar to participants, yielding practical advantages. Graph Mapping has excellent psychometric properties: Its convergent validity and reliability is comparable to the three leading traditional fluid reasoning tests. The convenient software allows a researcher to design the optimal test variant for a given study and sample. Graph Mapping can be downloaded from: https://osf.io/wh7zv/.

Fluid Intelligence Emerges from Representing Relations

Based on recent findings in cognitive neuroscience and psychology as well as computational models of working memory and reasoning, I argue that fluid intelligence (fluid reasoning) can amount to representing in the mind the key relation(s) for the task at hand. Effective representation of relations allows for enormous flexibility of thinking but depends on the validity and robustness of the dynamic patterns of argument-object (role-filler) bindings, which encode relations in the brain. Such a reconceptualization of the fluid intelligence construct allows for the simplification and purification of its models, tests, and potential brain mechanisms.

Uncovering the course of analogical mapping using eye tracking

Analogical mapping - the core component of analogical reasoning - consists of establishing the relational structure shared by two analogous situations and inferring the missing elements in a less familiar situation from a more familiar one. Several existing models of analogy predicted that the complete relational structure can be considered in parallel. Other models postulated that mapping can be less or more incremental - it can access only a relatively small part of the structure, and needs to move to its other parts in steps in order to construct the final relational correspondence. However, the precise time course of analogical mapping, especially in sufficiently complex analogies, to date was rarely studied empirically. In two studies, eye tracking was used to assess in a rigorous way the extent to which mapping can be incremental. In a newly designed geometric A:B::C:D task, pattern D was generated from C according to the same shape transformations that generated pattern B from A. The six possible response options differed systematically in the number of correct transformations, from no transformation matching, via partial relational match, up to the full match. In Study 1, the relational match of options fixated on by participants was initially low but increased monotonically over the course of analogy. The number of corresponding eye fixations predicted 68% variance in relational match of the final response. The correct option was chosen only if fixated on for a sufficiently long time. Study 2 replicated the findings using a more ecologically valid and less demanding task variant that required to map the changes in people's appearance. The results support these theoretical models of analogy which postulate strictly incremental mapping.

You Can Play the Game Without Knowing the Rules – But You’re Better Off Knowing Them

Abstract. Due to their high item difficulties and excellent psychometric properties, construction-based figural matrices tasks are of particular interest when it comes to high-stakes testing. An important prerequisite is that test preparation – which is likely to occur in this context – does not impair test fairness or item properties. The goal of this study was to provide initial evidence concerning the influence of test preparation. We administered test items to a sample of N = 882 participants divided into two groups, but only one group was given information about the rules employed in the test items. The probability of solving the items was significantly higher in the test preparation group than in the control group ( M = 0.61, SD = 0.19 vs. M = 0.41, SD = 0.25; t(54) = 3.42, p = .001; d = .92). Nevertheless, a multigroup confirmatory factor analysis, as well as a differential item functioning analysis, indicated no differences between the item properties in the two groups. The results suggest that construction-based figural matrices are suitable in the context of high-stakes testing when all participants are provided with test preparation material so that test fairness is ensured.

How to Get Better: Taking Notes Mediates the Effect of a Video Tutorial on Number Series

In recent studies, test-score increases have been shown in rule-based intelligence test tasks, such as number series, after watching a corresponding video tutorial. An open question remains regarding the mechanisms involved. Specifically, taking notes to describe the relations between numbers might be linked to test scores, and is hypothesized to mediate the effect of a number series video tutorial on number series test scores. Therefore, an experimental group (EG) watching a number series video tutorial (n = 58) was compared with a control group (CG) watching an irrelevant tutorial (n = 52) before working on number series items. Results revealed higher number series scores in the EG than the CG (d = .48), more items with provided notes in the EG than in the CG (d = .41), and substantial correlations between the number of items with notes and the number series sum scores in both groups (EG: r = .66; CG: r = .75). The effect of the video tutorial on the number series sum score was mediated by the number of items with notes (indirect effect = 3.41, SE = 1.74). Theoretical and practical implications as well as future research directions are discussed.

Test Preparation in Figural Matrices Tests: Focus on the Difficult Rules

It is well documented that training the rules employed in figural matrices tests enhances test performance. Previous studies only compare experimental conditions in which all or no rules were trained and therefore ignore the particular influence of knowledge about the easy and difficult rules. With the current study, we wanted to provide some first insights into this topic. Respondents were assigned to four groups that received training for no rules, only the easy rules, only the difficult rules, or for all rules. The results show that a training only for the difficult rules was more effective than the other trainings. This applies also to performance in the easy rules that were actually not part of the training. A possible explanation for this finding is a facilitation of the solution process that is primarily driven by knowledge about the difficult rules. In conclusion, our results demonstrate that taking differences between the rules into account may provide a deeper understanding of the effects of trainings for figural matrices tests.

Metacognitive Awareness Scale, Domain Specific (MCAS-DS): Assessing Metacognitive Awareness During Raven's Progressive Matrices

Metacognition, the cognition about cognition, is closely linked to intelligence and therefore understanding the metacognitive processes underlying intelligence test performance, specifically on Raven's Progressive Matrices, could help advance the knowledge about intelligence. The measurement of metacognition, is often done using domain-general offline questionnaires or domain-specific online think-aloud protocols. This study aimed to investigate the relationship between metacognitive awareness and intelligence via the design and use of a novel Meta-Cognitive Awareness Scale - Domain Specific (MCAS-DS) that encourages reflection of task strategy processes. This domain-specific scale was first constructed to measure participants' awareness of their own metacognition linked to Raven's Progressive Matrices (SPM). Following discriminatory index and Exploratory Factor Analysis, a 15-item scale was derived. Exploratory Factor Analysis showed five factors: Awareness of Engagement in Self-Monitoring, Awareness of Own Ability, Awareness of Responding Speed/Time, Awareness of Alternative Solutions and Awareness of Requisite Problem-Solving Resources. The intelligence level of ninety-eight adults was then estimated using Raven's Standard Progressive Matrices. Participants also completed the MCAS-DS, and further items that examined their test-taking behavior and Confidence level. Metacognitive awareness was positively correlated to standardized IQ scores derived from the SPM whilst Over-Confidence derived using the Confidence level measure was negatively correlated to SPM. Despite some limitations, this study shows promise for elucidating the relationship between metacognitive awareness and intelligence using the task-specific scale.

Within-Subject Performance on a Real-Life, Complex Task and Traditional Lab Experiments: Measures of Word Learning, Raven Matrices, Tapping, and CPR

In this data report, we describe a three-session experiment spanning six months. Several well-controlled laboratory tasks (Word Learning, Raven Matrices, and Tapping) and Cardiopulmonary Resuscitation (CPR), a complex but well-defined real-world task, were administered. Data are reported from 50 participants for the first session, 40 for the second, and 34 for the third. CPR is a useful domain for studying real-world performance inside the laboratory because clear performance standards can be applied to quantifying learners' proficiency covering both the first steps that need to be taken prior to the initiation of CPR (declarative knowledge) as well as the compressions and ventilations themselves (procedural skill). This research resulted in a rich dataset with a range of different measures for all participants. For all tasks, the complete set of raw data are made available along with relevant aggregate performance scores (see https://osf.io/m8bxe/). The raw data in particular will enable other researchers to explore potential analyses and modeling beyond the scope of our own. The details of the data collection protocol and available data are documented here to facilitate this process.

Use of cognitive correction training improves learning for children with mathematics learning disability

Children with normal intelligence may experience varying degrees of mathematics learning disabilities (MD). This study aims to conduct training to improve the brain's cognitive ability for mathematics learning by focusing on two important mathematical cognitive abilities. This was a prospective study of 70 children in grades 2-5 from two primary schools in Changzhou and with MD enrolled from June 2015 to February 2017. The children were randomized 1:1 to the training and control groups. A training cycle included 40 sessions (5/weeks) (30 min each session). The efficacy of learning was assessed by assessing number learning and graph reasoning, and by using the Raven standard reasoning test score. In the training group, backward number memory (from 6.1 ± 1.8 to 6.7 ± 1.3, P = 0.02), number sequential connection (from 54.4 ± 14.5 to 47.1 ± 12.1, P < 0.01), and rapid graph judgment (from 531.9 ± 76.3 to 557.8 ± 85.7, P = 0.04) were improved by training, while there was no effect on forward number memory (P = 0.13). There were significant differences in total score and scores of b, c, and e series before and after training (all P < 0.05). The children in the control group had no improvement after 8 weeks. There was a correlation between the ability of rapid graphic judgment before and after training and the score of the Raven E series (r = 0.384, P = 0.024), and between the score of the Raven C series and the score of the Raven D series (r = 0.468, P = 0.013). Cognitive correction training improved the sensitivity to numbers and mathematics learning in children with MD.

Is the Correlation between Storage Capacity and Matrix Reasoning Driven by the Storage of Partial Solutions? A Pilot Study of an Experimental Approach

Working memory capacity (WMC) and reasoning abilities-as assessed by figural matrices tests-are substantially correlated. It is controversially discussed whether this correlation is only caused by controlled attention or also by storage capacity. This study aims at investigating storage of partial solutions as a possible mechanism by which storage capacity may contribute to solving figural matrices tests. For this purpose, we analyzed how an experimental manipulation of storage demands changes the pattern of correlations between WMC and performance in a matrix task. We manipulated the storage demands by applying two test formats: one providing the externalization of partial solutions and one without the possibility of externalization. Storage capacity was assessed by different types of change detection tasks. We found substantial correlations between storage capacity and matrices test performance, but they were of comparable size for both test formats. We take this as evidence that the necessity to store partial solutions is not the limiting factor which causes the association between storage capacity and matrices test. It is discussed how this approach can be used to investigate alternative mechanisms by that storage may influence performance in matrices tests.