Reconstructing fine-grained cognition from brain activity

Tracking the Cognitive Band in an Open-Ended Task

Open-ended tasks can be decomposed into the three levels of Newell's Cognitive Band: the Unit-Task level, the Operation level, and the Deliberate-Act level. We analyzed the video game Co-op Space Fortress at these levels, reporting both the match of a cognitive model to subject behavior and the use of electroencephalogram (EEG) to track subject cognition. The Unit Task level in this game involves coordinating with a partner to kill a fortress. At this highest level of the Cognitive Band, there is a good match between subject behavior and the model. The EEG signals were also strong enough to track when Unit Tasks succeeded or failed. The intermediate Operation level in this task involves legs of flight to achieve a kill. The EEG signals associated with these operations are much weaker than the signals associated with the Unit Tasks. Still, it was possible to reconstruct subject play with much better than chance success. There were significant differences in the leg behavior of subjects and models. Model behavior did not provide a good basis for interpreting a subject's behavior at this level. At the lowest Deliberate-Act level, we observed overlapping key actions, which the model did not display. Such overlapping key actions also frustrated efforts to identify EEG signals of motor actions. We conclude that the Unit-task level is the appropriate level both for understanding open-ended tasks and for using EEG to track the performance of open-ended tasks.

Profiling cognitive workload in an unmanned vehicle control task with cognitive models and physiological metrics

In the present study, we use Cognitive Metrics Profiling (CMP) to capture variance in cognitive load within a complex unmanned vehicle control task. We aim to demonstrate convergent validity with existing workload measurement methods, and to decompose workload into constituent cognitive resources to aid in diagnosing causes of workload. A cognitive model of the task was developed and examined to determine the extent to which it could predict behavioral performance, subjective workload, and validated physiological workload metrics. We also examined model activity to draw insights regarding loaded cognitive capacities. We found that composite workload from the model predicted physiological metrics, performance, and subjective workload. Moreover, the model indicates that differences in workload were driven largely by procedural, declarative, and temporal memory demands. We have found preliminary evidence of correspondence between workload predictions of a CMP model and physiological measures of workload. This suggests our approach captures interesting aspects of workload in a complex task environment and may provide a theoretical link between behavioral, physiological, and subjective metrics. This approach may provide a means to design effective workload mitigation interventions and improve decision-making about personnel tasking and automation.

Discovering skill

This paper shows how identical skills can emerge either from instruction or discovery when both result in an understanding of the causal structure of the task domain. The paper focuses on the discovery process, extending the skill acquisition model of Anderson et al. (2019) to address learning by discovery. The discovery process involves exploring the environment and developing associations between discontinuities in the task and events that precede them. The growth of associative strength in ACT-R serves to identify potential causal connections. The model can derive operators from these discovered causal relations just as does with the instructed causal information. Subjects were given a task of learning to play a video game either with a description of the game's causal structure (Instruction) or not (Discovery). The Instruction subjects learned faster, but successful Discovery subjects caught up. After 20 3-minute games the behavior of the successful subjects in the two groups was largely indistinguishable. The play of these Discovery subjects jumped in the same discrete way as did the behavior of simulated subjects in the model. These results show how implicit processes (associative learning, control tuning) and explicit processes (causal inference, planning) can combine to produce human learning in complex environments.