Perspective-taking in dialogue as self-organization under social constraints

What Dynamic Approaches Have Taught Us About Cognition and What They Have Not: On Values in Motion and the Importance of Replicable Forms

Over the past several decades, research in the cognitive sciences has foregrounded the importance of active bodies and their continuous dependence on the changing environment, strengthening the relevance of dynamical models. These models have been steadily developed within the ecological psychology approach to cognition, which arguably contributes to the "ecological turn" we are witnessing today. The embodied and situated nature of cognition, regarded by some as a passing trend, is presently becoming a largely accepted assumption. In this paper, I claim that in light of these developments, ecological psychology, in alliance with related approaches, such as enactivism and interactivism, has the potential to deeply transform our perspectives on cognition and action, restoring their pertinence to humans as persons. However, an important challenge to the realization of this potential has to be noted: neither the mainstream information-processing approach nor the dynamics-oriented perspective on cognition provides an account of how the capacity of humans to use language and think "symbolically" can be derived from the continuous flow of agent-environment interaction. I will attempt to show that posing the "dynamical" and "computational" hypotheses about the nature of cognition as mutually exclusive approaches to cognition results in undesirable reductionism, which makes it difficult to meet this challenge. There are good reasons, advanced over half a century ago by, for example, Michael Polanyi or Howard Pattee, to think that we need complementary descriptions to understand cognizing systems, in order to grasp the fact that they are governed both by physical laws and by emergent historical constraints. Details of such a complementarity-based approach still await elucidation, but some proposed solutions have the potential to ease the tension between the information-processing and dynamical approaches to cognition and to lead to a better understanding of their interrelation.

Learning about others: Modeling social inference through ambiguity resolution

Bayesian accounts of social cognition successfully model the human ability to infer goals and intentions of others on the basis of their behavior. In this paper, we extend this paradigm to the analysis of ambiguity resolution during brief communicative exchanges. In a reference game experimental setup, we observed that participants were able to infer listeners' preferences when analyzing their choice of object given referential ambiguity. Moreover, a subset of speakers was able to strategically choose ambiguous over unambiguous utterances in an epistemic manner, although a different group preferred unambiguous utterances. We show that a modified Rational Speech Act model well-approximates the data of both the inference of listeners' preferences and their utterance choices. In particular, the observed preference inference is modeled by Bayesian inference, which computes posteriors over hypothetical, behavior-influencing inner states of conversation partners-such as their knowledge, beliefs, intentions, or preferences-after observing their utterance-interpretation behavior. Utterance choice is modeled by anticipating social information gain, which we formalize as the expected knowledge change, when choosing a particular utterance and watching the listener's response. Taken together, our results demonstrate how social conversations allow us to (sometimes strategically) learn about each other when observing interpretations of ambiguous utterances.

What is retained about common ground? Distinct effects of linguistic and visual co-presence

Common ground can be mutually established between conversational partners in several ways. We examined whether the modality (visual or linguistic) with which speakers share information with their conversational partners results in memory traces that affect subsequent references addressed to a particular partner. In 32 triads, directors arranged a set of tangram cards with one matcher and then with another, but in different modalities, sharing some cards only linguistically (by describing cards the matcher couldn't see), some only visually (by silently showing them), some both linguistically and visually, and others not at all. Then directors arranged the cards again in separate rounds with each matcher. The modality with which they previously established common ground about a particular card with a particular matcher (e.g., linguistically with one partner and visually with the other) affected subsequent referring: References to cards previously shared only visually included more idea units, words, and reconceptualizations than those shared only linguistically, which in turn included more idea units, words, and reconceptualizations than those shared both linguistically and visually. Moreover, speakers were able to tailor references to the same card appropriately to the distinct modality shared with each addressee. Such gradient, partner-specific adaptation during re-referring suggests that memory encodes rich-enough representations of multimodal shared experiences to effectively cue relevant constraints about the perceptual conditions under which speakers and addressees establish common ground.

Why are listeners sometimes (but not always) egocentric? Making inferences about using others' perspective in referential communication

Theory of Mind (ToM) is the ability to understand others’ mental states, and that these mental states can differ from our own. Although healthy adults have little trouble passing conceptual tests of ToM (e.g., the false belief task [1]), they do not always succeed in using ToM [2,3]. In order to be successful in referential communication, listeners need to correctly infer the way in which a speaker’s perspective constrains reference and inhibit their own perspective accordingly. However, listeners may require prompts to take these effortful inferential steps. The current study investigated the possibility of embedding prompts in the instructions for listeners to make inference about using a speaker’s perspective. Experiment 1 showed that provision of a clear introductory example of the full chain of inferences resulted in large improvement in performance. Residual egocentric errors suggested that the improvement was not simply due to superior comprehension of the instructions. Experiment 2 further dissociated the effect by placing selective emphasis on making inference about inhibiting listeners’ own perspective versus using the speaker’s perspective. Results showed that only the latter had a significant effect on successful performance. The current findings clearly demonstrated that listeners do not readily make inferences about using speakers’ perspectives, but can do so when prompted.

Interacting Timescales in Perspective-Taking

Through theoretical discussion, literature review, and a computational model, this paper poses a challenge to the notion that perspective-taking involves a fixed architecture in which particular processes have priority. For example, some research suggests that egocentric perspectives can arise more quickly, with other perspectives (such as of task partners) emerging only secondarily. This theoretical dichotomy-between fast egocentric and slow other-centric processes-is challenged here. We propose a general view of perspective-taking as an emergent phenomenon governed by the interplay among cognitive mechanisms that accumulate information at different timescales. We first describe the pervasive relevance of perspective-taking to cognitive science. A dynamical systems model is then introduced that explicitly formulates the timescale interaction proposed. This model illustrates that, rather than having a rigid time course, perspective-taking can be fast or slow depending on factors such as task context. Implications are discussed, with ideas for future empirical research.

The Effect of e-Book Vocabulary Instruction on Spanish-English Speaking Children

PURPOSE

This study aimed to examine the effect of an intensive vocabulary intervention embedded in e-books on the vocabulary skills of young Spanish-English speaking English learners (ELs) from low-socioeconomic status backgrounds.

METHOD

Children (N = 288) in kindergarten and 1st grade were randomly assigned to treatment and read-only conditions. All children received e-book readings approximately 3 times a week for 10-20 weeks using the same books. Children in the treatment condition received e-books supplemented with vocabulary instruction that included scaffolding through explanations in Spanish, repetition in English, checks for understanding, and highlighted morphology.

RESULTS

There was a main effect of the intervention on expressive labeling (g = 0.38) and vocabulary on the Peabody Picture Vocabulary Test-Fourth Edition (g = 0.14; Dunn & Dunn, 2007), with no significant moderation effect of initial Peabody Picture Vocabulary Test score. There was no significant difference between conditions on children's expressive definitions.

CONCLUSION

Findings substantiate the effectiveness of computer-implemented embedded vocabulary intervention for increasing ELs' vocabulary knowledge.

IMPLICATIONS

Computer-assisted vocabulary instruction with scaffolding through Spanish explanations, repetitions, and highlighted morphology is a promising approach to facilitate word learning for ELs in kindergarten and 1st grade.

Step by step: Harvesting the dynamics of delay discounting decisions

People show a tendency to devalue rewards when they are delayed in time. This so-called delay discounting often happens to an extent that seems irrational from an economical perspective. Research studying outcomes of delay discounting decisions has successfully derived descriptive models for such choice preferences. However, this outcome-based approach faces limitations in integrating the influence of contextual factors on the decision. Recently, this outcome-centred perspective on delay discounting has been complemented by a focus on the process dynamics leading to delay discounting decisions. Here, we embrace and add to this approach: We show how an attractor model can extend discounting descriptive discounting curves into the temporal dimension. From the model, we derive three predictions and study the predictions in a delay discounting experiment based on mouse tracking. We find differences in discounting depending on the order of option presentation and more direct movements to options presented first. Together with the analysis of specific temporal patterns of information integration, these results show that considering the continuous process dynamics of delay discounting decisions and harvesting them with continuous behavioural measures allow the integration of contextual factors into existing models of delay discounting, not as noise but as a signal on their own.

Process dynamics in delay discounting decisions: An attractor dynamics approach

How do people make decisions between an immediate but small reward and a delayed but large one? The outcome of such decisions indicates that people discount rewards by their delay and hence these outcomes are well described by discounting functions. However, to understand irregular decisions and dysfunctional behavior one needs models which describe how the process of making the decision unfolds dynamically over time: how do we reach a decision and how do sequential decisions influence one another? Here, we present an attractor model that integrates into and extends discounting functions through a description of the dynamics leading to a final choice outcome within a trial and across trials. To validate this model, we derive qualitative predictions for the intra-trial dynamics of single decisions and for the inter-trial dynamics of sequences of decisions that are unique to this type of model. We test these predictions in four experiments based on a dynamic delay discounting computer game where we study the intra-trial dynamics of single decisions via mouse tracking and the inter-trial dynamics of sequences of decisions via sequentially manipulated options. We discuss how integrating decision process dynamics within and across trials can increase our understanding of the processes underlying delay discounting decisions and, hence, complement our knowledge about decision outcomes.

The Virtual Teacher (VT) Paradigm: Learning New Patterns of Interpersonal Coordination Using the Human Dynamic Clamp

The Virtual Teacher paradigm, a version of the Human Dynamic Clamp (HDC), is introduced into studies of learning patterns of inter-personal coordination. Combining mathematical modeling and experimentation, we investigate how the HDC may be used as a Virtual Teacher (VT) to help humans co-produce and internalize new inter-personal coordination pattern(s). Human learners produced rhythmic finger movements whilst observing a computer-driven avatar, animated by dynamic equations stemming from the well-established Haken-Kelso-Bunz (1985) and Schöner-Kelso (1988) models of coordination. We demonstrate that the VT is successful in shifting the pattern co-produced by the VT-human system toward any value (Experiment 1) and that the VT can help humans learn unstable relative phasing patterns (Experiment 2). Using transfer entropy, we find that information flow from one partner to the other increases when VT-human coordination loses stability. This suggests that variable joint performance may actually facilitate interaction, and in the long run learning. VT appears to be a promising tool for exploring basic learning processes involved in social interaction, unraveling the dynamics of information flow between interacting partners, and providing possible rehabilitation opportunities.

The coordination dynamics of social neuromarkers

Social behavior is a complex integrative function that entails many aspects of the brain’s sensory, cognitive, emotional and movement capacities. Its neural processes are seldom simultaneous but occur according to precise spatiotemporal choreographies, manifested by the coordination of their oscillations within and between brains. Methods with good temporal resolution can help to identify so-called “neuromarkers” of social function and aid in disentangling the dynamical architecture of social brains. In our ongoing research, we have used dual-electroencephalography (EEG) to study neuromarker dynamics during synchronic interactions in which pairs of subjects coordinate behavior spontaneously and intentionally (social coordination) and during diachronic transactions that require subjects to perceive or behave in turn (action observation, delayed imitation). In this paper, after outlining our dynamical approach to the neurophysiological basis of social behavior, we examine commonalities and differences in the neuromarkers that are recruited for both kinds of tasks. We find the neuromarker landscape to be task-specific: synchronic paradigms of social coordination reveal medial mu, alpha and the phi complex as contributing neuromarkers. Diachronic tasks recruit alpha as well, in addition to lateral mu rhythms and the newly discovered nu and kappa rhythms whose functional significance is still unclear. Social coordination, observation, and delayed imitation share commonality of context: in each of our experiments, subjects exchanged information through visual perception and moved in similar ways. Nonetheless, there was little overlap between their neuromarkers, a result that hints strongly of task-specific neural mechanisms for social behavior. The only neuromarker that transcended both synchronic and diachronic social behaviors was the ubiquitous alpha rhythm, which appears to be a key signature of visually-mediated social behaviors. The present paper is both an entry point and a challenge: much work remains to determine the nature and scope of recruitment of other neuromarkers, and to create theoretical models of their within- and between-brain dynamics during social interaction.

Pooling the ground: understanding and coordination in collective sense making

Common ground is most often understood as the sum of mutually known beliefs, knowledge, and suppositions among the participants in a conversation. It explains why participants do not mention things that should be obvious to both. In some accounts of communication, reaching a mutual understanding, i.e., broadening the common ground, is posed as the ultimate goal of linguistic interactions. Yet, congruent with the more pragmatic views of linguistic behavior, in which language is treated as social coordination, understanding each other is not the purpose (or not the sole purpose) of linguistic interactions. This purpose is seen as at least twofold (e.g., Fusaroli et al., 2014): to maintain the systemic character of a conversing dyad and to organize it into a functional synergy in the face of tasks posed for a dyadic system as a whole. It seems that the notion of common ground is not sufficient to address the latter character of interaction. In situated communication, in which meaning is created in a distributed way in the very process of interaction, both common (sameness) and privileged (diversity) information must be pooled task-dependently across participants. In this paper, we analyze the definitions of common and privileged ground and propose a conceptual extension that may facilitate a theoretical account of agents that coordinate via linguistic communication. To illustrate the usefulness of this augmented framework, we apply it to one of the recurrent issues in psycholinguistic research, namely the problem of perspective-taking in dialog, and draw conclusions for the broader problem of audience design.