Interaction between language and vision: it's momentary, abstract, and it develops

Graph methods to infer spatial disturbances: Application to Huntington's Disease's speech

OBJECTIVE

Huntington's Disease (HD) is an inherited neurodegenerative disease caused by the mutation of the Htt gene, impacting all aspects of living and functioning. Among cognitive disabilities, spatial capacities are impaired, but their monitoring remains scarce as limited by lengthy experts' assessments. Language offers an alternative medium to evaluate patients' performance in HD. Yet, its capacities to assess HD's spatial abilities are unknown. Here, we aimed to bring proof-of-concept that HD's spatial deficits can be assessed through speech.

METHODS

We developed the Spatial Description Model to graphically represent spatial relations described during the Cookie Theft Picture (CTP) task. We increased the sensitivity of our model by using only sentences with spatial terms, unlike previous studies in Alzheimer's disease. 78 carriers of the mutant Htt, including 56 manifest and 22 premanifest individuals, as well as 25 healthy controls were included from the BIOHD & (NCT01412125) & Repair-HD (NCT03119246) cohorts. The convergence and divergence of the model were validated using the SelfCog battery.

RESULTS

Our Spatial Description Model was the only one among the four assessed approaches, revealing that individuals with manifest HD expressed fewer spatial relations and engaged in less spatial exploration compared to healthy controls. Their graphs correlated with both visuospatial and language SelfCog performances, but not with motor, executive nor memory functions.

CONCLUSIONS

We provide the proof-of-concept using our Spatial Description Model that language can grasp HD patient's spatial disturbances. By adding spatial capabilities to the panel of functions tested by the language, it paves the way for eventual remote clinical application.

What have we learned from research on the "geometric module"?

This article is an overview of the research and controversy initiated by Cheng's (Cognition, 23(2), 149-178, 1986) article hypothesizing a purely geometric module in spatial representation. Hundreds of experiments later, we know much more about spatial behavior across a very wide array of species, ages, and kinds of conditions, but there is still no consensus model of the phenomena. I argue for an adaptive combination approach that entails several principles: (1) a focus on ecological niches and the spatial information they offer; (2) an approach to development that is experience-expectant: (3) continued plasticity as environmental conditions change; (4) language as one of many cognitive tools that can support spatial behavior.

Language systematizes attention: How relational language enhances relational representation by guiding attention

Language can affect cognition, but through what mechanism? Substantial past research has focused on how labeling can elicit categorical representation during online processing. We focus here on a particularly powerful type of language-relational language-and show that relational language can enhance relational representation in children through an embodied attention mechanism. Four-year-old children were given a color-location conjunction task, in which they were asked to encode a two-color square, split either vertically or horizontally (e.g., red on the left, blue on the right), and later recall the same configuration from its mirror reflection. During the encoding phase, children in the experimental condition heard relational language (e.g., "Red is on the left of blue"), while those in the control condition heard generic non-relational language (e.g., "Look at this one, look at it closely"). At recall, children in the experimental condition were more successful at choosing the correct relational representation between the two colors compared to the control group. Moreover, they exhibited different attention patterns as predicted by the attention shift account of relational representation (Franconeri et al., 2012). To test the sustained effect of language and the role of attention, during the second half of the study, the experimental condition was given generic non-relational language. There was a sustained advantage in the experimental condition for both behavioral accuracies and signature attention patterns. Overall, our findings suggest that relational language enhances relational representation by guiding learners' attention, and this facilitative effect persists over time even in the absence of language. Implications for the mechanism of how relational language can enhance the learning of relational systems (e.g., mathematics, spatial cognition) by guiding attention will be discussed.

Different reference frames on different axes: Space and language in indigenous Amazonians

Spatial cognition is central to human behavior, but the way people conceptualize space varies within and across groups for unknown reasons. Here, we found that adults from an indigenous Bolivian group used systematically different spatial reference frames on different axes, according to known differences in their discriminability: In both verbal and nonverbal tests, participants preferred allocentric (i.e., environment-based) space on the left-right axis, where spatial discriminations (like "b" versus "d") are notoriously difficult, but the same participants preferred egocentric (i.e., body-based) space on the front-back axis, where spatial discrimination is relatively easy. The results (i) establish a relationship between spontaneous spatial language and memory across axes within a single culture, (ii) challenge the claim that each language group has a predominant spatial reference frame at a given scale, and (iii) suggest that spatial thinking and language may both be shaped by spatial discrimination abilities, as they vary across cultures and contexts.

Visual foundations of Euclidean geometry

Geometry defines entities that can be physically realized in space, and our knowledge of abstract geometry may therefore stem from our representations of the physical world. Here, we focus on Euclidean geometry, the geometry historically regarded as "natural". We examine whether humans possess representations describing visual forms in the same way as Euclidean geometry - i.e., in terms of their shape and size. One hundred and twelve participants from the U.S. (age 3-34 years), and 25 participants from the Amazon (age 5-67 years) were asked to locate geometric deviants in panels of 6 forms of variable orientation. Participants of all ages and from both cultures detected deviant forms defined in terms of shape or size, while only U.S. adults drew distinctions between mirror images (i.e. forms differing in "sense"). Moreover, irrelevant variations of sense did not disrupt the detection of a shape or size deviant, while irrelevant variations of shape or size did. At all ages and in both cultures, participants thus retained the same properties as Euclidean geometry in their analysis of visual forms, even in the absence of formal instruction in geometry. These findings show that representations of planar visual forms provide core intuitions on which humans' knowledge in Euclidean geometry could possibly be grounded.

Examining the role of external language support and children's own language use in spatial development

This research investigated whether an experimental manipulation providing children with external language support reflects developmental processes whereby children come to use language within spatial tasks. A total of 121 3- to 6-year-old children participated in language production and spatial recall tasks. The Production task measured children's task-relevant descriptions of spatial relations on the testing array. The Recall task assessed children's delayed search for hidden object locations on the testing array relative to one or more spatial reference frames (egocentric, room-centered, and intrinsic). During the Recall task, the experimenter provided children with either descriptive or nondescriptive verbal cues. Results showed that children's task-relevant language production improved with age and the effects of language support on spatial performance decreased with age. However, children's production of task-relevant language did not account for effects of language support. Instead, children benefited from language support irrespective of their task-relevant language production. These results suggest that verbal encoding is not a spontaneous process that young children use in support of their spatial performance. In addition, experimental manipulations of language support are not fully reflective of the ways in which children come to use language within spatial tasks.

The role of redundant verbal labels in 8- and 10-month-olds' working memory

Verbal labels have been shown to help preverbal infants' performance on various cognitive tasks, such as categorization. Redundant labels also aid adults' visual working memory (WM), but it is not known if this linguistic benefit extends to preverbal infants' WM. In two eye-tracking studies, we tested whether 8- and 10-month-old infants' WM performance would improve with the presence of redundant labels in a Delayed Match Retrieval (DMR) paradigm that tested infants' WM for object-location bindings. Findings demonstrated that infants at both ages were unable to remember two object-location bindings when co-presented with labels at encoding. Moreover, infants who encoded the object-location bindings with labels were not significantly better than those who did so in silence. These findings are discussed in the context of label advantages in cognition and auditory dominance.

Spatial Language of Young Children During Block Play in Kindergartens in Urban China

Spatial language is an important predictor of spatial skills and might be inspired by peer interaction and goal-oriented building behaviors during block play. The present study investigated the frequency, type and level of children’s spatial language during block play and their associations with the level of block play by observing 228 young children in classrooms equipped with unit blocks and allowing free play on a daily basis. The findings showed that during block play, young children used more words about spatial locations, deictic terms, dimensions, and shapes and fewer words about spatial features or properties and spatial orientations or transformations. Spatial locations were used most frequently, and young children tended to use vertical location words to represent the corresponding location. Most young children used gestures in conjunction with spatial deictic terms. Among shape words, tetragon words were frequently used, and the representation of spatial shapes showed alternatives, collective tendencies and gender differences. The use of spatial language during the play process had a significant positive correlation with age, the construction structure, and form of block building.

Speech and Gesture Production Provide Unique Insights Into Young Children's Spatial Reasoning

This study took a novel approach to understanding the role of language in spatial development by combining approaches from spatial language and gesture research. It analyzed forty-three 4.5- to 6-year-old's speech and gesture production during explanations of reasoning behind performance on Spatial Analogies and Children's Mental Transformation Tasks. Results showed that speech and gesture relevant for solving the trials (disambiguating correct choices) predicted spatial performance when controlling for age, gender, and spatial words and gestures produced. Children performed the spatial tasks well if they produced relevant information either verbally through speech or nonverbally through gesture. These results highlight the importance of not only focusing on concepts children can reference but also on how such concepts are used in spatial tasks.

Using eye-tracking to understand relations between visual attention and language in children's spatial skills

Relations between children's spatial language and spatial skills raise questions regarding whether the effects are unique to language or reflect non-linguistic processes. Different paradigms provided mixed evidence: experimenter-provided language supports spatial performance more than visual cues; however, children's non-verbal attention predicts their spatial performance more than their language production. The current study used eye-tracking during spatial recall to compare effects of language versus visual cues. Four- to five-year-old children completed two tasks requiring memory for the location of a toy under one of four cups in an array of cups and landmarks after a 5 s delay and array rotation. Children first completed the baseline task with non-specific cues, followed by the cue-manipulation task with either language, visual, or non-specific cues provided by the experimenter. As in prior studies, language cues were most effective in facilitating recall. Children's visual attention was directed by both language and visual cues to support their recall. However, visual attention only partially mediated the effects of language: language supported recall above and beyond directing visual attention. These results indicate that visual attention supports spatial recall, but language has additional unique influences. This may result from language providing a more coherent or redundant code to visual information, or due to the pragmatic nature of language cueing relevance in ways visual cues do not. Additionally, differences across conditions may reflect more benefit from endogenous versus exogenous attentional control. Through using eye-tracking, this research provided new insights into processes by which language and visual attention influence children's spatial cognition.

Measuring Spontaneous Focus on Space in Preschool Children

Previous work on children's Spontaneous Focus on Numerosity (SFON) has shown the value of measuring children's spontaneous attention within naturalistic interactions. SFON is the spontaneous tendency to focus attention on, and explicitly enumerate the exact number of, items in a set. This measure predicts later math skills above and beyond general IQ and other cognitive factors such as attention. The utility of SFON suggests that a parallel construct for space is a worthy pursuit; spatial cognition underlies many of our mathematical skills, especially as children are first learning these skills. We developed a measure of children's Spontaneous Focus on Space - the spontaneous tendency to attend to absolute and relative spatial components of the environment - and studied its relation to reasoning about the important spatial-numerical concept of proportions. Fifty-five 3- to 6-year-olds were tested at a local children's museums in New York City. Children participated in tasks designed to measure their spontaneous focus on space and number, and their ability to reason about spatial proportions. Results indicate that as children grow older, their Spontaneous Focus on Space becomes more complete and is positively related to proportional reasoning performance. These findings suggest that spatial awareness is rapidly increasing in the preschool years, alongside numerical awareness and spatial-numerical proportional reasoning.

Children's attention to task-relevant information accounts for relations between language and spatial cognition

Children's spatial language reliably predicts their spatial skills, but the nature of this relation is a source of debate. This investigation examined whether the mechanisms accounting for such relations are specific to language use or reflect a domain-general mechanism of selective attention. Experiment 1 examined whether 4-year-olds' spatial skills were predicted by their selective attention or their adaptive language use. Children completed (a) an attention task assessing attention to task-relevant color, size, and location cues; (b) a description task assessing adaptive language use to describe scenes varying in color, size, and location; and (c) three spatial tasks. There was correspondence between the cue types that children attended to and produced across description and attention tasks. Adaptive language use was predicted by both children's attention and task-related language production, suggesting that selective attention underlies skills in using language adaptively. After controlling for age, gender, receptive vocabulary, and adaptive language use, spatial skills were predicted by children's selective attention. The attention score predicted variance in spatial performance previously accounted for by adaptive language use. Experiment 2 followed up on the attention task (Experiment 2a) and description task (Experiment 2b) from Experiment 1 to assess whether performance in the tasks related to selective attention or task-specific demands. Performance in Experiments 2a and 2b paralleled that in Experiment 1, suggesting that the effects in Experiment 1 reflected children's selective attention skills. These findings show that selective attention is a central factor supporting spatial skill development that could account for many effects previously attributed to children's language use.

When a word is worth more than a picture: Words lower the threshold for object identification in 3-year-old children

A large literature shows strong developmental links between early language abilities and later cognitive abilities. We present evidence for one pathway by which language may influence cognition and development: by influencing how visual information is momentarily processed. Children were asked to identify a target in clutter and either saw a visual preview of the target or heard the basic-level name of the target. We hypothesized that the name of the target should activate category-relevant information and, thus, facilitate more rapid detection of the target amid distractors. Children who heard the name of the target before search were more likely to correctly identify the target at faster speeds of response, a result that supports the idea that words lower the threshold for target identification. This finding has significant implication for understanding the source of vocabulary-mediated individual differences in cognitive achievement and, more generally, for the relation between language and thought.

Parents' Spatial Language Mediates a Sex Difference in Preschoolers' Spatial-Language Use

Do boys produce more terms than girls to describe the spatial world-that is, dimensional adjectives (e.g., big, little, tall, short), shape terms (e.g., circle, square), and words describing spatial features and properties (e.g., bent, curvy, edge)? If a sex difference in children's spatial-language use exists, is it related to the spatial language that parents use when interacting with children? We longitudinally tracked the development of spatial-language production in children between the ages of 14 and 46 months in a diverse sample of 58 parent-child dyads interacting in their homes. Boys produced and heard more of these three categories of spatial words, which we call "what" spatial types (i.e., unique "what" spatial words), but not more of all other word types, than girls. Mediation analysis revealed that sex differences in children's spatial talk at 34 to 46 months of age were fully mediated by parents' earlier spatial-language use, when children were 14 to 26 months old, time points at which there was no sex difference in children's spatial-language use.

An adaptive cue combination model of human spatial reorientation

Previous research has proposed an adaptive cue combination view of the development of human spatial reorientation (Newcombe & Huttenlocher, 2006), whereby information from multiple sources is combined in a weighted fashion in localizing a target, as opposed to being modular and encapsulated (Hermer & Spelke, 1996). However, no prior work has formalized this proposal and tested it against existing empirical data. We propose a computational model of human spatial reorientation that is motivated by probabilistic approaches to optimal perceptual cue integration (e.g. Ernst & Banks, 2002) and to spatial location coding (Huttenlocher, Hedges, & Duncan, 1991). We show that this model accounts for data from a variety of human reorientation experiments, providing support for the adaptive combination view of reorientation.

Producing Spatial Words Is Not Enough: Understanding the Relation Between Language and Spatial Cognition

Prior research has investigated the relation between children's language and spatial cognition by assessing the quantity of children's spatial word production, with limited attention to the context in which children use such words. This study tested whether 4-year-olds children's (N = 41, primarily white middle class) adaptive use of task-relevant language across contexts predicted their spatial skills. Children were presented with a spatial scene description task, four spatial tasks, and vocabulary assessments. Children's adaptive use of task-relevant language was more predictive of their spatial skills than demographic and language factors (e.g., quantity of spatial words produced). These findings identify new links between language and spatial cognition and highlight the importance of understanding the quality, not just quantity, of children's language use.

The development of spatial category representations from 4 to 7 years

Representation of spatial categories was assessed in 4- to 7-year-olds. Across nine spatial categories (In, On, Under, In Front, Behind, Above, Below, Left, and Right), children were asked to pick the odd-one-out from four images, three of which displayed the same spatial relationship between two objects, and one which showed a different spatial relationship. Results support our proposed model of spatial category representation. Children progressed through three levels of understanding: from rigid (level 1), to abstract (level 2) to broad (including non-prototypical category exemplars) (level 3) understanding of spatial category membership. This developmental pattern was common to all spatial categories, and the ages at which children reached each level varied across categories, in line with the order in which category representations emerge in infancy.

How Children and Adults Encode Causative Events Cross-Linguistically: Implications for Language Production and Attention

This study investigates the implications of language-specific constraints on linguistic event encoding for the description and on-line inspection of causative events. English-speaking and Greek-speaking adults, 3-year-olds, and 4-year-olds viewed and described causative events, which are composed of Means and Result subevents, in an eyetracking study. The results demonstrate cross-linguistic differences in the informational content of causative event descriptions: Greek speakers across age groups were more likely than English speakers to mention only one causative subevent. Developmental changes in the tendency to encode information about causative events in language were also evident: in both language groups, adults were more likely than children to mention both Means and Result subevents. Finally, for both adult and child speakers of both languages, preparing different types of event descriptions changed the way that events were visually inspected, shifting attention toward to-be-encoded subevents. These findings offer some of the first evidence about the development of the language production system, the attentional mechanisms that it employs, and its workings in speakers of different languages.

Language and memory for object location

In three experiments, we investigated the influence of two types of language on memory for object location: demonstratives (this, that) and possessives (my, your). Participants first read instructions containing demonstratives/possessives to place objects at different locations, and then had to recall those object locations (following object removal). Experiments 1 and 2 tested contrasting predictions of two possible accounts of language on object location memory: the Expectation Model (Coventry, Griffiths, & Hamilton, 2014) and the congruence account (Bonfiglioli, Finocchiaro, Gesierich, Rositani, & Vescovi, 2009). In Experiment 3, the role of attention allocation as a possible mechanism was investigated. Results across all three experiments show striking effects of language on object location memory, with the pattern of data supporting the Expectation Model. In this model, the expected location cued by language and the actual location are concatenated leading to (mis)memory for object location, consistent with models of predictive coding (Bar, 2009; Friston, 2003).

Language supports young children's use of spatial relations to remember locations

Two experiments investigated the role of language in children's spatial recall performance. In particular, we assessed whether selecting an intrinsic reference frame could be improved through verbal encoding. Selecting an intrinsic reference frame requires remembering locations relative to nearby objects independent of one's body (egocentric) or distal environmental (allocentric) cues, and does not reliably occur in children under 5 years of age (Nardini, Burgess, Breckenridge, & Atkinson, 2006). The current studies tested the relation between spatial language and 4-year-olds' selection of an intrinsic reference frame in spatial recall. Experiment 1 showed that providing 4-year-olds with location-descriptive cues during (Exp. 1a) or before (Exp. 1b) the recall task improved performance both overall and specifically on trials relying most on an intrinsic reference frame. Additionally, children's recall performance was predicted by their verbal descriptions of the task space (Exp. 1a control condition). Non-verbally highlighting relations among objects during the recall task (Exp. 2) supported children's performance relative to the control condition, but significantly less than the location-descriptive cues. These results suggest that the ability to verbally represent relations is a potential mechanism that could account for developmental changes in the selection of an intrinsic reference frame during spatial recall.

Gesture is More Effective than Spatial Language in Encoding Spatial Information

The present research investigates whether producing gestures with and without speech facilitates route learning at different levels of route complexity and in learners with different levels of spatial skills. It also examines whether the facilitation effect of gesture is stronger than that of spatial language. Adults studied routes with 10, 13, and 16 steps and reconstructed them with sticks, either without rehearsal or after rehearsal by producing gestures with speech, gestures alone, or speech only. For all levels of route complexity and spatial skills, participants who were encouraged to gesture (with or without speech) during rehearsal had the best recall. Additionally, we found that number of steps rehearsed in gesture, but not that rehearsed in speech, predicted the recall accuracy. Thus, gesture is more effective than spatial language in encoding spatial information, and thereby enhancing spatial recall. These results further corroborate the beneficial nature of gesture in processing spatial information.