Left-right coding of past and future in language: the mental timeline during sentence processing

SNARC-like effect for tempo is consistent for fast and full tempo ranges but still controversial for slow tempo range

Recent evidence suggested the existence of a spatial associations for music tempo with faster left-hand responses to relatively slow tempos and faster right-hand responses to relatively fast tempos. We refer to a study that systematically explored these spatial associations across different tempo ranges, revealed a clear effect only in the fast tempo range (DOI 10.3758/s13414-019-01945-8). The present study further investigated whether a spatial association exists across different tempo ranges (i.e., "full", "slow" or "fast" tempo range). In particular, the present study was conducted aiming (1) to test the spatial associations for tempo in the full tempo range (Experiment 1) and (2) to further investigate the occurrence of this spatial associations in the slow and fast tempo ranges (Experiment 2). Experiment 1 revealed a spatial association for tempo occurs in the full tempo range (40-200 bpm). Experiment 2 confirmed this association in the fast tempo range (133-201 bpm) but showed contradictory results in the slow tempo range (40-104 bpm). This suggests that a spatial association is plausible in the slow tempo range, although further research is needed to clarify this phenomenon.

Writing direction influences the spatial representations of past- and future-tense forms: Evidence from eye tracking

The present study tests the hypothesis that the directionality of reading habits (left-to-right or right-to-left) impacts individuals' representation of nonspatial events. Using the blank screen paradigm, we examine whether eye movements reflect culture-specific spatial biases in processing temporal information, specifically, grammatical tense in Russian and Hebrew. Sixty-two native speakers of Russian (a language with a left-to-right reading and writing system) and 62 native speakers of Hebrew (a language with a right-to-left reading and writing system) listened to verbs in the past or future tense while their spontaneous gaze positions were recorded. Following the verb, a visual spatial probe appeared in one of the five locations of the screen, and participants responded manually to indicate its position. While participants' response latencies to the spatial probe revealed no significant effects, their gaze positions along the horizontal axis for past- and future-tensed verbs aligned with the reading and writing direction in their language. These results provide novel evidence that eye movements during auditory processing of grammatical tense are influenced by culturally specific reading and writing conventions, shifting leftward or rightward on the horizontal plane depending on the stimuli's time reference (past or future) and the participants' language (Russian or Hebrew). This spatial bias indicates a common underlying cognitive mechanism that uses spatial dimensions to represent temporal constructs.

Keeping track of time: Horizontal spatial biases for hours, days, and months

In many Western cultures, the processing of temporal words related to the past and to the future is associated with left and right space, respectively - a phenomenon known as the horizontal Mental Time Line (MTL). While this mapping is apparently quite ubiquitous, its regularity and consistency across different types of temporal concepts remain to be determined. Moreover, it is unclear whether such spatial mappings are an essential and early constituent of concept activation. In the present study, we used words denoting time units at different scales (hours of the day, days of the week, months of the year) associated with either left space (e.g., 9 a.m., Monday, February) or right space (e.g., 8 p.m., Saturday, November) as cues in a line bisection task. Fifty-seven healthy adults listened to temporal words and then moved a mouse cursor to the perceived midpoint of a horizontally presented line. We measured movement trajectories, initial line intersection coordinates, and final bisection response coordinates. We found movement trajectory displacements for left- vs. right-biasing hour and day cues. Initial line intersections were biased specifically by month cues, while final bisection responses were biased specifically by hour cues. Our findings offer general support to the notion of horizontal space-time associations and suggest further investigation of the exact chronometry and strength of this association across individual time units.

The time course of the spatial representation of 'past' and 'future' concepts: New evidence from the STEARC effect

Humans use space to think of and communicate the flow of time. This spatial representation of time is influenced by cultural habits so that in left-to-right reading cultures, short durations and past events are mentally positioned to the left of long durations and future events. The STEARC effect (Space Temporal Association of Response Codes) shows a faster classification of short durations/past events with responses on the left side of space and of long durations/future events with responses on the right side. We have recently showed that during the classification of time durations, space is a late heuristic of time because in this case, the STEARC appears only when manual responses are slow, not when they are fast. Here, we wished to extend this observation to the semantic classification of words as referring to the 'past' or the 'future'. We hypothesised that the semantic processing of 'past' and 'future' concepts would have engaged slower decision processes than the classification of short versus long time durations. According to dual-route models of conflict tasks, if the task-dependent classification/decision process were to proceed relatively slowly, then the effects of direct activation of culturally preferred links between stimulus and response (S-R), i.e., past/left and future/right in the case of the present task, should attain higher amplitudes before the instruction-dependent correct response is selected. This would imply that, at variance with the faster classification of time durations, during the slower semantic classification of time concepts, in incongruent trials, the direct activation of culturally preferred S-R links should introduce significant reaction time (RT) costs and a corresponding STEARC at the fastest manual responses in the experiment too. The study's results confirmed this hypothesis and showed that in the classification of temporal words, the STEARC also increased as a function of the length of RTs. Taken together, the results from sensory duration and semantic classification STEARC tasks show that the occurrence, strength and time course of the STEARC varies significantly as a function of the speed and level of cognitive processing required in the task.

Space-time mapping on the sagittal axis in congenital blindness

Previous evolutionary perspectives proposed that the space-time mapping on the sagittal axis originates from visuo-locomotion coupling when walking/running forward. Accordingly, the congenitally blind could not have developed a sagittal mental timeline if the latter depends on such a visuo-locomotion coupling. However, this conclusion was reached in only a single empirical study (Rinaldi et al. in J Exp Psychol General 147:444-450, 2018), and its theoretical underpinnings are not entirely convincing as locally static and continuous auditory input undergoes a relatively similar change as function of self-locomotion, but this type of sensory-locomotion coupling is spared even in congenital blindness. Therefore, the present study systematically explored whether the congenitally blind show space-time mappings on the sagittal axis using different paradigms in three experiments. In Experiment 1, using a typical implicit RT task, the congenitally blind showed the same preferred space-time mapping in the sagittal dimension as normally sighted participants did. In Experiment 2, this space-time mapping occurred even automatically when temporal relations were task-irrelevant in a naming task. In Experiment 3, in an explicit space-time mapping task, the congenitally blind were more likely to locate the past behind and the future in front of their bodies. Moreover, most blind participants used spatial metaphors for their space-time mapping on the sagittal axis. These results supported the conclusion that the congenitally blind have a sagittal mental timeline, and that their sensory-locomotion coupling experience was either more similar to that of sighted participants or not critical for the space-time mapping. The present study, thus, also helps to clarify the origin of the sagittal mental timeline.

Modal and amodal cognition: an overarching principle in various domains of psychology

Accounting for how the human mind represents the internal and external world is a crucial feature of many theories of human cognition. Central to this question is the distinction between modal as opposed to amodal representational formats. It has often been assumed that one but not both of these two types of representations underlie processing in specific domains of cognition (e.g., perception, mental imagery, and language). However, in this paper, we suggest that both formats play a major role in most cognitive domains. We believe that a comprehensive theory of cognition requires a solid understanding of these representational formats and their functional roles within and across different domains of cognition, the developmental trajectory of these representational formats, and their role in dysfunctional behavior. Here we sketch such an overarching perspective that brings together research from diverse subdisciplines of psychology on modal and amodal representational formats so as to unravel their functional principles and their interactions.

Mental association of time and valence

Five experiments investigated the association between time and valence. In the first experiment, participants classified temporal expressions (e.g., past, future) and positively or negatively connotated words (e.g., glorious, nasty) based on temporal reference or valence. They responded slower and made more errors in the mismatched condition (positive/past mapped to one hand, negative/future to the other) compared with the matched condition (positive/future to one hand, negative/past to the other hand). Experiment 2 confirmed the generalization of the match effect to nonspatial responses, while Experiment 3 found no reversal of this effect for left-handers. Overall, the results of the three experiments indicate a robust match effect, associating the past with negative valence and the future with positive valence. Experiment 4 involved rating the valence of time-related words, showing higher ratings for future-related words. Additionally, Experiment 5 employed latent semantic analysis and revealed that linguistic experiences are unlikely to be the source of this time-valence association. An interactive activation model offers a quantitative explanation of the match effect, potentially arising from a favorable perception of the future over the past.

Does walking/running experience shape the sagittal mental time line?

A growing body of evidence suggested that time could be separately represented either on the lateral or sagittal axis. And the lateral mental time line has an origin associated with sensorimotor experience, e.g., reading/writing. However, it is still not clear whether the sagittal mental time line also originates from sensorimotor experience, e.g., walking/running. To address this question, we examined how the movement experience affected the space-time mapping on the lateral and sagittal axes using the virtual reality technique in two experiments. The results showed that the virtual movement experience had significant effects on the space-time mapping on the lateral axis (Experiment 1), but not on the sagittal axis (Experiment 2). This finding supported that the space-time mapping on the lateral axis does originate from sensorimotor experience, while the space-time mapping on the sagittal axis more likely originates from spatial metaphors in languages or other cultural experiences.

How time gets spatial: factors determining the stability and instability of the mental time line

Left-to-right readers classify faster past events with motor responses on the left side of space and future events with responses on the right side. This suggests a left-to-right spatial organization in the mental representation of time. Here, we show that the significance and reliability of this representation are linked to the joint use of temporal and spatial codes in the task at hand. In a first unimanual Go/No-Go Implicit Association Test (IAT), attending selectively to "past" or to "future" words did not activate corresponding "left" or "right" spatial concepts and vice versa. In a second IAT, attending to both temporal (i.e., "past" and "future") words and spatial targets (i.e., "left" and "right") pointing arrows produced faster responses for congruent rather than incongruent combinations of temporal and spatial concepts in task instructions (e.g., congruent = "Go with past words and left-pointing arrows"; incongruent = "Go with past words and right-pointing arrows"). This effect increased markedly in a STEARC task where spatial codes defined the selection between "left-side" and "right-side" button presses that were associated with "past" and "future" words. Two control experiments showed only partial or unreliable space-time congruency effects when (a) participants attended to superordinate semantic codes that included both spatial "left"/"right" or temporal "past/future" subordinate codes; (b) a primary speeded response was assigned to one dimension (e.g., "past vs. future") and a nonspeeded one to the other dimension (e.g., "left" vs. "right"). These results help to define the conditions that trigger a stable and reliable spatial representation of time-related concepts.

Combining the Inner Self with the Map of the Body: Evidence for White Matter Contribution to the Relation Between Interoceptive Sensibility and Nonaction-oriented Body Representation

Very recent studies on healthy individuals suggest that changes in the sensibility toward internal bodily sensations across the lifespan affect the ability to mentally represent one's body, in terms of action-oriented and nonaction-oriented body representation (BR). Little is known about the neural correlates of this relation. Here we fill this gap using the neuropsychological model provided by focal brain damage. Sixty-five patients with unilateral stroke (20 with left and 45 with right brain damage, LBD and RBD, respectively) participated in this study. Both action-oriented BR and nonaction-oriented BR were tested; interoceptive sensibility was assessed as well. First, we tested whether interoceptive sensibility predicted action-oriented BR and nonaction-oriented BR, in RBD and LBD separately. Then, a track-wise hodological lesion-deficit analysis was performed in a subsample of twenty-four patients to test the brain network supporting this relation. We found that interoceptive sensibility predicted the performances in the task tapping nonaction-oriented BR. The higher interoceptive sensibility was, the worse patients performed. This relation was associated with the disconnection probability of the corticospinal tract, the fronto-insular tract, and the pons. We expand over the previous findings on healthy individuals, supporting the idea that high levels of interoceptive sensibility negatively affect BR. Specific frontal projections and frontal u-shaped tracts may play a pivotal role in such an effect, likely affecting the development of a first-order representation of the self within the brainstem autoregulatory centers and posterior insula and of a second-order representation of the self within the anterior insula and higher-order prefrontal areas.

Replacing vertical actions by mouse movements: a web-suited paradigm for investigating vertical spatial associations

The number of web-based studies in experimental psychology has been growing tremendously throughout the last few years. However, a straightforward web-based implementation does not exist for all types of experimental paradigms. In the current paper, we focus on how vertical response movements-which play a crucial role in spatial cognition and language research-can be translated into a web-based setup. Specifically, we introduce a web-suited counterpart of the vertical Stroop task (e.g., Fox & Shor, in Bull Psychon Soc 7:187-189, 1976; Lachmair et al., in Psychon Bull Rev 18:1180-1188, 2011; Thornton et al., in J Exp Psychol Hum Percept Perform 39:964-973, 2013). We employed nouns referring to entities typically located in lower or upper vertical space (e.g., "worm" and "bird", respectively) in Experiments 1 and 2, and emotional valence words associated with a crouched or an upward bodily posture (e.g., "sadness" and "excitement", respectively) in Experiment 3. Depending on the font color, our participants used their mouse to drag the words to the lower or upper screen location. Across all experiments, we consistently observed congruency effects analogous to those obtained with the lab paradigm using actual vertical arm movements. Consequently, we conclude that our web-suited paradigm establishes a reliable approach to examining vertical spatial associations.

Embodied time: Effect of reading expertise on the spatial representation of past and future

How do people grasp the abstract concept of time? It has been argued that abstract concepts, such as future and past, are grounded in sensorimotor experience. When responses to words that refer to the past or the future are either spatially compatible or incompatible with a left-to-right timeline, a space-time congruency effect is observed. In the present study, we investigated whether reading expertise determines the strength of the space-time congruency effect, which would suggest that learning to read and write drives the effect. Using a temporal categorization task, we compared two types of space-time congruency effects, one where spatial incongruency was generated by the location of the stimuli on the screen and one where it was generated by the location of the responses on the keyboard. While the first type of incongruency was visuo-spatial only, the second involved the motor system. Results showed stronger space-time congruency effects for the second type of incongruency (i.e., when the motor system was involved) than for the first type (visuo-spatial). Crucially, reading expertise, as measured by a standardized reading test, predicted the size of the space-time congruency effects. Altogether, these results reinforce the claim that the spatial representation of time is partially mediated by the motor system and partially grounded in spatially-directed movement, such as reading or writing.

The Variability of Mental Timeline in Vertical Dimension

People often use concrete spatial terms to represent abstract time. Previous studies have shown that mental timeline (MTL) is represented along a horizontal axis. Studies of the mental timeline have demonstrated that compared with English speakers, Mandarin speakers are more likely to think about time vertically (up-down) than horizontally (left-right/front-back). Prior studies have suggested that MTL in the up and down dimensions originated from temporal-spatial metaphors in language. However, there are still a large number of perceptual experiences in the up and down dimensions, such as visual and sensorimotor experience. Then does the visual experience in daily life affect the MTL in the vertical dimension? This study is aimed to investigate whether visual experience can promote or activate the opposite direction of MTL from implicit and explicit levels. The results showed that when the time information in the task was not prominent, the direction of vertical MTL cannot be affected by ascending or descending perceptual experience. While when the time information was prominent, whether the task was implicit or explicit, compared with the control group, watching the top-down scene significantly increased the top-down direction selection, while in the implicit task, watching the bottom-up scene made the top-down MTL disappear. To the best of our knowledge, our study provides the first evidence that the flexibility of space–time associations in vertical dimension extends beyond explicit and embraces even implicit levels. This study shows that the vertical MTL is activated in certain conditions and could be affected by the visual experience.

Is rushing always faster than strolling? A reaction time study on the processing of sentences containing manner of motion verbs

In the context of the embodied cognition debate, an effect of motion verb associated speed information has previously been detected using eye-tracking, functional magnetic resonance imaging (fMRI), and reaction times (RT). The latter, for instance, was implemented by Wender and Weber (1982), who observed that participants were faster in detecting motion in sentences associated with fast motion compared to sentences associated with slow motion after having formed mental images of the sentences' content. It remains open whether the reported effects of speed are associated with automatic lexical-semantic retrieval processes or whether they reflect higher top-down cognitive processes. To answer this question, the paradigm by Wender and Weber (1982) was adopted and further elaborated in the present study. In Experiment 1 visualization instructions were eliminated. Additionally, the stimulus material was manipulated in regards to the agent of the described movement (human vs. object motion) in order to determine the representation's modality (visual vs. motoric). In Experiment 2, the task to detect motion was replaced by the task to judge sensicality. The results suggest that the prompt to perform mental imagery is not a precondition for the engagement of modal representations in this speed of motion paradigm and that the involved representations' modality is visual rather than motoric. However, the modal representations' involvement is dependent on the task. They thus do not seem to be part of the invariant semantic representation of manner of motion verbs.

The categorical use of a continuous time representation

The abstract concept of time is mentally represented as a spatially oriented line, with the past associated with the left space and the future associated with the right. Although the line is supposed to be continuous, most available evidence is also consistent with a categorical representation that only discriminates between past and future. The aim of the present study was to test the continuous or categorical nature of the mental timeline. Italian participants judged the temporal reference of 20 temporal expressions by pressing keys on either the left or the right. In Experiment 1 (N = 32), all words were presented at the center of the screen. In Experiment 2 (N = 32), each word was presented on the screen in a central, left, or right position. In Experiment 3 (N = 32), all text was mirror-reversed. In all experiments, participants were asked to place the 20 temporal expressions on a 10-cm line. The results showed a clear Spatial–TEmporal Association of Response Codes (STEARC) effect which did not vary in strength depending on the location of the temporal expressions on the line. However, there was also a clear Distance effect: latencies were slower for words that were closer to the present than further away. We conclude that the mental timeline is a continuous representation that can be used in a categorical way when an explicit past vs. future discrimination is required by the task.

Mental representation of autobiographical memories along the sagittal mental timeline: Evidence from spatiotemporal interference

Time is usually conceived of in terms of space: many natural languages refer to time according to a back-to-front axis. Indeed, whereas the past is usually conceived to be "behind us", the future is considered to be "in front of us." Despite temporal coding is pivotal for the development of autonoetic consciousness, little is known about the organization of autobiographical memories along this axis. Here we developed a spatial compatibility task (SCT) to test the organization of autobiographical memories along the sagittal plane, using spatiotemporal interference. Twenty-one participants were asked to recall both episodic and semantic autobiographical memories (EAM and SAM, respectively) to be used in the SCT. Then, during the SCT, they were asked to decide whether each event occurred before or after the event presented right before, using a response code that could be compatible with the back-to-front axis (future in front) or not (future at back). We found that performance was significantly worse during the non-compatible condition, especially for EAM. The results are discussed in light of the evidence for spatiotemporal encoding of episodic autobiographical memories, taking into account possible mechanisms explaining compatibility effects.

Associations Between Abstract Concepts: Investigating the Relationship Between Deictic Time and Valence

The present study examines whether deictic time and valence are mentally associated, with a link between future and positive valence and a link between past and negative valence. We employed a novel paradigm, the two-choice-sentence-completion paradigm, to address this issue. Participants were presented with an initial sentence fragment that referred to an event that was either located in time (future or past) or of different valence (positive or negative). Participants chose between two completion phrases. When the given dimension in the initial fragment was time, the two completion phrase alternatives differed in valence (positive vs. negative). However, when the given dimension in the initial fragment was valence, the two completion phrase alternatives differed in time (future vs. past). As expected, participants chose completion phrases consistent with the proposed association between time and valence. Additional analyses involving individual differences concerning optimism/pessimism revealed that this association is particularly pronounced for people with an optimistic attitude.

Can mental time lines co-exist in 3D space?

A growing body of evidence suggested that time can be represented separately either on the lateral, sagittal or vertical axis. However, it is still not clear whether these mental time lines from different origins could co-exist, or compete with each other such that if one is selected, the others are inhibited? The present study addressed this question using a multi-dimensional free-choice paradigm with Mandarin speakers in three experiments. The results showed that significant spatial-temporal congruency effects were found both on the lateral and sagittal/vertical axes in the horizontal/coronal plane either in a temporal judgment relevant or irrelevant task. By contrast, the spatial-temporal congruency effects did not appear at the same time on the sagittal and vertical axes in the sagittal plane. These results supported that lateral mental time line could co-exist with the other two, while sagittal and vertical mental time lines could not co-exist with each other. This finding implied that the space-time mapping mechanism is different between the lateral axis and the sagittal and vertical axes, whereas it is the same for the latter two axes.

Where Are the Months? Mental Images of Circular Time in a Large Online Sample

People may think about time by mentally imaging it in some spatial form, or as "spacetime." In an online survey, 76,922 Norwegian individuals positioned two dots corresponding to the months of December and March on what they imagined to be their appropriate places on a circle. The majority of respondents placed December within a section of the circumference ranging from 11:00 to 12:00 o'clock, but a group of respondents chose positions around the diametrically opposite 6:00 o'clock position. A similar relationship occurred for March, where most respondents chose a position ranging from 2:30 to 3:00 o'clock but a group of respondents chose positions around 9:00 o'clock. About half of the respondents (N = 39,797) continued to fill out an online questionnaire probing their mental images related to the "year" concept. This clarified that 75% of respondents "saw" the months unfolding in a clockwise direction versus 19% in a counter clockwise fashion. Moreover, while a majority (70%) stated that they imagined the year as a "circle," the rest indicated the use of other mental images (e.g., ellipses and spirals, lines and squares, idiosyncratic or synesthetic spatial forms). We found only weak effects or preferences for spatial forms based on respondents' gender, handedness, age, or geographical location.

Action consequences affect the space-time congruency effect on reaction time

According to the metaphoric mapping hypothesis, people code time in terms of space. Consistent with this hypothesis, several reaction time studies have demonstrated that participants respond faster with a left (right) response to stimuli that convey temporal information about the past (future) than when this stimulus-response mapping is reversed (past → right, future → left). The present experiment examines whether the side of the response key or of the (visual) action effect elicited by the response is the crucial factor of this space-time congruency effect. In a response-effect (R-E) compatible group, a response to a temporal stimulus produced a visual action effect on the same side as the response location (left response → left action effect, right response → right action effect). In an R-E incompatible group, however, response and action effect occurred on opposite sides (left response → right action effect, right response → left action effect). A typical space-time congruency effect was obtained in the R-E compatible group, but the congruency effect interacted with group and was descriptively reversed in the R-E incompatible group. This result pattern suggests that the typical congruency effect is determined by the location of the action consequences rather than the location of the response key. Based on this result, we suggest that the space-time congruency effect is based on an abstract spatial mental representation that embraces action events in the external space.

A standard conceptual framework for the study of subjective time

Research on the mental representation of time ('subjective time') has provided broad insights into the nature of time perception and temporal processing. As the field comprises different scientific disciplines, such as psychology, philosophy, and neuroscience, studies differ with regard to the basic terms and concepts used. For this reason, research on subjective time lacks a coherent conceptual system. We argue that research in the field of subjective time should aim at establishing such a system, i.e., a more standardized terminology, in order to strengthen its theoretical basis and to support an efficient communication of results. Based on key empirical findings and concepts that are commonly (but inconsistently) used in the literature, we argue for a conceptual framework for the study of subjective time that differentiates between three types of mental representations of time: basic temporal processing, time perception in terms of passage, and time perception in terms of duration.

The Space-Time Congruency Effect: A Meta-Analysis

Several reaction time (RT) studies report faster responses when responses to temporal information are arranged in a spatially congruent manner than when this arrangement is incongruent. The resulting space-time congruency effect is commonly attributed to a culturally salient localization of temporal information along a mental timeline (e.g., a mental timeline that runs from left to right). The present study aims to provide a compilation of the published RT studies on this time-space association in order to estimate the size of its effect and the extent of potential publication bias in this field of research. In this meta-analysis, three types of task are distinguished due to hitherto existing empirical findings. These findings suggest that the extent to which time is made relevant to the experimental task has a systematic impact on whether or not the mental timeline is activated. The results of this meta-analysis corroborate these considerations: First, experiments that make time a task-relevant dimension have a mean effect size of d = 0.46. Second, in experiments in which time is task irrelevant, the effect size does not significantly deviate from zero. Third, temporal priming studies have a surprisingly high mean effect size of d = 0.47, which, however, should be adjusted to d = 0.36 due to publication bias.

Space and time in episodic memory: Effects of linearity and directionality on memory for spatial location and temporal order in children and adults

Episodic memory is a critical capacity that involves remembering past events along with their spatial and temporal contexts. Relatively little is known about the relations between spatial and temporal information in long-term memory in children or adults. The present research examined the influence of the mental timeline (linear horizontal display extending from the left to right direction for English speakers) on memory for events and their spatial and temporal features in 7-year-olds, 9-year-olds, 11-year-olds and young adults (N = 146). During encoding, participants studied triplets of objects, varying on two dimensions of the mental timeline: linearity (whether objects were presented in linear succession or not) and direction (whether objects were presented from left-to-right or right-to-left). After a delay, during retrieval, participants were tested on their memory for individual objects, and either the spatial location or temporal order of the objects. We found that overall accuracy for spatial location was higher than accuracy for temporal order, and there was a parallel developmental trajectory for both these aspects of memory. Across age groups we found that memory for temporal order, but not spatial location, was influenced by linearity and direction (i.e., match to mental timeline). Thus, in both children and adults the spatiotemporal mental timeline supported memory for temporal order, converging with predictions generated within domains of language and thought and enhancing our understanding of how space and time are represented in the mind.

The effect of temporal concept on the automatic activation of spatial representation: From axis to plane

Temporal concepts could be represented horizontally(X-axis) or vertically (Y-axis). However, whether the spatial representation of time exists in the whole plane remains unclear. In this study, we investigated whether processing temporal concepts would automatically activate spatial representations in a whole plane without any guidance or cue. Participants first indicated whether a word was past-related or future-related, then, they identified a target in different visual fields. In Experiment 1, the results demonstrated that past time mapped onto the left and top in a plane or axis, while future time mapped onto the right and bottom, with the horizontal effect being stronger than the vertical effect. In Experiment 2, an index of eye movement showed a similar data pattern. Thinking about temporal concepts activates spatial schema automatically without guidance or cue, and the time-space metaphor is represented not only as an axis but also as a whole plane. The results were discussed in terms of the possible cultural differences that made the Chinese participants tend to be more flexible in spatial representation of time because of their comprehensive thinking.

Nature and nurture effects on the spatiality of the mental time line

The nature-nurture debate regarding the origin of mental lines is fundamental for cognitive neuroscience. We examined natural-nurture effects on the mental time line, applying three different challenges to the directionality of time representation. We tested (1) patients with left-neglect and healthy participants, who are (2) left-to-right or right-to-left readers/writers, using (3) a lateralized left-right button press or a vocal mode in response to a mental time task, which asks participants to judge whether events have already happened in the past or are still to happen in the future. Using lateralized responses, a spatial-temporal association of response code (STEARC) effect was found, in concordance with the cultural effects. With vocal responses (no lateralization), past and future events showed similar results in both cultures. In patients with neglect, who have a deficit of spatial attention in processing the left side of space, future events were processed more slowly and less accurately than past events in both cultures. Our results indicate the existence of a “natural” disposition to map past and future events along a horizontal mental time line, which is affected by the different ways in which spatial representation of time is introduced.

The spatial representation of number, time, and serial order following sensory deprivation: A systematic review

The spatial representation of numerical and temporal information is thought to be rooted in our multisensory experiences. Accordingly, we may expect visual or auditory deprivation to affect the way we represent numerical magnitude and time spatially. Here, we systematically review recent findings on how blind and deaf individuals represent abstract concepts such as magnitude and time (e.g., past/future, serial order of events) in a spatial format. Interestingly, available evidence suggests that sensory deprivation does not prevent the spatial "re-mapping" of abstract information, but differences compared to normally sighted and hearing individuals may emerge depending on the specific dimension considered (i.e., numerical magnitude, time as past/future, serial order). Herein we discuss how the study of sensory deprived populations may shed light on the specific, and possibly distinct, mechanisms subserving the spatial representation of these concepts. Furthermore, we pinpoint unresolved issues that need to be addressed by future studies to grasp a full understanding of the spatial representation of abstract information associated with visual and auditory deprivation.

Mixed metaphors: Electrophysiological brain responses to (un)expected concrete and abstract prepositional phrases

Languages around the world use spatial terminology, like prepositions, to describe non-spatial, abstract concepts, including time (e.g., in the moment). The Metaphoric Mapping Theory explains this pattern by positing that a universal human cognitive process underlies it, whereby abstract concepts are conceptualized via the application of concrete, three-dimensional space onto abstract domains. The alternative view is that the use of spatial propositions in abstract phrases is idiomatic, and thus does not trigger metaphoric mapping. In the current study, event-related potentials (ERPs) were used to examine the time-course of neural processing of concrete and abstract phrases consisting of the prepositions in or on followed by congruent and incongruent nouns (e.g., in the bowl/plate and in the moment/mend). ERPs were recorded from the onset of reference nouns in 28 adult participants using a 128-channel electrode net. Results show that congruency has differential effects on neural measures, depending on whether the noun is concrete or abstract. Incongruent reference nouns in concrete phrases (e.g., on the bowl) elicited a significant central negativity (an N400 effect), while incongruent reference nouns in abstract phrases (e.g., on the moment) did not. These results suggest that spatially incongruent concrete nouns are semantically unexpected (N400 effect). A P600 effect, which might indicate rechecking, reanalysis and/or reconstruction, was predicted for incongruent abstract nouns, but was not observed, possibly due to the variability in abstract stimuli. Findings cast doubt on accounts claiming that abstract uses of prepositions are cognitively and metaphorically linked to their spatial sense during natural, on-line processing.

Die Beziehung zwischen sprachlicher und nicht-sprachlicher Kognition

Zusammenfassung. Die Beziehung zwischen sprachlicher und nicht-sprachlicher Kognition ist wenig verstanden. In der Vergangenheit wurden einige Arbeiten veröffentlicht, deren Ergebnisse in der Regel so interpretiert wurden, dass die sprachliche Kognition kaum oder gar nicht die nicht-sprachliche Kognition beeinflusst. Dies hat unseres Erachtens in der Vergangenheit dazu geführt, dass sprachliche und nicht-sprachliche Kognition eher separat beforscht wurden und zu der Vorstellung beigetragen, dass Sprache ein eigenständiges Modul innerhalb des kognitiven Systems besitzt, das funktional unabhängig von nicht-sprachlichen Prozessen ist. In jüngerer Zeit werden diese beiden Bereiche der Kognition im Rahmen des Grounded-Cognition-Ansatzes zusammengeführt, wobei allerdings linguistische Gesichtspunkte unberücksichtigt bleiben (z. B. die kompositionale Struktur sprachlicher Bedeutung). Daher sind die theoretischen Grenzen dieses Ansatzes als umfassendes Modell des menschlichen Sprachverstehens schon jetzt erkennbar. Nicht ganz unabhängig davon, erlebt die Forschung über das Verhältnis von Sprache und Denken in den letzten Jahren eine Wiederbelebung, die im Gegensatz zum Grounded-Cognition-Ansatz eher für die Vorstellung einer funktionalen Trennung von sprachlicher und nicht-sprachlicher Kognition spricht. Wir vertreten hier die These, dass das Verhältnis von sprachlicher und nicht-sprachlicher Kognition stark davon abhängt, welche Repräsentationsformen die beiden Bereiche involvieren. Wir diskutieren dabei zwei Sichtweisen (eine Zwei-Format-Sichtweise und eine Ein-Format-Sichtweise) und analysieren deren Implikationen für das theoretische Verständnis über den Zusammenhang von und sprachlicher und nicht-sprachlicher Kognition. Wir verdeutlichen diese Konzepte anhand aktueller Forschungsergebnisse. Unserer Meinung nach generiert die Ein–Format-Hypothese derzeit mehr Fragen als sie beantwortet. Im Gegensatz dazu erscheint uns die Zwei-Format-Hypothese als Forschungsansatz für die Kognitionspsychologie vielversprechender.

The spatial alignment of time: Differences in alignment of deictic and sequence time along the sagittal and lateral axes

People use space in a variety of ways to structure their thoughts about time. The present report focuses on the different ways that space is employed when reasoning about deictic (past/future relationships) and sequence (earlier/later relationships) time. In the first study, we show that deictic and sequence time are aligned along the lateral axis in a manner consistent with previous work, with past and earlier events associated with left space and future and later events associated with right space. However, the alignment of time with space is different along the sagittal axis. Participants associated future events and earlier events-not later events-with the space in front of their body and past and later events with the space behind, consistent with the sagittal spatial terms (e.g., ahead, in front of) that we use to talk about deictic and sequence time. In the second study, we show that these associations between sequence time and sagittal space are sensitive to person-perspective. This suggests that the particular space-time associations observed in English speakers are influenced by a variety of different spatial properties, including spatial location and perspective.

Enacted Reading Comprehension: Using Bodily Movement to Aid the Comprehension of Abstract Text Content

We report a design study that assessed the feasibility of Enacted Reading Comprehension (ERC), an intervention designed to teach 3rd and 4th grade students (n = 40 and 25, respectively) to use gestures to understand an increasingly abstract set of texts. Students were taught to use gestures to understand the idea of "opposing forces" in a concrete setting-the forces at play as tectonic plates move past each other-and then taught to use the gestures to understand opposing forces in more abstract situations. For example, students were taught to use gestures to understand the opposing sides of an argument, and to understand the internal conflicts that arise as individuals are faced with moral dilemmas. The results of our design study suggest that ERC has promise as a method for introducing students to the idea of using gesture to understand text content, and to employ this strategy in a range of reading contexts.

How Body Orientation Affects Concepts of Space, Time and Valence: Functional Relevance of Integrating Sensorimotor Experiences during Word Processing

The aim of the present study was to test the functional relevance of the spatial concepts UP or DOWN for words that use these concepts either literally (space) or metaphorically (time, valence). A functional relevance would imply a symmetrical relationship between the spatial concepts and words related to these concepts, showing that processing words activate the related spatial concepts on one hand, but also that an activation of the concepts will ease the retrieval of a related word on the other. For the latter, the rotation angle of participant's body position was manipulated either to an upright or a head-down tilted body position to activate the related spatial concept. Afterwards participants produced in a within-subject design previously memorized words of the concepts space, time and valence according to the pace of a metronome. All words were related either to the spatial concept UP or DOWN. The results including Bayesian analyses show (1) a significant interaction between body position and words using the concepts UP and DOWN literally, (2) a marginal significant interaction between body position and temporal words and (3) no effect between body position and valence words. However, post-hoc analyses suggest no difference between experiments. Thus, the authors concluded that integrating sensorimotor experiences is indeed of functional relevance for all three concepts of space, time and valence. However, the strength of this functional relevance depends on how close words are linked to mental concepts representing vertical space.

The Mental Timeline in a Crossed-Hands Paradigm

Abstract. The existence of a lateral mental timeline is well established; in left-to-right writing cultures, past is associated with the left, future with the right. Accordingly, participants respond faster with the left to past, and with the right to future. Recent studies indicate that this association does not reverse when participants respond with their hands crossed. We investigated the role of instruction for this association in a crossed-hands paradigm. Participants classified the temporal reference of words by pressing a key on the left with their right hand, or a key on the right with their left. Half of the participants were instructed to respond with their right or left hand; the other half were instructed to respond with the left or right key. An interaction between time and key showed only for participants instructed to respond with the key, providing support for the role of extracorporal space for the mental timeline.

Reading sentences describing high- or low-pitched auditory events: only pianists show evidence for a horizontal space-pitch association

This study explored differences between pianists and non-musicians during reading of sentences describing high- or low-pitched auditory events. Based on the embodied model of language comprehension, it was hypothesized that the experience of playing the piano encourages a corresponding association between high-pitched sounds and the right and low-pitched sounds and the left. This pitch-space association is assumed to become elicited during understanding of sentences describing either a high- or low-pitched auditory event. In this study, pianists and non-musicians were tested based on the hypothesis that only pianists show a compatibility effect between implied pitch height and horizontal space, because only pianists have the corresponding experience with the piano keyboard. Participants read pitch-related sentences (e.g., the bear growls deeply, the soprano singer sings an aria) and judged whether the sentence was sensible or not by pressing either a left or right response key. The results indicated that only the pianists showed the predicted compatibility effect between implied pitch height and response location. Based on the results, it can be inferred that the experience of playing the piano led to an association between horizontal space and pitch height in pianists, while no such spatial association was elicited in non-musicians.

Non-musicians also have a piano in the head: evidence for spatial-musical associations from line bisection tracking

The spatial representation of ordinal sequences (numbers, time, tones) seems to be a fundamental cognitive property. While an automatic association between horizontal space and pitch height (left-low pitch, right-high pitch) is constantly reported in musicians, the evidence for such an association in non-musicians is mixed. In this study, 20 non-musicians performed a line bisection task while listening to irrelevant high- and low-pitched tones and white noise (control condition). While pitch height had no influence on the final bisection point, participants' movement trajectories showed systematic biases: When approaching the line and touching the line for the first time (initial bisection point), the mouse cursor was directed more rightward for high-pitched tones compared to low-pitched tones and noise. These results show that non-musicians also have a subtle but nevertheless automatic association between pitch height and the horizontal space. This suggests that spatial-musical associations do not necessarily depend on constant sensorimotor experiences (as it is the case for musicians) but rather reflect the seemingly inescapable tendency to represent ordinal information on a horizontal line.

Depicting Visual Motion in Still Images: Forward Leaning and a Left to Right Bias for Lateral Movement

What artistic conventions are used to convey the motion of animate and inanimate items in still images, such as drawings and photographs? One graphic convention involves depicting items leaning forward into their movement, with greater leaning conveying greater speed. Though this convention could derive from the natural leaning forward of people and animals as they run, it is also applied to depictions of inanimate objects (eg cars and trains). It is proposed that it is this convention that allows the italicization of text to convey notions of motion and speed. Evidence for this is obtained from three sources: the use of italicization on book covers (in book titles); judgments of typeface connotations; and performance measures during the semantic classification of words appearing in italicized and non-italicized fonts. Inspection of the availability of italic fonts in Hebrew indicates an additional artistic convention for conveying motion, based on a fundamental bias, yet to be confirmed, for people to expect to see, or prefer to see, lateral movement (real or implied) in a left to right direction, rather than a right to left direction. Evidence for such a bias is found in photographs of a range of animate and inanimate items archived on Google Images. Whereas a rightward bias is found for photographs of animate and inanimate items in motion (the more so, the faster the motion being conveyed), either no bias or a leftward bias is found for the same items in static pose. Possible origins of a fundamental left to right bias for visual motion, and future lines of research able to evaluate them, are identified.

Are introspective reaction times affected by the method of time estimation? A comparison of visual analogue scales and reproduction

In this study, we investigated whether the method of time estimation plays a role in the apparent limits of introspection in dual-task processing. Previous studies showed that when participants reported introspective reaction times after each trial of a dual task by clicking on a visual analogue scale, they appeared to be unaware of the dual-task costs in their performance. However, visual analogue scales have seldom been used in interval estimation, and they may be inappropriate. In the present study, after each dual-task trial, participants reported their introspective reaction times either via a visual analogue scale or via the method of reproduction. The results replicated the previous findings, irrespective of method. That is, even though responses to the second task slowed down with increasing task overlap, this slowing was only very weakly reflected in the introspective reaction times. Thus, the participants' failure to report the objective dual-task costs in their reaction times is a rather robust finding that cannot be attributed to the method employed. However, introspective reaction times reported via visual analogue scales were more closely related to the objective reaction times, suggesting that visual analogue scales are preferable to reproduction. We conclude that introspective reaction times represent the same information regardless of method, but whether that information is temporal in nature is as yet unsettled.

Exploring Temporal Progression of Events Using Eye Tracking

This study investigates the representation of the temporal progression of events by means of the causal change in a patient. Subjects were asked to verify the relationship between adjectives denoting a source and resulting feature of a patient. The features were presented either chronologically or inversely to a primed event context given by a verb (to cut: long-short vs. short-long). Effects on response time and on eye movement data show that the relationship between features presented chronologically is verified more easily than that between features presented inversely. Post hoc, however, we found that the effects of temporal order occurred only when subjects read the features more than once. Then, the relationship between the features is matched with the causal change implied by the event context (contextual strategy). When subjects read the features only once, subjects respond to the relationship between the features without taking into account the event context.

Response mode does not modulate the space-time congruency effect: evidence for a space-time mapping at a conceptual level

Previous studies reported a space-time congruency effect on response time, supporting the notion that people's thinking about time is grounded in their spatial sensorimotor experience. According to a strong view of metaphoric mapping, the congruency effect should be larger for responses that differ in their spatial features than for responses that lack such differences. In contrast, a weaker version of this account posits that the grounding of time is based on higher-level spatial concepts. In this case, response mode should not modulate the size of the space-time congruency effect. In order to assess these predictions, participants in this study responded to temporal stimuli either manually or vocally. Response mode did not modulate the space-time congruency effect which supports the weaker view of metaphoric mapping suggesting that this effect emerges at a higher cognitive level.

What's up? Emotion-specific activation of vertical space during language processing

The relationship between language processing and vertical space has been shown for various groups of words including valence words, implicit location words, and words referring to religious concepts. However, it remains unclear whether these are single phenomena or whether there is an underlying common mechanism. Here, we show that the evaluation of word valence interacts with motor responses in the vertical dimension, with positive (negative) evaluations facilitating upward (downward) responses. When valence evaluation was not required, implicit location words (e.g., bird, shoe) influenced motor responses whereas valence words (e.g., kiss, hate) did not. Importantly, a subset of specific emotional valence words that are commonly associated with particular bodily postures (e.g., proud→upright; sad→slouched) did automatically influence motor responses. Together, this suggests that while the vertical spatial dimension is not directly activated by word valence, it is activated when processing words referring to emotional states with stereotypical bodily-postures. These results provide strong evidence that the activation of spatial associations during language processing is experience-specific in nature and cannot be explained with reference to a general mapping between all valence words and space (i.e., all positive and negative words generally relate to spatial processing). These findings support the experiential view of language comprehension, suggesting that the automatic reactivation of bodily experiences is limited to word groups referring to emotions or entities directly associated with spatial experiences (e.g., posture or location in the world).

How are overlapping time intervals perceived? Evidence for a weighted sum of segments model

This study investigated the way in which people time two overlapping intervals. Timing models already proposed in the literature predict different effects of the degree of overlap on each estimate, and empirical findings were compared to these predictions. Two unimodal experiments (in which each to-be-timed interval was a visual stimulus) and one bimodal experiment (in which one to-be-timed interval was auditory and the other visual) were conducted. The estimate of the first interval was either unaffected or decreased, and the estimate of the second interval consistently increased as the intervals were more temporally separated. The only model in the literature that could account for such result patterns is a single pacemaker single accumulator structure with an additional recency weighting (see the weighted sum of segments model). That is, participants appear to segment the two overlapping intervals into three non-overlapping and overlapping segments, time these segments separately, and then combine them to estimate each interval. Importantly, a recency weighting, determined by the time that has passed since the end of that segment, is also applied to each segment in the summation process. Further, in the bimodal experiment the order in which the stimuli of different modalities were presented affected the way in which they were timed, a finding that none of the current models can explain. This highlights that a comprehensive model of interval timing must consider not only the modalities of to-be-timed intervals but also the order in which different modalities must be timed.

Musical metaphors: evidence for a spatial grounding of non-literal sentences describing auditory events

This study investigated whether the spatial terms high and low, when used in sentence contexts implying a non-literal interpretation, trigger similar spatial associations as would have been expected from the literal meaning of the words. In three experiments, participants read sentences describing either a high or a low auditory event (e.g., The soprano sings a high aria vs. The pianist plays a low note). In all Experiments, participants were asked to judge (yes/no) whether the sentences were meaningful by means of up/down (Experiments 1 and 2) or left/right (Experiment 3) key press responses. Contrary to previous studies reporting that metaphorical language understanding differs from literal language understanding with regard to simulation effects, the results show compatibility effects between sentence implied pitch height and response location. The results are in line with grounded models of language comprehension proposing that sensory motor experiences are being elicited when processing literal as well as non-literal sentences.

Can conceptual congruency effects between number, time, and space be accounted for by polarity correspondence?

Conceptual congruency effects have been interpreted as evidence for the idea that the representations of abstract conceptual dimensions (e.g., power, affective valence, time, number, importance) rest on more concrete dimensions (e.g., space, brightness, weight). However, an alternative theoretical explanation based on the notion of polarity correspondence has recently received empirical support in the domains of valence and morality, which are related to vertical space (e.g., good things are up). In the present study we provide empirical arguments against the applicability of the polarity correspondence account to congruency effects in two conceptual domains related to lateral space: number and time. Following earlier research, we varied the polarity of the response dimension (left-right) by manipulating keyboard eccentricity. In a first experiment we successfully replicated the congruency effect between vertical and lateral space and its interaction with response eccentricity. We then examined whether this modulation of a concrete-concrete congruency effect can be extended to two types of concrete-abstract effects, those between left-right space and number (in both parity and magnitude judgment tasks), and temporal reference. In all three tasks response eccentricity failed to modulate the congruency effects. We conclude that polarity correspondence does not provide an adequate explanation of conceptual congruency effects in the domains of number and time.

Do we map remembrances to the left/back and expectations to the right/front of a mental timeline? Space-time congruency effects with retrospective and prospective verbs

Recent experimental studies have shown that people code time in terms of a mental timeline which typically runs from left to right or from back to front. Determining the cognitive function of this mental timeline for language processing, however, is still an unsettled issue. Whereas the studies of Ulrich and Maienborn (2010) and Ulrich et al. (2012) argue against an automatic activation of the mental timeline for the interpretation of tense and temporal adverbials at sentence level, Sell and Kaschak (2011) observe an automatic activation for the processing of past- and future-related sentences in small stories. The present paper reports the results of three experiments which examine the processing of sentences with retrospective and prospective verbs (e.g., to remember, to regret vs. to expect, to announce) in present tense, which locate a second, embedded event in the past or the future. When temporal information was task-relevant, a space-time congruency effect emerged (Experiment 1). This suggests that the mental timeline is not only linked to overtly deictic linguistic material but may also be construed in a more intricate way through the compositional construction of sentence meaning. The congruency effect disappeared, however, when temporal information was task-irrelevant (Experiments 2 and 3), suggesting that the mental timeline is not functionally involved in the cognitive processing of these especially demanding two-event sentences. The results of the present study support the conclusion that the relevant factor driving an automatic activation of the mental timeline is not the number of linguistically expressed events, but might rather be the number of sentential units.

On the efficacy of procedures to normalize Ex-Gaussian distributions

Reaction time (RT) is one of the most common types of measure used in experimental psychology. Its distribution is not normal (Gaussian) but resembles a convolution of normal and exponential distributions (Ex-Gaussian). One of the major assumptions in parametric tests (such as ANOVAs) is that variables are normally distributed. Hence, it is acknowledged by many that the normality assumption is not met. This paper presents different procedures to normalize data sampled from an Ex-Gaussian distribution in such a way that they are suitable for parametric tests based on the normality assumption. Using simulation studies, various outlier elimination and transformation procedures were tested against the level of normality they provide. The results suggest that the transformation methods are better than elimination methods in normalizing positively skewed data and the more skewed the distribution then the transformation methods are more effective in normalizing such data. Specifically, transformation with parameter lambda -1 leads to the best results.

[The future is in your right hand: an embodied temporal concept]

Previous, studies demonstrated that participants in left-to-right writing cultures showed a strong preference to associate the past with left space and the future with right space. The present studies investigated whether these spatial associations involved body-part-centered or extracorporal space. In Experiment 1, participants categorized words as referring to the past or the future by pressing button on the left with the left hand or a button on the right with the right hand. In Experiments 2 and 3, participants crossed their hands and were instructed to categorize words by pressing the left or right buttons (Experiment 2) or by moving their left or right hand (Experiment 3). Irrespective of the relative spatial positions of the response buttons, past words were more quickly categorized with the left hand and future words with the right hand. In addition, reaction times were slowerr in Experiment 2 than in Experiment 1, whereas there was no significant difference beween Experiments 1 and 3. These results suggest that temporal concepts such as past and future are more strongly associated with embodied space than visual space.