The redundancy of recursion and infinity for natural language

LadRa-Net: Locally Aware Dynamic Reread Attention Net for Sentence Semantic Matching

Sentence semantic matching requires an agent to determine the semantic relation between two sentences, which is widely used in various natural language tasks, such as natural language inference (NLI) and paraphrase identification (PI). Much recent progress has been made in this area, especially attention-based methods and pretrained language model-based methods. However, most of these methods focus on all the important parts in sentences in a static way and only emphasize how important the words are to the query, inhibiting the ability of the attention mechanism. In order to overcome this problem and boost the performance of the attention mechanism, we propose a novel dynamic reread (DRr) attention, which can pay close attention to one small region of sentences at each step and reread the important parts for better sentence representations. Based on this attention variation, we develop a novel DRr network (DRr-Net) for sentence semantic matching. Moreover, selecting one small region in DRr attention seems insufficient for sentence semantics, and employing pretrained language models as input encoders will introduce incomplete and fragile representation problems. To this end, we extend DRr-Net to locally aware dynamic reread attention net (LadRa-Net), in which local structure of sentences is employed to alleviate the shortcoming of byte-pair encoding (BPE) in pretrained language models and boost the performance of DRr attention. Extensive experiments on two popular sentence semantic matching tasks demonstrate that DRr-Net can significantly improve the performance of sentence semantic matching. Meanwhile, LadRa-Net is able to achieve better performance by considering the local structures of sentences. In addition, it is exceedingly interesting that some discoveries in our experiments are consistent with some findings of psychological research.

A novel approach to investigate recursion and iteration in visual hierarchical processing

We describe a new method to explore recursive cognition in the visual domain. We define recursion as the ability to represent multiple hierarchical levels using the same rule, entailing the ability to generate new levels beyond those previously encountered. With this definition recursion can be distinguished from general hierarchical embedding. To investigate this recursion/hierarchy distinction in the visual domain, we developed two novel methods: The Visual Recursion Task (VRT), in which an inferred rule is used to represent new hierarchical levels, and the Embedded Iteration Task (EIT), in which additional elements are added to an existing hierarchical level. We found that adult humans can represent recursion in the visuo-spatial domain, and that this ability is distinct from both general intelligence and the ability to represent iterative processes embedded within hierarchical structures. Compared with embedded iteration, visual recursion correlated positively with other recursive planning tasks (Tower of Hanoi), but not with specific visuo-spatial resources (spatial short-term memory and working memory). We conclude that humans are able to use recursive representations to process complex visuo-spatial hierarchies and that our visual recursion task taps into specific cognitive resources. This method opens exciting opportunities to explore the relationship between visual recursion and language.

How children perceive fractals: hierarchical self-similarity and cognitive development

The ability to understand and generate hierarchical structures is a crucial component of human cognition, available in language, music, mathematics and problem solving. Recursion is a particularly useful mechanism for generating complex hierarchies by means of self-embedding rules. In the visual domain, fractals are recursive structures in which simple transformation rules generate hierarchies of infinite depth. Research on how children acquire these rules can provide valuable insight into the cognitive requirements and learning constraints of recursion. Here, we used fractals to investigate the acquisition of recursion in the visual domain, and probed for correlations with grammar comprehension and general intelligence. We compared second (n=26) and fourth graders (n=26) in their ability to represent two types of rules for generating hierarchical structures: Recursive rules, on the one hand, which generate new hierarchical levels; and iterative rules, on the other hand, which merely insert items within hierarchies without generating new levels. We found that the majority of fourth graders, but not second graders, were able to represent both recursive and iterative rules. This difference was partially accounted by second graders' impairment in detecting hierarchical mistakes, and correlated with between-grade differences in grammar comprehension tasks. Empirically, recursion and iteration also differed in at least one crucial aspect: While the ability to learn recursive rules seemed to depend on the previous acquisition of simple iterative representations, the opposite was not true, i.e., children were able to acquire iterative rules before they acquired recursive representations. These results suggest that the acquisition of recursion in vision follows learning constraints similar to the acquisition of recursion in language, and that both domains share cognitive resources involved in hierarchical processing.

Probing recursion

The experimental probing of recursion in human performance is faced with non-trivial problems. Here, I analyse three case studies from the literature and argue that they tell us little about the underlying mental processes at play within each of these domains: (a) the question of whether experimental participants employ recursive rules in parsing artificial strings of nonsense syllables; (b) the role of self-embedded structures in reasoning and general cognition; and (c) the reputed connection between structural features of a given object and the corresponding, recursive rules needed to represent/generate it. I then outline what a recursive process would actually look like and how one could go about probing its presence in human behaviour, concluding, however, that recursive processes in performance are very unlikely, at least as far as fast, mandatory, and automatic modular processes are concerned.

Emergence in the central nervous system

"Emergence" is an idea that has received much attention in consciousness literature, but it is difficult to find characterizations of that concept which are both specific and useful. I will precisely define and characterize a type of epistemic ("weak") emergence and show that it is a property of some neural circuits throughout the CNS, on micro-, meso- and macroscopic levels. I will argue that possession of this property can result in profoundly altered neural dynamics on multiple levels in cortex and other systems. I will first describe emergent neural entities (ENEs) abstractly. I will then show how ENEs function specifically and concretely, and demonstrate some implications of this type of emergence for the CNS.