Distributional learning of subcategories in an artificial grammar: Category generalization and subcategory restrictions

Distributional Lattices as a Model for Discovering Syntactic Categories in Child-Directed Speech

Distribution information plays an important role in word categorization. In this paper, we present a novel distributional model, distributional lattices to discover syntactic categories in child directed speech. A distributional lattice is a hierarchy formed by closed sets of words that are distributionally similar. Such a hierarchy is potentially useful for capturing syntactic categories by clustering words with associate patterns they occur in. In order to empirically support the suggestion that the distributional lattice is effective at categorizing words, we present a distributional lattice analysis of the Brent corpus of child-directed speech. The results show that distributional lattices are able to yield extremely accurate syntactic categories.

Lexical category acquisition is facilitated by uncertainty in distributional co-occurrences

This paper analyzes distributional properties that facilitate the categorization of words into lexical categories. First, word-context co-occurrence counts were collected using corpora of transcribed English child-directed speech. Then, an unsupervised k-nearest neighbor algorithm was used to categorize words into lexical categories. The categorization outcome was regressed over three main distributional predictors computed for each word, including frequency, contextual diversity, and average conditional probability given all the co-occurring contexts. Results show that both contextual diversity and frequency have a positive effect while the average conditional probability has a negative effect. This indicates that words are easier to categorize in the face of uncertainty: categorization works best for words which are frequent, diverse, and hard to predict given the co-occurring contexts. This shows how, in order for the learner to see an opportunity to form a category, there needs to be a certain degree of uncertainty in the co-occurrence pattern.

Statistical language learning: computational, maturational, and linguistic constraints

Our research on statistical language learning shows that infants, young children, and adults can compute, online and with remarkable speed, how consistently sounds co-occur, how frequently words occur in similar contexts, and the like, and can utilize these statistics to find candidate words in a speech stream, discover grammatical categories, and acquire simple syntactic structure in miniature languages. However, statistical learning is not merely learning the patterns presented in the input. When their input is inconsistent, children sharpen these statistics and produce a more systematic language than the one to which they are exposed. When input languages inconsistently violate tendencies that are widespread in human languages, learners shift these languages to be more aligned with language universals, and children do so much more than adults. These processes explain why children acquire language (and other patterns) more effectively than adults, and also may explain how systematic language structures emerge in communities where usages are varied and inconsistent. Most especially, they suggest that usage-based learning approaches must account for differences between adults and children in how usage properties are acquired, and must also account for substantial changes made by adult and child learners in how input usage properties are represented during learning.

Infant learning: Historical, conceptual, and methodological challenges

Acquiring knowledge about the underlying structures of the environment presents a number of challenges for a naive learner. These challenges include the absence of reinforcement to guide learning, the presence of numerous information sources from which only a select few are relevant, and the uncertainty about when an underlying structure may have undergone a change. A crucial implication of these challenges is that the naive learner must make implicit decisions about when to generalize to novel inputs and when to restrict generalization because there are multiple underlying structures. An historical perspective on these challenges is presented and some potential solutions are proposed.